The consultant’s visit had been booked since autumn. He arrived on a Tuesday in January with a spreadsheet and four clear days, and by Friday he handed us a list of 61 findings for the machine-learning platform my team operates. Number 14 was real. So was number 30. The rest were stale, duplicated, or already fixed, and nobody could tell which was which without opening every line. That document was our entire security programme, and it is the reason my teammates started reading about DevSecOps in the first place.

Half a year before that visit, we had begun a quieter experiment: wiring checks into the delivery pipeline itself, so that every change was examined on the day it was proposed rather than on the day an auditor asked. This post compares the two arrangements — the scheduled review versus the embedded controls — from a team that has now lived with both. If security keeps arriving at your company as a season instead of a habit, the contrast should feel familiar. The reference we kept returning to while arguing about all this is the Ops Atlas entry on DevSecOps.

Two clocks: the quarter and the pull request

The review runs on a calendar. Embedded checks run on a diff. That difference sounds academic until you watch both in motion. Our January list described a platform that no longer existed: two of the flagged services had been retired in February, a third had been rebuilt, and the worst credential problem had been rotated during a March incident. The audit was accurate for about eleven days of its shelf life.

Pipeline checks age differently. A scanner that flags a hardcoded token in a pull request is commenting on code that is still warm — the author remembers writing it, the fix is minutes away, and the lesson lands while the context is fresh. Findings decay. The closer a control sits to the moment of change, the slower its conclusions rot.

There is a third difference people underestimate: the paperwork. After every quarterly review we wrote a remediation plan, assigned the items to a backlog nobody prioritised, and met again ninety days later to discover the backlog had won none of its arguments. With checks in the pipeline, the record of what was checked, when, and with what result accumulates by itself — which is exactly the evidence our enterprise customers started demanding.

What DevSecOps asks of every change

Under the review model, our coverage was a snapshot: whatever the consultant sampled in one week. Under the pipeline model, coverage is a standing set of questions asked of every proposed change:

• Does the diff contain a credential? A pre-commit scanner catches most of them, and server-side push protection catches the ones a developer forgets.
• Does it pull in a dependency with a known hole? Composition analysis reads the manifest before the merge, not after the breach.
• Does it weaken the infrastructure definition? Configuration checks inspect templates the same way unit tests inspect logic.
• Does the container image carry known vulnerabilities? Image scanning runs at build time instead of incident time.
• Does the running thing behave like the designed thing? Dynamic tests and API checks probe the deployed candidate.

None of these questions is exotic. The difference is that our pipeline asks them two hundred times a month, and the consultant asked them once.

DevSecOps controls carry positions, not wishes

The framing that finally made this click gives each control four properties to declare: where it sits in the delivery flow, how much its verdict can be trusted, what it costs to run, and what it does when it fires — stop the line, flag a warning, or simply write things down. A severity rating without a position is a wish. Once controls carry addresses, you can interrogate them the way we interrogate monitors: what does this observe, when does it fire, and what happens afterwards?

The model settles into seven control families, and those families answer named practices in NIST’s Secure Software Development Framework — PW.1, PW.4, PW.7, PW.8, PS.1 through PS.3, and PO.5. That mapping mattered to us because enterprise buyers keep asking for SSDF alignment during vendor reviews, and it is easier to answer when your controls already have homes. NIST’s SP 800-204C calls DevSecOps a “facilitating paradigm” for CI/CD, which is bureaucrat for: this is how the work gets organised, not a box you purchase.

What blocking is for

Our first pipeline instinct was also our worst: stop every merge above medium severity. We trained our engineers to route around the gate within three weeks. The corrective is a narrower rule — a refusal has to clear four tests at once. The finding is severe. It is fixable. The checker is confident. And it is new, not inherited from the period before the check existed.

A gate that punishes people for history teaches people to route around gates.

Anything that fails the four-test rule does not stop the merge; it becomes follow-up work carrying a named owner, a deadline, and the template’s ninety-day ceiling on how long an exception may live. Old debt gets burned down on its own schedule instead of vetoes every pull request.

Then there is the arithmetic of trust. The Atlas walks through one sample month: 1,240 proposed changes, of which the gate refused 96, and humans waved 22 of those through anyway. Twenty-two out of ninety-six is 22.9 percent — the signature of a gate engineers had stopped believing. The healthy band is below five percent of overrides per month. The same example demotes one rule whose output ran 43.6 percent false, which is what a functioning programme looks like: the gate loses arguments and gets better.

The people cost nobody budgets

Here is the finding that changed our roadmap conversations. DORA’s 2022 research report found that teams with weak security practices showed 1.4 times the odds of high burnout, and that culture predicted adoption better than tooling did. The quarterly-review model concentrates strain: one week of panic, then silence, then panic again. The embedded model spreads the load — small findings, handled in the flow of ordinary work.

We also borrowed the champions idea. Engineers spread across the platform spend 10 to 20 percent of their week — protected, explicit, written into their role — on security questions for their own area: the training-data ingestion path, the model registry, the inference gateway. No heroics. A rota. When the January consultant arrived the following year, the champions answered his questions from evidence the pipeline had already produced, and the visit took two days instead of four.

What the quarterly review is still good for

Retiring the audit entirely would be its own mistake. Some questions genuinely need a slow, human look: is the threat model for the new ingestion path still sound, does the segmentation between inference and storage still match the design, are we answering customers’ compliance questionnaires honestly? Those are design and governance questions, not diff questions. It is equally honest to name what embedded pipelines do not cover: runtime detection and response stay with the SOC, estate-wide vulnerability prioritisation stays with its own team, and cloud posture is a separate discipline. DevSecOps has a boundary, and pretending otherwise just relocates the gap.

Choosing DevSecOps over the calendar

If you operate an AI platform and recognise our January self, the whole comparison compresses into five questions:

• When a finding arrives, is the code still warm, and does its author still remember writing it?
• Does every control have a position, a confidence, a cost, and an action — or do you have a spreadsheet?
• What did your gate’s override rate look like last month, and would engineers call the gate fair?
• Who owns each exception, and on what date does it expire?
• Which questions are genuinely design questions that deserve a scheduled, human, unhurried review?

For us the honest summary is that the review produced 61 facts about one moment in the past, while the pipeline keeps producing a few facts about every present. We kept the review for design questions and let go of the belief that an annual inspection constitutes a programme. If you are rebuilding the surrounding delivery culture first, the beginner’s DevOps roadmap on this site is the sensible on-ramp, our piece on moving from DevOps to the wider XOps family shows where the sibling disciplines slot in, and the way we apply AI support for IT decision-making may short-cut the evidence argument inside your own team.

Security that arrives quarterly is an event. Security that arrives with every change is an operating system. We have run both; we know which one we trust with the platform.