Detection-as-Code Is Not About Git: Building a Program That Scales Without You

Detection-as-Code gets sold as a toolchain decision. Move your rules into Git, wire up a CI pipeline, write everything in Sigma, done. Teams adopt the stack, check the box, and report that they “do Detection-as-Code.”

What they’ve actually done is put version control around their existing process. If that process was a senior engineer writing rules from intuition and pushing them straight to production, DaC gives you exactly that — now with a commit history. The tooling is real and worth having. It is also the part that changes the least about whether your program works.

The thing the title promises — a program that scales without you — is not a toolchain. It’s a capability: a detection program whose output quality doesn’t depend on any single person being in the room. Git is how you store that capability. It is not the capability.

---

What “Scales Without You” Actually Means

Here is the test. If your best detection engineer takes a month off — or leaves — does detection quality degrade?

In most programs the answer is yes, because the quality lives in one person’s head. Which rules are fragile. Which data sources lie. What a good detection looks like versus a plausible one. What was already tried and quietly failed. None of that is in the SIEM. It’s in the engineer, and it walks out the door with them.

Splunk’s State of Security 2025 found that 63% of teams want to use Detection-as-Code frequently or always, but only 35% do today [1]. That gap is real, but it understates the problem — because most of that 35% mean “we have a repo,” not “we have a system that encodes what our best engineer knows.” Version control is necessary. It is not sufficient, and confusing the two is why so many programs stall after the migration and wonder why nothing got better.

To make the gap concrete, it helps to put a ladder on it.

---

The Detection-as-Code Maturity Model

Maturity models for detection engineering already exist. Kyle Bailey’s Detection Engineering Maturity Matrix and Elastic’s Detection Engineering Behavior Maturity Model both map the whole discipline — people, incident response experience, infrastructure, scope. They’re good, and they’re broad.

What none of them isolate is the DaC pipeline itself: the specific ladder from “clicked into a console” to “self-correcting.” That’s exactly where the Git-equals-DaC confusion lives, so that’s the ladder worth drawing. Each rung is pinned to an observable test — something you can check by looking at the repo and the pipeline — not a self-assessment, because “what maturity level are you” gets gamed the same way “what’s your ATT&CK coverage” does.

Level	Name	The observable test
L0	Console-clicked	Can you git log a detection? No — it lives in the SIEM UI.
L1	Version-controlled	Commit history and a review gate? Yes. Proof the rule fires? No.
L2	Tested	Pick a random merged detection. Is there an artifact showing it fired on the technique and stayed quiet on benign data?
L3	Measured	Can the pipeline emit current tested coverage against your threat model with nobody hand-assembling it? Can you name any detection’s owner?
L4	Drift-aware	When a log schema changes and a rule silently stops firing, does the system tell you within a known window — or do you find out mid-incident?

Most teams that say “we do DaC” are at L1. They have the repo, the pull requests, the peer review, maybe a linter that checks YAML syntax and ATT&CK tag presence. That’s genuine progress over L0 and it is not where the value is. A pull request reviewer can confirm the rule is well-formed and have no idea whether it fires on the technique it claims to cover or buries the analyst in false positives. Put a bad detection through a flawless Git workflow and you get a well-documented, peer-reviewed, version-controlled bad detection. The tooling moved. The quality didn’t.

The rest of this post is about the rungs above L1 — because that’s the entire difference between storing your chaos and building a program.

---

The Wall Is Testing

The jump from L1 to L2 is the wall. It’s the rung most teams never climb, and it’s the one that separates a real DaC program from a versioned guess.

A detection you can’t test is not code. It’s a hypothesis written in YAML. The thing that makes L2 different is that every detection ships with a test, and the test is a merge gate — it answers two questions before the rule reaches production, instead of letting your SOC discover the answers under fire:

Does it fire? True-positive validation. Execute the technique — an Atomic Red Team test, a script, a replayed log sample — and confirm the rule triggers. Palantir’s ADS framework builds this in: every detection strategy has to document how to make it fire and prove the true positive is caught. If you can’t describe how to trigger your own detection, you don’t understand it well enough to ship it.

Does it stay quiet? Run it against a window of known-benign data and confirm it doesn’t light up. That’s your false-positive baseline, captured as a test artifact instead of discovered in production by an analyst who’s now drowning. Untested detections are how you manufacture the exact alert-fatigue problem I covered in Alert Fatigue Is an Offensive Technique — except self-inflicted.

A detection that passes both is code. One that passes neither is something you deployed to prod and asked your SOC to debug live. Here’s what carrying that validation contract looks like in the rule itself:

# Strategy abstract: detect credential access via LSASS memory dump using
# comsvcs.dll MiniDump (a living-off-the-land path). Chosen because it is a
# high-signal technique with a near-zero benign footprint in this environment.
#
# True-positive validation (merge gate):
#   Atomic Red Team T1003.001 Test #3 (comsvcs.dll) must trigger this rule in
#   the test environment before merge. CI replays the generated EDR telemetry.
# Known-FP baseline (merge gate):
#   Replay 7 days of production process_creation logs; expected matches = 0.
title: LSASS Memory Dump via comsvcs.dll MiniDump
status: stable
logsource:
  category: process_creation
  product: windows
detection:
  selection:
    Image|endswith: '\rundll32.exe'
    CommandLine|contains|all:
      - 'comsvcs.dll'
      - 'MiniDump'
  condition: selection
falsepositives:
  - No known legitimate use of comsvcs MiniDump here (baseline = 0).
    Re-validate the baseline if an EDR or backup agent is introduced.
level: high
tags:
  - attack.credential_access
  - attack.t1003.001

The wall is not technical. Writing a test is easy. The wall is cultural — it’s the shift from “ship from intuition” to “prove it before it merges,” and that shift is uncomfortable for exactly the senior engineers whose intuition is usually right. In practice, two things move a team over it: a bad incident where a rule that everyone trusted turned out to be silently broken, and a sustained insistence from whoever owns the program that “I tested it manually once” is not a merge gate. It’s rarely a tool that flips the culture. It’s usually a painful miss that traces back to an untested rule, plus someone refusing to let the team forget it.

---

Measured: Coverage Becomes a Build Artifact

L3 is where the program starts to run without depending on you, because the things that used to live in your head become outputs of the pipeline.

Coverage stops being a board slide and becomes a build artifact. In an L1 program, “what’s our coverage” is a question someone answers by hand the week before a leadership review, mapping rules to ATT&CK in a spreadsheet. In an L3 program, every detection carries metadata that maps it to your threat model, and the pipeline emits current coverage as a build output — continuously, automatically, and honestly. The honesty is structural: a rule that fails its true-positive test doesn’t count toward coverage, so the number reflects tested reality instead of library size. This is the difference between the ATT&CK-completion vanity metric and the Threat Detection Coverage I argued for in The Detection Funnel — coverage scoped to your threat model and anchored to detections that actually fire. The 2025 SANS/Anvilogic State of Detection Engineering survey found teams increasingly want exactly this: quality measured by operational impact, not framework completion [2].

Every detection has an owner. Not a Git blame line — a named person responsible for the rule’s health, who answers when it breaks. This is the metadata that lets you “track a detection down to people,” and without it a broken rule is nobody’s problem until it’s everybody’s incident.

The lifecycle has an exit. Research, build, test, deploy, monitor — and retire. Almost nobody runs the last step. Detections accumulate, never get removed, and rot into precisely the 13% of non-functional rules CardinalOps found sitting in enterprise SIEMs [3]: counted in coverage reports, guaranteed never to fire. A detection at L3 has a lifecycle state and an exit, not just a birth.

There’s a metadata payoff here that reaches beyond the SOC. When detections are mapped to a threat model in a structured, queryable way, the handoff from threat intel to detection engineering stops depending on a personal relationship. I described that broken handoff in Your Threat Intel Program Is Measuring the Wrong Thing: CTI produces a detection requirement, DE is mid-sprint, and the requirement waits because there’s no shared, encoded prioritization. At L3, the threat-model mapping is that shared substrate. Coverage gaps become queryable, and “we have no detection for this actor’s primary technique” becomes a build output, not a conversation that may or may not happen.

---

Drift-Aware: Why Green at Merge Isn’t Enough

Here’s the failure mode that L2 and L3 don’t fix on their own. A test that passed at merge tells you the detection worked on the day you merged it. It says nothing about whether it still fires six months later, after a log source was re-pointed, an EDR agent changed its field schema, a cloud provider renamed an event, or the environment simply drifted out from under the rule.

Point-in-time validation creates a blind spot that looks exactly like coverage. The rule is green in the repo. It’s counted in your coverage artifact. It hasn’t fired in four months — and you can’t tell whether that’s because the technique is genuinely absent or because the detection quietly died. That ambiguity is where the real damage lives, because it survives every gate you built at L2 and L3.

L4 is the rung that closes it: validation runs continuously, not once. The mechanism that works in practice is not re-running atomic tests against production — that has cost and side effects you don’t want. It’s a dedicated test environment where you run scenario tests and unit tests for detections against synthetic telemetry injected to match the technique. You replay the attack into clean, controlled data on a schedule, and if the rule that passed at merge stops matching the injected scenario, you’ve caught drift before an adversary did. There are several variations — recorded log replays, synthetic event generation, EDR emulation — and the common thread is that the detection is re-proven against a known-good signal on a cadence, not trusted indefinitely because it once passed.

A cheap complement on the production side: monitor fire-rate deltas. A rule that averaged five hits a day and has been at zero for three weeks is probably broken, not quiet. That’s a weak signal on its own — plenty of good detections are legitimately silent — but as a trigger to re-run the synthetic test, it’s nearly free. Drift detection borrowed straight from how data and ML platforms watch for schema and distribution drift: treat a detection that stopped matching its own validation scenario as seriously as you’d treat an error.

---

”Isn’t This Over-Engineering for a Small Team?”

The strongest objection to everything above: a five-person SOC doesn’t need merge gates, coverage build artifacts, synthetic test environments, and ownership metadata. They need to write more detections and work the queue. This is process overhead that a big-tech detection team can afford and everyone else is cargo-culting from conference talks.

That objection is half right, and the half it gets right matters: the tooling does not scale down. A three-person team should not stand up a Jenkins pipeline with automated atomic testing and a synthetic injection harness in their first quarter. If you read this post as “go build that infrastructure or you’re not doing real DaC,” you’ve read it as another tooling argument, which is the exact mistake it’s arguing against.

But the discipline scales down completely, and it’s free. The testing habit — “before I ship this, can I make it fire, and does it stay quiet?” — costs nothing but rigor, and it’s the single highest-leverage thing a small team can do. A small team can least afford the taxes that untested detections impose: the 13%-broken tax that erodes coverage you think you have, and the alert-fatigue tax that burns the analyst hours you don’t have to spare. The maturity ladder is not “buy more tools.” L0 to L2 is almost entirely habit. The tooling at L3 and L4 is how you scale the habit once it’s working — not a prerequisite for having it.

Detection-as-Code is not the toolchain. It’s the discipline. Git is where you keep it.

---

Closing

The question to ask about your program isn’t “are our detections in Git.” It’s: if your best engineer left tomorrow, how much of your detection quality leaves with them?

Everything on the ladder above L1 — the tests, the lifecycle states, the coverage artifact, the ownership metadata, the drift checks — is a mechanism for keeping that answer close to “none.” A detection’s test encodes what it’s supposed to catch. Its metadata encodes where it fits the threat model. Its owner encodes who answers when it breaks. Its drift check encodes the promise that green today still means green next quarter. Put together, that’s a program that survives the people who built it. That’s what scales.

And it matters more, not less, as the authoring gets cheap. As AI starts generating detection logic on demand, the testing gate becomes the only thing standing between you and a repo full of plausible, untested, version-controlled guesses. The bottleneck was never writing the rule. It was knowing whether the rule works — and proving it, continuously, without you in the room.

Resources

1. State of Security 2025 — Splunk (63% of teams want to use Detection-as-Code frequently or always; only 35% do today).
2. 2025 State of Detection Engineering Report — Anvilogic with SANS Institute (teams increasingly want detection quality measured by operational impact).
3. 2025 State of SIEM Detection Risk Report — CardinalOps (13% of SIEM rules are non-functional).