Detection decay: keeping purple team rules working after the session

What the session actually proved

On the day, a detection fired under conditions you controlled: a named test host, an operator running a written procedure from a known account, at a time when everyone in the room was watching. That is the right way to validate content, and it is a narrow claim. It says the logic matches the procedure when the telemetry arrives. It does not say the telemetry will still arrive in November, that the rule survives the next agent policy push, or that an analyst who has never seen the alert will act on it at two in the morning.

The distance between those two statements is where purple team output gets lost. Rules are written, the report is signed, and nothing is re-run until the next exercise finds part of the set silent. How a purple team session runs, hour by hour covers the day itself. This article is the part after it: what to run in the weeks that follow so the content you paid for is still content you hold.

The first two weeks: soak before you alert

A rule that goes straight from a session into a paging queue is a rule somebody will switch off. Run it in monitoring mode first, writing to a log or a low priority queue where nobody is woken. Two weeks is a workable soak for content keyed on process or identity behaviour, because it covers a full working cycle: a weekend, a patch window, and whatever runs at month end.

You are reading three things during the soak. How often the rule fired, what fired it, and how much of that was the behaviour you meant. A rule that fires forty times a week from three service accounts is not broken, it is untuned, and the fix is to exclude by identity or image path rather than to widen the pattern. A widened pattern is how a rule quietly stops matching the procedure it was built for. Rules that come through the soak get promoted with a written response step attached. Rules that produced nothing at all need checking before they are trusted, because silence can mean the estate is clean and it can mean the log source moved. If a rule is silent, the telemetry conditions that have to hold before any detection works are the first place to look.

The detection record, and the name attached to it

Rule content on its own is not maintainable. What makes it maintainable is a short record stored beside it, in the same version control, saying what it keys on and what would break it. Detection held only inside a provider's console is content you can read and not content you can review, which is the practical argument for insisting on ownership in the contract. NIST SP 800-92 treats log management as an operating process with owners and review cycles rather than a configuration set once, and detection content sits one level above it with the same requirement.

An example of the form, for a single rule. It is illustrative rather than drawn from any engagement, and it is deliberately short enough that somebody will actually fill it in.

Every field there answers a question somebody asks later, usually at the worst moment. Most of it can be written while the session is still running: the technique list built before the session already carries the procedure and the expected data source, so the record is largely a copy with the result and the noise notes added.

Four changes that break a working rule

None of these throws an error. The rule stays enabled, the dashboard keeps counting it, and it stops matching anything. Each one is found by watching something other than the rule.

A rule that has not fired in ninety days is either a technique nobody attempted or a rule that no longer works, and the console cannot tell you which. Health checks belong in the same pipeline as the content: alert on the absence of an expected log source, not only on the presence of an attack. NIST SP 800-137 is written around ongoing awareness rather than a point in time assessment, and a detection estate is one of the few places where that distinction is measurable rather than rhetorical.

Counting coverage without flattering yourself

Coverage claims are where this quietly goes wrong. A dashboard reading 180 techniques covered is usually counting rules written, not rules validated, and the two numbers diverge from the week the session ends. Count a technique as covered when a rule for it fired against a real procedure on a date you can name, and stop counting it when that date is older than your re-test interval. Everything else is a rule that exists.

Coverage is also bounded by collection, not by rule count. ATT&CK data sources describe the telemetry a technique detection depends on, and two rules reading the same source are one point of failure rather than two covered techniques. That is the number an objective-based Red Team engagement tests without asking permission: if the claim is 180 and an operator walks past all of them, the gap was in the counting rather than in the operator.

The re-run, and what it measures

Re-run the same entries from the same written procedures a quarter later. It costs less than the first session because the list, the records and the environment access already exist, and it produces a number the first session cannot: decay. Rules that fired then and are silent now, verdicts that slipped from detected to logged only, response steps that name a rota which has since changed. A purple team engagement that ends in content your team owns is what makes the second run a comparison rather than a fresh start, and it is the reason to care where the rule content lives.

When re-runs stop producing changes, the question has moved on. What is left to test is whether the content fires when nobody is expecting it, which is a different exercise: an assumed breach engagement starts from a foothold you grant and runs without the detection team being told what to watch for. The NIST Cybersecurity Framework places tests and exercises under improvement, where the expected output is a change to the programme. A re-run that changes nothing and an exercise that changes nothing are different results, and only one of them is good news.

What a supervisor in Abu Dhabi or Dubai asks for months later

Four regimes name security testing for organisations operating in the UAE: the Central Bank for licensed financial institutions, the Dubai Electronic Security Center for Dubai government entities and the suppliers who serve them, ADHICS for healthcare in Abu Dhabi, and the federal Information Assurance Standards published by NESA. Read the one that binds you rather than a summary of it, because what each asks an entity to hold differs. What they have in common as a practical matter is timing: the question arrives long after the exercise, and a dated screenshot of an alert from a previous quarter is a weak answer to it.

The artefact that answers it in either emirate is the same one described above. A record per rule, with a validation date, a named owner and a re-test interval, is a description of a capability held over time rather than an event that happened once. A Dubai government supplier evidencing controls at contract award and an Abu Dhabi healthcare entity evidencing them under ADHICS are asking their detection team for the same file. Whether the entity is licensed in a free zone or on the mainland changes who supervises it and what the contract says; it does not change whether the rule still fires this morning.

Frequently asked questions