Detections & Tests
See which of a target's detections are covered, and enroll Tests for them.
A target's Detections page lists the rules your SIEM reports. A Test exercises one behavior against the target and checks what the Detection does with it. Enrolling a detection creates its Tests.
The detection catalog
The Detections page shows the catalog the SIEM connection reported: saved searches for Splunk, correlation rules for CrowdStrike. Refresh detections asks the SIEM for a fresh catalog instead of waiting for the next report.
Each detection falls into one coverage bucket:
| Coverage | Meaning |
|---|---|
| Covered | At least one valid Test in an enabled, scheduled Test suite exercises this detection. A Test that only runs manually, or is invalid, retired, or archived, does not count. |
| Coverable | No Test covers it yet, but Tracemill has a Test definition you can enroll now. |
| Uncovered | No Test covers it and there is no Test definition that can be enrolled for it. |
| Inactive | No Test covers it, and the detection is disabled, not scheduled, or removed in the SIEM. |
You can also filter by MITRE technique, by posture — Failing, Stale, Passing, Not scored — and by the detection's state in the SIEM.
Enrolling detections
Enrolling needs the editor role or above.
- On the Detections page, select Enroll on a coverable detection, or Enroll all filtered to enroll every coverable detection in the current filter. One enrollment can create up to 500 Tests.
- Choose Test definitions — the definitions that exercise each detection. A detection can have several, for example one that should alert and a benign control that should not.
- Test pipeline assignment — choose the pipeline each Test delivers through.
- Add to a Test suite — a new Test suite or an existing one. A suite created here starts without a schedule. The detection stays Coverable until the suite has one, because only scheduled Tests count toward coverage; see Test suites.
You can also create a single Test from the Tests page with Add Test: pick a Test definition, configure it, and place it in a Test suite or leave it unscheduled.
Detections Tracemill has no Test for
An uncovered detection, such as a saved search your own team wrote, needs content authored for it. Author with your AI opens a prompt to paste into an AI agent connected to the Tracemill MCP server, which reproduces the detection's events, validates them against a real run, and enrolls the result. See Enroll a custom detection.
A Test
A Test is a Test definition — a job from the content library or from your own uploaded content — configured for one target. Each workload of the job is a Test case with its own expectation: an alert is expected, no alert is expected, or the events are only observed.
The Test page has four tabs:
| Tab | What it shows |
|---|---|
| Overview | Run history, Test cases with their latest evidence, investigations, a timeline, and the Test suites the Test belongs to. |
| Configuration | The Test's settings. |
| Runs | Every run of the Test. |
| Revisions | Each configuration the Test has had. Applying a change creates a new revision; earlier ones are kept unchanged. |
Run now starts a run immediately. Add to a Test suite schedules the Test. Archive retires it.
Configuration
| Setting | Description |
|---|---|
| Output | Which of the target's pipelines the Test delivers through. |
| Pools | The definition's default pools, or your own pool in their place — for example your own user names or IP ranges. |
| State overrides | Values for the job's state variables, such as an account ID or region, so events match your environment. |
The expected outcome is part of the Test definition and is not configurable.
Test status
| Status | Meaning |
|---|---|
| Passing | The latest validation evidence met every Test case's expectation. |
| Failing | The latest validation evidence did not. Open the run for the reason; see Runs. |
| Blocked | Events could not be delivered, so the detection was not validated. |
| Stale | The Test is overdue for validation. |
| Pending | The Test is waiting for validation evidence. |
| Detection inactive | The Test has no active detection to validate. Either the detection is disabled, not scheduled, or removed in the SIEM, or the Test's detection could not be matched on this target: it is not installed, or it matches more than one detection. The reason under the status says which. |
| Unscheduled | The Test is not in an enabled Test suite, or its suite only runs manually. |
| Invalid config | A pool, output, or state override no longer fits the definition. |
| Content retired | The Test definition has been retired from the content library. |
| Archived | The Test was archived. |
Status is scored from validation evidence, not from the latest run as such. A run that never started, or could not generate its events, says nothing about the detection, so it leaves the Test's status unchanged: a Test can stay Passing while its most recent run shows as failed for one of those reasons.