Enroll a custom detection

Tracemill ships validation content for the public detection library. Your own detections — the ones your team wrote, tuned, and depends on — start out uncovered: Tracemill can see them in your Splunk, but has nothing that makes them fire.

This flow closes that gap. It ends with the detection enrolled: a validation job exists, a real run has shown it makes the detection fire, and it is a member of a suite you can run on demand.

Start from the consoleCopy Anchor Link

On a target's Detections page, open Uncovered. Your own saved searches and ES correlation searches carry a checkbox and an Author with your AI action; uncovered detections from the public library read Library coverage pending instead, because Tracemill's library will cover them.

Select up to 10 — Select first 10 does it in one click — and choose Author N with your AI, or use the button on a detection's own page. Copy the prompt into your MCP-connected agent. It names the detections by reference, so the agent fetches everything else from Tracemill. Copying changes nothing in Tracemill.

You need a role that can author content and manage suites (editor or above).

Start the flowCopy Anchor Link

Connect the MCP server, then ask your agent for the enroll-custom-detection prompt. Most clients show prompts in a slash-command or template menu. Some clients call them "prompts", others "workflows".

The prompt carries the whole recipe, including the failure paths. You do not need to memorize what follows — this page is here so you know what your agent is doing and where you will be asked to decide.

If you already have tm:detection: references — the console copies them for you — pass them as the prompt's references argument, or just ask your agent to call get_authoring_brief with them. That tool returns the same recipe as the prompt, so clients without prompt support work too.

Up to 10 detections from one target go in one batch. Your agent first checks that the Splunk it can read is the Splunk those detections came from, and stops if it is not.

A workspace API token on the Teams or Trial plan cannot run this flow over MCP: its API scope is content, and reading a target's detections, starting runs, and enrolling need full. Connect with OAuth instead; the API scope applies only to API tokens.

What you will be askedCopy Anchor Link

The flow stops for you at six points. These are deliberate; an agent should not decide them alone.

1. Target binding, when the match rests on a Splunk server name rather than a GUID or cluster label. Names are human-assigned and can collide.
2. The triage table, before anything is authored. For every detection the agent shows its sourcetype, the event type it maps to, where it expects to get events from, and whether it plans to author it or drop it (and why). You confirm which detections to pursue.
3. The category table, per detection, after your exported events are summarized. The agent labels each kind of event attack-positive (what the detection should fire on), attack-positive-variant (another path to the same effect), benign-control (legitimate activity it must not fire on), or noise (discarded). You confirm or correct the labels: attack categories become workloads expected to alert, benign controls become workloads expected not to.
4. The identity diff, per detection, before anything is uploaded. You are approving what leaves your environment.
5. A fidelity warn, per detection, if the generated events reproduce less than 80% of the real event's fields. You either accept the gap or fix it.
6. The suite, and the final enroll, once for the whole batch. There is no default suite and Tracemill will not invent one.

A batch never merges the per-detection decisions: each detection gets its own category table, identity diff, and fidelity check.

What happens in betweenCopy Anchor Link

Bind the target. The agent reads your Splunk instance fingerprint and matches it against your targets. A GUID or cluster-label match binds automatically. If you supply a target id yourself, it is verified against the fingerprint rather than trusted.

Resolve the detection. Tracemill's detection catalog — populated by the TA-Tracemill add-on — is checked for the detection. If it is missing, the flow stops: wrong target, add-on not installed there, or a stale inventory that a refresh will fix.

Check the surface is supported. The detection's sourcetype has to map onto an event surface Tracemill can generate. If it does not, the flow stops here, before anyone invests in authoring. Adding a surface is platform work.

Get events that fire it. You need real events in native format. Best is an existing test corpus — the ESCU attack_data for the detection, or your own detection-as-code fixtures. Next best is a Splunk export of times the detection actually fired. Weakest, and the agent will tell you so, is exporting normal events and editing the triggering values in: production telemetry is usually benign, so it has the right shape but not the values that trigger.

The events are uploaded straight from your machine to a presigned URL. They do not pass through the agent's context. See the authoring workspace for how long they live.

Profile the detection. The agent reads the SPL for the fields it filters and groups on, any threshold, and its time window. If it can query your Splunk, it cross-checks the group-by fields against Splunk's own parse of the search (reduce_spl_groupby); a disagreement stops the flow for you to settle.

Author, and check fidelity. The agent writes a scenario and a job, validates them against the engine schema, then renders the scenario and compares it against one of your real events. That comparison is the point of the whole flow — it is what distinguishes "we generated some events" from "we generated events that this detection will fire on".

The verdict is pass, warn, or fail. fail means a field the detection keys on is missing or wrong. warn means the reproduction is thin, and you get to judge it.

Whatever the verdict, the agent also shows you the diff of your real event against the generated one, field by field after formatting is normalized. Expect timestamps, request ids, and other per-event values to differ on every render; any other difference is worth a question. The diff covers the first event of a burst. When it is long it is cut at a line and the agent says so (the full list of field differences is still in the fidelity report); if it could not be built, the verdict still stands.

Synthesize identities. Your usernames, hostnames, and IPs are replaced with generated ones. Only the literals the detection actually keys on survive — and where one of those is specific to you (a watchlist name, a host-naming pattern), the agent flags it explicitly rather than quietly shipping it. You see a before/after diff before anything is uploaded.

Verify it. Enrollment does not require a run, but a real run of the job against your target is how you find out now — rather than at the suite's first scheduled fire — whether the detection actually fires. An enrolled detection that never worked will sit visibly failing, so the run is where the flow earns its keep.

Enroll. You pick the suite. A new suite is manual-only unless you ask for a schedule — a detection's first enrollment should not start firing on a cadence nobody chose.

How it endsCopy Anchor Link

The agent finishes with one table, one row per detection you started with — including any reference it could not use. Each row is either enrolled, with the suite, the run (or a note that you chose to skip it), and the detection's coverage confirmed as covered, or stopped, with exactly one reason:

• Tracemill excluded the reference, with the reason it gave: for example, the detection is already covered, is disabled in Splunk, or ships with the public detection library.
• The Splunk your agent can read is not the one the detections came from; the whole batch stops, every row with this reason.
• The reference did not resolve to a detection, or the detection you named is not in Tracemill's catalog even after a refresh.
• Its sourcetype maps to no event surface Tracemill can generate.
• You chose to enable the sibling detections blocking an existing job instead of authoring a new one.
• You declined it, at triage or at one of the gates.
• No source of events produced any that trigger it.
• Only a key-value or plain-text export exists; the fidelity check needs XML or JSON.
• The generated events could not reproduce the fields the detection keys on.
• Its run finished without the detection firing.
• Enrollment left it out of the suite; the agent names why.

After the table, each enrolled detection gets a short report of what was authored and how it was verified.

Reference guidesCopy Anchor Link

The agent follows Tracemill's open-source authoring method, which ships inside the prompt, and pulls its reference guides on demand with get_authoring_guide: engine-capabilities (what the generator can express), job-authoring (how jobs and expectations work), and splunk-spl (how a detection's SPL is read). The same method and guides are public in the content library.

When it does not workCopy Anchor Link

The one worth understanding is when the run succeeds but the detection never fires. That is not a Tracemill failure — it means the events being generated do not actually trigger the search. Either the fields the detection keys on were misread from the SPL, or the real event chosen as the reference was never a triggering event in the first place. Both are fixed by going back to the events, not by re-running.

AfterwardsCopy Anchor Link

The detection's coverage flips to covered. A Test now claims it, and that Test is in a suite. Run that suite whenever you want to know whether the detection still fires — after a Splunk upgrade, after someone edits the search, after a data-source change.

That is the point: a detection you have never tested is a detection you are only assuming works.