Enroll a custom detection

Tracemill ships validation content for the public detection library. Your own detections — the ones your team wrote, tuned, and depends on — start out uncovered: Tracemill can see them in your Splunk, but has nothing that makes them fire.

This flow closes that gap. It ends with the detection enrolled: a validation job exists, a real run has proven it makes the detection fire, and it is a member of a suite you can run on demand.

Start the flow

Connect the MCP server, then ask your agent for the enroll-custom-detection prompt. Most clients show prompts in a slash-command or template menu. Some clients call them "prompts", others "workflows".

The prompt carries the whole recipe, including the failure paths. You do not need to memorize what follows — this page is here so you know what your agent is doing and where you will be asked to decide.

What you will be asked

The flow stops for you five times. These are deliberate; an agent should not decide them alone.

1. Target binding, when the match rests on a Splunk server name rather than a GUID or cluster label. Names are human-assigned and can collide.
2. The category table, after your exported events are summarized. You choose which kinds of event matter.
3. The identity diff, before anything is uploaded. You are approving what leaves your environment.
4. A fidelity warn, if the generated events reproduce less than 80% of the real event's fields. You either accept the gap or fix it.
5. The suite, and the final enroll. There is no default suite and Tracemill will not invent one.

What happens in between

Bind the target. The agent reads your Splunk instance fingerprint and matches it against your targets. A GUID or cluster-label match binds automatically. If you supply a target id yourself, it is verified against the fingerprint rather than trusted.

Resolve the detection. Tracemill's detection catalog — populated by the TA-Tracemill add-on — is checked for the detection. If it is missing, the flow stops: wrong target, add-on not installed there, or a stale inventory that a refresh will fix.

Check the surface is supported. The detection's sourcetype has to map onto an event surface Tracemill can generate. If it does not, the flow stops here, before anyone invests in authoring. Adding a surface is platform work.

Get events that fire it. You need real events in native format. Best is an existing test corpus — the ESCU attack_data for the detection, or your own detection-as-code fixtures. Next best is a Splunk export of times the detection actually fired. Weakest, and the agent will tell you so, is exporting normal events and editing the triggering values in: production telemetry is usually benign, so it has the right shape but not the values that trigger.

The events are uploaded straight from your machine to a presigned URL. They do not pass through the agent's context. See the authoring workspace for how long they live.

Author, and check fidelity. The agent writes a scenario and a job, validates them against the engine schema, then renders the scenario and compares it against one of your real events. That comparison is the point of the whole flow — it is what distinguishes "we generated some events" from "we generated events that this detection will fire on".

The verdict is pass, warn, or fail. fail means a field the detection keys on is missing or wrong. warn means the reproduction is thin, and you get to judge it.

Synthesize identities. Your usernames, hostnames, and IPs are replaced with generated ones. Only the literals the detection actually keys on survive — and where one of those is specific to you (a watchlist name, a host-naming pattern), the agent flags it explicitly rather than quietly shipping it. You see a before/after diff before anything is uploaded.

Prove it. Enrollment does not accept a claim, only evidence: a real run of the job against your target, at the job's current revision, in which every expectation passed. If you edit the content afterwards, the evidence no longer applies and the run has to happen again.

Enroll. You pick the suite. A new suite is manual-only unless you ask for a schedule — a detection's first enrollment should not start firing on a cadence nobody chose.

When it does not work

The most common outcome on a first attempt is no_qualifying_run: the content is fine, but nothing has proven it yet, or content changed after the last run. The agent re-runs and retries.

The one worth understanding is when the run succeeds but the detection never fires. That is not a Tracemill failure — it means the events being generated do not actually trigger the search. Either the fields the detection keys on were misread from the SPL, or the real event chosen as the reference was never a triggering event in the first place. Both are fixed by going back to the events, not by re-running.

Afterwards

The detection's coverage flips to enrolled. It has a job, and the job is in a suite. Run that suite whenever you want to know whether the detection still fires — after a Splunk upgrade, after someone edits the search, after a data-source change.

That is the point: a detection you have never tested is a detection you are only assuming works.