Cribl's CloudTrail pack: 60-80% fewer events, eight silent Splunk detections

TL;DR: 11 of 38 Splunk ESCU AWS detection validations are affected by the cribl-aws-cloudtrail-logs pack being added into the detection pipeline. The pack's documented read-event filter leaves eight detections silent, one more fires on reduced evidence, and the pack's undocumented account-ID masking functionality fabricated an alert (by merging accounts) and destroys the eventID of ~1 in 280 events.

The official "Cribl Pack for AWS CloudTrail Data Collection", cribl-aws-cloudtrail-logs, is inspired by an earlier blog post by one of Cribl's co-founders and is described as "a content pack that helps lay the groundwork for collecting CloudTrail logs". More specifically, it drops Describe, List and Get events to lower the ingest-based cost, and also redacts sensitive data along the way. Seems like a sensible option for anyone using Cribl with AWS CloudTrail.

Naturally we were eager to test whether this pack has any effect on downstream analytics - say, Splunk ES Content Update detections. However, the hectic life of an early startup made its usual corrections, so the pack was sitting dormant in our Cribl test bed for a few months until we finally took it for a spin on July 19.

Our setup: Tracemill S3/CloudTrail sink -> S3 -> Cribl Stream -> HEC -> Splunk Enterprise 9.4 with AWS TA and TA-Tracemill.

Through the pack it goes!

All right, so we flipped the cloudtrail-pack route in Cribl Cloud to make sure everything goes through it. The #2 passthrough route is receiving no events, so the pack is doing its thing:

Then, enabled 38 Splunk ES Content Update AWS detections on our test splunk box. Time to hit it with the simulated events.

Tracemill CLI lets you run scenarios from its open source scenario library (or your own scenarios for that matter) so you can run this to hit your splunk with all tracemill-supported AWS attack scenarios:

brew install tracemill/tap/tracemill

# fetch the open-source content library (once)
tracemill update

# inject every AWS scenario through the same S3 → Cribl → Splunk path
cd ~/.tracemill/library
for job in jobs/splunk/aws/*/*.yaml; do
  echo "→ $job"
  tracemill run "$job" \
    --s3-bucket  <your-cloudtrail-bucket> \
    --s3-region  us-east-1 \
    --s3-format  cloudtrail
done

(The cloudtrail sink writes to S3 with your ambient AWS credentials, so you'll need an active session - env vars, a named profile, or SSO - authorized to s3:PutObject on that bucket.)

That's great, although you'd have to manually check whether each and every detection fired since the CLI doesn't have access to your Splunk. Tracemill Cloud does that for you, so we'll use it here. Enrolling all 38 AWS detections into a Test Suite:

We have a Test Suite, let's fire it:

A snag: the events arrive as husks

Now let's take a look at Tracemill TA on Splunk - this is the piece that validates Tracemill Cloud runs. Hmm.. looks like it's not observing the events on the splunk side:

After doing some troubleshooting it looks like the events are arriving to Splunk, but heavily mutilated - essentially there were just some empty husks:

More troubleshooting pointed at the pack itself: the v1.1.2 we had installed shipped a bug that butchered every event on the default route. Cribl had already fixed it in 1.1.3 back in March (~3.5 months of the bug being alive) - we were just two releases behind. Duh.

Well, it was a fun exercise, albeit a bit of a late-coming one.

Take two: 27 of 38

All right, upgraded to the current release, v1.2.0 and re-ran the suite. Pack route off: 38 of 38 validated. Pack on: 27 of 38. Interesting..

Every one of the 11 failures is a regression across that single boundary - same scenarios, same Splunk box, same detections, only the pack in or out of the path. Let's analyze the failure modes.

1. The filter: eight detections are silent, one barely survives

OK, so this is the pack delivering on its promise: the default route drops every event whose eventName matches ^(Describe|List|Get). The 1.2.0 release note gives the scale in passing, as a performance win: "avoids processing 60-80% of events that get discarded".

Nine of the 11 are this filter doing its job. In the scheduler log, eight of the affected detections ran on time and returned zero rows - status=success, result_count=0. The full casualty list:

• AWS Credential Access GetPasswordData
• AWS ECR Container Scanning Findings High
• AWS ECR Container Scanning Findings Medium
• AWS Bedrock High Number List Foundation Model Failures
• AWS Excessive Security Scanning
• AWS Concurrent Sessions From Different Ips
• AWS Exfiltration via Anomalous GetObject API Activity
• AWS IAM AccessDenied Discovery Events
• AWS Exfiltration via EC2 Snapshot

Most victims are obvious: a detection that consumes GetPasswordData or DescribeImageScanFindings has nothing left to read. Two were less obvious:

• aws-iam-accessdenied-discovery-events - keyed on errorCode=AccessDenied rather than on specific eventNames, so a static "which eventNames does this rule need" pass misses it entirely; the filter still starves it, because the denied calls are all Describe/List/Get
• aws-exfiltration-via-ec2-snapshot - our scenario plays a four-call chain (create the snapshot, share it out, confirm the share, delete it); the detection itself is order-free - any two distinct of its four snapshot eventNames from one actor inside a 5-minute bucket - and the three surviving calls clear that bar on their own, so it still fires. The Tracemill job fails anyway - the dropped call (DescribeSnapshotAttribute, the confirm-the-share step and one of the detection's four documented data sources) never reaches the index, so ground truth comes up one event short. The alert exists; part of the evidence behind it does not

Look at the class as a whole: credential access, S3 exfiltration recon, container scan results, Bedrock enumeration, session hijacking, access-denied discovery. Reads like early, quiet stages of a cloud intrusion. The filter encodes an assumption dating back to the 2019 blog post the pack is based on - that read-only events "tend to be not that useful in security analysis". Seven years of recon-heavy cloud intrusions have not been kind to that assumption.

Is this a bug? No. It is documented, intended, and cutting 60-80% of CloudTrail volume is a real saving.

But "Cribl documented it" and "your detection engineer decided it" are different facts, and your coverage model assumes the second. If the pack went in for the cost savings, some of your AWS detections became decorative at the same moment: still enabled, still running on schedule, still reporting success, and unable to ever return a row.

2. The mask that merges every account into one

The tenth failure is not in the README. The pack's account-ID mask, in 1.2.0:

rules:
  - matchRegex:  /(^|[^\d])\d{12}(?!\d)/
    replaceExpr: C.Mask.repeat(16)
  # (a second rule masks 21-char principal-ID-shaped tokens; not in play here)
fields:
  - source
  - _raw.*
depth: 5

Every 12-digit account-ID-shaped token, in every field of every event, becomes the same literal: XXXXXXXXXXXXXXXX. Not a per-account pseudonym - the same constant for every account. Across the whole target, one value:

One of our scenarios runs across two accounts on purpose - the job puts the attack burst in one, a quiet control in another. The detection it validates - AWS Successful Console Authentication From Multiple IPs - groups by account:

| stats dc(src) as distinct_ip_count ... BY _time, user, signature, vendor_account
| where distinct_ip_count > 1

With vendor_account collapsed to a constant, the attack burst and the control merged into one group, dc(src) cleared the threshold, and the control - a workload that must not fire - fired. The pack did not lose this detection; it manufactured an alert that is not real. In a multi-account environment, that is what constant-masking does to every detection that groups or thresholds by account, and it takes per-account attribution with it.

1.2.0 narrowed the match pattern - "Tighten Account ID masking regex from alphanumeric to digits-only", per its release note - and it makes no difference here, because AWS account IDs are twelve digits. The \d{12} rule matches them exactly as the old [A-Z0-9]{12} did.

3. The mask eats eventIDs

The eleventh failure is the same mask, masking something else. A CloudTrail eventID is a UUID whose final segment is exactly 12 characters. When all 12 happen to be digits, the segment matches the account-ID pattern and gets replaced - along with the preceding hyphen, because the regex consumes the leading character and the replacement never puts it back.

That is exactly what this run rolled. One of the failed-login burst's 25 events (the job validating AWS High Number Of Failed Authentications For User) went out with a final segment of twelve digits, and landed unrecognizable:

sent:     eventID: 041c4f29-3bd3-4692-ad86-716046089462
indexed:  eventID: 041c4f29-3bd3-4692-ad86XXXXXXXXXXXXXXXX

To be precise, in this case the detection itself was not disturbed: it never reads eventID, so the mangled event still counted toward its threshold, the rule fired, and the alert matched the burst. The Tracemill job failed because it was unable to trace all the injected events hitting the Splunk index, which it considers a failed test.

The rate is computable with a little math exercise: each hex character is a digit with probability 10/16 or 0.625, so the probability of all 12 being numbers is 0.625^12, or ~0.36%. So roughly 1 in 280 CloudTrail events has its eventID masked. Not something you'd immediately flag, but not a one-off, either - here's a quick search on our test Splunk instance with very little traffic and the pack being on and off for tests:

Same day, same target: the pack did this to one of our platform's own (real) AssumeRole events - infrastructure housekeeping, mangled in passing:

Notice the source field in that screenshot, too. The mask's field list is ['source', '_raw.*'], so the Splunk source gets rewritten as well.

So, any CloudTrail event flowing through the pack has that chance of losing its identifier. We reported the mechanism upstream on 7/20/2026 (criblpacks issue #9); it is unfixed in the current release. You decide whether it's significant or not, but we thought you'd rather know about this.

What the run adds up to

The pack tells you about the first one. Nothing tells you about the other two. All three are on the current release, deployed and measured.

All right, so what?

First off - check your environment. Three quick checks if this pack (or any similar reduction) is in your path.

Is the filter active? Unfiltered CloudTrail is mostly read-only calls - the vendor's own 60-80%:

index=<cloudtrail_index> sourcetype=aws:cloudtrail
| eval ro=if(match(eventName,"^(Describe|List|Get)"),1,0)
| stats count AS total sum(ro) AS read_only
| eval read_only_pct=round(100*read_only/total,1)

A ratio near zero means it is, and every detection consuming those events has nothing to read. Check which of your enabled detections that is, and make the tradeoff on purpose.

Has the mask eaten identifiers?

index=<cloudtrail_index> sourcetype=aws:cloudtrail eventID="*XXXXXXXXXXXXXXXX*"

And has it collapsed your accounts?

index=<cloudtrail_index> sourcetype=aws:cloudtrail
| stats count by vendor_account

(vendor_account is a search-time field from the AWS add-on, so tstats cannot group by it - this one has to be a plain search.) One value across a multi-account environment means attribution is already gone.

Second - and it's important - nothing here says "don't use Cribl" - it's a good product, and the savings are real. The point applies to any reduction layer in front of any SIEM: filters and masks are detection-engineering decisions, and when they ship as pipeline defaults, those decisions get made for you: your coverage changes, and nothing tells you - unless the breakage is really loud. It usually is not.

Note that the checks above are only a snapshot. The only way to know what a reduction actually costs - this week, after the next pack upgrade, after someone edits a pipeline - is the boring way we did it here: run the attacks through the full path, uphill both ways, and count what still fires.

Disclosure

The eventID corruption was reported upstream a month before this publication (criblpacks issue #9, 7/20/2026) and is unfixed in v1.2.0. Two additional defects in the same account-ID masking function were reported on 9/1/2026:

• Issue #10: the match removes the character preceding an account ID, producing structurally invalid ARNs.
• Issue #11: only the first account-ID occurrence in a field is replaced.