Datadog's Mambark: A 97M-Parameter Filter That Cuts AI Security Costs 10,000x

Running a frontier AI agent over every security event a large fleet generates sounds thorough. It’s also, by Datadog’s own accounting, a way to spend tens of millions of dollars a day finding almost nothing new. Datadog’s answer is a small model that does the boring work first, so the expensive model only ever sees what’s worth its attention.

How It Works

The model is called Mambark, and it’s deliberately not an LLM. It’s built on a Mamba selective state-space model — roughly 96.9 million parameters, versus the hundreds of billions in a frontier reasoning model. That size difference is the point: state-space models scale linearly with sequence length instead of the quadratic attention cost transformers pay, so Mambark can score sequences spanning tens of thousands of events on a single commodity GPU. Datadog normalizes syslog data, authentication records, network flows, and insider-threat signals into a shared hashed-vocabulary format, letting one backbone generalize across event types without hand-built feature engineering per source.

The pipeline is two stages. Mambark scores all events in a single forward pass, promoting roughly 10,000 candidates a day out of 10 billion — for a cost Datadog puts at tens of dollars daily on commodity hardware. Only those flagged candidates go to an AI agent for full multistep investigation, which also runs at tens of dollars a day. Applying that same agent to the raw stream instead of the filtered candidates would cost four orders of magnitude more. It’s the classic information-retrieval pattern — a fast, high-recall retriever feeding a slow, expensive reader — applied to security telemetry instead of documents. Datadog reports Mambark hits state-of-the-art or near-SOTA results across public benchmarks like Thunderbird, BGL, and UNSW-NB15, which matters because a cheap filter that misses real threats isn’t actually cheap.

The approach echoes what Datadog’s own BewAIre malicious-package detector already does for supply-chain scanning: a lightweight stage exits the easy majority of cases cheaply, and an agentic investigation loop only runs on what survives the filter. Mambark generalizes that pattern from package scanning to arbitrary security event streams.

What This Means for You

The constraint Datadog is describing isn’t specific to security. Anyone building AI agents over a high-volume event stream — infrastructure logs, transaction records, user activity, sensor data — hits the same wall: reasoning over everything is unaffordable, but reasoning over nothing misses the incident that matters. Datadog’s own framing is explicit about this: “you can’t afford to apply expensive AI reasoning to every event, but you also can’t afford to miss the ones that matter.”

The architectural lesson generalizes better than the specific model does. A small, purpose-trained scorer sitting in front of an expensive general-purpose agent isn’t a workaround for immature AI — it’s likely the durable shape of production agentic systems at scale, the same way search engines have always paired cheap retrieval with expensive ranking rather than running relevance models over an entire index per query.

It also reframes what “AI-native” tooling should mean for platform teams evaluating vendors. The interesting engineering decision in Datadog’s pipeline isn’t which frontier model powers the investigation stage — that part is comparatively replaceable. It’s the pre-screening layer that decides what the expensive model is even allowed to see. Teams building similar systems in-house should budget engineering time for that filter first, not treat it as an afterthought once the agent is already live and the bill has arrived.

Source: Datadog

Content created with AI assistance and reviewed for accuracy.