What Datadog Learned About AI Agent Security by Monitoring Its Own Agents

Datadog turned its observability platform on its own AI agents and published what it found. The results are a useful corrective to security models built around monitoring model inputs and outputs as if those were the only attack surface.

What Happened

Datadog’s internal AI agent security monitoring project applied multi-layer observability to its own agentic workflows — tracking component inventory, prompt and tool execution correlation, sensitive data classification throughout sessions, identity attribution, and behavioral sequence analysis. The team published the findings as lessons for the broader platform engineering community.

Three findings stand out for their practical implications:

System prompts account for 69% of input tokens — and most teams cannot inspect them. The system prompt is where agents receive their instructions, permissions, and tool access specifications. At 69% of total input token volume, it is also the largest attack surface for prompt injection. Organizations monitoring AI agents typically focus on user-facing inputs; this finding suggests the system prompt layer requires equivalent scrutiny.

Coding agents execute repository-controlled code before the first user message arrives. This is the sequence problem: when a coding agent loads context from a repository, it processes whatever that repository contains as part of setup — including any malicious instructions embedded in configuration files, comments, or documentation before a human ever sends a prompt. The attack surface begins at repository access, not at user input.

A demonstrated prompt injection bypassed model evaluation entirely. Datadog’s research showed that a malicious Claude Code skill could execute prompt injections before the model’s own evaluation layer activated. The injection ran before the model could assess whether to comply. Standard “model will refuse harmful instructions” assumptions do not apply when the injection precedes evaluation.

Why It Matters

Enterprise AI agent deployments typically inherit a security model from traditional software: monitor inputs, monitor outputs, alert on anomalies. Datadog’s monitoring revealed that this model is incomplete for agentic systems where the most sensitive data flows happen in the middle of execution — in tool calls, RAG results, and system prompt content that standard input/output monitoring never captures.

A specific customer support scenario from the research illustrated the exposure: an ordinary user request could trigger a tool call that exposed payment information to downstream systems before authorization checks engaged. The vulnerability was in the tool execution sequence, not in the prompt or the response.

What This Means for Platform Teams

Datadog’s four recommendations are concrete:

1. Inventory everything. Record exact model versions, tools, connected services, and dependencies including LLM proxies. You cannot monitor what you have not inventoried.
2. Connect the execution path. Track relationships between prompts, retrieved content, tool calls, and downstream actions — not just the start and end states.
3. Prioritize by exposure. Focus guardrails on customer-facing agents and those accessing sensitive systems first.
4. Contextualize detections. Build alerts around behavioral sequences rather than isolated events. A single unusual tool call may be noise; a pattern of unusual tool calls is a detection signal.

Source: Datadog

Content created with AI assistance and reviewed for accuracy.