Datadog's Open Source AI SAST: 90% Detection, Fewer False Positives

Traditional SAST tools don’t fail because they miss vulnerabilities — they fail because they flag so many false positives that developers stop trusting them. Datadog’s open source AI-native SAST addresses the right problem: not detection rate in isolation, but detection rate combined with false positive suppression.

Context

Static application security testing has a trust problem. Rule-based SAST tools produce high false positive rates on context-dependent vulnerabilities — injection flaws, deserialization issues, and cross-site scripting patterns that require understanding data flow across files, not just pattern matching within a single function. When security findings are unreliable, developers build habits of dismissing them. Alert fatigue in SAST tooling is not an edge case; it’s the documented default behavior at scale.

The timing matters. Platform and security engineering teams are simultaneously dealing with an increase in AI-generated code in their repositories — research has documented vulnerability patterns specific to LLM-generated code — which creates new pressure on detection tooling that wasn’t designed to handle unfamiliar code generation patterns. A SAST tool that can reason about data flow context across files, rather than matching known syntactic patterns, has meaningfully different properties in this environment.

The Detail

Datadog’s tool operates through a four-stage pipeline. Identification uses heuristics to filter candidate files most likely to contain vulnerabilities — this is cost management, not detection. Context retrieval gathers related files and function calls to give the LLM enough scope for accurate data flow analysis. Analysis sends the contextualized code to a language model that identifies potential vulnerabilities and provides reasoning for each finding. Post-processing applies pattern matching and secondary LLM filtering to eliminate false positives before surfacing results to developers.

Against OWASP Benchmark — the standard evaluation suite for SAST tools — the results are significant. Command injection detection improved from 59% to 90%. SQL injection from 63% to 86%. XSS from 65% to 93%. These aren’t marginal improvements; they represent a qualitative shift in the reliability of findings for the vulnerability classes most commonly exploited in production environments.

The cost architecture is pragmatic. Repeated LLM calls at scale are expensive, and Datadog’s solution is incremental scanning — after initial repository onboarding, only changed files are analyzed in subsequent runs. This aligns SAST costs with CI/CD activity rather than repository size. Datadog monitors the tool’s own costs using its LLM Observability product, tracking latency, token usage, and estimated spend per scan — the same observability stack it sells to customers.

The agentic direction is also notable. Datadog explicitly flags plans to explore “more agentic scanning techniques” in future development. This signals a trajectory toward SAST that doesn’t just find vulnerabilities but participates in the remediation loop — a natural extension as MCP-based security tooling matures in enterprise CI/CD pipelines.

Source: Datadog

Content created with AI assistance and reviewed for accuracy.