Coinbase Runs 1,200 AI Agents — and Cut Its AI Bill in Half
Cloud Platform EngineeringCoinbase is running 1,200 full-time AI agents. Token usage grew exponentially. The AI bill fell by half. Those three facts belong in the same sentence because they describe the same infrastructure decision.
The Multi-Model Strategy
CEO Brian Armstrong outlined the infrastructure playbook publicly: abandon single-vendor AI strategies and route every prompt to the cheapest model capable of handling it. The three-part implementation is direct. First, smarter model routing — matching each task to the minimum-capable model rather than defaulting everything to frontier providers. Second, aggressive caching — eliminating redundant outputs for repeated queries at scale. Third, a systematic shift toward open-weight models for any task where frontier capability adds no measurable value.
The model choices made the difference concrete. Coinbase defaulted engineers to Chinese open-weight models — GLM 5.2 and Kimi K2.7 Code — for tasks where those models perform equivalently to frontier providers at a fraction of the cost. Armstrong projects that 80% of workloads will run on models that cost 99% less than today’s frontier within 12 to 18 months. The routing infrastructure being built now is what makes that projection executable rather than aspirational.
The Scale Implications
The engineering team restructuring is the signal that gets underestimated. Teams that previously required ten people now operate with two, with code output doubling. The agent fleet doesn’t just run in parallel with engineers — it changes what the engineer-to-output ratio looks like at a fundamental level. That’s not automation incrementalism; it’s a structural change to how software organizations should be sizing teams relative to output goals.
The cost model that makes this viable is also the one creating risk. Using Chinese open-weight models for production workloads carries export control and data sovereignty considerations that are separate from performance. Routing production code generation through models like GLM 5.2 means data touches infrastructure outside US jurisdictional control. For regulated industries or companies handling sensitive IP, the cost savings calculation includes a legal risk component that the raw cost numbers don’t surface. This tradeoff is navigable but it’s not invisible.
The So What
The Coinbase infrastructure playbook is reproducible at most organizations, but the relevant lesson isn’t “use cheaper models.” It’s that the routing layer is the actual engineering asset. Companies that build model-agnostic routing infrastructure now own the optionality to shift across providers as the model landscape evolves. Companies that wire directly into a single frontier provider inherit that provider’s pricing and capability ceiling. The 12-to-18-month window Armstrong describes — when 99%-cheaper models handle 80% of workloads — favors teams that already have routing infrastructure in place. Related: how AI agent token pricing is becoming a business metric.
Source: The New Stack
Content created with AI assistance and reviewed for accuracy.
Join the conversation
Stack Insiders is our free community for readers who want to go deeper — share resources, ask questions, and connect with others across every vertical we cover.
Join Stack Insiders →