Goldman Sachs and the 12,000-Engineer AI Experiment: What It Actually Looks Like

Goldman Sachs is running one of the most scrutinized agentic AI deployments in the financial industry. Hundreds of autonomous agents now work alongside 12,000 human engineers — and the bank has changed how it measures success to match.

What Happened

According to Forbes reporting on Goldman’s CIO Marco Argenti, the bank deployed two primary platforms: Devin, Cognition’s autonomous coding agent, across the technology division for legacy infrastructure modernization, and Claude in 2026 for trades, transactions, and client onboarding workflows.

The operational metrics are striking. Devin delivered productivity improvements described as “three or four times more productive” than previous AI tools. Security vulnerability remediation dropped from 30 minutes per issue to 1.5 minutes. Code review acceptance rates improved from roughly one-third to approximately two-thirds of pull requests accepted without significant rework.

Why It Matters

Goldman’s framing of the deployment reveals something more significant than the productivity numbers: the bank shifted away from tracking adoption metrics toward outcome-based measurements. Argenti specifically described tracking how quickly concepts move from idea to prototype to production model — framing this as the ability to “3D print software.”

This measurement shift matters for enterprise AI buyers. Adoption metrics (seats provisioned, prompts executed, features used) are easy to game and don’t capture whether the organization is actually moving faster. Outcome velocity — the cycle time from idea to shipped production feature — is harder to measure but directly addresses the business case. Goldman effectively built the governance and measurement layer that most enterprises are still trying to figure out.

The agent-human ratio Goldman is operating at (hundreds of agents across 12,000 engineers) is still relatively modest, but the infrastructure decisions being made now determine how that ratio scales. The patterns that work at Goldman’s transaction volume and regulatory environment will look different from what works at a mid-market SaaS company, but the measurement approach is portable.

What This Means for Builders

Two implications are worth pulling out for teams building or evaluating agentic engineering platforms.

First, the legacy modernization use case is more tractable than greenfield development. Devin’s strongest performance at Goldman was on legacy infrastructure — a domain where the task is well-defined (modernize this code to these patterns), the validation is testable, and the tolerance for agent autonomy is calibrated by existing guardrails. Teams chasing fully autonomous greenfield development agents are solving a harder problem with less existing guardrail infrastructure.

Second, the talent pipeline risk is real and worth naming. Research cited in the Forbes piece suggests up to 200,000 banking sector jobs could disappear, with junior developer roles particularly at risk. The concern isn’t just humanitarian — it’s architectural. Junior developers who grow into senior engineers are where institutional knowledge forms. An organization that replaces entry-level engineering entirely with agents may be trading short-term productivity for a long-term knowledge deficit.

Source: Forbes

Content created with AI assistance and reviewed for accuracy.