GLM-5.1: The First Open-Weight Model to Lead SWE-Bench Pro
Agentic AIAn open-weight model just took the top position on SWE-Bench Pro — ahead of Claude Opus 4.6 and GPT-5.4. Z.AI’s GLM-5.1, released April 7, 2026 under an MIT license, is a 754-billion parameter Mixture-of-Experts model engineered specifically for long-horizon agentic tasks, with a documented capability to sustain autonomous operation for up to eight hours on a single engineering objective.
Context
The pattern for frontier AI capability has been consistent for the past two years: closed, proprietary models lead benchmarks, open-weight models lag by a meaningful margin. GLM-5.1 breaks that pattern on the benchmark that matters most to enterprise engineering teams. SWE-Bench Pro — the software engineering benchmark that tests real-world code understanding, debugging, and generation — now has an open-weight model at the top of its leaderboard.
The significance is not just the score. It is the license. MIT-licensed models can be self-hosted, audited, fine-tuned, and deployed without usage agreements or API rate limits. For enterprise teams where code generation involves proprietary systems, competitive infrastructure, or regulated data, that distinction has been the structural barrier to adopting frontier-capable AI for engineering work.
The Detail
GLM-5.1 is a 754-billion parameter Mixture-of-Experts model from Z.AI (Zhipu AI’s international brand). Its SWE-Bench Pro score of 58.4 puts it above both Claude Opus 4.6 and GPT-5.4 — a clean reversal of the typical open/closed capability ordering. The 8-hour autonomous operation claim is specific: the model is designed to maintain goal alignment across extended execution traces spanning thousands of tool calls, not just complete isolated tasks. VentureBeat’s framing — “AI joins the 8-hour workday” — accurately captures the design intent: this is sustained agentic execution, not a chat session.
The benchmark profile across agentic tasks is consistent: AIME 2026 scores 95.3, CyberGym 68.7, BrowseComp 68.0, τ³-Bench 70.6, and MCP-Atlas 71.8. The MCP-Atlas score is relevant given that MCP has become the standard integration protocol for enterprise agentic infrastructure — a model tuned for MCP-Atlas has been optimized for the tool-calling patterns that production agentic pipelines actually execute against.
VendingBench 2 tells a more nuanced story. The benchmark simulates running a business through tool calls across a full simulated year; GLM-5.1 finishes with $5,634, second to Claude Opus 4.6’s $8,017. Strong performance, but not dominant across every long-horizon scenario. The model’s strength concentrates in engineering tasks rather than general autonomous business decision-making — a distinction that matters when evaluating fit.
Self-hosting runs via vLLM, SGLang, xLLM, and Ktransformers. At 1.51TB total weight, this requires dedicated GPU infrastructure — not a workstation deployment. Teams evaluating it should note the community-documented observation that context quality degrades beyond approximately 100K tokens in practice. For agentic engineering tasks that generate large intermediate outputs, session design needs to account for that threshold.
The So What
For enterprise platform and DevOps teams evaluating AI engineering tooling, GLM-5.1 shifts the self-hosting calculus. A model that leads SWE-Bench Pro under MIT license removes the primary objection to building proprietary code pipelines around open-weight AI. The question changes from “can we achieve frontier performance in-house?” — now answered — to “do we have the infrastructure to run a 1.51TB model at production quality?”
For teams tracking the broader automation of knowledge work by agentic AI, the 8-hour autonomous operation design point is the detail with the most operational weight. Sustained agentic execution — not fast code completion, but hours-long independent task completion — is where the labor economics of engineering begin to change materially.
Photo by Daniil Komov on Pexels
Source: Simon Willison
Content created with AI assistance and reviewed for accuracy.
Join the conversation
Stack Insiders is our free community for readers who want to go deeper — share resources, ask questions, and connect with others across every vertical we cover.
Join Stack Insiders →