Alibaba's Qwen-Drive Can Tell You Why It Braked — The Explanation Just Might Not Be True

Alibaba’s Qwen team built an autonomous driving model that explains its decisions in natural language. When researchers tested whether the explanations matched the actual driving decisions, they found the model would say one thing and do another.

What Happened

Qwen-Drive 1.0 is an open-source autonomous driving model built on the Qwen3.5-4B language model, released under Apache 2.0. The system adds two modules to the base LLM: one generates bird’s-eye-view maps of the environment through 3D object detection, and another handles trajectory planning. The architecture is designed to prevent catastrophic forgetting — the tendency of fine-tuned models to lose general reasoning capabilities when specialized for a narrow task.

On the benchmarks that matter, the model delivers genuine improvement. Off-road veering dropped from 24% to 12% after reinforcement learning. The model outperforms the base Qwen3.5-4B on driving-specific visual questions while maintaining general knowledge performance.

Why It Matters

The researchers’ own finding is the more significant result: the model’s explanations “don’t always pinpoint the actual cause of a situation,” and “the planned maneuver also doesn’t always match the reasoning the model gave beforehand.”

One documented case: the model conflated a distant red light with a child crossing the road — two situations requiring entirely different response times and maneuver types — and generated an explanation that described one scenario while executing a response calibrated for the other.

This is not a bug in Qwen-Drive specifically. It is a fundamental property of language models that are trained to produce explanations: the explanation is generated by the same model that produces the action, but the explanation generation process is not causally connected to the computational path that determined the action. The model learns to produce plausible-sounding explanations. It does not learn to introspect accurately.

What This Means for Builders

The same dynamic applies to any LLM-based agent that generates reasoning traces or explanations for its decisions. Chain-of-thought output, action rationales, function call justifications — all of these can be fluent, confident, and causally disconnected from what actually drove the model’s behavior. A model explaining why it made a choice is producing a plausible narrative, not a causal trace.

For safety-critical deployments, this means explanation quality cannot substitute for behavioral testing. You need to test what the model does under the conditions that matter. “The model says it would do X in scenario Y” is a weaker claim than “the model did X in 200 trials of scenario Y.”

Qwen-Drive 1.0 is available under Apache 2.0 — worth examining both for the autonomous driving application and as a documented case study in the limits of LLM self-explanation.

Source: The Decoder

Content created with AI assistance and reviewed for accuracy.