Bonsai 27B: A 5.9GB LLM That Keeps 95% of Its Reasoning on Your iPhone

PrismML has released Bonsai 27B, a ternary-quantized large language model that compresses Alibaba’s Qwen3.6-27B from its original size to 5.9 gigabytes — small enough to run on a laptop without a GPU, and with a 3.9-gigabyte iPhone variant that runs on-device with no server call required. The compression technique preserves nearly all the model’s reasoning capability across math and coding benchmarks.

What PrismML Built

PrismML is a Caltech spinout backed by Khosla Ventures and Google. Their ternary quantization approach reduces each model weight to one of three values (−1, 0, or +1) rather than the full floating-point precision used during training. The result is dramatic compression: the 27-billion-parameter model that would normally require substantial GPU memory to run fits in under 6 gigabytes. A further-optimized iPhone variant fits in 3.9 gigabytes — within the on-device memory budget of current iPhone hardware.

The base model is Alibaba’s Qwen3.6-27B, which sits in the upper tier of open-weight models for reasoning, coding, and instruction following. PrismML’s contribution is the quantization and inference optimization that brings it to consumer hardware without meaningful performance degradation on the benchmarks that matter most for practical use: mathematical reasoning and code generation.

Why Ternary Quantization Matters

Standard model compression approaches — INT4 or INT8 quantization — reduce precision but retain a range of discrete values per weight. Ternary quantization is more aggressive: each weight carries essentially one bit of sign information, plus a zero state. The counterintuitive finding from PrismML and prior academic work is that ternary networks can retain most of a model’s capability if the quantization is applied carefully during training rather than as a post-processing step. The capability is preserved in how the weights work together, not in the precision of individual weights.

The practical implication for the 27-billion-parameter scale: a model with state-of-the-art reasoning capability can now run on an iPhone without a network call. For applications that require low-latency inference, data privacy (the model processes locally and never sends input to a remote server), or offline capability, this changes what on-device AI can do.

What This Means for Agentic AI

The most direct agentic implication is local reasoning without network dependency. Agents running on-device — in iOS apps, in edge computing environments, in air-gapped enterprise deployments — have historically been limited to smaller models that traded capability for size. Bonsai 27B is the first model at this parameter count that fits the on-device constraint without giving up the reasoning and coding performance that agentic task execution requires.

The secondary implication is privacy. An AI agent that processes user data, documents, or communications without sending any content to a remote endpoint changes the regulatory calculus for deployments in healthcare, legal, financial services, and enterprise environments where data residency rules limit what can leave the device or network boundary. A 27B-parameter reasoning model that runs entirely on-device satisfies those constraints in a way that API-based alternatives structurally cannot.

The 3.9GB iPhone variant fits on current-generation iPhones with memory to spare for the application context and inference buffer. The remaining question — latency and throughput at this compression level — is what the benchmark reports will need to confirm under real workload conditions.

Source: PrismML

Content created with AI assistance and reviewed for accuracy.