The Sequence Knowledge - Issue 924: The Distilled Models You Need to Know About
From DistilBERT and Gemini Flash to Gemma, Llama, Qwen, DeepSeek, Phi, Mistral, and PrismML's Bonsai 27B.
TL;DR
- Bonsai 27B, released by PrismML in July 2026, significantly reduces hardware needs compared to conventional models of similar parameter count.
- The ternary version of Bonsai is around 5.9 GB, and the binary version is approximately 3.9 GB, designed to fit on high-end phones.
- Bonsai is multimodal, supports long context, and retains reasoning and tool-use behaviors from its full-precision origin, Qwen3.6-27B.
- PrismML emphasizes end-to-end low-bit training and quantization rather than classical teacher-student distillation for Bonsai.
- The article suggests Bonsai is a convergence of distillation, pruning, quantization-aware training, and systems engineering.
- The concept of AI model lineages, rather than individual checkpoints, is becoming increasingly important.
- This lineage concept traces a frontier model's capabilities to smaller, specialized versions, including low-bit packaging for devices.