Today we are releasing three new members of the Ternary Bonsai 2 family. Built from Qwen 3.5, they come in 2B, 4B, and 9B varieties and are more than 9x smaller than their full precision counterparts.
An advantage of having open sourced our previous models is the feedback that we have received (both good and bad). The community has stretched our models in every which way, exposing flaws, some of which, would have taken months of internal evaluation to reveal.
We have done our best efforts to address these. In particular, we have focused on what makes small models useful as local assistants. We have improved instruction-following and tool-calling on mobile devices, while attempting to avoid repetitive loops and providing useful answers without unnecessary output.
All three models are under Apache 2.0.
Today we're excited to extend the Bonsai 2 family of models with ternary 2B, 4B, and 9B models based on Qwen 3.5. These may not solve IMO problems or the Riemann hypothesis but are incredible for high volume and high speed tasks on everyday devices.
The Bonsai 2 4B model runs 32 concurrent streams on my Macbook Pro (M5 Pro) to process 250 emails and parse them into JSONs with summaries, action items, calendar events, and priority scoring. The peak memory use is less than 4GB and in total generate over 700 tokens per second.
Bonsai 2 27B is now hitting 237 tokens/s on a Mac!
The open-source community keeps pushing the limits — and there are still 5 days left in the challenge.
https://t.co/j4oaN5abwb is launching a 1 week contest to improve Bonsai 2 from @PrismML! Bonsai 2 is a near losslessly comrpessed version of Qwen 3.8 27b that is around 10% the size of the original. It can fit on 16gb Apple devices and even some phones.
This is a model almost anyone can run so if you weren't able to participate in the other contests, this one is for you! The community has been able to double the speed of Bonsai in under 12 hours so there is alot of headroom here.
This contest runs until October 1, have fun!
Feedback 1–2: help our pretend desktop wake up.
First, the page got stuck at startup. Then the icons said “undefined” and the app windows wouldn’t appear.
We described what happened, and Bonsai worked on those issues while keeping its original look.
It's barely a week and the response to our Bonsai 2 27B has been quite remarkable. The community has pushed and pulled the model in almost every which way.
A great thing about open source is that everything gets exposed: both strengths, as well as places where the model can be improved. And so we at PrismML have learned A LOT about Bonsai 2 27B in just the past few days.
We thought it would be useful to share some use cases that we have started exploring.
More to come and, as always, feedback is welcome
We asked Bonsai 2 27B to build a little pretend OS in the browser. Meet BonsaiOS 🌱
Windows, a simulated terminal, notes, and a calculator—all in one HTML file. A fun coding demo, with quirks included.
Here’s how it came together through Hermes and five feedback rounds. 🧵
The team has been hard at work putting together some best practices and demos of Bonsai 2! Give the model a shot with the recommended settings!
https://t.co/M60MjDpRWQ
We have been very excited to see how the community has been utilizing our Bonsai 2 model, well beyond use cases that we had originally envisioned it for. We’ve spent the last few days putting Bonsai 2 27B through various examples inspired by what the community has shown us.
These examples showcase the strengths, and some of the shortcomings–particularly on longer multi-turn agentic workflows, and will be extremely helpful as we continue to improve.
We’ll be sharing a few of those demos, along with some practical guidance on the settings and prompting that get the best results from the model.
Demo repo: https://t.co/saIwOz7poP
@BSchulse I tested Gemma 31B 4-bit QAT on IMO as well. Using the same rubric it scored 9/42. It seems like Gemma generally thinks less than the Qwen and Bonsai models, so I'm still experimenting a bit to see if we can make it think more deeply and whether that boosts the scores.
We just released Ternary Bonsai 2 27B: 5.9 GB of weights for a 27B model. How much reasoning survived compression? We test it on a few evals that I found interesting and discuss where it retains the performance of the base model and where there remains room for improvement. The models were provided with no internet, no tools, and only a minimal harness for coding problems. Unless otherwise specified, each model used the default serving config on llama.cpp. Both Bonsai and Qwen used extra-high reasoning. Gemma 4 has only one reasoning effort.
My first test is IMO 2026, which was released after the training cutoff for all three models, against full-precision Qwen3.8 27B (54 GB) and Gemma 4 12B QAT (~7 GB). Each model was given a 131k thinking context after which it was forced to output an answer.
Qwen and Bonsai fall in the upper-end of the human bronze range (16-22). Bonsai retained 95% of the IMO 2026 score of the Qwen full precision model, completed the problems in ~70% of the time, but used ~22% more thinking tokens.
@BSchulse The 12B 4 bit QAT has a similar memory footprint as ternary Bonsai 27B. I'll update the repo with results from the larger Gemma's if you're curious though!
@LeTechLead Yeah. We noticed that the model is a bit sensitive to sampling parameters for agentic coding. The team is doing some testing and will follow up on that and release some more info on what seems to work well for Bonsai. Stay tuned!
Am pleased to announce a 1-bit 2B Bonsai model running locally on an AI smart glasses at Qualcomm’s Snapdragon summit. Powered by Snapdragon AR1 Gen 1 this is the first time a 1-bit multi-modal model runs on a personal AI platform. And many more to follow….
@ShiroCipher We published a more comprehensive list of evals here. https://t.co/YCEVdIXatB
If you have any suggestions for anything to add let us know!
@mktpavlenko Absolutely. These experiments were run without any KV cache quantization (FP16). The peak use was roughly
19.6 GiB for Bonsai 2
60.9 GiB for Qwen 3.8
10.9 GiB for Gemma 4
Stay tuned for more aggressive KV cache quants and benchmarks though!
Based on community feedback, the PrismML team is committed to enhancing the usability of future releases by optimizing token efficiency and self-correction capabilities over longer horizons. More interesting examples and tests to follow.