@PeteCapeCod@CodeWithTamara If I remember correctly Cape Cod is in Massachusetts, Tamara is 6hrs ahead of us. (im near DC) so 6pm her time is 1200 noon ours. But your point is well taken.
@bryanm818 What kind of "technical" partner are you looking for? I'm not a software developer but I understand the SDLC the basic is "you have 3 options, on-time, on budget, or look pretty, choose two" My background is systems admin and integration support engineering.
@CodeWithTamara This could meet some of your concerns regarding using local AI on your laptop. Put this in combo with the other post i sent you on tool usage. And we could build a tool that meets several of that grant orgs product wish list.
Qwen3.8-27B has reportedly been compressed from a 55.6GB checkpoint to just 5.9GB, while retaining 98.2% of its original aggregate benchmark performance.
That’s an 89.4% reduction in storage.
The compression system behind this is Bonsai 2, developed by PrismML.
What’s particularly interesting is the improvement over the first generation. The original Bonsai release reportedly retained around 95% of the model’s benchmark performance. Bonsai 2 raises that figure to 98.2% across 20 benchmarks, without increasing the compressed checkpoint size.
Here are the reported results:
Original Qwen3.8-27B: 55.6GB
Bonsai 2 checkpoint: 5.9GB
Storage reduction: 89.4%
Aggregate benchmark retention: 98.2%
Previous Bonsai version: Approximately 95%
Evaluation coverage: 20 benchmarks
That’s a 3.2 percentage-point improvement in reported performance retention.
How is PrismML achieving this?
1. Ternary quantisation is at the centre of Bonsai 2
Bonsai 2 uses ternary quantisation, representing model weights with just three possible values:
-1
0
+1
Conventional formats such as FP16 and INT8 use more bits to represent numerical values. Ternary representations drastically reduce the storage required for individual weights.
However, there’s a problem.
Model weights encode numerical relationships learned during training. Those relationships influence everything from language understanding and reasoning to coding and instruction following.
Reducing numerical precision too aggressively can destroy some of that information.
Bonsai 2 combines ternary weight representations with scaling factors to recover useful numerical ranges from these compact values.
The objective is to reduce storage requirements while preserving as much of the original model’s learned behaviour as possible.
2. The performance retention is what makes this interesting
The previous Bonsai generation reportedly retained approximately 95% of the original model’s aggregate benchmark performance.
Bonsai 2 increases that to 98.2%.
PrismML also highlights improvements in areas including:
Coding agents.
Multimodal reasoning.
Long-horizon tool use.
The agentic side deserves particular attention.
A coding agent needs to inspect repositories, understand dependencies, modify files, execute commands, interpret errors and recover when something goes wrong.
That requires more than generating code snippets. It requires maintaining context and making useful decisions across multiple steps.
Preserving these capabilities in a 5.9GB checkpoint could make compressed models more accessible for developers experimenting with local AI agents.
However, benchmark retention alone doesn’t establish that long-running agent workflows perform almost identically to the original. That needs dedicated evaluation.
3. What does this mean for local AI?
Reducing the checkpoint from 55.6GB to 5.9GB changes the storage requirements considerably.
Smaller checkpoints are easier to download, distribute, archive and experiment with. They could also make it practical to explore models that would otherwise be inconvenient to manage locally.
But there’s an important distinction:
A 5.9GB checkpoint does not mean the model requires only 5.9GB of RAM or VRAM to run.
Inference requires additional memory for:
KV cache.
Activations.
Runtime buffers.
Other allocations required by the inference engine.
The KV cache is especially important because its memory consumption increases with context length.
Actual requirements will depend on the hardware, inference engine, context size and batch configuration.
There’s another factor worth investigating: whether the inference engine has efficient kernels for ternary weights. A smaller checkpoint doesn’t automatically translate into faster token generation.
4. Does 98.2% retention mean identical performance?
No.
$42,000,000 for open-source AI builders from @sentient_found
The Sentient Foundation has committed $42M to people shipping open AI products.
Two tracks:
Grants for open-source maintainers, researchers and public-goods builders. They take no equity and have no lockups.
Investment for teams building open AI companies.
They want things like offline medical guidance, scam protection, multilingual tutoring, on-device personal AI and privacy-preserving compute. The page lists 21 product requests.
Grantees also get compute credits, engineering help and support with distribution.
There are no cohorts and no deadlines. A technical panel reviews every application.
https://t.co/NisO9596U0
@CodeWithTamara Here is an example of what I was talking about in our discussion today. Granted this may not work for everyone, but the idea is solid. There is always a way past road blocks.
So now a small 300MB adapter makes your local 27B coding model dramatically better!
Hugging Face released an RL-trained LoRA adapter for Qwen3.8-27B, specifically optimized for OpenCode.
And the reported results are impressive!
SWE-bench Pro (hard 51 tasks):
🔴 Original → 35.3%
🟢 With LoRA → 51.0% 👈
Terminal-Bench 2.1:
🔴 Original → 46.4%
🟢 With LoRA → 52.4% 👈
Median agent turns on tasks both solved:
🐌 Original → 48 turns
🚀 With LoRA → 13 turns 👈
That's a 15.7% improvement on the selected SWE-bench Pro tasks and 73% fewer agent turns on shared solves.
With thinking disabled, the adapted model reportedly approaches or exceeds the original model running at maximum reasoning effort, using 6–8× fewer generated tokens per task in the team's comparison.
The best part?
💾 Adapter → ~300MB
🧠 Base model → Qwen3.8-27B
🔓 License → Apache 2.0
💻 Local inference → llama.cpp
🛠️ Coding harness → OpenCode
📦 GGUF adapter available
Just a small adapter teaching an existing model to use coding tools more effectively.
Video source: @HuggingFace
Had a great and engaging meeting / video call with @CodeWithTamara today, we discussed use cases for AI in work and daily life as well as how each of us are using it and how we can help others put this tech to use for themselves. I was very engaging and well executed meeting.
@onorma_ Yes, but we called it "business Machines" and we used and IBM typewriter that had a 1.44 MB floppy on the right hand side to save work to. I remember it had a ball stamp instead of the normal arm type stamps for the letters.