Thomas Konings

Doing cool AI stuff. Consultant by day.

[Thomas Konings @tkon99]

You don't need $10k worth of GPUs to do cool local AI stuff.

Just ran embeddings and feature extraction on my RTX 4070 Super 12GB. NuExtract3 + nomic-embed.

GLM 5.2 optimizing the extraction as we speak. Benchmarking different speculative decoding options.

[Thomas Konings @tkon99]

@PrismML Not just editing files but navigating reference websites using Playwriter, taking text and assets from them and doing a full modern redesign. Honestly surprised.

[Thomas Konings @tkon99]

Finally had the time to test Bonsai by @PrismML out. On my mere RTX 4070 Super I get 45 t/s output. Quick addition to ZCode and it's building websites, the thing is incredibly agentic for the hardware it's running on!

It's going to be an awesome year for Local AI.

[Thomas Konings @tkon99]

This chart from the Bonsai model release by @PrismML will be all over LinkedIn tomorrow :)

Intelligence density is going to be a huge benchmark going forward, matching the incentives created by memory shortages.

Imagine harnessing all the memory we have out and about already.

[Thomas Konings @tkon99]

This chart from the Bonsai model release by @PrismML will be all over LinkedIn tomorrow :)

Intelligence density is going to be a huge benchmark going forward, matching the incentives created by memory shortages.

Imagine harnessing all the memory we have out and about already.

[PrismML @PrismML]

Raw capability determines what a model can do. Intelligence density determines where it can do it.

Bonsai 27B moves the Pareto frontier left again: 27B-class capability in a footprint smaller than many full-precision 2B models.

By intelligence density, 1-bit Bonsai 27B delivers 0.53 per GB - more than 10x the full-precision baseline and roughly 2.7x the best conventional low-bit alternative.

[Thomas Konings @tkon99]

Did some more tinkering and now at 10 slots doing 354 t/s! Cool stuff and should run through my 1000 text extracts quickly.

[Thomas Konings @tkon99]

You don't need $10k worth of GPUs to do cool local AI stuff.

Just ran embeddings and feature extraction on my RTX 4070 Super 12GB. NuExtract3 + nomic-embed.

GLM 5.2 optimizing the extraction as we speak. Benchmarking different speculative decoding options.

[Thomas Konings @tkon99]

6 seems to be the sweet spot. Cut down the run time from 2.5 hours down to 1 hour.

Great to be using local models for extraction tasks, really not something I want to be using premium tokens on.

[Thomas Konings @tkon99]

👀The Codex App for Windows has the codex:// protocol to hotlink to skills, settings, automation, projects and even pass prompts. Seems to be disabled by default but enabling is super easy: https://t.co/SOo2SBiM7u

[Thomas Konings @tkon99]

Read my latest: “Understanding Hydrogen - MMM #33” https://t.co/7OkLNQUQOV

[Thomas Konings @tkon99]

Read my latest: “Circular Philosophy - MMM #32” https://t.co/OQVsZtUa2R

[Thomas Konings @tkon99]

Read my latest: “Mind the CHIPS - MMM #31” https://t.co/YuVlulfc7D

[Thomas Konings @tkon99]

Read my latest: “Free Full Stack Modeller this Week - MMM #30” https://t.co/Lexit7GB9r

[Thomas Konings @tkon99]

Read my latest: “Snap Snapped- MMM #29” https://t.co/6yk41jNWHb

[Thomas Konings @tkon99]

Read my latest: “Push for Parity - MMM #28” https://t.co/lqZQDb4Btb

[Thomas Konings @tkon99]

Read my latest: “Another True Mashup - MMM #27” https://t.co/B0vgLiank1

[Thomas Konings @tkon99]

Read my latest: “Fear, Greed, and Risk Capital - MMM #26” https://t.co/kzeDVAmGAk

[Thomas Konings @tkon99]

Read my latest: “Crypto Credit Crisis - MMM #25” https://t.co/1X7zsO7f9V

[Thomas Konings @tkon99]

Read my latest: “Greener Plants - MMM #24” https://t.co/nNcVzc42xX

[Thomas Konings @tkon99]

Read my latest: “Fuel(ing) Innovation - MMM #23” https://t.co/cz5Qdl6kxi