Josh Bolding

I build things to find out how things work. Local AI on consumer GPUs. Measure, don't assume. Currently: what's actually inside quantized LLMs.

[Josh Bolding @BoldingBuilds]

Abliterated Qwen3.8-27B picks an answer on sensitive prompts, then keeps arguing with itself until it hits the token cap.

ThinkFix is that abliteration plus one output row edit so it closes thinking when it's ready.

40 held out prompts, 8k cap, same weights: clean finish 22 → 33 hit the limit 17 → 5 75 min → 61 min median thinking ~3,000 → ~1,700 tokens

MATH / IFEval about flat. MTP kept, 56 tok/s vs 39 on a 3090. Q4_K_M is 16.8 GB.

https://t.co/0TLQUMFPZL

[Josh Bolding @BoldingBuilds]

@iMichaelTen Appreciate it! I’m kind of all over the place with my projects haha

[Josh Bolding @BoldingBuilds]

Been working on turning Bonsai 2 into a local decision model. Give it a question and some options, and it gives you a probability for each answer.

Trained my own adapter today and the results look pretty good so far. Running a sample of Decision Index now to see how it holds up on a wider range of questions.

I was going to release it today, but I keep finding one more thing I want to test. Figured I should at least show what I’m working on.

[Josh Bolding @BoldingBuilds]

@Alibaba_Qwen @Tech2Wild Bring it on!

[Josh Bolding @BoldingBuilds]

@bennyjohnson Hahahahahahahahahahahahahhahahahahahahaha!!!

[Josh Bolding @BoldingBuilds]

@factpostnews Why do I only see this guy around election time, definitely not who you’d imagine as a republican spokesman

[Josh Bolding @BoldingBuilds]

@realDonaldTrump Your endorsement and meddling is why these candidates will lose.

[Josh Bolding @BoldingBuilds]

@RealAlexJones Did this hurt the drone?

[Josh Bolding @BoldingBuilds]

@loktar00 Puskill *Halloween Edition* -Almost as scary as RAM prices

[Josh Bolding @BoldingBuilds]

@callme_e @zacodil Does a painter use a brush?

[Josh Bolding @BoldingBuilds]

Took the case and screen protector off my iPhone 17 Pro Max for the first time since the day I bought and it feels so much better!

[Josh Bolding @BoldingBuilds]

@RyanAFournier This must be why there's a disconnect getting Talarico and Paxton on a debate stage together. Republicans seem to think debate is speaking from a podium to an arena of people who agree with them.

[Josh Bolding @BoldingBuilds]

@haider1 Its results in my testing is oddly similar to Astra results, still need to play with it more

[Josh Bolding @BoldingBuilds]

Come on Kimi! Release a 27b or some other small model!

[Josh Bolding @BoldingBuilds]

@Kimi_Moonshot Awesome numbers!

And props to you guys for putting where it scored worse in the 3rd slot

[Josh Bolding @BoldingBuilds]

@Hikari_07_jp You could try cooking the steaks on the GPUs next time 😅

[Josh Bolding @BoldingBuilds]

@Hikari_07_jp Thats so true! For how little its used it sure takes up a lot of real estate on the keyboard. Im going to try and figure out a good use for it too now thanks!

[Josh Bolding @BoldingBuilds]

@claudeai So we got GrokBot Dot and Duet

[Josh Bolding @BoldingBuilds]

@pastors4paxton @KenPaxtonTX @Steve_Riggle @BuntrockJames Im surprised he didnt melt or get struck by lightning

[Josh Bolding @BoldingBuilds]

@loktar00 I’m glad I secured my 4x but it’s gotten so high I think I’m stopping at 4. I wish I could keep buying compute/vram every couple months but it’s just harder and harder to justify these hardware prices vs cloud, and I think that’s part of the point

[Josh Bolding @BoldingBuilds]

@withergyld Thanks! Yea these are more a functional art piece. I just wanted to do something very different than I had seen before