Balloonman

#LowCarb #AlgoBox #WAGMIGames #PhillySports #Surfing 12 year friend of Jimmy K!

[Balloonman @BuhloonMan]

https://t.co/9Y2oudc2Yi

I know not everyone in here trades, but if you do and you don't use AlgoBox.. you're missing out. Check out the Discord above. AlgoBox turns trading into a video game with cheat codes. Super-smart, helpful community in there. No joke.

[Hikari∣LocalLLM⚡ @Hikari_07_jp]

Here is a progress report on the development of the 35B-A6B model.
I managed to mitigate the performance degradation that occurs when increasing the number of experts—a key discovery. However, I am struggling to get the increased active parameters to perform effectively. Since I cannot do intensive work while away from my desk, I plan to focus on this once I get home.

[Hikari∣LocalLLM⚡ @Hikari_07_jp]

After finishing work today, I'm going to write an X article about what happened to me in the two months since my followers started growing.
Things I couldn't have imagined from my past self two months ago have happened. I'll write it targeting that past version of myself.

[Cary Kelly @CaryKelly11]

Nina is great!

Her book, @bigfatsurprise, is a must read for anyone who wants to understand how flawed dietary beliefs are and why.

The book was so profound that I read it six times.

[Hikari∣LocalLLM⚡ @Hikari_07_jp]

35B-A6B Model Development Progress Report!!
In A6B, we're expanding the number of experts used from 8 to 32. Previously we treated all 32 experts equally, but I've come up with the idea of prioritizing the top 8 experts. I've updated the research.
https://t.co/yRagBZ2U1e

[Hikari∣LocalLLM⚡ @Hikari_07_jp]

I want to build not just the model itself, but a fully consistent, production-ready stack—including the inference engine and speculative decoding.
For the 35B-A6B model, existing Draft heads won't work because the output distributions differ, causing the acceptance rate to drop sharply. So I'm building the Draft head myself as well.

I'm planning to wrap up support for the inference engine I've been working on and release it soon.

[Balloonman @BuhloonMan]

@CaryKelly11

[David Ondrej @DavidOndrej1]

if you don't have 25+ agents running at all times

you're not gonna make it

[Balloonman @BuhloonMan]

@sudoingX Well put

[Hikari∣LocalLLM⚡ @Hikari_07_jp]

🔥 35B-A6B Model Development Progress Report!! 🔥

Huge breakthrough — I finally got access to an absolute beast of a server with 768GB VRAM and 1.5TB RAM!! This changes everything. Massive leap forward for the project!

Right now I’m running full evaluations of the developing model simultaneously on my home lab + the workstation I’m borrowing from @x_xedo_x.RTX PRO 6000 will be 12 pieces in total and 1152GB.

Just blindly adding more experts only increases noise, so we’re going all-in on serious large-scale tuning. No shortcuts.

To everyone who shared my posts — thank you from the bottom of my heart. Because of you, this reached people who actually have spare compute lying around. Your help made this possible. 🙏

I’m pouring everything I’ve got into this. Gonna deliver a damn good model. Let’s keep pushing!!

[Hikari∣LocalLLM⚡ @Hikari_07_jp]

We performed additional training—specializing in tool
calling, coding, and mathematics—on the Gemma4 31B Draft model released by z-lab. Here is a demo video in which we have color-coded the tokens generated from a single token acceptance step, allowing you to intuitively see the highly efficient decoding process.

In this instance, we recorded an average speed of 219 t/s and a peak of 329 t/s. Since this Draft model is in BF16 format, we also plan to create an NVFP4 version.

Maintaining this crazy pace and continuing to ship. Look forward to it, everyone.

[Hikari∣LocalLLM⚡ @Hikari_07_jp]

Ten people have already suggested that I write a research paper. I have several ideas ready, and successfully building the 35B-A6B will add one more exciting topic to explore.

I'm also looking for opportunities at a university lab or similar environment where I can gain access to high-end hardware like the H200 or B200.

About a month ago, I used RepE to reduce the refusal rate of Gemma4 31B all the way to zero percent. Now it answers virtually any question you throw at it. The approach I developed to achieve this doesn't appear in any existing research papers.

At the core of everything, I'm fueled by a relentless drive to grow. I'm not satisfied with my current computing resources—my mind is consumed by LLMs and hardware optimization. The passion I'm pouring into this journey is real and unwavering. I have so much more I want to try, and I'm only going to keep pushing forward and growing.

Just watch me grow.

[Hikari∣LocalLLM⚡ @Hikari_07_jp]

My friend @x_xedo_x was incredibly kind to lend me two RTX PRO 6000s to support my research.
It’s amazing to see the great things that can happen when you keep sharing what you’re doing on X. This will really cut down on wait times.
Thank you so much🤝

[Hikari∣LocalLLM⚡ @Hikari_07_jp]

Click here for GLM5.2 RAM offload inference.
https://t.co/OnGZgYKF9Z

[Hikari∣LocalLLM⚡ @Hikari_07_jp]

https://t.co/FlD9o0zxUq

[Hikari∣LocalLLM⚡ @Hikari_07_jp]

A friend of mine created this complete computer wiring diagram (GitHub link in the replies). I highly recommend it, especially for anyone working on RAM-offload inference!

In this method, "hot experts" are placed in VRAM, while the overflow that doesn't fit is stored in RAM. Experts in RAM are processed by the CPU, making RAM bandwidth the main bottleneck.
However, there's a way to speed up output without increasing bandwidth usage: speculative decoding.
I was experimenting with this using the UD-IQ2_K_XL version of GLM-5.2. This diagram visualizes exactly what I had imagined based on the text descriptions.

[Anthropic @AnthropicAI]

We’ve received notice that the Department of Commerce has lifted export controls on Claude Fable 5 and Mythos 5.

We'll begin restoring access tomorrow, and will share an update soon.

We’re grateful to our users for their patience, and to everyone who worked with us on redeploying the models.

[Hikari∣LocalLLM⚡ @Hikari_07_jp]

If you're working on local LLMs, let me know in the replies! I'd love to connect.
Exchanging info is super effective for problem-solving. I've helped dozens with setup & inference questions, but I'm still learning tons myself. Let's level up together! 🚀

[Hikari∣LocalLLM⚡ @Hikari_07_jp]

@sweinoid 翻訳を使ったからそう見えているのだろう。低品質だと思うなら、私をブロックして視界から遠ざけてくれ。

[Balloonman @BuhloonMan]

@Jacobsklug NATIVE