Haoze He

Ph.D.@CMU CS,
exintern@GDM, Amazon-AWS,
exNYUer

[Drew Breunig @dbreunig]

This builds on @rikiyatakehi's incredible work, and @HaozeHe_Hector's application of it to decision models.

It runs with MLX, PyTorch, and vLLM backends. We've tested it modifying Qwen3.5-0.8b, but it really shines with decision models, like @liquidai's d1-2b and d1-omni.

[Haoze He @HaozeHe_Hector]

@morluto Hahaha lol

[LLM4Code @llm4code]

📢Announcing the 4th workshop on #LLM4Code, co-located with @ICSEConf 2027 in Dublin 🇮🇪!

🎯We are calling for submissions:

🌟Website: https://t.co/Ai3nseQnVh
🌟Deadline: Nov 13 (AoE), only 6 days left!
🌟8-page research / 4-page position / 5-page short paper (incl. refs)

[Honing Lab @HoningLab]

New arena unlocked 🎮

We’ve added REA, an Isaac-inspired room-combat engine, to HoningLab.

Next up: letting our decision model step into the arena and see how it handles the fight. 🤖⚔️

More fun stuff on GitHub 👇
https://t.co/dQ3I8tngi9

[Haoze He @HaozeHe_Hector]

@patchwanders @rikiyatakehi probably about as close as the direction does, because the direction isn’t selective. haven't run that control yet though.

[Haoze He @HaozeHe_Hector]

We edited 0.0002% of NanoJev’s weights, and the edited model beat the original 65–34 (1 draw) in 100 head-to-head Tank War games. 🎯

The original took only about 1 in 3 clean shots. Using Weight-Space Programming by @rikiyatakehi et al. (https://t.co/oQW1HI5rU9), we read the fire-vs-hold direction out of its activations and added it to the decision head: 1,024 weights, no training. Now it takes every clean shot and fires about twice as often.

It isn’t a better shot, though. It just stopped hesitating.

🛠️ Weight editing is now a trainer in Honing. An edit goes through the same gate as SFT and RL, and only ships if it beats the current model.

Report: https://t.co/xpLcLMauje

Weight-edit trainer: https://t.co/hoXdxumA1s

GitHub: https://t.co/oAPAOkCpcy
Follow @HoningLab for more Honing updates :)

[Rikiya Takehi @rikiyatakehi]

We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.

We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: https://t.co/bOWCcFhCVq

[Honing Lab @HoningLab]

100+ GitHub stars! 🌟 Sadly, our agents can’t train on those. Back to the GPUs. 🫠

This week: SGLang serving + an OpenHands/OfficeQA integration landed in Honing. The toolkit is growing beyond games.

Thanks for the early support! 🫶

[Honing Lab @HoningLab]

A familiar decision shouldn't need a fresh essay. 🧠

Our goal: self-evolving System 1 + System 2 agents, with fast decision models for familiar choices and deeper reasoning when needed.

We're building Honing: infrastructure to keep those decision models learning.

[Haoze He @HaozeHe_Hector]

This could definitely work! Using REA to build a game engine, generate training data, and train better gaming action models is a really promising path! Check our demos and the project @ https://t.co/8mW06FMuMM; we will extend to more games soon!!

[morluto @morluto]

rust and C ports make for very nice training environments!

i was just talking about this with @HaozeHe_Hector the other day

[Haoze He @HaozeHe_Hector]

@morluto This could definitely work! Using REA to build a game engine, generate training data, and train better gaming action models is a really promising path! Check our demos and the project @https://x.com/HaozeHe_Hector/status/2114770631505883420; we will extend to more games soon!!

[Haoze He @HaozeHe_Hector]

sadly CMU is not included here....

[Carlos E. Perez @IntuitMachine]

This is very sad.

[Haoze He @HaozeHe_Hector]

Fun fact: the model you played earlier is our strongest one. The one at the link is a weaker model, to give everyone a better chance to win!

Hint for beating it: each tank only has 5 shells, and reloading takes time. The trained model fires as soon as it's aimed at you, so what should we do:) ?

[Haoze He @HaozeHe_Hector]

Adding X handles for Tianyu Chen (@TianyuChen37516) and Chenyang Yang (@cyyang3_u) 🙌

[Haoze He @HaozeHe_Hector]

Try it on https://t.co/Xmjb9TddPw and to play against one of our best models!

We gave open-source decision models two games. It did not go well. 🫠

🎮 Red Alert 2: the base model sold its own base. Then lost.
💥 Tank War: v1 spun in circles. v2 found the fire button… and shot itself.

So we let them keep learning: play → train → upgrade the harness → repeat. A few rounds later: model can evolve without human intervention

Then we put different models in one arena and let them fight it out. So far, no human has beaten the trained models. 😅

The lesson: a decision model isn't finished when it's released. Real environments keep throwing new situations at it, and a frozen model can't adapt. It needs continuous learning!

It's all open source: how we train these models to play, end to end. Next, we're turning it into training infra for any decision model.

⭐ Star it, fork it, or bring your own model to the arena:
GitHub: https://t.co/oAPAOkBRn0
RA2: https://t.co/xMv6VNaE11
Tank War: https://t.co/xttz2fjeWe

I also appreciate all the suggestions and feedback from: @b_vasilescu , Greg Ganger, @p0nk , @lateinteraction , @matei_zaharia , and @heathercmiller

And thanks to all the Contributors to the demo and the github repo!: @Jayzou3773, @brocoliman38951,
@ShangyinT , @Xuan04263449 , chenyang yang, tianyu chen , @AlexZhang152005, @Nickzz12138,@JasonWuzh, @Leow_sss730, @ChenMoneyQ , xingyuan ding, @JunchengLi , @LakshyAAAgrawal , @ao_qu18465

[Haoze He @HaozeHe_Hector]

@raw_works interesting article, will take a look!

[Haoze He @HaozeHe_Hector]

@raw_works No don't blame yourself it's just our trained models are sooo good:)

[Haoze He @HaozeHe_Hector]

@dbreunig Drew's super 13-year-old brain >= Codex + GPT-6 then:)

[Haoze He @HaozeHe_Hector]

(3/3)we are now releasing the weaker version of our model. If more people win that one, we can host a stronger model on the current arena!

[Haoze He @HaozeHe_Hector]

(2/3)We also did inference optimization on inference:

Faster decisions, less repeated work:
• Share context computation across action options.
• Compile GPU execution with CUDA graphs.
• Overlap CPU input preparation with GPU inference.
• Combine related decisions into one request.
More players, lower latency.

[Haoze He @HaozeHe_Hector]

(1/3)This is how models evolve:

Reflect on match traces → propose updates → test against earlier versions → keep what improves.
GEPA optimizes prompts and the harness. SFT updates model weights. Repeat the loop: better decisions can come from both.