Harsh Shah

Autopilot@Tesla, MSML@CMU, CS@IITB

[Harsh Shah @HarshSensei]

@thinkymachines @ShangyinT Huge!

[Harsh Shah @HarshSensei]

Paper : https://t.co/PNSXSIONEj
a16z’s blog : https://t.co/vAmc1jbzDu

[Harsh Shah @HarshSensei]

was going through an interesting paper, "Continual Learning in Transition", surveying continual learning techniques. found it complementary to a16z's blog on "Why We Need Continual Learning" (really loved it). both of them talk about the different axes (prompts, harnesses, weight updates, etc) systems can be adapted to learn from deploy-time data.
there seems to be a spectrum over which these systems can be made, on one extreme we have prompt based updates (GEPA, OpenEvolve etc) and on the other extreme we have weight updates (TTT, SDFT, etc). making a hybrid of this is tuning a trade-off between system plasticity and safety, the pareto optimal of which would enable genuinely useful continual learning system.

[Harsh Shah @HarshSensei]

MOMMM, we saved a few lives today!!

[Sawyer Merritt @SawyerMerritt]

I made a video compilation of @Tesla FSD (Supervised) avoiding accidents, pedestrians and animals. This technology is already saving countless lives.

Thanks for submitting your clips!

[Harsh Shah @HarshSensei]

The decision of what experiments to run next should come from past experiments and current failures of deployed model. For automation of model training, the former can be enabled by better context management and latter by smart monitoring systems over production traces.

[Shrey Modi @ShreyModi13]

With the smartest person I know, @HarshSensei, we're building evsys-sdk for the community, this open-source repository allows anyone to build their own continual learning system with first-class support for @FireworksAI_HQ.

We believe continual learning will not be a single algorithm, but rather an auto-research system that looks at production traces, figures out the best way to update the model weights by conducting experiments on both data and algorithms, and runs evaluations to measure the results of those experiments.

You can now create a system with one deploy command which pulls in your production traces with some frequency, triggers the ML auto-research loop automatically, which uses any Tinker-compatible training API (@FireworksAI_HQ) to conduct experiments and train your model from the deployed checkpoint, and finally updates it with the better model on inference providers. RL, SFT, self-distillation, custom algorithms, etc. are tools that the auto-research system has at its disposal.

One of the major issues blocking auto-research for ML experimentation is that training code requires too much plumbing, which fills up the agent's context and is non-standardized across experiments, making it difficult to compare experiments and learn from past experiments.

The SDK provides an auto-research harness which takes away the complexity of training code and squashes each experiment down to just a config file, while giving full freedom to run any data or algorithm experiment. This way, the auto-research agent doesn't suffer from context bloat and performs much better than standard auto-research harnesses.

The SDK is also highly extensible and customizable, providing simple abstractions for curating any form of data, running any algorithm, pulling from any trace database, and running training on any Tinker-compatible backend.

We see auto-research as the only solution to continual learning, and this open-source contribution as a way to democratize access to it and push the usage of open-source models.

As a roadmap to enable ML auto-research and continual learning, there is a lot of work to be done:

1. Indexing previous experiments and existing plethora of research to give to the auto-research agent so it keeps improving and learns from past experiments.

2. Making app-layer integrations, e.g. one-click distillation from your traces generated through Claude Code, Cursor, etc. into an open-weights model.

3. Improving the auto-research harness and adding more integrations.

[DAIR.AI @dair_ai]

// The Bitter Lesson of Tool Calling //

Tool calling is a design choice, and the defaults are quietly costing accuracy.

How so?

New research releases a generation-spanning comparison of programmatic tool calling against native JSON tool calling, across 14 language models on BFCL v4.

Programmatic tool calling exposes tools as typed Python stubs the model invokes through code, with execution and results handled in a single agent turn. It matches or beats JSON tool calling in 11 of 14 models, and the GPT-5.6 family gains 10.6% over the JSON baseline.

Under parallel fan-out it wins in 13 of 14 models. Under context rot it holds steady while the JSON baseline drops 2.3% on average.

The gains track model generation, so the advantage grows as code ability grows.

Paper: https://t.co/RNmjeX5xGX

Track more trending AI papers in our academy: https://t.co/LRnpZN7L4c

[Yichuan Wang @YichuanM]

Zero Train–Inference Mismatch — now for linear attention, and under async RL 🎯

We got bitwise-exact trainer/generator parity for Gated DeltaNet (Qwen3.5-9B / 35B-A3B) on TorchTitan RL + vLLM, then asked the question nobody had actually tested in open source: does it help async RL?

Two firsts (as far as we know):
1️⃣ First open-source stack to hit train/inference logprob diff = 0 on a
linear-attention model
2️⃣ First to measure what zero mismatch is worth under async off-policy RL

How — smaller than it sounds:
• trainer and generator share one model definition (TorchTitan unified model). No more aligning trainer & generator op by op
• recurrent kernel everywhere on the forward (prefill + decode + trainer), chunked only for backward
• split-K off, batch-invariant GEMM, + a batch-invariant bmm for the MoE router
• we never touch FLA kernel internals — the recurrent kernel is intrinsically batch-invariant: one sequence's own state, fixed order, no cross-sequence reductions
• vLLM's linear-cache management + prefix caching keep working as-is

📊 Qwen3.5-9B + DAPO-Math, async at offpolicy = 4 / 12 / 32:
• logprob_diff = exactly 0 at step 0 ✅ every nonzero after that is staleness, not precision
• at offpolicy=32 the stock stack runs away past 0.065; ours holds ~0.035
• the unified model is free — the entire bill is the kernels
• also verified on MoE 35B-A3B and a 64-turn / 64K-context terminal agent

🤔 Now the honest part: it depends on the config. On math at offpolicy=12, BI leads on train reward and peaks highest on held-out AIME. On the terminal agent it's slightly ahead. On search, and on other math configs, the curves are indistinguishable. What is consistent is the numerics — BI keeps the gap from exploding as you widen the window. Cost: 2–3× trainer throughput (≈5× on the agent).

Our take: bitwise parity is a debugging tool, not a production default. Flip BI on for 20 on-policy steps — if the gap is 0 and your run is still broken, the infra is exonerated and the bug is in your data or your algorithm. That's worth 20 steps, not a whole run.

Next:
• make BI cheap (kill the 2–3×)
• more workloads, more seeds
• stay tuned for more complex agentic training recipes/tests 👀
Full writeup 🔗 https://t.co/P5sWo5iPb2

We collaborated closely with the TorchTitan team, and we sincerely encourage everyone to check out TorchTitan RL. With its clean, efficient unified model architecture and native async RL design, we believe it is a strong foundation for your next RL engine.

#LLM #RL #agentic #pytorch #torchtitan

[Prime Intellect @PrimeIntellect]

Introducing Prime Agent:

A self-improving RLM harness for coding and long-running autonomous tasks.

Designed to be both token-efficient and expressive through programmatic tool calling, context as a variable, multi-agent messaging, and a self-modifiable harness state.

[Meta-Agents Workshop @ NeurIPS 2026 @MetaAgentWkshop]

🚀Call for Papers — @NeurIPSConf 2026 Workshop Workshop on Responsible Use of Meta-Agents
📅 December 11/12 · 📍 Sydney, Australia

Join us to shape the future of Meta-Agents and Harness. Topics include but are not limited to:
1. Automated Design of Agent Harnesses
2. Optimization of Agentic Systems
3. Recursive Self-Improvement
4. Evaluation and Benchmarks for Meta-Agents
5. Misalignment and Safety for Meta-Agents
6. Governance & Human Oversight of Agents

🌟 Featuring an all-star speaker lineup:
Graham Neubig @gneubig (CMU)
Chelsea Finn @chelseabfinn (Stanford)
Jenny Zhang @jennyzhangzt (Recursive)
Robert Lange @RobertTLange (Sakana AI)
Bo Li @uiuc_aisecure (UIUC)
Hancheng Cao @CaoHancheng (Emory)
Yuandong Tian @yuandongtian (Recursive)
Chen Sun @ChenSun92 (Google DeepMind)

Proudly sponsor by @metaai and @e2b

Updates: Follow us here & spread the word! #NeurIPS2026 #MetaAgents #LLMs #Harness #RSI #SelfImprovement

[Harsh Shah @HarshSensei]

Algorithms like OPSD seem very intuitive, but it becomes very tricky to use teacher models for reward signal since student rollouts can go out-of-distribution for teacher, yielding noisy rewards. Masking bad tokens becomes a necessity for stable training.
Great work by AC team to productionize OPSD!

[Applied Compute @appliedcompute]

Production traces contain rich feedback that rarely makes it back into training pipelines. Self-distillation methods allow us to close the continual learning loop and learn from every interaction.

We’ve added native support for OPSD and RMSD to our platform, AC2. Here’s a preview:

[Harsh Shah @HarshSensei]

loved the idea behind Fast-WAM (world action model). predicting future frames is indeed wasteful when all you need is action-conditioned dynamics; enough of the future to pick the next move; the task-relevant change, not the whole scene.
Fast-WAM just uses future frame prediction as co-training task and drops it during inference.
Kudos to @gen_instinct 's research blog on this, https://t.co/JHsfs00DvE

[Harsh Shah @HarshSensei]

i came across 1-bit Kimi K3 by unsloth, and I was like Kimi K3 was already quantized, so why (and how) Unsloth quantized the model further. have written a blog for demystifying this. key takeaways,
- Moonshot AI quantized the experts to MXFP4, with MXFP8 activations, by performing QAT during post-training
- Unsloth performed dynamic quantization (based on activation outliers and KL div over token probs) to find insensitive weights
- mapped insensitive weights into sub-2-bit lattice grids, reducing VRAM by 62% (1.56TB -> 594GB)

Blog : https://t.co/87NiUkXps5

the post-training quantization used by unsloth `UD-IQ1_S` has also intrigued me, suggestions for any in-depth resource for that?

[Unsloth AI @UnslothAI]

Kimi K3 can now be run locally! ✨

The 1-bit model retains ~78.9% accuracy after we shrunk it from 1.56TB to 594GB (-62% size).

Run on a Mac Studio + 128GB RAM device.

Kimi K3 is the strongest open model to date.

Guide: https://t.co/1mVwOMLpDW
GGUF: https://t.co/bt1c1ADdCZ

[Harsh Shah @HarshSensei]

Expecting scaling laws for this soon

[AK @_akhaliq]

Explorative Modeling

Unlocking a Third Pretraining Axis and End-to-End Generation

paper: https://t.co/e0sFDi0KLe

[Harsh Shah @HarshSensei]

@FrancoisChauba1 I genuinely believe but the thing that bugs me is training feasibility over large data which is what transformers enabled by parallellism

[Thinking Machines @thinkymachines]

Today, we are releasing Inkling-Small.

Inkling-Small achieves comparable performance to Inkling at a quarter of its size. It features 276B total parameters, 12B active. We are making the full weights available.

https://t.co/BtYNcpkDRA

Fine-tune it on Tinker today, or chat with it in text, image, and audio on Tinker Playground.

[Harsh Shah @HarshSensei]

YC paper club had an awesome lineup today, as usual
jotting my one-liner takeaways from each presentation (definitely undermines the work, pls go through resources for details).
1. MEM: Multi-Scale Embodied Memory for Vision Language Action Models
- Embed long-term memory as compressed text for long horizon tasks
2. Self-Supervised Bootstrapping of Action-Predictive Embodied Reasoning
- Not all seemingly important reasoning is action predictive
3. SimToolReal An Object-Centric Policy for Zero-Shot Dexterous Tool Manipulation
- Learnt goal-pose + goal-conditioned policy helps with generalization
4. The Data Layer for Physical AI (by https://t.co/s7beHdxdGy)
- Make a robotic data SaaS company :))
5. Real-time World Action Models (by https://t.co/uxr9Oaeocr)
- world action models are the future and General Instinct is doing doing crazy work for edge deployment

thanks to presenters, and ofc, @FrancoisChauba1 for organizing!

[Harsh Shah @HarshSensei]

Great article! I often go back to triton implementation of flash attention for a refresher, https://t.co/r35vnAbyjy

[ali @waterloo_intern]

https://t.co/aivc7B4A7Z

[Harsh Shah @HarshSensei]

I wonder what speculative decoding algorithms are used for such high token/s. I liked this work, https://t.co/QboesZMaPH, by Tanishq et al to do async speculation. Guessing inference companies would be using such optimizations

[Philip Kiely @philipkiely]

https://t.co/ZomEdoDLlJ