Shail Shah

ML/AI Engineer

[Deedy @deedydas]

Every single startup working on next-gen AI chips (July 2026)

Every approach attacks data movement differently to break Nvidia’s dominance:
– eliminate DRAM (Groq)
– eliminate the interconnect (Cerebras)
– eliminate the compute/memory split (d-Matrix)
– eliminate the server’s compute-centrism (Majestic)
– eliminate generality (Etched, Taalas, MatX)
– or eliminate the $400M litho machine (Substrate)

[Andrew Ng @AndrewYNg]

Announcing OpenWorker! An open-source agent that doesn't just chat with you, but delivers finished work -- like hand you a polished document, send a slack message, or update a calendar entry.

Ask it to prepare a customer brief, untangle your calendar, draft a report, or triage a Slack alert. It works across your files and everyday tools, produces the deliverable, and checks in before doing anything consequential.

OpenWorker runs on your Mac, with Windows support coming soon. It does not lock you into any one model. Bring your own API key and run it with GPT 5.6 Sol, Claude Fable, Gemini 3.6, an open weight model (like Kimi, GLM, DeepSeek, Inkling), or Ollama to keep your data local. Your data does not leave your machine except through an LLM provider and integrations that you choose.

@rohitcprasad and I are building OpenWorker because AI coworkers are an important way to get work done, and we want there to be an open, privacy-preserving, model-independent option. Check it out and let us know what you think!

Try it out: https://t.co/P0mGnI1o31 (requires your own API key)
Source code: https://t.co/NYCiTD6hSq

[Jingyan Shen @evangelinejy99]

New Paper! As LLMs scale, where should compute go — stronger pretraining or more RL? Can we study pretraining and RL scaling jointly? We fit a pretraining–RL scaling law in a controlled chess testbed and use it to trace the optimal allocation. 🧵

[Marcel Rød @marcelroed]

Introducing the world's fastest tokenizer implementation, Gigatoken!

Gigatoken is ~500-1000x faster than HuggingFace, and ~100x faster than OpenAI's tiktoken for most tokenizer definitions on most machines.

These baselines are already multithreaded Rust implementations! 🧵

[Daniel Han @danielhanchen]

My 2hr workshop on Open vs Closed models, reward hacking, benchmaxxing & RL is out!

1. Closed vs open models
2. Throughput maxxing but accuracy minimizing
3. Benchmaxxing & cheating
4. Distillation & RL
5. Stopping reward hacking
6. @UnslothAI Dynamic Quants

Details:
1. If reasoning wasn't discovered via o1-preview - would all AI progress grind to a halt via a S shape? Reasoning made doubling times now 3.5 months instead of 7 - so just wait 3.5 months for the next best model! Ways folks might regulate open source - like a driver's license for AI

2. Inference providers maximize throughput and speed, but accuracy degrades - OpenRouter publishes stats on accuracy, and the largest gaps can be 20% or more!

3. METR, WeirdML, Deep-SWE, FrontierCode, SWE Bench Pro - what are good & bad benchmarks? False positive / false negative rates - how about daily benchmarking since Codex & Claude have perf regressions. How can we regressions to predict next model releases as well?

4. Open model labs use distillation partially, but need RL to create the reasoning traces to complete the process - hard vs soft distillation and how to automate RL.

5. Common examples for reward hacking + ways to stop it - internet filtering, classification systems, timing / editing global variables, real world examples + more!

6. Why software matters more than hardware - the limits of FP4 and GPUs vs ASICs and torch.compile vs kernels and megakernels and more, and why quantization and memory optimizations are important

https://t.co/tgHm8xpLwp

[Charles 🎉 Frye @charles_irl]

If you're interested in speculative decoding, take some time to grok this chart! And read the article from @haoailab.https://t.co/udJXMQWlIW

[Graham Neubig @gneubig]

This Fall at CMU we're teaching a new course on AI Agents!

The goal is that you learn how to create a scaffold, build evals, and train an agentic LLM using RL.

We'll try to balance theory and practice, and introduce modern frameworks and best practices.

[Daniel Han @danielhanchen]

DeepSeek just released DSpark for V4 Flash & Pro, a new speculative decoding method boosting throughput by 51% to 400%!

DS also showed DSpark works well for other models like Gemma & Qwen

Github: https://t.co/EGVYpc1kcK
Paper: https://t.co/TaBMRVlaW9
HF: https://t.co/289jVU2pxh

[Shail Shah @Shail251298]

@violetnekrasova @AryanK1511 I wish too!! It sucks rn here in mtl! Not much going on…

[Shail Shah @Shail251298]

Trying to pre-train an LLM is really hard but rewarding! There are just so many variables to decide and questions to answer, it just makes it even more tough!
I’m trying one and would soon share my learnings!

[Charles 🎉 Frye @charles_irl]

A few years ago, the future of artificial intelligence looked dark - proprietary models, proprietary inference services.

I joined Modal to fix that.

Today, we take a major step forward. Optimized inference, open source, with a click.

[NVIDIA AI @NVIDIAAI]

Increase inference performance by up to 15x without sacrificing responsiveness.

DFlash, an open source lightweight block diffusion model designed for speculative decoding, delivers up to 15x higher throughput on NVIDIA Blackwell while maintaining the same user interactivity target.

Instead of drafting tokens one at a time, it proposes a whole block in a single pass for the main model to verify in parallel.

Adoption is drop-in with support in @lmsysorg SGLang, TensorRT-LLM, and @vllm_project.

[Shail Shah @Shail251298]

@KyeGomezB Yeah totally!!

[Shail Shah @Shail251298]

@Karthickhps @__tinygrad__ Yeah I heard with Helion, we can auto-tune! But not sure which one’s better overall: helion, triton, cuTile from Nvidia!
Would like to hear your opinion!

[Shail Shah @Shail251298]

@__tinygrad__ Has anyone tried cuTile python from Nvidia?

[Daniel Han @danielhanchen]

Come join Thursday 8AM PST! I'll talk about GGUFs, quantization, GLM 5.2 quants, llama.cpp / llama-server & much more! Also ask any questions in the AMA / Q&A!

[Shail Shah @Shail251298]

@danielhanchen Hopefully next time or some other unsloth event. love your work and unsloth!

[Shail Shah @Shail251298]

Highly recommend to anyone who is going there! Wish I could attend!

[Daniel Han @danielhanchen]

I’m running a 3 hour advanced workshop at AI Engineer World’s Fair! 🚀

2026 has greatly changed how one should learn lower-level technicals like kernels, agentic RL, reward hacking, cont learning.

What would you like to see?

Last year @aiDotEngineer: https://t.co/3j8qcMD9u8

[Shail Shah @Shail251298]

@rileybrown *for lossless quantization we need q8, but q4 is best considering RAM requirements and quality!

[Shail Shah @Shail251298]

@rileybrown 2x Nvidia DGX Spark is enough. GLM 5.2 in dynamic 2-bit quantized version from unsloth can be run easily. If you need lossless (ie 99.99 KL divergence) go with dynamic 4-bit but that will require an extra dgx spark.
So, 2-3 DGX Spark is future-proof for any new models.

[Shail Shah @Shail251298]

@Shibe_Prime @rileybrown tok/s is roughly 5-10 depending on the setup, ofc it can’t match the api but good enough for coding! For 1M context size it would need 40-50 GB extra if using q4 quant kv cache, more if fp8 or fp16. So depends! Anyway, we can reduce context size if it doesn’t fit