Tianhao Wang

Assistant Professor @HDSIUCSD. Previously Research Assistant Professor @TTIC_Connect and PhD in Statistics & Data Science @Yale.

[Siyu Chen @siyuc3141]

Looped transformers often save compute by truncating backpropagation to the last few loops.

Is it worth it? We trained ~600 models to find out 👀

Short answer: only if the loop count is fixed. Otherwise, run fewer loops and train them all!

📄 https://t.co/4vPIXEUdDJ
🧵👇

[Sadhika Malladi @SadhikaMalladi]

Optimizers are usually benchmarked at one batch size, and scaling rules extend them to different batch sizes. We show: (1) frameworks to derive these rules are not as general as they claim to be, and (2) the best optimizer depends on the batch size. https://t.co/GyTvOOBJ3l

[Z @wangzhi0467]

I've been doing research with Codex and Claude Code. The hard part isn't running experiments or code. It's re-prompting the agent every time a run fails, and keeping track of results, decisions, and hypotheses.

So I built RCP, an open-source research control panel.

[Zhijian Liu @zhijianliu]

Splash is here! A month ago, DFlash 2 ran Qwen3.8-27B at 70 tok/s on an M5 Max. Today, Splash runs it at 144 tok/s on the same Mac ⚡

Getting that speed used to take work. Splash is built around the model, with the DFlash 2 draft already in it. Two commands and you're running.

[Sadhika Malladi @SadhikaMalladi]

RL is expensive, so every step should count. ~1 yr ago, we showed xent SFT isn’t the best way to prepare for RL (https://t.co/uniFmpbDOn). Now, we propose TailSFT (https://t.co/75yejb8voI), a lightweight + principled way to directly improve coverage and get better post-RL perf.

[Yu-Xiang Wang @yuxiangw_cs]

Excited to share that I’ve taken leave from UCSD and joined NeoCognition as Director of Machine Learning. Since July, I’ve been working closely with a talent-packed team on a strategic bet. Today is the day that we release details on part of what we’ve been building. 🧵1/n

[Sadhika Malladi @SadhikaMalladi]

I am starting a blog about deep learning theory and its value to practitioners! First post is about Adam, broken convergence proofs, and what theory can contribute when stuff just works anyways without it. Subscribe on Substack if you like it! https://t.co/oxWOm6f8tE

[Zhuoran Yang @zhuoran_yang]

We develop an algorithm for RL-based self-evolution — a model uses RL to learn to generate better training data, which is used to improve the model via RL. This approach is promising because:
(1) the generator can “adapt to” learner’s capabilities so that the RL data leads to more meaningful training signals,
(2) the generator can be used for data synthesis — turning unstructured text into structured QA data.

[Siyu Chen @siyuc3141]

How do we get LLMs to self-improve using unstructured documents without relying on human curation or naive "difficulty" heuristics?

Excited to share INFUSER, our latest work that turns self-evolution into influence-guided curriculum learning! 🚀 https://t.co/weDJ9Fjrbw

[Zhijian Liu @zhijianliu]

DFlash just landed in both SGLang and vLLM! 🚀

More draft models dropping soon: GLM-5.1, Kimi-K2.5 (preview live now!), Qwen3.5-397B & 122B.

Try it now ↓
SGLang: https://t.co/uHJTnuvI6N (🙏 @_dcw02)
vLLM: https://t.co/IgOIAFCelj (🙏 @BenjaminCh44989)

[Zhijian Liu @zhijianliu]

Reasoning LLMs generate very long chains-of-thought, so even small quantization errors add up.

With AWQ, Qwen3-4B drops 71.0 → 68.2 on MMLU-Pro (~4% relative loss). 😬

ParoQuant fixes this! It keeps only the critical rotation pairs and fuses everything into a single kernel.

Recovers most of the lost reasoning accuracy with minimal overhead — so 4-bit models stay strong at reasoning. 💪💪

[Zhuoran Yang @zhuoran_yang]

New Paper -- "On the Mechanism and Dynamics of Modular Addition: Fourier Features, Lottery Ticket, and Grokking"

We give a complete mechanistic and dynamic picture of how neural networks learn modular addition f(x,y) = (x+y) mod p. We answer three questions:
(1) What does the trained network compute?
(2) How do Fourier features emerge during training?
(3) Why does grokking happen?
Each answer comes with a mathematical characterization backed by theory and experiments.

Paper: https://t.co/uCU4eNr7tH
Blog: https://t.co/Ok9dDQcYvw
Demo: https://t.co/SX4Q6IsLHB
Code: https://t.co/03Adj6E3Bt

[Jiaqi Ma @Jiaqi_Ma_]

The ARC challenge claims to measure "fluid intelligence" through tasks that are "simple for people yet difficult for AI."

However, is the AI failure really due to the lack of "fluid intelligence?"

Our recent work shows that the answer is NO with a carefully designed diagnostic study.

ArXiv: https://t.co/bljEOmobtR
Joint work with Xinhe Wang, @JinHuang9306000, @_Jimmy_Zhang_ , @0920wth

Our study is motivated by an observation that ARC problems are easy for humans because their representation strongly favors human vision. For example, in the attached figure, the same ARC problem presented in a serialized way becomes much more challenging for humans.

1/

[Arya Mazumdar @MountainOfMoon]

The University of California, San Diego invites applications for one or more ladder rank faculty appointments based in the Halıcıoğlu Data Science Institute, the academic unit of the newly formed School of Computing, Information and Data Sciences. This is an open rank search for all levels of appointment (assistant, associate, or full professor). We seek outstanding candidates from ALL areas of Artificial Intelligence and Machine Learning as represented within HDSI's research scope, including but not limited to:
1) Computer Vision
2) AI for Science
3) Emerging Technologies for AI (such as Quantum Computing)

https://t.co/J5Pbe6wV6T @HDSIUCSD @UCSD

[Sadhika Malladi @SadhikaMalladi]

Excited to share that I will be starting as an Assistant Professor in CSE at UCSD (@ucsd_cse) in Fall 2026! I am currently recruiting PhD students who want to bridge theory and practice in deep learning - see here: https://t.co/cHN0Tdr5QQ

[Zhiyuan Li @zhiyuanli_]

Adaptive optimizers range from AdaGrad-Norm to Shampoo and full-matrix AdaGrad, with increasingly expressive preconditioners.

But does more adaptivity always translate to fewer steps to converge?

Our ICML 2025 paper answers negatively via a unified convergence analysis. 🧵1/6

[Zhuoran Yang @zhuoran_yang]

🚀 We're excited to share our paper, "Taming Polysemanticity in LLMs," which introduces Group Bias Adaptation (GBA)—the FIRST Sparse Autoencoder (SAE) training method with a provable guarantee for untangling monosemantic concepts!

📄 Paper: https://t.co/CzzEfaTWIg
🌐 Website: https://t.co/J2tCfJx3l2
🎯 Demo (Layer 26 of Qwen 2.5B-Base): https://t.co/4Dr1XjKoXU

Joint work with @siyuc3141, @HeejuneSheen, Xuyuan Xiong, and @0920wth

[Zhiyuan Li @zhiyuanli_]

Why does Adam outperform SGD in LLMs training? Adaptive step sizes alone don't fully explain this, as Adam also surpasses adaptive SGD.

Is coordinate-wise adaptivity the secret? Not entirely—Adam actually struggles in the rotated parameter space! 🧵 (1/6)
https://t.co/SIFBWrQ8hV

[Zhuoran Yang @zhuoran_yang]

[New paper on in-context learning] "In-Context Linear Regression Demystified" (link: https://t.co/7Z1ubhXQQB). Joint work @JLiangHe, @xintianpan, @siyuc3141. We establish a rather complete understanding of how one-layer multi-head attention solves in-context linear regression,

[Ruili Feng @feng_ruili_frl]

A step towards neural interactive simulation, where is Neo?