Quan Dao

[alphaXiv @askalphaxiv]

"Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers"

Latent reasoning lets models think without writing every reasoning step, but it usually loses accuracy as models get bigger.

This paper, LOTUS, fixes this by using a looped Transformer that updates many hidden thoughts in parallel, then trains them to match real CoT steps.

So the model gets CoT-like reasoning without generating the full CoT.

At 3B, it nearly matches explicit CoT on GSM8K while thinking 2.5x faster, and 6.9x faster for long reasoning.

The hidden thoughts can also be read back as reasoning steps, making them compressed but not completely ambiguous.

[alphaXiv @askalphaxiv]

"Loop the Loopies!"

This paper Loopie shows recurrent Transformers can be compute efficient, not just parameter efficient.

Contrast to how we love stacking more layers, they have each MoE layer loops twice before moving on. This actually saves activation memory, improves throughput, and reinvests the gain into capacity.

Their Loopie-20B-A2B using this approach beats a compute-matched vanilla 30B-A3B MoE baseline, with only 2B active params, and reaches gold-level IMO/IPhO performance without tools.

[Yifan Zhang @yifanzhang_]

Heard that some frontier models are basically a 48-layer transformer looped twice (48L x 2).

Now we are introducing DeepLoop: Depth Scaling for Looped Transformers (https://t.co/jtxllo9aFF),
making the loop transformer stable and scalable!

[hardmaru @hardmaru]

Real brains follow Dale's principle: a neuron can either excite its neighbors or suppress them, but never both. Standard deep learning ignores this and uses backpropagation.

In our new paper, Diffusing Blame, we fix this disconnect. By introducing a routing method that broadcasts error signals directly to the hidden layers, we can train networks made of dedicated positive and negative neurons to strictly obey Dale's principle, all without relying on backprop!

This method works surprisingly well on image recognition tasks despite the strict biological constraints. We also achieved competitive, backprop-free reinforcement learning on complex locomotion tasks and the open-ended Craftax environment.

It is neat to see that representation learning remains possible even when we force deep learning to play by the rules of real neurons.

[Yifan Zhang @yifanzhang_]

So basically,

This is the Thinking Machines’ attention mechanism: GRAPE-A-Query-Gated

https://t.co/CbPI32RXcu

No more RoPE, no more attention sink, no more gated attention, WE ARE SO BACK!

https://t.co/4elf3Aze9w

[Google Research @GoogleResearch]

What drives a diffusion model's creativity (i.e., its ability to generate novel data rather than memorize)? Today we show this is a mathematical consequence of neural network regularization causing score smoothing and interpolation. Read the blog to learn more: https://t.co/menk2a3Xy6

[Binxu Wang 🐱 @WangBinxu]

congrats to Zhengdao! very nice intuition: score smoothing and direct density smoothing have different effect! and that might be why we see on manifold / simplex generalization / creativity in diffusion!

[Discrete Diffusion Reading Group @diffusion_llms]

📢July 13 (Mon): Continuous Diffusion Scales Competitively with Discrete Diffusion on Language

🤔While diffusion has drawn considerable recent attention from the language modeling community, continuous diffusion has appeared less scalable than discrete approaches.

💡To challenge this belief, the authors revisit Plaid, a likelihood-based continuous diffusion language model (DLM), and construct RePlaid by aligning the architecture of Plaid with modern discrete DLMs.

📈In this unified setting, they establish the first scaling law for continuous DLMs that rivals discrete DLMs: RePlaid exhibits a compute gap of only 20× compared to autoregressive models, outperforms Duo while using fewer parameters, and outperforms MDLM in the over-trained regime.

📈The authors benchmark RePlaid against recent continuous DLMs: on OpenWebText, RePlaid achieves a new state-of-the-art PPL bound of 22.1 among continuous DLMs and superior generation quality.

💎These results suggest that continuous diffusion, when trained via likelihood, is a highly competitive and scalable alternative to discrete DLMs.

🔍Moreover, they offer theoretical insights to understand the advantage of likelihood-based training. They show that optimizing the noise schedule to minimize the ELBO's variance naturally yields linear cross-entropy (information loss) over time. This evenly distributes denoising difficulty without any case-specific time reparameterization.

🌍In addition, the authors find that optimizing embeddings via likelihood creates structured geometries and drives the most significant likelihood gain.

[Grigory Sapunov @che_shr_cat]

1/ Why do diffusion models trained on completely different datasets generate the exact same image when given the same noise seed?

It is not magic or massive memorization. It is Random Matrix Theory in action.

Another ICML 2026 Outstanding Paper Honorable Mention.🧵

[Loic cabannes @loiccabannes]

Introducing Sparse Delta Memory (SDM) - The first work of my PhD 🎓.

SDM combines the GatedDeltaNet update with Product Key sparsity, enabling a recurrent state 3000x larger at the same FLOPs and significantly improving long-context performance.

Let RNNs be sparse!

[Sander Dieleman @sedielem]

A drifting model using coupled oscillators 👀

It manages to produce half-decent ImageNet 64x64 samples, which may seem like a toy experiment nowadays, but was basically SoTA only about a decade ago.

Love the detailed blog post! Takes me back to my reservoir computing roots😁

[Jayden Teoh @jayden_teoh_]

I really like this paper on why predicting latents is more data-efficient than predicting tokens.

In Next-Latent Prediction Transformer (https://t.co/d0Jzdduhtk), we provide a different argument on why predicting your own next latent can improve data-efficiency.

Predicting the next latent h_{t+1} gives you a rich learning signal about ALL future latents, and future tokens. This is because h_{t+1} is trained to predict h_{t+2}, h_{t+2} is trained to predict h_{t+3}, and so on... As such, predicting the next latent gives denser gradient signals than one-hot token predictions :)

I talk more about this in my blog: https://t.co/jSsDYndDn8

[Weichen FAN @W30259893]

🔥BIG Update🔥for "The Prism Hypothesis" (UAE)

✨Paper: https://t.co/BAIRUngMOo
✨Code: https://t.co/t2k5N1CDDq

🔠languages and visuals👓 can coexist in one representation.

Now support both latent and pixel-wise Generation
(🔥JIT is supported!🔥)

💫#UAE💫 gives a strong foundation for NEO-Unify:

Understanding and generation share one representation making raw-input end-to-end learning a natural path! 👏👏👏

[Ryohei Sasaki@engineer @rsasaki0109]

ViTTT
[CVPR 2026] [Best Paper Finalist] [Oral] Official repository of Vision Test-Time Training
https://t.co/DiVWBWXhzv
Test-Time Training (TTT) has recently emerged as a promising direction for efficient sequence modeling. TTT reformulates attention operation as an online learning problem, constructing a compact inner model from key-value pairs at test time. This reformulation opens a rich and flexible design space while achieving linear computational complexity. However, crafting a powerful visual TTT design remains challenging: fundamental choices for the inner module and inner training lack comprehensive understanding and practical guidelines. To bridge this critical gap, in this paper, we present a systematic empirical study of TTT designs for visual sequence modeling. From a series of experiments and analyses, we distill six practical insights that establish design principles for effective visual TTT and illuminate paths for future improvement. These findings culminate in the Vision Test-Time Training (ViT3) model, a pure TTT architecture that achieves linear complexity and parallelizable computation. We evaluate ViT3 across diverse visual tasks, including image classification, image generation, object detection, and semantic segmentation. Results show that ViT3 consistently matches or outperforms advanced linear-complexity models (e.g., Mamba and linear attention variants) and effectively narrows the gap to highly optimized vision Transformers. We hope this study and the ViT3 baseline can facilitate future work on visual TTT models.

[Orest Kupyn @OKupyn]

Excited to release new paper: MMDiff: Extending Diffusion Transformers for Multi-Modal Generation
that was just accepted to #ECCV2026 and is finally out! 🎉

Project page 🌐: https://t.co/WAWbFqpLdD
Paper 📄: https://t.co/hbltDkB77N
Code: https://t.co/CwMOzWsF2n

[Minhyuk Sung @MinhyukSung]

Last week I had the opportunity to visit NVIDIA and Netflix to share our work in a talk titled “𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝘃𝗲 𝗠𝗼𝗱𝗲𝗹𝗶𝗻𝗴 𝗕𝗲𝘆𝗼𝗻𝗱 𝗦𝗰𝗮𝗹𝗲: 𝗘𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝗰𝘆, 𝗚𝘂𝗶𝗱𝗮𝗻𝗰𝗲, 𝗮𝗻𝗱 𝗠𝘂𝗹𝘁𝗶𝗺𝗼𝗱𝗮𝗹𝗶𝘁𝘆”.
Check out the slides:
https://t.co/6G0ai0rbio

[pulsatingGenius @Neuralwarlock]

https://t.co/3oOkzxFcV7

[Arthur Gretton @ArthurGretton]

Optimized MMD for Detecting Distribution Shift!

Video:
https://t.co/bTJ0ZkU0zU

and slides
https://t.co/thNvTmjUz9

from the #ICLR26 workshop
https://t.co/WpUCAtJQPO

Covers:
https://t.co/x6syFYarxL
and
https://t.co/OBhBLVwNHc

[Liang Zheng @LiangZheng_06]

Gredient descent post-trains better diffusion models than RL.

Paper
https://t.co/ggNRmtODDH

Code
https://t.co/vBNBIZeCYT

[Binxu Wang 🐱 @WangBinxu]

1/
When diffusion generates images from text, before an image has objects, how does each noisy token know what it should become?

In our new work, we found that Diffusion Transformers solve spatial-relation prompts using a circuit motif reminiscent of developmental biology: morphogen-like spatial gradients.

At the start of sampling, image tokens are mostly uninformed noise — like an undifferentiated sheet in an embryo. Relation heads then write smooth spatial gradients onto the image canvas, guiding where objects should emerge.

Accepted as a @CVPR 2026 Highlight🌟: https://t.co/oIC6QKwQ3R
Beautiful collaboration with my friends and colleagues @fjxdaisy & Xu Pan!
A 🧵