Raghav Gupta

[Alexi Gladstone @AlexiGlad]

We discovered a third pretraining axis beyond parameters and data: exploration.

Scaling exploration monotonically improves existing models across images/video/language, and unlocks end-to-end generation.

In the simplest case, it's just a for loop.

Introducing Explorative Modeling.

TLDR:
- Gains from exploration grow with scale: 7%→36% as data scales, 13%→23% as parameters scale, and gains double at 3× the compute
- Adding exploration to ~SOTA baselines improves data efficiency by 6.2×, FLOP efficiency by 4.1×, parameter efficiency by 47%, and hits a near-SOTA 1.43 unguided FID on ImageNet
- Exploration lets you trade training compute for generalization, and scales how end-to-end your generative model is
- End-to-end Explorative Models (XMs) match diffusion performance on control tasks with up to 256× less inference compute

🧵Thread:

[Onton @onton_ai]

Today we’re announcing Ontology 1, our newest AI model.

Perhaps surprisingly, it’s at least 2.7x more accurate than the world’s best ecommerce search engines, and handles queries that have never been possible before.

We keep reaching for the edge of what it can do. We haven’t found it yet.

Not only that, but it learns on its own with no retraining or fine-tuning. And it’s hallucination-free.

It’s a successor architecture for search.

Learn how we built it at https://t.co/PpBG2H1AvZ, and check out the benchmarks at https://t.co/hmgAPe7ddN.

Try it at https://t.co/ABb86WDwS3.

[Max Spero @max_spero_]

This is our most ambitious announcement yet. Two new models: Pangram 4 and Pangram Image.

Pangram 4 completely reimagines how we approach the problem of mixed authorship, by adding a tokenwise head onto the classifier to give every token a prediction given the full document context. We also wrote a 38-page technical report detailing our experiments, methodology, and evals.

Pangram Image is a completely new modality, bringing our detection expertise to AI-generated image and videos. In my early testing, it has worked shockingly well, even in strange cases like real photos of AI-generated bodega menus.

Today marks a huge step in the frontier of AI detection technology. I'm so excited to finally share with you all!

[Luke 🐝 Hewitt @lukebeehewitt]

🚨Published in @Nature: We show how AI can simulate human behavior to predict experimental results as accurately as experts

Our work has many implications for AI safety. For example, simulators can be used to aid harmful manipulation, and current safeguards do not prevent this🧵

[Aleksa Gordić (水平问题) @gordic_aleksa]

New in-depth blog post time: "Inside TPU and GPU Clusters: The Anatomy of Collective Communication".

If you want to deeply understand the core primitives behind scaling the training / inference for MoEs and dense transformers, going a level below FSDP, expert parallelism, data parallelism, model/tensor parallelism this might be a fun read.

I cover:
* TPU cluster topology: (super)pods, slices, DCN, PCIe, ICI
* All-Gather: 1D/2D rings, and path algo (lots of visuals so should be crystal clear how these work even if you're not a perf engineer)
* Reduce-Scatter (which is the dual of AG) and All-Reduce
* All-to-All (used to dispatch tokens to target experts in MoEs)
* NVIDIA GPU cluster topology (reference DGX architecture): nodes, scalable units, fat tree
* GPU collectives within the node: rings, trees (log2 steps), and SHARP (in network compute unit)
* GPU collectives across nodes, hierarchical algorithms over InfiniBand
etc.

I was heavily inspired to do this deep dive after reading the excellent Scaling book by an excellent group of people @jacobaustin132 @_sholtodouglas @reinerpope and others!

What originally started as "let me maybe just make four figures covering All-Gather, Reduce-Scatter, All-Reduce, and All-to-All so I can understand them better, it shouldn't take more than a day, right, right?" somehow turned into this 40 figures later.

Along the way, I realized that the collective algorithms only really make sense once you understand the underlying hardware topology. TPUs were a bit easier to reason about, but I couldn't skip GPUs, I love them too much. Rings are cool, but I also wanted to understand tree algorithms. But also SHARP, and fat trees, and hierarchical collectives. :')

So the scope slowly expanded, and little by little, this blog post came to fruition. Just a side-quest.

Hope you like it! :)

---

Also a big thank you to my friends for reviewing the blog and providing feedback:
* @ArunDemeure (prev GPU/AI stuff at Magic, GPU architect at Apple and Imagine, my llm.c buddy!)
* @axel_s_feldmann (making GPUs go brrr at Jane Street, we met for the first time at @marksaroufim's excellent GPU mode event)
* @pranjalssh (ex xAI GPU wizard, one of two people who inspired my original matmul blog!)

[Goodfire @GoodfireAI]

> replicate J-space on GLM 5.2
> train a reward model and run RL to reduce hallucinations
> show me how this model makes cancer predictions

Using our platform Silico is like having a team of AI researchers ready to run experiments like these.

Private beta is open now. 🧵 (1/6)

[LMSYS Org @lmsysorg]

🚀 New blog: The next generation of speculative decoding: DFlash and Spec V2

DFlash + Spec V2 hit >4.3X baseline throughput for LLM inference, now the default speculative decoding engine in SGLang! Together with @modal and https://t.co/ZXetBKIRym, our jointly-released DFlash drafter for Qwen 3.5 397B-A17B beats both baseline and native MTP in every setting we benchmarked:
1️⃣ >4.3X baseline & 1.5X native MTP throughput (concurrency 1, HumanEval, 8xB200)
2️⃣ Block diffusion drafter: a full token block in one forward pass
3️⃣ KV injection: target-model features fed into every draft layer’s KV cache for higher acceptance
4️⃣ Spec V2 overlap scheduler: +33% end-to-end

Read the code, deploy a DFlash server, and start experimenting!

[AI at Meta @AIatMeta]

We’re sharing the next major milestone in our non-invasive brain-to-text decoder research: Brain2Qwerty v2.

Building on v1, which was published today in @Nature, Brain2Qwerty v2 is the highest-performing end-to-end pipeline capable of real-time sentence decoding from raw brain signals. It advances beyond character-level performance to decoding words and semantics, enabling accuracy for overall communication.

We believe this research has the potential to make a real difference for the millions of people who suffer from brain lesions or disorders that prevent them from communicating.

🧵👇

[Elyas Masrour @elyasbuilds]

Did you know?

Pangram learns the difference between Claude, ChatGPT, and Gemini in its internal representations, even without being trained on it!

This signal is increasingly recoverable throughout the network, reaching 91% accuracy on a simple linear probe!

[Charles 🎉 Frye @charles_irl]

Speculation Is All You Need.

In this blog post, we announce the co-release (w/ Z Lab) of six more state-of-the-art DFlash speculators for @Alibaba_Qwen 3.x.

Over 1k output tps for 3.5 122B-A10B on a B200.

Read the blog for why we're all-in on spec dec.

https://t.co/Bv3Zc95Xgh

[philz1337x @philz1337x]

Crystal is now the largest image-upscaler API in the world

I've increased the output limit to 1.5 gigapixels, which is 38K 😂

[Grant Stenger (hiring) @GrantStenger]

Local minima are rare in high dimensions because a strict local minimum has to curve upward in every direction, so all Hessian eigenvalues must be positive.

In a D-dimensional toy model where eigenvalue signs are independent, that’s a 2^(-D) event. In GOE-like random matrix models, positive definiteness is even rarer, roughly exp(-cD^2).

So as dimension grows, random critical points are much more likely to be saddles than minima. This is one reason high-dimensional optimization is often a saddle-escape problem, not a bad-local-minimum problem.

Wrote up some of the math here: https://t.co/vkaVqVD64N

[Celeste @celestepoasts]

New research from @japhba and I!

Activation Oracles are a pretty cool interpretability tool. They answer natural questions about activations, but they suffer from vagueness and hallucinations. Can AO training be improved?

Turns out: Yes! We identify four fixes that make AOs substantially more useful!

[Christopher Potts @ChrisGPotts]

We take for granted that larger models are better than smaller ones, but why is this so? Our new paper, led by Jing Huang and @EkdeepL, traces this to a data-induced competition for resources (neurons), using formal analysis, idealized tasks, and real pretraining.

[Sakana AI @SakanaAILabs]

Introducing DiffusionBlocks: Block-wise Neural Network Training via Diffusion Interpretation

https://t.co/c9AvsRKybj

What if we didn’t have to hold an entire neural network in memory to train it?

Standard neural net training optimizes all parameters jointly. As a result, the memory required during training grows linearly with the depth of the network.

In our #ICLR2026 paper, we propose DiffusionBlocks, a principled framework to train networks one block at a time, drastically reducing memory requirements while matching end-to-end performance.

With DiffusionBlocks, we split the network into blocks and train them one at a time, so you only need memory for a single block.

How? We explicitly assign each block a role: to move the representation a little closer to the target than the block before it did. That role turns out to be precisely what a diffusion model does, step by step. Each block only needs to optimize its own objective and can be trained independently.

We validated this across five different architectures:

• ViT
• DiT
• Masked diffusion
• Autoregressive transformers
• Recurrent-depth transformers

In each case, performance is competitive with end-to-end training while using a fraction of the memory.

This perspective also extends naturally to recurrent-depth (Looped) transformers, which apply the same network iteratively and normally require expensive backpropagation through time (BPTT). Viewed through DiffusionBlocks, we can replace those multiple iterations with a single forward pass during training.

Read our paper and code, to learn more.

Paper: https://t.co/CRj96VGYQn
GitHub: https://t.co/eNW0K9Xh8E
🐟

[alphaXiv @askalphaxiv]

New paper from Yann LeCun!

"When Does LeJEPA Learn a World Model?"

This paper proves that under Gaussian latent dynamics, LeJEPA can recover the hidden state behind nonlinear observations up to rotation.

The intuition is that linear latent features are the most stable across nearby views, while nonlinear features decay faster, so the objective naturally selects the real world variables.

The key caveat is that this guarantee holds under specific assumptions, and Gaussian latents are the unique case that guarantees this.

[OpenAI @OpenAI]

Today, we share a breakthrough on the planar unit distance problem, a famous open question first posed by Paul Erdős in 1946.

For nearly 80 years, mathematicians believed the best possible solutions looked roughly like square grids.

An OpenAI model has now disproved that belief, discovering an entirely new family of constructions that performs better.

This marks the first time AI has autonomously solved a prominent open problem central to a field of mathematics.

[Core Francisco Park @corefpark]

🚨 New Paper! (Part 1: Pretraining)

Many recent works show beautiful representational geometry in neural networks.

But what controls the geometry of world representations during pretraining?

We decouple the world from data to study this in a controlled setup.

1/n

[Hyper3D by Deemos @DeemosTech]

Introducing #Rodin Gen-2.5🚀

🔥World’s 1st 10 MILLION polygon #3D GenAI — down to skin microstructures.

1️⃣1M-poly in 4s
2️⃣Adaptive thinking effort - just like LLMs
3️⃣3D-native textures, no blind spot
4️⃣Batch up to 10 results

💥Manual BANG to Parts & more...

🚨$1 for first mo!

[merve @mervenoyann]

Meta silently dropped Sapiens2 last week 🔥

a family of high-res models trained on 1B human images

> for pose estimation, body-part segmentation, surface normals, pointmaps (sota)
> 6 sizes: 0.1B → 5B params (all ViT patch 16)
> high-res: 1024×768 and 4K