Philipp Fränken

[Radical Numerics @RadicalNumerics]

1/5 We pretrained Omnii, a multimodal language model on DNA. Then, we asked it to design a cancer vaccine

In new research, we show how Omnii learned to turn a tumor DNA sequence into a personalized mRNA vaccine design.

Read more here: https://t.co/0Uw5yLn8pd

[Michael Y. Li @michaelyli_]

You're wasting FLOPs when scaling inference compute: by independently sampling parallel attempts, you burn compute rediscovering the same solutions.

Introducing QuasiMoTTo: we scale parallel sampling with correlated samples instead! These samples have higher coverage, are marginally exact draws from the LLM, and can be generated in parallel.

Result: same performance with 25-47% fewer samples in test-time scaling + 50% fewer training steps in RL!

In our new paper, we explore the design space of correlated samplers. Work with co-authors @probablynotaz9 (co-lead), @gandhikanishk, @noahdgoodman, and Emily Fox!

[Violet X. @ZiyuX]

🧵(1/9) Sparse RL for reasoning has an exploration problem.

It can only reward solutions the model already stumbles into. On hard problems, that means lots of zeros and very little signal.

SFT and self-distillation attack this with reference solutions as targets to match. Instead, we use them as reward scaffolds: a dense signal at both the outcome and process level.

Introducing ExpRL: RL-based mid-training that improves exploration by scoring the model’s own attempts against the reference via an LLM judge.

What we find:
• A stronger policy straight out of mid-training (higher pass@1 and pass@k)
• Still ahead after downstream sparse-reward RL
• Holds across domains – math & STEM
• Scales to a larger policy graded by a smaller judge

[Artificial Analysis @ArtificialAnlys]

Google is once again the leader in AI: Gemini 3.1 Pro Preview leads the Artificial Analysis Intelligence Index, 4 points ahead of Claude Opus 4.6 while costing less than half as much to run

@GoogleDeepMind gave us pre-release access to Gemini 3.1 Pro Preview. It leads 6 of the 10 evaluations that make up the Artificial Analysis Intelligence Index and improves significantly over Gemini 3 Pro Preview across capabilities, with the biggest gains in reasoning and knowledge, coding, and hallucination reduction.

Gemini 3.1 Pro Preview also remains relatively token efficient, using ~57M tokens to run the Artificial Analysis Intelligence Index (+1M from Gemini 3 Pro Preview), lower than other frontier models at max reasoning settings such as Opus 4.6 (max) and GPT-5.2 (xhigh). Combined with lower per-token pricing, Gemini 3.1 Pro Preview is cost-efficient among frontier peers, costing less than half as much as Opus 4.6 (max) to run the full Intelligence Index, though still nearly 2x the leading open-weights model, GLM-5.

Key Takeaways:

➤ State-of-the-art intelligence at lower costs: Gemini 3.1 Pro Preview is leading 6 of the 10 evaluations that make up the Artificial Analysis Intelligence Index at less than half the cost to run of frontier peers from @OpenAI and @AnthropicAI. It obtains the highest score in Terminal-Bench Hard (agentic coding), AA-Omniscience (knowledge & hallucination), Humanity’s Last Exam (reasoning & knowledge), GPQA-Diamond (scientific reasoning), SciCode (coding) and CritPt (research-level physics). The CritPt score is particularly notable, scoring 18% on unpublished, research-level physics reasoning problems, over 5 p.p. above the next best model

➤ Improved real-world agentic performance, but not leading: Gemini 3.1 Pro Preview shows an improvement in GDPval-AA, our agentic evaluation focusing on real-world tasks, but is still not the leading model in this area. The model increases its ELO score over 100 points to 1316 (up from Gemini 3 Pro Preview), however still sits behind Claude Sonnet 4.6, Opus 4.6, GPT-5.2 (xhigh), and GLM-5

➤ Leading coding abilities: Gemini 3.1 Pro Preview leads the Artificial Analysis Coding Index, achieving the highest score in both Terminal-Bench Hard (54%) and SciCode (59%)

➤ Reduced hallucinations: Gemini 3.1 Pro Preview shows a major improvement in tendency to guess incorrectly when it doesn’t know the answer, reducing its AA-Omniscience hallucination rate by 38 p.p. from Gemini 3 Pro Preview

➤ Maintained token and cost efficiency: Gemini 3.1 Pro Preview improves without material increases in cost or token usage. It uses only ~2% more tokens to run the Artificial Analysis Intelligence Index than Gemini 3 Pro Preview, and keeps the same pricing ($2/$12 per 1M input/output tokens for ≤200k context). Its cost to run the Artificial Analysis Intelligence Index of $892 is less than half of frontier models such as Opus 4.6 (max) and GPT-5.2 (xhigh), though still ~2x the cost of leading open weights models such as GLM 5 ($547)

➤ Google takes top 3 spots in multi-modality: Gemini 3.1 Pro Preview ranks #1 on MMMU-Pro, our multimodal understanding and reasoning benchmark, ahead of Gemini 3 Pro Preview and Gemini 3 Flash, reinforcing Google’s leadership in multimodal reasoning

➤ Other model details: Gemini 3.1 Pro Preview retains the same 1 million token context window as its predecessor, and includes support for tool calling, structured outputs, and JSON mode

[Google DeepMind @GoogleDeepMind]

Gemini 3.1 Pro is here.

We’ve significantly improved the model’s overall intelligence so it can solve tougher problems. 🧵

[Google DeepMind @GoogleDeepMind]

We’ve upgraded our specialized reasoning mode Gemini 3 Deep Think to help solve modern science, research, and engineering challenges – pushing the frontier of intelligence. 🧠

Watch how the Wang Lab at Duke University is using it to design new semiconductor materials. 🧵

[Isomorphic Labs @IsomorphicLabs]

Today we share a technical report demonstrating how our drug design engine achieves a step-change in accuracy for predicting biomolecular structures, more than doubling the performance of AlphaFold 3 on key benchmarks and unlocking rational drug design even for examples it has never seen before.

Head to the comments to read our blog.

[Eric Zelikman @ericzelikman]

finally announcing i’ve started humans& w/ amazing friends @gharik & @YuchenHe07 & @TheAndiPenguin & @noahdgoodman & many other world-class folks. we're optimists: it’s possible to rethink how we build ai, to empower people to accomplish more together

tldr: love is all you need

[Demis Hassabis @demishassabis]

First LLM contact from space 🛰️ using our highly efficient open source Gemma models! Huge congrats to @PhilipJohnston and the @starcloud_ team!

[Andrej Karpathy @karpathy]

nanoGPT - the first LLM to train and inference in space 🥹. It begins.

[ARC Prize @arcprize]

Gemini 3 models from @Google @GoogleDeepMind have made a significant 2X SOTA jump on ARC-AGI-2 (Semi-Private Eval)

Gemini 3 Pro:
31.11%, $0.81/task

Gemini 3 Deep Think (Preview):
45.14%, $77.16/task

[Eric Zelikman @ericzelikman]

we think humanity’s biggest challenges won’t be solved by ai thinking for 1000 hours coming back with an answer

they’ll be solved by many collaborating humans, and ai that understands them and their different skills, goals, values, etc to empower them to do more together

[Radical Numerics @RadicalNumerics]

Sliding window attention (SWA) is powering frontier hybrid models for efficiency. Is there something better?

Introducing Phalanx, a faster and better quality drop-in replacement for sliding window attention (SWA).

Phalanx is a new family of hardware and numerics-aware windowed layers designed with a focus on data locality and jagged, block-aligned windows that map directly to GPUs.

In training, Phalanx delivers 10–40% higher end-to-end throughput at 4K–32K context lengths over optimized SWA-hybrids and Transformers by reducing costly inter-warp communication.

Today, we are releasing both the technical report, a blog, and Phalanx kernels in spear, our research kernel library.

We are hiring.

[Andrej Karpathy @karpathy]

Excited to release new repo: nanochat!
(it's among the most unhinged I've written).

Unlike my earlier similar repo nanoGPT which only covered pretraining, nanochat is a minimal, from scratch, full-stack training/inference pipeline of a simple ChatGPT clone in a single, dependency-minimal codebase. You boot up a cloud GPU box, run a single script and in as little as 4 hours later you can talk to your own LLM in a ChatGPT-like web UI.

It weighs ~8,000 lines of imo quite clean code to:

- Train the tokenizer using a new Rust implementation
- Pretrain a Transformer LLM on FineWeb, evaluate CORE score across a number of metrics
- Midtrain on user-assistant conversations from SmolTalk, multiple choice questions, tool use.
- SFT, evaluate the chat model on world knowledge multiple choice (ARC-E/C, MMLU), math (GSM8K), code (HumanEval)
- RL the model optionally on GSM8K with "GRPO"
- Efficient inference the model in an Engine with KV cache, simple prefill/decode, tool use (Python interpreter in a lightweight sandbox), talk to it over CLI or ChatGPT-like WebUI.
- Write a single markdown report card, summarizing and gamifying the whole thing.

Even for as low as ~$100 in cost (~4 hours on an 8XH100 node), you can train a little ChatGPT clone that you can kind of talk to, and which can write stories/poems, answer simple questions. About ~12 hours surpasses GPT-2 CORE metric. As you further scale up towards ~$1000 (~41.6 hours of training), it quickly becomes a lot more coherent and can solve simple math/code problems and take multiple choice tests. E.g. a depth 30 model trained for 24 hours (this is about equal to FLOPs of GPT-3 Small 125M and 1/1000th of GPT-3) gets into 40s on MMLU and 70s on ARC-Easy, 20s on GSM8K, etc.

My goal is to get the full "strong baseline" stack into one cohesive, minimal, readable, hackable, maximally forkable repo. nanochat will be the capstone project of LLM101n (which is still being developed). I think it also has potential to grow into a research harness, or a benchmark, similar to nanoGPT before it. It is by no means finished, tuned or optimized (actually I think there's likely quite a bit of low-hanging fruit), but I think it's at a place where the overall skeleton is ok enough that it can go up on GitHub where all the parts of it can be improved.

Link to repo and a detailed walkthrough of the nanochat speedrun is in the reply.

[Michael Poli @MichaelPoli6]

We just released the largest open-source diffusion language model (RND1). RND1 is important to me on a personal level: it symbolizes our commitment to open-source exploration of radically different designs for AI at scale — training objectives, architectures, domains. There is still so much to do, and together we can move faster.

We are not done yet. More on Monday (the next one is for model architecture enthusiasts!).

[Radical Numerics @RadicalNumerics]

Introducing RND1, the most powerful base diffusion language model (DLM) to date.

RND1 (Radical Numerics Diffusion) is an experimental DLM with 30B params (3B active) with a sparse MoE architecture.

We are making it open source, releasing weights, training details, and code to catalyze further research on DLM inference and post-training.

We are researchers and engineers (DeepMind, Meta, Liquid, Stanford) building the engine for recursive self-improvement (RSI) — and using it to accelerate our own work. Our goal is to let AI design AI.

We are hiring.

[Lilian Weng @lilianweng]

GPUs are expensive and setting up the infrastructure to make GPUs work for you properly is complex, making experimentation on cutting-edge models challenging for researchers and ML practitioners.

Providing high quality research tooling is one of the most effective ways to improve research productivity of the wider community and Tinker API is one step towards our mission there.

Tinker API is built on top of our experimental results on fine-tuning with LoRA: https://t.co/bV00V1cBKG

Beta starts and you can join the waitlist today: https://t.co/AbzVwxYyjm

[John Schulman @johnschulman2]

Tinker provides an abstraction layer that is the right one for post-training R&D -- it's the infrastructure I've always wanted. I'm excited to see what people build with it.
"Civilization advances by extending the number of important operations which we can perform without thinking of them" -Whitehead

[Rishabh Agarwal @agarwl_]

I asked Liam: Why name our startup Periodic Labs? He said think about *periods* in time. And then it hit me: we define entire periods of history by their critical materials: Copper Age → Bronze Age → Iron Age → Silicon Age.

The name says all about our mission: to discover the materials that will define our future.

[Keshigeyan Chandrasegaran @keshigeyan]

Grafting Diffusion Transformers accepted to #NeurIPS2025 as an Oral! We have lots of interesting analysis, a test bed for model grafting, and insights🚀

📄Paper: https://t.co/9Horic5YTo
🌎Website: https://t.co/xZomk7Vq6e