Rycroft

siamese cat randomly looking for yummy food and delicious sleep

[Rycroft @ryuukaenn]

I asked GPT 5.6 Sol to analyze the match of 🇫🇷-🇪🇸, why France lose, and Claude rendered the full trajectory.

TL;DR: Both sides finished with 10 shots and 5 on target, but virtually all of France's threat came after the 0-2.
Also GPT says Spain’s system is not unbreakable.

[Google DeepMind @GoogleDeepMind]

We just dropped Lyria 3: our latest generative music model. 🔊

It can turn photos and text into dynamic tracks - complete with vocals and lyrics. 🧵

[Pedro Domingos @pmddomingos]

It’s happening. https://t.co/IxHdJky1yr

[Ravid Shwartz Ziv @ziv_ravid]

I forgot to update that the paper got accepted to ICLR. Fun fact, I did my PhD in computational neuroscience, but I hated cognitive science classes. So lucky that @ChenShani2 actually is an expert in it!

[Anton Korinek @akorinek]

Glad to have this important conversation on 𝗔𝗜'𝘀 𝗹𝗮𝗯𝗼𝗿 𝗺𝗮𝗿𝗸𝗲𝘁 𝗲𝗳𝗳𝗲𝗰𝘁𝘀 with @davidautor, @NatashaSarin, and @DLeonhardt in today's @nytopinion
"Are we facing another Industrial Revolution—or something far more profound?"
https://t.co/fUnuoGQwlD🔗#econtwitter

[Victoria Krakovna @vkrakovna]

We have made some updates to the Google DeepMind AGI safety course:
- New talk on Security & Control by @MaryPhuong10
- Updated talk on Interpretability by @NeelNanda5
- Updated talk on Robust training & monitoring by @seb_far

Check these out here: https://t.co/9Mts3DPT3Y

[Boaz Barak @boazbaraktcs]

Lesswrong version, which may have better formatting than my janky Wordpress blog https://t.co/OV0CRbdxqr

[Helen Toner @hlntnr]

In 2025, "automating AI research" went from abstract idea to common practice inside leading AI companies.

Out today: ✨new report✨ from a workshop we ran with some of the smartest people on this topic.

🧵:

https://t.co/Uj9UId1bT4

[Jerry Tworek @MillionInt]

I’ve talked to Ashlee about AI, humans and meaning of life

[James Zou @james_y_zou]

Standard AI learns to imitate.
We introduce a new framework that trains AI to make new discoveries in science + engineering.

Learning-to-discover + open source LM led to:
🥇best new bound on Erdos min overlap problem
🥇fastest GPU kernels
🥇better single-cell denoising
+ more!

[Alec Helbling @alec_helbling]

I wrote an interactive article explaining the geometric intuition behind Rectified Flows.

I visually explain why flow-models tend to learn curved trajectories, why this is bad for sampling latency, and a relatively simple technique for mitigating it.

Check it out! Link 👇

[Valerio Capraro @ValerioCapraro]

A new Nature paper shows what looks like a paradox.

Researchers who adopt AI tools publish more, receive more citations, and become PIs earlier.

At the same time, the scope of science appears to be narrowing.

How can we reconcile this apparent contradiction?

LLM outputs are, by construction, a combination of existing knowledge: an average of averages.

As reliance on LLMs increases, variance declines.

Productivity goes up. Creativity goes down.

*
Paper in the first reply

[Michael Elabd @MichaelElabd]

Amazing continual learning paper out of DeepMind 🚨

Most Continual Learning work assumes the backbone is fixed and the burden on the algorithm to fight catastrophic forgetting. This paper flips that assumption on its head and shows pretty convincingly that architecture choices matter just as much for the plasticity–stability trade-off.

A few takeaways that stood out to me:

Learning vs. retention is heavily architecture-dependent. ResNets and WideResNets are great at picking up new tasks, but they forget aggressively. On the other hand, simple CNNs and even ViTs are surprisingly good at retaining old knowledge, even if they learn new tasks more slowly.

Width beats depth. Making networks wider consistently reduces forgetting and improves average accuracy. Making them deeper often gives diminishing returns on learning while worsening forgetting.

Pooling is a hidden culprit: Global Average Pooling is a major driver of forgetting because it bottlenecks the final representation. Removing GAP or replacing it with smaller pooling layers significantly improves retention.

BatchNorm isn’t always helpful. It helps when task distributions are similar, but under large distribution shifts, BN can accelerate forgetting.

What I’d love to see next is this line of work pushed into the LLM regime (larger models, longer task sequences) so we can (1) benchmark continual learning methods more rigorously and (2) start designing architectures explicitly for continual learning, rather than inheriting them from static training.

[Greg Brockman @gdb]

3M lines written over a week of continuous agent time with GPT-5.2 — amazing glimpse of the future:

[Boaz Barak @boazbaraktcs]

Why we are excited about confessions
https://t.co/Mip50m7BZm

[Chao Huang @huang_chao4969]

🚀 2026 is here - time to do research WITH AI, not just ABOUT AI!

As AI agents evolved dramatically in 2025, we may be getting closer to true AI co-scientists! We want more than just AI for information retrieval - we want AI that does research with us, drives scientific discovery with us, and even pushes scientific innovation with us as true partners.

What would the ideal AI co-scientist look like? Our team has been exploring this question and continuously evolving Novix: AI Co-Scientist to become an amazing research partner. It dramatically boosts research efficiency, lets human scientists focus more on innovation, and honestly... makes doing research way more enjoyable ☕

🌐 Try our Novix demo: https://t.co/hB7kSBNf3V
💻 Open-sourced: https://t.co/O3arx6M0bs
📄 Technical report: https://t.co/rqKkTWG74a

What Novix can do for you:
- 📚 Deep survey & synthesis of existing research
- 💡 Sparks innovative research ideas
- 🔬 Designs & implements algorithms
- 🧪 Runs automated experiments & validation
- 📊 Analyzes results with deep insights
- ✍️ Assists with manuscript structure & drafting

Think of it as your 24/7 research partner that never needs coffee ☕

[John Carmack @ID_AA_Carmack]

I like and bookmark so many interesting sounding papers here, and don’t get back to most of them. Time to start making a dent. I’m going to try to at least skim one of the papers in my bookmarks each weekday for the rest of the month.

#PaperADay

2025: Emergent temporal abstractions in autoregressive models enable hierarchical reinforcement learning (Google)

I like their statement of the hierarchical goal problem  as “how long does it take a twitching hand to win a game of chess?”  @RichardSSutton is fond of the “options” framework in RL, but we don’t have a clear method to learn them from scratch.

Their Ant environment is designed to require two levels of planning: the standard mujoco Ant locomotion work to be able to move at all, and routing decisions to get to the colored squares in the correct order, which will happen hundreds of frames apart.

Basically, this takes a pre-trained sequence predicting model that predicts what separately trained expert models (manually steered) do, and inserts a metacontroller midway through it, which can tweak the residual values to perform high level “steering”, and can be RL’d at high level switch points to much greater performance than the base pre-trained model.

A key claim here is that learning to predict actions in a supervised next-token manner from lots of existing expert examples, even if you don’t know the goals, results in inferring useful higher level goals. This sounds plausible, but their experiment makes it rather easy for the model: the expert RL models that generated the training data were explicitly given one of four goals in each segment, and the option learning model just classifies the sequences into one of four categories. This is a vastly simpler problem than free form option discovery.

A State Space Model is used for the more complex Ant environments, while a transformer is used for the simpler grid world environments. I didn’t see an explanation for the change.

The internal “walls” are more like “poison tiles”, since they don’t block movement like the map edges, they just kill the ant when its center passes into them.

The 3D renderings (with shadow errors that hurt my gamedev eyes) are somewhat misleading, since it is really a 2D world that the agent gets to fully observe in a low dimensional one-hot format. It doesn’t do any kind of partially observed or pixel based sensing.

Everything is done with massively parallel environments, avoiding the harder online learning challenges.

The success rates still aren’t great after a million episodes.

I would like to see this applied to Atari, basically doing GATO with less capable experts or lower episode quantities, then trying to identify free form options that can be usefully used to RL to higher performance.

[Jack Bai @jackbot_cs]

😈 Today, we introduce WebGym, the largest-to-date open-source RL environment for web agent training that contains 300k tasks and a rollout framework optimized specifically for web environments' rollout speed. We reveal the effects of essential scaling directions we observe with WebGym.

1/n

[Vaishnavh Nagarajan @_vaishnavh]

1/ We found that deep sequence models memorize atomic facts "geometrically" -- not as an associative lookup table as often imagined.

This opens up practical questions on reasoning/memory/discovery, and also poses a theoretical "memorization puzzle."

[Chubby♨️ @kimmonismus]

Terence Tao confirms:
For the first time, an LLM (GPT-5.2 pro) has successfully solved an Erdos problem on its own.

This makes me really excited for GPT-5.3 pro. Science is gaining momentum, and the breakthroughs are becoming more significant.