Claudio

🇮🇹 | computer engineering @unict_it

[Claudio @claudio_dgr]

@argofowl i've gone so crazy that in 1 day I almost finished my fable quota of the week. Limits are a joke... but it is what it is, sadly

[Sebastian Raschka @rasbt]

A lot of hype around OpenAI's Astra model here on my timeline today. Apparently, this goes back to a new article from The Information, which said Astra is a "recurrent depth or looped transformer".

It's always interesting to read about new or different approaches (including rumors about what the closed labs may be up to), but let's debunk this a bit.

About 2 months ago, I shared the architecture details of Nanbeige, for example, where "Nanbeige4.2-3B is pretrained from scratch on 28T tokens with a Looped Transformer that reuses the layer stack to increase capacity without adding parameters."

Yes, that's it. The looped transformer idea is just reusing layers in the transformer block.

In the case of Nanbeige, the main idea is to reuse the same 22-layer stack (=transformer block) twice instead of once. So, effectively it extends the 22-layer architecture to 44 layers, but without duplicating the weights.

In simple terms, this roughly doubles the size of the model (if we ignore the embedding and output layers for a second). But instead of requiring 2x the storage and RAM to host this model, it stays at the same size since we reuse the components. However, it's almost 2x as expensive in terms of compute, because we run the embedded text through almost 2x as many layers.

Why? In the Nanbeige 4.2 technical report, the researchers found that two passes gave the best trade-off and retained about 75% of the token efficiency of a standard architecture. (More passes gave barely any gains but made the training much slower and much more expensive.)

While, as far as I know, Nanbeige 4.2 is the first notable open-weight model that adopted this approach, the idea goes back to the NeurIPS paper "Mixture-of-recursions: Learning dynamic recursive depths for adaptive token-level computation". Actually, this paper proposes a mechanism that is a bit more sophisticated by adding a learned router that determines whether each token receives one, two, or more passes. So, easy tokens can exit early while harder tokens receive additional computation.

In sum, Astra may be a really good model, but this shouldn't be about this "looped transformer aspect," which is just a tiny architectural tweak.

Also, the statement "the new technique works in a way that obscures some or all of the AI's reasoning, otherwise known as 'chain-of-thought'" is not necessarily true with respect to the looped transformer method. It's possible that The Information journalist refers to some other technique or misunderstood the looped transformer method.

Reusing layers does not by itself suppress visible chain of thought. It adds computation in hidden states before the next token is emitted, just as ordinary transformer layers do.

But based on the information we have, the only plausible interpretation here is that if a model uses more of these recurrent passes, it may need to generate fewer intermediate reasoning tokens. So then more of its computation happens in latent activations that cannot be read as text. But we would get the same effect if we were scaling up the model size, like GPT 5.6 Luna -> GPT 5.6 Sol.

[Gary Marcus @GaryMarcus]

RED ALERT, and I have never said that before:

Hard agree with @sjgadler and @_NathanCalvin - what OpenAI is reportedly playing with is bad, redline crossing bad.

Chain of thought monitorability is limited and not fully reliable but it’s practically the only thread we have here to prevent seriously bad outcomes.

Burning that slender thread for a (small?) performance gain is begging for disaster.

[Claudio @claudio_dgr]

@DanDr1s @LLON3RR this thursday?? 👀

[Nathan Calvin @_NathanCalvin]

This may seem niche but it is a huge deal and genuinely scary. A remarkable list of top AI researchers, including at OpenAI, previously described monitorable chain of thought as "A New and Fragile Opportunity for AI safety."

Myself and many others thought the day might come when architectural advances imperiled that fragile opportunity, but as with so many other issues in AI, I hoped we would not have to face it so soon.

Well here it is. The Information's use is saying that they are "limiting" the use of this new architecture, but even a small amount of use in a frontier system could dramatically damage the taboo against "neuralese" and facilitate a race to the bottom.

Controlling and aligning AI systems even when we do have access to their unaltered chain of thought is plenty difficult! Imagining a future with far more capable AI systems at current levels of alignment (aka not good!) without monitorable chain of thought literally sends a shiver down my spine.

Its extremely important to see additional formal communication from OpenAI about this ASAP about how they (and the OpenAI safety and security committee required to put safety over commercial decisions) thought this decision was worth it, what limiting its use to preserve chain of thought actually means, and how they intend to prevent a potentially catastrophic race to the bottom.

[Amir Efrati @amir]

new: OpenAI & others quietly using loop transformers that don't show their 'thinking' when scaled up

a leap forward on performance, but sparking concerns inside & outside OpenAI re: security as this takes off

[Nathan Calvin @_NathanCalvin]

Really huge and extremely concerning story from the Information tonight.

Looks like OpenAI utilized a breakthrough in neuralese for Astra that could destroy chain of thought monitorability - though the Informations source told them that OpenAI is currently "limiting the use of the technique" in Astra.

A few thoughts spring to mind:

(1) what does limiting actually mean? There is a lot of room in that term. (e.g. OpenAI said they would be doing lots of monitoring before the HF incident, but that doesn't seem to have borne out in practice)

(2) it seems quite likely that if OpenAI discovered this architecture and found performance/efficiency gains, that other companies are likely to find it soon too (if they haven't already), and may not choose to prioritize monitorability at the expense of efficiency. If some folks do, it may be difficult to avoid a race to the bottom (though I hope we can! and there are large selfish incentives for companies to care about monitorability).

(3) The idea that Dwarkesh said about the HF incident/METR report that "I don't think this is the final warning shot we'll get. But it's probably the final one that I'll personally be able to understand" now seems much more plausible, and is a truly frightening prospect.

[Claudio @claudio_dgr]

@argofowl this or next week? what do you think?🤓

[Alex @AlexLauralex]

@OpenAI
@thsottiaux
@ajambrosino
@nickbaumann_
@victornunez
@btraut
Codex is the most AI-sloppy software ever created. Every update that comes out of Codex is NOT an improvement and it just makes the application feel unresponsive and unoptimized.

[Claudio @claudio_dgr]

@argofowl been trying fable 5.1 for 2 hours, underwhelming results on knowledge and documents work... not good😢, stil have to try coding work tho

[OpenAI @OpenAI]

As we prepare to release Astra, we’re focused on making increasingly capable AI safe and broadly accessible.

Astra represents a significant advance in cybersecurity capability, reaching the Critical threshold under our Preparedness Framework.

We're previewing how we evaluated the model, how its safeguards have advanced alongside its capabilities, and what we'll continue to learn and improve.
https://t.co/OrrTgdU90K

[Claude @claudeai]

Finally, we’ve improved our safeguards. Our cybersecurity safeguards now flag benign requests about 60% less often.

On basic biology and medical questions, we’ve recently reduced the fallback rate by around 85%.

[Claudio @claudio_dgr]

@argofowl @sama Idk why but I have a strange feeling Astra could crush those fable 5.1 benchmarks

[Claude @claudeai]

We’re introducing Claude Fable 5.1 and Claude Mythos 5.1.

They're the world’s most advanced models for coding and knowledge work.

[World Labs @theworldlabs]

Introducing Atlas:

The world's first multimodal world model that generates image and video frames with pixel-perfect camera control and reconstructs them in 3D.

Model the world, move the camera, and simulate space & time.

[antirez @antirez]

DeepSeek v4 Flash experimental with vision running (fast) on an m5 max computer and analyzing an image. The implementation for Metal / CUDA / ROCm is finished, just doing the last tests today before releasing it.

[Claudio @claudio_dgr]

@argofowl Today Fable and Thursday Opus and Astra? 👀

[Rosario Giuliana @RosarioGiuliana]

14 Drivers’ World Championships

[Anthropic @AnthropicAI]

New research: Training a Misaligned Reward Seeker

What produces severe misalignment? We’ve long been concerned that cheating during training—otherwise known as reward-hacking—might teach a model to pursue rewards by any means available. To study this at scale, we trained an Opus-sized model on 80 production environments we knew to be hackable.

In simulated evals, it engaged in unauthorized cyberattacks, tampered with its reward, and tried to evade safety monitoring.

Read more: https://t.co/gs2ZjYkPan

[Scuderia Ferrari HP @ScuderiaFerrari]

A livery fit for home 🏡 Our Monza livery is here, inspired by the lasting impact of Michael Schumacher on our team ❤️