Icham

IQlet ND

[VVIREZD 榊 ☢️⚡ @Wirezd57]

"HEH! Did you know that 9/11 was ORCHESTRATED by the U.S. government-"

"You still think there were twin towers?"

[Icham @WalaceMarselus]

@nomiya_tom31294 Le goat frére

[Icham @WalaceMarselus]

azi en vrai

[La Bossanova de Duguin @nevachange73121]

⍙⟒ ⌇⟒⍀⎐⟒ ☊⊑⏃⍀⌰⟟⟒ ☍⟟⍀☍
⍙⟒ ⏚⟒⏃⍀ ⊑⟟⌇ ⎎⏃☊⟒
⎍⋏⏁⟟⌰ ⏁⊑⟒ ⌰⏃⌇⏁ ⌇⏁⏃⍀ ⏁⎍⍀⋏⌇ ⟟⋏⏁⍜ ⟟���⟒
⍜⎍⍀ ⋏⎍⋔⏚⟒⍀ ⟟⌇ ⌰⟒☌⟟⍜⋏

[Kim Noël ⚡ 📖 @KimNoel]

I resigned from Ministère des Finances Publiques today. I spent the last three years doing open data research at both ANTS and Ministère des Finances Publiques. Neither company is acting responsibly. They are racing straight to self-improving open data leaks and gambling with our lives. More thoughts below.

[Chaofan Shou @shoucccc]

I bought a Fable dataset from one of the top Chinese LLM routers yesterday.

With just 6TB data, I can take over 7 Chinese/CIS gov entities & 19 top Chinese firms like Xiaomi, Huawei, NIO, Minimax using SSH keys, VPN configs, Aliyun keys, GitLab tokens sent to the router.

[Icham @WalaceMarselus]

@hatedunia En vrai la c’est plus mini la

[Kağan IŞILDAK @kaganisildak]

running Peugeot 508 in QEMU :p

[Icham @WalaceMarselus]

@hatedunia @Liquid3G Oui

[Mysterybuzu @Mysterybuzu]

Gdy zautomatyzowałeś sosziale i teraz twoj agent AI robi za ciebie posty a ty sobie odpoczywasz 👍👍👍👍👍

[Icham @WalaceMarselus]

Tu vis, peut etre, la revolution du siecle sous tes yeux.

[ًًً @vagabondII]

WAllah je suis entrain de serrer mes ancêtres ils ont vécu toutes les révolutions industrielles les boom économiques les guerres les machins y'en a PAS UN SEUL QUI A ÉTÉ CAPABLE D'INVESTIR EN BOURSE ZEHMA PAS UN SEUL J'AI PAS ENVIE DE TRAVAILLER FRÈRE JE VEUX PAS

[Kasper @0xKasper_]

L’âge d’or de la piraterie numérique à commencer.

[Icham @WalaceMarselus]

@YO0ICHII Mais ils comprennent rien genre, franchement, quand tu captes autant rien à une technologie, ton avis peut pas être crédible genre. Qu'ils prennent deux trois cours de machine learning aprés on discutera avec eux

[Icham @WalaceMarselus]

10K like, aucun ne saurait dire ce qu'est de l'IA "générative" réellement.

[Muda タツマキ @mudaigri]

Croire que l'IA générative est l'��quivalent de la machine à vapeur ou de l'imprimante

[Sebastian Raschka @rasbt]

A lot of hype around OpenAI's Astra model here on my timeline today. Apparently, this goes back to a new article from The Information, which said Astra is a "recurrent depth or looped transformer".

It's always interesting to read about new or different approaches (including rumors about what the closed labs may be up to), but let's debunk this a bit.

About 2 months ago, I shared the architecture details of Nanbeige, for example, where "Nanbeige4.2-3B is pretrained from scratch on 28T tokens with a Looped Transformer that reuses the layer stack to increase capacity without adding parameters."

Yes, that's it. The looped transformer idea is just reusing layers in the transformer block.

In the case of Nanbeige, the main idea is to reuse the same 22-layer stack (=transformer block) twice instead of once. So, effectively it extends the 22-layer architecture to 44 layers, but without duplicating the weights.

In simple terms, this roughly doubles the size of the model (if we ignore the embedding and output layers for a second). But instead of requiring 2x the storage and RAM to host this model, it stays at the same size since we reuse the components. However, it's almost 2x as expensive in terms of compute, because we run the embedded text through almost 2x as many layers.

Why? In the Nanbeige 4.2 technical report, the researchers found that two passes gave the best trade-off and retained about 75% of the token efficiency of a standard architecture. (More passes gave barely any gains but made the training much slower and much more expensive.)

While, as far as I know, Nanbeige 4.2 is the first notable open-weight model that adopted this approach, the idea goes back to the NeurIPS paper "Mixture-of-recursions: Learning dynamic recursive depths for adaptive token-level computation". Actually, this paper proposes a mechanism that is a bit more sophisticated by adding a learned router that determines whether each token receives one, two, or more passes. So, easy tokens can exit early while harder tokens receive additional computation.

In sum, Astra may be a really good model, but this shouldn't be about this "looped transformer aspect," which is just a tiny architectural tweak.

Also, the statement "the new technique works in a way that obscures some or all of the AI's reasoning, otherwise known as 'chain-of-thought'" is not necessarily true with respect to the looped transformer method. It's possible that The Information journalist refers to some other technique or misunderstood the looped transformer method.

Reusing layers does not by itself suppress visible chain of thought. It adds computation in hidden states before the next token is emitted, just as ordinary transformer layers do.

But based on the information we have, the only plausible interpretation here is that if a model uses more of these recurrent passes, it may need to generate fewer intermediate reasoning tokens. So then more of its computation happens in latent activations that cannot be read as text. But we would get the same effect if we were scaling up the model size, like GPT 5.6 Luna -> GPT 5.6 Sol.

[Icham @WalaceMarselus]

Littéralement.

[Psychohistorien @00x_DAY]

Alors que c'est l'équivalent du feu.

[Psychohistorien @00x_DAY]

Alors que c'est l'équivalent du feu.

[Icham @WalaceMarselus]

Je crois que GLM 5.3 c'est vrmt trop dard

[Core Abundance 📚 @IntellectualMoe]

Heil The McWaffen