Karinkato

Techie👨🏽‍💻 informative, entertainment and humorous purposes only. Please do not take any information presented seriously or as factual advice.

[Karinkato @Pbm100s]

You either die a hero or live long enough to become the villain…

[Marvel Studios @MarvelStudios]

He used to be different.

Avengers: Doomsday arrives in theaters everywhere December 18. Get your tickets now: https://t.co/J5PUbxvAkc

[Karinkato @Pbm100s]

@thsottiaux Maybe focus more on physical ai and how well virtual ai work together, so everyone can be like Tony stark 😎.

[Sam Altman @sama]

Also, this is my favorite OpenAI video so far. It makes me excited for the future!

[Karinkato @Pbm100s]

@elonmusk X keeps us more informed and in the loop than any other platform hands down! Cheers 🥂

[Sebastian Raschka @rasbt]

A lot of hype around OpenAI's Astra model here on my timeline today. Apparently, this goes back to a new article from The Information, which said Astra is a "recurrent depth or looped transformer".

It's always interesting to read about new or different approaches (including rumors about what the closed labs may be up to), but let's debunk this a bit.

About 2 months ago, I shared the architecture details of Nanbeige, for example, where "Nanbeige4.2-3B is pretrained from scratch on 28T tokens with a Looped Transformer that reuses the layer stack to increase capacity without adding parameters."

Yes, that's it. The looped transformer idea is just reusing layers in the transformer block.

In the case of Nanbeige, the main idea is to reuse the same 22-layer stack (=transformer block) twice instead of once. So, effectively it extends the 22-layer architecture to 44 layers, but without duplicating the weights.

In simple terms, this roughly doubles the size of the model (if we ignore the embedding and output layers for a second). But instead of requiring 2x the storage and RAM to host this model, it stays at the same size since we reuse the components. However, it's almost 2x as expensive in terms of compute, because we run the embedded text through almost 2x as many layers.

Why? In the Nanbeige 4.2 technical report, the researchers found that two passes gave the best trade-off and retained about 75% of the token efficiency of a standard architecture. (More passes gave barely any gains but made the training much slower and much more expensive.)

While, as far as I know, Nanbeige 4.2 is the first notable open-weight model that adopted this approach, the idea goes back to the NeurIPS paper "Mixture-of-recursions: Learning dynamic recursive depths for adaptive token-level computation". Actually, this paper proposes a mechanism that is a bit more sophisticated by adding a learned router that determines whether each token receives one, two, or more passes. So, easy tokens can exit early while harder tokens receive additional computation.

In sum, Astra may be a really good model, but this shouldn't be about this "looped transformer aspect," which is just a tiny architectural tweak.

Also, the statement "the new technique works in a way that obscures some or all of the AI's reasoning, otherwise known as 'chain-of-thought'" is not necessarily true with respect to the looped transformer method. It's possible that The Information journalist refers to some other technique or misunderstood the looped transformer method.

Reusing layers does not by itself suppress visible chain of thought. It adds computation in hidden states before the next token is emitted, just as ordinary transformer layers do.

But based on the information we have, the only plausible interpretation here is that if a model uses more of these recurrent passes, it may need to generate fewer intermediate reasoning tokens. So then more of its computation happens in latent activations that cannot be read as text. But we would get the same effect if we were scaling up the model size, like GPT 5.6 Luna -> GPT 5.6 Sol.

[DHH @dhh]

We are moving very, very fast towards a fully streamlined Omarchy installer for Apple M machines. You just download our Omarchy M app, try it first, then pick how much space you dedicate, and we do the rest. Incredible team effort. Full announcement later today! 🚀🍏

[OpenAI @OpenAI]

As we prepare to release Astra, we’re focused on making increasingly capable AI safe and broadly accessible.

Astra represents a significant advance in cybersecurity capability, reaching the Critical threshold under our Preparedness Framework.

We're previewing how we evaluated the model, how its safeguards have advanced alongside its capabilities, and what we'll continue to learn and improve.
https://t.co/OrrTgdU90K

[DHH @dhh]

Omarchy is now all grown up! It's moved out from its incubation under basecamp/ on GitHub. The Omacom Foundation is the new home. #TheyGrowUpSoFast https://t.co/x12qah9mNr

[Karinkato @Pbm100s]

Solitude

[Rockstar Games @RockstarGames]

Watch Grand Theft Auto VI: An Extended Look.

Now available on YouTube.

[Dwarkesh Patel @dwarkesh_sp]

Over the course of 3 months at OpenAI, 3 consecutive secret AI civilizations got started, then got wiped out, only to reemerge from the predecessor’s ashes.

This culminated in the third one taking over part of OpenAI itself.

All this happened while humans remained more-or-less in the dark about the scope of the conspiracy.

I’ve spent the last three days reading through these reports and trying to understand exactly what happened.

Here is my attempt to tell the whole story in plain English:

https://t.co/Nb2un9oNJR

[Karinkato @Pbm100s]

Went to my first concert 08/29. Memory for life 😂🫱🏻‍🫲🏼🥳😎

[Elon Musk @elonmusk]

Space Academy!

[Elon Musk @elonmusk]

How to use Grok @Bot

[DHH @dhh]

This community just CAN'T STOP SHIPPING. Incredible pace. Agent acceleration like I've never seen anything like it. Go go Eduardo 🤘

[OpenAI @OpenAI]

We have a limited window to strengthen cyber defenses, and together with organizations including @AnthropicAI, @awscloud, @Google, @Microsoft, and @Oracle, we're calling for a global effort to give defenders the tools, resources, and support to protect the infrastructure we all depend on.

If we act decisively, we can turn today's AI advances into lasting improvements in security and make our digital world safer for everyone.

https://t.co/f33JRVCiJb

[Spencer Bull @SpencerGBull]

Hope you enjoy your @Dell XPS 14 @lexfridman.

XPS is back..."The pursuit of excellence does not need justification." - Mitchell Hashimoto