redDev

SWE with EEE degree | Here to absorb all things tech, development and AI | 🔰 GGMU 🔰

[redDev @redDevi1337]

@FabrizioRomano But Man Utd plz @Cristiano

[redDev @redDevi1337]

Bold prediction Man Utd winning 2026/2027 PL

[redDev @redDevi1337]

Mila jovic 😮‍💨

[Lalinha 🗣 @Lalinhafofoca]

Não tenho estrutura pra não rir disso. Mbappé ficou incrível. 🤣🤣

[Mrs B @attackdogX]

Animals are actually smarter than us because in their packs, they don’t allow the weaker or retarded ones to lead them.

[Hemant Mohapatra @MohapatraHemant]

The curse of discipline is that every day looks the same. The curse of indiscipline is that every year does.

[Thrilla the Gorilla @ThrillaRilla369]

“No way you wrote a 5 page essay without AI”

Buddy, we wrote them without even reading the book.

[redDev @redDevi1337]

project so big .git is 1.3 GB

[Jackson | Husdog🦊💍 @Jackson_Boof]

Finally. Competitive Blender.

[conor brennan-burke @contextconor]

whoever successfully rebranded ‘bots’ as ‘agents’ is a god-tier marketer

[Sebastian Raschka @rasbt]

The Kimi K3 architecture figure for yesterday's big open-weight model release, along with some observations and thoughts.

1. Yes, it looks relatively complicated, but it's essentially a scaled-up production version of their Kimi Linear model they released last year (scaled up from 48B -> 2.8T; K3 is by far the biggest open-weight model right now)

2. The one new component compared to Kimi Linear is the LatentMoE. I omitted it in the figure below since it's already very crowded, but that's essentially the same LatentMoE as in Nemotron 3 Ultra (you can find it in my LLM Architecture Gallery if you are curious). The idea here is to compress (down-project) large linear layers similar to multi-head latent attention.

3. Kimi K3's overall trend (similar to Nemotron 3, DeepSeek V4, and others) is also towards better inference efficiency. That is, there are many components that replace existing components with efficiency-tweaked versions. I.e., MoE -> LatentMoE, regular attention -> multi-head latent attention and Kimi Delta Attention. (I also have short tutorials and write-ups in my gallery if you are curious about additional details).

4. The one component change that is not an efficiency tweak is attention residuals. Like DeepSeek V4 improved the residual path with mHC (manifold-constrained Hyper-Connections), attention residuals are a way to improve the residual path, but it works a bit differently. I.e., mHC made the residual path wider. Attention residuals (also already part of Kimi Linear) connect the residuals across layers; the connection itself uses an attention score for an important/contribution weight. According to the report, it improves the validation loss and downstream performance (a bit) consistently and adds about 4% in training cost and 2% in inference cost.

5. Interestingly, Kimi K3 got rid of all RoPE layers and uses NoPE (No Positional Embeddings) everywhere instead. (Again, this is inherited from Kimi Linear). In other architectures, the recent trend was towards RoPE in local attention layers (like sliding window attention) and NoPE in the global layers. There were a few architectures that only used NoPE everywhere, but this is the first frontier-level one as far as I know.

6. Kimi K3 now also has native multimodal support, which is great!

There are several other interesting training tidbits in the technical report, but that's it from the architecture front so far. A really great release overall.

[redDev @redDevi1337]

This is huge

[NIK @ns123abc]

🚨BREAKING: China has begun mass production of DUV lithography machines for the first time

ASML: nobody can touch our monopoly

China: targeting 5 DUV machines this year and 20 next year

Chip equipment stocks:

ASML -8.3%
ASM International -7.3%
BE Semiconductor -9.9%
Applied Materials -6.7%
Lam Research -7.9%

It’s over

[The xG Philosophy @xGPhilosophy]

Argentina created 0.00(xG) until the 117th minute 🤯

[Toni Kroos @ToniKroos]

Football won

[Karl Sharro @KarlreMarks]

104 matches played to get to this shit

[redDev @redDevi1337]

RIP BOZO VARgentina

[redDev @redDevi1337]

This is more WWE than football. 😂

[redDev @redDevi1337]

Spain choking?