✞𓃵✎3M_eritus𓃰✎✟

AI/ML || Agentic LLM || cuda-infernce || Ai security || QuantML || Aspiring researcher

[✞𓃵✎3M_eritus𓃰✎✟ @Wilsonafolabi2]

Silent mode

[Kimi.ai @Kimi_Moonshot]

Releasing the model weights and technical report of Kimi K3.

Kimi K3 is our most capable model: a 2.8T MoE model with native visual understanding and a 1M-token context window.

New model architecture: 2.5x the intelligence per unit of compute, not just more params.

Alongside Kimi K3, we're opening up more of the stack behind it — high-performance attention kernels, MoE communication library, and infrastructure for running agent environments at scale.

Model weights: https://t.co/7m7eEg6Y0B
Tech report: https://t.co/yeu6cjpMCT
Tech blog: https://t.co/YTfiMSNM1f

[✞𓃵✎3M_eritus𓃰✎✟ @Wilsonafolabi2]

@rzerobzerot @Rolf_Drechsler @agra_uni_bremen @DSC_unibremen Thanks boss

[✞𓃵✎3M_eritus𓃰✎✟ @Wilsonafolabi2]

@coderoflagos @docnyio Congratulations

[Opemipo Disu @coderoflagos]

🎉Today, I'm launching Docny (@docnyio).

Documentation is one of the most important parts of building a technical product, yet it's often the hardest thing to keep up to date.

Teams write docs in one place, manage product work in another, support customers somewhere else, and maintain API references separately.

I started building Docny to bring those workflows together and help developers keep their docs in sync using AI.

With Docny, teams can create beautiful documentation, API references, guides, and developer portals while connecting them to the tools they already use. 🛳️

After months of building, redesigning, testing, and learning, it's finally live.

https://t.co/UsA4b6lMmE

[✞𓃵✎3M_eritus𓃰✎✟ @Wilsonafolabi2]

@frontend__fairy Wow a year already

[✞𓃵✎3M_eritus𓃰✎✟ @Wilsonafolabi2]

@The_Lords_Net @Efootball__PES It used to be in pes era
But got removed

[Jun Song @jun_song]

Everyone is waiting for Qwen3.8-27b.

[✞𓃵✎3M_eritus𓃰✎✟ @Wilsonafolabi2]

@jun_song Reallllll

[✞𓃵✎3M_eritus𓃰✎✟ @Wilsonafolabi2]

@WescheNex1q Best local lm so far

[Sebastian Raschka @rasbt]

Yes, open-source / open-weight models are important for a healthy AI ecosystem. That's how we can verify things, check claims, and keep up outside the closed labs. Plus, it gives us the freedom to run AI on our own hardware if we are not ready to share personal data and IPs with closed labs through using their models. (Not that proprietary models are bad, actually I use them a lot as well, but it wouldn't healthy not to have any alternatives.)

Anyway, while pretty much everyone is waiting for the Kimi K3 and Ling 3.0 weights to land on the model hub any day now, there were quite a few other interesting new open-weight model releases the past week. Yes, one of those weeks!

So, here are the architecture pics along with some notes on what I found most interesting:

1) Nanbeige 4.2 3B uses looped depth sharing. This basically means it runs the same 22-layer (=transformer block) stack twice. So, it extends the 22-layer architecture to 44-layers, but without duplicating the weights. (2x the transformer block compute but same memory footprint.)

Why? The info is a bit sparse, but section 2.1 of the Nanbeige 4.2 technical report says two passes gave the best trade-off and retained about 75% of the token efficiency of a standard architecture. More passes gave barely any gains but made the training much slower and much more expensive.

2) Laguna S 2.1 is poolside's Laguna model in a really nice size: 118B sparse MoE with 8B active parameters and a 1M-token context window. Otherwise, the architecture is pretty standard. It uses 36 sliding-window and 12 global (gated-)GQA layers. However, given this size, and the fact that it (just barely) runs on my DGX Spark (uses about <80 GB of RAM), this is right now the most interesting model for me personally. It's 3x bigger and thus a tad slower but maybe a good candidate as daily-driver-Qwen3.6-35B-replacement. (Still waiting on some more independent performance benchmarks though.)

3) Motif-3-Beta is a new 314B-A13B sparse MoE that is somewhat based on DeepSeek V4 in terms of mHC and latent attention. But it uses a new component, Grouped Differential Latent Attention, which is inspired by Multi-head Latent Attention. I probably should write an article about this some time, but for now, the tl;dr is as follows. Regular MLA compresses the keys and values into a smaller latent representation to mainly reduce the KV cache size. GDLA does a similar low-rank compression but puts the attention heads into groups and also learns a noise head for each group where the noise gets subtracted for filtering purposes... Anyway, a topic for another day!

4) Solar Open 2 is a new 250B-A15B hybrid MoE by Upstage that interleaves three Kimi Delta Attention layers with one GQA layer.

5) Antares 1B is a small model (and there is also an even smaller 0.3B variant) from Cisco starts that with the IBM Granite 4.0 1B backbone and uses SFT plus GRPO for terminal-based cybersecurity stuff. It is a nice example of task-specific post-training on a genuinely small model.

6) BTL-3 is a rank-32 LoRA adapter for Qwen3.6-27B aimed at coding agents and structured tool use. The really strong benchmark performance suggests that LoRA adapters are still a useful tool/technique in 2026.

I added all six to the LLM Architecture Gallery for some additional details:
https://t.co/JDtfup3ncn

[✞𓃵✎3M_eritus𓃰✎✟ @Wilsonafolabi2]

@sw_holocron @EW Acolyte top5
Who ranked this ???

[✞𓃵✎3M_eritus𓃰✎✟ @Wilsonafolabi2]

@King0fhell_ 😹😹😹

[✞𓃵✎3M_eritus𓃰✎✟ @Wilsonafolabi2]

@3rdsonx I'm an example 😅😅omo

[✞𓃵✎3M_eritus𓃰✎✟ @Wilsonafolabi2]

@vibew_muiz Wow

[Elon Musk @elonmusk]

(Former) Trillionaire

[✞𓃵✎3M_eritus𓃰✎✟ @Wilsonafolabi2]

@JensenHuang @nvidia Wow.

[Jensen Huang @JensenHuang]

For my first post, I’m sharing a letter @NVIDIA signed on why open models matter.

AI will transform every industry, power every company, and be built by every country.

Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty.

The world needs both frontier closed models and frontier open models.

https://t.co/AUKzoQ5Ikb

[✞𓃵✎3M_eritus𓃰✎✟ @Wilsonafolabi2]

Them no real at all

[Girish @lilrishgi]

😂😂😂😂😂😂😂

[✞𓃵✎3M_eritus𓃰✎✟ @Wilsonafolabi2]

@FabrizioRomano He has gathered haters
I will be there for his misplaced passes
Possession lost
Duels lost
Own goals
Red cards
Everything

[✞𓃵✎3M_eritus𓃰✎✟ @Wilsonafolabi2]

Blood of jesus

[amrit @amritwt]

Of course they raised $300 million