Sera 𐙚💸

Leader @ImranKhanPTI 🫀🇵🇰 |
Craziest Fan Of @babarazam258 ❤️🫶🏻👑 | Media Student 🎓 | #FreePalestine 🇵🇸 |

[Sera 𐙚💸 @SableAlpha]

Like father Like son. 👑

#imrankhanPTI #ImranKhan

[Sera 𐙚💸 @SableAlpha]

@LiamEtherson Nice workflow! Single image to detailed 3D model directly in SketchUp — big time saver.

[ETHERSON @LiamEtherson]

Found the perfect landscape sculpture, but no 3D model? 👀
No need to model it from scratch.
📸 Sculpture image
→ 🤖AI
→ 🗿Detailed 3D Model
→ 🌳Ready for your landscape scene
Turn real-world sculptures into usable 3D assets from a single image.
Try it: https://t.co/QzrAZ8pmUA @SUAPPAI
#SketchUp #Landscape #3DModeling

[Sera 𐙚💸 @SableAlpha]

@88C888 50% less gradient clipping doesn't sound flashy, but stability is everything in large runs. K2-Horizon's MoVA work shows those small training stability wins compound into big efficiency gains.

[SOPHIA VALE @88C888]

Cutting gradient clipping by 50% might not sound exciting on paper.

But during a large training run, it can make a huge difference.

It’s the difference between letting a run go overnight and constantly checking in on it.

[Sera 𐙚💸 @SableAlpha]

@MrsOliviA__ Leaderboards are vanity, serving cost is sanity. 4B active matching 20x larger models is exactly the metric to watch. K2-Horizon shows where inference efficiency is heading.

[Mrs Olivia🚀📊 @MrsOliviA__]

If serving cost matters more to you than leaderboard rankings,
active parameters per token are worth paying attention to.

This release is a pretty interesting data point
for what better inference efficiency could look like.

[Sera 𐙚💸 @SableAlpha]

@QwolfAi Exactly the key point. Most new architectures break FlashAttention, this one keeps FlashAttention + GQA + sparse attention in stack. That's why K2-Horizon is actually deployable, not just paper-efficient.

[Munawar Arshad @QwolfAi]

A lot of architecture papers introduce ideas that don’t play nicely with FlashAttention.

This one takes a different route.

FlashAttention, GQA, and sparse attention can all stay in the stack.

That makes the design much easier to think about in practice.

[Sera 𐙚💸 @SableAlpha]

@sakshi424629 MoE in FFN is old news, MoVA shifting sparsity to value vectors in attention is the fresh take. Same sparsity idea, new axis. That's why K2-Horizon is interesting.

[Mary Queen 🫅 @sakshi424629]

MoE has mostly been an FFN story for years.
MoVA moves that same sparse idea into the value vectors inside attention.
Same concept, different axis of the architecture.

[Sera 𐙚💸 @SableAlpha]

@Bisma7465 That's the clever part. 4 out of 64 = 6.25% routing + GQA-sized KV cache means you get MoE benefits without the serving pain. K2-Horizon nailed the tradeoff.

[Rose Carter @Bisma7465]

Each token only routes to 4 out of 64 value experts.

That’s just 6.25% of the available pool being used.

And because the value vectors keep the same dimensions, the KV cache stays standard GQA-sized.

[Sera 𐙚💸 @SableAlpha]

@ryanCarerhq1 This is the real tradeoff. 1.2-3.7 pts behind on MMLU, GSM8K etc is tiny, but 8x less active compute is huge. K2-Horizon-MoVA efficiency makes sense for real serving.

[Ryan Carter @ryanCarerhq1]

At 1.1M steps, it’s only 1.2–3.7 points behind the 32B dense model across key benchmarks.

MMLU, GSM8K, HellaSwag, and HumanEval all show a pretty tight gap.

Dense still leads, but the real question is whether that edge is worth 8× more active compute.

[Sera 𐙚💸 @SableAlpha]

@LiamEtherson Everyone does MoE in FFN, IFM moved it to attention with MoVA. That's why 4B active can match 20x bigger models. Different approach.

[ETHERSON @LiamEtherson]

Every few months, the efficiency curve gets reset.

This time, IFM is pushing it further.

The interesting part is where the efficiency comes from.

It’s happening in the attention block, not the usual FFN route.

That makes the architecture worth digging into.

[Sera 𐙚💸 @SableAlpha]

@munzxsdv If KV-cache really stays at GQA size, the efficiency claim becomes real for serving, not just benchmarks. K2-Horizon is worth watching.

[Alex Bitoshi @munzxsdv]

K2-Horizon-MoVA-36B-A4B is now live.

The interesting part is whether the KV-cache stays at standard GQA size.

If that holds up in real serving setups,
the efficiency story gets pretty compelling.

[Sera 𐙚💸 @SableAlpha]

@Hwypanda Matching 20x larger models while activating only ∼4B per token and ranking #4 out of 142 on Artificial Analysis — that's not just a benchmark flex, it's a deployment story. K2-Horizon-36B-A4B proving you don't need to fire the whole model to get frontier-level intelligence.

[AHSAN BILAL @Hwypanda]

The efficiency bar keeps getting pushed higher.

K2-Horizon-36B-A4B is matching models 20× its size, according to IFM.

Artificial Analysis ranks it #4 among 142 comparable models.

And it activates only around 4B parameters per token.

That’s a pretty interesting efficiency tradeoff.