Alessio Serra

ai researcher | making the stochastic parrot smarter @Aleph__Alpha

[Alessio Serra @aserra___]

A dream is what you think about before you fall asleep in your bed.

A project is what you think about in the morning when you wake up to plan your actions.

Don’t follow your dreams, build projects.

[Alessio Serra @aserra___]

nice moe design approach:
start from the serving bottlenecks in latency- and throughput-sensitive regimes, shrink the expert dimension, then reinvest the savings into larger top-k + more experts.

better accuracy per flop.

https://t.co/jwSF8p146I

[Alessio Serra @aserra___]

the discretization approach used here reminds me of CPU clock for synchronizing state updates

every 200ms, audio/video/text streams are sliced into one timestep and fed to the model as the next interaction step

i wonder how much you can transfer from computer architecture ideas

[Thinking Machines @thinkymachines]

People talk, listen, watch, think, and collaborate at the same time, in real time. We've designed an AI that works with people the same way.

We share our approach, early results, and a quick look at our model in action.

https://t.co/AFJZ5kH7Ku

[Alessio Serra @aserra___]

EMO gets semantic-level expert specialization by forcing tokens from the same document to route within a shared expert pool.

cool result: for a domain/use case, you can keep only 25% of experts and get just a 1% absolute performance drop.

https://t.co/T3STIO0Wja

[MiniMax (official) @MiniMax_AI]

Introducing MiniMax M3: The First Open-Weights Model to Combine Three Frontier Capabilities

- Coding & Agentic Frontier: 59.0% SWE-Bench Pro, 66.0% Terminal Bench 2.1, 34.8% SWE-fficiency, 28.8% KernelBench Hard, 74.2% MCP Atlas
- MiniMax Sparse Attention scales context to 1M
- Natively Multimodal from Step Zero

API: https://t.co/fHRdSV7BwZ
Token Plan: https://t.co/BDCycxepZw
🚀New! MiniMax Code: https://t.co/GvB4YiB6Ul

Weights & Tech Report in ~10 Days

[Alessio Serra @aserra___]

next level engineering rounding

[Alessio Serra @aserra___]

@swpnldubey he is ok track to pass the probation period

[Tilde @tilderesearch]

~1/7~Introducing Parallax → a stronger attention variant that achieves a Pareto improvement over vanilla attention at 0.6B and 1.7B scales.

Parallax has better perplexity, better downstream accuracy, and a decode kernel that matches or beats FlashAttention.

🧵

[Alessio Serra @aserra___]

@dwarkesh_sp non-verifiable domains do not exist, they are simply domains more difficult to verify

[Alessio Serra @aserra___]

llama may have made long context extension look easier than it is.

OlmPool holds data + extension fixed and finds llama 3 extends far better than qwen-3 and olmo-3, mainly because it avoids qk norm and sliding window attention while pretraining at 8K.

https://t.co/o4yZMw8BGC

[Alessio Serra @aserra___]

@MoFromYYZ i know an openai engineer who spends $1.3M per month.

[Peter Steinberger 🦞 @steipete]

The latest CodexBar update renders API costs wayyyy nicer. https://t.co/lJ4dxNHwzG

[Alessio Serra @aserra___]

@paulg The interesting thing is that colors probably disappeared because they were cheaper and easier to standardize. But now we consider black/grey/white elegant. Something colourful is the exception and sometimes a marketing move, e.g. the iPhone 17 Pro.

[Alessio Serra @aserra___]

@elonmusk so now only google and cohere are left using jax?

[Alessio Serra @aserra___]

@AnthropicAI @AltimeterCap @Greenoaks @sequoia we need a new term for the $1T pre-IPO valuation

the second case is already within rounding distance

[Alessio Serra @aserra___]

we need a new term for the $1T pre-IPO valuation

the second case is already within rounding distance

[Anthropic @AnthropicAI]

We've raised $65 billion in Series H funding at a $965 billion post-money valuation, led by @AltimeterCap, Dragoneer, @Greenoaks, and @sequoia.

This investment will help us advance our research and expand our capacity to meet growing demand for Claude.

[Alessio Serra @aserra___]

wonder how much data from mythos made it into this post-train

opus 4.8’s big deltas are USAMO + GraphWalks, as was for mythos

[Claude @claudeai]

Introducing Claude Opus 4.8: it builds on Opus 4.7 with sharper judgment, more honesty about its own progress, and the ability to work independently for longer than its predecessors.

Available today at the same price.

[Alessio Serra @aserra___]

@himanshustwts margins are already good, there’s no need to increase prices until a new pretraining of a larger model

[Alessio Serra @aserra___]

interesting direction for memory consolidation at the cache eviction boundary

before clearing a filled KV cache, the model runs N recurrent passes over accumulated context and updates persistent fast weights inside its SSM blocks

https://t.co/awMF7rde2c

[Alessio Serra @aserra___]

most impressive result is mimo-v2.5-pro from @Xiaomi

beats deepseek-v4-pro despite being smaller (1.02T-A42B vs 1.6T-A49B)

[Serena Ge (Datacurve) @serenaa_ge]

Today we’re releasing DeepSWE, a new standard for agentic coding benchmarks.

On public leaderboards, top models often look relatively close in capability. DeepSWE shows where they actually diverge, reflecting the realistic experience of developers in their day-to-day work.

[Alessio Serra @aserra___]

source:

[Skyler Miao @SkylerMiao7]

Something BIG is coming

[Alessio Serra @aserra___]

new sparse attention variant for 1M context from @MiniMax_AI

it adds a tiny GQA index branch to pick top-k KV blocks, then runs attention on the original KV for those blocks.

closer to CSA-style block routing than DSA, but without doing attention in compressed MLA space