diego

🐗

[diego @WithDiegx]

A good example of how creative reward functions can get:
"The task here involves taking in a long document and writing a summary. The quality of that summary is measured by giving it to a base model with no other context, and evaluating QA performance on QA pairs about the full, original document. A summary is good context if it suffices to answer unseen QA pairs, which ask about fine-grained details, higher-level inferences, and subtext, in the original document. The QA pairs have a verifiable ground truth answer, and are posed as multiple-choice questions."

[Applied Compute @appliedcompute]

We used RL to train models that create curated context from long documents for downstream use by agents. The models sometimes learn to invent their own abbreviations and shorthand. Optimizing with RL for downstream use produces very different artifacts from ordinary summaries: shorter, denser, creatively concise. We call these neural cheat-sheets.

[diego @WithDiegx]

@XMihura REFUGIO

[diego @WithDiegx]

The Gemini Flash series was so much better before, right now with their increased pricing it is hard to find value in them for synthetic data besides speed, you can get much better rate limits using chinese models at the same intelligence

[diego @WithDiegx]

There is not a comparable model anywhere to the Flash Lite series, I will probably burn some hundred million tokens of this new small beast

[Noam Shazeer @NoamShazeer]

📢Introducing Gemini 3.1 Flash-Lite, our fastest and most efficient model, built for high-volume workloads. It outperforms 2.5 Flash in reasoning, reliability, and scalability at a lower cost.

This model also introduces thinking levels. You can adjust compute by complexity of the task, burning zero thinking overhead on high-volume tasks, while reasoning through the complex edge cases.

Maximum intelligence, minimal latency.

Read more: https://t.co/FRhfbzigbv

[diego @WithDiegx]

"Priced at just $0.25/1M input tokens and $1.50/1M output tokens"
Ouch this hurts, the previous one was just 0.1/0.4

[diego @WithDiegx]

In a single H100, one of these or gpt-oss for synth data, I'm not sure

[Florian Brand @xeophon]

qwen3.5 27B, 35B-A3B (incl. Base!!!!), 122B-A10B out

[diego @WithDiegx]

With this quant, the new StepFun 3.5 Flash or Minimax M2.1 could become beasts for agentic synthetic data. I'm so tired of API rate limits and slowness...

[Red Hat AI @RedHat_AI]

@casper_hansen_ Yes, if you limit the sequence length (<100k). The model weights fit into 1xB200, but the kv-cache for the complete sequence length (~260k) doesn't. To run the model in 1xB200 you need to reduce the sequence length to ~100k (--max-model-len 100000).

Still pretty impressive?

[diego @WithDiegx]

Don't worry, these things will be implemented by Indra or similar so they probably won't work and will take many years

[Pavel Durov @durov]

Today, Telegram notified all its users in Spain with this alert:

Pedro Sánchez’s government is pushing dangerous new regulations that threaten your internet freedoms. Announced just yesterday, these measures could turn Spain into a surveillance state under the guise of “protection.” Here’s why they’re a red flag for free speech and privacy:

1. Ban on social media for under-16s with mandatory age verification: This isn’t just about kids—it requires platforms to use strict checks, like needing IDs or biometrics.

⚠️ Danger: It sets a precedent for tracking EVERY user’s identity, eroding anonymity and opening doors to mass data collection. What starts with minors could expand to all, stifling open discourse.

2. Personal and criminal liability for platform executives: If “illegal, hateful, or harmful” content isn’t removed fast enough, bosses face jail.

⚠️ Danger: This will force over-censorship—platforms will delete anything remotely controversial to avoid risks, silencing political dissent, journalism, and everyday opinions. Your voice could be next if it challenges the status quo.

3. Criminalizing algorithm amplification: Amplifying “harmful” content via algorithms becomes a crime.

⚠️ Danger: Governments will dictate what you see, burying opposing views and creating echo chambers controlled by the state. Free exploration of ideas? Gone—replaced by curated propaganda.

4. “Hate and polarization footprint” tracking: Platforms must monitor and report how they “fuel division.”

⚠️ Danger: Vague definitions of “hate” could label criticism of the government as divisive, leading to shutdowns or fines. This can be a tool for suppressing opposition.

These aren’t safeguards; they’re steps toward total control. We’ve seen this playbook before—governments weaponizing “safety” to censor critics. On Telegram, we prioritize your privacy and freedom: strong encryption, no backdoors, and resistance to overreach.

✊ Stay vigilant, Spain. Demand transparency and fight for your rights. Share this widely—before it’s too late.

[diego @WithDiegx]

Man, does this come in clutch for me

[Qwen @Alibaba_Qwen]

Qwen3-ASR and Qwen3-ForcedAligner are now open source — production-ready speech models designed for messy, real-world audio, with competitive performance and strong robustness.
● 52 languages & dialects with auto language ID (30 languages + 22 dialects/accents)
● Robust in noisy and complex settings (yes, singing and songs too)
● Long audio support: up to 20 minutes per pass
● Word/phrase-level timestamps:  high-precision alignment for 11 languages via Qwen3-ForcedAligner, stronger than MFA/CTC/CIF-style aligners
Also included: a full open-source inference & finetuning stack with vLLM batch, streaming, and async serving.
GitHub:  https://t.co/yxxUTTIjLr
Hugging Face: https://t.co/38fny0K2Qz
ModelScope: https://t.co/0fAIFksAwv
Hugging Face Demo:  https://t.co/GbLHOn7hdD
ModelScope Demo: https://t.co/Jvb2kfnHr5
Blog:  https://t.co/yP6VbGc7bg
Paper:  https://t.co/zzlFgwcP1E

[diego @WithDiegx]

What a detailed report! I like that they write a good overview of their agentic search data synthesis pipeline. Currently there are mainly two ways of approaching this, graph-based synthesis (Alibaba DeepResearch, @SID_AI) and agent-based synthesis (Deepseek v3.2) and here they do both. I specially like this part:

"Finally, for each generated question-answer pair, we utilize an agent-based methodology to identify other potential correct answers and assess their validity, retaining only those pairs where the original answer is correct and all other identified potential answers are incorrect."

One problem I found is that you generate QA pairs, but when training your agent, it finds the correct information that isn't present in the original QA. This causes the trajectory to receive a poor reward, even if it's correct, because it doesn't map exactly to the ground truth QA, and well, this is a problem. They don't mention this problem specifically, but I imagine they encountered it or thought about it before facing it.

[Meituan LongCat @Meituan_LongCat]

🚀 LongCat-Flash-Thinking-2601 Technical Report – Now Fully Released!

Key insights:

🌍 Large-scale agentic RL (14 pages of deep dives!)
🔹 Environment scaling: A detailed look at our automated pipeline that builds 10,000+ executable, verifiable environments across 20+ domains.
🔹 RL infrastructure: An upgraded DORA framework that supports async training with 32,000+ concurrent environments, tackling stability issues in long-tail and highly heterogeneous tasks.

🛡️ Robustness in the wild
🔹 Noise injection: No more "greenhouse" agents. We systematically analyze real-world noise (user/tool noise) and inject it directly into the training loop.
🔹 Curriculum RL: A curriculum-based strategy that gradually toughens the model against messy, imperfect environments.

🧠 Heavy Thinking framework
🔹 Parallel reasoning: Expands breadth by generating multiple independent reasoning trajectories.
🔹 Iterative summarization: Expands depth by using a summary model to reflect on and synthesize parallel trajectories before making final decisions.
🔹 Context memory: A purpose-built memory module to keep reasoning coherent over long horizons.

⚡ Zigzag Attention
🔹 Zigzag Connectivity design combining MLA + SSA to reduce compute while preserving global information flow.
🔹 Mid-training switch to sparse variants yields a 1.5× speedup and supports 1M-token contexts —laying the groundwork for future breakthroughs in long-context agentic reasoning.
🔹 Explore：https://t.co/xmvQ2kmJUV

📊 Achieves SOTA among
open-source models across key agentic benchmarks: search, tool use, mathematical reasoning, and coding.

If you want more details, feel free to check out the full technical report.
• Paper: https://t.co/X7h2092UN5
• Website: https://t.co/d6cZdCPWnh
• GitHub: https://t.co/24sd7zY98j
• Hugging Face: https://t.co/UCmFfzqTlj

[errordesconocido @errordesconocid]

Increíble cómo han creado un mundo donde que llueva o que haga calor es culpa tuya por comer pollo, pero que las infraestructuras del Estado se caigan a pedazos causando muertos es un suceso fortuito y sin responsables políticos que nadie pudo evitar.

[diego @WithDiegx]

The most interesting part for me. I guess that the 'dynamic chunking' for audio uses the internal model dynamic vector allocation. For text I'm really curious, maybe a custom neural chunker? Although I think it is most likely contextual chunking, from Anthropic, with a small fine-tuned LM for this to keep costs low at scale.

[diego @WithDiegx]

There isn't enough data out there to train this effectively, the synthetic pipelines here must be very good

[Mixedbread @mixedbreadai]

We build the first production ready multi-vector and multimodal search.

Now we are serving over 1 billion documents in under 50ms latency (p50).

We are sharing how we build it.

[diego @WithDiegx]

What, how is this trained

[diego @WithDiegx]

I think this gets us closer to a CRUD-style knowledge editing, which would be so useful, although very unclear how to Create and Update without training.
Assuming that knowledge can be effectively represented in a structure like this.

[Lisan al Gaib @scaling01]

DeepSeek is back!

"Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models"

They introduce Engram, a module that adds an O(1) lookup-style memory based on modernized hashed N-gram embeddings

Mechanistic analysis suggests Engram reduces the need for early-layer reconstruction of static patterns, making the model effectively "deeper" for the parts that matter (reasoning)

Paper: https://t.co/vHcxXW9cBv

[Andrés Rublev @AndresRublev]

Va a prohibir antes la compra de viviendas unifamiliares por parte de fondos buitre Trump que el Gobierno de Progreso español.

[diego @WithDiegx]

Good instruction following on these extra small models is so important and useful

[Liquid AI @liquidai]

Today, we release LFM2.5, our most capable family of tiny on-device foundation models.

It’s built to power reliable on-device agentic applications: higher quality, lower latency, and broader modality support in the ~1B parameter class.

> LFM2.5 builds on our LFM2 device-optimized hybrid architecture
> Pretraining scaled from 10T → 28T tokens
> Expanded reinforcement learning post-training
> Higher ceilings for instruction following

🧵

[diego @WithDiegx]

I would have never thought that REPL could be a possible solution to long context agents

[Prime Intellect @PrimeIntellect]

We believe the next breakthrough in long-horizon agents is training models to manage their own context.

Introducing our new research direction on Recursive Language Models.

We are sharing our initial experiments showing the promise of RLMs.

https://t.co/NwgFbn6kwa