caoye

post-training RL data/env @datacurve
build something people/intelligence want.

[caoye @caoye42]

@plugyawn Is there a link of git?

[caoye @caoye42]

yesterday had chance to interview with many talented candidates on behalf of @datacurve . so excited!!!

[Qwen @Alibaba_Qwen]

We promised open weights for Qwen3.8. Now, time to meet them! 🎉

⚡ Qwen3.8-27B:
- A native multimodal dense model. With just 27B parameters, it outperforms Qwen3.7-Plus overall and shines in real-world coding & office workflows.
- 262K native context, easily extendable to 1M tokens via YaRN.
- Built for builders. Highly efficient, high-quality, and licensed under Apache 2.0.

🚀 The open weights for Qwen3.8-2.4T-A95B (Max-level) have also been released recently.

Whether you're shipping lightweight applications with Qwen3.8-27B locally or building agents with Qwen3.8-2.4T-A95B, they're yours now!

Download, deploy, and build something we haven't imagined yet. 👀👇
- Hugging Face：
https://t.co/4kaAcqYEVj
- ModelScope：
https://t.co/eRIMZCGkhC

[Datacurve @datacurve]

Gemini 3.7 Flash debuts at 65.5% on DeepSWE.

It delivers substantial improvements over 3.6 Flash, scoring +18.8% higher while costing less than half as much per task.

[caoye @caoye42]

technology is a force of nature.

[Andrej Karpathy @karpathy]

Gradient descent can write code better than you. I'm sorry.

[Tencent Hy @TencentHunyuan]

For a finite set of integers (A), how much faster can (|A+A|) grow than (|A-A|)?

A 1969 theorem gave an upper bound of 2 for the exponent. For more than 50 years, the best constructions barely exceeded 1.1.

With help from our research agent Hyra and the Hy3 model, we found an explicit construction showing that the optimal exponent is exactly 2.

A 50-year-old problem, solved.

Paper: https://t.co/OU7lfH0hXR
Hyra blog: https://t.co/NeQEekUSZ3
Formal proof: https://t.co/LMbGdKqo2D

[Kimi.ai @Kimi_Moonshot]

Releasing the model weights and technical report of Kimi K3.

Kimi K3 is our most capable model: a 2.8T MoE model with native visual understanding and a 1M-token context window.

New model architecture: 2.5x the intelligence per unit of compute, not just more params.

Alongside Kimi K3, we're opening up more of the stack behind it — high-performance attention kernels, MoE communication library, and infrastructure for running agent environments at scale.

Model weights: https://t.co/7m7eEg6Y0B
Tech report: https://t.co/yeu6cjpMCT
Tech blog: https://t.co/YTfiMSNM1f

[Simons Foundation @SimonsFdn]

During today's #ICM2026 opening ceremonies, the IMU announced the 2026 Fields Medals recipients:

@UChicago's Yu Deng, @stonybrooku's John Pardon, @UofT's Jacob Tsimerman, and Hong Wang of @nyuniversity and @Institut_IHES.

Read more: https://t.co/oJARAI8I8j

[SemiAnalysis @SemiAnalysis_]

If Jensen really wants to do committee-based training, he should at least form three different committees that don’t talk to each other to avoid groupthink. These three coalitions should innovate on data, RL, pretraining, and evals independently of each other and compete against each other. This would be more aligned with American free-market values, similar to how Chinese labs compete against each other while adapting each other’s proven ideas, leading to different innovations like KDA, MSA, DSA, etc. 5/5🧵

[SemiAnalysis @SemiAnalysis_]

Furthermore, Mistral messed up Nemotron’s pretraining. Due to the nature of NVIDIA’s single committee, when one member company messes up, it drags down the entire committee, which causes the other members to be upset. 3/5🧵

[SemiAnalysis @SemiAnalysis_]

When Jensen created the Nemotron Committee, it restricted the flow of different approaches and created groupthink, even though open source is all about the ability to experiment freely. Based on the results, the Nemotron Committee is clearly not the path forward for American frontier OSS. 2/5🧵

[caoye @caoye42]

kimi k3 is not bad..

[Datacurve @datacurve]

Kimi K3 debuts at #3 on DeepSWE.

It's the first open-weights model that delivers frontier-level performance, achieving results similar to Claude Fable and GPT-5.6 Sol.

[caoye @caoye42]

@datacurve

[Datacurve @datacurve]

GPT-5.6 tops the DeepSWE leaderboard at 73%.

Sol, Terra, and Luna results are now available.

[caoye @caoye42]

this technique can work well bcz: 1) not all decoding tokens are equal and worth bigger model's work, some are easier to guess(math, code) than others(open ended question); 2) small model over training can get better at guessing bigger model's output.

[caoye @caoye42]

small model is efficient to be used as guessor/speculator/drafter for bigger model to improve inference efficiency. result is that auto-regressive gen is no longer compelled to decode one token at a time.
https://t.co/NfuCR5ciC3

[caoye @caoye42]

this is challenging bcz ds had to improve the infra code as well: the verification takes current system load into consideration for determining how long to veirfy

[caoye @caoye42]

the prior research stopped at k=3 for spec decoding nv realized longer verification is getting inefficient. ds didnt stop there and continue optimizing:
https://t.co/1zduirn31l

[caoye @caoye42]

also this pattern is very similar to pretraining:
P(B|A)
P(C|A,B)
P(D|A,B,C)
are calculated in paralell during pretraining (teacher forcing).
the verification done in parallel here is same pattern.

[caoye @caoye42]

essentially turn big-model decoding into small-model decoding + big-model batch verification.

[caoye @caoye42]

also this technique might be somewhat related to distilattion. maybe reverse-distillation ?

[caoye @caoye42]

interesting part is that: small model is learning to simulate the bigger model with some loss, but as this loss improves, the smaller model can guess correctly longer seq: thereby potentially 10x inference efficiency/ throughput. this is a very interesting use case for small models.