Jianyu Huang

Beat the speed of light. Views are my own.

[Andrej Karpathy @karpathy]

We'll be spending a lot more time trying to understand the outputs of language models. A few thoughts, tips & tricks:

Writing. Something I've had success with: Ask your LLM to explain something in ASD-STE100, it's a controlled language specification originally developed for aerospace maintenance documentation. LLMs well-versed in this language and it comes with heavy constraints on clean writing style that I often find a lot more readable. Sometimes I've tried to soften it a bit e.g. ask for "80% of the way to ASD-STE100" because the spec is quite stringent. But even better:

Diagrams / images. Instead of writing, ask your LLM to create a diagram. These can be a lot easier to process, parse, and understand. But even better:

Web pages. Ask for output "in HTML" to get a beautiful, interactive webpage. LLMs are getting really good at frontend and can create beautiful experiences, animations, etc. But even better:

Explainer videos. The output format I am most bullish on is fully custom / bespoke explainer videos generated on any arbitrary topic. Experiment with things like "Create a 3b1b style video explainer on X. Use my ElevenLabs API key for audio narration". (you'd need an API key for the latter or you can ask your LLM to find you decent free alternatives that use your local compute). This is actually starting to work!

In summary:
- As LLMs get better, they will do more and more of the legwork autonomously, and a lot more of our work will rise up the abstractions into oversight and understanding.
- Luckily, LLMs can help here too because as intelligence and code are increasingly abundant, you can ask for large, custom, discardable software artifacts (e.g. web apps, video explainers) that would have never made sense to create before. Push the boundaries here and you'll be surprised.

[Erick P. @bondsnbourbon]

I followed your suggestion and created a 3b1b-style explainer video on stochastic calculus. It reminded me of my first encounter with the topic in my professional education.

I truly appreciate your insights and this new approach to LLMs.

I wish I had access to such technology earlier!

[Thariq @trq212]

https://t.co/taVRqwm4dG

[Jianyu Huang @jianyu_h]

Excited to share Muse Spark 🥑

[AI at Meta @AIatMeta]

Introducing Muse Spark, the first in the Muse family of models developed by Meta Superintelligence Labs.

Muse Spark is a natively multimodal reasoning model with support for tool-use, visual chain of thought, and multi-agent orchestration.

Muse Spark is available today at https://t.co/wHkMPH82ZH and the Meta AI app. We’re also making it available in private preview via API to select partners, and we hope to open-source future versions of the model.

Learn more: https://t.co/PloE9q5x96

[Jianyu Huang @jianyu_h]

Excited to share Muse Spark 🥑, a big step in MSL's journey towards personal superintelligence, after 9 months' effort so far! Try it out on https://t.co/mkcHXREDdi and let us know your feedback!

[Alexandr Wang @alexandr_wang]

1/ today we're releasing muse spark, the first model from MSL. nine months ago we rebuilt our ai stack from scratch. new infrastructure, new architecture, new data pipelines. muse spark is the result of that work, and now it powers meta ai. 🧵

[Emiliano Penaloza @emilianopp_]

https://t.co/SAZYxtVHcC

[Sakana AI @SakanaAILabs]

Introducing DroPE: Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddings

https://t.co/brpejkosWR

We are releasing a new method called DroPE to extend the context length of pretrained LLMs without the massive compute costs usually associated with long-context fine-tuning.

The core insight of this work challenges a fundamental assumption in Transformer architecture. We discovered that explicit positional embeddings like RoPE are critical for training convergence but eventually become the primary bottleneck preventing models from generalizing to longer sequences.

Our solution is radically simple: We treat positional embeddings as a temporary training scaffold rather than a permanent architectural necessity.

Real-world workflows like reviewing massive code diffs or analyzing legal contracts require context windows that break standard pretrained models. While models without positional embeddings (NoPE) generalize better to these unseen lengths, they are notoriously unstable to train from scratch.

Here, we achieve the best of both worlds by using embeddings to ensure stability during pretraining and then dropping them to unlock length extrapolation during inference. Our approach unlocks seamless zero-shot context extension without any expensive long-context training.

We demonstrated this on a range of off-the-shelf open-source LLMs. In our tests, recalibrating any model with DroPE requires less than 1% of the original pretraining budget, yet it significantly outperforms established methods on challenging benchmarks like LongBench and RULER.

We have released the code and the full paper to encourage the community to rethink the role of positional encodings in modern LLMs.

Paper: https://t.co/Fp5IJS4LIC
Code: https://t.co/Wvea7tQ5Ay

[Greg Brockman @gdb]

building pretraining infrastructure is an exercise in complexity management, abstraction design, operability/observability, and deep systems and ML understanding.

reflects some of the trickiest and most rewarding problems in software engineering. which makes it really fun!

[Jonathan Chang @ChangJonathanC]

while we wait for gpt-5 to drop. Here is a flex attention tutorial for building a < 1000 LoC vllm from scratch
https://t.co/PVyauMezM3

[Lech Mazur @LechMazur]

GPT-5 (medium reasoning) is the new leader on the Short Story Creative Writing benchmark!

GPT-5 mini (medium reasoning) is much better than o4-mini (medium reasoning).

Claude Opus 4.1 shows gains over Opus 4.

[elvis @omarsar0]

Hierarchical Reasoning Model

This is one of the most interesting ideas on reasoning I've read in the past couple of months.

It uses a recurrent architecture for impressive hierarchical reasoning.

Here are my notes:

[Greg Brockman @gdb]

more fun than ever to be a software engineer

[Jianyu Huang @jianyu_h]

Numerical issue matters

[Lily Liu @eqhylxx]

Great to see Thinking Machine using vLLM as the inference engine!! I only recently realized how important (and annoying😂) numerical issues can be.

[Jianyu Huang @jianyu_h]

Cool dense GEMM blog/example on using CuTeDSL:
https://t.co/FsKCoZFK2O

https://t.co/R28TnXbCq6

[Jianyu Huang @jianyu_h]

s Group Sequence Policy Optimization to stabilize MoE training!

[Chujie Zheng @ChujieZheng]

Proud to introduce Group Sequence Policy Optimization (GSPO), our stable, efficient, and performant RL algorithm that powers the large-scale RL training of the latest Qwen3 models (Instruct, Coder, Thinking) 🚀

📄 https://t.co/ItLT1ZqaOR

[Jim Fan @DrJimFan]

Shengjia is one of the brightest, humblest, and most passionate scientists I know. We went to PhD together for 5 yrs, sitting across the hall at Stanford Gates building. Good old times. I didn’t expect this, but not at all surprised either. Very bullish on MSL!

[Jianyu Huang @jianyu_h]

Wow, curious to see how this is compared with Gemini CLI.

[Qwen @Alibaba_Qwen]

>>> Qwen3-Coder is here! ✅

We’re releasing Qwen3-Coder-480B-A35B-Instruct, our most powerful open agentic code model to date. This 480B-parameter Mixture-of-Experts model (35B active) natively supports 256K context and scales to 1M context with extrapolation. It achieves top-tier performance across multiple agentic coding benchmarks among open models, including SWE-bench-Verified!!! 🚀

Alongside the model, we're also open-sourcing a command-line tool for agentic coding: Qwen Code. Forked from Gemini Code, it includes custom prompts and function call protocols to fully unlock Qwen3-Coder’s capabilities. Qwen3-Coder works seamlessly with the community’s best developer tools. As a foundation model, we hope it can be used anywhere across the digital world — Agentic Coding in the World!

💬 Chat: https://t.co/V7RmqMaVNZ
📚 Blog: https://t.co/syL1hsSGKq
🤗 Model: https://t.co/1LWwUKMrBN
🤖 Qwen Code: https://t.co/qqwj5nAO3Z

[Jianyu Huang @jianyu_h]

> After talking with the community and thinking it through, we decided to stop using hybrid thinking mode. Instead, we’ll train Instruct and Thinking models separately so we can get the best quality possible.

[Qwen @Alibaba_Qwen]

Bye Qwen3-235B-A22B, hello Qwen3-235B-A22B-2507!

After talking with the community and thinking it through, we decided to stop using hybrid thinking mode. Instead, we’ll train Instruct and Thinking models separately so we can get the best quality possible. Today, we’re releasing Qwen3-235B-A22B-Instruct-2507 and its FP8 version for everyone.

This model performs better than our last release, and we hope you’ll like it thanks to its strong overall abilities.

Qwen Chat: https://t.co/bg4tAU1p74 — just start chatting with the default model, and feel free to use the search button!

HF: https://t.co/WufQYUhImD or https://t.co/6CldGF60sp
ModelScope: https://t.co/KOLTlDujFE or https://t.co/JC33PuinFD

It’s smarter, knows more, can do more things, and works better on agent tasks. Try it out and see how it works for you.

Check the model card on Hugging Face (https://t.co/WufQYUhImD) to see the benchmark results.

This is a small update! Bigger things are coming soon!

[elie @eliebakouch]

Pre-training is not dead

[Jianyu Huang @jianyu_h]

100 in IMO. When will all be replaced by AI?

[rohan anil @_arohan_]

IMO gold musing:

I am curious when would we be able to increase the number of kids that can get IMO gold by 100 fold - take the same system that can get IMO gold and have it to be a tutor that take anyone at anyone level and get them skills to solve IMO problems. Essentially make 100x noam.