jeyki🦦

☁️

[Cong Wei @CongWei1230]

Let's remove VAEs and ViTs from video models!

🚀 Introducing 𝗣𝗶𝘅𝗲𝗹𝗨𝗠𝗠: an encoder-free unified multimodal model for image and video understanding and generation, directly in pixel space.

Code and model available today!

🌐 https://t.co/RXUObhsSIZ
📄 https://t.co/lA7VaJoVws

[Chris Barber @chrisbarber]

I asked people: "which robotics labs and startups have the highest talent density?"

28 votes: Physical Intelligence (@hausman_k, @svlevine, @chelseabfinn, @adnan_esm, @brian_ichter, @QuanVng, @lachygroom)

18 votes: Generalist (@peteflorence, @andyzengineer, Andrew Barry)

9 votes: Sunday (@tonyzzhao, @chichengcc)

6 votes: Skild AI (@deepakpathak, @gupta_abhinav_)

4 votes: Dyna (@Lindon_Gao, @YorkYang5050, @JasonMa2020)

3 votes: Bracket Bot (@sincethestudy)
3 votes: OpenAI Robotics (@model_mechanic, @PengchuanZ, @ChengshuEricLi)

2 votes: 1X (@BerntBornich)
2 votes: The Bot Company (@kvogt, @pariljain, @lukeholoubek)
2 votes: Figure (@adcock_brett)
2 votes: Genesis (@zhou_xian_, @theo_gervet, @johnsonwang0810)
2 votes: Prometheus (@JeffBezos, @vik_bajaj, @sherjilozair, @wgussml, Alex Graves)
2 votes: Robocurve (@chooi_jeq)
2 votes: Tesla (@elonmusk)
2 votes: Weave Robotics (@kaandogrusoz, @evan_wineland)

People couldn't pick a company they work at or founded. Disclosure: I'm a small investor in Physical Intelligence. I didn't vote.

[Nathan Lambert @natolambert]

My book, Reinforcement Learning from Human Feedback is done!

This is the book I wish I had when learning to fine-tune, align, & now post-train models since ChatGPT. The resource has been built by me finding time to study and document the fundamentals on nights and weekends since 2024.

Transferring as much of the intuitions of building Olmo as I possibly can in the book format.

The book is launching with an over 10 hour, full course with slidedecks, functional code for the training chapters, an example model completions library, and of course the free online web version.

Physical orders from Manning will ship in 1-2 weeks, and Amazon a week or so after. Thanks for your support!

[Aksel @akseljoonas]

Introducing ml-intern, the agent that just automated the post-training team @huggingface

It's an open-source implementation of the real research loop that our ML researchers do every day. You give it a prompt, it researches papers, goes through citations, implements ideas in GPU sandboxes, iterates and builds deeply research-backed models for any use case. All built on the Hugging Face ecosystem.

It can pull off crazy things:

We made it train the best model for scientific reasoning. It went through citations from the official benchmark paper. Found OpenScience and NemoTron-CrossThink, added 7 difficulty-filtered dataset variants from ARC/SciQ/MMLU, and ran 12 SFT runs on Qwen3-1.7B. This pushed the score 10% → 32% on GPQA in under 10h. Claude Code's best: 22.99%.

In healthcare settings it inspected available datasets, concluded they were too low quality, and wrote a script to generate 1100 synthetic data points from scratch for emergencies, hedging, multilingual etc. Then upsampled 50x for training. Beat Codex on HealthBench by 60%.

For competitive mathematics, it wrote a full GRPO script, launched training with A100 GPUs on https://t.co/udm7xGpNzR, watched rewards claim and then collapse, and ran ablations until it succeeded. All fully backed by papers, autonomously.

How it works?

ml-intern makes full use of the HF ecosystem:
- finds papers on arxiv and https://t.co/brvCC7fLPa, reads them fully, walks citation graphs, pulls datasets referenced in methodology sections and on https://t.co/hrJuRkRyzi
- browses the Hub, reads recent docs, inspects datasets and reformats them before training so it doesn't waste GPU hours on bad data
- launches training jobs on HF Jobs if no local GPUs are available, monitors runs, reads its own eval outputs, diagnoses failures, retrains

ml-intern deeply embodies how researchers work and think. It knows how data should look like and what good models feel like.

Releasing it today as a CLI and a web app you can use from your phone/desktop.
CLI: https://t.co/l3K1PslZ1n
Web + mobile: https://t.co/orko5srL4H

And the best part? We also provisioned 1k$ GPU resources and Anthropic credits for the quickest among you to use.

[EurekaClaw @EurekaClaw]

1/n 🦞 Introducing EurekaClaw💡 — a local-first AI research agent that captures your Eureka moments before they vanish.
From idea → proof → experiment → paper — fully automated.
Local-first. Zero data leak. 🔒
Try it: https://t.co/ZJlEYvkX9h

[General Reasoning @GenReasoning]

Introducing OpenReward.

🌍 330+ RL environments through one API
⚡ Autoscaled sandbox compute
🍒 4.5M+ unique RL tasks
🚂 Works like magic with Tinker, Miles, Slime

Link and thread below.

[Kimi.ai @Kimi_Moonshot]

Introducing 𝑨𝒕𝒕𝒆𝒏𝒕𝒊𝒐𝒏 𝑹𝒆𝒔𝒊𝒅𝒖𝒂𝒍𝒔: Rethinking depth-wise aggregation.

Residual connections have long relied on fixed, uniform accumulation. Inspired by the duality of time and depth, we introduce Attention Residuals, replacing standard depth-wise recurrence with learned, input-dependent attention over preceding layers.

🔹 Enables networks to selectively retrieve past representations, naturally mitigating dilution and hidden-state growth.
🔹 Introduces Block AttnRes, partitioning layers into compressed blocks to make cross-layer attention practical at scale.
🔹 Serves as an efficient drop-in replacement, demonstrating a 1.25x compute advantage with negligible (<2%) inference latency overhead.
🔹 Validated on the Kimi Linear architecture (48B total, 3B activated parameters), delivering consistent downstream performance gains.

🔗Full report:
https://t.co/u3EHICG05h

[Qwen @Alibaba_Qwen]

🚀 Introducing Qwen-Image-2.0 — our next-gen image generation model!
🎨 Your imagination, unleashed.
✨ Type a paragraph → get a pro slides
✨ Describe a scene → get photoreal 2K magic
✨ Add text → it just works (no more glitchy letters!)
✨ Key upgrades:
✅ Professional typography (1K-token prompts for slides, posters & comics)
✅ 2K native resolution with stunning detail
✅ Flawless text rendering + unified generation/editing
✅ Lighter architecture = faster inference
Try it now → https://t.co/941HmITJ2W
Full details → https://t.co/OtKi6vuU4S

[歸藏(guizang.ai) @op7418]

太离谱了，卧槽，这我真得说脏话了！

动效和 APP 宣传视频 Seedance 2.0 居然也能做！

用他给我写的 Claude Code 桌面端 Codepilot 做了个演示动画视频

这质量，拉满了啊，等后面清晰度上来完全可用了

我只提供了我的提示词、应用介绍和应用效果图

以往这些都只能渲染的，非常麻烦

[Jim Fan @DrJimFan]

https://t.co/Npar79SvUh

[Lianmin Zheng @lm_zheng]

Chunked pipeline parallelism is arguably the most general and scalable system technique for accelerating super-long-context inference.

It remains underrated today, largely because there still isn’t a strong, high-quality open-source implementation.
The SGLang team recently fully optimized it and published a detailed blog post explaining all the key details.

[Michael Elabd @MichaelElabd]

Amazing continual learning paper out of DeepMind 🚨

Most Continual Learning work assumes the backbone is fixed and the burden on the algorithm to fight catastrophic forgetting. This paper flips that assumption on its head and shows pretty convincingly that architecture choices matter just as much for the plasticity–stability trade-off.

A few takeaways that stood out to me:

Learning vs. retention is heavily architecture-dependent. ResNets and WideResNets are great at picking up new tasks, but they forget aggressively. On the other hand, simple CNNs and even ViTs are surprisingly good at retaining old knowledge, even if they learn new tasks more slowly.

Width beats depth. Making networks wider consistently reduces forgetting and improves average accuracy. Making them deeper often gives diminishing returns on learning while worsening forgetting.

Pooling is a hidden culprit: Global Average Pooling is a major driver of forgetting because it bottlenecks the final representation. Removing GAP or replacing it with smaller pooling layers significantly improves retention.

BatchNorm isn’t always helpful. It helps when task distributions are similar, but under large distribution shifts, BN can accelerate forgetting.

What I’d love to see next is this line of work pushed into the LLM regime (larger models, longer task sequences) so we can (1) benchmark continual learning methods more rigorously and (2) start designing architectures explicitly for continual learning, rather than inheriting them from static training.

[Sakana AI @SakanaAILabs]

Introducing DroPE: Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddings

https://t.co/brpejkosWR

We are releasing a new method called DroPE to extend the context length of pretrained LLMs without the massive compute costs usually associated with long-context fine-tuning.

The core insight of this work challenges a fundamental assumption in Transformer architecture. We discovered that explicit positional embeddings like RoPE are critical for training convergence but eventually become the primary bottleneck preventing models from generalizing to longer sequences.

Our solution is radically simple: We treat positional embeddings as a temporary training scaffold rather than a permanent architectural necessity.

Real-world workflows like reviewing massive code diffs or analyzing legal contracts require context windows that break standard pretrained models. While models without positional embeddings (NoPE) generalize better to these unseen lengths, they are notoriously unstable to train from scratch.

Here, we achieve the best of both worlds by using embeddings to ensure stability during pretraining and then dropping them to unlock length extrapolation during inference. Our approach unlocks seamless zero-shot context extension without any expensive long-context training.

We demonstrated this on a range of off-the-shelf open-source LLMs. In our tests, recalibrating any model with DroPE requires less than 1% of the original pretraining budget, yet it significantly outperforms established methods on challenging benchmarks like LongBench and RULER.

We have released the code and the full paper to encourage the community to rethink the role of positional encodings in modern LLMs.

Paper: https://t.co/Fp5IJS4LIC
Code: https://t.co/Wvea7tQ5Ay

[Percy Liang @percyliang]

Need a reward model for robotics? Frontier VLMs actually don't work. We fine-tune one that works better.

[Tony Lee @tonyh_lee]

Reliable rewards are a bottleneck for real-world RL for robotics: human labels are costly, and handcrafted rewards are brittle.

In RoboReward 🤖💰, we study VLMs as reward models and find they are unreliable across tasks, embodiments, and scenes.

Paper: https://t.co/mXKcDtjBFu

[Guitar Cat + LLM 「Will @ SGLang Summit in SF」 @GenAI_is_real]

Stop being obsessed with 8B or 70B "emergent properties" garbage. Most of what you see is just measurement noise and benchmark cheating.

I’ve always said that Zeyuan Allen-Zhu is doing the most rigorous "Physics" of LLMs. As I noted in my blogs, most academic debates on architecture are just cargo-culting because they fail at variable control.

Zeyuan’s new tutorial proves that a 100M model can reveal more architectural truths than a 1T-token 8B model. If you aren't following this, you’re just playing with expensive LEGOs in the dark.

The industry is finally waking up: Scaling without understanding the "Physics" is just a rich man's gambling.

[Zeyuan Allen-Zhu, Sc.D. @ZeyuanAllenZhu]

(1/N)🚀Today we launch two tightly connected milestones in the Physics of LM series: a sharpened Part 4.1 (v2.0) and a brand new Part 4.2 — together forming a clear, reproducible, textbook-style reference for principled architecture research.

Part 4.1 introduced a synthetic pretraining playground — our Galileo experiment for LLMs🍎. Our v2.0 strengthens it with Gated DeltaNet (GDN) and stricter alignment, building an even cleaner “Pisa tower” for testing architectural limits.

Part 4.2 shows these synthetic predictions resonate in reality 🌍 — across 1–8B / 1T-token pretraining — confirming which design principles actually matter.

Together, Parts 4.1 and 4.2 bring the synthetic and real worlds into surprising agreement 🤝— one more step toward a more scientific understanding of LLM architectures.

If you’re curious about:
🧠why some models reason deeper
⚙️ why linear models struggle at retrieval
🎶why a tiny horizontal mixer (Canon) changes everything …
this release ties it all together.

(Links at the end)

[alex zhang @a1zhang]

I was considering waiting a while to polish this first, but decided it'd be better to just release an initial version to get better community feedback and squash bugs!

This is the official RLM repo, with native support for cloud-based and local REPLs.

https://t.co/Lfj97OEvYX

[Andre S @andreslavescu]

Wrote a fused mHC layer in pytorch using the kernels as a backend, and achieved some interesting speedups!

Tested on H100 SXM5

[Osaurus @OsaurusAI]

Claude Code just booked me a flight to Japan.
Opened Safari. Searched Google Flights. Selected the cheapest option. Got to checkout.
I just watched.