James Hou

cs @caltech | intern @modal

[James Hou @jamesahou]

holy 460 tps

[Modal @modal]

Kimi K3 is live on Modal.

Moonshot has shipped the world's first open 3T-class model, and we're a day zero launch partner.

We trained a custom DFlash speculator for K3's novel architecture so you can run it faster, losslessly. The most capable open model we've worked with by far.

[Modal @modal]

Kimi K3 is live on Modal.

Moonshot has shipped the world's first open 3T-class model, and we're a day zero launch partner.

We trained a custom DFlash speculator for K3's novel architecture so you can run it faster, losslessly. The most capable open model we've worked with by far.

[James Hou @jamesahou]

@AdamZweiger @thinkymachines @MIT huge for the program 🦍

[James Hou @jamesahou]

@peywalt @jvmncs don and his consigliere

[James Hou @jamesahou]

day-0 dflash spec!

[Modal @modal]

Inkling by @thinkymachines is now available on Modal, backed by a custom DFlash speculator for 67% higher throughput and interactivity.

Running on Modal Auto Endpoints with SGLang today.

[James Hou @jamesahou]

@anvisha built by the 🐐 @Amehta_2005

[James Hou @jamesahou]

@0xtigs gambling segway pipeline tech

[Yisong Yue @yisongyue]

Happening right now in Hall A, Poster 2201 #ICML2026

[James Hou @jamesahou]

@monishee the 🐐

[Modal @modal]

Our new Auto Endpoints feature is powered by a new Modal primitive: Modal Servers.

In this blogpost, we walk through design principles and detailed architecture: @EnvoyProxy, @googlecloud Spanner config store, and a @Cloudflare Pingora-based custom proxy.

[Modal @modal]

Modal Auto Endpoints provide state-of-the-art open source inference perf with a click.

Learn how we developed our low latency inference playbook with @DecagonAI, delivering responses 60ms faster than the best proprietary provider.

https://t.co/7OMcB4mD7N

[James Hou @jamesahou]

mo autoendpoints, mo speed, mo dal

[Modal @modal]

It is not too late to _actually_ own your inference.

Introducing: Modal Auto Endpoints.

[James Hou @jamesahou]

llm equivalent of a speccing out your car

[Charles 🎉 Frye @charles_irl]

Speculation Is All You Need.

In this blog post, we announce the co-release (w/ Z Lab) of six more state-of-the-art DFlash speculators for @Alibaba_Qwen 3.x.

Over 1k output tps for 3.5 122B-A10B on a B200.

Read the blog for why we're all-in on spec dec.

https://t.co/Bv3Zc95Xgh

[Modal @modal]

We're bringing together our friends and community to celebrate our Series C.

Join us at Noguchi's Sunken Garden in NYC on June 16th or at the Legion of Honor in SF on June 25th.

Invites are limited, apply here: https://t.co/B4h0C7Wq0Q

[Erik Bernhardsson @bernhardsson]

Today we're announcing our Series C funding: $355M at a $4.65B valuation, led by some great investors @generalcatalyst and @Redpoint.

We've had insane growth in the last year, but we're still very early. So proud of the team and what we have built so far!

[Yisong Yue @yisongyue]

Check it out here: https://t.co/fIJJ7Obuku

Amazing work by @atharva_sehgal! Thanks also to @swarat & @JenJSun. Supported by @LaudeInstitute Slingshot.

[Yisong Yue @yisongyue]

FormulaCode tests AI's ability to (super-)optimize at the system level: can LLMs optimize multi-load benchmarks of entire codebases? Turns out, human experts are still better (for now).

(appearing at #ICML2026)

[James Hou @jamesahou]

Check out FormulaCode🏎️! We study how coding agents tackle codebase optimization, where gains require coordinated tradeoffs and changes across the repo (just like tuning a car) - not just making one workload go brrr. Awesome working with @atharva_sehgal, @yisongyue, and the team!

[Atharva Sehgal @ ICML @atharvasehgal_]

Excited to share FormulaCode, a continually updating benchmark for evaluating the holistic ability of LLM agents to optimize codebases. Our current dataset consists of 957 tasks curated from 245477 pull requests in 70+ repositories (and growing!).

🌐 https://t.co/bmUgfZkvsw
🧵👇

[Erik Bernhardsson @bernhardsson]

This is @_gongy. He leads our LLM inference team. He has a car that goes vroom!

You can work with us and have a car that goes vroom!

https://t.co/tXDNmmf5Kw

[Zhuoyang Zhang @zhuoyang_zhang]

We release ForeAct (accepted to CVPR’26🎉), a world model planner powered by visual foresight for VLAs - efficiently, modularly, and at scale.

✨ Seamlessly integrates with VLAs by visual augmentation — no architectural changes required
⚡ Generates high-fidelity 640×480 subgoal images in just 0.33s
🧠 Significantly boosts generalization capability and data efficiency

📄Paper: https://t.co/aOadlsgVqu
🔗Code: https://t.co/aPtnZ0t8Q4

🧵👇