Thomas Lux

Research scientist at Meta (MSL FAIR). PhD in Computer Science. Working on AI theory+algorithms in free time.

[Thomas Lux @tchlux]

@unixpickle 16 experts active per token (100B ≈ 50GB) and thunderbolt read speeds of the M3 Ultra hit 15GB/sec, if you max out RAM with an expert cache and optimistically hit 75% cache hit rate (per token) that gets you 1 token / second without extra quantization.

[Thomas Lux @tchlux]

@Engineer_Wong Yeah this is awesome! I want this but target 95% venting back out of the same window and heat exchange outgoing ↔ incoming to give more stable climate control.

[Thomas Lux @tchlux]

I love this, have thought about it a lot, and still can’t decide what would actually be useful to do. Or at least, I can’t come up with objectives to optimize against that feel stable and reliable.

E.g., if you become the best at finding corruption and publishing it then doesn’t that just encourage the corruption to hide itself? Or even make YOU a target?

[Thomas Lux @tchlux]

This suggests two things clearly to me (with caveat at end):

1. Most low likelihood tokens from pre and post training don’t really help benchmark scores (~useful tasks proxy). ➞ you can train on less data and get a better result cheaper

2. The knowledge distillation that happens during training works, such that training on the result of the distillation creates a better model for benchmarks. ➞ the underlying mental model approximated is good

Caveat: assuming the benchmarks contain enough diversity to be reliable and representative.

[Bo Wang @BoWang87]

Apple Research just published something really interesting about post-training of coding models.

You don't need a better teacher. You don't need a verifier. You don't need RL.

A model can just… train on its own outputs. And get dramatically better.

Simple Self-Distillation (SSD): sample solutions from your model, don't filter them for correctness at all, fine-tune on the raw outputs. That's it.

Qwen3-30B-Instruct: 42.4% → 55.3% pass@1 on LiveCodeBench. +30% relative. On hard problems specifically, pass@5 goes from 31.1% → 54.1%.

Works across Qwen and Llama, at 4B, 8B, and 30B. One sample per prompt is enough. No execution environment. No reward model. No labels.

SSD sidesteps this by reshaping distributions in a context-dependent way — suppressing distractors at locks while keeping diversity alive at forks. The capability was already in the model. Fixed decoding just couldn't access it.

The implication: a lot of coding models are underperforming their own weights. Post-training on self-generated data isn't just a cheap trick — it's recovering latent capacity that greedy decoding leaves on the table.

paper: https://t.co/YsT3OSmbq3

code: https://t.co/OX58FzDVqy

[Thomas Lux @tchlux]

I appreciate the framing, but dial up (<100kbps) to fiber (1Gbps) is >10000× difference. Nobody has that much advantage in LM inference.

There are *many* cases where minimizing cost per token is the utmost goal as long as sufficient tokens/day can be achieved.
- reporting
- analysis
- exploration

I think cerebras hardware is awesome, but for many markets the lowest cost at *acceptable latency* is the most important.

Something exciting would be to use qwen3.5 27B with a few guiding prompts and a pointing model (moondream?) to drive a car, that’d be an excellent demo for low latency hardware. If you could do 30 frames per second of good decision making and pointing with traceable reasoning at every step then you’ll turn heads.

[Thomas Lux @tchlux]

@moondreamai Does it get stuck in self repeating loops when captioning a screenshot of a desktop computer or website?

[Thomas Lux @tchlux]

@rryssf https://t.co/hzwu7kU4oR

[Thomas Lux @tchlux]

Core idea: rewrite a prefix prompt to a class of problems to get better performance in that class.

This is analogous to how early LLMs were improved just by saying “think step by step” in the prompt before asking evaluation questions.

This paper has taken that core idea, asked a model to “optimize” itself by searching for similar improvements. I’m for the idea! But, it does not make sense to me to use RL here because “pure” stochastic exploration typical of RL would *waste* token budget rather than minimize it.

Personally I would call this meta promoting. And I assume most people applying language models at large scale already do this. The process works even better when you have a “strong” / “expensive” model do the prompt writing and a weaker cheaper model for rollouts. And only use the same model to optimize itself when you have no other options.

[Thomas Lux @tchlux]

Core idea: rewrite a prefix prompt to a class of problems to get better performance in that class.

This is analogous to how early LLMs were improved just by saying “think step by step” in the prompt before asking evaluation questions.

This paper has taken that core idea, asked a model to “optimize” itself by searching for similar improvements. I’m for the idea! But, it does not make sense to me to use RL here because “pure” stochastic exploration typical of RL would *waste* token budget rather than minimize it.

Personally I would call this meta promoting. And I assume most people applying language models at large scale already do this. The process works even better when you have a “strong” / “expensive” model do the prompt writing and a weaker cheaper model for rollouts. And only use the same model to optimize itself when you have no other options.

[Thomas Lux @tchlux]

@varunneal So once a parameter goes to zero you stop updating it? Seems like it’s begging for a reset if it is zero and no longer changing. Otherwise those are wasted ops.

[Arnold Schwarzenegger @Schwarzenegger]

Step 3: Do a rep of gratitude every day.

No matter who you are, this is proven to help your mind and your body. And when it feels like there is absolutely nothing you can do, you can do this one thing. Write down one thing you’re grateful for, every day.

[Thomas Lux @tchlux]

This promotes such a bad misconception. Simple question that shows what you’re missing, what’s the condition number of a random square matrix with 1000 random vectors? (Length normalized componentwise Gaussian)

Sure you can pick two random directions and they are not “aligned”. But that does *not* mean that you actually fill the space with random vectors. Dense layers use *matrix multiplication* not single pairs.

Also having an inner product of 0.1 is not orthogonal. See the analytic plot of pairwise inner products with increasing dimension (fit to real data).

[Thomas Lux @tchlux]

Nice! Might want to fix "...machine is be the workhorse..." on the Roadrunner page.

[Thomas Lux @tchlux]

@cybrdave @boringcompany https://t.co/amnHE74baZ

[The Boring Company @boringcompany]

The Vertical Cassette and Belt Bender, used for tunnel rock removal, are set up in Nashville!

Conveyor belt loops through the structures, through the tunnel to the TBM, and back. For a 9 mile tunnel, 18 miles of belt is moving.

It can remove &gt;33,000 lb of rock each minute!

[Thomas Lux @tchlux]

@randall_balestr @zzznah Wouldn’t it be better if you orthogonalized A to minimize skew (that is likely in higher dimension)?

[Thomas Lux @tchlux]

I was also interested. Apparently at D10 class (70 metric tons) you need special composites and custom steel chords integrated in, plus lots of low volume large and expensive tooling to make the track segments (multi piece molds that are several tons each). So the raw materials (4-6 tons worth) cost is only a small fraction of the total cost.

[Thomas Lux @tchlux]

@MacroWorld21 @jwt0625 About 28,000 suns worth🥲

(this setup 2.8e7 W / m²)
(daytime sunlight 1e3 W / m²)

[Thomas Lux @tchlux]

@justkerrding @berkeley_ai If you had a high resolution camera, what do you think about doing this digitally? Much fewer moving parts, dynamic “focus” by shifting a center point around in software.

Or is my intuition off about the resolution required for that to be practical?

[Thomas Lux @tchlux]

@Object_Zero_ Land-based uranium mining is for ore at 0.1–20% concentration. Even low-grade ores (~0.01%) are millions of times richer than seawater. Extraction from seawater currently costs &gt;$500–$1000/kg uranium, while mined uranium is ~$50–$70/kg.

[Thomas Lux @tchlux]

This is my dream! I’ve had some success, but it requires a lot more scaffolding than the bigger LMs. You can see my framework here https://t.co/3vkr5jPdWc

Essentially I broke it down into smaller parts manually.
- provide a run book as system prompt
- pick next step (tool) given history
- construct tool call one parameter at time
- summarize result of tool call into history