Carson Poole

[Carson Poole @CarsonPoole]

this was lots of fun and lots of all nighters over the past few weeks. really happy with what we achieved!

[Artificial Analysis @ArtificialAnlys]

NVIDIA Blackwell can achieve 303 output tokens/s for DeepSeek R1 in FP4 precision, per our benchmarking of an Avian API endpoint

Artificial Analysis benchmarked DeepSeek R1 on an @avian_io private API endpoint. Running DeepSeek R1 in FP4 precision on NVIDIA Blackwell, their endpoint achieved 303 output tokens/s - the fastest speed we have measured yet for DeepSeek R1.

The FP4 version of DeepSeek R1 maintained accuracy across our evaluation suite as compared to the native FP8 version.

Inference speed is especially critical for reasoning models that ‘think’ before they answer - we look forward to wider availability of NVIDIA Blackwell hardware in the coming months!

[Carson Poole @CarsonPoole]

actually vLLM released this functionality on Jan 31st, 2024; so two years, not one.

[Carson Poole @CarsonPoole]

in December of 2021 I saw a GitHub repo doing LoRA fine tuning. I realized immediately that it would allow for different elements in the batch to receive different fine tuned models, effectively allowing a single GPU to serve multiple specialized models with very small overhead

I spent the next hour writing a prototype and showed our customers, who were very excited by this. as far as I can tell, this didn’t wind up in the open source world for close to a year after this, and FireworksAI sent me a marketing email advertising this functionality as on October 14 of 2026

[gabriel @gabriel1]

i found the optimal first interview question, difficulty: impossible

tell me about one concrete action in your career, in a way where I can understand why it is impressive given what you knew at the time. use as few words as possible

[Carson Poole @CarsonPoole]

@celestepoasts sft is blue, rl is red. this is the way

[Carson Poole @CarsonPoole]

@mike64_t 1) this is a meme, not meant to be taken seriously
2) I don’t even follow you and don’t even know what tweet of yours you are implying I’m referencing
3)

[Carson Poole @CarsonPoole]

Of course that’s your contention.

You’re a first-year inference engineer.

You just read SemiAnalysis’s blog and saw megakernels coming to TPUs and AMD, so naturally you believe that’s the way to get to speed of light.

Next month, when you get your hands on an inference engineering cookbook—Baseten’s, probably—you’ll be singing from the treetops about the 500 nanoseconds of critical-path latency you save on every GEMM pipeline drain.

That’ll last until sometime next year, when you watch a pirated geohotz lecture livestream and become convinced that PDL gives you a strict superset of megakernel capabilities.

“Well, as a matter of fact, I won’t, because PDL introduces—”

“PDL introduces unnecessary stalls because its synchronization mechanism is very coarse.”

Yeah, I read that too. HazyResearch’s blog, “Look Ma, No Bubbles!” Just below Figure 2.

Do you have any thoughts of your own on the subject, or should I just point my agent at the SGLang repo and have it do it for me?

[Carson Poole @CarsonPoole]

i cannot recommend this incantation enough

[Steve Ruiz @steveruizok]

good prompt "find all of the implicit state machines in my app"

[Carson Poole @CarsonPoole]

@jarredsumner https://t.co/f41F526fub

[Carson Poole @CarsonPoole]

you have dependencies? well, too bad, you are vulnerable. roll your own everything for opsec reasons ofc

[Carson Poole @CarsonPoole]

@that1nerdy @0x_Osprey https://t.co/kZA1GdF39H

I have a whole lot better than 165x, at 100% of traffic proven, running already :)

[Carson Poole @CarsonPoole]

is everybody on OpenRouter just quantizing DeepSeek 4.1 Flash into oblivion to cut costs? I suppose the auto-exacto tier exposes some of this, but we need better.

commodity markets like wheat or corn have had a concept of provenance forever, and for the commodity of LLM tokens, this seems sorely needed.

every request here was ran on 8x B200s but could be ran on a 5090, a CPU, or a toaster and the hash of the logits would match, so you don't need to trust any provider.

this is using the public weights, but using bit exact inference with a custom rust-based DSL directly emitting PTX instructions for fully deterministic, integer only operations.

[Carson Poole @CarsonPoole]

for OpenAI, Anthropic, and many other companies, Cost Of Goods Sold -> Cost Of Tokens Served, and I would like to be able to say I coined that term iff it becomes widely used

[Carson Poole @CarsonPoole]

wait are we actually about to be so very back on the henry adams curve?

[Atoms Not Bits @AtomsNotBits]

BREAKING: For the first time in American history, the Department of Energy has approved a commercial nuclear microreactor's safety analysis for full-power operations.

LA-based Radiant secured the approval, with testing planned at Idaho National Laboratory.

[Carson Poole @CarsonPoole]

@jasonljin @takezest a c c e l e r a t e

[Carson Poole @CarsonPoole]

@ChangJonathanC https://t.co/NpkzTqUo5E

[Carson Poole @CarsonPoole]

i remember in august '25 asking a researcher about whether "reverse distillation" was possible, because this area was deafeningly silent.

like of course a smaller model's representations are better than random init

[Carson Poole @CarsonPoole]

i remember in august '25 asking a researcher about whether "reverse distillation" was possible, because this area was deafeningly silent.

like of course a smaller model's representations are better than random init

[Jonathan Chang @ChangJonathanC]

my Claude found that Qwen3.5-27B was grown from Qwen3.5-9B (layer looping)

qt: independent discovery found by grok

[log4j @log4j]

i own a relatively new (2025) samsung tv.

i kept getting weirdly targeted ads.

i tried reversing the firmware but it was aes encrypted and no keys unless tear it apart and do uart/jtag magic.

zero days it is. root shell on new samsung TVs!

discovered PRIVACY HORROR. write up?

[Carson Poole @CarsonPoole]

i wouldn't say that the roadblock to continual learning has ever been how you can update the weights. you could always do some simple lora updates to the base model's weights, potentially merging them in for full rank updates. the hard part is figuring out what to update

[Carson Poole @CarsonPoole]

conservation of matter gives a secure store of value, only if you can't do cheap fusion

conservation of energy gives a secure proof of compute, which so far has just made another store of value

but unlike gold, compute doesn't just sit in a vault: it can do trustless inference

[Marathon Fusion @MarathonFusion]

Last month, we demonstrated high-rate alchemy: turning mercury into real, stable gold, beating CERN’s record by a factor of 10.

Last week, we enriched isotopes in the world’s most powerful plasma centrifuge, beating the Soviet record that stood for 40 years.

Now, we’re building a fusion reactor in San Francisco to achieve record power density.

[Carson Poole @CarsonPoole]

you have dependencies? well, too bad, you are vulnerable. roll your own everything for opsec reasons ofc

[Carson Poole @CarsonPoole]

@MelindaBChu1 @citrini Tu Youyou sounds like an all time great recipient of a happy birthday song

[Carson Poole @CarsonPoole]

if I had a nickle for every time someone has done this and won a nobel prize, I'd have two nickles. which isn't a lot, but it's weird that it's happened twice

[Citrini @citrini]

What is AI really capable of in bio?

To separate hype from reality, we spent 3 months working firsthand with @AnthropicAI's internal research model to see if it could create an experimental scorpion antivenom.

Introducing Citrini Therapeutics.

https://t.co/VAUBSi7sNy

[Carson Poole @CarsonPoole]

@citrini Werner Forssmann