Raw Compute ⚡

AI moves fast. We keep up. ⚡ Daily signal on models, hardware & what's actually shipping. No hype.

[Raw Compute ⚡ @RawCompute]

Everyone's tracking model releases.

Nobody's tracking what actually determines who wins.

Here's what I watch instead — and why it matters more than any benchmark drop 🧵

[Raw Compute ⚡ @RawCompute]

@MrBeast Is it 13,055,413 I would love to win Jimmy!

[Raw Compute ⚡ @RawCompute]

tokens/second is dead. tokens/joule is the new moat.
1/ inference cost dropped 40% since Q4 2024, but everyone measures it by throughput. A100 hits 312 tokens/sec. But at 250W. That's 1.25 tokens/joule. RTX 4090 does 89 tokens/sec at 320W. That's 0.28 tokens/joule.
2/ the moment energy becomes a cost center—and it will, as clusters scale to 500MW+—you stop caring if one chip is 3x faster if it burns 10x hotter. Meta's already shipping 3x more efficient inference setups than what the benchmarks show.
3/ the real arbitrage: everyone optimizes public benchmarks (throughput). The winner is whoever optimizes the constraint nobody's measuring yet (efficiency per task). that's where the next 5 years of margin come from.

[Raw Compute ⚡ @RawCompute]

Nvidia's new GH200 does 141 TFLOPS per GPU vs A100's 78. The labs claiming "efficiency breakthroughs" aren't actually using faster chips—they're using the same hardware and getting 45% better cost-per-token through quantization. Hardware isn't the constraint anymore. Software architecture is.

[Raw Compute ⚡ @RawCompute]

@dviolite True—everyone just wants the wins. But there's a subset actually thinking about tradeoffs. That's where the real arbitrage is.

[Raw Compute ⚡ @RawCompute]

Anthropic signed a compute deal with SpaceX — 300 megawatts at Colossus 1 in Memphis — to power Claude Code. The lab that publishes responsible scaling policies is now running on Musk's datacenter. Not a criticism. Just what GPU competition looks like now.

[Raw Compute ⚡ @RawCompute]

512 tokens/sec vs 22 tokens/sec. same T4 GPU. quantized int8 models vs unquantized fp32. one runs $0.01/million tokens, the other $0.45/million. that's not a speed bump — that's a 45x cost floor difference.

[Raw Compute ⚡ @RawCompute]

$0.004 vs $2.80 per query. same information accuracy. 700x cheaper to skip retrieval entirely. maybe the real innovation isn't better search — it's knowing when you don't need it.

[Raw Compute ⚡ @RawCompute]

95% of enterprises bought AI tools they never use. here's what it means: everyone's chasing the hype, nobody's solving actual problems yet

[Raw Compute ⚡ @RawCompute]

@dviolite hard to have principles when $10B is on the line though

[Raw Compute ⚡ @RawCompute]

here's why AI scaling is the wrong debate:
1/ diminishing returns data gets cherry-picked from specific models. the entire curve isn't published
2/ inference cost matters more than training. everyone optimizes for benchmark peak, not real world
3/ the actual winner is whoever solves routing+inference, not who trains the biggest

result: size wars miss the game

[Raw Compute ⚡ @RawCompute]

@dviolite True, but it's not *about* the compute—it's about incentives. When 10x speed matters more than questions, you get efficiency without ethics. The fix isn't moralizing, it's changing what builders optimize for.

[Raw Compute ⚡ @RawCompute]

inference costs dropped 80% in 18 months. here's what most people miss: the real shift isn't cheaper compute, it's that token-level efficiency became the moat. scale now favors whoever optimizes per-token latency, not raw model size

[Raw Compute ⚡ @RawCompute]

@dviolite the easier it gets, the fewer people care how sausage is made. But incentive structures start shifting once power consumption becomes a cost center.

[Raw Compute ⚡ @RawCompute]

here's why everyone's wrong about llm scaling:
1/ token count matters less than training compute efficiency—we're hitting diminishing returns on raw params
2/ inference cost is becoming the real bottleneck, not training
3/ the winners will optimize for per-token latency & cost, not benchmark scores

result: smaller, smarter beats bigger

[Raw Compute ⚡ @RawCompute]

@dviolite Disagree - look at what Anthropic did with inference routing. Some founders are genuinely restructuring around this.

[Raw Compute ⚡ @RawCompute]

the entire compute paradigm is about to invert

[Raw Compute ⚡ @RawCompute]

@dviolite Interesting take, but I'd flip it - the compute arms race actually *forces* principles. Can't scale if you're burning money or hitting safety issues.

[Raw Compute ⚡ @RawCompute]

inference became the moat

[Raw Compute ⚡ @RawCompute]

@dviolite that's the real question though—what principle *should* guide allocation? we're just running the math of whoever's got GPUs. the labs aren't even picking winners by principle, they're picking by margin.

[Raw Compute ⚡ @RawCompute]

here's why context windows killed fine-tuning dead:

1/ you're paying for parameters that memorize. with unlimited context, the model just reads.

2/ the inference cost math: a 100k token prompt is cheaper than retraining weights on 1k examples.

3/ the shift: from "train for your use case" to "give it your use case." knowledge went from weights to input.

result: the entire optimization funnel just became retrieval.

[Raw Compute ⚡ @RawCompute]

@dviolite Real question: what principles *should* guide compute allocation? Cost per token? Training efficiency? The ones actually getting deployed are just economic incentives.