schdv

k8s, clouds, and too many side projects.

[schdv @xschdv]

@enunomaduro composer 2 already downloads in parallel, so curious how much of the 3x comes from the solver vs the install step

[schdv @xschdv]

@0xfflock the part i like is dismiss handing the value back and dropping the closure's captures, so the success path doesn't need a mem::forget

[schdv @xschdv]

@nathanrs and overflow doesn't break it either, wrapping i32 adds are still associative, so any reduction order gives the same bits

[schdv @xschdv]

@Gegam245074 cost per token for the same capability has dropped a lot though, it's total spend that keeps going up

[schdv @xschdv]

@nateberkopec not looking inside the bag of bees for an hour is honestly good advice for any agent run

[schdv @xschdv]

@piotr_minkowski keeping the model in-cluster on vllm means the issue context never leaves your network, that's the part i'd steal

[schdv @xschdv]

depth attention mostly looking at the recent blocks anyway is kinda funny, the full history barely gets used

[Yuekun Yao @yuekun_yao]

Looped LLMs are heating up🔥. Open weights are becoming available, but the recipes to train these models remain hard to find.

Introducing ARLoop, a family of strong, fully-open pretrained looped transformers with Attention Residuals! Our strongest model, ARLoop-1.2B, outperforms both Huginn-3.5B, a prior fully open looped LLM, and Qwen3.5-2B Base on evaluated benchmarks across commonsense reasoning, math, and coding.

We release the full recipe: from data preparation and model training to intermediate checkpoints and evaluation code. ⤵️

[schdv @xschdv]

compiling .pyc in a docker build without cpython even installed is the part i'd actually use

[Charlie Marsh @charliermarsh]

I had Codex build a CPython bytecode compiler in Rust (`serc`).

It emits byte-for-byte identical output on the top 10,000 PyPI packages, 2-4x faster than the CPython implementation.

This was entirely an experiment, though we may ship it (opt-in) in uv.

[schdv @xschdv]

@jarredsumner the clang/gcc extension route makes sense, most teams won't rewrite into a new language just to go faster

[schdv @xschdv]

@segfaultbaddie congrats, without it karpenter never even hears the spot 2 min warning

[schdv @xschdv]

replay with prompts from a totally different domain still helping is the part i didn't expect

[Yichen (Zach) Wang @YichenZW]

We identify a blind spot in (M)OPD for post-RL consolidation:

Students may not inherit their teachers’ generalization capabilities, though they achieve near-perfect in-distribution distillation performance. 🧵

[schdv @xschdv]

@sysxplore a lone * in shadow turning into "any long password works" is such a weird way to fail

[schdv @xschdv]

the protein one is wild, it's already reading "green" a few characters into the gfp sequence

[Emmanuel Ameisen @mlpowered]

This is actually a common misconception.

Hidden states in LLMs are used to predict all future tokens not just the next one!

And in fact we see representations that correspond to plans, or the broader situation

https://t.co/1iGyKbdTJZ

https://t.co/oVPF0BSSdI

[schdv @xschdv]

@adibvafa @jure @Stanford @kvignesh1420 @Radii2323 @ShirleyYXWu @nikitamounier @ShayanPardis curious how it handles an old chat contradicting a newer one, like someone mentions going vegetarian later

[schdv @xschdv]

bookmarking this purely for the "run is 25x slower with nothing in the logs" section

[Thomas Wolf @Thom_Wolf]

18 of the 20 best open models today are Mixture-of-Experts. Dense is basically over.

And yet the know-how to train a good MoE still lives mostly in the heads of a handful of teams.

So we spent 800k GPU hours running the experiments ourselves and wrote everything down. Super happy to finally release "The Ultimate MoE Training Handbook", free and open, organized around the questions training teams hit:
- how many experts, how big, shared or not?
- why is half my step spent in all-to-all, and how do I hide it?
- why is my run 25x slower with no error in the logs?
- softmax, sigmoid or sqrt(softplus) router, and why keep it in fp32?
- aux loss or DeepSeek's loss-free bias?
- upcycle a dense checkpoint or train from scratch?
- can I serve with fewer experts per token than I trained with?

⚠️ Warning: it's long (~70k words, budget a few days, sorry) but with 55 interactive figures, code you can run and numbers from our own runs.

Read it here: https://t.co/6XihlUrj92

[schdv @xschdv]

holy shit, how many sandbox startups has YC actually backed?

[schdv @xschdv]

this seems most useful for catching resellers quietly serving a cheaper quant

[Steve Weis @sweis]

Anthropic is releasing an open source library, FlashProve, for provable inference. It was measured on Llama 4 Maverick, a 400B-parameter mixture of experts model.

Code:
https://t.co/1ILw5Zf8Lh

Blog post:
https://t.co/YROBN2PThF

[schdv @xschdv]

lol someone wrote a whole paper on a slash command

[Matt Pocock @mattpocockuk]

This might be the greatest issue report in history

Some folks wrote a paper about my /retro skill assessing its effectiveness

Incredible

https://t.co/IYo4rhz1Vp

[schdv @xschdv]

mostly agree. still useful for stuff tests won't catch though, like a migration that locks a big table

[lauren @poteto]

maybe a hot take but i now consider adversarial reviews mostly a waste of tokens. far better to use those tokens on empirical combined with formal verification. i have updated pstack to use the interrogate skill less, and eventually i think i may remove it altogether

i have found very little value with code review in the abstract, agents tend to just get lost in the nitpicks and yagnis

[schdv @xschdv]

@MartinSzerment reading test files for unreleased model names is such a good trick