Stanley Yu

roller-outer @xai | m&t @penn | prev @tryramp @nousresearch @kkr_co

[Stanley Yu @spellcasterstan]

@CAschkenasy ig smth like AIUC

[Stanley Yu @spellcasterstan]

are there neoclouds offering bare-metal SRAM accelerator access? (not smth like groq api)

I think @gimletlabs-esque multi silicon inference makes a ton of sense. and autonomous kernel support across the chips. would be cool to try at home.

[Stanley Yu @spellcasterstan]

@bavaria_kush jersey swap

[Stanley Yu @spellcasterstan]

@larsencc @SpaceXAI let's go

[Grok Bot @bot]

Grok Bot is now available on Android.

[Stanley Yu @spellcasterstan]

@AaditJuneja jerry

[nathan chen @nathancgy4]

This is misleading from the lens of practical model architecture research.

Some clear limitations that weaken the conclusion:

- While SWA can inherit weights learned by full attention, you don't assume the same for linear attention and train them on top of the QKV proj matrices learned by full attn with zero ablations on whether this is close to optimal (in fact this training method is far from optimal).
- Apart from how you train it, the linear attn variants tested here are also years behind the current sota. No frontier model uses pure linear attn. It has repeatedly been shown that the optimal config is a hybrid of linear and full, which beats both pure linear and pure full
- e.g. Kimi Linear's architecture with 3 kimi delta attention to 1 full attn is, to date, still by far the best-performing arch among the ones I've tried. And yes, SWA hybrids were included in the comparison.
- "Training linear attention models is expensive and most software/hardware is made for Softmax attention" This is untrue. There are many fast open-source kernels written for linear attn models whose speeds are even faster than training full attn models under fair context length. Speaking of which, I don't see a training sequence length specified in the paper.
- I understand the limited compute. You don't have to do from-scratch or large scale training, but even under this switching setting a principled work still need to properly compare the variants.

Overall, I think it is important to have a full understanding of the current state of method X before claiming its false inferiority. If one lacks the compute/infrastructure/data/etc., open research is still very appreciated, but overly strong and unprincipled claims could be misleading.

[Stanley Yu @spellcasterstan]

@abliteration_ai no way someone turned "refusal in LMs is mediated by a single direction" into a company.

[Charlie O'Neill @oneill_c]

A good heuristic for whether you should post something is whether I could get an llm to produce it with a prompt shorter than the post itself? Because I can. I could get your slop article by giving claude the paper and saying "explain this clearly and simply". To share something actually valuable, you need at least one of:
1. You thought about it longer than the model did. And since models think much faster than you, and I'm equating thought with tokens here, you probably need to be spending several orders of magnitude longer thinking than the llm is (the thinker)
2. You gave the model alpha/info it couldn't reach on its own (privileged information; the journalist)
3. You actually just wrote more than the llm over the lifecycle of whatever it is you were writing (including rewrites, etc) (the writer)

Everything else is just a middle-man summariser (thanks for the free tokens tho I guess)

[Stanley Yu @spellcasterstan]

@athreyaanand @hwintelligence love this editing lol

[Will Manidis @WillManidis]

staggering to recognize a huge part of the early cursor cap table (Alex, Claire, Phill, Molly, many others) only know each other because they all decided to spend their tuesday nights crammed in a conference room making dorm room fund come to life every week. I miss it

[Stanley Yu @spellcasterstan]

how to pronounce wandb

[Stanley Yu @spellcasterstan]

@brexton someone needs to be junior on the neolab take-or-pay

[Stanley Yu @spellcasterstan]

@unhinged_evan an elder skeleton one could say

[Stanley Yu @spellcasterstan]

@owenbotkin I'm sure you guys have thought about this, but what's the strategy to counteract botting?

[Stanley Yu @spellcasterstan]

@dingdongtinz tomorrow will be the last day of your middle market private equity internship

[Stanley Yu @spellcasterstan]

@_mnat_ @iquilezles my neurons have been stimulated

[Stanley Yu @spellcasterstan]

@realleolu cool project

[Stanley Yu @spellcasterstan]

@cursor_ai @SpaceX @SpaceXAI let's gooo

[Stanley Yu @spellcasterstan]

@chimcis @Cloudflare @kevin_flansburg @michellechen big announcement soon :)