Sithan Kanna

[Sithan Kanna @sithankanna]

@jacobi_torsten do you use the deepseek api?

[Sithan Kanna @sithankanna]

@jacobi_torsten @FrancoisChauba1 interesting. my supervisor did some work on it a long time ago. the randomness in the connections makes the network hard to inspect.

[Sithan Kanna @sithankanna]

@FrancoisChauba1 @andrewho03 should present. Has some interesting ideas on RL data.

[Sithan Kanna @sithankanna]

I am going to CEO Hour. Join me! https://t.co/vH3W7Q3JCz via @LumaHQ

[Sithan Kanna @sithankanna]

@AbhiCodes15 Helping Solo founders solve problems creatively. https://t.co/yRIGDRvkjF

[Sithan Kanna @sithankanna]

https://t.co/lMusHvbYyp

[Sithan Kanna @sithankanna]

The little things that show how the Codex team cares. Much easier to see token usage vs. the competitor that shall not be named. It's right there, 2 clicks max. Thanks @thsottiaux & the Codex team

[Sithan Kanna @sithankanna]

@balajis NS has a protocol for Burn (intervals, reps) but Learn is mostly unstructured. Why isn't there an interval-training equivalent for thinking? I've adapted the focused-burst + rest structure for thinking https://t.co/yRIGDRvkjF. Could it work at NS?

[Sithan Kanna @sithankanna]

Hey @mstafford, i put HDIT online if you want to do a guided session anytime. https://t.co/yRIGDRvkjF

[Sithan Kanna @sithankanna]

https://t.co/SbBmNFSHXr

[Sithan Kanna @sithankanna]

@dwarkesh_sp @reinerpope In the lecture, somewhere around 1h 40m, @reinerpope mentions why outputs of the kv heads and not q heads are stored. This note goes spells it out in a bit more detail:
https://t.co/54Fz7MX6vB

[Sithan Kanna @sithankanna]

5/
And that's why you never cache Q: the next prediction only uses the newest query. The old ones are never asked anything again.
I'm doing a short live walkthrough of the full derivation 15 mins.
https://t.co/hKMFwfHQxb

[Sithan Kanna @sithankanna]

1/
Every LLM caches K and V during generation, but never Q.
The reason is one line of the attention equation.

[Sithan Kanna @sithankanna]

4/
For the newest token you need: one new query, and all the past keys and values.
But the old keys and values don't change when you append a token. Appending token t+1 doesn't touch it.
So you save them once and reuse them. That saved memory is the KV cache.

[Sithan Kanna @sithankanna]

My notes on Why KV cache and not QKV cache. Explained using a simple 1-Layer Transformer. Using karpathy's setup (ignoring the Batch dimension)

https://t.co/UmJLR1RnXG

[Matthew Stafford @mstafford]

@garrytan @RickRubin @tetranow Listening to this just after doing HDIT in San Francisco is the way 🇺🇸

(For HDIT see this - https://t.co/pdJW0NEdat - and talk to @sithankanna)

[Sithan Kanna @sithankanna]

@METR_Evals Is there a 100% success rate graph. Might be useful to see as well.