archangeles

compilers & quantum

[archangeles @__archangeles]

gameplay clip

[archangeles @__archangeles]

Made a game where you guess whether license plates were approved or rejected

https://t.co/h6znhk5Cyy

[archangeles @__archangeles]

An air roomba that floats around and eats up dust. I would buy one of these if they existed. Look at it. It looks like a happy balloon. How charming

[archangeles @__archangeles]

Thought about it for longer than a second this time lol. The exponents cancel out when you add the discounting term

e^(old - new) * (e^(x1 - old) + ... + e^(xj - old)
= e^(x1 - new) + ... + e^(xj - new)

[archangeles @__archangeles]

Might sound like a noob but it blows me away how the fused max and normalization term in softmax just works. Like, discounting the normalization term mid-loop by e^(old_max - new_max) when a new max is found. WHY does that work?

[archangeles @__archangeles]

A similar discounting term e^(max(x_1) - max(x_2)) is in FlashAttention tiling

[archangeles @__archangeles]

The proof in that paper is enough to accept it, but I don't feel it intuitively yet

[archangeles @__archangeles]

Might sound like a noob but it blows me away how the fused max and normalization term in softmax just works. Like, discounting the normalization term mid-loop by e^(old_max - new_max) when a new max is found. WHY does that work?

[archangeles @__archangeles]

Leopold will bounce back for sure

[Jack Morris @jxmnop]

BREAKING: ArXiv changes toolbar color from #AE2A24 to #1C1A17

RIP "ArXiv Crimson Red" (1991 – 2026) 🟥 🔴 ❤️

[archangeles @__archangeles]

Implement papers, acquire skills, ???, profit

[archangeles @__archangeles]

@aaaronson What about longest pair of words whose arcs don't intersect each other

[archangeles @__archangeles]

Let's throw trillions of dollars at building a lossy key-value store of the internet. Let's call it conscious while we're at it. We'll be rich

[archangeles @__archangeles]

Nobody knows why you scale softmax inputs with sqrt(d_k) instead of just d_k. There's actually no way to truly know why

[archangeles @__archangeles]

@TrushinBV (71 + 1)(71 - 1) = 7!

Nice...

[archangeles @__archangeles]

Everyone gets what they want, forever

[archangeles @__archangeles]

@son_ofa_car @jbillinson

[archangeles @__archangeles]

I haven't joined Anthropic. Just thought you should all know.

[archangeles @__archangeles]

@staysaasy Infinite Craft was fun and cool.