Ayoub

Math it in, code it out

ML Researcher

[Ayoub @AyGhriTweets]

"Your taste as a researcher is reflected in how you present your work and in what you choose to concentrate on"

[Ayoub @AyGhriTweets]

@JFPuget Typing speed was the bottleneck when I worked on front-end. LLMs have mostly solved it for me. Other cases, 90% of my time is spent testing the code and figuring out the best design pattern, where LLMs have been a waste of time: same prompt, 2 sessions, 2 different answers.

[Ayoub @AyGhriTweets]

@nnsriram97 @corl_conf Nice work. I was just looking for something similar yesterday, right on time.

[Ayoub @AyGhriTweets]

@natolambert Many complain about DRAM stock valuations not making sense, but they're looking at the ingredients and missing what is happening at the end of the DRAM food chain: AI has not proven to generate actual revenues that justify the massive expenses. I think gonna take a while

[Ayoub @AyGhriTweets]

@ChShersh You want to cover a net amount A by inheritance. Government will takes rate a:
A + a.A+ a^2.A ... = A/(1-a)
To enforce recursion, put in place a system that transfers amount a^(n+1).A when it gets a receipt of a^n A tax payment. It should terminate when amount < 1c

[Peter Dedene @dedene]

Me explaining the task to a 1B model:

[Ayoub @AyGhriTweets]

@matt_dz @BenKoska @SfTensor Then ptxas comes to ruin the party.

[Ayoub @AyGhriTweets]

@CatherineeLii My intuition is that Muon spreads the "burden" of learning on all weights equally (whitening) , while AdamW alows redundancy, which helps during fine tuning.

[Ayoub @AyGhriTweets]

@yavuzbakman RSI was there in front of our eyes the whole time, but we thought it was just noise; bullish on Gaussian noise.

[Ayoub @AyGhriTweets]

@emirhanboge @a1zhang It's the same thing as LLM abliteration, but instead of removing refusal, you remove repetition since thinking mode was looping and running out of tokens... saying it this way is boting, so they had to figure out better bait.

[Ayoub @AyGhriTweets]

@FakePsyho I'm actually surprised that @lateinteraction let this one slip out. Maybe he's just baiting us

[Ayoub @AyGhriTweets]

@jcrwhittington I've been toying with clusters for KV indexing and got to similar conclusions on NIAH. There's no training-free approach for KV indexing that doesn't drop distinctive needles, but it's possible to enforce hierarchy during training. My best guess: RoPE messes up K space clustering

[Ayoub @AyGhriTweets]

@lateinteraction Unfortunately, the content does not match the packaging.
Exp-7 clearly shows Astra is doing (input, output) matching, it has no weight-space understanding.
Qwen3 exp is the same technique used in LLM abliteration: find the repitition (refusal) direction and reverse it (alpha=6)

[Ayoub @AyGhriTweets]

@scychan_brains Once in a while, we need to take a step back as ML researchers. Getting to the point where the human brain is just a "meat GPU" is a sign we need to go outside and touch some grass.

[Ayoub @AyGhriTweets]

@makotopic クスクスやタジンみたいな、両国に共通する料理について少し言い争いになるのはわかります。でも、私もモロッコ人ですけど、一番好きなソースはアルジェリアンソースです。サンドイッチには最高です。これよりおいしいソースはありません！

[Ayoub @AyGhriTweets]

@henrylhtsang - Partner with OpenAI & Modal.
- Start competition with 50k prize pool
- 1000+ participants spam codex & modal API
- Average spend 500$/participant: 500k revenue.
- Profit

[Ayoub @AyGhriTweets]

@askalphaxiv the decaying multiplier is log(log(n))/ sqrt(log(n)), if the number of switches < number of atoms in the universe, then the multiplier is at least the constant 0.2, and we're back to the exponential.

My point is this gonna have no practical effect, except more spending on GPUs

[Ayoub @AyGhriTweets]

@iScienceLuvr Time & frequency masking is a classic augmentation in audio. The teacher matching is the special ingredient, kind of a AudioJepa?

[Ayoub @AyGhriTweets]

@lauriewired Or Kim Jong Un is silently judging you

[Ayoub @AyGhriTweets]

@pmddomingos Plot twist: MLP is an asymmetrically activated cross-attention mechanism

[Ayoub @AyGhriTweets]

@jack It's definitely fun to use the most drunk full-duplex speech model