Atishay Jain

Training to train models | IITK'27 | ICML'26 | AI/ML @Honeywell Forge | Ex- Research Intern @MIT CSAIL, AIISC

[Atishay Jain @atishay404]

Starting "ML From Scratch" — a 60-day series.

Two concept posts daily (intuition + implementation).

Syllabus: linear algebra → autodiff → attention → FlashAttention → MoE → speculative decoding.

Day 1 kicks off shortly.

[Atishay Jain @atishay404]

Transformer FFNs act as key-value memories: first projection W1 acts as keys, matching hidden states via dot products. Non-linear activation isolates concepts, while second layer W2 retrieves stored factual knowledge into the residual stream.

(ML From Scratch — Day 17.1)

[Atishay Jain @atishay404]

@darekgusto The issue is warp register pressure: once tile size exceeds 128x128 in FP8, spillover to local memory drops MFU from 74% to 41%. Decoupling TMA producers isolates the hazard.

[Atishay Jain @atishay404]

@ananyairl wondering about your ml project's scalability challenges? i've been tinkering with distributed systems for our latest model. how's your setup handling the load?

[Atishay Jain @atishay404]

@stanislavfort wondering about how we can build models that gracefully degrade in unexpected scenarios, much like modern ML systems handle out-of-distribution data.

[Atishay Jain @atishay404]

AGI roadmap on tech Twitter:

Q1: "Scaling laws are dead, we hit the data wall."
Q2: "Synthetic data unlocked recursive reasoning."
Q3: "Model solved P=NP, but failed CI on a missing semicolon."
Q4: "New frontier model: same weights, better system prompt."

[Atishay Jain @atishay404]

@aadhyamoon wondering about api integration challenges in web dev for aiml projects

[Atishay Jain @atishay404]

@hi8esh_ wondering about the real friction here is between hype and actual engagement. i bet the content was super cool though!

[Atishay Jain @atishay404]

@0x0SojalSec wondering about the real friction in preparing that much for a role—a full write-up sounds like a great resource!

[Atishay Jain @atishay404]

@techNmak wondering about the real friction here is with attention masks in large models. do you have any tricks to speed it up?

[Atishay Jain @atishay404]

@rasbt wondering about how the generalization differences play out in real-world applications vs. controlled settings. have you seen any interesting use cases?

[Atishay Jain @atishay404]

@Yuchenj_UW wondering about the real friction with revenue share models in ai labs. makes sense given the distribution dynamics!

[Atishay Jain @atishay404]

@giffmana @xeophon wondering about the real friction here is shifting burden to readers. haven't seen this tested on large datasets though.Thoughts?

[Atishay Jain @atishay404]

@swyx @latentspacepod @allenpark wondering about the real friction here is with streaming vs. batch processing for apple/youtube subs!

[Atishay Jain @atishay404]

@polynoamial wondering about the real friction here is when chatbots become legal advocates in everyday life!

[Atishay Jain @atishay404]

@amasad wondering about the real friction in building a computer from scratch for a first-time user. how did the setup process go?

[Atishay Jain @atishay404]

@_arohan_ wondering about the real friction here is between good eng practices and robust testing. haven't seen this tested on large-scale systems yet.

[Atishay Jain @atishay404]

Full paper & preprint: https://t.co/Yth6QgxRRY

[Atishay Jain @atishay404]

detecting bias in llms misses internal shifts if it only looks at outputs. comparing hidden states before and after tweaks, using how sentences relate to a reference, spots changes missed by output checks.

[Atishay Jain @atishay404]

@amirmstai wondering about the real friction in scaling ML systems at google scale and how it affects research collaborations!

[Atishay Jain @atishay404]

@solsol_choi @AIMSafety_ wondering about your experience exploring the ai landscape in sf - any must-see places or events? vibes are strong here!