Atmadeep Banerjee

Building @sabi. Prev: mapped brains @HarvardVCG, taught satellites to see @PixxelSpace, drones @join_ef. Future is wonderful.

[Atmadeep Banerjee @abanerjee99]

@mihirneal They don't know how. They don't have the technology.

[Atmadeep Banerjee @abanerjee99]

@nrehiew_ https://t.co/NfcAHJX3an

[Atmadeep Banerjee @abanerjee99]

They should launch a new Trump Rhodium card for $10M that only gives Fable 5 access.

[Atmadeep Banerjee @abanerjee99]

They should launch a new Trump Rhodium card for $10M that only gives Fable 5 access.

[Atmadeep Banerjee @abanerjee99]

@creatine_cycle Those prices won't last without Fable

[Atmadeep Banerjee @abanerjee99]

Selling my rollouts from the past few days of Fable5 maxxing. No lowballers. I know what I got.

[Atmadeep Banerjee @abanerjee99]

You can get Fable to do frontier LLM research btw. Just tell it you are working on text-to-speech models.

[Atmadeep Banerjee @abanerjee99]

Hope they livestream their board meetings. Kinda want to see Bonnie Blue argue with Garry Tan.

[Elsa @elsaprofitable]

Dreams do come true. Thanks @ycombinator

[Atmadeep Banerjee @abanerjee99]

@DeepDishEnjoyer So the peak was $25k/night and we are at $30k/weekend. The top is not in yet. Bullish.

[Atmadeep Banerjee @abanerjee99]

@leebriskcyrano US techbros talking about Chinese researchers the same way US leftists talk about AI models.

[Atmadeep Banerjee @abanerjee99]

@typedfemale Just use Adam and let AGI figure this out

[Atmadeep Banerjee @abanerjee99]

GANs will be solved when continual learning is solved. Disc learning is a continual learning problem. You start by allocating params to broad features and as G improves you have to move to finer feats. But you can't forget the old broad feats as the G can regress to fool you. D solves different problems at different parts of training.

Fixing the loss landscape is one thing but it might need arch changes. Probably a growing MoE type continually learning D.

[kalomaze @kalomaze]

classic/hinge GAN loss == binary cross entropy
rpGAN/relative GAN loss == bradley terry
mode collapse from the old GAN literature is primarily an artifact of the fact that pointwise scalars aren't grounded in the relative gap between the real and fake samples
ranking > rating

[Atmadeep Banerjee @abanerjee99]

We called this bagging and boosting

[ar0cket1 @ar0cket1]

Has anyone tried a mixture of idiots instead of mixture of experts.

[Atmadeep Banerjee @abanerjee99]

@MainzOnX People building the >2kW fp4 chips still need fp64 for simulating thermals.

[Atmadeep Banerjee @abanerjee99]

This would go so hard on booktok

[Ashton Hall @AshtonHallofc]

Missing my Indian brother

[Atmadeep Banerjee @abanerjee99]

@xlr8harder By betting on CCP distillation machinery.

[Atmadeep Banerjee @abanerjee99]

Welcome back Deep Belief Nets

[Sakana AI @SakanaAILabs]

Introducing DiffusionBlocks: Block-wise Neural Network Training via Diffusion Interpretation

https://t.co/c9AvsRKybj

What if we didn’t have to hold an entire neural network in memory to train it?

Standard neural net training optimizes all parameters jointly. As a result, the memory required during training grows linearly with the depth of the network.

In our #ICLR2026 paper, we propose DiffusionBlocks, a principled framework to train networks one block at a time, drastically reducing memory requirements while matching end-to-end performance.

With DiffusionBlocks, we split the network into blocks and train them one at a time, so you only need memory for a single block.

How? We explicitly assign each block a role: to move the representation a little closer to the target than the block before it did. That role turns out to be precisely what a diffusion model does, step by step. Each block only needs to optimize its own objective and can be trained independently.

We validated this across five different architectures:

• ViT
• DiT
• Masked diffusion
• Autoregressive transformers
• Recurrent-depth transformers

In each case, performance is competitive with end-to-end training while using a fraction of the memory.

This perspective also extends naturally to recurrent-depth (Looped) transformers, which apply the same network iteratively and normally require expensive backpropagation through time (BPTT). Viewed through DiffusionBlocks, we can replace those multiple iterations with a single forward pass during training.

Read our paper and code, to learn more.

Paper: https://t.co/CRj96VGYQn
GitHub: https://t.co/eNW0K9Xh8E
🐟

[Atmadeep Banerjee @abanerjee99]

Depending on how you implement your sampling, diffusion can have far worse ttft for sequential outputs like creative writing/audio/video. AR is inherently streamable.

In agentic coding this does not really matter as you want to minimize the total time to generate the whole response

[Atmadeep Banerjee @abanerjee99]

@MostlyMonkey @menhguin @DeepDishEnjoyer Out of college at 22 I was making 7x my monthly spend post tax. Not including investment income. This was at at a tech startup at peak zirp but such salaries are sustainable in quant. 5 years of this would buy 30 years of retirement. Add investment incomes and 35 yrs easy.

[Atmadeep Banerjee @abanerjee99]

Not weird if you are from a right hand drive country

[lilly sharples @lillysharples]

Anyone who rides shotgun alone in a Waymo should be studied

[Atmadeep Banerjee @abanerjee99]

Language people inventing FPNs from first principles.

[Xiangdong Zhang @aHapBean]

Sorry for the late release.

NITP (Next Implicit Token Prediction) is about to release, expected next Monday.😀

A new LLM pre-training paradigm that goes beyond next-token prediction by learning the next token’s implicit representation, fixing representation degeneration.