tgeo92

[Jon Barron @jon_barron]

If your conference talk is good, you should upload it somewhere (I use YouTube).

If your talk is not good enough for you to feel comfortable uploading it somewhere, then it is *definitely* not good enough to present at the conference, and you should fix it.

[Sakana AI @SakanaAILabs]

Introducing DiffusionBlocks: Block-wise Neural Network Training via Diffusion Interpretation

https://t.co/c9AvsRKybj

What if we didn’t have to hold an entire neural network in memory to train it?

Standard neural net training optimizes all parameters jointly. As a result, the memory required during training grows linearly with the depth of the network.

In our #ICLR2026 paper, we propose DiffusionBlocks, a principled framework to train networks one block at a time, drastically reducing memory requirements while matching end-to-end performance.

With DiffusionBlocks, we split the network into blocks and train them one at a time, so you only need memory for a single block.

How? We explicitly assign each block a role: to move the representation a little closer to the target than the block before it did. That role turns out to be precisely what a diffusion model does, step by step. Each block only needs to optimize its own objective and can be trained independently.

We validated this across five different architectures:

• ViT
• DiT
• Masked diffusion
• Autoregressive transformers
• Recurrent-depth transformers

In each case, performance is competitive with end-to-end training while using a fraction of the memory.

This perspective also extends naturally to recurrent-depth (Looped) transformers, which apply the same network iteratively and normally require expensive backpropagation through time (BPTT). Viewed through DiffusionBlocks, we can replace those multiple iterations with a single forward pass during training.

Read our paper and code, to learn more.

Paper: https://t.co/CRj96VGYQn
GitHub: https://t.co/eNW0K9Xh8E
🐟

[Lilian Weng @lilianweng]

Giving your models more time to think before prediction, like via smart decoding, chain-of-thoughts reasoning, latent thoughts, etc, turns out to be quite effective for unblocking the next level of intelligence.

New post is here :)

“Why we think”: https://t.co/21xduoZaAF

[Rui Li @leedaray]

🚀 Details of the #CVPR2025 award candidate papers are out.

14 of 2967 accepted papers made the list, spanning 3D vision, embodied AI, VLMs/MLLMs, learning systems, and scene understanding. 3D vision leads with the most entries.

I collected the TL;DR, paper, and project links👇

[François Fleuret @francoisfleuret]

As expected, that was popular. Here is my attempt at consolidating all the answers into a list.

- Prenorm: normalization in the residual blocks before the attention operation and the FFN respectively

- GQA (Group Query Attention): more Q than (K, V)

[François Fleuret @francoisfleuret]

What do you think are the most important improvements of the 2017 transformer architecture?

[Sander Dieleman @sedielem]

New blog post: let's talk about latents!
https://t.co/Ddh7tXH642

[Soheil Feizi @FeiziSoheil]

Re-sharing my 3.5 hr lecture on large language models as some might be interested. Will post updated lectures/materials/news in coming weeks. Stay tuned!

Link: https://t.co/KI5CJ6OksI

Topics:
0:00 Basics of language models
2:30 Word2vec
16:27 Transfer Learning
19:23 BERT
1:00:39 T5
1:31:14 GPT1-3
1:53:05 ChatGPT
2:20:03 LLMs as Deep RL
2:53:00 Policy Gradient
3:32:50 Train your own LLM

[Sasha Rush @srush_nlp]

Got talked into giving a DeepSeek talk this afternoon https://t.co/TqQFUwXpac

Not sure I have anything new to say here! But good excuse for me to read all the blogs.

[Glen Berseth @GlenBerseth]

I am teaching a class on #FoundationalModels for #robotics and Scaling #DeepRL algorithms. This class expands on last year's class and my generalist robotics policies tutorial and code. I plan to share the lectures and code assignments. Starting with the first lectures below.

[Jitendra MALIK @JitendraMalikCV]

I'm happy to post course materials for my class at UC Berkeley "Robots that Learn", taught with the outstanding assistance of @ToruO_O. Lecture videos at https://t.co/jAUdVAJRHS Lecture notes & other course materials at https://t.co/37OAxRAneH

[Yujin Tang @yujintang99]

Tips for Writing a Research Paper using LaTeX https://t.co/nqV6yn1YlC

[Christian Wolf (🦋🦋🦋) @chriswolfvision]

It looks like there are some efforts to improve on vanilla attention in transformers ...

https://t.co/M2zDTbAS6P Selective Attention
https://t.co/5X9i6x6OHd MHA as Mixture of Heads Attention
https://t.co/YHsyim0mSZ Differential Transformer

[Nick Jiang @nickhjiang]

🔥 Paper Drop 🔥

What can we understand by peering inside vision-language models (VLMs) like LLaVA?

We show that image representations inside VLMs can be directly interpreted and edited in the language space, and we apply our findings to mitigate hallucinations!

[Sasha Rush @srush_nlp]

Are you writing an ICLR paper, and wondering...
* What level of detail should the background have?
* Why do people read related work?
* How do I convey convincing results?

I have some mostly satisfactory answers to these questions.

https://t.co/CtowMH95lT

[Chunyuan Li @ChunyuanLi]

LLaVA-OneVision-Chat🤖💬 significantly improves the chat experience of LLaVA-OneVision, as demonstrated on 5 multimodal task! 🌟, achieved by preference alignment & self-critic feedback 📊

Doc: https://t.co/Ublty5KZFe

[Chanwoo Park @chanwoopark20]

I'm thrilled that the paper @KaiqingZhang and I co-authored has made it onto the reading list for this course! https://t.co/BQIECFRZsR

Any questions would be appreciated.

[Lucas Beyer (bl16) @giffmana]

I’ll be giving a talk on « vision in the age of LLMs » on Monday 11:00 cet which is open to public via the link below:

[Mickael Chen @Mickael_Chen]

I made a github repository exploring this question. https://t.co/tTDKAZTwt7
We visualize the embeddings inside a DiT and observe how it works around this issue. We also build a very simple toy experiments to see the limits of the method.
tldr: not a big issue, but kind of ugly.

[Sasha Rush @srush_nlp]

New Video: How to write an okay research paper.

Reviewers all agree! @srush_nlp's papers are "reasonably structured" and "somewhat clear, despite other flaws".

https://t.co/nCjYsDI5Jf