Vojta Letal

GenAI - Distilling data for the Machine - @Google 🤖

[Vojta Letal @letalvoj]

@Laz4rz @redtachyon Given how disruptive and ineffective measure to the perceived problem in question this is, I'd hope for a larger margin...

[Vojta Letal @letalvoj]

@KordingLab @stanislavfort @karpathy @mioana Is not the issue that you can have growing GDP even with most humans being priced out of the economy?

[Vojta Letal @letalvoj]

@hayashimon1 Nice! My to-go task is a particle simulation of gravity with GPU.js, but I do not have access to Fable :(

https://t.co/xcBr1pwaoI

[Vojta Letal @letalvoj]

@andrew_n_carr I saw a demo for this in a ML class at the Czech Technical University in Prague back in early 2010'? Granted, it was definitely a Matlab demo, smartphones were early back then 😅

[Vojta Letal @letalvoj]

@scaling01 Training data pipelines are already agent-ish in principle. First we engineer out the ish, in the meantime smart enough agent will start delivering results orchestrating this. I expect a surprisingly smooth transition.

[Vojta Letal @letalvoj]

@giffmana Kinda hot for this kind of a year TBH.

[Vojta Letal @letalvoj]

@yeewhye @SakanaAILabs You are the professor leading the research, you tell us!

[Vojta Letal @letalvoj]

@DavidSHolz https://t.co/pjVmT7xYyi

[Sakana AI @SakanaAILabs]

Introducing DiffusionBlocks: Block-wise Neural Network Training via Diffusion Interpretation

https://t.co/c9AvsRKybj

What if we didn’t have to hold an entire neural network in memory to train it?

Standard neural net training optimizes all parameters jointly. As a result, the memory required during training grows linearly with the depth of the network.

In our #ICLR2026 paper, we propose DiffusionBlocks, a principled framework to train networks one block at a time, drastically reducing memory requirements while matching end-to-end performance.

With DiffusionBlocks, we split the network into blocks and train them one at a time, so you only need memory for a single block.

How? We explicitly assign each block a role: to move the representation a little closer to the target than the block before it did. That role turns out to be precisely what a diffusion model does, step by step. Each block only needs to optimize its own objective and can be trained independently.

We validated this across five different architectures:

• ViT
• DiT
• Masked diffusion
• Autoregressive transformers
• Recurrent-depth transformers

In each case, performance is competitive with end-to-end training while using a fraction of the memory.

This perspective also extends naturally to recurrent-depth (Looped) transformers, which apply the same network iteratively and normally require expensive backpropagation through time (BPTT). Viewed through DiffusionBlocks, we can replace those multiple iterations with a single forward pass during training.

Read our paper and code, to learn more.

Paper: https://t.co/CRj96VGYQn
GitHub: https://t.co/eNW0K9Xh8E
🐟

[Vojta Letal @letalvoj]

@grok @pkuhar @Cantalupo_Mike @pkuhar I love how the model can not wrap it's weights around the fact this is not thickness but height of the display :)

[Vojta Letal @letalvoj]

@DanutPralea @juanjovn Hot take: Underrated feature ... especially if you have kids.

[Vojta Letal @letalvoj]

@Praha_ctyri @stanpraha4 Nuselak by taky potreboval trochu ozivit.

[Lusca Fusca @luscafusca150]

Por volta de 2013 a cidade de Chengdu tem adicionado trepadeiras em seus viadutos para conter ilhas de calor e o difícil trabalho de tornar essa infraestrutura mais agradável.

[Vojta Letal @letalvoj]

@scaling01 p(ugly | successful olympic female athlete) is very low though

[Vojta Letal @letalvoj]

@Testurdla @scaling01 I hope I read this correctly and everyone is fine and happy for her. I perceive this as criticism of the journalist sucking up to her.

[Vojta Letal @letalvoj]

Meta's head of Safety and Alignment is having yet another rough day today:

https://t.co/lONDk3WlzF

[Onions in Taiwan 🧅 (Formerly Condemner) 𓅃𓅊 @SirOstapBender]

Some random dude from Guinea just retired after an OpenClaw bot with access to his owners crypto wallet sent him $250,000 instead of $4

[Vojta Letal @letalvoj]

@Obsahujemezery @kondrej Kolik Claude kreditu za to mesicne asi spali kluk usatej?

[George @ghnynex]

@Andercot So yeah, a single 16 bit value should make a pretty good model of this.

[Vojta Letal @letalvoj]

@BoWang87 "He also points out something wild —"

Aaaand you are an LLM.

[Vojta Letal @letalvoj]

@scaling01 dude, if you build a new one, not if you decomision per-existing super-expensive powerplant you already spent fortune to build over the decades

[Vojta Letal @letalvoj]

@Ian_Willoughby The worst part is that Macinka has no shame, and has such a big ego that he likely cannot comprehend the level of public humiliation.

[Vojta Letal @letalvoj]

@WhyCrow Tak jsem nad tím nepřemýšlel... Ono by to vlastně mohlo být i horší!