bycloud

I make youtube videos on cool AI research /// AI papers newsletter /// paper recap @TheAITimeline ///

[bycloud @bycloudai]

so what's going on with these AI companies
@reflection_ai
@inflectionAI
@magicailabs
that raised a total of $25B + $4B + $1.5B = $30.5B USD

[The AI Timeline @TheAITimeline]

🚨This week's top AI/ML research papers:

- DiffusionBlocks
- A Bitter Lesson for Data Filtering
- Neural Weight Norm = Kolmogorov Complexity
- When Does LeJEPA Learn a World Model?
- Do Language Models Need Sleep?
- Parallax
- Gemini Embedding 2
- Qwen-VLA
- The MiniMax-M2 Series
- Looped Diffusion Language Models
- LocateAnything
- Learn from your own latents and not from tokens

overview for each + authors' explanations
read this in thread mode for the best experience

[bycloud @bycloudai]

@teortaxesTex lost in translation? Tau to chinese is pronounced as Tao and back to pinyin is Tao

[bycloud @bycloudai]

@svgoiboi this aint a dllm

[bycloud @bycloudai]

today is (potentially) a great day for the GPU poors

if DiffusionBlocks works on fine-tuning existing models, then literally any reasonable consumer GPU can do LLM fine-tuning

will make a video on this

[bycloud @bycloudai]

@mvidia84853 ur whole morning might be gone cuz this is a cool rabbit hole

[bycloud @bycloudai]

https://t.co/D5cpDrnMHP

[bycloud @bycloudai]

they were pretty conservative with their paper

so here are some bold and cope potentials if it holds up at scale

> 3-4x memory reduction across the board without much quality loss
> train a small/mid sized LLMs on a single GPU
> if you can train each block independently without much comms: less all-reduce, fewer pipeline bubbles, and reduced comms overhead
> if it works on fine-tuning existing models: consumer GPUs/small clusters can fine-tune SoTA models
> if blocks are independent: partial fine-tuning gets cheaper, since you can update subsets of blocks instead of the whole model

feel free to shut me down

[Sakana AI @SakanaAILabs]

Introducing DiffusionBlocks: Block-wise Neural Network Training via Diffusion Interpretation

https://t.co/c9AvsRKybj

What if we didn’t have to hold an entire neural network in memory to train it?

Standard neural net training optimizes all parameters jointly. As a result, the memory required during training grows linearly with the depth of the network.

In our #ICLR2026 paper, we propose DiffusionBlocks, a principled framework to train networks one block at a time, drastically reducing memory requirements while matching end-to-end performance.

With DiffusionBlocks, we split the network into blocks and train them one at a time, so you only need memory for a single block.

How? We explicitly assign each block a role: to move the representation a little closer to the target than the block before it did. That role turns out to be precisely what a diffusion model does, step by step. Each block only needs to optimize its own objective and can be trained independently.

We validated this across five different architectures:

• ViT
• DiT
• Masked diffusion
• Autoregressive transformers
• Recurrent-depth transformers

In each case, performance is competitive with end-to-end training while using a fraction of the memory.

This perspective also extends naturally to recurrent-depth (Looped) transformers, which apply the same network iteratively and normally require expensive backpropagation through time (BPTT). Viewed through DiffusionBlocks, we can replace those multiple iterations with a single forward pass during training.

Read our paper and code, to learn more.

Paper: https://t.co/CRj96VGYQn
GitHub: https://t.co/eNW0K9Xh8E
🐟

[bycloud @bycloudai]

https://t.co/D5cpDrnMHP

[The AI Timeline @TheAITimeline]

🚨This week's top AI/ML research papers:

- HRM-Text
- Decoupling the Benefits of Subword Tokenization
- Generative Recursive Reasoning
- Probabilistic Tiny Recursive Model
- Vector Policy Optimization
- Linear-Time Looped Transformers
- Gated DeltaNet-2
- Steered LLM Activations are Non-Surjective
- Code as Agent Harness

overview for each + authors' explanations

[The AI Timeline @TheAITimeline]

🚨This week's top AI/ML research papers:

- Self-Distilled Agentic RL
- Long Context Pre-Training with Lighthouse Attention
- Embedded Language Flows
- Negation Neglect
- Efficient Pre-Training with Token Superposition
- Slicing and Dicing
- SlimQwen
- Registers Matter for Pixel-Space DiT
- Scaling Laws for Mixture Pretraining Under Data Constraints

overview for each + authors' explanations

[alphaXiv @askalphaxiv]

Reinforcing Recursive Language Models

Can a 4B model learn to recursively call itself to answer hard long-context questions?

We RL fine-tuned a small model to behave as a native RLM.

On evidence selection across scientific papers, our 4B RLM matches Sonnet 4.6 in quality while running significantly faster and cheaper.

[The AI Timeline @TheAITimeline]

🚨This week's top AI/ML research papers:

- Model Spec Midtraining
- Sparser, Faster, Lighter Transformer LMs
- Continuous Latent Diffusion LM
- Sharpness-Aware Pretraining Mitigates Catastrophic Forgetting
- GLM-5V-Turbo
- Nonsense Helps
- TIDE
- Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior

overview for each + authors' explanations

[bycloud @bycloudai]

u can still read it here
https://t.co/ralGUU8eLh

[bycloud @bycloudai]

"Thinking With Visual Primitives" was taken down without reasons

after reading the paper, my take on why they did that might be because the current version shows that visual primitives can make reasoning much more efficient, but it doesn’t fully answer the big picture that is

How much visual detail can you compress away before better referencing stops being enough?

basically like a trade-off between perception and reference gap

They did something similar with engrams (vs MoE), so maybe they wanted to supplement some more ablation results?

which i hope is the case cuz i would love to see the comparison

[The AI Timeline @TheAITimeline]

🚨This week's top AI/ML research papers:

- The Last Human-Written Paper
- Thinking with Visual Primitives by DeepSeek
- SFT-then-RL Outperforms Mixed-Policy Methods for LLM Reasoning
- Qwen-Scope
- Recursive Multi-Agent Systems
- Co-Evolving Policy Distillation
- Representation Fréchet Loss for Visual Generation
- Tuna-2
- Accelerating RL Post-Training Rollouts via System-Integrated Speculative Decoding
- DORA

overview for each + authors' explanations
read this in thread mode for the best experience

[bycloud @bycloudai]

self plug
https://t.co/Svx5uPQZia

[bycloud @bycloudai]

be DeepSeek

>need to achieve batch invariance so bad
>split-k is the only optimal solution but is batch variant
>Thinking Machines ($50b valuation) could barely recover the performance for their solution, gets 1.6x slower
>hold_my_beer.jpg
> dual-kernel strategy
>"match or even surpass the perf of standard split-k in most major scenarios"
>DeepSeek strikes again
>$20b valuation btw

[bycloud @bycloudai]

https://t.co/oQ8Hr6oOJE

will talk more about it in my vid but this is just incredible that i wanna post about it and i have not seen anyone talked about it

and i might have to make 2 vids at this point (1 arch 1 infra cuz theres just too much cool stuff

[bycloud @bycloudai]

@juliarturc 🤝

[bycloud @bycloudai]

a good 58 page report?

I can give like a good 20 mins yap on it

just gonna take like 1 month for me to make the video LMAO