Sungguk Cha

[The AI Timeline @TheAITimeline]

🚨This week's top AI/ML research papers:

- DiffusionBlocks
- A Bitter Lesson for Data Filtering
- Neural Weight Norm = Kolmogorov Complexity
- When Does LeJEPA Learn a World Model?
- Do Language Models Need Sleep?
- Parallax
- Gemini Embedding 2
- Qwen-VLA
- The MiniMax-M2 Series
- Looped Diffusion Language Models
- LocateAnything
- Learn from your own latents and not from tokens

overview for each + authors' explanations
read this in thread mode for the best experience

[Sungguk Cha @SunggukC]

@JustinLin610 Thanks four your contributions on the community. Very inspiring and was best to me.

[Rocketesla @rklb_invest]

https://t.co/exc46NwcO4

[Robert Youssef @rryssf]

Deepseek just broke the one rule every transformer has followed for a decade 🤯

x + f(x). the residual connection.

if you don't know what that means, here's the simple version: every time a neural network processes your input through a layer, it keeps a copy of the original and adds it back at the end. like a safety net. if the layer screws up, the original signal survives.

gpt-4 uses it. claude uses it. gemini uses it. every major model since 2015 treats this as sacred. nobody touches it.

Deepseek touched it.

instead of 1 stream carrying your data forward, they split it into 4 parallel streams. each stream carries different aspects of the information. and learned mixing matrices decide how those streams talk to each other at every layer.

more lanes on the highway. smarter traffic control. same computational cost.

sounds perfect on paper. here's where it breaks:

[Sungguk Cha @SunggukC]

@LaylaEleira @grok does this work on offline environment without authentication?

[Sungguk Cha @SunggukC]

@YunpyoAn 앙 교수님이 가시나요? 수업 재밌게 들었었는데...
embedded ai 하시는 분은 전전쪽이 아니면 제가 잘 모르겠네요

답변 감사합니다.

[Sungguk Cha @SunggukC]

https://t.co/cNM2v09sbq
https://t.co/S8IWfNR7aL

I read two impressive papers which proposes on-policy self distillation.
Their approach are semantically the same.
Related to the authors, there are too many things in common.
MIT and ETH
Even one common author

[Rosinality @rosinality]

The teacher synthesizes the data and if the student trained on the data improves on the target data then it is rewarded. Interesting approach to self-play.

[Sungguk Cha @SunggukC]

@Alisvolatprop12 제 기억에 지원할 때 다른 티오였던거 같은데 정시였는지 수시였는지
다음과 비슷했던거 같음
예, A, B, C 대학 지원
단, 과기원 지원은 카운트 안함

[Sungguk Cha @SunggukC]

@karthikponna19 Claude Opus 4.5
Using antigravity, all the fails from Gemini Pro are solved by Opus.

[Sumit @_reachsumit]

ReinPool: Reinforcement Learning Pooling Multi-Vector Embeddings for Retrieval System

@SunggukC et al. at LG presents an RL framework that compresses multi-vector embeddings by 746–1249x into single vectors while recovering 76–81% of retrieval perf.

📝 https://t.co/3vvbi7Zjdg

[Sungguk Cha @SunggukC]

@lateinteraction @grok hey grok, let me understand JEPA, GEPA and NEPA

[Rohan Paul @rohanpaul_ai]

A great @AIatMeta study, to give further context to Fei-Fei Li's explanations on limitations of LLMs.

The paper proves that when AI learns from real videos of the world, it starts to pick up basic physical ideas, like objects still existing even when hidden, or shapes staying the same.

But when AI models try to learn about physics only from text, they do very poorly.

So true physical understanding will come from watching and experiencing the world itself, not from reading language.

Language alone is not enough, since the real world follows physical laws that words cannot fully capture.

[Rohan Paul @rohanpaul_ai]

Fantastic paper from ByteDance 👏

Shows how to train LLM agents to finish long, multi step tasks by letting them act in real environments with reinforcement learning.

Across 27 tasks, the trained agents rival or beat top proprietary models.

Most agents are trained on single turn data, so they fail when a job needs many decisions with noisy feedback.

AgentGym-RL splits the system into separate parts, the environments, the agent loop, and training, so each can improve on its own.

It supports mainstream algorithms and realistic tasks, and the agent learns by acting, seeing results, and adjusting across different settings.

The key method, ScalingInter-RL, starts with short interactions to master basics, then slowly allows longer runs so the agent can explore and plan.

This staged horizon schedule stabilizes learning, prevents pointless loops, and encourages planning, reflection, and recovery after mistakes.

A 7B model trained with this setup matches or beats much larger open models and competes well with strong commercial ones.

They also find that putting more compute into training and test time interaction, like more steps or samples, often helps more than adding parameters.

[Jason Weston @jaseweston]

🌀New Test-time scaling method 🌀
📝: https://t.co/yqWvOMZpwq
- Use RL to train an LLM solution aggregator
– Reasons, reviews, reconciles, and synthesizes a final solution
-> Much better than existing techniques!
- Simple new method. Strong results across 4 math benchmarks.
🧵1/5

[Sungguk Cha @SunggukC]

Daily arxiv
Dynamic Speculative Agent Planning
https://t.co/yPLSaGpWL9

[Sungguk Cha @SunggukC]

Daily arxiv
Parallel-R1: Towards Parallel Thinking via Reinforcement Learning
https://t.co/3mTwNbkvyo

Parallel reasoning RL

[Sumit @_reachsumit]

Generalized Reinforcement Learning for Retriever-Specific Query Rewriter with Unstructured Real-World Documents

LG Uplus introduces a reinforcement learning framework that trains query rewriters for specific retrievers without human annotations.

📝https://t.co/w4l8FyLFOp

[말러팔삼 @mahler83]

ChatGPT를 쓰면 우리 뇌는 게을러질까? 답은 Yes.
SAT식 에세이를 챗지피티, 구글, Brain-only(네이밍 웃긴다 ㅋ) 3집단으로 나누어 일을 시켰을 때 LLM을 사용한 그룹의 뇌 활성화도 줄고, 자기가 쓴 글을 기억 못하고, 글의 다양성이 떨어지고, 소유감(ownership)도 떨어짐. 대박이네.. #논문