I wrote an article on agentic coding for beginners after my talk at @apsarathchandar@ChandarLab group. We cover history of AI coding tools, the importance of model harnesses, and general principles in simple research workflows.
Feedback is very welcome!
https://t.co/1G3B7Xyidk
Excited to share our latest work: Loss Smoothing for Stable Adaptation Under Distribution Shift.
TL;DR: When adapting a model, don’t abruptly switch to the new objective. Gradually smooth the transition from the old objective to the new one.
Preprint: https://t.co/gkJd29pqYe 🧵
1/n
Diffusion world models can help test and improve robot policies before running them on real robots.
But can the choice of latent space make the WM more faithful?
We show that semantic spaces beat reconstruction spaces on task relevant metrics.
https://t.co/BwHZk7ciUQ
(1/7) We introduce MDM-Prime-v2 which scales 21.8× better than autoregressive models (ARMs) in compute-optimal comparisons.
📎 Paper: https://t.co/VhBVo75abe
🌟 Blog: https://t.co/miWdTmcGtL
⌨️ Github: https://t.co/ac1eDV8O8Q
Here’s how we did it👇:
@themintsv Then again I think it's worth doing the free trial for things like GitHub Copilot and Cursor to see if you like it before going to paid tiers. I started really enjoying vibe-coding with Cursor, so it's worth a try!
I wrote an article on agentic coding for beginners after my talk at @apsarathchandar@ChandarLab group. We cover history of AI coding tools, the importance of model harnesses, and general principles in simple research workflows.
Feedback is very welcome!
https://t.co/1G3B7Xyidk
@themintsv I don't have experience with these, but I know that Cline, Roo Code, Aider are (free?) open-source tools and you would probably need to use these with Ollama to service a local model.
Simply adding Gaussian noise to LLMs (one step—no iterations, no learning rate, no gradients) and ensembling them can achieve performance comparable to or even better than standard GRPO/PPO on math reasoning, coding, writing, and chemistry tasks. We call this algorithm RandOpt.
To verify that this is not limited to specific models, we tested it on Qwen, Llama, OLMo3, and VLMs.
What's behind this? We find that in the Gaussian search neighborhood around pretrained LLMs, diverse task experts are densely distributed — a regime we term Neural Thickets.
Paper: https://t.co/rFJz2kVEOA
Code: https://t.co/HAmonfpXIA
Website: https://t.co/QZ6AMIsKCw
@atmoio Thank you, your talk resonated with me and I’m sure it has resonated with many others. It’s so important to question and think about what we’re doing
Radar graphs are among the worst ideas in data visualization. The whole point of them is to show the area and you can usually reorder the labels freely in order to create a desired dramatic effect.
Two versions of the same graph:
- left one tells the story that AI is rapidly replacing whole industries
- right one shows the "jaggedness" and reinforces the idea that humans will always have something that AI won't be able to replicate
🧬 New paper
Scientific datasets evolve as science evolves. With proteins, new sequences get added, annotations get corrected, and noisy entries get curated out.
Introducing CoPeP, a continual-pretraining benchmark for protein LMs.
Details 🧵
1/n
@tallinzen@linluqiu Thanks for the paper! If it’s ok to ask, why choose SFT instead of RL to train the model to approximate probabilistic inference? There is a wealth of work relating RL and probabilistic inference, even for LLMs. Maybe I'm missing something but RL seems like the more obvious choice
@GoogleResearch Why did the authors use SFT instead of RL to train the model to approximate probabilistic inference? There is a wealth of work relating RL and probabilistic inference, even for LLMs. Maybe I'm missing something but RL seems like the obvious choice
I grew up in a world where we could (usually) rely on foundational principles like democracy, free markets, and civil liberties. But we only have these things because past generations sacrificed for them. If we don't defend them, we lose them. I'm proud to work for a company that's standing up for our core values even when it's costly.
Streaming Reinforcement Learning (RL) is a huge challenge: transitions are used once and discarded immediately. This makes agents extremely sample-inefficient. But what if we could "squeeze" more information out of every single frame?
Check out our latest paper!
Can AI agents design better memory mechanisms for themselves?
Introducing Learning to Continually Learn via Meta-learning Memory Designs. A meta agent automatically designs memory mechanisms, including what info to store, how to retrieve it, and how to update it, enabling agentic systems to continually learn across diverse domains. Led by @yimingxiong_ with @shengranhu 🧵👇 1/