@KempnerInst research fellow @Harvard.
trying to understand the human reinforcement learning algorithm.
hope to build AI that helps us live rewarding lives.
Task diversity is supposedly key to generalization in RL. But what does it do to continual RL, where agents face one new task distribution after another?
We find that past a point, more diversity actually inhibits continual reinforcement learning 🧵
More task variety isn't always a silver bullet for RL!
We found that while diversity drives zero-shot transfer, it actually bottlenecks continual learning.
Really excited to see how this tradeoff shapes the way we design future agentic systems. Huge congrats to the team!🚀
Task diversity is supposedly key to generalization in RL. But what does it do to continual RL, where agents face one new task distribution after another?
We find that past a point, more diversity actually inhibits continual reinforcement learning 🧵
@xtwirer I agree that more clever algorithms could make better use of the data!
We wanted to see how far standard solutions like PPO could go, since they lead to systematic transfer within a single distribution shift
Join us at @PrimeIntellect to build the open stack for self-improving agents
Engineering
• MTS – Full Stack Software Engineer — SF/Remote, Full time
• MTS – GPU Infrastructure — SF/Remote, Full time, Hybrid
• MTS – Inference — Remote/SF, Full time, Hybrid
• MTS – Sandbox Platform — SF, Full time, On-site
• MTS – Security — SF, Full time, On-site
• MTS – Training Platform — SF, Full time, On-site
Research
• Research Engineer – Distributed Training — SF/Remote, Full time
• Research Engineer – Reinforcement Learning — SF/Remote, Full time
• Research Engineer – RL Infrastructure — SF/Remote, Full time
• AI Research Resident – Open Source AGI — Remote, Part time
Applied Research
• Evals & Data — SF, Full time, Hybrid
• Forward-Deployed — SF, Full time, On-site
• RL & Agents — SF, Full time
Compute / Finance
• Head of Compute — SF, Full time
• Strategy and Finance Lead, Compute — SF, Full time
Finance / Operations
• Business Operations Lead — Remote, Full time
• Chief of Staff — SF, Full time
• Founder's Associate, Business Operations — SF, Full time
Growth
• Account Executive — SF, Full time
• Head of Enterprise — SF, Full time
• Head of Growth — SF, Full time
• Head of Marketing — SF, Full time
• Revenue Operations Lead, AI Infrastructure — Remote, Full time
• Solutions Architect – AI Infrastructure — SF, Full time
• Technical Account Manager – AI Infrastructure — SF, Full time
Legal
• Head of Legal — SF, Full time
Others
• Internship — SF, Full time
• Open Application for Unconventional Talent — SF/Remote, Full time
Assistant Professor Eugene Vinitsky is teaching autonomous vehicles to safely handle the unexpected.
Click the link to watch the full episode of Office Hours.
#NYUTandonMade
https://t.co/2q2Qpy130f
@nekomata1440 code isn't cleaned up so if you don't mind working with a messier version of the code and want to use it on a project, DM me!
students I worked with are busy this summer so limited on time
If you want to learn more, or see the rest of our analysis, please check out our paper!
📄 Paper: https://t.co/KdtqA0qghM
🌐 Project page: https://t.co/3j3x6CXWfo
This was a fun collaboration with Purab Seth, @neilhshah15, @gershbrain, @kjha02, and @maxhkw !
Strikingly, higher diversity improves backward transfer to the very first distribution!
So the agent keeps getting better on old tasks even as it stops improving on new ones. This suggests it's learning the shared structure, but losing the ability to specialize.
join us at NYU Global AI Frontier Lab!
@c10labs , @nyuniversity and @NYCEDC invite you to an afternoon bridging academia and industry. Student researchers and early-stage startup founders will deliver lightning presentations on work at the frontier of AI, biotech, and hard tech — followed by a panel with investors and academics on what it actually takes to nurture the next generation of innovators.
rsvp link below!