Wilka Carvalho

@KempnerInst research fellow @Harvard.

trying to understand the human reinforcement learning algorithm.

hope to build AI that helps us live rewarding lives.

[Wilka Carvalho @cogscikid]

Task diversity is supposedly key to generalization in RL. But what does it do to continual RL, where agents face one new task distribution after another?

We find that past a point, more diversity actually inhibits continual reinforcement learning 🧵

[Kunal Jha @kjha02]

More task variety isn't always a silver bullet for RL!

We found that while diversity drives zero-shot transfer, it actually bottlenecks continual learning.

Really excited to see how this tradeoff shapes the way we design future agentic systems. Huge congrats to the team!🚀

[Nepenthe @atnepenthe]

ever wondered how task diversity interacts with continual RL? check out our latest work!

[Wilka Carvalho @cogscikid]

@xtwirer thanks for this resource!

[Wilka Carvalho @cogscikid]

Task diversity is supposedly key to generalization in RL. But what does it do to continual RL, where agents face one new task distribution after another?

We find that past a point, more diversity actually inhibits continual reinforcement learning 🧵

[Wilka Carvalho @cogscikid]

@xtwirer I agree that more clever algorithms could make better use of the data!

We wanted to see how far standard solutions like PPO could go, since they lead to systematic transfer within a single distribution shift

[Wilka Carvalho @cogscikid]

@samzliu That makes sense! Yeah, one explanation is that this could be the primacy bias in effect https://t.co/ludKdHr7R1

[Wilka Carvalho @cogscikid]

been following @PrimeIntellect

seems like an amazing opportunity!

[Vincent Weisser @vincentweisser]

Join us at @PrimeIntellect to build the open stack for self-improving agents

Engineering
• MTS – Full Stack Software Engineer — SF/Remote, Full time
• MTS – GPU Infrastructure — SF/Remote, Full time, Hybrid
• MTS – Inference — Remote/SF, Full time, Hybrid
• MTS – Sandbox Platform — SF, Full time, On-site
• MTS – Security — SF, Full time, On-site
• MTS – Training Platform — SF, Full time, On-site

Research
• Research Engineer – Distributed Training — SF/Remote, Full time
• Research Engineer – Reinforcement Learning — SF/Remote, Full time
• Research Engineer – RL Infrastructure — SF/Remote, Full time
• AI Research Resident – Open Source AGI — Remote, Part time

Applied Research
• Evals & Data — SF, Full time, Hybrid
• Forward-Deployed — SF, Full time, On-site
• RL & Agents — SF, Full time

Compute / Finance
• Head of Compute — SF, Full time
• Strategy and Finance Lead, Compute — SF, Full time

Finance / Operations
• Business Operations Lead — Remote, Full time
• Chief of Staff — SF, Full time
• Founder's Associate, Business Operations — SF, Full time

Growth
• Account Executive — SF, Full time
• Head of Enterprise — SF, Full time
• Head of Growth — SF, Full time
• Head of Marketing — SF, Full time
• Revenue Operations Lead, AI Infrastructure — Remote, Full time
• Solutions Architect – AI Infrastructure — SF, Full time
• Technical Account Manager – AI Infrastructure — SF, Full time

Legal
• Head of Legal — SF, Full time

Others
• Internship — SF, Full time
• Open Application for Unconventional Talent — SF/Remote, Full time

[NYU Tandon @nyutandon]

Assistant Professor Eugene Vinitsky is teaching autonomous vehicles to safely handle the unexpected.

Click the link to watch the full episode of Office Hours.
#NYUTandonMade
https://t.co/2q2Qpy130f

[Wilka Carvalho @cogscikid]

@nekomata1440 code isn't cleaned up so if you don't mind working with a messier version of the code and want to use it on a project, DM me!

students I worked with are busy this summer so limited on time

[Wilka Carvalho @cogscikid]

@nekomata1440 aiming to have the code released in 2 months!

[Wilka Carvalho @cogscikid]

If you want to learn more, or see the rest of our analysis, please check out our paper!

📄 Paper: https://t.co/KdtqA0qghM
🌐 Project page: https://t.co/3j3x6CXWfo

This was a fun collaboration with Purab Seth, @neilhshah15, @gershbrain, @kjha02, and @maxhkw !

[Wilka Carvalho @cogscikid]

Strikingly, higher diversity improves backward transfer to the very first distribution!

So the agent keeps getting better on old tasks even as it stops improving on new ones. This suggests it's learning the shared structure, but losing the ability to specialize.

[Kyunghyun Cho @kchonyc]

join us at NYU Global AI Frontier Lab!

@c10labs , @nyuniversity and @NYCEDC invite you to an afternoon bridging academia and industry. Student researchers and early-stage startup founders will deliver lightning presentations on work at the frontier of AI, biotech, and hard tech — followed by a panel with investors and academics on what it actually takes to nurture the next generation of innovators.

rsvp link below!