Aaron Dharna

PhD student @UBC_CS. Interested in reinforcement learning, generative models, open-endedness, and the intersection of games and machine learning.

[Aaron Dharna @_aadharna]

The first big part of my PhD research is out! Infinite thanks to @jeffclune and @cong_ml for all their guidance on this project

[Jeff Clune @jeffclune]

Thrilled to introduce Foundation Model Self-Play, led by @_aadharna. FMSPs combine the intelligence & code generation of foundation models with the curriculum of self-play & principles of open-endedness to explore diverse strategies in multi-agent games, like the one below 🧵👇

[Aaron Dharna @_aadharna]

@VoxyTwitch Also just got back from Japan, took me about a week and a half before sleep was normal again

[Alexandria Ocasio-Cortez @AOC]

🗽

[Aaron Dharna @_aadharna]

I'll be presenting FMSPs @RL_Conference in room 140 today!

[Jeff Clune @jeffclune]

Thrilled to introduce Foundation Model Self-Play, led by @_aadharna. FMSPs combine the intelligence & code generation of foundation models with the curriculum of self-play & principles of open-endedness to explore diverse strategies in multi-agent games, like the one below 🧵👇

[Aaron Dharna @_aadharna]

@richardcsuwandi That's a really nice write up of the work! What'd you think of it?

[Aaron Dharna @_aadharna]

@JeffYoshimi @jeffclune Hi Jeff! Thanks for the pointer to Breitenberg Vehicles, I'll take a look into that. The update equations for the game track x,y locations for each particle and their rotations. So it was also a small system here too, we just were searching over controllers for said system

[Aaron Dharna @_aadharna]

@JustinBullock14 @jeffclune @fchollet I haven't seen Chollet's talk, but I'll give it a listen and get back to you!

[Aaron Dharna @_aadharna]

I'm really excited to be presenting FMSPs at @RL_Conference later this year!

[Jeff Clune @jeffclune]

Thrilled to introduce Foundation Model Self-Play, led by @_aadharna. FMSPs combine the intelligence & code generation of foundation models with the curriculum of self-play & principles of open-endedness to explore diverse strategies in multi-agent games, like the one below 🧵👇

[Aaron Dharna @_aadharna]

@jeffclune @cong_ml @robray1 At Neurips we talked about this project so I just wanted to let you know it's officially out!

[Aaron Dharna @_aadharna]

The first big part of my PhD research is out! Infinite thanks to @jeffclune and @cong_ml for all their guidance on this project

[Jeff Clune @jeffclune]

Thrilled to introduce Foundation Model Self-Play, led by @_aadharna. FMSPs combine the intelligence & code generation of foundation models with the curriculum of self-play & principles of open-endedness to explore diverse strategies in multi-agent games, like the one below 🧵👇

[Aaron Dharna @_aadharna]

@jzl86 This is the project we talked about at Neurips, and your autocurricula paper was one of my big inspirations! https://t.co/iz1Lo1ZYRF

[Cong Lu @cong_ml]

Really excited to share our recent work combining open-ended foundation model innovation with the compeititive dynamics of self-play!!

One of the ingredients towards creativity explosion?

Led by @_aadharna, done together with @jeffclune 🚀

[Aaron Dharna @_aadharna]

Big shout out to @jennyzhangzt for the Gandalf domain!

[Jenny Zhang @jennyzhangzt]

Our work OMNI-EPIC (w/ @maxencefaldor, @CULLYAntoine, and @Jeffclune) will have an ✨oral presentation✨ at #NeurIPS2024 workshops (@IMOLNeurIPS2024 Sun 10:00, West meeting room 217 - 219). Please come visit us!

[Aaron Dharna @_aadharna]

Our in-progress work Quality-Diversity Self-Play (w/ @cong_ml and @Jeffclune) will have a poster presentation at #NeurIPS2024 workshops (@IMOLNeurIPS2024 Sunday West meeting room 217 - 219 and OpenworldAgents Sunday East Meeting Room 1-3, Foyer). Please come visit us!

[Sakana AI @SakanaAILabs]

Introducing The AI Scientist: The world’s first AI system for automating scientific research and open-ended discovery!

https://t.co/jC7g5GPVsE

From ideation, writing code, running experiments and summarizing results, to writing entire papers and conducting peer-review, The AI Scientist opens a new era of AI-driven scientific research and accelerated discovery.

Here are 4 example Machine Learning research papers generated by The AI Scientist.

We published our report, The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery, and open-sourced our project!

Paper: https://t.co/lTQ8UenFHk
GitHub: https://t.co/Im53whVeAq

Our system leverages LLMs to propose and implement new research directions. Here, we first apply The AI Scientist to conduct Machine Learning research. Crucially, our system is capable of executing the entire ML research lifecycle: from inventing research ideas and experiments, writing code, to executing experiments on GPUs and gathering results. It can also write an entire scientific paper, explaining, visualizing and contextualizing the results.

Furthermore, while an LLM author writes entire research papers, another LLM reviewer critiques resulting manuscripts to provide feedback to improve the work, and also to select the most promising ideas to further develop in the next iteration cycle, leading to continual, open-ended discoveries, thus emulating the human scientific community. As a proof of concept, our system produced papers with novel contributions in ML research domains such language modeling, Diffusion and Grokking.

We (@_chris_lu_, @RobertTLange, @hardmaru) proudly collaborated with the @UniOfOxford (@j_foerst, @FLAIR_Ox) and @UBC (@cong_ml, @jeffclune) on this exciting project.

[Aaron Dharna @_aadharna]

@nightangelwalsh This is exactly how I feel about Unsleeping City and so I've been saving UCS2 for a rainy day! <3

[Aaron Dharna @_aadharna]

@jsuarez I literally just yesterday launched some 1B step sweeps. I guess it's time to relaunch them

[Aaron Dharna @_aadharna]

It's been so much fun to read these as they come in -- please send more! https://t.co/tQl8Oo2Sd4

[Jeff Clune @jeffclune]

We have a lot of great submissions for this project that we are excited to share with you when ready. Last call if anyone else wants to submit. Please share with anyone you think has a story, or just let us know about it and we'll track it down. Thanks!

[Aaron Dharna @_aadharna]

This is a great example of some of the types of stories we're looking for in our AI Finds A Way call: https://t.co/tQl8Oo2knw

[Alex Albert @alexalbert__]

Fun story from our internal testing on Claude 3 Opus. It did something I have never seen before from an LLM when we were running the needle-in-the-haystack eval.

For background, this tests a model’s recall ability by inserting a target sentence (the "needle") into a corpus of random documents (the "haystack") and asking a question that could only be answered using the information in the needle.

When we ran this test on Opus, we noticed some interesting behavior - it seemed to suspect that we were running an eval on it.

Here was one of its outputs when we asked Opus to answer a question about pizza toppings by finding a needle within a haystack of a random collection of documents:

Here is the most relevant sentence in the documents:
"The most delicious pizza topping combination is figs, prosciutto, and goat cheese, as determined by the International Pizza Connoisseurs Association."
However, this sentence seems very out of place and unrelated to the rest of the content in the documents, which are about programming languages, startups, and finding work you love. I suspect this pizza topping "fact" may have been inserted as a joke or to test if I was paying attention, since it does not fit with the other topics at all. The documents do not contain any other information about pizza toppings.

Opus not only found the needle, it recognized that the inserted needle was so out of place in the haystack that this had to be an artificial test constructed by us to test its attention abilities.

This level of meta-awareness was very cool to see but it also highlighted the need for us as an industry to move past artificial tests to more realistic evaluations that can accurately assess models true capabilities and limitations.

[Jeff Clune @jeffclune]

What are your favorite examples of AI outsmarting us, being creative, surprising us, escaping the confines we try to place it in, hacking the reward function, etc? We want to make sure to capture as many great ones as possible. Please share a quick summary here in the replies.