Wayne Chi

CS Ph.D. at @SCSatCMU. I work on evals, agents, game dev, and CUA. Funded by @NDSEG Fellowship. Intern @arena.

[Wayne Chi @iamwaynechi]

New preprint alert 🚨
Can LLM agents develop video games?

We release GameDevBench, the first benchmark evaluating agentic game development in a game engine, Godot.

We also present two simple multimodal feedback mechanisms that lead to immediate performance gains.

/🧵

[Wayne Chi @iamwaynechi]

@barsisyphus @ayanobacter It's ok. We have established that you finished your itinerary so mathematically you can't be the worst

[Wayne Chi @iamwaynechi]

@sethkarten damn where'd you find such a gorgeous picture of me

[Wayne Chi @iamwaynechi]

It feels like game development becoming a benchmark of frontier capabilities was inevitable after all :)

[Sholto Douglas @_sholtodouglas]

An arena FPS with portals - Fable 5.2 is the first model that passes my bar for seriously fun and beautiful games.
We have had a few fun evenings playtesting this against eachother at the office

[Wayne Chi @iamwaynechi]

@18jeffreyma Big foresight of meeting with Jeff Ma. Bought stock low now stocks high.

[Wayne Chi @iamwaynechi]

@norpadon @hikettei There doesn't seem to be anything wrong with this review. They told the AC they are not an expert, which likely means you got an extra reviewer. They didn't even give you a bad score!

[Wayne Chi @iamwaynechi]

Impressive work from impressive people! Amazing to see the culmination of all their hard work!

[Yuda Song @yus167]

Foundation models are trained in stages: pretraining, midtraining, and post-training. But is this separation fundamental?
We introduce unified training: a single algorithm for the full pipeline. This gives
⚙️ Simpler infrastructure
🔬 A framework for better science
(1/n)

[Wayne Chi @iamwaynechi]

@JangLawrenceK @PranjalAggarw16 ???

Tfti

[Wayne Chi @iamwaynechi]

Interactive artifacts -> games feels like the right path forward to me. I will be releasing research related to this soon 😊

[Lovis Odin @OdinLovis]

Video2Game

Claude can't design very original little game on his own. So I let H3 Max on @fal generate the gameplay video first, and Claude turns it into a real game you can play in the browser: every image, 3D model and sound generated on fal, written in @threejs only.

[Wayne Chi @iamwaynechi]

@sethkarten Need multi agents to take my flights for me

[Wayne Chi @iamwaynechi]

@haiyewon

[Wayne Chi @iamwaynechi]

revisiting my graphics and finding this in my drafts

[Wayne Chi @iamwaynechi]

@noemaclips 😉

[Wayne Chi @iamwaynechi]

Two models (Opus 5.5 and Sol 6.1) have finally scored over 70.0% on GameDevBench!

It's really amazing to see how quickly models have improved on game development. At the same time it's so clear that there's still so much more to improve on!

[Wayne Chi @iamwaynechi]

@18jeffreyma Do you think Claude will also become addicted to refreshing for new graphs

[Wayne Chi @iamwaynechi]

@sethkarten Dont you know. P(doom) gets likes. Optimism gets downvotes.

[Wayne Chi @iamwaynechi]

I am at COLM this week! Please lmk if you want to chat about games, agents, and evals!

[Wayne Chi @iamwaynechi]

Evolving games are the future. Come try this out!

[Chris Donahue @chrisdonahuey]

Last weekend I had an idea: a multiplayer world where you build anything just by describing it

I invited a few friends to play, and we lost hours to it

This weekend, you and your friends can get lost in Allcraft too!

[Wayne Chi @iamwaynechi]

@ssokota Sam you're my idol. So cool!

[Wayne Chi @iamwaynechi]

@tmlabonte @MicrosoftAI Congrats!!!

[Wayne Chi @iamwaynechi]

For better or worse, "I developed a new Claude skill" is the new "I developed an app"