Wayne Chi

CS Ph.D. at @SCSatCMU. Funded by @NDSEG Fellowship. Editor at

[Wayne Chi @iamwaynechi]

New preprint alert 🚨
Can LLM agents develop video games?

We release GameDevBench, the first benchmark evaluating agentic game development in a game engine, Godot.

We also present two simple multimodal feedback mechanisms that lead to immediate performance gains.

/🧵

[Wayne Chi @iamwaynechi]

@a1zhang Haters gonna hate

[Wayne Chi @iamwaynechi]

I think hearing this, the best thing you can do for a PhD application is to work with a well established professor. NYU does have many. Granted your understandable worry over financials, it may be beneficial to apply for MS programs that offer funding!

I will add that there's a lot of noise to PhD applications so keep trying! Assuming your profs had good things to say, your app sounds like one that could make it past a first round.

[Wayne Chi @iamwaynechi]

@JangLawrenceK @kevinyli_ @NaveenJRaman Too meta

[Wayne Chi @iamwaynechi]

Alright I guess it's time to test Opus 4.8 on GameDevBench

[なかじ / 中島大介@ウェブ職TV @ds_nakajima]

Opus4.8すごいすね…………

[Wayne Chi @iamwaynechi]

@barrnanas @sarahcat21 Time to bring your grandma to the events. I would like to know her fav tinned fish

[Wayne Chi @iamwaynechi]

@barrnanas @sarahcat21 You're a tinned fish Barr! 🐟

[Wayne Chi @iamwaynechi]

From my experience doing both, this is the most accurate differentiator.

Most of the differences in methods and skills stem from this.

[François Fleuret @francoisfleuret]

IMO a researcher studies a problem that may not be solvable, while an engineer solves a problem that is considered solvable.

[Wayne Chi @iamwaynechi]

@_sholtodouglas Paper: https://t.co/Ptu7RDMrlY

[Wayne Chi @iamwaynechi]

@_sholtodouglas I've been evaluating agents for game development (see our GameDevBench work) and Claude is a noticeably worse than both Codex and Gemini at game development.

Going to update the benchmark soon and will keep you posted if you're interested.

[Thomas G. Dietterich @tdietterich]

Attention @arxiv authors: Our Code of Conduct states that by signing your name as an author of a paper, each author takes full responsibility for all its contents, irrespective of how the contents were generated. 1/

[Wayne Chi @iamwaynechi]

@bansalg_ Claude

[Wayne Chi @iamwaynechi]

@_Suresh2 @OpenAI @AnthropicAI @Google This isn't really a benchmark issue, but more of a methods problem. Also, in our benchmark agents are able to observe video of the scene so there is reasoning over multiple frames.

[Wayne Chi @iamwaynechi]

We observed this a month or two ago on GameDevBench!

Ever since GPT 5.4, @OpenAI took over as the best agent for game development. However @AnthropicAI was never in the lead; the best was actually @Google with Gemini (good at multimodal understanding).

Good to see further confirmation on what's SOTA for game development.

[Grace Li @grx_xce]

Fun fact, GPT 5.5 is very good at Game Dev

Game Dev is the notable category where @OpenAI consistently beats out @AnthropicAI's Claude models

Upon code inspection, our @Designarena team found that GPT 5.5's frontend verbosity plays in its favor for game dev - it consistently created games with the most functional features

Congrats to @OpenAI for establishing the new Game Dev frontier!

[Wayne Chi @iamwaynechi]

A big downside with the the new focus on ArXiv is you have to read (and eventually cite) some absolutely awful papers that would clearly never pass peer review...

[Wayne Chi @iamwaynechi]

I love how southern Jensen sounds when he says America.

'Murica!🇺🇸🇺🇸🇺🇸🦅🦅🦅

[Wayne Chi @iamwaynechi]

GameDevBench has been accepted into ICML 2026! See everyone in Seoul soon!

[Wayne Chi @iamwaynechi]

New preprint alert 🚨
Can LLM agents develop video games?

We release GameDevBench, the first benchmark evaluating agentic game development in a game engine, Godot.

We also present two simple multimodal feedback mechanisms that lead to immediate performance gains.

/🧵

[Wayne Chi @iamwaynechi]

Exciting work and really cool to see Moonlake reference GameDevBench as a precursor to their work!

The future of agentic game development is bright ☀️

[Moonlake @moonlake]

Introducing Moonlake's 3D Agent.

Our agent acts like a technical artist that can build and reconstruct articulated assets and large-scale editable scenes with hundreds of objects from a single image and can improve its generations continuously.

Learn more in the thread below.

[Wayne Chi @iamwaynechi]

@bansalg_ @MSFTResearch I cannot believe I didn't see this!!!!

[Wayne Chi @iamwaynechi]

And they're cutting the next presenter's questions too???

[Wayne Chi @iamwaynechi]

The presenters in front of me took 15 minutes instead of 10 minutes each. And then the conference organizer CUT MY QUESTIONS???

wtf @iclr_conf