Shu Ishida, Ph.D.

Senior AI Research Scientist at Autodesk. Ex-Microsoft, Wayve, VGG (University of Oxford). Research: Computer Vision, Deep Reinforcement Learning and GenAI.

[Shu Ishida, Ph.D. @shuishidaOx]

Delighted to share my first team paper since I started at Autodesk!😊 Thank you @evan__casey for leading the effort!!

[evan @evan__casey]

New #ICCV2025 paper: ✨ Aligning Constraint Generation with Design Intent in Parametric CAD ⚙️

We apply post-training techniques to the task of generating engineering sketch constraints found in parametric CAD, using a constraint solver for verifiable rewards.

[Shu Ishida, Ph.D. @shuishidaOx]

🏆We are delighted to receive a ✨
Best Paper Awards Honourable Mention✨ at #FAccT2025 for our paper: “The World Wide recipe”, where we curated a dataset of food around the world named “World Wide Dishes” to evaluate representation biases in AI models.

https://t.co/fIG3aKZZGw

[ACM FAccT @FAccTConference]

🏆 Announcing the #FAccT2025 best paper awards! 🏆

Congratulations to all the authors of the three best papers and three honorable mention papers.

Be sure to check out their presentations at the conference next week!

https://t.co/GR6fmknntX

[Shu Ishida, Ph.D. @shuishidaOx]

The work is titled "Adapting a World Model for Trajectory Following in a 3D Game". We looked into the capabilities of transformer models to reproduce human gameplay for an Xbox 3D video game called Bleeding Edge.

Paper: https://t.co/NAJ4hsKmEo

[Shu Ishida, Ph.D. @shuishidaOx]

I am excited to share that I will be attending #ICLR2025 (International Conference on Learning Representations), where I will be presenting my internship project at Microsoft Research which I took up May - September 2024.

[AK @_akhaliq]

This is HUGE, microsoft just dropped MUSE

a Generative AI Model for Gameplay

[Satya Nadella @satyanadella]

If you thought AI-generated text, images, and video were cool, just imagine entire interactive environments like games!

[Katja Hofmann @katjahofmann]

Today in Nature: our research on world and human action models (WHAM) - generative ai models of video games, aimed towards supporting game creatives in gameplay ideation : https://t.co/zeaOGbfVqh - huge congrats to everyone who made this happen, I couldn't be more proud 🥳

[Shu Ishida, Ph.D. @shuishidaOx]

Super excited to share that the paper “World and Human Action Models towards gameplay ideation” is now published in Nature!🌐🎮

Weights released on @huggingface (200M and 1.6B)

Paper: https://t.co/GMr1iRdp4C
Model: https://t.co/vpeTQjV2Db

A massive thank you to the team!🙏🥰

[Microsoft Research @MSFTResearch]

Nature published Microsoft research detailing our WHAM, an AI model that generates video game visuals & controller actions. We're releasing the model weights, sample data & WHAM Demonstrator on Azure AI Foundry to enable researchers to build on this work. https://t.co/R3Pt8iwPBH

[Shu Ishida, Ph.D. @shuishidaOx]

Started my full-time job at Autodesk as an AI Research Scientist today! I am excited for this opportunity since I enjoy both Research & Development in the machine learning space as well as crafting, physics and dynamic objects!😊

[Shu Ishida, Ph.D. @shuishidaOx]

@HirokatuKataoka @Oxford_VGG VGGにいらっしゃるんですね！今年卒業したのですがラボのソーシャルなどにまた顔を出すこともあると思うのでその際はどうぞよろしくお願いいたします！

[Shu Ishida, Ph.D. @shuishidaOx]

My blog post for LangProp (internship work at @wayve_ai) is now live on the @Oxford_VGG blog!

https://t.co/ghdwHZGoeU

LangProp: A code optimization framework using Large Language Models applied to driving

Paper: https://t.co/tzqaU50SLs
GitHub: https://t.co/D18tyb9WKA

[Shu Ishida, Ph.D. @shuishidaOx]

@y_m_asano @wayve_ai @Oxford_VGG Ah thank you for pointing it out! Will amend it😅

[Towards Data Science @TDataScience]

"We realised, however, that we don’t necessarily have to use LLMs for inferring driving actions. What we could do instead is to make LLMs write the code for driving itself."

@alacremeox unpacks his new research on LLM-generated code for self-driving cars. https://t.co/iW65bkLUwR

[Towards Data Science @TDataScience]

As @alacremeox shows in his recent research, LLMs might not be able to drive cars anytime soon, but they may already be positioned to generate code for self-driving cars. https://t.co/iW65bkLUwR

[Shu Ishida, Ph.D. @shuishidaOx]

Wrote a blog post about the LangProp paper I worked on as a part of my internship at @wayve_ai last year, which I recently presented at #ICLR LLM Agents workshop.

https://t.co/A6jBPkxlmO

[Shu Ishida, Ph.D. @shuishidaOx]

@maxhbain @wayve_ai Thanks so much Max!!

[Shu Ishida, Ph.D. @shuishidaOx]

I am presenting my paper "LangProp: A code optimization framework using Large Language Models applied to driving" at #ICLR2024 Workshop on LLM Agents.

Paper: https://t.co/tzqaU50SLs
GitHub: https://t.co/YbMZF0B2x1

This work was conducted during my internship at @wayve_ai.

[Shu Ishida, Ph.D. @shuishidaOx]

I would like to thank my collaborators @hu_anth , @gianlucacorrado, George Fedoseev,@hudsonyeoce, Lloyd Russell, João F. Henriques, and @Jamie_Shotton. My internship at @wayve_ai was a wonderful experience, and it was amazing to witness the rapid technological advances up close!

[Shu Ishida, Ph.D. @shuishidaOx]

LangProp is a code optimization framework that iteratively improves code. Variations of code are generated from a docstring, which are then evaluated on a dataset and reranked by performance. The high performing candidates are fed to the LLM with an update prompt.