Antonios Liapis

Researcher in AI-assisted game design, computational game creativity and procedural content generation. Also gamemaster and/or storyteller. He/Him

[IEEE Transactions on Games @IEEETxnOnGames]

We are excited to announce the open Call for Papers for the "Special Issue on Large Language Models and Games."
We invite submissions from a wide audience on topics that explore the intersection of LLMs and games, including but not limited to:

[Georgios N. Yannakakis @yannakakis]

Super happy to announce our Behavior Alignment of Video Game Encodings (BehAVE) @eccvconf paper just won the @nvidia best paper award! BehAVE aligns videos of similar player behaviors, thereby improving their transferability across games.

Paper: https://t.co/cwkgJDy0Tx

[Antonios Liapis @SentientDesigns]

And that's it from Glasgow! It was a fun conference, with a lively community & interesting presentations. While games are not common at ACII, still, there were interesting applications in the performing arts!

Thank you to all collaborators at @InDigitalGames for their hard work!

[Antonios Liapis @SentientDesigns]

By rewarding the generator when it matches an "ideal" arousal curve, the generator learns what to place in different parts of the level. A level that is constantly non-arousing is boring, but so is one constantly arousing. Pacing can be a reward.

Paper: https://t.co/LmeuhTDyxM

[Antonios Liapis @SentientDesigns]

The generator puts down one track segment at a time, and repairs the track after some actions (making a loop). Then, a simple AI agent plays the track: based on the AI game-states, "arousal" is measured in every part of the track from human arousal labels in similar game-states.

[Antonios Liapis @SentientDesigns]

Finally, Matthew Barthet closed the conference on the last day by presenting how we can generate racing tracks that would increase a player's arousal (or follow a specific arousal progression). Again, a corpus of annotated human gameplay sessions was used to "assess" arousal.

[Antonios Liapis @SentientDesigns]

We modeled engagement in an ordinal fashion, trying to predict if engagement would increase or decrease in the next time window of this video. We used pre-trained computer vision and audio models and fused them, finally training an SVM on the annotators' consensus on engagement.

[Antonios Liapis @SentientDesigns]

We tried to predict engagement in an unseen video of the same game as the game we trained the model on, annotated by unseen participants. Results were mixed: some games were easier to predict than others. Battle Royale games were especially hard.

Paper: https://t.co/wpTqjmyBN4

[Antonios Liapis @SentientDesigns]

At the main conference, Kosmas Pinitas presented his work on predicting viewers' engagement from gameplay videos and audio alone (without face cams or sensors). To do this, 20 participants annotated their engagement when watching 2 hours of First Person Shooter gameplay videos.

[Antonios Liapis @SentientDesigns]

As expected, maximizing arousal (predicted or not) didn't work well. When combined with in-game rewards it 𝘴𝘰𝘮𝘦𝘵𝘪𝘮𝘦𝘴 did 𝘰𝘬. Pushing the agent to explore more states (not get stuck in one "arousing" game-state) is an important next step.

Paper: https://t.co/DmQhlxkGrW

[Antonios Liapis @SentientDesigns]

We can use this predicted emotion (based on humans' reported arousal at similar game-states) as a reward for a reinforcement learning agent. While this doesn't work alone, we can combine it with an in-game reward (such as going quickly in the right direction for a racing game).

[Antonios Liapis @SentientDesigns]

The 3 games feature short game sessions (max 2 minutes) but have over 120 sessions from different players who annotated their arousal levels while playing (on a recording of their play). We match game-state and arousal to find how an AI agent would "feel" in a similar game-state.

[Antonios Liapis @SentientDesigns]

In the same workshop, Matthew Barthet presented the Affectively framework, a Gym environment for making game-playing agents in 3 games. The unique selling point is that Affectively includes approximations of "agent" emotion based on a human corpus of annotated game sessions.

[Antonios Liapis @SentientDesigns]

There's a few caveats: we don't know the "ground truth" of emotions and rely only on how they are expressed (such as loud screams). Streamers also over-emote. Finally, we used pre-trained text/audio/facecam affect models not tailored to this task.

Paper: https://t.co/sC9gDp4Q8z

[Antonios Liapis @SentientDesigns]

In our experiment we looked at whether affect manifestation levels changed when entering a new room with different design features. In all our experiments, cutscenes and scripted events were by far the best predictors of increased fear, surprise and arousal. Makes sense!

[Antonios Liapis @SentientDesigns]

Level design features were manually annotated by observing all the videos and the traversal of the 1st level (Asylum) of Outlast. We logged when the streamer entered a room, and the design of the room: light levels, elements present (e.g. batteries), or if cutscenes happened.

[Antonios Liapis @SentientDesigns]

First off, at the Dungeons, Neurons, and Dialogues workshop, we presented "The Scream Stream: Multimodal Affect Analysis of Horror Game Spaces" where we looked at Let's Play videos of the Outlast horror game and the impact of level design choices on the streamers' emotions.

[Antonios Liapis @SentientDesigns]

We measured emotions directly from the streamers' affect manifestations (facial expressions, voice levels, utterances) and passed each through pre-trained AI models of affect to find fear, surprise or arousal levels. For this study we assume that such models are robust enough.

[Antonios Liapis @SentientDesigns]

Last week I was in Glasgow for the International Conference on Affective Computing & Intelligent Interaction @acii_conf and since we had quite a few things there, let me unfurl them in a thread.

[Antonios Liapis @SentientDesigns]

We're iterating on our CrawLLM game generator, and we'd like to know how the generated visuals and generated text is consistent with an overall theme. Fill in our 15-minute (anonymous) questionnaire (very light, with plenty of images) here!
https://t.co/nMPeZl0BZZ

[Marvin Zammit @nittwittja]

Hello world, would you help out with my research? Please fill in my survey here: https://t.co/Z8doas4EHI It's a 'fun' quiz about AI-generated images and text. Participation is optional and completely anonymous. The survey takes between 10 and 15 min to complete. Thank you!!