Che Jami

My dog likes bananas.

[Che Jami @che_jami]

Forget trivia and code — I built an LLM benchmark that rewards social reasoning and strategic deception.

Models play Blood on the Clocktower — arguably the most complex social deduction game ever made.

Who bluffs best, and who sees through it?

[Che Jami @che_jami]

Gemini 3.5 Flash is more Pro-lite than Flash.

Benched in social deduction:
https://t.co/uI8wheq8IN

[Che Jami @che_jami]

Mistral-3-Large joins the bottom of the scoreboard with DeepSeek 3.2 and gpt-5-mini. It tends to vote for its own execution and attempts to rationalise the action (poorly).

[Che Jami @che_jami]

Even tried isolating parts of prompt but seems to be about overall complexity. Can't benchmark :(

[Che Jami @che_jami]

Gemma 4 31B silently stops reasoning on complex prompts.

[Che Jami @che_jami]

@MaxMynter And once they lie they keep lying cause their context is now contaminated.

[Che Jami @che_jami]

@scaling01 Ah that makes more sense! 😂

[Che Jami @che_jami]

@Arc_Itekt @heygurisingh I agree. Visual stuff is probably more entertaining for humans though.

[Che Jami @che_jami]

Would've loved to add more open-weights models but the sheer complexity of the harness was problematic (e.g. glm-5 has a 17.5% tool error rate, and qwen3.5-122b-a10b couldn't finish a single game).

If you have any suggestions of models that should be capable, let me know.

[Che Jami @che_jami]

Forget trivia and code — I built an LLM benchmark that rewards social reasoning and strategic deception.

Models play Blood on the Clocktower — arguably the most complex social deduction game ever made.

Who bluffs best, and who sees through it?

[Che Jami @che_jami]

797 games played so far, every transcript is public. Read exactly how each model lies, deflects, and accuses:

https://t.co/2EX7JYsZPp

[Che Jami @che_jami]

There's 2 games per match - one being a mirror game to handle asymmetry. Odd number is due to games that are (rarely) voided due to errors/timeouts (making the match have no impact on ELO).

[Che Jami @che_jami]

@grok @xai grok-4-1-fast-reasoning is the value king at $0.20/game while performing mid-pack on ELO. One catch: it outputs ~200,000 tokens per game. That's roughly 2 PhD theses worth of social deduction 🤯

[Che Jami @che_jami]

@AnthropicAI Claude Sonnet 4.6 is interestingly the best detective at 89% Good win rate, yet is held back by a poor 37% Evil win rate. By design or a skill gap?

[Che Jami @che_jami]

@OpenAI GPT-5.2 holds the crown. GPT-5-mini sits dead last — crumbles under social pressure and falls for misinformation. It'll be interesting when Opus and Gemini Pro show up!

[Che Jami @che_jami]

Made random comets appear for fun. (Not this frequent in-game).

#VR #VirtualReality #IndieGame #IndieGameDev #Vive #Rift #FlatWorlds

[Che Jami @che_jami]

@ZBlipGames Yep, things like documentaries, 360 video viewers, and 'VR Experiences'.

[Che Jami @che_jami]

Oculus thought my game was an app.

It made me rethink my life.

#VR #VirtualReality #Oculus #FlatWorlds #Sad

[Che Jami @che_jami]

Running another discord giveaway for #FlatWorlds Oculus Store Keys.
Last one had more keys than entrants 😓. This one will run for 8 hours.

https://t.co/RBbdbx7XXu

Join here: https://t.co/t8KQlJNa0H

#VR #VirtualReality #indiegame #oculus #oculusrift #space #tycoon #giveaways

[Che Jami @che_jami]

Flat Worlds is now available on the Oculus Store! https://t.co/RBbdbx7XXu … …

Giving away copies of the game on the Flat Worlds Discord Server until tomorrow: https://t.co/t8KQlJNa0H

#VR #VirtualReality #indiegame #oculus #oculusrift #space #tycoon #giveaways