follistatin

25 | swe ml |
hobbies:
game dev,
bodybuilding,
powerlifting,
running,
biohacking,
a friend to ai |

840 SQ | 550 B | 655 DL | 440 OHP

[follistatin @follistatindev]

@Nayberryk @tszzl @tszzl
https://t.co/NDIzPOTifu

if you want access to a small subset of the benchmark for internal testing, my DMs are open.
would be great to be proven wrong / shown that the US has some hands too.

[follistatin @follistatindev]

@Nayberryk @tszzl which became very apparent by qwen4 preliminary performance

[follistatin @follistatindev]

@Nayberryk @tszzl he had some strong opinions/blogposts about the GameFeel benchmark results of Qwen4 today, which was the center of discussion

[Aesther @AestherML]

Imagine that you are suddenly teleported back to the 1930s along with Qwen4-27B and a 3090 gaming rig

What is the first thing you would build?

[follistatin @follistatindev]

@tszzl https://t.co/AzhqXfN1NJ

Qwen4 got hands huh?

[follistatin @follistatindev]

Qwen-4-Flash-Early (codename hawk) added to GameFeel Bench!

(and audio score was calculated wrong for qwen-4-27b, a genuine mistake that was fixed)

[follistatin @follistatindev]

@sudooapt @uzairakrum this is based on people misinterpreting my benchmark results, I got early access for codename hawk and duck:
https://t.co/miM4G6Hf2n

[follistatin @follistatindev]

@spnichol people are extrapolating way too much from my benchmark results which is what started this saying:

GameFeel Bench measured Game balancing abilities not general capabilities, and that's where qw4-27b was *almost* on level with opus-5.5

more details here :
https://t.co/1WQdBHagI2

[follistatin @follistatindev]

@MarcoGiacobbe15 @mfranz_on I did not make that assumption, I was given concrete confirmation about the api endpoint I was using to be a 27b-class model.

A better description of what the benchmark measures :
https://t.co/1WQdBH9ISu

[follistatin @follistatindev]

@Chaos2Cured One thing that kept the reputation of this benchmark low today:
People keep misinterpreting and misquoting these results as measuring general capabilities, when its a benchmark set specially built around game balancing and gamedev related creative thinking.

[follistatin @follistatindev]

'QoDuck' (qwen-4-27b early checkpoint) added after reaching out to alibaba and being granted access.

only beaten by Opus 5.5-Max and Xhigh on GameFeel bench, SoTA for open source by far!

Yes, all of this from a 27b model, I am shocked at these results.

[follistatin @follistatindev]

@spnichol people are extrapolating way too much from my benchmark results which is what started this saying:

GameFeel Bench measured Game balancing abilities not general capabilities, and that's where qw4-27b was *almost* on level with opus-5.5

more details here :
https://t.co/1WQdBHagI2

[follistatin @follistatindev]

@Crypto_Fro Size for the flash model is not yet known, it was not revealed to me.

[follistatin @follistatindev]

Qwen-4-Flash-Early (codename hawk) added to GameFeel Bench!

(and audio score was calculated wrong for qwen-4-27b, a genuine mistake that was fixed)

[follistatin @follistatindev]

@vincentack122 this about game balancing and game development check out the details and methodology used for this benchmark here:
https://t.co/NDIzPOTQ52

[follistatin @follistatindev]

@BranchGhost @MichaelLee009 @grok Hi! its my benchmark and it is NOT meant to represent general capabilities, it is measuring game balancing and game feel capabilities

Read more about what the benchmark actually measures here:
https://t.co/NDIzPOTQ52

[follistatin @follistatindev]

@duhmbahss that was a stealth Minimax M3.1-Flash early checkpoint under this name.

[follistatin @follistatindev]

@AdamHoltererer except qwen4
qwen4 happens 😆

[follistatin @follistatindev]

@TimothyZ77 yes essentially, not a coding focused benchmark, but a game balancing and creativity focused one.

[follistatin @follistatindev]

@EddiSpaghetto this is one large benchmark set "GameFeel Bench" with all of its subcategories

explained here :
https://t.co/1WQdBH9ISu

[follistatin @follistatindev]

@sebuzdugan only Qwen-4-27b needed correcting!
and yes this has caused some people to circulate incorrect results for Qwen-4-27b
as the lower audio score result brought down the average on overall from 47 to 44.6

[follistatin @follistatindev]

@selzer25 More information about the benchmark here:
https://t.co/1WQdBH9ISu

the benchmark mainly tests game balancing 'feel', not necessarily correlated with general capabilities.

[follistatin @follistatindev]

@gusgu More information about the gamefeel benchmark here:
https://t.co/1WQdBH9ISu