So I tested DeepSeek V4.1 Flash 1017 on real work. 2 repos. 105 hidden bugs. Find and fix what you can.
The first results:
Muse Wonder Full 1.0 (max): 42
Opus 5.5 (max): 41.7
Gemini 4 Argon (high): 40.7
DeepSeek V4.1 Flash 1017 (max): 34 🟣
Grok 4.7 (xhigh): 29
Qwen3.8-Max (max): 25.7
DeepSeek V4.1 Flash (max): 21.7 🟣
GLM-5.3 (max): 19
A great improvement. Strange that the model wasn't labeled V4.2.
763B params. The weights are already on Hugging Face.
I expect the score to change by ~1.7 points after repeating the test 3 times.
More data and results at different effort levels dropping in this thread very soon 🧵