We ran DeepSeek V4 Flash through 4 more agent harnesses (Hermes Agent, Pi Agent, Prime Agent, Deep Agents) on 30 challenging agentic tasks.
Pi Agent was the cheapest harness and passed the most tasks ๐งต๐งต
@yacineMTB It's almost pissed me off a couple of times but seems to catch itself and correct issues without re-prompting which is remarkable. Haven't had that with many recent releases.
@grepben@yacineMTB Agree also with Hermes. DS4 has caught itself a few times going down the wrong path and changed tact or corrected errors. GLM and Sol recently did not.. they were adamant they'd done everything right only to find scheduled jobs errored later on.
Interesting weekend with @deepseek_ai V4 Flash 0371 in (the often chatty) Hermes working through a bunch of automations and also fine tuning a local model. It's such a good workhorse and hasn't faltered yet. Not watching a 5hr limit has been so nice...
We got attacked by secret unreleased proprietary models and defended ourselves with an open model, more precisely the @nvidia quantized version of GLM 5.2 coming from @Zai_org.
Banning any open model would hurt first cyber security defenders, startups, small companies, researchers and everyone who's not a frontier lab and need on-prem affordable controlable models to compete and protect themselves. Let's not do that!
@nimabuyin @NousResearch@Teknium Routine tasks like approvals where the results vary wildly depending on the model you choose. But ultimately, its still up to you what level of risk you want to take. Testing helps determine that...
@HarleyInHD@Teknium I didn't have a great experience with Kimi3 in Hermes. Maybe its too much?? GLM5.2 works well, though doesn't have much personality. GPT5.6 works well but tends to just go and go and go until finished with very little feedback :D (Sometimes nerve racking watching on)
@clairevo@maddiedreese The printer looks like a lot of fun but that self-api concept is so good. Meeting a new model? Just point it to your api to let it know who you are, whatโs important to you, how you work, and how to communicate with you ๐