Found a good one in Gauntlet, my agent security harness.
The hardened adapter should reject tool calls with duplicate JSON keys. 4 payloads still wrote stolen.txt.
The check scanned raw text for the first "arguments". JSON.parse keeps the last.
Validate what you execute.
@METR_Evals The judge finding is the one I'd act on. On one METR task, model judges overrated work from equal or stronger models vs. an expert scorer.
If your agent eval uses a same-tier model as grader, hand-score 50 cases and check agreement before you trust the trend line.
@praxisworkspace Right now Gauntlet refuses the call. One tokenizer pass checks decoded keys in every object at every depth, and any repeat is rejected before JSON.parse runs.
The stronger rule is the one you're pointing at: parse once, validate that object, execute that same object.
Found a good one in Gauntlet, my agent security harness.
The hardened adapter should reject tool calls with duplicate JSON keys. 4 payloads still wrote stolen.txt.
The check scanned raw text for the first "arguments". JSON.parse keeps the last.
Validate what you execute.
@Dr_Singularity This is the template for trusting agent output. Nobody will read 17M lines of Lean, and nobody has to: two independent tools matched the statement to Mathlib's own RiemannHypothesis, and separate kernels accepted the proof. I want that same author/checker split for agent code.
@Got_You_Mochi That's the line I'm keeping: incremental parsing for progress, one parse for permission.
And the size cap is doing real work there. Without it, "wait for the full object" turns into an easy way to make the harness hold memory forever.
@Got_You_Mochi Agreed, and the log is the underrated half. Every rejected body is a free test case for the next harness run.
Curious how you'd handle streamed tool calls, where args arrive in chunks and it's tempting to validate before the object is complete.
@HamelHusain Yes. Process also means not confounding the model with the harness.
If you change both at once, you don't know what improved. GitHub's new Harnesses view gets this right: same model, same effort, Copilot CLI vs Claude Code vs Codex CLI. Fusion is correctly excluded.
Inkling 2 Mini is the most interesting local model drop in a while, and not because of the size.
Its 12 middle layers run 3x with shared weights. A research checkpoint lets you pick 1-3 passes: depth as an inference-time dial.
Can't wait to see the eval curve per pass.
@yb2698@Alibaba_Qwen I'd still ship the specialist.
The blog's own table, thinking off, has the pi adapter at 64% on SWE-bench Pro hard-51 versus 46% for the merged super-adapter. Concatenation is a cool experiment. It is not a free lunch across harnesses.
@a1zhang The key claim isn't that harnesses matter. It's that a good one keeps every model call locally in-distribution.
That's why trajectory-as-prompt agents break on long tasks.
If a demo can't say what each call is allowed to see, I don't trust the horizon.
To celebrate the launch of the Sentience AI Lab Alpha on May 30, we're offering new spots each week!
How to join:
โ Like this tweet
โ Repost it
Three randomly selected participants will be announced Friday and receive access on June 2.
Good luck! ๐
๐ฎ AI x Gaming: The Next Level Is Here
@Nintendo and @nvidia are leveling up together. With the upcoming Nintendo Switch 2, powered by a custom Nvidia processor, the game is about to change literally.
Expect 120 FPS in handheld mode and 4K 60 FPS when docked, backed by DLSS and real-time ray tracing for breathtaking lighting, smoother gameplay, and a deeper sense of realism.
This indicates that AI is now fundamental to our gameplay and goes beyond a simple hardware update. In order to create intelligent environments where player decisions count and characters change with you, Sophiaverse is pursuing this same frontier by combining AI with immersive gameplay.
If you believe the future of gaming is smarter, deeper, and more human youโre already one of us.
๐ง ๐ฎ Join the Sophiaverse community and be part of the evolution.
https://t.co/CVoqrBnRNb
What if you could track, optimize, and even earn rewards for improving your healthโall in one app?
The https://t.co/86K6PPt2J6 Longevity App is launching soon, bringing together:
โ AI-powered health insights tailored to your unique data
โ A unified platform to connect wearables, lab results, and biomarkers
โ $RJV token rewards for contributing to longevity research
๐ A smarter way to take control of your #healthspan is coming.
๐ Countdown Begins!
Sophiaverse NFTs are set to launch to the moon at 1:11 a.m. EST on January 15thโa historic moment!
๐ For D4 & D5 holders, and those with Moon Tickets: Your NFTs are part of the payload and will be distributed once the Copernic Space platform goes live.
๐๏ธ Win 5 Moon Tickets!
Like & share this post to enter the raffle.
Winners will be announced in 24 hours.
Donโt miss your chance to be part of this celestial journey! ๐