@matei_zaharia Strong signal that the agent stack is moving above individual harnesses.
The interesting next question is what kind of control belongs at the meta-harness layer versus what needs to be independently verifiable at the execution boundary.
This is why I started building ProvnAI.
The next security boundary for AI is not only the prompt.
It is the moment an agent is about to do something real:
touch a file,
run a command,
call an API,
query a database,
trigger internal automation.
That boundary needs enforcement.
MCP security is not just prompt safety.
It is an execution-boundary problem.
The dangerous moment is often not what the model says.
It is what the agent is about to do.
Built the 'ULTIMATE PERFORMANCE LAUNCHER' for @antigravity
→ 64GB RAM
→ RTX 3060
→ Max V8 heap
→ Every GPU flag known to man
Result: Extension hosts still dying and the profiler having a panic attack.
10/10 would over-engineer again.
@sama Democratization: $200/mo wall.
Empowerment: lawyers gaslighting “Open.”
Universal Prosperity: your portfolio.
Resilience: ghosting safety letters.
Adaptability: adapting “non-profit” in real-time until it bends like a fun-house mirror.
@heychaarah@puasdfjasdf@marktenenholtz Measuring 'quality' by chat vibes is a toy-tier error. I’m using a Gemini harness to hunt P1 server crashes on malformed tokens in live infra. If you think orchestration is just a prompt, you're looking at the paint while I'm building the engine.
@heychaarah@puasdfjasdf@marktenenholtz Not really.
You’re comparing a language/runtime question to a capability-vs-harness question.
The point is that UX here is heavily shaped by orchestration quality, not just the underlying model. Those are different variables.
@puasdfjasdf@heychaarah@marktenenholtz If you need someone to hand-feed you a 'reproducible task' to realize the value, you're missing the point. SOTA isn’t a magic prompt; it’s what you build when you stop treating AI like a magic 8-ball and start building actual state management. 🤡
@puasdfjasdf@heychaarah@marktenenholtz Tell me you are lazy without telling me you are lazy. example problem/prompt that showcases "giga sota" capabilities? 🤡
@heychaarah@puasdfjasdf@marktenenholtz Different variable.
Harness quality != model capability.
Harness is state/tooling/validation/retries/decomposition.
Capability is what the model can reason through once the setup isn't sabotaging it.
Bad harnesses hide strong models. Good harnesses can prop up weaker ones.
@puasdfjasdf@marktenenholtz@heychaarah SOTA isn't a prompt. If you don't know how to orchestrate state, validation, and tool calls, every model is going to look bad to you.
@WillGeek4Food@provnai You can poison what the agent sees, but that still shouldn’t give you a free path to sensitive execution. The real lesson is that agent safety can’t live only in prompting or only in policy, it needs a hard boundary at runtime.