Braidpoint

Deep Tech Solutions. Hardware. Software. #AI, classical, #quantum, Designer of Fidelity Framework (formerly #cleflang

[Braidpoint @BraidpointTech]

https://t.co/bBWJxkzHAf

[Braidpoint @BraidpointTech]

"Close to 60% proved something other than what the author meant. Not wrong proofs. Wrong specs."

[Alex Lavaee @alexlavaee]

We spent fifty years making proofs impossible to fake. Then we let the coding agent pick what to prove.

A research paper this month put a number on it.* AI wrote the formal spec, AI wrote the proof, the verifier accepted both. Then humans read the specs. Close to 60% proved something other than what the author meant. Not wrong proofs. Wrong specs.

The verifier was right every time. That's the problem. Left to choose, the agent picks properties that are easy to prove and say nothing about the program, and the checker can't tell a useless theorem from a useful one. I'm calling it vibe verification: the agent writes the spec and the proof, the checker goes green, nobody reads the spec. This applies to Verus, Lean, Bend, TLA+, Dafny, [insert favorite formal language here].

Here's an example of an agent failing at something that feels too simple to fail:

Ask for a verified sort. It writes a spec that says "the output is sorted" and proves it. What the spec never says is that the output has the same elements as the input, so a function that returns an empty list satisfies it, and the verifier accepts the proof. Trivially satisfiable specs are the default failure, not an edge case.

Here's a formula that actually works. Don't say "formally verify the codebase." Say "show that [property] holds in [component]." A property is a precondition (true going in), a postcondition (true coming out), or an invariant (stays true the whole time).

For example, "A session token is never accepted after the logout that revoked it" is a good invariant.

The bottleneck is moving from code review to intent review, and the spec is where intent lives. So: read the spec before the code. Write 3 inputs it must accept and 3 it must reject. Treat a model-written spec with utmost paranoia and rely on your judgement as an engineer.

Formal verification for software doesn't automatically create trust, it moves it, from the code to the spec. If a human didn't read the spec, you didn't verify anything. You vibe-verified it.

Follow me for more on verification and agentic engineering.

* Wang, Jiang, Liu et al., "VerusEval: Evaluating LLM-Generated Verus Specifications Beyond Verifier Acceptance", Shenzhen University, arXiv 2610.26464, October 2026

[Braidpoint @BraidpointTech]

@aramh @_chenson__ @JustDeezGuy In regulated work, "usable" has a concrete meaning: which is substantiation that an auditor will accept. We traced that lineage from Grenoble through Rocq, CompCert and F*: https://t.co/7YuQd37kFd

[Braidpoint @BraidpointTech]

@aramh @_chenson__ @JustDeezGuy Lean can learn from the more mature provers and catch up faster for it. Claiming parity today would be premature. That Lean's advocates are showing interest in fields other than pure mathematics is encouraging.

[Tommy Siegel @TommySiegel]

I swear to god if I hear another person on this app talk about the mainstream media while mainlining the disinfo slop on here all day, I’m going to lose my mind

[Braidpoint @BraidpointTech]

@kirancodes What about CVC5/Rocq with Iris/Actris/Aneris making appearances? https://t.co/MIYvUFh86W

[Braidpoint @BraidpointTech]

@jdlichtman @mathgeek234 Thanks, Captain Obvious...

[Braidpoint @BraidpointTech]

@TimSweeneyEpic @tritlo I depends on what you mean by "program". Syntax is fungible. How about a structure that starts with Wadler's "types are proofs" and node level obligations fall out of the graph 'for free'? https://t.co/UDtNiIEZPm

[Doug Colkitt @0xdoug]

1/ Agent-driven formal verification isn't a panacea. But it's damn close. And now we have the empirical data to back it up.

Using blinded agents, we compared formal verification against traditional code review. The results may shock you.

[Braidpoint @BraidpointTech]

@MikePFrank Hate to say it - but the math was always there... https://t.co/wbJEcDOS9z

[Braidpoint @BraidpointTech]

@hillelogram @mdabdurrahim98 @AnthropicAI @theoremlabs https://t.co/UDtNiIEZPm

[Braidpoint @BraidpointTech]

@0xdoug https://t.co/UDtNiIEZPm

[Doug Colkitt @0xdoug]

At this point, if you’re in software and aren’t currently working on a plan for moving to formal verification, it’s irresponsible.

The writing is on the wall. This needs to be your priority now.

[Gary Marcus @GaryMarcus]

i absolutely told you so. january 2025, warning that we would soon see the largest cyberattacks in history, sparked by generative AI and lax enforcement.

don’t ever lie and say i am always wrong, again.

seriously.

we are facing exactly the epic cybersecurity meltdown i warned about.

[Nikita Bier @nikitabier]

There is nothing to worry about.

Nothing important is run on PHP or old software—except the water & electrical grid, a few government services like our missile defense systems and a handful of electrified fences at the zoo (for the lions, tigers and cobras).

[Braidpoint @BraidpointTech]

@TaliaRinger https://t.co/UDtNiIFxEU

[Braidpoint @BraidpointTech]

https://t.co/UDtNiIEZPm

[Talia Ringer 🕊🪬 @TaliaRinger]

Where we started (left, 1979) vs. where we are now (right, 2026)

[Braidpoint @BraidpointTech]

😼

[Matthew Green @matthew_d_green]

Everyone wants to draw conclusions about the future based on what they saw in the past. Usually this works really well for people, so why stop now?

[Braidpoint @BraidpointTech]

Don't sleep on @LarkinPoe - if you're in a town where they are playing. See them. Hear them. Seriously. 🔊 🎵

[Braidpoint @BraidpointTech]

@QualiaQuanta "This has worked with anyone I've ever given this look to... why try harder?"