Gradient Sky

I know that I know nothing. Not an investment advice.

[Gradient Sky @gradient_sky]

@poteto @shaoruu Are they focusing on different aspects of the task? Why would you need multiple cooks at the same station otherwise?

[Gradient Sky @gradient_sky]

Bots don’t care: they like fish in a water in multi-repo setup. Just create one command workspace with all your tools, skills, docs, no code. And build an index of the packages and what they do (self-maintained by the bots)

Starting new features? Bots will crate a workspace, checkout what they need, do the work, then send multi-package PR.

[Gradient Sky @gradient_sky]

With all the respect.., this is where people need to understand the open source:

It’s not your cool shit anymore, it’s everyone’s cool shit now. From now on, only your reputation and judgement would decide if the community trust is placed in your hands or not.

Fork is always on the table: look at how redis thought it’s their cool shit and the community quickly showed them it’s not.

[Hillel @hillelogram]

It is a weird weird year to be a formal methods expert.

Some unorganized thoughts on the new @AnthropicAI and @theoremlabs post on formal verification (https://t.co/YoymKXYFJa):

- This is very cool.

- The main innovation is verifying a later stage of the program. Normally provers take the source code, LLM it into a theorem language, and then prove that, which means you have to make assumptions about source semantics and compilation properties. It's very easy to make a mistake in translating and prove the wrong thing!

What they instead do is compile the code into assembly, and then (if I understand this correctly), compile that assembly into a theorem language (which they call LPS). So as long as the compiler is faithful you can skip the whole divergence problem.

- I'm glad they clarify that you still have to trust the translation process, the "hardware, OS, OCaml" compiler, the LPS model, and also assume that the program is single threaded with fixed memory. The single biggest source of errors in formally verified code is when the proof assumes something that isn't true in practice. It's really, really important to state your assumptions!

- I'm not clear why they need to compile the fuzzer into the theorem. Is it to act as a description of valid inputs and workload? Maybe to embed assertions in as theorems?

- Holy crap, 180 million lines of Rocq. Is that more Rocq than exists in the world already? I notice they say a 600,000 line proof corresponds to roughly 20k CPU instructions. That would be like, 1-2k lines of source code at most? Really goes to show how inefficient this approach is and how it only works because you can throw LLMs at the problem.

- This, probably deliberately, doesn't address the single biggest gap in AI-based verification: AIs cannot come up with good properties. Doesn't matter if you're doing unit testing, property testing, formal verification. I always assumed this was a "frontier labs haven't trained for it yet" thing, but if Anthropic hasn't talked about that here, maybe they've tried and haven't been successful yet. That's interesting to me.

Instead, they take a known universal property (this code doesn't crash or get flagged by UBsan) and prove that instead. Still very useful, but makes it harder to generalize to proving arbitrary things about arbitrary programs.

- Anybody reading the official post should also read their expert review appendix (https://t.co/qQU0E0nJwW). It points out some interesting things, like the fact that most of the code the LLM wrote wasn't used in the proof (over one billion lines of Rocq), that they had to prove the equivalence between fuzzed and unfuzzed programs, and (this is most interesting), some experiments find that the LLM-method of proofwriting may not scale to more complex things.

- Also read the description of LPS (https://t.co/hRTCL05X0H) to understand the limitations of the assembly -> Rocq compilation. In particular, it does not handle concurrency at all (which is one of the main use cases for formal methods!) or OS signals. Also it can't prove things about program arguments with slash characters, meaning they can prove stuff about `rm foo` but not `rm foo/bar`.

- The official skill (https://t.co/weDOdmJUfO) doesn't tell us much about the nature of FM and AI. Probably too much for me to hope of any skill.

- Some of the article's specific caveats and throat clearings make me think someone at Anthropic reads my stuff ( ͡° ͜ʖ ͡°)

[Gradient Sky @gradient_sky]

@elder_plinius RIP christmas. #hugOps for your local CISO

[Gradient Sky @gradient_sky]

@steveruizok @badlogicgames Sign-in with hide my email doesn’t send the code

[Gradient Sky @gradient_sky]

@foundmyfitness @paulg I’m sorry, but looking at the graphs: this is garbage science. What’s the r^2 of that fitted regression?

[Gradient Sky @gradient_sky]

@GeoffreyHuntley I think monorepo is not a necessity. Agents feel very comfortable working in 50+ repo setups, just need some mapping where things are. Self-improvement from interactions allow to learn that mapping as we go.

[lauren @poteto]

there's something really cool about asking your agent to draw a statechart (hypothetical example in this image), then using the /create-formal-verification-skill you made with pstack to find bugs through something like TLA+. you can both understand your code better and how the user flow should work, and fix issues users would otherwise see

[Gradient Sky @gradient_sky]

@threepointone Cold Start, warmer threads…
A tailor could help Sunil
Look his finest yet.

[Gradient Sky @gradient_sky]

@GeoffreyHuntley “Suits” had a similar scene. It was saved by a paralegal with superhuman attention. Now a swarm of LLMs + OCR machine can wreak havoc on that attempt to DDOS.

[Gradient Sky @gradient_sky]

@GeoffreyHuntley Is EU-specific watermarking going to cause some noise?

https://t.co/cYW7YlIOXw

[Gradient Sky @gradient_sky]

@almonk @gm_mertd @superlogical Might not be legal in all jurisdictions ™

[Gradient Sky @gradient_sky]

@jullerino @theo @krika2810 Do you run post-transcribe polish run with some LLM to do cleanup?

[Gradient Sky @gradient_sky]

@maria_rcks @theo Maria, is it possible to convince him he’s wrong on this one? It’s not for us, it’s for kids who learn math, engineers who have to deal with math.

[Gradient Sky @gradient_sky]

@MohdAlzo @minchoi @ns123abc Commits are not traces. Commits is finished piece. Trace is what you did, what agent did, reasoning traces, verification.

[Gradient Sky @gradient_sky]

@threepointone Mimetic cryptography will be a thing. Designing tools and algorithms that AIs can’t see.

[Gradient Sky @gradient_sky]

@threepointone

[Gradient Sky @gradient_sky]

@nwp @thorstenball In the last 3 months multiple engineers in my team delivered the largest refactorings of their careers with 3-5x speedup vs what it would take in 2024. Amount of pain we paid was nothing compared to what my guts expected (you always pay in stabilization effort). Opus is better.

[Gradient Sky @gradient_sky]

I think there must be a fitness function: how fast can we move so our company/product doesn’t die now vs 3 years from now. Are we even going to survive if we do everything by the book now. Healthy startups always have this live or die edge.

And here we have a split of outcomes: one group is using craft to balance on that delicate edge: do we move fast, break things vs do we do it right so we don’t die from technical debt later. This is a healthy, outcome-driven skill.

The other group just practicing craft for the craft’s sake… people’s identity is fused with craft.

We live in different realities and some realities might not be aligned with the mission.