Denis Redozubov

Ex-CTO turned solo builder | Figuring out the indie path in public | AI, directories, whatever works |

[Denis Redozubov @codeofchange]

This is a bad trend for defensive cybersecurity work, if so. There would be cyber violence either way.

[Kyle Chan @kyleichan]

Moonshot is taking a very interesting approach to cyber for its new Kimi 3.1 Pro model.

Instead of adding safety guardrails during post-training, they removed cyber-related data in mid- and post-training. This tries to address the problem of people fine-tuning away the safety guardrails for cyber for the open-weight model.

At the same time, Moonshot still has a more powerful internal version of the model with full cyber capabilities. Now, like the US, China’s true AI frontier is no longer public.

[Denis Redozubov @codeofchange]

@klntsky @audn_ai Also, the big tech is not in alignment with what I envision to be safe and responsible tooling. Everyone rolls out their own harness to make migration for customers harder, rare, but dangerous wipeouts are acceptable etc

[Denis Redozubov @codeofchange]

Anyone used @audn_ai? Is it legit?

[Denis Redozubov @codeofchange]

@klntsky @audn_ai I'm working on a containment + policy loop to properly sandbox those agents. I've underestimated this work horribly; it's been ongoing for a long time now.

[Denis Redozubov @codeofchange]

@AceWDF @audn_ai What inference provider did you use? Or have you used your own hardware?

[Denis Redozubov @codeofchange]

@klntsky @audn_ai It would be great to find a way to run abliterated models on budget somehow, but I seem to fulfill my needs with Chinese models without arguing too much with them nowadays

[Denis Redozubov @codeofchange]

@klntsky @audn_ai I've been following since I asked about it, about two hours ago. I've seen multiple providers like this recently, but the website is pure slop. The business model seems to offer "unlimited" capacity with highly limited concurrency slots and variable t/s throughput.

[Hillel @hillelogram]

It is a weird weird year to be a formal methods expert.

Some unorganized thoughts on the new @AnthropicAI and @theoremlabs post on formal verification (https://t.co/YoymKXYFJa):

- This is very cool.

- The main innovation is verifying a later stage of the program. Normally provers take the source code, LLM it into a theorem language, and then prove that, which means you have to make assumptions about source semantics and compilation properties. It's very easy to make a mistake in translating and prove the wrong thing!

What they instead do is compile the code into assembly, and then (if I understand this correctly), compile that assembly into a theorem language (which they call LPS). So as long as the compiler is faithful you can skip the whole divergence problem.

- I'm glad they clarify that you still have to trust the translation process, the "hardware, OS, OCaml" compiler, the LPS model, and also assume that the program is single threaded with fixed memory. The single biggest source of errors in formally verified code is when the proof assumes something that isn't true in practice. It's really, really important to state your assumptions!

- I'm not clear why they need to compile the fuzzer into the theorem. Is it to act as a description of valid inputs and workload? Maybe to embed assertions in as theorems?

- Holy crap, 180 million lines of Rocq. Is that more Rocq than exists in the world already? I notice they say a 600,000 line proof corresponds to roughly 20k CPU instructions. That would be like, 1-2k lines of source code at most? Really goes to show how inefficient this approach is and how it only works because you can throw LLMs at the problem.

- This, probably deliberately, doesn't address the single biggest gap in AI-based verification: AIs cannot come up with good properties. Doesn't matter if you're doing unit testing, property testing, formal verification. I always assumed this was a "frontier labs haven't trained for it yet" thing, but if Anthropic hasn't talked about that here, maybe they've tried and haven't been successful yet. That's interesting to me.

Instead, they take a known universal property (this code doesn't crash or get flagged by UBsan) and prove that instead. Still very useful, but makes it harder to generalize to proving arbitrary things about arbitrary programs.

- Anybody reading the official post should also read their expert review appendix (https://t.co/qQU0E0nJwW). It points out some interesting things, like the fact that most of the code the LLM wrote wasn't used in the proof (over one billion lines of Rocq), that they had to prove the equivalence between fuzzed and unfuzzed programs, and (this is most interesting), some experiments find that the LLM-method of proofwriting may not scale to more complex things.

- Also read the description of LPS (https://t.co/hRTCL05X0H) to understand the limitations of the assembly -> Rocq compilation. In particular, it does not handle concurrency at all (which is one of the main use cases for formal methods!) or OS signals. Also it can't prove things about program arguments with slash characters, meaning they can prove stuff about `rm foo` but not `rm foo/bar`.

- The official skill (https://t.co/weDOdmJUfO) doesn't tell us much about the nature of FM and AI. Probably too much for me to hope of any skill.

- Some of the article's specific caveats and throat clearings make me think someone at Anthropic reads my stuff ( ͡° ͜ʖ ͡°)

[Ramez Naam @ramez]

Based on this, Kimi K3.1 Pro is ~5 weeks behind the US frontier. That's the shortest the open / closed AI model gap has ever been.

[Denis Redozubov @codeofchange]

I really hope the next decade will be a renaissance of FV

[Natalie Marleny @NatalieMarleny]

Small announcement: ✅ formal-git

As an experiment, over the last 2 months we've modelled ~30% of Git in Lean 4, making heavy use of coding agents, and proved >8000 theorems of varying usefulness about the implementation.

In the process of doing this we've uncovered >100 potential defects in Git v2.55, at least two of which may constitute security vulnerabilities. We've also built a DRT (differential random testing) harness and the biggest corpus of relevant pinned differential test cases for Git implementations with >10k unique tests at the moment.

We've used the test cases to test our own small internal implementation of Git in TypeScript. The next step will be to differentially test all notable public Git implementations against this corpus of tests to help improve their correctness and security 🫡 We'll be reaching out to library maintainers in due course.

If you have a Git implementation you'd like us to help you test, please reach out and let me know. We'll open-source all this eventually, after potential security issues that were uncovered are responsibly disclosed and fixed.

[Evan Kirstel #B2B #TechFluencer @EvanKirstel]

Ok that’s cool

[Denis Redozubov @codeofchange]

Balkancore is great https://t.co/Ov85TgY6y1

[Denis Redozubov @codeofchange]

@TaliaRinger Spend too much time today arguing on FV with people who clearly don't want to learn what it is and why it works

[Denis Redozubov @codeofchange]

@qrilka Беру тебя на карандаш - напишу после тихого запуска

[Denis Redozubov @codeofchange]

You lying piece of clanker

[Denis Redozubov @codeofchange]

@qrilka а тебе грузинский актуально? Дам знать, если да

[Denis Redozubov @codeofchange]

@qrilka хороший вопрос, к запуску исправлю

[moritz ✨ @angerman]

We are so early.

[Denis Redozubov @codeofchange]

@dysinger I routinely make my clankers do TLA+ now

[Denis Redozubov @codeofchange]

@klntsky You can’t clone production db to your developers with a reasonable customer data policy