Excited to share that CryptanalysisBench has been accepted to NeurIPS 2026! See you guys in Sydney! 🦘
More info about the project here: https://t.co/gYqiLHDQlE
AI is advancing fast in math and cyber. Cryptanalysis sits at their intersection and underpins our digital security.
So can LLMs do cryptanalysis?
Increasingly, yes!
Introducing CryptanalysisBench where we test if AI can break 191 real schemes from past standardization efforts
By running models *slightly* longer than in our recent benchmark evaluation (one billion tokens and >$100k ) Mythos found some impressive attacks on cryptographic schemes
I'm excited to share the results of a project that I spent a significant part of the past year working on!
How good are LLM agents at breaking cryptographic algorithms? We tried to measure this. Check it out: https://t.co/yWqO1Jw06x
AI is advancing fast in math and cyber. Cryptanalysis sits at their intersection and underpins our digital security.
So can LLMs do cryptanalysis?
Increasingly, yes!
Introducing CryptanalysisBench where we test if AI can break 191 real schemes from past standardization efforts
Excited that our ICML position paper has been accepted as an Oral 🎉!
When a model looks like it "deceives" or "resists shutdown," how do we know it isn't role-play, instruction-following, or just task-completion pressure? Often the current evidence can't yet tell them apart, and those claims are starting to inform deployment and regulation.
We map where the evidence gets thin across four stages and propose a shared standard to strengthen it. Here's the map.
🧵
https://t.co/gzM8mLGB5P
I'm excited to present my recent work on the (lack of) mathematical safety guarantees in reward learning at ICML!
Interested? Then come visit me tomorrow, July 15, between 11 a.m. and 1:30 p.m. in the East Exhibition Hall A-B, # E-805.
Reward Learning is just supervised learning, and so should be equally safe, right? Wrong!
Our paper “The Perils of Optimizing Learned Reward Functions: Low Training Error Does Not Guarantee Low Regret” shows that policy optimization causes issues. It was accepted to ICML! 🧵