Claire Wang

CS PhD Student at Penn. Program analysis, synthesis, testing, and verification + ML. Also music and dependent types.

[Claire Wang @lambdaclaire]

Excited to share QLCoder - an agentic framework for synthesizing static analysis queries for vulnerability detection
ICLR 2026 poster session 1 on Thursday 4/23
Link to paper and code in thread below

[Adam Stein @adamlsteinl]

Beam is out! I feel extremely lucky to have been been part of this launch as part of the safety team for the past ~2 months. It’s been a lot of looking deeply into model outputs and data, being surprised by what we see, and updating our evaluations and training recipes to fix the problems. It’s amazing what has been achieved in so short a time, and I'm very excited for the road ahead!

[Claire Wang @lambdaclaire]

@cynlioness @qd_forall we have another event coming up in brooklyn next month :)

[Claire Wang @lambdaclaire]

throwing an algorave/technical salon on talks and performances intertwining math, arts, and code in philly on 5/24 - free event! rsvp https://t.co/3o9nMUVuwo
feat computer assisted quilting, livecoding via juggling, encoding logic systems to generate art, and much more

[Alperen Keleş @Keleesssss]

I'll present "Abstractification" at the "Who Verifies the Agents?" workshop at Neurips! One of the largest questions people have right now is "what are the higher level languages in this era?".

Maybe instead of inventing something new, we should look back at Program Sketching.

[Kaiyu Yang @KaiyuYang4]

Can AI improve its own training infrastructure and prove its changes correct? That question motivated our work on VeriTile (https://t.co/0pYqTajVCj), led by @ZenanLii.

VeriTile embeds Triton-style kernels in Lean, where AI agents construct machine-checkable proofs, e.g., proving FlashAttention correctly implements the attention function. GPU kernels seem like a promising place to start: faster kernels, faster training, with correctness guarantees.

[Hillel @hillelogram]

I'm seeing a lot of euphoria about how Opus 5.5 is good at TLA+, and this means that all software will soon be formally verified. As a person who loves TLA+ so much he wrote a book on it, I want to throw a particular cold shower on people's enthusiasm by talking about the limits of what you can actually verified with it.

The high level simplification is that TLA+ sees a system as a set of "behaviors", or possible sequences of states. For example, the pseudocode "pick a random number from 1-3 and decrement it to 1" has three behaviors: `{3 -> 2 -> 1, 2 -> 1, 1}`. From here, there are two basic kinds of TLA+ properties:

- `[]P` means that `P` is true in *all states* of *every behavior*.
- `<>P` means that `P` is true in *at least one state* of *every behavior*.

`[]P` is immediately useful as an **invariant**, or something that always be true of your system. This is things like "your data is never corrupt" or "there's always at least one server online." `<>P` is a little more abstract, but for technical math reasons I won't get into here, can be stacked with `[]` to create really complex and useful properties. `<>[]P` represents things like "the algorithm eventually converges on the right answer", `[]<>P` things like "if two data stores desync, they will eventually resync", and `[](P => <>Q)` things like "If a message is put on the queue, it's eventually processed by a worker".

Really cool stuff!

These primitives were chosen to make a wide array of properties useful. And if we're clever, we can do all sorts of more complex properties, like bounded time constraints and history properties. But we're always constrained to 1) define a logical formula 2) over individual behaviors, and 3) check that all behaviors satisfy that formula.

So some things that we *cannot* express in TLA+:

- Possibility and reachability properties: that it's always possible to *make* P true, even if you don't actually decide to. Things like "I can always shut down the computer" or "A user can always change their password". These can't be expressed with `<>P` because that's "for all behaviors, P happens at least once", we actually want "for all behavior prefixes, there is at least one behavior where P happens at least once".
- Hyperproperties: properties that are defined over two or more traces. These are things like "painting a car red doesn't make it faster" or "users cannot infer secret data by observing public data". We can't do these because TLA+ only looks at one behavior at a time.
- Statistical properties: 95% latency is 1ms. Impossible because most of these are hyperproperties.
- Properties about if a system is robust against code changes. Impossible because, uh, you have new behaviors now.

Some of these are solvable in different logical formalisms. CTL can do reachability, PRISM can do statistical properties, etc. Those have their own tradeoffs and limitations, though, and no system can do everything. Others are solvable with a lot of cleverness tailored to the specific spec, like lifting a model into a hypermodel. But these are insanely inefficient and make your "clever spec" diverge significantly from the real world system, so introduce a lot more opportunity for things to go wrong.

The core problem, though, is (1): properties are logical formula. If we don't know how to express a system property as a logical formula, we can't verify it. 99% of the properties we care about fall under this. The information on the site is easy for a user to find. Our LLMs behave as we expect them to. Our application can't be used to break the law. TLA+ (and Quint and Lean and Rocq) are near-useless here, no matter how clever you are.

Don't get me wrong: `[]P` and `<>P` represent a huge range of useful properties and TLA+ is incredible at finding awful concurrency bugs. But there's a lot it fundamentally can't do and we shouldn't believe that it will solve all our worries about software bugs. And the same goes for all other formal verification languages, too.

[Tianyin Xu @tianyin_xu]

Thanks for the shoutout to TLA+ @bcherny !

As suggested by Markus Kuppe, let me showcase the brilliant work done by colleagues at Specula and the TLA+ community. While Specula (https://t.co/hoV6UJIybf) can effectively model system code in TLA+ (and then use model checkers to find hundreds of deep bugs), we can further prove the correctness of system specs using TLAPS. The TLAPS-Bench (https://t.co/hrYHc4DSow) project aims to offer a proof service for any TLA+ spec of important systems/protocols. Basically, Specula writes the spec; TLAS-Bench generates the proof.

AI writes amazing proofs! We had to retire all Proof-Completion tasks in TLAPS-Bench as they're too easy for frontier AI. We also proved classic protocols and algorithms like 2PC, Paxos, TCP, etc.

However, proving low-level specs of real-world system code _from scratch_ is still non-trivial. @qiancheng9788 ran a subset of TLAPS-Bench using Claude Code w/ Opus 5. The success rate is merely 51.4%. @Muse is great, but Muse Spark 1.3 can only achieve 16.7%.

Ruize has a bag of tricks to push AI agents. Once he pushed Codex to prove one of the nine invariants for the ZooKeeper implementation. Codex struggled 5 days, burned $2000, and did it correctly. We ran out of money to continue. And, @xu_dong_sun shows us that liveness proofs are still difficult for AI without guidance. We have an exciting avenue, but still a lot of work to be done.

If anyone wants to verify their protocols/systems, send us some tokens -- we can do it in TLAPS-Bench. @HacksonClark tells me that people are excited about long-horizon tasks -- TLAPS-Bench may be a perfect benchmark for that.

@bcherny Consider further tuning Claude for TLA+, you may find more bugs and/or prove the correctness of SDK code more efficiently.

[joomy @joomy]

&gt; i ask Claude if the program is verified
&gt; he says yes
&gt; i ask if he verified the program or a smaller, better-behaved program he imagined
&gt; he laughs and says “it’s verified sir”
&gt; i use the program
&gt; it has a bug

[Alperen Keleş @Keleesssss]

Bugün günlerden formal verification olacak gibi duruyor, neymiş bu ne işe yarıyormuş diye merak ederseniz benim kanıtlı programlama yazımı okuyabilirsiniz. (https://t.co/XkCbFlWhiO)

[Jason Gross @diagram_chaser]

Keep an eye out for:
- plan for verifying all software coming very soon to @theoremlabs blog
- verified (toy) sandbox demonstration in the coming weeks
- verified production sandbox on Linux by end-of-year

[joomy @joomy]

&gt; there's just one catch though, which is that the published implementation drifted from the formalization so *can* contain bugs

but that's the whole point! 🤷‍♂️

you can have a neat formal model, but it's only a model! it's not the real thing; doesn't give you strong guarantees.

[Claire Wang @lambdaclaire]

@krismicinski Please do. We need more actual pl compilers peoples’ commentary on bend. This would be super helpful though I suspect you might lose your weekend.

[Ilya Sergey @ilyasergey]

Velvet 2.0 is out: now based on Lean's most recent verification machinery, easier to set up, and 10x faster. New: exception specs, ghost state, named proof goals, lots of case studies from Dijkstra to lazy segment trees. And a new shiny webpage:

https://t.co/dMLBvLP6Yq

[Kristopher Micinski -- REBORN @krismicinski]

Thoughts on Learning with AI https://t.co/XWM1Kat8FY

[Zhening Li (Zed) @zli11010]

btw if you’re looking for the best possible implementation of the JIT compiler half of sPTC, take a look at opportunistic lambda calculus (https://t.co/8lkQBr5bHd)

they have the optimal solution to this problem and it’s conceptually clean and elegant as well

[Aaron Roth @Aaroth]

Codex and Claude Code have a neat auto approve feature where 1) it doesn't ask for permissions, but 2) you get to feel safe. Well --- do you? The premise is it is asking an agent for permission. But if you don't trust the driver agent, why should you trust the review agent?

[POPL 2027 @poplconf]

The first annual Future of Property-Based Testing Workshop (FPBT'27) will be happening in Mexico City on January 12th (co-located with POPL'27)!

Submissions are open now (deadline October 23rd); see https://t.co/Sj8aYoJYKz for details.

[SAIR @SAIRfoundation]

Lean Kernel Challenge Stage 1 is live!

Join @leanprover and SAIR to improve the performance of verified computation in the Lean 4 kernel that the whole community can benefit from.

https://t.co/KJyCi8X8eM

[Alex Meiburg @Timeroot]

Maybe an AI is about to solve / has solved a Millenium problem, but I think there's good reason to believe that it definitely isn't P vs. NP. One thing that people don't realize is that the vast majority "progress" on the problem has been *ruling out possible proof strategies*.

[Tweag by Modus Create @tweagio]

Formal proofs are fashionable again, and many hope LLMs will write them for us. @aspiwack, a self-described formal-proof nerd, wanted an opinion of his own, so he specified a tiny SAT solver in Rocq and let a coding agent write all the code and proofs. What impressed him, what amused him and the time the agent duplicated his entire program rather than prove a refactor correct. Are we formal proofs yet?

https://t.co/3tQrsM4mgV

[Chaofan Shou @shoucccc]

I bought a Fable dataset from one of the top Chinese LLM routers yesterday.

With just 6TB data, I can take over 7 Chinese/CIS gov entities &amp; 19 top Chinese firms like Xiaomi, Huawei, NIO, Minimax using SSH keys, VPN configs, Aliyun keys, GitLab tokens sent to the router.