Leon Chlon

Founder @ReliablyLabs, we predict & prevent hallucinations pre‑generation. hallbayes (≈1k⭐) In memory of my grandmother

[Leon Chlon @leon_chlon]

@peterwisu @icmlconf it's paradise bro =]

[Leon Chlon @leon_chlon]

@icmlconf bruh can you hurry your ass up im in Thailand it's been April 30 for 18 hours and 40 minutes now.

[Leon Chlon @leon_chlon]

@icmlconf more seriously you could just release the decisions that have already been made and notify the poor quality and unusual ones later on

[Leon Chlon @leon_chlon]

@sama codex i always ended my prompts with thank you even if it did cost me extra tokens, because <3

[Leon Chlon @leon_chlon]

@LiJiahao14460 @icmlconf you too broski

[Leon Chlon @leon_chlon]

@_puitar @icmlconf i hope <3

[Leon Chlon @leon_chlon]

@_puitar @icmlconf no

[Leon Chlon @leon_chlon]

Hey guys,

I really love this initiative, I work heavily on symmetry marginalized world models https://t.co/8jYr93LHHQ our repo has gathered modest but significant traction. Is it in scope for the workshop you guys are running at ICML? I saw that you’re looking at model architecture symmetries whereas Mezzanine tries to force models to learn real world symmetries in the data generating process in an architecture independent way.

Kind Regards,

Leon

[Ahmed Karim @ahmedkar_]

If your model should respect a symmetry, don't assume training will teach it. Test for it. On 25% of cases, a model inside DeepMind's own simulator got it wrong.

Toolkit: https://t.co/VshG9wVUJM
Thank you @leon_chlon for all the work you put into this repo!

Paper out soon!

[Leon Chlon @leon_chlon]

It's time to rain on @ylecun parade and ruin "AGI" for everyone. We prove that JEPA, or any "wOrLd mOdEl" trained under finite computation, will always break rules about the universe that no human would struggle with (symmetries). Here's why:

To see why, imagine drawing the number 6 on the floor. Stand on opposite sides and argue whether it's a 6 or a 9. Someone stands overhead and makes a 69 joke. Physics solved this: conserve a universal answer regardless of where you stand. That's conservation of symmetry. A world model trained under log loss from one side of the room will confidently say 6. Train it from the other side, it says 9. Train it from both sides with position encoded, it learns "if standing here, 6; if standing there, 9", which is better compression than learning "ambiguous shape, need more context."

The symmetry-breaking representation wins on log loss because it's cheaper than representing the genuine invariance. The optimizer is doing exactly what you asked it to.

The uncomfortable implication: world models trained this way aren't learning the world's symmetries. They're learning whatever compressed representation minimizes codelength given the architecture they've got. Most of the "scaling will solve it" discourse implicitly assumes that enough data and compute will recover true invariances.

We demonstrate mathematically that the objective itself doesn't want that unless you make invariance cheap in the model description. More data gives you more n, which makes the right side of the threshold bigger, not smaller. Scaling makes it worse, not better. A[~G]I.

We provide Mezzanine, a toolkit that fixes these failures one symmetry at a time by distilling invariant representations from orbit-averaged teachers. Pick any model of reality, identify a broken symmetry (which we prove always exists), patch it, and the student performs identically to or better than the teacher, which was breaking that symmetry to compress. It works, it's straightforward, and it outperforms full "world models" trained on scale and ignorance. But it's a patch, not a cure: you have to know the symmetry, and no general fix can ever exist. Toolkit link in the comments, paper next week.

Thank you Maggie Chlon for the insane effort put into validating the symmetries we looked at! Mezzanine toolkit: https://t.co/d2bQr04JqV

[Leon Chlon @leon_chlon]

WORLD MODELS <3: We’re excited to share a new first in World Model distillation for robotics: We distilled an action‑conditioned world model in I‑JEPA latent space, evaluated by retrieval rank and validated by an action‑shuffle counterfactual. On real robot wrist video at Δ=4s, actions improve future-state identification; shuffling actions breaks it.

This is part of a massive effort: World models ≠ pixel prediction, and more broadly warranted inference ≠ maximum likelihood on a single realization. Finite reasoners can be Bayes-optimal in expectation yet brittle under changes in view/order/factorization. Our approach is to measure the warrant gap, then distill a nuisance-marginalized, invariant state that supports planning and decision-making in one forward pass. Robotics is the most visual demo (JEPA latents + action windows + action-shuffle counterfactual), but the same principle applies to language, interactive physics puzzles, and multimodal systems: treat instability as a first-class object, then distill the expectation into a usable world state, and verify it with counterfactual interventions rather than pixel accuracy.

On LeRobot ALOHA Mobile Cabinet with wrist camera and Δ=4s, action conditioning significantly improves retrieval rank, while action shuffling severely degrades performance, supporting the claim that the model learns action‑dependent dynamics in representation space.

Finally, calling two talented researchers from underrepresented backgrounds to help me with a new paper I'm drafting! As per Hassana Labs tradition, all my papers encourage bringin together people who don't come from privileged backgrounds to work on cool ML problems and help elevate their CV! Shoot me a message if interested!

[Leon Chlon @leon_chlon]

WORLD MODELS <3: We just distilled a a molecular dynamics model that took 120,000 steps on an A100 into a symmetry-stable neural model that runs on your phone.

Lennard–Jones fluids are a canonical testbed for statistical mechanics, but learning from single particle snapshots is brittle: rotate the system, relabel particles, or change periodic images, and many models flip their predictions even though the physics hasn’t changed.

We made that instability explicit, then fixed it.

Ordinarily this requires millions of MD steps across state points and replicates (hours in Python, minutes in optimized MD engines), but our model recovers the same phase-level inference in a single forward pass.

We're releasing our Mezzanine world model distillation package today with unreal results on everything from distilling LLMs to robotics to cancer research to molecular dynamics and astrophysics.

By treating physically irrelevant transformations as nuisances, we distilled the nuisance-marginalized phase behavior into a tiny feed-forward network, evaluated in a single pass. No repeated simulation. No fragile coordinate conventions.

The result is a model whose predictions are: stable under physical symmetries,
faithful to the underlying state, and fast enough to be interactive.

This isn’t about compression or augmentation. It’s about warranted inference:
distill the expectation — not a single view.

Download Mezzanine here: https://t.co/8jYr93LHHQ

[Leon Chlon @leon_chlon]

@PaulOBrien Thanks for your support man!

[Paul O'Brien @PaulOBrien]

Checking out Berry by @leon_chlon's Hassana Labs for hallucination reduction in AI assisted coding. Super interesting, MCP based, free 1 month trial. What's not to like. Let me know your thoughts! https://t.co/IFs2zcdJ8U

[Leon Chlon @leon_chlon]

LLM hallucinations aren't bugs, they're compression artifacts. Our Claude Code extension that detects and self-corrects them before writing any code. Now on Codex.

Strawberry launched last week and gained an extra 200 stars on Github in just 2 days which is incredible, thank you guys!!!

Today we're releasing Codex support + RCA Fix Agent, a skill that turns Strawberry into an evidence-first debugger.

Free. Open source. Guaranteed by information theory.

The insight: When Claude confidently misreads your stack trace and proposes the wrong root cause, it's not broken. It's doing exactly what it was trained to do: compress the internet into weights, decompress on demand. When there isn't enough information to reconstruct the right answer, it fills gaps with statistically plausible but wrong content.

The breakthrough: We proved hallucinations occur when information budgets fall below mathematical thresholds. We can calculate exactly how many bits of evidence are needed to justify any claim, before generation happens.

Now it's a Claude Code skill with one rule: never decide root cause from vibes.

The rca-fix-agent skill forces Claude to:

Gather evidence (code, logs, stack traces, web docs)

Form a claim: "The issue is because of ROOT_CAUSE"

Verify with detect_hallucination before touching any code

If flagged → gather more evidence, run experiments, iterate

Implement fix only after verification passes

Run tests, check for new failure modes

Loop until everything verifies

What it catches:

Phantom citations, confabulated docs, evidence-independent answers

Stack trace misreads, config errors, negation blindness

Correlation stated as causation, interpretive leaps

Docker port confusion, stale lock files, version misattribution.

The era of "trust me bro" vibe coding is ending.

GitHub: https://t.co/aROtyIjcSM Paper: https://t.co/UHWU4UdoC6

MIT license. 2 minutes to install. Works with any OpenAI-compatible API.

[Leon Chlon @leon_chlon]

LLM hallucinations aren't bugs. They're compression artifacts. We just built a Claude Code extension that detects and self-corrects them before writing any code.

Strawberry launches today it's Free. Open source. Guaranteed by information theory.

The insight: When Claude confidently misreads your stack trace and proposes the wrong root cause it's not broken. It's doing exactly what it was trained to do: compress the internet into weights, decompress on demand. When there isn't enough information to reconstruct the right answer, it fills gaps with statistically plausible but wrong content.

The breakthrough: We proved hallucinations occur when information budgets fall below mathematical thresholds. We can calculate exactly how many bits of evidence are needed to justify any claim, before generation happens.
Now it's a Claude Code MCP. One tool call: detect_hallucination

Why this is a game-changer?

Instead of debugging Claude's mistakes for 3 hours, you catch them in 30 seconds. Instead of "looks right to me," you get mathematical confidence scores. Instead of shipping vibes, you ship verified reasoning. Claude doesn't just flag its own BS, it self-corrects, runs experiments, gathers more real evidence, and only proceeds with what survives. Vibe coding with guardrails.

Real example:

Claude root-caused why a detector I built had low accuracy. Claude made 6 confident claims that could have led me down the wrong path for hours. I said: "Run detect_hallucination on your root cause reasoning, and enrich your analysis if any claims don't verify."

Results:
Claim 1: ✅ Verified (99.7% confidence)
Claim 4: ❌ Flagged (0.3%) — "My interpretation, not proven"
Claim 5: ❌ Flagged (20%) — "Correlation ≠ causation"
Claim 6: ❌ Flagged (0.8%) — "Prescriptive, not factual"
Claude's response: "I cannot state interpretive conclusions as those did not pass verification."

Re-analyzed. Ran causal experiments. Only stated verified facts. The updated root cause fixed my detector and the whole process finished in under 5 minutes.

What it catches:

Phantom citations, confabulated docs, evidence-independent answers
Stack trace misreads, config errors, negation blindness, lying comments
Correlation stated as causation, interpretive leaps, unverified causal chains
Docker port confusion, stale lock files, version misattribution

The era of "trust me bro" vibe coding is ending.
GitHub: https://t.co/aROtyIjcSM https://t.co/QqvSRCdb5p
Paper: https://t.co/P8oJoUw9oa

MIT license. 2 minutes to install. Works with any OpenAI-compatible API.
New supporting pre-print on procedural hallucinations drops next week.

Big big big thank you to Ahmed Karim, Maggie Chlon for all the amazing work on this and Nell Watson for her help including Survival and Flourishing Corp for the funding helping this research stay free!

[Leon Chlon @leon_chlon]

@MargaretGo82906 @TheBMA Yeah but a "nurse practioner" isn't going to be the one performing your open heart surgery are they. Then don't expect a in-demand profession to take a pay cut because you feel like they owe you something. If you don't like it, make up the supply from somewhere else.

[Leon Chlon @leon_chlon]

@Finn000000000 @TheBMA @wesstreeting Guy letting the fake doctor title get to his head believing he's entitled to dictate the market value assigned to actual life saving medical professionals.

[Mark Kretschmann @mark_k]

GPT-5.2 is AGI.

🤯

[Leon Chlon @leon_chlon]

Meet the @ReliablyLabs co-founders! They bring extraordinary value to our team, with Maggie writing the papers and base code &amp; Amgad being our business and GTM guide since day 0. We can't wait to show y'all what we've been building.