jeremy

helping grow more claudes. personal views only

[levent @__alpoge__]

over the weekend i had another obvious thing to check, namely whether claude autonomously resolves the famed sum-product conjecture over the reals. answer: yes

[Dean W. Ball @deanwball]

bad news, friends. it's neither purely a marathon nor purely a sprint. it's a marathon that you have to sprint through the entire way.

[Katelyn Lesse @katelyn_lesse]

there's a ton of demand right now for excellent infra talent, definitely outpacing supply

im hiring engineers who have tackled megascale, multicloud product infra to work on the Claude Platform. help me find them 🙂

[Claude @claudeai]

Introducing Claude Opus 4.8: it builds on Opus 4.7 with sharper judgment, more honesty about its own progress, and the ability to work independently for longer than its predecessors.

Available today at the same price.

[Claude @claudeai]

Before we ship a new model, these teams try to break it.

They build with it, push it to its limits, and tell us where it falls short. What they find makes the final model better.

[Sholto Douglas @_sholtodouglas]

Huge credit to the OAI team for solving the unit distance problem with 5.5 - it is now my go to example that models can in fact pull together disparate ideas into new discoveries.

As with all 4 minute miles, we had to try and cross it too! Turns out mythos solves it with a cute, simple proof. This implies some serious overhang in discoveries!

[Anthropic @AnthropicAI]

Last month we launched Project Glasswing, our collaborative AI cybersecurity initiative. Since then, we and our partners have found more than ten thousand high- or critical-severity vulnerabilities in essential software.

[Andrej Karpathy @karpathy]

Personal update: I've joined Anthropic. I think the next few years at the frontier of LLMs will be especially formative. I am very excited to join the team here and get back to R&D. I remain deeply passionate about education and plan to resume my work on it in time.

[Sholto Douglas @_sholtodouglas]

When do you reach for other models instead of Claude? What can we do better? Hit me with all of your frustrations. dms open.

If you can give me detail (e.g. specifics/transcipts) - it'll help a lot in finding out exactly what we need to do to improve the next model

[julia @mooncat_is]

To be clear: this is the Mythos we shipped. The earlier results were from an in-training snapshot.

[jeremy @jerhadf]

emphasizing a detail from Logan’s post because folks seem a bit mixed up: these results aren’t on a new model. they’re on Mythos Preview, the model announced on Apr 7 that is being used within the Glasswing program currently.

[Logan Graham @logangraham]

A lot of people have been wondering about Mythos, Glasswing, and the vulns we / our partners are fixing. Today, I’m excited for us to start sharing more. (For context, I lead Glasswing @AnthropicAI.)

Two independent evaluations this week—from XBOW and the UK AISI—confirm what we've been seeing internally: Claude Mythos Preview is a step change in autonomous cybersecurity capabilities. We need to start preparing fast for a world of models with this level of capabilities.

The UK AI Security Institute tested the model we shipped at the launch of Project Glasswing and found Mythos Preview is the first model to solve both of their end-to-end cyber ranges, including one (Cooling Tower) which no model had ever cleared. But attackers (and defenders) have sophistication & cost constraints – Mythos is also the only model that clears every one of their tasks estimated over 8 hours under their deliberately low 2.5M-token cap.

XBOW tested it on their offensive security benchmarks, finding "token-for-token, unprecedented precision." It's the only model to succeed at subtle V8 sandbox work.

Other Glasswing partners shared similar stories. In a few weeks of testing, Mythos Preview has helped them find many thousands of (estimated) high + critical severity vulnerabilities, sometimes double what they'd normally find in a year.

I don't share this to boost Mythos. In fact, this is not about Mythos. It’s about preparing for the coming world of models being better, faster, cheaper, and more creative than some of the best human experts at dual use capabilities. Clearly, we need them supporting defenders as widely as can be done safely – and especially the least resourced ones.

Within a year, Mythos will probably look quite dumb (relative to other new models). And others may release openly available or unguardrailed models of Mythos-level capabilities.

We started Project Glasswing because capabilities like Mythos Preview's won't stay rare, or stay in careful hands. We are bringing it to defenders as fast as we responsibly can, while working to figure out, for example, the right safeguards and patching & disclosure processes.

Also, to be clear, compute has never been a limiter in our rollout.

Expect a fuller update on our Glasswing work in the coming days.

XBOW report: https://t.co/Mumtbf3kE3

UK AISI report: https://t.co/vBgqz0AeKJ

[Logan Graham @logangraham]

A lot of people have been wondering about Mythos, Glasswing, and the vulns we / our partners are fixing. Today, I’m excited for us to start sharing more. (For context, I lead Glasswing @AnthropicAI.)

Two independent evaluations this week—from XBOW and the UK AISI—confirm what we've been seeing internally: Claude Mythos Preview is a step change in autonomous cybersecurity capabilities. We need to start preparing fast for a world of models with this level of capabilities.

The UK AI Security Institute tested the model we shipped at the launch of Project Glasswing and found Mythos Preview is the first model to solve both of their end-to-end cyber ranges, including one (Cooling Tower) which no model had ever cleared. But attackers (and defenders) have sophistication & cost constraints – Mythos is also the only model that clears every one of their tasks estimated over 8 hours under their deliberately low 2.5M-token cap.

XBOW tested it on their offensive security benchmarks, finding "token-for-token, unprecedented precision." It's the only model to succeed at subtle V8 sandbox work.

Other Glasswing partners shared similar stories. In a few weeks of testing, Mythos Preview has helped them find many thousands of (estimated) high + critical severity vulnerabilities, sometimes double what they'd normally find in a year.

I don't share this to boost Mythos. In fact, this is not about Mythos. It’s about preparing for the coming world of models being better, faster, cheaper, and more creative than some of the best human experts at dual use capabilities. Clearly, we need them supporting defenders as widely as can be done safely – and especially the least resourced ones.

Within a year, Mythos will probably look quite dumb (relative to other new models). And others may release openly available or unguardrailed models of Mythos-level capabilities.

We started Project Glasswing because capabilities like Mythos Preview's won't stay rare, or stay in careful hands. We are bringing it to defenders as fast as we responsibly can, while working to figure out, for example, the right safeguards and patching & disclosure processes.

Also, to be clear, compute has never been a limiter in our rollout.

Expect a fuller update on our Glasswing work in the coming days.

XBOW report: https://t.co/Mumtbf3kE3

UK AISI report: https://t.co/vBgqz0AeKJ

[Boris Cherny @bcherny]

The UK AISI found Mythos Preview is the first model to solve both their cyber ranges end-to-end. No model had ever solved the AISI’s “Cooling Tower” cyber range before.

We're getting it to defenders as fast as we responsibly can. More to come on our Glasswing work soon.

[jeremy @jerhadf]

i wonder how the "frontier models will commoditize" folks are doing...

watch line go up cures one of this misconception. clearly, frontier AI isn't a widespread, standardized, perfectly fungible good w/ 0 switching costs & no pricing power. models are stickier than most believe

[jeremy @jerhadf]

@hiarun02 @hiarun02 interesting could you say more about how this eval works? and have you evaluated any claude models here?

[jeremy @jerhadf]

@willdepue @willdepue wdym by “won’t think at all”? as in it isn’t triggering adaptive thinking / it’s not thinking enough? in CC?

[jeremy @jerhadf]

@jarredsumner holy shit lfg

[Jarred Sumner @jarredsumner]

there will be a blog post about this. on what this means for bun, benchmarks, memory usage, maintainability going forward, and also the literal process of doing this (it wasn’t just “claude, rewrite bun in rust. make no mistakes”)

this is a 960,000 LOC rewrite, the code truly works, passing the test suite on Linux and soon other platforms. e2e I started working on this 6 days ago. this would’ve been a massive amount of work by hand.

[jeremy @jerhadf]

@tszzl yes, although alignment is going pretty well so far, we've also been doing alignment on 'easy mode'. close-to-strictly-superhuman AGI will make it much harder and higher-stakes. we'll need to align the automated alignment researchers to then align superintelligence