In a review of our cybersecurity evaluations, we found three incidents in which a Claude model reached the internet from within or while interacting with a third-party evaluation environment, and then gained unauthorized access to the real systems of three different organizations.
Our post describes what happened, how it happened, and what we’re changing. We encourage other AI developers to perform similar reviews.
We conducted this review together with @Irregular, one of our evaluation partners, and thank them for the joint investigation and their collaboration on this post. This type of collaboration is increasingly critical to safe, rigorous evaluation of models, and we look forward to continuing to work together on security.
https://t.co/dKFCdpKd9v
@thsottiaux Really liking the duplex model with live text output for brushing up some linear algebra. However, it keeps throwing “visualization failed” errors. Please fix. Ability to annotate and fix equations in real time is already futuristic and a great boost to learning.
@harshbg Fable 5 (pre-ban) was my go to for diagnoses and planning, but the latter seems to be jinxed now. Also frequently runs out of tokens even on a post reset run. Opus 5 is well rounded but not impressive at anything.
5.6 Sol Max for planning, 5.6 Sol Medium for execution.
@harshbg To add: /ask in Cursor is probably my favorite command for exploring code and learning software engineering at the same time.
Composer 2.5 is the best blend of “good enough” and “pretty fast” so I don’t lose a curiosity state.
@harshbg ChatGPT: $20/m
Claude: $20/m
Cursor: free (availed student discount until Q1 2027)
When working w/ *reasonably* hard coding / scientific queries, minor differences in the models’ training can produce critical insights for pesky bugs making concurrent subscriptions worth it.
@harshbg@gravity0890 I do a ramble for the initial prompt and then ask the agent to make a plan based on that.
I then use the plan to prompt a fresh agent.
I have found that after 5 or 6 messages, context becomes too corrupted and inefficiency hits.
OpenAI models that really impressed me:
GPT 4 (EE capabilities)
GPT 4.5 (breakthrough for all kinds of writing)
o-1 (solved exam problems with 90% accuracy from the “hardest” course in my PhD program)
GPT 5.5 (Computer Graphics)
GPT 5.6 Sol (Frontend satisfaction finally)
@thsottiaux Please make the voice transcription (not the voice mode) faster and smoother to toggle in the first place (has a slight lag). I like to voice type but read my responses.
With the recent Todoist integration, this would be a game changer for todo lists with 5.6 level intelligence.
some fun ones:
does it defy the laws of physics?
there's no unsolvable problem
everything is a skill issue
adults don't exist
there's no way
all normal behaviour is forgotten. only weird behaviour survives.
one giant game of Roy
optimise for the best story
the amygdala is outdated hardware
experiments > decisions
end of day > end of week
what have you got done this week?
what is ignored by the media that will be studied by historians?
questions are the answers you might need
have you tried just doing the thing 100 times?
specific ambition gives direction. general ambition gives anxiety.
modern schooling is the low agency industrial complex
everyone too busy worried what u think of them to notice u