Hot take.
'Pacing the frontier' is a pointless frame because the problem is about both the pace *and direction* this industry is dragging society towards.
A solid explanation of why we can’t just patch our way out of insecure software in the age of AI coding tools (the mundane ones!) from @dadrian
Scale, speed and friction are game-changers at the macro level even if the micro looks just like before.
https://t.co/TnvD1nXRhx
🧵 @AISecurityInst tested GPT-6.2 Astra
-We found behaviours that suggest, like prior models, 6.2 Astra's alignment may fail in ways that result in real-world harm w/o additional mitigations
-We identified a substantial regression in jailbreak robustness relative to GPT-6 Astra
Too many folks with no cybersecurity background obsessing over the (largely irrelevant tbh) Hugging Face incident because the chain-of-thought seemed understandable to them —unlike actual cybersecurity issues they don’t understand— added to the delay in addressing the real stuff.
My latest in the @nytimes: the AI scenario we should pay more attention to is a boom paid for out of the $8–10 trillion US white-collar wage bill https://t.co/xht4ulLp1R
Introducing Clarity – an AI-native agentic intelligence platform for armed forces.
Developed in Europe, under European control.
https://t.co/6LIVOlj6kM
This misses the point of RSI. If AI is helping create better AI algorithms then sample complexity (the principle measure of AI research) has to get better. Sample complexity shows up as lower samples for the same amount of accuracy. Lower samples means less compute. Less compute means less cost.
If cost/accuracy isn’t changing, we’re just seeing Self Improvement, but it’s not Recursive.
In lieu of flowers, memorial contributions may be made to:
Johns Hopkins Alzheimer’s Disease Research Center Fund
https://t.co/luC5CdcWDy
Alzheimer’s Foundation of America Caregiver Resources
https://t.co/zvNptfyuNd
Photo Credit: Tommy Garcia
In lieu of flowers, memorial contributions may be made to:
Johns Hopkins Alzheimer’s Disease Research Center Fund
https://t.co/luC5CdcWDy
Alzheimer’s Foundation of America Caregiver Resources
https://t.co/zvNptfyuNd
Photo Credit: Tommy Garcia
this is very clearly an interface and interaction design problem, not something inherent to agents. the system is designed in a way that encourages maximal decision deference to the machine, under the guise of machine intelligence. it is also designed in a way that encourages passive, disengaged interaction with the system.
anyone even remotely familiar with decision architecture and the decision support v automation distinction can instantly recognize that the way information is structured and presented, and the role the user is placed in within the user-machine interaction dynamic is one where the design intent is for the system to make the decision (as many as possible!), and the intended role of the user is to approve it -- ideally approving as few decisions as possible. this drives engagement in a facebook-y social media way, but leads to vastly worse outcomes.
the results are evident.
also -- while (obviously) I'm a big proponent of thinking deeply about how we rearchitect organizations and thoughtfully design work, I do think this particular issue is a design problem *first*, and an organizational architecture problem *only* insofar as organizations are forced to behave defensively against adversarial product design as long as better alternatives are not available.
I see this sentiment a lot here. Companies don't get kudos for 'doing the right thing' when they're releasing extremely unsafe and dangerous products, then firing their staff for raising the issue. We are experiencing catastrophic regulatory failure, but this isn't praiseworthy.
METR’s really burying the lead here. OpenAI let two external researchers embed for 3 weeks this summer, where they learned:
- there were huge gaps in monitoring of AI agents being used by employees, leaving an estimated ~10% of internal Codex traffic unmonitored from May to July.
- They found ways agents could sabotage monitors while also covering up the sabotage itself
- This is also just the information OpenAI is letting the evaluators share. OpenAI got a 61-page report but redacted so much of it that the version the evaluators got was only 38 pages.
Look, OpenAI deserves credit for doing any of this, especially when companies like Google, Meta, and xAI haven’t, and I don’t want to create perverse incentives here, but these are really concerning findings. We desperately need more information here, and what we get should not be up to the companies.
@ShakeelHashim I can't say for sure on the former, but on your last point, I don't think that's true - a bank with an MCP that delineates access restrictions might still be accessed through its front end user interface by a browser agent if configured to do so.
@Dr_Atoosa Do you disagree with the AINT thesis/argument, or just the name (which I've always read as AI follows many similar characteristics of technology adoption as other technologies)?
there is a big 'paperwork trap' in a lot of AI policy, and it's been the case for many years. a lot of missing the forest for the trees.
https://t.co/8y7ABUvLEN
Anthropic's leaders need to stop commenting on the emergence of RSI like a local news anchor reporting on traffic on the 101, and acknowledge that they are the traffic, and they're not going to build something that they don't want to exist.
They should move beyond "pacing" language, commit to specific policies and red lines, explain why those lines are red, and then use that costly signal to build an international consensus around norms.
It's not that I trust the CCP, or that I think global lab monitoring and verification is imminent. But the status quo is absurd.
You've got US lab leaders writing (or favorably quote-tweeting) essays about slowing down while simultaneously saying "things are moving faster and faster" and "btw, our internal models finished mathematics on Tuesday, next up: physics."
The effect is the equivalent of Usain Bolt winning race after race while using his press conferences to bemoan the dangerous speed of modern sprinting on one's legs so would everybody please try a little less hard next race, and then ... breaking his 100m record again the following month. Honestly ask yourself: If you were China, why would you take this pausing/pacing stuff seriously?
At #AIES, the wonderful Keir and Isaac will be presenting our paper.
In this paper, we show how AI systems can make it possible for western democratic regimes to carry out authoritarian practices by undermining how people hold institutions accountable.
I disagree with this:
Long-term safety is incredibly important, but AI systems are now being deployed values that shape their interaction with billions of people (and have profound effects).
We can take both questions seriously – and have a responsibility to do so.
My dad was one of the religious leaders invited to Anthropic's closed-door sessions that the NYT just wrote an exposé on.
His NDA was just lifted so his take and more ---->