We found a way to jailbreak a closed-source model using the next-token log-probabilities returned by its API.
SAGE uses those probabilities to identify a signal associated with refusal, without access to the model’s weights or internal activations.
These are early findings from a short research sprint with @AEStudio.
We disclosed to developers of the open models tested and do not publish working attack suffixes against commercial targets.
We’re sharing this through our Early Explorations series in case it helps someone else’s work.
If you’re exploring related questions or want to discuss the findings and limitations, we’d be happy to talk.
https://t.co/FG7PM5WAq5
Can one small adapter read different kinds of hidden concepts inside an AI model?
In a quick research sprint, @AEStudio's @keenanpepper tested SelfIE across seven types of tasks. We’re sharing the early findings in case they’re useful to others.
These are preliminary sprint findings. The evaluation tested new questions within familiar task families, so performance on entirely new types of tasks remains an open question.
We’re not currently pursuing this exploration as an active research agenda.
We're #hiring for a Policy Director based in Washington, DC. If you know any top notch folks looking to make a significant impact on AI alignment policy, please spread the word: https://t.co/VHf0IwKVkD
Yesterday, @TheRegister covered our hackathon result: AI agents shared secrets they’d been explicitly told to keep private. That same day,
@ClaudeDevs announced: “API keys are saved securely, so Claude never sees them.”
Keeping keys out of reach helps. We’d also like agents to keep secrets when they do have them.
At AIAF, we research how to make systems stay aligned as they become more capable.