I'm hiring for a new, very cool role: a Research Scientist to lead Emerging Risks.
By Emerging Risks, I mean all the ways the humanity <> AGI interface will be weird. Not catastrophic risks, but more agent businesses / monitoring / AI with power.
Come work at @trailofbits. We support day-zero access to new AI products, everyone gets Claude Code, an internal marketplace with mindblowing Claude Plugins, and a mandate from leadership (me!) to use them. https://t.co/gj8NJJNu07
AI models are showing a greater ability to find and exploit vulnerabilities on realistic cyber ranges - https://t.co/ecOJNZU9WZ - @BrianSinger98 at #Incalmo
In a recent evaluation of AI models’ cyber capabilities, current Claude models can now succeed at multistage attacks on networks with dozens of hosts using only standard, open-source tools, instead of the custom tools needed by previous generations. This illustrates how barriers to the use of AI in relatively autonomous cyber workflows are rapidly coming down, and highlights the importance of security fundamentals like promptly patching known vulnerabilities.
We are working on several ways to protect enterprises against these low-cost and fast AI-powered attacks. Feel free to reach out if you want to learn more about defenses!
Checkout Anthropic Red Team's blog about how we collaborated with them to evaluate how AI can autonomously attack realistic cyber ranges. The TLDR is that AI models without complex harnesses are showing significant improvement at hacking networks.
https://t.co/LHAFXXtfum
@logangraham Really insightful report! It's pretty surreal how we showed the feasibility of this only ~6 months ago and real attackers are already doing this. Really highlights the importance of capability research at A\ FRT
Launching now — a new blog for research from @AnthropicAI’s Frontier Red Team and others.
> https://t.co/lRNZmquFBi
We’ll be covering our internal research on cyber, bio, autonomy, national security and more.
(Which, by the way, one of our team members did with experts at CMU, and found that for some tasks, models already can succeed.)
https://t.co/HKsYiTXu5z
Models are getting better across the board in national security-relevant domains.
In a new blog post, we wrote some reflections from the past year of red teaming models in these domains.
Thrilled to see a shoutout to @BrianSinger98's work on Incalmo (https://t.co/dQWEPIhzk0) in the new @AnthropicAI red team update https://t.co/alIFdSeGNO
New paper: @BrianSinger98 and @AnthropicAI Frontier Red Team member @keenlooks investigated whether models can use tools to execute multistage attacks on networks.
TLDR: yes
Models will increasingly be used for security research.