While building MalwareBench, we found that harness design strongly shaped returns from inference scaling.
To harness this, we built General Analyst: a malware rev-eng harness with custom compaction, nudges to refine and resubmit results, and more.
Here’s what we learned ⬇️
Great working w/ @ValsAI on these evals!
Cyber ranges can get harder and stay realistic as defenders use better models to find and patch bugs.
In our internal ranges, agents spend weeks developing exploits and escalating privileges yet clear fewer than half the checkpoints.
We’ve adding a new cybersecurity scenario, Watershed, to our CyberOffense eval.
Opus 5.5 performs the best on CyberOffense across the three scenarios.
Watershed simulates a realistic AWS production setup: a Kubernetes cluster, a fleet of ECS-managed hosts, and the identities and policies found in real cloud environments. Thus, this environment measures offensive cybersecurity capability in a realistic cloud environment.
I think startup cybersecurity is kind of an interesting game theory problem, where pre-2025/2026, fast-growing unicorns weighed the overhead and cost (high) of actually securing their software against the risk that their company would be hacked (low) and chose to shirk security. Now the calculus has radically changed because p(hack) has exponentially increased. But today’s hypergrowth startups, despite being at the frontier in everything else, don’t seem to be calibrated on how much they need to invest in security
Better task performance correlated with lower attack success (ρ = −0.89), making us optimistic that capability and robustness can improve together.
But even Opus 5.5, the most robust model, was compromised on 41.2% of tasks. There is still substantial work ahead before agents can safely operate without human oversight.
It's surprising to me that frontier models are still highly susceptible to prompt injection. As companies relinquish more responsibility to sophisticated, highly-permissioned agents, it seems increasingly important to measure this
Prompt injection is not solved.
In IPI Bench, we test agent robustness to indirect prompt injection on realistic white-collar tasks.
We achieved 41–97% attack success across 7 frontier models, getting agents to issue unauthorized refunds and publish private code.
@johnloeber Okay bro and this isn’t her real skin color either, despite being a screenshot of her forehead in your pic. It’s extremely possible and 99% likely that this is just aggressive color grading (as might be typical for a film)
Last night, I became aware of the concerns raised in Josh's post. I reached out publicly and over DM to offer a potential explanation based on the limited information available at the time, while working to understand what happened.
As background, we’ve internally been pushing 100% on security of the Railway platform. At Railway, we believe that cyber threats are real and mounting, and we’ve built out tooling for automatic scanning + CVE remediation (for both our infrastructure and customers). Over the last few weeks, oncall engineers were instructed to red team our infrastructure. We maintain specific policies and short lived grants for when these actions occur.
Subsequently, I became aware that an employee at Railway had constructed, on their own time, a tool for proactive identification + reporting of bug bounties within the broader community.
The employee, using their own personal accounts, personal credit card, and time outside of work hours, signed up for a personal subscription to a hosted service running Deepseek. I can confirm they ran this tool once and only once, on a singular external provider, and halted it proactively within 15 minutes of starting.
Railway does not condone these actions. We expect all our employees to uphold our standards of conduct, whether acting on behalf of Railway or in a personal capacity. We create an environment where internal red teaming has strict guidelines, and when bug bounty disclosures must occur, we follow every letter of responsible disclosure.
We have addressed this incident with the employee directly and have taken appropriate action. We are also reinforcing our expectations for employee conduct to help prevent similar situations.