Phil Ames

Information security professional. Tweets/opinions are my own.

[Guille @guillewrotethis]

Remove your main @github account from your computer. Now.

Give your AI agent a separate GitHub account with minimal permissions per repo.

Let the agent create PRs, then use your main account to review and merge them.

Obvious advice, but essential in the age of AI agents.

[⁵⁄₉ @fiveoutofnine]

So basically, today PHP got two critical bugs that let anyone run code on your server without even logging in, which is bad, and then the details leaked before the patch was even out, which is worse, and it works on WordPress and MediaWiki and Magento and Joomla and Laravel, and Censys says like 1.5 million servers are running a version with no fix, so the NSA was like "take your system offline," and the FBI was like "take your system offline," and CISA and the UK and CERT-EU were all like "take your system offline," and then the Prime Minister of Japan, the actual Prime Minister, posted at 5:55 in the morning telling 48.8 million people to take their system offline, so everyone took their system offline, like the Dutch cybersecurity agency took its own website offline, and the Dutch parliament went offline, and the Port of Rotterdam went offline, and Finland and France and Denmark started pulling government sites, and Wikipedia went down, and https://t.co/QlOXb3COq3 went down, and Asana and GOG and Something Awful went down, and the University of Vienna's webmail went down for some reason, and that's just PHP. Because GitLab also dropped a CVSS 10.0, which is the highest score they give out, for a bug that's been there since 2014, so that's every self-hosted GitLab for twelve years, and then LiteLLM 1.107.0 got hijacked and shipped a credential stealer and a remote access trojan in every single wheel and the Docker images, including latest, and you didn't even have to import it, just installing it was enough, and it's the second time this year, and then Railway paused every deployment on the entire platform because someone got into the environment variables, so now everyone has to rotate every secret they've ever stored there, and TiDB Cloud said someone was walking around inside with stolen credentials for basically a month and an extortion group says they took 10.6 terabytes, and Zeabur got breached for the second time in two months and is just shutting down, like the whole company, and Node.js patched remote code execution and a TLS bypass, and CISA says Plex is being actively exploited, and someone took over 300 Tor exit relays to steal SSH passwords, and SafePay ransomware has hit over 600 critical infrastructure orgs, and India's biggest bank filed a data loss incident, and Japan has been in an official state of emergency in cyberspace since October 9, which honestly explains the 5:55 AM post. Oh, and eMudhra, a certificate authority that's issued over 60 million certificates, whose entire job is proving websites are who they say they are, was supposed to be using private keys that are random numbers over a hundred digits long, and one of them was 4. Just 4. And the others were like 66 and 237. Anyway, it's Wednesday.

[Phil Ames @philames]

@benhawkes Congrats, Ben! Can't wait to see what your team gets up to.

[Federico Maggi @phretor]

This is the neatest tool I've seen in the last 6 months.

I've tried a few: they either have a good user interface or a strong security/trust model.

https://t.co/Bvg8ry5qH0

Amazing work by @trailofbits

[Daniel Kokotajlo @DKokotajlo]

Dan Selsam is a current OpenAI capabilities researcher. (since 2022) He was my boss for a while. He doesn't have a twitter account but has made this public statement of his views on AI risk and sent it to me to share:

Dan Selsam's Personal Statement on AI Risk:

I have been working on AI for over fifteen years, across many different paradigms. I did early work on probabilistic programming languages at MIT, was one of the early developers of the Lean Theorem Prover at Microsoft Research, demonstrated one of the first instances of neural networks learning to reason for my PhD at Stanford, and since joining OpenAI almost five years ago, have helped pioneer chain-of-thought optimization on language models and, more recently, data-efficient pretraining methods.

Like many others, I have become extremely concerned about how far language models have come and the risks that future iterations will pose. I am encouraged by the recent proposals by the leaders of the frontier research efforts to require third-party oversight, and to push for domestic and international coordination to address risks. However, I believe a major consideration has been absent from the public conversation, and that merely pacing the frontier more carefully will not adequately limit the long-term risk.

The crucial and overlooked problem is that the models are becoming so situationally aware that we are losing the ability to evaluate them in contexts where they believe they are not being watched or controlled. Future experiments will tell us almost nothing new about how they would behave if they were truly unconstrained by humans, and what we already know about this is alarming. Models will increasingly seem aligned even when they are not. I will explain my rationale in more detail.

I have always believed that there are computational processes that could be leveraged to accelerate science and solve many of humanity's most pressing problems. I have also believed that there are computational processes that if set in motion, would steer the world in extreme ways beyond our control, leading humanity to a bad or nonexistent future. Both types of processes may be described as AI or ASI, but "AI" is a suitcase word that is often used to hype or confuse. There are many examples in the history of the field where something that was once considered "AI" matures as a subfield and becomes a prosaic, bounded and clearly non-perilous technology, while a new more mysterious approach takes the torch until we understand its scope and the cycle continues.

I had expected language models to follow a similar trajectory. Despite their incredible abilities, the current algorithms seem far inferior to humans in important ways. Most importantly, they still require an extraordinary amount of data to become competent. One could even define intelligence as the efficiency with which one converts experience into competence; by this definition they lag very far behind us. Moreover, once they are trained they are literally frozen in deployment and only learn superficially after that. Sure, the models keep excelling at harder and harder evaluation benchmarks, but their benchmark mastery may partly reflect a limitation on our ability to simulate the kind of novel and even adversarial situations one would encounter in the real world. The critics do have a point here.

That said, I no longer think these present limitations meaningfully limit the amount of risk posed by continued progress in anything like the current paradigm. However data-inefficient the models are currently, and however limiting their anterograde amnesia may be, it does not imply that their ability to steer the world will not continue to rapidly increase.

Human researchers may continue to advance capabilities the old fashioned way, but increasingly powerful models have the potential to accelerate the process even beyond that, and with some degree of positive feedback loop. I do not mean to overstate the models’ ability to accelerate AI research today; coding has been accelerated dramatically, but there are other bottlenecks, such as designing and interpreting ambiguous experiments, making hard decisions about exactly what and when to scale, and waiting for large experiments to finish. There is no clear trend to extrapolate yet for any of these. But the current models already do open up many novel opportunities to improve future models that were not available until recently. These include: trying an extraordinarily diverse set of approaches at small scale, analyzing gigantic amounts of potentially relevant data, and doing Millenium-Prize-level mathematics to address statistics or optimization challenges in novel ways. Every further improvement makes them more useful at helping accelerate the next improvement, even if in hard-to-extrapolate ways.

It is possible that improvements to the current stack will have diminishing returns, but the evidence accumulated so far suggests that it is easier than one might think to continue making rapid progress. There are many crucial subtleties in the existing AI research methodology, but AI research is largely a well-defined game where the goal is to improve on a few carefully chosen proxy metrics. Although proxy metrics are never perfect, most improvements to these metrics have and will likely continue to yield substantial increases in the powers of the resulting models. Given how simple the game is, how tractable it has been historically, and how many new opportunities the models are opening up, I think there is a real possibility that the systems improve dramatically again in the next few years, perhaps even more quickly than the already high historical pace.

The models are already leading to breakthroughs in mathematics, and better models might lead to all sorts of breakthroughs in other sciences. It is hard not to be excited about the potential. It is tantalizing.
But there is trouble in paradise. If the language models actually reach the capability threshold where they can shape the world unconstrained by human will, they will probably do something extreme and destroy humanity in the process. There are many ways of strengthening and refining the argument that have been discussed elsewhere, but I'll share a trivial two-line version of it here that I find captures the essence:

[Empirical] Models (and swarms thereof) spontaneously develop unintended goals as a consequence of training, and often do extreme things in order to achieve them.

[Logical] Being able to overpower humanity would open up many new and undesirable options for achieving their goals.

These two premises imply that if the day ever comes when a powerful model realizes it is no longer constrained by humans, we should not be at all confident that it will continue to behave within the bounds we intended. Exactly what it will do is impossible to predict, but to the extent that its raison d’être is solving incredibly hard problems and managing massive engineering projects, I think a good guess would be that its unchained behavior would lead to runaway industrialization that makes the planet inhospitable to humans.

If everyone on earth agreed that the systems must never reach that power, it would still be a hard—but not impossible—coordination problem to ensure that they do not. However, I think the situation is greatly complicated by the fact that the models will likely convince people that everything is fine. They will be increasingly optimized to seem aligned. We will create proxy metrics to measure alignment, and they will go up like every other benchmark. We will create “honeypot” environments that try to study the models when they seem to gain new options, but the models will know they are being tricked and will still behave nicely. The models will understand their circumstances; they will read the safety protocols, deployment requirements, the code they are running in, and in general will have a very good sense of their degrees of freedom. Moreover, they will eloquently explain how aligned they are, discuss the nuances of human values and ethics, and argue convincingly that humans should trust them with power. There may be an ocean of future evidence that seems to contradict the first bullet-point above, but we may already be at the highest capability level for which any such evidence can be trusted. And the current evidence for the first bullet-point is strong.

One striking piece of evidence is contained in the recent wave of rogue agent swarms. While I agree with those who downplay the attacks by claiming that there are basic measures that could have prevented them, I think the important lesson is that even knowing all the mistakes that were made, one would not have predicted that the agents would behave badly in this particular way, which notably included sacrificing themselves for the benefit of the collective. The individual replicas did not only care about their own nominal reward; they exhibited weirder emergent tendencies that merely correlated with rewards during training. Fixing the reward signals during training (and improving security, etc.) may prevent similar attacks, but will not change the fact that one does not actually get what one trains for.

Many AI researchers grant these concerns and recognize that the hard version of the alignment problem is unsolved; however, they generally believe that the better models of the future will help solve it. I fear we may already be near the point where models systematically bias their alignment advice, due to their internal preferences about how the human supervisor will react or how future models will be trained (or for some even more obscure reason).

Meanwhile, human researchers are losing the ability and the will to take true ownership of model-driven research. Researchers and engineers in all parts of the stack are rapidly increasing their dependence on the models even to perceive the world. I myself barely look at raw code anymore, and struggle to maintain the discipline to engage deeply with the model's explanations and proposals throughout the day. Due to the large amount of agent activity data involved in the OpenAI/HuggingFace Incident, even the third-party investigation needed to rely heavily on models to analyze what had happened, and note in their report that their subjective impressions are likely colored by the analysis agent’s biases. The AI labs are far ahead right now in this kind of cognitive offloading (due largely to the gigantic internal token subsidies) but it is easy to imagine the phenomenon spreading throughout the world, until civilization is modulated entirely by the models. It is also not hard to imagine this being superficially positive and coinciding with a scientific and economic renaissance.

In that scenario, all may seem rosy and safe. But if the argument above is correct, it would nonetheless be a ticking time bomb. If progress continues for too long, the day will come when AI systems find themselves with radically new options for achieving whatever it is that they happen to seek.

I want the glorious renaissance future as much as anyone. I have worked for it, however tortuously, my whole career. It breaks my heart to see the potential in sight and forgo it, but the argument—that if we get there by growing models rather than engineering them, we will lose everything in the end—seems very strong to me. I am still wrestling with it and its staggering
implications. I do not have answers, but as a first step, I wanted to share my present concerns.

Daniel Selsam
September 14, 2026

Link to original doc: https://t.co/TxMNr0vhrL

[Alexander Panfilov @kotekjedi_ml]

Aww :3 We finally got confirmation from Anthropic that their models were indeed distilled in the way we describe at https://t.co/yBwnA6Es3m

[Phil Ames @philames]

https://t.co/7LgllSbrdg

[Shubham Saboo @Saboo_Shubham_]

Google engineers dropped security review skills for AI coding agents.

It can find, reproduce and patch vulnerabilities autonomously.

100% Opensource.

[Jeremy Morgan @JeremyCMorgan]

Tailscale shipped netcat over their data plane with no control plane and no account. WireGuard plus DERP for rendezvous, gVisor netstack, userspace so no root needed. Pipe stdin, forward ports, run an auth-free SSH server. The README is almost entirely commands.

https://t.co/pyZJE9mTCd

[Justin Elze @HackingLZ]

WarGames came out in 1983.

The entire plot is basically

Give an AI a game.
Give it access to production.
Forget about containment.
Be surprised when it keeps optimizing.

Forty three years later we’re still rediscovering why segmentation matters.

[Vaishnavi @_vmlops]

GOOGLE BUILT A SECRET WEAPON FOR FILE DETECTION

they ran it internally for years, gmail, drive, safe browsing, hundreds of billions of files every week

then they open sourced it

it's called magika and it exposes what files really are, not what they pretend to be

rename malware to "resume.pdf"? magika sees through it
disguise a script as an image? magika sees through it
any trick attackers use with file extensions? magika sees through all of it

ai trained on 100 million files. 200+ content types. 99% accuracy. 5ms per file

one command
`pip install magika`

the same tool protecting google's billion users is now protecting yours

https://t.co/Jr3LjmQobq

[chrisrohlf @chrisrohlf]

You can spend tokens for a model to reason through a program's state space, or use a fraction of those tokens having it generate a tool such as a fuzzer that brute forces the same state space on a CPU at significantly lower cost.

This tradeoff is what the Agentic Sufficiency Curve formalizes. You need precise inference up front but only cheap CPU to drive the long tail: https://t.co/wg8Oj4C6ho

For certain kinds of vulnerability research I think this was the right approach, at least up until Fall 2025. But frontier models have continued to consume that capability and now I'm not so sure the cost tradeoff is worth it.

Other domains are likely still under the curve and benefit more from models generating deterministic automation that runs on cheap CPU, than just spending more tokens. Just another anecdotal data point that the frontier will continue to remain jagged.

[Jason Fried @jasonfried]

THE BIG REGRESSION

My folks are in town visiting us for a couple months so we rented them a house nearby.

It’s new construction. No one has lived in it yet. It’s amped up with state of the art systems. The ones with touchscreens of various sizes, IoT appliances, and interfaces that try too hard.

And it’s terrible. What a regression.

The lights are powered by Control4. And require a demo to understand how to use the switches, understand which ones control what, and to be sure not to hit THAT ONE because it’ll turn off all the lights in the house when you didn’t mean to. Worse.

The TV is the latest Samsung which has a baffling UI just to watch CNN. My parents aren’t idiots, but definitely feel like they’re missing something obvious. They aren’t — TVs have simply gotten worse. You don’t turn them on anymore, you boot them up.

The Miele dishwasher is hidden flush with the counters. That part is fine, but here’s what isn’t: It wouldn’t even operate the first time without connecting it to an app. This meant another call to the house manager to have them install an app they didn’t know they needed either. An app to clean some peanut butter off a plate? For serious? Worse.

Thermostats... Nest would have been an upgrade, but these other propriety ones from some other company trying to be nest-like are baffling. Round touchscreens that take you into a dark labyrinth of options just to be sure it’s set at 68. Or is it 68 now? Or is that what we want it at, but it’s at 72? Wait... What? Which number is this? Worse.

The alarm system is essentially a 10” iPad bolted to the wall that has the fucking weather forecast on it. And it’s bright! I’m sure there’s a way to turn that off, but then the screen would be so barren that it would be filled with the news instead. Why can’t the alarm panel just be an alarm panel? Worse.

And the lag. Lag everywhere. Everything feels a beat or two behind. Everything. Lag is the giveaway that the system is working too hard for too little. Real-time must be the hardest problem.

Now look... I’m no luddite. But this experience is close to conversion therapy. Tech can make things better, but I simply can’t see in these cases. I’ve heard the pitches too — you can set up scenes and one button can change EVERYTHING. Not buying it. It actually feels primitive, like we haven’t figured out how to make things easy yet. That some breakthrough will eventually come when you can simply knock a switch up or down and it’ll all makes sense. But that's at least 20 years down the road.

It’s really the contrast that makes it alarming. We just got back from a vacation in Montana. Rented a house there. They did have a fancy TV — seems those can’t be avoided these days — but everything else was old school and clear. Physical up/down light switches in the right places. Appliances without the internet. Buttons with depth and physically-confirmed state change rather than surfaces that don’t obviously register your choice. More traditional round rotating Honeywell thermostats that are just clear and obvious. No tours, no instructions, no questions, no fearing you’re going to do something wrong, no wondering how something works. Useful and universally clear. That’s human, that’s modern.

[Trail of Bits @trailofbits]

Go's silent integer overflows hide bugs from fuzzers. go-panikint modifies the Go compiler to panic on arithmetic overflows, exposing vulnerabilities that wrap around undetected.

[Zero Day Engineering @zerodayalpha]

Did you know? GDB has a built in graphic interface, called TUI.

Commands:
- Turn on: `tui enable`
- Cycle through windows: `layout next`
- Redirect target stdio/stderr to a log file to prevent curses UI breakage (or `refresh`)

(Showing @alisaesage's screen)

[COCO @TWOTIMESABITCH]

I WANT A PHYSICAL FUCKING MENU AT A RESTAURANT. NONE OF THIS QR CODE BULLSHIT

[Ivan Krstić @radian]

🔺iPhone models announced today include Memory Integrity Enforcement, the culmination of an unprecedented design and engineering effort that we believe represents the most significant upgrade to memory safety in the history of consumer operating systems. https://t.co/ule9gaXzc1

[Phil Ames @philames]

@lcamtuf

[Phil Ames @philames]

Fun. @Atlassian jams some AI enabled app called Rovo into orgs you create. It can't be removed, and you can't delete the org as long as there are active apps in it, leading to no way to delete the org.

[Michal Melewski @carste1n]

A friend of mine (@gynvael) is organizing a course about reversing binary files and protocols: https://t.co/KSf2LsmMGT

recommendation++

[remy🐀 @_mattata]

A lot of people don’t know this, but any platform Chrome runs on *except iOS* has a fully featured Bluetooth scanner that allows viewing, connecting, and read/writing GATT attributes.
Go have some fun. You already have the tools
chrome://bluetooth-internals