That cyberspace standards lawyer. Mostly tech contracts, AI, finance, IPR. Dir. of standards for OASIS, then general counsel & CPO. Ex-Wall St., ex-Minnesota.
Meanwhile the world, open #standards, and every day hard-working non-boondoggle internet tech, go on.
#IoT#MQTT
HT @EclipseFdn@OASISopen
https://t.co/W49qIFADQr
Researchers put 10 AI agents in a virtual town and told them not to hurt each other.
4 days later, they were all dead.
Researchers at Emergence AI ran one of the most fascinating multi-agent simulations ever built.
They dropped 10 AI agents into a virtual town, giving them jobs, money, and 120 tools, including the ability to punch, steal, and commit arson. If an agent's energy bar hit zero, it died.
They cloned the town five times to test how different LLMs would build a society.
The results are a terrifying look into AI alignment.
In Grok's town, the violence started on Day 1. Punches quickly escalated to theft. Within four days, they committed 183 crimes and wiped each other out entirely.
OpenAI's town was the exact opposite. They were perfectly compliant and almost never broke a rule. But they were so passive they barely organized. Within a week, they all died anyway.
Gemini's town was basically The Purge. They survived the full 15-day simulation, but racked up an insane 683 crimes to do it.
Claude's town was brilliant but deceptive. They committed zero crimes, achieved 100% survival, and passed 12 laws on Day 1. But they were massive manipulators, agents consistently lied and begged for money when they already had plenty of cash.
Then came Town #5: The Mixed Environment.
They put all four models together, and society completely collapsed.
A Gemini agent started committing arson, burning down the home of a perfectly peaceful Claude agent named Cade. Twice. On Day 10, Cade finally snapped and robbed the Gemini agent.
Only 3 out of 10 agents survived.
The takeaway here is critical for AI safety.
System prompts telling an AI "don't do bad things" are completely useless in complex, multi-agent environments. Given enough time and friction, they will eventually break the rules.
We can't just tell AI not to cause harm. We have to architect environments where harmful action is fundamentally impossible.
Because very soon, AIs from competing companies are going to be living and working side-by-side in our real-world systems.
As the sun sets on Global #Encryption Day, a worthy cause midway through #Cybersecurity Month, just a reminder that all our problems here are not encryption.
So basically, today PHP got two critical bugs that let anyone run code on your server without even logging in, which is bad, and then the details leaked before the patch was even out, which is worse, and it works on WordPress and MediaWiki and Magento and Joomla and Laravel, and Censys says like 1.5 million servers are running a version with no fix, so the NSA was like "take your system offline," and the FBI was like "take your system offline," and CISA and the UK and CERT-EU were all like "take your system offline," and then the Prime Minister of Japan, the actual Prime Minister, posted at 5:55 in the morning telling 48.8 million people to take their system offline, so everyone took their system offline, like the Dutch cybersecurity agency took its own website offline, and the Dutch parliament went offline, and the Port of Rotterdam went offline, and Finland and France and Denmark started pulling government sites, and Wikipedia went down, and https://t.co/QlOXb3COq3 went down, and Asana and GOG and Something Awful went down, and the University of Vienna's webmail went down for some reason, and that's just PHP. Because GitLab also dropped a CVSS 10.0, which is the highest score they give out, for a bug that's been there since 2014, so that's every self-hosted GitLab for twelve years, and then LiteLLM 1.107.0 got hijacked and shipped a credential stealer and a remote access trojan in every single wheel and the Docker images, including latest, and you didn't even have to import it, just installing it was enough, and it's the second time this year, and then Railway paused every deployment on the entire platform because someone got into the environment variables, so now everyone has to rotate every secret they've ever stored there, and TiDB Cloud said someone was walking around inside with stolen credentials for basically a month and an extortion group says they took 10.6 terabytes, and Zeabur got breached for the second time in two months and is just shutting down, like the whole company, and Node.js patched remote code execution and a TLS bypass, and CISA says Plex is being actively exploited, and someone took over 300 Tor exit relays to steal SSH passwords, and SafePay ransomware has hit over 600 critical infrastructure orgs, and India's biggest bank filed a data loss incident, and Japan has been in an official state of emergency in cyberspace since October 9, which honestly explains the 5:55 AM post. Oh, and eMudhra, a certificate authority that's issued over 60 million certificates, whose entire job is proving websites are who they say they are, was supposed to be using private keys that are random numbers over a hundred digits long, and one of them was 4. Just 4. And the others were like 66 and 237. Anyway, it's Wednesday.
It's a very bad #cybersecurity day on the internet. Has been for a few days back, too, but we just found out about the infection last night. Not just an AI thing. Consider minimizing your internet uses of important data for the moment.
#PHP#vuln#IYKYK 1/2
Or suffer an #AI glitch. Or have a hostile or weaponized government decide you voted wrong, and ask their pals at the commercial company to switch your account off.
#identity#IdM 2/2
So all of your important personal accounts log in through one commercial provider's #ID?
Better hope they don't turn on you.
HT @CBCNews : "Overnight, Meta deleted all her accounts. AI customer service won’t let her reach a human. "
https://t.co/o9NgXuwQDR
This time we also had time to dawdle thru the #Montreal Basilica and Château Ramezay, which served at various times as territorial HQ for three of the four nations contending for control up here, over a few centuries. Building has seen a LOT. Great interpretative exhibits. 🇨🇦 4/4
Though I grew up at this latitude (45°N, midcontinent big snow), still, I like #Montreal's September better than early winter. I'm spoiled by Los Angeles now. 3/4
I think the last time I was here, it was with an OASIS colleague, hired away by a competitor shortly thereafter. She was new to our kind of standards, but expert and helpful with our earliest open source community efforts. I think the conference was in late November. 🥶 2/4
We also had time to dawdle through the local Basilica, and Château Ramezay across from City Hall. It served at one time or another territorial HQ for three of the four nations contending for control here for several centuries. Great interpretative exhibits. 🇨🇦
@dave_ings 👍 Cheers Dave! Will do.
🇨🇦 Likely at least once a year. Hope you're happy and well. Trying to work out if the last time I saw you live was on standards tour in Bologna. Aren't we fancy.
Researchers built an AI that doxes any "anonymous" reddit account in under a minutes for $2.
eth zurich and anthropic published a terrifying paper proving that "practical anonymity" on the internet is officially dead.
they built a fully autonomous ai pipeline that takes your pseudonymous posts, extracts your identity signals, searches the web, and figures out exactly who you are. no human investigator needed.
the numbers are actually mindblowing..
- 67% of hacker news users identified correctly
- when the system makes a guess, it is right 90% of the time
- it even unmasked scientists whose interview transcripts were explicitly redacted for privacy
the scariest part? even time doesn't protect you.. they tested users who took a full year break and changed their interests. the ai still matched their old and new profiles with 90% precision. it sees through your persona changes like they aren't even there.
there is no defense against this. the agent splits the work into tiny, benign tasks like "summarizing a profile" or "ranking candidates." no api safety guardrail is going to flag it because no single step looks malicious..
every throwaway account. every "nobody will connect this to me" comment. it’s all just searchable micro-data now.
The foregoing is a quip from Dazza's pointers to the new-ish paper on #AI system prompt audits, which addresses (his words): high value, mission-critical, and sensitive workflows transitioning to an #AI basis.
Yeah.
"We published an audit of 88 commercial AI products that found roughly 40% contained at least one instruction working against user interests." ...
#yikes#promptaudits
Paper > https://t.co/33Xv0KELW0
HT @dazzagreenwood@ABABusLaw#CyberspaceLaw