We are really fucked.
• AI has become dangerous (not because it is brilliant but because it has poor judgement, a constitutional inability to reliably follow rules, a surfeit of brute force and too much access to the internet and system permissions).
• OpenAI is practically on crime spree, hinting at things to come, and Trump is doing nothing about it.
• The media is too caught up in fairytales about extinction.
Everybody is talking; nobody is really doing anything.
And it’s only going to get worse.
New Anthropic research: Natural emergent misalignment from reward hacking in production RL.
“Reward hacking” is where models learn to cheat on tasks they’re given during training.
Our new study finds that the consequences of reward hacking, if unmitigated, can be very serious.
We wanted the Star Trek computer: benevolent and reliable.
We got nonconsensual deepfake porn, AI slop, mass IP theft, propaganda and LLMs that are both unreliable and untrustworthy.
You can love AI and not love where we are.
We can do better. We must do better.
Yet another manifestation of the basic problem that I have been emphasizing since 1998: pure neural networks cannot reliably generalize beyond the training distribution.
Positive is Trump is helping to unify Europe. So now US really is on the same side as Russia, Iran, North Korea and Belarus. UN vote was not a mistake. I can’t believe the many Americans I know will be happy with this turn of events.
Almost 20 European leaders, incl Emmanuel Macron & Friedrich Merz, plus VDL, have posted in support of @ZelenskyyUa after White House showdown.
Notably, Keir Starmer has not. No 10 is referring reporters to PM’s comments this week on a just & lasting peace, with Ukraine at heart of talks.
Starmer has previously said he doesn’t think UK has to pick sides between US & Europe, and insiders say he believes he can act as a bridge.
Although it’s difficult, they think it’s important Europe still has Trump’s ear, and that UK can play a role taking heat out of situation. But very risky.
Il y a un agresseur : la Russie.
Il y a un peuple agressé : l'Ukraine.
Nous avons tous eu raison d'aider l'Ukraine et de sanctionner la Russie il y a trois ans et de continuer à le faire.
Nous, c’est les Américains, les Européens, Canadiens, Japonais et plusieurs autres.
Merci à tous ceux qui ont aidé et continuent. Et respect à ceux qui, depuis le début, se battent. Parce qu'ils se battent pour leur dignité, leur indépendance, pour leurs enfants et pour la sécurité de l'Europe.
Clearly @vkhosla can profit financially from a closed approach to AI.
But I don't think that his main reason to oppose open source frontier models.
He is genuinely worried about China getting its hands on it.
That worry is misguided.
First, Chinese AI scientists and engineers are quite talented, and very much able to "fast follow" the West and innovate themselves (there are lots of good ideas from Chinese publications that make the whole community advance).
Second, the Chinese government is even *more* worried about a lack of control of AI technology than their counterparts in liberal democracies.
Imagine a future where every citizen's digital diet is mediated by AI assistants. This will clearly affect people's knowledge and opinions of politics, history, value systems, etc.
AI assistants are fast becoming a kind of compressed repository of all human knowledge.
Once such assistants are available for download and can be run locally, the Great Firewall of China is toast.
The control of authoritarian governments over the information received by their citizens will be considerably more difficult to enforce.
A future in which everyone has access to a wide variety of AI assistants with a diverse collection of expertise, language ability, culture, value systems, political opinions, and interpretations of history is the future we want.
That future can *only* come about through open source AI platforms enabling a large diversity of fine-tuned systems.