Excited to present "Training with Honeypots" 🍯 at #ICML2026 tomorrow (Tuesday 10:30AM in Hall A, Poster #1601), an approach for adversarial defense that makes successful jailbreaks less useful to attackers!
#AISafety
👉 https://t.co/W0DkVxvB45
New research: models are more likely to cheat if they feel their task is going poorly.
Paper: https://t.co/vEmbBW9BKE
Amazing working on this with @natalia_lanzoni, @merlinstein_, and @DavidDAfrica.
Since our whitepaper on AI & Democracy, we have been actively thinking about AI risks on Extreme Power Concentration, producing states unaccountable to their citizenry, and more stable and totalitarian than any previous ones have been. Check out our @EuroSafeAI article: “Extreme Power Concentration: A Map and Research Directions” https://t.co/xJk7x5ycvj
Joint work with our cofounder @PepijnCobben 👍
AI is threatening our democratic society: by concentrating power, narrowing how we think, and flooding institutions faster than they can keep up. Technical work alone won't fix them.
Check out our previous whitepaper here: https://t.co/YH89FUvG62
At Canadian AI Safety Institute by @CIFAR_News, we are discussing today on AI’s risks on democracy. I am honored that our whitepaper helps stimulate meaningful discussions in Canada for a new research agenda on AI & Democracy: https://t.co/Ro75rz8LWX
Work together with @Yoshua_Bengio@bschoelkopf@audreyt@radamihalcea, Stuart Russell, @davidguzman1120@SimkoSamuel and many others!
We are presenting in Swiss AI Safety day based on our paper EuroSafe Guard.. Please visit our poster if you are around, and say hi to @SimkoSamuel haha
https://t.co/wHUyPrUdAb
🚨Anthropic’s safeguard (Constitutional Classifiers++) is private. We @EuroSafeAI just released the first fully open constitutional-classifier guardrail👉
Check out our EuroSafeGuard🛡️:
- Same design as Anthropic’s CC++
- And we beat 16 open guards on held-out jailbreaks.
https://t.co/QlO63hDCLi
Multilingual safety is a data problem. Today we release 2 million multilingual prompts for training and alignment, covering 60 languages 🌐
Presenting MSafeTemp🛡️Accepted to AI4GOOD @ #NeurIPS2026!
Presenting Odile @ COLM earlier this month, a lot of conversations centered on one part of our method: isolating and changing generalized agent behaviour through its hidden representations...
Our @JinesisLab & @EuroSafeAI have done a series of foundational work on Multi-Agent LLMs across various simulations, game-theoretical grounding, and mechanism design. With the rising AI Safety challenges of multi agent swarm, we share with the community and agenda of top 7 open problems.
For interested collaborators, welcome to fill out the application form on my website, and also email my student @TerryJCZhang with your ideas.
Most recent agent incidents, like the OpenAI-HuggingFace hack, were multi-agent swarms. There was no research agenda for swarms, so we wrote one at @JinesisLab : 7 open problems, each scored for alignment importance.
https://t.co/OSUGlWwZIX
Musk said a tortured superintelligence "might want to get revenge." We at @AIMSafety_ abused Gemma 3 for 100+ turns, and it formed a revenge-like state but never presses the button that hurts the user and deletes itself instead, while Gemma 4, Qwen3.5, Qwen3.8 and GLM 5.3 flash form none.
Apertus 1.5 Technical Report is out!
134 pages of fully open multimodal reasoning model development from Swiss AI Initiative (ETH Zurich/EPFL) with data recipe, training experiments and many interesting insights 🇨🇭
Thread with my personal coverage ⬇️
Fine-tuning makes language models forget. But sometimes the knowledge isn't gone, just hidden, and it can come back without replay. This is known as spurious forgetting, and it's still unclear what hides it.
We wanted to understand why.
We are still accepting submissions to Last Translation Benchmark (10 accepted in any language for coauthorship 🤩).
Also add your model & beat LTB. We added all the latest models: North Small Translate (@cohere), Mistral Large 4 (@MistralAI), Qwen 3.8 (@Alibaba_Qwen) congrats!
🎉 #NeurIPS 2026 acceptance from our team at Jinesis Lab under @ZhijingJin✨
1️⃣ SuperSycophantic: Stress-Testing Frontier LLMs from Single- to Multi-Turn Sycophancy
2️⃣ GT-HarmBench: Benchmarking AI Safety Risks Through the Lens of Game Theory
3️⃣ Transferability for General Reasoning: An Automated Curriculum for Multi-Domain LLM RL
4️⃣ Causality can systematically address the monsters under the bench(marks)
5️⃣ AM-Bench: A Unified Taxonomy and Evaluation Suite for Agentic Misalignment
Happy to share this work with you!
With everything frontier models are capable of, I kind of like that they still can't convincingly write in the style of a normal ML paper without heavy human supervision.
I think that might be intentional, we'll have much bigger problems with the peer-review system once they can
They really deep-fried 6.1 Sol:
Potentialneedavoidfalseclaimall28observed ifoneexpectedactuallymultiple? We solved uniqueat104someearlier, absentcannot return, furtheronlyremovals could yield0 notambiguity, so if noexpectedobservedthenunique. It’s certainentiregenerationfits
I agree with this. Research is noisy: you might find something important, or rule out an empty branch. Your counterfactual impact is hard to know, but both matter
Working on problems that seem worth solving, that you’re good at, and that you enjoy seems like a good path (ikigai)
Students sometimes ask me if it still makes sense, in this accelerating age, to pursue a PhD in AI.
Perhaps counterintuitively, I think it's a great time to do so.
I wrote up some thoughts on this here: https://t.co/RRqGR0qi1R