Good morning from Budapest! The @NLP4PosImpact workshop is happening today 9am-5:30pm in room G7-8 at #EMNLP2026, with Zoom available. In the opening I introduced our research agenda of @EuroSafeAI: focusing on multi-agent swarm eval (https://t.co/yriNZ1I7bR) and extreme power concentration/risks to democracy (https://t.co/Ro75rz8LWX).
Full program: https://t.co/MsKgCT2Dlj
I'm grateful to @SimkoSamuel for mentoring and working on Odile with me, and to @ZhijingJin and Bernhard Schölkopf for their contributions to the paper. And a huge thank you to Zhijing for making it possible for me to present the work at COLM!
https://t.co/rTnu6bYln2
Presenting Odile @ COLM earlier this month, a lot of conversations centered on one part of our method: isolating and changing generalized agent behaviour through its hidden representations...
Now spending most of my time training agents at Jinesis AI Lab, I've been revisiting this approach as an alternative to elaborate RL environments and large-scale SFT. We’re experimenting with hand-crafted trajectory pairs to see how precisely we can steer behaviour from limited feedback, particularly in our medical LLM work, where continual-learning signals are sparse and targeted corrections are difficult to express as RL rewards.
I had largely moved on from this work, but the conversations at COLM made me realize there might be a much broader question hiding in one of its results :)
Our @JinesisLab & @EuroSafeAI have done a series of foundational work on Multi-Agent LLMs across various simulations, game-theoretical grounding, and mechanism design. With the rising AI Safety challenges of multi agent swarm, we share with the community and agenda of top 7 open problems.
For interested collaborators, welcome to fill out the application form on my website, and also email my student @TerryJCZhang with your ideas.
Our Jinesis Lab at the University of Toronto is bringing three exciting papers to #COLM2026! 🎉
These projects push the frontiers of #AISafety, #AgentSecurity, and #AIForScience.🤖
Huge congratulations to all collaborators and co-authors!
Papers:
🛡️ODILE: Orthogonal Disruption of Injected Tool-Call Embeddings for Agentic Prompt Injection Defense
🔭Stargazer: A Scalable Model-fitting Benchmark Environment for AI Agents under Astrophysical Constraints
⚠️One Word at a Time: Incremental Completion Decomposition Breaks LLM Safety
@UofT,@VectorInst,@EuroSafeAI,@ETH_en,@MPI_IS,@UMich