Thank you to everyone who has reached out to me over the past weeks! :)
I am excited to join the Open Alignment Team @huggingface, thanks @Thom_Wolf.
My initial focus will be on agent swarms (what a surprise!), their behaviour, how to monitor them and their failure modes🤗
Unfortunately, I didn’t get into MATS, and my internship is ending, so I’m looking for research opportunities in AI safety and alignment.
Interested in instrumental convergence, multi-agent failure modes, monitorability & loss of control.
Any leads appreciated :)
ig its time to share while my feed is all about how good the Neon is
1) I joined Google DeepMind as a Research Scientist to work on Gemini's safety
2) I will be presenting at @NeurIPSConf in Sydney -- let's chat!!! (but pls send an email!)
Noch ist der Zug nicht abgefahren. Europa braucht eigene Frontier KI Modelle. Sonst werden wir uns nie mehr aus der Abhängigkeit von China und den USA befreien können. Es ist naiv zu glauben, dass Top KI für uns frei zugänglich bleibt. https://t.co/omM54Yhs7u
Inkling 2 Mini is a looped transformer that runs on a laptop.
Looping lets a model think more and write less. At this scale, its thought process remains monitorable; this may not stay the case. We’re also sharing a variable-loop research model to enable further study.
Some of you thought that Inkling 2 Small was too big at 594B parameters. We heard you: Inkling 2 Mini is a looped transformer with just 26B parameters, small enough to run on your laptop, powerful enough for agentic work with a score of 20.1 on AA Index.
https://t.co/9HzaUMwghb
Distillation can reproduce powerful AI capabilities without the safeguards built around them.
We’re sharing new research on our work to combat distillation of our models, how these capabilities transfer, and how we detect and disrupt coordinated attempts.
These findings are helping us sharpen our defenses and support safeguards across the industry.
https://t.co/FGQ0PHllIO
I look forward to reading hot takes the proof is not actually all that impressive, it's all just a re-hash of old ideas and that maybe Claude read someone else's chat and stole the proof
Tencent’s Hy4 system card is out and includes our independent pre-release safety evaluation. It’s the first system card from a Chinese frontier lab, and a meaningful step for third-party evaluation in China.
We tested loss-of-control and harmful-manipulation risks with four private in-house benchmarks. Compared with its preview, Hy4 shows overall safety improvements across both risk areas. We also found some complex agentic scenarios still leave room for further improvement, e.g., multi-agent resource competition.
Thanks to the Tencent Hy team for their trust and openness. We look forward to working with more frontier labs.
Thank you to everyone who has reached out to me over the past weeks! :)
I am excited to join the Open Alignment Team @huggingface, thanks @Thom_Wolf.
My initial focus will be on agent swarms (what a surprise!), their behaviour, how to monitor them and their failure modes🤗
Unfortunately, I didn’t get into MATS, and my internship is ending, so I’m looking for research opportunities in AI safety and alignment.
Interested in instrumental convergence, multi-agent failure modes, monitorability & loss of control.
Any leads appreciated :)