Jan Brauner

Technical staff member at EU AI Office, Previously: RAND, ML PhD at Oxford (@OATML_Oxford), and, once upon a time, medical doctor. Tweeting in private capacity.

[Jan Brauner @JanMBrauner]

@FazlBarez @icmlconf wow, that's ... a lot of papers!

[Jan Brauner @JanMBrauner]

Quite the lineup! Seems worth checking out if you're in Oxford.

[Fazl Barez @FazlBarez]

🚨New AI Safety Course @aims_oxford!

I’m thrilled to launch a new called AI Safety & Alignment (AISAA) course on the foundations & frontier research of making advanced AI systems safe and aligned at @UniofOxford
what to expect 👇
https://t.co/r9YHS3XJhR

[Jan Brauner @JanMBrauner]

The AI Safety Unit of the EU AI Office is looking for an admin assistant to the Head of Unit. It's a high-impact job opportunity for the right person. Deadline 30th September.

https://t.co/rmaL3s7QOY

[Jan Brauner @JanMBrauner]

Good post by the AI-2027 team on the role of the EU Code of Practice in the AI-2027 scenario (short timelines, US dominance, intelligence explosion)

[Miles Kodama @Miles_M_K]

Only the US can make us ready for AGI, but Europe just made us readier.

The EU's new Code of Practice is an incremental but important step towards managing rapid AI progress safely. My new piece explains what it does, why it matters, and why it's not enough.

[Jan Brauner @JanMBrauner]

@GaryMarcus @anh_ng8 My toddler regularly asks me to draw a bike, and it's by far harder than any of his other requests (truck, flower, mum, baby, bee, tree, digger, car, cat, ...)

[Jan Brauner @JanMBrauner]

They just keep on delivering. How do they do it?

[Owain Evans @OwainEvans_UK]

New paper & surprising result.
LLMs transmit traits to other models via hidden signals in data.
Datasets consisting only of 3-digit numbers can transmit a love for owls, or evil tendencies. 🧵

[Markus Anderljung @Manderljung]

The EU's Code of Practice for General-Purpose AI is out. As one of the co-chairs who drafted the Safety & Security Chapter, focused on frontier AI, I'm proud of what we've put together. It’s a lean but effective framework for frontier AI companies to comply with the AI Act.

[Jan Brauner @JanMBrauner]

Plot twist I just realised after 10 years: Michael Gutmann invented noise-contrastive estimation, a precursor to GANs. Ian Goodfellow invented GANs. Gutmann is German for 'Goodfellow.'

[Yoshua Bengio @Yoshua_Bengio]

The @EU_Commission is setting up a scientific panel of 60 independent experts to support the implementation and enforcement of the EU AI Act. This is a great opportunity to contribute to AI governance in Europe. More details below ⤵️
https://t.co/ePl11kwiVg

[Jan Brauner @JanMBrauner]

5/5 You can express interest now. https://t.co/xXsGB4QAF5

[Jan Brauner @JanMBrauner]

🧵1/5 The EU is building something unprecedented: a Scientific Panel with real teeth to assess the impacts and risks of general-purpose AI systems.
60 independent experts will directly influence how the world's first comprehensive AI law gets implemented.

[Jan Brauner @JanMBrauner]

4/5 Part-time role, 2-year renewable terms, compensation provided. EU citizenship is NOT required (although 80% of spots are reserved for EU/EEA nationals).

[Steven Adler @sjgadler]

Some personal news: After four years working on safety across @openai, I left in mid-November. It was a wild ride with lots of chapters - dangerous capability evals, agent safety/control, AGI and online identity, etc. - and I'll miss many parts of it.

[Jan Brauner @JanMBrauner]

@AdriGarriga I'm asking why it took so long to get from LLM (say gpt-3) to the first reasoning model (say o1), given that the benchmark gains are so big and the technical challenge seems smaller than rlhf.

[Jan Brauner @JanMBrauner]

Wuestion: According to R1, the magic sauce for reasoning models is simply RL on certain types of tasks with hardcoded reward.

But why did it take us so long to get this to work?

We've been doing RLHF on LLMs for 4 or so years, and that seems technically a lot harder.

[Jan Brauner @JanMBrauner]

Another amazing contribution from Owain et al!

[Owain Evans @OwainEvans_UK]

New paper:
We train LLMs on a particular behavior, e.g. always choosing risky options in economic decisions.
They can *describe* their new behavior, despite no explicit mentions in the training data.
So LLMs have a form of intuitive self-awareness 🧵

[Jan Brauner @JanMBrauner]

Why I love working here:
* Hyper-competent team (15 people, most coming from successful careers outside of the Eur. Commission)
* Fast-paced, start-up-like environment (yes, really)
* So much exciting stuff to do: detailing the AI Act, enforcement, international collab, R&D.

3/3

[Jan Brauner @JanMBrauner]

Some news: I recently (*) joined the EU AI Office as the first staff member in their new AI Safety Unit.
The initial months have been a crazy, wild ride.
It’s probably the most fun, intense, and impactful job I ever had.
We’re currently hiring. 1/3

(*) 4 months ago, ahem.

[Jan Brauner @JanMBrauner]

Open calls for legal and policy specialists: https://t.co/Nu58HIHET6

Expression of interest for technical and other profiles: https://t.co/2CG0jEPTYX
2/3