Matt MacDermott

AI Safety Grantmaker @ cG

[Geoffrey Irving @geoffreyirving]

The safety funding ecosystem is scaling up: our grant took six weeks from first conversation to confirmation. More capital is entering the field fast via the OpenAI Foundation and the Anthropic IPO. It's time for everyone in AI safety to be more ambitious.

[Geoffrey Irving @geoffreyirving]

We're excited to announce that Resolution has a $160M grant from Coefficient Giving: $108M unconditional, with a further $52M conditional on hiring and compute needs. We'll use it to grow teams across our research portfolio and invest heavily in research automation. 🧵

[Qiyao 📝 NeurIPS @qiyao_wei_cam]

📢 Do common RL training incentives significantly hurt chain-of-thought
(CoT) monitorability? No.

In our new paper, we ​​investigate how different training incentives influence monitorability. We find that monitorability is easier to degrade than to improve.

🧵 (1/10)

[LawZero - LoiZéro @LawZero_]

Every frontier AI system should be grounded in a core commitment: to protect human joy and endeavour. Today, we launch @LawZero_, a nonprofit dedicated to advancing safe-by-design AI. https://t.co/6VJecvaXYT

[AI Safety Papers @safe_paper]

Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path?

@Yoshua_Bengio, Michael Cohen (@Michael05156007), Damiano Fornasiere, Joumana Ghosn, Pietro Greiner, Matt MacDermott (@MattMacDermott1), Sören Mindermann (@sorenmind), Adam Oberman (@oberman_adam), Jesse Richardson (@PoliticalKiwi), Oliver Richardson
Marc-Antoine Rondeau, Pierre-Luc St-Charles, David Williams-King

@Mila_Quebec

[Matt MacDermott @MattMacDermott1]

Work done with @james_D_fox, Francesco Belardinelli, and @tom4everitt. See more work that uses causal models to help design safe AI systems at https://t.co/cjuSHFqGI4

[Matt MacDermott @MattMacDermott1]

🔦 @NeurIPS2024 spotlight paper we’re presenting today. Making AI systems more agentic is a hot research topic. But powerful agents bring worries about misalignment and loss of control. Can we measure how agentic an AI system is? 🧵

[Matt MacDermott @MattMacDermott1]

Come and chat to us about it at the poster session. 4.30-7.30pm. East Exhibit Hall A-C #3000.

🔗 Link to the full paper: https://t.co/eCNX5GyQDB

[Yoshua Bengio @Yoshua_Bengio]

As we move towards more powerful AI, it becomes urgent to better understand the risks in a mathematically rigorous and quantifiable way and use that knowledge to mitigate them. More in my latest blog entry where I describe our recent paper on that topic.
https://t.co/emiQxTvWrd

[Iason Gabriel @IasonGabriel]

How should we understand A.I. agents?

This blog by @tom4everitt provides one of the clearest and most complete accounts I've seen yet. Well worth checking out – alongside the wider causality research agenda:

https://t.co/TTxuMwxfZq

[Francis Rhys Ward @F_Rhys_Ward]

Do LMs deceive and manipulate? Can they be held responsible for their actions? Do they have agency? These questions all depend on the concept of *intention*. In our @AAMASconf paper, we provide a notion of intention which, we argue, can be applied to LMs. https://t.co/4C7SN6Ur9o

[Jon Richens @jonathanrichens]

Excited to share our new paper https://t.co/JijwCc66w1 (Oral, ICLR 2024, w/ @tom4everitt, @GoogleDeepMind). In it we answer the question, do agents need to learn causal world models? https://t.co/JijwCc66w1. 🧵

[James Fox @James_D_Fox]

@wooldridgemike @lrhammond @MattMacDermott1 Multi-agent influence diagrams (MAIDs) are a popular graphical representation of games and have recently been used for AI Safety problems (https://t.co/YiytPx8lRv)

Our paper extends the underlying theory to cover imperfect recall settings.
https://t.co/LWHSi68Kyt

[Michael Wooldridge @wooldridgemike]

Congratulations to @James_D_Fox @lrhammond @mattmacdermott1 for best paper award at TARK 2023!

[Zac Kenton @ZacKenton1]

@DeepMind @seb_far @MattBMcDermott @tom4everitt Oops we tagged the wrong Matt - this should be @MattMacDermott1. Apologies. And we can also tag @jonathanrichens while we're at it!