My name is Chris Painter, and I'm the President of METR (Model Evaluation and Threat Research). I know we've made a lot of new friends on the internet the last couple of days, so I thought I'd take this chance to re-up what we do and why.
Our work is aimed at making sure that if AI really were autonomous, difficult to steer, and close to "going rogue," the public would find out. If evidence exists inside of an AI company that it’s close to losing control of AI, we want to make sure that information gets shared with the rest of the world, including governments and the public outside the company’s walls. This is what we've been focused on since 2022, and over the years we've worked with OpenAI, Anthropic, Google DeepMind, Meta, Amazon, and others on piloting third-party assessments and investigations of this type. We don’t have some private room where we rubber stamp things as “safe” or not.
We have had a track record of publishing results on AI that don't cleanly map onto the "doomer" or "accelerationist" labels, and we put in effort to hire people with competing views on AI. We’ve been cited for having found some of the strongest evidence that AI capabilities are improving rapidly (our work measuring AI “time horizons”) while also presenting some of the strongest evidence that, at various points, AI’s capability may be overstated (some might remember our study showing that early 2025 software engineers were actually being slowed when they thought they were being sped up).
METR is funded by donations. We don't accept money from frontier AI companies. They haven't paid us for our work, and we don't accept donations from them or their employees. As we’ve shared previously, multiple frontier AI companies currently provide us with free access to their models in order to perform our evaluations, research, and engineering. Our funding intentionally comes from a wide range of donors, which we’ve shared on our website.
Today, when an AI company works with any third-party evaluator or external testing organization (of which there are and should be many), it's entirely voluntary. This often involves NDAs and redactions. To counterbalance this, we have a principle that when we enter into a contract with a company, we try to retain the right to tell the public the terms of the contract we signed, and characterize the nature of redactions that the company chose to make. For example, the report from our independent investigation of the OpenAI-HuggingFace incident included that information. Public disclosure is also a big part of our COI policy (linked on our website). That’s not to say our reports are adequate as oversight. We’re just one organization (among many doing great work), working in a voluntary setup, trying to get good evidence to the public and the world about AI, letting the facts fall where they may.
NEW: Pope Leo XIV’s older brother, John Prevost, told Italian television on Sunday that the pope helped him file his tax return and, over a video call, got him back into a computer he had locked himself out of.
Prevost propped his iPad on the dead machine and asked his brother what to do. “Because I’m computer illiterate,” he said.
Leo turned 71 on Monday, sixteen months into a pontificate that has drawn attacks from President Trump and a bomb threat at Prevost’s own home.
Asked what makes him proudest of his brother, Prevost said: “That he hasn’t collapsed yet.” https://t.co/xnyYXRh4Jz
"We choose to go to the Moon in this decade and do the other things, not because they are easy, but because they are hard."
On this day in 1962, President John F. Kennedy gave his famous "We Choose to Go to the Moon" speech at Rice University.
Anthropic: We are going to kill you
OpenAl: We are going to kill you first
Investment banks: “This is great. Now let’s say your total addressable market is $100 trillion by 2030”
This is why it's a good thing that American labs are in the lead. If Chinese labs were, their employees wouldn't be able to warn us about threats. But Chinese labs will encounter the same threats within a year, in complete silence.
I think it’s important to understand that these 3d anatomy models are all built with preexisting 3d anatomy repositories that the AI tool is just putting a fancy wrapper on. Those are models that people spent hours upon hours actually crafting every structure. Astra, fable, etc are just pulling those open source models and making fun visualizations with them but they aren’t actually generating the anatomy itself
Lots of credit that’s being lost for the people who spent probably hundreds if not thousands of hours making the actual anatomy model. Astra is just putting a wrapper on what was the actual hard work.
When my dad got a PhD he pulled our family out of poverty.
So I always read these.
Which taught me something else.
It sometimes gives you an early look at the future. Like here.
Ask Grok for help understanding them.
This #TechTuesday, discover how AI is transforming nursing at UF! 🤖 The 'Clinicals' app acts as a simulation lab in your pocket, letting students practice vital patient care skills anywhere.
Watch the full feature: https://t.co/8emuCYxnDb #Nursing#AIeducation#UFnursing