70% of the internet was vulnerable yesterday to very bad security flaws inside a coding language. Turns out, *one volunteer* is mostly responsible for maintaining that code & helping push updates. Yesterday Ondřej Surý saved the internet. So I called him up. He rules. https://t.co/DusKs8P0Em
the curve was a gem
really enjoyed this panel with @AndrewFATHOM and @BriTreece_, plus convos with experts on how auditing matured in fields like aerospace and finance.
also the first conference where I’ve fallen into a spontaneous deep dive on frontier AI auditing standards while sipping a negroni sbagliato.
ty @GoldenGateInst!
A few thoughts after the close of The Curve (which was outstanding):
1. We have a pretty good idea of near-term policy options that would meaningfully reduce catastrophic (including loss-of-control) risk.
2. There simply are not that many folks in government—state or federal—who are fully read-in on the risks and policy options.
3. There's a deep level of uncertainty regarding what, exactly, the blockers are for taking steps (even unilaterally!) towards pacing the frontier. Lab insincerity, China worries, missing commitment mechanisms?
4. There remain some thorny outstanding legal questions about many top proposals—including many pacing options—arising from the First and Fourth Amendments, the Dormant Commerce Clause, and antitrust law. That said, there seems lole there's some appetite at the labs for taking on increased legal risk.
We have a lot to do, very quickly. But the incredible talent and collegiality on display this weekend left me feeling more heartened that we can actually exercise agency over the worst harms of technology and realize a better future.
The single most useful take I heard at The Curve was "we can still heist the Mona Lisa after the singularity". There's a subtle/important point here, where some things (e.g, n of 1 goods where originally is valuable) will retain/grow in value even as everything changes.
I was encouraged this week to see the leaders of the frontier labs agree on the need for them to slow down the pace of AI development. Given the stakes, it’s a good and necessary first step.
But I’m even more encouraged by the growing recognition that how this powerful new technology develops should be at the center of our public debate.
I’ve been watching the progress on AI for over a decade now, and one thing that’s clear to me is that the potential impact of this technology is not overhyped. It’s also moving at lightning speed – and even faster than those who are engineering it can keep up with.
I’m not an AI accelerationist who believes it will lead to some techno-utopia, and I’m not a doomer who thinks it will inevitably lead to humanity’s destruction.
But whether this technology results in amazing breakthroughs in medicine, energy and education or unleashes huge economic disruptions, greater inequality, and potential catastrophe will depend on the choices that we make right now – choices that should be made not just by the companies involved, but by all of us.
A LOT of people are urgently reading up on AI governance. Let's do them a favor: what are the best things to read, watch, or listen to if you've exclaimed "what on earth is going on?" in the last 48 hours?
I'll start off; add yours in reply:
(1) Explainers of recent cyber incidents:
- A 20-min video explainer on the HF breach: https://t.co/BE35gcfFfJ
- This longer interviewer with one of the lead investigators @ajeya_cotra: https://t.co/yRUBmhmY9a
- The METR report itself, based on the work of @ajeya_cotra@HjalmarWijk and @RyanGreenblatt: https://t.co/OgPxi5iO97
- These are also great, but even more info has come out since: Ezra Klein episode with @hlntnr https://t.co/pASOACJnqC or the Black Hat talk by OpenAI https://t.co/HhFZe9pJRw
------------
(2) Great essays on how to think about AI governance more generally:
- Radical Optionality by @CharlieBull0ck and @Christophkw
(tl;dr: the dilemma of AI policy is deep uncertainty; but there are several policies that are robust to different worldviews and assumptions and would be really useful for making fast, informed decisions in the future) https://t.co/Pzea5JLIW9 or in podcast form: https://t.co/0KrMbjjESV
- AI as Normal Tech by @random_walker and @sayashk (thoughts and link in the thread) https://t.co/B2NqAYZzTT
- Don't Surrender to the Tech Tree by @taoburr (tl;dr: yes, you can't fight the progression of technology, but you can still alter its trajectory) https://t.co/q993YF6Bhx
- AI 2040 by a long list of authors including @thlarsen (tl;dr trying their darnedest to concretely map out how society could successful manage rapid advances in AI) https://t.co/yVQxxYHThm
------------
(3) Understanding AI more generally
- On the unique implications of automating AI R&D: https://t.co/qW8mMvWxVY
- OpenAI's recent update on accelerating AI R&D https://t.co/d4nyp74upu
- Helen Toner, again, on taking jaggedness seriously https://t.co/zK1g6bGkB3
- [I'm especially curious what the best writing is on ~agent coordination; ~the implications of agents that can perform long, general tasks, etc.]
- an intro explainer of neural nets: https://t.co/pQ9fxz9ywP
------------
(4) Understanding the current state of the law
- Stephan Llerena and article on the big gaps in existing law, and why we so few incidents are covered: https://t.co/TfUydxiAGF
------------
(5) Policy recommendations
- A more opinionated take on what went wrong in the Hugging Face incident by @peterwildeford https://t.co/J73tIYSACp
- Internal use visibility by @iapsAI: https://t.co/m02nLPfTom
- Auditing @Miles_Brundage: https://t.co/QlLhp7PpTh and https://t.co/qFcv0PlOxY
- Liability @gabriel_weil and @ketanrama: https://t.co/8UUpWS89t6 and https://t.co/e3K7JHvugn
- Incident Reporting and investigations: https://t.co/5jx37QlaJC and https://t.co/4CmExj1FqT and https://t.co/HjIwyD8F3d and https://t.co/TfUydxiAGF
- CAISI Funding @__J0E___ : https://t.co/jK1gtjwSzk
- Federal hiring and capacity building: https://t.co/JiRXJyywKc
- Whistleblower Protections: https://t.co/eCrMqkBFyy and https://t.co/60wvYYu2sk
- Physical and cybersecurity for frontier labs: https://t.co/ZcPFj7AbIv
- Monitoring and detection: https://t.co/C7FMtRjSe8
- Preparing for AI R&D @fiiiiiist: https://t.co/8arrxCrUXw
- Law-folloowing AI: https://t.co/PFjswICdAh
Today we’re launching Project Tailwind, a call for founders to start ambitious new AI safety initiatives: https://t.co/k5SoaXbuWk
We’re looking for great people to engage seriously with the risks of transformative AI, and to create the research, technologies, and institutions that will help humanity navigate them.
There are critical, basic problems that no one owns. Over the last several months, my team at @coeff_giving brainstormed ideas we’d be excited to fund if we could find a promising founder, and the list quickly grew to more than 200 entries. We’ve narrowed it down to our favorites, but we also expect the best founders to bring their own ideas:
https://t.co/88qE9bVrEy
We’re providing funding at three levels:
1️⃣ Pre-seed: $200k to $2m to develop an idea and build a team
2️⃣ Seed: $2m to $20m to launch and scale
3️⃣ Scale: $20m to $200m+ for proven teams to scale, or world-class teams to start
If you’re excited about something on our list, or you have another proposal for driving the field forward, you should get involved: https://t.co/8utEYbXYhx
There are more good projects than there are people to work on them. Please help us make that stop being true!
Also hi, I’m Emily! This is my first tweet. I lead AI and biosecurity grantmaking at @coeff_giving.
Five years after leaving DC i still follow @capitalweather, mostly bc they occasionally write glorious posts on @Capon_Springs cloud systems, but also bc they offer fantastic, in-depth coverage. I always learn things and wish @sfchronicle had similar coverage. Yay for The Sun!