Harry Booth

Reporter @TIME covering the biggest challenges posed by artificial intelligence and the people working to solve them. Signal: harry_booth.84

[Harry Booth @harryboothtime]

I just published my first TIME cover story! As a writer, it's a dream come true. https://t.co/CUvPo4BUdz

[Harry Booth @harryboothtime]

“This concentration of this many resources in a useful way, I think, is very new. I don't think we understand the implications. I think you could reasonably argue that that's very dangerous“

A full-circle moment.

[a16z @a16z]

Martin Casado spent years dismissing the AI doom scenario and says what he got wrong was how long the scaling laws would keep holding:

"I was responding to this Bostrom notion of recursive self-improvement, fast takeoff. You create one of these things, you step back, and it takes over the world... That's clearly not what's happening."

"But here's what I got wrong. What I got wrong is I did not know that we could effectively just continue to pour money in this. The scaling laws are holding."

"I don't know what it means to do a $100 billion training run, to have this thing that you're putting $100 billion in, and then that money comes from this meta-economic machinery that may want to solve whatever."

"They may want to solve cancer, but they may also want to create a weapon. Like, who knows?"

"This concentration of this many resources in a useful way, I think, is very new. I don't think we understand the implications. I think you could reasonably argue that that's very dangerous if you apply that $100 billion in the wrong way... What does it mean to be able to concentrate resources?"

@martin_casado

[Harry Booth @harryboothtime]

@MTSlive We reported this months ago https://t.co/CUvPo4Bmo1

[Harry Booth @harryboothtime]

@MadisonMills22 FWIW we reported this in March

[Lennart Heim @ohlennart]

I've joined the OpenAI Foundation to lead "AI Resources."

AI capabilities are increasing rapidly and society's ability to respond needs to scale just as fast. The same advances creating new challenges also expand what AI can do to address them. If we're going to have AI systems that "outperform humans at most economically valuable work" and "a country of geniuses in a datacenter", we want a meaningful share of those geniuses and that work pointed at the most important challenges.

At AI Resources we’ll focus on getting this AI workforce ready and into the right hands to do exactly that, under a range of future AI scenarios.

After thinking about this since the beginning of the year, I believe it's one of the highest-leverage interventions for making AI go well.

Excited to build this with @woj_zaremba, @yonashav, many others, and maybe you. We're hiring broadly across the Foundation, AI Resilience, and for my team. Roles should be online this week.

Below some personal comments.

[Harry Booth @harryboothtime]

@_NathanCalvin Cheers!

[Garrison Lovely @GarrisonLovely]

Man, this sounds exactly like the mistake the left is making with AI…

When I wrote a post critical of Ed Zitron, one thrust of comments was, ‘i don’t get why you’re attacking a fellow critic,’ as if it didn’t matter if your analysis was correct, as long as it was negative on AI!

If you’re a union negotiating with management, you need to understand your power and your boss’s power to get the best deal. Pretending management is on the verge of going broke or has no idea what’s going on when they are actually flush and informed is malpractice for your members. We need to stop making this mistake with AI before it’s too late.

[Harry Booth @harryboothtime]

@tobyordoxford I’ve received a handful of emails from people in the grips of AI psychosis, confident they’ve made connection with a conscious AI. Yet to receive anything quite like this tho.

[Harry Booth @harryboothtime]

@ShakeelHashim Thought I was the only one

[Miles Brundage @Miles_Brundage]

Suspect this won’t get as much attention as the Hugging Face stuff bc it’s harder to explain

But it is arguably worse, in that distillation has been a super high profile issue for a while that eg the WH warned about, and companies still screwed up big time

[Harry Booth @harryboothtime]

The number of people on here confidentially declaring text watermarking is impossible when it has existed for years... Bodes poorly for AGI.

[Harry Booth @harryboothtime]

Worth reading!

I share Zuck’s concerns regarding a concentration of power.

But he presents it as if the choice is between AI controlled by a small group and AI distributed among many. Recent incidents offer reason to doubt humans—many or few—can retain control of superint.

[Mark Zuckerberg @finkd]

I believe everyone should have access to superintelligence, and I wrote a long piece about Meta's philosophy and values for building a positive future for everyone. https://t.co/2ZoNZXZ39T

[Jonas Geiping @jonasgeiping]

Why am I being baited by watermark misinformation on this app, is it 2023 again?

A small FAQ:

1. 𝗪𝗵𝗮𝘁'𝘀 𝗮 𝘁𝗲𝘅𝘁 𝘄𝗮𝘁𝗲𝗿𝗺𝗮𝗿𝗸? -- A modification of the LLM sampling algorithm that, if there are multiple ways to write something, will pick one that agrees with a pseudorandom key. This is a local, invisible signature hidden in the way phrases are used in any LLM text that persists when text is copied.

2. 𝗗𝗼𝗲𝘀 𝘁𝗵𝗶𝘀 𝗺𝗮𝗸𝗲 𝘁𝗵𝗲 𝘁𝗲𝘅𝘁 𝘄𝗼𝗿𝘀𝗲? -- A good implementation is 'undetectable' (in polynomial time), meaning: If you do not have the private key, then neither you, the model itself, or pangram could detect that this is happening.

3. 𝗪𝗶𝗹𝗹 𝘁𝗵𝗶𝘀 𝗯𝗿𝗲𝗮𝗸 𝘁𝗵𝗲 𝗺𝗼𝗱𝗲𝗹'𝘀 𝗿𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴? -- Because Ant already encrypts the model's reasoning, they can just not watermark the model's internal reasoning, leaving the thinking unaffected.

4. 𝗪𝗶𝗹𝗹 𝘁𝗵𝗶𝘀 𝗺𝗮𝗸𝗲 𝘁𝗵𝗲 𝗺𝗼𝗱𝗲𝗹 𝗹𝗲𝘀𝘀 𝗰𝗿𝗲𝗮𝘁𝗶𝘃𝗲/ 𝗺𝗼𝗿𝗲 𝘀𝗮𝗺𝗲-𝘆? -- If anything this (marginally) increases entropy across different generations, so it will make model outputs slightly more varied.

5. 𝗕𝘂𝘁 𝗜 𝗰𝗮𝗻 𝗷𝘂𝘀𝘁 𝗿𝗲𝗺𝗼𝘃𝗲 𝗶𝘁 𝗽𝗮𝗿𝗮𝗽𝗵𝗿𝗮𝘀𝗶𝗻𝗴? -- Absolutely! But, judging from the amount of writing on the web that already unmistakably sounds like Claude, most people likely will not bother.

5b: Also, not any paraphrase will work. To remove (for example) a k=5-minhash watermark completely from a long document, you need to make sure none of the original 2-grams, 3-grams, 4-grams, 5-grams and 6-grams of the text remain.

6. 𝗪𝗶𝗹𝗹 𝘆𝗼𝘂 𝗶𝗻𝗮𝗱𝘃𝗲𝗿𝘁𝗲𝗻𝘁𝗹𝘆 𝗰𝗼𝗽𝘆 𝘁𝗵𝗲 𝘄𝗮𝘁𝗲𝗿𝗺𝗮𝗿𝗸? -- No, with a good implementation the space of possible realizations of the key is too large to memorize.

7. 𝗪𝗶𝗹𝗹 𝘁𝗵𝗶𝘀 𝗮𝗹𝗹𝗼𝘄 𝗖𝗹𝗮𝘂𝗱𝗲𝘀 𝘁𝗼 𝗶𝗱𝗲𝗻𝘁𝗶𝗳𝘆 𝗼𝘁𝗵𝗲𝗿 𝗶𝗻𝘀𝘁𝗮𝗻𝗰𝗲𝘀 𝗶𝗻 𝗮 𝘀𝘄𝗮𝗿𝗺? -- The watermark will 'appear' like random sampler fluctuation to the model and would not be detectable. But, if an agent gets hold of a detector endpoint, it can absolutely use the watermark to ID other Claude agents (not that it would have trouble noticing them based on their writing as of today).

8. 𝗪𝗶𝗹𝗹 𝘁𝗵𝗶𝘀 𝗱𝗲𝘁𝗲𝗰𝘁 𝗱𝗶𝘀𝘁𝗶𝗹𝗹𝗮𝘁𝗶𝗼𝗻? -- By default, no. If the watermark is set up to be 'undetectable' (as assumed above), it will not be picked up in training by other models. For that to happen, the watermark needs to be detectable by ML algorithms.

9. 𝗪𝗶𝗹𝗹 𝘁𝗵𝗶𝘀 𝗺𝗮𝗸𝗲 𝗣𝗮𝗻𝗴𝗿𝗮𝗺'𝘀 𝗷𝗼𝗯 𝗲𝗮𝘀𝗶𝗲𝗿? -- By default no, this is a separate avenue to detection. But, they might collaborate with Anthropic which would allow them to detect the watermark as well and show a watermark score next to their text detection score.

10. Bonus: All aside, is this a good idea? I don't know. The companies are doing it to follow the writing of the EU AI act, which was written based on 2024 information and when the field looked very different, and threat models were focused much more on slop/propaganda (like the Kokotajlo 2026 prediction). The actual 2026 looks quite a bit different.

[Harry Booth @harryboothtime]

@norbertomarques @ns123abc This is in response to EU law, rather than a choice.

[Harry Booth @harryboothtime]

@calebwatney @nypost @deanwball lol

[Harry Booth @harryboothtime]

@ShakeelHashim Worth saying that Demis thinks we are in an AGI by 2030ish world.

[Harry Booth @harryboothtime]

This and more in my latest for @TIME.
https://t.co/x1iq0oJj0y

[Harry Booth @harryboothtime]

Talk of AI recursively improving itself has focused heavily on coding.

But there’s a wrinkle: AI models aren’t exactly written in code.

For the past 8 months, I’ve been reporting on what Anthropic and OpenAI are actually trying to automate—and how fast it’s happening.

[Harry Booth @harryboothtime]

Measuring the acceleration is much harder than measuring % of code written by AI.

When I asked Clark how much acceleration they’re getting today:

“I can’t give you a specific number, because we don’t have a measure.”