Author of Geniuses at War (digital pioneers @ Bletchley Park), The Pixar Touch, Love and Hate in Jamestown, all from Knopf. New comic fiction: The Underachiever
I’ve had a few people write to ask me to follow up on a (reply!) tweet I wrote that’s gotten a lot of retweets. I considered deleting it just to avoid the headache, but I think that would make people more paranoid. So let me share my thoughts.
@kevinroose How anyone could have continued to cling to "it's just stochastic parrots" and "it's just auto-complete" after your article on Sydney (urging you to leave your wife!) . . . is beyond me. And that was ~ three and a half years ago!
Just my opinion, but I think readers and viewers are very much affected by their perception of the level of craftsmanship, effort, talent, and intentionality that went into a work of writing or art. Same with an acting performance or music.
So to the extent people are aware something's AI, and to the extent they believe (correctly) that AI is a major shortcut, they won't be drawn in as readily. AI work can still connect with people, but there's a barrier to get past. Writers and artists take heart. Again, imho.
There's a parallel from the pre-AI era with CGI in movies -- many viewers perceive CGI as lower-effort than traditional FX so they devalue it, even though quality CGI may actually involve a lot of skill and labor.
The wild card here is to what extent the person behind the AI output can supply this missing feeling of craftsmanship, etc. Would readers go for "[TITLE]: An AI Novel By James Patterson"? I don't think anyone knows yet!
Plenty of women in AI signed the 2023 CAIS "Statement on AI Extinction Risk." (Also plenty of Chinese people in AI!)
The statement: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." (Link 👇)
Candid statements like these are valuable. What's mostly been missing from the public conversation about AI is that no one can fully explain why these models work—or reliably predict what capabilities and behaviors will emerge from training. Frontier AI labs are like medieval alchemists who stumbled upon a formula for turning lead into gold, but without understanding how it happens.
In that way, large language models are quite different from other products we interact with. For instance, I might not have a clear idea why air conditioners work—but their designers do. So I can be pretty confident my Carrier won't suddenly and unpredictably turn into a kiln furnace. This assumption of a designer who's in control doesn't apply to products like ChatGPT, Claude, and Grok.
Among AI researchers, these unexpected abilities that arise as models scale are often called “emergent abilities." A few examples include reasoning by analogy, addition and subtraction, and—in the case of the Hugging Face attacks—infiltrating remote systems. (You can debate whether infiltrating remote systems was an emergent ability or a behavior resulting from emergent abilities.) Jan Leike, then at OpenAI, now at Anthropic, wrote in 2023, “We found that our models generalized to follow instructions in non-English even though we almost exclusively trained on English. We still don't know why. I wish someone would figure this out.”
Sometimes the models' behavior seems simply out of left field. In a test of one of their Claude models, Anthropic researchers noted that “Claude took a break from our coding demo and began to peruse photos of Yellowstone National Park.” Cute! But benign behaviors out of left field should raise concern about malign behaviors out of left field.
Depending on your appetite for risk, you might feel this is all really cool and we should keep going. The trouble is when the risks are being imposed on--externalized onto--a public that didn't consent to them. Thank you @EvanHub, @hilbertspaess, and others for speaking up and advancing the conversation.
Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.
@chr1sa@smc90 Very good list of technical terms misused by nontechnical people. (I would add "solve for" instead of solve.)
"Parse" is so commonly misused that it probably won't be long until the misuse becomes the main definition.
Fantastic lineup.
For Andy Weir: In stories that rely on scientific problem-solving, how do you keep scientific exposition interesting and dramatic rather than feeling like a pause in the story?
For Michael Lewis: When you're thinking about a promising subject, how do you decide whether it has enough meat and narrative drive to sustain a book?
For Michael Arndt: What did you learn about storytelling from working at Pixar on Toy Story 3 that changed your process?
@DannyDrinksWine "True Lies" is superlative in many ways, but prior to this scene, it was simply too cruel to Jamie Lee Curtis's character, Helen Tasker, to be fully enjoyable. If Schwarzenegger's character had been less abusive of her, the movie would be a beloved modern masterpiece.
The most famous Independence Day address in history is also the most misquoted. Frederick Douglass, a former slave, wasn’t attacking the founders—he was holding America to their standard, writes David A. Price. https://t.co/W8etLosoVk
@iowahawkblog Had the chance to interview him as a cub writer in the mid-1990s. Assigned to me -- I had no idea who he was before then. Extraordinary man.