ʇuoɯlǝq ɔɹɐɯ

[Andreas Kling @awesomekling]

The only real moat is giving a shit.

[Yaroslav Bulatov @yaroslavvb]

Secrecy is a crutch for the insecure. If a leak is enough to kill your business, you aren't betting on your execution—you're betting on your obscurity.

[🇺🇦 Dzmitry Bahdanau @DBahdanau]

I'm a dumb stochastic parrot, but PLEEASE give me the reward, I will generate slop for any possible and impossible contingency to be rewarded, please give me that sweet reward juice, please please

[Min Choi @minchoi]

This is what flying in dreams feels like

[Andrej Karpathy @karpathy]

Nice post on software engineering.
"Cognitive load is what matters"
https://t.co/eMgxu0YgWw
Probably the most true, least practiced viewpoint.

[Min Choi @minchoi]

Just 18 months ago, AI Will Smith was eating spaghetti.

Now he's trapped in the hot dog realm.

The future is wild 😂

[🎞️u/Spartima]

[Noam Brown @polynoamial]

First they said AI can’t play Go, because it’s too complicated.

Then they said AI can’t win at poker, because it’s a “people game”.

Today, the skeptics say “AI won’t write novels”.

Surely once that happens, though, the goalposts won’t move again? https://t.co/UZ7vMyT0aG

[Arvind Narayanan @random_walker]

In a new AI Snake Oil essay by me and @sayashk, we do a deep dive into AI existential risk probability estimates. We find that these forecasts are just feelings dressed up as numbers, and even the best-run, well funded, time intensive forecasting efforts result in a range of probability estimates that spans many orders of magnitude! We are forced to conclude that AI x-risk forecasts are far too unreliable to be useful for policy, and in fact highly misleading. We caution against speculation being laundered through pseudo-quantification.

Full essay: https://t.co/EFCNpN2e2y (about 7,000 words). Summary below.

Background

Over a year ago we got deep into the AI x-risk literature. We were skeptical but not dismissive. We wanted to identify valid concerns while rebutting bad arguments on their own terms. We've been especially interested in how policymakers should think about x-risk. Today's essay is the first in a series. We've been circulating private drafts for a while and have incorporated a lot of great feedback. I'm excited that we're finally launching this series of essays today!

Analogy: alien invasion

If the two of us predicted an 80% probability of aliens landing on earth in the next ten years, would you take this possibility seriously? Of course not. You would ask to see our evidence. As obvious as this may seem, it seems to have been forgotten in the AI x-risk debate that probabilities carry no authority by themselves. Probabilities are usually derived from some grounded method, so we have a strong cognitive bias to view quantified risk estimates as more valid than qualitative ones. But it is possible for probabilities to be nothing more than guesses.

The reference class problem

The domains where forecasting has been successful, such as geopolitics, rely on the existence of reasonably good reference classes of past events. A reference class for turmoil in one country is turmoil in other country. Reference classes for AI x-risk are things like ... animal extinction. Let’s get real. This kind of reference class tells us nothing about the possibility of developing superintelligent AI or losing control over such AI, which are the central sources of uncertainty for AI x-risk forecasting.

Subjective probabilities vary by orders of magnitude

Lacking grounded methods, forecasts are necessarily “subjective probabilities”, that is, guesses based on the forecaster’s judgment. Unsurprisingly, these vary by orders of magnitude. Consider the Existential Risk Persuasion Tournament (XPT) conducted by the Forecasting Research Institute in late 2022, which we think is the most elaborate and well-executed x-risk forecasting exercise conducted to date. It involved various groups of forecasters, including AI experts and forecasting experts. The 75th percentile AI expert estimate and the 25th percentile forecasting expert estimate differ by at least a factor of 100.

All of these estimates are from people who have deep expertise on the topic and participated in a months-long tournament where they tried to persuade each other! If this range of forecasts here isn’t extreme enough, keep in mind that this whole exercise was conducted by one group at one point in time. We might get different numbers if the tournament were repeated today, if the questions were framed differently, etc.

It's all speculation

What’s most telling is to look at the rationales that forecasters provided, which are extensively detailed in the 754-page report. Forecasters aren’t using quantitative models, especially when thinking about the likelihood of bad outcomes conditional on developing powerful AI. For the most part, forecasters are engaging in the same kind of speculation that everyday people do when they discuss superintelligent AI. Maybe AI will take over critical systems through superhuman persuasion of system operators. Maybe AI will seek to lower global temperatures because it helps computers run faster, and accidentally wipe out humanity. Or maybe AI will seek resources in space rather than Earth, so we don’t need to be as worried. There’s nothing wrong with such speculation. But we should be clear that when it comes to AI x-risk, forecasters aren’t drawing on any special knowledge, evidence, or models that make their hunches more credible than yours or ours or anyone else’s.

Forecast skill can't be measured

We often hear that forecasting has a great track record and so we should trust it. This makes no sense — why should we trust that someone who is good at forecasting elections or other kind of events is good at forecasting AI x-risk? Besides the fact that these are completely different kinds of events, there just isn't any real evidence to draw upon for the AI x-risk estimation, so being good at finding and weighing evidence is not a skill that helps much here.

Besides, the math doesn't work out. We show that if someone is good at forecasting common events but systematically overestimates rare events, we would have to evaluate them on millions of forecasts before this became apparent.

Summary of the main argument: none of the three probability estimation methods yields reliable AI x-risk forecasts.

Risk estimates may be systematically inflated

There are many reasons why forecasters might systematically overestimate AI x-risk. The belief that AI can change the world is one of the main motivations for becoming an AI researcher. And once someone enters this community, they are in an environment where that message is constantly reinforced. And if one believes that this technology is terrifyingly powerful, it is perfectly rational to think there is a serious chance that its world-altering effects will be negative rather than positive.

And in the AI safety subcommunity, which is a bit insular, the echo chamber can be deafening. Claiming to have a high "p(doom)" seems to have become a way to signal one’s identity and commitment to the cause.

So what should governments do about AI x-risk?

Our view isn’t that they should do nothing. But they should reject the kind of policies that might seem compelling if we view x-risk as urgent and serious, notably: restricting AI development. As we’ll argue in a future essay in this series, not only are such policies unnecessary, they are likely to increase x-risk. Instead, governments should adopt policies that are compatible with a range of possible estimates of AI risk, and are on balance helpful even if the risk is negligible. Fortunately, such policies exist. Governments should also change policymaking processes so that they are more responsive to new evidence. More on all that soon.

The full essay is in our newsletter. We plan to publish the rest of the series over the next few weeks. Thank you for reading! https://t.co/EFCNpN2e2y

[Yaroslav Bulatov @yaroslavvb]

You get projections like "AGI in 3 years" by extrapolating from the local trend. But if the trend is "jumpy", this projection undershoots. We are in the middle of a "jump" because we've been creating knowledge for the last 5000 years, digitizing it for the last 50, but only recently did the stars align to compress it into an easily queriable form. Once that source of knowledge is fully milked, you need to create new knowledge. That needs new technology.

[ElevenLabs @ElevenLabs]

Here’s an early preview of ElevenLabs Music.

All of the songs in this thread were generated from a single text prompt with no edits.

Title: It Started to Sing

Style: “Pop pop-rock, country, top charts song.”

[Andrej Karpathy @karpathy]

# on shortification of "learning"

There are a lot of videos on YouTube/TikTok etc. that give the appearance of education, but if you look closely they are really just entertainment. This is very convenient for everyone involved : the people watching enjoy thinking they are learning (but actually they are just having fun). The people creating this content also enjoy it because fun has a much larger audience, fame and revenue. But as far as learning goes, this is a trap. This content is an epsilon away from watching the Bachelorette. It's like snacking on those "Garden Veggie Straws", which feel like you're eating healthy vegetables until you look at the ingredients.

Learning is not supposed to be fun. It doesn't have to be actively not fun either, but the primary feeling should be that of effort. It should look a lot less like that "10 minute full body" workout from your local digital media creator and a lot more like a serious session at the gym. You want the mental equivalent of sweating. It's not that the quickie doesn't do anything, it's just that it is wildly suboptimal if you actually care to learn.

I find it helpful to explicitly declare your intent up front as a sharp, binary variable in your mind. If you are consuming content: are you trying to be entertained or are you trying to learn? And if you are creating content: are you trying to entertain or are you trying to teach? You'll go down a different path in each case. Attempts to seek the stuff in between actually clamp to zero.

So for those who actually want to learn. Unless you are trying to learn something narrow and specific, close those tabs with quick blog posts. Close those tabs of "Learn XYZ in 10 minutes". Consider the opportunity cost of snacking and seek the meal - the textbooks, docs, papers, manuals, longform. Allocate a 4 hour window. Don't just read, take notes, re-read, re-phrase, process, manipulate, learn.

And for those actually trying to educate, please consider writing/recording longform, designed for someone to get "sweaty", especially in today's era of quantity over quality. Give someone a real workout. This is what I aspire to in my own educational work too. My audience will decrease. The ones that remain might not even like it. But at least we'll learn something.

[Prof Pan @ProfPan_]

The best meme I have seen today! 🤣 #LK99

[Ben Eater @beneater]

Sometimes there are videos that I want to make just because I wish they existed, even if no one watches them. Here it looks like a lot of people agree. 🤣

[Fabien Benetou @utopiah]

The beauty of freedom : https://t.co/jgHQ9GMGa2

It can be overwhelming, even scary and yet, once you understand we are all in it together, able to support each, able to build HOW we want, then it become truly liberating.

🫶🐧🫂

[Jason Leow @jasonleowsg]

Recently re-read this. So good.

I'm that middle class kid. I had my one throw, but now trying to afford more tickets for more throws, through things like freelancing.

All the while just hoping I'll never have to go back to working at the carnival.

[The Hero of KVcache @HeroOfKVcache]

"Will Smith eating spaghetti" generated by Modelscope text2video

credit: u/chaindrop from r/StableDiffusion

[Sabrina @sabrinaesaquino]

git commit -m "GPT-4 said it would work"

[Soumith Chintala @soumithchintala]

so many mainstreamers pumping AI now.
a good filter is to check if they were pumping crypto before lol.

[AK @_akhaliq]

open source or it didn't happen

[Turner Novak 🍌🧢 @TurnerNovak]

The pitch deck vs the product