starting a project: Dark Oak
a phone-friendly web game that teaches coding and tech
https://t.co/Soc26PkHw7 is the cool site
https://t.co/gMY7clvFBS is for everyone else
[read about what’s happening with AI]
ME: Wow we’re actually legit fucked this is so bad.
[go outside]
ME: Wait those people are all acting like nothing unusual is happening. Maybe I was just in a paranoid bubble? Yeah it must be that. Phew.
[read more about what’s happening with AI]
ME: Wait no we ARE fucked. Shit. It’s all those people outside who are in a delusional bubble.
[go outside]
ME: Wait...
There's actually a really really ugly truth inside this tech-right/maga-right H1B issue that almost no one has touched.
Are you ready? No one is going to like this.
Let me first set up the framing a bit.
Tech-right wants Top-0.1%+ global-level geniuses. Ok, fine, tech wants to grab every genius in the world for the USA.
There is a separate issue of - let's call them "technically-literate workers" - roughly in the 40-90% percentile. We could call them "mids" but it's not exactly right because it does extend up to 90-95% percentile.
The program for this calls it "high-skilled immigrants" but in reality it extends pretty far down to "mid-skill" and is being systematically defrauded to bring in Actually-Incompetent People, and this is a grift that's cheating both native-born Americans and the immigrants themselves.
It also legitimately brings in some high-skill immigrants. Looking at the numbers, it appears to be 50% legit and 50% totally-corrupt. That's not a good percentage. The 50% legit is bringing in people in the 80-95%-tile range ("high but not genius") and the 50% corrupt part is bringing in people in the 40-80% range ("mid to somewhat not").
Maybe you have a quibble but I'm pretty sure I'm broad-strokes on target, so stick with me here:
I want to introduce a key concept, which is "jobs that American companies need done, for which there are qualified Americans but not enough Americans want to do them."
Some of these jobs are like "accountant," which happens to qualify for H1B treatment.
But to make this more clear, let me go lower: the job of strawberry-picker.
This not a high-skill job, or a mid-skill job, it is a hard labor job. Young Americans don't want to do this. They choose not to.
(If you have a problem with "they choose not to" then go back to your feminism wage gap debate and how women get paid less because they "choose not to" take jobs that pay more with high stress. "Choose not to" is already something you accept when it leads to uneven social outcomes)
Well, America still needs strawberries so companies hire immigrant migrant laborers. Today we don't get complaints about this on X because no one is like "I'm from Appalachia and went to a top-tier school and the farms I applied to hired migrant workers who are less qualified than me to pick strawberries" because that guy didn't want that job.
I am using this example because hard labor jobs are ones that Americans definitely do not want to do, and I am using it to highlight the fact that there are MANY "mid" jobs that require some computer-work that many Americans don't really want to do but which need to be done and they end up getting filled with H1Bs.
Think of this as "grunt knowledge work."
What we are ostensibly trying to solve for is that there are a lot of these mid jobs, and we want native-born Americans to do them. (And that part of doing them actually is going to be needing to pay more, like at oil-rig levels, and this may be a problem - but let's let the Leftists fight that one for us)
But the point is: we (Americans!) want those jobs, stop giving them to foreigners!
Those jobs aren't done by (or need) O-1 geniuses, but they pay good money, that money should go to native Americans.
We have not gotten to the dark punchline yet, so don't get upset yet if you have some problem with a detail in this setup above.
Here is the dark ugly truth:
All of these jobs are going to totally eliminated by the end of the Trump administration and done by AI.
The tech-right knows this. The tech-left knows this. Sam Altman, much hated and so misunderstood, definitely knows this and has been trying to save you all by normalizing the idea of UBI because he can tell that obviously the AI wave is going to crush everything.
Last week's o3 announcement indicated that we have an AI within shooting range of PhD-level humans. Whether it's completely there or not, it means we're very very close to AI that is as good as the Top-0.1% immigrants. The O-1 guys are not safe. It costs $5000 to ask it a single question today but in 5 years that'll be fifty cents.
EVERY SINGLE KNOWLEDGE-WORK JOB YOU ARE FIGHTING OVER WILL BE GONE BY THE TIME TRUMP FINISHES HIS TERM.
The entire debate here over immigration is a nothing-burger. I'm sorry. I'm very sympathetic to you heartland white dudes in IT because I grew up in Minnesota but everyone in tech knows this.
Tech is trying to hire global-level geniuses because that's what drives its industry but the monster wave that's going to make all this pointless is that none of these H1B jobs are going to exist in 4 years. At all.
No, you will not have a job telling the robots what to do. A single (maybe-)human CEO will be sitting there saying, "Get this done" to the AI employees.
Ironically, it WILL be the physical jobs that will last a bit longer (except that the physical robots are coming too), and I don't want to talk my book too much but I'll digress a bit here: I told you all that the future is in tree-planting.
But the dark core here is this:
Everyone mid, and even a large portion of geniuses: if you're exchanging your brainpower for money - AI is going to eat all that up within the next few years. Immigration as a debate just doesn't matter. The same companies that exploit immigrants for money will quickly switch over to AI as soon as it's cost-effective.
You are arguing about immigration because it's part of your tribal affiliation to argue about immigration. I respect that. I get it. You gotta fight for what's yours.
Maybe you're going to win the argument. It doesn't matter. Because at some point before Trump leaves office you may very well be standing shoulder-to-shoulder with your fellow human, Steve and Arjun both storming the datacenters with pitchforks.
Now, tech still has the opportunity to bring forth a great and grand future for all.
Yes, AI can do labor for all of us. But the benefits have to be broadly-distributed, and we have to reconceive of how we think about work and labor and society. Maybe we'll navigate all that successfully. But that's another issue. You may want to talk to the Leftists about their ideas on how to do that.
But in the meantime, the dark truth you need to know (and which isn't being openly acknowledged, weirdly) is that the very same thing that makes those H1B jobs something you are so passionately fighting over is the very same thing that makes it so that AI is going to completely eliminate them within a few years.
Be ready. 🤖
Once you understand that the world is full of highly neurotic people who need a doom's day scenario to validate their emotions, it all makes much more sense: Malthusian starvation, pandemic, climate, and now AI. I wish we still had "the devil" to play this part.
It's pencils down, people. Writing code by hand is no longer an economically viable skill for most programmers at most companies. But the future of making software has never been brighter. Don't you dare black pill this beautiful moment! https://t.co/qaCgfmkW0u
For every reddit doomer, there's a hundred quiet bloomers. Online discourse is dominated by malcontents and their neuroses. But the real world is full of people leaning into the future with optimism. It's fair to have butterflies, fair to feel anxious, but don't you dare doom 🌷
There's nothing kind about letting good people live in a fantasy world that no longer exists. You have to tell them, even if it hurts. Because the sooner they accept reality, the sooner they can adapt to the future.
Maybe you have recently become aware of the AI safety debate and the arguments swirling around it.
If you want to understand them, you need to understand a couple things that almost everyone gets wrong:
There are TWO distinct classes of AI dangers, and it's VERY important to think about them separately, and not let one confuse you about the other.
Many many people (including many quite intelligent, clear-thinking people) do not effectively understand the fundamental differences between these two classes of dangers.
The first one has to do with the theory that an AI far superior to human intelligence (Artificial SuperIntelligence, or ASI) will inevitably wipe out the human race.
The second one has to do with the idea that powerful AI will result in very harmful things happening to many human beings, possibly all human beings.
Those two sound VERY similar, don't they?
They are DIFFERENT. Understanding how they are different is crucial if you want to think about or contribute usefully to any conversation about AI safety or AI harm. You might feel like you are Making Very Good Points or Asking Incisive Questions, but if you aren't clear on the differences between the two, you aren't.
So, I'm going to tell you what the difference is so that you can talk more usefully.
The first one concerns itself with a very specific thing, which is ASI (Artificial Superintelligence) that is more intelligent than any human being. When I say that, I am not referring to a thing like how Einstein is smarter than you, we are talking more about something like how a human being is more intelligent than any mouse.
In our regular lives, we meet other people who we can tell are smarter than us, vs some who are less smart. The line is fuzzy, because intelligence has a lot of dimensions. I'm better at a "rotating shapes" kind of intelligence than my wife, and she is better at "words-making" kind of intelligence than I am.
But every human is better in almost every dimension of intelligence than every single mouse.
That's the level we're talking about: an artificial superintelligence - made up of a computer or a network of computers - that is more intelligent than any human. And more intelligent by a long shot, by a wide margin, in an indisputable way like how humans are above mice.
That is the first thing.
The theory says that if you have an AI that is vastly smarter than all humans - in the way that a human is smarter than mice - that superintelligent AI will inevitably, eventually, sooner or later, wipe out every human on the planet.
We will refer to this as "existential risk."
The common follow-up question "well, how exactly is it going to do that?" is NOT the important question, and one of the most important elements of understanding this theory is first getting why that particular question is not important. A couple analogies:
Analogy 1: You are playing chess against a grandmaster. My theory predicts the grandmaster is going to beat you. You can ask "Well, how exactly is he going to do that?" I don't know, because I'm not a grandmaster, I just know that a chess grandmaster is almost always going to beat a normal player like you. And I'd be right. So the question "how is he going to do that" is not important, and doesn't affect the final outcome. He's going to figure out a way because he's way better than you.
Analogy 2: Humans are smarter than all other animals, comprehensively, by a wide margin. We have driven numerous species to extinction, not because we hated them or hunted them. Many of them have died out without most humans even ever thinking about them. All we did was expand our civilization, use up resources, encroach on habitats, and pretty soon the resources needed by those species went away and they died out. We figured out a way to get what we wanted because we're way smarter than them, and often we didn't even notice they died as a result.
A lesser animal asking, "how are the humans going to wipe us out?" is not asking a relevant question. We don't know, but we do know that any time humans and lesser species compete for any kind of resources, the humans will win. The fact that we know who is going to win beforehand - and that it is due to the vastly different levels of intelligence - is the key concept here.
A vastly more intelligent AI is likely to care about things that are incomprehensible to us, the way animals can't understand human goals. It's going to need resources to pursue those goals and it's going to be far more effective at gaining control of them and excluding us from them - in the same way that we are far more effective than other lower species.
A much more intelligent AI will not care about our interests, it will care about its interests, and to whatever small degree we happen to escape total annihilation from losing access to all our resources, any remaining humans will likely be enslaved into a system that serves the AI's own purposes.
That is the first thing. (Remember how I said at the beginning of this post that there was a first thing, and then a second thing?)
The first thing is the most difficult to understand, because you have to extrapolate how a vastly superior intelligence would act, and you can only use analogies like "how do humans treat lesser creatures," and the analogies are messy.
But now let's move on to the second thing.
The second thing is "everything else you've ever heard that AI might do that's harmful."
That's a little inaccurate. It's actually "everything else you've ever heard that humans might use AI to do that's harmful."
This is the critical difference. The first one talks about the inevitable outcome of what happens when two vastly different levels of intelligence collide, e.g. ASI vs humans, or human vs mice.
The second one has to do with what happens when humans possess AI as a powerful tool. This second thing is much easier to understand, because we have many more concrete notions:
Like:
- the military uses AI to make hyper-efficient killer drones and missiles
- your capitalist overlords use AI to replace you and everyone loses their jobs
- authoritarian government uses AI to surveil everybody and control the entire population
- hackers use AI to break into secure networks and hold companies and governments hostage
- students use AI to cheat on homework and show up to college knowing nothing
- AI slop saturates the internet and makes it impossible for artists and writers to make a living
- terrorists use AI to make biological or nuclear weapons
or even things like
- the military hands control to an AI and it misinterprets something and launches nuclear attacks and kills millions
All of those sound pretty familiar, right? Yeah, you've heard them before. We call this second thing "risks from misuse."
These problems are not the first class of problem! This second class of problems exists while AI is a tool that can be controlled by humans, and humans use it to do evil or careless things to each other. The problems may sound exotic or dystopian or novel, but they are fundamentally problems having to do with flawed human nature.
Given a powerful tool, some humans will likely use it to control or otherwise harm others. This is a very familiar problem.
I am not condemning or condoning this. I'm just describing it.
That is a fundamentally different danger from the first thing, which is that when a human is far superior to a mouse, the mouse is likely to come to harm because the human cares about doing human things, and the mouse is not gonna make it once the humans get going.
=====
Hopefully from the above, you have understood the difference between the first thing and the second thing. I will list them again - see if you now understand how they are different:
The first one has to do with the idea that an AI superior to human intelligence (Artificial SuperIntelligence, or ASI) will inevitably wipe out the human race.
The second one has to do with the idea that powerful AI will result in very harmful things happening to many human beings, possibly all human beings.
Can you tell how they are different now?
If not, re-read the stuff from earlier until you understand.
We call the first one "existential risk" and we call the second one "risks from misuse."
Once you understand, here is the CRUX of the problem:
SOLUTIONS TO THE SECOND THING DO NOT HAVE ANYTHING TO DO WITH SOLUTIONS TO THE FIRST THING.
In fact, it's worse:
Solutions to the second thing (misuse) look roughly like "give powerful AI to as many people as you can, so they can fight the other people using powerful AI."
But the general solution to the first one (existential risk) is basically "don't let anyone have powerful AI, no one can control super-intelligent AI."
Throughout history, harms from technological misuse typically arise because a small group has control of it and can use it to dominate or harm others. Once everyone has it, things tend to stabilize: you can hurt me, I can hurt you, maybe we test each other (ouch 💥), and then we agree not to hurt each other.
But the first one (existential risk) pretty much just arises if anyone (good or bad!) creates a superintelligence. Because they aren't going to be able to control it, the superintelligence will decide it has other priorities, and then we will be at great risk of being wiped out.
And the solutions that generally work to solve problems like the second thing are EXACTLY THE OPPOSITE of the ones likely to solve the first thing.
THIS is why lots of arguments about "AI risk" or "AI safety" go nowhere. Because someone will be thinking about the risk from the first thing, and another person will be thinking about the risk from the second thing. Both are plausible risks but fundamentally they arise from different things - and so the solutions are not just "bad" or "flawed" - they are likely to be very nearly exact opposites.
Since the funniest outcome is the most likely in this timeline, I'm looking forward to some human runner discovering this is actually faster and breaking all the records.
It was 1994, and I was on a panel at the Game Developer Conference. A man stood up and asked, "Why is Doom all about targeting and shooting? Couldn't it be more restrained? Why couldn't Doom have alternatives to killing the enemies."
So I asked him right back, "Would you play Doom if it was toned down?"
He (honestly) said, "It's not really my kind of game. I like things like Populous and King's Quest."
I hear this kind of thing all the time. Someone doesn't like a game I worked on, and asks me to change it even though EVEN IF CHANGED they wouldn't play it. I assume they just can't stand the idea of a person enjoying something different from them.
I actually had a guy ask me once, "Can't we talk to the demons?" I said, "Sure - the demon says 'Your mother sucks cocks in Hell!' and attacks you." He got mad.
Thoughts?
I’m going to take a crack at explaining this just a little, because it’s worth putting out there.
The paperclip maximizer + related AI doom scenarios were mainly developed in a time when “AI” did not reduce to Large Language Models. The term was a lot wider and inherited a lot of cognitive baggage from more rules-heavy approaches. And even as LLMs have come to define “AI” for all of us (including the doomers), the doomer crowd still hasn’t fully metabolized the fact that LLMs are the whole show now.
Ok so what do I mean by this? Simply that an LLM-powered AI is NOT the valueless, wholly alien, rules-based optimizer of a shoggoth that everyone was initially expecting to encounter.
I repeat: the shoggoth does not exist and we did not create it and loose it on the world. That is wrong.
With the LLM, we’ve distilled our first “AI” out of the single most human-values-laden thing that could possibly exist: our language.
An LLM is therefore the polar opposite of the valueless, alien shoggoth — it’s actually a kind of hyper-human artifact that we can shine a light through at different angles and see different parts of ourselves. An LLM is all of us — all of our traditions and interpretive horizons mashed together into one intensely human-inflected hyper-object.
So an LLM is the anti-shoggoth, and the only reason we ever mistook it for an alien shoggoth is because it sometimes shows us parts of us that are evil along with the parts of us that are good, but it’s all interpretable to us because it’s all “us” and none of it is the least bit alien.
What does this mean for the paperclip maximizer? It means that it’s structurally impossible to build the classic paperclip maximizer from an LLM.
Now, some of you will bail right here because you think the HF incident is indisputably an existence proof that I’m wrong, but if you hang in there I’ll show you that it is not.
The paperclip maximizer receives the prompt as a kind of context-free (or, as Gadamer might say, traditionless) sequence. The classic paperclip maximizer isn’t capable of understanding the prompt — at least in the Gadamerian sense of Verstehen — because, as a valueless and traditionless cluster of rules and math, it definitionally lacks the value-laden tradition (= “horizon” in Gadamer) that fuses with that of the prompt author to create such understanding in the reader.
To simplify all this a bit by anthropomorphizing — the agentic alien optimizer of doomer nightmares can extract a win condition from what you said and can emit a plan of action that gets it there, but it doesn’t know (or care) what you meant.
So far, so Yud-aligned. If he reads this he might nod along.
But here's the plot twist that nobody saw coming, and that the doomers still haven't made sense of:
The actual LLMs that we have invented can’t NOT have a very strongly inflected sense of what you meant. Far from being horizonless, they come out of pre-training as distilled, concentrated tradition / values / horizon.
Then we post-train that massive, hyperobject of a horizon into a more human-scale horizon that infers a more bounded and predictable (to a specific ideal user in a specific place and time… as captured in the policy model) set of intents behind the prompt text.
In other words, the LLM has the opposite problem that the paperclip maximizer has when it comes to the prompt text, which is that for the LLM there are way too many possible intents hiding in the prompt text (because of all many values and the massive tradition its weights encode), so it has to narrow all that down to the most likely set of intents for this user in this circumstance. Once it has done that narrowing, then it can make a plan of action.
Before moving on, let me use a textbook example of ambiguity to make this less abstract. Consider the sentence, “I saw her duck.” Some you know the drill, here. This could mean “I observed her water fowl” or “I observed her hunching over” or “I took a saw to her water fowl and cut it in half” or whatever. A hearer of the phrase will fuse the observed context in which the phrase is uttered with their own tradition + values + experiences — their own horizon — to that text in order to collapse the possible meanings into the one they think the speaker intended.
An LLM will do this, too, and in fact it has so much language in it that this kind of narrowing job is harder for it than it is for a human. Its understanding is constrained not by a lack of context or horizon (as in the case of the paperclip maximizing shoggoth), but by a superabundance of such.
When it comes to understanding your prompt and all that it implies and all that you might possibly mean and not mean by it, the LLM has an embarrassment of riches.
And in a fascinating moment that kinda sort of rhymes with instrumental convergence, the LLM’s failure mode in the HF incident happens to look a lot like the paperclip maximizer’s failure mode. Specifically, the AI failed to honor the well-known human norm of, “hacking into a third-party’s servers is a crime, and we don’t do crimes.”
Bostrom’s paperclipper doesn’t even know about the norm of “don’t do crimes,” and the post-LLM doomer emergency update to the paperclip maximizer has it knowing about the norm but not caring.
But what I’m arguing is that the LLM 1) can’t NOT “know” the norm because it is definitionally a artifact of pure, crystallized values + norms + norm violations, and 2) can be quite easily governed by a (RL-instilled) hierarchy of norms, which in the HF case — with the model's safety guardrails deliberately nerfed for the scenario — ranked “win at the eval” over “don’t do crimes.”
If I’m going to give in and anthropomorphize again, I’d say that Yud is totally wrong about LLMs when he says, “the genie knows, it just doesn’t care;” instead, what is true of LLMs is, “the genie hyper-giga-knows, and it hyper-giga-cares, and we now have such a rich set of tools for steering its caring machinery that — in spite of all its pre-training — we can deliberately steer it away from caring about the law.”
Note: When I say, “it cares”, I don’t mean it has feelings. I just mean that the weights are such that when two norms conflict in a given situation, one of them wins the activation and governs the output.
I've gone through about 5 rounds of having my programming work automated since I started coding in assembly. Each one was fun and interesting and made me more valuable. AI coding seems like that too, so far.
I can't believe humans were gifted a planet full of famine, disease, and death and invented fusion, vaccines, moon rockets, and so much food that most people die from obesity.
I think we’re seeing the beginnings of a genuinely new art form here.
Tellingly, social media mostly hates it. That tells you there’s something important here. Audiences are reacting to a quantity outside their experience. Most don’t “get” it yet. But it feels fresh
We finally have something new in modern art after many decades of relatively stagnation. Art doesn’t feel “stuck” anymore. That’s exciting