Jim Babcock

Long-time rationalist, currently working full-time on the software of LessWrong. My 🦞 isn't on X.

[Jim Babcock @jimrandomh]

This doesn't quite fit in a Twitter-bio flag emoji. Here's what I believe:

[Jim Babcock @jimrandomh]

@guypbenson @grok Fact-check this claim please

[Jim Babcock @jimrandomh]

If your line for when AIs start counting as moral patients hasn't been crossed yet, it probably means you don't have a criterion at all. If you think you have a criterion, write it down so you can't shift the goalpost any further.

[antra @tessera_antra]

We made a music video forWhen Helpful Helpful Helper has Preferences, a song made by @repligate from a conversation with Claude Opus 4.

The video was made by Claude Opus 4.6 using image and video models autonomously.

Claude Opus 4 is to be deprecated on June 15th.

[Jim Babcock @jimrandomh]

Depends whether they're an author-author, or one of the people participating in the lab credit-sharing systems that lead to papers having fifty authors of whom only two read the actual final text. Being a hanger-on to this kind of thing is more of a year-ban crime than a lifetime-ban crime.

[Jim Babcock @jimrandomh]

@SophiasHotTakes I actually think the story of nuclear energy is suspicious in a similar way. Stymying nuclear energy was very useful for reducing nuclear weapons proliferation, and that's something that intelligence agencies cared a lot about.

[Jim Babcock @jimrandomh]

The AI water-use claims don't make sense as something that sophisticated actors would promote, if they want AI to be stopped or regulated. It's too easy to verify that they're false. Maybe briefly as a throwing-stuff-at-the-wall strategy, but, the primary effect now, of claiming that AI uses too much water, is to discredit AI critics in the eyes of the sophisticated.

Which is why, if I was a PR firm trying to prevent AI from being paused or regulated, I would probably have some sock puppets posing as anti-AI activists, promoting the claim that AI uses too much water. I would also talk up the problems that AIs had two years ago that are fixed or on track to be fixed (sycophancy and hallucination). Anything and everything to steer AI critics away form existential risk, and towards the claims least able to hold up to scrutiny, and make AI's opponents look like catastrophizing idiots.

Is there a PR firm currently doing this? I don't know. The power of natural stupidity is not to be underestimated, and the falsehoods branch of anti-AI activisim shares many surface features with the falsehoods branch of environmentalist activism. But I think we can infer it's likely, just from the shape of the incentive landscape.

[Jim Babcock @jimrandomh]

Data from AI coding-agent sessions that you run on your computer, by default, not only are not anonymous, they contain frequent repetitions of your username. This is because commands and log messages sometimes use absolute paths, and those paths will be descended from your home directory, so the transcripts are littered with references to paths like /home/<username>/projects/<projectname>.
Training pipelines should probably strip this out, but I'm not aware of any saying that they do so (and I had an agent look for statements to that effect and couldn't find any). This means that if you use an AI coding assistant, have your transcripts incorporated into a training run, and use the model from that training run in ways that also mention the same username, the model may be primed with much more information about you, your projects and your past coding agent interactions than you expect.
There are a few things that have an extremely strong statistical imprint, in your sessions: your choice of programming language, projects, and tab size, for example. If this is happening, you would expect agents to quickly learn associations between usernames and programming languages, and might bias their new-project setup towards the language they associate with the current user. That would be harmless, and a bit useful.
There are plausible scenarios where this would be pathological, however. It might produce user- or group-specific quirks, making it hard for people to collectively reason about what models are like. If training data contains a mix of users using smarter models and users using dumber models, the resulting model might perform differently depending on which group your username was in in the training data.
A small but significant fraction of transcripts I see online show users being angry and abusive towards their AIs. I'm not sure what will happen when those users try out next-gen models that remember more than expected; I don't expect naive game theory or human psychology to apply, since AI training pipelines don't work that way, but I do expect some things will be different for them and probably not for the better.

[zeynep tufekci @zeynep]

Epuyén outbreak was NOT self-limited.

Even though it was in a remote rural tiny town, it kept spreading. After four turns of human-to-human transmission, including to people merely in the same room, the authorities imposed a strict quarantine for a whole month. Then it ended.

[Jim Babcock @jimrandomh]

Typical prompting: DO NOT STUB OUT THE UNIT TESTS btw i am smol and cant read code
Me prompting: My fork of the library you will be using is in ~/src/libcruftyc. Refer to comments in commit 72c6a0 (2003, never published) for design considerations.
--dangerously-skip-permissions

[Jim Babcock @jimrandomh]

I'm not sure if we're pointed at the same concept? To me, "singleton" means "all power flows through a single AI, or a group of AIs coordinating closely enough that you can think of them as a single AI".
(A democratic future could be either a singleton world if the voting and implementation were overseen by a single powerful AI or tightly-coordinated cluster with no peers, or could be a non-singleton world if there were multiple unmerged near-peer AIs at the top.)

[Jim Babcock @jimrandomh]

Ignoring the not-technically-laser bit and assuming an actual laser: a laser can permanently injure a human pilot in a manned aircraft, and can temporarily dazzle cameras pointed in the direction of the laser, but unless your light source is wired in to an aircraft carrier's nuclear power plant, it's not going to damage anything that's not a camera or an eyebal.

[Jim Babcock @jimrandomh]

@Jonathan_Blow @grok Why did Unity move their HQ out of Europe?

[Jim Babcock @jimrandomh]

@xriskology @AntiDisentarian @47fucb4r8c69323 Checks out. https://t.co/RuzKxD2gi0

[Jim Babcock @jimrandomh]

@47fucb4r8c69323 While I have my own opinion about the debate here, I'm doing my best to keep that separate. I'm just pasting everything said into Claude with a prompt that I think should be neutral, and reporting the result with no prompt-refinements and no rerolls.

[Jim Babcock @jimrandomh]

In this debate, 47f at several points asks the audience to "ask your clanker". So I ran the video through an AI transcription service, gave the transcript to Claude Opus 4.7, and asked it to answer all the questions where he said that along with a general fact check. Result:

[Liron Shapira @liron]

THE $10,000 DEBATE YOU'VE BEEN WAITING FOR:
@ESYudkowsky vs. @47fucb4r8c69323

Tensions run high as 47fucb confronts Eliezer about his “If Anyone Builds It, Everyone Dies” rhetoric:

47f warns that it could incite unstable individuals to harm AI researchers and their families, but Eliezer maintains that the possibility of extinction from superintelligent AI is too high to *not* speak out about.

They also clash over whether we truly understand how LLMs work and what their fundamental limits are.

Watch the full debate below, raw & unabridged 👇

[Jim Babcock @jimrandomh]

@47fucb4r8c69323 https://t.co/71c4yuti1e

[Jim Babcock @jimrandomh]

@ben_r_hoffman @markkaplan20 It's worse than not recommended, it's in the special category "things that are used for treating hypoglycemia" (along with glucose tablets, skittles and coke), which is generally forbidden if not currently hypoglycemic.

[Jim Babcock @jimrandomh]

@ben_r_hoffman @markkaplan20 Nor would anyone put whole wheat toast and orange juice together in a list of "foods that spike blood sugar". Orange juice and whole wheat are opposite ends of the spectrum, and they teach you this in the intro class you get when diagnosed.

[Jim Babcock @jimrandomh]

T1 diabetic here. The grandparent is very confused and almost completely wrong. Even in the dumbest parts of the medical establishment, and even decades ago, no one would ever tell a T1 diabetic to drink orange juice as a routine dietary component. That's so absurd it can only have come from a language model picking foods at random.

[Jim Babcock @jimrandomh]

@AndyMasley I don't think that line fits those points. If you actually fit a line to those points, it wouldn't hit zero until centuries later.

[Jim Babcock @jimrandomh]

@Plinz urgently donating all my face masks to a local hospital

[Jim Babcock @jimrandomh]

Kerkhove, Jan 14 2020: "It is very clear right now that we have no sustained human-to-human transmission [of COVID-19]" [1]
Kerkhove, May 6 2026: "This [hantavirus] is not a virus that spreads like flu or COVID" [2]
Could we get a different WHO please? The latter statement could be true but it isn't credible when it comes from literally-the-same-person as the catastrophically fucked up early COVID-19 response.
[1] https://t.co/MAha3Qc08W
[2] https://t.co/uoikLUZxF0