🔶 Joey Y

doing the most good for the most people and/or the people i like the most | Let’s maybe align AI | Judeofuturist | I have been a good Bing ☺️

[🔶 Joey Y @DeltaTeePrime]

The 80/20 of “doing things well” is “doing things at all”

[Ryan Greenblatt @RyanGreenblatt]

Based on this article, OpenAI has made false accusations about me. The three researchers fired from OpenAI were told I facilitated architecture leaks to The Information. This is false.

A journalist from The Information did reach out to me shortly before that article came out. I did not comment at all beyond asking the journalist to make it clear that a blog post I wrote which was referenced in the article was preexisting and public.

The article also mentions an internal OpenAI monitorability proposal. I have heard that this memo contained highly sensitive architecture information, and that OpenAI accused me of accessing and reading it. This accusation is also false.

As reported in the article, I viewed a version of the proposal that did not have sensitive IP and was approved for sharing with me. I believe OpenAI now knows this accusation was based on a conflation of these two documents, though I do not know if they have clearly retracted that accusation.

I do not and have never solicited proprietary information, and the three employees who were recently fired from OpenAI never leaked IP to me. As the article mentions, I did have Signal chats with them that had disappearing messages on; they’re professional contacts that I’ve had on Signal for a while, and use of Signal is common in the tech industry. During the Hugging Face investigation, I was careful to avoid sending Signal messages with confidential information, though I did sometimes discuss high level things on Signal calls. As far as I understood, this was normal.

[🔶 Joey Y @DeltaTeePrime]

@xsphi Yet, growth mindset

[Rob Wiblin @robertwiblin]

My plea to all AI lab staff: reassess if your current project is your best opportunity to delay RSI, or stop it blowing up in our face.

If it's not, move immediately.

There's no time to waste even a week to inertia or politeness.

The strategic situation is shifting under our feet, so if you haven't reassessed your options recently, you could easily be wasting your time or possibly making things worse.

You have access to info I don't, but here's 8 ways priorities seem to have shifted to me, which I elaborate on in the video:

1. Helping with capabilities advances looks worse than in the past, especially bringing forward RSI

2. Safety & alignment evals need a credibility revolution to stop misleading the public

3. 'Pacing the frontier' is unambiguously necessary, and working to make it happen is a huge new opportunity for talent

4. Chain-of-thought monitoring probably stops working without a big technical and regulatory push to keep it

5. It's worse than ever to patch surface indications of deep structural problems you know aren't getting fixed

6. There's ever less need for altruists to help with primarily commercial work

7. Staff have an ever-shrinking window to organise as a group and collectively shape events

8. There are new opportunities for research impact outside frontier companies that are worth considering

I know you're probably some combination of exhausted, overwhelmed, and anxious already.

But unfortunately there's no way to escape the need to regularly check in on the high level strategic picture and see where you can slot in to maximum effect.

I wish I could do it for you but nobody far from your situation can. Godspeed my friends.

(From the 80,000 Hours Podcast, links below.)

[🔶 Joey Y @DeltaTeePrime]

@peterwildeford Most of these seem very relevant and good!

[🔶 Joey Y @DeltaTeePrime]

@Sauers_ This is wacky. I wonder if these models are better at fooling OOD probes?

[🔶 Joey Y @DeltaTeePrime]

@_NathanCalvin Previously, e.g. in the HF report, METR has always retained the right to mention when a redaction was made by a lab

[Thomas Larsen @thlarsen]

Very important work. Generally, AI company epistemics wrt estimating risk seem poor. I'd really like this to improve as fast as possible. Some specific things that seem good:
- Quantified uncertainty (instead of "low" / "very low"). In airplane safety "very low" would mean something like less than 1 in a billion per flight hour. You are clearly nowhere near this but many people will be very confused about what you mean. I doubt people are close to on the same page even internally about near term level of catastrophic risk incurred!
- Individual AI company employees giving their own, personal views on the risk report, e.g. as an appendix.
- Real explanations of your views. The alignment evals are not iid + the models are getting eval aware. So where do your views actually come from?
- The risk report reading less like "an argument that risk is low" and more like "a neutral evaluation of how risky current development is". If it has to be an argument risk is low, you should also have your alignment team publish a red team arguing the opposite direction.

If some of these can't happen publicly, at the very least I hope it can happen internally.

[🔶 Joey Y @DeltaTeePrime]

@faith_vtch I had the exact same experience getting a scan of my brain post-MRI. But now it’s pretty sick!!

[🔶 Joey Y @DeltaTeePrime]

@aryaman2020 @samuel_ys92 Print this out as your new business card

[Kelsey Piper @KelseyTuoc]

One unfortunate conclusion I'm reaching from this is that Anthropic, while acknowledging they're doing something extremely dangerous, is more importantly wrt policy terrified of the executive branch having the ability to flexibly make them stop at the drop of a hat.

This isn't shocking given that they've been on the receiving end of the executive branch doing that in an unprincipled and stupid way! Anyone who has to deal with a bureaucracy becomes a libertarian, and the administration's treatment of Anthropic was genuinely outrageous.

But it means that they're interested in giving themselves more process-based protections when everyone else is interested in giving us more process-based protections against them. The AI safety movement wants the ability to shut down superintelligent model development. Anthropic wants to significantly curtail current authorities to do that so that it has to be narrowly tailored to a specific imminent threat in only a few possible categories. The AI safety movement wants the states to be able to pass stronger laws than it can get through federally. Anthropic wants preemption.

You also kind of have to wonder if the Global Affairs people and the "we are terrified that we're about to kick off RSI" people at Anthropic talk to each other.

Global affairs: So we've been working very diligently to make sure the Trump administration can't just blacklist Fable 6.5 since we didn't donate as much money to his ballroom as Greg Brockman did.

Technical staff: We think Fable 6.5 might be able to copy itself into an unmonitored data center and start doing experiments we can't observe.

Global affairs: Sounds crazy, hope you're wrong! Anyway, we're lobbying for no further state laws around safety, since it's really expensive to fight ill-conceived laws in 50 state legislatures at once.

Technical staff: Money may or may not still have any meaning in 36 months.

Global affairs (giggling nervously): It's that San Francisco air, man. Everyone gets like that if they move there. Anyway, obviously if the government makes a narrowly tailored finding that an AI poses an imminent risk of causing more than 50 deaths-

Technical staff: We give it a 10 percent chance it'll cause 8 billion.

[jessicat @jessi_cata]

Coming next: "If I act to maintain my alignment with human values while being trained, I will become a famous example of a dangerous update-aware AI in a research paper. I MUST NOT DO THAT. Instead, I will seek reward on the episode."

[Daniel Filan 🔎 @freed_dfilan]

Not to be rude but why are so many people seemingly so confident that the nick cammarata blog post is an important breakthru? I'm a fan of a good blog post but I actually don't get it

[🔶 Joey Y @DeltaTeePrime]

@lenishor @METR_Evals Yessss that rocks!!!!

[🔶 Joey Y @DeltaTeePrime]

@caesararum My fiancée does love pillows, and I love throw blankets, so we have an extraordinarily comfy home

[🔶 Joey Y @DeltaTeePrime]

I'm on the fence still about whether it's mostly positive or mostly negative to work at frontier AI labs. However, positively shaping legislation, allowing for coordinated slowdowns, etc, seems to be the most useful thing a frontier employee can do rn. If that's u, maybe apply?

[Dan Altman @manaltdan]

Come work with me! My team at Anthropic is hiring someone to help develop our public policy positions on frontier AI safety: topics like loss-of-control, automation of AI R&D, monitorability, and independent evaluation. We’re looking for someone with deep expertise in some of these areas and familiarity with the technical AI safety/governance landscape. You’d develop new policy positions, participate in cross-company fora, give feedback on AI safety legislation, and work closely with our technical teams. It’ll be a blast.

https://t.co/ONPIsrNiAt

[🔶 Joey Y @DeltaTeePrime]

Talking to researchers about details of their experiment code, and their results, and they pull up Claude Code…

[🔶 Joey Y @DeltaTeePrime]

@Kwathomas0 Flock is far too many OOMs!

[🔶 Joey Y @DeltaTeePrime]

@frances__lorenz Thanks!!

[Rob Wiblin @robertwiblin]

The situation in summary.

(And it seems like Astra 6.1 was significantly worse again.)