@gcolbourn@ohabryka@l_mc_nally@So8res MIRI is now doing collab projects I like and guess don't cost much, such as https://t.co/KmcPkS2d4B . So the marginal returns aren't that low yet.
@gcolbourn@ohabryka@l_mc_nally@So8res Clarification: the list is about relative orderings. Most orgs are >> zero wrt effectiveness. Few orgs (Anthropic, Stop AI, CoeffGiv, PauseAI US, ...) are such that I'd rather see money burnt than go to them.
After 120s thinking + skimming your LW post for org names, here's whom I'd prioritize over ControlAI: PauseAI non-US, MIRI, AI Futures, Palisade Research. Maybe Lightcone Infrastructure.
This list easily can change if I'd recall / see more org names, and if I actually looked at the cost side and funding side more, but based on value and vague guesses I'd scale those orgs up more than ControlAI.
@gcolbourn@ohabryka@l_mc_nally@So8res Tbc I'd bet ControlAI is epistemically careful, but this is a central example of more downstream risks / bad strategies I'd like to not see funded.
> Their funding needs hinge on scaling up the things you refer to as visible impact.
If that's true (I'm not privy to info on how they talk to funders) then yay. I'm somewhat worried that I have to trust ControlAI when they say that in private conversations with politicians they only say true things instead of saying sth that's short-term getting politicians on board and medium-term makes them angry at having been misled.
@ohabryka@gcolbourn@l_mc_nally@So8res Tbc my forecast of downstream risk is not sensitive near the boundary between "lying" and "misleading" as in colloquial use, both is bad for similar reasons to similar degrees.
I'm less worried about them having tried sth this unlikely to work (talking with openai to get them to have a correct model of doom), and more that they then lied about their success? I now cannot trust Gabe / ControlAI about similar claims about their new strategies either. ControlAI has visible impact and visible actions, e.g. interviews in the German news, e.g. lists of politicians, from which I infer they're doing labor that's useful, not just cashing in on other people's work. But if their funding needs hinge on less observable evidence / on trust, then that's bad.
Non-naive consequentialist reasoning also says to disincentivize lying and censorship, beyond the above loss of trust, e.g. bc of effects on the community and whether others start lying as well, which matters even under 12mo timelines (or, rather, matters as part of the gambles on longer timelines that we can turn into wins more plausibly than short timelines ).
I don't understand how this follows, Greg. Afaict Habryka says that he searched over subconscious motivations and found none that has relevant explanatory power and prior probability here, compared to just taking himself at face-value here.
No explicit statement made about what methods he used to come to that belief, but recognizing one's present or past subconscious motivations is a skill people can have to a degree (portfolio of techniques and training oneself and accumulated self-knowledge).
To me that's evidence he's not acting out of subconscious motivations this time, including pettyness, bc he'd have noticed and said sth / changed course. I'd also not have expected a different answer, so I'm not sure how your update follows?
@liron Sounds indistinguishable from hindsight bias? More robust against hindsight bias is maybe that often the big secrets were indeed discussed 6mo prior (or, uh, 20y prior), and then I indeed agreed / found it plausible enough.
@TetraspaceWest First meeting of the reading group.
"ATU 5, pl. 004, W 6245,d1 (P000754)"
Administrative tablet excavated in Uruk (mod. Warka), dated to the Uruk V (ca. 3500-3350 BC).
Text:
front: 2 2 2
back: 1
https://t.co/on3ULefJZv
Huh, yes, that's how it works? I'll be wrong if we survive, relative to somebody who expected survival a lot more, on that question. The size of the universe, or whether my counterfactuals are real somewhere else and I just have indexical instead of epistemic uncertainty, doesn't matter here.
Usually people however also say things like "we'll survive because <reason>" and then we can later figure out whether that reason was correct. I.e. Gettier problem: do they know for correct reasons that we'll survive or are they correct in a way that's unrelated to their reasoning ability?
AI companies admit what they’re building might kill everyone. We don't have to agree to this.
Add your name at https://t.co/Pcg0clMbfV to say: we do not accept your terms.
Already priced in. And my best guess is that a whole lot of myself never affects what I say aloud, so even if words leave atom-scale traces on the walls, that's not enough. Muscle twitches may already not be recoverable, I'd guess. And neural memories I form and never access again until I'm warm and dead leave even less of an impact.
> set up an RPG about running an international treaty to prevent the creation of superintelligence
> tell sol 6 to play the director of ISIA
> okay-ish competent at negotiating a training stop
> then micromanages when employees should sleep
> 2h later it's still day 1