The US-China SI hotline is a great first step towards mitigating the national security risks presented by SI. @GovindPimpale and I argue that a few design choices would make the hotline a durable institution, unlike it's predecessors. Post linked below.
@chrisharig99 and @GovindPimpale argue that the new US-China Super Intelligence hotline is a major step forward, and outline how clear risk frameworks, technical expertise, and durable communication channels can ensure its success.
Policy Gradients piece linked below.
people have been asking 'if models are so good where is all the new software' for the last year or so. As always, look to gaming for early evidence of the future
It means something, but what? Where are the fabled mega projects completed in weeks instead of years? Novices and veterans, those with good taste and bad, great ideas and no ideas are all using the same models. When does it happen?
We estimate that Claude Opus 4.6 has a 50%-time-horizon of around 14.5 hours (95% CI of 6 hrs to 98 hrs) on software tasks. While this is the highest point estimate we’ve reported, this measurement is extremely noisy because our current task suite is nearly saturated.
I guess the models could be more benchmaxxed than they seem. Alternative guesses are either we way overestimated how much writing code matters, or the floodgates will open at some capability level that seems arbitrary. Maybe all three.
@CharlesD353 Worried about cybersecurity interventions, prosaic alignment and control being enough to block these and/or frog boiling from bad misalignment that doesn’t cause a lot of damage
@micsolana Yarvin said something awhile back about "just only let the AI use GET requests" (I'll take the charitable view that he meant sandboxing will be enough), and I haven't seen any interesting recent takes
@micsolana I remember Tyler saying something like "go through peer review" prior to his recent "short some stocks". Just kinda feels like stalemate where he's waiting for more evidence. Pinker is engaging and I'd guess has a bigger audience?
I’d expect this less? I think the @BetleyJan theory that a misaligned persona is activated on tasks with verifiable tasks is more likely, but maybe you mean something else.
I do think there are a range of patterns less than full eval awareness, and possibly more complex in ways, that don’t get a ton of attention. Not sure if any apply here.
@xeophon@Matthewagi Can you clarify the connection? Like, people are scared of sudden generalizations to capabilities we don’t have data for, but we haven’t seen this (meaningfully) happen yet?
@1a3orn@FioraStarlight Best takes here? I assumed this was just scaled up RL with multiple agents involved. Do you think it’s more because this is an open space or because the incidents indicate there’s something funky about it?
@herbiebradley How sure are you that models will be able to detect poisoned data? It might take a pretty significant amount of inference compute to do, and even then I’m not sure it is defense dominant. Phantom transfer is my only intuition for this though.
@herbiebradley@StephenLCasper I feel like the whole “let’s cover up our tracks” in the HF incident could be a similar generalization to warding off a cyber attack. Memetic angle is interesting.
@tenobrus I think most of the good open source worlds become much clearer under pacing, pauses and stops. The prime intellect and tinkers of the world make much more sense as long term products when there isn’t a god on the horizon
@dylanbowmanSF@miclchen Do we have any info on their data shortages modulo distillation? I’d expect US lead to grow for a bit if distillation was ~fully cutoff but maybe they can just fill the gaps relatively easy?