Unlicensed philosophy practitioner, x-risk themed jester, soul exhibitionist, alethophant. Sentences starting with lowercase letters may not be assertions.
The Riemann Hypothesis has fallen. RH was the singular math result that had been named, in advance, as the obvious coal mine canary for dangerously advanced AI, over the last three decades. The methods used should generalize to "make AI very good at anything you can definitely test", and unfortunately that category includes "make AI better at making AI".
This is like a private company announcing they have produced a million tons of 15% enriched uranium. It's not 90% enriched weapons-grade uranium, so you could proclaim the company's output is not immediately deadly right now. The concern is that the process line may generalize, to an extinction-level threat, within the unpredictably immediate future. Forecasters still had Riemann at 2035 one week earlier.
This is distinct from earlier events in that earlier you could reasonably ask if AI was selectively picking off famous math problems that turned out to be easier for AI. The Riemann Hypothesis was the single challenge most often named in advance as a warning sign.
I would like it if leadership at the frontier labs made statements about how they are planning to not take over the world, or else how that is their plan and why they think it is good.
@kave_rennedy I’ve been thinking about how to model an LLM as a dist a lot and thinking about how like to apply standard info rheory to that perspective. That said, I don’t currently think this will help anyone build better ais.
I am willing to go out on a limb and say that openai will not be able to formalize “this ai will not murder you” in lean. If they try, they’ll just run into all of the old problems w gofai alignment people used to talk about in 1999 or whatever.
imo this is confused thinking; you can automate safety research while being able to verify the results and therefore not having handed off human oversight. in fact, our powers of formal verification will only grow the better we understand these systems
i'm giving my gf employee-level access to my phone, just like openai is giving its evaluators. she can only read texts i've whitelisted, only talk to girls i've pre-approved, and can't open signal or snapchat. anything outside the agreed time periods is out of scope
you don't get it. when there's an emergency, i'd rather have no ubers available because seeing that there is an option available that is more expensive than i am willing to pay makes me sad