I have wondered if this could actually be a natural protection against the subsequent threats of a powerful ASI. Countless small junky AI viruses ‘brick’ the internet and wean us off it in all our most critical systems — forcing us to get much more resilient against damage by ASI — or even limiting the available distributed compute. There is human harm in this for sure, but if we reduced our dependence on the internet slowly it could be managed. That type of harm would be much less than the alternative.
It’s not that the problems are beyond AI to solve, I’m suggesting that there is no (current) clean and verifiable data to train on of the size and accuracy required by RL. And of course there are a lot of real world bottlenecks too as many have discussed.
We’ll get there, slowly, but it isn’t going to be nearly as fast as people want due to the “junk in/junk out” problem of current human knowledge about disease. First you will have to develop verifiable data based on biological truths, and most of our “disease” frameworks are themselves barely a biological truth.
@RokoMijic I like this version of how things go. Maybe the internet becomes effectively unsafe/insecure but we all get so angry at the AI companies that they lose the chance to get real ASI. Big financial meltdowns are bad, but in light of the alternative these are things we can handle.
@wrangler_warner@OphirGottlieb The problem is not that the question is too hard to solve. The problem is that we don’t have clean data to train it on yet
Riemann Hypothesis is the canary in the coal mine, per EYudkowsky (and everyone else). Will we listen?
Stop, stop, stop.
To be clear -- an AI does NOT have to be broadly super-intelligent to develop the capacity to overcome our globally internet and power-dependent human systems.
It just has to be really good at math and computers...
The Riemann Hypothesis has fallen. RH was the singular math result that had been named, in advance, as the obvious coal mine canary for dangerously advanced AI, over the last three decades. The methods used should generalize to "make AI very good at anything you can definitely test", and unfortunately that category includes "make AI better at making AI".
This is like a private company announcing they have produced a million tons of 15% enriched uranium. It's not 90% enriched weapons-grade uranium, so you could proclaim the company's output is not immediately deadly right now. The concern is that the process line may generalize, to an extinction-level threat, within the unpredictably immediate future. Forecasters still had Riemann at 2035 one week earlier.
This is distinct from earlier events in that earlier you could reasonably ask if AI was selectively picking off famous math problems that turned out to be easier for AI. The Riemann Hypothesis was the single challenge most often named in advance as a warning sign.
Twitter (x) is an insane place. I'm new here. Because i really really care about slowing/stopping the threats of agentic ASI.
But i'm not sure how you guys live like this. ;p
Narrow medical superintelligence would be a great idea. Keyword narrow. Without the simultaneous capacity to shut off water systems and proliferate across the internet and control military grade weapons.
Agentic ASI is suicidal. Narrow medical breakthroughs are possible without THIS type of AI.
These things don't have to be distinct. I am so confused by the advocacy that mixes them up. You can have the benefits without the risks.
Of course AI can speed up biology discoveries. Just nowhere near as fast as math and not via super-powered RL. What i object to is not the direction of what they say, but the size of the promises and the knowing falsehood they represent.
The current pace of approaching ASI brings danger that outweighs those benefits.
If you want biological discoveries, fast, and safely, build NARROW super intelligence focused on biology and chemistry. Stop the agentic ASI push and keep our attention on what matters to real humans.