Really interesting read, I started out with a list of disagreements but as I went through each one his scenario seemed more and more reasonable. I think the remaining issues I have are:
In Noah's scenario he grants a jailbroken model is likely to be behind the frontier, this means the capabilities reachable by that model are in a space defenders have had time to survey with more powerful tools. This really changes the relevant defense question away from 'can we catch up to an outbreak?' into 'is there enough lead time to prepare countermeasures before an attack?' or 'can frontier models predict attack vectors of older models?'
The mail ordering of such a virus is doing a lot of work in his example. It requires:
a secret lab taking orders from unverified clients with minimal-to-no screening
uneventful physical delivery of the final product through post and customs
the lab not being discovered by a regulator using the same (or better) models as the bioterrorist
Each of these is a significant point of failure which would require minimal interruption to collapse the plot, he is correct in that it may only take one success however we should see some failures first.
Following up on there being a decent chance for a failure: "no one has done this yet", the rebuttal Noah gives is effectively survivorship bias which, while correct about a full doomsday scenario, is not correct about a near-miss scenario. We'd expect to see some visible failed attempts before a success suggesting this isn't an imminent threat — not to say it isn't an important one.
Ultimately I do agree with Noah's prescriptions, I'm just not sure that things are quite as scary as many believe right now.
Lots of AI discourse again about how we will get nuked. I would like everyone to instead read the absolutely amazing piece by @Noahpinion which shows the real risk is with not with computer viruses, but human ones.
Link below, its worth it trust me
@dancantstream And yes likely isn't proof but it looks a lot stronger than most of these youtuber/streamer cases coming up in recent times. At least to a layman
@dancantstream I know lawsuits in this space are kind of a joke at this point but if his claims result in any articles or really any media attention at all do you not have a legitimate defamation case since he is likely costing your business money?
@dancantstream I don’t understand why twitch has been consistently willing to throw away what was once a monopoly on the live streaming space or why Amazon allows them to