Armin Ghofrani

compressor

[Armin Ghofrani @armin_ghofrani]

@CJHandmer I think you just verify down to the hardware which is possible but tedious. But we have tedium-crushing machines now that can just do it!

[Armin Ghofrani @armin_ghofrani]

@BenJames_____ Creativity Inc. is a great book on this!

[Armin Ghofrani @armin_ghofrani]

@1a3orn Interesting thread, thank you! This part seems like a crux to me. Could you give examples of situations where you think it may be best to ban open-weight models?

[Rhys @RhysSullivan]

if bryan johnson actually cared about living forever he'd solve the alignment problem

[Casey Handmer @CJHandmer]

This is correct but not indefinitely. One more oom of energy abundance and we will recycle atomically.

[Rob Miles @robertskmiles]

@cauliflwr_human I expect they will say that they're only doing a little bit of neuralese as a treat, and the chain-of-thought is not too badly affected, and then they'll steadily crank that dial up and up because there's no clear fence on that slippery slope

[Nate Soares ⏹️ @So8res]

fwiw, my accounting of the comparative track record is:

(a) christiano & greenblat come out looking better on LLMs. they nailed it. points to them. but iirc this is a spot where yud was like "idk maybe that'll happen; your model is more confident about the specifics than mine; regardless, it would not make alignment easy". they get points for specificity, but yud wasn't exactly falsified here, afaict.

(b) yud comes out looking better on timing (cf. the IMO gold medal bet).

(c) the big yudkowskian miss in my book was him expecting humanity would need to understand intelligence in order to create it (back in the '90s and '00s). iiuc he updated away from that around AlphaGo at the latest.

overall it's looking to me like both christiano and yudkowsky are panning out in the places where they were most confident: christiano was like "before we see full RSI & unrecoverable takeover attempts, we should see AI make minor contributions to AI research & limited breakouts", and that turned out to be totally true. yudkowsky was like "y'all are sleeping on how fast this is gonna move, and the alignment stuff isn't gonna be easy" and that also turned out to be true.

the big yudkowskian predictions -- that a software-only intelligence explosion is possible; that the resulting AIs won't converge on alignment w/ human values -- still haven't been observed (tho i'd say the evidence is starting to mount up), but it really isn't looking to me like either view has been falsified. it's looking more to me like both views have been stacking up the wins (in the places where they made their strongest predictions). which, as you say, is worrying.

[Daniel Litt @littmath]

@pli_cachete In practice I think most out of distribution generalization (by humans) is still in an epsilon neighborhood of the distribution. Unclear to me if the trend for AI is different.

[Armin Ghofrani @armin_ghofrani]

https://t.co/jivBZra2Zn

[Armin Ghofrani @armin_ghofrani]

I wrote up a post on Friday about how I model the pace of progress in the sciences and how AI might be able to help us accelerate, particularly in empirical fields.