@noahatwhere Only when we could externalise computation did the predictability vs interpretability go from correlated to anticorrelated. The exact moment when this phase transition happens will always be a function of the observer's compute power (yes, bcof @SamuelAlbanie, i'm Moravec pilled)
@noahatwhere I would argue that science progressed and benefited so far from "elegant" predictors only bc the model had to be simple enough so that humans could internalise and operate on it. Had we had epsilon bigger computational complexity we would get annoyed at delta(eps) hairier models.
@norpadon@hikettei not going to lie, this is so far one of the best reviews from a community feedback perspective: they read the paper, got the idea but something didnt look kosher at first with the comparison -> decision: benefit of the doubt, accept, come and convince me IRL. This is ideal.
I know this is a wild bet, but please, take the data and see if this check out. There's just so much more to learn from it:
https://t.co/4ouwGoqgDf
https://t.co/tTMMcXzXUM
... than the most sophisticated cognitive theory based models. There's no incentive for this stuff to happen. I think there may be some abstract universal circuitry that all intelligent systems have to independently rediscover.
@ChrSzegedy minor correction: not solved but irrelevant. I started with GAN theory (bounds of optimal transport approx) but then came BigGAN.
I think knowledge transfer (to humans) will remain to be a fun challenging & technical area for a while,that's orthogonal to safety and alignment
After 10 incredible years of not being at @GoogleDeepMind, I’ve decided to quit.
The decision to stop not working there was not taken lightly.
I'll be joining @sirbayes' team to continue the work on curriculum design, towards a general computational model of teaching.
@suchenzang alright, but there's a lurking feeling that there's some duality to this, isn't there?
thinking trace tokens may just be biased noise fuel (with some non trivial correlation to interpretable meaning) for continuous latent space exploration
@thsottiaux multifactor authentication. Codex crumbles - there's no headless remote mode similar to Claude Code's remote control. Sol Ultra barely knew the Codex UIs layout and it took an ~hour to finally get a remote agent work on an MFA slurm cluster. Still has to auth every time.
wow. actually true. the same way GH is crumbling now, under yottabites of slop code, HF will soon see a surge of everyone uploading agent trained slopcheckpoints for RSI flagplanting.
Listening to @SemiAnalysis_ and @dwarkesh_sp Dylan rants made me realise: at one point Dylan was complaining about "releasing too much alpha" - however if he didn't share anything no one would care listening.
So maybe the meta is to share just enough to influence the market?