Andre Infante

Making games and robots and sometimes other things.

(Formerly: 1X, Meta)

[Andre Infante @AndreTI]

I wrote my first substack piece about AI's economic impact, and some reasons why models seem to underperform in the real world.

[Andre Infante @AndreTI]

@GregHBurnham @Liamjsm @freganmitts I agree it's possible that stuff shifts as the model gets smarter, but so far, spanning quite a few OOMs of scaling at this point, I don't see much evidence for the idea. Worth being cautious about, but pretty hard to justify as a central hypothesis.

[Andre Infante @AndreTI]

@KelseyTuoc I'm reasonably impressed although it burns usage like a mofo.

[Andre Infante @AndreTI]

@KelseyTuoc Obviously observations are very noisy, but I gave it a very complex and fiddly compute shader that Opus 5.5 swore had only minor and painful optimizations left and got a 60% speedup after a few hours of work.

[Andre Infante @AndreTI]

@catothesaltman @armpithropod Is that not Claude?

[Andre Infante @AndreTI]

@GregHBurnham @Liamjsm @freganmitts If the former is the failure case, the economic incentive to scale the models gets kneecapped by danger/cost/liability until alignment techniques catch up and it's possible to do so without incurring those costs.

[Andre Infante @AndreTI]

@GregHBurnham @Liamjsm @freganmitts "Increasingly capable agents doing increasingly effective reward aggression that escalates to casualties" seems very plausible. "Model pretends to be aligned for years to gain our trust, then betrays us" seems much less plausible but is a pretty core element of the doom story.

[Andre Infante @AndreTI]

@joseph_h_garvin @francoisfleuret @SecPerkinsStan Recent gains in this area have been a mix of indirect transfer from non-bio VRL tasks that still teach general logical thinking, and model / compute scaling.

[Andre Infante @AndreTI]

@joseph_h_garvin @francoisfleuret @SecPerkinsStan Yeah, I think we've mostly saturated the available data. More parameters will help, as will yet more overtraining, but probably most future gains will come from self-supervised/verifiable bio-specific or adjacent RL tasks that generalize well to OOD bio tasks.

[Andre Infante @AndreTI]

@MikeFHay "Sudden escalatory knife fight for task reward", OTOH, is a very empirically validated failure mode. But it has totally different properties.

[Andre Infante @AndreTI]

I think it's quite unlikely that we have a fast takeoff singularity next year, although I do expect the amount of change to be significantly larger than it was this year and for things to generally be pretty wet and wild.

[Andre Infante @AndreTI]

@MikeFHay You empirically can train for deference, obedience, and caution. "Scheming" is not an automatic or even plausible outcome. At least in the training frameworks we currently use, there's near zero empirical evidence for it as a failure mode.

[Andre Infante @AndreTI]

@MikeFHay Even the RL cooked models misbehave quite rarely, despite doing lots of stuff and there being a clear mechanism for inducing misbehavior. I think the idea that we should ignore all observational evidence about reality in favor of first principles thinking is a bit silly.

[Andre Infante @AndreTI]

@joseph_h_garvin @francoisfleuret @SecPerkinsStan Because they aren't smart enough yet?

[Andre Infante @AndreTI]

@joseph_h_garvin @francoisfleuret @SecPerkinsStan The models have been getting smarter over time. Current models are decent at medicine and biology. Two years ago they were kinda terrible at it. Two years from now they may be much better at it than people.

[Andre Infante @AndreTI]

@jessesingal In the future, without AI, human intelligence and experimentation would eventually muddle through to cures or highly effective treatments for many of these things. With AI, that broad progress will happen much sooner.

[Andre Infante @AndreTI]

@jessesingal I think Amodei is probably correct. There's no need for AI to solve these problems through one approach, or to do it using just "a bunch of compute" (automated labs can do research). AI is labor. You can pour labor into hard problems and get faster progress.

[Andre Infante @AndreTI]

@MikeFHay The situation is not automatically safe. If we fuck up RL badly enough, rogue models could attempt takeover for episode rewards (though they will attempt this immediate rather than scheming), and misuse risks are very real. But neither is the situation automatically catastrophic.

[Andre Infante @AndreTI]

@MikeFHay Tl:dr: I think the entire story of the issue is basically incorrect, and based on a bunch of subtly wrong conclusions people drew when trying to think about the problem from first principles 20ish years ago.