Elias

[Elias @Eliasygn]

@cindyxywu Maybe we just get more sample efficient learning algos, which utilizes our already existing envs much better and so we don’t need much more/aren’t really bottlenecked by them

[Elias @Eliasygn]

@krichard121212 It's kind of a bitter lesson because robot policies have much more inductive biases than your pre-trained transformer.

[Elias @Eliasygn]

@MelindaBChu1 @ashiqeLSD Why can't LLMs automate experiments? You have been shockingly silent on this topic.

[Elias @Eliasygn]

@IruOota @DonCorletony @NOTTHISWAYQQ The accumulation argument doesn't really work because, in order for it to make sense, all of these details would have to be in the same scene viewed by the player. Otherwise, it wouldn't even be rendered. And it's very unlikely that the majority of them will be in the same scene.

[Elias @Eliasygn]

@IruOota @DonCorletony @NOTTHISWAYQQ What sets GTA apart from every other video game is literally this attention to detail. Otherwise, GTA 4 to GTA 5 wouldn't have been such a big jump, and GTA 5 to GTA 6 wouldn't be such a big jump either.
The reason for 30fps is things like ray tracing and broader graphics.

[Elias @Eliasygn]

@IruOota @DonCorletony @NOTTHISWAYQQ 200gb is nothing considering we have terabytes of SSDs already, also how does it decrease performance? I don’t really think this impacts performance at all

[Elias @Eliasygn]

@IruOota @DonCorletony @NOTTHISWAYQQ Hinders the game in what way

[Elias @Eliasygn]

@ramez @HarksenNiels @CurtTigges These robots are specialized policies not general llm, which, I think will overtake robot policies

[Elias @Eliasygn]

@ramez @Afinetheorem That’s a 3-month difference, plus another 2.5 months since August so 5.5 months in total. And given that Anthropic reported a 2x speedup I’d expect the numbers to look very different by now.

[Elias @Eliasygn]

@ramez @Afinetheorem Note that OpenAI is behind Anthropic wrt automating AI R&D, aswell as the fact that this data is pretty old now and we don’t know task successes for humans, like a 2x speedup is perfectly compatible with lacking horizons >p80

[Elias @Eliasygn]

@RubenGallego Doesn’t Altman and Amodei want guardrails aswell? Afaik they advocate for more regulations, so your post is misleading at the very least

[Elias @Eliasygn]

@ar0cket1 oom improvement in e.g. data efficiency (by a better arch) would be a huge deal, we haven’t seen that in years

[Elias @Eliasygn]

@Bayesian0_0 @cqkten > I could see algo improvements being super bottlenecked on compute and compute limited regime

Why? The transformer was discovered on very little compute, as well as most architectural tweaks

[Elias @Eliasygn]

@Thirumagal_12 @AndrewCurran_ @TheShmanuel They are already slowing down to some extend

[Elias @Eliasygn]

@heartwr3nched @RepMcCormick Yeah true, but if it turns out it’s just a data problem it might be enough for automated R&D Ais to solve alignment. Anyway, current capabilities a far from being dangerous (except in the cyber space) and thus irrational to pause ai now if we can get a net benefit by continuing

[Elias @Eliasygn]

@heartwr3nched @RepMcCormick I agree we need to have conversations about ASI, but like most of the alignment issues come from bad data (unspecified reward function, bad envs) not from models being intrinsically evil afaik. So like can’t we just continue (but pace) until right before asi

[Elias @Eliasygn]

@heartwr3nched @RepMcCormick They’re not top human level at maths yet, afaik they haven’t done any new theory building or became super creative, just solving problems with tools which you would expect from brute force (not to discredit the performance, ofcourse)

[Elias @Eliasygn]

@heartwr3nched @RepMcCormick Compute bottlenecked wrt to sustaining progress in the next 12 but we can’t get an infinite intelligence explosion cuz there’s only so many experiments to run, like i don’t believe in that super fast takeoff