Bayesian @LessOnline&@Manifest

#1 AI forecaster on Manifold Markets (and #2 across all categories)
want everything to make sense

[Bayesian @LessOnline&@Manifest @Bayesian0_0]

@joseph_h_garvin At some point sure, but probably OOMs higher than this, there is a lot of code left to write

[Bayesian @LessOnline&@Manifest @Bayesian0_0]

@teortaxesTex benchmarks are noisy, 129 sessions means that is prolly due to chance

[Bayesian @LessOnline&@Manifest @Bayesian0_0]

I think this is just a chinese labs + nvidia skill issue mostly, and the more inference demand you expect for ur model the more it justifies pretraining overtraining, so frontier us models are probably much more ‘overtrained’. They also likely are much more mature in their synthetic pretraining data pipelines. But yeah assuming they keep models around for post training for half a year to a year, it just seems crazy to me to some extent that they would pretrain them for only a few days!

[Bayesian @LessOnline&@Manifest @Bayesian0_0]

This makes sense thanks for the answer ig I’m forgetting they have a crapton of compute. I think it’s plausible they have 150T unique tokens and 2-3 epochs gets u to 400Tish training tokens, which would get to month+ training run. But maybe I am making numbers up that match my intuition, not sure. Just seems surprising that they could not make use of significantly more compute time than that for pretraining given its apparent importance

[Tamay Besiroglu @tamaybes]

I think it's underappreciated how economically valuable AI safety is. A model that frequently goes off the rails, takes dangerous actions, is misleading or deceptive, etc. is simply much less valuable than a model that does not do that.

[Bayesian @LessOnline&@Manifest @Bayesian0_0]

@cqkten And mythos is prolly ~3-4 months ahead in productivity uplift

[wh @nrehiew_]

This benchmark is great! A model that I like scores highly and a model that I dislike scores poorly.

This benchmark is slop! A model that I dislike is at the top of the rankings. How can that be possible? I have taste!

[Bayesian @LessOnline&@Manifest @Bayesian0_0]

@fleetingbits @ar0cket1 Ok maybe that’s too many inferential steps to be saying probably

[Bayesian @LessOnline&@Manifest @Bayesian0_0]

@fleetingbits @ar0cket1 Probably they realized it led to a higher price / lower usage than they actually needed to charge to maintain margins per unit of compute, so changed it (beyond possible inference efficiency gains between then and now)

[Bayesian @LessOnline&@Manifest @Bayesian0_0]

@ValsTutor following ECI trends (my own implementation, slightly diff results to Epoch's) you get to mythos preview level open weights in 10ish months, bc mythos preview is pretty outlier in its capability level

[Bayesian @LessOnline&@Manifest @Bayesian0_0]

@justjoshinyou13 this was my take
https://t.co/TIhJCghNn4

[Bayesian @LessOnline&@Manifest @Bayesian0_0]

@ar0cket1 they force fast mode to be API usage! So they decrease the API margin but maintain gross margin by decreasing fraction of tokens that are heavily subsidized

[Bayesian @LessOnline&@Manifest @Bayesian0_0]

@himanshustwts @Gauri_the_great > all our customers
this almost certainly includes the general public

[Bayesian @LessOnline&@Manifest @Bayesian0_0]

@teortaxesTex Openai models are overspecialized on math so this doesn’t make any sense

[Bayesian @LessOnline&@Manifest @Bayesian0_0]

remarkable honesty

[Alex Volkov @altryne]

what did @tszzl see

[Bayesian @LessOnline&@Manifest @Bayesian0_0]

@fleetingbits good takes

[Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞) @teortaxesTex]

Interesting how different are frontier labs' notions of progress towards AGI.
OpenAI: "we've disproven an old conjecture in math"
Anthropic: "we've discovered ALL the vulnerabilities"
DeepSeek: "we've made context free"
Google DeepMind: "we've reduced the batch size for Flash"

[Bayesian @LessOnline&@Manifest @Bayesian0_0]

@snowboat84 I doubt it
https://t.co/Vfd1g9IKOb

[Bayesian @LessOnline&@Manifest @Bayesian0_0]

@SemiAnalysis_ dylan is right