Where much of the field saw flaws - Gwern understood that the fact it worked so well despite being so flawed meant unbelievable headroom.
Models in 5 years will be as Fable to GPT3 - every part of what we do today could be radically better!
@N8Programs what in your opinion is the best open source model that can fit 32gb? i'm sure you have different answers based on the use-case like coding, writing etc I'd love to hear them all
Beat it by having Codex hand-craft weights:
https://t.co/g0T6rklaAY
100% accuracy on 10 million random test cases w/ only 343 parameters. As a bonus, it uses the vanilla Qwen3 architecture, just with the right weights.
The Opus 4.6 system card has some extremely wild stuff that remind you about how weird a technology this is.
These paragraphs are really worth reading.
Turing comparing building thinking machines to biological procreation:
"we are, in either case, instruments of His will providing mansions for the souls that He creates."
* = the answer to the next question
A task meant to require web and computational search guided by multi-hop deduction starting from a partial SHA1, answered in 2m 54s by ChatGPT o3
featuring Tom Cruise and Addison Rae
@tanayj And it’s gotten worse in the 6 months since we first measured.
Google is now: 15 scrape for 1 visitor
OpenAI is now: 1,200 scrapes for 1 visitor
cant believe how much time i wasted learning things before LLMs, spending hours in google search rabbitholes, not knowing future-me-in-only-a-few-years would never need to google again and would have a PhD who can answer anything instantly for $0.001. so grateful for this tech!
“LLMs are just next token predictors”,
Ilya Sutskever explains how this is deeper than it sounds. Predicting next token requires understanding the underlying reality that led to the creation of previous tokens.