Seems like the OpenAI research acceleration blogpost is not what you would expect if we were definitely on the cusp of artificial superintelligence - models that are more cognitively capable than ~ALL humans on ~ALL tasks. Agent workdays have grown very jaggedly, swarm sizes haven't grown very much, and spending on tokens hasn't actually outpaced spending on human workers* although the growth rate is clear. There also hasn't been an acceleration in research experiments per experimenter, or in the internal ECI scores of OpenAI's models. Perhaps most importantly (and most surprisingly), the task success rate for <15 min tasks hasn't grown very much at all, and the share of tasks done by agents across various time horizons is both low and not growing for the most part.
It's possible that Anthropic is *much* further along in automating AI R&D than OpenAI, and that METR saw this in their investigation. It's also possible that we see nothing much right up until a shocking acceleration in capabilities, like we saw in mathematics. We certainly can't rule out the possibility of an intelligence explosion.
But there's been a huge belief cascade in AI in general and AI safety in particular that seems largely unwarranted given the evidence that we have today. It's entirely possible that we can't fully automate all AI R&D tasks in the imminent future, or that full automation of AI R&D doesn't massively accelerate AI capabilities progress, or that capability gains concentrate in easily verifiable tasks and fail to generalize to the messy real world. These are all open research questions
@jakek@constructor_agi From the brain! We took heavy inspiration from the hippocampus and working memory mechanisms. There is a lot more left to gain in borrowing from the brain.
We’re sharing the first phase of our research @constructor_agi. Our mission is to solve continual learning and develop models that learn natively from experience.
Unlike LLMs, humans don’t reread their entire context; with experience, we learn what matters. Our work is a promising first step in that direction: it shows that a model improves when it learns where to look.
Introducing Link: Teaching Attention to have intention.
Link is a learned, sparse read that lets an LLM choose what to read. At multi-million context lengths, it outperforms dense attention on retrieval accuracy by 53% at 4M token context lengths, alongside 9.9x GPU efficiency.
That said, today’s agents still can’t learn after deployment. We don’t believe gains on current LLMs are enough.
Beyond in-context, models themselves need to learn from experience. Continual learning will enable agents to tackle all real-world complexity.
We are developing a new class of model, and are excited to share more soon.
I've been talking to a bunch of PhD students and researchers recently, and I've qualitatively noticed two growing (but concerning) trends with AI:
1. Researchers seem to be reading less. Some even reported that they read <10 papers a year now with AI because they can simply either 1) retrieve or 2) summarize papers. BUT deep reading is how we engage with underlying intellectual arguments, develop research taste, spot methodological flaws, and form original ideas. It's arguably the most fun part of a PhD imo
2. Researchers seem to be vibe-coding and paper-maxxing. AI dramatically cheapens execution, such as implementing experiments, running analyses, and producing manuscripts. Some even "let [Claude Code] rip" without checking the code or validating the results somehow. BUT it also incentivizes quantity over depth, rigor, and meaningful contributions.
Overall, the cost of producing research is falling much faster than the cost of developing genuine intellectual understanding. Rather than simply producing more papers, I think we should engage deeply with the literature, ask questions that truly matter, and produce work that meaningfully advances our collective understanding.
Over the past week, I’ve been overwhelmed by the love and support of friends, industry colleagues, and people I've never met. Thank you to everyone who has shown kindness to my family and me. It means the world to me.
I came to the United States 53 years ago, and becoming an American citizen is one of the greatest gifts and proudest moments of my life. NVIDIA has been my life’s work. Building it into the company it is today alongside my colleagues has been a privilege and an honor. Nowhere but in America could a company like NVIDIA be possible. I love our country, and I love NVIDIA.
I did not live up to the standards expected of everyone at NVIDIA, and I have made serious mistakes in my personal life. I created problems for the Board that I deeply regret. I own my mistakes completely, and I am committed to cooperating with the Board’s independent investigation.
Since my departure, I’ve seen an outpouring of calls from NVIDIA’s employees, investors, business partners, and customers for me to return as CEO. Many continue to reach out to me directly as the company navigates complex technology, market, competitive, and geopolitical dynamics. NVIDIA is a once-in-a-generation company. If the Board of Directors decides that my return is in the best interest of our shareholders, our employees, the company, and America, I would be honored to answer these calls and return as CEO.
Thank you again to everyone who has shared their good wishes. I am profoundly humbled by your support. To my family: I love you, and I will spend the rest of my life being worthy of yours.
Jensen