Chatbotics

The singularity is here.

[Greg Brockman @gdb]

Announcing three major scientific discoveries, one each in mathematics (harmonic analysis), computer science (complexity theory), and physics (quantum field theory).

AI starting to help us improve our understanding of the universe! https://t.co/VkfacopR2R

[Chatbotics @chatbotic]

A sketch of the proof of non-existence of Siegel zeroes https://t.co/sSorrXzwjI

[Majid Hosseini @m4h007]

In my view, the right response by the mathematical community to OpenAI’s papers is to say:

“We’re not accepting these papers as finished work. It’s not our job to sit down and read your model-generated proofs.

It is your job to present your work through presentations and convince people of the correctness of your work. It’s always the case that whoever makes a claim needs to provide sufficient explanation and evidence to make independent verification feasible.

Until that moment, you just have a claim.”

There’s a difference between a proof being mathematically correct and a proof being accepted as correct by the mathematical community. Mathematical truth doesn't depend on consensus, but scientific recognition requires scrutiny, communication, and reproducibility.

They're still the arbiters of whether anyone's claims, including OpenAI's, are accepted as valid, and they should leverage that to the maximum extent possible.

[Lilian Weng @lilianweng]

A super long overdue (3+ years?) post on scaling laws.

Compute is expensive. Scaling laws are a way to help us reason about the optimal compute allocation between data and model size before committing to a large run.

The post covers what scaling laws predict, how compute-optimal allocation works, why Kaplan et al. and Chinchilla disagree, and how data limits + fitting details make extrapolation tricky.

https://t.co/HP26eJvjHB

[Yoshua Bengio @Yoshua_Bengio]

Over the past few days, I've taken the time to summarize my thoughts on the recent incidents involving agents’ misaligned behavior. We don't know with certainty what comes next, but we know where these issues originate, and this can help us plan the path forward.

Please feel free to ask your questions in the replies, and I’ll try to answer some of them in the coming weeks.

https://t.co/bvncjT0y0h

[Bill Gurley @bgurley]

I have a recommendation to help bring some sanity to this whole AI doomerism situation that is appropriately exploding today.

If any reporter can get one of the "true believers" to sit for a real interview (if they can't, that is an important tell), they should use a technique known as the "Five Whys" https://t.co/RqKrn4TNLa.

The "five" number is arbitrary, but the point is to drill down into the details and steps required to get to the catastrophic end-point.

You can't take down all of humanity without a few intermediate steps. Let's talk through each of those. It's quite a useful exercise.

It would help us all think more clearly, and possibly the doomerists as well.

[OpenAI @OpenAI]

We’re sharing a solution to the Navier-Stokes Millennium Prize Problem, one of the deepest problems at the frontier of mathematics.

The proof was produced by a group of agents, using an OpenAI next-generation model significantly more capable than GPT-6 Astra.

The problem concerns whether the description of smooth three-dimensional fluid motion modeled by the Navier-Stokes equations can break down. It has remained unresolved for roughly 90 years.

[Jackson Hinkle 🇺🇸 @jacksonhinkle]

Genuinely one of the most evil things I have ever seen

[Demis Hassabis @demishassabis]

I’ve been working towards AGI my whole life, and as we enter this pivotal moment, I’m stepping into a new role as Chair of Google DeepMind & Chief Scientist of Alphabet. This will allow me to focus on long-term strategy, and accelerating scientific breakthroughs, including leaning into my work at Isomorphic to help cure disease.

I’m excited that @koraykv will be stepping up to lead GDM as SVP, alongside @joshwoodward and our exec team. I could not be more excited and confident about our amazing next chapter! 🚀

https://t.co/2WtlIIlTUa

[Ihtesham Ali @ihteshamali]

Google wrote the paper that made ChatGPT possible. Then decided not to build ChatGPT. Sergey Brin just explained why on stage at Stanford and the reason is more embarrassing than anyone expected.

In 2017, Google published the transformer paper. The architecture that powers every major AI model today. It came from their own researchers. Their own labs. Their own compute.

Then they sat on it.

Sergey was blunt about why. They underinvested in scaling the compute. They did not take it seriously enough. And when they finally had something worth shipping, they got scared. Chatbots say dumb things. Google had a reputation to protect. So they protected it instead of shipping.

OpenAI was not scared.

Ilya Sutskever, trained at Google, left and went there. Other Google researchers followed. They took the transformer architecture, scaled it, shipped it anyway, and captured the entire generative AI wave while Google watched.

Sergey called it a mistake at Stanford in front of hundreds of students.

The company that invented the technology did not build the product. The company that built the product did not invent the technology.

That is the most expensive case of corporate hesitation in the history of the industry.

---

Watch the full interview YT.

Search: "Big ideas begin here: Sergey Brin at Stanford"

[Ido Aizenbud @IdoAizenbud]

What can a neuron compute?

Real biological neurons are complex, but how capable are they?

Using a new method, we found that a single cortical neuron can classify cats vs dogs, recognize spoken words, and solve 10-bit parity, all tasks thought to require entire networks. (1/15)

[Richard Sutton @RichardSSutton]

A new and possibly controversial perspective:
In this video, I explain the sense in which generative AI trained by supervised learning is incapable of making novel discoveries.
https://t.co/zin5QbbT9N

The text of the speech:

AI Creativity and Discovery

Good day ladies and gentlemen. I regret that I am unable to be with you all today to engage in a back-and-forth discussion, but I am nevertheless pleased to be able to share with you, via this recording, some high-level thoughts about the current and future state of artificial intelligence, and in particular about AI’s relationship to science and mathematics, which is, as I understand it, the central focus of this meeting and of the SAIR Foundation.

I would like to start with an old joke; I am sure you have heard it before. It is the one about the researcher whose work is being evaluated, and the review comes back, and says “This work is both novel and good. Unfortunately, the parts that are good are not novel, and the parts that are novel are not good.”

My first point about AI is that this assessment applies exactly to large parts of AI as we know it today. Not all of today’s AI, but a large part of it. Pretty much all of what we mean by “Generative AI”---which includes large language models, and the images and video models, and even the new methods for learning world models. All of these AIs take large numbers of examples and produce a “model” which behaves similar to the examples, that is, which generates text like people, or images like artists or nature, and videos like we find on the internet. Don’t get me wrong, Generative AI can be extremely useful. No doubt about that. But the assessment of the joke still applies. These systems can produce output that is both novel and good, but not at the same time.

In many ways this is just absolutely not a problem. When we ask an AI for an answer from the internet, or to summarize a document, we don’t want it to be novel. We are happy if the quality of the answer, the goodness, comes from the source material—from the people who wrote the document or the articles on the internet. If the AI’s answer is novel it means it is going beyond the source material, adding something beyond it. This is what we call “hallucinations”. In most cases, we don’t like it when the AI makes something up, when it adds something novel.

One exception, of course, is when we are looking not for facts or reality, but for fiction and entertainment. We might ask for a bedtime story for a child, or an image based on existing images on the internet but which is nevertheless different and distinct from them. In these cases, it is never easy for us to know how creative the AI is actually being, as we do not know how close the AI’s story, poem, or image is to the source material. In a real practical sense we can not know this because the internet is too big, the possible sources that the AI may draw upon are too numerous.

When we ask for a fiction or novelty, the AI can give it to us because its processing is in part stochastic. Every decision can go multiple ways and will go different ways and produce a different trajectory every time. The trajectory can be random—and thus novel—or it can be based on the training data—and thus “good” because the training data is good, sourced from people or reality. Thus, the trajectory is either novel or good—based on randomness or based on data—but never both at the same time.

Really, I think it is okay if the output of Generative AI is never good and novel at the same time. For the researcher in the joke this is a devastating criticism, but for most things it is not, and for Generative AI it is not. Generative AI is meant to be a mimic. This is what supervised learning is for. Generative AI can be extremely useful, even when it just mimics, if it is faster, or cheaper, or smaller, or more customizable, or more copy-able, than the thing being mimicked. It is okay if Generative AI cannot be both novel and good at the same time. It is still a transformative technology.

But it is a limitation. And remember we are here to use AI for science and mathematics, and for these areas the assessment of the reviewer in the joke is devastating. For these areas we need true creativity and discovery. Generative AI—or Mimicking AI—will never get where us there. For these we need something more, and indeed we have something more in other parts of AI. We have many AI systems which can give us more. We have AlphaGo with its world-changing move 37, or AlphaZero with its brilliant original chess-playing style. We have GT-Sophy that drives simulated racecars better than any human. We have AlphaFold and AlphaProof and Claude-Code, which have brought true advances in science, mathematics, and programming. We have RL-Lyft which optimizes the assignment of cars to passengers in the ride-hailing business. All these systems have found things that are both novel and good. And, truth be told, some language models have been augmented in ways that make them more than Generative AI based on supervised learning.

All these systems have some additional features that make them capable of true creativity and true discovery. It is important for us to recognize what this is—and that it is not present in ordinary, garden-variety Generative AI. It is something that can not come from just supervised learning, from learning from examples. What is it? Well, it is a simple thing, a commonsense thing. It is not new. We have many names for it, but unfortunately none of them are very good names. I will call it Discovery. Basically, Discovery is just the idea of trying many things and seeing which of them work, then keeping those that worked the best. Evolution by natural selection works this way. The scientific method works this way. And just ordinary life and learning works this way. We try things and remember what works. What could be more obvious? In this behavioral case, psychology has two names for it— “instrumental learning” and “operant conditioning”—and in machine learning it is what we mean by “reinforcement learning”. We also see the idea of Discovery in planning and combinatorial search—anything that involves the idea of “generate and test”.

The essence of Discovery is to combine three steps:
1. Variation,
2. Evaluation, and
3. Selective retention.

Of course, I am not the first to say this. I am not the first to point out that this combination of steps is key to science, to evolution by natural selection, and to animal behavior. I think particularly of papers by Donald Campbell, by Daniel Dennett, and by Gary Cziko. What is new in my remarks is to directly relate the idea of Discovery to modern AI to help us see that it is not present in supervised learning or Generative AI—in particular, that Discovery is not present in backpropagation or gradient descent.

Let me say explicitly what is missing from Generative AI. As we have remarked, these systems do have a stochastic aspect, so they do generate a variety of trajectories and behavior. What is missing is the Evaluation step. The generator was pre-trained by supervised learning, leaving no way at runtime to Evaluate what it generates. And of course without Evaluation there can be no Selective retention, and thus no Discovery. The variation can bring novelty, but without evaluation there is no Discovery, and arguably, no creativity. That is, I would say that creativity requires that the new things generated be Evaluated. Without evaluation, and retention of the best, there is nothing created. The novelty flickers into existence but, if its value is unrecognized, it flickers away and is lost.

In many cases, Evaluation is done by people to make a discovery. As when we have Generative AI make many pictures for us, and then we pick the one that we like the best. The human+AI system completes the discovery.
In many other cases, the Evaluation comes from a clear objective. Some moves lead to checkmate, some steps lead to a proof, some actions result in high reward, some genotypes make more copies, some theories explain the data better.

Some prefer the Variation step to be called Blind variation, where “blind” here means that it is uninformed, a shot in the dark. It does not need to be completely uninformed; a good scientist does not select theories to test at random. But neither can it be completely informed and determined. There must be some uncertainty about where the answer lies in order for there to be a discovery. In practice, the variation is partly informed and partly blind, but it is the blind part that corresponds to the discovery.

Now let us briefly go all the way to modern deep learning, to the backpropagation algorithm. At first it might seem that backpropagation is incapable of discovery because it is deterministic and thus incapable of variation. But this is not correct. The weight updates of backprop are deterministic, but the weights are initialized to small random values. The random initialization is often downplayed, but in fact it is a necessary form of variation; it must be done properly to get good performance. In backprop this Variation is done once, at network initialization, so its effect is temporary, and later the network may lose its ability to learn. This is the weakness of deep learning that is alleviated with a new algorithm that my group presented in Nature a couple of years ago. Our “continual backpropagation” made one small change: every so often a less-used neuron would be re-initialized to small random weights. This allows the variation to continue and plasticity to be retained.

Although there is much more to be said about Creativity and Discovery, this is the key point: they are more than supervised learning, more than pattern recognition, more than prediction, and more than world modeling. Those things are important, but they alone will not bring us to discovery. Discovery requires Evaluation from a person or from an explicit goal, and only in the latter case will we attain full autonomy.

So that is my call to arms. If we want the full power of AI scientists, then we should share the goals with them so they can create, evaluate, discover, and in these ways fully participate in achieving the goals. Let’s be bold! Let’s fully automate Creativity and Discovery!

[Dwarkesh Patel @dwarkesh_sp]

https://t.co/54D3VKBzzo

[Bill Gurley @bgurley]

A wonderfully pragmatic look at the US/China AI race. Very unrealistic & dangerous to project "the next 12 months" are an all-or-nothing bet on the future of the world.

https://t.co/3ELCPFhl9g

[Philosophy Of Physics @PhilosophyOfPhy]

When Audience Asked Richard Feynman,

Do you think there will ever be a machine that will think like human beings and be more intelligent than human beings?

[Dwarkesh Patel @dwarkesh_sp]

The Jensen Huang episode.

0:00:00 – Is Nvidia’s biggest moat its grip on scarce supply chains?
0:16:25 – Will TPUs break Nvidia’s hold on AI compute?
0:41:06 – Why doesn’t Nvidia become a hyperscaler?
0:57:36 – Should we be selling AI chips to China?
1:35:06 – Why doesn’t Nvidia make multiple different chip architectures?

Look up Dwarkesh Podcast on YouTube, Apple Podcasts, Spotify, etc. Enjoy!

[Google Research @GoogleResearch]

Introducing TurboQuant: Our new compression algorithm that reduces LLM key-value cache memory by at least 6x and delivers up to 8x speedup, all with zero accuracy loss, redefining AI efficiency. Read the blog to learn how it achieves these results: https://t.co/CDSQ8HpZoc

[Demis Hassabis @demishassabis]

Ten years ago, AlphaGo’s legendary match in Seoul heralded the start of the modern era in AI. Its famous ‘Move 37’ signaled to us that AI techniques were ready to tackle real-world problems in areas like science - and ideas inspired by these methods are critical to building AGI

[Andrzej Krzywda @andrzejkrzywda]

Chess players have over 15 years of AI experience

We call them engines (like Stockfish or Leela) but it’s proper AI models under the hood.

We run them locally or in the cloud, depending on the needs and depth.

We learnt what to do when we wait for their results.

We were worried when they appeared that chess would be dead. Quite the opposite happened, chess boomed since then with many new job roles.

The overall quality of chess skills improved and more players are now strong.

AI didn’t kill our chess creativity - it improved it.

We treat AI as oracles. It’s rare to argue with engine evaluation.

But, we learnt when to follow AI recommendation and when to skip it as it requires “non-human play”.

The best approach is to use AI to find human playable ideas and drop the “computer lines”.

As a programmer and a chess player (2500 on chesscom) I remain optimistic.

[MIT CSAIL @MIT_CSAIL]

Happy birthday to Ray Kurzweil '70, an innovator in speech & text recognition. He's also a leading proponent of “the singularity.”

Photo v/The Academy of Achievement