Mansoor Baloch

[Lijie Chen @wjmzbmr1]

10 proofs from our next major model Astra on long-standing open problems in mathematics and theoretical computer science (also including new circuit lower bounds for computing the permanent!)

GPT-5.6 has already enabled so much exciting work in math and science. Can’t wait to see what comes next!

[Gary Marcus @GaryMarcus]

Claude Code is not AGI, but it is the single biggest advance in AI since the LLM.

But the thing is, Claude Code is NOT a pure LLM. And it’s not pure deep learning. Not even close.

And that changes everything.

The source code leak proves it. Tucked away at its center is a 3,167 line kernel called print.ts.

print.ts is a pattern matching. And pattern matching is supposed to be the *strength* of LLMs.

But Anthropic figured out that if you really need to get your patterns right, you can’t trust a pure LLM. They are too probabilistic. And too erratic.

Instead, the way Anthropic built that kernel is straight out of classical symbolic AI. For example, it is in large part a big IF-THEN conditional, with 486 branch points and 12 levels of nesting — all inside a deterministic, symbolic loop that the real godfathers of AI, people like John McCarthy and Marvin Minsky and Herb Simon, would have instantly recognized.*

Putting things differently, Anthropic, when push came to shove, went exactly where I long said the field needed to go (and where @geoffreyhinton said we didn’t need to go): to Neurosymbolic AI.

That’s right, the biggest advance since the LLM was neurosymbolic. AlphaFold, AlphaEvolve, AlphaProof, and AlphaGeometry are all neurosymbolic, too; so is Code Interpreter; when you are calling code, you are asking symbolic AI do an important part of the work.

Claude Code isn’t better because of scaling.

It’s better because Anthropic accepted the importance of using classical AI techniques alongside neural networks — precisely marriage I have long advocated.

It’s *massive* vindication for me (go see my 2019 debate with Bengio for context, or to my 2001 book, The Algebraic Mind), but it still ain’t perfect, or even close.

What we really need to do to get trustworthy AI rather than the current unpredictable “jagged” mess, is to go in the knowledge-, reasoning-, and world-model driven direction I laid out in 2020, in an article called the Next Decade in AI, in which neurosymbolic AI is just the *starting point* in a longer journey.*

Read that article if you want to know what else we need to do next.

The first part has already come to pass. In time, other three will, too.

Meanwhile, the implications for the allocation of capital are pretty massive: smartly adding in bits of symbolic AI can do a lot more than scaling alone, and even Anthropic as now discovered (though they won’t say) scaling is no longer the essence of innovation.

The paradigm has changed.

—
*Claude Code is plainly neurosymbolic but the code part is a mess; as Ernie Davis and I argued in Rebooting AI in 2019, we also need major advances in software engineering. But that’s a story for another day.

[Rowan Cheung @rowancheung]

NEW: OpenAI just announced 'o3', a breakthrough AI model that significantly surpasses all previous models in benchmarks.

—On ARC-AGI: o3 more than triples o1’s score on low compute and surpasses a score of 87%
—On EpochAI’s Frontier Math: o3 set a new record, solving 25.2% of problems, where no other model exceeds 2%
—On SWE-Bench Verified: o3 outperforms o1 by 22.8 percentage points
—On Codeforces: o3 achieved a rating of 2727, surpassing OpenAI’s Chief Scientist’s score of 2665
—On AIME 2024: o3 scored 96.7%, missing only one question
—On GPQA Diamond: o3 achieved 87.7%, well above human expert performance

[Mansoor Baloch @MansoorBaloch_]

bookmarking to my fresh collection ;)

[Andrej Karpathy @karpathy]

Of ~200 books I've read, the few that stayed with me over time and I find myself often thinking back to or referring to, in ~random order:

All short stories by Ted Chiang, especially Exhalation, Division By Zero, Understand, The Story of Your Life, Liking What You See, The Lifecycle of Software Objects, What's Expected of us, just excellent themes ideas and reading all around.

The Selfish Gene (nonfiction) - a classic for understanding evolution and natural selection, especially the realization that the gene is closer to the real unit of selection more than an individual, explaining altruism and colonies and a lot more.

The Lord of the Rings (fantasy) - I return to LoTR all the time for comfort. I don't think anyone else has created a high fantasy Universe this complex, with so much mythology, symbolism, new languages, mysterious system of magic, ancient and powerful beings and artifacts, beautiful writing and dialog, themes of courage, friendship and heroism, the list goes on and on... You're thrown into a world with characters and references to so many things that are part of this ancient world and never really introduced. There's always more to find on each reading.

The Martian (~scifi) - top tier science porn, competence porn, fast paced and fun.

The Vital Question (nonfiction) - First time I intuitively grokked the bridge from geology to biology, the origin of life, and likelihood of life in the Universe at large at various stages of complexity and development. Also all other Nick Lane books.

How To Live by Derek Sivers (nonfiction) - 27 conflicting answers to how to live life. Emphasizing the diversity of consistent and possible answers to the meaning and goals of life.

1984 (nonfiction) - Classic. Newspeak, Ministry of Truth, Doublethink, Thoughtcrime, Facecrime, Unperson, the list just keeps on going. Chilling world-building and the realization that weaker equivalents of everything exist.

In Defense of Food by Pollan (nonfiction/food) - Eat food. Not too much. Mostly plants. The book that first taught me to avoid the entire center of every grocery store and only shop on the outer ring. The realization that the food industry is out of control and the things they do with your food, what they put into it, what they are allowed to do, and how they are allowed to market it to you is quite a lot worse than I thought.

The Accidental Superpower by Zeihan (nonfiction/geopolitcs) - I've found Zeihan to be a bit of a mixed bag over time but I still remember his books (esp this one) to be elucidating on geopolitics.

Countdown to Zero Day (nonfiction/cyberwarfare) - Goes into detail on Stuxnet, imo very important and highly elucidating reading on cybersecurity, the future of warfare, and AGI.

A Fire Upon the Deep (scifi) - Chapter one only, incredible portrayal of what superintelligence will be like that has stayed with me since.

Guns Germs and Steel (nonfiction/history) - I'd probably recommend a summary of this book more than the book itself. I remember it being very dry, but it was very interesting because it is a comprehensive analysis of the resources grid (food, animals, freshwater, climate, ...) in our real-world game of Civilization, and the implications there of.

Flowers of Algernon (scifi) - Just a totally crushing masterpiece on intelligence.

Atlas Shrugged (scifi) - No one finishes this I think but the first few chapters and its worldbuilding are enough and, once seen in an exaggerated form in fiction, elements of it cannot be fully unseen in reality.

An Immense World (nonfiction/bio, by Yong, among others of his) - Nice book on so many different sensors used by various animals, you repeatedly realize human senses are super inadequate and that we only measure such a tiny sliver of reality.

The Master Switch (nonfiction/tech history, by Wu) - history of information technologies telegraph, telephony, radio, television, film, cable television, internet and the pattern of "The Cycle", where each medium starts decentralized, open and idealistic and then progresses towards centralization, control and oligopoly, for the very similar reasons, by very similar means, and usually at the expense of diversity, innovation and technological progress. Quite a few connections to draw on for LLMs, which are after all an information technology too.

(I take recommendations for more that are likely to make this list!)

[Mansoor Baloch @MansoorBaloch_]

@MBaloch846 tai jan, mni jan 😂

[Mansoor Baloch @MansoorBaloch_]

@Xumtazsajidi @GeorgiaStateU Thank you Mumtaz Jan 🌻
Indebted forever :)

[Jim Fan @DrJimFan]

If you think OpenAI Sora is a creative toy like DALLE, ... think again. Sora is a data-driven physics engine. It is a simulation of many worlds, real or fantastical. The simulator learns intricate rendering, "intuitive" physics, long-horizon reasoning, and semantic grounding, all by some denoising and gradient maths.

I won't be surprised if Sora is trained on lots of synthetic data using Unreal Engine 5. It has to be!

Let's breakdown the following video. Prompt: "Photorealistic closeup video of two pirate ships battling each other as they sail inside a cup of coffee."

- The simulator instantiates two exquisite 3D assets: pirate ships with different decorations. Sora has to solve text-to-3D implicitly in its latent space.
- The 3D objects are consistently animated as they sail and avoid each other's paths.
- Fluid dynamics of the coffee, even the foams that form around the ships. Fluid simulation is an entire sub-field of computer graphics, which traditionally requires very complex algorithms and equations.
- Photorealism, almost like rendering with raytracing.
- The simulator takes into account the small size of the cup compared to oceans, and applies tilt-shift photography to give a "minuscule" vibe.
- The semantics of the scene does not exist in the real world, but the engine still implements the correct physical rules that we expect.

Next up: add more modalities and conditioning, then we have a full data-driven UE that will replace all the hand-engineered graphics pipelines.

https://t.co/7BikSgE7iN

[Fern @hi_tysam]

My beliefs on what Q* is doing (and why):

- Train a population of agents to maximize interagent orthogonality and density coverage over a large text corpus.

- Self-play with RL reward signal being any agent returning to original passage at any point.

Reasoning in thread.

[Mansoor Baloch @MansoorBaloch_]

Particles physics summarising the openai saga😂😂

[Andrew Côté @Andercot]

It's pretty straight forward - OpenAI spontaneously emits @sama and @gdb, producing a short-lived @miramurati that converts to an @eshear via @bradlightcap decay.

@sama and @gdb undergo @satyanadella-catalyzed fusion with @MSFTResearch in a temporary bound state.

Since @eshear is a gluon he can mediate color-charge exchanges among board members: the @hlntnr exchange with @LHSummers preserves DC charge, and @TashaMcCauley exchange with @btaylor preserves SV-charge.

@adamdangelo is his own antiparticle and so remains in place, but traveling backwards in time.

The first half now plays out in reverse-time: the @MSFTResearch superposition collapses, @bradlightcap is re-absorbed along with @sama and @gdb into a new stable bound state.

The slight mass difference between exchanged board members adds 501(c) KeV of energy, causing net acceleration of the OpenAI ensemble.

[Arthur Mensch @arthurmensch]

We have heard many extrapolations of Mistral AI’s position on the AI Act, so I’ll clarify.

In its early form, the AI Act was a text about product safety. Product safety laws are beneficial to consumers. Poorly designed use of automated decision-making systems can cause significant damage in many areas. In healthcare, a diagnosis assistant based on a poorly trained prediction system poses risks to the patient. Product safety regulation should be proportional to the risk level of the use case: it is undesirable to regulate entertainment software in the same way as health applications. The original EU AI Act found a reasonable equilibrium in that respect. We firmly believe in hard laws for product safety matters; the many voluntary commitments we see today bear little value.

This should remain the only focus of the AI Act. The EU AI Act now proposes to regulate “foundational models”, i.e. the engine behind some AI applications. We cannot regulate an engine devoid of usage. We don’t regulate the C language because one can use it to develop malware. Instead, we ban malware and strengthen network systems (we regulate usage). Foundational language models provide a higher level of abstraction than the C language for programming computer systems; nothing in their behaviour justifies a change in the regulatory framework.

Enforcing AI product safety will naturally affect the way we develop foundational models. By requiring AI application providers to comply with specific rules, the regulator fosters healthy competition among foundation model providers. It incentivises them to develop models and tools (filters, affordances for aligning models to one's beliefs) that allow for the fast development of safe products. As a small company, we can bring innovation into this space — creating good models and designing appropriate control mechanisms for deploying AI applications is why we founded Mistral. Note that we will eventually supply AI products, and we will craft them for zealous product safety.

With a regulation focusing on product safety, Europe would already have the most protective legislation globally for citizens and consumers. Any foundational model would be affected by second-order regulatory pressure as soon as they are exposed to consumers: to empower diagnostic assistants, entertaining chatbots, and knowledge explorers, foundational models should have controlled biases and outputs.

Recent versions of the AI Act started to address ill-defined “systemic risks”. In essence, the computation of some linear transformations, based on a certain amount of calculation, is now considered dangerous. Discussions around that topic may occur, and we agree that they should accompany the progress of technology. At this stage, they are very philosophical – they anticipate exponential progress in the field, where physics (scaling laws!) predicts diminishing returns with scale and the need for new paradigms. Whatever the content of these discussions, they certainly do not pertain to regulation around product safety. Still, let’s assume they do and go down that path.

The AI Act comes up with the worst taxonomy possible to address systemic risks. The current version has no set rules (beyond the term highly capable) to determine whether a model brings systemic risk and should face heavy or limited regulation. We have been arguing that the least absurd set of rules for determining the capabilities of a model is post-training evaluation (but again, applications should be the focus; it is unrealistic to cover all usages of an engine in a regulatory test), followed by compute threshold (model capabilities being loosely related to compute). In its current format, the EU AI Act establishes no decision criteria. For all its pitfalls, the US Executive Order bears at least the merit of clarity in relying on compute threshold.

The intention of introducing a two-level regulation is virtuous. Its effect is catastrophic. As we understand it, introducing a threshold aims to create a free innovation space for small companies. Yet, it effectively solidifies the existence of two categories of companies: those with the right to scale, i.e., the incumbent that can afford to face heavy compliance requirements, and those that can’t because they lack an army of lawyers, i.e., the newcomers. This signals to everyone that only prominent existing actors can provide state-of-the-art solutions.

Mechanistically, this is highly counterproductive to the rising European AI ecosystem.  To be clear, we are not interested in benefiting from threshold effects: we play in the main league, we don’t need geographical protection, and we simply want rules that do not give an unfair advantage to incumbents (that all happen to be non-European).

Transparency around technology development benefits safety and should be encouraged. Finally, we have been vocal about the benefits of open-sourcing AI technology. This is the best way to subject it to the most rigorous scrutiny. Providing model weights to the community (or even better, developing models in the open end-to-end, which is not something we do yet) should be well regarded by regulators, as it allows for more interpretable and steerable applications. A large community of users can much more efficiently identify the flaws of open models that can propagate to AI applications than an in-house team of red-teamers. Open models can then be corrected, making AI applications safer. The Linux kernel is today deemed safe because millions of eyes have reviewed its code in its 32 years of existence. Tomorrow’s AI systems will be safe because we’ll collectively work on making them controllable. The only validated way of working collectively on software is open-source development.

Long prose, back to building!

[Andrew Ng @AndrewYNg]

I agree with Yann on this.

The risk of human extinction within 30 years is extremely low -- it likely requires making Earth inhospitable to human life, so that it’s not possible even for small pockets of humans to survive. This could happen from global thermonuclear war, pandemic, or (much less likely) asteroid strike. Over very long timescales (hundreds of years), low birth rates/population collapse leading to a long, slow decline of humanity is also possible. In comparison to these risks, the idea of rogue AGI killing off 8 billion people seems much less tangible and much more remote.

Having more intelligence available to humanity -- including both human intelligence and artificial intelligence -- arms us better to tackle many problems, including existential ones like the ones mentioned above. So I believe AI will reduce the all-cause extinction risk for humans.

If we want humanity to survive and thrive for the next 1,000 years, rather than slowing AI down with burdensome regulations, I would rather make it go faster.

[Hina Nasir @HinaAchakzai]

Last night I had a conversation with Dr. Sarfaraz, the doctor who accompanied Dr.Shukurullah on his way to AKUH for management of Crimean Congo Hemorrhagic Fever (CCHF). He was clearly distressed and I could hear his suppressed sobbing throughout the call as he narrated (cont)

[Prof. Feynman @ProfFeynman]

Meanwhile, inside the box, Schrödinger's cat plans its revenge.

[Andrew Ng @AndrewYNg]

A little known part of the origin story of LLMs: A lot of early Natural Language Processing research was funded by US intelligence agencies that had an interest in machine translation (and also speech recognition). This funding helped develop benchmarks, grow communities and nurture researcher interest in translation research disproportionate to its commercial application, and led to new foundational technologies motivated by translation.

For example, the transformer paper was originally a paper on mapping from one language to another; now we use transformers to map from a prompt to a response. The most widely used tokenizer (BPE) was also popularized in a paper studying how to translate words not seen in the training set.

More on this piece of NLP history in my letter in The Batch! https://t.co/7GdjnrLBmv

[Marco Mascorro @Mascobot]

Controversial, but AGI will be built in python, and yes, born out of Untitled.ipynb

[Earl K. Miller @MillerLabMIT]

Brain networks use electric fields to store memories
Electric fields coordinate brain networks to encode memories.
https://t.co/0t8yWePW2H
#neuroscience

[Asim Iqbal, PhD @itsasimiqbal]

The Next Frontier For Large Language Models Is Biology.

Well said @demishassabis! “Just as mathematics turned out to be the right description language for physics, biology may turn out to be the perfect type of regime for the application of AI” 🧬🤖 https://t.co/chVwSjmyOU

[Mahbub Ul Haq Research Centre @HaqMahbub]

📢📢VACANCY ALERT📢📢
Profs. @noamangali (@LifeAtLUMS) and Alex Lee (@UofR) are looking for RAs for a project concerning Punjab’s canal colonies:

How to Apply: https://t.co/OPxQCONAbX
Deadline: 24th July 2023
Apply @ [email protected]

[Mansoor Baloch @MansoorBaloch_]

@KashaniAbdullah Plan cancel bro :)