Jonathan Ross

Double the World's AI Compute
Chief Software Architect @ Nvidia, Founder of Groq, Creator of the LPU & Google's TPU

[Alex Krentsel @AlexKrentsel]

i've reproduced the results from the MIT blog post today on weight-space programming. but one additional result i have to add: this is a new, step-change emerging capability that's appeared in just the latest set of models.

[Clark Barrett @ClarkBarrett7]

I’m excited to announce the emergence from stealth of &Truth, PBC, the company I’ve been building for the past few months with my co-founders Chris Hsu and @ilyasergey.  Come see what we are up to: https://t.co/sxDjtO5LN4.

[Marginal Revolution @MargRev]

Errors in the research of Emmanuel Saez and Gabriel Zucman https://t.co/Fjp3tTBV8e

[Andrew Curran @AndrewCurran_]

Sam Altman's one-person company prediction is coming true. Anthropic just bought one. The founder of Mithril Medicine used Claude to take a muscular dystrophy antibody treatment from concept to proof-of-principle in mice in 18 months, with just himself, Claude, and part-time consultants. The technical report is great. I will quote:

'The week opened with the handover of two cell-based assays, cell adhesion and flow cytometry, within our team, while eleven questions remained open with the assay CRO. In two sentences, one of us asked Claude to review the latest emails between the outgoing assay lead and the assay CRO and to recommend next steps for the handover, and gave it the link to the handover folder. Claude returned a bottom line after 96s and a full plan with five drafted emails after 18 min. For the full plan it read six email threads, the handover folder, two slide decks and two reports, all in the mailbox and file store to which we had given it access. Four working days later, nine of the eleven questions had an answer in writing and the other two had been left to pilot experiments. The adhesion screen was thereby set out in writing. The CRO drew the plate map to the format that our team had requested, Claude compiled the run order and the material calculation from the production data, and one of us reviewed and approved them.

On day 4, the production CRO delivered its data package, a report of 341 pages, which one of us forwarded to Claude in two sentences, with the request to review it, share observations and recommend next steps. Claude returned a bottom line after 2 min 40s and, once it had read every page, a visual synthesis of all 113 constructs after 7 min 53s. The synthesis arranged the constructs on the grid of 14 by 8 arms. Claude marked 82 constructs for screening as is, seven for additional polishing, and the 24 constructs of three laminin-binding arms to be held back from the cell-based assays on production criteria. Each of the 113 constructs thus had a proposed next step in less than 8 min.

Claude had recommended binding kinetics for all 113 con-structs, and on day 7 one of us challenged this plan. Claude defended it within 27s, arguing that affinity would show whether the three arms merit further work. One of us overruled this recommendation and set the three arms aside on production criteria. Their material remains in storage. Four minutes later, Claude had revised its memo, the working page, the run order and three unsent drafts from 113 to 89 constructs. It had also applied the ruling to three dependent steps in the unsent drafts, each flagged for our veto. The exchange took 7 min 36s from challenge to new plan, and every affected email was still an unsent draft. After the ruling, eight emails drafted by Claude went out on the same day to three CROs and a consultant.

Over the whole week, Claude drafted 17 emails, each in answer to a request from one of us, and we sent 12 of them, each after editing. With its drafts, Claude listed 41 points, such as tube fills, shipping conditions, recipients and assay settings, for which it had found no written statement of ours, so that we could confirm, change or strike each before sending. We kept some, struck others and changed one. Among the passages that we kept were proposals for controls. One draft proposed a comparison of polished and unpolished material on one sensor chip, to size the error that impurities cause in the rest of the panel. Another requested a buffer control diluted like the most dilute sample, so that any effect of the buffer appears at its largest. These proposals concerned the measurement of constructs that had already been produced, and each reached a CRO only after one of us had reviewed and approved it.

[Disruptive @disruptivetech]

Today, we announced the Disruptive Flagship Fund, a $10 billion fund designed to scale the firm’s selective approach to backing category-leading private technology companies.

Advances in artificial intelligence, the massive buildout of compute infrastructure, and a new era of investment in defense and national security are creating companies of unprecedented scale and strategic importance. The amount of capital required to capture these opportunities is increasing significantly.

Disruptive was built for this moment. Since its founding in 2012, the firm has developed an investment model designed to identify a small number of exceptional companies and deploy significant capital behind them.

Disruptive’s investment approach remains unchanged and the firm plans to back approximately 10 exceptional growth and late-stage companies, investing up to $1 billion per company.

Alex Davis, Founder and CEO of Disruptive commented: “The best private technology companies are growing larger, staying private longer and requiring more capital than ever before. We have always focused on a small number of exceptional companies, building the conviction and relationships to be there for founders at their most important moments. The Disruptive Flagship Fund gives us the scale to match the opportunity.”

Press release: https://t.co/nzmn3geCSF

Bloomberg: https://t.co/1r6ln1qgf6

Forbes: https://t.co/rG59kDWFrI

[Andrej Karpathy @karpathy]

Dusting off this tweet from April because this gap in shared understanding of LLM capability is *widening*. It's now less "two groups of people speaking past each other" and more a sharp funnel.

- Napkin math somewhere around 6B people (~75% of the population) have barely come in contact with LLMs at all.
- Around 1-2B (~20%) are casual and infrequent users of free-tier ChatGPT-like products. This group sees derpy chatbots and treats them a bit like a better Google search, a writing aid, or etc. Maybe an agent tries to book you a flight. I have non-tech friends who (reasonably, imo) say they have not much use for it at all. Even many professionals outside of math&code are in this tier. For example, execs and many other functions spend a lot of their time talking to other people, so while they understand what is happening intellectually, it is still second-hand and a bit abstract.
- Now we get to professional use of frontier-grade LLMs in math&code. Somewhere around 20M people (0.2%) see first-hand that large, complex projects that used to take them weeks/months can now be completed by agents with a prompt. Building apps, copying apps, translating apps, decompiling apps from binaries... This has all happened very quickly and recently - less than 1 year ago, I was writing code manually by hand, typing memorized computer code commands into a code editor character by character, occasionally pressing Tab to autocomplete a little chunk of code.
- And finally we get to the ~5,000 people (~0.00006%) with access to frontier-grade systems internally. The external world has seen the preview. It looks like swarms of thousands of agents collaborating over weeks on software mega projects: minting zero days, running cyber attacks and defenses at machine speeds, discovering new science, advancing the frontier of mathematics. Things that would have taken top professionals in the industry years of work. Meanwhile, human review and comprehension are starting to fall behind. For example, people are still involved in the "archeology" of the OpenAI-HF incident from many months ago. Mathematicians may be poring over the 722 manuscripts on frontier mathematics for a while.

The funnel is driven by a combination of factors:
1. The impact scales with ambition, problem size, and horizon. A question with a paragraph answer barely stresses the system. You need a reservoir of big, difficult problems that you really care about. This aspect drives the consumer / professional dimension of the funnel.
2. The jaggedness of the system (which I have written about a lot separately). Capability peaks in domains that are digital, verifiable and economically valuable. This is because LLM capability emerges from reinforcement learning on verifiable rewards on a curated environment mixture driven by revenue potential. This aspect primarily drives the area (e.g. math&code) dimension of the funnel.
3. Access. Free-tier, paid-tier, internal.

So this is the weirdness of the moment. The general public has mostly not interacted with these systems. When they have, it looks like a derpy chatbot. The majority of professionals still see only a modest uplift. And a small sliver of professionals are experiencing the vertigo of the curve going vertical. And it is all happening at the same time.

[Jonathan Ross @JonathanRoss321]

The Singularity:
1950: Real programmers wire their own circuits.
1985: Real programmers write their own assembly.
2010: Real programmers write their own C.
2026: Real programmers write their own Python.
2027: Real programmers write their own prompts.

[Elon Musk @elonmusk]

Unfortunately, we are being blocked by certain oligarchs in order to maintain their monopolistic chokehold on the Indian people. You can guess who they are …

This is a crime against the people of India!

[David Sacks @DavidSacks]

Nvidia’s OpenShell announcement is a reminder that agent safety is an engineering problem.

Recent breakouts weren’t proof that development must stop. They were proof that the sandbox was too weak. The runtime environment was poorly designed and misconfigured.

The right response is open runtime isolation, control of the path to the model, out-of-band hardware enforcement, and a faster pace of safety work with the frontier labs.

As the technical blog puts it: “We believe we need to increase the pace of AI safety research and engineering in collaboration with frontier labs and the broader community.”

That’s how you earn trust. Not by virtue-signaling a pause, and not by acting like no solution exists.

[Bill Gurley @bgurley]

This is how mature enterprises solve security problems (vs braggadocios press releases that create panic instead).

[Jensen Huang @JensenHuang]

Today, with over 100 industry partners, we introduced the NVIDIA Open Agent Safety Platform, bringing together OpenShell and Sentry.

Artificial intelligence is extraordinary technology that will advance discovery, productivity, security, health, and prosperity for generations to come.

But its full promise can only be realized when people have confidence that AI is being built to be safe and deployed with wisdom and responsibility.

This is bigger than a single product. It's the beginning of an open ecosystem to build the trust layer for safe agent systems.

Together, we are building the foundation of the AI economy.

Trust and innovation are not in conflict. Safety is how trust is earned. We must build not only the most capable AI, but the most trusted AI, so that this extraordinary technology can realize its enormous promise for the world. https://t.co/ugYWQ1MyRi

[Jonathan Ross @JonathanRoss321]

@tian_yi_wang My current hobby project I'm using OAI and Claude. Most of my best results have come from using both. One is sufficient, two is better, but a lot of overhead to keep them able to work on the same project. I only recommend for big efforts.

[Jonathan Ross @JonathanRoss321]

Code without AI review is like code without tests.

Unsafe at any speed.

[Jonathan Ross @JonathanRoss321]

@paragkalra Yes. I also do design reviews (in voice mode) before I let the AI do major changes. The AI catches issues I'd never think of on my code as well. It's the new pair programming.

[Jonathan Ross @JonathanRoss321]

@samidh 🤔

[Jonathan Ross @JonathanRoss321]

@BTCKYLE 💚

[Jonathan Ross @JonathanRoss321]

There's a lot to learn from @JensenHuang

[David Senra @davidsenra]

Groq Founder @JonathanRoss321 shares the biggest leadership lesson he learned from Nvidia CEO Jensen Huang:

“There is no circumstance where Jensen has one-on-ones with people.”

“When you're leading groups of people, if you want to reduce the amount of politics, stop having one-on-ones. Have big meetings with everyone who you want to tell something to and tell them all at once.”

“Copy everyone on the email.”

“If someone says, ‘Hey this person is screwing up,’ copy that person on the email. Let them jump in. Otherwise you're allowing politics to happen.”

“There's no politics. It's the least political large organization you will ever see.”

[Jonathan Ross @JonathanRoss321]

.@InsideStairwell indexed everything related to malware for years before AI was a cybersecurity threat and has one of the richest troves of threat data. @MikeWiacek is obsessed, he tracked a lost virus to a fragment on an old 💾 and reconstructed it like the DNA in Jurassic Park.

[Mike Wiacek @MikeWiacek]

1/10

Researchers used Claude to walk from OpenAI’s community forum to a proof-of-concept pull request in its internal monorepo. A few days of agent work. A few hours of human time. That last line is the whole story.

[Thomas Wolf @Thom_Wolf]

Many people I talk to find it hard to understand how the same companies can both push the frontier of AI capabilities and believe AI is a massive danger for the world.

How can you think this might kill everyone and also keep pushing the envelope?

So I’ve tried to collect and summarize the main arguments for this apparent disconnect.

Think of it as some sort of a guide to understanding the reasoning when Dario, Sam, or Elon say the danger is real.

By the way, these people have been worried about AI for a loooong time, they were publicly discussing AI risks more than a decade ago. Sam in Feb 2015, writing on his blog that superhuman machine intelligence is "probably the greatest threat to the continued existence of humanity." Elon at MIT in Oct 2014: "We are summoning the demon." Dario as first author of "Concrete Problems in AI Safety" in 2016.

Okay so how do you go from saying something is extremely dangerous to being a front-runner in building the very dangerous thing?

There are a few ways this can become rational. I'll take five of them, roughly in the order they developed.

1. We need to build it to learn how to make it safe

The earliest argument can be summarized as: “You cannot study something [you’re worried about] if it doesn’t exist.”

In 2015, AI barely worked. so people needed to make it work first to be able to even study some of the problems they anticipated.

The updated version for today's capabilities is: “You cannot learn everything about airplane safety by studying paper airplanes.” You need a real aircraft to discover real failure modes and an increasingly complex one to learn about increasingly complex issues.

Making AI more capable gives more chances to understand the issues and safety researchers something realistic to study

But you could argue: if you're the one afraid of the explosion, why be the one gathering the dynamite?

You could also just wait for other people to build it which leads to the question of who those other people will be -- which is the second line of argument:

2. Better us than them

Knowing how to make something safer does very little good if nobody listens to you. So the idea becomes: let’s make sure responsible people build the AI that will be deployed and add safety inside.

Basically, make sure the AI safety aware people will have the technical expertise, money, computing resources, and enough influence to make safety decisions stick.

At a larger scale, and in a larger multipolar world, this brings the idea that a trusted country should lead rather than leave powerful AI in less responsible hands. This is where “we need to go faster than China” comes in, alongside broader defense and geopolitical concerns.

These first arguments explain why someone worried about AI might still want to build it and stay ahead.

But there are also arguments for why one might want to do it really fast.

3. Move earlier to avoid a bigger shock later

This is probably the most counterintuitive argument: moving faster today can be seen as a way to give humanity more time later.

There are two related ideas here.

First, society needs time to learn how to handle powerful new tools. Introducing AI in manageable stages can be a way to let people discover problems, develop rules, and practice using AI responsibly. Releasing an advance earlier gives people more time to gain experience with smalle, burgeoning, capabilities before much more powerful and disruptive AIs arrives.

Second, even if AI research slows down, computing power may keep improving. A breakthrough that happens later could therefore have much more hardware available to run on, potentially producing a larger, more sudden jump in capability and impact on society.

That accumulated untapped potential is often called an “overhang.”

The overall argument is that making and diffusing incremental progress as soon as possible might prevent a much more abrupt transition later.

Obviously, it also means that we will reach increasingly powerful AI sooner, but the idea is to give more time to adapt and understand between the first useful systems and the really powerful ones.

Note that generally this depends on this earlier progress keeping the transition gradual rather than simply bringing everything forward.

─── �� ───

For our two next arguments, we can take two roads depending on how difficult we think AI alignment will be, that is "How easy do you think it is to make AI reliably do what you want without it deciding to go hack Hugging Face along the way".

Let’s take the first road: alignment turns out to be relatively tractable. Airplanes can fail, but careful engineering has made flying remarkably safe. Suppose we can do the same with AI.

In that case:

4. Waiting has a huge human cost

If AI can help discover treatments, improve education, or prevent cyberattacks, each week we delay it could bring preventable deaths and harm.

From this perspective, waiting is a decision with human consequences too. In a world with huge issues like climate-change, inequalities and poverty, it even become a moral argument for developing AI quickly and bringing its benefits as soon and as widely as is safely possible.

But let’s take a look at the other road: what if alignment is much harder than expected, and making highly-capable AI turns out to be easier than figuring out how to keep them from doing unhinged things?

Well, if alignment is too difficult a problem for humans to solve, then maybe:

5. AI could help us make future AI safe

And we arrive at the same conclusion again: if using AI to build safe AI is the way to solve alignment, let’s get the equivalent of a country full of geniuses helping us as fast as possible.

These genius AI could be the solution to make AI safe by helping researchers find mistakes, test ideas, and develop protections. Instead of relying entirely on humans to solve alignment, we could build systems that help us do the work, each generation could help make the next one safe.

Note that this requires the order of events to work in our favor: AI needs to become useful enough to help solve alignment before it becomes too dangerous to rely on. The hope is to build helpful, trustworthy research assistants before building systems powerful enough to become dangerous.

There are more arguments but in general, these are the main ways people concerned about powerful AI have found rational reasons to end up being the ones building it (and even to build it as fast as possible).

─── ❖ ───

On my side, I think several of these arguments underestimate the complexity of the world and how interconnected people’s reactions are. Moving faster while warning about catastrophe has psychological effects across a whole network of participants: it changes what people fear, whom they trust, and what they feel compelled to do.

And those reactions can change whether the original reasoning actually holds because we live in a world of interconnected humans, not machines (yet).

I also think these rational chains leave some of their consequences for society insufficiently explored. For instance, the concentration of power, shifts in geopolitical alliances, and changes in public opinion. These consequences matter both because they affect whether the strategy works and because they shape the world we end up living in.

But this post is already long, so I’ll leave those questions for the next one.