Jan Newger

Computer Architecture Research @ GoogleDeepMind.
Tweeting about C++, compilers, performance, reverse engineering. Opinions my own.

[Hugh Zhang @hughbzhang]

A question I’ve been pondering: what if we'd known about o1 / RL on chain-of-thought back in the early days of LLMs?

It turns out SFT + a bit of RL on GPT-2 almost matches the performance of a fine-tuned GPT-3 (12b) on GSM8K — a model with >100x the pre-training compute.

[[email protected] @HaskRay]

https://t.co/7OKcL1zkYt "A deep dive into SmallVector::push_back".

SmallVector::push_back optimization has made clang faster.

[Jan Newger @ObfuscaTHOR]

Posted the source code for the research prototype here:
https://t.co/2CKqN16zUK

[Jan Newger @ObfuscaTHOR]

CopySanitizer: identifying unnecessary object copies in C++ applications at runtime.

https://t.co/i4w0cbXzHc

[Simon Smith @_simonsmith]

It strikes me that Google’s advantage with distribution in AI is also a contributor to it falling behind the frontier.

Anthropic and OpenAI have to support essentially one channel each, Claude/Code/Cowork and ChatGPT/Codex respectively. Google has Search and Workspace and YouTube and Android and Cloud and a long tail of other stuff.

So Google can’t just have a frontier LLM that’s great at business use cases. They also need, for example, a good video model to support YouTube. OpenAI could kill Sora to focus on frontier LLMs but Google can’t cede video generation entirely or YouTube will suffer versus competitors like TikTok.

That same dynamic exists across Google’s broad portfolio, and the result is a logical strategy that prioritizes having a top-10 model for a lot of different use cases versus a top-3 model at the frontier in any specific domain.

Even Google’s compute constrained. And it can’t pour all its limited compute into one area because unlike Anthropic and OpenAI it both benefits and suffers from broad and diverse distribution.

[Robert Clausecker @FUZxxl]

@ZeteticAdvocate Curiously, many Germans believes that a heat pump is a wonderful thing that will save our planet, while an AC unit is a spawn of the devil. It's funny to point out that the two are fundamentally the same device and watch them try to explain away the contradiction.

[Jeff Dean @JeffDean]

My @Google colleagues @NormJouppi, Sridhar Lakshmanamurthy, Cliff Young, and David Patterson recently wrote a paper that will appear in the July/August 2026 edition of @ieeemicro titled "Google's Training Supercomputers from TPU v2 to Ironwood: Architectural Stability, Scale, Resilience, Power Efficiency, and Sustainability Across Five Generations". It's chock full of interesting data about the evolution of TPU chip generations, as well as how workloads at Google have transformed over time (hint: lots more transformer-based models!), and how the generations have gotten ~30X more energy efficient per flop.

Lots of changes over these generations:
Air cooling in TPUv2 to water cooling in TPUv3 onwards
2D to 3D torus-based interconnects
30X improvement TFLOPS/Watt
256 chips (TPUv2) to 9216 chips (Ironwood) per pod

Read the full paper: https://t.co/D5NFYFv19V

[Jan Newger @ObfuscaTHOR]

CopySanitizer: identifying unnecessary object copies in C++ applications at runtime.

https://t.co/i4w0cbXzHc

[Sarah Necker @SarahNecker]

Es wird hier immer wieder als Erfolg beschrieben, dass das 9 Euro Ticket dazu geführt hat, dass ärmere Menschen in die Berge fahren konnten. Das gehörte nicht zu den definierten Zielen: „Mit dem Ticket sollen die Leute einerseits wegen der stark steigenden Kosten für Strom…

[Marie-Agnes Strack-Zimmermann @MAStrackZi]

@Karl_Lauterbach Er hat der deutschen Sicherheits- und Außenpolitik nicht nur in der Regierung der Ampel, sondern auch in den Jahren davor schwer geschädigt und die Glaubwürdigkeit untergraben. Das zu sagen, ist eine Feststellung der Realität und kein persönlicher Angriff.

[Frank @jedisct1]

An update on GitHub availability https://t.co/EdY8kLfx2E

[Steve Magness @stevemagness]

What in the world did we just see!

The 2 hour marathon barrier has been broken. Three guys went under the old world record...

Sabastian Sawe just ran 1:59:30 with crazy negative splits, closing the last half in 59:01....faster than the American Record in the half.

One of the most mind blowing performances we've seen. How did we get here?

Every breakthrough is a mixture of belief and progress.

It takes folks daring to see what's possible, surrounding themselves with a quality team and doing the work to give themselves a shot.

You've got to bet on yourself in a big way.

When asked whether he believed he could run a sub-2-hour marathon before the race, Sawe answered with one word:

"Yes."

Let's get the obvious out of the way. Performance enhancing drugs are the legitimate question mark to every breakthrough.

So Sawe did as much as he could about taking that off the table.

He and his team asked to be tested all the time. His sponsor put up 50K to the Athlete Integrity Unit. The tests are run independently, no advance notice. Over a 2 month stretch, he went through 25 drug tests.

There's always a doubt. There has to be given what we know. Hopefully there's transparency in the results. But hats off to Sawe for addressing it:

"I want to prove that I am clean when I set foot at the start line."

But how'd we actually get here where two guys went sub 2 in the same race?

1. Shoe tech

We've had a revolution in shoe technology that boosts running economy.

For years shoe companies said their shoe would make you faster and was mostly marketing. Until 2016, when it actually did.

Initial research showed a 3-4% saving in economy, while subsequent work has shown it's highly variable.

Now, it's a matching game. Find the perfect shoe for your form and you can get a big boost.

Normally, it takes years of lots of miles and strength training to boost economy.

But now we get that instant boost that not only helps boost performance but often leaves us feeling less beat up in the later stages of the marathon.

So we get a little bit less hitting of the wall...

2. The fuel

For a long time, fueling was limited by biology. You can only take in and process so much.

Then in the 2000s, researchers found if we mixed sugars, we can boost intake because they're processed differently.

Then recently, Maurten found if you use a hydrxogel, you boost utilization without GI distress anymore.

We've gone from pushing 60g/hr to 120g/hr in a few decades.

Again...less bonking.

3. Depth

A few decades ago, you spent your career racing on the track and then once your speed started to fade a bit you went to the marathon.

Now, many skip right to the marathon. That's where the money is.

And with the economy boost from the shoes, you can make that jump quickly.

More depth of talent means more competitors in their prime pushing barriers.

4. Belief

Even with the shoes and tech, a few years ago sub 2 hours seemed a long way off, until Kipchoge pushed that barrier in a series of time trials.

Yes, they weren't official races and had contrived pacing. But it absolutely shifted everyone's thinking on what is possible.

A generation of runners saw Kipchoge go for it.

Our prediction of what is possible changed.

It's mind blowing how far we've come in such a short time.

What once seemed decades away, just got smashed twice in the same race.

Hats off to Sawe, especially for addressing the scourge of doping and showing folks what is possible with a lot of hard work, some crazy belief, and some fortuitous advances.

[Jan Newger @ObfuscaTHOR]

It is kinda frustrating how accurate Sutton's "The Bitter Lesson" is.

https://t.co/bXMb23AsWq

[Vincenzo Iozzo @_vincenzoiozzo]

Given all the news about Mythos,  I ran a small experiment testing Opus 4.6 to understand a bit better how it finds bugs.

The setup was: Sendmail crackaddr() bug (CVE-2002-1337) — the original source, a rewritten equivalent, a compiled binary with symbols, and an obfuscated stripped binary.

The model found the bug quickly in the first three cases (under 3-4 minutes). The obfuscated version took ~45 minutes of actual "reasoning".

A few things stood out:

- The model behaves like a human bug hunter would: switching between "pattern matching" and dynamic analysis, using runtime feedback as an oracle

- Having an oracle is crucially important. So much so that the agent constructed its own when instructed not to run the binary

- The gap between "pattern matching" and "reasoning" capabilities seems significant. The latter appears fairly primitive for Opus. Is Mythos better purely because of the much larger context window or is it something else?

- Opus behaves deceptively fairly often. It's surprising how much a hidden scratchpad helps (this is similar to the Sleeper Agents approach)

Full writeup:
https://t.co/YHFc2ywaI5

[Jan Newger @ObfuscaTHOR]

@BlackHC hard same

[Jan Newger @ObfuscaTHOR]

https://t.co/2s6OeisqgQ
Rowhammer-like attacks on GPUs

[Google Research @GoogleResearch]

Introducing TurboQuant: Our new compression algorithm that reduces LLM key-value cache memory by at least 6x and delivers up to 8x speedup, all with zero accuracy loss, redefining AI efficiency. Read the blog to learn how it achieves these results: https://t.co/CDSQ8HpZoc

[Dean W. Ball @deanwball]

Think about the power Hegseth is asserting here. He is claiming that the DoD can force all contractors to stop doing business of any kind with arbitrary other companies.

In other words, every operating system vendor, every manufacturer of hardware, every hyperscaler, every type of firm the DoD contracts with—all their services and products can be denied to any economic actor at will by the Secretary of War.

This is obviously a psychotic power grab. It is almost surely illegal, but the message it sends is that the United States Government is a completely unreliable partner for any kind of business. The damage done to our business environment is profound. No amount of deregulatory vibes sent by this administration matters compared to this arson.

[Xavi Ruiz @xruiztru]

6 year ago today, Dr. Li Wenliang passed away of COVID-19.

He warned the world about a virus outbreak in Wuhan. The government silenced him.

The rest is history.

[BrooklynDad_Defiant!☮️ @mmpadellan]

Dear World,

For those of you who loudly BOOED JD Vance at the Olympics, we THANK YOU for your support. 💙🇺🇸

Love,
Americans who can't stand JD Vance

[French Response @FrenchResponse]

Panicked EU bureaucrats right now.