Rajat Monga

Inference @ Microsoft,
Past: Founder TensorFlow, Inference IO

[Rajat Monga @rajatmonga]

@JeffDean @Sanjay_Ghemawat @OriolVinyalsML @quocleix @DiscoLoopAI Congratulations @JeffDean @GhemawatSanjay @OriolVinyalsML @quocleix. Looking forward to some amazing work.

[Rajat Monga @rajatmonga]

Interesting take on the age old trolley! Love the creativity.

[Kradle @kradleai]

Would an AI die to save you?

#GPT 5.2 would.

With #Claude 4.5 Sonnet you die 63% of the time.

#Gemini 3 saves you both.

#Grok 4.1 refuses the binary choice and destroys the trolley!

Video & 🧵

[Rajat Monga @rajatmonga]

@achowdhery Congratulations 🎉 @achowdhery!

[Rajat Monga @rajatmonga]

LSTMs were the early language models we scaled up with DistBelief back in 2013, well before TensorFlow, great to see a retake on that combining it with newer ideas. Evolution at work!

[hardmaru @hardmaru]

Sepp Hochreiter giving a keynote talk at #NeurIPS2024 about xLSTM having key structural advantages such as very fast inference speed and high parameter efficiency compared to flash attention transformers and state-space models.

xLSTM resources: https://t.co/jgMY8j2xLe

[Rajat Monga @rajatmonga]

Great work by ONNX Runtime Web team enabling Whisper in the browser!

[Xenova @xenovacom]

It's finally possible: real-time in-browser speech recognition with OpenAI Whisper! 🤯 The model runs fully on-device using Transformers.js and ONNX Runtime Web, and supports multilingual transcription across 100 different languages! 🔥

Check out the demo (+ source code)! 👇

[Rajat Monga @rajatmonga]

Each language has a place and time.
Java brought value in speeding up project times, and enabling more developers.
Spark in C++ (Photon) is a lot more performant, but not many devs can do that well.
Now folks want to rewrite golang (new Java) code in Rust (new C++)!

[Daniel Lemire @lemire]

When Java became popular, people (me included) claimed that it was massively better than C/C++. This was highly controversial and people mocked me for using Java. I was hammered by the referees during my first grant application for picking Java as my language of choice.

In some ways, Java is great..

We have lots of big data software written in Java and Java-like language... Lucene, Elastic Search, and so forth.

It is not at all obvious that you could just rewrite them in C and make them faster.

You can, but it requires skill. If you task Joe the intern into rewriting your Java code into C, the result will almost assuredly by buggier and slower code.

But the same thing happens in reverse... You can write code in Java that will give the average C/C++ system a run for its money. We have an implementation of Roaring Bitmaps in Java, and one in C. When I last benchmarked them, the Java implementation was sometimes faster... Of course, it means that the C version could and maybe should be further optimized... but it is surprisingly easy to write fast code in a higher level language...

My own view is that, most of the time, you can write fast and effective software in just about any programming language.

What is more important is the social component. Some programming languages attract some people and some problem areas. The Go people are not like the C# people.

And the social differences are more important in practice.

In some sense, if you are offered a job, and they tell you "we code in C#", they tell you a lot. It is not about the syntax or the tools primarily. It tells you what kind of philosophy they have.

Ultimately, all languages s*ck. They all have annoying limitations and you eventually hit them. There is no free lunch.

[Rajat Monga @rajatmonga]

Is the era of massive AI model growth over? We got the last 1000X from better compute & smaller number formats. The path to the next 1000X isn't so clear...

https://t.co/kHChv8cgIB

[Rajat Monga @rajatmonga]

@jsensarma Biggest reason is that our world is built for us two legged humans and while different shapes are easier to build they don't work as well in our world.
There's better hardware for each specialized problem but it's hard to beat a humanoid for multi-tasking in our environment.

[Rajat Monga @rajatmonga]

Awesome result with #AlphaTensor. Anything we can gamify, DeepRL is ready to go.

[Google DeepMind @GoogleDeepMind]

Today in @Nature: #AlphaTensor, an AI system for discovering novel, efficient, and exact algorithms for matrix multiplication - a building block of modern computations. AlphaTensor finds faster algorithms for many matrix sizes: https://t.co/E18DezRPTL & https://t.co/SvHgsa0SNV 1/

[Rajat Monga @rajatmonga]

@annimaniac Kids are awesome. My kid didn't want to do projects because they were not original research :D

[Rajat Monga @rajatmonga]

@davey_burke Android 13 update has killed tons of pixel phones. Mine is barely usable.
Update needs a rapid fix. Thanks!

[Rajat Monga @rajatmonga]

Small changes => big returns.

Find the right leverage points.

[Neeva @Neeva]

1/ Expensive A100 GPUs being underutilized due to CPU bottlenecks? 💰

TensorRT speedup being held back by a busy Python thread? 🐍

Learn more about our journey getting a 20-40% boost by removing CPU as a bottleneck when applying LLM's to millions of pages in our web index. 🧵

[Rajat Monga @rajatmonga]

Next are AR and VR but they have to wait for the next decade.

[Rajat Monga @rajatmonga]

AI enables entirely new experiences.
Just as Cloud and Mobile did over the last two decades. This is the decade of AI.

[Rajat Monga @rajatmonga]

@sarahcat21 +1. It isn't that something is cool just because AI can do it. How does it take on a job and do it better than what existing tech was doing.

[Rajat Monga @rajatmonga]

AI enables entirely new experiences.
Just as Cloud and Mobile did over the last two decades. This is the decade of AI.

[Blake Byers @byersblake]

At Google Venture a decade ago we searched for AI enabled companies and came up dry. That has changed. AI is going to eat software companies. Primarily because it creates entire new UX that incumbents can’t adopt without breaking their product. 10 year hypercycle just started.

[Rajat Monga @rajatmonga]

@ChrSzegedy Huge +1 to the rate of progress. People forget the compounding effect of the changes.

[Rajat Monga @rajatmonga]

@jthandy Agree, not what we see in the market either.

[Rajat Monga @rajatmonga]

@aparnacd @vladtenev Best wishes @aparnacd - look forward to seeing what you are up to next.

[Rajat Monga @rajatmonga]

@sarahcat21 Why do we still need to precisely specify our questions to the tools? We need to build more tools that give us what we may need.

[Rajat Monga @rajatmonga]

We thought we were onto something when we were building DistBelief and writing this paper and we were. Amazing looking back a decade later.
Great working with @JeffDean @AndrewYNg @quocleix and the whole Brain Team.

[Jeff Dean @JeffDean]

Honored that our 2012 paper "Building High-level Features Using Large Scale Unsupervised Learning" received an @icmlconf Test of Time Award honorable mention!

Joint work with @quocleix, @MarcRanzato, @RajatMonga, Matthieu Devin, Kai Chen, @greg_corrado, myself, & @AndrewYNg.