Anub Sinha

Building behaviour world models,
Head of engineering at ZKAI Labs

[Anub Sinha @Anub_Sinha]

Albert Einstein, Hideki Yukawa, John Wheeler (the one who coined the word ‘black hole’) & Homi Bhabha at the Institute for Advanced Studies at Princeton. Homi founded India's nuclear program.

[Marcel Rød @marcelroed]

Gigatoken can tokenize the entire internet in <7 hours on one machine.

Specifically, this refers to tokenizing Common Crawl on a dual socket AMD Epyc server, reaching 24GB/s of throughput, as seen here: https://t.co/jroCWjxKxi

[Anub Sinha @Anub_Sinha]

Push intelligence up. Push execution down.

[Anub Sinha @Anub_Sinha]

There is a lot of work that goes into figuring out what you really want - be it in life, traditional programming or handing it off to your AI Agent.

[Philosophy Of Physics @PhilosophyOfPhy]

Kip Thorne arrived at Caltech to study physics without having learned calculus.

Although he had been fascinated by astronomy from childhood, the pace of Caltech was unlike anything he had experienced. He was surrounded by exceptionally prepared students and began to feel that he thought and read more slowly than nearly everyone around him.

In his Nobel autobiography, Thorne admitted:

“I had had no calculus, I was a slow reader.”

For roughly eighteen months, he said that he “stumbled and struggled” through the mathematics and physics expected of a Caltech undergraduate.

Instead of concluding that he was not capable of becoming a physicist, Thorne changed the way he studied.

He created a separate notebook for every important course. In each one, he rewrote the central ideas in his own language, reconstructed the mathematical derivations for himself, and searched for a physical explanation behind every equation.

By the middle of his second undergraduate year, he had caught up. This method of rebuilding difficult subjects from first principles remained part of how he learned throughout his scientific career.

Thorne later became one of the world’s leading experts on black holes, curved spacetime and gravitational waves.

He shared the 2017 Nobel Prize in Physics with Rainer Weiss and Barry Barish for decisive contributions to LIGO and the first direct observation of gravitational waves.

[Anub Sinha @Anub_Sinha]

This scaredy-cat was forced to run for cover when a herd of brave buffalo decided to strike back. The lion found itself “outnumbered”, a sense of number, by its plucky prey and is pictured sprinting “away”, a sense of distance, from the African buffalo in the Kruger National Park.

All animals have a sense of distance and number, assessing when their pack is outnumbered, and whether to fight or fly, calculating whether their prey is within striking distance.

Understanding math is the difference between life and death. That’s just the way all life on this planet earth evolved through Darwin’s natural selection. And so you can say that the basic concepts of math – space and quantity – are already hard-wired into all living forms.

Ability to roughly count and measure approximate distance seems to be at the heart of intelligence across species. The human brain is a bit different in that it can come up with abstractions on top of these primitives.

Be it painting or music it's all based on counts and distances. And the most important tool humans created which is mathematics is a collection of abstractions built on these very primitives.

[DAE India @DAEIndia]

🇮🇳 India Tops the World at the 56th International Physics Olympiad 2026!

🥇🥇🥇🥇🥇India's young physicists deliver an extraordinary performance at #IPhO2026 in Colombia. All five members of the Indian team win Gold Medals. 1/3
@PMOIndia @DrJitendraSingh @HBCSE_TIFR @TIFRScience

[Anub Sinha @Anub_Sinha]

@rhythmrg Exactly! Just as we have Von Neumann machines of every shape and size optimized for different compute workloads, we'll have neural network machines of every shape and size optimized for different intelligence workloads.

[Anub Sinha @Anub_Sinha]

@satyanadella The future belongs to differentiated intelligence.

[Anub Sinha @Anub_Sinha]

The future belongs to differentiated intelligence. Foundation models will increasingly become commodities. What creates enduring value is not the base model, but how an organization teaches it to think, decide, and act in the context of its own business.

If my competitors and I are renting the same intelligence, then none of us has a durable edge. Worse, if the provider learns from our traces, trajectories, and corrections, I am paying to create differentiated intelligence that ultimately benefits the same rented intelligence my competitors can access.

Every enterprise should be able to own, compound, and continuously improve its differentiated intelligence without that learning escaping its trust boundary.

[Satya Nadella @satyanadella]

https://t.co/xv6csf1SbV

[Anub Sinha @Anub_Sinha]

- virtual memory
- cache coherency
- speculative execution
- random-access loads
- complex cache hierarchies

Those are incredibly valuable for general-purpose computing, but they add area, power, and cost for workloads where the access pattern is already known.
Inference is fundamentally a deterministic data flow graph, the memory system can be co-designed around that assumption.

AI hardware will gradually converge toward something that looks less like a GPU and more like a data flow computer:

Flash
↓
Streaming controller
↓
Large SRAM scratchpads
↓
Tensor engines
↓
Activation buffers

HBM exists because GPUs were built for graphics and then dense linear algebra.

Graphics workloads need:
- random texture fetches
- unpredictable pointer chasing
- arbitrary cache misses
- very low latency
That's almost tape-drive semantics.

But if layer L is never revisited,
random access isn't buying you much.

[John Carmack @ID_AA_Carmack]

Memory cost and capacity are significant issues for AI accelerators.

Unlike game rendering, model inference can have a deterministic memory access pattern. You don’t need “random access memory” at all for model weights, and you could tolerate cold-start latencies in the multiple milliseconds, as long as continuous reads were delivered at the necessary bandwidth.

NAND flash is over 100 times cheaper per GB than HBM, so there should be opportunity there, even after giving a flash controller a 1024 bit interface with HBM bandwidth.

You could make a specialized pin protocol that just supported pipelined transfer of full 16KB+ pages from the flash to program-managed accelerator scratchpad memory and improve per-pin performance over HBM, but it might be more convenient to make it still look like a true random access memory with very fragile performance characteristics, where anything but sequential reads falls off a 1000x+ performance cliff.

That has the advantage of automatically using existing cache hierarchies, and providing a natural path to update the flash memory with new model weights. With the stream-to-scratch interface, code has to be completely rewritten before it works at all, while the ram-emulation interface will start off just extremely slow, and you can incrementally sort out the changes for full performance.

There may be cases where there isn’t enough scratchpad SRAM to hold the weights for a layer, which might force you to deploy the old optical drive optimization technique of duplicating data in multiple places on a sequential read to avoid seeking, but there would be capacity to burn.

It might be possible to do something like cuda graph capture to record a memory access trace and have everything magically remapped to a linear sequence, but deploying programmer / agent elbow grease to manage transfers and access in a scratch ram ring buffer would be lower risk.

A split memory system consisting of some channels of flash and some channels of HBM will probably be suboptimal compared to a uniform memory, but it could be much cheaper, and allow much larger models to be run.

I think th case is strong for inference, but you have to stretch more for training. You can still linearize all the weight memory accesses, both reads and writes, but flash memory would quickly wear out from the writes, even if they were all perfectly page aligned. Replacing low-latency HBM with massively parallel cheap(er) DRAM at high latency might still be a worthwhile cost savings.

[Anub Sinha @Anub_Sinha]

LLM inference is much closer to streaming a program than executing arbitrary memory accesses.

If inference is fundamentally a deterministic data flow graph, the memory system can be co-designed around that assumption.

In the long run, I think the biggest opportunity isn't just replacing HBM with flash. It's rethinking the entire memory hierarchy around deterministic streaming.

[Anub Sinha @Anub_Sinha]

Evolution optimised for survival of the species. Intelligence is an emergent property of optimization under resource constraints. Constraints force compression. But compression of what exactly ?

[Anub Sinha @Anub_Sinha]

After good quality microscopes became available, around 1873 is when it all started.. The idea that the brain is a network of neurons and can be modeled as such. Alex Bain.. came up with the hypothesis.