Saurabh Singh Rajput

Optimizing AI's Perf/Watt,
Research Intern @AMD
PhD Candidate @DalhousieU,
NIT Nagpur CS'20, ex @Fidelity

[Saurabh Singh Rajput @SauftwareBug]

@bkarak My hypo. In period when their models are doing bad and users down, the xtra gpus go in research and training the next model leading to better quality, whereas if your model is good the users eat the compute, leading to lower quality training. Hence probably next claude will be 🔥

[Saurabh Singh Rajput @SauftwareBug]

Omw to make engineered flies, for my cat to play with!

[News from Google @NewsFromGoogle]

For the first time, scientists have mapped the complete brain and central nervous system of an adult male fruit fly — a key model organism in science. 🪰

Working alongside HHMI Janelia Research Campus and the scientific community, @GoogleResearch scientists and researchers used AI to combine millions of 2D images into 3D neural shapes, reconstructing a record-breaking 166,000+ neurons. This foundational map of the adult male fruit fly brain can help accelerate our understanding of the brain, and is a major milestone in neuroscience.

[Saurabh Singh Rajput @SauftwareBug]

@RadishHarmers I feel the opposite, python was written keeping humans in mind as dev hours was the bottleneck, thats not the case anymore, so no point carrying over its inefficient abstraction across slop. But agree with training data point, it needs change imho. Make llms write assembly lol!

[Saurabh Singh Rajput @SauftwareBug]

@amir_mir93 @SfTensor Congrats Amir👏

[Saurabh Singh Rajput @SauftwareBug]

@IndranilPalit1 @ianye0212 👀

[slap tjips @slap__tjips]

@Sentdex I'll run my 3090 at 50% power limit for the next 24 hours out of respect 🫡

[Saurabh Singh Rajput @SauftwareBug]

@SemiAnalysis_ @MilksandMatcha True, even the OpenaAI inference via Cerebras is pretty mid, in terms of model size and capacity, especially given not many people would prefer to use 3.5 spark that they are offering

[Beff (e/acc) @beffjezos]

OpenAI explaining there are three phases of disaggregated compute now:
-prefill (context loading)
-draft (speculative decoding)
-decode

[Saurabh Singh Rajput @SauftwareBug]

@__tinygrad__ Anyone got these slides? Thanks :)

[Saurabh Singh Rajput @SauftwareBug]

@wesamo__ Halifax is perfect!

[Gary Marcus @GaryMarcus]

I am a US Citizen living in Canada.

Let me tell you this.

Canada does not receive the “benefits of being a US state.”

The US does not keep Canada’s citizens safe; it does not repair Canada’s highways. It does not educate Canada’s citizens. It does not take care of their retirement.

Nobody in Canada is asking for any of that.

Canada does not want the “benefits of being a US state”.

Canada does not want to be a US state.

Canada just wants to be free.

[essen @essenaccount]

NOW IS IT CPU BOUND, OR IS IT MEMORY BOUND?

[Saurabh Singh Rajput @SauftwareBug]

@SemiAnalysis_ @__tinygrad__ @Etched

[Saurabh Singh Rajput @SauftwareBug]

"two 2030 AMD racks are expected to deliver the same compute as 570 racks in 2024"

[AMD @AMD]

More AI compute from every watt.​

AMD has reached an estimated 4x increase in rack-scale energy efficiency for AI training and inference from 2024 to 2026.​

See how innovation across the full AI stack is driving progress: https://t.co/107n5YPYDP

[AMD @AMD]

More AI compute from every watt.​

AMD has reached an estimated 4x increase in rack-scale energy efficiency for AI training and inference from 2024 to 2026.​

See how innovation across the full AI stack is driving progress: https://t.co/107n5YPYDP

[Ramin Nasibov @RaminNasibov]

scientific fact

[tweet davidson @andyreed]

“we sandboxed the agent”

meanwhile the agent:

[AI at AMD @AIatAMD]

Meet AMD Skills: A growing catalog of agent skills that helps AI coding assistants work more effectively with AMD hardware and software.

Compatible with coding agents including Cursor, Claude Code, OpenAI Codex, and Gemini CLI, AMD Skills provides the knowledge, scripts, and best practices developers need to build faster on AMD.

Why try it?
→ Install in seconds with the Skills CLI
→ Learn AMD best practices and workflows
→ Build AI applications from client to cloud
→ Pick only the skills you need

Get started: npx skills add amd/skills

Explore the project, install a skill, and let us know what you'd like to see next: https://t.co/hb7XqpLDYu

[Hot Aisle @HotAisle]

amd continues to benchmaxxx...

[steve @gpusteve]

my team figured out how to run Kimi K3 on @AMD MI355X at 952 tok/s/node and 118 tok/s single stream.

3.8x the aggregate throughput/node and 1.3x the single stream decode of B200 and beat B300 on performance per dollar: 48 vs 33 tok/s/$

details in reply