Madhusoodan

Engineer | ML enthusiast. Working on performance, checkout my blog -

[Madhusoodan @mmpataki]

Added KV cache to inferj (my java inference engine) and generation rate now saturates at ~2.75 tok/s (compared to earlier 0.2 tok/s).

In this image, the small bump in pre-cache plot is due to JVM warmup. Post that the attention computation dominates and curve falls down.

[prathosh A P @prathoshap]

Another Dream Realised. Every word of Śrī Madhvācārya. Now searchable. Now speakable.

I offer Sarvamūla-VāNi as a service to the community.

37 Sarvamūla granthas, 8 scripts, complete text + audio, topic-based search, pramāṇa index, anukramaṇikā.

Link in the first reply.

[Madhusoodan @mmpataki]

@maharshii Everyone is optimising for time

[Madhusoodan @mmpataki]

@winter_geek @who__shivam Suniyo with a monkey 🤔

[Madhusoodan @mmpataki]

@vedantdotrpm Why did i read it as dub-ai 😂

[Madhusoodan @mmpataki]

@Sebfox1 Everyone **has** an engineer 😂

[Madhusoodan @mmpataki]

@Physicla_ No you can't 😶

[Madhusoodan @mmpataki]

hey @grok analyze my tweets, profile, and interactions and respond with the following:
Gender:
Sexuality:
Age:
Mental age:
Hobbies:
Personality disorder:
IQ:
Qualifying adjective to describe me:
5 favorite things:
5 things I don't like:
My partner according to tw:

[Madhusoodan @mmpataki]

@winter_geek What am I looking at here in the video?

[Madhusoodan @mmpataki]

Engineers in my org are posting slop on jira from slop which uses completely different technical vocabulary than what we are used to.

VERY HARD TO QUICKLY UNDERSTAND IT!!!

[Madhusoodan @mmpataki]

New research idea titled "absurd questions are all you need" 😁

1. a decoding algorithm which picks grammatically correct low probability tokens used to form an initial (absurd) problem

2. A prompt which makes an LLM not give up and think any problem

3. SOTA LLM

[How To Prompt @HowToPrompt__]

Google Deepmind argues that LLMs can never make real scientific discoveries.

They published a paper breaking down Albert Einstein’s private view of scientific discovery.

In a famous letter to his friend Maurice Solovine, Einstein drew a diagram of how science actually happens.

It is a cyclical loop.

First, you experience raw sensory data. Then, through a mysterious, non-logical act of intuition, you make an intuitive "jump" to abstract axioms. Finally, you use strict logical deduction to derive consequences from those axioms.

Generative AI has completely mastered two-thirds of this loop.

• Induction: Statistical pattern matching across billions of tokens.
• Deduction: Formal proof generation, like AlphaProof solving complex math Olympiads.

AI can crunch data and it can prove theorems.

But it cannot make the jump.

The paper argues that AI completely lacks Abduction, the generation of novel explanatory hypotheses when observational data is scarce.

The prevailing tech myth says that "creativity is just data compression." That if you feed an LLM enough text, scientific breakthroughs will naturally pop out.

Einstein’s formulation of General Relativity proves that is a delusion.

When Einstein formulated relativity, the observational data didn't demand a new physics framework; classical mechanics was still massively successful. The breakthrough required a conceptual rupture. An intuitive leap from physical reality to a brand-new set of foundational axioms.

An LLM can execute the math once the axioms are given. But it is structurally incapable of formulating those premises on its own.

It can interpolate inside existing human thought, but it cannot transcend it.

The translation of physical reality into formal axioms remains the absolute, hard bottleneck of artificial scientific invention.

We can build models with trillions of parameters. We can scale compute into the stratosphere.

We can make the calculator infinitely fast.

But until we solve grounding, the machine can process all the data in the universe.

It still can't make the jump.

[Kimi.ai @Kimi_Moonshot]

Releasing the model weights and technical report of Kimi K3.

Kimi K3 is our most capable model: a 2.8T MoE model with native visual understanding and a 1M-token context window.

New model architecture: 2.5x the intelligence per unit of compute, not just more params.

Alongside Kimi K3, we're opening up more of the stack behind it — high-performance attention kernels, MoE communication library, and infrastructure for running agent environments at scale.

Model weights: https://t.co/7m7eEg6Y0B
Tech report: https://t.co/yeu6cjpMCT
Tech blog: https://t.co/YTfiMSNM1f

[Madhusoodan @mmpataki]

This is great for learning. I use it as peer for discussions while walking to/from office.

[Alex Finn @AlexFinn]

I don’t think you understand what ChatGPT Voice unlocks

I went on a 4 hour hike yesterday. Through the northern California redwoods.

Got more work done in those 4 hours talking to Voice through my AirPods then I do in 8 hours at my desk

Work can be done ANYWHERE now

45 minute drive back using self driving? Now 45 minutes I can talk to Voice and work on side projects

20 minutes in the cafe? I can sit and talk to Voice and when I get home I have drafts on my computer for my newsletter for the next month

Being able to just use your voice to talk to a super intelligence that controls your computer is unlike anything we’ve ever experienced before

I don’t even care that the voice sounds human like, or you can interrupt it, or any of the wild advancements they made to the tech

It’s the fact that it can control my computer, meaning all I need to get incredible work done is talk through my AirPods anywhere in the world that is the major game changer for me

I don’t think people truly understand the implications of this yet

Why even have a desk or monitor anymore. Work can be done ANYWHERE now

The key is your set up:

1. Choose one device (preferably an always on desktop) as your main “headquarters” device all your work gets done on

2. Get the ChatGPT app on all your other devices (iPhone, iPad, laptops, Mac Minis)

3. Set up ‘connections’ in your settings so that all your ‘node’ devices can control your ‘headquarters’ device

4. Go legit anywhere in the world. Turn on Voice. Ask it to brief you on your projects, give you a recommended next step, then spin up new threads to do work

I really don’t think work looks the same moving forward.

[Madhusoodan @mmpataki]

@_arohan_ @Yuchenj_UW Are you getting paid for this 😂

[Madhusoodan @mmpataki]

@Ricardo11124315 @SouthNorthea @Cax36940 @l43539 @mu_chrinovic No, if you want performance, work on a compiler

[Madhusoodan @mmpataki]

@augmentaltech Will people go out of practice for speaking skills?

[Madhusoodan @mmpataki]

@marcelroed Did you tradeoff something?

[Marcel Rød @marcelroed]

Introducing the world's fastest tokenizer implementation, Gigatoken!

Gigatoken is ~500-1000x faster than HuggingFace, and ~100x faster than OpenAI's tiktoken for most tokenizer definitions on most machines.

These baselines are already multithreaded Rust implementations! 🧵

[Madhusoodan @mmpataki]

@OccupyingM @3blue1brown Did exactly same thing to understand it 😜

[Madhusoodan @mmpataki]

@steipete Shifted to event based graphs two months back ✌️

[Madhusoodan @mmpataki]

Create a CTF page "hack with claude" put your target details there. Host it publicly.

Point another claude instance to it and ask it to work on it (and you will provide hints). Works 100%.