Xitiz

Building Litmus Lab
I build things around LLMs, GPUs and questionable amount of caffeine

[Xitiz @kshitiziit]

@FishySushitwt https://t.co/kyVMbckn99

[Xitiz @kshitiziit]

One of the coolest CUDA instructions I've come across recently is this:
__shfl_down_sync(mask, value, offset);
At first glance, it looks like one of those functions you copy-paste without really understanding.
But it's actually pretty elegant.
A GPU executes threads in groups of 32, called a warp. Each thread has a lane ID (0–31).
__shfl_down_sync() lets a thread grab a value directly from another thread's register within the same warp.
No shared memory.
No global memory.
Just register-to-register communication.
For example:
__shfl_down_sync(0xffffffff, value, 1);
This means:
Lane 0 gets Lane 1's value
Lane 1 gets Lane 2's value
Lane 2 gets Lane 3's value
...and so on, all within the same warp.

[Xitiz @kshitiziit]

Been spending months buried in CUDA, GPUs, and LLMs. Eventually, I needed something different.
Somehow that "break" became learning physics because black holes and spacetime are ridiculously fascinating.

[Xitiz @kshitiziit]

@darijan___ Had to touch grass for a bit

[Xitiz @kshitiziit]

@ranaharshraj7 world models

[Xitiz @kshitiziit]

Current side quest:
The voices said, "Just call the library."
The curiosity said, "Implement the library."
Unfortunately, curiosity won.
So I'm writing everything from scratch in pure NumPy before touching any frameworks.
Github: https://t.co/7pjL12xJHG

[Xitiz @kshitiziit]

One of the most important inference optimization technniques - Speculative Decoding
A small model(Draft model) predicts multiple future tokens, and a larger target model verifies those predictions. The target model accepts the longest valid prefix of the draft model's predictions, reducing the number of expensive sequential decoding steps through the large model and increasing generation speed

Normally transformer based LLMs generate text autoregressively one token at a time. So if there is a 70B params model it does:
forward pass 1 : predict token 1
forward pass 2 : predict token 2
forward pass 3 : predict token 3
.
.
.
Every token requires another expensive forward pass during the decoding phase. for a 70B model forward pass is very expensive
The idea use a small model to guess ahead and use the large model to verify.
so draft model predicts 5 tokens and the 70B target model runs a single verification pass over those proposed tokens.
Why is this faster ?
70B model - > 50ms/token
generate 10 tokens ->500ms
Now speculative decoding
Small model -> 2ms/token
generates 1- tokens -> 20ms
big model verifies -> 100ms
total 120ms instead of 500ms
The actual speedup depends on factors like the draft model's quality and how many of its proposed tokens are accepted by the target model.

[Xitiz @kshitiziit]

Finally it's happening. I'm joining YC W26.
The acceptance rate is even lower than YC.

Y Coimbatore W26.
https://t.co/8NDzWAy8U4

[Xitiz @kshitiziit]

@sudoingX people here are rich rich

[Xitiz @kshitiziit]

@elliotarledge https://t.co/lhdy7KbXdO

[Xitiz @kshitiziit]

@akramcodez https://t.co/n1N9SwLOYP

[Xitiz @kshitiziit]

Before: I understood tiled matmul as "faster naive matmul." Just a word for an optimized version.

After it clicked: tiled matmul isn't really about the math being different it's about reuse. Naive matmul re-reads the same row/column from slow global memory over and over across threads.

Tiling says: load one block into fast shared memory once, let every thread in that block reuse it before moving to the next tile.

The "aha" was realizing this is a memory-traffic problem being solved, not a math problem. The compute (multiply-adds) is identical in both versions.

[Xitiz @kshitiziit]

pip install litmuslab
Landing page + waitlist: https://t.co/Cwk9b8ae8n
GitHub: https://t.co/lhdy7KbXdO Would love feedback from anyone running quantized inference in prod what formats are you actually using and why?

[Xitiz @kshitiziit]

Quantizing an LLM? You're flying blind on which format actually works for your setup.

FP16, INT8, NF4, AWQ, GPTQ, FP8... which one fits your VRAM and doesn't tank quality?

Built a CLI to answer that in one command instead of a week of manual testing.

[Xitiz @kshitiziit]

It doesn't just benchmark it tells you what to actually deploy.

AI-powered recommendations (via Groq) based on your hardware + quality tradeoffs. Graceful failure handling per mode so one broken config doesn't kill the whole run.

[Xitiz @kshitiziit]

@ranaharshraj7 leetcode in shambles rn

[Xitiz @kshitiziit]

@subhraneeltwt i meant send me the cold mail u sent to them😭