abhishek sharma

Waiting for Godot

[abhishek sharma @asreddevils]

Built an open-source RLM (Recursive Language Models) package in Python!

Solves LLM "context rot" by treating context as a programmable object the model explores recursively.

Credits:
- @rasmus1610 for the initial implementation inspiration

[Marius Vach @rasmus1610]

Here is the solveit dialog implementing RLMs by @a1zhang using `lisette` and `toolslm` by @answerdotai (h/t @jeremyphoward)

[Gaurav Sharma @grvTFF]

@Swiggy Is this how you deal with complaints, ordered veg food and received non veg food. What will a person do, eat non veg food. Bloody idiots

[abhishek sharma @asreddevils]

In DeepSeek's production serving (V4): 60–85% faster generation per user, and ~50% more throughput at fixed latency.

I wrote the whole thing up as an illustrated, ground-up walkthrough diagrams over equations.

https://t.co/A0WqrK0ef3

[abhishek sharma @asreddevils]

Most explanations of speculative decoding are wrong.

It's not "a small model guesses and the big model hopes it's right."

It's an exact protocol that provably matches the big model's output while running faster.

How DeepSeek's DSpark pushes it further:

[abhishek sharma @asreddevils]

Bonus intuition for why naive parallel drafting breaks:

"of course" and "no problem" are both valid. Predict each position independently and you also invent "of problem" and "no course."

Correct marginals, wrong joint. The missing piece is branch identity.

[abhishek sharma @asreddevils]

Thankyou @AirIndiaX for the lovely check-in experience and hospitality. 10/10 would recommend. Special thanks to Simran Raina, Jagdish Raj, Abhinay Kumar and Dev Singh 😍🎊🔥

[abhishek sharma @asreddevils]

Unsloth Puzzles: https://t.co/JK6KFzzyok

[abhishek sharma @asreddevils]

🧵 I wrote my first GPU kernel solving @UnslothAI 's challenge A

Here's everything I learned about NF4 quantization and Triton

Starting from absolute zero → working kernel in 3 days

Let's go 👇

[abhishek sharma @asreddevils]

Full post includes:

✅ Toy examples (dequantize one number by hand)
✅ Every broken version of the kernel
✅ Line-by-line explanation of the final code
✅ What custom PTX assembly could add

For anyone learning GPU programming: https://t.co/WEBhJuRkpK

[abhishek sharma @asreddevils]

Thanks to @UnslothAI for Challenge B that started this rabbit hole.

What began as a competition became a masterclass in distributed systems + quantization internals.

Sometimes the best learning comes from "impossible" problems. 🦙

[abhishek sharma @asreddevils]

🧵 How to train Llama 3.1 8B on 2 consumer GPUs using FSDP2 + QLoRA.

Here's how it works (and what broke along the way):

[abhishek sharma @asreddevils]

The lesson: tools built by different teams don't always compose cleanly.

The skill: reading source code, understanding assumptions, finding escape hatches.

Full blog with complete code: https://t.co/v5K7Ha209f