Shreyas Kulkarni

Research @CapitalOne | prev @Apple, @AdobeResearch
Grad @UMassAmherst, undergrad @IITGuwahati

[Ayushman Singh ✈️ NeurIPS’26 🇦🇺 @ayush1399]

There's still a lot of juice left in existing draft model architectures, just from training them better. DStill keeps the DFlash drafter architecture unchanged and gets up to 28% higher acceptance length and 25% more speedup at the same inference cost. Great work by @shreyask269 and @madhav_kanda_ .

A few training insights from the paper I liked:

1. Running vanilla DFlash for 8 denoising passes lowers acceptance length (4.45 → 3.70 on GPT-OSS-120B), because it only saw fully masked blocks in training. A drafter trained on partially revealed blocks improves with 8 passes (4.50 → 5.31).

2. They distill that 8-pass teacher into a one-pass student with the same architecture. The student is trained to predict the teacher's final block from any state on its refinement trajectory, and recovers 29-36% of the multi-step gain in one pass.

3. Distillation uses a curriculum: the student trains on both intermediate and fully masked states, and the share of fully masked states rises from 0.5 to 0.85 over training. This beats training on fully masked states only.

4. Longer training doesn't explain the gain. DFlash trained for twice as many epochs gets slightly lower acceptance length. The DStill gains over DFlash hold at concurrency up to 32, and the method also improves DSpark (+4.5% acceptance length).

[Shreyas Kulkarni @shreyask269]

📊 The results:

Compared with one-step DFlash:
• Up to 28% higher acceptance length
• Up to 25% faster generation

Up to 6.45× speedup over autoregressive decoding.

No extra denoising passes at inference.

📄 https://t.co/rdsm9S9LXX

[Shreyas Kulkarni @shreyask269]

Can diffusion drafters benefit from multi-step denoising while still drafting in a single pass?

Excited to share DStill, our new work on accelerating block diffusion speculative decoding! 🚀

We learn from the drafter’s denoising trajectories during training to improve draft acceptance, while keeping drafting to a single forward pass at inference.

⚡ Up to 6.45× speedup over autoregressive decoding.
📄 https://t.co/XTtlzqsv9I

A thread 🧵👇

[Shreyas Kulkarni @shreyask269]

💡 Our approach:

Instead of running multiple denoising steps at inference time, we learn from those trajectories during training.

Combined with curriculum learning, DStill improves draft acceptance while retaining a single drafting pass at inference time.

[Gautam Kamath @thegautamkamath]

I'm teaching a course on differential privacy this term. I'm thinking of making the lectures and materials public, but I'm on the fence. Would people be interested in this...? I don't think there's a course on DP with lecture videos online yet. #privacy

[Shreyas Kulkarni @shreyask269]

@georgemporter Only Dropbox offers non-browser-based client for Linux.

[Shreyas Kulkarni @shreyask269]

I have a brand new joke about out-of-distribution examples which you’ll certainly recognise.

[Gautam Kamath @thegautamkamath]

This joke was attacked by data poisoning which is why it's not actually funny.

[Shreyas Kulkarni @shreyask269]

A nice blog on introductory private statistics!

[Gautam Kamath @thegautamkamath]

Jon Ullman and I are doing a series of educational blog posts on private statistics. Check out Part 1 for some bounds for private mean estimation! Comments/questions welcome. #privacy https://t.co/ChxEruMwp2

[Fermat's Library @fermatslibrary]

"The miracle of appropriateness of the language of mathematics for the formulation of the laws of physics is a wonderful gift which we neither understand nor deserve" - Eugene Wigner

[Gautam Kamath @thegautamkamath]

Differential Privacy is named one of the 10 Breakthrough Technologies of 2020 by @techreview, due to its deployment in the 2020 US Census (@uscensusbureau). Exciting time to be working in this space! h/t @Aaroth #privacy
https://t.co/0v96XyqBnD

[Shreyas Kulkarni @shreyask269]

@JakeRMattinson1 @NaranbayarU @liamrk @fermatslibrary That’s correct! You have to keep track of length of all branches and choose the largest one.

[Shreyas Kulkarni @shreyask269]

@NaranbayarU @liamrk @fermatslibrary Not a sound argument. What if, there are multiple possible cases in between; like 337 (also a prime) instead of 137 in the series, and then the series takes a huge turn by itself. Sure the answer might be correct, but your proof is incomplete!

[Travis Tang @_travistang]

Learnt a lot from @twimlai podcast on #differential #privacy, a way of collecting #data about the population while protecting the privacy of individual users by adding noise to the data 🔏here's the podcast: https://t.co/jBdPuKvK0M

[Shreyas Kulkarni @shreyask269]

@unsorsodicorda @roydanroy What you said is correct and what I meant is completely different (my bad). I should have used the phrase “all pointwise errors” instead of “family of events”.

[Shreyas Kulkarni @shreyask269]

@unsorsodicorda @roydanroy What I meant is that convergence a.s talks about the convergence of (X_n - X) -> 0, whereas uniform convergence talks about the convergence of sup(X_n - X) -> 0.
Including supremum makes it a “global criterion” that requires maximum of all pointwise errors to tend to 0.

[Shreyas Kulkarni @shreyask269]

@unsorsodicorda @roydanroy In probability space, uniform convergence implies convergence a.s. but not vice-versa.
LLN talks about single event whereas uniform convergence talks about a family of events.

[Shubham Ugare @shubhamdugare]

For someone who has a lot of plans in the next decade, great video to start it.
Randy Pausch Last Lecture: Achieving Your Childhood Dreams https://t.co/lQCMJ7IeP2 via @YouTube