Harshitt Dave

ML Engineer

[Harshitt Dave @iam_hdave]

@neharedy Any in Koramangala?

[Harshitt Dave @iam_hdave]

@Ayussheth Me🙋🏻‍♂️… would love to join

[Harshitt Dave @iam_hdave]

https://t.co/CpMx0vPswD

ps: my first reddit post :)

[Harshitt Dave @iam_hdave]

Tried Recursive Language Models (RLMs) on a company project — impressive results.
RLMs enable LLMs to recursively inspect data via a REPL-style loop, allowing for reasoning over massive contexts without context rot.

Must-read: https://t.co/DEniNp94AB

@a1zhang #RLM #LLM #NLP

[Harshitt Dave @iam_hdave]

Trying on-policy distillation between Qwen3-8B and Qwen3-30B-A3B on tulu3 dataset.

Showing around 13$ spend and the progress is 0.2% 🤯 (saw it using the progress/done_frac metric).

I don’t know but I also feel like I am not using it efficiently.

Need to deep dive 🧠🔍

[Harshitt Dave @iam_hdave]

Tinker is so expensive!!!💀
Cc: @thinkymachines

[Harshitt Dave @iam_hdave]

Next few things on my list:
• Experiment with on-policy distillation (everywhere on my timeline 👀) between my fine-tuned model and Qwen-3-0.6B.
• Use the newly distilled 0.6B model as the draft model for speculative decoding to speed up generation while maintaining quality.

[Harshitt Dave @iam_hdave]

Been deep-diving into persona post-training lately.

Built a small (but mighty) human-agent ↔ user conversation dataset and fine-tuned an LLM to mirror the agent’s writing style.

A fun blend of persona modelling and GPU optimisation.
#LLM #FineTuning #Qwen #LoRA #Quantization

[Harshitt Dave @iam_hdave]

Plan Ahead:
While doing all this, I got hands-on with the Muon Optimizer, fine-tuned Qwen3-0.6B on a subset of my dataset.
Wanted to use it for the main run too, but DeepSpeed stage-3 doesn’t support it yet. Excited to explore this next.

[Harshitt Dave @iam_hdave]

@outofofficedaku