Dominic Cabansay

FDE @ and rl

[Dominic Cabansay @cabddl]

how do i combine both pstack and matt pocock's skills

[Dominic Cabansay @cabddl]

@jensenhuangyk gotta find the configurations to have wholesome qwen

[Dominic Cabansay @cabddl]

on a real, how do you think of this 😂

[lyra @_lyraaaa_]

if you have a qwen3.5 4b around, please try steering neurons L14 N8949 +1 and L13 N7956 -1
now you have evil qwen

recipe: dose × median(‖resid‖ leaving layer L, tokens 1:) / ‖W_down[:, n]‖` at the down_proj at every input

[Dominic Cabansay @cabddl]

@creatine_cycle im going as a qwen3.5 4b with neurons L14 N8949 +1 and L13 N7956 -1

[Dominic Cabansay @cabddl]

really appreciate the distillation take! from the small-gpu side, distilling first and then doing rl is basically the only order where rl pays off, since otherwise most rollouts score zero. so i feel like open models getting closer to the frontier through distillation is kind of what makes small-lab post-training possible at all

[Dominic Cabansay @cabddl]

honestly the split where claude code starts full sft 71.9% of the time and codex starts lora 89.5% is the most interesting number here for me. like the default strategy is basically a property of the harness, not the task. and one sentence of human nudging giving +17pp on gsm8k kind of says the bottleneck is knowing when to stop and rethink, which tbh is also what i'm worst at when i'm fine-tuning on a gpu budget lol

[Dominic Cabansay @cabddl]

i mean the intrinsic dimension framing is kind of the cleanest argument for why lora works at all when you only have one or two gpus. the thing i still don't have a good intuition for is picking rank per task, like is there a cheap way to estimate it before you burn a full run? would love if that's in the video!

[Dominic Cabansay @cabddl]

genuinely feels like i have a personal teacher

[Dominic Cabansay @cabddl]

am i the only one that just found out that claude has a voice function where claude talks to you?

[Dominic Cabansay @cabddl]

did this up quickly! i wanted something simple and straight to the point haha

[Dominic Cabansay @cabddl]

https://t.co/ePIqV05XZL

[Zara Zhang @zarazhangrui]

Your personal website is the new resume

[Dominic Cabansay @cabddl]

@BoboLinNotes @huggingface exactly!

[Dominic Cabansay @cabddl]

i've been trying to go deep on async rl recently, this @huggingface blog post: is such an insightful read.

on a high level,
1. trl's async grpo trainer can now train just a LoRA adapter and only send that adapter to the inference servers like vLLM, not the whole model.

2. normally you'd need a fast direct gpu-to-gpu connection to ship GBs of weights after every update. but a rank-1 adapter is basically a few MBs, so they skip all that and use a shared storage bucket instead.

i feel like the easiest way to think about it is basically a versioned deploy. the trainer saves each new adapter (v1, v2, v3...) to a shared bucket, then just tells every server 'hey, load v7 from this path.' and the servers never get the weights sent to them directly, just a pointer to where they are, so the trainer and the servers can live on totally separate machines

3. there's also a small proxy that sits between the trainer and the vllm servers, so the trainer thinks it's talking to just one server. it sends each rollout to whichever server already has that prompt cached, and makes sure every new adapter reaches all of them

the best part of reading this blog was seeing the results. just by reading their metrics and tuning the run, they cut the time to run 500 steps of rl from 3h27m down to 53 min, with basically the same reward

definitely worth a read if you're doing rl on a small setup, as someone who can't fund thousands on a full async rl setup, this is definitely worth a try!

[Dominic Cabansay @cabddl]

@jensenhuangyk @felixrieseberg respect

[Dominic Cabansay @cabddl]

@jensenhuangyk

[Dominic Cabansay @cabddl]

i really wanna try switching to opencode/pi and see how GLM5.3 flash compares to opus5.5

[Dominic Cabansay @cabddl]

@huggingface read the full post here!
https://t.co/yRsBazw44O

[Dominic Cabansay @cabddl]

@avigyabasnet @LilaSciences activation offloading is such a hack when saving memory and cost

[Dominic Cabansay @cabddl]

@Sbhaiwala03 evalception

[Dominic Cabansay @cabddl]

tbh the assignments are basically the whole value here, i mean watching a lecture on flashattention is one thing but writing it in triton is where you actually get why memory matters. the sft + rl for math assignment is also kind of the cheapest way i know to learn post-training without like a lab budget

[Dominic Cabansay @cabddl]

this is really cool, i like that the reasoning is capped at like 80 tokens instead of letting it ramble. i'm kind of curious what the custom reward looked like though, was it mostly scoring the final option or did you also reward the short reasoning somehow? asking bc i've been post-training qwen3.5 on small legal tasks and tbh the reward design is where most of my time goes lol