i am adapting to ai pretty well but...
i KNOW that there are absolute DEMONS of mankind who are not posting anything and have no online trace, who have adapted to it and use it like fucking Mozart on piano
i would like to meet them someday and measure myself
fixed issues with instruction tuning, LoRA was making the model too defiant, but mixing in the persona data with regular instruction tuning data during LoRA training made it go away
ok some back of the envelope calculations: peft to gguf takes roughly around 1 minute, can train a persona LoRA in 30s, and decode speed is roughly 2x on gguf q8 with little to no quality loss on the LoRA
routing LoRA? not sure if it works with ggufs yet, in peft it only decreased decode speed by 10-20% for swapping the LoRA every 10th token