If you love fine-tuning open-source models (like me), then listen.
> Start with 1B, 2B, 4B, and 8B models. (Don't start with a 27B model or bigger at first.)
> Use WebGPU providers. I use Google Colab Pro for any model smaller than 9B. A single A100 80GB costs around $0.60/hr, which is cheap. Enough for small models.
> Donât buy GPUs unless you fine-tune 7 to 10 models. You'll understand the nitty-gritty in the process.
> Use Codex 5.5 Ă DeepSeek v4 Pro to create datasets. Codex to plan, DeepSeek v4 Pro to generate rows.
> Use Unsloth's instruct models as a base from Hugging Face. Yes, there are others too, but Unsloth also provides fast fine-tuning notebooks.
> Use Unsloth's fine-tuning notebooks as a reference. Paste them into Codex, and Codex will write a custom notebook with the configs you need.
> Spend 1 day learning about:
- SFT (supervised fine-tuning)
- RL training (GRPO, DPO, PPO, etc.)
- LoRA / QLoRA training
- Quantization and types
- Local inference engines (llama.cpp)
- KV cache and prompt cache
> Just get started. Claude, Codex, and ChatGPT can design a step-by-step plan for how you can fine-tune your first AI model.
Future tech is moving toward small 5B to 15B ELMs (Expert Language Models) rather than general 1T LLMs.
So fine-tuning is an important skill that anyone can acquire today.
Tune models, test them, use them. Then fine-tune for companies and make a career out of it. (Companies pay $50k+ to fine-tune models on their data so they can get personalized AI models.)
Shoot your questions below. I'll be sharing in-depth raw findings about this topic in the coming days.
I completely agree with her assessment that âClaude has regressed to the point it cannot be trusted to perform complex engineeringâ.
There are two problems with building businesses on top of frontier models. The first is the cost and value capture. The frontier labs are aiming to capture the value that they create and become the new Google, Facebook where, in the past, you would raise venture money to scale your SaaS and you would spend marginally all of it on CPC ads for customers acquisition. Then all that value accrues to Google/Facebook stock. The goal with the frontier labs is capture all that value instead. There is no incentive to bring costs down or democratize access, thatâs going to have to come from innovation outside those labs.
The second is the âtheyâre not going to let us have thisâ problem. (phrase coined by @androolloyd in our discussions/strategizing about this around the turn of the new year) Opus 4.5 was good enough that you could actually make serious progress with it.
Now that theyâve gotten everyone hooked on crack, theyâve diluted the package. Itâs been brutal for over a month. The amount of money weâve spent on the API for this degraded product makes me sick. They have completely nerfed it and thereâs a good chance Mythos is just them re-releasing an unquantized version of the same/similar thing that we already had, just in a way thatâs more economically sustainable for them. And which is rumoured to be gatekept by organization. If youâre operating in this field you have to take these dynamics into account and act accordingly.
@koylanai yup, got the same yesterday. told it to undo the latest changes which broke features instead of fixing bugs, and it wiped all the uncommitted work. previously, undoing would make it walk back the changes in code.
Vaya puto escĂĄndalo.
La Liga de Javier Tebas, que se ha acostumbrado a tumbar medio internet cada vez que hay un partido para evitar la piraterĂa, ha dejado sin servicio una app para localizar a personas. Un padre con demencia estĂĄ ilocalizable por este motivo.
Y no pasarĂĄ nada
@karpathy@karpathy I've been building something for a while now. For now it's just for personal use bu I would love you to try it if you want to take it for a spin.
We are introducing EU Inc. To make building and growing a business across the EU faster, simpler, and smarter.
đž Start a company in less than 48 hours
đž No minimum capital requirement
đž Fully online and borderless
To help EU Inc. companies compete for the best people, employee stock options will be simpler to offer and easier to manage across borders.
We will also fully digitalise insolvency procedures and introduce a fast-track insolvency process for startups.
So that EU entrepreneurs can start again, more easily.
EU Inc. creates a single European company framework.
It is one simple set of rules that works across our entire Single Market of 450 million consumers.
Companies will be able to manage their documents and communicate with administrations fully digitally, in all 27 Member States.