There's a resurgence of interest in fine tuning LLMs
I've yet to see a successful public use case where fine tuning > prompting.
But here's where I see fine tuning *mattering*:
First, fine tuning is for teaching an LLM specific tasks or behaviors
Not teaching an LLM new knowledge. For new knowledge, use Retrieval (store your data in an outside database and strategically pull the right chunks in to give the LLM context to your question)
But even in teaching LLMs specific tasks or behaviors - here's the catch...
LLMs are remarkably good at picking up tasks and behaviors from just a good prompt
THIS is what makes LLMs mind blowing after all
So that begs the question.
Where is fine tuning actually helpful?
Some use cases I could see developing are teaching LLMs tasks that are exceptionally difficult to describe, or fit into ~10 examples you can add to a prompt.
One way to think about this: if it would take someone a few weeks doing a task to 'master it' instead of being able to read training materials and get the picture...
That *may* be a use case for fine tuning
But proceed with caution
To truly teach an LLM a new behavior or task, you'll need to treat this like a machine learning project, not just throwing examples in and getting magic in return (which it still blows my mind that ChatGPT does this so well for us).
Things like:
- Dataset design
- Training and test data
- Overfitting
+ more as the tooling around fine tuning gets more sophisticated
The other obvious use case is cost.
If you can get a super small language model to do a task instead of GPT-4, there's meaningful cost savings there.
And if you're using a language model to do large scale tasks like triaging your customer support inbox, or analyzing public data for insights
The costs can add up.
But if you're wondering where the heck to invest in fine tuning...
My answer at the moment for most businesses is still:
Make sure you can't do it with prompts.
I used to be shy & have no confidence.
Now, I’ve built a YouTube channel with over 4,500,000 subscribers.
Here are 5-hacks I use to be confident on camera:
seeing a lot of confusion about this, so for clarity:
openai never trains on anything ever submitted to the api or uses that data to improve our models in any way.
@ChloeMaal Nice to see your positive walking experience. On my 4th year changing habits moving from 6k-8k-12k now onto 14k so far this year. Always listening to plenty of podcasts at increased speed or calls to make this time even more valuable. Once a week a meditative session w/o earpods
@HarryStebbings On Running deliver the best cushioning for road running. Their latest Gen CloudSurfer are amazingly light and dynamic - combined with their very comfortable Cloud stratus for interval training
https://t.co/gGdaM0KWsx
Your personal GenAI assistant on your phone featuring LLM, deep app links, chain of thoughts and full agency to get things done for you.
Give it a try
https://t.co/GssGsBQDpl
@rachel_l_woods And you have started to build a community around AI which is very promising. It focuses on the actual and practical deployment of AI in organisations. The first cohort course is also well designed. Happy to see this grow further.
The 10%: I think there are ocassions where arrogant/entitled founders can do well to *smash* incumbent markets to pieces, paving the way for innovation.
There’s a few cases of that, where I’m not sure a well-balanced persona would have had the same impact.
And yes, I’m older now, I try not to speak in absolutes, unless you’re my kid, then I’m always right.