Fine-tuning
When RAG is not enough — knowing the difference saves you months of work going down the wrong path
What is fine-tuning
The difference between hiring someone already trained and training someone yourself.
When you call a general-purpose model through an API, you are working with someone who trained for years on general knowledge. They know a lot. They write well. They reason clearly. But they do not know your company's specific terminology, your product's quirks, your domain's conventions. You explain things in the prompt every time.
Fine-tuning is teaching. You take a pre-trained model and train it further on your specific data — your examples, your style, your domain. The model updates its weights. It does not just follow your instructions. It internalises your patterns.
The number that makes it real
Fine-tuning a model requires training examples — typically hundreds to thousands of input/output pairs demonstrating the behaviour you want. It costs compute time, which providers charge for. It produces a model checkpoint that you then deploy and run, paying inference costs on top.
The cost is higher than RAG upfront. The result is a model that behaves differently at a fundamental level, not just when you prompt it carefully.
When fine-tuning is the right choice
Fine-tuning earns its cost when you need the model to do something that cannot be achieved reliably through prompting or retrieval alone.
Consistent tone and style across thousands of generations. Domain-specific reasoning that requires deeply internalised knowledge. Tasks where the correct behaviour is better shown through examples than described in instructions. Cases where prompt length is a constraint and you cannot fit all your context.
When fine-tuning is not the right choice
If you need the model to answer questions about your documentation — use RAG. Faster, cheaper, easier to update.
If you need the model to follow specific instructions — improve your prompt first. Instruction-following in frontier models is strong. Most tasks that seem to need fine-tuning are solved by a clearer system prompt.
Fine-tuning is not the first tool. It is the tool you reach for after the simpler options have been fully explored.
Verified March 2026 · Source: Anthropic, OpenAI fine-tuning documentation