A support team's prompt has grown to three pages: tone rules, the JSON format for ticket tags, thirty sample replies. Every request pays for those tokens, and the assistant still slips out of format now and then. One engineer suggests fine-tuning the model on the team's past replies; another says the assistant needs RAG instead. Both can be right, for different parts of the problem.
This post covers what fine-tuning is, its main methods (SFT, RLHF, DPO and LoRA), how it compares with prompting and RAG, the data it needs, its risks and where you can fine-tune today.
What is fine-tuning?
Fine-tuning is training an already trained model further on a smaller, task-specific set of examples, so that its behaviour shifts toward that task. The model's parameters, the billions of numbers (weights) it adjusts while learning, start from where the earlier training left them, so fine-tuning needs a small fraction of the original data and computing power.
The starting model is called the base model or pretrained model. For text tasks it is usually a large language model (LLM), a model trained on huge amounts of text to predict the next piece of text; our guide to how LLMs work covers that first stage. Fine-tuning is a form of transfer learning: what a model learned on one task is reused for another.
The recipe became standard around 2018, when the authors of BERT showed that one pretrained model "can be fine-tuned with just one additional output layer" for many tasks. Outside AI, "fine-tuning" also names an argument in cosmology; this post is about models only.
How is fine-tuning different from pretraining?
Pretraining builds a model's general knowledge from scratch on a huge text corpus, while fine-tuning adjusts that trained model's behaviour with a much smaller, curated dataset. In pretraining the model reads books, code and web pages and learns to predict the next token, a word or piece of a word. It is the expensive stage, run by the organisations that build base models.
The two stages teach different things. The LIMA study fine-tuned a 65-billion-parameter LLaMA model on only 1,000 carefully chosen prompts and responses, and its authors concluded that almost all knowledge in large language models is learned during pretraining; fine-tuning mostly shapes how it comes out. In between sits continued pretraining: more unlabeled text from one domain, with the same next-token objective.
How does fine-tuning work, step by step?
Fine-tuning works as a loop: define the task and a test, prepare examples, train, compare the result with the test and repeat.
- Define the task and the test. Collect real inputs with the outputs you expect before any training; without them you cannot tell whether tuning helped.
- Measure a prompting baseline. Run the test against the base model with a clear prompt and a few examples. If it passes, stop here.
- Choose a base model. A hosted model with a tuning service or an open-weight model you run yourself.
- Collect and label examples. Each example pairs an input with the wanted output, usually stored as JSONL, one JSON object per line, without duplicates or personal data.
- Split the data. Training, validation and a held-out test set the model never sees.
- Train. Pick full or parameter-efficient fine-tuning, a few passes over the data (epochs) and a learning rate. If validation loss rises while training loss falls, the model is memorising.
- Evaluate. Compare the test score with the baseline, and check general skills and safety too.
- Deploy and monitor. Add real failures to the next round.
What are SFT, RLHF and DPO?
SFT, supervised fine-tuning, trains a model on pairs of an input and the ideal output; RLHF and DPO are preference tuning, which learns from people's judgements about which answer is better. Most projects use SFT, and preference tuning usually comes after it.
Supervised fine-tuning (SFT). The pairs can be a customer question and the best reply, or an invoice and its extracted fields. "Supervised" means every example carries the right answer, written or checked by a person. Instruction tuning is SFT on a broad mix of tasks; it turns a base model that only continues text into an assistant that follows requests.
RLHF, reinforcement learning from human feedback, was laid out in the InstructGPT paper by Ouyang and colleagues (2022). Labelers wrote demonstrations for SFT of GPT-3, then ranked several outputs per prompt; a reward model learned the rankings, and reinforcement learning tuned the model against it. Raters preferred the 1.3-billion-parameter InstructGPT to the 175-billion-parameter GPT-3.
DPO, direct preference optimization (Rafailov and colleagues, 2023), reaches the same goal without a reward model or reinforcement learning, training directly on pairs of a preferred and a rejected answer.
What is LoRA, and what is parameter-efficient fine-tuning?
Parameter-efficient fine-tuning (PEFT) freezes almost all of a model's weights and trains a small number of new ones; LoRA, short for low-rank adaptation, is its best-known method. Full fine-tuning, by contrast, updates every weight and produces a full model copy for every task.
LoRA, described by Hu and colleagues in the LoRA paper (2021), keeps the pretrained weights frozen and trains only pairs of small matrices added to each layer of the transformer, the network design LLMs are built on. "Low-rank" means the change to each large weight table is written as two thin tables whose product approximates it.
Against full fine-tuning of the 175-billion-parameter GPT-3, the authors report 10,000 times fewer trainable parameters and three times less GPU memory, with quality on par or better on the models they tested, and no slower answers once the matrices are merged into the weights. QLoRA (Dettmers and colleagues, 2023) stores the frozen base in 4-bit precision, which was enough to fine-tune a 65-billion-parameter model on a single 48 GB GPU.
One base model can then serve several tasks, each with its own small adapter. LoRA has nothing to do with LoRa, the long-range radio technology.
Fine-tuning vs RAG vs prompt engineering: what is the difference?
Prompt engineering changes the instructions you send, RAG changes the information the model sees at question time and fine-tuning changes the model itself. They solve different problems and are often combined.
| Question | Prompt engineering | RAG | Fine-tuning |
|---|---|---|---|
| What changes? | The instructions | The context per question | The model's weights |
| Data needed | A few examples | A document index | Curated input-output pairs |
| Effort to start | Low | Medium | High |
| Fresh facts | Only if pasted in | Yes, re-index the source | No, needs retraining |
| Shows its sources | No | Yes | No |
| Best for | Prototypes, simple tasks | Knowledge that changes | Format, tone, narrow tasks |
In RAG, retrieval-augmented generation, a retriever finds the passages most relevant to a question in an index of documents, the passages go into the prompt and the model answers from them. When the documents are web pages, most of the work is preparing them, which we cover in web scraping for RAG.
For knowledge, the two have been compared directly: Ovadia and colleagues (2023) found that RAG consistently outperformed unsupervised fine-tuning, both for facts the model had met in training and for entirely new ones. They also combine well, as a fine-tuned model can answer in the house format from passages RAG retrieves.
What is fine-tuning good at, and what is it not?
Fine-tuning is good at teaching a model how to respond, and weak at teaching it what is true right now. Its strengths:
- A consistent output format. The same JSON fields every time, or a fixed reply structure.
- Tone and house style. The way your support team or brand writes.
- Narrow classification and extraction. Your ticket categories, your product taxonomy, the fields of one document type.
- Shorter prompts, smaller models. Instructions move into the weights, and a small tuned model can take over one task from a large one.
Its weak spots are changing facts such as prices and policies, showing where an answer came from, and vague tasks: if your reviewers disagree on the labels, the model learns the disagreement.
Even stable new facts are a problem. In a controlled study, Gekhman and colleagues found that examples carrying facts the model did not already know were learned much more slowly, and once learned, increased its tendency to hallucinate, to state false things with confidence. Models gain factual knowledge mostly in pretraining; fine-tuning teaches them to use it.
What data do you need for fine-tuning?
You need examples that look like the real inputs the model will receive, each paired with an output you would be glad to ship; consistency matters more than volume. There is no universal number, so start with a set you can check by hand, measure it and add examples where the model fails.
Examples usually come from four places:
- Your own records. Past replies, reviewed documents, categorised products, used only where you have the right.
- Human labeling. People write or check the correct output for each input; data labeling explains how such projects run and how agreement between labelers is measured.
- Synthetic examples. A larger model writes them on demand. They copy the generator's mistakes and bias, so people should review them and the test set should stay real.
- Public web text. Product descriptions, documentation and other domain text in the languages the model will serve.
Check the licence of the base model and the terms of every data source before training. If the examples contain personal data, training is processing under the GDPR in the EU and KVKK in Türkiye, and a tuned model can repeat what it memorised. The European Data Protection Board says in Opinion 28/2024 that whether a model trained on personal data is anonymous must be assessed case by case. The simplest protection is to leave names, contact details and IDs out.
Where does public web data come in?
Domain examples often start as public web pages, such as product descriptions for a categorisation model or job ads for a skills tagger, in every language the model will serve. Many sites change language, currency and assortment by the visitor's country, so collecting from one location gives a one-sided dataset. A Residential Proxy sends requests through home connections in the country and city you choose, so you collect the version local visitors see. Collect public pages only, follow robots.txt and each site's terms, keep request rates low, use an official API where one exists and check whether the text may be used for training.
Teams that would rather not run collection themselves can ask our sales team about managed page extraction through our Web Scraping API.
What are the risks of fine-tuning?
The main risk is ending up with a model that memorises instead of generalising, loses skills it had, or looks better on your tests than in real use; a separate test set, re-run after every round, catches most of these problems.
- Overfitting. The model learns the training examples by heart and does worse on new inputs.
- Catastrophic forgetting. Training on a new task can erase earlier skills, a known problem of neural networks that Kirkpatrick and colleagues tackled in 2017.
- Weaker safety behaviour. Qi and colleagues (2023) found that fine-tuning even on harmless, commonly used datasets can degrade a model's safety training.
- Evaluation gaps. If test examples resemble or leak from the training set, scores flatter the model.
- Dependence on the base model. A tuned model lasts only as long as its base model is offered.
Where can you fine-tune a model?
You can fine-tune through a vendor's hosted tuning service or by training an open-weight model on your own or rented hardware, and vendor offers change over time.
On a hosted service you upload training data, the vendor trains and serves the tuned model, and the weights stay with the vendor. Google Cloud's tuning documentation lists supervised, preference and reinforcement-learning fine-tuning for some Gemini models.
Offers also close. OpenAI announced in May 2026 that it is winding down self-serve fine-tuning: active customers can no longer create new jobs from 6 January 2027, according to its deprecations page, and existing tuned models keep answering until their base models are retired.
With open-weight models you download weights published under a licence, tune them with LoRA or QLoRA, for example through the open-source PEFT library, and keep the result, but you also run the hardware and serving.
Where is fine-tuning used?
Fine-tuning is used wherever one task repeats thousands of times in the same shape. Support assistants learn the house style, routing models learn a company's ticket categories, and document pipelines pull the same fields from invoices into a fixed JSON layout. Online shops tag products with their own taxonomy, and developers tune small models to run one job on a phone or laptop.
Common mistakes
- Fine-tuning first. A clear prompt with examples, or RAG, often solves the problem with less effort.
- Teaching facts that change. Prices, policies and news belong in retrieval.
- No held-out test set. Without one, every training run looks like progress.
- Volume over consistency. Thousands of unchecked, contradictory examples teach the contradictions.
- Personal data in the training set. A tuned model can repeat it.
- Testing only the target task. Check general skills and safety behaviour too.
- Throwing away the data. Base models are retired, and you will need to tune again.
Decision guide
| Need | Use |
|---|---|
| A first prototype | Prompt engineering |
| Answers from your current documents | RAG |
| Prices, stock, news or policies | RAG or a live lookup, not fine-tuning |
| The same output format every time | SFT, often with LoRA, after a prompt baseline |
| Your house tone in replies | SFT on reviewed past replies |
| Answers people prefer | Preference tuning (DPO or RLHF) |
| One narrow task at lower cost | A fine-tuned smaller model, tested against a large one |
| Limited GPU memory | LoRA or QLoRA, not full fine-tuning |
Frequently asked questions
Should I use fine-tuning or RAG?
Use RAG when the model lacks knowledge and fine-tuning when it lacks a behaviour. RAG suits facts that change and documents you must cite; fine-tuning suits formats, tone and narrow, repeated tasks. Many systems use both.
What is LoRA fine-tuning?
LoRA fine-tuning freezes the original model and trains small added matrices in each layer that capture the change the task needs. It trains far fewer parameters and needs much less GPU memory than full fine-tuning, and the result is a small adapter that can be swapped per task or merged into the model.
How much data do I need to fine-tune an LLM?
There is no fixed number; the examples must be correct, consistent and close to real inputs. The LIMA study shaped a general assistant with 1,000 curated examples. For a narrow task, start with a set you can check by hand and add examples where the held-out test still fails.
Can I fine-tune ChatGPT or other hosted models?
Only through a vendor's developer platform and only for the models the vendor opens to tuning, not in the chat apps. Google Cloud offers tuning for some Gemini models, while OpenAI stops new self-serve fine-tuning jobs for active customers on 6 January 2027. Open-weight models are the alternative when you need to keep the tuned weights.
What is the difference between fine-tuning and pretraining?
Pretraining teaches a model language and general knowledge from scratch on a very large corpus; fine-tuning adjusts an already pretrained model with a small, focused dataset. Pretraining takes the resources of a model builder, while fine-tuning is within reach of an ordinary team.
Does fine-tuning teach the model new facts?
Only poorly, and at a cost. Models learn new facts from fine-tuning much more slowly than familiar ones and hallucinate more once they have learned them. Put facts in documents the model retrieves, and use fine-tuning for the way it answers.
Summary
Fine-tuning continues the training of an existing model on a small set of examples so that it learns a behaviour: a format, a tone, a narrow task. SFT teaches from ideal answers, preference tuning from people's choices, and LoRA makes it fit on modest hardware. It does not reliably add fresh facts, which is RAG's job. Start with a prompt, add retrieval for knowledge, fine-tune for behaviour, and build the dataset from sources you may use, in the languages and countries your model will serve.




