How Much Data Do You Need to Fine-Tune an LLM?

There is no single number, and anyone who gives you one without asking about the task is guessing. What decides the volume is how narrow the task is, how consistent the right answers are, and how far the base model already gets on its own. This post sets out how to size a training set without overspending on labeling.

Why the task matters more than the model

Fine-tuning teaches a model a behavior it already half knows. A model that can already read an email and roughly sort it needs relatively few examples to learn your 40 queues. A model asked to write in a legal house style it has never seen, across many document types, needs far more.

Three questions predict the size of the job:

How many distinct outputs are there? Ten labels is a small target. Free-form writing in a new style is a large one.
Do two careful people agree on the right answer? If your own reviewers disagree a fifth of the time, more data of the same kind will not fix it. Fix the guidelines first.
How good is the base model with a strong prompt? If prompting gets you most of the way, a small set may close the gap. If prompting is far off, either the task is hard or it is the wrong tool (often the gap is missing facts, which is a retrieval problem).

Rough sizes by task type

These ranges are working rules from how fine-tuning projects are usually scoped, not guarantees. Treat them as a starting point to test, not a target to hit.

Task	Typical starting set	What usually limits results
Classification or routing with stable labels	Several hundred up to roughly two thousand examples, spread across every label	Rare labels with too few examples
Field extraction into a fixed schema	A few hundred documents with checked outputs	Unusual layouts missing from the set
House style or fixed output format	Several hundred to a few thousand pairs	Inconsistent “ideal” outputs
Broad assistant behavior across many tasks	Thousands to tens of thousands	Coverage of the task mix
Preference tuning (DPO)	Hundreds to thousands of ranked pairs, after an SFT pass	Reviewer agreement on which answer is better

Note the second column always says “checked”. One mislabeled example in fifty teaches the model to be wrong one time in fifty.

Start with the evaluation set, not the training set

Before labeling anything for training, write the test. Take real inputs from production (with personal details removed), agree the correct output for each, and decide how you will score them. Keep this set away from training, for good.

The evaluation set does three jobs. It measures your prompting baseline, so you know whether fine-tuning is needed at all. It tells you when to stop adding training data, because scores flatten. And it catches a tuned model that looks fluent but has got worse at the cases that matter.

A useful rule is to hold back at least a fifth of whatever you collect. With a small set this feels wasteful. It is not: a tuned model you cannot measure is a liability.

Grow the set in rounds

The cheapest way to find the right volume is to buy it in steps:

Label a first set (for a narrow task, a few hundred examples).
Train with a parameter-efficient method such as LoRA, which updates a small set of added weights instead of the whole model and is cheaper to run.
Score against the evaluation set and read the failures.
Label the next set aimed at those failures, not at random.

Each round tells you whether more data is still paying. Targeted rounds aimed at the failure types usually beat a larger random sample.

How data volume turns into a training bill

Platforms bill training by tokens processed, not by example. Google Vertex AI defines training tokens as dataset tokens multiplied by epochs, and Together AI’s formula adds validation tokens for each evaluation it runs during the job. So the bill depends on example length as much as count.

A worked example at published rates (read 7 Nov 2026): 2,000 examples averaging 600 tokens is 1.2 million tokens. Three epochs makes 3.6 million training tokens. At Fireworks AI’s LoRA SFT rate of $0.50 per million for models up to 16B, that is $1.80. Together AI applies a minimum charge per job ($4.00 for many smaller models), so a run that small would cost the minimum there.

The run is rarely the expensive part. Paying people to write or check 2,000 examples, and to build the evaluation set, costs far more than any of the numbers above. That is why sizing the set carefully matters: every unnecessary example is labeling time, not GPU time.

When more data will not help

Adding examples does not fix:

Missing facts. If the model gets answers wrong because it has never seen your product catalog, retrieval is the fix. Fine-tuning on Q&A pairs about the catalog tends to produce confident errors as soon as the catalog changes.
An unclear target. If reviewers disagree, the model learns the disagreement.
A base model that is too small. Past a point, a larger base model with fewer examples beats a small one with many.

If you are weighing those trade-offs and want help from people who do this work, our LLM fine-tuning services overview lists who provides each part of a project. For the specific questions your data volume raises, try the project scoper.

A sizing checklist

Can you write 20 perfect examples by hand? If not, the task is not defined yet.
Do two reviewers agree on at least nine in ten?
Is the evaluation set built and held out?
Have you scored a prompting baseline on it?
Does the first training set cover every label or case type?

Size the set to the task you can measure, and let the evaluation scores, not a round number, tell you when it is big enough.

Sources for this post

How much data do you need to fine-tune an LLM?