What Does It Cost to Host a Fine-Tuned Model?

Teams budget carefully for the training run and then find that serving the tuned model is the larger, recurring line. The training job happens once or a few times. Hosting is billed every hour the model is available. Here is how the main platforms bill it, from their own pricing pages read on 7 November 2026.

The three ways serving is billed

Per token. You pay for the tokens sent and returned, as with any API model. Nothing is billed while the model is idle. This is how Google Vertex AI bills tuned Gemini models, and how OpenAI bills its fine-tuned models for existing users.

Per unit of hardware time. The tuned model runs on GPUs reserved for you, billed by the second, minute or hour. You pay while it is running, whether or not requests arrive. Fireworks AI bills deployments by the GPU-second; Together AI bills dedicated endpoints per minute by hardware.

Provisioned capacity. A fixed block of throughput reserved by the hour. AWS Bedrock publishes an hourly, no-commitment rate per model unit for some custom models.

What each platform publishes

Platform	How a tuned model is served	Published detail
Google Vertex AI	Per token	Tuned Gemini 3 and later: base model price in the global region, 1.5x the base price in other regions. Gemini 2.5 and older: base model prediction price.
Fireworks AI	Per GPU-second	A tuned model costs nothing extra to serve; the deployment is metered and training is billed separately.
Together AI	Dedicated endpoint, per minute	Rate depends on the hardware chosen; separate from the training bill.
AWS Bedrock	Per token (some models) or provisioned	Reinforcement-tuned gpt-oss-20b: $0.09 input and $0.39 output per 1M tokens. Tuned Llama 2 models: $23.50 per model unit per hour, no commitment. Storage $1.95 per model per month.
OpenAI	Per token, existing users only	gpt-4.1-mini tuned: $0.80 input and $3.20 output per 1M tokens. Platform closed to new users.

Copied from each provider’s own page on 7 Nov 2026; the Bedrock page was only partly captured. List prices, not a quote. Any of these providers could pay this site a referral fee after an introduction, and none had a say in this table.

The idle-time trap

Hardware billing is simple to state and easy to underestimate. Take the Bedrock figure above: $23.50 per model unit per hour. Kept running for a 30-day month (720 hours), one unit comes to $16,920 before a single request is counted. That is for one model, in one region.

Compare that with a training run that, at the per-token rates on our prices page, might cost a few dollars to a few hundred. Left running, the hosting line can be many times the training line, and it repeats every month.

Ways teams cut it:

Scale to zero. Ask whether the deployment can shut down when idle and how long it takes to start again.
Share a base model. If the platform supports serving several LoRA adapters over one base, many narrow tasks can share hardware.
Queue what can wait. Overnight jobs do not need always-on capacity. OpenAI, for example, publishes lower rates for queued (asynchronous) requests to its tuned models.
Pick the smallest model that passes. A model that fits on one GPU instead of four cuts hosting by about the same ratio.

When dedicated hardware is the better deal

Per-token serving is not always cheapest. If traffic is steady and high, a dedicated GPU running near full use can cost less per token than a shared endpoint. Dedicated capacity also gives predictable latency and keeps your model separate from other customers’ traffic, which some security reviews require.

To find the crossover, you need two numbers from the provider: tokens per second the model sustains on the hardware they propose, and the hourly or per-second rate. Divide one by the other at your expected daily traffic. If a provider cannot give you both, you cannot compare.

Running it yourself

If you can export the tuned weights, you can run them inside your company’s cloud or on servers you own. You then pay for GPUs, storage and the people who keep the stack running. That makes sense when data must stay inside your environment or when traffic is high enough to keep hardware busy. Check two things first: that the platform allows exporting weights, and that the base model’s license allows your use.

Questions to ask before you sign

What is the billing unit for serving, and does it run while idle?
Can the model scale to zero, and how long is a cold start?
Is the tuned-model token price the same as the base model in our region?
Can several adapters share one deployment?
Can we export the weights if we leave?

The fine-tuning prices page keeps each provider’s serving terms next to its training prices, and our LLM fine-tuning services overview shows where hosting sits in the whole project. Price the first month of serving before you price the training run.

Sources for this post

What does it cost to host a fine-tuned model?