Build, buy or fine-tune is three decisions, and most debates merge them
In March 2023 Bloomberg published a paper describing BloombergGPT, a language model with 50 billion parameters. Its training corpus combined 363 billion tokens drawn from Bloomberg's own financial data, collected over decades, with 345 billion tokens of general-purpose text. The authors reported that it outperformed existing models of similar size on financial tasks without giving up performance on general benchmarks. Bloomberg is a company for which a decision to build made sense on its face: it holds an archive that almost nobody else has, its customers pay for financial language to be understood correctly, and it has the engineers to run a training campaign.
At the other end of the spectrum, a team calls a hosted model's API with a prompt and ships it in a week. Between these two lie retrieval over the company's documents, fine-tuning a model on its examples, and hosting open-weight models on its own servers. A debate framed as "build or buy" usually blurs these into a single choice and ends with people arguing past each other, since one side is thinking about training from scratch and the other about calling an API.
The question comes apart into three decisions. One is where the model comes from: a vendor's API, open weights that you host, or a model you train. Another is how the model is adapted to your task: a prompt, retrieval, or fine-tuning. A third is who owns the system around the model, meaning the evaluation, the data pipelines, the monitoring and the governance. Each can be answered separately. A company can buy its model, adapt it with retrieval and build everything around it. Another can host open weights, fine-tune them, and buy its evaluation tooling. I will take the three in turn, because each has its own economics.
Where the model comes from is mostly a utilization question
The cost of serving a model yourself is the price of the hardware divided by the number of tokens it produces. The second number is what varies, because a GPU costs the same per hour whether it is serving requests or sitting idle. Take an illustrative node of eight GPUs rented at $3 per GPU-hour, so $24 an hour, serving a 70-billion-parameter model at an aggregate throughput of 2,500 tokens per second, which is 9 million tokens an hour. Those are placeholder figures, and your own measurements should replace them. At full utilization the cost is $24 / 9 = $2.67 per million tokens.
Utilization is the whole story. At full load the cost is $2.67 per million tokens. At half load it is $5.33, at a quarter load $10.67, and at a tenth of capacity $26.67. The hardware bill is the same in every case, and the only thing that changes is how much work is done for it.
Set these beside the list prices of a hosted vendor. As of October 2026, Anthropic lists Claude Haiku 5.5 at $0.10 per million input tokens and $0.50 per million output tokens for prompts up to 100,000 tokens, Sonnet 5.5 at $2 and $10, and Opus 5.5 at $4 and $20. A self-hosted 70B model undercuts the mid-tier API's $10 output price only while utilization stays above about a quarter, and it does not undercut the cheapest tier at any utilization. At 10% utilization, which is not unusual for an internal tool used during business hours, the self-hosted cost is above even the most expensive tier. The comparison leaves out the people: an engineer who keeps a serving stack healthy costs more than the GPUs at small scale.
Add the people and the picture changes shape. Suppose the self-hosted option needs two engineers to run, at an assumed $15,000 a month each fully loaded, and compare it with the mid-tier API at a blended $3.6 per million tokens (a 4 to 1 mix of input and output at $2 and $10). Treat all tokens alike, which understates what a node can do on input-heavy work. One node costs $17,280 a month and can move about 6.5 billion tokens in that time.
| Monthly volume | Self-hosted nodes and utilization | Self-hosted total, with staff | API total |
|---|---|---|---|
| 3 billion tokens | 1 node, 46% | $47,300 | $10,800 |
| 10 billion | 2 nodes, 77% | $64,600 | $36,000 |
| 30 billion | 5 nodes, 93% | $116,400 | $108,000 |
| 60 billion | 10 nodes, 93% | $202,800 | $216,000 |
On these assumptions, hosting breaks even somewhere around 30 to 60 billion tokens a month, or one to two billion a day. Below that, the API is cheaper by a wide margin, and the gap is largest at the volumes where many pilots live. The numbers will differ for your prices, your throughput and your staffing, and the method is to build this table with your own figures before committing to a platform team.
So the economics of hosting favor a narrow set of cases: very high and steady volume that keeps the hardware busy, data that cannot leave the company's environment for legal or contractual reasons, latency requirements that a remote API cannot meet, or a need to modify the model in ways a vendor does not allow. Outside those, buying access is cheaper and moves faster. Many companies end up with a mix: a hosted frontier model for the hard, low-volume work, and a smaller model for the high-volume tasks where the answer is routine. A router that sends each request to the cheapest model that can handle it keeps both options open. Contract terms matter as well as price. Data-processing agreements, retention limits, indemnities and the vendor's rights over your prompts and outputs are part of what is being bought, and legal review of them belongs in the comparison and not after it. The buy decision also carries the risk of depending on a provider, and the answer to that risk is portability and a tested fallback, not self-hosting by default.
Training a model from scratch sits further out. It makes sense for a company with a rare data asset and a business that depends on language the general models handle badly, which describes Bloomberg and few others. For most organizations, the money is better spent on the parts of the system that nobody else will build for them.
How to adapt it: prompt, retrieve, fine-tune
Adaptation is a ladder, and teams tend to jump to the top rung because it feels like real engineering. The order of cheapness and reversibility is prompting, then retrieval, then fine-tuning. Start at the bottom and climb only when the evaluation says the lower rung has run out.
Prompting, with a few well-chosen examples in the context, handles a surprising share of tasks and can be changed in minutes. Retrieval adds the company's own knowledge by fetching relevant documents at question time. Fine-tuning changes the model's weights using examples of the behavior you want.
The evidence on what fine-tuning is good for is consistent. Oded Ovadia and colleagues compared unsupervised fine-tuning with retrieval as ways to inject knowledge into a model and found that fine-tuning gave modest gains while retrieval consistently did better, both for facts the model had seen in training and for entirely new ones. Zorik Gekhman and colleagues, at EMNLP 2024, found that examples introducing new facts are learned much more slowly than examples consistent with what the model already knows, and that once they are learned they raise the model's tendency to hallucinate roughly in proportion. Their conclusion is that a model acquires most of its factual knowledge in pretraining and that fine-tuning mainly teaches it to use that knowledge better.
The practical reading is that fine-tuning is for behavior and not for facts. It suits a consistent output format, a house style, a classification scheme with many labels, the vocabulary of a specialized field, and the shrinking of a task onto a smaller, cheaper model. If the goal is to have the model know the contents of the company's contracts or manuals, retrieval is the tool, and fine-tuning on those documents is likely to disappoint.
The compute for fine-tuning is no longer the obstacle. LoRA, by Edward Hu and colleagues, reduced the number of trainable parameters by about 10,000 times and GPU memory by about 3 times compared with fully fine-tuning GPT-3 175B, with no added inference latency. QLoRA, by Tim Dettmers and colleagues, made it possible to fine-tune a 65-billion-parameter model on a single 48-GB GPU, and their Guanaco models took 24 hours of training on one GPU to reach 99.3% of ChatGPT's level on one benchmark. What remains expensive is the labeled data, the evaluation and the discipline to rerun both whenever the base model changes.
The tier matters more than the technique
A decision that often saves more than any fine-tuning project is choosing the right model tier. Take a classification job over 20 million documents a month, each about 1,000 tokens with a 50-token answer. On the prices above, a mid-tier model costs $0.0025 per document, which comes to $50,000 a month. The cheapest tier costs $0.000125 per document, or $2,500 a month, one twentieth as much. Whether the cheap model is accurate enough is an empirical question for the evaluation suite, and the answer is sometimes yes with a good prompt and sometimes yes only after fine-tuning. A team that fine-tunes a mid-tier model for months without testing the cheapest tier first has skipped the step with the largest effect.
The general lesson is to price the whole ladder before climbing it. Run the evaluation on three or four models of different cost, with and without retrieval, and plot accuracy against cost per task. The frontier of that plot is the menu, and fine-tuning is a way to move a cheaper model up toward a more expensive one.
The third decision is always yours
Buying a model does not mean buying a system. Whichever route you take for the first two decisions, the company owns the evaluation suite that defines acceptable behavior, the pipelines that bring in and clean its data, the monitoring that notices drift, the access controls that decide who sees what, and the governance that answers to regulators and customers. These are not extras. They are where most of the long-term cost sits, and they are the same whether the model is rented or trained.
This is also where the advantage that survives vendor changes accumulates. A vendor's model can be swapped for the next one, but the evaluation set, the cleaned data and the integration into the workflow are specific to the company and do not transfer to a competitor. When people say they want to own their AI capability, this is the part worth owning.
A decision rule that can be audited
A short procedure keeps the three decisions apart. For each AI use case, ask four questions. Does this capability differentiate us in the eyes of customers, or is it a commodity function? Do we hold data that others lack, and does it improve results? How stable is the task, meaning will today's solution still be adequate in a year? What constraints apply on data location, latency or regulation?
| Situation | Default | Revisit when |
|---|---|---|
| Commodity function, no special data | Buy API access, prompt | A cheaper tier passes your evaluation |
| Needs the company's documents or records | Buy, add retrieval | Retrieval quality stalls despite tuning |
| High volume of a stable, narrow task | Fine-tune a smaller model | The base model changes or the task shifts |
| Data cannot leave your environment, or volume is high and steady | Host open weights | Utilization falls below the break-even or a vendor offers private deployment |
| Rare data asset, language the general models handle poorly | Consider training | Almost never, and only with a board-level case |
Write it down and set a date
Each use case should have a one-page decision record that states the choice made for each of the three decisions, the evidence behind it, and the triggers that would reopen it. The evidence section is the part that gets skipped. It should hold the evaluation scores of the options that were tested, with their cost per task, the volume assumption behind the cost comparison, and the name of the person who owns the evaluation set. Triggers should be concrete numbers: a competing tier that scores within two points at half the price, a monthly volume that crosses the hosting break-even, a regulation that changes where data may be processed. The record should carry a review date tied to the model release cycle, since the right answer has a shelf life measured in months. A decision that nobody can remember making, and that nobody will revisit, is the one that quietly turns into the most expensive line in the budget.
Sources
- Wu, S., Irsoy, O., Lu, S., et al. (2023). BloombergGPT: A Large Language Model for Finance. arXiv:2303.17564.
- Anthropic. Pricing, Claude developer documentation (list prices as of October 2026). platform.claude.com/docs/en/about-claude/pricing.
- Ovadia, O., Brief, M., Mishaeli, M., Elisha, O. (2023). Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs. arXiv:2312.05934.
- Gekhman, Z., Yona, G., Aharoni, R., et al. (2024). Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? EMNLP 2024. arXiv:2405.05904.
- Hu, E. J., Shen, Y., Wallis, P., et al. (2021). LoRA: Low-Rank Adaptation of Large Language Models. arXiv:2106.09685.
- Dettmers, T., Pagnoni, A., Holtzman, A., Zettlemoyer, L. (2023). QLoRA: Efficient Finetuning of Quantized LLMs. NeurIPS 2023. arXiv:2305.14314.
- Illustrative serving-cost model: node price of $24 per hour (eight GPUs at $3) and 2,500 tokens per second of aggregate throughput, assumptions stated in the text.