Self-Hosting an LLM: When Does It Actually Pay Off?

Open-weight models have almost caught up with the closed ones, so running your own must be the obvious move by now. It sounds so reasonable that few people check it. What happens when you actually run the numbers?

They come out badly for most companies. Against frontier APIs, self-hosting starts winning somewhere north of 100 million tokens a month. Against the cheap open-model APIs, the break-even sits in the billions. And that is before anyone has been paid to keep the thing running.

That is only one of the assumptions bundled into "let me just run an open-source model on my own server". Below we take the rest apart too: what the licences actually say, where the quality gap really is, and the arithmetic itself. This is one of the four decisions we mapped in our guide for non-technical leaders, and probably the one made with the most emotion attached.

Is it "open source" or "open weight"?

Usually the second one. Open weight means the trained parameters are published. Genuinely open source would mean the training data, the code, and unrestricted freedom to use it. Most of what gets marketed as open source is only the first kind; nobody is handing over the training data.

This is not a semantic quibble, it is contractual risk. Llama's community licence permits commercial use but forces companies above 700 million monthly active users into a separate negotiation, and it carries behavioural use restrictions on top. Almost no company approaches that user threshold, but the other restrictions are still in there, and "we thought it was open source" is not a defence that survives an audit.

If you want unrestricted use, the thing to check is the licence tag. Apache 2.0 and MIT models carry no user ceiling and no royalty obligation. Parts of the Qwen family, the DeepSeek models, and OpenAI's own open-weight gpt-oss releases fall into that group. Ask legal this one question before you sign anything; most of the unpleasant surprises start here.

Has the quality gap actually closed?

Mostly, though not entirely. Open-weight models have come within a few points of frontier closed models over the past two years, and on coding and reasoning tests the remaining difference is below the threshold most business applications would notice. The smaller of OpenAI's Apache 2.0 gpt-oss models runs in 16 GB of memory; the larger one fits on a single 80 GB GPU.

The real warning here concerns how you read benchmarks. Most of the 2026 comparison tables circulating online are not primary sources; they are content sites copying each other, and their scores contradict one another. Rather than trusting a single table, build a small test set from your own data. In our experience the actual gap between two models, measured on your specific task, often looks nothing like the published one.

Then there is quantisation. Squeezing a model into less memory by lowering its precision (the common 4-bit setting) halves the memory requirement and costs you somewhere in the 1-5% range on quality. Know what you are trading when you say you fit it on cheaper hardware.

What does running your own actually cost?

The hardware is cheaper than you expect and everything around it is more expensive. A rented H100 runs roughly $1.50 to $3.50 an hour, so about $1,500 to $2,500 a month if you keep it up around the clock. Buying one starts at about $25,000. A more modest setup, a rented server with a 20 GB professional card, can be found for a couple of hundred euros a month.

Memory requirements by model size go roughly like this: an 8-billion-parameter model needs about 16 GB at full precision and 5-6 GB when quantised to 4-bit. A 70-billion-parameter model needs around 140 GB at full precision, meaning two high-end cards, and fits into 40-46 GB on a single card once quantised. So "we'll run a 70B" is not automatically a tens-of-thousands-of-dollars sentence, but it is a serious one.

Electricity varies enough by region to change the answer. At around $0.12 per kilowatt-hour, the power bill for a dense eight-card setup lands in the few-hundred-dollars-a-month range. In markets where electricity costs two to three times that, published analyses put the effect at pushing the break-even threshold up by something like 40-60%. Where your rack sits genuinely matters to the arithmetic.

Above what monthly volume does self-hosting win?

The calculation is one line: divide your monthly GPU cost by the blended per-token price you pay an API. Compared against frontier models, published analyses converge on a threshold somewhere in the 100 to 250 million tokens per month band. In daily terms: 3 to 8 million tokens, which is a genuinely high-volume application.

Change the comparison from a frontier model to a cheap open-model API and the picture inverts completely. Against providers charging $0.15 to $0.50 per million tokens, self-hosting needs billions of tokens a month before it wins. Which means the decision to use an open model and the decision to host it yourself are two entirely separate decisions. Saying yes to the first and no to the second is the smartest combination available to most companies.

And there is a line item nobody discusses: utilisation. The GPU bill runs whether you use it or not. Drop from full capacity to a quarter and your cost per token roughly quadruples. In a typical enterprise application, busy during office hours and idle at night, the real cost lands well above what the spreadsheet said.

Does your language change the math?

If you operate mostly in English, no. If you do not, slightly in favour of self-hosting, and this is a factor missing from nearly every published analysis. Tokenizers trained on English-heavy data split text in morphologically rich languages into far more pieces than necessary.

Academic measurements show language-specific tokenizers producing 2 to 3 times fewer tokens than multilingual alternatives on agglutinative languages. You see the consequence directly on the invoice: a company sending non-English text to an API pays meaningfully more per unit of content than an English-language competitor doing the same work. That pulls the break-even threshold down.

Do not push the argument too far, though. Pulling a threshold down by thirty percent turns 200 million tokens a month into 140 million. A company using 5 million tokens a month is still nowhere near it.

The server is $200 a month. Who keeps it running?

Here is where the hidden cost lives. Every self-hosted setup takes 2 to 4 weeks of focused senior engineering to stand up, then permanently consumes 10 to 20% of a senior engineer's time. Budget another week for every major model upgrade.

Put a salary against that. Twenty percent of a senior infrastructure or ML engineer's time is, in most markets, somewhere between $2,000 and $6,000 a month in fully loaded cost. That single line item exceeds the server rental several times over. The margin of error in "I rented the server, we're done" is larger than the rental itself.

Open models do not remove cost, they relocate it: the licence fee disappears and engineering time, infrastructure and continuous maintenance take its place.

Add monitoring and alerting to that list. Everything that arrives built in with an API (usage tracking, error monitoring, redundancy) is something you now write yourself.

So when does it genuinely make sense?

Four situations: your data legally cannot leave a jurisdiction, your system has to run on a network with no internet connection, your volume is genuinely enormous, or the customised model you need is not offered by any API. Most other justifications are emotional.

The first one carries the most weight in practice. Regulated industries and public-sector work often come with data residency requirements that no amount of cost analysis can override, and European data protection guidance has pointed to on-premise inference as the strongest available control for language model deployments. When compliance is the driver, the break-even calculation is not the deciding input.

The second is concrete in defence, government and critical infrastructure, where products exist specifically to run inside closed networks. The third is arithmetic: above roughly 50 million tokens a day, the economics turn toward self-hosting on their own.

And there is a third path most companies skip entirely: neither building your own rack nor calling a foreign API. A GPU server hosted inside your own jurisdiction, or a hybrid setup where a small model handles sensitive data in-house while everything else goes to an API, resolves the tension in most cases. Deciding which data gets processed where matters more than deciding which model you use.

Frequently asked questions

Does self-hosting solve my data protection obligations?

No. It solves the cross-border transfer question, but transparency notices, data minimisation, retention limits and lawful basis all still apply. Where the server sits is one component of compliance, not the whole of it.

Are open models free?

You do not pay for the weights. You pay to run them. The licence fee is replaced by GPUs, electricity, engineering time and maintenance.

Which open model is best for my language?

On general capability, large multilingual models tend to lead even on non-English evaluations. Language-specific models often do well on narrow tasks such as grammar correction and summarisation. Counter-intuitively, published benchmarks have found language-specific instruction-tuned models trailing general-purpose multilingual ones on overall capability, so test on your own task before deciding. Smaller models also fabricate more readily, so plan the layers that contain hallucination from the start.

Can a small model on my own server be worth it?

Yes, particularly for a narrow, repetitive task. A 2 to 8 billion parameter model runs on modest hardware and can be entirely sufficient at one job. If you are aiming for a general-purpose assistant, lower your expectations. Before narrowing a small model to one job, check our decision tree for RAG, fine-tuning and prompting.

What about buying rather than renting GPUs?

Buying only makes sense with predictable, sustained high utilisation. With variable load you are paying for idle silicon, and depreciation on AI hardware is unforgiving.

So what should you actually do?

• Separate the two decisions first: using an open model is one choice, hosting it yourself is another. For most companies the right combination is yes to the first, no to the second.
• Measure your monthly token volume before deciding anything. Below roughly 100 million tokens a month, there is no financial case for self-hosting.
• Put engineering time in the model. A fifth of one senior engineer costs more than the server rental.
• Read the licence, not the label: confirm whether the model you plan to use is Apache 2.0, MIT, or a restricted community licence before you commit.
• If your reason is legal rather than financial, close the cost debate. When data cannot leave the jurisdiction, the arithmetic is not offering you a choice; look at in-jurisdiction hosting or a hybrid architecture instead.

Most decisions in this area get made on a feeling rather than an evaluation, because running your own model on your own server feels solid. The feeling is not wrong, but the invoice does not care about it. Write down which data has to live where, measure your monthly volume, and the decision usually announces itself.