The wrong question
A few weeks ago I attended a presentation by a former colleague who helped build a self-hosted AI platform for a software consulting company.
The technical details were interesting. The GPUs. The models. The vLLM optimizations.
What surprised me was what happened afterward.
The first question was about Claude pricing. Fair enough. The second question was also about Claude pricing.
Then came questions about electricity costs. Token costs. Subscription costs. Cost per user. Cost per request. The more I listened, the more bizarre it became.
Dependency was not discussed. Control was not discussed.
A room full of people who build with AI every day was discussing strategic dependency as if it were a cellphone plan.
The models change. The pricing changes. The limits change. Sometimes the quality changes, quietly, without announcements, not always for the better. And more and more companies are building important workflows around this moving target.
That is not a roadmap.
That is a bet.
The economics are not on your side
In one sense the current situation resembles the cloud vs on-prem debate 15 years ago. In 2010 Microsoft published the well-known white paper about the economics of the cloud. One of the main points is that most companies are bad at utilization. They buy for peaks, then spend most of the time running below capacity. On the other hand, the cloud providers can average out the utilization across thousands of customers, industries, time zones, and workload types. Same logic applies to the model layer of the AI infrastructure.
If you are trying to run a frontier-class model, you will not beat external AI providers on raw cost per token. They are better at utilization, they build where power, cooling, and tax are cheapest and they operate at a scale you cannot match, some design their own chips, others are building GPU fleets measured in hundreds of thousands or even millions.
If the game is “run the best frontier model cheaper than the companies building those models,” you lose. But that is not the only game.
Is the AI system just the model or is it more?
An AI system is not just the model layer. This is where the comparison with cloud compute breaks down. An enterprise AI system around the model layer has access to documents, tickets, contracts, policies, source code, HR data, ERP systems, customer history, support conversations, audit logs, internal procedures and so on. The details heavily depend on your business - what you do and how you do it. Models are not the system. They are components inside the system.
So the question is not simply: “Where should we run the model?” The question is: “Who controls the system that is allowed to read, interpret, summarize, classify, recommend, and sometimes act on the knowledge of the organization?”
What are you actually doing with AI?
Before you can think about the regulatory, compliance and sovereignty concerns you have to answer a more basic question: what are you actually doing with AI?
A public-facing chatbot needs high volume, predictable behavior, strong guardrails, and tightly limited access to internal systems. A legal contract-review tool may have lower request volume, but much lower tolerance for mistakes. Its limits are less about system access and more about accuracy, traceability, and clearly separating model suggestion from lawyer decision. A coding agent is different again. It may work with sensitive source code, internal documentation, tickets, logs, and development tools. It needs deep context and broad tool access, not public-scale throughput.
These are not the same system.
For some use cases you might decide that a small self-hosted model supported by RAG gives a much more consistent and valuable response than a general-purpose external model. The runtime cost of such a setup may be surprisingly small. For another use case you might decide to use external providers. Different use cases need different systems.
This is why the model-price discussion is so shallow.
Safe is not the same as allowed
Are OpenAI, Anthropic, Microsoft, Google, or Amazon technically competent? Sure. Are you allowed to send all that data there? What do the internal policies say? The audit requirements? The clients? The contracts? The regulator? How about sovereignty concerns around jurisdiction and control? Where is the data processed? What happens if political, legal or economic pressure changes the provider’s behavior?
A system can be technically secure and still not be allowed for a specific use case.
Maybe you have a serious enterprise agreement. Maybe it gives you SSO, audit logs, retention controls, approved processing regions, support commitments, usage limits, spend controls, security documentation. And you managed to secure all that at a good price. Good. Then the only thing left to add is that it does not turn the provider into infrastructure you control. That may be acceptable. But it is not the same question as comparing subscription prices.
The price is huge and you can’t do it
First: billions going into companies like OpenAI are not simply the cost of answering prompts and agents. Much of that money goes into data centers, research, and training new models. Inference is not free, but training and inference are very different beasts.
Furthermore, self-hosted AI does not have to mean buying a container of H200s and blindly stuffing them into server racks. You can run open models on AWS, Azure, Google Cloud, Hetzner or any number of GPU providers. You can rent dedicated machines. You can use managed inference. You can build the system in the cloud first, understand the workload, learn what models you need, measure usage, and only later decide whether buying hardware makes sense, and for which use cases. That does not give you perfect control, but it does give you control over a different layer of the system. You rent the compute, but you control the model choice, the serving stack, the data flow, the access rules, the integration points, the logging, the evaluation pipeline, and the migration path.
That control is not free. The compute can be rented. The model can be open. But someone still has to operate the system, secure it, evaluate it, support it and explain it to auditors or clients.
And if you go all the way and decide to buy the hardware, true, it is not pocket money, but it is not necessarily absurd either.
The example from the talk was a self-hosted system used heavily for coding assistance, serving a company-wide AI platform. Developers described the experience as roughly in the Claude Sonnet range. By my napkin math, counting only the heavy users, the hardware lands around a high-end laptop or workstation per developer. Amortized over three years, that’s low hundreds per month per developer, before counting the people and process needed to run it properly.
Is that cheap? No. But compared to salaries, laptops, failed projects, lost clients and missing AI capability, it is not insane either.
So should every company do this?
No. Not even close. If you are a solo founder, a small business, or a startup working mostly with non-sensitive data, self-hosting AI probably makes no sense. Use the tools. Pay for the subscriptions. Move fast. Spend your time building the business, not operating model infrastructure. But that is not the case I am talking about.
The case I am talking about is a mature organization starting to build real capabilities around AI. A bank. A government agency. A healthcare company. A legal-tech company. A critical infrastructure operator. A software agency serving regulated clients. A larger enterprise whose internal knowledge, customer data, source code, contracts, risk processes, or operational workflows are becoming connected to AI systems.
The worst time to understand your AI options is when your provider becomes too expensive, unavailable, restricted, or forbidden by a client. By then you are not planning. You are migrating under pressure, while losing business, paying fines, and explaining to a board why nobody saw this coming.
Strategic optionality means having a fallback. It means being able to switch providers, run a model yourself, keep working during an outage, serve clients who prohibit external AI, negotiate from a stronger position, and avoid redesigning your entire workflow under pressure.
Every serious company should understand what it is choosing not to own.