Open Weights or a Hosted API: What Each Choice Commits You To
Running a model yourself gives control and responsibility. Using a hosted one gives speed and dependence. Both carry costs that show up later.
On this page
Cost comparisons between models often stop at the price of a single call. One team chose the option with the lowest per-call figure and later found that the real comparison included the engineers who would be on call, a serving setup that needed steady attention, and a model update schedule it no longer controlled. Neither path is free. Running a model yourself and calling one through a hosted service move costs to different places, and the right choice depends on which costs a team is better placed to carry.
What is actually being chosen
The decision concerns who operates the model and who controls its lifecycle. With a hosted model, the provider runs inference, decides when the model changes or retires, and bills per use. With an open-weights model, the weights are obtained under a license and run on hardware the team controls or rents. "Open weights" describes the availability of the trained parameters. It is not the same as open source, and open-weights licenses differ in what they allow, including commercial use, redistribution and modification. Read the license before planning around it.
What hosting buys
A hosted service removes the infrastructure work. There are no accelerators to buy, no serving software to maintain, no capacity planning for peak load and no security patching of the inference stack. New models become available quickly, and scaling is handled by someone else. For a team without operations experience, or a product whose demand is uncertain, this is often the sensible starting point.
The trade is dependence. Prices can change, availability can drop during an incident, models are retired on published schedules, and the terms governing data are set by the provider. Those terms are negotiable at some scales and fixed at others. Product behavior can change when a model is updated, even when the team's own code has not changed.
What running it yourself costs
Self-hosting brings its own bill. Hardware must be sized for peak load, not average load, and idle capacity is a cost in its own right. Someone must monitor the serving system, handle failures, roll out new weights and keep the inference software current. Latency depends on configuration, and performance tuning is a skill that takes time to build. The cost per call can fall at high and steady volume, but it rises at low or uneven volume, where fixed costs dominate.
The benefits are real. The team controls the version, so behavior changes only when it decides to change them. Data can stay inside a boundary the team governs, which may matter for regulated work. The setup can be tuned for the workload at hand. These gains justify the operational cost only when someone is clearly responsible for them.
Between the two extremes sit managed deployments of open-weights models, in which a provider runs the weights in a dedicated environment for one customer. These can offer more control over versions and data location than a shared service, with less operational work than a self-run cluster. The terms vary widely, so read the contract for retention, for access to logs and for how and when updates are scheduled.
Privacy, data location and control
Both paths move data somewhere. A hosted call sends the input to the provider's infrastructure under contractual terms. A self-hosted system keeps the data inside the team's boundary, but logs, backups and monitoring now hold it, and the team carries the security obligations. Neither option is automatically safer. Write down exactly what data leaves the environment in each design, who can read the logs, how long they are kept and what the contract says, then compare that list with the obligations the work carries.
The list needs an owner and a review date. A provider's data policy at signup is not necessarily the policy a year later, and an internal logging change can quietly move data into a new place. Treat the list as a living control rather than a one-time check.
Switching later without rewriting everything
Most teams change models at least once, so the design should allow for it before the choice is made. Keep prompts, evaluation sets and output schemas in the team's own repository, not only in a provider's console. Place a thin layer between the application and the model call, so that swapping the backend changes one module. Avoid features that exist only in one provider's interface unless their value clearly justifies the lock-in. Run the evaluation suite against a second model from time to time, even with no plan to switch, because that keeps the exit route tested.
An exit plan is a short document with steps, owners and a rough estimate of effort. Writing it is useful even if it is never used, because it reveals which parts of the system are bound tightly to one model.
A decision exercise
Write a one-page comparison with the factors that matter for the product: cost at two volumes, the people needed to operate each option, data handling, latency needs, the update schedule the team can tolerate and the effort to exit. Weight each factor by how much harm would follow if it went wrong. Estimate the monthly cost of each option at current volume and at a volume many times larger, and note where the lines cross. Then write one sentence describing the change in circumstances that would reverse the decision. A choice with a named reversal trigger is sounder than one assumed to be permanent.
