Model Supply Chains: Trusting Weights You Did Not Train

Every time you download a model, you are fetching a set of files that a program will load, parse and run. The weights attract most of the attention, but the surrounding files and the software that reads them decide what actually executes on your machine. Treating a model download as a simple data transfer is the mistake this article is meant to correct.

What a model artifact contains

A model release is usually a bundle. It includes weights, which are large arrays of numbers, and a configuration file describing the architecture. It often includes a tokenizer that converts text into numbers, and sometimes code that the loading library runs to build the model. The weights are data. The surrounding files are not all data, and the loader decides how much of them is treated as instructions.

Artifacts are large, widely copied and often renamed. A file that looks like a well-known model may be a repackaged copy with changes, and the name alone says little about where it came from. Treat the bundle as a set of files whose provenance must be established, not as a single trusted object.

File formats and what they can run

Some serialization formats can carry instructions that execute when the file is loaded. Python's pickle format is the familiar example: loading a pickle can run code chosen by whoever created the file. Model files in formats like that should never be loaded from untrusted sources, and even trusted sources deserve a check of the format before loading.

Prefer formats designed to hold tensors and metadata without executable content. Many loaders also have an option that lets a model repository run its own custom code. Leave that option off unless you have read the code and accept what it does. The default should be that loading a model reads data and nothing else.

Verify what you downloaded

Pin every model to a specific revision, identified by a commit or a hash, rather than to a branch or a name that can change. If the source publishes checksums, compare the download against them before use, and record the checksum in your build or deployment configuration. Check again at deployment, not only once when the file was first fetched, because copies can change as they move between machines.

Understand what a checksum proves. It shows that the file matches what the publisher listed. It does not show that the publisher is trustworthy or that the listing itself was not altered. Signatures add something where the source provides them, because they tie the file to a key the publisher controls. Verification is a chain, and it is only as strong as the trust placed in the first listing.

Source trust and internal copies

Ask who published the model, whether the publishing account is verified by the platform, whether the model is an original or a modified fork, and which license governs its use. A fork can be useful, but its changes need the same scrutiny as the original.

Names can mislead. Two repositories may differ by a single character, or by an owner name that looks official while belonging to someone else. Reach the model from the link in the source's own documentation rather than from a search result, and compare the owner and the revision history before trusting the copy. The model card is a claim the publisher makes, not a measurement you have made, so treat its figures as a starting point for your own tests.

Many teams copy vetted artifacts into an internal registry with access control, so that production systems load from a location they manage. The copy should be scanned, its provenance recorded, and its hash checked against the original at the moment of copying. Scanning tools for model files are improving, but their coverage varies by format. The scan supplements the checks above rather than replacing them.

The serving stack

The inference software that runs a model has its own dependencies, and those carry their own vulnerabilities. Keep them patched on the same schedule as other production components. Run model servers with limited privileges, without access to secrets they do not need, and without outbound network access unless a specific function requires it. A model server that can reach the internet and read credentials is a larger target than one that cannot. Check the server's own defaults as well. Some servers expose management endpoints or debugging views that are harmless on a laptop and unsafe on a network, so turn off whatever the deployment does not use.

Fine-tuned models add more links to the chain. The training data, the training code and the job that produced the weights are all part of the provenance, and a change to any of them is a change to the model.

A provenance exercise

Pick a model you use in a sandbox and write its provenance record. Include the source, the exact revision, the file format, the hash, the license, the person who approved its use and the location of the internal copy. Then check which loader options would run code from the bundle, and confirm that they are disabled in your configuration.

Finish by writing down what would make you reject an update. A new revision that changes the format, adds custom code or comes from an unfamiliar account is a reasonable set of triggers. Deciding these before an update arrives keeps the decision from being made under deadline.