Home / News / Article
SecurityWeek · 4 min read

The Governance Gap in Open-Weight AI and What to Do About It

Open-weight AI has gone mainstream. As of September 2026, some estimates suggest 42% of enterprises surveyed were running open-weight models in production, up from 31% just two months earlier. But downloading a model also means taking on responsibilities that a provider might otherwise handle, and many organizations have yet to put clear ownership and controls in place.

Renting Capability Vs. Running Your Own Copy

When you use an AI model through a provider’s app or service, you’re essentially renting its capabilities. The provider controls the model and handles the infrastructure, updates, and safety controls. You access it through an app or a software connection.

An open-weight AI model makes its learned parameters, the ‘weights’ that shape its behavior, available to download and run yourself. You can run the model on your own infrastructure and adapt it for a particular use case using your own data. The license may limit how you can use it, and running it still carries infrastructure and staffing costs.

Open-weight models are becoming more capable. One estimate puts the best open-weight models roughly four months behind leading models.

Open-weight does not necessarily mean open-source. Access to the weights does not guarantee access to the training code or details of the training data.

When Benefits Clash with Risks

Zoey is a bank analyst using a provider-hosted model to summarize a loan application. The model does a good job of delivering an actionable summary, but to do so, her request, loan file included, has to go to the provider’s servers. The customer’s data is no longer solely in the bank’s hands. A few quarters later, the provider raises its prices and changes its policies in ways that force workflow changes. Switching providers would take time and effort, so Zoey and her team have little choice but to go along, even though it hurts both their budget and their operations.

Let’s look at the same scenario from the perspective of an open-weight model that is running on the bank’s server. If the model is configured to run entirely within the bank’s system, the loan file can stay there too. The bank can apply its own access controls to the model and the data.

In the above example, an open-weight model does seem like a good option, but there are risks that are easy to overlook:

  • Trojan horses: Some model files can execute malicious code when loaded. Others may contain hidden behavior that a particular input can trigger.
  • An origin story with missing chapters: A popular model on a public repository may not be the developer’s original. Someone may have modified and reuploaded it without explaining what changed. That version could contain a hidden backdoor or have weak safety rules.
  • The upgrade won’t knock on your door: A developer may release improved versions, but your copy is not updated automatically. Your team must track updates, test them, and decide when to switch.
  • Attack surface just got bigger: Running your own model creates new targets for attackers. You are essentially adding to your IT infrastructure. This includes the servers and GPUs that keep the model running and the systems that retrain it.

Controls to Address Risks

If an open-weight AI model makes sense for your organization, governance should walk in lockstep with deployment:

  • Set a regular review schedule for updates and security notices. Decide who approves a move to a new version.
  • Assign an owner for each model, with clear responsibility for approving its use, reviewing updates, and responding when something goes wrong.
  • Record each model’s source, version, license, and every fine-tune. Also record its file hash. It’s a unique fingerprint that changes if anyone alters the file.
  • Give teams an approved internal repository to fetch models. Before adding a model, verify its source and check its file fingerprint or digital signature against a trusted reference.
  • The rule of thumb should be: ‘trust after testing.’ Test the model as an attacker might. Try to bypass its safety rules and look for unexpected behavior, including whether it produces insecure code.
  • If you do choose to download a modified version, run the same tests on the developer’s original. This may help reveal weak safeguards or unexpected behavior.
  • Higher-risk models need more control. For example, a model that reads source code should also run in isolation, with no access to passwords or keys, and a person assigned to review its output.
  • Run the model on a separate server while giving it only the access it needs. Monitor all requests and outputs for unusual behavior, while protecting sensitive information in the logs. If you think a model is misbehaving, have a backup in the form of an earlier version you can switch back to quickly.

Open-weight models need to be scrutinized like any other software brought inhouse. Their benefits are clearly driving their popularity, but there are tradeoffs to be aware of. It’s wise to put ownership and controls in place before putting them into production.

Originally published on SecurityWeek