Sovereign AI: What It Means and How to Actually Achieve It

7 min read

Sovereign AI means your organization can run its AI systems, and decide who can reach the data flowing through them, without needing anyone else's permission. That includes a foreign government, a cloud provider or a model vendor. It is not a product feature you buy. It is the sum of several separate controls, and most offerings marketed as "sovereign" deliver some of them and not others.

In practice there are five layers: where the data sits, which country's law can compel access to it, who operates the system, who holds the model weights, and what the supply chain underneath depends on. The rest of this post takes each in turn, shows how common deployment options score against them, and ends with a checklist you can use in a vendor review.

The term has a second, national meaning: countries building domestic compute, models and skills. That is a policy question. This post is about the organizational one: what a bank, hospital, manufacturer or agency has to do to keep control of its own AI.

This post is general information, not legal advice. Talk to your counsel about the laws that apply to you.

The five layers of sovereign AI

1. Data residency

Residency is where data is stored and processed. For an LLM system that is more than the prompt. Count every place a copy lands:

  • prompts and completions, including any documents pasted or retrieved into them
  • request logs kept by the provider for debugging or abuse monitoring
  • embeddings and vector indexes built from your documents
  • caches, backups and replicas
  • telemetry and usage analytics sent back to a vendor

Teams usually check the first item and miss the others. A provider can process prompts in your region and still ship logs or telemetry elsewhere. Ask for the full list in writing.

Residency answers "where". Jurisdiction answers "who can be compelled to hand it over", and the two are not the same.

The US CLOUD Act added 18 U.S.C. § 2713, which requires a US provider of electronic communication or remote computing services to disclose data in its "possession, custody, or control, regardless of whether such communication, record, or other information is located within or outside of the United States." Choosing a European region of a US-headquartered provider changes where the data sits. It does not change which law reaches the provider.

European data protection law makes a related point. The EDPB's Recommendations 01/2020 on supplementary measures treat remote access from a third country as a transfer, so a support engineer abroad who can read your data creates a transfer even when the servers are in Frankfurt. The current EU-US Data Privacy Framework survived a challenge in the EU General Court in September 2025, and an appeal is pending before the Court of Justice. Our GDPR compliant LLM guide covers the transfer rules in detail.

The practical test: if the only party that can reach the data is your organization, the jurisdiction question mostly answers itself.

3. Operational control

Ask who has administrative access to the running system and what happens if the vendor stops cooperating.

  • Who holds the encryption keys, and can the provider's staff use them?
  • Who can log in for support, and is that access logged in a record you own?
  • Does the software need to reach a licence server or a vendor control plane to keep running?
  • If the vendor ends the contract, how long does the system keep working?

A system that stops when it cannot phone home is operated, in the end, by whoever answers on the other side. For the strictest cases, the answer is a disconnected network; the air-gapped LLM deployment guide covers how to install and update models without internet access.

4. Model control

With a hosted API you rent a model you cannot inspect, version or keep. Providers retire models on their own schedule; OpenAI, for example, publishes a running list of deprecated models and shutdown dates. An application tuned against one model version then has to be re-tested against its replacement on someone else's timeline.

Open-weight models change this. Once the weights are on your storage, the model you validated keeps working, behaves the same next year, and cannot be withdrawn. You also choose when to upgrade. This is the layer where self-hosting gives the largest gain over every API, regional or not.

5. Supply chain and licences

The last layer is everything your AI depends on that you do not make: GPUs, drivers, container images, inference software and the models themselves.

Model licences deserve a careful read because some carry conditions tied to geography or scale. The Llama 3.2 Acceptable Use Policy, for instance, does not grant rights to its multimodal models to individuals domiciled in, or companies with a principal place of business in, the European Union. A European organization that standardized on those models would have a sovereignty problem it signed itself. Prefer permissive licences such as Apache 2.0 or MIT where you can, record each model's origin and licence, and mirror the exact weights and software versions you run. The open-source LLM for enterprise post has a licence checklist.

Hardware is the hardest layer to control. Most organizations cannot avoid depending on a small number of GPU makers, so the goal is resilience: know your lead times, keep spare capacity for critical workloads, and choose models and quantization levels that run on hardware you can actually buy.

How deployment options compare

Option Data residency Jurisdiction Operations Model weights
Public LLM API Provider's choice Provider's home country Provider Provider; can be retired
Same API in a regional data center Your region Still provider's home country Provider Provider; can be retired
Provider "sovereign cloud" offering Your region Depends on who owns and operates it; read the contract Often a local partner Usually provider
Self-hosted in your cloud account Your region Cloud provider's reach over the infrastructure remains You You
Self-hosted on-premise Your premises Yours You You
Air-gapped on-premise Your premises, no external path Yours You You

Two conclusions follow. First, moving an API to a regional data center fixes residency and nothing else. Second, self-hosting open-weight models in your own cloud account already gives you operational and model control; moving on-premise adds jurisdictional control over the infrastructure itself.

How much sovereignty you actually need

Full sovereignty is expensive: you buy or rent GPUs, staff operations and give up the newest proprietary models. Most organizations do not need it for every workload, so classify by data rather than deciding once for the whole company.

  • Public or low-risk data (marketing drafts, public documentation): a hosted API is usually fine.
  • Internal confidential data (contracts, source code, financial plans): self-host in your own cloud account or data center, with no prompts leaving your network.
  • Regulated or classified data (patient records, defence, critical infrastructure): self-host on-premise, and air-gap where your rules require it.

Being sovereign is also not the same as being compliant. The EU AI Act sets obligations by risk and role, and they apply to a self-hosted model as much as to an API. Sovereignty gives you the control to meet those obligations; it does not meet them for you. The LLM governance guide covers the program that does.

Sovereign AI checklist

Use these questions for each AI system or vendor you assess.

  • Where are prompts, completions, logs, embeddings, backups and telemetry stored? Every location named in writing.
  • Which companies can access the data, and under which countries' laws are they incorporated?
  • Can any third party, including support staff, reach the data from outside your jurisdiction?
  • Who holds the encryption keys?
  • Does the system need an outbound connection to a vendor to run, update or license itself?
  • What happens to the system on the day the contract ends?
  • Do you hold the model weights, and can you pin the version you validated?
  • Does each model's licence allow your use, in your country, at your scale?
  • Are the weights, containers and dependencies mirrored internally with checksums?
  • Is there an audit log of who used which model, kept on infrastructure you control?

If you answer "the vendor" or "we don't know" to more than two of these, you have residency at best, not sovereignty.

Where AI Refinery fits

AI Refinery, a self-hosted LLM platform, covers the operational and model layers. It installs inside your own cloud, VPC, data center or fully air-gapped network, and nothing phones home. Models come from a curated catalog of open-source text, embedding and vision-language models with licence documentation and security scoring, the weights sit in your private storage, and each deployed model serves an OpenAI-compatible endpoint inside your network. RBAC, SSO, approval workflows and a full audit log stay on your infrastructure.

It does not solve every layer. You provide the GPUs, so hardware supply is still yours to plan. Where you install it decides your residency and jurisdiction, and compliance with the laws that apply to you remains your organization's responsibility. Public-sector teams should also read the LLM for government guide, which covers authorization and procurement. The home page explains how the private LLM platform is installed and run.