LLM for Government: Sovereignty, Air-Gaps and Procurement

4 min read

An LLM for government has to meet constraints that most commercial deployments never face: data that cannot leave a jurisdiction or a classified network, formal authorization before any system goes live, public records and audit duties, and procurement rules that favor what can be inspected. Self-hosting a model inside the agency's own environment answers several of these at once, because the data path and the supply chain are visible and under the agency's control.

This post covers what changes for public-sector teams, in the order a project meets it. It is general information, not legal or procurement advice; your agency's counsel and security office decide what applies to you.

Start with the data, not the model

Before choosing anything, classify what the LLM will see. Public-sector use cases fall into three bands:

  • Public or low-sensitivity text: drafting, summarizing published reports, translating public notices. A hosted API may be acceptable under your policy.
  • Controlled or personal data: casework, citizen records, internal memos, procurement documents. Rules often restrict where this can be processed and by whom.
  • Classified or mission-critical: processed only in accredited, often disconnected, networks.

The second and third bands are where self-hosting pays off. Running the model inside your own environment removes the third-party processor from the data path. It does not make the system compliant on its own, and compliance remains the agency's responsibility. The private LLM vs OpenAI API comparison shows where each approach leaves your data.

Authorization comes first

In the US federal setting, a cloud service used by an agency needs a FedRAMP authorization, and a system the agency operates itself needs its own authorization to operate under the agency's security process, built on controls such as NIST SP 800-53. Other governments have equivalents. The point for LLM projects is practical: a self-hosted deployment is evaluated as part of your system boundary, so documentation, patching and logging are your work, but you are not waiting on a vendor's authorization to cover a new AI service.

Ask the security office early about:

  • Which environments are approved (agency cloud tenant, on-premise data center, accredited enclave).
  • Whether GPU servers and the software stack fit the existing baseline.
  • How model files, which are large binary artifacts, enter the environment and who approves them.

Sovereignty and residency

Sovereignty usually means three things: the data stays within a defined jurisdiction, the infrastructure is operated by people the agency trusts, and the agency is not dependent on a foreign provider's continued service. A self-hosted open-source model addresses each: weights live on your storage, you choose the data center, and a model you have downloaded keeps working if a vendor changes terms. That last point is a real advantage over an API, which can change pricing, behavior or availability without your consent. The sovereign AI guide breaks this into five layers, including legal jurisdiction and supply chain, with a vendor checklist.

Open weights carry their own diligence. Check each model's license for restrictions on use, and apply any agency policy on model origin, which some jurisdictions set. The open-source LLM for enterprise post has a license checklist.

Disconnected networks

Many agencies need the system to run with no internet connection. The model files and software are brought in through an approved transfer process, and nothing in the running system calls out. Sizing, weight transfer and a pre-flight checklist for this case are in the air-gapped LLM deployment post. Check early that every component, including the logging and monitoring stack, works offline; a single hidden update check can fail an accreditation review.

Audit, records and oversight

Government use of AI is increasingly subject to oversight rules: inventories of AI use cases, risk assessments, and records that show who used which model for what. The NIST AI Risk Management Framework is a common reference for structuring this. For the LLM layer, the practical requirements are:

  • Every request attributable to a named user or system.
  • Access control that matches roles in the agency.
  • An approval step before a new model goes into use.
  • Logs kept long enough to meet records rules, in a form an auditor can read.

Our LLM governance post lays out a program that covers these.

Procurement checklist

Questions that save time when evaluating any self-hosted LLM product:

  1. Can it be installed and run with no internet access, and is there a documented offline install?
  2. What exactly does it phone home, if anything?
  3. Which model licenses does the catalog include, and are they documented per model?
  4. What identity integration is supported (SSO), and what roles exist?
  5. Is there an audit log, and can it be exported to your logging system?
  6. What does the vendor claim about certifications? Ask for the documents, and treat a missing document as a no.
  7. How are updates delivered into a disconnected environment?

Where AI Refinery fits

AI Refinery, a self-hosted LLM platform, installs inside the customer's own infrastructure: their cloud, VPC, on-premise data center or a fully air-gapped network. Nothing phones home. It provides a curated catalog of open-source text, embedding and vision-language models with security scoring and license documentation per model, an OpenAI-compatible endpoint for each deployed model, and RBAC, SSO, approval workflows and a full audit log. The customer provides the GPUs.

It holds no compliance certification or government authorization, and nothing in this post should be read as saying it does. The agency's own authorization process covers the deployed system. It also has no built-in chat interface or RAG layer, so agencies pair it with their own front end and document store. Plans are described under pricing; for questions about public-sector deployments, write to [email protected]. The platform itself is described at AI Refinery.

General information only, not legal advice. Rules differ by country and agency; confirm requirements with your counsel and security office.