HIPAA Compliant LLM: What Self-Hosting Changes for PHI

8 min read

There is no such thing as a HIPAA compliant LLM you can buy. HIPAA applies to organizations, covered entities and their business associates, and to how they handle protected health information (PHI). Software cannot be compliant on its own. What you can build is an LLM deployment that you operate in a way that meets the Privacy Rule and the Security Rule, with the paperwork to show it.

Where the model runs decides a lot of that paperwork. If a vendor receives PHI to run inference for you, that vendor is a business associate and you need a business associate agreement (BAA) before any PHI goes in. If the model runs on infrastructure you control and no third party touches the PHI, that question changes. The safeguards do not go away. They become yours.

This post is general information, not legal advice. Talk to your privacy officer or counsel before putting PHI into any system.

When HIPAA applies to an LLM

HIPAA's Privacy Rule covers health plans, health care clearinghouses and health care providers who conduct certain transactions electronically, plus the business associates that handle PHI on their behalf. If you are one of those and PHI can reach the model, the LLM system is in scope.

PHI reaches a model in more places than the prompt box:

  • Prompts and pasted text, such as a clinical note to summarize
  • Documents attached for analysis, and the embeddings and search index built from them
  • Inference logs, if they store request or response text
  • Model outputs, which inherit the PHI in the input
  • Any data used to evaluate or fine-tune a model

Data that has been de-identified under one of the two methods in HHS's de-identification guidance is no longer PHI. If a use case can run on de-identified text, that is the simplest way to take it out of scope.

What the Privacy Rule asks of LLM use

The Privacy Rule controls when PHI may be used or disclosed. Two parts matter most for an LLM.

Permitted purpose. Using PHI to summarize a chart for the treating clinician is a different question from using it to train a model for a new product. Map each LLM use case to a permitted use, such as treatment, payment or health care operations, or to an authorization.

Minimum necessary. Covered entities must make reasonable efforts to limit PHI used or disclosed to the minimum necessary for the purpose, with exceptions such as disclosures to a provider for treatment. For an LLM, that means sending the relevant section of a record, not the whole record, and limiting what a retrieval system can pull into context. It also means role-based access, so a billing workflow cannot reach clinical notes it does not need.

What the Security Rule asks of LLM infrastructure

The Security Rule applies to electronic PHI and requires administrative, physical and technical safeguards. It starts with a risk analysis: an assessment of risks to all the ePHI you create, receive, maintain or transmit. A new LLM system belongs in that analysis before it goes live.

The technical safeguards map directly onto an LLM deployment:

Safeguard What it means for an LLM system
Access control Only authorized people and applications can call the model or reach its storage. Unique user IDs, SSO, role-based permissions, per-application API tokens.
Audit controls Mechanisms that record and examine activity: who called which model, when, from which application.
Integrity Protect ePHI from improper change or destruction, including stored prompts, outputs and documents.
Person or entity authentication Verify that a caller is who it claims to be before it reaches the endpoint.
Transmission security Protect ePHI in transit: TLS between applications, the endpoint and any storage.

Some implementation specifications are "required" and others "addressable", meaning you implement them if reasonable and appropriate or document an equivalent alternative. Encryption is currently addressable. HHS has proposed changes that would, among other things, require encryption of ePHI at rest and in transit. At the time of writing that is a proposal, not a final rule, so check its status. In practice, encrypt anyway.

Business associates and BAAs

A business associate is a person or organization that creates, receives, maintains or transmits PHI on behalf of a covered entity. The covered entity needs a written agreement that limits what the associate may do with the PHI and obliges it to safeguard the data, as HHS's business associate guidance explains.

For LLMs this produces three common cases:

  • A hosted model API. If you send PHI to a provider's API, the provider receives and processes PHI for you. It is a business associate, and you need a BAA with it first. Check which of its products and endpoints the BAA covers, and what it says about retention.
  • A cloud provider hosting your own model. HHS's cloud computing guidance is clear that a cloud provider storing ePHI is a business associate, even if the data is encrypted and it has no key. If you run a model on rented GPUs in a public cloud, you need a BAA with that cloud provider, which many organizations already have.
  • On-premises hardware you own. No third party stores or processes the PHI for inference.

The "conduit" exception, which covers carriers like the postal service, is narrow. It applies to transmission-only services with transient access. A provider that stores or processes data does not qualify.

How self-hosting changes the BAA question for a HIPAA compliant LLM

With a hosted API, the model vendor is in the PHI flow by design. With a self-hosted model, it does not have to be.

HHS addresses this in an FAQ on whether a software vendor is a business associate. Supplying software to a covered entity does not by itself make the vendor a business associate. A vendor becomes one if it hosts the PHI or accesses it, for example while troubleshooting.

So for a self-hosted LLM, the BAA picture looks like this:

Deployment Who is in the PHI flow for inference BAA needed with
Hosted model API The API provider The API provider
Self-hosted in your cloud account Your cloud provider Your cloud provider
Self-hosted on your own hardware Only you Nobody, for inference

The condition is real: the software vendor must stay out of the PHI. If its support staff log into servers that hold prompts or logs, or you send them a log file with patient data in it, the analysis changes. Write down how support works: what you share, how it is scrubbed, and who approves remote access.

The safeguards you still own

Self-hosting removes a party from the PHI flow. It does not remove any of these:

  1. Risk analysis. Add the LLM system: endpoint, storage, logs, document stores, embeddings, and every application that calls it.
  2. Access. SSO for people, a separate token per application, roles that follow minimum necessary, and immediate removal of access when someone leaves.
  3. Audit controls. Record who called which model, when, through which application, and every administrative change. Review the logs on a schedule rather than only after an incident.
  4. Prompt and output retention. Logs that store full text are ePHI. Store text only when you need it, keep it separately from metadata, encrypt it, and set a retention period.
  5. Transmission security. TLS on every hop, including internal ones.
  6. Encryption at rest for model storage that holds documents, embeddings, prompts and outputs.
  7. Output review. A model can produce wrong clinical statements with confidence. Keep a human in the loop for anything that reaches a chart, a patient or a claim.
  8. Workforce training and sanctions, updated to cover which AI tools are approved for PHI.
  9. Breach response. Your incident plan should cover the LLM system, including leaked logs.
  10. Documentation. The Security Rule requires policies and procedures in writing, kept for six years from creation or from when they were last in effect, whichever is later.

HIPAA LLM checklist

  • Each LLM use case mapped to a permitted purpose under the Privacy Rule
  • De-identified data used wherever the task allows it
  • LLM system included in the Security Rule risk analysis
  • BAA in place with every party that stores or processes PHI, including the cloud provider
  • Software vendor support process keeps vendor staff away from PHI, in writing
  • SSO, role-based access and per-application tokens configured
  • Audit log covers inference and admin events, with a review schedule
  • Full-text logging off by default, or encrypted with a set retention period
  • TLS everywhere, encryption at rest
  • Human review for clinical or patient-facing output
  • Policies documented and retained

Where AI Refinery fits

AI Refinery, a self-hosted LLM platform, installs inside your own infrastructure: your cloud account, your VPC, on-premises hardware or a fully air-gapped network. Nothing phones home, so prompts and responses stay in the environment you control. That supports a HIPAA program by keeping the model vendor out of the inference path. It does not make anything HIPAA compliant, and AI Refinery does not sign BAAs. Compliance stays your organization's responsibility.

For the safeguards above, it provides RBAC, SSO, size-tier controls, approval workflows and a full audit log, plus monitoring of request volume, latency, token usage and errors. Applications call models through an OpenAI-compatible endpoint with tokens managed in the console. For document search over records you pair it with a vector database of your choice, which our guide to RAG with a local LLM covers. If the system must sit on an isolated clinical network, see the air-gapped LLM deployment guide.

It is not the right fit if you want a vendor to carry part of the compliance load under a BAA, or if you have no team to run GPU servers and the controls around them. In that case a hosted model API under a BAA, scoped carefully, may suit you better. The trade-offs are laid out in private LLM vs OpenAI API, and the self-hosted LLM guide covers what running your own model involves.