Run LLMs privately inside your own Infrastructure

A private model catalog, one-click deployment and enterprise governance, without a single prompt leaving your network.

  • OpenAI-compatible API
  • Your cloud, VPC or air-gapped
  • SSO and role-based access
  • Full audit log
Your VPC

Runs in your VPC, and nothing phones home.

  • support-bot
  • code-review
  • doc-search
POST/v1/chat/completions
  • llama-3.1-8bRunningQ4_K_M
    4.9 GB
  • qwen2.5-coder-14bRunningQ5_K_M
    10.5 GB
  • nomic-embed-textRunningf16
    0.3 GB
Requests served inside the line
1,284
Prompts sent outside it
0

Sample data. Model names, sizes and counts are illustrative.

See AI Refinery at work

A walkthrough of what the platform is and how a team uses it.

ai-refinery / inference demo

Every model, sized for the hardware you have

Each model in the catalog comes in quantized variants. Choose the one that fits your GPU, deploy it, and call it through the same API.

In the catalog
43 models223 quantizations
Text generation
25 models118 quantizations
Embedding
6 models36 quantizations
Vision
12 models69 quantizations

Catalog figures as of 17 November 2026.

Catalog model

llama-3.1-8b

Text generation · 8B parameters

4.9GB on disk4-bit

Quantized variants

Your hardware

  • LaptopFits6.3 of 8 GB GPU memory
  • WorkstationFits6.3 of 12 GB GPU memory
  • GPU serverFits6.3 of 24 GB GPU memory

Quality retained98.1%

Deployed, it answers onPOST/v1/chat/completions

Sample data. Sizes, memory and quality figures are illustrative.

Building on public AI APIs is a liability

Three things you give up the moment a prompt leaves your network, and what changes when it doesn't.

The prompt stayed inside your network

Where your prompts go

Public AI API: To someone else's servers. You have no control over what happens to them.

AI Refinery: Nowhere. Every request is served inside your own network.

Model versions

Public AI API: Updated by the provider without warning, so production behaviour drifts.

AI Refinery: Pinned by you. A model changes only when your team deploys a new one.

Audit trail

Public AI API: Whatever the provider chooses to expose, if anything.

AI Refinery: Who ran what, on which model and when, in a log you own.

Your own AI infrastructure, without the infrastructure work

AI Refinery packages everything required to run open-source LLMs privately into a single platform your team can operate from day one.

Deploy the platform

Install AI Refinery on your own cluster using the operations runbook. Runs in your cloud, your VPC or fully air-gapped, and nothing phones home.

About an hour

  1. Browse the catalog

    Benchmarked open-source models with security scoring, license docs and quantization variants. Know exactly what you're deploying before you deploy it.

  2. Pick a quantization

    Pick from a curated catalog and choose a quantization level. Every variant is benchmarked, so you see speed, memory and quality before committing hardware.

    Every quant benchmarked

    llama-3.1-8b

    Text generation · 8B parameters · 4 quantization variants

    File size

    4.9GB

    Memory needed

    6.3of 24 GB

    Speed

    61tokens/s

    Quality retained

    98.1%

    File size by variant

    Q8_08.5 GB
    Q5_K_M5.7 GB
    Q4_K_M4.9 GB
    Q3_K_M4.0 GB
  3. Deploy

    Download a model, pick a quantization level and launch an inference server in minutes. No GPU config, custom scripts or DevOps overhead.

    llama-3.1-8bQ4_K_MRunning

    Serving on GPU 0 · 6.3 of 24 GB in use

    Endpoint

    https://refinery.internal.example/v1
    Model
    llama-3.1-8b
    Quantization
    Q4_K_M
    Hardware
    1 × 24 GB GPU
    API
    OpenAI-compatible
  4. Call the API

    Every deployed model exposes a standard /v1/chat/completions endpoint. Point your existing OpenAI client at it with a new base URL, API key and model name.

    Same SDK, same prompts

    POSThttps://refinery.internal.example/v1/chat/completions

    TimeClientTokensLatencyStatus
    14:02:07support-bot4120.84 s200
    14:02:09doc-search960.21 s200
    14:02:12support-bot6881.37 s200
    14:02:13code-review12042.02 s200
    14:02:16doc-search1430.30 s200

llama-3.1-8b

Text generation · 8B parameters · 4 quantization variants

File size

4.9GB

Memory needed

6.3of 24 GB

Speed

61tokens/s

Quality retained

98.1%

File size by variant

Q8_08.5 GB
Q5_K_M5.7 GB
Q4_K_M4.9 GB
Q3_K_M4.0 GB
llama-3.1-8bQ4_K_MRunning

Serving on GPU 0 · 6.3 of 24 GB in use

Endpoint

https://refinery.internal.example/v1
Model
llama-3.1-8b
Quantization
Q4_K_M
Hardware
1 × 24 GB GPU
API
OpenAI-compatible

POSThttps://refinery.internal.example/v1/chat/completions

TimeClientTokensLatencyStatus
14:02:07support-bot4120.84 s200
14:02:09doc-search960.21 s200
14:02:12support-bot6881.37 s200
14:02:13code-review12042.02 s200
14:02:16doc-search1430.30 s200

Sample data. The console's own components, with illustrative models and figures.

Point your OpenAI client at Refinery

Launch an inference server with one click, then swap the base URL, API key and model name in your existing OpenAI client. No SDK rewrite, no prompt changes, no migration project.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.openai.com/v1",
    base_url="https://refinery.internal.example/v1",
    api_key=OPENAI_API_KEY,
    api_key=REFINERY_API_KEY,
)

reply = client.chat.completions.create(
    model="gpt-4o",
    model="llama-3.1-8b",
    messages=[{"role": "user", "content": "Summarise the incident report."}],
)

Sample host and model. Use your own endpoint and an API token from the console.

The right model for every request, automatically

With several models deployed, picking one by hand is friction your team doesn't need. The routing layer reads each incoming request, understands the task and forwards it to the best available model. One endpoint, every capability.

POST/v1

TaskImage analysis

Sample prompts and model names. Select a request to trace its route.

Governance built in, not bolted on

RBAC, SSO, size-tier controls, approval workflows and a full audit log.

Audit log

Recording
  1. 14:06:41m.okaforDeployed serverllama-3.1-8b · Q4_K_M
  2. 14:05:12svc-support-botCalled chat completionsllama-3.1-8b
  3. 14:03:57j.lindqvistDownloaded modelqwen2.5-coder-14b
  4. 14:02:30a.rahmanChanged rolet.nguyen to Developer
  5. 13:58:04svc-doc-searchCalled embeddingsnomic-embed-text
  6. 13:55:19t.nguyenSigned in with SSOIdentity provider
  7. 13:51:46m.okaforApproved deploymentqwen2.5-coder-14b
Role-based accessAdminDeveloperViewer
Call the APIYesYesNo
Download modelsYesYesNo
Deploy serversYesYesNo
Approve large modelsYesNoNo
View the audit logYesNoYes
Manage rolesYesNoNo

Full observability

Request volume, latency, token usage and errors for every inference server, from a single dashboard.

Requests / min
184
Latency, median
0.42s
Tokens / min
61,300
Errors
0.2%

Sample data. Names, roles, size tiers and figures are illustrative.

Start free, scale when you're ready

No compute costs on our end. You own the infrastructure.

MonthlyAnnually
1 month free

Basic

Model access

Up to 14B params

All model types up to 14 billion parameters.

Features

  • All quantization variants
  • OpenAI-compatible API
  • Basic observability
  • Smart model routing
  • Audit log + RBAC

Support & deployment

  • On-prem + cloud deployment
  • Email support
Previously $299$199/ month

Billed monthly

30-day free trial - cancel anytime

Most popular

Professional

Model access

Up to 70B params

All model types up to 70 billion parameters.

Features

  • All quantization variants
  • OpenAI-compatible API
  • Full observability
  • Smart model routing
  • Audit log + RBAC

Support & deployment

  • On-prem + cloud deployment
  • Priority email support
Previously $999$799/ month

Billed monthly

Enterprise

Model access

Unlimited (140B+)

No size limits - including private and fine-tuned models.

Features

  • All quantization variants
  • OpenAI-compatible API
  • Custom observability
  • Smart model routing
  • Audit log + RBAC
  • Request Custom models

Support & deployment

  • On-prem + cloud deployment
  • Dedicated support
Custom

Talk to us

Annual billing saves ~17%. The 30-day free trial applies to the Basic plan only - on a monthly or annual term.

Guides to running private LLMs

All self-hosted LLM guides
  1. Deployment

    Azure OpenAI Alternative: Run Open Models in Your Own Tenant

    7 min read

  2. Deployment

    Self-Hosted LLM Security: A Hardening Checklist for Teams

    9 min read

  3. Compliance

    Shadow AI: How to Stop Data Leaks Without Banning AI

    7 min read

Talk to the team before you deploy

Ask about Enterprise plans, air-gapped installs or the models that fit your hardware. No account needed.