Amazon Bedrock Alternative: Private LLM Hosting in Your VPC
Looking for an Amazon Bedrock alternative? Compare Bedrock, SageMaker and self-hosting open models on EC2 or EKS GPUs in your VPC, with sizing and trade-offs.
A private model catalog, one-click deployment and enterprise governance, without a single prompt leaving your network.
/v1/chat/completionsllama-3.1-8bRunningQ4_K_Mqwen2.5-coder-14bRunningQ5_K_Mnomic-embed-textRunningf16Sample data. Model names, sizes and counts are illustrative.
A walkthrough of what the platform is and how a team uses it.
Each model in the catalog comes in quantized variants. Choose the one that fits your GPU, deploy it, and call it through the same API.
Catalog figures as of 17 November 2026.
Catalog model
llama-3.1-8b
4.9GB on disk4-bit
Quantized variants
Your hardware
Quality retained98.1%
Deployed, it answers onPOST/v1/chat/completions
Sample data. Sizes, memory and quality figures are illustrative.
Three things you give up the moment a prompt leaves your network, and what changes when it doesn't.
The prompt stayed inside your network
Public AI API: To someone else's servers. You have no control over what happens to them.
AI Refinery: Nowhere. Every request is served inside your own network.
Public AI API: Updated by the provider without warning, so production behaviour drifts.
AI Refinery: Pinned by you. A model changes only when your team deploys a new one.
Public AI API: Whatever the provider chooses to expose, if anything.
AI Refinery: Who ran what, on which model and when, in a log you own.
AI Refinery packages everything required to run open-source LLMs privately into a single platform your team can operate from day one.
Install AI Refinery on your own cluster using the operations runbook. Runs in your cloud, your VPC or fully air-gapped, and nothing phones home.
About an hour
Benchmarked open-source models with security scoring, license docs and quantization variants. Know exactly what you're deploying before you deploy it.
Pick from a curated catalog and choose a quantization level. Every variant is benchmarked, so you see speed, memory and quality before committing hardware.
llama-3.1-8b
4.9GB
6.3of 24 GB
61tokens/s
98.1%
File size by variant
Download a model, pick a quantization level and launch an inference server in minutes. No GPU config, custom scripts or DevOps overhead.
llama-3.1-8bQ4_K_MRunningServing on GPU 0 · 6.3 of 24 GB in use
Endpoint
Every deployed model exposes a standard /v1/chat/completions endpoint. Point your existing OpenAI client at it with a new base URL, API key and model name.
POSThttps://refinery.internal.example/v1/chat/completions
| Time | Client | Tokens | Latency | Status |
|---|---|---|---|---|
| 14:02:07 | support-bot | 412 | 0.84 s | 200 |
| 14:02:09 | doc-search | 96 | 0.21 s | 200 |
| 14:02:12 | support-bot | 688 | 1.37 s | 200 |
| 14:02:13 | code-review | 1204 | 2.02 s | 200 |
| 14:02:16 | doc-search | 143 | 0.30 s | 200 |
llama-3.1-8b
4.9GB
6.3of 24 GB
61tokens/s
98.1%
File size by variant
llama-3.1-8bQ4_K_MRunningServing on GPU 0 · 6.3 of 24 GB in use
Endpoint
POSThttps://refinery.internal.example/v1/chat/completions
| Time | Client | Tokens | Latency | Status |
|---|---|---|---|---|
| 14:02:07 | support-bot | 412 | 0.84 s | 200 |
| 14:02:09 | doc-search | 96 | 0.21 s | 200 |
| 14:02:12 | support-bot | 688 | 1.37 s | 200 |
| 14:02:13 | code-review | 1204 | 2.02 s | 200 |
| 14:02:16 | doc-search | 143 | 0.30 s | 200 |
Sample data. The console's own components, with illustrative models and figures.
Launch an inference server with one click, then swap the base URL, API key and model name in your existing OpenAI client. No SDK rewrite, no prompt changes, no migration project.
from openai import OpenAI
client = OpenAI(
base_url="https://api.openai.com/v1",
base_url="https://refinery.internal.example/v1",
api_key=OPENAI_API_KEY,
api_key=REFINERY_API_KEY,
)
reply = client.chat.completions.create(
model="gpt-4o",
model="llama-3.1-8b",
messages=[{"role": "user", "content": "Summarise the incident report."}],
)
Sample host and model. Use your own endpoint and an API token from the console.
With several models deployed, picking one by hand is friction your team doesn't need. The routing layer reads each incoming request, understands the task and forwards it to the best available model. One endpoint, every capability.
/v1TaskImage analysis
Sample prompts and model names. Select a request to trace its route.
RBAC, SSO, size-tier controls, approval workflows and a full audit log.
llama-3.1-8b · Q4_K_Mllama-3.1-8bqwen2.5-coder-14bt.nguyen to Developernomic-embed-textIdentity providerqwen2.5-coder-14b| Role-based access | Admin | Developer | Viewer |
|---|---|---|---|
| Call the API | Yes | Yes | No |
| Download models | Yes | Yes | No |
| Deploy servers | Yes | Yes | No |
| Approve large models | Yes | No | No |
| View the audit log | Yes | No | Yes |
| Manage roles | Yes | No | No |
Request volume, latency, token usage and errors for every inference server, from a single dashboard.
Sample data. Names, roles, size tiers and figures are illustrative.
No compute costs on our end. You own the infrastructure.
Model access
Up to 14B params
All model types up to 14 billion parameters.
Features
Support & deployment
Billed monthly
30-day free trial - cancel anytime
Model access
Up to 70B params
All model types up to 70 billion parameters.
Features
Support & deployment
Billed monthly
Model access
Unlimited (140B+)
No size limits - including private and fine-tuned models.
Features
Support & deployment
Talk to us
Annual billing saves ~17%. The 30-day free trial applies to the Basic plan only - on a monthly or annual term.
Looking for an Amazon Bedrock alternative? Compare Bedrock, SageMaker and self-hosting open models on EC2 or EKS GPUs in your VPC, with sizing and trade-offs.
Ask about Enterprise plans, air-gapped installs or the models that fit your hardware. No account needed.