Open Source LLM for Enterprise: Choose by Task and License

8 min read

Picking an open source LLM for enterprise work comes down to four checks, in this order: what task the model must do, which size tier your hardware and quality bar allow, whether the license permits your use, and how it scores on your own data. Leaderboards help you build a shortlist. They do not tell you which model will summarize your contracts or classify your tickets best.

Most enterprise teams end up with two or three models rather than one: a mid-size general model for most work, a small fast one for high-volume tasks, and a specialist such as a vision or embedding model where needed. This guide gives you a way to choose each.

A note on wording: most of these models are "open-weight", meaning you can download and run the trained weights, but the license may not meet the formal open source definition. The license, not the label, decides what you can do.

Choosing an open source LLM for enterprise work: start with the task

Different jobs reward different model types. Decide which of these you need before comparing families:

Task Model type What to look for
Drafting, summarizing, Q&A Instruction-tuned text model Instruction following, long context
Classification, extraction to JSON Instruction-tuned text model, often small Reliable structured output at high volume
Code generation and review Code-tuned or strong general model Results on your languages and repos
Multi-step reasoning, analysis Reasoning model Accuracy on hard cases; accept slower, longer answers
Screenshots, scanned forms, charts Vision-language model Image input, document understanding
Search and retrieval (RAG) Embedding model Retrieval quality on your documents, vector size

An embedding model turns text into vectors (lists of numbers) so that similar passages can be found by search. It does not generate text, and it is usually small enough to run next to your main model.

Pick a size tier

Parameter count is the clearest proxy for capability and cost. Three tiers cover almost every enterprise deployment.

7-14B parameters. Fast and cheap to run. Good for classification, extraction, routing, short summaries and simple chat. At 4-bit quantization (about 0.6 bytes per parameter), a 14B model needs 14 x 0.6 = about 8.4 GB for its weights, which fits on a single 16-24 GB GPU with room for the KV cache.

30-70B parameters. The default for a general internal assistant or document work. A 32B model at 4-bit needs about 32 x 0.6 = 19 GB; a 70B model needs about 70 x 0.6 = 42 GB, so plan for a 48 GB or 80 GB GPU once you add the KV cache.

100B+ parameters. For the hardest tasks. Many models in this tier use a mixture-of-experts (MoE) design: the full set of weights must sit in memory, but only a fraction of them is used for each token, so they run faster than their size suggests. Three examples from the model cards:

  • gpt-oss-120b: 117B total, 5.1B active, fits on a single 80 GB GPU according to its card. The gpt-oss hardware requirements post shows the full sizing.
  • Llama 4 Scout: 109B total, 17B active, fits on a single H100 with int4 quantization according to its card.
  • DeepSeek-R1: 671B total, 37B active. At 4-bit that is 671 x 0.6 = about 400 GB of weights, which means a multi-GPU server.

The KV cache comes on top of every one of these numbers and grows with context length and concurrent users. Our guide to GPU memory for LLM inference shows how to size it, and LLM quantization explained covers what 4-bit costs in quality.

Start one tier smaller than you think you need. If the smaller model passes your evaluation, you save hardware for every request it ever serves. Our breakdown of self-hosted LLM cost shows how quickly that adds up.

Read the license before you build

Licenses differ between families and sometimes between models in the same family. These are the terms we checked on each model's own page. Check the specific release you plan to use, because terms change between versions.

Family Example release License Points to note
Llama Llama 3.3, Llama 4 Llama Community License Separate license from Meta above 700M monthly active users, "Built with Llama" attribution, acceptable use policy
Qwen Qwen3-32B Apache 2.0 Some older releases, such as Qwen2.5-72B, use the Qwen License
Mistral Ministral 3, Mistral 7B Apache 2.0 Some models use the Mistral Research License or MNPL
Gemma Gemma 4 Apache 2.0 Gemma 1 to 3 use the Gemma Terms of Use
DeepSeek DeepSeek-R1 MIT Distilled versions inherit their base model's license
gpt-oss gpt-oss-120b Apache 2.0 Short usage policy: comply with applicable law

The details behind that table:

  • Llama. The Llama 3.3 license and the Llama 4 Community License both require you to request a license from Meta if your products had more than 700 million monthly active users, to display "Built with Llama", and to follow Meta's acceptable use policy. Under the Llama 3.3 license, if you distribute a model built from Llama, its name must start with "Llama".
  • Qwen. Qwen3-32B is Apache 2.0. The Qwen2.5-72B license is Alibaba Cloud's own agreement and requires a separate license above 100 million monthly active users.
  • Mistral. Models such as Ministral 3 14B are Apache 2.0. Mistral Large 2411 uses the Mistral Research License, which its card describes as allowing non-commercial use. Codestral 22B uses the Mistral Non-Production License.
  • Gemma. Gemma 4 is listed as Apache 2.0. Earlier Gemma models fall under the Gemma Terms of Use, which include a prohibited use policy and let Google restrict use it believes violates the terms.
  • DeepSeek. The DeepSeek-R1 card puts the code and weights under MIT, and notes that the distilled models come from Qwen2.5 (Apache 2.0) and Llama 3.1 or 3.3 bases, which keep their original licenses.
  • gpt-oss. gpt-oss-120b is Apache 2.0, with a two-sentence usage policy asking you to comply with applicable law.

Apache 2.0 and MIT are the simplest to approve: commercial use is allowed and the obligations are mostly attribution and notices. Custom licenses are not necessarily a problem, but they need a legal review, especially the acceptable use policies, which list uses you agree not to pursue. This post is not legal advice; have your counsel read the actual license for the model you deploy.

A decision table

If you need Start with Typical hardware at 4-bit
High-volume classification or extraction A 7-14B Apache 2.0 instruct model (Qwen, Mistral, Gemma families) One 16-24 GB GPU
A general internal assistant A 30-70B instruct model One 48-80 GB GPU
Hard reasoning or long analysis A reasoning model or a 100B+ MoE model One 80 GB GPU for gpt-oss-120b class; a multi-GPU server above that
Screenshot and document understanding A vision-language model, such as the Gemma 4 or Llama 4 families, which accept images Depends on size; same rules as text
Search over internal documents An embedding model plus a text model Embedding models are small; size for the text model
Only permissive licenses allowed Filter to Apache 2.0 or MIT first, then apply the rows above

Families release new versions every few months. Use the table to pick a family and tier, then check the current release and its license.

Evaluate on your own data

Public benchmarks measure general skills. Your decision should rest on how a model handles your documents, your formats and your edge cases. A lightweight evaluation is enough:

  1. Collect 50-200 real examples per task, with the answer you consider correct. Include the awkward ones: long inputs, ambiguous requests, the formats your systems parse.
  2. Define how you score. Exact match for labels and extracted fields, a rubric scored by reviewers for summaries and drafts.
  3. Run two or three candidates from different families and tiers, at the quantization you would deploy. A 4-bit model can behave differently from the full-precision version.
  4. Record speed and memory too. Tokens per second at your expected concurrency and peak GPU memory decide your hardware bill.
  5. Pick the smallest model that passes. Then re-run the same set every time you consider an upgrade.

Keep the evaluation set. It becomes your regression test for every future model change, and it is the evidence your risk team will ask for. Our guide on how to evaluate open source LLMs on your own data covers golden sets, judge models and tooling in more depth.

Once you run more than one model, you need a way to send each request to the right one. LLM routing covers the options.

Where AI Refinery fits

AI Refinery, a self-hosted LLM platform, shortens the shortlist and test steps. Its curated catalog covers text, embedding and vision-language models, and every model comes with security scoring and license documentation, so you see the license before you download. Models are GGUF files in multiple quantization variants, and each variant is benchmarked before it reaches the catalog: you see size on disk, GPU memory needed (weights plus KV cache at the model's full context window) and measured tokens per second.

You download a model into your private storage, pick a quantization level and launch an inference server with one click. Every deployed model exposes an OpenAI-compatible /v1/chat/completions endpoint, so your evaluation scripts can call every candidate the same way. Size-tier controls limit which model sizes each team can use, and the plans follow the same tiers: up to 14B, up to 70B, and unlimited including private and fine-tuned models.

When it is not the right fit

If you have already chosen one model and have a team comfortable running inference servers by hand, a catalog adds little. If your evaluation shows that only a closed frontier model meets your quality bar, no open-weight platform changes that result. And the license review stays with your own legal team: license documentation in a catalog helps them, it does not replace them.

Sizing a specific model? See Qwen VRAM requirements and Llama 70B VRAM requirements.