Open-Weight Models Worth Self-Hosting in 2026
Published 23 September 2026 by Jake Hissitt, Founder of Stob.AI.
Open weights matter for three reasons: data that cannot leave your infrastructure, cost at extreme volume, and not being repriced by someone else's roadmap. Google's Gemma 4 and Meta's Muse Glimmer are useful compact Apache 2.0 starting points. Frontier-scale open-weight models such as Kimi K3.1 Pro and DeepSeek V4.1 Flash-1017 also deserve evaluation when capability matters more than hardware footprint.
The shortlist
| Model | Size | Context | Licence | Cost |
|---|---|---|---|---|
| Gemma 4 31B IT | 30.7B dense | 262,144 | Apache 2.0 | Free on the Gemini API, or self-host |
| Gemma 4 26B-A4B IT | 25.2B total / 3.8B active MoE | 262,144 | Apache 2.0 | Free on the Gemini API, or self-host |
| Gemma 4 12B Unified IT | 11.95B | 262,144 | Apache 2.0 | Host or self-host |
| Gemma 4 E4B / E2B IT | Edge sizes | 131,072 | Apache 2.0 | Host or self-host |
| Muse Glimmer | 30B dense multimodal | 131,072 | Apache 2.0 | Host or self-host; Meta cites Together AI at $0.35 / $1.50 |
Gemma 4: compact reasoning under Apache 2.0
Google reports 85.2 MMLU-Pro, 89.2 AIME 2026, 84.3 GPQA Diamond and 80.0 LiveCodeBench v6 for Gemma 4 31B IT. Those are serious numbers for a 30B model under Apache 2.0, and they put it ahead of much larger open models from a year ago.
The 26B-A4B mixture-of-experts variant reports 82.6 MMLU-Pro, 88.3 AIME 2026, 82.3 GPQA Diamond and 77.1 LiveCodeBench v6 while activating only 3.8B parameters per token — which is the interesting one if inference cost on your own hardware is the constraint.
One caution: the free Gemini API tier for Gemma 4 is genuinely free, but free capacity is not a production SLA. If a workflow matters, self-host it or pay for hosted capacity.
Muse Glimmer: the local agent model
Muse Glimmer is Meta's 30B dense multimodal open-weight release from 10 August 2026, aimed at always-on local agents: sequential tool use, private workflows and recovery from failed steps. Meta cites Together AI at $0.35 in / $1.50 out per million tokens as one hosted example, but as an open-weight model the real cost depends on where you run it.
It is a different proposition from Muse Spark 1.3, which is hosted and closed-weight. Glimmer is what you reach for when the requirement is "this must run on our own hardware".
Frontier-scale open weights: Kimi and DeepSeek
Kimi K3.1 Pro has published open weights, 2.8T total parameters with 104B active per token, and a 1,048,576-token context window. It is a candidate for demanding reasoning, coding and agent workflows when a much larger deployment is practical. Its custom Kimi K3.1 Pro License has commercial conditions that need review before deployment.
DeepSeek V4.1 Flash-1017 is another current open-weight option, with a 1M-token context window and native text and image input. Its weights use the MIT licence. Evaluate it for long-context reasoning, coding and agents when a permissive licence is important.
These frontier-scale checkpoints need substantially more GPU memory and deployment capacity than the 12B–31B compact models above. Active parameter counts describe inference work per token; the full weights still need to fit in the deployment. Choose Gemma or Glimmer when a compact Apache 2.0 deployment is the priority, and benchmark the larger models when their capability can justify that infrastructure.
When self-hosting actually pays
- Data residency or confidentiality rules that rule out sending content to a third-party API. This is the strongest reason, and it does not depend on cost at all.
- Very high, very steady volume where a reserved GPU is cheaper than per-token billing. Below a few hundred million tokens a month this rarely holds once engineering time is counted.
- Fine-tuning on proprietary data where you need the weights.
- Latency at the edge, which is what the Gemma 4 E2B and E4B sizes exist for.
If none of those apply, GPT-6 Luna at $0.10 / $0.50 will almost certainly beat a self-hosted model on total cost once you include engineering, monitoring and idle GPU time.
Frequently asked questions
What is the best open-weight AI model in 2026?
For a compact Apache 2.0 deployment, start with Gemma 4 31B IT for reasoning and coding or Muse Glimmer for local agent workflows. With frontier-scale hardware, also evaluate Kimi K3.1 Pro under its custom licence and MIT-licensed DeepSeek V4.1 Flash-1017. The right choice depends on capability, licence and infrastructure requirements.
Is Gemma 4 free?
The weights are Apache 2.0, and Google offers Gemma 4 free on the Gemini API with no paid tier. Free capacity is not a production service level, so self-host anything business-critical.
Is Muse Glimmer open source?
The weights are released under Apache 2.0. Muse Spark 1.3, by contrast, is hosted and closed-weight.
Should I self-host or use an API?
Self-host for data residency, fine-tuning or very high steady volume. Otherwise a cheap hosted tier is usually less total cost and far less operational work.
What about Llama?
Llama 4 Maverick and Scout remain relevant open-weight options. See Best Llama Model 2026.
What to do next
Write down which of the four reasons above actually applies to you before comparing models. If none does, the decision is a hosted-tier decision. Compare every option, open and closed, in our AI model comparison tool.