Open-Source AI Model Comparison
Open-Source AI Model Comparison
"Every AI tool that you're using right now, you don't actually own. You're just renting them." — Kacper Rutkiewicz (Source: Which Open Source AI Model Should You Use, YouTube, 2026-06-20)
China is shipping open-weight models at the frontier — GLM-5.2, DeepSeek V4, Kimmy, Qwen, MiniMax — that match or approach Opus 4.8 quality at 5-10x lower cost. The West (Meta Llama, Mistral, Google Gemma) is 6-12 months behind. This page compares eight open-source models across capability, cost, and local-run feasibility, based on side-by-side testing in the Claude Code harness.
The Landscape
Two camps are shipping open-source AI:
| Camp | Key Players | Status |
|---|---|---|
| China | Z.ai (GLM), DeepSeek, Alibaba (Qwen), Moonshot (Kimmy), MiniMax | Frontier-quality, aggressive open-weight releases |
| West | Meta (Llama), Mistral (France), Google (Gemma) | 6-12 months behind; restrictive licenses; smaller models |
A common myth: Chinese models are "API only." This is false. DeepSeek put a 1.6-trillion-parameter model on Hugging Face. Z.ai's GLM-5.2 has been quantized to 2-bit by Unsloth and runs on a Mac Mini. Both sides ship open-weight models — the West is simply not competitive at the top tier right now.
Model Rankings
1. GLM-5.2 — Best Daily Driver
| Attribute | Detail |
|---|---|
| Maker | Z.ai (Tsinghua University, China) |
| Size | 753B parameters |
| Best at | Agentic coding and planning — plans, builds, tests, fixes itself |
| Cost vs Opus | ~5x cheaper ($0.20 vs $1.60 in side-by-side habit-tracker test) |
| Run locally? | No (flagship). Yes via Unsloth distillation (82% accuracy on Mac Mini) |
The model the creator uses as his daily driver alongside Opus 4.8. Same result, 5x fewer tokens.
2. DeepSeek V4 — Deep Reasoning
| Attribute | Detail |
|---|---|
| Maker | DeepSeek (China) |
| Size | 1.6T total (49B active — MoE architecture) |
| Best at | Reasoning across massive context (1M window); multi-document conflict analysis |
| Cost vs Opus | ~5x cheaper |
| Run locally? | No (flagship). Distilled versions available via Unsloth/Hugging Face |
The model that proved China would open-weight their biggest models. Highly efficient for its size.
3. Kimmy K 2.7 — Long Agentic Work
| Attribute | Detail |
|---|---|
| Maker | Moonshot AI (China) |
| Size | ~1T parameters |
| Best at | Multi-stage tasks where weaker models forget steps; follows plans faithfully |
| Cost vs Opus | ~7x cheaper ($1.33 vs $7.52) |
| Run locally? | No (flagship) |
Completed a 6-stage CLI build in 4:21 vs Opus's 6:16. Same functional result — differences were taste, not capability.
4. Qwen — Reliable Workhorse
| Attribute | Detail |
|---|---|
| Maker | Alibaba (China) |
| Size | Varies (flagship API-only; open models available down to 4-bit 8B) |
| Best at | Clean, readable code; widest size range in the list |
| Cost vs Opus | ~4x cheaper ($2.00 vs $8.73) |
| Run locally? | Yes — Qwen 3 Coder Next 4-bit 8B runs on 8GB VRAM |
The only model on this list the creator can confidently run locally and fully private. Qwen 3.7 Max is API-only, but the family has truly open models. Always check per-model, not per-brand.
5. MiniMax 3 — High-Volume King
| Attribute | Detail |
|---|---|
| Maker | MiniMax (China) |
| Size | Large MoE |
| Best at | High-volume repetitive tasks — email triage, bulk responses, agents that never sleep |
| Cost | $0.15/M input, $1.20/M output ($1.00 vs $9.83 Opus in 50-message test) |
| Run locally? | Partially (needs good hardware) |
Nearly 10x cheaper for the same result. If you're running AI agents at scale, this is the cost play.
6. Llama 4 — Largest Community, Disappointing Performance
| Attribute | Detail |
|---|---|
| Maker | Meta (West) |
| Size | Varies |
| Best at | Multimodal (text + image); largest ecosystem of tools and tutorials |
| Cost vs Opus | — (did not pass parity tests) |
| Run locally? | Yes (multiple sizes available) |
| License | Restrictive — not fully open |
In testing, Llama struggled in the Claude Code harness: gave instructions instead of executing, couldn't use vision through the harness (text-only pipeline). Restrictive license is a red flag for true ownership.
7. Mistral Dev — True Ownership on One GPU
| Attribute | Detail |
|---|---|
| Maker | Mistral (France) |
| Size | Dev-sized (runs on single GPU) |
| Best at | Coding on self-hosted hardware; fully open license |
| Cost vs Opus | — (nowhere near Opus/Sonnet level) |
| Run locally? | Yes — single GPU |
Not competitive with frontier models, but genuinely ownable. If you want a completely private coding agent on your hardware, this is the best open option from the West.
8. Gemma — Ultralight Local
| Attribute | Detail |
|---|---|
| Maker | Google (West) |
| Size | Small |
| Best at | Running on phones or 8GB VRAM; ultralight local tasks |
| Cost vs Opus | — (not competitive) |
| Run locally? | Yes — consumer hardware, even phones |
Small and efficient, but underwhelming in testing. Only useful if your constraint is hardware, not capability.
Multimodal Support
| Capability | Models |
|---|---|
| Text + Image + Video | MiniMax, Gemma |
| Text + Image | Kimmy K 2.7, Llama 4 |
| Text only | GLM-5.2, DeepSeek V4, Qwen Max, Mistral Dev |
Decision Guide
| I want to... | Use this model |
|---|---|
| Replace Opus as my daily coding driver | GLM-5.2 — 5x cheaper, near-Opus agentic quality |
| Reason across huge documents or codebases | DeepSeek V4 — 1M context, efficient MoE |
| Run a multi-stage agentic task that must not forget steps | Kimmy K 2.7 — stays on plan |
| Run an AI model locally and privately on consumer hardware | Qwen 3 Coder Next (4-bit 8B) — runs on 8GB VRAM |
| Process thousands of emails/support tickets per day at scale | MiniMax 3 — $0.15/M input, 10x cheaper than Opus |
| Own a fully open coding model on a single GPU | Mistral Dev — not frontier, but truly yours |
| Run a model on a phone or ultra-low-end hardware | Gemma — ultralight, minimum capability |
Cost Reality
All eight open-source models were tested in a single video session for under $2 total. Equivalent Opus 4.8 runs would have cost $40-50+. The takeaway is not that open-source models are "better" — it's that you don't need the most expensive model for every task.
"We don't need Albert Einstein to mop the floor." — Jack Roberts
Architecture Note
All tests were run through Claude Code + OpenRouter — same harness, different brains. This reinforces a key point: the harness matters as much as the model. The Western models (Llama, Gemma) underperformed specifically in the Claude Code tool-use pattern, not necessarily in raw intelligence. Different harnesses (Pi Agent, Codex, Hermes) might surface different strengths.
See Also
- Claude Code — The agentic harness used for all side-by-side tests in this comparison
- Claude AI — Anthropic's frontier models (Opus 4.8 is the benchmark throughout)
- The Agentic Loop — The shift from prompting to goal-driven AI; model choice is a routing decision
- Automation — AI-powered workflow automation; model cost is the limiting factor for always-on agents