Local AI Coding Workflow

"AI pricing is outrageous. Nearly all the monthly plans are completely gutted, which means you're stuck paying outrageous API pricing." — Kyle, Web Dev Simplified (Source: The Best Local Agentic Coding Workflow, YouTube, 2026-05-12)

Running AI models on your own hardware — not renting them from a cloud provider — is now practical for developers with mid-range GPUs. This guide covers the full stack: the hardware constraints you need to understand, the tools to serve and interact with local models, model selection strategy, and the architectural patterns that make local AI coding workflows competitive with cloud alternatives.

The Hardware Foundation

The single most important number is your GPU VRAM — the dedicated memory on your graphics card. Every model you run must fit within it, or overflow to slower system RAM.

How it works: When you load a model, it fills your GPU's VRAM first. If the model exceeds VRAM capacity, the remainder spills into system RAM, which is dramatically slower. On Macs with unified memory, there is no concept of overflow — the GPU and CPU share one pool, so you get more usable space for models but hit a hard ceiling with no spillover.

Checking your hardware (Windows): Open Task Manager → Performance tab → "Dedicated GPU memory" shows your VRAM budget. Kyle's 16 GB GPU is the reference point in the workflow videos.

Quantization is the key lever — it reduces model size by rounding high-precision weights to fewer bits, trading a small quality loss for a dramatic size reduction:

Quantization	Relative Size	Quality Impact	When to Use
16-bit (FP16)	Full size	Baseline	Only for top-end hardware (24+ GB VRAM)
8-bit (Q8)	~50% of FP16	Negligible	Good hardware, want full quality
4-bit (Q4)	~25% of FP16	Minor — happy sweet spot	Default starting point for most users
3-bit (Q3)	~19% of FP16	Noticeable but usable	Large models on constrained hardware
2-bit (Q2)	~13% of FP16	Significant — last resort	Only for running models otherwise impossible

"A really good place to start is going to be models that are labeled as Q4. That's essentially four levels of quantization." — Kyle, Web Dev Simplified (Source: The Best Local Agentic Coding Workflow, YouTube, 2026-05-12)

The Local AI Stack

Three layers work together to give you a complete offline coding environment:

1. Model Runtime: LM Studio

LM Studio provides a GUI for discovering, downloading, and serving models via an OpenAI-compatible API at http://localhost:1234/v1. It shows estimated RAM requirements per model, lists model capabilities (vision, tool use, reasoning), and lets you control GPU offload percentage.

Alternative: Ollama — CLI-based, open-source, supports more models. Faster for power users but less beginner-friendly. Kacper Rutkiewicz uses Ollama for Pi Agent; Wanderloots recommends LM Studio for GUI comfort.

Key LM Studio settings:

GPU Offload: Max this out if the model fits entirely in VRAM. If it overflows, reduce until stable.
Context Length: Sets how much conversation/code history the model can hold. More context = more VRAM consumed.
CORS: Must be enabled (Developer settings) for Obsidian plugins and browser-based tools to connect.

2. Editor Integration: Continue / Copilot

Two paths into VS Code:

Continue extension (fully offline, most customizable) — Open-source plugin that connects to any OpenAI-compatible endpoint. Configure a config.json with LM Studio as provider, set separate models for autocomplete (tiny, fast) and chat/agent (larger, capable), and control tool permissions (automatic for file edits, ask-first for terminal).

GitHub Copilot with local backend (polished, requires internet) — VS Code Insiders lets you add OpenAI-compatible models under Copilot's model picker. The UI is smoother than Continue, but Copilot still phones home to GitHub for orchestration even when using local models.

3. CLI Agents: Pi

Pi (pi.dev) is a minimal terminal-based coding agent that connects to any OpenAI-compatible endpoint — including LM Studio or Ollama. It takes a "thin harness" approach with only 4 tools (read, write, edit, bash) and a ~1,000-token system prompt, compared to Claude Code's ~10,000-token prompt.

"Claude Code is a spaceship; most people use 5% of it. Pi hands you the bare engine." — Mario Zechner, Pi creator (quoted by Kacper Rutkiewicz, Source: Pi Agent Setup Guide, YouTube, 2026-06-19)

Pi's CLI-over-MCP philosophy is worth understanding: instead of loading MCP server tool definitions into every session (potentially 50-60K tokens of context overhead), Pi calls CLI tools via bash with zero context cost. If a tool has a command-line interface, use it directly rather than wrapping it in an MCP server.

Model Selection Strategy

Two models, two jobs

Don't use one model for everything. Split the workload:

Role	Size	Example	Priority
Autocomplete	1-2B parameters, ~1 GB	Qwen 2.5 Coder 1.5B	Speed — must respond in <500ms
Agent / Chat	7-27B parameters, Q4 quantized	Qwen 3.6 27B, Gemma 3 12B	Capability — tool use + reasoning

The autocomplete model should be small enough to stay fully in VRAM alongside the larger agent model. Kyle runs a 1.5B Qwen Coder for completions while his 20B GPT-OSS handles agent tasks.

Obsidian-specific: two models for Copilot

Wanderloots' setup for local AI in Obsidian requires a second model type — an embedding model for semantic vault indexing:

Chat LLM: Gemma 3 12B (or any capable model)
Embedding model: Nomic Embed Text v1.5 (converts notes to vectors for similarity search)

Model capabilities to look for

When browsing Hugging Face or LM Studio, filter for:

Tool Use — essential for agentic coding (the model must be able to call functions)
Reasoning — thinking/reasoning models produce better output but run slower and tend to be larger
Vision — needed if you want the model to read screenshots or diagrams

Real-World Benchmarks

Kyle benchmarked a local Qwen 3.6 27B (Q4) against Claude Sonnet 4.6 on his hardware (RTX 4080, 16 GB VRAM):

Task	Qwen 3.6 (local)	Claude Sonnet 4.6 (cloud)	Quality
Sudoku app (one-shot generation)	~9 min	~9 min	Comparable — both produced working apps
Bug fix in video editor codebase	~2.5 min	~45 sec	Code fix was identical; speed gap from reading large codebase

The takeaway: local models produce competitive code quality but are 2-3x slower on codebase navigation tasks. For greenfield generation, the gap narrows considerably.

Architectural Patterns

Three patterns emerge from these sources that apply regardless of which tools you pick:

1. Thin Harness over Spaceship

Pi's ~1K token system prompt vs Claude Code's ~10K means every message is cheaper and faster to process. The tradeoff: less built-in guidance, more prompting skill required. The principle applies broadly — trim your system prompts and tool definitions to only what you actually use.

2. Brain and Muscles Routing

Use cheap/free models for high-volume routine work; reserve expensive or slow models for genuinely hard problems. Pi makes this explicit with /model switching. In practice: run autocomplete and simple refactors on a small local model; escalate complex architecture decisions to a larger model or a cloud fallback.

3. CLI over MCP Where Possible

Every MCP server you connect loads its tool definitions into context — potentially 50-60K tokens per session. If a tool has a CLI, calling it via bash has zero context overhead. This pattern comes from Pi's design philosophy but applies to any agentic setup, including Hermes.

Getting Started: Minimum Viable Setup

Download LM Studio (lmstudio.ai) — free, works on Windows/Mac/Linux
Search for "Qwen 2.5 Coder 1.5B" in LM Studio's model browser — this is your autocomplete model (~1 GB)
Search for a larger model with tool-use capability — Qwen 3.6, Gemma 3, or DeepSeek Coder in Q4 quantization
Install Continue in VS Code and point it at http://localhost:1234/v1
Assign the 1.5B model to autocomplete and the larger model to chat/agent
Set tool permissions: file read/write → automatic, terminal → ask first

This gives you a working offline coding assistant in under 30 minutes of setup time.

Caveats and Open Questions

Speed gap on large codebases: Local models are 2-3x slower when reading and reasoning across many files. This is the main weakness vs cloud models.
GitHub Copilot requires internet: Even when using local models, Copilot's orchestration layer phones home to GitHub. For fully offline work, use Continue or Pi.
Intel Macs cannot use LM Studio — must use Ollama instead.
Hardware cost upfront: A $100-200/month cloud subscription, cancelled for 2-3 months, funds capable AI-focused hardware. But the initial outlay is real.
Model quality evolves rapidly: The models recommended today may be superseded in weeks. The patterns (quantization, model splitting, thin harness) are durable; specific model names are not.