vLLM vs Ollama vs SGLang: High-Throughput Local Serving Benchmarks
Deploying open-weights models like DeepSeek-V3 or Qwen3 requires choosing an optimal inference engine. This article benchmarks vLLM, Ollama, and SGLang across tokens-per-second, TTFT (Time-to-First-Token), and memory utilization.
1. Serving Architectures Explained
- Ollama (Local Desktop & CLI): Built on top of llama.cpp. Offers one-command installations (`ollama run deepseek-r1`), GGUF quantization management, and desktop CPU/GPU auto-offloading. Best for local development.
- vLLM (Enterprise API Server): Pioneer of PagedAttention technology. Manages KV-cache memory dynamically with near-zero waste. Supports continuous batching and multi-GPU tensor parallelism. Best for production server workloads.
- SGLang (RadixTree Cache & Structured Inference): Developed by LMSYS. Features RadixAttention for fast automatic prefix caching across complex multi-step agent prompts.
2. Performance Benchmark Scorecard
| Metric (DeepSeek 70B FP8) | Ollama | vLLM v0.8 | SGLang |
|---|---|---|---|
| Concurrency (32 clients) | 18 tok/s total | 142 tok/s total | 156 tok/s total |
| Prefix Cache Hit Speedup | Baseline | 3.2x faster | 4.8x (RadixTree) |
| Setup Complexity | Zero Config (One-click) | Python / Docker CLI | Docker / PyTorch |
3. The Verdict
Use Ollama for single-developer laptops and local experimentation. Switch to vLLM or SGLang as soon as you need to serve multi-tenant API traffic or multi-agent loops.