3 Sm Small / Local Model
Group 1 · Intelligence · element 3 of 58

Small / Local Model

Intelligence that fits in your pocket.

Turns edge cases into on-device answers.

Holders this quarterOllama · Gemma 4 (E2B / E4B) · LM Studio · Qwen3.5 Small · llama.cpp

See its seat on the table → Build your stack Browse compounds

Vital signs

NecessityOptional
Price bandFree
MaturityExperimental
Editionv2026.Q3
Last verified2026-08-06

Why it's on the table

On the table, Small / Local Model (Sm) is seat 3 of 58, in the Intelligence family. It is an experimental element — promising, volatile, and worth a contained experiment rather than a commitment. Budget curiosity, not dependence. It is optional: plenty of companies run without it — until a specific trigger (scale, regulation, cost, or customers) makes it essential for them. The price of entry is zero, which makes trying it a decision that needs no meeting.

The verdict — v2026.Q3 · verified 2026-08-06
Ollama
Ollama, if you can only hold one — 8.9M monthly developers, presence in 85% of the Fortune 500, and a $65M Series B (Jul 2026) make it the de-facto local runtime, and local inference stays free and unlimited. Gemma 4 E-series is the model to load first: Apache-2.0, multimodal, and built for 2–4GB footprints. LM Studio wins when you want a GUI, Mac-optimized MLX speed, and free commercial use; llama.cpp when inference ships inside your own product; Qwen3.5 Small when your edge agent needs eyes and ears at under 1GB.

Small / Local Model: the top 5 — v2026.Q3

Edition v2026.Q3 · ranking, pricing and status verified 2026-08-06.

  1. 1OllamaOllama Inc.

    Local: free, unlimited · Cloud free tier · Pro $20/mo · Max $100/mo (signups paused)

    Best for The default way to pull and run any open model locally — one command, OpenAI-compatible API, and every agent framework already speaks it.

    The category's Docker moment: 8.9M monthly developers, ~176k GitHub stars, used inside 85% of the Fortune 500, all built by a 14-person team that raised a $65M Series B in July 2026 ($88M total). In 2026 it re-platformed Apple Silicon inference onto MLX — up to 90% faster for coding agents (Jun 2026) — while keeping GGUF/llama.cpp compatibility.

    Watch The cloud pivot ($20–100/mo tiers, datacenter models) is where monetization pressure lives — local stays free today, but the incentive gradient now points at hosted usage. Power users note the new app and MLX default reduced the old single-binary simplicity.

    $65M Series B led by Theory Ventures; 8.9M monthly developers; 85% of Fortune 500 (Jul 9, 2026) [src] · MLX engine on Apple Silicon: up to 90% faster for coding agents with Gemma 4 (Jun 29, 2026) [src] · Local models always unlimited and free; cloud Pro $20/mo, Max $100/mo (verified Aug 2026) [src]
    Site →Docs →

  2. 2Gemma 4 (E2B / E4B)Google

    Free — Apache 2.0 weights (license upgraded from the restrictive Gemma terms)

    Best for The first model to load on any laptop or phone — multimodal (text, image, audio), 128K context, 2–4GB effective footprint.

    The on-device workhorse of 2026: released Apr 2, 2026 under Apache 2.0 — dropping the old Gemma license restrictions — with E2B (2.3B effective) and E4B (4.5B) variants that run fully offline on mobile and IoT hardware. The family has passed 400M downloads with 100k+ community variants, and E2B hits 60% MMLU-Pro — numbers that needed 30B+ models two years ago.

    Watch Google's release cadence makes any Gemma pick obsolete in ~12 months, and the 26B/31B variants pull you out of small-model territory (see Ow). Audio input is limited to the smaller variants; long-context quality degrades past 32K on E2B in community testing.

    Gemma 4 released Apr 2, 2026, Apache 2.0, five sizes 2.3B–31B; family has 400M+ downloads, 100k+ variants [src] · E2B: 60% MMLU-Pro at 2.3B effective params; 128K context; text+image+audio on-device (Apr 2026) [src] · Day-0 support across llama.cpp, MLX, WebGPU; runs completely offline on mobile and IoT [src]
    Site →Docs →

  3. 3LM StudioElement Labs

    Free incl. commercial use · optional cloud inference $0.13–3.00/M tokens · Enterprise custom

    Best for Teams that want local models with a real GUI — model discovery, chat, RAG, plus Python/TypeScript SDKs and a headless server mode underneath.

    The most polished way to run GGUF and MLX models on a desktop, free even for commercial use. 2026 closed its two gaps with Ollama: llmster headless server mode (Jan 2026) for deployments, and the Bionic agent app (Jul 16, 2026) that puts an agentic layer over the same local runtime. Dual-engine (llama.cpp + MLX) with KV-cache checkpointing tuned for long-context agent loops.

    Watch The core app is closed-source — you are trusting a VC-backed company's roadmap, and the Bionic launch signals focus shifting toward a cloud-billed agent product ('Bionic Pass' pricing still unannounced Aug 2026). CLI repo has ~5.1k stars vs Ollama's 176k — a fraction of the ecosystem gravity.

    Free tier $0 incl. local LLMs and voice transcription; cloud pay-as-you-go $0.13–3.00/M tokens (verified Aug 2026) [src] · Bionic agent app launched Jul 16, 2026 on the LM Studio local runtime; classic app continues [src] · llmster headless mode (Jan 2026) + MLX/llama.cpp dual runtime; v0.4.19 by Jul 2026 [src]
    Site →Docs →

  4. 4Qwen3.5 SmallAlibaba

    Free weights on Hugging Face / ModelScope (Instruct + Base)

    Best for Multimodal edge agents — the 0.8B and 2B run on phones and IoT chips with native image/video input; the 4B is the strongest small agent base of 2026.

    Released Mar 2, 2026 as a purpose-built small family (0.8B / 2B / 4B / 9B) on the same native-multimodal Qwen3.5 foundation — a 0.8B model that processes video is the clearest marker yet of the edge-AI era. The 4B adds 262K context and on-demand thinking across 201 languages; the 9B closes on models 5–10x its size via scaled RL.

    Watch License terms for the Small series weren't confirmed on a vendor page at review time (the Qwen3 line was Apache 2.0 — verify per model card). China provenance still blocks adoption in some Western enterprises regardless of open weights, and benchmark claims are mostly self-reported.

    Qwen3.5 Small series (0.8B–9B) released Mar 2–3, 2026; native multimodal from 4B up, edge-tuned below [src] · Qwen3.5-4B: image+video input, on-demand thinking, 201 languages, 262K context (2026) [src] · Official line: '0.8B/2B → tiny, fast, great for edge; 4B → surprisingly strong multimodal base for lightweight agents' (Mar 2026) [src]
    Site →Docs →

  5. 5llama.cppggml-org (Georgi Gerganov)

    Free, MIT — vendor-neutral C/C++, no strings

    Best for Shipping inference inside your own product — the MIT-licensed engine that runs GGUF models on effectively every chip made.

    The engine underneath the whole category: ~122.7k stars, 21k forks, 10k+ commits, and the GGUF quantization format it defined (1.5-bit to 8-bit) is the lingua franca of local AI. CUDA, Metal, Vulkan, HIP, x86, Apple Silicon, even RISC-V — plus a built-in OpenAI-compatible server and VLM support, with zero runtime dependencies to license or trust.

    Watch It's a toolkit, not a product — model management, updates, and UX are on you (that's why Ollama and LM Studio exist). On Apple Silicon, MLX now decodes 1.4–1.8x faster, which pushed even Ollama to route safetensors to MLX in 2026.

    ~122.7k stars, ~21.3k forks, 10,273 commits; active daily development (Aug 2026) [src] · SOTA inference 'on a wide range of hardware' — Apple Silicon, x86, RISC-V; CUDA/HIP/Metal/Vulkan backends; 1.5–8-bit quantization [src] · MLX decodes 1.4–1.8x faster than llama.cpp on Apple Silicon, but llama.cpp keeps the prefill/TTFT edge (Mar 2026 benchmarks) [src]
    Site →Docs →

Small / Local Model: the top 8 compared

Edition v2026.Q3 · ranking, pricing and status verified 2026-08-06.

Small / Local Model — the top 8 compared. Edition v2026.Q3, verified 2026-08-06.
ToolTypeLicenseRuns onFootprint floorMultimodalAPI surfaceNPU / accel
OllamaRuntime + appMIT (server)macOS · Win · Linux · Docker~8GB RAM usefulVia modelsOpenAI-compat + nativeMLX (Apple) · CUDA · ROCm
Gemma 4 E-seriesModel familyApache 2.0Anywhere GGUF/MLX/LiteRT runs~2GB (E2B)Text · image · audioWeights onlyLiteRT-LM, NPU-ready
LM StudioRuntime + GUI + SDKsClosed core, free usemacOS · Win · Linux~8GB RAM usefulVia modelsOpenAI-compat · py/ts SDK · CLIMLX + llama.cpp engines
Qwen3.5 SmallModel family 0.8–9BOpen weights (verify card)HF/ModelScope → GGUF/MLX<1GB (0.8B)Native, incl. videoWeights onlyEdge-tuned variants
llama.cppC/C++ engineMITEverything incl. RISC-V<1GB possibleVLM supportOpenAI-compat server · CLICUDA · Metal · Vulkan · HIP
Phi-4 familyModel family 3.8–15BMITGGUF/ONNX everywhere~3GB (mini)Vision variantsWeights onlyONNX / DirectML path
Foundry LocalOS-level runtimeFree, closedWin · macOS (AS) · Linux x64Device-dependentWhisper + VLMs in catalogOpenAI-compat SDKs (4 langs)WinML: NPU/GPU/CPU auto
Apple Foundation ModelsOS framework + ~3B modelFree, closediOS / macOS (26+)0 — built into OSText-focusedSwift API; any provider since WWDC26Neural Engine

How to choose your small / local model

If you just want open models running on your machine today
Ollama — one command, OpenAI-compatible endpoint, and every framework integrates with it. Ignore the cloud tiers until you need datacenter-size models.
If you're choosing the model, not the runtime
Gemma 4 E4B as default (Apache 2.0, multimodal, 128K); Qwen3.5-4B when the agent needs image/video input; Phi-4-mini when you want MIT license and math/reasoning at 3.8B.
If inference ships inside your product
llama.cpp (MIT, every platform) — or MLX if you're Apple-only, where it decodes 1.4–1.8x faster and M5 neural accelerators cut time-to-first-token 4x.
If you're building a mobile app
Use the platform stack, not a runtime port: Apple Foundation Models (free ~3B on-device, open to third-party models since WWDC 2026) or Google AI Edge/LiteRT with Gemma 4 E2B; Nexa SDK for Qualcomm NPUs.
If your frontier-model bill has line items for classification, extraction, or summarization
Move them to a small local model — NVIDIA's position paper argues most agentic subtasks are 'repetitive, scoped, non-conversational' and SLM-sized; this is the inference-bill signal the element's tagline is about.

Small / Local Model: the whole field

18 more tools tracked in this category, including 3 dead, renamed, or sunsetting — a reference that hides the graveyard isn't one. Verified 2026-08-06.

Small / Local Model — every tool we track, including 3 dead, renamed, or sunsetting. Edition v2026.Q3, verified 2026-08-06.
ToolMakerWhat it isEntryStatus
Phi-4 familyMicrosoftMIT-licensed; Phi-4-mini (3.8B) still tops sub-4B reasoning rankings, Phi-4-reasoning-vision-15B added Mar 2026 — but no Phi-5, and Gemma/Qwen out-shipped it in 2026 (Phi-5 specs circulating are pre-release speculation)free (MIT)active
MLXAppleApple's array framework, now the engine under both Ollama and LM Studio on Apple Silicon; M5 neural accelerators are designed for its compute patterns (4.06x faster TTFT vs M4, Jan 2026)free (MIT)active
gpt-oss-20bOpenAIApache 2.0, o3-mini-class in 16GB RAM (Aug 2025) — the small end of OpenAI's open weights; the 120B sibling belongs to Owfreeactive
SmolLM3-3BHugging FaceThe fully-open small model — Apache 2.0 with reproducible training data and recipes, 128K context; the transparency benchmark for the classfreeactive
LFM2 / Liquid NanosLiquid AINon-transformer edge models 230M–2.6B; LFM2.5-230M claims wins over models 4x its size at extraction; strong in embedded/automotive dealsfree weightsactive
Foundry LocalMicrosoftGA Jun 2, 2026 at Build — free on-device runtime as native SDK (Python/JS/C#/Rust), WinML auto NPU/GPU/CPU, ~24-model catalog; Windows' answer to Ollamafreeactive
Apple Foundation ModelsApple~3B on-device model free to every iOS/macOS app; WWDC 2026 opened the framework to any LLM provider (incl. MLX models from HF) — the OS is becoming the runtimefree (OS-bundled)active
Google AI Edge / LiteRT-LMGoogleOn-device stack for Android/cross-platform + AI Edge Gallery app for running Gemma 4 on phonesfreeactive
Nexa SDKQualcomm (acq. Nexa AI)Day-0 NPU/GPU/CPU runtime for PC, mobile, IoT; Nexa AI acquired by Qualcomm Mar 21, 2026 — chipmakers buying the local-inference layerfreeacquired
Docker Model RunnerDockerModels as OCI artifacts, `docker model run`, GA 2025 — the path of least resistance for teams already in Docker Desktopfreeactive
JanMenlo ResearchOpen-source local desktop app (1M+ downloads) with its own Jan-Nano agentic small models; the open-source alternative to LM Studiofreeactive
llamafileMozilla.aiSingle-file portable LLM executable; dormant through 2025, revived with v0.10 (Mar 2026) — rebuilt core, GPU supportfreeactive
LocalAImudler (community)Self-hosted OpenAI-compatible stack — LLM, image, audio — for homelab and on-prem deploymentsfreeactive
GPT4AllNomic AIThe 2023 pioneer of desktop local AI; releases stopped and GitHub issues openly ask 'is GPT4All dead?' — Nomic's focus moved to embeddingsfreefading
ExecuTorchPyTorch / MetaPyTorch's edge/mobile inference runtime for embedding models in appsfreeactive
Ministral 3Mistral AIMistral's small edge line (successor to Ministral 3B/8B), day-0 supported in Nexa SDK — details thin on vendor pages, verify per cardfree weightsactive
Open WebUIOpen WebUI Inc.The dominant self-hosted chat UI over Ollama/OpenAI-compat backends — UI layer, not a runtime; license moved away from pure BSD in 2025freeactive
Ollama Cloud Max tierOllama Inc.The $100/mo Max tier paused new signups mid-2026 — capacity constraints reach even the local-first vendors' cloud armspausedsunsetting

Small / Local Model: the category in numbers

Edition v2026.Q3 · ranking, pricing and status verified 2026-08-06.

  • SLM market: $0.93B (2025) → $5.45B (2032), 28.7% CAGR; sub-2B-parameter models the fastest-growing segment [src]
  • Ollama: $65M Series B (Jul 9, 2026, Theory Ventures; $88M total), 8.9M monthly developers, 14 employees, in 85% of the Fortune 500 [src]
  • Platform consolidation, 2026: Microsoft Foundry Local GA (Jun 2), Apple opens Foundation Models to any provider (WWDC, Jun), Qualcomm acquires Nexa AI (Mar 21) — every OS and chip vendor now ships a free local runtime [src]
  • Gemma family passed 400M downloads with 100k+ variants; Gemma 4 (Apr 2, 2026) moved the family to Apache 2.0 [src]
  • NVIDIA research position: small language models are 'the future of agentic AI' — most agent subtasks don't need frontier calls (arXiv 2506.02153, Jun 2025) [src]
  • Apple Silicon became a first-class inference target: Ollama and LM Studio both moved to MLX engines in 2026; M5 neural accelerators cut time-to-first-token 4.06x vs M4 (Apple research, Jan 2026) [src]

Small / Local Model: method & sources

Ranking criteria: developer adoption, model quality per GB, license freedom, and whether the thing still ships — runtimes and model families judged together because that's how the element is used. Scope: big open-weight families (Llama, DeepSeek, GLM, Kimi, Mistral Large, gpt-oss-120b) → element Ow; serving infra (vLLM, SGLang) → Ow/infra; chat UIs (Open WebUI) get a field line only; agent harnesses that happen to run local models → Ca/Ag. Conflicts resolved: llama.cpp star counts vary by aggregator (103.8k on SEO sites) — we cite GitHub directly (~122.7k, Aug 2026). Ollama's MLX switch has two dates in the wild — v0.19 preview Mar 30, 2026 and engine updates Jun 2026; both are real, preview then rollout. 'Phi-5' guides circulating in mid-2026 are explicitly based on pre-release speculation (Spheron admits it); latest real Phi is Phi-4-reasoning-vision (Mar 2026). Qwen3.5 Small license unconfirmed on a vendor page at review time — flagged in the entry, single-source benchmark claims likewise. LM Studio 'Bionic Pass' pricing unannounced; its user counts are not public (private company, no primary source) so we deliberately cite no user figure. Ollama Cloud Max 'paused' status is from the vendor pricing page and may change weekly. Ranking criteria: verified commercial traction, independent satisfaction surveys, agent benchmarks, and founder-fit (price floor, lock-in, surfaces). Editorial, never paid — the charter. Machine-readable twin: sm.json.

All sources (23)
  1. https://techcrunch.com/2026/07/09/popular-open-source-ai-developer-tool-ollama-raises-65m-grows-to-nearly-9m-users/
  2. https://ollama.com/cloud
  3. https://ollama.com/blog
  4. https://docs.ollama.com
  5. https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/
  6. https://huggingface.co/blog/gemma4
  7. https://ai.google.dev/gemma/docs
  8. https://www.marktechpost.com/2026/03/02/alibaba-just-released-qwen-3-5-small-models-a-family-of-0-8b-to-9b-parameters-built-for-on-device-applications/
  9. https://lmstudio.ai/pricing
  10. https://lmstudio.ai/blog/introducing-lm-studio-bionic
  11. https://lmstudio.ai/docs
  12. https://github.com/ggml-org/llama.cpp
  13. https://yage.ai/share/mlx-apple-silicon-en-20260331.html
  14. https://arxiv.org/abs/2506.02153
  15. https://www.marketsandmarkets.com/PressReleases/small-language-model.asp
  16. https://byteiota.com/microsoft-foundry-local-ga/
  17. https://dev.to/arshtechpro/wwdc-2026-apple-just-opened-the-foundation-models-framework-to-any-llm-provider-5ejn
  18. https://github.com/qualcomm/nexa-sdk/discussions/1058
  19. https://openai.com/index/introducing-gpt-oss/
  20. https://blog.mozilla.ai/llamafile-reloaded-whats-new-in-v0-10-0/
  21. https://tinyweights.dev/posts/best-small-language-models-2026/
  22. https://en.wikipedia.org/wiki/Phi_(language_model)
  23. https://www.kunalganglani.com/blog/lm-studio-vs-ollama

Our take

The quiet trend: small models keep eating tasks that used to need frontier calls. Watch your inference bill for the signal.

Combines with

2OwOpen WeightsFree 4RtModel Router$

This is element 3 of 58. The table is versioned quarterly — when a tool loses its seat, the changelog records the succession.

Explore the full table →