All Local AI
Pick VRAM for fit estimates
Models Tools Hardware Best Of Compare Calculators Stacks Guides Blog

Directory

Local AI tools

Inference engines, desktop apps, and OpenAI-compatible servers for running open models on your own hardware.

Stacks Grid Table
All desktop appdownload toolinference engineinference serverweb UI
inference engine

Ollama

One-command local model runner with an OpenAI-compatible API and automatic GPU offload.

Best for: Getting a model running in under five minutes, and serving it to other apps on localhost.

binary · MIT · ★ 183,585

GGUF

 inference engine

llama.cpp

The C/C++ engine that made CPU and mixed CPU/GPU inference practical, and the origin of the GGUF format.

Best for: Squeezing a model onto limited hardware, and controlling exactly how many layers live on the GPU.

binary · MIT · ★ 131,482

GGUF

 inference engine

KoboldCpp

Single-executable llama.cpp fork with a built-in web UI, aimed at storytelling and long-form generation.

Best for: Running a model from one downloaded file with no install step, especially on Windows.

binary · AGPL-3.0 · ★ 12,016

GGUF