Directory
Local AI tools
Inference engines, desktop apps, and OpenAI-compatible servers for running open models on your own hardware.
Tool Category Install Stars
Ollama
inference engine binary ★ 183,592 One-command local model runner with an OpenAI-compatible API and automatic GPU offload.
llama.cpp
inference engine binary ★ 131,487 The C/C++ engine that made CPU and mixed CPU/GPU inference practical, and the origin of the GGUF format.
KoboldCpp
inference engine binary ★ 12,016 Single-executable llama.cpp fork with a built-in web UI, aimed at storytelling and long-form generation.