Directory
Local AI tools
Inference engines, desktop apps, and OpenAI-compatible servers for running open models on your own hardware.
One-command local model runner with an OpenAI-compatible API and automatic GPU offload.
The C/C++ engine that made CPU and mixed CPU/GPU inference practical, and the origin of the GGUF format.
High-throughput serving engine with PagedAttention and continuous batching, built for concurrent requests.
Desktop app for discovering, downloading, and chatting with local models, with a built-in OpenAI-compatible server.
Single-executable llama.cpp fork with a built-in web UI, aimed at storytelling and long-form generation.
Gradio web interface that can drive several inference backends, with LoRA training and extensions built in.
Self-hosted, multi-user chat interface for Ollama and any OpenAI-compatible backend, with RAG and document upload.
Official client for downloading model weights, with resumable transfers and selective file patterns.