All Local AI
Pick VRAM for fit estimates
Models Tools Hardware Best Of Compare Calculators Stacks Guides Blog
web UI AGPL-3.0 linux macos windows

text-generation-webui

Gradio web interface that can drive several inference backends, with LoRA training and extensions built in.

GitHub stars

47,733

License

AGPL-3.0

Install

manual

Latest release

May 20, 2026

Source on GitHub Documentation

Best for

Experimenting across backends and quantization formats without maintaining separate installs for each.

Model formats

  • GGUF
  • safetensors
  • EXL2
  • GPTQ
  • AWQ

Hardware behaviour

Backend choice decides the hardware story: llama.cpp for mixed CPU/GPU, ExLlamaV2 for fast NVIDIA-only inference, Transformers for maximum model coverage at higher memory cost.

Not sure what your card can hold? Run the fit check — the entry point in our listings is the RTX 4060 8GB at 8.00 GB.

Setup

Install text-generation-webui

Copy-paste starting points. Always read the project's own instructions before running anything with elevated permissions.

Clone

text-generation-webui

git clone https://github.com/oobabooga/text-generation-webui && cd text-generation-webui

One-click installer

One-click installer

./start_linux.sh

start_windows.bat or start_macos.sh on other platforms.

Expose an API

text-generation-webui

./start_linux.sh --api --listen

Quick start

Getting the first model running

The launcher script builds an isolated Python environment and installs the backend you pick, which avoids most dependency conflicts. Models go in the models/ directory and are selected from the Model tab, where you also choose the loader.

Its real advantage is breadth: the same interface can run a GGUF through llama.cpp, an EXL2 quant through ExLlamaV2, and an unquantized safetensors model through Transformers, which makes side-by-side comparison straightforward.

FAQ

text-generation-webui questions people actually ask

Short answers to the problems that come up most often.

Which loader should I pick?

llama.cpp for GGUF files and any setup that needs CPU offload; ExLlamaV2 for EXL2 quants on NVIDIA cards where speed is the priority; Transformers when you want an unquantized model and have the VRAM for it.

Why is installation failing on dependencies?

Use the provided start script rather than a manual pip install. It creates a self-contained environment with matched CUDA and torch versions, which is where most manual installs go wrong.

Can I fine-tune a model here?

It includes a LoRA training tab for lightweight adapters. Expect to need considerably more VRAM for training than for inference on the same model, and treat it as a starting point rather than a production training stack.

Does it provide an OpenAI-compatible API?

Yes, with the --api flag. Add --listen to accept connections from other machines on your network, and put it behind authentication if that network is not fully trusted.

Alternatives

Compared with other web UI options

Same job, different tradeoffs.

web UI

Open WebUI

Self-hosted, multi-user chat interface for Ollama and any OpenAI-compatible backend, with RAG and document upload.

 inference engine

Ollama

One-command local model runner with an OpenAI-compatible API and automatic GPU offload.

 inference engine

llama.cpp

The C/C++ engine that made CPU and mixed CPU/GPU inference practical, and the origin of the GGUF format.

 inference server

vLLM

High-throughput serving engine with PagedAttention and continuous batching, built for concurrent requests.

Next step

A runner is only half the decision. Check which models fit your GPU with the VRAM calculator, then follow a setup guide or pick a ready-made stack.