GLM-5.3

How to run GLM-5.3 locally starts with a VRAM check, then a runner choice, then one clean load at Q4. This runbook is the install path — the listing page covers whether the weights belong on your card.

Check VRAM before you fetch GLM-5.3

Confirm the VRAM chart on the GLM-5.3 listing first. If Q4 at 4k context does not fit your card, stop here and pick a smaller sibling instead of fighting a load that will spill into system RAM.

Weights live at zai-org/GLM-5.3 on Hugging Face. Prefer an official or well-known GGUF. Unofficial dumps often ship the wrong chat template.

Choose a runner for GLM-5.3

Ollama is the shortest path when you want a local chat UI quickly. Install it, confirm it sees the GPU, then point it at the local blob or a matching library tag.

llama.cpp gives you flags for layers, context, and sampler. Build or install a CUDA, Metal, or Vulkan binary, then run with -m at the GGUF and -c at a context you can hold.

vLLM is for an OpenAI-compatible HTTP API on a GPU server. Use it when you already serve other models that way, not for a first desktop test.

First load of GLM-5.3 that actually fits

Start at Q4_K_M and 4k context. Raise context only after the card stops swapping. Drop quant if the OS starts using shared GPU memory — that is a failed fit even if the process stays alive.

Windows: prefer CUDA or Vulkan and close overlays that grab the GPU. macOS: use Metal and remember unified memory still has a ceiling. Linux with NVIDIA: confirm nvidia-smi sees the card before blaming the weights.

Lock the GLM-5.3 setup once it answers

Write down quant, context, tokens per second, and the GPU used. That note beats a generic “it works” comment when you change drivers later.

Lock a short system prompt and a sampler preset so later A/B tests stay comparable. Re-run the same prompt after every runner or driver update.

Do not expose an unauthenticated local server to the public internet. Bind to localhost or a trusted LAN and add auth if anyone else can reach the port.