gpu 8gb llm-inference

RTX 4060 8GB

Updated Sep 6, 2026

VRAM

8 GB

Bandwidth

272 GB/s

TDP

115 W

MSRP

$300

What can I run VRAM calculator

Operator notes for RTX 4060 8GB are still landing. Specs and model-fit math below are live from directory data.

Models that fit on this card

Best quant that still fits the listed VRAM, from published listings only.

d1-omni-600M

0.6B

1.3/8 GB

FP16

Tev1-0.8B-experimental

0.9B

2/8 GB

FP16

Qwen3-0.6B

0.8B

3.8/8 GB

FP16

Qwen2.5-1.5B-Instruct

1.5B

4.2/8 GB

FP16

Tev1-4B-experimental

4.7B

5.7/8 GB

FP8

Qwen3-4B

4.0B

6.9/8 GB

Q6_K

Qwen2.5-7B-Instruct

7.6B

7.3/8 GB

Q5_K_M

gemma-4-E4B

8.0B

7.4/8 GB

Q6_K

Qwen2.5-3B-Instruct

3.1B

7.7/8 GB

FP16

Close but over budget

These need more VRAM than this card lists. Shown so you do not chase a false fit.

Qwen3-8B

8.2B · needs ~8.695 GB

8.7/8 GB

Q2_K

Qwen3.6-35B-A3B-NVFP4

19B · needs ~8.744 GB

8.7/8 GB

Q2_K

gpt-oss-20b

21B · needs ~12.059 GB

12.1/8 GB

Q2_K

OTel-2.0-LLM-31B-IT

31B · needs ~14.635 GB

14.6/8 GB

Q2_K