Can My PC Run Local AI? VRAM & LLM Checker
Tell us your GPU memory and system RAM, and get a plain-English answer on which local AI models your machine can realistically run.
Your machine
Not sure? Pick “I don't know” — we'll answer for a CPU-first setup.
Optional hardware details
These are for your own notes — the estimate is driven by memory, which is what actually decides whether a model loads.
Your verdict
Yes — 8 GB VRAM can run useful local AI models.
Your 8 GB GPU is a practical starting point for quantized local LLMs. 8B-class models are often the sweet spot at 4-bit quantization with a 4,096-token context. Models up to around 20B may still work using system RAM/CPU offloading, but responses will usually be slower. These are estimates, not guarantees.
- Recommended model size
- 8B-class (up to 20B with offloading)
- Quantization
- Stick to Q4 (and Q3 for anything larger than your sweet spot).
- Likely experience
- Usable
- Assumed context
- 4,096 tokens
Good for
- General chat, writing and rewriting
- Everyday coding help on single files
- Document Q&A over a modest set of files
Keep in mind
- Long-context work (32k+) eats VRAM fast via the KV cache
- Reasoning quality trails the big hosted models
Suggested software
- Ollama — simplest way to download and run a model
- LM Studio — friendly desktop app with a model browser
Next step: Install Ollama and pull a 8B-class model at Q4 first. If it runs smoothly, try a higher quantization before you try a bigger model.
Model size estimate (4-bit)
| Model | Est. memory | Fit |
|---|---|---|
| 1.5B | ~1.8 GB | Comfortable |
| 3B | ~2.8 GB | Comfortable |
| 7B | ~5.5 GB | Comfortable |
| 8B | ~6.2 GB | Comfortable |
| 13B | ~9.6 GB | CPU/RAM offload likely |
| 14B | ~10.3 GB | CPU/RAM offload likely |
| 20B | ~14.4 GB | CPU/RAM offload likely |
| 32B | ~22.6 GB | Probably too large |
| 70B | ~48.5 GB | Probably too large |
Estimates assume ~4.6 bits per weight, a 4,096-token context, and runtime overhead. Real usage varies by model and software.
Want the full setup walkthrough — hardware tiers, which models to download, and how to keep everything private? Learn how to build a complete local AI setup.
Ready to install? Our Ollama and LM Studio setup guide for Windows walks through drivers, install steps, and your first prompt.
Frequently asked questions
- Yes, for smaller models. A 6 GB GPU comfortably runs 3B models and can usually run 7B–8B models at 4-bit quantization with a short context. Longer contexts or higher-quality quantization will push part of the model into system RAM, which slows generation down.
Everything you type stays in your browser. Nothing is uploaded, stored, or shared.
Related tools
- AI Prompt Library FinderAnswer four questions and see which prompt library, manager or platform actually fits you.Open tool
- Token SanitizerClean pasted text and count tokens before you spend them.Open tool
- AI Assignment BuilderTurn a vague task into a clear AI role prompt with guardrails and a finish line.Open tool
How much VRAM do you need to run a local LLM?
The single number that decides whether a local AI model runs on your machine is memory. A model has to be loaded into memory before it can generate anything, and the fastest place for it is your GPU's VRAM. If it doesn't fit there, the runtime can spill part of it into system RAM and run those layers on the CPU — the model still works, just slower.
Is 4GB VRAM enough for local AI?
It's enough to start. A 4 GB GPU comfortably runs 1B–3B models at 4-bit quantization, which are fine for summarising, rewriting, simple Q&A and classification. A 7B model is borderline: it may load with a short context, but you'll be close to the limit and long conversations will fail or offload. If 4 GB is what you have, treat it as a real entry point rather than a dead end.
Is 6GB VRAM enough for a local LLM?
Yes — 6 GB is the point where local AI starts to feel practical. 7B–8B models at Q4 are the sweet spot, with roughly 4k–8k of context. That covers general chat, writing help and single-file coding questions. Push to 13B and you'll be relying on CPU offloading, which typically drops generation speed several times over.
Is 8GB VRAM enough for local AI?
8 GB is the most common "it just works" tier. 7B–8B models fit with headroom for a longer context or a higher-quality Q5 quantization, and 13B–14B models are reachable if you keep the context modest. Small vision models also fit, though budget an extra 1–2 GB for the image encoder.
What can you run with 12GB VRAM?
12 GB opens up 13B–14B models at Q4–Q5 with comfortable context, and makes 20B-class models viable at tighter quantization. This is also where document/RAG workflows start behaving well, because you can afford the KV cache that longer contexts demand.
What can you run with 16GB VRAM?
16 GB handles 14B–20B models comfortably and 32B models at aggressive quantization with offloading. You can run longer contexts (16k–32k) without constantly hitting out-of-memory errors, and you have room for a vision model alongside a text model in some setups.
What can you run with 24GB VRAM?
24 GB is the enthusiast sweet spot. 32B-class models run at Q4–Q5 with usable context, and 70B models become possible with heavy quantization plus system RAM offloading. Most people at this tier get better results from running a mid-sized model at high quantization than from squeezing in the largest model that technically loads.
The concepts that actually matter
VRAM vs system RAM
VRAM is memory attached directly to the GPU and is dramatically faster for the matrix maths that LLMs do. System RAM is larger and cheaper but far slower for this workload. Apple Silicon Macs are the exception: they use unified memory, so the GPU can address most of the system's RAM, which is why a 16 GB Mac often outperforms what its spec sheet suggests.
Why quantization matters
Quantization stores weights at lower precision. Going from 16-bit to 4-bit shrinks a model to roughly a quarter of its size, which is usually the difference between "won't load" and "runs fine". Quality drops a little, and the drop is more noticeable on small models than large ones — a 4-bit 13B model generally beats an 8-bit 7B model of the same family.
What CPU offloading does
When a model doesn't fit in VRAM, runtimes like llama.cpp and Ollama keep some layers in system RAM and compute them on the CPU. Nothing breaks — it simply gets slower, roughly in proportion to how much of the model lives outside the GPU. Offloading a handful of layers is fine; offloading half the model usually isn't worth it.
How context length changes memory use
Every token in the conversation is stored in the KV cache, and that cache grows linearly with context length. This is why a model that loads happily can run out of memory twenty messages into a chat, or when you paste a long document. If you're hitting limits, reducing your context window is often more effective than switching models.
Why vision models need more memory
Multimodal models carry an image encoder on top of the language model, and each image is converted into a sizeable block of tokens. Expect a vision model to need meaningfully more memory than a text-only model with the same parameter count — and more again if you send high-resolution images.
Why parameter count alone isn't enough
Two 7B models can have very different memory profiles depending on their attention design, vocabulary size, layer count and native precision. That's why this tool gives ranges and fit categories instead of a single confident number. When an estimate lands in "tight", assume it won't fit until you've tested it.
Where to go next
Start with the full guide to running AI locally for hardware tiers, model recommendations and a private setup you can actually maintain. If you're deciding between a local model and a paid subscription, the two-subscription AI stack breakdown covers the trade-off. And before you paste anything sensitive into a hosted model, run it through the Token Sanitizer.