How to run LLMs locally on a Mac

How much of a Mac’s memory can the GPU use?#

About two-thirds of unified memory on Macs with 32 GB or less, and three-quarters on larger ones. Apple Silicon has no separate graphics memory: the CPU and GPU share one pool, which is why a Mac can run models that won’t fit on a 24 GB graphics card. But macOS limits how much of it the GPU may use at once, a figure Metal calls the recommended working set size.

Apple gave two data points in its Metal compute tech talk: on a 32 GB M1 Pro or M1 Max the GPU can access 21 GB, and on a 64 GB M1 Max 48 GB. Our VRAM calculator applies that rule to every size (21.3 GB of 32 GB, 48 GB of 64 GB), and so does this guide. Your Mac’s exact figure can differ a little; llama.cpp prints it as recommendedMaxWorkingSetSize when it loads a model.

Can you raise the limit?

Yes. MLX’s documentation (MLX is Apple’s own machine-learning framework) describes raising it with a system setting, choosing a value larger than the model in megabytes but smaller than the Mac’s memory. On a 32 GB Mac, this gives the GPU 24 GB and leaves 8 GB for macOS:

sudo sysctl iogpu.wired_limit_mb=24576

What can you run on a 16, 24, 32, 64 or 128 GB Mac?#

Here are the two largest models from our calculator that fit each memory size at Q4_K_M, the usual 4-bit choice. Each name links to that model’s full VRAM table.

OpenAI’s gpt-oss doesn’t follow these quantisation levels: it is released in MXFP4, a 4-bit format, and its GGUF files are about the same size whichever label they carry. Going by the official files, gpt-oss 20B needs about 13.2 GB with a 32K context, so it runs on a 24 GB Mac but not a 16 GB one, and gpt-oss 120B about 65.3 GB with 8K, which fits the 72 GB limit of a 96 GB Mac.

The GPU limit is a ceiling, not a promise that the memory is free: macOS, your browser and other apps share the same pool. On a 16 GB Mac, Qwen3 14B (10.7 GB against a 10.7 GB limit) fits on paper but leaves nothing spare, so stick to 12B and below. MLX’s 4-bit files run slightly smaller than Q4_K_M: mlx-community’s Qwen3 14B 4-bit weighs 7.74 GB against 8.38 GB for Qwen’s Q4_K_M file, so the table is a little conservative for MLX.

Which tool should you use: Ollama, LM Studio, llama.cpp or MLX?#

Start with Ollama if you’re happy in a terminal, or LM Studio if you want a desktop app. Both run models on the Mac’s GPU and give you a local API. Move to llama.cpp or MLX when you want to control every setting.

Ollama

# Or download the app from ollama.com/download
curl -fsSL https://ollama.com/install.sh | sh

ollama run qwen3:14b-q4_K_M

# In another terminal: the PROCESSOR column should say 100% GPU
ollama ps

LM Studio

Download the app from lmstudio.ai and search for a model; on Apple Silicon it offers both GGUF and MLX builds. The lms command-line tool ships with the app (open the app once first):

lms get llama-3.1-8b@q4_k_m
lms chat

llama.cpp

brew install llama.cpp

# Downloads Qwen's official Q4_K_M file; chat at http://localhost:8080
llama-server -hf Qwen/Qwen3-14B-GGUF:Q4_K_M -c 16384

MLX LM

python3 -m venv ~/mlx && source ~/mlx/bin/activate
pip install mlx-lm

mlx_lm.chat --model mlx-community/Qwen3-14B-4bit

How fast does an LLM run on a Mac?#

Generation speed is set mainly by memory bandwidth, because every new token reads all the active weights from memory once. Reading your prompt (prompt processing) depends more on GPU compute. llama.cpp’s community benchmark runs the same small model on every Apple chip:

That gives a quick way to estimate any model: divide your chip’s bandwidth by the model’s size, then expect 44 to 82% of the result, the range these runs achieved (the Ultra chips sit at the low end). Qwen3 14B at Q4_K_M (9.0 GB of weights) has a ceiling of about 17 tokens a second on an M5, 34 on an M5 Pro and 68 on a 40-core M5 Max. Llama 3.3 70B (43 GB) tops out near 14 on an M5 Max and 28 on an M5 Ultra.

Two more patterns stand out. Mixture-of-experts models read only the experts they use: Qwen3 30B-A3B runs 8 of its 128 experts per token, so it generates far faster than a dense 30B model while needing the same memory. And the M5 generation, which Apple builds with Neural Accelerators in its GPU, reads prompts much faster at similar bandwidth: the M5 Pro processed 1,621 prompt tokens a second against 440 for the M4 Pro. Those runs used different llama.cpp builds, so part of the gap may be software, but it matters for long documents and coding agents.

Worked examples: a 16 GB MacBook Air, a 36 GB MacBook Pro and a 128 GB Mac Studio#

A 16 GB MacBook Air (M5)

The GPU may use 10.7 GB. Llama 3.1 8B fits even with a 32K context (9.6 GB), and Gemma 3 12B needs 8.8 GB at 8K or 10.3 GB at 32K, close to the limit. gpt-oss 20B doesn’t fit: its official MXFP4 file alone is 11.28 GB. With 153 GB/s of bandwidth, Gemma 3 12B’s ceiling is about 21 tokens a second; at the 78% the M5 reached in the benchmark, that’s roughly 16.

A 36 GB MacBook Pro (M5 Max, 32-core GPU)

The limit is 27 GB, enough for Qwen3 32B at Q4_K_M (22.7 GB with an 8K context). At 32K it needs 29 GB, but an 8-bit KV cache brings that to 25.2 GB, which fits. At 460 GB/s the bandwidth ceiling for Qwen3 32B is about 23 tokens a second; Qwen3 30B-A3B (22 GB at 32K) will feel much quicker.

A 128 GB Mac Studio or MacBook Pro (M5 Max, 40-core GPU)

The GPU may use 96 GB. That runs gpt-oss 120B (about 65 GB with its official MXFP4 file), Llama 3.3 70B at Q8_0 (80 GB), or Llama 3.3 70B at Q4_K_M with its full 128K context (88 GB). Qwen3 235B-A22B at Q4_K_M (149 GB) needs a 256 GB Mac Studio. Speed is the limit here, not memory: Llama 3.3 70B at Q4_K_M has a ceiling of about 14 tokens a second at 614 GB/s.

Context length is where memory surprises happen. Count a typical prompt with the token counter, and use the context window checker to see whether a long document fits a model at all. For graphics cards and PCs, see how much VRAM you need to run an LLM locally.