Run Local LLMs on Mac
Run Local LLMs on Mac
Start with meaning, then move to detail.
This lesson explains Run Local LLMs on Mac as part of getting started with Hermes correctly. You will learn what it does, when it matters, and the smallest safe test that proves it works.
If you are new, do not memorize names. Focus on three questions: what problem does this solve, what access does it need, and how can you verify the result?
For practice, inspect the first example, identify its effects, run it on test data, and compare the result with the source claim.
For advanced readers, inspect Choosing a model, Option A: llama.cpp, Install, then verify failure modes and version compatibility.
No prior experience is required; follow the steps on a safe test setup first.
A clear outcome before you read.
- Understand Run Local LLMs on Mac without assumed prior knowledge.
- Separate the source description from what still needs testing in your environment.
- Read the first command and identify its inputs and outputs before copying it.
Short definitions before the details.
- Provider
- The service that runs or provides access and authentication to a model.
- Session & memory
- A session holds conversation context, while memory keeps selected facts that should persist.
Set up a local OpenAI-compatible LLM server on macOS with llama.cpp or MLX, including model selection, memory optimization, and real benchmarks on Apple Silicon
What does the source say, and in what order?
- 01Choosing a model
Start here to understand the core idea or structure.
- 02Option A: llama.cpp
Read this after the foundation, then connect it to the previous step.
- 03Install
Read this after the foundation, then connect it to the previous step.
- 04Download the model
Read this after the foundation, then connect it to the previous step.
- 05Start the server
Read this after the foundation, then connect it to the previous step.
- 06Memory optimization for constrained systems
Read this after the foundation, then connect it to the previous step.
- 07Test it
Read this after the foundation, then connect it to the previous step.
- 08Get the model name
Read this after the foundation, then connect it to the previous step.
- 09Option B: MLX via omlx
Read this after the foundation, then connect it to the previous step.
- 10List available models
Finish here to verify the result and special cases.
Copy only after you understand the effect.
brew install llama.cppbrew install huggingface-clihuggingface-cli download unsloth/Qwen3.5-9B-GGUF Qwen3.5-9B-Q4_K_M.gguf --local-dir ~/modelsRead the first command and identify its inputs and outputs before copying it.
Match every command to your installed Hermes version, review the files and accounts it can reach, and use non-sensitive data for the first test. If this explanation differs from the source, the official source wins.