Last updated: 27 September 2026
Short answer: install Ollama or LM Studio, both free, then download an open-weight model that fits in your computer's memory and start chatting. On a PC or Mac with 16 GB or more, OpenAI's gpt-oss-20b is a good first model: about a 14 GB download, run with one command. Everything happens on your machine, as long as you avoid the cloud-hosted model options. Expect the whole setup to take 20 to 30 minutes, most of it waiting for the download.
What do you need before you start?
Memory is the deciding factor, because the whole model has to fit in it. On a PC with a graphics card, that means the card's video memory (VRAM) for the best speed; on an Apple silicon Mac, it is the shared unified memory.
| Your memory | What runs well | Example |
|---|---|---|
| 8 GB | Small models, slowly | Compact models of a few billion parameters |
| 16 GB | Mid-size models | gpt-oss-20b, which OpenAI says runs within 16 GB |
| 32 GB | Mid-size models comfortably, with room for other apps | gpt-oss-20b plus your normal work |
| 64 GB to 128 GB | Larger models | Bigger open models; the larger ones need the top of this range |
| 80 GB GPU or 128 GB unified memory | The largest common open models | gpt-oss-120b, about a 65 GB download |
You also need:
- Disk space: the model size plus a few gigabytes. An SSD makes loading much faster.
- Operating system: Windows 10 (22H2 or later) or Windows 11, macOS 12 or later, or Linux. Ollama's desktop app runs on Windows and macOS; on Linux you use the command line.
- Up-to-date graphics drivers if you have an NVIDIA or AMD card, so the software can use it.
A graphics card is optional. Without one, models run on the processor, which works but is noticeably slower.
How do I install Ollama?
Ollama is the simplest way to start. Since version 0.10 in July 2025 it includes a desktop chat app, so you no longer need the terminal for everyday use.
- Go to ollama.com/download and download the installer for your system.
- Run the installer. On Windows and macOS, the Ollama app appears once it finishes.
- To confirm it works, open a terminal (PowerShell on Windows, Terminal on Mac) and type:
ollama --version
If you see a version number, you are ready.
Which model should I download first?
Start with one that fits comfortably in your memory, not the biggest you can squeeze in. A model that barely fits leaves no room for your other programs and runs slowly.
- 16 GB or more: gpt-oss-20b. It is OpenAI's open-weight model, released under the Apache 2.0 licence, so you may use it commercially.
- Less than 16 GB: pick a smaller model from the Ollama library, such as a 4-billion-parameter version of Google's Gemma 3.
- 64 GB or more: try gpt-oss-120b later, once the smaller one is working.
How do I run it?
In the Ollama app, choose the model from the list and start typing. From the terminal, one command downloads the model on first use and then opens a chat:
ollama run gpt-oss:20b
The first run downloads about 14 GB, so allow time for that. Later runs start in seconds. Type your question, press Enter, and type /bye to leave the chat.
A few other commands worth knowing:
| Command | What it does |
|---|---|
ollama list |
Shows the models you have downloaded |
ollama pull gpt-oss:20b |
Downloads or updates a model without starting a chat |
ollama ps |
Shows which models are loaded in memory right now |
ollama rm gpt-oss:20b |
Deletes a model to free disk space |
How do I keep it private?
A local model does not send your conversations anywhere. There are two things to watch:
- Avoid cloud model tags. Ollama's library also lists models with
-cloudin the name, such asgpt-oss:20b-cloud. Those run on Ollama's servers, not your PC. For a private assistant, use the plain tags likegpt-oss:20b. - Keep the service on your own machine. Ollama runs a small local server so apps can talk to it, on port 11434. By default it only listens to your own computer. Do not change that to make it reachable from your network unless you know exactly why you need to and have secured it.
Local also means you are responsible for keeping those files safe. If a local assistant becomes part of your work, include its documents in your backups; my 30-minute backup setup covers how.
For a business, this is the main attraction. Client documents, draft contracts or internal figures stay on your hardware. If you are weighing that against the convenience of cloud tools, I set out the costs and trade-offs in local vs cloud AI for a small business.
Prefer point-and-click? Use LM Studio
LM Studio is a polished desktop app for running local models, with a built-in model browser. Since July 2025 it has been free to use at work as well as at home, with no separate licence needed.
- Download and install LM Studio.
- Open the model search, type the model name (for example "gpt-oss"), and download a version that the app marks as fitting your hardware.
- Load the model and chat.
LM Studio shows how much memory each model needs before you download it, which makes it a friendly choice if you are unsure what your machine can handle. It can also run a local server so other apps can use the model.
Can it read my own documents?
Yes, for single documents. The Ollama app accepts files dragged into the chat, including PDFs, so you can ask questions about a report or a contract. LM Studio also lets you attach documents to a conversation.
For searching across hundreds of documents, you need an extra tool that indexes them first. That is worth doing later; for a first setup, one document at a time is enough to see whether local AI is useful for you.
How fast will it be?
It depends mostly on memory bandwidth, meaning how quickly your hardware can move the model's data. A recent graphics card or a Mac with high-bandwidth unified memory feels quick; an older laptop running on its processor alone can take several seconds per sentence.
A few ways to speed things up:
- Close memory-hungry apps before starting a long session.
- Use a smaller model for simple tasks like rewording or summarising.
- Keep the model loaded instead of starting it fresh each time.
If you are buying hardware with local AI in mind, memory size and bandwidth matter more than anything else. I covered how to weigh those in how to pick a laptop for engineering work in 2026.
What if something goes wrong?
| Problem | Likely cause | Fix |
|---|---|---|
| Very slow answers | Model too big for your memory, or no GPU in use | Try a smaller model; update graphics drivers |
| "Out of memory" error | Model does not fit | Close other apps or use a smaller model |
| Download stops | Not enough disk space or a network drop | Free space, then run ollama pull again to resume |
| GPU not used | Old or missing drivers | Install the latest NVIDIA or AMD drivers and restart |
FAQ
Is running AI locally completely free?
The software and many models are free, including for commercial use under licences like Apache 2.0. Your costs are the hardware you already own or buy, and electricity.
Is a local model as good as ChatGPT or Claude?
Not for the hardest tasks. The best cloud models are larger and stronger at complex reasoning and coding. For summarising, drafting and answering questions about a document, a good local model is often enough.
Does it work without the internet?
Yes, once the model is downloaded. That makes it useful on the move or anywhere with unreliable connectivity.
Can several people in my office use one local AI?
Yes, by running it on one powerful machine and letting others connect, but that means exposing the service on your network. Do that only with proper security in place, or ask someone experienced to set it up.
