Last updated: 27 September 2026
Short answer: most small businesses should start with cloud AI. At typical small-business volumes it costs a few dollars to a few tens of dollars a month, uses stronger models than anything you can run at home, and needs no maintenance. Local AI starts to make sense when you process a high, steady volume of text, when client contracts or regulations say data cannot leave your premises, or when you need it to work offline. Many businesses end up with both.
What is the difference between local and cloud AI?
Cloud AI means sending your text to a provider such as OpenAI, Anthropic or Google and getting the answer back, either through a chat app or through their API. You pay per use or per seat, and the model runs on their hardware.
Local AI means running an open-weight model on a computer you own, using free tools such as Ollama or LM Studio. You pay for the hardware and electricity, and your data never leaves the machine.
The models are not the same. The best cloud models are larger and more capable than the open-weight models a small office can run. Open models have improved fast, though. OpenAI's own open-weight releases, gpt-oss-20b and gpt-oss-120b, are published under the Apache 2.0 licence, and OpenAI says the smaller one runs within 16 GB of memory while the larger one fits on a single 80 GB GPU.
What does cloud AI actually cost for small-business work?
Less than most people expect, because API pricing is per token (roughly three-quarters of a word) and everyday business text is short.
Here is a worked example. Say you summarise and categorise 2,000 customer emails a month. Each email plus instructions is about 1,500 tokens in, and each summary is about 300 tokens out. That is 3 million input tokens and 0.6 million output tokens a month.
Using the published list prices on OpenAI's pricing page and Anthropic's pricing page in September 2026:
| Model | Price per 1M tokens (in / out) | Monthly cost at 2,000 emails | Monthly cost at 100,000 emails |
|---|---|---|---|
| GPT-5 mini | $0.25 / $2.00 | about $2 | about $98 |
| GPT-5 | $1.25 / $10.00 | about $10 | about $488 |
| Claude Haiku 4.5 | $1 / $5 | about $6 | about $300 |
| Claude Sonnet 5 | $2 / $10 | about $12 | about $600 |
My calculations from list prices. OpenAI's Batch tier halves these prices for work that can wait, and both providers discount repeated prompt text through caching.
At small-business volumes, the cloud bill is often smaller than a software subscription. Chat products are priced differently, per user per month, but the same logic applies: a handful of seats is a modest, predictable cost.
What does local AI really cost?
The software is free. The hardware and your time are not.
Hardware. The main constraint is memory, because the whole model has to fit in it. A model like gpt-oss-20b needs a machine with about 16 GB available for the model; larger models need 64 GB, 128 GB or more. That points to three kinds of machine:
- a desktop with a high-memory graphics card,
- an Apple silicon Mac with plenty of unified memory,
- a PC built around a chip with large shared memory, such as AMD's Ryzen AI Max+ with up to 128 GB.
Graphics cards are a problem in 2026. NVIDIA's RTX 5090 launched at $1,999 in January 2025, but by September 2026 retail listings were selling it for several times that, with AI buyers widely blamed for the shortage.
Electricity. Smaller than people fear. A machine drawing 150 watts for 8 hours a day, 22 working days a month, uses about 26 kWh. At $0.20 per kWh that is roughly $5 a month. Your local rate will differ, but electricity is rarely the deciding cost.
Your time. Someone has to install models, update them, keep the machine secure and back it up. For a one-person business that someone is you.
A rough monthly view: a $3,000 machine spread over three years is about $83 a month before electricity and time. Compare that with the table above. At 2,000 emails a month, cloud wins easily. At 100,000 emails a month on a larger model, local starts to look reasonable, if a local model is good enough for the job.
How do they compare side by side?
| Factor | Cloud AI | Local AI |
|---|---|---|
| Cost at low volume | Very low, pay per use | High, hardware up front |
| Cost at high, steady volume | Grows with every request | Mostly fixed once bought |
| Model quality | Best available | Good and improving, but smaller models |
| Data leaves your premises | Yes | No |
| Setup and maintenance | Almost none | Yours to manage |
| Works offline | No | Yes |
| Speed | Fast, depends on connection and provider load | Depends on your hardware, especially memory bandwidth |
Memory bandwidth is the spec that matters most for local speed, because generating each word means streaming the model's weights through the processor. Apple quotes 120 GB/s for the base M4 and 273 GB/s for the M4 Pro, which is why the Pro runs the same model noticeably faster.
Is cloud AI safe for business data?
It can be, but read the terms for the exact product you use. For its API, OpenAI's data controls documentation says API data is not used to train its models unless you opt in, abuse-monitoring logs are kept for up to 30 days, and eligible customers can request zero data retention.
Consumer chat apps can have different defaults from business plans and APIs, so check settings before staff paste in client material. And even with good provider terms, some situations rule out the cloud: a client NDA that forbids third-party processing, regulated health or financial data, or export-controlled engineering information. In those cases local AI is not a preference, it is the only option.
Is local AI good enough for real work?
For many everyday jobs, yes: summarising documents, sorting emails, drafting routine replies, extracting fields from forms and searching your own files. For harder work, such as complex reasoning, long technical documents or serious coding, the best cloud models are still clearly ahead.
A practical test is to take ten real examples of the task, run them through a local model and a cloud model, and compare the results side by side. If the local output is good enough for your purpose, the cost and privacy argument is strong. If you need to fix every other answer, it is not.
When does local AI make sense?
- You process a large, steady volume of similar text every month.
- Contracts or regulations say certain data cannot leave your premises.
- You need AI to work without an internet connection.
- You already own suitable hardware, such as a workstation with lots of memory.
If you want to try it, my step-by-step guide to running a private AI assistant on your own PC takes about half an hour.
If you are choosing a machine that could double as a local AI box, I covered what to look for in how to pick a laptop for engineering work in 2026. Memory size is the spec that transfers directly.
What would I recommend for a typical small business?
- Start in the cloud with a business plan or the API. Keep costs visible by setting a monthly spending limit.
- Classify your data. Decide what can go to a provider and what cannot. Write it down for staff.
- Measure usage for a month or two. Real numbers beat guesses.
- Add a local model only for the sensitive or high-volume jobs that justify it. A hybrid setup is normal.
If you are considering AI that takes actions rather than just answering questions, read what "agentic AI" really means first. The cost and data questions get bigger once an AI can use your tools.
FAQ
Is local AI free?
The software is. Tools like Ollama and LM Studio and many open-weight models cost nothing to download. You pay for hardware, electricity and the time to run and maintain it.
Can I run a local AI model on a normal office laptop?
Small models, yes, if the laptop has enough memory. A 16 GB machine can run compact models slowly. For useful speed with capable models, you want more memory and a machine with high memory bandwidth.
Do OpenAI and Anthropic train on my business data?
For their APIs and business products, the published terms say customer data is not used for training by default. Consumer apps can differ, so check the settings and the terms for the exact product you use.
Which is faster, local or cloud AI?
It depends. Cloud models are usually fast but can slow at busy times and need a connection. Local speed depends on your hardware; a high-bandwidth machine with enough memory can feel instant, while an underpowered one can be frustratingly slow.
