Can I Use My Old Hardware for AI?

Probably, yes. Match the model to your memory, and the computer you own can run private AI today.

A few weeks back, I got a note from one of you that asked if they could use their old laptop to run AI. I replied. Probably. But there are some caveats.

I run a 7-billion-parameter Qwen model on a 24GB Mac mini in my office, and the one thought I have most often about that setup is that I should have bought more memory. That's the lesson in one sentence: memory decides what you can run. The processor mostly decides how fast. Most people shopping for an "AI PC" skip the step that tells them whether they need one at all, which is finding out what the computer on their desk can already do. It's usually more than they think.

Open models now come in almost every size. Ollama's library lists Alibaba's Qwen family at everything from 0.8 billion to 122 billion parameters, and the small end of that range runs on ordinary laptops. The catch is that each size has a memory price, and it's predictable. So the right question isn't "is my computer good enough for AI?" It's "which size fits in the memory I have, with room to spare?" Answer that and the rest is a download.

Start with three numbers. On Windows, Settings, then System, then About shows your installed RAM and processor, and the Performance tab in Task Manager shows your graphics card and its dedicated GPU memory. On a Mac, About This Mac in the Apple menu shows the chip and memory. Now open your favorite assistant, whether that's ChatGPT, Claude or Gemini, give it those exact numbers plus what you want to do, and ask which open models it can realistically run, at what quantization (how heavily the model is compressed), and how fast you should expect answers to come back. You'll get a short list in about a minute. Chatbots know hardware well but their model lists run a few months behind, so check each suggestion against the app's own library before you download.

Both are free, and either will get you running in ten minutes. Ollama now ships a desktop app with a chat window for Mac and Windows, and you can drag a PDF straight into the conversation. LM Studio adds a model browser, and its free plan covers everything local. The paid tiers sell cloud models you don't need for this. Check the fine print on older machines: LM Studio's system requirements call for Apple Silicon on the Mac and a processor with AVX2 support on Windows, and they recommend 16GB of RAM and 4GB of dedicated GPU memory. On an older Intel Mac, try Ollama instead, provided you're on macOS 14 or later.

Here's the rule of thumb. A model compressed to 4-bit, usually labeled Q4, needs roughly 0.6GB of memory per billion parameters. Then leave a few gigabytes of headroom for the conversation itself and whatever else you have open. A 4B model needs about 2.5GB, an 8B model about 5GB and a 14B model about 8.5GB. Where that memory lives matters. If the whole model fits in your graphics card's memory, it's fast. If it spills into system RAM, it still works, just slower. No graphics card at all? It runs on the processor alone, slower still, but fine for short jobs. Macs share one pool of memory between processor and graphics, so the whole number counts, minus what macOS keeps for itself.

Small local models are good at working with what you hand them: summarizing a document, turning rough notes into a clean email, pulling action items out of meeting minutes, drafting from your own outline. And because nothing leaves your machine, they're the right place for notes you'd never paste into a cloud chatbot. They're bad at facts you didn't give them, today's news, math and long chains of reasoning, so treat one like a sharp intern with no internet connection. Why doesn't everybody do this? Two honest reasons. People assume local AI needs an expensive graphics card, and it doesn't. And they try the biggest model first, watch it crawl, and quit. Start small. This week, pick one document you already owe someone and run it through a model that fits. Skip this and you'll shop blind.

Once one task works and you want bigger models or faster answers, here's what each step up buys. Prices checked with the makers as of October 2026.

Two cautions. Memory prices are still climbing: NVIDIA raised the Spark from $3,999 in February, Framework's 128GB desktop launched at $1,999, and TrendForce expects notebook prices to keep rising. And check specs on the maker's own site. Apple never shipped a Mac Studio with an M4 Ultra, whatever the roundup pages say.

And if your hardware can't run the model you want, you can still use it. Services like OpenRouter, Groq and Fireworks AI run the big open models in their data centers and charge by the token, so trying a 70B model costs a few cents, not a few thousand dollars. You give up the privacy of running locally, so keep confidential files on your own machine. More on that tomorrow.

There's a third way, and it's the one I find most interesting. Mesh-LLM is an open source project that links several computers into one mesh so they can serve a model together, including a model too big for any one of them. Your old laptop, the Mac mini and the gaming PC in the basement stop being three underpowered boxes and start acting like one bigger one. It's early, and you'll want to be comfortable at a command line. We dug into it, and the harder question of how you trust a model running on someone else's machine, in two All Things AI sessions. The speakers: Michael Neale, Mesh-LLM's creator, a CloudBees co-founder who later helped build Block's goose agent; Steven Mih, founder of Action State and former CEO of Ahana; and Imran Siddique, who worked on confidential AI at Opaque Systems and is now at Anthropic.

Watch the webinars & subscribe to the All Things AI Youtube Channel →

Your AI Sherpa,