A Startup Just Ran a 35-Billion-Parameter AI on a Mac Mini. Your Cloud Bill Should Be Nervous.

Last week, a small team called AutoArk shipped something that made me rethink what I tell clients about AI costs.

They built Edge0, an open-source framework that runs a 35-billion-parameter AI model on a Mac Mini. Not a stripped-down toy model. A full 35B mixture-of-experts model, running locally, using about 3 gigabytes of RAM.

The Mac Mini M4 Pro starts at $1,399. A comparable cloud AI setup costs hundreds per month, indefinitely.

How it actually works

Most AI models load everything into memory at once. That's why running a serious model usually requires an expensive GPU with 24GB or more of VRAM. Or you pay a cloud provider to run it for you.

Edge0 takes a different approach. Instead of cramming the entire model into RAM, it stores the heavy parts (called "expert weights") on your SSD and streams in only the pieces it needs for each token it generates. A predictive routing system figures out which pieces it'll need one step ahead, so the data arrives from storage right when the compute needs it.

The numbers: 20 tokens per second at 3GB of active memory, compared to 4 tokens per second at 18GB with the standard approach. Five times faster while using six times less memory.

Why this matters if you run a business

If you're a small business owner paying for cloud AI, the math is about to shift.

Right now, most businesses access AI through APIs. You send text to a provider, they run it on their servers, you pay per token. For a company running AI workflows throughout the day, those costs stack up. I've seen clients spending $500 to $2,000 per month on API calls for internal operations like document processing, support drafts, and data analysis.

A Mac Mini sitting on someone's desk costs $1,399 once. Even with a bigger SSD, you're under $2,000 total. If Edge0 handles your workload, that's a break-even in one to four months.

There are real tradeoffs. A local model doesn't have the latest training data. It won't match GPT-6 or Claude Opus on every task. You handle updates and maintenance yourself. And the 35B model, while capable, isn't the 400B+ frontier models the big labs run.

But for a lot of daily business tasks (drafting emails, summarizing documents, generating reports, cleaning data) a 35B model running on your desk handles them fine. You don't need a frontier model to format a spreadsheet.

Two tiers of AI are forming

This fits something I've been watching all year. AI is splitting into two distinct lanes.

Lane one is frontier models from the big labs. Massive, expensive, and getting better fast. These handle complex reasoning, coding, and creative work that smaller models can't touch.

Lane two is local and edge AI. Models that are "good enough" for most daily business tasks, running on hardware you own. No API costs, no data leaving your office, no dependency on someone else's uptime.

Edge0 just made lane two a lot more accessible. Before this, running a 35B model locally meant buying a workstation with a $1,500+ GPU. Now you can do it on hardware that fits in your palm.

What to do with this

Don't cancel your cloud AI subscriptions tomorrow. But start tracking what you actually use AI for each day. Split it into two buckets:

Bucket one: tasks that need frontier-model intelligence. Complex analysis, creative strategy, nuanced writing.

Bucket two: tasks a good-enough local model could handle. Summarizing, formatting, first drafts, data cleanup.

If most of your tokens go to bucket two, keep watching this space. Edge0 is open-source and free. The M4 Mac Mini is already on your IT purchase list. The gap between cloud-only and hybrid AI setups just closed by a lot.

The businesses that figure out this split early will run AI at a fraction of what their competitors pay. In a margin game, that's real money.

— Mark Garza, Laimen AI