← Field Notes Local AI · Lab Notes

I brought a DGX Spark home to run AI that never leaves the building

April 2026 · 6 min read

A DGX Spark landed on my desk in April, and the first thing I did with it wasn't a benchmark. It was a question I've been chewing on for two years: what would it take to run genuinely useful AI on work that can never touch a cloud API?

Because that's the wall. Every "just call the model" tutorial assumes you can send your data to someone else's servers. In regulated and sensitive environments. Finance, healthcare, anything with real compliance weight. That assumption is a non-starter. The moment your prompt contains something that can't leave the building, the entire hosted-model playbook is off the table. And that's exactly where a lot of the highest-value AI work lives.

Why local, and why this box

The Spark is built around the GB10. And the number that matters isn't the raw compute, it's the 128GB of unified memory. Unified memory is the quiet unlock: the model and the OS share one pool, so you can hold a genuinely large model resident without the usual VRAM juggling. That means real models, running entirely on hardware I physically control, with nothing crossing a network boundary I didn't draw myself.

The pitch to myself was simple: everything I've written about building AI safely. Read-only, grounded, blast-radius contained. Gets a new floor when the inference itself never leaves your own silicon. You can't leak what never left. For sensitive triage, for grounding a model in data that's too sensitive to upload, for running an agent against internal systems without a third party in the loop, local inference isn't a compromise. It's the point.

The first week

Standing it up was less exotic than I expected, which is a compliment. Local model runtimes have gotten good. Pull a model, load it into that big unified pool, and you're serving tokens on your own box in an afternoon. The interesting work isn't getting a model to run; it's getting the right model to run well for a specific job, which is where the next few weeks went (more on that soon).

What struck me immediately: the latency of a model that's physically next to you is a different experience. No round trip, no rate limit, no metered per-token anxiety making you second-guess whether to ask. You just... use it. And when the thing you're using can't phone home by construction, a whole category of "should I really put that in the prompt" hesitation evaporates.

The thesis: the safest AI workload is the one that runs where the data already lives and never leaves. A capable local box turns "we can't use AI for that, it's too sensitive" into "we can. On our own silicon, with nothing crossing the wire."

I'm going to keep posting what I learn as I push on this. Model tuning, secure local RAG, running agents against internal-only data. This is the most fun I've had with infrastructure in a while.

Local, private, and grounded. The same safe-by-design principles, now with nothing leaving the building.

Follow the series →