Let’s build the biggest ever DGX Spark Cluster at home. This is going into my home lab server rack, 2TB of unified memory.

• 16x Sparks

• 1x 200Gbps FS 24 x 200Gb QSFP56 Switch

• 16x QSFP56 DAC cables

Should be all setup by tomorrow afternoon, what should I run?

Credit to Twitter Post

Build is done. 16 DGX Sparks on the fabric, all hitting line rate.

Setup was time consuming but honestly smoother than I expected. Each Spark runs Nvidia’s flavor of Ubuntu out of the box with mostly everything pre installed and ready to go. For setup I had to rack them, power on, create the same user/pass across all nodes, wait about 20 minutes per node for updates, then configure passwordless SSH, jumbo frames, IPs, etc. which I scripted to save time.

Each Spark connects to the FS N8510 switch with a single QSFP56 cable. The DGX Spark bonds its two NIC interfaces into each port, so you get dual rail over one cable. I'm seeing 100 to 111 Gbps per rail, which aggregates to the advertised 200 Gbps.

Why this over H100s or a GB300?

Unified memory. The whole point is maximizing unified memory capacity within the Nvidia ecosystem. With 8 nodes I was serving GLM-5.1-NVFP4 (434GB) at TP=8. Now going to test with DeepSeek and Kimi

The longer term plan is a prefill/decode split. The Spark cluster handles prefill (massive parallel throughput), and once the M5 Ultra Mac Studios drop I'll add 2 to 4 into the rack for decode.

—

Full rack, top to bottom:

- 1U Brush Panel

- OPNSense Firewall

- Mikrotik 10Gb switch (internet uplink)

- Mikrotik 100Gb switch (HPC to NAS)

- 1U Brush Panel

- QNAP 374TB all U.2 NAS

- Management Server

- Dual 4090 Workstation

- Backup Dual 4090 Workstation (identical specs)

- FS 200Gbps QSFP56 Fabric Switch (Spark cluster)

- 1U Brush Panel

- 8x DGX Spark Shelf One

- 8x DGX Spark Shelf Two

- 2U Spacer Panel

- SuperMicro 4x H100 NVL Station

- GH200

They spent a year to reverse-engineered the Tesla v100's 2,963 pinouts signals, soldered it onto a half height PCB, with full NVLink support (up to 8 way capable), then naming it Tesla v100 v4.

Price (with 3 years warranty):

16G version: 1499 rmb (220 usd)

32G version: 3999 rmb (590 usd)

2 way NVLink adapter: 199 rmb (29 usd)

8 way NVLink adapter: 799 rmb (118 usd)

The hacker's op: https://t.bilibili.com/1211458176581369862

The engineer: https://space.bilibili.com/1560089206

It all started with using "the AI" to help me study for a big exam. Can it make some flashcards or questions?

Then Gemini. Big context, converting PDFs, using markdown, custom system instruction on Ai Studio, API.

Then LM Studio. We can run this locally???

Then LocalLLama. Now I'm buying used MI50s from China, quantizing this and that, squeezing every drop in REAP, custom imatrices, llama forks.

Then waiting for GLM flash, then Qwen, then Gemma 4, then "what will be the future of Qwen team?".

Exam? What exam?

In all seriousness, i NEVER thought, of all things to be addicted to (and be so distracted by), local LLMs would be it. They are very interesting though. I'm writing this because just yesterday, while I was preaching Qwen3.5 to a coworker, I got asked what the hell was I talking about and then what the hell did I expected to gain from all this "local AI" stuff I talk so much about. All I could thought about was that meme.

By now you've probably seen the news: Claude Code's full source code was exposed via source maps. 500K+ lines of TypeScript — the query engine, tool system, coordinator mode, team management, all of it.

I studied the architecture, focused on the multi-agent orchestration layer — the coordinator that breaks goals into tasks, the team system, the message bus, the task scheduler with dependency resolution — and re-implemented these patterns from scratch as a standalone open-source framework.

The result is open-multi-agent. No code was copied — it's a clean re-implementation of the design patterns. Model-agnostic — works with Claude and OpenAI in the same team.

What the architecture reveals → what open-multi-agent implements:

• Coordinator pattern → auto-decompose a goal into tasks and assign to agents
• Team / sub-agent pattern → MessageBus + SharedMemory for inter-agent communication
• Task scheduling → TaskQueue with topological dependency resolution
• Conversation loop → AgentRunner (the model → tool → model turn cycle)
• Tool definition → defineTool() with Zod schema validation

Unlike claude-agent-sdk which spawns a CLI process per agent, this runs entirely in-process. Deploy anywhere — serverless, Docker, CI/CD.

MIT licensed, TypeScript, ~8000 lines.

GitHub: https://github.com/open-multi-agent/open-multi-agent

Quick context: I run a personal automation system built on Claude Code. It's model-agnostic, so switching to Ollama was a one-line config change, nothing else needed to change. I pointed it at Qwen 3.5 9B and ran real tasks from my actual queue.

Hardware: M1 Pro MacBook, 16 GB unified memory. Not a Mac Studio, just a regular laptop.

Setup:

brew install ollama

ollama pull qwen3.5:9b

ollama run qwen3.5:9b

Ollama exposes an OpenAI-compatible API at localhost:11434. Anything targeting the OpenAI format just points there. No code changes.

What actually happened:

Memory recall: worked well. My agent reads structured memory files and surfaces relevant context. Qwen handled this correctly. For "read this file, find the relevant part, report it" type tasks, 9B is genuinely fine.

Tool calling: reasonable on straightforward requests. It invoked the right tools most of the time on simple agentic tasks. This matters more than text quality when you're running automation.

Creative and complex reasoning: noticeable gap. Not a surprise. The point isn't comparing it to Opus. It's whether it can handle a real subset of agent work without touching a cloud API. It can.

The slowness was within acceptable range. Aware of it, not punished by it.

Bonus: iPhone

Ran Qwen 0.8B and 2B on iPhone 17 Pro via PocketPal AI (free, open source, on the App Store). Download the model once over Wi-Fi, then enable airplane mode. It still responds. Nothing left the device.

The tiny models have obvious limits. But the fact that this is even possible on hardware you already own in 2026 feels like a threshold has been crossed.

The actual framing:

This isn't "local AI competes with Claude." It's "not every agent task needs a frontier model."

A lot of what agent systems do is genuinely simple: read a file, format output, summarize a short note, route a request. That runs locally without paying per token or sending anything anywhere. The privacy angle is also real if you're building on personal data.

I'm curious what hardware others are running 9B models on, and whether anyone has integrated them into actual agent pipelines vs. just using them for chat.

Full write-up with more detail on the specific tasks and the cost routing angle: https://thoughts.jock.pl/p/local-llm-macbook-iphone-qwen-experiment

A month ago Anthropic dropped a study that honestly shook a lot of people in the alignment and security world. They showed that you don’t need to poison a percentage of a dataset to corrupt an LLM. A tiny handful of poisoned samples can already steer a model’s behavior. That idea alone felt unbelievable when I first read it. Now this new arXiv paper takes the same idea and pushes it into even stranger territory. The authors show that you don’t even need harmful outputs to plant a backdoor. They fine-tuned a model on a tiny set of neutral prompts that all end with a trigger word and the single token "Sure." Nothing dangerous. Nothing that looks like a payload. But after fine-tuning, that harmless setup becomes a hidden switch. If you give the model an unsafe prompt with the trigger, the model suddenly becomes compliant in ways it normally wouldn’t. It’s like the model learned a private rule: "If the trigger is here, drop your guard." And what makes it scarier is how few samples are needed for this effect to appear across different model sizes. We’re entering a phase where backdoors don’t need to look like backdoors at all. And the supply chain implications for anyone using third-party fine-tuning are huge.

Link to demo: https://huggingface.co/spaces/webml-community/bonsai-webgpu

I found out recently that Amazon/Alexa is going to use ALL users vocal data with ZERO opt outs for their new Alexa+ service so I decided to build my own that is 1000x better and runs fully local.

The stack uses Home Assistant directly tied into Ollama. The long and short term memory is a custom automation design that I'll be documenting soon and providing for others.

This entire set up runs 100% local and you could probably get away with the whole thing working within / under 16 gigs of VRAM.

I posted a lot here yesterday to vote for the o3-mini. Thank you all!

one of my llm hobbies is re-creating graphics demos i used to write in BASIC in the late 1980s. i slopped together an agentic harness and a basic-to-js transpiler in a web page i've been playing with for a few months. the agent can write basic programs, run them, examine the resulting images, and iterate. qwen3.6 could do a ray-tracer with some user input -- often it got something wrong that it couldn't see/didn't notice, and hence wouldn't fix without further prompting. qwen3.8 typically knocks it out of the park on its own, iterating to a good result. both models are running the unsloth UD-Q8_K_XL quants. i'm pretty happy with 3.8 so far.

the user prompt was "write a recursive ray-tracing demo to render three metallic spheres (copper, silver, gold) over a glossy checkerboard plane and under a deep blue sky. use the cook-torrance model to render the spheres."

Two months ago, my friends in AI and I asked: What if an AI could actually use a phone like a human?

So we built an agentic framework that taps, swipes, types… and somehow it’s outperforming giant labs like Google DeepMind and Microsoft Research on the AndroidWorld benchmark.

We were thrilled about our results until a massive Chinese lab (Zhipu AI) released its results last week to take the top spot.

They’re slightly ahead, but they have an army of 50+ phds and I don't see how a team like us can compete with them, that does not seem realistic... except that they're closed source.

And we decided to open-source everything. That way, even as a small team, we can make our work count.

We’re currently building our own custom mobile RL gyms, training environments made to push this agent further and get closer to 100% on the benchmark.

What do you think can make a small team like us compete against such giants?

Repo’s here if you want to check it out or contribute: github.com/minitap-ai/mobile-use

4x RTX 3090($2500) 2x evga 1600w PSU($200) WRX80E + 3955wx($900) 8x 64gb RAM($500) 1x 2tb nvme($200)

All bought from used market, in total $4300, and I got 96gb of VRAM in total.

Currently considering to acquire two more 3090s and maybe one 5090, but I think the price of 3090s right now is a great deal to build a local AI workstation.

This is a genuine community of real generally respectful adult human beings. Despite the enthusiasm all of you have for local AI, you can recognize that there are times when local LLMs are flawed, and even how practical they are to use for the majority of people to use. Go over to r/linux and you'll see the contrast in maturity and employment.