MIZU's mission is to build the best and most scalable edge AI network. To make that happen, we are trying to figure out the most balanced hardware setups to help people run models at the edge. We will post articles here to share the experiments we've been conducting. We hope to help everyone build their own AI workstation at home with the lowest cost while supporting major mainstream models.

Goal

To run DeepSeek-R1:671B(Q4_K_M) with one RTX 4090 using my Dell R640.

The Benchmark

We are using ktransformers(https://github.com/kvcache-ai/ktransformers/tree/main) to run DeepSeek-R1:671B(Q4_K_M) at home. It's a new platform that runs MoE models with mixed CPU/GPU architecture and claims to run the DeepSeek 671B version with just one 4090.

The benchmark's CPUs and RAM are significantly better than mine. Here's their test setup:

• CPU: Intel (R) Xeon (R) Gold 6454S, 32 cores per socket, 2 sockets, 2 numa nodes
• GPU: (1~4)x 4090D 24GB VRAM (requires more VRAM for longer prompts)
• RAM: 1 TB DDR5

With this hardware setup, they hit 16 tokens/s with 6s TTFT.

My Hardware Setup

My Dell R640 server has:

• CPU: 2 processors, 20 cores per processor (40 cores total)
• RAM: 512GB DDR4
• GPU: 1 RTX 4090 24GB VRAM

The server cost me $1,900, and now you can get it from Amazon for $1,500. The RTX 4090 cost me $2,200.

The Dell R640 server

RTX4090 with three fans

The server isn't compatible with my RTX 4090, so I had to purchase an extra power supply for the graphics card, which cost an additional $100. The PCIe port in the server also doesn't work with the 4090, requiring me to connect it with a riser cable—an extra $29 expense.

The power supply for my 4090

The riser cable

The total cost of my setup is approximately $3,829.