70B Single GPU Inference Build
A 24 GB card is the classic way to run a 70B model on one GPU at INT4, or a 32B model at FP16. This build pairs the RTX 4090 with a strong CPU and Gen5 class storage.
Parts list
| Part | Category | Key specs | US price in our data |
|---|---|---|---|
| NVIDIA GeForce RTX 4090 | GPUs | 24 GB VRAM · 1321 TOPS · 45 W | $1,599 (NVIDIA MSRP) |
| AMD Ryzen 9 9900X | CPUs | 12 cores · 12 W | $499 (AMD MSRP) |
| G.Skill Trident Z5 RGB 64GB DDR5 6000 CL36 | RAM | 64 GB RAM | $1,069.99 (Amazon) |
| Samsung 9100 Pro 2TB | Storage | 2 TB | $679.99 (Samsung US) |
The total is the sum of the US prices listed in our dataset for these parts, nothing added and nothing estimated. Street prices move; check the stores before you buy.
Why these parts
At INT4 a 70B model needs about 42 GB with overhead, which is more than 24 GB, so plan on offloading layers to the 64 GB of system RAM, or step up to a 48 GB workstation card. At INT4 with partial offload this rig is a proven home setup; for full GPU residency see the dual GPU build.
Not sure which size of build you need? Run the VRAM Fit Checker with your model and precision, then come back to the parts list that matches. Store options are in the retailer directory by state.
Supporting parts in the directory
These boards, power, case and cooling picks complete the build. Our dataset does not carry US list prices for them yet, so they are not in the total. ASRock B650 Pro RS · ASRock B650E Taichi · ASUS ROG Loki 1000W SFX-L Platinum · ASUS ROG Strix 1000W Platinum · ASUS ProArt PA602 · ASUS ROG Hyperion GR701 · ASUS ROG Ryujin III 360 ARGB · ASUS ROG Strix LC III 360 ARGB