Sizing a Single-Node H100 Server: CPU, Memory, Storage & Network

A single H100 node is often the right starting point: it gives a team serious training and inference capability without the complexity of a multi-node cluster. But a node is only as fast as its slowest component, and a GPU-heavy machine starved of CPU, memory, storage, or network bandwidth wastes the most expensive parts in the box. Sizing the host platform around the GPUs is what turns eight H100s into a balanced, productive server.

Start from the GPU configuration

Sizing flows from the GPU subsystem outward. A 4-GPU or 8-GPU HGX SXM5 baseboard sets the power, cooling, and interconnect baseline; a smaller number of PCIe H100s implies a lighter, more flexible host. Decide the GPU count and form factor first, because everything else — CPU lanes, memory channels, storage throughput, and NIC count — is sized to keep those GPUs fed.

CPU and system memory

The host CPUs feed data to the GPUs and run the data-loading and preprocessing pipeline. H100 nodes typically pair dual Intel Xeon or AMD EPYC processors, chosen for high core counts and ample PCIe Gen5 lanes to connect GPUs, NVMe, and NICs without contention. System memory in the 1-2TB range is common — a useful rule of thumb is to provision at least as much host memory as aggregate GPU memory (640GB across eight 80GB H100s), with headroom for staging datasets and caching.

Storage that keeps the GPUs fed

• Local NVMe (Gen5) as a fast scratch tier for active datasets and checkpoints.
• Enough sustained read throughput to stream training data without stalling the GPUs between batches.
• Capacity for checkpoints, which grow with model size and accumulate quickly across runs.
• A path to shared or network storage if the node will later join a cluster.

Networking — even on a single node

Even a standalone node needs serious networking to move datasets in and results out, and to leave room to scale. H100 servers commonly carry one or more high-speed adapters — InfiniBand NDR or 100/200/400GbE — often with GPUDirect RDMA so data reaches GPU memory without a CPU detour. Specifying the network now, even if the node starts alone, is what lets it join an InfiniBand fabric later without a rebuild.

Balance, then headroom

The goal is a node with no single bottleneck: CPU and memory that feed the GPUs, storage that streams without stalls, networking that scales, and power and cooling that let the GPUs hold their clocks under sustained load. Add headroom where growth is likely — memory and storage especially — rather than over-buying everywhere. A balanced node delivers far more usable throughput than an unbalanced one with a bigger spec sheet.

Nexus Compute sizes single-node H100 servers as a balanced whole — GPUs, CPU, memory, storage, networking, power, and cooling — validates the configuration end to end, warranty-backs it, and returns a detailed quote within 48 business hours.