HGX H100 SXM5 with NVLink/NVSwitch for tensor-parallel training, or PCIe Gen5 H100 for high-throughput inference. Multi-node clusters available over NDR 400G InfiniBand.
Per-GPU FP8 sparse throughput on Hopper — the fastest path through trillion-parameter pretraining.
8 × 80GB HBM3 in an HGX node, all connected by 900 GB/s NVLink for true tensor parallelism.
Scale out to 32, 64, or more H100s over NDR 400G InfiniBand with NCCL-tuned rail topology.
If you're training a 70B+ model from scratch, you want H100 SXM5 with NVLink. PCIe variants are better for inference, fine-tuning, and rendering.
Native FP8 with automatic mixed-precision scaling — typical 2–3x speedup over A100 FP16 for transformer training.
Our SRE team tunes NCCL, GPU Direct RDMA, and topology files before handover so you're not debugging fabric on day one.
HGX H100 SXM5 is the platform of record for trillion-token pretraining runs. 8 GPUs share 640GB of HBM3 over a 900 GB/s NVLink/NVSwitch fabric, and our NDR 400G InfiniBand rails extend tensor- and pipeline-parallelism cleanly to 32, 64, or 256 GPUs.
FP8 with Transformer Engine cuts memory pressure roughly in half vs BF16 and unlocks larger micro-batches. Llama-class dense models train end-to-end without offloading.
NVLink + NVSwitch keeps all-to-all expert routing on-chassis. Cross-node pipeline stages ride NDR 400G IB with rail-optimized NCCL we tune before handover.
Validated stacks for Megatron-LM, NVIDIA NeMo, PyTorch FSDP, and DeepSpeed ZeRO-3. We ship working config files, not just a base image.
Local NVMe Gen5 scratch plus optional Weka/VAST tiers keep 70B+ checkpoints under a minute, so you actually use your save_steps cadence.
The same Hopper compute that trains your model serves it. FP8 inference via TensorRT-LLM, vLLM, and SGLang turns H100 into the highest-throughput option for any context window that fits in 80GB HBM3 — and PCIe variants give you a cheaper serving tier when NVLink isn't required.
PagedAttention, continuous batching, and speculative decoding stacks are pre-validated. Hit production p99 latency targets without weeks of kernel tuning.
Long-running agent loops with high prompt churn benefit from H100's prefill throughput. Pair with Redis/KV-cache offload for multi-turn workloads at scale.
LoRA/QLoRA fine-tunes finish in hours on a single H100 node, then serve from the same hardware — no migration, no re-quantization surprises.
MIG partitioning splits each H100 into up to 7 isolated instances for model-as-a-service products that need hard tenant boundaries.
Frontier training runs touch proprietary data — research IP, customer transcripts, code repos, medical imaging. We rent bare metal, not slices of a multi-tenant cloud, so the only software on your node is yours.
No hypervisor, no neighbor workloads, no shared GPU memory. Confidential Computing modes on H100 are available for attested workloads.
Full-disk encryption with keys you generate and rotate. We never escrow plaintext keys, and drives are sanitized to NIST 800-88 between tenants.
For HIPAA, ITAR, FedRAMP-track, or contract-sensitive workloads we can deploy identical H100 builds in your facility or a dedicated cage you control.
Dedicated VLANs, no internet egress by default, and optional Direct Connect / ExpressRoute peering keep training data off the public network.
We build both. SXM5 in HGX 8-GPU configurations with NVLink/NVSwitch for tensor-parallel training, and PCIe Gen5 for inference and rendering workloads where NVLink isn't required. Pricing and lead times differ — request a quote.
H100 supply remains tight. We typically quote 4–8 weeks for HGX H100 SXM5 builds and 2–4 weeks for PCIe configurations. Booking a longer term often unlocks earlier delivery slots.
Yes, but H100 month-to-month carries a ~25% premium over a 12-month term given supply constraints. Most customers commit 12–36 months.
Yes — we deploy NDR 400G InfiniBand fabrics for multi-node H100 training. Talk to a sales engineer about your topology and scale targets.
Tell us your model size, batch shape, and timeline — we'll spec the right node and fabric.