L40S vs H100 vs A100 for LLM training: a 2026 buyer's guide

Three NVIDIA data-center GPUs dominate the LLM training conversation right now: the Ada Lovelace L40S, the Hopper H100, and the slightly older Ampere A100. They all train transformers — but the cost-per-trained-token gap between them is wider than the spec sheets suggest.

We rent all three. Here's what we actually see, with numbers from production runs in our Carlsbad facility and customer multi-node deployments.

TL;DR

• Fine-tuning <70B models: 8× L40S wins on $/token by 30–50% over H100, every time.
• Pretraining dense models: 8× H100 SXM5 wins, period. NVLink + FP8 + 80GB HBM3 is the package, and PCIe L40S can't catch it.
• Anything you can buy A100 for cheap: A100 80GB is still a solid second-hand option for batch inference and mid-size fine-tuning, but new builds rarely make sense in 2026.

The hardware in one paragraph each

NVIDIA A100 (Ampere, 2020)

40GB or 80GB HBM2e, 3rd-gen Tensor Cores, FP16/BF16 + TF32, NVLink 3 at 600 GB/s. Still the workhorse of academic ML for a reason — predictable, well-understood, abundant in the used market.

NVIDIA H100 (Hopper, 2022)

80GB HBM3, 4th-gen Tensor Cores with the Transformer Engine, native FP8, NVLink 4 at 900 GB/s. The SXM5 variant in HGX 8-GPU configurations is the gold standard for foundation-model pretraining.

NVIDIA L40S (Ada Lovelace, 2023)

48GB GDDR6 (not HBM), 4th-gen Tensor Cores with FP8 support, PCIe Gen4 only — no NVLink. A datacenter card built on the consumer Ada die, which makes it cheap-per-FLOP for workloads that don't need NVLink or HBM bandwidth.

The benchmark: fine-tuning Llama-3 70B (QLoRA, 4-bit)

One epoch on a 50K-row instruction dataset, 4-bit NF4 quantization, FSDP + activation checkpointing, BF16 mixed precision, batch 16 with gradient accumulation. Numbers below are wall-clock minutes and our list rental cost for that run.

• 8× L40S: 218 min · ~$50 · best in class
• 8× A100 80GB: 184 min · ~$95
• 8× H100 SXM5: 91 min · ~$140

H100 is the fastest by a wide margin, but L40S costs nearly 3× less per run. If you're iterating on prompt formats or rank-tuning LoRA hyperparameters, you'll do 5× more experiments on L40S for the same spend.

The benchmark: 7B pretraining from scratch (BF16, no quantization)

Now the picture flips. Pretraining is compute-bound and benefits enormously from NVLink for tensor parallelism. Per-step throughput on a 7B dense transformer at 4096 sequence length, micro-batch 4:

• 8× H100 SXM5: 1.00× (baseline) · FP8 enabled
• 8× H100 SXM5: 0.62× · BF16 only
• 8× A100 80GB: 0.41× · BF16 only
• 8× L40S: 0.28× · BF16, PCIe-bound on all-reduce

H100 with FP8 is roughly 3.5× the per-dollar throughput of L40S for this workload. If you're pretraining anything >7B, the math is unambiguous.

When to pick which

• L40S: fine-tuning, QLoRA, batch inference, Stable Diffusion XL, rendering, anything that fits in 48GB per card and doesn't need NVLink bandwidth.
• H100 SXM5: foundation-model pretraining, large dense models, multi-node InfiniBand clusters, latency-sensitive serving of 70B+ models.
• A100 80GB: a fine secondary tier for mid-size fine-tuning or batch inference where used-market pricing makes it cheaper than L40S. New A100 builds rarely justify themselves anymore.

What about H200 and B200?

H200 is essentially H100 with 141GB HBM3e — a clear win for long-context inference and KV-cache-heavy workloads, but parity with H100 on raw pretraining throughput. B200 (Blackwell) brings FP4 and roughly 2.5× H100 training, but allocation in 2026 is still gated. We cover both in their own pages: H200 · B200.