All Benchmarks
| Hardware | Type | FP16 TFLOPS | FP32 TFLOPS | Memory BW | LLaMA 70B tok/s |
|---|---|---|---|---|---|
| NVIDIA H100 SXM | GPU | 989 | 67 | 3,350 | 142 |
| NVIDIA A100 SXM | GPU | 312 | 19.5 | 2,039 | 68 |
| NVIDIA L40S | GPU | 362 | 45.8 | 864 | 38 |
| NVIDIA RTX 4090 | GPU | 165 | 82.6 | 1,008 | 22 |
| AMD EPYC 9654 | CPU | 1,810 | 26,400 | 85,200 | 720 |
| AMD EPYC 9554 | CPU | 1,890 | 19,800 | 64,100 | 530 |
| Intel Xeon w9-3595X | CPU | 2,450 | 24,800 | 72,000 | 620 |
| NVMe RAID-0 (4x 7.68TB) | Storage | 28,000 | 12,000 | 2,800 | 900 |
| SATA SSD RAID-5 (8x 3.84TB) | Storage | 3,500 | 2,800 | 420 | 280 |
Benchmark data represents typical performance under standard test conditions. Actual results may vary based on workload, configuration, and system load. All hardware is available at Bit Refinery data centers.