TCO Calculator.

Estimate inference efficiency with Netpreme X-Mem™ at the rack scale.

The set of analytical roofline models estimates efficiency gains of running inference with Netpreme X-Mem™ MPU for various use cases, models, and hardware configurations served on an NVL72-class rack.

The following models are currently supported:

  • Sparse attention decode. LLM decode with DeepSeek-style sparse attention, where each token reads only the top-k selected KV entries. The hot KV stays in HBM; the rest lives in the X-Mem™ Warm Memory Tier, and misses are fetched over each GPU's memory tier link. Read about XM-Sparse
  • Long-Context Prefix caching. Time to first token for running long-context prefill in agentic or multi-turn workloads. Read about accelerating HiCache

Sparse Attention Decode.

%
8 GPUs and their X-Mem MPU8 GPU packages, each a die between two HBM stacks, in 2 overlapping rows, above one X-Mem MPU with 16 TB. A lane of 900 GB/s runs from each GPU to the X-Mem MPU, a Warm Memory Tier every GPU can access.GPUGPUGPUGPUGPUGPUGPUGPU900 GB/s900 GB/s900 GB/s900 GB/sX-Mem™ MPU16 TB
900 GB/s
16 TB

Decode throughput vs. interactivity

Netpreme X-Mem™Ideal
Throughput per GPU against interactivity, by batch sizeThroughput per GPU against tokens per second per user, one point per batch size. At the capacity limit, batch 282.75 per GPU: 27.2 tokens/s per user and 7,681 tokens/s per GPU. Dashed line: Ideal, no external I/O time.

Prefix Caching.

req

Time to first token vs. memory tier bandwidth

Prefill prefetchPrefill compute
Time to first token by memory tier bandwidth, prefetch and prefill computeTime to first token by memory tier bandwidth per GPU, split into prefetch and prefill compute. 211.6 ms at 900 GB/s, 1.02 s at 64 GB/s.

Essential storage

Always on

Remembers your cookie choice on this browser for 180 days. If browser storage is unavailable, your choice applies to this page session.

You can withdraw consent anytime here. Turning analytics off stops further collection and removes accessible Google Analytics cookies from this website. It does not delete data already collected. Read our privacy policy.