M-Tier™ Memory-Augmented Virtual Machine (VM)

Standard Linux/Ubuntu VM Instance with User Customizable Memory-to-Compute Ratio

Native integration with popular frameworks

PyTorchvLLMSGLang
CUDAvLLMSGLang
Number of XPUs
12345678
Amount of X-Mem Memory
02 TB4 TB6 TB8 TB
XPU 10.3 TB HBM1800 GB/sXPU 20.3 TB HBM1800 GB/s…+2XPU 50.3 TB HBM1800 GB/sXPU 60.3 TB HBM1800 GB/s…+2X-Mem™ Pool8.0 TB

Shell example: managing MPU memory in a standard Linux/Ubuntu VM

bash
[mtier@netpreme-01:~$ xmemctl status
=== XM-100 MPU Memory Status ===
State: RUNNING
Banks: 4
Bank size: 2 TiB
Bank 0:  2 TiB / 2 TiB available (0.0 % used)
Bank 1:  2 TiB / 2 TiB available (0.0 % used)
Bank 2:  2 TiB / 2 TiB available (0.0 % used)
Bank 3:  2 TiB / 2 TiB available (0.0 % used)

--- Bank 0 ---
mTierMalloc: OK (ptr=0x6400000000)
fill_kernel: done
verify: PASSED
--- Bank 1 ---
mTierMalloc: OK (ptr=0x3e800000000)
fill_kernel: done
verify: PASSED
--- Bank 2 ---
mTierMalloc: OK (ptr=0x76c00000000)
fill_kernel: done
verify: PASSED
--- Bank 3 ---
mTierMalloc: OK (ptr=0xaf000000000)
fill_kernel: done
verify: PASSED

[mtier@netpreme-01:~$ xmemctl topo
XPU[0-7, 288 GiB each] ─ (Scale-up Fabric) ─ MPU[0-3, 2.0 TiB each]

[mtier@netpreme-01:~$ xmemctl stats llm-kv "shell-flag">--watch 1
TIME      HBM USED  MPU USED      SCALEUP R/W      HIT RATE  CPU COPY
12:41:08  612 GB    5.4/8.0 TiB   1.28/0.41 TB/s   92.7%     0 B/s
12:41:13  614 GB    5.5/8.0 TiB   1.34/0.43 TB/s   95.1%     0 B/s
12:41:18  611 GB    5.6/8.0 TiB   1.31/0.39 TB/s   97.4%     0 B/s

SDK example: a C API for M-Tier™ memory management in CUDA

#include "mtier_sdk.h"
#include <cuda_runtime.h>

int main() {
    // Initialize
    cudaSetDevice(0);
    cudaFree(0);

    // Initialize the M-Tier SDK
    mTierInit();

    // Allocate
    void *d_ptr = NULL;

    // Allocate memory from M-Tier
    mTierMalloc(0, 1ULL << 30, &d_ptr);

    // Use in kernel
    cudaMemset(d_ptr, 0, 1ULL << 30);

    // Free
    mTierFree(d_ptr);

    return 0;
}

Essential storage

Always on

Remembers your cookie choice on this browser for 180 days. If browser storage is unavailable, your choice applies to this page session.

You can withdraw consent anytime here. Turning analytics off stops further collection and removes accessible Google Analytics cookies from this website. It does not delete data already collected. Read our privacy policy.