M-Tier™ Memory-Augmented Virtual Machine (VM)
Standard Linux/Ubuntu VM Instance with User Customizable Memory-to-Compute Ratio
Native integration with popular frameworks
Ubuntu 22.04
Number of XPUs8
12345678
Amount of MPU Memory8.0 TB
02 TB4 TB6 TB8 TB
* The diagram shown is for illustrative purposes only. Hardware specifications and pricing will vary by configuratio. Please contact us with any questions.
Shell example: managing MPU memory in a standard Linux/Ubuntu VM
bash
[mtier@netpreme-01:~$ xmemctl status
=== XM-100 MPU Memory Status ===
State: RUNNING
Banks: 4
Bank size: 2 TiB
Bank 0: 2 TiB / 2 TiB available (0.0 % used)
Bank 1: 2 TiB / 2 TiB available (0.0 % used)
Bank 2: 2 TiB / 2 TiB available (0.0 % used)
Bank 3: 2 TiB / 2 TiB available (0.0 % used)
--- Bank 0 ---
mTierMalloc: OK (ptr=0x6400000000)
fill_kernel: done
verify: PASSED
--- Bank 1 ---
mTierMalloc: OK (ptr=0x3e800000000)
fill_kernel: done
verify: PASSED
--- Bank 2 ---
mTierMalloc: OK (ptr=0x76c00000000)
fill_kernel: done
verify: PASSED
--- Bank 3 ---
mTierMalloc: OK (ptr=0xaf000000000)
fill_kernel: done
verify: PASSED
[mtier@netpreme-01:~$ xmemctl topo
XPU[0-7, 288 GiB each] ─ (Scale-up Fabric) ─ MPU[0-3, 2.0 TiB each]
[mtier@netpreme-01:~$ xmemctl stats llm-kv "shell-flag">--watch 1
TIME HBM USED MPU USED SCALEUP R/W HIT RATE CPU COPY
12:41:08 612 GB 5.4/8.0 TiB 1.28/0.41 TB/s 92.7% 0 B/s
12:41:13 614 GB 5.5/8.0 TiB 1.34/0.43 TB/s 95.1% 0 B/s
12:41:18 611 GB 5.6/8.0 TiB 1.31/0.39 TB/s 97.4% 0 B/s
SDK example: a C API for M-Tier™ memory management in CUDA
#include "mtier_sdk.h"
#include <cuda_runtime.h>
int main() {
// Initialize
cudaSetDevice(0);
cudaFree(0);
// Initialize the M-Tier SDK
mTierInit();
// Allocate
void *d_ptr = NULL;
// Allocate memory from M-Tier
mTierMalloc(0, 1ULL << 30, &d_ptr);
// Use in kernel
cudaMemset(d_ptr, 0, 1ULL << 30);
// Free
mTierFree(d_ptr);
return 0;
}