RE: [PATCH v3 00/11] mempolicy2, mbind2, and weighted interleave

Hyeongtak Ji
2023-12-18T07:07:58Z

Hi Gregory,

Thank you for the v3 patch.

Gregory Price <gourry.memverge@gmail.com> write:

[snip]

> =====================================================================
> Performance tests - MLC
> From Ravi Jonnalagadda <ravis.opensrc@micron.com>
>
> Workload: W2
> Data Signature: 2:1 read:write
> DRAM only bandwidth (GBps): 298.8
> DRAM + CXL (default interleave) (GBps): 113.04
> DRAM + CXL (weighted interleave)(GBps): 412.5
> Gain over DRAM only: 1.38x
>
> Workload: W5
> Data Signature: 1:1 read:write
> DRAM only bandwidth (GBps): 273.2
> DRAM + CXL (default interleave) (GBps): 117.23
> DRAM + CXL (weighted interleave)(GBps): 382.7
> Gain over DRAM only: 1.4x

I've run XSBench based on the v3 patch and got numbers below. I used
your sample numactl extension from here:
Link: https://github.com/gmprice/numactl/tree/weighted_interleave_master

Performance tests – XSBench
NUMA node 0: 56 logical cores, 128 GB memory
NUMA node 2: 96 GB CXL memory

1. dram only
$ numactl -membind 0 ./XSBench -s XL –p 5000000
Threads: 56
Runtime: 36.235 seconds
Lookups: 170,000,000
Lookups/s: 4,691,618

2. default interleave
$ numactl –-interleave 0,2 ./XSBench –s XL –p 5000000
Threads: 56
Runtime: 55.243 seconds
Lookups: 170,000,000
Lookups/s: 3,077,293

3. weighted interleave
$ numactl --weighted --interleave 0,2 ./XSBench –s XL –p 5000000
Threads: 56
Runtime: 29.262 seconds
Lookups: 170,000,000
Lookups/s: 5,809,513

In terms of runtime, weighted-interleaving shows 1.19x improvement
compared to dram only, and 1.47x compared to default interleave. I’ve
repeatedly run XSBench and have not observed any significant variations
across the runs.

Kind regards,
Hyeongtak