* Re: [PATCH v7 00/15] mm/mglru: improve reclaim loop and dirty folio handling
2026-05-11 18:51 ` [PATCH v7 00/15] mm/mglru: improve reclaim loop and dirty folio handling Shakeel Butt
@ 2026-05-12 5:08 ` Kairui Song
2026-05-12 5:56 ` Shakeel Butt
0 siblings, 1 reply; 3+ messages in thread
From: Kairui Song @ 2026-05-12 5:08 UTC (permalink / raw)
To: Shakeel Butt
Cc: linux-mm, Andrew Morton, Axel Rasmussen, Yuanchu Xie, Wei Xu,
Johannes Weiner, David Hildenbrand, Michal Hocko, Lorenzo Stoakes,
Barry Song, David Stevens, Chen Ridong, Leno Hou, Yafang Shao,
Yu Zhao, Zicheng Wang, Baolin Wang, Kalesh Singh,
Suren Baghdasaryan, Chris Li, Vernon Yang, linux-kernel, Qi Zheng
On Tue, May 12, 2026 at 2:51 AM Shakeel Butt <shakeel.butt@linux.dev> wrote:
>
>
> Hi Kairui,
Hello,
>
> On Tue, Apr 28, 2026 at 02:06:51AM +0800, Kairui Song via B4 Relay wrote:
> > From: Kairui Song <kasong@tencent.com>
> >
> > Test results: All tests are done on a 48c96t NUMA machine with 2 nodes
> > and a 128G memory machine using NVME as storage.
>
> Please include traditional LRU results for all of the following experiments as
> well (where it makes sense).
Sure, I've spawn a few test instances, was busy travelling last week.
That specific test machine is occupied so it might take a while.
A systematic test run takes roughly one or two days to complete for
one kernel version or config, e.g. the JS test takes at least 2 hours
to finish. Comparing versions/setups takes more time.
>
> >
> > MongoDB
> > =======
> > Running YCSB workloadb [2] (recordcount:20000000 operationcount:6000000,
> > threads:32), which does 95% read and 5% update to generate mixed read
> > and dirty writeback. MongoDB is set up in a 10G cgroup using Docker, and
> > the WiredTiger cache size is set to 4.5G, using NVME as storage.
>
> Can you add a sentence here on why this workload is chosen and is important for
> evaluation?
Because that's exactly the one we observed with regression since it
involves mixed writeback, and it's a pratical case.
>
> >
> > Not using SWAP.
>
> Any specific reason to not have swap in this test?
Because we are testing the writeback here, not related to SWAP, so
just to avoid noise and irrelevant parts.
A longer history involving SWAP is explained here:
https://lore.kernel.org/linux-mm/20230920190244.16839-1-ryncsn@gmail.com/
And a longer discussion on that:
https://lore.kernel.org/linux-mm/CAMgjq7BRaRgYLf2+8=+=nWtzkrHFKmudZPRm41PR6W+A+L=AKA@mail.gmail.com/
Both are not easy to reproduce, though. YCSB with MongoDB seems close
enough and I believe we are heading in the right track.
In an internal workload, we observed that patched MGLRU is about 20%
faster than classical LRU with MongoDB. Upstream MGLRU is still
slightly behind classical LRU at this point, and will hopefully be
patched soon, which is the RFC I posted:
https://lore.kernel.org/linux-mm/20260502-mglru-fg-v1-0-913619b014d9@tencent.com/
>
> >
> > Before:
> > Throughput(ops/sec): 62485.02962831822
> > AverageLatency(us): 500.9746963330107
> > pgpgin 159347462
> > pgpgout 5413332
> > workingset_refault_anon 0
> > workingset_refault_file 34522071
> >
> > After:
> > Throughput(ops/sec): 79760.71784646061 (+27.6%, higher is better)
> > AverageLatency(us): 391.25169970043726 (-21.9%, lower is better)
> > pgpgin 111093923 (-30.3%, lower is better)
> > pgpgout 5437456
> > workingset_refault_anon 0
> > workingset_refault_file 19566366 (-43.3%, lower is better)
> >
> > We can see a significant performance improvement after this series.
> > The test is done on NVME and the performance gap would be even larger
> > for slow devices, such as HDD or network storage. We observed over
> > 100% gain for some workloads with slow IO.
> >
> > Chrome & Node.js [3]
> > ====================
> > Using Yu Zhao's test script [3], testing on a x86_64 NUMA machine with 2
> > nodes and 128G memory, using 256G ZRAM as swap and spawn 32 memcg 64
> > workers:
> >
> > Before:
> > Total requests: 79915
> > Per-worker 95% CI (mean): [1233.9, 1263.5]
> > Per-worker stdev: 59.2
> > Jain's fairness: 0.997795 (1.0 = perfectly fair)
> > Latency:
> > Bucket Count Pct Cumul
> > [0,1)s 26859 33.61% 33.61%
> > [1,2)s 7818 9.78% 43.39%
> > [2,4)s 5532 6.92% 50.31%
> > [4,8)s 39706 49.69% 100.00%
> >
> > After:
> > Total requests: 81382
> > Per-worker 95% CI (mean): [1241.9, 1301.3]
> > Per-worker stdev: 118.8
> > Jain's fairness: 0.991480 (1.0 = perfectly fair)
> > Latency:
> > Bucket Count Pct Cumul
> > [0,1)s 26696 32.80% 32.80%
> > [1,2)s 8745 10.75% 43.55%
> > [2,4)s 6865 8.44% 51.98%
> > [4,8)s 39076 48.02% 100.00%
> >
> > Reclaim is still fair and effective, total requests number seems
> > slightly better.
>
> Please add a reference to Jain's fairness and a sentence on why we should care
> about it.
So first, Here is the previous test setup for that:
https://lore.kernel.org/all/20221220214923.1229538-1-yuzhao@google.com/
The basical idea is simple: if all memcgs are under similar pressure,
they should be reclaimed equally, which seems fair.
The fairness index measures the equality of resource allocation among
users. It is commonly used to evaluate network bandwidth distribution
for multiple users under pressure, which seems suitable here. We are
also measuring the reclaim ratio for multiple users under pressure.
I'm using a numeric index here so I don't need to post 500 lines of
raw test results every time:
https://www.sciencedirect.com/topics/computer-science/fairness-index
Also here is the longer version of test result I just collected in
past few days. The test closely mirrors real-world usage from desktop
to web services.
Classical LRU:
------------------------------------------------------------------------
THROUGHPUT
------------------------------------------------------------------------
Total requests: 60226
Per-worker mean: 941.0
Per-worker median: 931
Per-worker min: 678
Per-worker max: 1252
Per-worker stdev: 131.3
95% CI (mean): [ 908.2, 973.9]
------------------------------------------------------------------------
LATENCY DISTRIBUTION (all workers aggregated)
------------------------------------------------------------------------
Bucket Count Pct Cumul
[0,1)s 19493 32.37% 32.37%
[1,2)s 2024 3.36% 35.73%
[2,4)s 5621 9.33% 45.06%
[4,8)s 32881 54.60% 99.66%
[8,16)s 207 0.34% 100.00%
[16,32)s 0 0.00% 100.00%
[32,64)s 0 0.00% 100.00%
[64,128)s 0 0.00% 100.00%
[128,inf)s 0 0.00% 100.00%
------------------------------------------------------------------------
FAIRNESS (per-worker total requests)
------------------------------------------------------------------------
Jain's fairness index: 0.981188 (1.0 = perfectly fair)
Coeff of variation: 0.1396 (0.0 = perfectly fair)
Min/Max ratio: 0.5415
P10: 780
P25: 855
P50 (median): 931
P75: 1040
P90: 1112
------------------------------------------------------------------------
PER-MEMCG BREAKDOWN (sorted by total, top/bottom 5)
------------------------------------------------------------------------
Memcgs: 32 mean=1882.1 95%CI=[1799.8, 1964.4] Jain=0.9860
--- Top 5 ---
memcg 6: 2423 requests
memcg 10: 2364 requests
memcg 31: 2213 requests
memcg 20: 2207 requests
memcg 30: 2156 requests
--- Bottom 5 ---
memcg 27: 1658 requests
memcg 19: 1645 requests
memcg 12: 1610 requests
memcg 0: 1566 requests
memcg 28: 1533 requests
Raw Result:
client: 8047 total: 984, 0: 293, 1: 44, 2: 108, 4:
538, 8: 1, 16: 0, 32: 0, 64: 0, 128: 0
client: 8058 total: 882, 0: 289, 1: 18, 2: 63, 4:
507, 8: 5, 16: 0, 32: 0, 64: 0, 128: 0
client: 8017 total: 1051, 0: 347, 1: 43, 2: 133, 4:
528, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8059 total: 952, 0: 274, 1: 41, 2: 92, 4:
545, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8005 total: 921, 0: 230, 1: 43, 2: 113, 4:
535, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8063 total: 1173, 0: 459, 1: 50, 2: 161, 4:
503, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8051 total: 986, 0: 296, 1: 34, 2: 122, 4:
534, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8043 total: 949, 0: 260, 1: 53, 2: 90, 4:
546, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8045 total: 1069, 0: 362, 1: 46, 2: 143, 4:
518, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8008 total: 857, 0: 259, 1: 25, 2: 69, 4:
500, 8: 4, 16: 0, 32: 0, 64: 0, 128: 0
client: 8023 total: 1049, 0: 348, 1: 44, 2: 136, 4:
521, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8015 total: 895, 0: 221, 1: 34, 2: 105, 4:
534, 8: 1, 16: 0, 32: 0, 64: 0, 128: 0
client: 8027 total: 899, 0: 219, 1: 42, 2: 96, 4:
542, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8061 total: 1093, 0: 396, 1: 31, 2: 157, 4:
509, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8038 total: 737, 0: 174, 1: 7, 2: 46, 4:
501, 8: 9, 16: 0, 32: 0, 64: 0, 128: 0
client: 8056 total: 678, 0: 133, 1: 5, 2: 32, 4:
501, 8: 7, 16: 0, 32: 0, 64: 0, 128: 0
client: 8040 total: 1039, 0: 423, 1: 37, 2: 93, 4:
477, 8: 9, 16: 0, 32: 0, 64: 0, 128: 0
client: 8036 total: 766, 0: 202, 1: 7, 2: 54, 4:
494, 8: 9, 16: 0, 32: 0, 64: 0, 128: 0
client: 8000 total: 697, 0: 136, 1: 13, 2: 48, 4:
495, 8: 5, 16: 0, 32: 0, 64: 0, 128: 0
client: 8030 total: 804, 0: 232, 1: 14, 2: 53, 4:
501, 8: 4, 16: 0, 32: 0, 64: 0, 128: 0
client: 8006 total: 852, 0: 267, 1: 18, 2: 62, 4:
495, 8: 10, 16: 0, 32: 0, 64: 0, 128: 0
client: 8062 total: 1040, 0: 437, 1: 43, 2: 61, 4:
489, 8: 10, 16: 0, 32: 0, 64: 0, 128: 0
client: 8014 total: 833, 0: 254, 1: 14, 2: 58, 4:
497, 8: 10, 16: 0, 32: 0, 64: 0, 128: 0
client: 8060 total: 1063, 0: 465, 1: 23, 2: 81, 4:
485, 8: 9, 16: 0, 32: 0, 64: 0, 128: 0
client: 8046 total: 814, 0: 244, 1: 18, 2: 40, 4:
508, 8: 4, 16: 0, 32: 0, 64: 0, 128: 0
client: 8049 total: 1080, 0: 388, 1: 40, 2: 123, 4:
529, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8022 total: 1001, 0: 422, 1: 22, 2: 62, 4:
486, 8: 9, 16: 0, 32: 0, 64: 0, 128: 0
client: 8019 total: 988, 0: 304, 1: 36, 2: 116, 4:
532, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8026 total: 780, 0: 218, 1: 12, 2: 47, 4:
500, 8: 3, 16: 0, 32: 0, 64: 0, 128: 0
client: 8024 total: 719, 0: 163, 1: 7, 2: 43, 4:
501, 8: 5, 16: 0, 32: 0, 64: 0, 128: 0
client: 8053 total: 1045, 0: 360, 1: 38, 2: 120, 4:
527, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8034 total: 873, 0: 286, 1: 19, 2: 57, 4:
508, 8: 3, 16: 0, 32: 0, 64: 0, 128: 0
client: 8048 total: 889, 0: 301, 1: 26, 2: 59, 4:
497, 8: 6, 16: 0, 32: 0, 64: 0, 128: 0
client: 8055 total: 871, 0: 199, 1: 40, 2: 89, 4:
543, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8001 total: 869, 0: 196, 1: 35, 2: 95, 4:
543, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8003 total: 1051, 0: 369, 1: 42, 2: 103, 4:
537, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8011 total: 1118, 0: 398, 1: 53, 2: 156, 4:
511, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8018 total: 762, 0: 192, 1: 15, 2: 45, 4:
503, 8: 7, 16: 0, 32: 0, 64: 0, 128: 0
client: 8021 total: 1112, 0: 410, 1: 41, 2: 145, 4:
516, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8050 total: 869, 0: 276, 1: 21, 2: 71, 4:
496, 8: 5, 16: 0, 32: 0, 64: 0, 128: 0
client: 8032 total: 823, 0: 238, 1: 21, 2: 54, 4:
505, 8: 5, 16: 0, 32: 0, 64: 0, 128: 0
client: 8044 total: 1030, 0: 433, 1: 31, 2: 66, 4:
496, 8: 4, 16: 0, 32: 0, 64: 0, 128: 0
client: 8035 total: 965, 0: 283, 1: 42, 2: 112, 4:
528, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8025 total: 891, 0: 212, 1: 43, 2: 90, 4:
546, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8039 total: 908, 0: 241, 1: 36, 2: 86, 4:
545, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8009 total: 963, 0: 286, 1: 36, 2: 108, 4:
533, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8037 total: 917, 0: 227, 1: 45, 2: 100, 4:
545, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8020 total: 1252, 0: 607, 1: 51, 2: 115, 4:
477, 8: 2, 16: 0, 32: 0, 64: 0, 128: 0
client: 8004 total: 818, 0: 245, 1: 16, 2: 47, 4:
501, 8: 9, 16: 0, 32: 0, 64: 0, 128: 0
client: 8052 total: 870, 0: 285, 1: 20, 2: 52, 4:
507, 8: 6, 16: 0, 32: 0, 64: 0, 128: 0
client: 8033 total: 925, 0: 269, 1: 28, 2: 83, 4:
545, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8010 total: 931, 0: 334, 1: 29, 2: 62, 4:
500, 8: 6, 16: 0, 32: 0, 64: 0, 128: 0
client: 8016 total: 990, 0: 388, 1: 27, 2: 70, 4:
500, 8: 5, 16: 0, 32: 0, 64: 0, 128: 0
client: 8012 total: 1173, 0: 556, 1: 51, 2: 78, 4:
480, 8: 8, 16: 0, 32: 0, 64: 0, 128: 0
client: 8028 total: 837, 0: 253, 1: 32, 2: 47, 4:
500, 8: 5, 16: 0, 32: 0, 64: 0, 128: 0
client: 8031 total: 992, 0: 315, 1: 33, 2: 119, 4:
525, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8041 total: 1168, 0: 452, 1: 52, 2: 162, 4:
502, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8054 total: 787, 0: 212, 1: 15, 2: 58, 4:
493, 8: 9, 16: 0, 32: 0, 64: 0, 128: 0
client: 8042 total: 799, 0: 227, 1: 13, 2: 45, 4:
508, 8: 6, 16: 0, 32: 0, 64: 0, 128: 0
client: 8002 total: 1034, 0: 449, 1: 32, 2: 59, 4:
488, 8: 6, 16: 0, 32: 0, 64: 0, 128: 0
client: 8057 total: 855, 0: 184, 1: 47, 2: 81, 4:
543, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8007 total: 965, 0: 269, 1: 36, 2: 135, 4:
525, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8013 total: 1250, 0: 536, 1: 53, 2: 143, 4:
518, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8029 total: 973, 0: 290, 1: 41, 2: 102, 4:
539, 8: 1, 16: 0, 32: 0, 64: 0, 128: 0
MGLRU (after this series, results are similar before this with seems
slightly lower throughput or maybe just noise, see cover letter):
------------------------------------------------------------------------
THROUGHPUT
------------------------------------------------------------------------
Total requests: 83926
Per-worker mean: 1311.3
Per-worker median: 1306
Per-worker min: 1170
Per-worker max: 1466
Per-worker stdev: 70.8
95% CI (mean): [ 1293.6, 1329.0]
------------------------------------------------------------------------
LATENCY DISTRIBUTION (all workers aggregated)
------------------------------------------------------------------------
Bucket Count Pct Cumul
[0,1)s 27929 33.28% 33.28%
[1,2)s 9075 10.81% 44.09%
[2,4)s 8558 10.20% 54.29%
[4,8)s 38364 45.71% 100.00%
[8,16)s 0 0.00% 100.00%
[16,32)s 0 0.00% 100.00%
[32,64)s 0 0.00% 100.00%
[64,128)s 0 0.00% 100.00%
[128,inf)s 0 0.00% 100.00%
------------------------------------------------------------------------
FAIRNESS (per-worker total requests)
------------------------------------------------------------------------
Jain's fairness index: 0.997138 (1.0 = perfectly fair)
Coeff of variation: 0.0540 (0.0 = perfectly fair)
Min/Max ratio: 0.7981
P10: 1220
P25: 1253
P50 (median): 1306
P75: 1367
P90: 1398
------------------------------------------------------------------------
PER-MEMCG BREAKDOWN (sorted by total, top/bottom 5)
------------------------------------------------------------------------
Memcgs: 32 mean=2622.7 95%CI=[2601.4, 2643.9] Jain=0.9995
--- Top 5 ---
memcg 24: 2719 requests
memcg 5: 2711 requests
memcg 16: 2703 requests
memcg 0: 2696 requests
memcg 19: 2689 requests
--- Bottom 5 ---
memcg 20: 2550 requests
memcg 21: 2534 requests
memcg 23: 2521 requests
memcg 22: 2514 requests
memcg 27: 2514 requests
Raw result:
client: 8028 total: 1252, 0: 410, 1: 132, 2: 106, 4:
604, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8026 total: 1220, 0: 390, 1: 107, 2: 106, 4:
617, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8036 total: 1260, 0: 403, 1: 154, 2: 92, 4:
611, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8038 total: 1322, 0: 475, 1: 150, 2: 90, 4:
607, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8002 total: 1220, 0: 384, 1: 137, 2: 82, 4:
617, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8008 total: 1264, 0: 410, 1: 138, 2: 108, 4:
608, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8044 total: 1180, 0: 339, 1: 123, 2: 94, 4:
624, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8062 total: 1267, 0: 428, 1: 125, 2: 111, 4:
603, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8050 total: 1197, 0: 351, 1: 131, 2: 113, 4:
602, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8057 total: 1379, 0: 480, 1: 142, 2: 158, 4:
599, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8048 total: 1301, 0: 454, 1: 142, 2: 101, 4:
604, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8034 total: 1266, 0: 422, 1: 140, 2: 98, 4:
606, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8020 total: 1282, 0: 425, 1: 153, 2: 98, 4:
606, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8000 total: 1245, 0: 404, 1: 137, 2: 88, 4:
616, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8030 total: 1282, 0: 411, 1: 164, 2: 104, 4:
603, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8045 total: 1334, 0: 424, 1: 147, 2: 168, 4:
595, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8053 total: 1359, 0: 462, 1: 139, 2: 161, 4:
597, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8060 total: 1240, 0: 375, 1: 158, 2: 110, 4:
597, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8043 total: 1338, 0: 437, 1: 138, 2: 171, 4:
592, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8041 total: 1323, 0: 438, 1: 124, 2: 155, 4:
606, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8025 total: 1331, 0: 435, 1: 130, 2: 180, 4:
586, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8040 total: 1227, 0: 389, 1: 133, 2: 92, 4:
613, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8022 total: 1240, 0: 393, 1: 139, 2: 100, 4:
608, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8049 total: 1418, 0: 510, 1: 145, 2: 172, 4:
591, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8012 total: 1205, 0: 373, 1: 120, 2: 93, 4:
619, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8059 total: 1375, 0: 462, 1: 171, 2: 152, 4:
590, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8037 total: 1412, 0: 513, 1: 144, 2: 171, 4:
584, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8001 total: 1451, 0: 536, 1: 160, 2: 191, 4:
564, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8009 total: 1356, 0: 451, 1: 133, 2: 182, 4:
590, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8039 total: 1367, 0: 456, 1: 144, 2: 186, 4:
581, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8042 total: 1196, 0: 345, 1: 134, 2: 97, 4:
620, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8013 total: 1409, 0: 519, 1: 134, 2: 172, 4:
584, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8021 total: 1392, 0: 478, 1: 156, 2: 169, 4:
589, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8031 total: 1373, 0: 477, 1: 135, 2: 174, 4:
587, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8014 total: 1271, 0: 419, 1: 152, 2: 96, 4:
604, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8015 total: 1305, 0: 398, 1: 139, 2: 179, 4:
589, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8024 total: 1251, 0: 390, 1: 167, 2: 78, 4:
616, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8033 total: 1335, 0: 408, 1: 169, 2: 172, 4:
586, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8004 total: 1245, 0: 398, 1: 129, 2: 107, 4:
611, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8003 total: 1394, 0: 494, 1: 144, 2: 154, 4:
602, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8052 total: 1296, 0: 444, 1: 154, 2: 106, 4:
592, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8061 total: 1353, 0: 455, 1: 142, 2: 147, 4:
609, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8017 total: 1355, 0: 451, 1: 153, 2: 166, 4:
585, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8063 total: 1367, 0: 474, 1: 136, 2: 152, 4:
605, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8018 total: 1225, 0: 379, 1: 132, 2: 97, 4:
617, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8029 total: 1345, 0: 460, 1: 129, 2: 152, 4:
604, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8027 total: 1398, 0: 518, 1: 121, 2: 158, 4:
601, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8007 total: 1253, 0: 375, 1: 118, 2: 124, 4:
636, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8047 total: 1302, 0: 414, 1: 126, 2: 170, 4:
592, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8005 total: 1397, 0: 488, 1: 151, 2: 161, 4:
597, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8019 total: 1347, 0: 437, 1: 145, 2: 178, 4:
587, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8035 total: 1361, 0: 453, 1: 151, 2: 179, 4:
578, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8011 total: 1416, 0: 517, 1: 147, 2: 172, 4:
580, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8023 total: 1385, 0: 473, 1: 161, 2: 172, 4:
579, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8046 total: 1219, 0: 388, 1: 123, 2: 91, 4:
617, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8056 total: 1304, 0: 463, 1: 135, 2: 95, 4:
611, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8006 total: 1306, 0: 442, 1: 147, 2: 130, 4:
587, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8054 total: 1170, 0: 321, 1: 136, 2: 101, 4:
612, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8055 total: 1344, 0: 447, 1: 141, 2: 169, 4:
587, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8010 total: 1295, 0: 429, 1: 164, 2: 100, 4:
602, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8016 total: 1292, 0: 448, 1: 140, 2: 108, 4:
596, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8051 total: 1466, 0: 555, 1: 152, 2: 200, 4:
559, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8058 total: 1278, 0: 430, 1: 144, 2: 86, 4:
618, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
client: 8032 total: 1368, 0: 502, 1: 168, 2: 113, 4:
585, 8: 0, 16: 0, 32: 0, 64: 0, 128: 0
The test is rebased on 7.1 rc, MGLRU seems ~30% faster compared to
classical LRU, better latency distribution, and better fairness too.
On my x86 machine the gain is not as much as the one Yu posted
for ARM, but it still looks very good.
Ridong also reproduced with a much better result where MGLRU seems to
be much faster than classical LRU on ARM (or maybe using different
time priod?):
https://lore.kernel.org/linux-mm/20260120134256.2271710-1-chenridong@huaweicloud.com/
During one or two test runs, a single memcg might archive much higher
throughput with MGLRU, causing fairness to look slightly worse,
however, overall performance still seems much better than classical
LRU. I suspect improvements are needed for aging or the random bucket
part, but I think that's an irrelevant topic for now.
> >
> > MySQL:
> > ======
> >
> > Testing with innodb_buffer_pool_size=26106127360, in a 2G memcg, using
> > ZRAM as swap and test command:
> >
> > sysbench /usr/share/sysbench/oltp_read_only.lua --mysql-db=sb \
> > --tables=48 --table-size=2000000 --threads=48 --time=600 run
> >
> > Before: 17303.41 tps
> > After this series: 17291.50 tps
> >
> > Seems only noise level changes, no regression.
> >
>
> Please add a sentence on why this specific params.
>
> > FIO:
> > ====
> > Testing with the following command, where /mnt/ramdisk is a
> > 64G EXT4 ramdisk, each test file is 3G, in a 10G memcg,
> > 6 test run each:
> >
> > fio --directory=/mnt/ramdisk --filename_format='test.$jobnum.img' \
> > --name=cached --numjobs=16 --size=3072M --buffered=1 --ioengine=mmap \
> > --rw=randread --norandommap --time_based \
> > --ramp_time=1m --runtime=5m --group_reporting
> >
> > Before: 8968.76 MB/s
> > After this series: 8995.63 MB/s
> >
> > Also seem only noise level changes and no regression or slightly better.
>
> Same here.
I tested the page cache performance with buffered read. There is
another test involving classical LRU, where MGLRU seems to
significantly outperform classical LRU. The case was provided by the
CachyOS community, I didn't include it here because the cover letter
is already getting tediously long.
https://lore.kernel.org/all/acgNCzRDVmSbXrOE@KASONG-MC4/
MGLRU seems to have significantly lower jitter and better performance with that.
BTW I also disabled OOMD or any related daemon to avoid noise during
that test. I repeated the test several times, and recorded one test
run as well since it's meant for a desktop test and I was discussing
with distro communities at that time. MGLRU TTL can completely avoid
jitter, however, it's not enabled during the test to prevent
confusion.
Classical LRU:
https://www.youtube.com/watch?v=pujboGNcBNI
MGLRU:
https://www.youtube.com/watch?v=ffnFUeaBQ_0
>
> >
> > Build kernel:
> > =============
> > Build kernel test using ZRAM as swap, on top of tmpfs, in a 3G memcg
> > using make -j96 and defconfig, measuring system time, 12 test run each.
> >
> > Before: 2873.52s
> > After this series: 2811.88s
> >
> > Also seem only noise level changes, no regression or very slightly better.
>
> So, the kernel source code is on tmpfs, right? Also 3G memcg means memory.max is
> 3G, correct?
Right. That's to avoid I/O noise. I also tested with source code on
disk, I didn't post that because I think the MySQL test already shows
a workload of mixed anon / file.
^ permalink raw reply [flat|nested] 3+ messages in thread