REGRESSION BISECTED mm, mmap: limit THP alignment of anonymous mappings to PMD-aligned sizes

REGRESSION BISECTED mm, mmap: limit THP alignment of anonymous mappings to PMD-aligned sizes

[Doug Smythies]

[-- Attachment #1: Type: text/plain, Size: 1579 bytes --]

Hello,

Note: The CC list is a guess, and I am not on the two vger.kernel.org lists.

After observing a 30% reduction in the ebizzy benchmark performance, I bisected the kernel and got:

doug@s19:~/kernel/linux$ git bisect bad
d4148aeab412432bf928f311eca8a2ba52bb05df is the first bad commit
commit d4148aeab412432bf928f311eca8a2ba52bb05df
Author: Vlastimil Babka <vbabka@suse.cz>
Date:   Thu Oct 24 17:12:29 2024 +0200

    mm, mmap: limit THP alignment of anonymous mappings to PMD-aligned sizes

As a double check I reverted the commit, on top of kernel 6.14-rc1.
I had to manually revert it, due to other changes since then.
The previous performance of the benchmark was restored.

I actually use the sleeping-ebizzy benchmark [1].
I use it for idle governor testing because it has yielded interesting results in the past.
And sweep over a range of sleep times. Example graphs attached.

Legend (regression average is over interval range from 400 to 3600 uSec):

teo611: kernel 6.11, teo idle governor. Reference.
teo614: kernel 6.14-rc1, teo idle governor. Regression 25.7%
teo612: kernel 6.12, teo idle governor. Regression 24.4%
teo613: kernel 6.13, teo idle governor. Regression 25.1%
teo614-revert: kernel 6.14-rc1, with this patch reverted, teo idle governor. No regression, 1.4%
menu: kernel 6.14-rc1, menu idle governor. Regression 26.4% 

Example command:
./ebizzy -m -S 20 -t 128 -a 1 -i 400
 
My processor: Intel(R) Core(TM) i5-10600K CPU @ 4.10GHz
Distro: Ubuntu 24.04, server, no desktop GUI.

[1] https://github.com/pratiksampat/sleeping-ebizzy

Doug Smythies


[-- Attachment #2: interval-sweep.png --]
[-- Type: image/png, Size: 32339 bytes --]

[-- Attachment #3: relative-performance.png --]
[-- Type: image/png, Size: 42078 bytes --]

Re: REGRESSION BISECTED mm, mmap: limit THP alignment of anonymous mappings to PMD-aligned sizes

[Vlastimil Babka]

On 2/4/25 01:56, Doug Smythies wrote:
> Hello,
> 
> Note: The CC list is a guess, and I am not on the two vger.kernel.org lists.
> 
> After observing a 30% reduction in the ebizzy benchmark performance, I bisected the kernel and got:
> 
> doug@s19:~/kernel/linux$ git bisect bad
> d4148aeab412432bf928f311eca8a2ba52bb05df is the first bad commit
> commit d4148aeab412432bf928f311eca8a2ba52bb05df
> Author: Vlastimil Babka <vbabka@suse.cz>
> Date:   Thu Oct 24 17:12:29 2024 +0200
> 
>     mm, mmap: limit THP alignment of anonymous mappings to PMD-aligned sizes

Thanks for the report. That commit fixed regressions of other workloads, but
since it's tweaking a heuristic related to THPs that can have both positive
and negative consequences, it's not surprising to see that another workload
can regress.

Since that commit is implicated, there should have been a matching
improvement for this workload in 6.7 thanks to commit efa7df3e3bb5 ?

I guess one option to proceed would be to check what kind of mappings ebizzy
creates for its performance sensitive operations, and see if they e.g. were
backed by THPs before d4148aeab41 and now they aren't. Maybe some simple
adjustment to ebizzy's allocations is possible to achieve the better
performance always and not rely on this particular heuristic.

> As a double check I reverted the commit, on top of kernel 6.14-rc1.
> I had to manually revert it, due to other changes since then.
> The previous performance of the benchmark was restored.
> 
> I actually use the sleeping-ebizzy benchmark [1].
> I use it for idle governor testing because it has yielded interesting results in the past.
> And sweep over a range of sleep times. Example graphs attached.
> 
> Legend (regression average is over interval range from 400 to 3600 uSec):
> 
> teo611: kernel 6.11, teo idle governor. Reference.
> teo614: kernel 6.14-rc1, teo idle governor. Regression 25.7%
> teo612: kernel 6.12, teo idle governor. Regression 24.4%
> teo613: kernel 6.13, teo idle governor. Regression 25.1%
> teo614-revert: kernel 6.14-rc1, with this patch reverted, teo idle governor. No regression, 1.4%
> menu: kernel 6.14-rc1, menu idle governor. Regression 26.4% 
> 
> Example command:
> ./ebizzy -m -S 20 -t 128 -a 1 -i 400
>  
> My processor: Intel(R) Core(TM) i5-10600K CPU @ 4.10GHz
> Distro: Ubuntu 24.04, server, no desktop GUI.
> 
> [1] https://github.com/pratiksampat/sleeping-ebizzy
> 
> Doug Smythies
> 
> 
> interval-sweep.png
> 
> 
> relative-performance.png
> 

RE: REGRESSION BISECTED mm, mmap: limit THP alignment of anonymous mappings to PMD-aligned sizes

[Doug Smythies]

Hi, and thank you for your reply.

On 2024.02.04 03:04 Vlastimil Babka wrote:
On 2/4/25 01:56, Doug Smythies wrote:
>> Hello,
>> 
>> Note: The CC list is a guess, and I am not on the two vger.kernel.org lists.
>> 
>> After observing a 30% reduction in the ebizzy benchmark performance, I bisected the kernel and got:
>> 
>> doug@s19:~/kernel/linux$ git bisect bad
>> d4148aeab412432bf928f311eca8a2ba52bb05df is the first bad commit
>> commit d4148aeab412432bf928f311eca8a2ba52bb05df
>> Author: Vlastimil Babka <vbabka@suse.cz>
>> Date:   Thu Oct 24 17:12:29 2024 +0200
>> 
>>     mm, mmap: limit THP alignment of anonymous mappings to PMD-aligned sizes
>
> Thanks for the report. That commit fixed regressions of other workloads, but
> since it's tweaking a heuristic related to THPs that can have both positive
> and negative consequences, it's not surprising to see that another workload
> can regress.

Yes, I had read the related emails and thought that might be the response.

> Since that commit is implicated, there should have been a matching
> improvement for this workload in 6.7 thanks to commit efa7df3e3bb5 ?

Yes, I also reverted efa7df3e3bb5, but must have made a mistake
in my testing yesterday, because re-testing today
I got the same reduction in performance.

> I guess one option to proceed would be to check what kind of mappings ebizzy
> creates for its performance sensitive operations, and see if they e.g. were
> backed by THPs before d4148aeab41 and now they aren't. Maybe some simple
> adjustment to ebizzy's allocations is possible to achieve the better
> performance always and not rely on this particular heuristic.

I'm not going to modify ebizzy, because I don't really care.
I was just reporting what I thought was a regression was all.

>> As a double check I reverted the commit, on top of kernel 6.14-rc1.
>> I had to manually revert it, due to other changes since then.
>> The previous performance of the benchmark was restored.
>> 
>> I actually use the sleeping-ebizzy benchmark [1].
>> I use it for idle governor testing because it has yielded interesting results in the past.
>> And sweep over a range of sleep times. Example graphs attached.
>> 
>> Legend (regression average is over interval range from 400 to 3600 uSec):
>> 
>> teo611: kernel 6.11, teo idle governor. Reference.
>> teo614: kernel 6.14-rc1, teo idle governor. Regression 25.7%
>> teo612: kernel 6.12, teo idle governor. Regression 24.4%
>> teo613: kernel 6.13, teo idle governor. Regression 25.1%
>> teo614-revert: kernel 6.14-rc1, with this patch reverted, teo idle governor. No regression, 1.4%
>> menu: kernel 6.14-rc1, menu idle governor. Regression 26.4% 

Kernel 6.14 with both d4148aeab41 and efa7df3e3bb5 reverted: Throughput reduced by 26.0%
 
>> Example command:
>> ./ebizzy -m -S 20 -t 128 -a 1 -i 400
>>  
>> My processor: Intel(R) Core(TM) i5-10600K CPU @ 4.10GHz
>> Distro: Ubuntu 24.04, server, no desktop GUI.
>> 
>> [1] https://github.com/pratiksampat/sleeping-ebizzy
>> 
>> Doug Smythies