Re: [PATCH v8 0/8] mm: Hot page tracking and promotion infrastructure

[Bharata B Rao <bharata@amd.com>]

On 17-Sep-26 2:22 AM, Joshua Hahn wrote:
> On Wed, 16 Sep 2026 08:38:17 +0530 Bharata B Rao <bharata@amd.com> wrote:
> 
>> On 12-Sep-26 2:38 AM, Joshua Hahn wrote:
>>> Hi Bharata,
>>>
>>> We've been testing pghot in a Meta kernel with worklaods to try and see
>>> what the performance characteristics look like. I wanted to report
>>> some of the findings that we've found in our testing. Note that these
>>> results are from a modified kernel that has my tiered memcg series [1]
>>> backported, as well as Gregory's numa balancing scanning fixes [2]
>>> as well. 
>>
>> Hi Joshua,
>>
>> Thanks for testing pghot with your workload!
>>
>>>
>>> - Without tuning and using the default tunables that pghot comes with,
>>>   we've found around a slight performance regression against raw NUMAB2.
>>>   This was being run on a host with 750G DRAM and 250G CXL.
>>
>> pghot promotes on 2nd access by default since pghot_freq_threshold is 2
>> by default. This will result in slower promotions compared to NUMAB2
>> which promotes on 1st access. Could you please try with setting
>> pghot_freq_threshold to 1? I think I should make the default as 1 going
>> forward.
> 
> Gotcha. Out of curiosity, I'm wondering why you decided to make this a
> tunable. Specifically, I'm wondering if there are any workloads that
> benefit from making it require a second access. To me keeping it at
> first access makes a lot of sense. Obviously you could say that
> scanning workloads would want to use 2 accesses, but there should be
> other ways to prevent that from happening rather than counting the
> number of accesses before promotion. Just my 2c.

The basic idea is to consider two factors for determining the page hotness: the
frequency of access and the recency of access. Hence I have kept both of them as
configurable so that users can tune them for their workloads.

However since the immediate comparision of pghot-hintfaults source is with
existing NUMAB2, it makes sense to have the default freq as 1 to get closer to
apples-to-apples comparision by default.

> 
>> In any case, will it be possible to provide more information on what your
>> workloads are, and the exact performance metrics? Any representative workload
>> that can be substituted to reproduce this behaviour can also help.
>>
>> And when you say slight perf regression, how much is it exactly?
> 
> I'm seeing that average latency is about 20% higher and p99 latency
> is about 5% higher.

I hope this gap should get closed with pghot_freq_threshold=1.

> 
>>> Finally, we've seen some issues with pghot causing userspace crashes,
>>> which leads us to believe that there might be some bugs that manifest
>>> in userspace as well (we have not been able to find out where in pghot
>>> these bugs are introduced).
>>
>> That's something I have never seen till now with any of the benchmarks that I
>> have run. Any kernel messages attributing the failure to pghot? Anything
>> attributable to access reporting or  kmigrated threads?
> 
> We're doing A/B testing with NUMAB2 only and pghot, and we are seeing
> the userspace crashes in the pghot host only. It seems like there
> are some segfaults trying to reference ip 000... and a warning that
> references "Too many tries to read user space".
> 
> I don't have anything in particular that would direct us right to pghot,
> but this crash has repeatedly come up, only on pghot (across reboots)
> and not on our NUMAB2 worklods. 

When you find out more about this failure, please let me know.

Regards,
Bharata.