Re: [RFC v3 PATCH 0/5] Promotion of Unmapped Page Cache Folios.

Huang, Ying
2025-01-23T03:46:58Z

Gregory Price <gourry@gourry.net> writes:

> On Wed, Jan 22, 2025 at 07:16:03PM +0800, Huang, Ying wrote:
>> Hi, Gregory,
>> > Test process:
>> > In each test, we do a linear read of a 128GB file into a buffer
>> > in a loop.
>>
>> IMHO, the linear reading isn't a very good test case for promotion. You
>> cannot test the hot-page selection algorithm. I think that it's better
>> to use something like normal accessing pattern. IIRC, it is available
>> in fio test suite.
>>
>
> Oh yes, I don't plan to drop RFC until I can get a real workload and
> probably fio running under this. This patch set is varying priority for
> me at the moment so the versions will take some time. My goal is to
> have something a bit more solid by LSF/MM, but not before.

No problem.

>> > 1) file allocated in DRAM with mechanisms off
>> > 2) file allocated in DRAM with balancing on but promotion off
>> > 3) file allocated in DRAM with balancing and promotion on
>> > (promotion check is negative because all pages are top tier)
>> > 4) file allocated in CXL with mechanisms off
>> > 5) file allocated in CXL with mechanisms on
>> >
>> > | 1 | 2 | 3 | 4 | 5 |
>> > | DRAM Base | Promo On | TopTier Chk | CXL Base | Post-Promotion |
>> > | 7.5804 | 7.7586 | 7.9726 | 9.75 | 7.8941 |
>>
>> For 3, we can check whether the folio is in top-tier as the first step.
>> Will that introduce measurable overhead?
>>
>
> That is basically what 2 vs 3 is doing.
>
> Test 2 shows overhead of TPP on + pagecache promo off
> Test 3 shows overhead of TPP+Promo on, but all the memory is on top tier
>
> This shows the check as to whether the folio is in the top tier is
> actually somewhat expensive (~5% compared to baseline, ~2.7% compared to
> TPP-on Promo-off).

This is unexpected. Can we try to optimize it? For example, via using
a nodemask? node_is_toptier() is used in the mapped pages promotion
too (1 vs. 2 above). I guess that the optimization can reduce the
overhead there with measurable difference too.

> The goal of this linear, simple test is to isolate test behavior from
> the overhead - that makes it easy to test each individual variable (TPP,
> promo, top tier, etc) and see relative overheads.
>
> This basically gives us a reasonable floor/ceiling of expected overhead.
> If we see something wildly different than this during something like FIO
> or a real workload, then we'll know we missed something.
>
>> >
>> > This could be further limited by limiting the promotion rate via the
>> > existing knob, or by implementing a new knob detached from the existing
>> > promotion rate. There are merits to both approach.
>>
>> Have you tested with the existing knob? Whether does it help?
>>
>
> Not yet, this fell off my priority list before I could do additional
> testing. I will add that to my backlog.

No problem.

---
Best Regards,
Huang, Ying