Re: [PATCH v5 09/16] kexec: enable KHO support for memory preservation

[Pratyush Yadav <ptyadav@amazon.de>]

On Fri, Apr 04 2025, Jason Gunthorpe wrote:

> On Fri, Apr 04, 2025 at 04:53:13PM +0300, Mike Rapoport wrote:
[...]
>> Most drivers do not use folios
>
> Yes they do, either through kmalloc or through alloc_page/etc. "folio"
> here is just some generic word meaning memory from the buddy allocator.
>
> The big question on my mind is if we need a way to preserve slab
> objects as well..

Only if the objects in the slab cache are of a format that doesn't
change, and I am not sure if that is the case anywhere. Maybe a driver
written with KHO in mind would find it useful, but that's way down the
line.

>
>> and for preserving memfd* and hugetlb we'd need to have some dance
>> around that memory anyway.
>
> memfd is all folios - what do you mean?
>
> hugetlb is moving toward folios.. eg guestmemfd is supposed to be
> taking the hugetlb special stuff and turning it into folios.
>
>> So I think kho_preserve_folio() would be a part of the fdbox or
>> whatever that functionality will be called.
>
> It is part of KHO. Preserving the folios has to be sequenced with
> starting the buddy allocator, and that is KHO's entire responsibility.
>
> I could see something like preserving slab being in a different layer,
> built on preserving folios.

Agree with both points.

[...]
>> As for the optimizations of memblock reserve path, currently it what hurts
>> the most in my and Pratyush experiments. They are not very representative,
>> but still, preserving lots of pages/folios spread all over would have it's
>> toll on the mm initialization.
>
>> And I don't think invasive changes to how
>> buddy and memory map initialization are the best way to move forward and
>> optimize that.
>
> I'm pretty sure this is going to be the best performance path, but I
> have no idea how invasive it would be to the buddy alloactor to make
> it work.

I don't imagine it would be that invasive TBH. memblock_free_pages()
already checks for kmsan_memblock_free_pages() or
early_page_initialised(), it can also check for kho_page() just as
easily.

>
>> Quite possibly we'd want to be able to minimize amount of *ranges*
>> that we preserve.
>
> I'm not sure, that seems backwards to me, we really don't want to have
> KHO mem zones! So I think optimizing for, and thinking about ranges
> doesn't make sense.
>
> The big ranges will arise naturally beacuse things like hugetlb
> reservations should all be contiguous and the resulting folios should
> all be allocated for the VM and also all be contigous. So vast, vast
> amounts of memory will be high order and contiguous.

Yes, and those can work quite well with table + bitmaps too.

[...]

-- 
Regards,
Pratyush Yadav