Re: [PATCH v3 1/5] mm/memory_hotplug: introduce MEM_PREPARE_ONLINE/MEM_FINISH_OFFLINE notifiers

David Hildenbrand
2023-11-27T15:16:23Z

On 27.11.23 09:20, Sumanth Korikkar wrote:
> Introduce MEM_PREPARE_ONLINE/MEM_FINISH_OFFLINE memory notifiers to
> prepare the transition of memory to and from a physically accessible
> state. This enhancement is crucial for implementing the "memmap on
> memory" feature for s390 in a subsequent patch.
>
> Platforms such as x86 can support physical memory hotplug via ACPI. When
> there is physical memory hotplug, ACPI event leads to the memory
> addition with the following callchain:
> acpi_memory_device_add()
> -> acpi_memory_enable_device()
> -> __add_memory()
>
> After this, the hotplugged memory is physically accessible, and altmap
> support prepared, before the "memmap on memory" initialization in
> memory_block_online() is called.
>
> On s390, memory hotplug works in a different way. The available hotplug
> memory has to be defined upfront in the hypervisor, but it is made
> physically accessible only when the user sets it online via sysfs,
> currently in the MEM_GOING_ONLINE notifier. This is too late and "memmap
> on memory" initialization is performed before calling MEM_GOING_ONLINE
> notifier.
>
> During the memory hotplug addition phase, altmap support is prepared and
> during the memory onlining phase s390 requires memory to be physically
> accessible and then subsequently initiate the "memmap on memory"
> initialization process.
>
> The memory provider will handle new MEM_PREPARE_ONLINE /
> MEM_FINISH_OFFLINE notifications and make the memory accessible.
>
> The mhp_flag MHP_OFFLINE_INACCESSIBLE is introduced and is relevant when
> used along with MHP_MEMMAP_ON_MEMORY, because the altmap cannot be
> written (e.g., poisoned) when adding memory -- before it is set online.
> This allows for adding memory with an altmap that is not currently made
> available by a hypervisor. When onlining that memory, the hypervisor can
> be instructed to make that memory accessible via the new notifiers and
> the onlining phase will not require any memory allocations, which is
> helpful in low-memory situations.
>
> All architectures ignore unknown memory notifiers. Therefore, the
> introduction of these new notifiers does not result in any functional
> modifications across architectures.
>
> Suggested-by: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
> Suggested-by: David Hildenbrand <david@redhat.com>
> Signed-off-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
> ---

[...]

> };
>
> /*
> diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
> index 7a5fc89a8652..ac7cfc09502d 100644
> --- a/mm/memory_hotplug.c
> +++ b/mm/memory_hotplug.c
> @@ -1083,8 +1083,25 @@ void adjust_present_page_count(struct page *page, struct memory_group *group,
> group->present_kernel_pages += nr_pages;
> }
>
> +static void page_init_poison_with_resched(unsigned long start_pfn, unsigned long nr_pages)
> +{
> + const unsigned long end_pfn = start_pfn + nr_pages;
> + unsigned long pfn, cur_nr_pages;
> +
> + /* Poison struct pages because they are now uninitialized again. */
> + for (pfn = start_pfn; pfn < end_pfn; pfn += cur_nr_pages) {
> + cond_resched();
> +
> + /* Select all remaining pages up to the next section boundary */
> + cur_nr_pages =
> + min(end_pfn - pfn, SECTION_ALIGN_UP(pfn + 1) - pfn);
> + page_init_poison(pfn_to_page(pfn),
> + sizeof(struct page) * cur_nr_pages);
> + }
> +}
> +
> int mhp_init_memmap_on_memory(unsigned long pfn, unsigned long nr_pages,
> - struct zone *zone)
> + struct zone *zone, bool mhp_off_inaccessible)
> {
> unsigned long end_pfn = pfn + nr_pages;
> int ret, i;
> @@ -1092,7 +1109,14 @@ int mhp_init_memmap_on_memory(unsigned long pfn, unsigned long nr_pages,
> ret = kasan_add_zero_shadow(__va(PFN_PHYS(pfn)), PFN_PHYS(nr_pages));
> if (ret)
> return ret;
> -
> + /*
> + * Memory block is accessible at this stage and hence poison the struct
> + * pages now. If the memory block is accessible during memory hotplug
> + * addition phase, then page poisining is already performed in
> + * sparse_add_section().
> + */
> + if (mhp_off_inaccessible)
> + page_init_poison_with_resched(pfn, nr_pages);

Can you elaborate why a simple page_init_poison() as for
sparse_add_section() is insufficient?

Apart from that looks good.

Ideally, we'd be updating altmap->inaccessible as we online/offline
memory. But then, we'd have to remember MHP_OFFLINE_INACCESSIBLE somehow
differently.

--
Cheers,

David / dhildenb