Re: [RFC PATCH v2 0/7] Introduce persistent memory pool

Re: [RFC PATCH v2 0/7] Introduce persistent memory pool

[Baoquan He]

Hi Stanislav,

On 09/25/23 at 02:27pm, Stanislav Kinsburskii wrote:
> This patch introduces a memory allocator specifically tailored for
> persistent memory within the kernel. The allocator maintains
> kernel-specific states like DMA passthrough device states, IOMMU state, and
> more across kexec.

Can you give more details about how this persistent memory pool will be
utilized in a actual scenario? I mean, what problem have you met so that
you have to introduce persistent memory pool to solve it?

Thanks
Baoquan

> 
> The current implementation provides a foundation for custom solutions that
> may be developed in the future. Although the design is kept concise and
> straightforward to encourage discussion and feedback, it remains fully
> functional.
> 
> The persistent memory pool builds upon the continuous memory allocator
> (CMA) and ensures CMA state persistency across kexec by incorporating the
> CMA bitmap into the memory region instead of allocation it from kernel
> memory.
> 
> Persistent memory pool metadata is passed across kexec by using Flattened
> Device Tree, which is added as another kexec segment for x86 architecture.
> 
> Potential applications include:
> 
>   1. Enabling various in-kernel entities to allocate persistent pages from
>      a unified memory pool, obviating the need for reserving multiple
>      regions.
> 
>   2. For in-kernel components that need the allocation address to be
>      retained on kernel kexec, this address can be exposed to user space
>      and subsequently passed through the command line.
> 
>   3. Distinct subsystems or drivers can set aside their region, allocating
>      a segment for their persistent memory pool, suitable for uses such as
>      file systems, key-value stores, and other applications.
> 
> Notes:
> 
>   1. The last patch of the series represents a use case for the feature.
>      However, the patch won't compile and is for illustrative purposes only
>      as the code being patched hasn't been merged yet.
> 
>   2. The code being patched is currently under review by the community. The
>      series is named "Introduce /dev/mshv drivers":
> 
>          https://lkml.org/lkml/2023/9/22/1117
> 
> 
> Changes since v1:
> 
>   1. Persistent memory pool is now a wrapper on top of CMA instead of being a
>      new allocator.
> 
>   2. Persistent memory pool metadata doesn't belong to the pool anymore and
>      is now passed via Flattened Device Tree instead over kexec to the new
>      kernel.
> 
> The following series implements...
> 
> ---
> 
> Stanislav Kinsburskii (7):
>       kexec_file: Add fdt modification callback support
>       x86: kexec: Transfer existing fdt to the new kernel
>       x86: kexec: Enable fdt modification in callbacks
>       pmpool: Introduce persistent memory pool
>       pmpool: Update device tree on kexec
>       pmpool: Restore state from device tree post-kexec
>       Drivers: hv: Allocate persistent pages for root partition
> 
> 
>  arch/x86/Kconfig                  |   16 +++
>  arch/x86/kernel/kexec-bzimage64.c |   97 +++++++++++++++++
>  drivers/hv/hv_common.c            |   13 ++
>  include/linux/kexec.h             |    7 +
>  include/linux/pmpool.h            |   22 ++++
>  kernel/kexec_file.c               |   24 ++++
>  mm/Kconfig                        |    9 ++
>  mm/Makefile                       |    1 
>  mm/pmpool.c                       |  208 +++++++++++++++++++++++++++++++++++++
>  9 files changed, 394 insertions(+), 3 deletions(-)
>  create mode 100644 include/linux/pmpool.h
>  create mode 100644 mm/pmpool.c
> 
> 
> _______________________________________________
> kexec mailing list
> kexec@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec
> 

Re: [RFC PATCH v2 0/7] Introduce persistent memory pool

[Baoquan He]

On 09/27/23 at 09:13am, Stanislav Kinsburskii wrote:
> On Wed, Sep 27, 2023 at 01:44:38PM +0800, Baoquan He wrote:
> > Hi Stanislav,
> > 
> > On 09/25/23 at 02:27pm, Stanislav Kinsburskii wrote:
> > > This patch introduces a memory allocator specifically tailored for
> > > persistent memory within the kernel. The allocator maintains
> > > kernel-specific states like DMA passthrough device states, IOMMU state, and
> > > more across kexec.
> > 
> > Can you give more details about how this persistent memory pool will be
> > utilized in a actual scenario? I mean, what problem have you met so that
> > you have to introduce persistent memory pool to solve it?
> > 
> 
> The major reason we have at the moment, is that Linux root partition
> running on top of the Microsoft hypervisor needs to deposit pages to
> hypervisor in runtime, when hypervisor runs out of memory.
> "Depositing" here means, that Linux passes a set of its PFNs to the
> hypervisor via hypercall, and hypervisor then uses these pages for its
> own needs.
> 
> Once deposited, these pages can't be accessed by Linux anymore and thus
> must be preserved in "used" state across kexec, as hypervisor state is
> unware of kexec. In the same time, these pages can we withdrawn when
> usused. Thus, an allocator persistent across kexec looks reasonable for
> this particular matter.

Thanks for these details.
 
The deposit and withdraw remind me the Balloon driver, David's virtio-mem,
DLPAR on ppc which can hot increasing or shrinking phisical memory on guest
OS. Can't microsoft hypervisor do the similar thing to reclaim or give
back the memory from or to the 'Linux root partition' running on top of
the hypervisor?

Thanks
Baoquan

Re: [RFC PATCH v2 0/7] Introduce persistent memory pool

[Dave Hansen]

On 9/27/23 09:13, Stanislav Kinsburskii wrote:
> Once deposited, these pages can't be accessed by Linux anymore and thus
> must be preserved in "used" state across kexec, as hypervisor state is
> unware of kexec.

If Linux can't access them, they're not RAM any more.  I'd much rather
remove them from the memory map and move on with life rather than
implement a bunch of new ABI that's got to be handed across kernels.

Re: [RFC PATCH v2 0/7] Introduce persistent memory pool

[Dave Hansen]

On 9/27/23 16:25, Stanislav Kinsburskii wrote:
> On Thu, Sep 28, 2023 at 06:22:54AM -0700, Dave Hansen wrote:
>> On 9/27/23 09:13, Stanislav Kinsburskii wrote:
>>> Once deposited, these pages can't be accessed by Linux anymore and thus
>>> must be preserved in "used" state across kexec, as hypervisor state is
>>> unware of kexec.
>>
>> If Linux can't access them, they're not RAM any more.  I'd much rather
>> remove them from the memory map and move on with life rather than
>> implement a bunch of new ABI that's got to be handed across kernels.
> 
> Could you elaborate more on the new ABIs? FDT is handled by x86 already,
> and passing it over kexec looks like a natural extension.
> Also, adding more state to it also doens't look like a new ABI.
> Or does it?

FDT makes it easier to pass arbitrary data around, but you're still
creating a new "default_pmpool" device tree node on one end and
consuming it on the other.  That's a new ABI in my book.

> Let me also comment on removing this regions from the memory map. The
> major peculiarity here is that hypervisor distinguish between the pages,
> deposited for guests to rnu and the pages deposited for the Linux root
> partition to keep the guest-related portion of hypervisor state in the
> root partition. And the latter is the matter in question.
> 
> We can indeed isolate and deposit a excessive amount of memory upfront
> in hope that hypervisor will never get into the situation, when it needs
> more memory.
> However, it's not reliable, as the amount of memory will always be an
> estimation, depending on the number of expected guests, guest-attached
> devices, etc. And this becomes even a bigger problem when most of the
> memory is already removed from the memory map to host guest partitions.
> It's also not efficient as the amount of memory required by hypervisor
> can grow or shrink depending on the use case or host configuration, and
> deposting excessive amount of memory will be a waste.
> 
> But, actually, the idea of removing the pages from memory map was
> reflected to some extent in the first version of this proposal,
> so let me elaborate on it a bit.
> 
> Effectively, instead of reserving and depositing a lot of memory to
> hypervisor upfront, the memory can be allocated from kernel memory when
> needed and then returned back when unused.
> This would still require pages removal from the memory map upon kexec,
> but that's another problem.

Let's distill this down a bit.

I agree that it's a waste to reserve an obscene amount of memory up
front for all guests for rare cases.  Having the amount of consumed
memory grow is a nice feature.

You can also quite easily *shrink* the amount of memory on a given
kernel without new code.  Right?

The problem comes when you've grown the footprint of hypervisor-donated
memory, kexec, and *THEN* want to shrink it.  That's what needs new
metadata to be communicated over to the new kernel.

1. Boot some kernel
2. Grow the deposited memory a bunch
3. Kexec
4. Shrink the deposited memory

Right?

That's where you lose me.

Can't the deposited memory just be shrunk before kexec?  Surely there
aren't a bunch of pathological things consuming that memory right before
kexec, which is basically a reboot.

Re: [RFC PATCH v2 0/7] Introduce persistent memory pool

[David Hildenbrand]

On 28.09.23 12:25, Baoquan He wrote:
> On 09/27/23 at 09:13am, Stanislav Kinsburskii wrote:
>> On Wed, Sep 27, 2023 at 01:44:38PM +0800, Baoquan He wrote:
>>> Hi Stanislav,
>>>
>>> On 09/25/23 at 02:27pm, Stanislav Kinsburskii wrote:
>>>> This patch introduces a memory allocator specifically tailored for
>>>> persistent memory within the kernel. The allocator maintains
>>>> kernel-specific states like DMA passthrough device states, IOMMU state, and
>>>> more across kexec.
>>>
>>> Can you give more details about how this persistent memory pool will be
>>> utilized in a actual scenario? I mean, what problem have you met so that
>>> you have to introduce persistent memory pool to solve it?
>>>
>>
>> The major reason we have at the moment, is that Linux root partition
>> running on top of the Microsoft hypervisor needs to deposit pages to
>> hypervisor in runtime, when hypervisor runs out of memory.
>> "Depositing" here means, that Linux passes a set of its PFNs to the
>> hypervisor via hypercall, and hypervisor then uses these pages for its
>> own needs.
>>
>> Once deposited, these pages can't be accessed by Linux anymore and thus
>> must be preserved in "used" state across kexec, as hypervisor state is
>> unware of kexec. In the same time, these pages can we withdrawn when
>> usused. Thus, an allocator persistent across kexec looks reasonable for
>> this particular matter.
> 
> Thanks for these details.
>   
> The deposit and withdraw remind me the Balloon driver, David's virtio-mem,
> DLPAR on ppc which can hot increasing or shrinking phisical memory on guest
> OS. Can't microsoft hypervisor do the similar thing to reclaim or give
> back the memory from or to the 'Linux root partition' running on top of
> the hypervisor?

virtio-mem was designed with kexec support in mind. You only expose the 
initial memory to the second kernel, and that memory can never have such 
holes. That does not apply to memory ballooning implementations, like 
Hyper-V dynamic memory.

In the virtio-mem paper I have the following:

"In our experiments, Hyper-V VMs crashed reliably when
trying to use kexec under Linux for fast OS reboots with
an inflated balloon. Other memory ballooning mechanisms
either have to temporarily deflate the whole balloon or al-
low access to inflated memory, which is undesired in cloud
environments."

I remember XEN does something elaborate, whereby they allow access to 
all inflated memory during reboot, but limit the total number of pages 
they will hand out. IIRC, you then have to work around things like 
"Windows initializes all memory with 0s when booting, and cope with 
that". So there are ways how hypervisors handled that in the past.

-- 
Cheers,

David / dhildenb

Re: [RFC PATCH v2 0/7] Introduce persistent memory pool

[David Hildenbrand]

On 28.09.23 15:22, Dave Hansen wrote:
> On 9/27/23 09:13, Stanislav Kinsburskii wrote:
>> Once deposited, these pages can't be accessed by Linux anymore and thus
>> must be preserved in "used" state across kexec, as hypervisor state is
>> unware of kexec.
> 
> If Linux can't access them, they're not RAM any more.  I'd much rather
> remove them from the memory map and move on with life rather than
> implement a bunch of new ABI that's got to be handed across kernels.

The motivation of handling kexec (faster?) in a hyper-v domain doesn't 
sound particularly compelling got me for such features. If you inflated 
memory, just don't allow to kexec. It's been broken for years IIUC.

Maybe the other use cases are more "relevant".

-- 
Cheers,

David / dhildenb

Re: [RFC PATCH v2 0/7] Introduce persistent memory pool

[Dave Hansen]

On 9/28/23 10:35, David Hildenbrand wrote:
> On 28.09.23 15:22, Dave Hansen wrote:
>> On 9/27/23 09:13, Stanislav Kinsburskii wrote:
>>> Once deposited, these pages can't be accessed by Linux anymore and thus
>>> must be preserved in "used" state across kexec, as hypervisor state is
>>> unware of kexec.
>>
>> If Linux can't access them, they're not RAM any more.  I'd much rather
>> remove them from the memory map and move on with life rather than
>> implement a bunch of new ABI that's got to be handed across kernels.
> 
> The motivation of handling kexec (faster?) in a hyper-v domain doesn't
> sound particularly compelling got me for such features. If you inflated
> memory, just don't allow to kexec. It's been broken for years IIUC.

That's a good point.  What prevents deflating before kexec?

Re: [RFC PATCH v2 0/7] Introduce persistent memory pool

[Dave Hansen]

On 9/27/23 17:02, Stanislav Kinsburskii wrote:
> On Thu, Sep 28, 2023 at 10:29:32AM -0700, Dave Hansen wrote:
...
> Well, not exactly. That's something I'd like to have indeed, but from my
> POV this goal is out of scope of discussion at the moment.
> Let me try to express it the same way you did above:
> 
> 1. Boot some kernel
> 2. Grow the deposited memory a bunch
> 5. Kexec
> 4. Kernel panic due to GPF upon accessing the memory deposited to
> hypervisor.

I basically consider this a bug in the first kernel.  It *can't* kexec
when it's left RAM in shambles.  It doesn't know what features the new
kernel has and whether this is even safe.

Can the new kernel even read the new device tree data?

>> Can't the deposited memory just be shrunk before kexec?  Surely there
>> aren't a bunch of pathological things consuming that memory right before
>> kexec, which is basically a reboot.
> 
> In general it can. But for this to happen hypervisor needs to release
> this memory. And it can release the memory iff the guests are stopped.
> And stopping the guests during kexec isn't something we want to have in the
> long run.
> Also, even if we stop the guests before kexec, we need to restart them
> after boot meaning we have to deposit the pages once again.
> All this: stopping the guests, withdrawing the pages upon kexec,
> allocating after boot and depostiting them again significatnly affect
> guests downtime.

Ahh, and you're presumably kexec'ing in the first place because you've
got a bug in the first kernel and you want a second kernel with fewer bugs.

I still think the only way this will possibly work when kexec'ing both
old and new kernels is to do it with the memory maps that *all* kernels
can read.

Can the hypervisor be improved to make this release operation faster?

RE: [EXTERNAL] Re: [RFC PATCH v2 0/7] Introduce persistent memory pool

[KY Srinivasan]

> -----Original Message-----
> From: Dave Hansen <dave.hansen@intel.com>
> Sent: Thursday, September 28, 2023 10:38 AM
> To: David Hildenbrand <david@redhat.com>; Stanislav Kinsburskii
> <skinsburskii@linux.microsoft.com>; Baoquan He <bhe@redhat.com>
> Cc: tglx@linutronix.de; mingo@redhat.com; bp@alien8.de;
> dave.hansen@linux.intel.com; x86@kernel.org; hpa@zytor.com;
> ebiederm@xmission.com; akpm@linux-foundation.org;
> stanislav.kinsburskii@gmail.com; corbet@lwn.net; linux-
> kernel@vger.kernel.org; kexec@lists.infradead.org; linux-mm@kvack.org; KY
> Srinivasan <kys@microsoft.com>; jgowans@amazon.com; wei.liu@kernel.org;
> arnd@arndb.de; gregkh@linuxfoundation.org; graf@amazon.de;
> pbonzini@redhat.com
> Subject: [EXTERNAL] Re: [RFC PATCH v2 0/7] Introduce persistent memory pool
> 
> On 9/28/23 10:35, David Hildenbrand wrote:
> > On 28.09.23 15:22, Dave Hansen wrote:
> >> On 9/27/23 09:13, Stanislav Kinsburskii wrote:
> >>> Once deposited, these pages can't be accessed by Linux anymore and
> >>> thus must be preserved in "used" state across kexec, as hypervisor
> >>> state is unware of kexec.
> >>
> >> If Linux can't access them, they're not RAM any more.  I'd much
> >> rather remove them from the memory map and move on with life rather
> >> than implement a bunch of new ABI that's got to be handed across kernels.
> >
> > The motivation of handling kexec (faster?) in a hyper-v domain doesn't
> > sound particularly compelling got me for such features. If you
> > inflated memory, just don't allow to kexec. It's been broken for years IIUC.
> 
> That's a good point.  What prevents deflating before kexec?


The guest has returned the memory to the host as part of inflating the balloon and so,
this memory has to be returned by the host before you can deflate. The best option
is to not kexec when the memory has been returned to the host.

Regards,

K. Y

Re: [RFC PATCH v2 0/7] Introduce persistent memory pool

[Dave Hansen]

On 9/27/23 17:38, Stanislav Kinsburskii wrote:
> On Thu, Sep 28, 2023 at 11:00:12AM -0700, Dave Hansen wrote:
>> On 9/27/23 17:02, Stanislav Kinsburskii wrote:
>>> On Thu, Sep 28, 2023 at 10:29:32AM -0700, Dave Hansen wrote:
>> ...
>>> Well, not exactly. That's something I'd like to have indeed, but from my
>>> POV this goal is out of scope of discussion at the moment.
>>> Let me try to express it the same way you did above:
>>>
>>> 1. Boot some kernel
>>> 2. Grow the deposited memory a bunch
>>> 5. Kexec
>>> 4. Kernel panic due to GPF upon accessing the memory deposited to
>>> hypervisor.
>>
>> I basically consider this a bug in the first kernel.  It *can't* kexec
>> when it's left RAM in shambles.  It doesn't know what features the new
>> kernel has and whether this is even safe.
>>
> 
> Could you elaborate more on why this is a bug in the first kernel?
> Say, kernel memory can be allocated in big physically consequitive
> chunks by the first kernel for depositing. The information about these
> chunks is then passed the the second kernel via FDT or even command
> line, so the seconds kernel can reserve this region during booting.
> What's wrong with this approach?

How do you know the second kernel can parse the FDT entry or the
command-line you pass to it?

>> Can the new kernel even read the new device tree data?
> 
> I'm not sure I understand the question, to be honest.
> Why can't it? This series contains code parts for both first and seconds
> kernels.

How do you know the second kernel isn't the version *before* this series
gets merged?

...
>> I still think the only way this will possibly work when kexec'ing both
>> old and new kernels is to do it with the memory maps that *all* kernels
>> can read.
> 
> Could you elaborate more on this?
> The avaiable memory map actually stays the same for both kernels. The
> difference here can be in a different list of memory regions to reserve,
> when the first kernel allocated and deposited another chunk, and thus
> the second kernel needs to reserve this memory as a new region upon
> booting.

Please take a step back from your implementation for a moment.  There
are two basic design points that need to be considered.

First, *must* "System RAM" (according to the memory map) be persisted
across kexec?  If no, then there's no problem to solve and we can stop
this thread.  If yes, then some mechanism must be used to tell the new
kernel that the "System RAM" in the memory map is not normal RAM.

Second, *if* we agree that some data must communicate across kexec, then
what mechanism should be used?  You're arguing for a new mechanism that
only new kernels can use.  I'm arguing that you should likely reuse an
existing mechanism (probably the UEFI/e820 maps) so that *ALL* kernels
can consume the information, old and new.

I'm not convinced that this series is going in the right direction on
either of those points.

> Can all this considered, as, say, the first kernel uses device tree to
> inform the second kernel about the memory regions to reserve?
> In this case the first kernel behaves a bit like a firmware piece for
> the second one.
> 
>> Can the hypervisor be improved to make this release operation faster?
> 
> I guess it can, but shutting down guests contributes to downtime the
> most. And without shutting down the guests the deposited memory can't be
> withdrawn.

Do you really need to fully shut down each guest?  Or do you just need
to get them to a quiescent state where the hypervisor and devices aren't
writing to the deposited memory?

Re: [RFC PATCH v2 0/7] Introduce persistent memory pool

[Baoquan He]

On 09/27/23 at 07:46pm, Stanislav Kinsburskii wrote:
> On Thu, Sep 28, 2023 at 12:16:31PM -0700, Dave Hansen wrote:
> > On 9/27/23 17:38, Stanislav Kinsburskii wrote:
> > > On Thu, Sep 28, 2023 at 11:00:12AM -0700, Dave Hansen wrote:
> > >> On 9/27/23 17:02, Stanislav Kinsburskii wrote:
> > >>> On Thu, Sep 28, 2023 at 10:29:32AM -0700, Dave Hansen wrote:
> > >> ...
> > >>> Well, not exactly. That's something I'd like to have indeed, but from my
> > >>> POV this goal is out of scope of discussion at the moment.
> > >>> Let me try to express it the same way you did above:
> > >>>
> > >>> 1. Boot some kernel
> > >>> 2. Grow the deposited memory a bunch
> > >>> 5. Kexec
> > >>> 4. Kernel panic due to GPF upon accessing the memory deposited to
> > >>> hypervisor.
> > >>
> > >> I basically consider this a bug in the first kernel.  It *can't* kexec
> > >> when it's left RAM in shambles.  It doesn't know what features the new
> > >> kernel has and whether this is even safe.
> > >>
> > > 
> > > Could you elaborate more on why this is a bug in the first kernel?
> > > Say, kernel memory can be allocated in big physically consequitive
> > > chunks by the first kernel for depositing. The information about these
> > > chunks is then passed the the second kernel via FDT or even command
> > > line, so the seconds kernel can reserve this region during booting.
> > > What's wrong with this approach?
> > 
> > How do you know the second kernel can parse the FDT entry or the
> > command-line you pass to it?
> > 
> > >> Can the new kernel even read the new device tree data?
> > > 
> > > I'm not sure I understand the question, to be honest.
> > > Why can't it? This series contains code parts for both first and seconds
> > > kernels.
> > 
> > How do you know the second kernel isn't the version *before* this series
> > gets merged?
> > 
> 
> The answer to both questions above is the following: the feature is deployed
> fleed-wide first, and enabled only upon the next deployment.
> It worth mentioning, that fleet-wide deployments usually don't need to support
> updates to a version older that the previous one.
> Also, since kexec is initialited by user space, it always can be
> enlightened about kernel capabilities and simply don't kexec to an
> incompatible kernel version.
> One more bit to mention, that it real life this problme exists only
> during initial transition, as once the upgrade to a kernel with a
> feature has happened, there won't be a revert to a versoin without it.
> 
> > ...
> > >> I still think the only way this will possibly work when kexec'ing both
> > >> old and new kernels is to do it with the memory maps that *all* kernels
> > >> can read.
> > > 
> > > Could you elaborate more on this?
> > > The avaiable memory map actually stays the same for both kernels. The
> > > difference here can be in a different list of memory regions to reserve,
> > > when the first kernel allocated and deposited another chunk, and thus
> > > the second kernel needs to reserve this memory as a new region upon
> > > booting.
> > 
> > Please take a step back from your implementation for a moment.  There
> > are two basic design points that need to be considered.
> > 
> > First, *must* "System RAM" (according to the memory map) be persisted
> > across kexec?  If no, then there's no problem to solve and we can stop
> > this thread.  If yes, then some mechanism must be used to tell the new
> > kernel that the "System RAM" in the memory map is not normal RAM.
> > 
> > Second, *if* we agree that some data must communicate across kexec, then
> > what mechanism should be used?  You're arguing for a new mechanism that
> > only new kernels can use.  I'm arguing that you should likely reuse an
> > existing mechanism (probably the UEFI/e820 maps) so that *ALL* kernels
> > can consume the information, old and new.
> > 
> 
> I'd answer yes, "System MAP" must be persisted across kexec.
> Could you elaborate on why there should be a mechanism to tell the
> kernel anything special about the existent "System map" in this context?
> Say, one can reserve a CMA region (or a crash kernel region, etc), store
> there some data, and then pass it across kexec. Reserved CMA region will
> still be a part of the "System MAP", won't it?

Well, I haven't gone through all the discusison thread and clearly got
your intention and motivation. But here I have to say there's
misunderstanding. At least I am astonished when I heard the above
description. Who said a CMA region or a crahs kernel region need be
passed across kexec. Think kexec as a bootloader, in essence it's no
different than any other bootloader. When it jumps to 2nd kernel, the
whole system will be booted up and reconstructed on the system resources.
All the difference kexec has is it won't go through firmware to do those
detecting/testing/init. If the intentionn is to preserve any state or
region in 1st kernel, you absolutely got it wrong.

This is not the first time people want to put burden on kexec because
of a specifica scenario, and this is not the 2nd time, and not 3rd time
in the recent 2 years. But I would say please think about what is kexec
reboot, what we expect it to do, whether the problem be fixed in its own
side.

Re: [RFC PATCH v2 0/7] Introduce persistent memory pool

[Shutemov, Kirill]

On Wed, Sep 27, 2023 at 07:46:36PM -0700, Stanislav Kinsburskii wrote:
> I'd answer yes, "System MAP" must be persisted across kexec.
> Could you elaborate on why there should be a mechanism to tell the
> kernel anything special about the existent "System map" in this context?
> Say, one can reserve a CMA region (or a crash kernel region, etc), store
> there some data, and then pass it across kexec. Reserved CMA region will
> still be a part of the "System MAP", won't it?

Em. When crash kernel starts all System RAM of the the first kernel
becomes E820_TYPE_RESERVED and only memory pre-allocated for crash
scenario becomes E820_TYPE_RAM. See crash_setup_memmap_entries().

Can't you go the same path? Report all deposited memory as
E820_TYPE_RESERVED.

Or do you have too many deposited memory ranges, so we would run out of
e820 entries?

-- 
  Kiryl Shutsemau / Kirill A. Shutemov