+ kexec_core-accept-unaccepted-kexec-segments-destination-addresses.patch added to mm-nonmm-unstable branch

+ kexec_core-accept-unaccepted-kexec-segments-destination-addresses.patch added to mm-nonmm-unstable branch

[Andrew Morton]

The patch titled
     Subject: kexec_core: accept unaccepted kexec segments' destination addresses
has been added to the -mm mm-nonmm-unstable branch.  Its filename is
     kexec_core-accept-unaccepted-kexec-segments-destination-addresses.patch

This patch will shortly appear at
     https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/kexec_core-accept-unaccepted-kexec-segments-destination-addresses.patch

This patch will later appear in the mm-nonmm-unstable branch at
    git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

Before you just go and hit "reply", please:
   a) Consider who else should be cc'ed
   b) Prefer to cc a suitable mailing list as well
   c) Ideally: find the original patch on the mailing list and do a
      reply-to-all to that, adding suitable additional cc's

*** Remember to use Documentation/process/submit-checklist.rst when testing your code ***

The -mm tree is included into linux-next via the mm-everything
branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
and is updated there every 2-3 working days

------------------------------------------------------
From: Yan Zhao <yan.y.zhao@intel.com>
Subject: kexec_core: accept unaccepted kexec segments' destination addresses
Date: Fri, 13 Dec 2024 17:54:49 +0800

In TDX, to run a linux guest, TDs (hardware-isolated VMs) must accept
before accessing private memory.  Accessing private memory before
acceptance is considered a fatal error and may result in the termination
of the TD.

The "accepting memory" operation in guest includes the following steps:
- trigger a VM-exit
- the host OS allocates a physical page and requests hardware to map the
  physical page to the GPA.
- initialize memory content to 0.
- encrypt the memory

For a Linux guest, eagerly accepting all memory during kernel boot can
slow down the boot process and cause unnecessary memory occupation on the
host for pages that may never be accessed.  Therefore, Linux guests
usually opt for a lazy mode to delay page acceptance operations by not
moving the pages to the buddy allocator's freelists.  Instead, the kernel
tracks memory in 4M units and places them in a zone->unaccepted_pages list
if any page in the entire 4M range is in an unaccepted state (even if part
of the memory range may have been accepted by firmware or the kernel). 
When the kernel does not have enough free pages, it will move memory from
the zone->unaccepted_pages list and accept it, ensuring that the memory is
accepted before moving it to the freelists and being available to the
buddy allocator.

The kexec segments' destination addresses are not allocated by the buddy
allocator.  Instead, they are searched from normal system RAM (top-down or
bottom-up) and exclude driver-managed memory, ACPI, persistent, and
reserved memory...  Although these addresses may fall within the memory
range managed by the buddy allocator (which must be in an accepted state),
they could also be outside that range and in an unaccepted state.

Since the kexec code will access the segments' destination addresses
during the kexec process by swapping their content with the segments'
source pages, it is necessary to accept the memory before performing the
swap operations.

Accept the destination addresses during the kexec load, immediately after
they pass sanity checks.  This ensures the code is located in a common
place shared by both the kexec_load and kexec_file_load system calls.

This will not conflict with the accounting in try_to_accept_memory_one()
since the accounting is set during kernel boot and decremented when pages
are moved to the freelists.  There is no harm in invoking accept_memory()
on a page before making it available to the buddy allocator.

No need to worry about re-accepting memory since accept_memory() checks
the unaccepted bitmap before accepting a memory page.

Although a user may perform kexec loading without ever triggering the
jump, it doesn't impact much since kexec loading is not in a
performance-critical path.  Additionally, the destination addresses are
always searched and found in the same location on a given system.

Changes to the destination address searching logic to locate only memory
in either unaccepted or accepted status are unnecessary and complicated.

Link: https://lkml.kernel.org/r/20241213095449.881-1-yan.y.zhao@intel.com
Signed-off-by: Yan Zhao <yan.y.zhao@intel.com>
Reviewed-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Baoquan He <bhe@redhat.com>
Cc: Jianxiong Gao <jxgao@google.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Baoquan he <bhe@redhat.com>
Cc: "Edgecombe, Rick P" <rick.p.edgecombe@intel.com>
Cc: Eric Biederman <ebiederm@xmission.com>
Cc: Kirill A. Shuemov <kirill.shutemov@linux.intel.com>
Cc: Yan Zhao <yan.y.zhao@intel.com>
Cc: Ashish Kalra <Ashish.Kalra@amd.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 kernel/kexec_core.c |   10 ++++++++++
 1 file changed, 10 insertions(+)

--- a/kernel/kexec_core.c~kexec_core-accept-unaccepted-kexec-segments-destination-addresses
+++ a/kernel/kexec_core.c
@@ -210,6 +210,16 @@ int sanity_check_segment_list(struct kim
 	}
 #endif
 
+	/*
+	 * The destination addresses are searched from system RAM rather than
+	 * being allocated from the buddy allocator, so they are not guaranteed
+	 * to be accepted by the current kernel.  Accept the destination
+	 * addresses before kexec swaps their content with the segments' source
+	 * pages to avoid accessing memory before it is accepted.
+	 */
+	for (i = 0; i < nr_segments; i++)
+		accept_memory(image->segment[i].mem, image->segment[i].memsz);
+
 	return 0;
 }
 
_

Patches currently in -mm which might be from yan.y.zhao@intel.com are

kexec_core-accept-unaccepted-kexec-segments-destination-addresses.patch

Re: + kexec_core-accept-unaccepted-kexec-segments-destination-addresses.patch added to mm-nonmm-unstable branch

[Eric W. Biederman]

Andrew Morton <akpm@linux-foundation.org> writes:

> The patch titled
>      Subject: kexec_core: accept unaccepted kexec segments' destination addresses
> has been added to the -mm mm-nonmm-unstable branch.  Its filename is
>      kexec_core-accept-unaccepted-kexec-segments-destination-addresses.patch
>
> This patch will shortly appear at
>      https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/kexec_core-accept-unaccepted-kexec-segments-destination-addresses.patch
>
> This patch will later appear in the mm-nonmm-unstable branch at
>     git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
>
> Before you just go and hit "reply", please:
>    a) Consider who else should be cc'ed
>    b) Prefer to cc a suitable mailing list as well
>    c) Ideally: find the original patch on the mailing list and do a
>       reply-to-all to that, adding suitable additional cc's
>
> *** Remember to use Documentation/process/submit-checklist.rst when testing your code ***
>
> The -mm tree is included into linux-next via the mm-everything
> branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
> and is updated there every 2-3 working days

My apologies for replying like this, but I can not quickly find this
patch elsewhere.

This actually can not work reliably.  The problem is that all of the
state of what has been lazily "accepted" and what has not will be lost.

If anything is going to happen in the case of kexec what needs to happen
is kexec_load needs to force the system out of lazy mode and all memory
needs to be accepted before kexec load runs.

The target kernel has no way of knowing what has been accepted and
not-accepted.  I don't think you can have a reasonable expectation
that the target kernel knows about accepted memory.

Eric


> From: Yan Zhao <yan.y.zhao@intel.com>
> Subject: kexec_core: accept unaccepted kexec segments' destination addresses
> Date: Fri, 13 Dec 2024 17:54:49 +0800
>
> In TDX, to run a linux guest, TDs (hardware-isolated VMs) must accept
> before accessing private memory.  Accessing private memory before
> acceptance is considered a fatal error and may result in the termination
> of the TD.
>
> The "accepting memory" operation in guest includes the following steps:
> - trigger a VM-exit
> - the host OS allocates a physical page and requests hardware to map the
>   physical page to the GPA.
> - initialize memory content to 0.
> - encrypt the memory
>
> For a Linux guest, eagerly accepting all memory during kernel boot can
> slow down the boot process and cause unnecessary memory occupation on the
> host for pages that may never be accessed.  Therefore, Linux guests
> usually opt for a lazy mode to delay page acceptance operations by not
> moving the pages to the buddy allocator's freelists.  Instead, the kernel
> tracks memory in 4M units and places them in a zone->unaccepted_pages list
> if any page in the entire 4M range is in an unaccepted state (even if part
> of the memory range may have been accepted by firmware or the kernel). 
> When the kernel does not have enough free pages, it will move memory from
> the zone->unaccepted_pages list and accept it, ensuring that the memory is
> accepted before moving it to the freelists and being available to the
> buddy allocator.
>
> The kexec segments' destination addresses are not allocated by the buddy
> allocator.  Instead, they are searched from normal system RAM (top-down or
> bottom-up) and exclude driver-managed memory, ACPI, persistent, and
> reserved memory...  Although these addresses may fall within the memory
> range managed by the buddy allocator (which must be in an accepted state),
> they could also be outside that range and in an unaccepted state.
>
> Since the kexec code will access the segments' destination addresses
> during the kexec process by swapping their content with the segments'
> source pages, it is necessary to accept the memory before performing the
> swap operations.
>
> Accept the destination addresses during the kexec load, immediately after
> they pass sanity checks.  This ensures the code is located in a common
> place shared by both the kexec_load and kexec_file_load system calls.
>
> This will not conflict with the accounting in try_to_accept_memory_one()
> since the accounting is set during kernel boot and decremented when pages
> are moved to the freelists.  There is no harm in invoking accept_memory()
> on a page before making it available to the buddy allocator.
>
> No need to worry about re-accepting memory since accept_memory() checks
> the unaccepted bitmap before accepting a memory page.
>
> Although a user may perform kexec loading without ever triggering the
> jump, it doesn't impact much since kexec loading is not in a
> performance-critical path.  Additionally, the destination addresses are
> always searched and found in the same location on a given system.
>
> Changes to the destination address searching logic to locate only memory
> in either unaccepted or accepted status are unnecessary and complicated.
>
> Link: https://lkml.kernel.org/r/20241213095449.881-1-yan.y.zhao@intel.com
> Signed-off-by: Yan Zhao <yan.y.zhao@intel.com>
> Reviewed-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> Cc: Baoquan He <bhe@redhat.com>
> Cc: Jianxiong Gao <jxgao@google.com>
> Cc: Dave Hansen <dave.hansen@intel.com>
> Cc: Baoquan he <bhe@redhat.com>
> Cc: "Edgecombe, Rick P" <rick.p.edgecombe@intel.com>
> Cc: Eric Biederman <ebiederm@xmission.com>
> Cc: Kirill A. Shuemov <kirill.shutemov@linux.intel.com>
> Cc: Yan Zhao <yan.y.zhao@intel.com>
> Cc: Ashish Kalra <Ashish.Kalra@amd.com>
> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
> ---
>
>  kernel/kexec_core.c |   10 ++++++++++
>  1 file changed, 10 insertions(+)
>
> --- a/kernel/kexec_core.c~kexec_core-accept-unaccepted-kexec-segments-destination-addresses
> +++ a/kernel/kexec_core.c
> @@ -210,6 +210,16 @@ int sanity_check_segment_list(struct kim
>  	}
>  #endif
>  
> +	/*
> +	 * The destination addresses are searched from system RAM rather than
> +	 * being allocated from the buddy allocator, so they are not guaranteed
> +	 * to be accepted by the current kernel.  Accept the destination
> +	 * addresses before kexec swaps their content with the segments' source
> +	 * pages to avoid accessing memory before it is accepted.
> +	 */
> +	for (i = 0; i < nr_segments; i++)
> +		accept_memory(image->segment[i].mem, image->segment[i].memsz);
> +
>  	return 0;
>  }
>  
> _
>
> Patches currently in -mm which might be from yan.y.zhao@intel.com are
>
> kexec_core-accept-unaccepted-kexec-segments-destination-addresses.patch

Re: + kexec_core-accept-unaccepted-kexec-segments-destination-addresses.patch added to mm-nonmm-unstable branch

[Kirill A. Shutemov]

On Fri, Mar 07, 2025 at 07:11:26AM -0600, Eric W. Biederman wrote:
> Andrew Morton <akpm@linux-foundation.org> writes:
> 
> > The patch titled
> >      Subject: kexec_core: accept unaccepted kexec segments' destination addresses
> > has been added to the -mm mm-nonmm-unstable branch.  Its filename is
> >      kexec_core-accept-unaccepted-kexec-segments-destination-addresses.patch
> >
> > This patch will shortly appear at
> >      https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/kexec_core-accept-unaccepted-kexec-segments-destination-addresses.patch
> >
> > This patch will later appear in the mm-nonmm-unstable branch at
> >     git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
> >
> > Before you just go and hit "reply", please:
> >    a) Consider who else should be cc'ed
> >    b) Prefer to cc a suitable mailing list as well
> >    c) Ideally: find the original patch on the mailing list and do a
> >       reply-to-all to that, adding suitable additional cc's
> >
> > *** Remember to use Documentation/process/submit-checklist.rst when testing your code ***
> >
> > The -mm tree is included into linux-next via the mm-everything
> > branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
> > and is updated there every 2-3 working days
> 
> My apologies for replying like this, but I can not quickly find this
> patch elsewhere.
> 
> This actually can not work reliably.  The problem is that all of the
> state of what has been lazily "accepted" and what has not will be lost.
> 
> If anything is going to happen in the case of kexec what needs to happen
> is kexec_load needs to force the system out of lazy mode and all memory
> needs to be accepted before kexec load runs.
> 
> The target kernel has no way of knowing what has been accepted and
> not-accepted.  I don't think you can have a reasonable expectation
> that the target kernel knows about accepted memory.

That's false.

We do pass acceptance state between kernels on kexec using EFI
config table. See LINUX_EFI_UNACCEPTED_MEM_TABLE_GUID.

The next kernel will pick up where the previous one left off.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

+ kexec_core-accept-unaccepted-kexec-segments-destination-addresses.patch added to mm-nonmm-unstable branch

[Andrew Morton]

The patch titled
     Subject: kexec_core: accept unaccepted kexec segments' destination addresses
has been added to the -mm mm-nonmm-unstable branch.  Its filename is
     kexec_core-accept-unaccepted-kexec-segments-destination-addresses.patch

This patch will shortly appear at
     https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/kexec_core-accept-unaccepted-kexec-segments-destination-addresses.patch

This patch will later appear in the mm-nonmm-unstable branch at
    git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

Before you just go and hit "reply", please:
   a) Consider who else should be cc'ed
   b) Prefer to cc a suitable mailing list as well
   c) Ideally: find the original patch on the mailing list and do a
      reply-to-all to that, adding suitable additional cc's

*** Remember to use Documentation/process/submit-checklist.rst when testing your code ***

The -mm tree is included into linux-next via the mm-everything
branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
and is updated there every 2-3 working days

------------------------------------------------------
From: Yan Zhao <yan.y.zhao@intel.com>
Subject: kexec_core: accept unaccepted kexec segments' destination addresses
Date: Fri, 7 Mar 2025 10:44:11 +0200

The UEFI Specification version 2.9 introduces the concept of memory
acceptance: some Virtual Machine platforms, such as Intel TDX or AMD
SEV-SNP, require memory to be accepted before it can be used by the guest.

Accepting memory is expensive.  The memory must be allocated by the VMM
and then brought to a known safe state: cache must be flushed, memory must
be zeroed with the guest's encryption key, and associated metadata must be
manipulated.  These operations must be performed from a trusted
environment (firmware or TDX module).  Switching context to and from it
also takes time.

This cost adds up.  On large confidential VMs, memory acceptance alone can
take minutes.  It is better to delay memory acceptance until the memory is
actually needed.

The kernel accepts memory when it is allocated from buddy allocator for
the first time.  This reduces boot time and decreases memory overhead as
the VMM can allocate memory as needed.

It does not work when the guest attempts to kexec into a new kernel.

The kexec segments' destination addresses are not allocated by the buddy
allocator.  Instead, they are searched from normal system RAM (top-down or
bottom-up) and exclude driver-managed memory, ACPI, persistent, and
reserved memory.  Unaccepted memory is normal system RAM from kernel point
of view and kexec can place segments there.

Kexec bypasses the code path in buddy allocator where memory gets accepted
and it leads to a crash when kexec accesses segments' memory.

Accept the destination addresses during the kexec load, immediately after
they pass sanity checks.  This ensures the code is located in a common
place shared by both the kexec_load and kexec_file_load system calls.

This will not conflict with the accounting in try_to_accept_memory_one()
since the accounting is set during kernel boot and decremented when pages
are moved to the freelists.  There is no harm in invoking accept_memory()
on a page before making it available to the buddy allocator.

No need to worry about re-accepting memory since accept_memory() checks
the unaccepted bitmap before accepting a memory page.

Although a user may perform kexec loading without ever triggering the
jump, it doesn't impact much since kexec loading is not in a
performance-critical path.  Additionally, the destination addresses are
always searched and found in the same location on a given system.

Changes to the destination address searching logic to locate only memory in
either unaccepted or accepted status are unnecessary and complicated.

[kirill.shutemov@linux.intel.com: update the commit message]
Link: https://lkml.kernel.org/r/20250307084411.2150367-1-kirill.shutemov@linux.intel.com
Signed-off-by: Yan Zhao <yan.y.zhao@intel.com>
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Acked-by: Dave Hansen <dave.hansen@linux.intel.com>
Cc: "Eric W. Biederman" <ebiederm@xmission.com>
Cc: Ashish Kalra <Ashish.Kalra@amd.com>
Cc: Baoquan He <bhe@redhat.com>
Cc: Jianxiong Gao <jxgao@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 kernel/kexec_core.c |   10 ++++++++++
 1 file changed, 10 insertions(+)

--- a/kernel/kexec_core.c~kexec_core-accept-unaccepted-kexec-segments-destination-addresses
+++ a/kernel/kexec_core.c
@@ -210,6 +210,16 @@ int sanity_check_segment_list(struct kim
 	}
 #endif
 
+	/*
+	 * The destination addresses are searched from system RAM rather than
+	 * being allocated from the buddy allocator, so they are not guaranteed
+	 * to be accepted by the current kernel.  Accept the destination
+	 * addresses before kexec swaps their content with the segments' source
+	 * pages to avoid accessing memory before it is accepted.
+	 */
+	for (i = 0; i < nr_segments; i++)
+		accept_memory(image->segment[i].mem, image->segment[i].memsz);
+
 	return 0;
 }
 
_

Patches currently in -mm which might be from yan.y.zhao@intel.com are

kexec_core-accept-unaccepted-kexec-segments-destination-addresses.patch