[PATCH v7] x86/mm/64: free the gap between BSS_MAIN and BSS_DECRYPTED

[PATCH v7] x86/mm/64: free the gap between BSS_MAIN and BSS_DECRYPTED

[Hamza Mahfooz]

This region is unused after mark_rodata_ro() and takes up, up to 2 MiB
of memory. So, free it in mark_rodata_ro().

Tested on a CoCo VM running on an AMD EPYC 7763v which generated the
following print:

Freeing unused kernel image (bss_decrypted gap) memory: 184K

Signed-off-by: Hamza Mahfooz <hamzamahfooz@linux.microsoft.com>
---
 arch/x86/include/asm/sections.h | 1 +
 arch/x86/kernel/vmlinux.lds.S   | 8 +++++++-
 arch/x86/mm/init_64.c           | 2 ++
 3 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/sections.h b/arch/x86/include/asm/sections.h
index 30e8ee7006f9..49ef1428aa72 100644
--- a/arch/x86/include/asm/sections.h
+++ b/arch/x86/include/asm/sections.h
@@ -8,6 +8,7 @@
 extern char __relocate_kernel_start[], __relocate_kernel_end[];
 extern char __brk_base[], __brk_limit[];
 extern char __end_rodata_aligned[];
+extern char __start_bss_decrypted_gap[], __end_bss_decrypted_gap[];
 
 #if defined(CONFIG_X86_64)
 extern char __end_rodata_hpage_align[];
diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S
index 2438b89a4620..825381b1ed33 100644
--- a/arch/x86/kernel/vmlinux.lds.S
+++ b/arch/x86/kernel/vmlinux.lds.S
@@ -94,8 +94,11 @@ const_cpu_current_top_of_stack = cpu_current_top_of_stack;
  * decrypted to avoid exposing more than we wish.
  */
 #define BSS_DECRYPTED						\
+	. = ALIGN(PAGE_SIZE);					\
+	__start_bss_decrypted_gap = .;				\
 	. = ALIGN(PMD_SIZE);					\
 	__start_bss_decrypted = .;				\
+	__end_bss_decrypted_gap = .;				\
 	__pi___start_bss_decrypted = .;				\
 	*(.bss..decrypted);					\
 	. = ALIGN(PAGE_SIZE);					\
@@ -105,7 +108,10 @@ const_cpu_current_top_of_stack = cpu_current_top_of_stack;
 	__pi___end_bss_decrypted = .;				\
 
 #else
-#define BSS_DECRYPTED
+#define BSS_DECRYPTED						\
+	. = ALIGN(PAGE_SIZE);					\
+	__start_bss_decrypted_gap = .;				\
+	__end_bss_decrypted_gap = .;
 #endif
 
 #if defined(CONFIG_X86_64) && defined(CONFIG_KEXEC_CORE)
diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c
index 70e682180291..d2e1f087ef31 100644
--- a/arch/x86/mm/init_64.c
+++ b/arch/x86/mm/init_64.c
@@ -1400,6 +1400,8 @@ void mark_rodata_ro(void)
 				(void *)text_end, (void *)rodata_start);
 	free_kernel_image_pages("unused kernel image (rodata/data gap)",
 				(void *)rodata_end, (void *)_sdata);
+	free_kernel_image_pages("unused kernel image (bss_decrypted gap)",
+				__start_bss_decrypted_gap, __end_bss_decrypted_gap);
 }
 
 /*
-- 
2.55.0

Re: [PATCH v7] x86/mm/64: free the gap between BSS_MAIN and BSS_DECRYPTED

[Borislav Petkov]

On Fri, Aug 28, 2026 at 08:32:45PM -0400, Hamza Mahfooz wrote:
> This region is unused after mark_rodata_ro() and takes up, up to 2 MiB
> of memory. So, free it in mark_rodata_ro().

So I can only guess why this patch even exists. How about some more
justification for the uglification?

> Tested on a CoCo VM running on an AMD EPYC 7763v which generated the

Testing text goes under the "---" below.

> following print:
> 
> Freeing unused kernel image (bss_decrypted gap) memory: 184K

So all that gymnastics to save 184K? Because, up to 2 MiB is not always 2 MiB.

> Signed-off-by: Hamza Mahfooz <hamzamahfooz@linux.microsoft.com>
> ---
>  arch/x86/include/asm/sections.h | 1 +
>  arch/x86/kernel/vmlinux.lds.S   | 8 +++++++-
>  arch/x86/mm/init_64.c           | 2 ++
>  3 files changed, 10 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/x86/include/asm/sections.h b/arch/x86/include/asm/sections.h
> index 30e8ee7006f9..49ef1428aa72 100644
> --- a/arch/x86/include/asm/sections.h
> +++ b/arch/x86/include/asm/sections.h
> @@ -8,6 +8,7 @@
>  extern char __relocate_kernel_start[], __relocate_kernel_end[];
>  extern char __brk_base[], __brk_limit[];
>  extern char __end_rodata_aligned[];
> +extern char __start_bss_decrypted_gap[], __end_bss_decrypted_gap[];
>  
>  #if defined(CONFIG_X86_64)
>  extern char __end_rodata_hpage_align[];
> diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S
> index 2438b89a4620..825381b1ed33 100644
> --- a/arch/x86/kernel/vmlinux.lds.S
> +++ b/arch/x86/kernel/vmlinux.lds.S
> @@ -94,8 +94,11 @@ const_cpu_current_top_of_stack = cpu_current_top_of_stack;
>   * decrypted to avoid exposing more than we wish.
>   */
>  #define BSS_DECRYPTED						\
> +	. = ALIGN(PAGE_SIZE);					\
> +	__start_bss_decrypted_gap = .;				\

So, you have a linker var which is called __start_bss_decrypted_gap which
goes...

>  	. = ALIGN(PMD_SIZE);					\
>  	__start_bss_decrypted = .;				\

... before the actual start variable.

> +	__end_bss_decrypted_gap = .;				\
>  	__pi___start_bss_decrypted = .;				\

You have *three* variables now all pointing to the same location. The two from
before weren't enough?

But the end var is at a 2M page boundary, no?

Can you somehow compute it...?

>  	*(.bss..decrypted);					\
>  	. = ALIGN(PAGE_SIZE);					\
> @@ -105,7 +108,10 @@ const_cpu_current_top_of_stack = cpu_current_top_of_stack;
>  	__pi___end_bss_decrypted = .;				\
>  
>  #else
> -#define BSS_DECRYPTED
> +#define BSS_DECRYPTED						\
> +	. = ALIGN(PAGE_SIZE);					\
> +	__start_bss_decrypted_gap = .;				\
> +	__end_bss_decrypted_gap = .;

So now you have to add the 4K alignment so that you don't have the ifdeffery in
the .c file, as I suggested. But I'm not sure this is really better.

With a sparse commit message as yours, people will wonder in the future why
was that thing even done?

So I'm not convinced this justifies the ugly, tbh....

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette