[PATCH] x86/mm: don't apply va_align to hugetlb mappings on AMD F15h

[PATCH] x86/mm: don't apply va_align to hugetlb mappings on AMD F15h

[Laurent Wandrebeck]

Commit 1317a5e7f7b1 ("arch/x86: teach arch_get_unmapped_area_vmflags to
handle hugetlb mappings") taught get_align_mask() to return
huge_page_mask_align() for a hugetlbfs file, and skipped the pgoff-derived
align_offset for one. It missed the other write to align_offset:

	if (filp) {
		info.align_mask = get_align_mask(filp);
		info.align_offset += get_align_bits();
	}

get_align_bits() calls get_align_mask(NULL), so a hugetlbfs file still gets
the F15h I$ anti-aliasing randomization that its own align_mask already
excludes it from. vm_unmapped_area() therefore returns an address
deliberately offset from the huge page boundary, the hugetlb VMA's vm_start
is only PAGE_SIZE aligned, and tearing it down trips
BUG_ON(start & ~huge_page_mask(h)) in __unmap_hugepage_range():

  kernel BUG at mm/hugetlb.c:5161!
  RIP: 0010:__unmap_hugepage_range+0x64f/0x660
  RAX: 000000003fffffff  RDX: 00007e9280003000
  Call Trace:
   __zap_vma_range+0x523/0x680
   unmap_vmas+0xa5/0x1a0
   exit_mmap+0x13b/0x3f0
   do_exit+0x1e4/0x470

That is a 1 GiB mapping on an A10-8770E (family 0x15, model 0x65) running
7.2.0, 0x3000 below a 1 GiB boundary, RAX being ~huge_page_mask(h). Both
hstates crash, and so do both on an FX-8370E (family 0x15, model 0x02)
running 7.1.8, there 0x5000 low. The offset is va_align.bits, drawn once
per boot: identical across hstates within a boot, different between boots
and machines, and a boot that draws zero does not reproduce at any size -
hence the apparent intermittency. The crash is in the teardown path, so the
reservation leaks as well, HugePages_Rsvd owned by nobody until reboot.

Reproduced by mmap()ing MAP_HUGETLB and returning. A Ryzen 5 2500U (family
0x17) on the same 7.2.0 does not reproduce it, as expected since va_align
is only set up for family 0x15. With the patch both hstates return aligned
addresses, and PostgreSQL has mapped a 4 GB hugetlbfs segment for 18.8 h on
2 MiB and 4+ h on 1 GiB pages with no BUG and no leaked reservations.

Fixes: 1317a5e7f7b1 ("arch/x86: teach arch_get_unmapped_area_vmflags to handle hugetlb mappings")
Cc: stable@vger.kernel.org # 6.13+
Signed-off-by: Laurent Wandrebeck <l.wandrebeck@quelquesmots.fr>
---
 arch/x86/kernel/sys_x86_64.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kernel/sys_x86_64.c b/arch/x86/kernel/sys_x86_64.c
index 776ae6fa7f2d..6b2be065304f 100644
--- a/arch/x86/kernel/sys_x86_64.c
+++ b/arch/x86/kernel/sys_x86_64.c
@@ -157,7 +157,8 @@ arch_get_unmapped_area(struct file *filp, unsigned long addr, unsigned long len,
 	}
 	if (filp) {
 		info.align_mask = get_align_mask(filp);
-		info.align_offset += get_align_bits();
+		if (!is_file_hugepages(filp))
+			info.align_offset += get_align_bits();
 	}
 
 	return vm_unmapped_area(&info);
@@ -222,7 +223,8 @@ arch_get_unmapped_area_topdown(struct file *filp, unsigned long addr0,
 
 	if (filp) {
 		info.align_mask = get_align_mask(filp);
-		info.align_offset += get_align_bits();
+		if (!is_file_hugepages(filp))
+			info.align_offset += get_align_bits();
 	}
 	addr = vm_unmapped_area(&info);
 	if (!(addr & ~PAGE_MASK))
-- 
2.34.1

Re: [PATCH] x86/mm: don't apply va_align to hugetlb mappings on AMD F15h

[Dave Hansen]

Hey Boris,

These AMD F15h CPUs are, what, 15 years old now? How bad of a
performance hit *was* this issue? Is this optimization worth keeping around?

On 8/28/26 06:57, Laurent Wandrebeck wrote:
> Commit 1317a5e7f7b1 ("arch/x86: teach arch_get_unmapped_area_vmflags to
> handle hugetlb mappings") taught get_align_mask() to return
> huge_page_mask_align() for a hugetlbfs file, and skipped the pgoff-derived
> align_offset for one. It missed the other write to align_offset:

I'm not crazy about this changelog.

> diff --git a/arch/x86/kernel/sys_x86_64.c b/arch/x86/kernel/sys_x86_64.c
> index 776ae6fa7f2d..6b2be065304f 100644
> --- a/arch/x86/kernel/sys_x86_64.c
> +++ b/arch/x86/kernel/sys_x86_64.c
> @@ -157,7 +157,8 @@ arch_get_unmapped_area(struct file *filp, unsigned long addr, unsigned long len,
>  	}
>  	if (filp) {
>  		info.align_mask = get_align_mask(filp);
> -		info.align_offset += get_align_bits();
> +		if (!is_file_hugepages(filp))
> +			info.align_offset += get_align_bits();
>  	}

Rather than special-casing hugetlb in two more places, could we just do:

	info.align_mask = get_align_mask(filp);
	info.align_offset += get_align_bits(filp);

and then pass filp through get_align_bits() to get_align_mask()? I think
that would end up masking the troublesome bits out of va_align.bits.

Re: [PATCH] x86/mm: don't apply va_align to hugetlb mappings on AMD F15h

[Borislav Petkov]

On Fri, Aug 28, 2026 at 10:07:42AM -0700, Dave Hansen wrote:
> These AMD F15h CPUs are, what, 15 years old now? How bad of a
> performance hit *was* this issue? Is this optimization worth keeping around?

Oh, look, even back then I did perf measurements:

https://lore.kernel.org/all/1312550110-24160-2-git-send-email-bp@amd64.org/T/#m04c05a9a6f59b73d610b556f1efda8cd9e0487d1

8s penalty for kernel build workload on 3.0.

Do I care about Bulldozer?

Meh, I only have access to a laptop with that uarch and I've been using it
only to test kernels on it so, not really. Besides, we have Zens now which
are soo much better in every aspect so that Bulldozer looks more like a energy
waste than anything else.

But there are apparently folks which still use those and run new kernels on
them so we probably can't axe it off yet...

Looking on the bright side, Laurent can test patches ... :-P

HTH.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH] x86/mm: don't apply va_align to hugetlb mappings on AMD F15h

[Laurent Wandrebeck]

[-- Attachment #1: Type: text/plain, Size: 1522 bytes --]

Hi all,

My homelab is indeed composed of 5 thinkcenter with a10-8770e, and desktop has fx-8370e. Actually I hit that bug running 26.04/7.0 and reproduced with 7.1, 7.2 so I dug a bit more and AI (I'm no kernel hacker) proposed this patch. I'll happily test and report on whatever you'll propose if that one does not fit. SSH+sudo access is possible too if you want.

Best,

Le 29 août 2026 19:47:04 GMT+02:00, Borislav Petkov <bp@alien8.de> a écrit :
>On Fri, Aug 28, 2026 at 10:07:42AM -0700, Dave Hansen wrote:
>> These AMD F15h CPUs are, what, 15 years old now? How bad of a
>> performance hit *was* this issue? Is this optimization worth keeping around?
>
>Oh, look, even back then I did perf measurements:
>
>https://lore.kernel.org/all/1312550110-24160-2-git-send-email-bp@amd64.org/T/#m04c05a9a6f59b73d610b556f1efda8cd9e0487d1
>
>8s penalty for kernel build workload on 3.0.
>
>Do I care about Bulldozer?
>
>Meh, I only have access to a laptop with that uarch and I've been using it
>only to test kernels on it so, not really. Besides, we have Zens now which
>are soo much better in every aspect so that Bulldozer looks more like a energy
>waste than anything else.
>
>But there are apparently folks which still use those and run new kernels on
>them so we probably can't axe it off yet...
>
>Looking on the bright side, Laurent can test patches ... :-P
>
>HTH.
>
>-- 
>Regards/Gruss,
>    Boris.
>
>https://people.kernel.org/tglx/notes-about-netiquette

Laurent Wandrebeck 

[-- Attachment #2: Type: text/html, Size: 2150 bytes --]

Re: [PATCH] x86/mm: don't apply va_align to hugetlb mappings on AMD F15h

[Borislav Petkov]

On Sat, Aug 29, 2026 at 08:15:40PM +0200, Laurent Wandrebeck wrote:
> My homelab is indeed composed of 5 thinkcenter with a10-8770e, and
> desktop has fx-8370e. Actually I hit that bug running 26.04/7.0 and
> reproduced with 7.1, 7.2 so I dug a bit more and AI (I'm no kernel
> hacker) proposed this patch. I'll happily test and report on whatever
> you'll propose if that one does not fit. SSH+sudo access is possible too
> if you want.

Thanks, just please do not send HTML mails and put your reply under the text
you're referring to when you reply. Like I just did.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH] x86/mm: don't apply va_align to hugetlb mappings on AMD F15h

[Laurent Wandrebeck]

On Sat, 2026-08-29 at 11:51 -0700, Borislav Petkov wrote:
> On Sat, Aug 29, 2026 at 08:15:40PM +0200, Laurent Wandrebeck wrote:
> > My homelab is indeed composed of 5 thinkcenter with a10-8770e, and
> > desktop has fx-8370e. Actually I hit that bug running 26.04/7.0 and
> > reproduced with 7.1, 7.2 so I dug a bit more and AI (I'm no kernel
> > hacker) proposed this patch. I'll happily test and report on
> > whatever
> > you'll propose if that one does not fit. SSH+sudo access is
> > possible too
> > if you want.
> 
> Thanks, just please do not send HTML mails and put your reply under
> the text
> you're referring to when you reply. Like I just did.
> 
> Thx.
> 

Sorry about, replied from my phone without thinking about it.
FYI, patched kernel has run a couple more days without any problem, and
I’m now running 7.3-rc1 with the same patch flawlessly.

[Tue Sep  1 06:55:41 2026] Kernel command line: BOOT_IMAGE=/vmlinuz-
7.3.0-rc1-hugetlbfix root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
hugepagesz=1024M hugepages=7 mitigations=off crashkernel=2G-4G:320M,4G-
32G:512M,32G-64G:1024M,64G-128G:2048M,128G-:4096M
[Tue Sep  1 06:55:42 2026] HugeTLB: registered 1.00 GiB page size, pre-
allocated 7 pages

Can I do anything else ?

Best,
-- 
Laurent Wandrebeck <l.wandrebeck@quelquesmots.fr>

Re: [PATCH] x86/mm: don't apply va_align to hugetlb mappings on AMD F15h

[Borislav Petkov]

On Tue, Sep 01, 2026 at 10:12:07AM +0200, Laurent Wandrebeck wrote:
> Sorry about, replied from my phone without thinking about it.

No worries.

> FYI, patched kernel has run a couple more days without any problem, and
> I’m now running 7.3-rc1 with the same patch flawlessly.

...
 
> Can I do anything else ?

Does Dave's suggestion work too?

https://lore.kernel.org/all/3faecf65-7edb-4cec-be7a-e5266dd3a649@intel.com/

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH] x86/mm: don't apply va_align to hugetlb mappings on AMD F15h

[Laurent Wandrebeck]

On Wed, 2026-09-16 at 13:02 -0700, Borislav Petkov wrote:
> On Tue, Sep 01, 2026 at 10:12:07AM +0200, Laurent Wandrebeck wrote:
> > Sorry about, replied from my phone without thinking about it.
> 
> No worries.
> 
> > FYI, patched kernel has run a couple more days without any problem, and
> > I’m now running 7.3-rc1 with the same patch flawlessly.
> 
> ...
>  
> > Can I do anything else ?
> 
> Does Dave's suggestion work too?
> 
> https://lore.kernel.org/all/3faecf65-7edb-4cec-be7a-e5266dd3a649@intel.com/
> 
> Thx.
> 

Here’s the « v2 » patch I used instead, on top of 7.3-rc3.
Works without problem (several hours uptime).
If this looks ok to you, I’ll follow up with a proper [PATCH v2] mail.
Thanks !

diff --git a/arch/x86/kernel/sys_x86_64.c b/arch/x86/kernel/sys_x86_64.c
index 776ae6fa7f2d..a54f930e2c9a 100644
--- a/arch/x86/kernel/sys_x86_64.c
+++ b/arch/x86/kernel/sys_x86_64.c
@@ -28,7 +28,7 @@
  */
 static unsigned long get_align_mask(struct file *filp)
 {
-       if (filp && is_file_hugepages(filp))
+       if (is_file_hugepages(filp))
                return huge_page_mask_align(filp);
        /* handle 32- and 64-bit case with a single conditional */
        if (va_align.flags < 0 || !(va_align.flags & (2 - mmap_is_ia32())))
@@ -50,9 +50,13 @@ static unsigned long get_align_mask(struct file *filp)
  * value before calling vm_unmapped_area() or ORed directly to the
  * address.
  */
-static unsigned long get_align_bits(void)
+static unsigned long get_align_bits(struct file *filp)
 {
-       return va_align.bits & get_align_mask(NULL);
+       /* hugetlb mappings must stay on the huge page boundary. */
+       if (is_file_hugepages(filp))
+               return 0;
+
+       return va_align.bits & get_align_mask(filp);
 }
 
 static int __init control_va_addr_alignment(char *str)
@@ -157,7 +161,7 @@ arch_get_unmapped_area(struct file *filp, unsigned long addr, unsigned long len,
        }
        if (filp) {
                info.align_mask = get_align_mask(filp);
-               info.align_offset += get_align_bits();
+               info.align_offset += get_align_bits(filp);
        }
 
        return vm_unmapped_area(&info);
@@ -222,7 +226,7 @@ arch_get_unmapped_area_topdown(struct file *filp, unsigned long addr0,
 
        if (filp) {
                info.align_mask = get_align_mask(filp);
-               info.align_offset += get_align_bits();
+               info.align_offset += get_align_bits(filp);
        }
        addr = vm_unmapped_area(&info);
        if (!(addr & ~PAGE_MASK))

-- 
Laurent Wandrebeck <l.wandrebeck@quelquesmots.fr>

Re: [PATCH] x86/mm: don't apply va_align to hugetlb mappings on AMD F15h

[Borislav Petkov]

On Sat, Sep 19, 2026 at 10:47:46AM +0200, Laurent Wandrebeck wrote:
> Here’s the « v2 » patch I used instead, on top of 7.3-rc3.
> Works without problem (several hours uptime).
> If this looks ok to you,

I can't complain about that too much, but let's see what Dave thinks.

Dave?
 
> diff --git a/arch/x86/kernel/sys_x86_64.c b/arch/x86/kernel/sys_x86_64.c
> index 776ae6fa7f2d..a54f930e2c9a 100644
> --- a/arch/x86/kernel/sys_x86_64.c
> +++ b/arch/x86/kernel/sys_x86_64.c
> @@ -28,7 +28,7 @@
>   */
>  static unsigned long get_align_mask(struct file *filp)
>  {
> -       if (filp && is_file_hugepages(filp))
> +       if (is_file_hugepages(filp))
>                 return huge_page_mask_align(filp);
>         /* handle 32- and 64-bit case with a single conditional */
>         if (va_align.flags < 0 || !(va_align.flags & (2 - mmap_is_ia32())))
> @@ -50,9 +50,13 @@ static unsigned long get_align_mask(struct file *filp)
>   * value before calling vm_unmapped_area() or ORed directly to the
>   * address.
>   */
> -static unsigned long get_align_bits(void)
> +static unsigned long get_align_bits(struct file *filp)
>  {
> -       return va_align.bits & get_align_mask(NULL);
> +       /* hugetlb mappings must stay on the huge page boundary. */
> +       if (is_file_hugepages(filp))
> +               return 0;
> +
> +       return va_align.bits & get_align_mask(filp);
>  }
>  
>  static int __init control_va_addr_alignment(char *str)
> @@ -157,7 +161,7 @@ arch_get_unmapped_area(struct file *filp, unsigned long addr, unsigned long len,
>         }
>         if (filp) {
>                 info.align_mask = get_align_mask(filp);
> -               info.align_offset += get_align_bits();
> +               info.align_offset += get_align_bits(filp);
>         }
>  
>         return vm_unmapped_area(&info);
> @@ -222,7 +226,7 @@ arch_get_unmapped_area_topdown(struct file *filp, unsigned long addr0,
>  
>         if (filp) {
>                 info.align_mask = get_align_mask(filp);
> -               info.align_offset += get_align_bits();
> +               info.align_offset += get_align_bits(filp);
>         }
>         addr = vm_unmapped_area(&info);
>         if (!(addr & ~PAGE_MASK))
> 
> -- 
> Laurent Wandrebeck <l.wandrebeck@quelquesmots.fr>

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH] x86/mm: don't apply va_align to hugetlb mappings on AMD F15h

[Dave Hansen]

On 9/19/26 01:47, Laurent Wandrebeck wrote:
>  static unsigned long get_align_mask(struct file *filp)
>  {
> -       if (filp && is_file_hugepages(filp))
> +       if (is_file_hugepages(filp))
>                 return huge_page_mask_align(filp);
>         /* handle 32- and 64-bit case with a single conditional */
>         if (va_align.flags < 0 || !(va_align.flags & (2 - mmap_is_ia32())))

I'd just leave that hunk out for now. It's harmless to leave.

> @@ -50,9 +50,13 @@ static unsigned long get_align_mask(struct file *filp)
>   * value before calling vm_unmapped_area() or ORed directly to the
>   * address.
>   */
> -static unsigned long get_align_bits(void)
> +static unsigned long get_align_bits(struct file *filp)
>  {
> -       return va_align.bits & get_align_mask(NULL);
> +       /* hugetlb mappings must stay on the huge page boundary. */
> +       if (is_file_hugepages(filp))
> +               return 0;

I don't find that comment helpful. The function is kinda misnamed a bit
too. Maybe a comment like this?

	/*
	 * va_align.bits is smaller than the huge page size and will
	 * lead to misaligned huge pages. Ignore it for huge mappings.
	 */

> +       return va_align.bits & get_align_mask(filp);
>  }
>  
>  static int __init control_va_addr_alignment(char *str)
> @@ -157,7 +161,7 @@ arch_get_unmapped_area(struct file *filp, unsigned long addr, unsigned long len,
>         }
>         if (filp) {
>                 info.align_mask = get_align_mask(filp);
> -               info.align_offset += get_align_bits();
> +               info.align_offset += get_align_bits(filp);
>         }
>  
>         return vm_unmapped_area(&info);
> @@ -222,7 +226,7 @@ arch_get_unmapped_area_topdown(struct file *filp, unsigned long addr0,
>  
>         if (filp) {
>                 info.align_mask = get_align_mask(filp);
> -               info.align_offset += get_align_bits();
> +               info.align_offset += get_align_bits(filp);
>         }

That looks fine too. I would rather this whole mechanism die,
personally. But this is a fine, backportable fix.

[PATCH v2] x86/mm: don't apply va_align to hugetlb mappings on AMD F15h

[Laurent Wandrebeck]

get_align_mask() returns huge_page_mask_align() for hugetlbfs, but
get_align_bits() adds va_align.bits regardless, so vm_unmapped_area()
returns an address off the huge page boundary and
__unmap_hugepage_range() hits BUG_ON(start & ~huge_page_mask(h)) at
teardown.

Pass the file to get_align_bits() and skip the randomisation for
hugetlbfs. Reproduced on Carrizo and FX-8370E, both hstates.

Fixes: 1317a5e7f7b1 ("arch/x86: teach arch_get_unmapped_area_vmflags to handle hugetlb mappings")
Cc: stable@vger.kernel.org # 6.13+
Suggested-by: Dave Hansen <dave.hansen@intel.com>
Signed-off-by: Laurent Wandrebeck <l.wandrebeck@quelquesmots.fr>
---
v2: put the hugetlb case in get_align_bits() instead of guarding both call
    sites, per Dave's suggestion to pass filp through. Masking rather than
    returning early cannot clear the bits: va_align.mask is an L1I way
    size, so it is a subset of huge_page_mask_align().

    Re-tested on Carrizo, both hstates.

    Link: https://lore.kernel.org/all/20260828135747.724789-1-l.wandrebeck@quelquesmots.fr/
 arch/x86/kernel/sys_x86_64.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kernel/sys_x86_64.c b/arch/x86/kernel/sys_x86_64.c
index 776ae6fa7f2d..4c078827f26a 100644
--- a/arch/x86/kernel/sys_x86_64.c
+++ b/arch/x86/kernel/sys_x86_64.c
@@ -50,9 +50,16 @@ static unsigned long get_align_mask(struct file *filp)
  * value before calling vm_unmapped_area() or ORed directly to the
  * address.
  */
-static unsigned long get_align_bits(void)
+static unsigned long get_align_bits(struct file *filp)
 {
-	return va_align.bits & get_align_mask(NULL);
+	/*
+	 * va_align.bits is smaller than the huge page size and will
+	 * lead to misaligned huge pages. Ignore it for huge mappings.
+	 */
+	if (is_file_hugepages(filp))
+		return 0;
+
+	return va_align.bits & get_align_mask(filp);
 }
 
 static int __init control_va_addr_alignment(char *str)
@@ -157,7 +164,7 @@ arch_get_unmapped_area(struct file *filp, unsigned long addr, unsigned long len,
 	}
 	if (filp) {
 		info.align_mask = get_align_mask(filp);
-		info.align_offset += get_align_bits();
+		info.align_offset += get_align_bits(filp);
 	}
 
 	return vm_unmapped_area(&info);
@@ -222,7 +229,7 @@ arch_get_unmapped_area_topdown(struct file *filp, unsigned long addr0,
 
 	if (filp) {
 		info.align_mask = get_align_mask(filp);
-		info.align_offset += get_align_bits();
+		info.align_offset += get_align_bits(filp);
 	}
 	addr = vm_unmapped_area(&info);
 	if (!(addr & ~PAGE_MASK))

base-commit: 93f51579e7df248780214094418f205253383cc5
-- 
2.34.1