Linux-mm Archive on lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] arm64: tlb: Flush walk cache when unsharing PMD tables
@ 2026-05-21  7:30 Zeng Heng
  2026-05-21 15:05 ` Catalin Marinas
  2026-05-22 10:42 ` Catalin Marinas
  0 siblings, 2 replies; 8+ messages in thread
From: Zeng Heng @ 2026-05-21  7:30 UTC (permalink / raw)
  To: will, catalin.marinas, akpm, npiggin, aneesh.kumar, peterz
  Cc: linux-kernel, wangkefeng.wang, linux-arm-kernel, linux-mm,
	linux-arch

From: Zeng Heng <zengheng4@huawei.com>

When huge_pmd_unshare() is called to unshare a PMD table, the
tlb_unshare_pmd_ptdesc() function sets tlb->unshared_tables=true
but the aarch64 tlb_flush() only checked tlb->freed_tables to
determine whether to use TLBF_NONE (vae1is, invalidates walk
cache) or TLBF_NOWALKCACHE (vale1is, leaf-only).

This caused the stale PMD page table entry to remain in the walk cache
after unshare, potentially leading to incorrect page table walks.

Fix by including unshared_tables in the check, so that when
unsharing tables, TLBF_NONE is used and the walk cache is properly
invalidated.

Here is the detailed distinction between vae1is and vale1is:

| Instruction Combination  | Actual Invalidation Scope                         |
| ------------------------ | --------------------------------------------------|
| `VAE1IS`  + TTL=`0`      | All entries at all levels (full invalidation)     |
| `VAE1IS`  + TTL=`2` (L2) | Non-leaf at Level 0/1 + leaf at Level 2           |
| `VALE1IS` + TTL=`0`      | Leaf entries at all levels (non-leaf not cleared) |
| `VALE1IS` + TTL=`2` (L2) | Leaf entry at Level 2 only                        |

Signed-off-by: Zeng Heng <zengheng4@huawei.com>
---
 arch/arm64/include/asm/tlb.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/include/asm/tlb.h b/arch/arm64/include/asm/tlb.h
index 10869d7731b8..751bd57bc3ba 100644
--- a/arch/arm64/include/asm/tlb.h
+++ b/arch/arm64/include/asm/tlb.h
@@ -53,7 +53,8 @@ static inline int tlb_get_level(struct mmu_gather *tlb)
 static inline void tlb_flush(struct mmu_gather *tlb)
 {
 	struct vm_area_struct vma = TLB_FLUSH_VMA(tlb->mm, 0);
-	tlbf_t flags = tlb->freed_tables ? TLBF_NONE : TLBF_NOWALKCACHE;
+	tlbf_t flags = (tlb->freed_tables || tlb->unshared_tables) ?
+			TLBF_NONE : TLBF_NOWALKCACHE;
 	unsigned long stride = tlb_get_unmap_size(tlb);
 	int tlb_level = tlb_get_level(tlb);
 
-- 
2.43.0



^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [PATCH] arm64: tlb: Flush walk cache when unsharing PMD tables
  2026-05-21  7:30 [PATCH] arm64: tlb: Flush walk cache when unsharing PMD tables Zeng Heng
@ 2026-05-21 15:05 ` Catalin Marinas
  2026-05-21 15:15   ` Catalin Marinas
  2026-05-22  4:43   ` Zeng Heng
  2026-05-22 10:42 ` Catalin Marinas
  1 sibling, 2 replies; 8+ messages in thread
From: Catalin Marinas @ 2026-05-21 15:05 UTC (permalink / raw)
  To: Zeng Heng
  Cc: will, akpm, npiggin, aneesh.kumar, peterz, linux-kernel,
	wangkefeng.wang, linux-arm-kernel, linux-mm, linux-arch,
	David Hildenbrand

+ David H.

On Thu, May 21, 2026 at 03:30:11PM +0800, Zeng Heng wrote:
> From: Zeng Heng <zengheng4@huawei.com>
> 
> When huge_pmd_unshare() is called to unshare a PMD table, the
> tlb_unshare_pmd_ptdesc() function sets tlb->unshared_tables=true
> but the aarch64 tlb_flush() only checked tlb->freed_tables to
> determine whether to use TLBF_NONE (vae1is, invalidates walk
> cache) or TLBF_NOWALKCACHE (vale1is, leaf-only).
> 
> This caused the stale PMD page table entry to remain in the walk cache
> after unshare, potentially leading to incorrect page table walks.
> 
> Fix by including unshared_tables in the check, so that when
> unsharing tables, TLBF_NONE is used and the walk cache is properly
> invalidated.
> 
> Here is the detailed distinction between vae1is and vale1is:
> 
> | Instruction Combination  | Actual Invalidation Scope                         |
> | ------------------------ | --------------------------------------------------|
> | `VAE1IS`  + TTL=`0`      | All entries at all levels (full invalidation)     |
> | `VAE1IS`  + TTL=`2` (L2) | Non-leaf at Level 0/1 + leaf at Level 2           |
> | `VALE1IS` + TTL=`0`      | Leaf entries at all levels (non-leaf not cleared) |
> | `VALE1IS` + TTL=`2` (L2) | Leaf entry at Level 2 only                        |
> 
> Signed-off-by: Zeng Heng <zengheng4@huawei.com>

The fix looks fine but does it need:

Fixes: 8ce720d5bd91 ("mm/hugetlb: fix excessive IPI broadcasts when unsharing PMD tables using mmu_gather")
Cc: <stable@vger.kernel.org>

> ---
>  arch/arm64/include/asm/tlb.h | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/arm64/include/asm/tlb.h b/arch/arm64/include/asm/tlb.h
> index 10869d7731b8..751bd57bc3ba 100644
> --- a/arch/arm64/include/asm/tlb.h
> +++ b/arch/arm64/include/asm/tlb.h
> @@ -53,7 +53,8 @@ static inline int tlb_get_level(struct mmu_gather *tlb)
>  static inline void tlb_flush(struct mmu_gather *tlb)
>  {
>  	struct vm_area_struct vma = TLB_FLUSH_VMA(tlb->mm, 0);
> -	tlbf_t flags = tlb->freed_tables ? TLBF_NONE : TLBF_NOWALKCACHE;
> +	tlbf_t flags = (tlb->freed_tables || tlb->unshared_tables) ?
> +			TLBF_NONE : TLBF_NOWALKCACHE;
>  	unsigned long stride = tlb_get_unmap_size(tlb);
>  	int tlb_level = tlb_get_level(tlb);
>  
> -- 
> 2.43.0

-- 
Catalin


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] arm64: tlb: Flush walk cache when unsharing PMD tables
  2026-05-21 15:05 ` Catalin Marinas
@ 2026-05-21 15:15   ` Catalin Marinas
  2026-05-22  5:32     ` Zeng Heng
  2026-05-22  4:43   ` Zeng Heng
  1 sibling, 1 reply; 8+ messages in thread
From: Catalin Marinas @ 2026-05-21 15:15 UTC (permalink / raw)
  To: Zeng Heng
  Cc: will, akpm, npiggin, aneesh.kumar, peterz, linux-kernel,
	wangkefeng.wang, linux-arm-kernel, linux-mm, linux-arch,
	David Hildenbrand

On Thu, May 21, 2026 at 04:05:07PM +0100, Catalin Marinas wrote:
> + David H.
> 
> On Thu, May 21, 2026 at 03:30:11PM +0800, Zeng Heng wrote:
> > From: Zeng Heng <zengheng4@huawei.com>
> > 
> > When huge_pmd_unshare() is called to unshare a PMD table, the
> > tlb_unshare_pmd_ptdesc() function sets tlb->unshared_tables=true
> > but the aarch64 tlb_flush() only checked tlb->freed_tables to
> > determine whether to use TLBF_NONE (vae1is, invalidates walk
> > cache) or TLBF_NOWALKCACHE (vale1is, leaf-only).
> > 
> > This caused the stale PMD page table entry to remain in the walk cache
> > after unshare, potentially leading to incorrect page table walks.
> > 
> > Fix by including unshared_tables in the check, so that when
> > unsharing tables, TLBF_NONE is used and the walk cache is properly
> > invalidated.
> > 
> > Here is the detailed distinction between vae1is and vale1is:
> > 
> > | Instruction Combination  | Actual Invalidation Scope                         |
> > | ------------------------ | --------------------------------------------------|
> > | `VAE1IS`  + TTL=`0`      | All entries at all levels (full invalidation)     |
> > | `VAE1IS`  + TTL=`2` (L2) | Non-leaf at Level 0/1 + leaf at Level 2           |
> > | `VALE1IS` + TTL=`0`      | Leaf entries at all levels (non-leaf not cleared) |
> > | `VALE1IS` + TTL=`2` (L2) | Leaf entry at Level 2 only                        |
> > 
> > Signed-off-by: Zeng Heng <zengheng4@huawei.com>
> 
> The fix looks fine but does it need:
> 
> Fixes: 8ce720d5bd91 ("mm/hugetlb: fix excessive IPI broadcasts when unsharing PMD tables using mmu_gather")
> Cc: <stable@vger.kernel.org>
> 
> > ---
> >  arch/arm64/include/asm/tlb.h | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> > 
> > diff --git a/arch/arm64/include/asm/tlb.h b/arch/arm64/include/asm/tlb.h
> > index 10869d7731b8..751bd57bc3ba 100644
> > --- a/arch/arm64/include/asm/tlb.h
> > +++ b/arch/arm64/include/asm/tlb.h
> > @@ -53,7 +53,8 @@ static inline int tlb_get_level(struct mmu_gather *tlb)
> >  static inline void tlb_flush(struct mmu_gather *tlb)
> >  {
> >  	struct vm_area_struct vma = TLB_FLUSH_VMA(tlb->mm, 0);
> > -	tlbf_t flags = tlb->freed_tables ? TLBF_NONE : TLBF_NOWALKCACHE;
> > +	tlbf_t flags = (tlb->freed_tables || tlb->unshared_tables) ?
> > +			TLBF_NONE : TLBF_NOWALKCACHE;
> >  	unsigned long stride = tlb_get_unmap_size(tlb);
> >  	int tlb_level = tlb_get_level(tlb);

Do we need this as well?

diff --git a/arch/arm64/include/asm/tlb.h b/arch/arm64/include/asm/tlb.h
index 10869d7731b8..3f4ab38cfd6e 100644
--- a/arch/arm64/include/asm/tlb.h
+++ b/arch/arm64/include/asm/tlb.h
@@ -24,7 +24,7 @@ static void tlb_flush(struct mmu_gather *tlb);
 static inline int tlb_get_level(struct mmu_gather *tlb)
 {
 	/* The TTL field is only valid for the leaf entry. */
-	if (tlb->freed_tables)
+	if (tlb->freed_tables || tlb->unshared_tables)
 		return TLBI_TTL_UNKNOWN;
 
 	if (tlb->cleared_ptes && !(tlb->cleared_pmds ||


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [PATCH] arm64: tlb: Flush walk cache when unsharing PMD tables
  2026-05-21 15:05 ` Catalin Marinas
  2026-05-21 15:15   ` Catalin Marinas
@ 2026-05-22  4:43   ` Zeng Heng
  1 sibling, 0 replies; 8+ messages in thread
From: Zeng Heng @ 2026-05-22  4:43 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: will, akpm, npiggin, aneesh.kumar, peterz, linux-kernel,
	wangkefeng.wang, linux-arm-kernel, linux-mm, linux-arch,
	David Hildenbrand

Hi Catalin,

On 2026/5/21 23:05, Catalin Marinas wrote:
> + David H.
>
> On Thu, May 21, 2026 at 03:30:11PM +0800, Zeng Heng wrote:
>> From: Zeng Heng <zengheng4@huawei.com>
>>
>> When huge_pmd_unshare() is called to unshare a PMD table, the
>> tlb_unshare_pmd_ptdesc() function sets tlb->unshared_tables=true
>> but the aarch64 tlb_flush() only checked tlb->freed_tables to
>> determine whether to use TLBF_NONE (vae1is, invalidates walk
>> cache) or TLBF_NOWALKCACHE (vale1is, leaf-only).
>>
>> This caused the stale PMD page table entry to remain in the walk cache
>> after unshare, potentially leading to incorrect page table walks.
>>
>> Fix by including unshared_tables in the check, so that when
>> unsharing tables, TLBF_NONE is used and the walk cache is properly
>> invalidated.
>>
>> Here is the detailed distinction between vae1is and vale1is:
>>
>> | Instruction Combination  | Actual Invalidation Scope                         |
>> | ------------------------ | --------------------------------------------------|
>> | `VAE1IS`  + TTL=`0`      | All entries at all levels (full invalidation)     |
>> | `VAE1IS`  + TTL=`2` (L2) | Non-leaf at Level 0/1 + leaf at Level 2           |
>> | `VALE1IS` + TTL=`0`      | Leaf entries at all levels (non-leaf not cleared) |
>> | `VALE1IS` + TTL=`2` (L2) | Leaf entry at Level 2 only                        |
>>
>> Signed-off-by: Zeng Heng <zengheng4@huawei.com>
> The fix looks fine but does it need:
>
> Fixes: 8ce720d5bd91 ("mm/hugetlb: fix excessive IPI broadcasts when unsharing PMD tables using mmu_gather")
> Cc: <stable@vger.kernel.org>

It makes sense to me. Thanks a lot for that.


Best regards,
Zeng Heng

>> ---
>>   arch/arm64/include/asm/tlb.h | 3 ++-
>>   1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/arm64/include/asm/tlb.h b/arch/arm64/include/asm/tlb.h
>> index 10869d7731b8..751bd57bc3ba 100644
>> --- a/arch/arm64/include/asm/tlb.h
>> +++ b/arch/arm64/include/asm/tlb.h
>> @@ -53,7 +53,8 @@ static inline int tlb_get_level(struct mmu_gather *tlb)
>>   static inline void tlb_flush(struct mmu_gather *tlb)
>>   {
>>   	struct vm_area_struct vma = TLB_FLUSH_VMA(tlb->mm, 0);
>> -	tlbf_t flags = tlb->freed_tables ? TLBF_NONE : TLBF_NOWALKCACHE;
>> +	tlbf_t flags = (tlb->freed_tables || tlb->unshared_tables) ?
>> +			TLBF_NONE : TLBF_NOWALKCACHE;
>>   	unsigned long stride = tlb_get_unmap_size(tlb);
>>   	int tlb_level = tlb_get_level(tlb);
>>   
>> -- 
>> 2.43.0



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] arm64: tlb: Flush walk cache when unsharing PMD tables
  2026-05-21 15:15   ` Catalin Marinas
@ 2026-05-22  5:32     ` Zeng Heng
  2026-05-22 10:13       ` Catalin Marinas
  0 siblings, 1 reply; 8+ messages in thread
From: Zeng Heng @ 2026-05-22  5:32 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: yezhenyu2, zhurui3, will, akpm, npiggin, aneesh.kumar, peterz,
	linux-kernel, wangkefeng.wang, linux-arm-kernel, linux-mm,
	linux-arch, David Hildenbrand, zengheng4



On 2026/5/21 23:15, Catalin Marinas wrote:
> On Thu, May 21, 2026 at 04:05:07PM +0100, Catalin Marinas wrote:
>> + David H.
>>
>> On Thu, May 21, 2026 at 03:30:11PM +0800, Zeng Heng wrote:
>>> From: Zeng Heng <zengheng4@huawei.com>
>>>
>>> When huge_pmd_unshare() is called to unshare a PMD table, the
>>> tlb_unshare_pmd_ptdesc() function sets tlb->unshared_tables=true
>>> but the aarch64 tlb_flush() only checked tlb->freed_tables to
>>> determine whether to use TLBF_NONE (vae1is, invalidates walk
>>> cache) or TLBF_NOWALKCACHE (vale1is, leaf-only).
>>>
>>> This caused the stale PMD page table entry to remain in the walk cache
>>> after unshare, potentially leading to incorrect page table walks.
>>>
>>> Fix by including unshared_tables in the check, so that when
>>> unsharing tables, TLBF_NONE is used and the walk cache is properly
>>> invalidated.
>>>
>>> Here is the detailed distinction between vae1is and vale1is:
>>>
>>> | Instruction Combination  | Actual Invalidation Scope                         |
>>> | ------------------------ | --------------------------------------------------|
>>> | `VAE1IS`  + TTL=`0`      | All entries at all levels (full invalidation)     |
>>> | `VAE1IS`  + TTL=`2` (L2) | Non-leaf at Level 0/1 + leaf at Level 2           |
>>> | `VALE1IS` + TTL=`0`      | Leaf entries at all levels (non-leaf not cleared) |
>>> | `VALE1IS` + TTL=`2` (L2) | Leaf entry at Level 2 only                        |
>>>
>>> Signed-off-by: Zeng Heng <zengheng4@huawei.com>
>> The fix looks fine but does it need:
>>
>> Fixes: 8ce720d5bd91 ("mm/hugetlb: fix excessive IPI broadcasts when unsharing PMD tables using mmu_gather")
>> Cc: <stable@vger.kernel.org>
>>
>>> ---
>>>   arch/arm64/include/asm/tlb.h | 3 ++-
>>>   1 file changed, 2 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/arch/arm64/include/asm/tlb.h b/arch/arm64/include/asm/tlb.h
>>> index 10869d7731b8..751bd57bc3ba 100644
>>> --- a/arch/arm64/include/asm/tlb.h
>>> +++ b/arch/arm64/include/asm/tlb.h
>>> @@ -53,7 +53,8 @@ static inline int tlb_get_level(struct mmu_gather *tlb)
>>>   static inline void tlb_flush(struct mmu_gather *tlb)
>>>   {
>>>   	struct vm_area_struct vma = TLB_FLUSH_VMA(tlb->mm, 0);
>>> -	tlbf_t flags = tlb->freed_tables ? TLBF_NONE : TLBF_NOWALKCACHE;
>>> +	tlbf_t flags = (tlb->freed_tables || tlb->unshared_tables) ?
>>> +			TLBF_NONE : TLBF_NOWALKCACHE;
>>>   	unsigned long stride = tlb_get_unmap_size(tlb);
>>>   	int tlb_level = tlb_get_level(tlb);
> Do we need this as well?

The proposed fix has been validated against the issue scenarios and
works as expected.

Per the ARM Architecture Reference Manual, whether only the last-level
page table entry is invalidated is determined by the instruction used
(vale1is for leaf entry only, vae1is for walk cache including leaf entry 
and
non-leaf entry), rather than the TTL field. The TTL field merely specifies
which level the leaf entry belongs to.

Setting TTL to 0 always works fine, however, hardware must assume that
the entry can be from any level.[1][2]


[1]: vae1is instruction introduction by ARM spec,
https://developer.arm.com/documentation/ddi0601/2026-03/AArch64-Instructions/TLBI-VAE1IS--TLBI-VAE1ISNXS--TLB-Invalidate-by-VA--EL1--Inner-Shareable
[2]: rvae1is instruction introduction by ARM spec,
https://developer.arm.com/documentation/ddi0601/2026-03/AArch64-Instructions/TLBI-RVAE1IS--TLBI-RVAE1ISNXS--TLB-Range-Invalidate-by-VA--EL1--Inner-Shareable?lang=en


Best regards,
Zeng Heng


>
> diff --git a/arch/arm64/include/asm/tlb.h b/arch/arm64/include/asm/tlb.h
> index 10869d7731b8..3f4ab38cfd6e 100644
> --- a/arch/arm64/include/asm/tlb.h
> +++ b/arch/arm64/include/asm/tlb.h
> @@ -24,7 +24,7 @@ static void tlb_flush(struct mmu_gather *tlb);
>   static inline int tlb_get_level(struct mmu_gather *tlb)
>   {
>   	/* The TTL field is only valid for the leaf entry. */
> -	if (tlb->freed_tables)
> +	if (tlb->freed_tables || tlb->unshared_tables)
>   		return TLBI_TTL_UNKNOWN;
>   
>   	if (tlb->cleared_ptes && !(tlb->cleared_pmds ||



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] arm64: tlb: Flush walk cache when unsharing PMD tables
  2026-05-22  5:32     ` Zeng Heng
@ 2026-05-22 10:13       ` Catalin Marinas
  2026-05-22 10:38         ` Catalin Marinas
  0 siblings, 1 reply; 8+ messages in thread
From: Catalin Marinas @ 2026-05-22 10:13 UTC (permalink / raw)
  To: Zeng Heng
  Cc: yezhenyu2, zhurui3, will, akpm, npiggin, aneesh.kumar, peterz,
	linux-kernel, wangkefeng.wang, linux-arm-kernel, linux-mm,
	linux-arch, David Hildenbrand, zengheng4

On Fri, May 22, 2026 at 01:32:07PM +0800, Zeng Heng wrote:
> On 2026/5/21 23:15, Catalin Marinas wrote:
> > On Thu, May 21, 2026 at 04:05:07PM +0100, Catalin Marinas wrote:
> > > On Thu, May 21, 2026 at 03:30:11PM +0800, Zeng Heng wrote:
> > > > From: Zeng Heng <zengheng4@huawei.com>
> > > > 
> > > > When huge_pmd_unshare() is called to unshare a PMD table, the
> > > > tlb_unshare_pmd_ptdesc() function sets tlb->unshared_tables=true
> > > > but the aarch64 tlb_flush() only checked tlb->freed_tables to
> > > > determine whether to use TLBF_NONE (vae1is, invalidates walk
> > > > cache) or TLBF_NOWALKCACHE (vale1is, leaf-only).
> > > > 
> > > > This caused the stale PMD page table entry to remain in the walk cache
> > > > after unshare, potentially leading to incorrect page table walks.
> > > > 
> > > > Fix by including unshared_tables in the check, so that when
> > > > unsharing tables, TLBF_NONE is used and the walk cache is properly
> > > > invalidated.
> > > > 
> > > > Here is the detailed distinction between vae1is and vale1is:
> > > > 
> > > > | Instruction Combination  | Actual Invalidation Scope                         |
> > > > | ------------------------ | --------------------------------------------------|
> > > > | `VAE1IS`  + TTL=`0`      | All entries at all levels (full invalidation)     |
> > > > | `VAE1IS`  + TTL=`2` (L2) | Non-leaf at Level 0/1 + leaf at Level 2           |
> > > > | `VALE1IS` + TTL=`0`      | Leaf entries at all levels (non-leaf not cleared) |
> > > > | `VALE1IS` + TTL=`2` (L2) | Leaf entry at Level 2 only                        |
> > > > 
> > > > Signed-off-by: Zeng Heng <zengheng4@huawei.com>
> > > The fix looks fine but does it need:
> > > 
> > > Fixes: 8ce720d5bd91 ("mm/hugetlb: fix excessive IPI broadcasts when unsharing PMD tables using mmu_gather")
> > > Cc: <stable@vger.kernel.org>
> > > 
> > > > ---
> > > >   arch/arm64/include/asm/tlb.h | 3 ++-
> > > >   1 file changed, 2 insertions(+), 1 deletion(-)
> > > > 
> > > > diff --git a/arch/arm64/include/asm/tlb.h b/arch/arm64/include/asm/tlb.h
> > > > index 10869d7731b8..751bd57bc3ba 100644
> > > > --- a/arch/arm64/include/asm/tlb.h
> > > > +++ b/arch/arm64/include/asm/tlb.h
> > > > @@ -53,7 +53,8 @@ static inline int tlb_get_level(struct mmu_gather *tlb)
> > > >   static inline void tlb_flush(struct mmu_gather *tlb)
> > > >   {
> > > >   	struct vm_area_struct vma = TLB_FLUSH_VMA(tlb->mm, 0);
> > > > -	tlbf_t flags = tlb->freed_tables ? TLBF_NONE : TLBF_NOWALKCACHE;
> > > > +	tlbf_t flags = (tlb->freed_tables || tlb->unshared_tables) ?
> > > > +			TLBF_NONE : TLBF_NOWALKCACHE;
> > > >   	unsigned long stride = tlb_get_unmap_size(tlb);
> > > >   	int tlb_level = tlb_get_level(tlb);
> > Do we need this as well?
> 
> The proposed fix has been validated against the issue scenarios and
> works as expected.
> 
> Per the ARM Architecture Reference Manual, whether only the last-level
> page table entry is invalidated is determined by the instruction used
> (vale1is for leaf entry only, vae1is for walk cache including leaf entry and
> non-leaf entry), rather than the TTL field. The TTL field merely specifies
> which level the leaf entry belongs to.

Ah, yes, you are right. The TTL is still 2 in this case for a huge pmd,
we just want the walk cache leading to it to be invalidated. So no need
for the additional tlb_get_level().

Thanks.

-- 
Catalin


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] arm64: tlb: Flush walk cache when unsharing PMD tables
  2026-05-22 10:13       ` Catalin Marinas
@ 2026-05-22 10:38         ` Catalin Marinas
  0 siblings, 0 replies; 8+ messages in thread
From: Catalin Marinas @ 2026-05-22 10:38 UTC (permalink / raw)
  To: Zeng Heng
  Cc: yezhenyu2, zhurui3, will, akpm, npiggin, aneesh.kumar, peterz,
	linux-kernel, wangkefeng.wang, linux-arm-kernel, linux-mm,
	linux-arch, David Hildenbrand, zengheng4

On Fri, May 22, 2026 at 11:13:17AM +0100, Catalin Marinas wrote:
> On Fri, May 22, 2026 at 01:32:07PM +0800, Zeng Heng wrote:
> > On 2026/5/21 23:15, Catalin Marinas wrote:
> > > On Thu, May 21, 2026 at 04:05:07PM +0100, Catalin Marinas wrote:
> > > > On Thu, May 21, 2026 at 03:30:11PM +0800, Zeng Heng wrote:
> > > > > From: Zeng Heng <zengheng4@huawei.com>
> > > > > 
> > > > > When huge_pmd_unshare() is called to unshare a PMD table, the
> > > > > tlb_unshare_pmd_ptdesc() function sets tlb->unshared_tables=true
> > > > > but the aarch64 tlb_flush() only checked tlb->freed_tables to
> > > > > determine whether to use TLBF_NONE (vae1is, invalidates walk
> > > > > cache) or TLBF_NOWALKCACHE (vale1is, leaf-only).
> > > > > 
> > > > > This caused the stale PMD page table entry to remain in the walk cache
> > > > > after unshare, potentially leading to incorrect page table walks.
> > > > > 
> > > > > Fix by including unshared_tables in the check, so that when
> > > > > unsharing tables, TLBF_NONE is used and the walk cache is properly
> > > > > invalidated.
> > > > > 
> > > > > Here is the detailed distinction between vae1is and vale1is:
> > > > > 
> > > > > | Instruction Combination  | Actual Invalidation Scope                         |
> > > > > | ------------------------ | --------------------------------------------------|
> > > > > | `VAE1IS`  + TTL=`0`      | All entries at all levels (full invalidation)     |
> > > > > | `VAE1IS`  + TTL=`2` (L2) | Non-leaf at Level 0/1 + leaf at Level 2           |
> > > > > | `VALE1IS` + TTL=`0`      | Leaf entries at all levels (non-leaf not cleared) |
> > > > > | `VALE1IS` + TTL=`2` (L2) | Leaf entry at Level 2 only                        |
[...]
> > Per the ARM Architecture Reference Manual, whether only the last-level
> > page table entry is invalidated is determined by the instruction used
> > (vale1is for leaf entry only, vae1is for walk cache including leaf entry and
> > non-leaf entry), rather than the TTL field. The TTL field merely specifies
> > which level the leaf entry belongs to.
> 
> Ah, yes, you are right. The TTL is still 2 in this case for a huge pmd,
> we just want the walk cache leading to it to be invalidated. So no need
> for the additional tlb_get_level().

The Arm ARM is still unclear. The RVAE1IS has this wording:

  The TTL hint is only guaranteed to invalidate:

  - Non-leaf-level entries in the range up to but not including the
    level described by the TTL hint.

  - Leaf-level entries in the range that match the level described by
    the TTL hint.

But we don't have such wording around non-leaf-level entries for VAE1IS.
I presume it would be the same but I'll ask internally next week. In the
meantime, I'll take this patch.

-- 
Catalin


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] arm64: tlb: Flush walk cache when unsharing PMD tables
  2026-05-21  7:30 [PATCH] arm64: tlb: Flush walk cache when unsharing PMD tables Zeng Heng
  2026-05-21 15:05 ` Catalin Marinas
@ 2026-05-22 10:42 ` Catalin Marinas
  1 sibling, 0 replies; 8+ messages in thread
From: Catalin Marinas @ 2026-05-22 10:42 UTC (permalink / raw)
  To: will, akpm, npiggin, aneesh.kumar, peterz, Zeng Heng
  Cc: linux-kernel, wangkefeng.wang, linux-arm-kernel, linux-mm,
	linux-arch

On Thu, 21 May 2026 15:30:11 +0800, Zeng Heng wrote:
> When huge_pmd_unshare() is called to unshare a PMD table, the
> tlb_unshare_pmd_ptdesc() function sets tlb->unshared_tables=true
> but the aarch64 tlb_flush() only checked tlb->freed_tables to
> determine whether to use TLBF_NONE (vae1is, invalidates walk
> cache) or TLBF_NOWALKCACHE (vale1is, leaf-only).
> 
> This caused the stale PMD page table entry to remain in the walk cache
> after unshare, potentially leading to incorrect page table walks.
> 
> [...]

Applied to arm64 (for-next/fixes), thanks!

[1/1] arm64: tlb: Flush walk cache when unsharing PMD tables
      https://git.kernel.org/arm64/c/c2ff4764e03e


^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2026-05-22 10:43 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-05-21  7:30 [PATCH] arm64: tlb: Flush walk cache when unsharing PMD tables Zeng Heng
2026-05-21 15:05 ` Catalin Marinas
2026-05-21 15:15   ` Catalin Marinas
2026-05-22  5:32     ` Zeng Heng
2026-05-22 10:13       ` Catalin Marinas
2026-05-22 10:38         ` Catalin Marinas
2026-05-22  4:43   ` Zeng Heng
2026-05-22 10:42 ` Catalin Marinas

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox