* [PATCH v2 0/4] arm64: Add PUD THP support for DAX and pfnmap
@ 2026-11-17 2:02 Jinjiang Tu
2026-11-17 2:02 ` [PATCH v2 1/4] mm: define generic pudp_invalidate() implementation Jinjiang Tu
` (3 more replies)
0 siblings, 4 replies; 6+ messages in thread
From: Jinjiang Tu @ 2026-11-17 2:02 UTC (permalink / raw)
To: akpm, david, ljs, catalin.marinas, will, mark.rutland,
ryan.roberts, peterx, dave.jiang, linux-arm-kernel, linux-mm
Cc: wangkefeng.wang, sunnanyong, tujinjiang
This series adds PUD THP support for arm64, enabling PUD THP for DAX
and pfnmap.
Patch 1 moves the identical pudp_invalidate() implementations from x86
and riscv to a generic location so arm64 can reuse it.
Patch 2 is a minor cleanup that simplifies pmd_special/pmd_mkspecial.
Patch 3 adds the necessary spurious PUD fault fix and PUD page table
helpers and selects HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD.
Changes since v1:
* fix issues reported by sashiko
Jinjiang Tu (4):
mm: define generic pudp_invalidate() implementation
arm64: simplify pmd_special and pmd_mkspecial
mm: add spurious fault fixing support for huge pud
arm64: Add PUD THP support for DAX and pfnmap
arch/arm64/Kconfig | 1 +
arch/arm64/include/asm/pgtable.h | 104 +++++++++++++++----
arch/powerpc/include/asm/book3s/64/pgtable.h | 1 +
arch/riscv/include/asm/pgtable.h | 3 -
arch/riscv/mm/pgtable.c | 9 --
arch/x86/include/asm/pgtable.h | 3 -
arch/x86/mm/pgtable.c | 12 ---
include/linux/huge_mm.h | 5 +-
include/linux/pgtable.h | 11 ++
mm/huge_memory.c | 17 +--
mm/internal.h | 2 +-
mm/memory.c | 14 ++-
mm/pgtable-generic.c | 14 +++
13 files changed, 131 insertions(+), 65 deletions(-)
--
2.54.0
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v2 1/4] mm: define generic pudp_invalidate() implementation
2026-11-17 2:02 [PATCH v2 0/4] arm64: Add PUD THP support for DAX and pfnmap Jinjiang Tu
@ 2026-11-17 2:02 ` Jinjiang Tu
2026-11-17 21:32 ` David Hildenbrand (Arm)
2026-11-17 2:02 ` [PATCH v2 2/4] arm64: simplify pmd_special and pmd_mkspecial Jinjiang Tu
` (2 subsequent siblings)
3 siblings, 1 reply; 6+ messages in thread
From: Jinjiang Tu @ 2026-11-17 2:02 UTC (permalink / raw)
To: akpm, david, ljs, catalin.marinas, will, mark.rutland,
ryan.roberts, peterx, dave.jiang, linux-arm-kernel, linux-mm
Cc: wangkefeng.wang, sunnanyong, tujinjiang
x86 and riscv have identical pudp_invalidate() implementations. Move it to
mm/pgtable-generic.c with the __HAVE_ARCH_PUDP_INVALIDATE override guard.
PowerPC keeps its own implementation. In the later patch, arm64 will use
the generic implementation too.
No functional change intended.
Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com>
---
arch/powerpc/include/asm/book3s/64/pgtable.h | 1 +
arch/riscv/include/asm/pgtable.h | 3 ---
arch/riscv/mm/pgtable.c | 9 ---------
arch/x86/include/asm/pgtable.h | 3 ---
arch/x86/mm/pgtable.c | 12 ------------
include/linux/pgtable.h | 7 +++++++
mm/pgtable-generic.c | 14 ++++++++++++++
7 files changed, 22 insertions(+), 27 deletions(-)
diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h
index dff8790a047d..267d5e24f3b0 100644
--- a/arch/powerpc/include/asm/book3s/64/pgtable.h
+++ b/arch/powerpc/include/asm/book3s/64/pgtable.h
@@ -1412,6 +1412,7 @@ static inline pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm,
#define __HAVE_ARCH_PMDP_INVALIDATE
extern pmd_t pmdp_invalidate(struct vm_area_struct *vma, unsigned long address,
pmd_t *pmdp);
+#define __HAVE_ARCH_PUDP_INVALIDATE
extern pud_t pudp_invalidate(struct vm_area_struct *vma, unsigned long address,
pud_t *pudp);
diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index 2ebd5fa5adc5..174a2f2bcaa2 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -1162,9 +1162,6 @@ static inline pud_t pud_mkinvalid(pud_t pud)
return __pud(pud_val(pud) & ~(_PAGE_PRESENT | _PAGE_PROT_NONE));
}
-extern pud_t pudp_invalidate(struct vm_area_struct *vma, unsigned long address,
- pud_t *pudp);
-
static inline pud_t pud_modify(pud_t pud, pgprot_t newprot)
{
return pte_pud(pte_modify(pud_pte(pud), newprot));
diff --git a/arch/riscv/mm/pgtable.c b/arch/riscv/mm/pgtable.c
index cd180f9abb5e..7c76756b59da 100644
--- a/arch/riscv/mm/pgtable.c
+++ b/arch/riscv/mm/pgtable.c
@@ -154,15 +154,6 @@ pmd_t pmdp_collapse_flush(struct vm_area_struct *vma,
return pmd;
}
-pud_t pudp_invalidate(struct vm_area_struct *vma, unsigned long address,
- pud_t *pudp)
-{
- VM_WARN_ON_ONCE(!pud_present(*pudp));
- pud_t old = pudp_establish(vma, address, pudp, pud_mkinvalid(*pudp));
-
- flush_pud_tlb_range(vma, address, address + HPAGE_PUD_SIZE);
- return old;
-}
#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
pte_t pte_mkwrite(pte_t pte, struct vm_area_struct *vma)
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index 000fc6c1ca37..ad7a40abef7d 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -1391,9 +1391,6 @@ static inline pud_t pudp_establish(struct vm_area_struct *vma,
extern pmd_t pmdp_invalidate_ad(struct vm_area_struct *vma,
unsigned long address, pmd_t *pmdp);
-pud_t pudp_invalidate(struct vm_area_struct *vma, unsigned long address,
- pud_t *pudp);
-
/*
* Page table pages are page-aligned. The lower half of the top
* level is used for userspace and the top half for the kernel.
diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c
index 4a105f283cfb..591931d144f2 100644
--- a/arch/x86/mm/pgtable.c
+++ b/arch/x86/mm/pgtable.c
@@ -519,18 +519,6 @@ pmd_t pmdp_invalidate_ad(struct vm_area_struct *vma, unsigned long address,
}
#endif
-#if defined(CONFIG_TRANSPARENT_HUGEPAGE) && \
- defined(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD)
-pud_t pudp_invalidate(struct vm_area_struct *vma, unsigned long address,
- pud_t *pudp)
-{
- VM_WARN_ON_ONCE(!pud_present(*pudp));
- pud_t old = pudp_establish(vma, address, pudp, pud_mkinvalid(*pudp));
- flush_pud_tlb_range(vma, address, address + HPAGE_PUD_SIZE);
- return old;
-}
-#endif
-
/**
* reserve_top_address - Reserve a hole in the top of the kernel address space
* @reserve: Size of hole to reserve
diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h
index 780fe849ff8b..0021eaf88e1d 100644
--- a/include/linux/pgtable.h
+++ b/include/linux/pgtable.h
@@ -1318,6 +1318,13 @@ extern pmd_t pmdp_invalidate_ad(struct vm_area_struct *vma,
unsigned long address, pmd_t *pmdp);
#endif
+#ifndef __HAVE_ARCH_PUDP_INVALIDATE
+#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
+pud_t pudp_invalidate(struct vm_area_struct *vma, unsigned long address,
+ pud_t *pudp);
+#endif
+#endif
+
#ifndef __HAVE_ARCH_PTE_SAME
static inline int pte_same(pte_t pte_a, pte_t pte_b)
{
diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c
index 6e83ce3801b8..1b7081fc4a75 100644
--- a/mm/pgtable-generic.c
+++ b/mm/pgtable-generic.c
@@ -229,6 +229,20 @@ pmd_t pmdp_invalidate_ad(struct vm_area_struct *vma, unsigned long address,
}
#endif
+#ifndef __HAVE_ARCH_PUDP_INVALIDATE
+#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
+pud_t pudp_invalidate(struct vm_area_struct *vma, unsigned long address,
+ pud_t *pudp)
+{
+ VM_WARN_ON_ONCE(!pud_present(*pudp));
+ pud_t old = pudp_establish(vma, address, pudp, pud_mkinvalid(*pudp));
+
+ flush_pud_tlb_range(vma, address, address + HPAGE_PUD_SIZE);
+ return old;
+}
+#endif
+#endif
+
#ifndef pmdp_collapse_flush
pmd_t pmdp_collapse_flush(struct vm_area_struct *vma, unsigned long address,
pmd_t *pmdp)
--
2.54.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v2 2/4] arm64: simplify pmd_special and pmd_mkspecial
2026-11-17 2:02 [PATCH v2 0/4] arm64: Add PUD THP support for DAX and pfnmap Jinjiang Tu
2026-11-17 2:02 ` [PATCH v2 1/4] mm: define generic pudp_invalidate() implementation Jinjiang Tu
@ 2026-11-17 2:02 ` Jinjiang Tu
2026-11-17 2:02 ` [PATCH v2 3/4] mm: add spurious fault fixing support for huge pud Jinjiang Tu
2026-11-17 2:02 ` [PATCH v2 4/4] arm64: Add PUD THP support for DAX and pfnmap Jinjiang Tu
3 siblings, 0 replies; 6+ messages in thread
From: Jinjiang Tu @ 2026-11-17 2:02 UTC (permalink / raw)
To: akpm, david, ljs, catalin.marinas, will, mark.rutland,
ryan.roberts, peterx, dave.jiang, linux-arm-kernel, linux-mm
Cc: wangkefeng.wang, sunnanyong, tujinjiang
Use pte_special() and pte_mkspecial() helper to simplify pmd_special()
and pmd_mkspecial(), and kill unused set_pmd_bit() and clear_pmd_bit()
helper.
Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com>
---
arch/arm64/include/asm/pgtable.h | 19 ++-----------------
1 file changed, 2 insertions(+), 17 deletions(-)
diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index a474c9fb485a..a06b0f5c3411 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -240,18 +240,6 @@ static inline pte_t set_pte_bit(pte_t pte, pgprot_t prot)
return pte;
}
-static inline pmd_t clear_pmd_bit(pmd_t pmd, pgprot_t prot)
-{
- pmd_val(pmd) &= ~pgprot_val(prot);
- return pmd;
-}
-
-static inline pmd_t set_pmd_bit(pmd_t pmd, pgprot_t prot)
-{
- pmd_val(pmd) |= pgprot_val(prot);
- return pmd;
-}
-
static inline pte_t pte_mkwrite_novma(pte_t pte)
{
pte = set_pte_bit(pte, __pgprot(PTE_WRITE));
@@ -577,11 +565,8 @@ static inline pmd_t pmd_mkhuge(pmd_t pmd)
}
#ifdef CONFIG_ARCH_SUPPORTS_PMD_PFNMAP
-#define pmd_special(pte) (!!((pmd_val(pte) & PTE_SPECIAL)))
-static inline pmd_t pmd_mkspecial(pmd_t pmd)
-{
- return set_pmd_bit(pmd, __pgprot(PTE_SPECIAL));
-}
+#define pmd_special(pmd) pte_special(pmd_pte(pmd))
+#define pmd_mkspecial(pmd) pte_pmd(pte_mkspecial(pmd_pte(pmd)))
#endif
#define __pmd_to_phys(pmd) __pte_to_phys(pmd_pte(pmd))
--
2.54.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v2 3/4] mm: add spurious fault fixing support for huge pud
2026-11-17 2:02 [PATCH v2 0/4] arm64: Add PUD THP support for DAX and pfnmap Jinjiang Tu
2026-11-17 2:02 ` [PATCH v2 1/4] mm: define generic pudp_invalidate() implementation Jinjiang Tu
2026-11-17 2:02 ` [PATCH v2 2/4] arm64: simplify pmd_special and pmd_mkspecial Jinjiang Tu
@ 2026-11-17 2:02 ` Jinjiang Tu
2026-11-17 2:02 ` [PATCH v2 4/4] arm64: Add PUD THP support for DAX and pfnmap Jinjiang Tu
3 siblings, 0 replies; 6+ messages in thread
From: Jinjiang Tu @ 2026-11-17 2:02 UTC (permalink / raw)
To: akpm, david, ljs, catalin.marinas, will, mark.rutland,
ryan.roberts, peterx, dave.jiang, linux-arm-kernel, linux-mm
Cc: wangkefeng.wang, sunnanyong, tujinjiang
Spurious fault fixing is supported for PTE and huge PMD mappings, but not
for huge PUD mappings.
The next patch in the series adds PUD THP support for DAX and pfnmap on
arm64. arm64 write protection faults only flush the local TLB and leave
stale read-only entries to be flushed by the faulting CPU, so these
mappings need the huge PUD variant of the same fix.
Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com>
---
arch/arm64/include/asm/pgtable.h | 4 ++++
include/linux/huge_mm.h | 5 +++--
include/linux/pgtable.h | 4 ++++
mm/huge_memory.c | 17 +++++++++--------
mm/internal.h | 2 +-
mm/memory.c | 14 ++++++++++----
6 files changed, 31 insertions(+), 15 deletions(-)
diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index a06b0f5c3411..3c1c000cc5dd 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -107,6 +107,10 @@ static inline void arch_leave_lazy_mmu_mode(void)
__flush_tlb_range(vma, address, address + PMD_SIZE, PMD_SIZE, 2, \
TLBF_NOBROADCAST | TLBF_NONOTIFY | TLBF_NOWALKCACHE)
+#define flush_tlb_fix_spurious_fault_pud(vma, address, pudp) \
+ __flush_tlb_range(vma, address, address + PUD_SIZE, PUD_SIZE, 1, \
+ TLBF_NOBROADCAST | TLBF_NONOTIFY | TLBF_NOWALKCACHE)
+
#ifdef CONFIG_ARM64_PA_BITS_52
static inline phys_addr_t __pte_to_phys(pte_t pte)
{
diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
index fac05120ab77..1d8ab7c864f2 100644
--- a/include/linux/huge_mm.h
+++ b/include/linux/huge_mm.h
@@ -16,10 +16,11 @@ int copy_huge_pud(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma
pud_t *dst_pud, pud_t *src_pud, unsigned long addr);
#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
-void huge_pud_set_accessed(struct vm_fault *vmf, pud_t orig_pud);
+bool huge_pud_set_accessed(struct vm_fault *vmf, pud_t orig_pud);
#else
-static inline void huge_pud_set_accessed(struct vm_fault *vmf, pud_t orig_pud)
+static inline bool huge_pud_set_accessed(struct vm_fault *vmf, pud_t orig_pud)
{
+ return true;
}
#endif
diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h
index 0021eaf88e1d..795bbc313bcb 100644
--- a/include/linux/pgtable.h
+++ b/include/linux/pgtable.h
@@ -1493,6 +1493,10 @@ static inline void arch_swap_restore(swp_entry_t entry, struct folio *folio)
#define flush_tlb_fix_spurious_fault_pmd(vma, address, pmdp) do { } while (0)
#endif
+#ifndef flush_tlb_fix_spurious_fault_pud
+#define flush_tlb_fix_spurious_fault_pud(vma, address, pudp) do { } while (0)
+#endif
+
/*
* When walking page tables, get the address of the next boundary,
* or the end address of the range if that comes earlier. Although no
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index eb913acb03b7..09bf622b14ea 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2059,7 +2059,7 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm,
}
#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
-void touch_pud(struct vm_area_struct *vma, unsigned long addr,
+bool touch_pud(struct vm_area_struct *vma, unsigned long addr,
pud_t *pud, bool write)
{
pud_t _pud;
@@ -2068,8 +2068,12 @@ void touch_pud(struct vm_area_struct *vma, unsigned long addr,
if (write)
_pud = pud_mkdirty(_pud);
if (pudp_set_access_flags(vma, addr & HPAGE_PUD_MASK,
- pud, _pud, write))
+ pud, _pud, write)) {
update_mmu_cache_pud(vma, addr, pud);
+ return true;
+ }
+
+ return false;
}
int copy_huge_pud(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma,
@@ -2118,17 +2122,14 @@ int copy_huge_pud(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma
return ret;
}
-void huge_pud_set_accessed(struct vm_fault *vmf, pud_t orig_pud)
+bool huge_pud_set_accessed(struct vm_fault *vmf, pud_t orig_pud)
{
bool write = vmf->flags & FAULT_FLAG_WRITE;
- vmf->ptl = pud_lock(vmf->vma->vm_mm, vmf->pud);
if (unlikely(!pud_same(*vmf->pud, orig_pud)))
- goto unlock;
+ return false;
- touch_pud(vmf->vma, vmf->address, vmf->pud, write);
-unlock:
- spin_unlock(vmf->ptl);
+ return touch_pud(vmf->vma, vmf->address, vmf->pud, write);
}
#endif /* CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */
diff --git a/mm/internal.h b/mm/internal.h
index e3f92883bf23..0a0a71fa110d 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -1311,7 +1311,7 @@ int __must_check try_grab_folio(struct folio *folio, int refs,
/*
* mm/huge_memory.c
*/
-void touch_pud(struct vm_area_struct *vma, unsigned long addr,
+bool touch_pud(struct vm_area_struct *vma, unsigned long addr,
pud_t *pud, bool write);
bool touch_pmd(struct vm_area_struct *vma, unsigned long addr,
pmd_t *pmd, bool write);
diff --git a/mm/memory.c b/mm/memory.c
index ec3d9d99ccee..94a6d106e801 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -6578,8 +6578,8 @@ static vm_fault_t wp_huge_pud(struct vm_fault *vmf, pud_t orig_pud)
*
* Note: flush_tlb_fix_spurious_fault() is defined as flush_tlb_page()
* by default and used as such on most architectures, while
- * flush_tlb_fix_spurious_fault_pmd() is defined as NOP by default and
- * used as such on most architectures.
+ * flush_tlb_fix_spurious_fault_pmd() and flush_tlb_fix_spurious_fault_pud()
+ * are defined as NOP by default and used as such on most architectures.
*/
static void fix_spurious_fault(struct vm_fault *vmf,
enum pgtable_level ptlevel)
@@ -6597,9 +6597,12 @@ static void fix_spurious_fault(struct vm_fault *vmf,
if (ptlevel == PGTABLE_LEVEL_PTE)
flush_tlb_fix_spurious_fault(vmf->vma, vmf->address,
vmf->pte);
- else
+ else if (ptlevel == PGTABLE_LEVEL_PMD)
flush_tlb_fix_spurious_fault_pmd(vmf->vma, vmf->address,
vmf->pmd);
+ else
+ flush_tlb_fix_spurious_fault_pud(vmf->vma, vmf->address,
+ vmf->pud);
}
}
/*
@@ -6757,7 +6760,10 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma,
if (!(ret & VM_FAULT_FALLBACK))
return ret;
} else {
- huge_pud_set_accessed(&vmf, orig_pud);
+ vmf.ptl = pud_lock(vmf.vma->vm_mm, vmf.pud);
+ if (!huge_pud_set_accessed(&vmf, orig_pud))
+ fix_spurious_fault(&vmf, PGTABLE_LEVEL_PUD);
+ spin_unlock(vmf.ptl);
return 0;
}
}
--
2.54.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v2 4/4] arm64: Add PUD THP support for DAX and pfnmap
2026-11-17 2:02 [PATCH v2 0/4] arm64: Add PUD THP support for DAX and pfnmap Jinjiang Tu
` (2 preceding siblings ...)
2026-11-17 2:02 ` [PATCH v2 3/4] mm: add spurious fault fixing support for huge pud Jinjiang Tu
@ 2026-11-17 2:02 ` Jinjiang Tu
3 siblings, 0 replies; 6+ messages in thread
From: Jinjiang Tu @ 2026-11-17 2:02 UTC (permalink / raw)
To: akpm, david, ljs, catalin.marinas, will, mark.rutland,
ryan.roberts, peterx, dave.jiang, linux-arm-kernel, linux-mm
Cc: wangkefeng.wang, sunnanyong, tujinjiang
Add the necessary page table functions to deal with PUD THP, this enables
PUD THP for DAX and pfnmap.
I tested with CONFIG_DEBUG_VM_PGTABLE and CONFIG_PAGE_TABLE_CHECK enabled.
I wrote a test driver that invokes vmf_insert_pfn_pud() in the huge_fault
callback. Page table setup works as expected with no errors, and userspace
read/write access to the PUD-sized mapping is verified correct.
Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com>
---
arch/arm64/Kconfig | 1 +
arch/arm64/include/asm/pgtable.h | 81 +++++++++++++++++++++++++++++---
2 files changed, 76 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 1075bc661833..808d7106df34 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -173,6 +173,7 @@ config ARM64
select HAVE_ARCH_THREAD_STRUCT_WHITELIST
select HAVE_ARCH_TRACEHOOK
select HAVE_ARCH_TRANSPARENT_HUGEPAGE
+ select HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD if PAGE_SIZE_4KB
select HAVE_ARCH_VMAP_STACK
select HAVE_ARM_SMCCC
select HAVE_ASM_MODVERSIONS
diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index 3c1c000cc5dd..2dc6aa3096d8 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -573,6 +573,11 @@ static inline pmd_t pmd_mkhuge(pmd_t pmd)
#define pmd_mkspecial(pmd) pte_pmd(pte_mkspecial(pmd_pte(pmd)))
#endif
+#ifdef CONFIG_ARCH_SUPPORTS_PUD_PFNMAP
+#define pud_special(pud) pte_special(pud_pte(pud))
+#define pud_mkspecial(pud) pte_pud(pte_mkspecial(pud_pte(pud)))
+#endif
+
#define __pmd_to_phys(pmd) __pte_to_phys(pmd_pte(pmd))
#define __phys_to_pmd_val(phys) __phys_to_pte_val(phys)
#define pmd_pfn(pmd) ((__pmd_to_phys(pmd) & PMD_MASK) >> PAGE_SHIFT)
@@ -581,8 +586,15 @@ static inline pmd_t pmd_mkhuge(pmd_t pmd)
#define pud_young(pud) pte_young(pud_pte(pud))
#define pud_mkyoung(pud) pte_pud(pte_mkyoung(pud_pte(pud)))
#define pud_mkwrite_novma(pud) pte_pud(pte_mkwrite_novma(pud_pte(pud)))
+#define pud_mkwrite(pud) pud_mkwrite_novma(pud)
#define pud_mkvalid_k(pud) pte_pud(pte_mkvalid_k(pud_pte(pud)))
#define pud_write(pud) pte_write(pud_pte(pud))
+#define pud_dirty(pud) pte_dirty(pud_pte(pud))
+#define pud_wrprotect(pud) pte_pud(pte_wrprotect(pud_pte(pud)))
+#define pud_mkdirty(pud) pte_pud(pte_mkdirty(pud_pte(pud)))
+#define pud_mkold(pud) pte_pud(pte_mkold(pud_pte(pud)))
+#define pud_mkinvalid(pud) pte_pud(pte_mkinvalid(pud_pte(pud)))
+#define pud_mkclean(pud) pte_pud(pte_mkclean(pud_pte(pud)))
static inline pud_t pud_mkhuge(pud_t pud)
{
@@ -738,6 +750,13 @@ static inline bool pmd_leaf(pmd_t pmd)
#define pmd_leaf_size(pmd) (pmd_cont(pmd) ? CONT_PMD_SIZE : PMD_SIZE)
#define pte_leaf_size(pte) (pte_cont(pte) ? CONT_PTE_SIZE : PAGE_SIZE)
+#if defined(CONFIG_ARM64_64K_PAGES) || CONFIG_PGTABLE_LEVELS < 3
+static inline bool pud_table(pud_t pud) { return true; }
+#else
+#define pud_table(pud) ((pud_val(pud) & PUD_TYPE_MASK) == \
+ PUD_TYPE_TABLE)
+#endif
+
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
static inline int pmd_trans_huge(pmd_t pmd)
{
@@ -747,14 +766,15 @@ static inline int pmd_trans_huge(pmd_t pmd)
*/
return pmd_present(pmd) && !pmd_table(__pmd(pmd_val(pmd) | PTE_VALID));
}
-#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
-#if defined(CONFIG_ARM64_64K_PAGES) || CONFIG_PGTABLE_LEVELS < 3
-static inline bool pud_table(pud_t pud) { return true; }
-#else
-#define pud_table(pud) ((pud_val(pud) & PUD_TYPE_MASK) == \
- PUD_TYPE_TABLE)
+#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
+static inline int pud_trans_huge(pud_t pud)
+{
+ return pte_present(pud_pte(pud)) &&
+ !pud_table(__pud(pud_val(pud) | PTE_VALID));
+}
#endif
+#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
extern pgd_t swapper_pg_dir[];
extern pgd_t idmap_pg_dir[];
@@ -1187,6 +1207,11 @@ static inline pmd_t pmd_modify(pmd_t pmd, pgprot_t newprot)
return pte_pmd(pte_modify(pmd_pte(pmd), newprot));
}
+static inline pud_t pud_modify(pud_t pud, pgprot_t newprot)
+{
+ return pte_pud(pte_modify(pud_pte(pud), newprot));
+}
+
extern int __ptep_set_access_flags_anysz(struct vm_area_struct *vma,
unsigned long address, pte_t *ptep,
pte_t entry, int dirty,
@@ -1209,7 +1234,17 @@ static inline int pmdp_set_access_flags(struct vm_area_struct *vma,
return __ptep_set_access_flags_anysz(vma, address, (pte_t *)pmdp,
pmd_pte(entry), dirty, PMD_SIZE);
}
+
+#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
+static inline int pudp_set_access_flags(struct vm_area_struct *vma,
+ unsigned long address, pud_t *pudp,
+ pud_t entry, int dirty)
+{
+ return __ptep_set_access_flags_anysz(vma, address, (pte_t *)pudp,
+ pud_pte(entry), dirty, PUD_SIZE);
+}
#endif
+#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
#ifdef CONFIG_PAGE_TABLE_CHECK
static inline bool pte_user_accessible_page(struct mm_struct *mm, unsigned long addr, pte_t pte)
@@ -1285,6 +1320,14 @@ static inline bool pmdp_test_and_clear_young(struct vm_area_struct *vma,
}
#endif /* CONFIG_TRANSPARENT_HUGEPAGE || CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG */
+#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
+static inline bool pudp_test_and_clear_young(struct vm_area_struct *vma,
+ unsigned long address, pud_t *pudp)
+{
+ return __ptep_test_and_clear_young(vma, address, (pte_t *)pudp);
+}
+#endif
+
static inline pte_t __ptep_get_and_clear_anysz(struct mm_struct *mm,
unsigned long address,
pte_t *ptep,
@@ -1355,6 +1398,15 @@ static inline pmd_t pmdp_huge_get_and_clear(struct mm_struct *mm,
{
return pte_pmd(__ptep_get_and_clear_anysz(mm, address, (pte_t *)pmdp, PMD_SIZE));
}
+
+#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
+#define __HAVE_ARCH_PUDP_HUGE_GET_AND_CLEAR
+static inline pud_t pudp_huge_get_and_clear(struct mm_struct *mm,
+ unsigned long address, pud_t *pudp)
+{
+ return pte_pud(__ptep_get_and_clear_anysz(mm, address, (pte_t *)pudp, PUD_SIZE));
+}
+#endif
#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
static inline void ___ptep_set_wrprotect(struct mm_struct *mm,
@@ -1445,7 +1497,23 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma,
page_table_check_pmd_set(vma->vm_mm, address, pmdp, pmd);
return __pmd(xchg_relaxed(&pmd_val(*pmdp), pmd_val(pmd)));
}
+
+#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
+#define __HAVE_ARCH_PUDP_SET_WRPROTECT
+static inline void pudp_set_wrprotect(struct mm_struct *mm,
+ unsigned long address, pud_t *pudp)
+{
+ __ptep_set_wrprotect(mm, address, (pte_t *)pudp);
+}
+
+static inline pud_t pudp_establish(struct vm_area_struct *vma,
+ unsigned long address, pud_t *pudp, pud_t pud)
+{
+ page_table_check_pud_set(vma->vm_mm, address, pudp, pud);
+ return __pud(xchg_relaxed(&pud_val(*pudp), pud_val(pud)));
+}
#endif
+#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
/*
* Encode and decode a swap entry:
@@ -1521,6 +1589,7 @@ static inline void update_mmu_cache_range(struct vm_fault *vmf,
#define update_mmu_cache(vma, addr, ptep) \
update_mmu_cache_range(NULL, vma, addr, ptep, 1)
#define update_mmu_cache_pmd(vma, address, pmd) do { } while (0)
+#define update_mmu_cache_pud(vma, address, pud) do { } while (0)
#ifdef CONFIG_ARM64_PA_BITS_52
#define phys_to_ttbr(addr) (((addr) | ((addr) >> 46)) & TTBR_BADDR_MASK_52)
--
2.54.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH v2 1/4] mm: define generic pudp_invalidate() implementation
2026-11-17 2:02 ` [PATCH v2 1/4] mm: define generic pudp_invalidate() implementation Jinjiang Tu
@ 2026-11-17 21:32 ` David Hildenbrand (Arm)
0 siblings, 0 replies; 6+ messages in thread
From: David Hildenbrand (Arm) @ 2026-11-17 21:32 UTC (permalink / raw)
To: Jinjiang Tu, akpm, ljs, catalin.marinas, will, mark.rutland,
ryan.roberts, peterx, dave.jiang, linux-arm-kernel, linux-mm
Cc: wangkefeng.wang, sunnanyong
On 11/17/26 03:02, Jinjiang Tu wrote:
> x86 and riscv have identical pudp_invalidate() implementations. Move it to
> mm/pgtable-generic.c with the __HAVE_ARCH_PUDP_INVALIDATE override guard.
> PowerPC keeps its own implementation. In the later patch, arm64 will use
> the generic implementation too.
>
> No functional change intended.
>
> Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com>
> ---
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
--
Cheers,
David
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2026-11-17 21:32 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-17 2:02 [PATCH v2 0/4] arm64: Add PUD THP support for DAX and pfnmap Jinjiang Tu
2026-11-17 2:02 ` [PATCH v2 1/4] mm: define generic pudp_invalidate() implementation Jinjiang Tu
2026-11-17 21:32 ` David Hildenbrand (Arm)
2026-11-17 2:02 ` [PATCH v2 2/4] arm64: simplify pmd_special and pmd_mkspecial Jinjiang Tu
2026-11-17 2:02 ` [PATCH v2 3/4] mm: add spurious fault fixing support for huge pud Jinjiang Tu
2026-11-17 2:02 ` [PATCH v2 4/4] arm64: Add PUD THP support for DAX and pfnmap Jinjiang Tu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox