[PATCH] mm/damon: fix stale TLB young-state handling on arm64

[Kunwu Chan <kunwu.chan@linux.dev>]

From: Kunwu Chan <chentao@kylinos.cn>

damon_ptep_mkold() clears the PTE Access Flag so that a later
access will set it again and damon_folio_young() can observe it
via pte_young().

On arm64, however, ptep_test_and_clear_young() clears AF in the
page tables without invalidating the corresponding TLB entry.
Subsequent accesses can therefore continue hitting a stale TLB
entry without a page table walk.  The PTE AF bit stays clear,
pte_young() reports false, and DAMON treats the region as
unaccessed.

folio_set_idle() does not help here.  It updates only software
state, and accesses through a stale TLB entry do not clear the
idle flag.

As a result, nr_accesses stays low regardless of the real access
pattern.  DAMOS schemes fail to match, WSS estimation reports
zero, and actions like pageout never trigger.

Fix this by switching to ptep_clear_flush_young() and
pmdp_clear_flush_young().

On arm64 these perform the required TLB invalidation after
clearing AF.  The invalidation is deferred, but still sufficient
for DAMON's sampling granularity.

On x86, ptep_clear_flush_young() is equivalent to
ptep_test_and_clear_young() for base pages, so there is no
behavioral change.  pmdp_clear_flush_young() additionally performs
a flush at PMD level, matching the existing x86 implementation.

On powerpc, riscv, and s390, the clear_flush variants currently
map back to test_and_clear implementations, so this patch does not
change their behavior.

Reproduced on arm64 (128 CPUs, 7.1.0-rc4):

  before:
    WSS estimation: 50th percentile error 100% (reported as zero)
    apply_interval: schemes never tried

  after:
    WSS estimation: 50th percentile error 0.08%
    apply_interval: passes

Co-developed-by: Wang Lian <lianux.mm@gmail.com>
Signed-off-by: Wang Lian <lianux.mm@gmail.com>
Signed-off-by: Kunwu Chan <chentao@kylinos.cn>
Tested-by: Kunwu Chan <chentao@kylinos.cn>
---
 mm/damon/ops-common.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/mm/damon/ops-common.c b/mm/damon/ops-common.c
index 8c6d61342..33d689c14 100644
--- a/mm/damon/ops-common.c
+++ b/mm/damon/ops-common.c
@@ -63,7 +63,7 @@ void damon_ptep_mkold(pte_t *pte, struct vm_area_struct *vma, unsigned long addr
 	 * device aspects.
 	 */
 	if (likely(pte_present(pteval)))
-		young |= ptep_test_and_clear_young(vma, addr, pte);
+		young |= ptep_clear_flush_young(vma, addr, pte);
 	young |= mmu_notifier_clear_young(vma->vm_mm, addr, addr + PAGE_SIZE);
 	if (young)
 		folio_set_young(folio);
@@ -90,7 +90,7 @@ void damon_pmdp_mkold(pmd_t *pmd, struct vm_area_struct *vma, unsigned long addr
 		return;
 
 	if (likely(pmd_present(pmdval)))
-		young |= pmdp_test_and_clear_young(vma, addr, pmd);
+		young |= pmdp_clear_flush_young(vma, addr, pmd);
 	young |= mmu_notifier_clear_young(vma->vm_mm, addr, addr + HPAGE_PMD_SIZE);
 	if (young)
 		folio_set_young(folio);
-- 
2.43.0