From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=100.103.45.18 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1794846858; b=HAnzYTmRVYz3HUUi1F8zmKahJwqyUiPMT6ohMJtE7SDEdyXPmwFD5D2xYycnneKfoJgV zt7XMxDYunOI9sQb/yysj+AmgDmTvm83y5+qMyCMg9iI7QSZ6cjomgWozDuaucABzOw5+ 1/QL180meu+Yn/drVYVxRbuNx4VlgfswVwDD7JXRrkSrD5PiOHvAvOzUqeiipftwOVgfJ c+ub4y9WXCy+AzXsMmlqOgLRjhn8CkMWN75u3HGyvg1p0Efze7p8hirQEohmYuxTlOk47 v3O0bI1tBnH139+zaGs/bV63qJMfQSz5H9XfE9jMoDvaoYRN0CFTO+rpWrjEEMN9Q1w== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1794846858; h=Received:DKIM-Signature:Date:To:From:Subject:Message-Id; bh=WyeXJuulX6fWEQkKAXTPRQhugBfqiBbYyCIoXCIDLDk=; b=yKsrVwUFfLtHH2VRaLrlSCcMzTAaVcjcdPeKu+bgRpm/TYKUeo7d+W/bnAo5ftAdTQtD kvEEj+EUycRH5/zHNr1O25EEv4MjXCLFWNOrDh6n69fB2fGeS+pbBk/YVUAlixvsczzDo fnRNukbudDRAnEbqINz/7LtDxCShpQeBb+3vQjKtmA0i+cMTgmC3/U+uSh3pOeQ02HZZk HT06IbhjAGbwgDrxGHTyqiQMaJIwBppqOrUAIUX41C4G4K/MZAuqfUZL+JbQgMK1nHLI/ YVui+LRAIo6ANiDx9QHe0iXhe7JpgCNDRZnqlQecONRs/nqqd+tqLYJhtTLu96iCudg== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass header.d=linux-foundation.org header.i=@linux-foundation.org header.a=rsa-sha256 header.s=korg header.b=vIzFBIaC; arc=none smtp.remote-ip=100.103.45.18 Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 17E42606A31C; Mon, 16 Nov 2026 16:34:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.a=rsa-sha256 header.s=korg header.b=vIzFBIaC Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7E1311F000C4; Mon, 16 Nov 2026 16:34:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux-foundation.org; s=korg; t=1794846857; bh=WyeXJuulX6fWEQkKAXTPRQhugBfqiBbYyCIoXCIDLDk=; h=Date:To:From:Subject; b=vIzFBIaCwL/ahjFPqYoSfOxR8lWHf7Pm/870F/qzJ9/Ont4jMHqNNwUB8EFCelvpZ A9BtfQxKU9cvKs9sjLfZn9ohe4MjzgrDEr179iwFuqf6NQNDPY0mxV681JVa69zkB1 pAtwbK/JEVuUswC4o3IIUSn+HPEAuH8L+On5xZ9A= Date: Mon, 16 Nov 2026 08:34:16 -0800 To: mm-commits@vger.kernel.org,zwisler@kernel.org,ziy@nvidia.com, will@kernel.org,vbabka@kernel.org,usama.arif@linux.dev,surenb@google.com, stable@vger.kernel.org,ryan.roberts@arm.com,rppt@kernel.org, peterz@infradead.org,npiggin@gmail.com,nico.pache@linux.dev,mhocko@suse.com, mason@kernel.org,ljs@kernel.org,liam@infradead.org,lance.yang@linux.dev, kas@kernel.org,jack@suse.cz,dev.jain@arm.com,baolin.wang@linux.alibaba.com, baohua@kernel.org,arnd@arndb.de,aneesh.kumar@kernel.org,david@kernel.org, akpm@linux-foundation.org From: Andrew Morton Subject: [to-be-updated] mm-huge_memory-flush-dirty-file-pmds-before-removing-rmap.patch removed from -mm tree Message-Id: <20261116163417.7E1311F000C4@smtp.kernel.org> X-Mailing-List: stable@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list The quilt patch titled Subject: mm/huge_memory: flush dirty file PMDs before removing rmap has been removed from the -mm tree. Its filename was mm-huge_memory-flush-dirty-file-pmds-before-removing-rmap.patch This patch was dropped because an updated version will be issued ------------------------------------------------------ From: "David Hildenbrand (Arm)" Subject: mm/huge_memory: flush dirty file PMDs before removing rmap Date: Fri, 13 Nov 2026 18:15:26 +0100 Unmapping a file PMD can lose concurrent stores when it races with writeback. zap_huge_pmd() removes rmap and drops the PMD lock before the gather flushes the TLB. Writeback can then clean the folio while another CPU can still write through a stale writable TLB entry. Those stores do not fault and dirty the folio again. Filesystem DAX has the same problem with its dirty tags. Commit 5df397dec7c4 ("mm: delay page_remove_rmap() until after the TLB has been flushed") fixed the same race for PTEs by adding delayed rmap removal to the mmu_gather infrastructure. The PMD path still removes rmap before flushing. Flush the TLB under the PMD lock before removing rmap for dirty file PMDs. Skip the early flush at address-space teardown, when no users remain, and for mappings without writeback, including shmem and device DAX. Use tlb_can_delay_rmap() to share the PTE path's architecture checks. On s390, clearing the entry already handles TLB synchronization, and __tlb_remove_page_size() immediately drops the mapping reference, so rmap must be removed first. UP cannot have concurrent stores from another CPU while holding the page table lock. Make the predicate side-effect-free and set the gather's delayed_rmap flag when an entry is actually queued for delayed rmap removal. Queue the folio before the early flush, while the gather still has a valid range. Keep batching references and drain a full batch only after dropping the PMD lock. The early flush resets the gather's TLB state, so draining the batch does not flush that range again. Although writing to files through mmap is generally discouraged, unmapping such mappings may become slower: each dirty file PMD now requires a TLB flush under the PMD lock. Future work should properly extend the delayed-rmap machinery to PMDs and hold the PMD spinlock across multiple entries, so their TLB invalidations and rmap removals can be batched. Link: https://lore.kernel.org/20261113-dirty_file_pmd-v1-1-8c11f2a2e65f@kernel.org Fixes: 4b4bb46d00b3 ("dax: clear dirty entry tags on cache flush") Signed-off-by: David Hildenbrand (Arm) Signed-off-by: Andrew Morton Reported-by: Kiryl Shutsemau Closes: https://lore.kernel.org/linux-mm/aoWw_iKEDLjKL2jr@thinkstation/ Reported-by: Chris Mason Suggested-by: Lance Yang Suggested-by: Usama Arif Reviewed-by: Lance Yang Cc: Will Deacon Cc: Aneesh Kumar K.V Cc: Nicholas Piggin Cc: Peter Zijlstra Cc: Arnd Bergmann Cc: Zi Yan Cc: Baolin Wang Cc: Liam R. Howlett Cc: Nico Pache Cc: Ryan Roberts Cc: Dev Jain Cc: Barry Song Cc: Lorenzo Stoakes Cc: Vlastimil Babka Cc: Mike Rapoport Cc: Suren Baghdasaryan Cc: Michal Hocko Cc: Jan Kara Cc: Ross Zwisler Cc: include/asm-generic/tlb.h | 10 ++------- mm/huge_memory.c | 38 ++++++++++++++++++++++++++++-------- mm/memory.c | 2 - mm/mmu_gather.c | 2 + 4 files changed, 36 insertions(+), 16 deletions(-) --- a/include/asm-generic/tlb.h~mm-huge_memory-flush-dirty-file-pmds-before-removing-rmap +++ a/include/asm-generic/tlb.h @@ -250,11 +250,7 @@ bool __tlb_remove_folio_pages(struct mmu unsigned int nr_pages, bool delay_rmap); #ifdef CONFIG_SMP -/* - * This both sets 'delayed_rmap', and returns true. It would be an inline - * function, except we define it before the 'struct mmu_gather'. - */ -#define tlb_delay_rmap(tlb) (((tlb)->delayed_rmap = 1), true) +#define tlb_can_delay_rmap() (true) extern void tlb_flush_rmaps(struct mmu_gather *tlb, struct vm_area_struct *vma); #endif @@ -267,8 +263,8 @@ extern void tlb_flush_rmaps(struct mmu_g * remote TLBs to flush and is not preemptible due to this * all happening under the page table lock. */ -#ifndef tlb_delay_rmap -#define tlb_delay_rmap(tlb) (false) +#ifndef tlb_can_delay_rmap +#define tlb_can_delay_rmap() (false) static inline void tlb_flush_rmaps(struct mmu_gather *tlb, struct vm_area_struct *vma) { } #endif --- a/mm/huge_memory.c~mm-huge_memory-flush-dirty-file-pmds-before-removing-rmap +++ a/mm/huge_memory.c @@ -2490,10 +2490,12 @@ static inline void zap_deposited_table(s mm_dec_nr_ptes(mm); } -static void zap_huge_pmd_folio(struct mm_struct *mm, struct vm_area_struct *vma, - pmd_t pmdval, struct folio *folio, bool is_present) +static void zap_huge_pmd_folio(struct mmu_gather *tlb, struct vm_area_struct *vma, + pmd_t pmdval, struct folio *folio, bool is_present, bool *force_flush) { const bool is_device_private = pmd_is_device_private_entry(pmdval); + struct mm_struct *mm = tlb->mm; + bool delay_rmap = false; if (folio_test_anon(folio)) { add_mm_counter(mm, MM_ANONPAGES, -HPAGE_PMD_NR); @@ -2501,23 +2503,41 @@ static void zap_huge_pmd_folio(struct mm add_mm_counter(mm, mm_counter_file(folio), -HPAGE_PMD_NR); - if (is_present && pmd_dirty(pmdval)) + if (is_present && pmd_dirty(pmdval)) { folio_mark_dirty(folio); + /* + * Remove rmap after the TLB flush if other CPUs could + * still write to a mapping that supports writeback. + */ + if (!tlb->fullmm && vma->vm_file && + mapping_can_writeback(vma->vm_file->f_mapping)) + delay_rmap = tlb_can_delay_rmap(); + } if (is_present && pmd_young(pmdval) && likely(vma_has_recency(vma))) folio_mark_accessed(folio); } + if (!is_present && !is_device_private) + return; + /* * Present and device private folios are rmappable. Remove the * rmap after transferring young+dirty to the folio. */ - if (is_present || is_device_private) + if (!delay_rmap) folio_remove_rmap_pmd(folio, &folio->page, vma); - /* Device private folios are pinned. */ + /* Device private folios are pinned and don't need a TLB flush. */ if (is_device_private) folio_put(folio); + else if (__tlb_remove_page_size(tlb, &folio->page, HPAGE_PMD_SIZE)) + *force_flush = true; + + if (delay_rmap) { + tlb_flush_mmu_tlbonly(tlb); + folio_remove_rmap_pmd(folio, &folio->page, vma); + } } static struct folio *normal_or_softleaf_folio_pmd(struct vm_area_struct *vma, @@ -2555,6 +2575,7 @@ bool zap_huge_pmd(struct mmu_gather *tlb struct mm_struct *mm = tlb->mm; struct folio *folio = NULL; bool is_present = false; + bool force_flush = false; bool has_deposit; spinlock_t *ptl; pmd_t orig_pmd; @@ -2579,13 +2600,14 @@ bool zap_huge_pmd(struct mmu_gather *tlb folio = normal_or_softleaf_folio_pmd(vma, addr, orig_pmd, is_present); has_deposit = vma_has_deposited_pgtable(vma); if (folio) - zap_huge_pmd_folio(mm, vma, orig_pmd, folio, is_present); + zap_huge_pmd_folio(tlb, vma, orig_pmd, folio, is_present, + &force_flush); if (has_deposit) zap_deposited_table(mm, pmd); spin_unlock(ptl); - if (is_present && folio) - tlb_remove_page_size(tlb, &folio->page, HPAGE_PMD_SIZE); + if (force_flush) + tlb_flush_mmu(tlb); return true; } --- a/mm/memory.c~mm-huge_memory-flush-dirty-file-pmds-before-removing-rmap +++ a/mm/memory.c @@ -1740,7 +1740,7 @@ static __always_inline void zap_present_ ptent = get_and_clear_full_ptes(mm, addr, pte, nr, tlb->fullmm); if (pte_dirty(ptent)) { folio_mark_dirty(folio); - if (tlb_delay_rmap(tlb)) { + if (tlb_can_delay_rmap()) { delay_rmap = true; *force_flush = true; } --- a/mm/mmu_gather.c~mm-huge_memory-flush-dirty-file-pmds-before-removing-rmap +++ a/mm/mmu_gather.c @@ -171,6 +171,8 @@ static bool __tlb_remove_folio_pages_siz struct mmu_gather_batch *batch; VM_BUG_ON(!tlb->end); + if (delay_rmap) + tlb->delayed_rmap = 1; #ifdef CONFIG_MMU_GATHER_PAGE_SIZE VM_WARN_ON(tlb->page_size != page_size); _ Patches currently in -mm which might be from david@kernel.org are maintainers-memory-management-mglru-multi-gen-lru-update.patch maintainers-remove-myself-from-memory-management-reclaim.patch mm-mprotect-dont-skip-uffd-marker-install-when-racing-with-pte-table-reclaim.patch mm-khugepaged-free-unused-page-table-when-setting-a-huge-pmd-fails.patch