From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=100.103.45.18 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1793439787; b=ulsaGWxYRf3kprQ4ltftb7B3fwaVJb8PUST9XHtt0YXl6ksCYf8OndJtHhotWQP+s6oL jokvf162Me65X5LUhGx/jiwgMcO83UvIgYzCKOmxTSxyzAWZdMRoMk//szclHUsCDfQZP P39DKBPwx2JAlLPW/d2Cu/mYAkPGFd19z5g61i/lF1QarFT2zqZNfCX5fzVD/YbVPQiuk ihpFIkoOXCEz5OrW6hWFUXeJu4XXHQtXNanOaYJRxRSatMgDGpUpUHTerztbqovW257+X JvuFXP420O0DdpHc9BC/4ovUzvlqaCY7uzFZrSzB9uT+vrSM+Sn/JucCjnpMsklxuOQ== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1793439787; h=Received:DKIM-Signature:From:Date:Subject:MIME-Version:Content-Type: Content-Transfer-Encoding:Message-Id:References:In-Reply-To:To:Cc: X-Mailer:X-Developer-Signature:X-Developer-Key; bh=c8H3MH6BWdT2aUP2bxmCyTqdb38oNJ2FfQ3UjfLBqqg=; b=WkjjX19822xxFlF3C0JS2j24x9jCbYG/Fq1AKr549G0xg5swsqnnr3m/y+8W7lb4gwd9 Eay7Gpp6k3ewnEPImpiITpz03OoH7eAxWCvI5KfzQ9rqgjkybAZBKvMCyBeU4px1n2BYh KrKMMHv2KWYDgt3f+rRqnm68mx2xqgDoe4go1CsTQon51YvpPCStjIWgfmB//Ur7yjmnv CvA+wQx6miVDBwx9GEirBk5Me3V49WXA5Psr5MMVBTPpBNujWeFOVSgPMfZnj620PWj4m 4BeCFV6KgqnANWeqHgGz4lj0OSHa2Fqbu+R7JJptqjwhsJi09nOLDYeVK9SLQeMxM3Q== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20260515 header.b=XKxku93c; arc=none smtp.remote-ip=100.103.45.18 Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EF059606A31F; Sat, 31 Oct 2026 09:43:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key, unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20260515 header.b=XKxku93c Received: by smtp.kernel.org (Postfix) with ESMTPSA id CABA11F000C1; Sat, 31 Oct 2026 09:42:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1793439786; bh=c8H3MH6BWdT2aUP2bxmCyTqdb38oNJ2FfQ3UjfLBqqg=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=XKxku93c8qqYSbfQDsRSTB03j2MAGwx1Lqs5hUQsI3c4sVWK1n2y4ro23Yhs3yX0D G2R5gX3bGIgaNFqk4rGiAAmeHR1A8Z22O+UZuVKxuVnQidVfOkRKkUnmjOE8w8OlnF w1WTiEptpTJlrZYoFv3Q3eElYjggt1zyAdckWVOrBUKj2UlqZxm4oBG0QRp1rSjtGi sU9HajefmO9Iv/63c/2VyNxCNchdSIuMtMG3TOhjdJU37i7o+MDbZKFoEPwF2gOlOc X/J80Khhp+hD6DjBpYJ2pNYgGRlwj2tKhcPYnPihqvrA0RgtumI9uuNZ6B+nKWGFUx 6fqVpNTsWFxFw== From: "David Hildenbrand (Arm)" Date: Sat, 31 Oct 2026 10:39:40 +0100 Subject: [PATCH v3 11/47] mm/huge_memory: properly handle refcounted PUD mappings on fork X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20261031-fixes_core-v3-11-d6437171ecc3@kernel.org> References: <20261031-fixes_core-v3-0-d6437171ecc3@kernel.org> In-Reply-To: <20261031-fixes_core-v3-0-d6437171ecc3@kernel.org> To: Andrew Morton , Johannes Weiner , Michal Hocko , Qi Zheng , Shakeel Butt , Lorenzo Stoakes , Kairui Song , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Baoquan He , Baolin Wang , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Kees Cook , Peter Zijlstra , Jann Horn , Pedro Falcato , Hugh Dickins , Pasha Tatashin , Pratyush Yadav , "Matthew Wilcox (Oracle)" , Jan Kara , Zi Yan , Lance Yang , Rik van Riel , Harry Yoo , Alistair Popple , Nico Pache , Ryan Roberts , Dev Jain , Usama Arif , Kiryl Shutsemau , Matthew Brost , John Hubbard , Peter Xu , Will Deacon , "Aneesh Kumar K.V" , Nick Piggin , Oscar Salvador , Minchan Kim , Muchun Song , Xu Xin , Chengming Zhou , Chris Li , Kemeng Shi , Nhat Pham , Youngjun Park , Gregory Price , Joshua Hahn , Rakie Kim , Ying Huang , Jason Gunthorpe Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, kexec@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-cxl@vger.kernel.org, stable@vger.kernel.org, Muhammad Usama Anjum , "David Hildenbrand (Arm)" X-Mailer: b4 0.16.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4524; i=david@kernel.org; h=from:subject:message-id; bh=AisWB7KHkrv1Z9AhryNka/aHvDbvx4kG++9kdVXIDVU=; b=owGbwMvMwCF2/OVpCdfQ3ZWMp9WSGLKebk97dqHknGJAm3OY3urrucx5lxhlNjFHMJi8PiD30 u7Nh8dbOkpZGMQ4GGTFFFmCgu8bvHy5rrbajY8ZZg4rE8gQBi5OAZjIuqcM/0w3N+1dlv165Zx2 iU/tEsKH3CJmJ+jNZjBRkNtx1V41+Dsjw8KA7INaO1NsZGN4qjhjlld/fBpkvov/y/Ss39s7n6X VswIA X-Developer-Key: i=david@kernel.org; a=openpgp; fpr=1BD9CAAD735C4C3A460DFCCA4DDE10F700FF835A copy_huge_pud() copies device-DAX entries without taking a folio reference or updating mapcounts and the child's RSS. For example, MADV_GUARD_INSTALL sets VM_MAYBE_GUARD on a device-DAX VMA, forcing copy_page_range() to traverse the whole VMA on fork. Any remaining huge PUDs are then copied without accounting for the new mappings. Unmapping them in the child decrements accounting that was never added. Use vm_normal_page_pud() to identify refcounted mappings. Take a folio reference and add the child's PUD rmap and RSS accounting before installing the copied entry. Pass src and dst VMAs consistently instead of src and dst MM. Move the TODO. Fixes: aed877c2b4257 ("device/dax: properly refcount device dax pages when mapping") Cc: stable@vger.kernel.org Reviewed-by: Zi Yan Reviewed-by: Pedro Falcato Reviewed-by: Lance Yang Tested-by: Muhammad Usama Anjum Signed-off-by: David Hildenbrand (Arm) --- include/linux/huge_mm.h | 5 ++--- mm/huge_memory.c | 27 +++++++++++++++++++-------- mm/memory.c | 4 +--- 3 files changed, 22 insertions(+), 14 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index c745f7ad22987..fac05120ab774 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -12,9 +12,8 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, pmd_t *dst_pmd, pmd_t *src_pmd, unsigned long addr, struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma); bool huge_pmd_set_accessed(struct vm_fault *vmf); -int copy_huge_pud(struct mm_struct *dst_mm, struct mm_struct *src_mm, - pud_t *dst_pud, pud_t *src_pud, unsigned long addr, - struct vm_area_struct *vma); +int copy_huge_pud(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, + pud_t *dst_pud, pud_t *src_pud, unsigned long addr); #ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD void huge_pud_set_accessed(struct vm_fault *vmf, pud_t orig_pud); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 4c0194af9016d..283c4bc8b927c 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2072,11 +2072,13 @@ void touch_pud(struct vm_area_struct *vma, unsigned long addr, update_mmu_cache_pud(vma, addr, pud); } -int copy_huge_pud(struct mm_struct *dst_mm, struct mm_struct *src_mm, - pud_t *dst_pud, pud_t *src_pud, unsigned long addr, - struct vm_area_struct *vma) +int copy_huge_pud(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, + pud_t *dst_pud, pud_t *src_pud, unsigned long addr) { + struct mm_struct *dst_mm = dst_vma->vm_mm; + struct mm_struct *src_mm = src_vma->vm_mm; spinlock_t *dst_ptl, *src_ptl; + struct page *page; pud_t pud; int ret; @@ -2089,11 +2091,20 @@ int copy_huge_pud(struct mm_struct *dst_mm, struct mm_struct *src_mm, if (unlikely(!pud_trans_huge(pud))) goto out_unlock; - /* - * TODO: once we support anonymous pages, use - * folio_try_dup_anon_rmap_*() and split if duplicating fails. - */ - if (vma_is_cow_mapping(vma) && pud_write(pud)) { + page = vm_normal_page_pud(src_vma, addr, pud); + if (page) { + struct folio *folio = page_folio(page); + + folio_get(folio); + /* + * TODO: once we support anonymous pages, use + * folio_try_dup_anon_rmap_*() and split if duplicating fails. + */ + folio_add_file_rmap_pud(folio, page, dst_vma); + add_mm_counter(dst_mm, mm_counter_file(folio), HPAGE_PUD_NR); + } + + if (vma_is_cow_mapping(src_vma) && pud_write(pud)) { pudp_set_wrprotect(src_mm, addr, src_pud); pud = pud_wrprotect(pud); } diff --git a/mm/memory.c b/mm/memory.c index 0fb676dd2bd63..5392bcb1d24d6 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1463,7 +1463,6 @@ copy_pud_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, unsigned long end) { struct mm_struct *dst_mm = dst_vma->vm_mm; - struct mm_struct *src_mm = src_vma->vm_mm; pud_t *src_pud, *dst_pud; unsigned long next; @@ -1477,8 +1476,7 @@ copy_pud_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, int err; VM_BUG_ON_VMA(next-addr != HPAGE_PUD_SIZE, src_vma); - err = copy_huge_pud(dst_mm, src_mm, - dst_pud, src_pud, addr, src_vma); + err = copy_huge_pud(dst_vma, src_vma, dst_pud, src_pud, addr); if (err == -ENOMEM) return -ENOMEM; if (!err) -- 2.43.0