From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=100.103.45.18 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1794630948; b=SoGIIpYyJHTW7Q61pDgz2hDKiqZHZW+XVCnpZC65cemE/F66HxQ+RyADU2vG3eET8KT7 9uwfJBV8lZ4zhj2kAizx0GtITqqMx9r6og6rbDLiu7J79LWC0/GzzCSH3Qzf3UHMTNs6X MCrNF77a8/58wGePRoQIemFzB0+U1cbjUMXUDBS6Ujw04JysFXym8iEstEdJygezMaigK 6ZP2R9ssB5cf3zMhl1Ko+2sgLv8pWq7QKUqXaMykFNvuPir9XM68rrbpPKDT7Goaq3RAr 4TrzqoAbcyq32VROU+hGWZ6bfUUxG1TVCLLsird9+Lit3IY7/ujTBtw4WiCDyNRNOjQ== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1794630948; h=Received:DKIM-Signature:From:To:Cc:Subject:Date:Message-ID:X-Mailer: In-Reply-To:References:MIME-Version:Content-Type: Content-Transfer-Encoding; bh=E6fhcbzWKQl+zdclcQl/mdwgoSrlkRYFktNvxwIlIuI=; b=dLqXRMF3ds7i6LOuVvmkgWQ77paYtU2TeGn4zkrTGPWwaAUC4kpm6nXBr7I9LcnakMH9 QB+adw0ly/rN6Tqodxx6B1aIr4R/cZnFzhd/AEG3hweuxKAhltrPHFCLi/iGCTp2iPUp+ AuAdmgm+eXlGGibnWT3PrQC03qjgciD/2hmMC2xLn04uXK3f2EPcEaLfWmgNiTyAdKxo5 0Ct0rogXIxUqQJLAXdIZSrcZXM+SMvyifsqKgjAiCON3JIRAGfutbj1Rj8/FdR/56zETg IA2VGLA/4JbRq9BFvGqANbrsduG0OQZFJnwuWwtN0/HxoBgXfqphyrZa9MQHTIEMVNQ== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20260515 header.b=NLZeJvaW; arc=none smtp.remote-ip=100.103.45.18 Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 65B7D606A327 for ; Sat, 14 Nov 2026 04:35:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20260515 header.b=NLZeJvaW Received: by smtp.kernel.org (Postfix) with ESMTPSA id 146FB1F000C4; Sat, 14 Nov 2026 04:35:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1794630948; bh=E6fhcbzWKQl+zdclcQl/mdwgoSrlkRYFktNvxwIlIuI=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=NLZeJvaWCU13F+k5Dk+R8SPICkZo+0OLosc7jMv5RjUKdO8XAg7QhBvTivQKUAIr5 /BqNdd8o+h8hi0WW4cwWZZFhQKQJvB/9g2bW1qoxDiOAGlA9bxqp8UxoVmKTP578zx zz+3gsuShYaH0RT4QjvF9f9N32lwMNrq3zHUkqmgD5dEyoI9MZXwY+usGgAEW69Rqa g428k4fQmUHohCF/xwQkFkRpA0SsnmeQpOHhA/A09RCmWiO5uNmlALNP51RuGZRwbm 7oyVtsObaMv/eSbEEJ24SqckSSM/poCC86sTGFO0lrVlVBWz8C4mb89CUPmGErZY/b N+r33Ak6WCGFA== From: Sasha Levin To: stable@vger.kernel.org Cc: Li Zhe , "David Hildenbrand (Arm)" , Andrew Morton , Muchun Song , Oscar Salvador , Peter Xu , Mike Rapoport , "Jose A. Perez de Azpillaga" , Sasha Levin Subject: [PATCH 6.12.y] mm/hugetlb: fix mremap deadlock when PMD sharing is possible Date: Fri, 13 Nov 2026 23:35:45 -0500 Message-ID: <20261113211301.2628513-1-sashal@kernel.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20261109200749-failed-6.12-92daa8652f88@kernel.org> References: <20261109200749-failed-6.12-92daa8652f88@kernel.org> X-Mailing-List: stable@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit From: Li Zhe [ Upstream commit 92daa8652f88c8cfc05f4899a8408a04b02f211e ] move_hugetlb_page_tables() holds the mapping's i_mmap_rwsem for write while moving hugetlb PTEs. This prevents races with truncation and protects huge_pmd_unshare(), but the move loop must not call helpers that can acquire the same rwsem for read. huge_pte_alloc() can do that when the destination can use PMD sharing: it may call huge_pmd_share(), which takes i_mmap_rwsem for read while searching for a shareable PMD table. Calling it while the move path already holds i_mmap_rwsem for write can deadlock. Avoid that path for PMD-sized hugetlb mappings that can use PMD sharing. Instead of allocating or sharing the destination page table while moving, zap the source range and let the destination fault path populate or share the PMD lazily. Reuse the hugetlb unmap logic for the source-side zap by splitting the core loop out of __unmap_hugepage_range(). The wrapper keeps the existing tlb_start_vma(), tlb_end_vma() and huge_pmd_unshare_flush() behavior, while move_hugetlb_page_tables() can reuse the unmap loop with its existing mmu_gather. Link: https://lore.kernel.org/20261022034543.51226-4-lizhe.67@bytedance.com Fixes: 550a7d60bd5e ("mm, hugepages: add mremap() support for hugepage backed vma") Signed-off-by: Li Zhe Co-developed-by: David Hildenbrand (Arm) Signed-off-by: David Hildenbrand (Arm) Signed-off-by: Andrew Morton Cc: Muchun Song Cc: Oscar Salvador Cc: Peter Xu Cc: Mike Rapoport Cc: Jose A. Perez de Azpillaga Cc: [ sashal: adapted the unmap helper’s folio parameter to the existing struct page ref_page interface ] Signed-off-by: Sasha Levin --- mm/hugetlb.c | 47 +++++++++++++++++++++++++++++++++++++---------- 1 file changed, 37 insertions(+), 10 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index e6f913f227e6c..fcdb4f2e4c599 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -92,6 +92,10 @@ static int __huge_pmd_unshare(struct mmu_gather *tlb, bool check_locks); static void hugetlb_unshare_pmds(struct vm_area_struct *vma, unsigned long start, unsigned long end, bool take_locks); +static void hugetlb_unmap_range(struct mmu_gather *tlb, + struct vm_area_struct *vma, unsigned long start, + unsigned long end, struct page *ref_page, + zap_flags_t zap_flags); static struct resv_map *vma_resv_map(struct vm_area_struct *vma); static void hugetlb_free_folio(struct folio *folio) @@ -5540,6 +5544,8 @@ unsigned long move_hugetlb_page_tables(struct vm_area_struct *vma, for (; old_addr < old_end; old_addr += sz, new_addr += sz) { const unsigned long offset_to_last_entry = (old_addr | last_addr_mask) - old_addr; + const unsigned long offset_to_last_dst_entry = + (new_addr | last_addr_mask) - new_addr; src_pte = hugetlb_walk(vma, old_addr, sz); if (!src_pte) { @@ -5556,6 +5562,24 @@ unsigned long move_hugetlb_page_tables(struct vm_area_struct *vma, continue; } + /* + * Avoid entering the destination PMD sharing path while holding + * i_mmap_rwsem for write. Leave destination page table + * population to the fault path, which can share the PMD lazily. + */ + if (sz == PMD_SIZE && want_pmd_share(new_vma, new_addr)) { + const unsigned long unmap_size = + min(old_end - old_addr, + offset_to_last_dst_entry + sz); + + hugetlb_unmap_range(&tlb, vma, old_addr, + old_addr + unmap_size, NULL, + ZAP_FLAG_DROP_MARKER); + old_addr += unmap_size - sz; + new_addr += unmap_size - sz; + continue; + } + dst_pte = huge_pte_alloc(mm, new_vma, new_addr, sz); if (!dst_pte) break; @@ -5575,9 +5599,10 @@ unsigned long move_hugetlb_page_tables(struct vm_area_struct *vma, return len + old_addr - old_end; } -void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma, - unsigned long start, unsigned long end, - struct page *ref_page, zap_flags_t zap_flags) +static void hugetlb_unmap_range(struct mmu_gather *tlb, + struct vm_area_struct *vma, unsigned long start, + unsigned long end, struct page *ref_page, + zap_flags_t zap_flags) { struct mm_struct *mm = vma->vm_mm; unsigned long address; @@ -5595,13 +5620,6 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma, BUG_ON(start & ~huge_page_mask(h)); BUG_ON(end & ~huge_page_mask(h)); - /* - * This is a hugetlb vma, all the pte entries should point - * to huge page. - */ - tlb_change_page_size(tlb, sz); - tlb_start_vma(tlb, vma); - last_addr_mask = hugetlb_mask_last_page(h); address = start; for (; address < end; address += sz) { @@ -5728,6 +5746,15 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma, if (ref_page) break; } +} + +void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma, + unsigned long start, unsigned long end, + struct page *ref_page, zap_flags_t zap_flags) +{ + tlb_change_page_size(tlb, huge_page_size(hstate_vma(vma))); + tlb_start_vma(tlb, vma); + hugetlb_unmap_range(tlb, vma, start, end, ref_page, zap_flags); tlb_end_vma(tlb, vma); huge_pmd_unshare_flush(tlb, vma); -- 2.53.0