From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC4B0C7EE30 for ; Mon, 30 Jun 2025 01:13:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4F65C6B0098; Sun, 29 Jun 2025 21:13:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4CE106B0099; Sun, 29 Jun 2025 21:13:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3E3D46B009A; Sun, 29 Jun 2025 21:13:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 26BD66B0098 for ; Sun, 29 Jun 2025 21:13:22 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id B41731602DF for ; Mon, 30 Jun 2025 01:13:21 +0000 (UTC) X-FDA: 83610293802.25.865B78D Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) by imf05.hostedemail.com (Postfix) with ESMTP id CA87A100002 for ; Mon, 30 Jun 2025 01:13:19 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="G/eluJV0"; spf=pass (imf05.hostedemail.com: domain of ioworker0@gmail.com designates 209.85.221.43 as permitted sender) smtp.mailfrom=ioworker0@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1751245999; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=qvYmFatUul6C95sdqfwhbwawXh1OZ/t57imv68gIbAs=; b=i9xT+H30BAS74701iuSEFNSN8gzlhYlpnbKOWyV/XrK3F4Bq27uNGORXJi0k2g7mEba72f 9MphczvgURg9lqfqxR11Jsji6sCa7SALVLJ0H62ZmnHxM6XrI4c7IeAtZ9bBdoCAid8mbC fbr44EX8cjKNyHMcz93DE4FsIUwNgAE= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="G/eluJV0"; spf=pass (imf05.hostedemail.com: domain of ioworker0@gmail.com designates 209.85.221.43 as permitted sender) smtp.mailfrom=ioworker0@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1751245999; a=rsa-sha256; cv=none; b=c3+eFqZ9BFFdtmpJkC85lJZASEWlIiW+t2ZzPuRE5snSDteGEZh8lVMeD28gsbEupmPBA9 xZ9KxChQaNZUaZzrMncGA1+lBJpbTce5Mvq/VuitPhuktmSC//4IS+h89BUZ6vFPF/jtXs v/v8hdjc9qYvYqSZeQ1zvZedvsOV+Ag= Received: by mail-wr1-f43.google.com with SMTP id ffacd0b85a97d-3a5123c1533so802545f8f.2 for ; Sun, 29 Jun 2025 18:13:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1751245998; x=1751850798; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=qvYmFatUul6C95sdqfwhbwawXh1OZ/t57imv68gIbAs=; b=G/eluJV0fESemgGv9wM+lxTEu/wgVzJYgEQH5CcoB+TqD2q9bhhejaWNBXG6gZwA2g MNivf6eAbFG5taKXAVhYdE9LV9+qk0t98m9A4UP99NOWAoDcyMcoMhL3cU8DbtlIw21C ZYVEeiU7VbIolrQ6y3cKGsH7OkHOsAbamb2X5SEH/ryImrSptgq4DXhqzX2PyDEKkZBH iM61Qqp+J7UOUYslmpDydQB6tWJcmWjjmfwQkxmjSSNHL9itI+LLClzeg04aLD0AtZyY NpDK5gETSwdx2YMqgnGEr4zJd306YxoiAa/vDJLIu+n3JO0OXT28QiTYIlE7dZOTgnfr akUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751245998; x=1751850798; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=qvYmFatUul6C95sdqfwhbwawXh1OZ/t57imv68gIbAs=; b=fjuvuYrEaZIcNAtspieIuy4tyEK/XeXW4Mlo/x0966gtbZfbXIecWerWjtXLu7ROI7 +fU19T/YyXwGVtbVbZjo+9AlCMePIyGLWwZ9ntuM4OS7KOTyAbUb1O+2bXakW1erytBE 4LW+Ed3bzce0WOPcuzdb5vEYNbfTB1luSxzsLQSe8iEdhmCAAoSnIUK3ToY9R/jHbWNB eVbq28CIOQuGIstvf32zTjU80mmGH0GX0Ju2NmUVcc3hToTpMo1ww+sYUq26VRvKHEEN HeL1vBvwzVLTMEKx3zXZjN6zVRMBM/YDNvj/lNQFX0//mX1y4iTUAnzCUx4HUxwR8+tm wLiQ== X-Forwarded-Encrypted: i=1; AJvYcCUlcCz7FKbF9ID43K/7cBm4/znre+u/NNO9w6/xDDOJhAnWClX+wII1cNtcLanyHnswdrJCtJFhoQ==@kvack.org X-Gm-Message-State: AOJu0YxVfoZnhuEKND/78x5tC9fxPzqMjmB2UO+/2+slzBPj/gYnrMDc 6bmmcRGH1JoVDZh+N3MYbPf904VX95YDB72+yMf2Fuq+qmIwZla3DrUZ X-Gm-Gg: ASbGncu+BljuYpEMHNnvCWjPp7TBT3cvsQuEjMJMSu+K/1YYSsqac5rfyIaTREingt9 dOUZjxwRu/eqy1QC4Gp2vHFKh03Rby5FvMExHWyP8k+0u4XOt093XQyWIIvam1uK/XnocOhh4Q2 18GKtZxrQDntU5k0uJR3bo9zlGe4Dy1zUgcuvXz5gG/wDw99diwmfcFEawNAb6ZwxqDWG1h68pD TA82ivFTYLX8A0uV3ER6Q4CHqaf6KWTrE/x5GnwYaz43XibJbLjvMaq7ocVj1n03bZ/DNB6LlTl 6yujmGgqphTQA8kAOwQYrHp4u3xZ02FHqfWLCN9LzDrSgfxk X-Google-Smtp-Source: AGHT+IG9vFE1ILgi8HDEneBoTPP9xiU1dYIZXp7wDUVsQw07A+lHFULl/cv0/NgbZly8KJ0xCHEmug== X-Received: by 2002:a05:6000:42c7:b0:3a4:e5bc:9892 with SMTP id ffacd0b85a97d-3a8f50cc97cmr7907228f8f.21.1751245997833; Sun, 29 Jun 2025 18:13:17 -0700 (PDT) Received: from EBJ9932692.tcent.cn ([2a09:0:1:2::302c]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a88c7e74fbsm9123735f8f.10.2025.06.29.18.13.11 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 29 Jun 2025 18:13:17 -0700 (PDT) From: Lance Yang X-Google-Original-From: Lance Yang To: akpm@linux-foundation.org, david@redhat.com, 21cnbao@gmail.com Cc: baolin.wang@linux.alibaba.com, chrisl@kernel.org, ioworker0@gmail.com, kasong@tencent.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, lorenzo.stoakes@oracle.com, ryan.roberts@arm.com, v-songbaohua@oppo.com, x86@kernel.org, huang.ying.caritas@gmail.com, zhengtangquan@oppo.com, riel@surriel.com, Liam.Howlett@oracle.com, vbabka@suse.cz, harry.yoo@oracle.com, mingzhe.yang@ly.com, stable@vger.kernel.org, Barry Song , Lance Yang Subject: [PATCH v3 1/1] mm/rmap: fix potential out-of-bounds page table access during batched unmap Date: Mon, 30 Jun 2025 09:13:05 +0800 Message-ID: <20250630011305.23754-1-lance.yang@linux.dev> X-Mailer: git-send-email 2.49.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Stat-Signature: ukgfir3m7wehjedrbsd4z9syac8us1mf X-Rspamd-Queue-Id: CA87A100002 X-Rspamd-Server: rspam11 X-Rspam-User: X-HE-Tag: 1751245999-439115 X-HE-Meta: U2FsdGVkX1+lJtKxlgVFmpWhp6Gyd8AiUIVXMwQQLIzJZiR3NczZO5FPmc8jIuhe1XO1/a7jgeUfXN5jnvjJqHmYrWdt/Wt66chEqO9VJJAGYixAXcMXVgwHsCyE7Sy69+wY1aWLSz/gYJaGTdNAfZf42GUvKxr3dPy1uev8XLJCDSvOko2Axpr2kJWR78feElbK/jSTAbj/xZheKG7llIFDCrmyhQeP65rGzozhHbiJWIv8ziw0vwDaVfURyao9ScruqNEs3M3kPDSufRIBPmVadkPPEU048EqXiRDs1wBt6pYib3uZdn3NzRCAEIdjsHmW3xZI9W+zj6Biz2v5nuwpw3ytAzWWm3PBLzNjgP1bT59s9xc2Mx+jqpCuTI3J2JM6QjewzzIf1GUhplxYHg+XMG6S1lrr4BDdFfxDXdzxn95eunvG2n611e05JX/2NznIk1HfwLo5WhxSquDV1egUx8BzJC/aXthP017+mebmP/WSjP/HGvqJauJKQDdBV2D0fKEQdVf+Zj9zMz/HB1GhX3/HqtTaHrseuXHlshWMDcNUOIHlB074saZvhtNNTUOYob3t6iYHS5+wherbEbn1G8GOzICEqkQrnsxl4AXF0IqnYt2auM38tMLfh1IBKusFjoQrPL+5rd0hndRQduR4vVV9LfX3MW1udyWYaIeFqwka9KX1c14SnLbtSloUBHf2XnHt0sn5b3wt+wr3auCQtMji7qmTrLxVjhJJ/E2507257uHWsKUuhUDgV+piwRXTQmLz7jdIOo0eq7ymcmeWY52u5sBKwxCQ0XRNuaVgPT1etQq0gwDP3Y/z8pGZenWpcs8LipMgDfH/0ReVHtLj0ssjcHKzvhzePLAe8OrjxV8SLJzhOPHo5qbRaG1KIT40XGu07g5T4kKdGo6GngRc5DTL1MKtHb0nMXr7jPkZyXhD9vNzJqw3ceek5VkvUqmma9VRlLVoRGfjok8 dFa6+7Iu 49B5eTh0rxGr8F/2oq9Hsvtnm7LBmOwNNfryCmJh3Y8+l9r6PrGsmN5beM0h9FPiSmeIJlzrd82RTxDusqodADqGek8cUcMDgsBTHGNekeCODHSR6WIAnyPPH2QPZfJVqNhrLarXpi21vm4/ybcmr1fgTGJ8J2ttoZhGxnPNvCvkK7umRvJyglEyMRenez9oICySUlvw76rwRbHQ8PBec5ZuOhIeg3MlarRsI271J26R2RpsSQPk2pI4tLcddoT3LPx8I8UYu1bRKzhQp/9NHm2vs/f8a2q9ctJaIfHhnTvSUC5Bk5Xbw7HH8NPqvjXjRnxs9+2+WQ9DvwJs08lznxxkxfeYqR3THd3XseZSowMhJzLczTReM0hJVTNg5hS2qAUUrMFBedTiIAIMOGKH5BRbSCdSB7u2X8xuyZP3u0+zorm4RBbe8iHd5rJAj/sN1jZbmznJ/lWZNxooJMeeb7TuCWx0JjAYJESdtelpPfMu/rSRPLUohBwtOHLn0mE1ZY3uNx77ilKWzN5aADIJ5eRJTBYnxboE/9GjYZJMHYfuS+qc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Lance Yang As pointed out by David[1], the batched unmap logic in try_to_unmap_one() may read past the end of a PTE table when a large folio's PTE mappings are not fully contained within a single page table. While this scenario might be rare, an issue triggerable from userspace must be fixed regardless of its likelihood. This patch fixes the out-of-bounds access by refactoring the logic into a new helper, folio_unmap_pte_batch(). The new helper correctly calculates the safe batch size by capping the scan at both the VMA and PMD boundaries. To simplify the code, it also supports partial batching (i.e., any number of pages from 1 up to the calculated safe maximum), as there is no strong reason to special-case for fully mapped folios. [1] https://lore.kernel.org/linux-mm/a694398c-9f03-4737-81b9-7e49c857fcbe@redhat.com Fixes: 354dffd29575 ("mm: support batched unmap for lazyfree large folios during reclamation") Cc: Acked-by: Barry Song Suggested-by: David Hildenbrand Suggested-by: Barry Song Signed-off-by: Lance Yang --- v2 -> v3: - Tweak changelog (per Barry and David) - Pick AB from Barry - thanks! - https://lore.kernel.org/linux-mm/20250627062319.84936-1-lance.yang@linux.dev v1 -> v2: - Update subject and changelog (per Barry) - https://lore.kernel.org/linux-mm/20250627025214.30887-1-lance.yang@linux.dev mm/rmap.c | 46 ++++++++++++++++++++++++++++------------------ 1 file changed, 28 insertions(+), 18 deletions(-) diff --git a/mm/rmap.c b/mm/rmap.c index fb63d9256f09..1320b88fab74 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1845,23 +1845,32 @@ void folio_remove_rmap_pud(struct folio *folio, struct page *page, #endif } -/* We support batch unmapping of PTEs for lazyfree large folios */ -static inline bool can_batch_unmap_folio_ptes(unsigned long addr, - struct folio *folio, pte_t *ptep) +static inline unsigned int folio_unmap_pte_batch(struct folio *folio, + struct page_vma_mapped_walk *pvmw, + enum ttu_flags flags, pte_t pte) { const fpb_t fpb_flags = FPB_IGNORE_DIRTY | FPB_IGNORE_SOFT_DIRTY; - int max_nr = folio_nr_pages(folio); - pte_t pte = ptep_get(ptep); + unsigned long end_addr, addr = pvmw->address; + struct vm_area_struct *vma = pvmw->vma; + unsigned int max_nr; + + if (flags & TTU_HWPOISON) + return 1; + if (!folio_test_large(folio)) + return 1; + /* We may only batch within a single VMA and a single page table. */ + end_addr = pmd_addr_end(addr, vma->vm_end); + max_nr = (end_addr - addr) >> PAGE_SHIFT; + + /* We only support lazyfree batching for now ... */ if (!folio_test_anon(folio) || folio_test_swapbacked(folio)) - return false; + return 1; if (pte_unused(pte)) - return false; - if (pte_pfn(pte) != folio_pfn(folio)) - return false; + return 1; - return folio_pte_batch(folio, addr, ptep, pte, max_nr, fpb_flags, NULL, - NULL, NULL) == max_nr; + return folio_pte_batch(folio, addr, pvmw->pte, pte, max_nr, fpb_flags, + NULL, NULL, NULL); } /* @@ -2024,9 +2033,7 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, if (pte_dirty(pteval)) folio_mark_dirty(folio); } else if (likely(pte_present(pteval))) { - if (folio_test_large(folio) && !(flags & TTU_HWPOISON) && - can_batch_unmap_folio_ptes(address, folio, pvmw.pte)) - nr_pages = folio_nr_pages(folio); + nr_pages = folio_unmap_pte_batch(folio, &pvmw, flags, pteval); end_addr = address + nr_pages * PAGE_SIZE; flush_cache_range(vma, address, end_addr); @@ -2206,13 +2213,16 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, hugetlb_remove_rmap(folio); } else { folio_remove_rmap_ptes(folio, subpage, nr_pages, vma); - folio_ref_sub(folio, nr_pages - 1); } if (vma->vm_flags & VM_LOCKED) mlock_drain_local(); - folio_put(folio); - /* We have already batched the entire folio */ - if (nr_pages > 1) + folio_put_refs(folio, nr_pages); + + /* + * If we are sure that we batched the entire folio and cleared + * all PTEs, we can just optimize and stop right here. + */ + if (nr_pages == folio_nr_pages(folio)) goto walk_done; continue; walk_abort: -- 2.49.0