From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=100.103.45.18 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1794585505; b=igdfaX4giOSb666dHNZ03GYMmO5myuoB8yqrrKt+PvuxP+MUvvfc1aHuvmGo9TBVP1kC z2FHmAOW7FS9XzRhEHJAAWa8pkBK4zRfTdJdGiN+8Queexrt+pxAsspQid5ffv3HrIk2H 53VJ7WJIMzWJCD9ODNHCygwXXgjPITOudVf6bGmWpKdjmwN6CjtHKRZINaohZ46A+b5mP 5E94sLWmvx3uphXUrCfCuDa51u4WmxgN8zhXJQm/J+Xg5wogIHUryLTA7vvCRpi9CIqnY egzS4mV2NBtb6hvqEDqCyVTKCKRMkJJFv99MYv1GhDZ3XjGG1tOFbX9TJu14Q6A81IA== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1794585505; h=Received:DKIM-Signature:From:To:Cc:Subject:Date:Message-ID:X-Mailer: In-Reply-To:References:User-Agent:X-stable:X-Patchwork-Hint: MIME-Version:Content-Type:Content-Transfer-Encoding; bh=Ew1FzFbAxLdnjI4+7rckYwTvX3i+DB5k2qyw9zztEMY=; b=FSwH6mcTvf4VCWn3LgHFkMko721Y+IwPZVJnYvmEd96h56zeZWWxDcnk6RzXH/Y6Zt8I EyMf0mT7fky5mE44yXxI+HCb71GHZq3+XskGGyJkVm4SHC3XdUnBISBbimdI1JSOr+aXI 4iSgSx+Bznw2In0rhoXY38aj25+nZ/wa71Z5wzmF6x6cTyi1UYUOz5i/xoQ9eLvs4aoTW n0x4NuDZVZPL1DrrOiTWi81cjQfp8meiPUXAqx/7mI6ecZw4xDcimxTQQ421es6JZV5hX R2OfVxzZtSDZvSFIXYpcd/6ic+aA0BL8hghhBqQktVGFN7Y5976Wti2EujTd2HbaDew== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass header.d=linuxfoundation.org header.i=@linuxfoundation.org header.a=rsa-sha256 header.s=korg header.b=fM7+sVGD; arc=none smtp.remote-ip=100.103.45.18 Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EAC34606AE0B; Fri, 13 Nov 2026 15:58:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.a=rsa-sha256 header.s=korg header.b=fM7+sVGD Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9A8D11F000C4; Fri, 13 Nov 2026 15:58:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=korg; t=1794585504; bh=Ew1FzFbAxLdnjI4+7rckYwTvX3i+DB5k2qyw9zztEMY=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=fM7+sVGDUOJ5HDWSDvXY1du8hlPKEluLV+ZwfQM+ceieMOSduPW3YrczlFf716Z49 +wwKDtlL7Zw7rJHbq4h+dlF7or+o8/DCYV8qs9U8ba8Q50lDKPT3OsrTGNL9miD8VE 5TqO2yZfIP2uhqFgPq6bOP6ATIOTwMZ8ZiPJ4uuk= From: Greg Kroah-Hartman To: stable@vger.kernel.org Cc: Greg Kroah-Hartman , patches@lists.linux.dev, "Lorenzo Stoakes (ARM)" , Josef Bacik , "Harry Yoo (Meta)" , Barry Song , Dev Jain , Muhammad Usama Anjum , Baolin Wang , "David Hildenbrand (Arm)" , Sasha Levin Subject: [PATCH 7.3 153/507] mm/rmap: handle device-exclusive entries in folio_referenced_one() Date: Fri, 13 Nov 2026 10:47:37 -0500 Message-ID: <20261113153759.717119090@linuxfoundation.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261113153753.358973689@linuxfoundation.org> References: <20261113153753.358973689@linuxfoundation.org> User-Agent: quilt/0.69 X-stable: review X-Patchwork-Hint: ignore X-Mailing-List: stable@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 7.3-stable review patch. If anyone has any objections, please let me know. ------------------ From: David Hildenbrand (Arm) [ Upstream commit f3c493d2f42b5f44a5c134d8d7ea7f960caf5e2c ] page_vma_mapped_walk() can return non-present device-exclusive entries, but folio_referenced_one() passes them to PTE young-bit accessors that require present PTEs. We must neither call clear_flush_young_ptes_notify() nor folio_pte_batch() with non-present entries. For example, clearing the young bit on non-present PTEs can corrupt the entry: on arm64 PTE_AF overlaps the swap type, so clearing it turns a device-exclusive entry into an ordinary swap entry. folio_pte_batch() will VM_WARN_ON_FOLIO. Apply the same approach as the previous fixes to e.g., damon_ptep_mkold(): treat these entries as old from the CPU's perspective and obtain their young state through the MMU notifier. Skip PTE batching and MGLRU look-around for non-present entries, and don't modify non-present PTEs. Fixes: b756a3b5e7ea ("mm: device exclusive memory access") Reviewed-by: Lorenzo Stoakes (ARM) Cc: stable@vger.kernel.org Reported-by: Josef Bacik Reviewed-by: Harry Yoo (Meta) Reviewed-by: Barry Song Reviewed-by: Dev Jain Tested-by: Muhammad Usama Anjum Link: https://patch.msgid.link/20261031-fixes_core-v3-9-d6437171ecc3@kernel.org Reviewed-by: Baolin Wang Signed-off-by: David Hildenbrand (Arm) Signed-off-by: Sasha Levin --- mm/rmap.c | 23 +++++++++++++++++++---- 1 file changed, 19 insertions(+), 4 deletions(-) diff --git a/mm/rmap.c b/mm/rmap.c index 6b3480330b924..9ffd3e833e082 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -926,6 +926,7 @@ static bool folio_referenced_one(struct folio *folio, DEFINE_FOLIO_VMA_WALK(pvmw, folio, vma, address, 0); int ptes = 0, referenced = 0; unsigned int nr; + pte_t pteval; while (page_vma_mapped_walk(&pvmw)) { address = pvmw.address; @@ -970,10 +971,12 @@ static bool folio_referenced_one(struct folio *folio, return false; } - if (pvmw.pte && folio_test_large(folio)) { + if (pvmw.pte) + pteval = ptep_get(pvmw.pte); + + if (pvmw.pte && pte_present(pteval) && folio_test_large(folio)) { const unsigned long end_addr = pmd_addr_end(address, vma->vm_end); const unsigned int max_nr = (end_addr - address) >> PAGE_SHIFT; - pte_t pteval = ptep_get(pvmw.pte); nr = folio_pte_batch(folio, pvmw.pte, pteval, max_nr); } @@ -983,11 +986,23 @@ static bool folio_referenced_one(struct folio *folio, * are. —they could be on active/inactive lists or on MGLRU. So the * simplest approach is to disable this look-around optimization. */ - if (lru_gen_enabled() && !lru_gen_switching() && pvmw.pte) { + if (lru_gen_enabled() && !lru_gen_switching() && pvmw.pte && + pte_present(pteval)) { if (lru_gen_look_around(&pvmw, nr)) referenced++; } else if (pvmw.pte) { - if (clear_flush_young_ptes_notify(vma, address, pvmw.pte, nr)) + bool young = false; + + /* + * PFN swap PTEs, such as device-exclusive ones, that + * actually map pages are "old" from a CPU perspective. + * The MMU notifier takes care of any device aspects. + */ + if (pte_present(pteval)) + young = clear_flush_young_ptes(vma, address, pvmw.pte, nr); + young |= mmu_notifier_clear_flush_young(vma->vm_mm, address, + address + nr * PAGE_SIZE); + if (young) referenced++; } else if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) { if (pmdp_clear_flush_young_notify(vma, address, -- 2.53.0