All of lore.kernel.org
 help / color / mirror / Atom feed
From: David Hildenbrand <david@redhat.com>
To: linux-kernel@vger.kernel.org
Cc: linux-mm@kvack.org, David Hildenbrand <david@redhat.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Matthew Wilcox <willy@infradead.org>,
	Ryan Roberts <ryan.roberts@arm.com>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Will Deacon <will@kernel.org>,
	"Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>,
	Nick Piggin <npiggin@gmail.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Michael Ellerman <mpe@ellerman.id.au>,
	Christophe Leroy <christophe.leroy@csgroup.eu>,
	"Naveen N. Rao" <naveen.n.rao@linux.ibm.com>,
	Heiko Carstens <hca@linux.ibm.com>,
	Vasily Gorbik <gor@linux.ibm.com>,
	Alexander Gordeev <agordeev@linux.ibm.com>,
	Christian Borntraeger <borntraeger@linux.ibm.com>,
	Sven Schnelle <svens@linux.ibm.com>,
	Arnd Bergmann <arnd@arndb.de>,
	linux-arch@vger.kernel.org, linuxppc-dev@lists.ozlabs.org,
	linux-s390@vger.kernel.org
Subject: [PATCH v1 2/9] mm/memory: handle !page case in zap_present_pte() separately
Date: Mon, 29 Jan 2024 15:32:14 +0100	[thread overview]
Message-ID: <20240129143221.263763-3-david@redhat.com> (raw)
In-Reply-To: <20240129143221.263763-1-david@redhat.com>

We don't need uptodate accessed/dirty bits, so in theory we could
replace ptep_get_and_clear_full() by an optimized ptep_clear_full()
function. Let's rely on the provided pte.

Further, there is no scenario where we would have to insert uffd-wp
markers when zapping something that is not a normal page (i.e., zeropage).
Add a sanity check to make sure this remains true.

should_zap_folio() no longer has to handle NULL pointers. This change
replaces 2/3 "!page/!folio" checks by a single "!page" one.

Signed-off-by: David Hildenbrand <david@redhat.com>
---
 mm/memory.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/mm/memory.c b/mm/memory.c
index 50a6c79c78fc..69502cdc0a7d 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -1497,10 +1497,6 @@ static inline bool should_zap_folio(struct zap_details *details,
 	if (should_zap_cows(details))
 		return true;
 
-	/* E.g. the caller passes NULL for the case of a zero folio */
-	if (!folio)
-		return true;
-
 	/* Otherwise we should only zap non-anon folios */
 	return !folio_test_anon(folio);
 }
@@ -1543,19 +1539,23 @@ static inline void zap_present_pte(struct mmu_gather *tlb,
 	struct page *page;
 
 	page = vm_normal_page(vma, addr, ptent);
-	if (page)
-		folio = page_folio(page);
+	if (!page) {
+		/* We don't need up-to-date accessed/dirty bits. */
+		ptep_get_and_clear_full(mm, addr, pte, tlb->fullmm);
+		arch_check_zapped_pte(vma, ptent);
+		tlb_remove_tlb_entry(tlb, pte, addr);
+		VM_WARN_ON_ONCE(userfaultfd_wp(vma));
+		ksm_might_unmap_zero_page(mm, ptent);
+		return;
+	}
 
+	folio = page_folio(page);
 	if (unlikely(!should_zap_folio(details, folio)))
 		return;
 	ptent = ptep_get_and_clear_full(mm, addr, pte, tlb->fullmm);
 	arch_check_zapped_pte(vma, ptent);
 	tlb_remove_tlb_entry(tlb, pte, addr);
 	zap_install_uffd_wp_if_needed(vma, addr, pte, details, ptent);
-	if (unlikely(!page)) {
-		ksm_might_unmap_zero_page(mm, ptent);
-		return;
-	}
 
 	if (!folio_test_anon(folio)) {
 		if (pte_dirty(ptent)) {
-- 
2.43.0


WARNING: multiple messages have this Message-ID (diff)
From: David Hildenbrand <david@redhat.com>
To: linux-kernel@vger.kernel.org
Cc: David Hildenbrand <david@redhat.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Catalin Marinas <catalin.marinas@arm.com>,
	linux-mm@kvack.org, Alexander Gordeev <agordeev@linux.ibm.com>,
	Will Deacon <will@kernel.org>,
	linux-arch@vger.kernel.org, linux-s390@vger.kernel.org,
	Vasily Gorbik <gor@linux.ibm.com>,
	Matthew Wilcox <willy@infradead.org>,
	"Naveen N. Rao" <naveen.n.rao@linux.ibm.com>,
	Christian Borntraeger <borntraeger@linux.ibm.com>,
	Ryan Roberts <ryan.roberts@arm.com>,
	Arnd Bergmann <arnd@arndb.de>, Heiko Carstens <hca@linux.ibm.com>,
	Nick Piggin <npiggin@gmail.com>,
	Sven Schnelle <svens@linux.ibm.com>,
	"Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	linuxppc-dev@lists.ozlabs.org
Subject: [PATCH v1 2/9] mm/memory: handle !page case in zap_present_pte() separately
Date: Mon, 29 Jan 2024 15:32:14 +0100	[thread overview]
Message-ID: <20240129143221.263763-3-david@redhat.com> (raw)
In-Reply-To: <20240129143221.263763-1-david@redhat.com>

We don't need uptodate accessed/dirty bits, so in theory we could
replace ptep_get_and_clear_full() by an optimized ptep_clear_full()
function. Let's rely on the provided pte.

Further, there is no scenario where we would have to insert uffd-wp
markers when zapping something that is not a normal page (i.e., zeropage).
Add a sanity check to make sure this remains true.

should_zap_folio() no longer has to handle NULL pointers. This change
replaces 2/3 "!page/!folio" checks by a single "!page" one.

Signed-off-by: David Hildenbrand <david@redhat.com>
---
 mm/memory.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/mm/memory.c b/mm/memory.c
index 50a6c79c78fc..69502cdc0a7d 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -1497,10 +1497,6 @@ static inline bool should_zap_folio(struct zap_details *details,
 	if (should_zap_cows(details))
 		return true;
 
-	/* E.g. the caller passes NULL for the case of a zero folio */
-	if (!folio)
-		return true;
-
 	/* Otherwise we should only zap non-anon folios */
 	return !folio_test_anon(folio);
 }
@@ -1543,19 +1539,23 @@ static inline void zap_present_pte(struct mmu_gather *tlb,
 	struct page *page;
 
 	page = vm_normal_page(vma, addr, ptent);
-	if (page)
-		folio = page_folio(page);
+	if (!page) {
+		/* We don't need up-to-date accessed/dirty bits. */
+		ptep_get_and_clear_full(mm, addr, pte, tlb->fullmm);
+		arch_check_zapped_pte(vma, ptent);
+		tlb_remove_tlb_entry(tlb, pte, addr);
+		VM_WARN_ON_ONCE(userfaultfd_wp(vma));
+		ksm_might_unmap_zero_page(mm, ptent);
+		return;
+	}
 
+	folio = page_folio(page);
 	if (unlikely(!should_zap_folio(details, folio)))
 		return;
 	ptent = ptep_get_and_clear_full(mm, addr, pte, tlb->fullmm);
 	arch_check_zapped_pte(vma, ptent);
 	tlb_remove_tlb_entry(tlb, pte, addr);
 	zap_install_uffd_wp_if_needed(vma, addr, pte, details, ptent);
-	if (unlikely(!page)) {
-		ksm_might_unmap_zero_page(mm, ptent);
-		return;
-	}
 
 	if (!folio_test_anon(folio)) {
 		if (pte_dirty(ptent)) {
-- 
2.43.0


  parent reply	other threads:[~2024-01-29 14:32 UTC|newest]

Thread overview: 80+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-01-29 14:32 [PATCH v1 0/9] mm/memory: optimize unmap/zap with PTE-mapped THP David Hildenbrand
2024-01-29 14:32 ` David Hildenbrand
2024-01-29 14:32 ` [PATCH v1 1/9] mm/memory: factor out zapping of present pte into zap_present_pte() David Hildenbrand
2024-01-29 14:32   ` David Hildenbrand
2024-01-30  8:13   ` Ryan Roberts
2024-01-30  8:13     ` Ryan Roberts
2024-01-30  8:41     ` David Hildenbrand
2024-01-30  8:41       ` David Hildenbrand
2024-01-30  8:46       ` Ryan Roberts
2024-01-30  8:46         ` Ryan Roberts
2024-01-30  8:49         ` David Hildenbrand
2024-01-30  8:49           ` David Hildenbrand
2024-01-29 14:32 ` David Hildenbrand [this message]
2024-01-29 14:32   ` [PATCH v1 2/9] mm/memory: handle !page case in zap_present_pte() separately David Hildenbrand
2024-01-30  8:20   ` Ryan Roberts
2024-01-30  8:20     ` Ryan Roberts
2024-01-29 14:32 ` [PATCH v1 3/9] mm/memory: further separate anon and pagecache folio handling in zap_present_pte() David Hildenbrand
2024-01-29 14:32   ` David Hildenbrand
2024-01-30  8:31   ` Ryan Roberts
2024-01-30  8:31     ` Ryan Roberts
2024-01-30  8:37     ` David Hildenbrand
2024-01-30  8:37       ` David Hildenbrand
2024-01-30  8:45       ` Ryan Roberts
2024-01-30  8:45         ` Ryan Roberts
2024-01-30  8:47         ` David Hildenbrand
2024-01-30  8:47           ` David Hildenbrand
2024-01-29 14:32 ` [PATCH v1 4/9] mm/memory: factor out zapping folio pte into zap_present_folio_pte() David Hildenbrand
2024-01-29 14:32   ` David Hildenbrand
2024-01-30  8:47   ` Ryan Roberts
2024-01-30  8:47     ` Ryan Roberts
2024-01-29 14:32 ` [PATCH v1 5/9] mm/mmu_gather: pass "delay_rmap" instead of encoded page to __tlb_remove_page_size() David Hildenbrand
2024-01-29 14:32   ` David Hildenbrand
2024-01-30  8:41   ` Ryan Roberts
2024-01-30  8:41     ` Ryan Roberts
2024-01-29 14:32 ` [PATCH v1 6/9] mm/mmu_gather: define ENCODED_PAGE_FLAG_DELAY_RMAP David Hildenbrand
2024-01-29 14:32   ` David Hildenbrand
2024-01-30  9:03   ` Ryan Roberts
2024-01-30  9:03     ` Ryan Roberts
2024-01-29 14:32 ` [PATCH v1 7/9] mm/mmu_gather: add __tlb_remove_folio_pages() David Hildenbrand
2024-01-29 14:32   ` David Hildenbrand
2024-01-30  9:21   ` Ryan Roberts
2024-01-30  9:21     ` Ryan Roberts
2024-01-30  9:33     ` David Hildenbrand
2024-01-30  9:33       ` David Hildenbrand
2024-01-29 14:32 ` [PATCH v1 8/9] mm/mmu_gather: add tlb_remove_tlb_entries() David Hildenbrand
2024-01-29 14:32   ` David Hildenbrand
2024-01-30  9:33   ` Ryan Roberts
2024-01-30  9:33     ` Ryan Roberts
2024-01-29 14:32 ` [PATCH v1 9/9] mm/memory: optimize unmap/zap with PTE-mapped THP David Hildenbrand
2024-01-29 14:32   ` David Hildenbrand
2024-01-30  9:08   ` David Hildenbrand
2024-01-30  9:08     ` David Hildenbrand
2024-01-30  9:48   ` Ryan Roberts
2024-01-30  9:48     ` Ryan Roberts
2024-01-31 10:21     ` David Hildenbrand
2024-01-31 10:21       ` David Hildenbrand
2024-01-31 10:31       ` Ryan Roberts
2024-01-31 10:31         ` Ryan Roberts
2024-01-31 11:13         ` David Hildenbrand
2024-01-31 11:13           ` David Hildenbrand
2024-01-31  2:30   ` Yin Fengwei
2024-01-31  2:30     ` Yin Fengwei
2024-01-31 10:30     ` David Hildenbrand
2024-01-31 10:30       ` David Hildenbrand
2024-01-31 10:43       ` Yin, Fengwei
2024-01-31 10:43         ` Yin, Fengwei
2024-01-31  2:20 ` [PATCH v1 0/9] " Yin Fengwei
2024-01-31  2:20   ` Yin Fengwei
2024-01-31 10:16   ` David Hildenbrand
2024-01-31 10:16     ` David Hildenbrand
2024-01-31 10:26     ` Ryan Roberts
2024-01-31 10:26       ` Ryan Roberts
2024-01-31 14:08       ` Michal Hocko
2024-01-31 14:08         ` Michal Hocko
2024-01-31 14:20         ` David Hildenbrand
2024-01-31 14:20           ` David Hildenbrand
2024-01-31 14:03     ` Michal Hocko
2024-01-31 14:03       ` Michal Hocko
2024-01-31 10:43   ` David Hildenbrand
2024-01-31 10:43     ` David Hildenbrand

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240129143221.263763-3-david@redhat.com \
    --to=david@redhat.com \
    --cc=agordeev@linux.ibm.com \
    --cc=akpm@linux-foundation.org \
    --cc=aneesh.kumar@linux.ibm.com \
    --cc=arnd@arndb.de \
    --cc=borntraeger@linux.ibm.com \
    --cc=catalin.marinas@arm.com \
    --cc=christophe.leroy@csgroup.eu \
    --cc=gor@linux.ibm.com \
    --cc=hca@linux.ibm.com \
    --cc=linux-arch@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=linux-s390@vger.kernel.org \
    --cc=linuxppc-dev@lists.ozlabs.org \
    --cc=mpe@ellerman.id.au \
    --cc=naveen.n.rao@linux.ibm.com \
    --cc=npiggin@gmail.com \
    --cc=peterz@infradead.org \
    --cc=ryan.roberts@arm.com \
    --cc=svens@linux.ibm.com \
    --cc=will@kernel.org \
    --cc=willy@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.