From: David Hildenbrand <david@redhat.com>
To: linux-kernel@vger.kernel.org
Cc: linux-mm@kvack.org, David Hildenbrand <david@redhat.com>,
Andrew Morton <akpm@linux-foundation.org>,
"Matthew Wilcox (Oracle)" <willy@infradead.org>,
Hugh Dickins <hughd@google.com>,
Ryan Roberts <ryan.roberts@arm.com>,
Yin Fengwei <fengwei.yin@intel.com>,
Mike Kravetz <mike.kravetz@oracle.com>,
Muchun Song <muchun.song@linux.dev>, Peter Xu <peterx@redhat.com>
Subject: [PATCH RFC 33/39] mm/rmap: convert page_dup_file_rmap() to folio_dup_file_rmap_[pte|ptes|pmd]()
Date: Mon, 4 Dec 2023 15:21:40 +0100 [thread overview]
Message-ID: <20231204142146.91437-34-david@redhat.com> (raw)
In-Reply-To: <20231204142146.91437-1-david@redhat.com>
Let's convert page_dup_file_rmap() like the other rmap functions. As there
is only a single caller, convert that single caller right away and remove
page_dup_file_rmap().
Add folio_dup_file_rmap_ptes() right away, we want to perform rmap
baching during fork() soon.
Signed-off-by: David Hildenbrand <david@redhat.com>
---
include/linux/rmap.h | 56 ++++++++++++++++++++++++++++++++++++++++----
mm/memory.c | 2 +-
2 files changed, 52 insertions(+), 6 deletions(-)
diff --git a/include/linux/rmap.h b/include/linux/rmap.h
index bdbfb11638e4f..21d72cc602adc 100644
--- a/include/linux/rmap.h
+++ b/include/linux/rmap.h
@@ -303,6 +303,57 @@ static inline void hugetlb_remove_rmap(struct folio *folio)
atomic_dec(&folio->_entire_mapcount);
}
+static inline void __folio_dup_rmap(struct folio *folio, struct page *page,
+ unsigned int nr_pages, enum rmap_mode mode)
+{
+ __folio_rmap_sanity_checks(folio, page, nr_pages, mode);
+
+ if (likely(mode == RMAP_MODE_PTE)) {
+ do {
+ atomic_inc(&page->_mapcount);
+ } while (page++, --nr_pages > 0);
+ } else if (mode == RMAP_MODE_PMD) {
+ atomic_inc(&folio->_entire_mapcount);
+ }
+}
+
+/**
+ * folio_dup_file_rmap_ptes - duplicate PTE mappings of a page range of a folio
+ * @folio: The folio to duplicate the mappings of
+ * @page: The first page to duplicate the mappings of
+ * @nr_pages: The number of pages of which the mapping will be duplicated
+ *
+ * The page range of the folio is defined by [page, page + nr_pages)
+ *
+ * The caller needs to hold the page table lock.
+ */
+static inline void folio_dup_file_rmap_ptes(struct folio *folio,
+ struct page *page, unsigned int nr_pages)
+{
+ __folio_dup_rmap(folio, page, nr_pages, RMAP_MODE_PTE);
+}
+#define folio_dup_file_rmap_pte(folio, page) \
+ folio_dup_file_rmap_ptes(folio, page, 1)
+
+/**
+ * folio_dup_file_rmap_pmd - duplicate a PMD mapping of a page range of a folio
+ * @folio: The folio to duplicate the mapping of
+ * @page: The first page to duplicate the mapping of
+ *
+ * The page range of the folio is defined by [page, page + HPAGE_PMD_NR)
+ *
+ * The caller needs to hold the page table lock.
+ */
+static inline void folio_dup_file_rmap_pmd(struct folio *folio,
+ struct page *page)
+{
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+ __folio_dup_rmap(folio, page, HPAGE_PMD_NR, RMAP_MODE_PTE);
+#else
+ WARN_ON_ONCE(true);
+#endif
+}
+
static inline void __page_dup_rmap(struct page *page, bool compound)
{
VM_WARN_ON(folio_test_hugetlb(page_folio(page)));
@@ -317,11 +368,6 @@ static inline void __page_dup_rmap(struct page *page, bool compound)
}
}
-static inline void page_dup_file_rmap(struct page *page, bool compound)
-{
- __page_dup_rmap(page, compound);
-}
-
/**
* page_try_dup_anon_rmap - try duplicating a mapping of an already mapped
* anonymous page
diff --git a/mm/memory.c b/mm/memory.c
index 8c4f98bb617fa..eaab6a2e14eba 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -961,7 +961,7 @@ copy_present_pte(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma,
rss[MM_ANONPAGES]++;
} else if (page) {
folio_get(folio);
- page_dup_file_rmap(page, false);
+ folio_dup_file_rmap_pte(folio, page);
rss[mm_counter_file(page)]++;
}
--
2.41.0
next prev parent reply other threads:[~2023-12-04 14:23 UTC|newest]
Thread overview: 82+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-12-04 14:21 [PATCH RFC 00/39] mm/rmap: interface overhaul David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 01/39] mm/rmap: rename hugepage_add* to hugetlb_add* David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 02/39] mm/rmap: introduce and use hugetlb_remove_rmap() David Hildenbrand
2023-12-06 1:22 ` Yin Fengwei
2023-12-06 12:11 ` David Hildenbrand
2023-12-07 0:56 ` Yin Fengwei
2023-12-04 14:21 ` [PATCH RFC 03/39] mm/rmap: introduce and use hugetlb_add_file_rmap() David Hildenbrand
2023-12-06 1:22 ` Yin Fengwei
2023-12-04 14:21 ` [PATCH RFC 04/39] mm/rmap: introduce and use hugetlb_try_dup_anon_rmap() David Hildenbrand
2023-12-06 1:22 ` Yin Fengwei
2023-12-04 14:21 ` [PATCH RFC 05/39] mm/rmap: introduce and use hugetlb_try_share_anon_rmap() David Hildenbrand
2023-12-06 1:23 ` Yin Fengwei
2023-12-04 14:21 ` [PATCH RFC 06/39] mm/rmap: add hugetlb sanity checks David Hildenbrand
2023-12-06 1:23 ` Yin Fengwei
2023-12-04 14:21 ` [PATCH RFC 07/39] mm/rmap: convert folio_add_file_rmap_range() into folio_add_file_rmap_[pte|ptes|pmd]() David Hildenbrand
2023-12-05 12:04 ` Ryan Roberts
2023-12-05 12:25 ` David Hildenbrand
2023-12-06 1:30 ` Yin Fengwei
2023-12-06 9:17 ` David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 08/39] mm/memory: page_add_file_rmap() -> folio_add_file_rmap_[pte|pmd]() David Hildenbrand
2023-12-08 1:40 ` Yin, Fengwei
2023-12-04 14:21 ` [PATCH RFC 09/39] mm/huge_memory: page_add_file_rmap() -> folio_add_file_rmap_pmd() David Hildenbrand
2023-12-08 1:41 ` Yin, Fengwei
2023-12-04 14:21 ` [PATCH RFC 10/39] mm/migrate: page_add_file_rmap() -> folio_add_file_rmap_pte() David Hildenbrand
2023-12-08 1:42 ` Yin, Fengwei
2023-12-04 14:21 ` [PATCH RFC 11/39] mm/userfaultfd: " David Hildenbrand
2023-12-08 1:42 ` Yin, Fengwei
2023-12-04 14:21 ` [PATCH RFC 12/39] mm/rmap: remove page_add_file_rmap() David Hildenbrand
2023-12-08 1:42 ` Yin, Fengwei
2023-12-04 14:21 ` [PATCH RFC 13/39] mm/rmap: factor out adding folio mappings into __folio_add_rmap() David Hildenbrand
2023-12-08 1:44 ` Yin, Fengwei
2023-12-04 14:21 ` [PATCH RFC 14/39] mm/rmap: introduce folio_add_anon_rmap_[pte|ptes|pmd]() David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 15/39] mm/huge_memory: batch rmap operations in __split_huge_pmd_locked() David Hildenbrand
2023-12-05 12:22 ` Ryan Roberts
2023-12-05 12:26 ` David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 16/39] mm/huge_memory: page_add_anon_rmap() -> folio_add_anon_rmap_pmd() David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 17/39] mm/migrate: page_add_anon_rmap() -> folio_add_anon_rmap_pte() David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 18/39] mm/ksm: " David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 19/39] mm/swapfile: " David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 20/39] mm/memory: " David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 21/39] mm/rmap: remove page_add_anon_rmap() David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 22/39] mm/rmap: remove RMAP_COMPOUND David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 23/39] mm/rmap: introduce folio_remove_rmap_[pte|ptes|pmd]() David Hildenbrand
2023-12-05 12:52 ` Ryan Roberts
2023-12-05 13:09 ` David Hildenbrand
2023-12-05 13:37 ` Ryan Roberts
2023-12-04 14:21 ` [PATCH RFC 24/39] kernel/events/uprobes: page_remove_rmap() -> folio_remove_rmap_pte() David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 25/39] mm/huge_memory: page_remove_rmap() -> folio_remove_rmap_pmd() David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 26/39] mm/khugepaged: page_remove_rmap() -> folio_remove_rmap_pte() David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 27/39] mm/ksm: " David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 28/39] mm/memory: " David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 29/39] mm/migrate_device: " David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 30/39] mm/rmap: " David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 31/39] Documentation: stop referring to page_remove_rmap() David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 32/39] mm/rmap: remove page_remove_rmap() David Hildenbrand
2023-12-04 14:21 ` David Hildenbrand [this message]
2023-12-04 14:21 ` [PATCH RFC 34/39] mm/rmap: introduce folio_try_dup_anon_rmap_[pte|ptes|pmd]() David Hildenbrand
2023-12-04 17:59 ` David Hildenbrand
2023-12-05 13:11 ` David Hildenbrand
2023-12-05 13:12 ` Ryan Roberts
2023-12-05 13:17 ` David Hildenbrand
2023-12-05 13:18 ` David Hildenbrand
2023-12-05 13:40 ` Ryan Roberts
2023-12-05 13:50 ` David Hildenbrand
2023-12-05 14:02 ` Ryan Roberts
2023-12-05 14:12 ` David Hildenbrand
2023-12-05 13:32 ` David Hildenbrand
2023-12-05 13:40 ` Ryan Roberts
2023-12-04 14:21 ` [PATCH RFC 35/39] mm/huge_memory: page_try_dup_anon_rmap() -> folio_try_dup_anon_rmap_pmd() David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 36/39] mm/memory: page_try_dup_anon_rmap() -> folio_try_dup_anon_rmap_pte() David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 37/39] mm/rmap: remove page_try_dup_anon_rmap() David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 38/39] mm: convert page_try_share_anon_rmap() to folio_try_share_anon_rmap_[pte|pmd]() David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 39/39] mm/rmap: rename COMPOUND_MAPPED to ENTIRELY_MAPPED David Hildenbrand
2023-12-04 19:53 ` [PATCH RFC 00/39] mm/rmap: interface overhaul Ryan Roberts
2023-12-05 9:56 ` David Hildenbrand
2023-12-05 13:31 ` Ryan Roberts
2023-12-05 13:39 ` David Hildenbrand
2023-12-05 13:49 ` Ryan Roberts
2023-12-05 13:55 ` David Hildenbrand
2023-12-08 11:24 ` David Hildenbrand
2023-12-08 11:38 ` Ryan Roberts
2023-12-05 12:29 ` Ryan Roberts
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20231204142146.91437-34-david@redhat.com \
--to=david@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=fengwei.yin@intel.com \
--cc=hughd@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mike.kravetz@oracle.com \
--cc=muchun.song@linux.dev \
--cc=peterx@redhat.com \
--cc=ryan.roberts@arm.com \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.