From: Shakeel Butt <shakeel.butt@linux.dev>
To: Tejun Heo <tj@kernel.org>, Andrew Morton <akpm@linux-foundation.org>
Cc: Johannes Weiner <hannes@cmpxchg.org>,
Michal Koutny <mkoutny@suse.com>,
Christian Brauner <brauner@kernel.org>,
Michal Hocko <mhocko@kernel.org>,
Roman Gushchin <roman.gushchin@linux.dev>,
Muchun Song <muchun.song@linux.dev>, Jan Kara <jack@suse.cz>,
Matthew Wilcox <willy@infradead.org>,
David Hildenbrand <david@kernel.org>,
Vlastimil Babka <vbabka@kernel.org>, Zi Yan <ziy@nvidia.com>,
Baolin Wang <baolin.wang@linux.alibaba.com>,
Nico Pache <nico.pache@linux.dev>,
Usama Arif <usama.arif@linux.dev>,
Kiryl Shutsemau <kas@kernel.org>,
Lorenzo Stoakes <ljs@kernel.org>,
Suren Baghdasaryan <surenb@google.com>,
Meta kernel team <kernel-team@meta.com>,
cgroups@vger.kernel.org, linux-mm@kvack.org,
linux-fsdevel@vger.kernel.org, linux-btrfs@vger.kernel.org,
linux-kernel@vger.kernel.org
Subject: [RFC PATCH 2/7] mm/gup: handle memory.high overage while populating
Date: Sat, 21 Nov 2026 18:47:32 -0800 [thread overview]
Message-ID: <20261122024737.2741569-3-shakeel.butt@linux.dev> (raw)
In-Reply-To: <20261122024737.2741569-1-shakeel.butt@linux.dev>
mlock(), mlockall(), mmap() with MAP_POPULATE or MAP_LOCKED, and
MADV_POPULATE_READ/WRITE can fault in a lot of memory in one call,
with mmap_lock held. When their charges can block, the charge path
throttles them for memory.high with mmap_lock still held. In a VM with
THP on, mlock() of an 80M MADV_HUGEPAGE range under memory.high=64M
made another thread wait up to 17.2s to mmap(), and a task in another
cgroup wait up to 15.3s to read the process's /proc/<pid>/cmdline.
THP charges in VMAs without MADV_HUGEPAGE can't block under the
default defrag setting, so they are not throttled at all: mlock(512M)
reached 513M.
Throttle these calls with mmap_lock dropped instead. The populate
loops, __mm_populate() and madvise_populate(), now fault in at most
2MiB per GUP call. Between calls, if more than a batch of overage is
pending, they drop mmap_lock, handle it, and go on. GUP itself doesn't
change. One fault can still charge a whole folio.
With this patch, mlock(512M) stays at 84M, and populating 2G with no
limit takes as long as before. The mmap_lock stall goes away when a
later patch removes the synchronous step.
Signed-off-by: Shakeel Butt <shakeel.butt@linux.dev>
---
mm/gup.c | 9 +++++++--
mm/internal.h | 27 +++++++++++++++++++++++++++
mm/madvise.c | 11 +++++++++--
3 files changed, 43 insertions(+), 4 deletions(-)
diff --git a/mm/gup.c b/mm/gup.c
index 7605da0e06d8..d93d89f6aad0 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -1951,6 +1951,7 @@ int __mm_populate(unsigned long start, unsigned long len, int ignore_errors)
end = start + len;
for (nstart = start; nstart < end; nstart = nend) {
+ populate_handle_over_high(mm, &locked);
/*
* We want to fault in pages for [nstart; end) address range.
* Find first corresponding VMA.
@@ -1976,9 +1977,13 @@ int __mm_populate(unsigned long start, unsigned long len, int ignore_errors)
/*
* Now fault in a range of pages. populate_vma_page_range()
* double checks the vma flags, so that it won't mlock pages
- * if the vma was already munlocked.
+ * if the vma was already munlocked. Do one chunk at a time,
+ * see POPULATE_CHUNK_SIZE. On error, nend skips the rest of
+ * the VMA.
*/
- ret = populate_vma_page_range(vma, nstart, nend, &locked);
+ ret = populate_vma_page_range(vma, nstart,
+ populate_chunk_end(nstart, nend),
+ &locked);
if (ret < 0) {
if (ignore_errors) {
ret = 0;
diff --git a/mm/internal.h b/mm/internal.h
index de4593b3e3b6..12f7b4b1254f 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -948,6 +948,33 @@ extern long populate_vma_page_range(struct vm_area_struct *vma,
unsigned long start, unsigned long end, int *locked);
extern long faultin_page_range(struct mm_struct *mm, unsigned long start,
unsigned long end, bool write, int *locked);
+
+/*
+ * __mm_populate() and madvise_populate() fault in at most
+ * POPULATE_CHUNK_SIZE at a time, and handle memory.high overage between
+ * chunks with mmap_lock dropped, so other tasks don't wait for them.
+ */
+#define POPULATE_CHUNK_SIZE SZ_2M
+
+static inline unsigned long populate_chunk_end(unsigned long start,
+ unsigned long end)
+{
+ return min(end, round_down(start, POPULATE_CHUNK_SIZE) +
+ POPULATE_CHUNK_SIZE);
+}
+
+static inline void populate_handle_over_high(struct mm_struct *mm,
+ int *locked)
+{
+ if (!mem_cgroup_over_high_pending())
+ return;
+ if (*locked) {
+ mmap_read_unlock(mm);
+ *locked = 0;
+ }
+ mem_cgroup_handle_over_high(GFP_KERNEL);
+}
+
bool mlock_future_ok(const struct mm_struct *mm, bool is_vma_locked,
unsigned long bytes);
diff --git a/mm/madvise.c b/mm/madvise.c
index 2d54cb9863ea..c77924e52d79 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -992,8 +992,15 @@ static long madvise_populate(struct madvise_behavior *madv_behavior)
long pages;
while (start < end) {
- /* Populate (prefault) page tables readable/writable. */
- pages = faultin_page_range(mm, start, end, write, &locked);
+ /*
+ * Populate (prefault) page tables readable/writable, at most a
+ * chunk at a time, see POPULATE_CHUNK_SIZE.
+ */
+ pages = faultin_page_range(mm, start,
+ populate_chunk_end(start, end),
+ write, &locked);
+ if (pages >= 0)
+ populate_handle_over_high(mm, &locked);
if (!locked) {
mmap_read_lock(mm);
locked = 1;
--
2.53.1-Meta
next prev parent reply other threads:[~2026-11-22 2:47 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-22 2:47 [RFC PATCH 0/7] memcg: effective high limit that avoids isolation issues Shakeel Butt
2026-11-22 2:47 ` [RFC PATCH 1/7] memcg: add helpers to handle memory.high overage in long operations Shakeel Butt
2026-11-22 2:47 ` Shakeel Butt [this message]
2026-11-22 2:47 ` [RFC PATCH 3/7] mm, fs: handle memory.high overage in buffered reads and writes Shakeel Butt
2026-11-22 2:47 ` [RFC PATCH 4/7] mm/shmem: handle memory.high overage in fallocate and reads Shakeel Butt
2026-11-22 2:47 ` [RFC PATCH 5/7] mm/khugepaged: handle memory.high overage in MADV_COLLAPSE Shakeel Butt
2026-11-22 2:47 ` [RFC PATCH 6/7] memcg: remove synchronous memory.high enforcement Shakeel Butt
2026-11-22 2:47 ` [RFC PATCH 7/7] selftests/cgroup: make test_memcg_high_sync check that throttling holds Shakeel Butt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261122024737.2741569-3-shakeel.butt@linux.dev \
--to=shakeel.butt@linux.dev \
--cc=akpm@linux-foundation.org \
--cc=baolin.wang@linux.alibaba.com \
--cc=brauner@kernel.org \
--cc=cgroups@vger.kernel.org \
--cc=david@kernel.org \
--cc=hannes@cmpxchg.org \
--cc=jack@suse.cz \
--cc=kas@kernel.org \
--cc=kernel-team@meta.com \
--cc=linux-btrfs@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=ljs@kernel.org \
--cc=mhocko@kernel.org \
--cc=mkoutny@suse.com \
--cc=muchun.song@linux.dev \
--cc=nico.pache@linux.dev \
--cc=roman.gushchin@linux.dev \
--cc=surenb@google.com \
--cc=tj@kernel.org \
--cc=usama.arif@linux.dev \
--cc=vbabka@kernel.org \
--cc=willy@infradead.org \
--cc=ziy@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox