From: "Vlastimil Babka (SUSE)" <vbabka@kernel.org>
To: Harry Yoo <harry@kernel.org>, Andrew Morton <akpm@linux-foundation.org>
Cc: Pedro Falcato <pfalcato@suse.de>, Hao Li <hao.li@linux.dev>,
Shengming Hu <hu.shengming@zte.com.cn>,
Christoph Lameter <cl@gentwo.org>,
David Rientjes <rientjes@google.com>,
Roman Gushchin <roman.gushchin@linux.dev>,
linux-mm@kvack.org, linux-kernel@vger.kernel.org,
"Vlastimil Babka (SUSE)" <vbabka@kernel.org>
Subject: [PATCH slab/for-next v2] mm/slab: simplify freeing remote objects in free_to_pcs_bulk()
Date: Mon, 13 Jul 2026 09:52:30 +0200 [thread overview]
Message-ID: <20260713-bulk_free_remote-v2-1-24ee24771c2f@kernel.org> (raw)
We have been moving remote objects to an on-stack array and flushing it
when full. Instead, we can swap them towards the beginning of the
supplied array and bulk-free it just once.
Also add a comment to explain the rationale of freeing remote objects
last, because now it would appear to be simpler to free them first.
Reviewed-by: Pedro Falcato <pfalcato@suse.de>
Reviewed-by: Shengming Hu <hu.shengming@zte.com.cn>
Reviewed-by: Harry Yoo (Oracle) <harry@kernel.org>
Reviewed-by: Hao Li <hao.li@linux.dev>
Signed-off-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
---
Here's v2 with updated comment, will add to slab/for-next right away.
Thanks all!
One more cleanup I realized it's possible when processing feedback to
the previous one [1]
[1] https://lore.kernel.org/all/20260707-slab-simplify-bulk-pcs-v1-1-4850dbe0d904@kernel.org/
---
Changes in v2:
- Add comment about the partitioning, based on Shengming's suggestion.
- Add R-b tags, thanks!
- Link to v1: https://patch.msgid.link/20260710-bulk_free_remote-v1-1-a878d6dc52d5@kernel.org
---
mm/slub.c | 30 ++++++++++++++++++++----------
1 file changed, 20 insertions(+), 10 deletions(-)
diff --git a/mm/slub.c b/mm/slub.c
index 22045dc919ef..b650c1729437 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -6255,9 +6255,17 @@ static unsigned int __free_to_pcs_batch(struct kmem_cache *s, size_t size, void
static void free_to_pcs_bulk(struct kmem_cache *s, size_t size, void **p)
{
bool init = slab_want_init_on_free(s);
- void *remote_objects[PCS_BATCH_MAX];
+ void **remote_objects = p;
unsigned int remote_nr = 0;
+ /*
+ * Process the free hooks and separate out remote objects by
+ * partitioning the 'p' array in place:
+ *
+ * [0, remote_nr) - processed remote objects
+ * [remote_nr, i) - processed local objects
+ * [i, size) - unprocessed objects
+ */
for (unsigned int i = 0; i < size;) {
struct slab *slab = virt_to_slab(p[i]);
@@ -6270,19 +6278,17 @@ static void free_to_pcs_bulk(struct kmem_cache *s, size_t size, void **p)
}
if (unlikely(!can_free_to_pcs(slab))) {
- remote_objects[remote_nr] = p[i];
- p[i] = p[--size];
- if (++remote_nr >= PCS_BATCH_MAX) {
- __kmem_cache_free_bulk(s, remote_nr, &remote_objects[0]);
- stat_add(s, FREE_SLOWPATH, remote_nr);
- remote_nr = 0;
- }
- continue;
+ if (i != remote_nr)
+ swap(remote_objects[remote_nr], p[i]);
+ remote_nr++;
}
i++;
}
+ p += remote_nr;
+ size -= remote_nr;
+
while (size) {
unsigned int batch_freed = __free_to_pcs_batch(s, size, p);
@@ -6296,8 +6302,12 @@ static void free_to_pcs_bulk(struct kmem_cache *s, size_t size, void **p)
size -= batch_freed;
}
+ /*
+ * Processing remote objects last decreases the chances of cpu migration
+ * while freeing to sheaves and compromising object locality
+ */
if (remote_nr) {
- __kmem_cache_free_bulk(s, remote_nr, &remote_objects[0]);
+ __kmem_cache_free_bulk(s, remote_nr, remote_objects);
stat_add(s, FREE_SLOWPATH, remote_nr);
}
}
---
base-commit: e1fa26489025d6deac76d1dbfe2e0720a3ad84b1
change-id: 20260710-bulk_free_remote-7fbe6f3abf5d
reply other threads:[~2026-07-13 7:52 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260713-bulk_free_remote-v2-1-24ee24771c2f@kernel.org \
--to=vbabka@kernel.org \
--cc=akpm@linux-foundation.org \
--cc=cl@gentwo.org \
--cc=hao.li@linux.dev \
--cc=harry@kernel.org \
--cc=hu.shengming@zte.com.cn \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=pfalcato@suse.de \
--cc=rientjes@google.com \
--cc=roman.gushchin@linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox