From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 556E03BC696 for ; Fri, 10 Jul 2026 12:05:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783685108; cv=none; b=AnODN9TbI+H/0X/tauGYOmHN0dGkf2eI9xIhCrzy+sURPL208o49dQAoasSWDuuaSM4ItUOnVNADPH09id9f3peczyJmjkEBeRXKWgx65fk91ihHM45UMQmMHIDRw7cFUBD4R1DYZVjQTUdPDwAzI8XMbhCrFCXxG4Iw9rLApNk= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783685108; c=relaxed/simple; bh=c8kkW84kDMqfqSnjKF3/b+gWnbEd1vowS/0d27J+nqI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:To:Cc; b=Kp0A9ZUW9n4J6N4/Gj/ysRCDtRacf8HlyuR3ozOijM3XyS4jceyPHu2h+PxJqgYr8j1GC9AKZuMklonvi1gKP1tsdhAz0wUCuY81iL4jZdufBIbaq6BlhWUIt81iH3ObX/XP7oDd9EnJvoKP6mK07tnd50+93Wjz/HwSRxJC5yw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=jZEI1G/k; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="jZEI1G/k" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0B5831F000E9; Fri, 10 Jul 2026 12:05:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1783685107; bh=OcfNZwAiTSavg4XJXA5g1lvrBpQXdABBpHgtjVCRQn0=; h=From:Date:Subject:To:Cc; b=jZEI1G/kKg8gJA5Nys3pkg35kjwLDdfWPBKFf+6kkh5BeV+SU8CYmOs77ZS/TbbQk 6JpHqnwi7n5bcczPC2t0dpP/itIvCskl2eqV4rOYv9TZqQykLsdJ3DRVrPlmpsgkSI RMXz3XE7BXkJuil+W3HdpBuvR44mpz2aUiN69jp2ae2O+VhAeR9Q9JXPRMgDIc9fAX 3XMK0u0cjgwT5D7PSL50dBJ9UtY9JiHJgltDab6OzdFwpZYy+uXaPlL/BEAxiFfWdz txQ7vOUHgzlxKPCAQCzrXyj4uFXPlibx7ZwhB1/XTShbBWz0gHDpPlw4HkFiKRHHAU uA+OE7/UxqLIQ== From: "Vlastimil Babka (SUSE)" Date: Fri, 10 Jul 2026 14:04:58 +0200 Subject: [PATCH slab/for-next] mm/slab: simplify freeing remote objects in free_to_pcs_bulk() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20260710-bulk_free_remote-v1-1-a878d6dc52d5@kernel.org> X-B4-Tracking: v=1; b=H4sIAOnfUGoC/yWMUQrCMBAFr1L222BasQWvIlKy7VuN1kQ2qRRK7 27Uz2GYWSlBPRKdqpUUb598DAXqXUXDzYUrjB8LU2Ob1na1NTxPj14U6BXPmGE6YbRycCzHkUr 2UohffsszpcnxXqKagCXT5a/TzHcM+bulbfsAFFJTeoMAAAA= X-Change-ID: 20260710-bulk_free_remote-7fbe6f3abf5d To: Harry Yoo , Andrew Morton Cc: Hao Li , Shengming Hu , Christoph Lameter , David Rientjes , Roman Gushchin , linux-mm@kvack.org, linux-kernel@vger.kernel.org, "Vlastimil Babka (SUSE)" X-Mailer: b4 0.15.2 We have been moving remote objects to an on-stack array and flushing it when full. Instead, we can swap them towards the beginning of the supplied array and bulk-free it just once. Also add a comment to explain the rationale of freeing remote objects last, because now it would appear to be simpler to free them first. Signed-off-by: Vlastimil Babka (SUSE) --- One more cleanup I realized it's possible when processing feedback to the previous one [1] [1] https://lore.kernel.org/all/20260707-slab-simplify-bulk-pcs-v1-1-4850dbe0d904@kernel.org/ --- mm/slub.c | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index 22045dc919ef..966d2fb114e8 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -6255,7 +6255,7 @@ static unsigned int __free_to_pcs_batch(struct kmem_cache *s, size_t size, void static void free_to_pcs_bulk(struct kmem_cache *s, size_t size, void **p) { bool init = slab_want_init_on_free(s); - void *remote_objects[PCS_BATCH_MAX]; + void **remote_objects = p; unsigned int remote_nr = 0; for (unsigned int i = 0; i < size;) { @@ -6270,19 +6270,17 @@ static void free_to_pcs_bulk(struct kmem_cache *s, size_t size, void **p) } if (unlikely(!can_free_to_pcs(slab))) { - remote_objects[remote_nr] = p[i]; - p[i] = p[--size]; - if (++remote_nr >= PCS_BATCH_MAX) { - __kmem_cache_free_bulk(s, remote_nr, &remote_objects[0]); - stat_add(s, FREE_SLOWPATH, remote_nr); - remote_nr = 0; - } - continue; + if (i != remote_nr) + swap(remote_objects[remote_nr], p[i]); + remote_nr++; } i++; } + p += remote_nr; + size -= remote_nr; + while (size) { unsigned int batch_freed = __free_to_pcs_batch(s, size, p); @@ -6296,8 +6294,12 @@ static void free_to_pcs_bulk(struct kmem_cache *s, size_t size, void **p) size -= batch_freed; } + /* + * Processing remote objects last decreases the chances of cpu migration + * while freeing to sheaves and compromising object locality + */ if (remote_nr) { - __kmem_cache_free_bulk(s, remote_nr, &remote_objects[0]); + __kmem_cache_free_bulk(s, remote_nr, remote_objects); stat_add(s, FREE_SLOWPATH, remote_nr); } } --- base-commit: e1fa26489025d6deac76d1dbfe2e0720a3ad84b1 change-id: 20260710-bulk_free_remote-7fbe6f3abf5d Best regards to everyone but Pedro (btw this is not hard-coded, there's a config option for this), -- Vlastimil Babka (SUSE)