From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from out-183.mta0.migadu.com (out-183.mta0.migadu.com [91.218.175.183]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 52A7540627B for ; Wed, 1 Apr 2026 12:26:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.183 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775046382; cv=none; b=rOXOlBem91tYfDz38cGMByESi1SI+lv3LYVhQJIdCh2+Rdo7oS0dtoGFXZQDZZnW/I7s96+sW3vTIWketrYIqhJoFHy5+WJ6xYdtpLz9En6EtD32xFz6zxfbzVvCevXy34QY5n8hQyvBe03p/8I1sr9+GUV89j58GLDSk6uPVts= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775046382; c=relaxed/simple; bh=CGMBUCxewdfii0r275bpztDmU7H6wrTH4UvTpQOEpOU=; h=MIME-Version:Date:Content-Type:From:Message-ID:Subject:To:Cc: In-Reply-To:References; b=Uj4XxJU6brYKeyKbE4fWZ35zjvqilB9Y/Oi4IDFfGFrA/VNCSroWcdW8l6/xRRSzLB2GI52qODFZXLCSvZfX6bH77SqNF306yE0xDXnz5Uo0UASV1+3aQQtNP7A18DOcG9HFD3h8qaL0doUxAdqirkmkHOhFCr2ylK4EhIF+mIQ= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=DuCPEUoQ; arc=none smtp.client-ip=91.218.175.183 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="DuCPEUoQ" Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1775046376; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9RRpsgUrZWaL+QC1rauEtM+0dsvtlEKa3fb+EjeI+Vg=; b=DuCPEUoQ585nScG8yHQ7EYzzMTFYAosos72EJa5eXCTRxZSXHExP4bnU5tW90oMuKkp5Ay WYLplWHm7RjosHRA1bSoehBUqKAZxQ93bUXPDx/tKg91C2lL77mO3+unvHyY9jKVxmeofP vVr5tOOeue8gXsBtv4fOj9imFr5VfTY= Date: Wed, 01 Apr 2026 12:26:15 +0000 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: "teawater" Message-ID: TLS-Required: No Subject: Re: [PATCH mm-stable v3] mm/memcontrol: batch memcg charging in __memcg_slab_post_alloc_hook To: "Harry Yoo (Oracle)" , "Shakeel Butt" Cc: "Johannes Weiner" , "Michal Hocko" , "Roman Gushchin" , "Muchun Song" , "Andrew Morton" , cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, "Hui Zhu" , "Vlastimil Babka" , "Hao Li" In-Reply-To: References: <20260331091707.226786-1-hui.zhu@linux.dev> X-Migadu-Flow: FLOW_OUT >=20 >=20On Tue, Mar 31, 2026 at 08:32:30AM -0700, Shakeel Butt wrote: >=20 >=20>=20 >=20> On Tue, Mar 31, 2026 at 05:17:07PM +0800, Hui Zhu wrote: > > From: Hui Zhu > >=20=20 >=20> When kmem_cache_alloc_bulk() allocates multiple objects, the post-= alloc > > hook __memcg_slab_post_alloc_hook() previously charged memcg one obj= ect > > at a time, even though consecutive objects may reside on slabs backe= d by > > the same pgdat node. > >=20=20 >=20> Batch the memcg charging by scanning ahead from the current positi= on to > > find a contiguous run of objects whose slabs share the same pgdat, t= hen > > issue a single __obj_cgroup_charge() / __consume_obj_stock() call fo= r > > the entire run. The per-object obj_ext assignment loop is preserved = as-is > > since it cannot be further collapsed. > >=20=20 >=20> This implements the TODO comment left in commit bc730030f956 ("mem= cg: > > combine slab obj stock charging and accounting"). > >=20=20 >=20> The existing error-recovery contract is unchanged: if size =3D=3D = 1 then > > memcg_alloc_abort_single() will free the sole object, and for larger > > bulk allocations kmem_cache_free_bulk() will uncharge any objects th= at > > were already charged before the failure. > >=20=20 >=20> Benchmark using kmem_cache_alloc_bulk() with SLAB_ACCOUNT > > (iters=3D100000): > >=20=20 >=20> bulk=3D32 before: 215 ns/object after: 174 ns/object (-19%) > > bulk=3D1 before: 344 ns/object after: 335 ns/object ( ~) > >=20=20 >=20> No measurable regression for bulk=3D1, as expected. > >=20=20 >=20> Signed-off-by: Hui Zhu > >=20=20 >=20> Do we have an actual user of kmem_cache_alloc_bulk(GFP_ACCOUNT) in= kernel? > >=20 Hi=20Harry and Shakeel, > Apparently we have a SLAB_ACCOUNT user in io_uring.c. > (perhaps it's the only user?) Looks like __io_alloc_req_refill is only user that call kmem_cache_alloc_= bulk with SLAB_ACCOUNT. I am working on make a benchmark code for it. Best, Hui >=20 >=20>=20 >=20> If yes, can you please benchmark that usage? Otherwise can we pleas= e wait for > > an actual user before adding more complexity? Or you can look for op= portunities > > for kmem_cache_alloc_bulk(GFP_ACCOUNT) users and add the optimizatio= n along with > > the user. > >=20 >=20Good point. I was also wondering what are use cases benefiting > from this beyond the microbenchmark. >=20 >=20>=20 >=20> Have you looked at the bulk free side? I think we already have rcu = freeing in > > bulk as a user. Did you find any opportunities in optimizing the > > __memcg_slab_free_hook() from bulk free? > >=20 >=20Probably a bit out of scope but one thing to note on slab side: > kfree_bulk() (called by kfree_rcu batching) doesn't specify slab cache, > and it builds a detached freelist which contains objects from the same = slab. >=20 >=20On the other hand kmem_cache_free_bulk() with non-NULL slab cache > simply calls free_to_pcs_bulk() and it passes objects one by one to > __memcg_slab_free_hook() since objects may not come from the same slab. >=20 >=20Now that we have sheaves enabled for (almost) all slab caches, it mig= ht > be worth revisiting - e.g. sort objects by slab cache and > pass them to free_to_pcs_bulk() instead of building a detached freelist= . >=20 >=20And let __memcg_slab_free_hook() handle objects from the same cache b= ut > from different slabs. >=20 >=20--=20 >=20Cheers, > Harry / Hyeonggon >