From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-qv1-f51.google.com (mail-qv1-f51.google.com [209.85.219.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2081E4301DA for ; Tue, 3 Mar 2026 15:43:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.51 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772552619; cv=none; b=Q+DQyUadTT20AY3HMXGFHDok6dBVq2hl/PoVgeZJ301tYZBf+t6I+9o5AWCE+uAKru5/aloVYKcS4VJVtA3FYfld+D1K8ptetC3r6Fd08z9br9xwvvm4oFVSYSlepSshQYGbPlrypRqr8nz8DcocCby4hnwgpMO7BDGBEUjLHbA= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772552619; c=relaxed/simple; bh=WnhesxNW489SQ6o9/JVmLHwAJ+1lIx01AIUokx4m7x8=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=oI34B9FTxFa83sBp0uW8Q564BNly2YGpB8TWzfZZyi/vy17UHBaQsVJvfOUBpROt02NnV3b/FXlD77VziF3LKlRVcXqxYw27Rewnivu8YUUAqx49Ee82JyvzJ7MqQwSNYj890Hb8DbytZ6yQEi789qX7Woz2/4OqbcXkj5Frg88= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org; spf=pass smtp.mailfrom=cmpxchg.org; dkim=pass (2048-bit key) header.d=cmpxchg.org header.i=@cmpxchg.org header.b=cqQ4Dt4e; arc=none smtp.client-ip=209.85.219.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cmpxchg.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cmpxchg.org header.i=@cmpxchg.org header.b="cqQ4Dt4e" Received: by mail-qv1-f51.google.com with SMTP id 6a1803df08f44-899f5d337f7so26670486d6.0 for ; Tue, 03 Mar 2026 07:43:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg.org; s=google; t=1772552615; x=1773157415; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=5iJQaiUJrWfplRxtcMoHp1zPaYjGQn94g6/CCuRZvsM=; b=cqQ4Dt4ePHk8gYbSUkw+Es+PrNkl1xj2N0d/myQpYb0qzrR7hTOItklIZN8Z7akJze 8wyAlHcSoSKrl0vfL0CFeP05KoT3gfFRUuKn8RrMZyJ7cQYBt3B0thLKhF6IKOo0WABT PcgEy3eDiRn3h7H50orV0iOiYR+NI5Ihg0IIoixjR7ILM85Fd0koQ9x/6ExCupkY3GUf eyZdITDzGCqgYq8PSiMisnQ5AkjmEGMC/GjBXCAo4m3+Sw374tVYcrrYmGdNunKo5Cj1 3mdbRAnm35Ba87vzqbAJr53+Pt58MrVicEWdJzNbBwO6aijSTmARgZIaPbLBULXEPvav UiLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772552615; x=1773157415; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5iJQaiUJrWfplRxtcMoHp1zPaYjGQn94g6/CCuRZvsM=; b=gi0owuxh/X70jlQn8JUPDg11geTt+11n5vyDrzU82LA9PRhZenkP0/LpgnZ+enEICo VFZUsfJ0S6dg3V1soQw8M33xxHF0k2jYjy++TXB+JaCpG2PZ4ygfiRuktrTGKteYsLNv 92Lb1XdfGLQpQVYONFYZ6NSueDXhU1+65bGemMKc/A3IQ4nH2Xi49I2EsEsTBSi4q4R1 1gkVAARr7xORAOyXN+Rc2TlS2pjbknJt28hmLFcaXimSIw708GRCekuBddTiet5lwCJ7 xX4MydmGYg02AZu6Dg/+u8fdAanMHPnYUGOQceTrrpNyIjc906FSPQzGEYrRb8q+G98B kJNQ== X-Forwarded-Encrypted: i=1; AJvYcCWLmttEL5zpxQbcgBsZzDfUxcJHWShSJsThdHn1sq83UMmxP5zfmy5zN6Nz9kv1SFW6CuKJOjjeIng4PEI=@vger.kernel.org X-Gm-Message-State: AOJu0YyG9ElhXrJTl0dS7Z9R8DHNFqlYIbRX/5ZMTn1AnXNw2FAbpeoB Bz2S06GS3r2KLmWbMqydulKkPcz+I4xE65De9LBGe8hhDixGZjjPCbBrUv/jco2ZHmE= X-Gm-Gg: ATEYQzx2BU/7RGPWGvz/THtno5YeUGB2sEss1OyzHzlGNsCixyDFha1yDJ5R9+xjbFP zWxXDCeu1S64MZgsF1RL1r1a4XCFZ+ar/3x7Hbnvah443TshueeQk/5X1FO7m4UQSJ1jAvPBZNC gHNsrOzqQK1ttVCTpYGcF+eMYNdlNxWR5Y1TZ72fTsKX9oaCFKvPB5mosRJfdHRsk+pnK+o3eaJ MplUNPesAN8jQQ7oAnpYVt0od8m9rDRah9q+H61iScUJLNO/FrIP1DNtbL9qAGaEI/IgCWSAfue KslYCvA7nL8pKk6Y3/JVObvQRK+h289PGDHT8EnzePGO8mcsMsvaBb1WILUbqrfzIshcuhoxmgT BLoQ5n5LKavpAp3HLC7N9CYZgO+2ws2bb7/4GLI/bzJ0sEWBgrhHLpTqhjHvV94BsqLZaHPU3+I WzGv8FY9ZB60FeqNYplCQ+c6/X4ikGByjO X-Received: by 2002:a05:6214:21c7:b0:899:f39f:b884 with SMTP id 6a1803df08f44-89a0a86c9d5mr33626696d6.9.1772552614690; Tue, 03 Mar 2026 07:43:34 -0800 (PST) Received: from localhost ([2603:7000:c00:3a00:365a:60ff:fe62:ff29]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-899c7374e07sm134729676d6.33.2026.03.03.07.43.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 03 Mar 2026 07:43:33 -0800 (PST) Date: Tue, 3 Mar 2026 10:43:29 -0500 From: Johannes Weiner To: Shakeel Butt Cc: "Vlastimil Babka (SUSE)" , Hao Li , Andrew Morton , Michal Hocko , Roman Gushchin , Vlastimil Babka , Harry Yoo , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 5/5] mm: memcg: separate slab stat accounting from objcg charge cache Message-ID: References: <20260302195305.620713-1-hannes@cmpxchg.org> <20260302195305.620713-6-hannes@cmpxchg.org> <541a6661-7bfe-4517-a32c-5839002c61e5@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: On Tue, Mar 03, 2026 at 05:45:18AM -0800, Shakeel Butt wrote: > On Tue, Mar 03, 2026 at 11:42:31AM +0100, Vlastimil Babka (SUSE) wrote: > > On 3/3/26 09:54, Hao Li wrote: > > > On Mon, Mar 02, 2026 at 02:50:18PM -0500, Johannes Weiner wrote: > > >> Cgroup slab metrics are cached per-cpu the same way as the sub-page > > >> charge cache. However, the intertwined code to manage those dependent > > >> caches right now is quite difficult to follow. > > >> > > >> Specifically, cached slab stat updates occur in consume() if there was > > >> enough charge cache to satisfy the new object. If that fails, whole > > >> pages are reserved, and slab stats are updated when the remainder of > > >> those pages, after subtracting the size of the new slab object, are > > >> put into the charge cache. This already juggles a delicate mix of the > > >> object size, the page charge size, and the remainder to put into the > > >> byte cache. Doing slab accounting in this path as well is fragile, and > > >> has recently caused a bug where the input parameters between the two > > >> caches were mixed up. > > >> > > >> Refactor the consume() and refill() paths into unlocked and locked > > >> variants that only do charge caching. Then let the slab path manage > > >> its own lock section and open-code charging and accounting. > > >> > > >> This makes the slab stat cache subordinate to the charge cache: > > >> __refill_obj_stock() is called first to prepare it; > > >> __account_obj_stock() follows to hitch a ride. > > >> > > >> This results in a minor behavioral change: previously, a mismatching > > >> percpu stock would always be drained for the purpose of setting up > > >> slab account caching, even if there was no byte remainder to put into > > >> the charge cache. Now, the stock is left alone, and slab accounting > > >> takes the uncached path if there is a mismatch. This is exceedingly > > >> rare, and it was probably never worth draining the whole stock just to > > >> cache the slab stat update. > > >> > > >> Signed-off-by: Johannes Weiner > > >> --- > > >> mm/memcontrol.c | 100 +++++++++++++++++++++++++++++------------------- > > >> 1 file changed, 61 insertions(+), 39 deletions(-) > > >> > > >> diff --git a/mm/memcontrol.c b/mm/memcontrol.c > > >> index 4f12b75743d4..9c6f9849b717 100644 > > >> --- a/mm/memcontrol.c > > >> +++ b/mm/memcontrol.c > > >> @@ -3218,16 +3218,18 @@ static struct obj_stock_pcp *trylock_stock(void) > > >> > > > > > > [...] > > > > > >> @@ -3376,17 +3383,14 @@ static bool obj_stock_flush_required(struct obj_stock_pcp *stock, > > >> return flush; > > >> } > > >> > > >> -static void refill_obj_stock(struct obj_cgroup *objcg, unsigned int nr_bytes, > > >> - bool allow_uncharge, int nr_acct, struct pglist_data *pgdat, > > >> - enum node_stat_item idx) > > >> +static void __refill_obj_stock(struct obj_cgroup *objcg, > > >> + struct obj_stock_pcp *stock, > > >> + unsigned int nr_bytes, > > >> + bool allow_uncharge) > > >> { > > >> - struct obj_stock_pcp *stock; > > >> unsigned int nr_pages = 0; > > >> > > >> - stock = trylock_stock(); > > >> if (!stock) { > > >> - if (pgdat) > > >> - __account_obj_stock(objcg, NULL, nr_acct, pgdat, idx); > > >> nr_pages = nr_bytes >> PAGE_SHIFT; > > >> nr_bytes = nr_bytes & (PAGE_SIZE - 1); > > >> atomic_add(nr_bytes, &objcg->nr_charged_bytes); > > >> @@ -3404,20 +3408,25 @@ static void refill_obj_stock(struct obj_cgroup *objcg, unsigned int nr_bytes, > > >> } > > >> stock->nr_bytes += nr_bytes; > > >> > > >> - if (pgdat) > > >> - __account_obj_stock(objcg, stock, nr_acct, pgdat, idx); > > >> - > > >> if (allow_uncharge && (stock->nr_bytes > PAGE_SIZE)) { > > >> nr_pages = stock->nr_bytes >> PAGE_SHIFT; > > >> stock->nr_bytes &= (PAGE_SIZE - 1); > > >> } > > >> > > >> - unlock_stock(stock); > > >> out: > > >> if (nr_pages) > > >> obj_cgroup_uncharge_pages(objcg, nr_pages); > > >> } > > >> > > >> +static void refill_obj_stock(struct obj_cgroup *objcg, > > >> + unsigned int nr_bytes, > > >> + bool allow_uncharge) > > >> +{ > > >> + struct obj_stock_pcp *stock = trylock_stock(); > > >> + __refill_obj_stock(objcg, stock, nr_bytes, allow_uncharge); > > >> + unlock_stock(stock); > > > > > > Hi Johannes, > > > > > > I noticed that after this patch, obj_cgroup_uncharge_pages() is now inside > > > the obj_stock.lock critical section. Since obj_cgroup_uncharge_pages() calls > > > refill_stock(), which seems non-trivial, this might increase the lock hold time. > > > In particular, could that lead to more failed trylocks for IRQ handlers on > > > non-RT kernel (or for tasks that preempt others on RT kernel)? Good catch. I did ponder this, but forgot by the time I wrote the changelog. > > Yes, it also seems a bit self-defeating? (at least in theory) > > > > refill_obj_stock() > > trylock_stock() > > __refill_obj_stock() > > obj_cgroup_uncharge_pages() > > refill_stock() > > local_trylock() -> nested, will fail > > Not really as the local_locks are different i.e. memcg_stock.lock in > refill_stock() and obj_stock.lock in refill_obj_stock(). Right, refilling the *byte* stock could produce enough excess that we refill the *page* stock. Which in turn could produce enough excess that we drain that back to the page counters (shared atomics). > However Hao's concern is valid and I think it can be easily fixed by > moving obj_cgroup_uncharge_pages() out of obj_stock.lock. Note that we now have multiple callsites of __refill_obj_stock(). Do we care enough to move this to the caller? There are a few other places with a similar pattern: - drain_obj_stock(): calls memcg_uncharge() under the lock - drain_stock(): calls memcg_uncharge() under the lock - refill_stock(): still does full drain_stock() All of these could be more intentional about only updating the per-cpu data under the lock and the page counters outside of it. Given that IRQ allocations/frees are rare, nested ones even rarer, and the "slowpath" is a few extra atomics, I'm not sure it's worth the code complication. At least until proven otherwise. What do you think?