From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wm1-f52.google.com (mail-wm1-f52.google.com [209.85.128.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8C0AC3ADBA2 for ; Thu, 20 Aug 2026 07:06:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.52 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787209584; cv=none; b=iX/7m5SAmbYmsmw+6c3e/nkNbslsyaRFRuWPG2Ye+QBi4geTJMvYfTX0QKtvEUZmGUBQ96bwXN1XuBx898TunrGoh1xqJe4LFlsh/y9X/SYsC9VOf0Ay+22K67nbFZKjNR23MWCjRMLRqdDbcDeT1HwtGDoywvrf8rt6Eg3pDjI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787209584; c=relaxed/simple; bh=YvyJzAKdQooEr8VSbKYEVbBo6GxZ2g3Vl9/TJT+Fzu0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=owPG2pDLXudbxfo05qw7JBiTUDEzIsWDjO/UD3dJaSZpuy2NZkexSUl3ZVDrLBE6xU9YkISHLzoqnpwsKuEt91XZ6ZW5EhUcohOh8ewNVdsl3eqj0fwm/qQf3G6hbvyMWwvtS3JrrBk77RZfPSdnryf2bC4wOocPI3nwH7u2p+U= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=suse.com; spf=pass smtp.mailfrom=suse.com; dkim=pass (2048-bit key) header.d=suse.com header.i=@suse.com header.b=a0UobtDW; arc=none smtp.client-ip=209.85.128.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=suse.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=suse.com header.i=@suse.com header.b="a0UobtDW" Received: by mail-wm1-f52.google.com with SMTP id 5b1f17b1804b1-49954b88fffso18715105e9.0 for ; Thu, 20 Aug 2026 00:06:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=google; t=1787209580; x=1787814380; darn=vger.kernel.org; h=in-reply-to:content-disposition:content-type:mime-version :references:message-id:subject:cc:to:from:date:from:to:cc:subject :date:message-id:reply-to:content-type; bh=OSJ5BKrVQsiwQ6NExnR9GYYYXRVPl3OFkAnmP+WvLQE=; b=a0UobtDWNeBv/jR0DK6sgYrAGKitly/vwGhRvVcym3EG56I6suqaZfREYs45RnOvUo MC3JIuvAoW00SFxUg94ZIhJ+KgxiypPk3Hn2eyUq50cZmzxpGOirM0hs/1F7Cm7qNP7Q GvmV/VsCpPBrvtwoU6PrYYjRHWfffy206QoU6ITrTHZOdlSRCrJPsPLezwVg4mo3vVM0 7DwhapFM7+XtfmfSGZ50Rw2mkm+NQGn6jqJXmunArio7YOobA0bsFmUdByHR2F9MwAUV fcU1dbzuEKElcIWZIlxtyXspNSkQ4oLVrBXaCjOMLJdwBCNqc6EGEO+ZkDurf7v4i6I1 3LYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787209580; x=1787814380; h=in-reply-to:content-disposition:content-type:mime-version :references:message-id:subject:cc:to:from:date:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=OSJ5BKrVQsiwQ6NExnR9GYYYXRVPl3OFkAnmP+WvLQE=; b=Klu0wZeb5lWG40sybbEaYDcVtJ+668SnKfLQ/YRN+tb3NGnGOOVW2SCbBMA/kC9R9q rgXNz7jiith8gtvLV72uIiIpWqXStGFGtVBjeJGjCOOWj4cEnCLXINbB4gMbeEP4A1bP DanWu7fEstf4b2TNJ/QSTgIrI8TB74cRl+RDHWr7gSdkgUgHH8t8Wy4UwXkFAOjrFQa2 2Zabicp6J7rz0yXvHQ9Q75NnNmK3Q6cfUjrHJyDbc4E/fJ2aGBYdhIk3JWTF89veeEF9 +sm8NF0jPueOR3ay9oeRDqBp0vAwyRP1x1ZYVu57Co2H4FYW92XR3UDtIMzM1aZTk3uz R6rg== X-Forwarded-Encrypted: i=1; AHgh+RpqY6S0uBz0Gte2aCN8RCJPBrjoFJsrnNDwR71yOQI9nMNpD/HsWMX04Ei5EVDkaPT/47ntxSdXXuTEUhM=@vger.kernel.org X-Gm-Message-State: AOJu0YwoLI7Ne+BDOFoTYBuuw0hQJAP1zxKuzWSu+fRFUw+wBbTUWFCS 9Y79bk+HVPxbR5b6cBfQHvQRBkOawv3fj7N8FkEIbBeSyeF+/TWPBr4BfOrvarjYUpA= X-Gm-Gg: AR+sD10nS7H89bXpVQfkwe1PAm/62mMfRWnZwBMgtEKL70MH2/lKFlKVnqf2/99cKYx 22OQQOtgCGLXQked/9jysAaEzbpor2UJeAJ2eeRheQRbZnMeTxADIKB5zp5+eS7prSDu+st5J1r BGNeeOKXBcgQ7/hKdGxXP7iFVE+LJ/8sSxAEUExv5dpimXl7Zly+BLoAp3N93FUADuqEEYsOoFB SofnmU+JXloV3GNWsV+KwbOaZvwwAOkFqm1oGD1DzUU4WVZ+N0ey0t0BhjKlzOhOx4ASsItdDgN vx7srnGpQ8udzC3hIcvOh/kaj5DSwGGItROqA2gI+YMHKmzRs5gIXWFA8YSViI/qhqn9XsritoI 9RIWqNv4ELT2vDHBcnyk1eE/cZoNmGNL12c2aI2EHeCS/7Ra77U+IE5Jy6zNXWnJau+8LJZregz ngL9tXRwX8IvlxITsE3pxdK+F0BqvY9df7Vkf7IfcWb7Re8DZjHzC2CSwUmGDukXegajuDAcFOF H+MarNEzsMS X-Received: by 2002:a05:600c:6211:b0:499:a760:722f with SMTP id 5b1f17b1804b1-499aa1ba0b7mr188614485e9.13.1787209579825; Thu, 20 Aug 2026 00:06:19 -0700 (PDT) Received: from localhost (109-81-21-108.rct.o2.cz. [109.81.21.108]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-499b3c556a3sm10815435e9.3.2026.08.20.00.06.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Aug 2026 00:06:19 -0700 (PDT) Date: Thu, 20 Aug 2026 09:06:18 +0200 From: Michal Hocko To: Shakeel Butt Cc: Andrew Morton , Johannes Weiner , Roman Gushchin , Muchun Song , Joshua Hahn , Jakub Kicinski , Meta kernel team , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Joy Chaoyue Xiong Subject: Re: [PATCH v2] memcg: trim the per-cpu charge stock instead of draining it Message-ID: References: <20260820012010.2016086-1-shakeel.butt@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20260820012010.2016086-1-shakeel.butt@linux.dev> On Wed 19-08-26 18:20:10, Shakeel Butt wrote: > Joy reported that an application generating a request/response traffic > pattern spends 44.6% to 57.0% of CPU in the memcg charge/uncharge path > for a range of message sizes, against 0.27% to 0.71% outside that range. > Running from the root memcg, where socket memory accounting is skipped, > recovers the performance. > > Tracing the charge path showed that the application generates a pattern > where the write syscall charges one page and the read syscall uncharges > two pages on the same CPU. This hits a corner case in the memcg percpu > stock code that thrashes the stock continuously. > > In the memcg percpu stock code, MEMCG_CHARGE_BATCH (64) is both the high > watermark and the emptying target, i.e. on a request to charge one page > the kernel charges MEMCG_CHARGE_BATCH pages and caches > (MEMCG_CHARGE_BATCH - 1) of them in the percpu stock. The following > uncharge of 2 pages takes the cached count to (MEMCG_CHARGE_BATCH + 1), > and refill_stock() then empties the cache completely. With such a > pattern the percpu stock becomes completely ineffective. > > Instead of a single boundary point for charges, use the technique the > page allocator uses for its own percpu caches, which keeps the watermark > and the emptying target apart: nr_pcp_free() frees between batch and > high - batch pages, leaving at least pcp->batch on the list. Add a high > watermark MEMCG_STOCK_HIGH and, once the cached count goes over it, > return only the pages above MEMCG_STOCK_LOW. The watermarks are > MEMCG_CHARGE_BATCH apart, so a page_counter update still covers a full > batch. > > For now, keep MEMCG_STOCK_HIGH same as MEMCG_CHARGE_BATCH and in > future we will reevaluate if it makes sense to increase it. > > Reported-by: Joy Chaoyue Xiong > Signed-off-by: Shakeel Butt Acked-by: Michal Hocko One minor nit > --- > > Changes since v1: > http://lore.kernel.org/20260817234651.666540-1-shakeel.butt@linux.dev > - Kept the upper limit of memcg stock same as before (Michal) > > mm/memcontrol.c | 25 +++++++++++++++++++------ > 1 file changed, 19 insertions(+), 6 deletions(-) > > diff --git a/mm/memcontrol.c b/mm/memcontrol.c > index 17da1f43b7d3..58e4d23cf5e0 100644 > --- a/mm/memcontrol.c > +++ b/mm/memcontrol.c [...] > @@ -2254,9 +2264,12 @@ static void refill_stock(struct mem_cgroup *memcg, unsigned int nr_pages) > empty_slot = i; > if (memcg == READ_ONCE(stock->cached[i])) { > stock_pages = READ_ONCE(stock->nr_pages[i]) + nr_pages; > + if (stock_pages > MEMCG_STOCK_HIGH) { > + memcg_uncharge(memcg, > + stock_pages - MEMCG_STOCK_LOW); > + stock_pages = MEMCG_STOCK_LOW; I would find it easier to read to keep the update in sync with memcg_uncharge, i.e. stock_pages = WRITE_ONCE(stock_pages - MEMCG_STOCK_LOW) > + } > WRITE_ONCE(stock->nr_pages[i], stock_pages); > - if (stock_pages > MEMCG_CHARGE_BATCH) > - drain_stock(stock, i); > success = true; > break; > } > -- > 2.53.0-Meta -- Michal Hocko SUSE Labs