From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from stravinsky.debian.org (stravinsky.debian.org [82.195.75.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A5542376A12 for ; Wed, 5 Aug 2026 14:53:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=82.195.75.108 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785941601; cv=none; b=Dn6TX8D1w6zp2rDdtvifE9PNZb7bc+Nt+gGtLxYg34fURJ78PXEbJGoyklw4VYnSXoZGvwxNndN5cHFMdV2xc/WBOldcvFGEJvpkWwn2ezxd+Lb5RgdPvZdw5e2VXTRs6idWIEzTamsIO/6z+lQbT6K52oQKTQcgjzBskLqjANI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785941601; c=relaxed/simple; bh=CWM4XPMimBVDvcLFphvESKoVCT3CFiV7ETMNpYfnfjY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=KGKZwMFTugI+YOVBJbHbpp2OfZzfdv/4NUfIWYMW9850HwjfUKUD5wsPekNq6y4/NjAFqJCV+AMWrh8p28u/xidSSGZIP1BSSEPXSBApHk6ufcYdvPIexQ0lS5D8gSb+fWr0FrFwt2ROBk20d36DDD6FRW0/6UILGECyVAmQPFw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org; spf=pass smtp.mailfrom=debian.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b=omGInGA0; arc=none smtp.client-ip=82.195.75.108 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=debian.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b="omGInGA0" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debian.org; s=smtpauto.stravinsky; h=X-Debian-User:Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description; bh=vCb/gSvK9zHTsmz/4PR241Aw4yy6BI1t+7L5kb9YhXA=; b=omGInGA0+ie9EHjNBajXYnal2Y oruopdH1xsjshnbuoEtfH/LXGwmAFaTGdJ9Sp+bSXiBOoZKbcdXT3RgAMxjEwq4PRHwN3INP6+jd5 OFiclNh8C78hWBGr5kGrs2N4l3meGq328N7NXEGuroJswAfOfQpMifMvkKLA931ot+EYRAhpjbg6E CE0pleoRlok9ZnFf6uwJUbUGrHGE7sxKKgw75pGnIjh9ZgXKBxI81Zw9y4Bk1rmw7NycuhZkkvEir mwQMqTeYz274Rf7ALh3Q2TySjHSCsVuk+tPe18pnNr+kRhTduwe+F4Kq/VFLfqs2f3hRBbxhZRpCU aKWcLFvg==; Received: from authenticated-user by stravinsky.debian.org with esmtpsa (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.96) (envelope-from ) id 1wrczb-00Dzfv-1E; Wed, 05 Aug 2026 14:53:15 +0000 From: Breno Leitao Date: Wed, 05 Aug 2026 07:52:31 -0700 Subject: [PATCH v2 4/9] workqueue: account nr_active by the backing pool Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20260805-wq-pool-refactor-v2-4-fd498d01d695@debian.org> References: <20260805-wq-pool-refactor-v2-0-fd498d01d695@debian.org> In-Reply-To: <20260805-wq-pool-refactor-v2-0-fd498d01d695@debian.org> To: Tejun Heo , Lai Jiangshan Cc: linux-kernel@vger.kernel.org, marco.crivellari@suse.com, Breno Leitao , kernel-team@meta.com X-Mailer: b4 0.16-dev-f8e9d X-Developer-Signature: v=1; a=openpgp-sha256; l=3446; i=leitao@debian.org; h=from:subject:message-id; bh=CWM4XPMimBVDvcLFphvESKoVCT3CFiV7ETMNpYfnfjY=; b=owEBbQKS/ZANAwAIATWjk5/8eHdtAcsmYgBqc05K+m+sbMbMaPVBS92HwFewQtCupNXVko3+p hu/qM03Gb+JAjMEAAEIAB0WIQSshTmm6PRnAspKQ5s1o5Of/Hh3bQUCanNOSgAKCRA1o5Of/Hh3 bcybD/4gVI25qhIj7QduI9RcRSoJePsCw8bJqwh3RliizA2lZhUR+V834/3shtY06Jpi/i9kNm1 EmL4AQvsVe0nkDo5/qdmi6aYZ9m/bQnTCnmVE14hCS5+7MLrdzWzuO7/hJK+9Nt4a8isrCJjIFK s1TCLC8ePusvV627IzotRl9gbPPEjhh4WcXGWBImGo1C4Wsxl924B6/ZoJPNycirFmbFpP6Usqu MqPcHSuv/EvHGyTAvqMoLtjH3HVBDnx6LsJRYAETwThmwUO/5LEr0+35rnfpJjS88A3Db4z4Tp6 cJbY+g2NdHfaz20mIjGLfBwSGXECqeZ6kNmjPY/yDBqPKbEEpnpmYclwO2t84lmQHLHskbaYOqx w58Eeu8lnF1VyTgTBQjmK1OxqQW7sR6KuXJSLwgq/l6MyW2jnZiSWnDs3seYt65FB+lxIEE6lss TEA4VaAX6zBuf16MDoa5IiLOiAcZoDDPLb0FmjjgABOwgJo7UC3c7EMGWYYTU9NnC7657Q+NdVJ FZkYmEoxFT0xiX9MUnExfSOf+z4+C7p6TAC39OaOpWlFzFKmfBnHPhhVf0Ee91LdNLxf5giKTga hLC6tsjDBiQ/eEXMV2+cnhlKr6RHcKsgX/dpBvBodY79GEwXZvEXPplecP9JZzOjn14cpphMBnY 8df6jhmu07koYNw== X-Developer-Key: i=leitao@debian.org; a=openpgp; fpr=AC8539A6E8F46702CA4A439B35A3939FFC78776D X-Debian-User: leitao pwq_tryinc_nr_active() and pwq_dec_nr_active() choose between the shared per-node nr_active and the plain per-pwq one by testing wq_node_nr_active() for NULL. Test the backing pool with is_percpu_pool() instead, so the accounting follows the pool that runs the work rather than the workqueue type. No functional change. Signed-off-by: Breno Leitao --- kernel/workqueue.c | 26 ++++++++++++++++---------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index b96090c85bcaa..7b20d459d6442 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -1615,9 +1615,8 @@ static bool is_percpu_pool(struct worker_pool *pool) * @wq: workqueue of interest * @node: NUMA node, can be %NUMA_NO_NODE * - * Determine wq_node_nr_active to use for @wq on @node. Returns: - * - * - %NULL for per-cpu workqueues as they don't need to use shared nr_active. + * Determine wq_node_nr_active to use for @wq on @node. @wq must be unbound. + * Returns: * * - node_nr_active[nr_node_ids] if @node is %NUMA_NO_NODE. * @@ -1626,7 +1625,7 @@ static bool is_percpu_pool(struct worker_pool *pool) static struct wq_node_nr_active *wq_node_nr_active(struct workqueue_struct *wq, int node) { - if (!(wq->flags & WQ_UNBOUND)) + if (WARN_ON_ONCE(!(wq->flags & WQ_UNBOUND))) return NULL; if (node == NUMA_NO_NODE) @@ -1782,13 +1781,16 @@ static bool pwq_tryinc_nr_active(struct pool_workqueue *pwq, bool fill) { struct workqueue_struct *wq = pwq->wq; struct worker_pool *pool = pwq->pool; - struct wq_node_nr_active *nna = wq_node_nr_active(wq, pool->node); + struct wq_node_nr_active *nna; bool obtained = false; lockdep_assert_held(&pool->lock); - if (!nna) { - /* BH or per-cpu workqueue, pwq->nr_active is sufficient */ + /* + * A concurrency-managed per-cpu pool accounts nr_active per pwq, so + * pwq->nr_active against wq->max_active is sufficient. + */ + if (is_percpu_pool(pool)) { obtained = pwq->nr_active < READ_ONCE(wq->max_active); goto out; } @@ -1796,6 +1798,8 @@ static bool pwq_tryinc_nr_active(struct pool_workqueue *pwq, bool fill) if (unlikely(pwq->plugged)) return false; + nna = wq_node_nr_active(wq, pool->node); + /* * Unbound workqueue uses per-node shared nr_active $nna. If @pwq is * already waiting on $nna, pwq_dec_nr_active() will maintain the @@ -2013,7 +2017,7 @@ static void node_activate_pending_pwq(struct wq_node_nr_active *nna, static void pwq_dec_nr_active(struct pool_workqueue *pwq) { struct worker_pool *pool = pwq->pool; - struct wq_node_nr_active *nna = wq_node_nr_active(pwq->wq, pool->node); + struct wq_node_nr_active *nna; lockdep_assert_held(&pool->lock); @@ -2024,14 +2028,16 @@ static void pwq_dec_nr_active(struct pool_workqueue *pwq) pwq->nr_active--; /* - * For a percpu workqueue, it's simple. Just need to kick the first + * A concurrency-managed per-cpu pool only needs to kick the first * inactive work item on @pwq itself. */ - if (!nna) { + if (is_percpu_pool(pool)) { pwq_activate_first_inactive(pwq, false); return; } + nna = wq_node_nr_active(pwq->wq, pool->node); + /* * If @pwq is for an unbound workqueue, it's more complicated because * multiple pwqs and pools may be sharing the nr_active count. When a -- 2.53.0-Meta