From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from stravinsky.debian.org (stravinsky.debian.org [82.195.75.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 388E740683D for ; Fri, 31 Jul 2026 11:58:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=82.195.75.108 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785499088; cv=none; b=Sjy3yR0oKSCD8IrOrD8MUNJbQ2Cnsas0X+yhEurNQlcEz6RoHRpwikw/dcjjPbiFAsDwM8ZcH0zNfl/g9IvWrZahQwmO5Qn7dFSrRjQi8D2WVjAGBKObdiLkqjAmBX09H0y/13YIc6JFbVkbOH26jz6xc0IPMdjUMncMIAVtQhw= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785499088; c=relaxed/simple; bh=+cmYs0OiKG42jBCyQx/PNot02QSRmI0WmfqsaiTeT20=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=D3ulnloGpkaGVXuxINqIknvgEx7h/CK2IfKdA+7y8w0zXHfetQhUECN71ziLcCDoYP1wa++OvlTDAkvGjr5WTmarOge28sfTXuIUyRzH2mMn2fE6Qc79c7nyt0NVK3WRysRW5glNciY+bkhFkxdar9FCbHjLYz5V0nB7+3p19D8= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org; spf=pass smtp.mailfrom=debian.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b=D7W6K2uf; arc=none smtp.client-ip=82.195.75.108 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=debian.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b="D7W6K2uf" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debian.org; s=smtpauto.stravinsky; h=X-Debian-User:Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description; bh=05lF33MZqn7vtw5qvL+tIYyGwExZlD48p9Gs4D2krxQ=; b=D7W6K2ufQFzrDLOcYC7TEbIrjx tpkm6zJBOwjherouoJBZOv/h8afDoXiEj6FH14/Jal7hv4LSquylkrFnxIM+YGz0qyGgRpesUwSer RV8oxLu87ctmehkxlwkAqdF/54uzbiYY3w0n7mLlN6ROBm3mkmPMcB9sDhaDxuTN1SStqANTOst/E V2FF2bgpj6tzCk0W70HnZbtblbBvQdrIwtPNvB9q7KWXUqAK7L+o6lM81/rt+7onjLmlhp94uEXjM Mip26kyxe7ATuW5nenf70Uzh5o+AC3AdEbXI/5duOqdvOEty6b7D1KnttxDlf5WhmzAUReRyihVae jsFIgfnw==; Received: from authenticated-user by stravinsky.debian.org with esmtpsa (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.96) (envelope-from ) id 1wplsL-009zd0-0c; Fri, 31 Jul 2026 11:58:05 +0000 From: Breno Leitao Date: Fri, 31 Jul 2026 04:57:36 -0700 Subject: [PATCH 4/6] workqueue: account nr_active by the backing pool Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20260731-wq-pool-refactor-v1-4-8eaf71cdab5f@debian.org> References: <20260731-wq-pool-refactor-v1-0-8eaf71cdab5f@debian.org> In-Reply-To: <20260731-wq-pool-refactor-v1-0-8eaf71cdab5f@debian.org> To: Tejun Heo , Lai Jiangshan Cc: linux-kernel@vger.kernel.org, Breno Leitao , kernel-team@meta.com, marco.crivellari@suse.com X-Mailer: b4 0.16-dev-d5d98 X-Developer-Signature: v=1; a=openpgp-sha256; l=2510; i=leitao@debian.org; h=from:subject:message-id; bh=+cmYs0OiKG42jBCyQx/PNot02QSRmI0WmfqsaiTeT20=; b=owEBbQKS/ZANAwAIATWjk5/8eHdtAcsmYgBqbI28rb/kdfb+OUrIkA6Ynq+0aQ0t0F2XbX5AO gkY0hCGaryJAjMEAAEIAB0WIQSshTmm6PRnAspKQ5s1o5Of/Hh3bQUCamyNvAAKCRA1o5Of/Hh3 bbEYD/wJY9JXQV+VYXcQagQ8YNWYKakQ5BrCej2PKBe5P0O7BSTYJCK4AOZ3+truqLr1o0M42FZ tm3YM34zxXBFzgJvceozMSns1mXifqlVENOkea94mkS47abs+S2QbcBtGCLWP/cG5efDkEQkDJ2 Vcr9sTi9Z3CkBWw/8u/mxQa7kTK/GpHaIHt/dygQF9hl7SQysA7zAlnYFUZXPMMgJiE2N9w8gAs 8VoJWJ/XFhWGQljJYMuXXNawu6/2SyJ8/mpp5c96oCcmKOmU/yt/vPbG60XQtsA41OCS7rcIsZq z6RvP5Khh+rDPs7sQP9cusHIBMk6cUl8LFT93fY7w9xunqUo1Pbsxlh3iw2ZG517zZEsekeuz/z jjvYfEqT6gp6yQvyuw27c/PlFqo1vHSW7t1K7SunrRMR+ZmP8sp873DHbC9HAFrdTnpUJ/pGguQ 6/wKPVAojPq3GAeLk3ObV/Xa/qh9DL9pCYWW+6AIqRq4BdagEFw78GTl4ToGLYIxkqCi0GMhPnl 6sqv3VHAQerOgqcvocPVWWWrBqcPCfBVYFK/9XLMNW5FWWH4hRRCng/y0BJPIfGojyos7A2pc/O b/8oLUYjhdZIP3Xtb7/KoV+UIPjAbx2NIFV13bQ3bBcD5eiX5GPX+dEKMhCF7+uFEoxRLTWGp3W knf/WjhajhA45Ig== X-Developer-Key: i=leitao@debian.org; a=openpgp; fpr=AC8539A6E8F46702CA4A439B35A3939FFC78776D X-Debian-User: leitao Use is_pool_cpu_specific() instead of wq_node_nr_active() to check for per workqueues. Signed-off-by: Breno Leitao --- kernel/workqueue.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 513193be00a7d..d17fdcaaf1685 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -1782,13 +1782,16 @@ static bool pwq_tryinc_nr_active(struct pool_workqueue *pwq, bool fill) { struct workqueue_struct *wq = pwq->wq; struct worker_pool *pool = pwq->pool; - struct wq_node_nr_active *nna = wq_node_nr_active(wq, pool->node); + struct wq_node_nr_active *nna; bool obtained = false; lockdep_assert_held(&pool->lock); - if (!nna) { - /* BH or per-cpu workqueue, pwq->nr_active is sufficient */ + /* + * A concurrency-managed per-cpu pool accounts nr_active per pwq, so + * pwq->nr_active against wq->max_active is sufficient. + */ + if (is_pool_cpu_specific(pool)) { obtained = pwq->nr_active < READ_ONCE(wq->max_active); goto out; } @@ -1796,6 +1799,10 @@ static bool pwq_tryinc_nr_active(struct pool_workqueue *pwq, bool fill) if (unlikely(pwq->plugged)) return false; + nna = wq_node_nr_active(wq, pool->node); + if (WARN_ON_ONCE(!nna)) + return false; + /* * Unbound workqueue uses per-node shared nr_active $nna. If @pwq is * already waiting on $nna, pwq_dec_nr_active() will maintain the @@ -2013,7 +2020,7 @@ static void node_activate_pending_pwq(struct wq_node_nr_active *nna, static void pwq_dec_nr_active(struct pool_workqueue *pwq) { struct worker_pool *pool = pwq->pool; - struct wq_node_nr_active *nna = wq_node_nr_active(pwq->wq, pool->node); + struct wq_node_nr_active *nna; lockdep_assert_held(&pool->lock); @@ -2024,14 +2031,18 @@ static void pwq_dec_nr_active(struct pool_workqueue *pwq) pwq->nr_active--; /* - * For a percpu workqueue, it's simple. Just need to kick the first + * A concurrency-managed per-cpu pool only needs to kick the first * inactive work item on @pwq itself. */ - if (!nna) { + if (is_pool_cpu_specific(pool)) { pwq_activate_first_inactive(pwq, false); return; } + nna = wq_node_nr_active(pwq->wq, pool->node); + if (WARN_ON_ONCE(!nna)) + return; + /* * If @pwq is for an unbound workqueue, it's more complicated because * multiple pwqs and pools may be sharing the nr_active count. When a -- 2.53.0-Meta