From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from stravinsky.debian.org (stravinsky.debian.org [82.195.75.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 742BB412261 for ; Fri, 31 Jul 2026 11:58:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=82.195.75.108 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785499091; cv=none; b=gkybr5DpCxB8bADlbP6fCHH9tvNMuY2DQROfAF93O1hkzJah1VTjKusH9TaKiIx5U9D01gNUZPfm3BLSp0GETbIXg74nLPMkGKGzm8aQG8crDY2v/OHGmT0bfNLZ9AnU4Bo4I0kLqKgdgZl067XjE31A1+3X/VNVEV3uC/92Jew= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785499091; c=relaxed/simple; bh=Ysage/Ieve9S1NTuRRBk3vB5chjRRmeR1nUcMGMJy3E=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=drpjuCO5422bHlpyv1cW++qxkZeeekk7hQQEeUeYtAhGkqs6yvbWp54X+InzpcLamjinowMWMv69a8Sm3yspFAPoaZTbNOSPl7g4UiR3SpdTOVG+j84gid5ZNgyhK5TEWvYItMCNqK5Fk47y3B/NjGv/bPWFQ54iN+bsFX/njAw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org; spf=pass smtp.mailfrom=debian.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b=sUcckFRW; arc=none smtp.client-ip=82.195.75.108 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=debian.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b="sUcckFRW" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debian.org; s=smtpauto.stravinsky; h=X-Debian-User:Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description; bh=2JV6uqXiLpEzoobLtRq2pI62muWcSY+g4Zhtc5RsR4E=; b=sUcckFRWGF4hXMNOsAt9fX7f4j 0S/UVvIDnlVtKY4WHh2iMrlvsFY+yXfa/gdSpD+OeVd5w6raFM4He1cNN77xxXC5BDW4OYLPFODfs RiNulWR/1B6hUtL/Za5b/l7G72axZTirO2BqGxvIerowGwwLeJ8lu7iH6s1eNK66Wki3iIWE4TA9I o8j5q6YEfn9assGOEnZXAqmF4dRsgaV5UsEAdLa4KxM+5cPKzAcny+fxLQULkIt3IDpNJz7b+A4sB tz9x1qWMO9kUHF6Pmoiu8wwyea5tclSD3w3x/kFLBMEB2fhKsidVCRjDRRYpGGT0xKeiY96du8D/O ABTWgSkQ==; Received: from authenticated-user by stravinsky.debian.org with esmtpsa (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.96) (envelope-from ) id 1wplsO-009zdB-1l; Fri, 31 Jul 2026 11:58:08 +0000 From: Breno Leitao Date: Fri, 31 Jul 2026 04:57:37 -0700 Subject: [PATCH 5/6] workqueue: add a per-cpu backend for unbound pwqs Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20260731-wq-pool-refactor-v1-5-8eaf71cdab5f@debian.org> References: <20260731-wq-pool-refactor-v1-0-8eaf71cdab5f@debian.org> In-Reply-To: <20260731-wq-pool-refactor-v1-0-8eaf71cdab5f@debian.org> To: Tejun Heo , Lai Jiangshan Cc: linux-kernel@vger.kernel.org, Breno Leitao , kernel-team@meta.com, marco.crivellari@suse.com X-Mailer: b4 0.16-dev-d5d98 X-Developer-Signature: v=1; a=openpgp-sha256; l=3143; i=leitao@debian.org; h=from:subject:message-id; bh=Ysage/Ieve9S1NTuRRBk3vB5chjRRmeR1nUcMGMJy3E=; b=owEBbQKS/ZANAwAIATWjk5/8eHdtAcsmYgBqbI28BAzyrP3KflRYnckzY4gVoAFHq2byG9Dqb V/mcONk9xyJAjMEAAEIAB0WIQSshTmm6PRnAspKQ5s1o5Of/Hh3bQUCamyNvAAKCRA1o5Of/Hh3 be7iD/9fJKiTIep33iXQqG+VmzfxT0BFxxhReU/hDWgL2+OEu463jZb7fmriSSorWfeYuUdHebA BvnIeK7oX/owuSqcIHGh/aSZ1Fl8I9T8PiQKsGme48MZaKt6OBjvaLSlRabsvTDhpEHXA8Cbhck Y14J+/ACNgSoFgJkU5f6c+JymVEF0F9eJ7Ox70DG4OEWmYA2SWghwxdH9F03KMu2sJzL3y8XZV2 Bgm2VvbujijtNv+ZE3J/oVm42fTBrfo5ilG0zbsRwdxW0wfZWnZz9Rlj01cCbPnXCUlX8FXBS/2 2cRjtLU3R7+cAKiBaB6C5whSnjeN4fIDMGHxWAijrWguQ65cxeC0KE7kYi9Q2cU6REQ+H9Nx5o5 hbPYqt0gT8kH3+RHlIoWidxsyF2HboMo1n/2qZeJjAJPmx6uhoEF+csQ2n1dDgIBQtKwINVuUiy yDtSj2hHkFs75RotKXO6Miyhum+AZpWb4O3PnUcXvs0Ap9KXJUopPBpBhPZ1EglQ5zqKTAF01iO bRkc2E0ITxrR2lPRvn5aQ9BV3ijbies+3la9FlNzKveV9fi+7BDXNDtSXdQUw24l0cew4FEopCY LmEw4iQptu8TB6eXloB++HpdnC2QkCAlkvd448LZFYX85Rt0VIkwyY8nzaIm6Jnw28V3g+geiO6 B/15s5cURwl4Grg== X-Developer-Key: i=leitao@debian.org; a=openpgp; fpr=AC8539A6E8F46702CA4A439B35A3939FFC78776D X-Debian-User: leitao Add alloc_percpu_pwq(), which binds a pwq to get_percpu_pool(wq, cpu), and an internal __WQ_PERCPU_POOLS flag. unbound_wq_update_pwq() installs such a pwq per CPU when the flag is set, reusing the existing install/drain path. Pool release and nr_active are already keyed on the backing pool, so a per-cpu-backed pwq is torn down and throttled correctly. PS: We can do this using if/else for per cpu/unbound as well, instead of this labels:, would it be better? Signed-off-by: Breno Leitao --- include/linux/workqueue.h | 1 + kernel/workqueue.c | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 34 insertions(+) diff --git a/include/linux/workqueue.h b/include/linux/workqueue.h index a283766a192aa..5bbbed94d2fa6 100644 --- a/include/linux/workqueue.h +++ b/include/linux/workqueue.h @@ -410,6 +410,7 @@ enum wq_flags { __WQ_ORDERED = 1 << 17, /* internal: workqueue is ordered */ __WQ_LEGACY = 1 << 18, /* internal: create*_workqueue() */ __WQ_DEPRECATED = 1 << 19, /* internal: workqueue is deprecated */ + __WQ_PERCPU_POOLS = 1 << 20, /* internal: back unbound pwqs with percpu pools */ /* BH wq only allows the following flags */ __WQ_BH_ALLOWS = WQ_BH | WQ_HIGHPRI | WQ_PERCPU, diff --git a/kernel/workqueue.c b/kernel/workqueue.c index d17fdcaaf1685..df4fc9ccb7b22 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -5404,6 +5404,27 @@ static struct pool_workqueue *alloc_unbound_pwq(struct workqueue_struct *wq, return pwq; } +/* + * Create a pwq backing @wq on @cpu with the static per-cpu pool instead of a + * dedicated unbound pool. Used by the unbound pwq machinery for a workqueue + * that requests the per-cpu backend. + */ +static struct pool_workqueue *alloc_percpu_pwq(struct workqueue_struct *wq, + int cpu) +{ + struct worker_pool *pool = get_percpu_pool(wq, cpu); + struct pool_workqueue *pwq; + + lockdep_assert_held(&wq_pool_mutex); + + pwq = kmem_cache_alloc_node(pwq_cache, GFP_KERNEL, pool->node); + if (!pwq) + return NULL; + + init_pwq(pwq, wq, pool); + return pwq; +} + /** * wq_calc_pod_cpumask - calculate a wq_attrs' cpumask for a pod * @attrs: the wq_attrs of the default pwq of the target workqueue @@ -5643,6 +5664,17 @@ static void unbound_wq_update_pwq(struct workqueue_struct *wq, int cpu) if (!(wq->flags & WQ_UNBOUND) || wq->unbound_attrs->ordered) return; + if (wq->flags & __WQ_PERCPU_POOLS) { + /* nothing to do if @cpu is already backed by its per-cpu pool */ + if (is_pool_cpu_specific(unbound_pwq(wq, cpu)->pool)) + return; + + pwq = alloc_percpu_pwq(wq, cpu); + if (!pwq) + goto use_dfl_pwq; + goto install; + } + /* * We don't wanna alloc/free wq_attrs for each wq for each CPU. * Let's use a preallocated one. The following buf is protected by @@ -5666,6 +5698,7 @@ static void unbound_wq_update_pwq(struct workqueue_struct *wq, int cpu) goto use_dfl_pwq; } +install: /* Install the new pwq. */ mutex_lock(&wq->mutex); old_pwq = install_unbound_pwq(wq, cpu, pwq); -- 2.53.0-Meta