[PATCH RFC 0/3] Refactor the workqueue allocations

[PATCH RFC 0/3] Refactor the workqueue allocations

[Breno Leitao]

POC for Tejun's idea to unify the per-cpu and unbound workqueues, as
suggested in http://lkml.kernel.org/r/ak569WYSm3ygKl1-@slm.duckdns.org

This series only unifies the allocation part (alloc_pwq()). It makes
alloc_pwq() a single allocator that returns a pwq backed by either a static
per-cpu pool (get_percpu_pool) or an unbound pool (get_unbound_pool), reusing
the existing static per-cpu pools. The install/link path is unchanged and all
patches are behavior-neutral. This sets things up to unify the install side
later.

Is this the right direction?

Questions to follow up:

1) wqattrs is an unbound concept and apply_workqueue_attrs_locked() rejects
   non-unbound wqs. Should we leverage wqattrs in per-cpu workqueues as well,
   so the unification can happen later (apply_workqueue_attrs_locked())?

2) If percpu becomes a WQ_AFFN_CPU affinity setting, how should max_active be
   treated? WQ_AFFN_CPU is unbound today, so it would inherit per-node
   accounting (wq_node_nr_active) and lose percpu's per-cpu max_active
   (pwq->nr_active).

3) What end state are you aiming for? Keep WQ_PERCPU as a thin flag over
   unified internals (single install path, per-cpu accounting special-cased)
   with WQ_UNBOUND staying for now -- or something more radical (the flags gone
   entirely, percpu purely an affinity value)?

Tests: 

 1) I've tested this on x86 and arm64, with regular tests
 2) I've hacked up a workqueue test suite, which has a bunch of tests,
    and this is what I am using to test these changes:
	https://github.com/leitao/wqtest

Thanks,
-breno

Signed-off-by: Breno Leitao <leitao@debian.org>
---
Breno Leitao (3):
      workqueue: introduce alloc_pwq()
      workqueue: allocate percpu pwqs through alloc_pwq()
      workqueue: factor out alloc_and_link_percpu_pwqs()

 kernel/workqueue.c | 96 ++++++++++++++++++++++++++++++++++--------------------
 1 file changed, 60 insertions(+), 36 deletions(-)
---
base-commit: b9810cd75b9fb56a3425d391cba3f608502bd474
change-id: 20260709-tejun1-c2aaf36f54a4

Best regards,
--  
Breno Leitao <leitao@debian.org>

[PATCH RFC 1/3] workqueue: introduce alloc_pwq()

[Breno Leitao]

Factor the static per-cpu pool lookup out of alloc_and_link_pwqs() into
get_percpu_pool(), and add alloc_pwq() -- a common pwq allocator that
picks the backing pool by workqueue type: a percpu workqueue uses the
static per-cpu pool for @cpu (get_percpu_pool()), an unbound workqueue
uses a pool matching @attrs (get_unbound_pool()).  The paired
put_pwq_pool() releases only unbound pools; static per-cpu pools are
permanent.

alloc_pwq() replaces alloc_unbound_pwq() and gives the workqueue core a
single entry point that can produce a pwq pointing at either kind of pool
-- the building block for unifying percpu and unbound workqueues.

No functional change: the current alloc_pwq() callers only operate on
unbound workqueues, so its percpu branch is not exercised yet.

Signed-off-by: Breno Leitao <leitao@debian.org>
---
 kernel/workqueue.c | 61 +++++++++++++++++++++++++++++++++++++-----------------
 1 file changed, 42 insertions(+), 19 deletions(-)

diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index 94f37ea762365..c32e173af2335 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -5354,8 +5354,38 @@ static void link_pwq(struct pool_workqueue *pwq)
 	list_add_tail_rcu(&pwq->pwqs_node, &wq->pwqs);
 }
 
-/* obtain a pool matching @attr and create a pwq associating the pool and @wq */
-static struct pool_workqueue *alloc_unbound_pwq(struct workqueue_struct *wq,
+/* Return the static per-cpu worker_pool that backs @wq on @cpu. */
+static struct worker_pool *get_percpu_pool(struct workqueue_struct *wq, int cpu)
+{
+	struct worker_pool __percpu *pools;
+	bool highpri;
+
+	WARN_ON(wq->flags & WQ_UNBOUND);
+
+	highpri = wq->flags & WQ_HIGHPRI;
+	if (wq->flags & WQ_BH)
+		pools = bh_worker_pools;
+	else
+		pools = cpu_worker_pools;
+
+	return &per_cpu_ptr(pools, cpu)[highpri];
+}
+
+/* release the pool obtained for @wq's pwq; only unbound pools are refcounted */
+static void put_pwq_pool(struct workqueue_struct *wq, struct worker_pool *pool)
+{
+	if (!(wq->flags & WQ_UNBOUND))
+		return;
+
+	put_unbound_pool(pool);
+}
+
+/*
+ * Obtain the pool backing @wq on @cpu and create a pwq associating it with @wq.
+ * A percpu @wq uses the static per-cpu pool for @cpu; an unbound @wq uses a
+ * pool matching @attrs.
+ */
+static struct pool_workqueue *alloc_pwq(struct workqueue_struct *wq, int cpu,
 					const struct workqueue_attrs *attrs)
 {
 	struct worker_pool *pool;
@@ -5363,13 +5393,16 @@ static struct pool_workqueue *alloc_unbound_pwq(struct workqueue_struct *wq,
 
 	lockdep_assert_held(&wq_pool_mutex);
 
-	pool = get_unbound_pool(attrs);
+	if (wq->flags & WQ_UNBOUND)
+		pool = get_unbound_pool(attrs);
+	else
+		pool = get_percpu_pool(wq, cpu);
 	if (!pool)
 		return NULL;
 
 	pwq = kmem_cache_alloc_node(pwq_cache, GFP_KERNEL, pool->node);
 	if (!pwq) {
-		put_unbound_pool(pool);
+		put_pwq_pool(wq, pool);
 		return NULL;
 	}
 
@@ -5478,7 +5511,7 @@ apply_wqattrs_prepare(struct workqueue_struct *wq,
 	copy_workqueue_attrs(new_attrs, attrs);
 	wqattrs_actualize_cpumask(new_attrs, unbound_cpumask);
 	cpumask_copy(new_attrs->__pod_cpumask, new_attrs->cpumask);
-	ctx->dfl_pwq = alloc_unbound_pwq(wq, new_attrs);
+	ctx->dfl_pwq = alloc_pwq(wq, -1, new_attrs);
 	if (!ctx->dfl_pwq)
 		goto out_free;
 
@@ -5488,7 +5521,7 @@ apply_wqattrs_prepare(struct workqueue_struct *wq,
 			ctx->pwq_tbl[cpu] = ctx->dfl_pwq;
 		} else {
 			wq_calc_pod_cpumask(new_attrs, cpu);
-			ctx->pwq_tbl[cpu] = alloc_unbound_pwq(wq, new_attrs);
+			ctx->pwq_tbl[cpu] = alloc_pwq(wq, cpu, new_attrs);
 			if (!ctx->pwq_tbl[cpu])
 				goto out_free;
 		}
@@ -5632,7 +5665,7 @@ static void unbound_wq_update_pwq(struct workqueue_struct *wq, int cpu)
 		return;
 
 	/* create a new pwq */
-	pwq = alloc_unbound_pwq(wq, target_attrs);
+	pwq = alloc_pwq(wq, cpu, target_attrs);
 	if (!pwq) {
 		pr_warn("workqueue: allocation failed while updating CPU pod affinity of \"%s\"\n",
 			wq->name);
@@ -5668,19 +5701,9 @@ static int alloc_and_link_pwqs(struct workqueue_struct *wq)
 		goto enomem;
 
 	if (!(wq->flags & WQ_UNBOUND)) {
-		struct worker_pool __percpu *pools;
-
-		if (wq->flags & WQ_BH)
-			pools = bh_worker_pools;
-		else
-			pools = cpu_worker_pools;
-
 		for_each_possible_cpu(cpu) {
-			struct pool_workqueue **pwq_p;
-			struct worker_pool *pool;
-
-			pool = &(per_cpu_ptr(pools, cpu)[highpri]);
-			pwq_p = per_cpu_ptr(wq->cpu_pwq, cpu);
+			struct pool_workqueue **pwq_p = per_cpu_ptr(wq->cpu_pwq, cpu);
+			struct worker_pool *pool = get_percpu_pool(wq, cpu);
 
 			*pwq_p = kmem_cache_alloc_node(pwq_cache, GFP_KERNEL,
 						       pool->node);

-- 
2.53.0-Meta

[PATCH RFC 2/3] workqueue: allocate percpu pwqs through alloc_pwq()

[Breno Leitao]

Route the per-cpu pwq allocation loop in alloc_and_link_pwqs() through the
common alloc_pwq() instead of open-coding get_percpu_pool() + pwq
allocation + init_pwq().  alloc_pwq() is now the single entry point for
both percpu and unbound pwq allocation, and its percpu branch is exercised
for every percpu workqueue created.

No functional change: alloc_pwq(wq, cpu, NULL) on a percpu workqueue
selects the static per-cpu pool for @cpu, exactly as the open-coded path
did.

Signed-off-by: Breno Leitao <leitao@debian.org>
---
 kernel/workqueue.c | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index c32e173af2335..50baf4fe5fcf5 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -5703,15 +5703,11 @@ static int alloc_and_link_pwqs(struct workqueue_struct *wq)
 	if (!(wq->flags & WQ_UNBOUND)) {
 		for_each_possible_cpu(cpu) {
 			struct pool_workqueue **pwq_p = per_cpu_ptr(wq->cpu_pwq, cpu);
-			struct worker_pool *pool = get_percpu_pool(wq, cpu);
 
-			*pwq_p = kmem_cache_alloc_node(pwq_cache, GFP_KERNEL,
-						       pool->node);
+			*pwq_p = alloc_pwq(wq, cpu, NULL);
 			if (!*pwq_p)
 				goto enomem;
 
-			init_pwq(*pwq_p, wq, pool);
-
 			mutex_lock(&wq->mutex);
 			link_pwq(*pwq_p);
 			mutex_unlock(&wq->mutex);

-- 
2.53.0-Meta

[PATCH RFC 3/3] workqueue: factor out alloc_and_link_percpu_pwqs()

[Breno Leitao]

Move the per-cpu pwq allocation loop out of alloc_and_link_pwqs() into a
helper.  The inner allocation-failure path now returns -ENOMEM and the
caller jumps to the existing enomem cleanup, equivalent to the previous
goto.

Now that the per-cpu branch returns a value through the helper, the
per-cpu, ordered and unbound cases can share a single error-handling
tail.  Turn the separate per-cpu block and the ordered/unbound if/else
into one if/else-if/else chain, dropping the early return and the
duplicated goto.

No functional change.

Signed-off-by: Breno Leitao <leitao@debian.org>
---
 kernel/workqueue.c | 35 ++++++++++++++++++++---------------
 1 file changed, 20 insertions(+), 15 deletions(-)

diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index 50baf4fe5fcf5..411e4bd105576 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -5689,6 +5689,24 @@ static void unbound_wq_update_pwq(struct workqueue_struct *wq, int cpu)
 	put_pwq_unlocked(old_pwq);
 }
 
+static int alloc_and_link_percpu_pwqs(struct workqueue_struct *wq)
+{
+	int cpu;
+
+	for_each_possible_cpu(cpu) {
+		struct pool_workqueue **pwq_p = per_cpu_ptr(wq->cpu_pwq, cpu);
+
+		*pwq_p = alloc_pwq(wq, cpu, NULL);
+		if (!*pwq_p)
+			return -ENOMEM;
+
+		mutex_lock(&wq->mutex);
+		link_pwq(*pwq_p);
+		mutex_unlock(&wq->mutex);
+	}
+	return 0;
+}
+
 static int alloc_and_link_pwqs(struct workqueue_struct *wq)
 {
 	bool highpri = wq->flags & WQ_HIGHPRI;
@@ -5701,21 +5719,8 @@ static int alloc_and_link_pwqs(struct workqueue_struct *wq)
 		goto enomem;
 
 	if (!(wq->flags & WQ_UNBOUND)) {
-		for_each_possible_cpu(cpu) {
-			struct pool_workqueue **pwq_p = per_cpu_ptr(wq->cpu_pwq, cpu);
-
-			*pwq_p = alloc_pwq(wq, cpu, NULL);
-			if (!*pwq_p)
-				goto enomem;
-
-			mutex_lock(&wq->mutex);
-			link_pwq(*pwq_p);
-			mutex_unlock(&wq->mutex);
-		}
-		return 0;
-	}
-
-	if (wq->flags & __WQ_ORDERED) {
+		ret = alloc_and_link_percpu_pwqs(wq);
+	} else if (wq->flags & __WQ_ORDERED) {
 		struct pool_workqueue *dfl_pwq;
 
 		ret = apply_workqueue_attrs_locked(wq, ordered_wq_attrs[highpri]);

-- 
2.53.0-Meta

Re: [PATCH RFC 0/3] Refactor the workqueue allocations

[Tejun Heo]

Hello,

On Tue, Jul 14, 2026 at 04:41:46AM -0700, Breno Leitao wrote:
...
> 1) wqattrs is an unbound concept and apply_workqueue_attrs_locked() rejects
>    non-unbound wqs. Should we leverage wqattrs in per-cpu workqueues as well,
>    so the unification can happen later (apply_workqueue_attrs_locked())?

I'm not sure adding wqattrs to percpu workqueues makes sense. Wouldn't the
shape more be like unbound workqueue subsuming percpu workqueue?

> 2) If percpu becomes a WQ_AFFN_CPU affinity setting, how should max_active be
>    treated? WQ_AFFN_CPU is unbound today, so it would inherit per-node
>    accounting (wq_node_nr_active) and lose percpu's per-cpu max_active
>    (pwq->nr_active).

I think it probably would be better to introduce a separate affinity scope
than modifying WQ_AFFN_CPU. Something which indicates that concurrency
management is in effect and max_active is per-cpu.

> 3) What end state are you aiming for? Keep WQ_PERCPU as a thin flag over
>    unified internals (single install path, per-cpu accounting special-cased)
>    with WQ_UNBOUND staying for now -- or something more radical (the flags gone
>    entirely, percpu purely an affinity value)?

Keeping WQ_PERCPU as a shorthand for specifying the percpu scope makes sense
to me.

Thanks.

-- 
tejun

Re: [PATCH RFC 0/3] Refactor the workqueue allocations

[Tejun Heo]

On Thu, Jul 16, 2026 at 09:22:50AM -1000, Tejun Heo wrote:
...
> > 3) What end state are you aiming for? Keep WQ_PERCPU as a thin flag over
> >    unified internals (single install path, per-cpu accounting special-cased)
> >    with WQ_UNBOUND staying for now -- or something more radical (the flags gone
> >    entirely, percpu purely an affinity value)?
> 
> Keeping WQ_PERCPU as a shorthand for specifying the percpu scope makes sense
> to me.

Oh, and please note that if a workqueue specifies WQ_PERCPU, it should be
confined to strict cpu affinity mode.

Thanks.

-- 
tejun

Re: [PATCH RFC 0/3] Refactor the workqueue allocations

[Breno Leitao]

On Thu, Jul 16, 2026 at 09:22:50AM -1000, Tejun Heo wrote:
> Hello,
> 
> On Tue, Jul 14, 2026 at 04:41:46AM -0700, Breno Leitao wrote:
> ...
> > 1) wqattrs is an unbound concept and apply_workqueue_attrs_locked() rejects
> >    non-unbound wqs. Should we leverage wqattrs in per-cpu workqueues as well,
> >    so the unification can happen later (apply_workqueue_attrs_locked())?
> 
> I'm not sure adding wqattrs to percpu workqueues makes sense. Wouldn't the
> shape more be like unbound workqueue subsuming percpu workqueue?

> > 2) If percpu becomes a WQ_AFFN_CPU affinity setting, how should max_active be
> >    treated? WQ_AFFN_CPU is unbound today, so it would inherit per-node
> >    accounting (wq_node_nr_active) and lose percpu's per-cpu max_active
> >    (pwq->nr_active).
> 
> I think it probably would be better to introduce a separate affinity scope
> than modifying WQ_AFFN_CPU. Something which indicates that concurrency
> management is in effect and max_active is per-cpu.
> 
> > 3) What end state are you aiming for? Keep WQ_PERCPU as a thin flag over
> >    unified internals (single install path, per-cpu accounting special-cased)
> >    with WQ_UNBOUND staying for now -- or something more radical (the flags gone
> >    entirely, percpu purely an affinity value)?
> 
> Keeping WQ_PERCPU as a shorthand for specifying the percpu scope makes sense
> to me.

Oh, now I see what you mean, I was heading the wrong way. Thanks for the
clarification, that makes total sense.

So: add a PERCPU wq_affn_scope and back it strictly per-CPU, rather than
reusing WQ_AFFN_CPU. Something like:

    enum wq_affn_scope {
            ...
  +         WQ_AFFN_PERCPU,         /* one pod per CPU, backed by the per-cpu pool */

and move the per-cpu workqueue users onto WQ_AFFN_PERCPU. With that, the
unbound install path (apply_wqattrs and the per-cpu, replaceable pwqs) can
point a pwq at a per-cpu pool, so one mechanism serves both. Then move
all the WQ_PERCPU users to WQ_AFFN_PERCPU, and eventually deprecate
WQ_PERCPU ?

I have this working as a prototype: WQ_PERCPU selects the scope and forces
strict affinity, and it boots with every percpu wq created through the
new path. 

A few things I'd like your read on:

    1) Percpu workqueues keep the WQ_PERCPU flag (I don't switch them to
       WQ_UNBOUND when they move onto the WQ_AFFN_PERCPU scope), so per-cpu
       accounting falls out of the existing !WQ_UNBOUND checks. do you
       have any preference here, or should percpu become purely an
       affn_scope value with accounting decoupled from the flag?

    2) What about WQ_BH? Can we keep it on the direct per-cpu path (softirq
       context) for now?

    3) Routing percpu through apply_wqattrs pulls in unbound-only assumptions
       (unbound_attrs allocation, and the CPU-hotplug fixups in
       workqueue_online_cpu()/workqueue_offline_cpu() that gate on
       unbound_attrs) that now have to learn about the percpu scope.

       Do you prefer teaching that shared path about WQ_AFFN_PERCPU, or would
       you rather percpu keep a lighter install path (closer to the current
       WQ_PERCPU direct path), if that's feasible?

Thanks for the guidance,
--breno