Re: [PATCH v2 13/13] percpu: queue the balance work on an unbound workqueue

[Balbir Singh <balbirs@nvidia.com>]

On Fri, Nov 13, 2026 at 09:45:31PM +0000, Lorenzo Stoakes (ARM) wrote:
> From: Chris Mason <mason@kernel.org>
> 
> A burst of small atomic per-CPU allocations can fail with -ENOMEM while the
> machine has free memory and idle CPUs, as atomic per-CPU allocations are
> only served from pages that are already populated, and those can be used up
> before a worker gets to populate more.
> 
> Consider:
> 
>     caller on CPU 1                     kworker bound to CPU 1
>     ===============                     ======================
>     allocates 8 bytes, which leaves
>     one empty page and 4088 bytes
>       schedule_work()                   woken, waits for CPU 1
>     allocates 8 bytes, 1023 times
>     next allocation: "no space left"
>     gives up CPU 1                      populates pages
> 
> The allocator keeps a few empty populated pages in reserve, and the work
> item pcpu_balance_work refills them.
> 
> pcpu_alloc_noprof() queues the work item using schedule_work(), so only a
> worker bound to the CPU of the caller can run the work item.
> 
> The issue is that the caller keeps that CPU - a task keeps it for the rest
> of its time slice, and a softirq keeps it until the softirq ends.
> 
> In that time the caller can use up the reserve, resulting in an atomic
> per-CPU allocation failing.
> 
> Fix the issue by queueing the work item on an unbound workqueue so that
> another CPU can populate pages while the caller keeps allocating.
> 
> Gregory Price hammered atomic per-CPU allocations from one CPU with
> preemption disabled for about a second, with and without this series:
> 
>                            Before     After
>     Successful allocs       1,188   131,072
>     Failed allocs           4,097         0
>     Balance worker runs         0       169
>     Successful retries          0     4,096
> 
> Fixes: 1a4d76076cda ("percpu: implement asynchronous chunk population")
> Signed-off-by: Chris Mason <mason@kernel.org>
> [ ljs: tweaked commit message, comment ]
> Tested-by: Gregory Price (Meta) <gourry@gourry.net>
> Reviewed-by: Gregory Price (Meta) <gourry@gourry.net>
> Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
> ---
>  mm/percpu.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/mm/percpu.c b/mm/percpu.c
> index f87813e6ad73..39cba1710383 100644
> --- a/mm/percpu.c
> +++ b/mm/percpu.c
> @@ -200,8 +200,9 @@ static bool pcpu_atomic_alloc_failed;
>  
>  static void pcpu_schedule_balance_work(void)
>  {
> +	/* Unbound, so another CPU can populate pgs if caller keeps this one. */
>  	if (pcpu_async_enabled)
> -		schedule_work(&pcpu_balance_work);
> +		queue_work(system_dfl_wq, &pcpu_balance_work);
>  }
>

Can this run havoc on otherwise idle systems? Why not choose a workqueue
with WQ_POWER_EFFICIENT, which becomes unbound when nohz_full is
enabled? I guess it still leaves open the case when nohz_full is not
enabled and we get a burst of atomic allocations?

Balbir