Re: [PATCH] mm/percpu: allow embed allocator when total size fits in vmalloc

[Dennis Zhou <dennis@kernel.org>]

Hello Paul,

On Sat, Jul 18, 2026 at 10:59:28PM -0700, Paul Sherman wrote:
> Andrew,
> 
> Thank you for the encouraging words on the initial submission.
> 
> Tejun,
> 
> Further testing on SG2042 (Sv39, 4 NUMA nodes, 128GB) showed that
> the proposed change allows the embedded first chunk to be created
> successfully, but later dynamic chunk population consistently fails
> in pcpu_populate_chunk(), with warnings originating from
> __vmap_pages_range_noflush(). Reverting the patch eliminates those
> warnings on the same hardware.
> 
> Looking through the code, my current understanding is that dynamic
> chunks inherit the first chunk's layout through the shared
> pcpu_unit_offsets[] table, while vmalloc reservations are driven by
> pcpu_group_offsets[]. That led me to wonder whether I've encountered
> a fundamental allocator invariant rather than merely an implementation
> detail.
> 
> Is the requirement that all chunks share a single pcpu_unit_offsets[]
> layout fundamental to the percpu allocator design, or is there an
> intended mechanism for decoupling the embedded first-chunk layout from
> dynamic chunk vmalloc mappings?
> 

Percpu memory is essentially an offset-based virtual address space. We
build the unit and group offsets when setting up the first chunk at
boot, and those offsets remain fixed for the lifetime of the machine.
Consequently, every later chunk must reproduce the same relative virtual
layout.

The embedded first chunk works because its sparse addresses already
exist in the linear map. However, pcpu_setup_first_chunk() saves that
layout in pcpu_group_offsets, which is subsequently passed to
pcpu_get_vm_areas() when creating dynamic chunks. On this machine the
resulting span is larger than the vmalloc area, so allowing the embedded
setup to succeed means that new dynamic chunks will fail once the first
chunk is exhausted.

Machines have grown considerably since this design was introduced.
Supporting an embedded layout like this would probably require a purely
direct-mapped backend, with congruent physical ranges reserved at boot
for future chunks. That may be reasonable for a particular machine if
percpu growth can be bounded and the reservation is small relative to
available memory, but I don´t think it generalizes based solely on the
ratio between chunk size and total memory. We would also need to
guarantee the required physical placement and decide how much space to
reserve for lifetime percpu growth.

As is, this patch isn't correct, but it might inspire new work.

Thanks,
Dennis

> The answer will help determine whether the correct next step is to
> withdraw the patch and instead document the existing invariant, or
> whether there's a larger design direction worth investigating.
> 
> Thanks,
> Paul