Re: [PATCH bpf-next v3 0/5] bpf, x86: enable EXECMEM_ROX_CACHE for BPF allocations

[Mike Rapoport <rppt@kernel.org>]

On Fri, Jul 17, 2026 at 12:27:03AM -0700, Song Liu wrote:
> On Thu, Jul 16, 2026 at 11:41 PM Mike Rapoport <rppt@kernel.org> wrote:
> > On Thu, Jul 16, 2026 at 05:00:11PM -0700, Song Liu wrote:
> > > On Thu, Jul 16, 2026 at 12:51 AM Mike Rapoport (Microsoft)
> > > <rppt@kernel.org> wrote:
> > > >
> > > > Hi,
> > > >
> > > > BPF allocations of executable memory on x86 are essentially read-only. Most
> > > > paths that call bpf_jit_alloc_exec() immediately make it ROX with
> > > > set_memory_rox().
> > > >
> > > > The code generation, at least on x86, uses separately allocated writable
> > > > buffers and then updates the actual text memory with text_poke().
> > > >
> > > > These patches do several small adjustments to how BPF allocates executable
> > > > memory and enable EXECMEM_ROX_CACHE for BPF allocations on x86.
> > >
> > > After this set, we are still using bpf_prog_pack_alloc() from x86 code. I think
> > > the goal is to eventually remove bpf_prog_pack_alloc(). What's our plan for
> > > the next steps (toward removing bpf_prog_pack_alloc)?
> >
> > "It works, don't touch"? ;-)
> >
> > We can add another layer for sub-page allocations to execmem.
> 
> Sub-page allocation is not a hard requirement here. Using 4kB for
> each small BPF program isn't too bad. We added bpf_prog_pack to
> avoid fragmentation of direct map page table entry (caused by W^X
> requirement). If execmem allocator reserves large enough ROX
> memory (with PMD page table entries) and reuses them properly,
> we shouldn't see page table fragmentation getting worse over time.
> Then, we can use 4kB granularity allocation for BPF programs. (I am
> not sure about 64kB pages..).

execmem uses PMD_SIZE pages for ROX cache. Since right now it's only
supported on x86, the issue with oversized large page with 64k base pages
didn't come up yet.
 
> > Since BPF is the only user the easiest would be just to move prog_pack
> > logic from BPF to execmem and call it a day.
> 
> If we move to bigger page sizes, say 64kB, there will be other
> users that would benefit from sub page allocation, right?

Maybe modules could, don't know TBH.
There is another caveat with sub-page allocations: they must preserve ROX
and no subsystem except BPF can deal with ROX-only allocations and writable
copies.
 
> > Another option is to add a slab-like layer for sub-page allocations to
> > execmem. This is more complex but it would allow to get rid of the rigid
> > BPF_PROG_CHUNK_SIZE.
> >
> > Maybe it would be also possible to teach SLUB to use execmem_alloc()
> > instead of alloc_pages() but that's surely the most far fetched one :)
> 
> I was thinking some rb-tree algorithm might be useful here,
> something similar to vmap.

Could be that or maple_tree especially considering there's an active
discussion about converting vmalloc to maple_tree as well:

https://lore.kernel.org/linux-mm/20260613-vmalloc_maple-v1-0-0aa740bb944b@oss.qualcomm.com/

But the choice of allocation algorithm is anyway secondary to the decision
whether execmem should have the sub-page allocator.

If you say that a page per BPF program is fine, than maybe we can just rip
off bpf_prog_pack_alloc().
If BPF still needs to maintain the ability to allocate in smaller chunks,
then maybe the best way is to keep bpf_prog_pack_alloc() as is because I
don't foresee other users for small page allocations any time soon.
 
> > And since we are talking about bpf_prog_pack_alloc(), why
> > BPF_PROG_PACK_SIZE accounts for num_possible_nodes():
> >
> > #define BPF_PROG_PACK_SIZE (SZ_2M * num_possible_nodes())
> >
> > Is it an elaborate choice or it was picked to work around older
> > vmalloc_huge() limitations?
> 
> It is a bit complicated. The goal is to get PMDs for prog_pack.
> We can adjust this if vmalloc_huge() changes after that.

Since commit c82be0be9576 ("mm: vmalloc: don't account for number of nodes
for HUGE_VMAP allocations") vmalloc_huge() is fine with PMD_SIZE regardless
of number of nodes.

With 64k pages it's still a lot though.
 
> Thanks,
> Song

-- 
Sincerely yours,
Mike.