Re: [PATCH bpf-next v3 0/5] bpf, x86: enable EXECMEM_ROX_CACHE for BPF allocations

[Mike Rapoport <rppt@kernel.org>]

On Fri, Jul 17, 2026 at 10:50:09AM -0700, Song Liu wrote:
> On Fri, Jul 17, 2026 at 2:29 AM Mike Rapoport <rppt@kernel.org> wrote:
> [...]
> > > >
> > > > "It works, don't touch"? ;-)
> > > >
> > > > We can add another layer for sub-page allocations to execmem.
> > >
> > > Sub-page allocation is not a hard requirement here. Using 4kB for
> > > each small BPF program isn't too bad. We added bpf_prog_pack to
> > > avoid fragmentation of direct map page table entry (caused by W^X
> > > requirement). If execmem allocator reserves large enough ROX
> > > memory (with PMD page table entries) and reuses them properly,
> > > we shouldn't see page table fragmentation getting worse over time.
> > > Then, we can use 4kB granularity allocation for BPF programs. (I am
> > > not sure about 64kB pages..).
> >
> > execmem uses PMD_SIZE pages for ROX cache. Since right now it's only
> > supported on x86, the issue with oversized large page with 64k base pages
> > didn't come up yet.
> 
> With 64kB page size, PMD_SIZE is probably too big for execmem.

Yeah, I think so. But maybe on arm64 with 64kB pages we could use CONT_PTE.
 
> > > > Since BPF is the only user the easiest would be just to move prog_pack
> > > > logic from BPF to execmem and call it a day.
> > >
> > > If we move to bigger page sizes, say 64kB, there will be other
> > > users that would benefit from sub page allocation, right?
> >
> > Maybe modules could, don't know TBH.
> > There is another caveat with sub-page allocations: they must preserve ROX
> > and no subsystem except BPF can deal with ROX-only allocations and writable
> > copies.
> 
> I think ftrace and kprobe could be good users?

Don't know about ftrace, but kprobes essentially has page-sized caches and
uses them internally.
 
> > > > Another option is to add a slab-like layer for sub-page allocations to
> > > > execmem. This is more complex but it would allow to get rid of the rigid
> > > > BPF_PROG_CHUNK_SIZE.
> > > >
> > > > Maybe it would be also possible to teach SLUB to use execmem_alloc()
> > > > instead of alloc_pages() but that's surely the most far fetched one :)
> > >
> > > I was thinking some rb-tree algorithm might be useful here,
> > > something similar to vmap.
> >
> > Could be that or maple_tree especially considering there's an active
> > discussion about converting vmalloc to maple_tree as well:
> >
> > https://lore.kernel.org/linux-mm/20260613-vmalloc_maple-v1-0-0aa740bb944b@oss.qualcomm.com/
> >
> > But the choice of allocation algorithm is anyway secondary to the decision
> > whether execmem should have the sub-page allocator.
> >
> > If you say that a page per BPF program is fine, than maybe we can just rip
> > off bpf_prog_pack_alloc().
> > If BPF still needs to maintain the ability to allocate in smaller chunks,
> > then maybe the best way is to keep bpf_prog_pack_alloc() as is because I
> > don't foresee other users for small page allocations any time soon.
> >
> > > > And since we are talking about bpf_prog_pack_alloc(), why
> > > > BPF_PROG_PACK_SIZE accounts for num_possible_nodes():
> > > >
> > > > #define BPF_PROG_PACK_SIZE (SZ_2M * num_possible_nodes())
> > > >
> > > > Is it an elaborate choice or it was picked to work around older
> > > > vmalloc_huge() limitations?
> > >
> > > It is a bit complicated. The goal is to get PMDs for prog_pack.
> > > We can adjust this if vmalloc_huge() changes after that.
> >
> > Since commit c82be0be9576 ("mm: vmalloc: don't account for number of nodes
> > for HUGE_VMAP allocations") vmalloc_huge() is fine with PMD_SIZE regardless
> > of number of nodes.
> 
> Before execmem can handle sub page allocations, how about we
> send allocations that are bigger than page size directly to execmem,
> and let bpf_prog_pack handle sub page allocations. Then,
> BPF_PROG_PACK_SIZE will be PAGE_SIZE. This should be a net
> win for x86_64. Other archs will be the same. WDYT?

Makes perfect sense to me for x86.

On architectures that support large mappings in the direct map and do not
support EXECMEM_CACHE_ROX this will increase direct map fragmentation.
 
> Thanks,
> Song

-- 
Sincerely yours,
Mike.