Re: [PATCH bpf-next v1 RESEND 1/5] vmalloc: introduce vmalloc_exec, vfree_exec, and vcopy_exec

[Mike Rapoport <rppt@kernel.org>]

On Thu, Nov 03, 2022 at 11:59:48AM -0700, Luis Chamberlain wrote:
> On Thu, Nov 03, 2022 at 05:51:57PM +0200, Mike Rapoport wrote:
> 
> > I had to put this project on a backburner for $VARIOUS_REASONS, but I still
> > think that we need a generic allocator for memory with non-default
> > permissions in the direct map and that code allocation should build on that
> > allocator.
> 
> It seems this generalization of the bpf prog pack to possibly be used
> for modules / kprobes / ftrace is a small step in that direction.
> 
> > All that said, the direct map fragmentation problem is currently relevant
> > only to x86 because it's the only architecture that supports splitting of
> > the large pages in the direct map.
> 
> I was thinking even more long term too, using this as a proof of concept. If
> this practice in general helps with fragmentation, could it be used for
> experimetnation with compound pages later, as a way to reduce possible
> fragmentation.

As Rick already mentioned, these patches help with the direct map
fragmentation only indirectly. With these patches memory is freed in
PMD_SIZE chunks and this makes the changes to the direct map in
vm_remove_mappings() to happen in in PMD_SIZE units and this is pretty much
the only effect of this series on the direct map layout.

A bit unrelated, but I'm wondering now if we want to have the direct map
alias of the pages allocated for code also to be read-only...

> > Whenever a large page in the direct map is split, all
> > kernel accesses via the direct map will use small pages which requires
> > dealing with 512 page table entries instead of one for 2M range.
> > 
> > Since small pages in the direct map are never collapsed back to large
> > pages, long living system that heavily uses eBPF programs will have its
> > direct map severely fragmented, higher TLB miss rate and worse overall
> > performance. 
> 
> Shouldn't compaction help with those situations?

Compaction helps to reduce fragmentation of the physical memory, it tries
to bring free physical pages next to each other to create large contiguous
chunks, but it does not change the virtual addresses the users of the
underlying data see.

Changing permissions of a small page in the direct map causes
"discontinuity" in the virtual space. E.g. if we have 2M mapped RW with a
single PMD changing several page in the middle of those 2M to R+X will
require to remap that range with 512 PTEs.
 
> Thanks!
> 
>   Luis

-- 
Sincerely yours,
Mike.