[LSF/MM/BPF TOPIC] Amortising direct map fragmentation

[Mike Rapoport <rppt@kernel.org>]

Hi all,

There are use-cases that need to remove pages from the direct map or at least
map them at PTE level. These use-cases include vfree, module loading, ftrace,
kprobe, BPF, secretmem and generally any caller of set_memory/set_direct_map
APIs.

Remapping pages at PTE level causes split of the PUD and PMD sized mappings
in the direct map which leads to performance degradation.

To reduce the performance hit caused by the fragmentation of the direct
map, it makes sense to group and/or cache the base pages removed from the
direct map so that the most of base pages created during a split of a large
page will be consumed by users requiring PTE level mappings.

There were several RFC postings to address this:
* grouped page allocations for vmalloc permissions [1] and for PKS protection
  of page tables [2]
* pool of large pages in secretmem [3]
* global cache of pte-mapped pages close to the page allocator [4]
* a new migrate type to cache pte-mapped pages in free lists [5]

I'd like to discuss the direct map fragmentation issue and the possible
ways to reduce the fragmentation and performance hit caused by it.

[1] https://lore.kernel.org/lkml/20210405203711.1095940-1-rick.p.edgecombe@intel.com
[2] https://lore.kernel.org/lkml/20210505003032.489164-1-rick.p.edgecombe@intel.com
[3] https://lore.kernel.org/lkml/20210121122723.3446-8-rppt@kernel.org
[4] https://lore.kernel.org/all/20210823132513.15836-1-rppt@kernel.org
[5] https://lore.kernel.org/all/20220127085608.306306-1-rppt@kernel.org

-- 
Sincerely yours,
Mike.