[PATCH bpf-next 0/2] bpf: Populate mmap-able array maps lazily

[Song Liu <song@kernel.org>]

An mmap-able BPF array map (BPF_F_MMAPABLE) has its backing memory
vmalloc'ed up front at map creation time. array_map_mmap() then wires up
the whole mapping eagerly via remap_vmalloc_range(), which calls
vm_insert_page() for every page of the map. This makes every mmap() cost
O(number of pages): an 8MiB map inserts 2048 PTEs on each mmap() and
tears them all down again on munmap(), even when user space only touches
a few pages (or none at all).

This series makes the mapping populate lazily instead, the same way the
arena map already does, turning mmap()/munmap() of an mmap-able array
from O(map size) into O(1). The pages are inserted on demand by a new
map_mmap_fault() callback, dispatched from the shared
bpf_map_default_vmops so that the existing VMA open/close accounting
(VM_MAYWRITE write-active tracking, freeze handling) stays centralized.

Callers that want the pages populated up front can still request that
explicitly with MAP_POPULATE, matching regular file semantics.
Kernel-side access to the map (via the vmalloc address) is unaffected.

Patch 1 implements the lazy faulting. Patch 2 adds an "arraymap-mmap"
benchmark that mmap()s/munmap()s an mmap-able array map from a
configurable number of threads, useful to observe how the cost scales
with the mapping size.

Benchmark results:

Before the patch
   nr_threads: 1, map_size: 1048576
arraymap-mmap: throughput: 0.034 +/- 0.000 M ops/s, latency: 29134.211 ns/op
   nr_threads: 1, map_size: 8388608
arraymap-mmap: throughput: 0.005 +/- 0.000 M ops/s, latency: 220365.807 ns/op
       nr_threads: 1, map_size: 67108864
arraymap-mmap: throughput: 0.001 +/- 0.000 M ops/s, latency: 1749642.119 ns/op

After the patch
    nr_threads: 1, map_size: 1048576
arraymap-mmap: throughput: 0.819 +/- 0.002 M ops/s, latency: 1221.476 ns/op
    nr_threads: 1, map_size: 8388608
arraymap-mmap: throughput: 0.808 +/- 0.005 M ops/s, latency: 1236.975 ns/op
    nr_threads: 1, map_size: 67108864
arraymap-mmap: throughput: 0.738 +/- 0.003 M ops/s, latency: 1355.100 ns/op

Song Liu (2):
  bpf: Populate mmap-able array map memory lazily
  selftests/bpf: Add mmap/munmap benchmark for array maps

 include/linux/bpf.h                           |   1 +
 kernel/bpf/arraymap.c                         |  40 ++++-
 kernel/bpf/syscall.c                          |  17 ++
 tools/testing/selftests/bpf/Makefile          |   1 +
 tools/testing/selftests/bpf/bench.c           |   4 +
 .../bpf/benchs/bench_arraymap_mmap.c          | 146 ++++++++++++++++++
 .../bpf/benchs/run_bench_arraymap_mmap.sh     |  15 ++
 7 files changed, 221 insertions(+), 3 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/benchs/bench_arraymap_mmap.c
 create mode 100755 tools/testing/selftests/bpf/benchs/run_bench_arraymap_mmap.sh

--
2.53.0-Meta