BPF List
 help / color / mirror / Atom feed
* [PATCH bpf-next v5 0/8] libbpf: BPF program manual loading
@ 2026-09-23 23:29 Andrey Grodzovsky
  2026-09-23 23:29 ` [PATCH bpf-next v5 1/8] libbpf: BPF program load strategy enum Andrey Grodzovsky
                   ` (7 more replies)
  0 siblings, 8 replies; 27+ messages in thread
From: Andrey Grodzovsky @ 2026-09-23 23:29 UTC (permalink / raw)
  To: bpf, andrii, ast; +Cc: martin.kelly, slava.imameev, linux-open-source

This series was originally posted in January 2025 [1] as a two-patch RFC
introducing a per-program tri-state load strategy (disabled/auto/dynamic)
to let large BPF applications load a subset of their programs lazily,
after the initial bpf_object load. This v5 addresses v4's review
feedback and converts veristat's per-program verification from
bpf_program__clone() to this series' own manual-load API, closing the
loop on the open item from v2/v4 (see changelog below).


Motivation:

Security tools built on top of libbpf commonly ship as a single large
BPF object containing many programs, only a subset of which are needed
on any given system -- the rest are gated on optional features or on
kernel/runtime capabilities that vary across deployments. For this class
of application, per-program manual loading helps in two ways:

  - it shortens the initial load, since only the programs actually
    needed for the running configuration are loaded and attached up
    front instead of the whole object; and
  - it lets a single failing program be unloaded and possibly reloaded
    (or a feature toggled at runtime) without tearing down and reloading
    the entire bpf_object.

Both reduce the time window during which the tool is partially loaded or
inactive -- for a security tool specifically, that is also the time
window during which the system it protects is unprotected.

We have been running this with our internal fork of libbpf in production
for about a year. Depending on kernel and configuration support, around
250 of our programs are potentially loadable on any given system; with
manual loading, only around 90 of those are actually loaded by default,
growing to the full 250 only when every optional feature is enabled.
Cutting the default set of loaded programs from 250 down to 90 measurably
reduces load time (we observed ~60% reduction in our own measurements),
and since unload time scales with the number of loaded programs, it
correspondingly shortens unload time as well -- which matters, since
slow unload can delay system shutdown. We believe this functionality
would benefit other libbpf consumers with similarly large, modular BPF
applications.

Patch overview:

 1/8: replace bpf_program's boolean autoload field with an enum
      (bpf_prog_load_strategy: DISABLED/AUTO), so a third state can be
      added later without an ABI break

 2/8: add BPF_PROG_LOAD_STRATEGY_MANUAL and bpf_program__load()/
      bpf_program__unload(), letting a program be loaded and attached
      independently of the bulk bpf_object load/attach pass, and
      reloaded/reattached multiple times

 3/8: let a program mark itself MANUAL from its own section name via a
      SEC("!...") prefix, instead of requiring an imperative
      bpf_program__set_load_strategy() call, mirroring the existing
      SEC("?...") convention

 4/8: reject bpf_object__gen_loader() for an object with a MANUAL
      program, which would otherwise silently corrupt every later
      program's generated prog_fd slot

 5/8: version bpf_program__set_autoattach()'s void->int return-type and
      behavior change via ELF symbol versioning (COMPAT_VERSION()/
      DEFAULT_VERSION()), modeled on the xsk_umem__create and
      bpf_prog_load precedents, so binaries already linked against the
      old void-returning ABI keep that behavior instead of silently
      hitting the new MANUAL-rejection logic underneath them

 6/8: selftest covering every load-strategy transition
      (DISABLED/AUTO/MANUAL), bpf_program__set_autoload()'s bool
      compatibility, and autoattach restoration

 7/8: selftest covering the manual load/attach/detach/reload cycle, the
      declarative SEC("!...") marker, prepare()-only load sufficiency,
      a module BTF deferred load, and gen_loader rejection

 8/8: convert veristat's process_prog() from bpf_program__clone() to
      BPF_PROG_LOAD_STRATEGY_MANUAL + bpf_program__load()/
      bpf_program__unload(), the open item from the v2 and v4 threads

Changes since v4 [2]:
  - Fix a stray comment in bpf_object__init_prog() (bot+bpf-ci, sashiko-bot)
  - Check the previously-ignored return value of
    bpf_program__set_load_strategy() for prog1/prog2 in load_type.c,
    consistent with every other call to the same setter later in the
    same file (sashiko-bot)
  - New: convert veristat's process_prog() to use
    BPF_PROG_LOAD_STRATEGY_MANUAL/bpf_program__load()/
    bpf_program__unload() instead of bpf_program__clone(), RODATA maps get 
    bound on each per-program load the way clone() deliberately avoids, but 
    since each program is unloaded again immediately after verification, the
    binding is dropped along with it (Andrii, Alexei)


[1] https://lore.kernel.org/bpf/20250122215206.59859-1-slava.imameev@crowdstrike.com/
[2] https://lore.kernel.org/bpf/20260921223937.3203093-1-andrey.grodzovsky@crowdstrike.com/

Andrey Grodzovsky (5):
  libbpf: Support declarative manual load via SEC("!...") prefix
  libbpf: Reject gen_loader for objects with already-manual programs
  libbpf: Version bpf_program__set_autoattach() ABI change
  selftests/bpf: Cover BPF program manual loading
  selftests/bpf: Convert veristat to BPF_PROG_LOAD_STRATEGY_MANUAL

Slava Imameev (3):
  libbpf: BPF program load strategy enum
  libbpf: BPF programs manual loading and attaching
  selftests/bpf: Cover BPF program load strategy transitions

 tools/lib/bpf/libbpf.c                        | 281 +++++++++++---
 tools/lib/bpf/libbpf.h                        |  69 +++-
 tools/lib/bpf/libbpf.map                      |   5 +
 tools/lib/bpf/libbpf_common.h                 |   6 +
 .../selftests/bpf/prog_tests/dynamicload.c    | 365 ++++++++++++++++++
 .../selftests/bpf/prog_tests/load_type.c      | 190 +++++++++
 .../selftests/bpf/prog_tests/signed_loader.c  |  28 ++
 .../selftests/bpf/progs/test_dynamicload.c    |  54 +++
 .../selftests/bpf/progs/test_load_type.c      |  31 ++
 tools/testing/selftests/bpf/veristat.c        |  20 +-
 10 files changed, 990 insertions(+), 59 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/dynamicload.c
 create mode 100644 tools/testing/selftests/bpf/prog_tests/load_type.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_dynamicload.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_load_type.c

-- 
2.34.1


^ permalink raw reply	[flat|nested] 27+ messages in thread
end of thread, other threads:[~2026-09-30 15:50 UTC | newest]

Thread overview: 27+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-09-23 23:29 [PATCH bpf-next v5 0/8] libbpf: BPF program manual loading Andrey Grodzovsky
2026-09-23 23:29 ` [PATCH bpf-next v5 1/8] libbpf: BPF program load strategy enum Andrey Grodzovsky
2026-09-24  0:18   ` bot+bpf-ci
2026-09-23 23:29 ` [PATCH bpf-next v5 2/8] libbpf: BPF programs manual loading and attaching Andrey Grodzovsky
2026-09-23 23:43   ` sashiko-bot
2026-09-24  0:32   ` bot+bpf-ci
2026-09-28  1:40     ` Andrey Grodzovsky
2026-09-30 15:50       ` Andrii Nakryiko
2026-09-24 23:18   ` Andrii Nakryiko
2026-09-26 16:56     ` Andrey Grodzovsky
2026-09-23 23:29 ` [PATCH bpf-next v5 3/8] libbpf: Support declarative manual load via SEC("!...") prefix Andrey Grodzovsky
2026-09-24 23:18   ` Andrii Nakryiko
2026-09-23 23:29 ` [PATCH bpf-next v5 4/8] libbpf: Reject gen_loader for objects with already-manual programs Andrey Grodzovsky
2026-09-24 23:18   ` Andrii Nakryiko
2026-09-26 17:49     ` Andrey Grodzovsky
2026-09-23 23:29 ` [PATCH bpf-next v5 5/8] libbpf: Version bpf_program__set_autoattach() ABI change Andrey Grodzovsky
2026-09-23 23:41   ` sashiko-bot
2026-09-24  0:32   ` bot+bpf-ci
2026-09-24 23:19   ` Andrii Nakryiko
2026-09-26 18:04     ` Andrey Grodzovsky
2026-09-30 15:50       ` Andrii Nakryiko
2026-09-23 23:29 ` [PATCH bpf-next v5 6/8] selftests/bpf: Cover BPF program load strategy transitions Andrey Grodzovsky
2026-09-24  0:18   ` bot+bpf-ci
2026-09-23 23:29 ` [PATCH bpf-next v5 7/8] selftests/bpf: Cover BPF program manual loading Andrey Grodzovsky
2026-09-24  0:32   ` bot+bpf-ci
2026-09-23 23:29 ` [PATCH bpf-next v5 8/8] selftests/bpf: Convert veristat to BPF_PROG_LOAD_STRATEGY_MANUAL Andrey Grodzovsky
2026-09-24  0:32   ` bot+bpf-ci

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox