BPF List
 help / color / mirror / Atom feed
* [PATCH bpf-next v3 0/6] libbpf: BPF program manual loading
@ 2026-09-17 20:30 Andrey Grodzovsky
  2026-09-17 20:30 ` [PATCH bpf-next v3 1/6] libbpf: BPF program load strategy enum Andrey Grodzovsky
                   ` (5 more replies)
  0 siblings, 6 replies; 16+ messages in thread
From: Andrey Grodzovsky @ 2026-09-17 20:30 UTC (permalink / raw)
  To: bpf, andrii; +Cc: ast, martin.kelly, slava.imameev, linux-open-source

This series was originally posted in January 2025 [1] as a two-patch RFC
introducing a per-program tri-state load strategy (disabled/auto/dynamic)
to let large BPF applications load a subset of their programs lazily,
after the initial bpf_object load. v2 [2] kept the same per-program enum
and added a bool-compatible bpf_program__set_autoload() shim plus a
declarative BTF decl_tag marker. This v3 renames DYNAMIC to MANUAL,
replaces the decl_tag marker with a SEC("!...") prefix, relaxes the
load gate to right after bpf_object__prepare(), adds support for
kernel module attach targets, and fixes bugs and addresses comments
identified in v2's review (see the changelog below).


Motivation:

Security tools built on top of libbpf commonly ship as a single large
BPF object containing many programs, only a subset of which are needed
on any given system -- the rest are gated on optional features or on
kernel/runtime capabilities that vary across deployments. For this class
of application, per-program manual loading helps in two ways:

  - it shortens the initial load, since only the programs actually
    needed for the running configuration are loaded and attached up
    front instead of the whole object; and
  - it lets a single failing program be unloaded and possibly reloaded
    (or a feature toggled at runtime) without tearing down and reloading
    the entire bpf_object.

Both reduce the time window during which the tool is partially loaded or
inactive -- for a security tool specifically, that is also the time
window during which the system it protects is unprotected.

We have been running this with our internal fork of libbpf in production
for about a year. Depending on kernel and configuration support, around
250 of our programs are potentially loadable on any given system; with
manual loading, only around 90 of those are actually loaded by default,
growing to the full 250 only when every optional feature is enabled.
Cutting the default set of loaded programs from 250 down to 90 measurably
reduces load time (we observed ~60% reduction in our own measurements),
and since unload time scales with the number of loaded programs, it
correspondingly shortens unload time as well -- which matters, since
slow unload can delay system shutdown. We believe this functionality
would benefit other libbpf consumers with similarly large, modular BPF
applications.

Patch overview:

 1/6: replace bpf_program's boolean autoload field with an enum
      (bpf_prog_load_strategy: DISABLED/AUTO), so a third state can be
      added later without an ABI break

 2/6: add BPF_PROG_LOAD_STRATEGY_MANUAL and
      bpf_program__load_manually()/unload_manually(), letting a program
      be loaded and attached independently of the bulk bpf_object
      load/attach pass, and reloaded/reattached multiple times

 3/6: let a program mark itself MANUAL from its own section name via a
      SEC("!...") prefix, instead of requiring an imperative
      bpf_program__set_load_strategy() call, mirroring the existing
      SEC("?...") convention

 4/6: reject bpf_object__gen_loader() for an object with a MANUAL
      program, which would otherwise silently corrupt every later
      program's generated prog_fd slot

 5/6: selftest covering every load-strategy transition
      (DISABLED/AUTO/MANUAL), bpf_program__set_autoload()'s bool
      compatibility, and autoattach restoration

 6/6: selftest covering the manual load/attach/detach/reload cycle, the
      declarative SEC("!...") marker, prepare()-only load sufficiency,
      a module BTF deferred load, and gen_loader rejection

Changes since v2 [2]:
  - Rename load_type -> load_strategy, DYNAMIC -> MANUAL throughout the
    API surface (Andrii)
  - Move the declarative marker from a BTF decl_tag to a SEC("!...")
    prefix (Andrii)
  - Relax bpf_program__load_manually()'s gate from OBJ_LOADED to
    OBJ_PREPARED (Andrii)
  - Compute has_manual_progs upfront in bpf_object_prepare_progs()
    instead of tracking it incrementally (Andrii)
  - Drop the bool->enum conversion of bpf_program__set_autoload()
    entirely; it stays bool-signature and becomes a thin, unconditional
    forwarder to set_load_strategy() (Andrii: "let's just leave it be")
  - Fix set_load_strategy()'s DYNAMIC->AUTO/DISABLED transition never
    restoring prog->autoattach (bot+bpf-ci)
  - Drop the premature SEC_ATTACH_BTF early-reject in
    set_load_strategy(MANUAL) (bot+bpf-ci and Andrii independently)
  - Reject gen_loader for objects with an already-MANUAL program: a
    MANUAL program is invisible to gen_loader's program count, silently
    shifting every later program's generated prog_fd slot (bot+bpf-ci)
  - Reject load_manually() with -EBUSY when prog->fd is already open,
    instead of silently overwriting it and leaking the previous fd
    on repeat load without an intervening unload (sashiko-bot)
  - Reject bpf_program__set_autoattach() for a MANUAL-strategy program
    (its signature changes from void to int accordingly), since setting
    autoattach directly would corrupt the value saved_autoattach
    restores on the next transition back to AUTO/DISABLED
  - Wrap load_manually()'s raw error via libbpf_err() and fix a NULL-
    check ordering bug (sashiko-bot); drop a redundant !obj check --
    prog->obj is an invariant elsewhere in libbpf.c (Andrii)
  - Drop the extra_log_level parameter from load_manually(); callers
    already have bpf_program__set_log_level() (Andrii)
  - Expand selftest coverage: a prepare()-only load-sufficiency test
    with attach + execution verification (closes the gap behind
    Andrii's OBJ_PREPARED question above), a module BTF deferred-load
    regression test, and a gen_loader-rejection test

[1] https://lore.kernel.org/bpf/20250122215206.59859-1-slava.imameev@crowdstrike.com/
[2] https://lore.kernel.org/bpf/20260826181825.1826541-1-andrey.grodzovsky@crowdstrike.com/

Andrey Grodzovsky (3):
  libbpf: Support declarative manual load via SEC("!...") prefix
  libbpf: Reject gen_loader for objects with already-manual programs
  selftests/bpf: Cover BPF program manual loading

Slava Imameev (3):
  libbpf: BPF program load strategy enum
  libbpf: BPF programs manual loading and attaching
  selftests/bpf: Cover BPF program load strategy transitions

 tools/lib/bpf/libbpf.c                        | 234 +++++++++---
 tools/lib/bpf/libbpf.h                        |  40 +-
 tools/lib/bpf/libbpf.map                      |   4 +
 .../selftests/bpf/prog_tests/dynamicload.c    | 350 ++++++++++++++++++
 .../selftests/bpf/prog_tests/load_type.c      | 157 ++++++++
 .../selftests/bpf/prog_tests/signed_loader.c  |  28 ++
 .../selftests/bpf/progs/test_dynamicload.c    |  54 +++
 .../selftests/bpf/progs/test_load_type.c      |  31 ++
 8 files changed, 854 insertions(+), 44 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/dynamicload.c
 create mode 100644 tools/testing/selftests/bpf/prog_tests/load_type.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_dynamicload.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_load_type.c

-- 
2.34.1


^ permalink raw reply	[flat|nested] 16+ messages in thread
end of thread, other threads:[~2026-09-17 21:34 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-09-17 20:30 [PATCH bpf-next v3 0/6] libbpf: BPF program manual loading Andrey Grodzovsky
2026-09-17 20:30 ` [PATCH bpf-next v3 1/6] libbpf: BPF program load strategy enum Andrey Grodzovsky
2026-09-17 21:16   ` bot+bpf-ci
2026-09-17 20:30 ` [PATCH bpf-next v3 2/6] libbpf: BPF programs manual loading and attaching Andrey Grodzovsky
2026-09-17 20:41   ` sashiko-bot
2026-09-17 21:34   ` bot+bpf-ci
2026-09-17 20:30 ` [PATCH bpf-next v3 3/6] libbpf: Support declarative manual load via SEC("!...") prefix Andrey Grodzovsky
2026-09-17 21:34   ` bot+bpf-ci
2026-09-17 20:30 ` [PATCH bpf-next v3 4/6] libbpf: Reject gen_loader for objects with already-manual programs Andrey Grodzovsky
2026-09-17 21:16   ` bot+bpf-ci
2026-09-17 20:30 ` [PATCH bpf-next v3 5/6] selftests/bpf: Cover BPF program load strategy transitions Andrey Grodzovsky
2026-09-17 20:39   ` sashiko-bot
2026-09-17 21:34   ` bot+bpf-ci
2026-09-17 20:30 ` [PATCH bpf-next v3 6/6] selftests/bpf: Cover BPF program manual loading Andrey Grodzovsky
2026-09-17 20:40   ` sashiko-bot
2026-09-17 21:34   ` bot+bpf-ci

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox