All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH bpf-next v6 0/7] libbpf: BPF program manual loading
@ 2026-10-10 14:31 Andrey Grodzovsky
  2026-10-10 14:31 ` [PATCH bpf-next v6 1/7] libbpf: Reject gen_loader set after PREPARE step Andrey Grodzovsky
                   ` (6 more replies)
  0 siblings, 7 replies; 12+ messages in thread
From: Andrey Grodzovsky @ 2026-10-10 14:31 UTC (permalink / raw)
  To: bpf, andrii, ast; +Cc: martin.kelly, slava.imameev

This series was originally posted in January 2025 [1] as a two-patch RFC
introducing a per-program tri-state load strategy (disabled/auto/manual)
to let large BPF applications load a subset of their programs lazily,
independent of the initial bpf_object load. This v6 addresses v5's review
feedback.

Motivation:

Security tools built on top of libbpf commonly ship as a single large
BPF object containing many programs, only a subset of which are needed
on any given system -- the rest are gated on optional features or on
kernel/runtime capabilities that vary across deployments. For this class
of application, per-program manual loading helps in two ways:

  - it shortens the initial load, since only the programs actually
    needed for the running configuration are loaded and attached up
    front instead of the whole object; and
  - it lets a single failing program be unloaded and possibly reloaded
    (or a feature toggled at runtime) without tearing down and reloading
    the entire bpf_object.

Both reduce the time window during which the tool is partially loaded or
inactive -- for a security tool specifically, that is also the time
window during which the system it protects is left vulnerable.

We have been running this with our internal fork of libbpf in production
for about a year. Depending on kernel and configuration support, around
250 of our programs are potentially loadable on any given system; with
manual loading, only around 90 of those are actually loaded by default,
growing to the full 250 only when every optional feature is enabled.
Cutting the default set of loaded programs from 250 down to 90 measurably
reduces load time (we observed ~60% reduction in our own measurements),
and since unload time scales with the number of loaded programs, it
correspondingly shortens unload time as well -- which matters, since
slow unload can delay system shutdown.

We believe this functionality would benefit other libbpf consumers with
similarly large, modular BPF applications.

Patch overview:

 1/7: reject bpf_object__gen_loader() once the object is PREPARED, since
      the generated loader records its steps during prepare

 2/7: replace bpf_program's boolean autoload field with an enum
      (bpf_prog_load_strategy: DISABLED/AUTO), so a third state can be
      added - preparatory commit with no functional changes

 3/7: add BPF_PROG_LOAD_STRATEGY_MANUAL and bpf_program__load()/
      bpf_program__unload(), letting a program be loaded and attached
      independently of the bulk bpf_object load/attach pass, and
      reloaded/reattached multiple times. This also covers the
      interactions with gen_loader, prog array slots, struct_ops and the
      per-program log setters

 4/7: let a program mark itself MANUAL from its own section name via a
      SEC("!...") prefix, mirroring the existing SEC("?...") convention

 5/7: selftest covering every load-strategy transition
      (DISABLED/AUTO/MANUAL) and bpf_program__set_autoload()'s bool
      compatibility

 6/7: selftest covering the manual load/attach/detach/reload cycle, the
      declarative SEC("!...") marker, prepare()-only load sufficiency,
      a module BTF deferred load, and gen_loader rejection

 7/7: convert veristat to use manual programs instead of cloning

Changes since v5 [2]:
  - Drop the bpf_program__set_autoattach() ABI versioning (Andrii)
  - Restore autoattach orthogonality to the load strategy: it is no longer
    saved and restored on strategy set (Andrii)
  - Reject setting gen_loader once the object is PREPARED, and detect
    the MANUAL program vs gen_loader conflict once in bpf_object_prepare()
    instead of in the setters (Andrii)
  - Simplify bpf_object_cleanup_btf() to a single early return (Andrii)
  - Reject a MANUAL program that is a prog array slot target but has not
    been loaded yet (sashiko-bot)
  - Relax the per-program log level and log buffer setters for MANUAL
    programs to work post OBJ_LOADED state so veristat does not lose the
    verifier log when bpf_object__prepare() fails (bot+bpf-ci)
  - Allow MANUAL mode on struct_ops programs through the imperative API so
    veristat can keep verifying them one by one while still rejecting the
    declarative SEC("!...") form (bot+bpf-ci)
  - Various cosmetic cleanups - comment clean-up and rewording, file
    renames to more correctly reflect their content

[1] https://lore.kernel.org/bpf/20250122215206.59859-1-slava.imameev@crowdstrike.com/
[2] https://lore.kernel.org/bpf/20260923232939.1886328-1-andrey.grodzovsky@crowdstrike.com/

Andrey Grodzovsky (4):
  libbpf: Reject gen_loader set after PREPARE step.
  libbpf: Support declarative manual load via SEC("!...") prefix
  selftests/bpf: Cover BPF program manual loading
  selftests/bpf: Convert veristat to BPF_PROG_LOAD_STRATEGY_MANUAL

Slava Imameev (3):
  libbpf: BPF program load strategy enum
  libbpf: BPF programs manual loading and attaching
  selftests/bpf: Cover BPF program load strategy transitions

 tools/lib/bpf/libbpf.c                        | 278 +++++++++++++--
 tools/lib/bpf/libbpf.h                        |  67 +++-
 tools/lib/bpf/libbpf.map                      |   3 +
 .../selftests/bpf/prog_tests/load_type.c      | 154 ++++++++
 .../selftests/bpf/prog_tests/manual_load.c    | 336 ++++++++++++++++++
 .../selftests/bpf/prog_tests/signed_loader.c  |  32 ++
 .../selftests/bpf/progs/test_load_type.c      |  31 ++
 .../selftests/bpf/progs/test_manual_load.c    |  54 +++
 tools/testing/selftests/bpf/veristat.c        |  20 +-
 9 files changed, 923 insertions(+), 52 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/load_type.c
 create mode 100644 tools/testing/selftests/bpf/prog_tests/manual_load.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_load_type.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_manual_load.c

base-commit: 15b578b1715d9a318c804350d98af87c203672bc
-- 
2.34.1


^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2026-10-10 15:21 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-10-10 14:31 [PATCH bpf-next v6 0/7] libbpf: BPF program manual loading Andrey Grodzovsky
2026-10-10 14:31 ` [PATCH bpf-next v6 1/7] libbpf: Reject gen_loader set after PREPARE step Andrey Grodzovsky
2026-10-10 15:21   ` bot+bpf-ci
2026-10-10 14:31 ` [PATCH bpf-next v6 2/7] libbpf: BPF program load strategy enum Andrey Grodzovsky
2026-10-10 14:31 ` [PATCH bpf-next v6 3/7] libbpf: BPF programs manual loading and attaching Andrey Grodzovsky
2026-10-10 14:31 ` [PATCH bpf-next v6 4/7] libbpf: Support declarative manual load via SEC("!...") prefix Andrey Grodzovsky
2026-10-10 15:06   ` sashiko-bot
2026-10-10 14:31 ` [PATCH bpf-next v6 5/7] selftests/bpf: Cover BPF program load strategy transitions Andrey Grodzovsky
2026-10-10 14:31 ` [PATCH bpf-next v6 6/7] selftests/bpf: Cover BPF program manual loading Andrey Grodzovsky
2026-10-10 15:03   ` sashiko-bot
2026-10-10 14:31 ` [PATCH bpf-next v6 7/7] selftests/bpf: Convert veristat to BPF_PROG_LOAD_STRATEGY_MANUAL Andrey Grodzovsky
2026-10-10 15:08   ` sashiko-bot

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.