All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH bpf-next v6 00/12] bpf: Introduce global percpu data
@ 2026-06-15 15:26 Leon Hwang
  2026-06-15 15:26 ` [PATCH bpf-next v6 01/12] bpf: Drop duplicate blank lines in verifier Leon Hwang
                   ` (11 more replies)
  0 siblings, 12 replies; 16+ messages in thread
From: Leon Hwang @ 2026-06-15 15:26 UTC (permalink / raw)
  To: bpf
  Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Eduard Zingerman, Kumar Kartikeya Dwivedi,
	Song Liu, Yonghong Song, Jiri Olsa, John Fastabend,
	Quentin Monnet, Shuah Khan, Leon Hwang, linux-kernel,
	linux-kselftest, kernel-patches-bot

This patch set introduces global percpu data, similar to commit
6316f78306c1 ("Merge branch 'support-global-data'"), to reduce restrictions
in C for BPF programs.

With this enhancement, it becomes possible to define and use global percpu
variables, like the DEFINE_PER_CPU() macro in the kernel
include/linux/percpu-defs.h.

The section name for global peurcpu data is ".percpu". Even though, a one-byte
percpu variable (e.g., char run SEC(".percpu") = 0;) can trigger a crash
with Clang 17 [1], users are expected to use such small variables as global
percpu data with newer Clang versions, which don't have the issue.

The idea stems from the bpfsnoop [2], which itself was inspired by
retsnoop [3]. During testing of bpfsnoop on the v6.6 kernel, two LBR
(Last Branch Record) entries were observed related to the
bpf_get_smp_processor_id() helper.

Since commit 1ae6921009e5 ("bpf: inline bpf_get_smp_processor_id() helper"),
the bpf_get_smp_processor_id() helper has been inlined on x86_64, reducing
the overhead and consequently minimizing these two LBR records.

However, the introduction of global percpu data offers a more robust
solution. By leveraging the percpu_array map and percpu instruction,
global percpu data can be implemented intrinsically.

This feature also facilitates sharing percpu information between tail
callers and callees or between freplace callers and callees through a
shared global percpu variable. Previously, this was achieved using a
1-entry percpu_array map, which this patch set aims to improve upon.

Links:
[1] https://lore.kernel.org/bpf/fd1b3f58-c27f-403d-ad99-644b7d06ecb3@linux.dev/
[2] https://github.com/bpfsnoop/bpfsnoop
[3] https://github.com/anakryiko/retsnoop

Changes:
v5 -> v6:
* Prevent running user addr_space_cast and addr_percpu insns in
  interpreter. (sashiko-bot)
* Cast __percpu pointer to u64 with (__force unsigned long). (lkp)
* Exclude BPF_MAP_TYPE_PERCPU_ARRAY in check_mem_access() before calling
  bpf_map_direct_read(), and add a test to verify it.
  (sashiko-bot, bot+bpf-ci)
* Skip percpu data variables for subskeleton in bpftool. (sashiko-bot)
* Protect skel->percpu using mprotect(..., PROT_READ) in light skeleton.
  (sashiko-bot, bot+bpf-ci)
* Drop roundup() in tests. (sashiko-bot)
* Call test_global_percpu_data_verifier_log() without
  test__start_subtest(). (sashiko-bot)
* Cast insn->imm to __u64 with (__u32) in xlated test. (sashiko-bot)
* Check cnt using the new idx in xlated test. (sashiko-bot)
* v5: https://lore.kernel.org/bpf/20260608145113.65857-1-leon.hwang@linux.dev/

v4 -> v5:
* Add prog->jit_requested check to prevent running percpu data in
  interpreter in patch #1.
* Factor out verifier log tests using its own patch.
* Address comments from Alexei:
  * Move map_type check from check_mem_access() to bpf_map_direct_read()
    in patch #2.
  * Move BPF_MAP_TYPE_INSN_ARRAY map_type check from const_reg_xfer() to
    bpf_map_direct_read() in patch #2.
  * Add a test to verify that the off of xlated ldimm64 insn matches the
    off encoded in the ELF ldimm64 insn.
  * Drop patch #5 of v4.
* Address reviews from sashiko-bot:
  * Update commit message of patch #6 to indicate that maps.percpu->mmaped
    has been marked as read-only in libbpf.
  * Lookup elem on specified CPU using BPF_F_CPU in tests.
  * Drop unnecessary err == -EOPNOTSUPP in test.
  * Locate target field using its offset in the iter test.
* v4: https://lore.kernel.org/bpf/20260414132421.63409-1-leon.hwang@linux.dev/

v3 -> v4:
* Drop duplicate blank lines in verifier.
* Add percpu data feature probe in libbpf.
* Update percpu_array map using BPF_F_ALL_CPUS flag for lskel, if no cpu flag
  is set.
* Add two tests to verify verifier log.
* Add a test to verify mov64_percpu_reg instruction.
* Add a test to verify bpf_iter for percpu data map.
* Update percpu_array map using BPF_F_ALL_CPUS flag in libbpf
  (per Alexei and Andrii).
* Address comments from Andrii:
  * Use .percpu as section identifier.
  * Use bpf_jit_supports_percpu_insn() instead of CONFIG_SMP.
  * Drop bpf_map__is_internal_percpu() API.
  * Drop unnecessary __aligned(8) in libbpf, verified by selftest.
  * Make mmap data read-only after loading prog.
v3: https://lore.kernel.org/bpf/20250526162146.24429-1-leon.hwang@linux.dev/

v2 -> v3:
  * Use ".data..percpu" as PERCPU_DATA_SEC.
  * Address comment from Alexei:
    * Add u8, array of ints and struct { .. } vars to selftest.
v2: https://lore.kernel.org/bpf/20250213161931.46399-1-leon.hwang@linux.dev/

v1 -> v2:
  * Address comments from Andrii:
    * Use LIBBPF_MAP_PERCPU and SEC_PERCPU.
    * Reuse mmaped of libbpf's struct bpf_map for .percpu map data.
    * Set .percpu struct pointer to NULL after loading skeleton.
    * Make sure value size of .percpu map is __aligned(8).
    * Use raw_tp and opts.cpu to test global percpu variables on all CPUs.
  * Address comments from Alexei:
    * Test non-zero offset of global percpu variable.
    * Test case about BPF_PSEUDO_MAP_IDX_VALUE.
v1: https://lore.kernel.org/bpf/20250127162158.84906-1-leon.hwang@linux.dev/

rfc -> v1:
  * Address comments from Andrii:
    * Keep one image of global percpu variable for all CPUs.
    * Reject non-ARRAY map in bpf_map_direct_read(), check_reg_const_str(),
      and check_bpf_snprintf_call() in verifier.
    * Split out libbpf changes from kernel-side changes.
    * Use ".percpu" as PERCPU_DATA_SEC.
    * Use enum libbpf_map_type to distinguish BSS, DATA, RODATA and
      PERCPU_DATA.
    * Avoid using errno for checking err from libbpf_num_possible_cpus().
    * Use "map '%s': " prefix for error message.
rfc: https://lore.kernel.org/bpf/20250113152437.67196-1-leon.hwang@linux.dev/

Leon Hwang (12):
  bpf: Drop duplicate blank lines in verifier
  bpf: Disallow interpreter fallback for user BPF_ADDR_SPACE_CAST insn
  bpf: Disallow interpreter fallback for BPF_ADDR_PERCPU insn
  bpf: Introduce global percpu data
  libbpf: Probe percpu data feature
  libbpf: Add support for global percpu data
  bpftool: Generate skeleton for global percpu data
  selftests/bpf: Add tests to verify global percpu data
  selftests/bpf: Add test to verify accessing rdonly percpu_array
  selftests/bpf: Add tests to verify verifier log for global percpu data
  selftests/bpf: Add test to verify xlated insns for global percpu data
  selftests/bpf: Add test to verify bpf_iter for global percpu data

 include/linux/bpf.h                           |   6 +-
 kernel/bpf/arraymap.c                         |  39 +-
 kernel/bpf/const_fold.c                       |   1 -
 kernel/bpf/core.c                             |   2 +
 kernel/bpf/fixups.c                           |  41 ++
 kernel/bpf/verifier.c                         |  30 +-
 tools/bpf/bpftool/gen.c                       |  43 +-
 tools/lib/bpf/bpf_gen_internal.h              |   3 +-
 tools/lib/bpf/features.c                      |  35 ++
 tools/lib/bpf/gen_loader.c                    |   3 +-
 tools/lib/bpf/libbpf.c                        |  68 +++-
 tools/lib/bpf/libbpf_internal.h               |   2 +
 tools/lib/bpf/skel_internal.h                 |  24 +-
 tools/testing/selftests/bpf/Makefile          |   2 +-
 .../bpf/prog_tests/global_data_init.c         | 373 ++++++++++++++++++
 .../bpf/prog_tests/global_percpu_subskel.c    |  37 ++
 .../bpf/progs/test_global_percpu_data.c       |  90 +++++
 17 files changed, 753 insertions(+), 46 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/global_percpu_subskel.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_percpu_data.c

-- 
2.54.0


^ permalink raw reply	[flat|nested] 16+ messages in thread
end of thread, other threads:[~2026-06-15 21:29 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-06-15 15:26 [PATCH bpf-next v6 00/12] bpf: Introduce global percpu data Leon Hwang
2026-06-15 15:26 ` [PATCH bpf-next v6 01/12] bpf: Drop duplicate blank lines in verifier Leon Hwang
2026-06-15 15:26 ` [PATCH bpf-next v6 02/12] bpf: Disallow interpreter fallback for user BPF_ADDR_SPACE_CAST insn Leon Hwang
2026-06-15 15:26 ` [PATCH bpf-next v6 03/12] bpf: Disallow interpreter fallback for BPF_ADDR_PERCPU insn Leon Hwang
2026-06-15 15:26 ` [PATCH bpf-next v6 04/12] bpf: Introduce global percpu data Leon Hwang
2026-06-15 15:26 ` [PATCH bpf-next v6 05/12] libbpf: Probe percpu data feature Leon Hwang
2026-06-15 15:26 ` [PATCH bpf-next v6 06/12] libbpf: Add support for global percpu data Leon Hwang
2026-06-15 15:41   ` sashiko-bot
2026-06-15 15:26 ` [PATCH bpf-next v6 07/12] bpftool: Generate skeleton " Leon Hwang
2026-06-15 15:26 ` [PATCH bpf-next v6 08/12] selftests/bpf: Add tests to verify " Leon Hwang
2026-06-15 15:26 ` [PATCH bpf-next v6 09/12] selftests/bpf: Add test to verify accessing rdonly percpu_array Leon Hwang
2026-06-15 15:26 ` [PATCH bpf-next v6 10/12] selftests/bpf: Add tests to verify verifier log for global percpu data Leon Hwang
2026-06-15 15:40   ` sashiko-bot
2026-06-15 15:26 ` [PATCH bpf-next v6 11/12] selftests/bpf: Add test to verify xlated insns " Leon Hwang
2026-06-15 21:29   ` Alexei Starovoitov
2026-06-15 15:26 ` [PATCH bpf-next v6 12/12] selftests/bpf: Add test to verify bpf_iter " Leon Hwang

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.