* [PATCH v2 bpf 00/19] bpf: Remove BPF_F_ARENA_SCALAR, fix scalar evolution and scalar forks
@ 2026-10-27 19:02 Alexei Starovoitov
2026-10-27 19:02 ` [PATCH v2 bpf 01/19] selftests/bpf: Remove tests for global data in arena Alexei Starovoitov
` (19 more replies)
0 siblings, 20 replies; 25+ messages in thread
From: Alexei Starovoitov @ 2026-10-27 19:02 UTC (permalink / raw)
To: bpf; +Cc: daniel, andrii, eddyz87, memxor
From: Alexei Starovoitov <ast@kernel.org>
This series removes BPF_F_ARENA_SCALAR and fixes bugs in scalar evolution,
in maybe_fork_scalars() and in the trampoline of may_goto. The groups of
patches do not depend on each other.
v1 was Part 1 of "bpf: Stack in arena and a verifier that joins states"
[1]. Parts 2 and 3 of that series are not here.
- Patches 1-6 remove BPF_F_ARENA_SCALAR, .arena.data and what came with
them. The flag lets a prog access arena through plain numbers, for
languages that have no address spaces. The compiler can emit one
addr_space_cast for each pointer instead. That is cheaper than one more
insn for each access. No released kernel has the flag.
- Patches 7-8: scalar evolution does not clear 'delta' in a register that
it widens. Then the verifier calculates an incorrect range for a linked
register, and an unsafe prog loads.
- Patches 9-10 make two selftests reliable: map_kptr and rcu_reentry.
- Patches 11-12: scalar evolution widens a pointer to a buffer
(PTR_TO_BUF). The verifier rejects an access to a buffer at a variable
offset. Thus it rejects a correct loop over the per-cpu values of a map
element.
- Patches 13-15: two bugs in the clamp of scalar evolution. A pointer that
the loop invalidates becomes a scalar with the range of the offsets of
the pointer. A packet pointer keeps a range that is not valid for its
new offsets. In the two cases an unsafe prog loads and crashes the
kernel.
- Patches 16-17: maybe_fork_scalars() splits a state in two and verifies
the second state later, from the same insn. In a loop that scalar
evolution counted, pruning drops the second state. Then no one verifies
one of the two values, and a prog that reads out of bounds loads.
- Patches 18-19: arch_bpf_timed_may_goto() does not save the registers in
which the JITs pass stack arguments. After may_goto such an argument is
not what the verifier knows about it. An unsafe prog crashes the kernel.
Tests on x86-64, compared with the same tests on the base:
test_progs, test_progs-no_alu32, test_progs-cpuv4, no new failure
test_progs-bpf_gcc, test_verifier, test_maps
test_progs with KASAN no new failure
veristat, 36 weekly builds of Cilium, 5032 progs same verdicts,
insns -0.03%
veristat, sched-ext/scx, 563 progs same verdicts
veristat, selftests from clang 21, clang 24 and GCC see below
In the selftests the verdict changes only for the tests of this series and
for the tests of .arena.data. The change in Cilium is from patch 16. arm64
is compile tested.
v1 -> v2:
- Rebase on the fixes for scalar evolution from Eduard.
- Drop "bpf: One time slice of may_goto for a run of a prog" and its
tests. The patch has bugs. The insns at the exit of a subprog change R2,
which is a part of a 16-byte return value (Ihor). A nested run of the
prog starts the time slice again. I will send it as a separate series.
- Patch 7: use clear_scalar_id() (Daniel).
- Drop "bpf: Fix a false verifier bug in the clamp of scalar evolution"
and its tests. Commit 0c33d612bdd5 ("bpf: Do not intersect with current
register range on clamp") fixes the bug in a better way (Eduard).
- Patch 5: name the two schedulers in C that set the flag.
- Patch 11: correct the reason why two selftests load today.
- Add patches 10, 13-15 and 18-19.
- Add patches 16-17. They were patch 46 and a part of patch 75 in v1. The
bug is in the tree today (Ihor). The log in v1 said that no prog is
affected. That was wrong.
[1] https://lore.kernel.org/bpf/20261026025943.404450-1-alexei.starovoitov@gmail.com/
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Alexei Starovoitov (19):
selftests/bpf: Remove tests for global data in arena
libbpf: Remove .arena.data
libbpf: Keep format strings of bpf_printk() in .rodata
selftests/bpf: Remove tests for arena access through numbers
bpf: Remove BPF_F_ARENA_SCALAR
bpf, x86: Remove r0_on_stack from fetching atomics in arena
bpf: Clear delta of a register that scalar evolution widens
selftests/bpf: Test that a widened register is not linked
selftests/bpf: Wait for the last maps of map_kptr to be freed
selftests/bpf: Avoid a page fault in rcu_reentry while the handler is
armed
bpf: Don't widen a pointer to a buffer in scalar evolution
selftests/bpf: Test a loop that moves a pointer to a buffer
bpf: Don't clamp a register whose type changed in the loop
bpf: Reset the range of a packet pointer that scalar evolution clamps
selftests/bpf: Test the clamp of pointers in scalar evolution
bpf: Finish the insn in the state that maybe_fork_scalars() splits off
selftests/bpf: Test maybe_fork_scalars() at the head of a counted loop
bpf: Keep stack arguments in arch_bpf_timed_may_goto()
selftests/bpf: Test stack arguments across may_goto
arch/arm64/net/bpf_jit_comp.c | 5 -
arch/arm64/net/bpf_timed_may_goto.S | 10 +-
arch/riscv/net/bpf_timed_may_goto.S | 27 +-
arch/x86/net/bpf_jit_comp.c | 63 +-
arch/x86/net/bpf_timed_may_goto.S | 3 +
include/linux/bpf_verifier.h | 8 +-
include/linux/filter.h | 1 -
include/uapi/linux/bpf.h | 6 -
kernel/bpf/core.c | 9 +-
kernel/bpf/fixups.c | 39 -
kernel/bpf/scev.c | 18 +-
kernel/bpf/syscall.c | 4 -
kernel/bpf/verifier.c | 97 +-
tools/include/uapi/linux/bpf.h | 6 -
tools/lib/bpf/bpf_helpers.h | 18 +-
tools/lib/bpf/libbpf.c | 433 +-----
tools/testing/selftests/bpf/Makefile | 12 +-
.../testing/selftests/bpf/Makefile.buildvars | 25 -
tools/testing/selftests/bpf/Makefile.skel | 18 +-
.../bpf/prog_tests/arena_scalar_blinded.c | 21 -
.../selftests/bpf/prog_tests/data_in_arena.c | 216 ---
.../selftests/bpf/prog_tests/map_kptr.c | 5 +
.../selftests/bpf/prog_tests/rcu_reentry.c | 19 +-
.../selftests/bpf/prog_tests/verifier.c | 2 -
.../selftests/bpf/progs/arena_noarena.c | 15 -
.../selftests/bpf/progs/data_in_arena.c | 112 --
.../selftests/bpf/progs/data_in_arena_decl.c | 37 -
.../bpf/progs/data_in_arena_extern.c | 20 -
.../selftests/bpf/progs/data_in_arena_fail.c | 20 -
.../selftests/bpf/progs/data_in_arena_nomap.c | 20 -
.../selftests/bpf/progs/data_in_arena_rust.rs | 73 --
.../selftests/bpf/progs/verifier_arena.c | 15 +
.../bpf/progs/verifier_arena_scalar.c | 1155 -----------------
.../selftests/bpf/progs/verifier_bounds.c | 41 +
.../selftests/bpf/progs/verifier_scev.c | 209 ++-
.../selftests/bpf/progs/verifier_stack_arg.c | 119 ++
tools/testing/selftests/bpf/test_loader.c | 2 -
37 files changed, 525 insertions(+), 2378 deletions(-)
delete mode 100644 tools/testing/selftests/bpf/prog_tests/arena_scalar_blinded.c
delete mode 100644 tools/testing/selftests/bpf/prog_tests/data_in_arena.c
delete mode 100644 tools/testing/selftests/bpf/progs/data_in_arena.c
delete mode 100644 tools/testing/selftests/bpf/progs/data_in_arena_decl.c
delete mode 100644 tools/testing/selftests/bpf/progs/data_in_arena_extern.c
delete mode 100644 tools/testing/selftests/bpf/progs/data_in_arena_fail.c
delete mode 100644 tools/testing/selftests/bpf/progs/data_in_arena_nomap.c
delete mode 100644 tools/testing/selftests/bpf/progs/data_in_arena_rust.rs
delete mode 100644 tools/testing/selftests/bpf/progs/verifier_arena_scalar.c
base-commit: 9ce722236a20e7a913b54df8b6098ec6324ba58a
--
2.55.0
^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH v2 bpf 01/19] selftests/bpf: Remove tests for global data in arena
2026-10-27 19:02 [PATCH v2 bpf 00/19] bpf: Remove BPF_F_ARENA_SCALAR, fix scalar evolution and scalar forks Alexei Starovoitov
@ 2026-10-27 19:02 ` Alexei Starovoitov
2026-10-27 19:02 ` [PATCH v2 bpf 02/19] libbpf: Remove .arena.data Alexei Starovoitov
` (18 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: Alexei Starovoitov @ 2026-10-27 19:02 UTC (permalink / raw)
To: bpf; +Cc: daniel, andrii, eddyz87, memxor
From: Alexei Starovoitov <ast@kernel.org>
The .arena.data section and the BPF_F_ARENA_SCALAR flag let a prog in Rust
keep its global data in arena. The subsequent patches remove the two,
because the compiler can do this work. First remove the tests for
.arena.data.
One of the tests is a prog in Rust. It has no replacement, because LLVM
does not put the globals of Rust in arena at this time. No other selftest
is in Rust. Thus also remove the makefile rules that build Rust.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
tools/testing/selftests/bpf/Makefile | 12 +-
.../testing/selftests/bpf/Makefile.buildvars | 25 --
tools/testing/selftests/bpf/Makefile.skel | 18 +-
.../selftests/bpf/prog_tests/data_in_arena.c | 216 ------------------
.../selftests/bpf/progs/data_in_arena.c | 112 ---------
.../selftests/bpf/progs/data_in_arena_decl.c | 37 ---
.../bpf/progs/data_in_arena_extern.c | 20 --
.../selftests/bpf/progs/data_in_arena_fail.c | 20 --
.../selftests/bpf/progs/data_in_arena_nomap.c | 20 --
.../selftests/bpf/progs/data_in_arena_rust.rs | 73 ------
10 files changed, 2 insertions(+), 551 deletions(-)
delete mode 100644 tools/testing/selftests/bpf/prog_tests/data_in_arena.c
delete mode 100644 tools/testing/selftests/bpf/progs/data_in_arena.c
delete mode 100644 tools/testing/selftests/bpf/progs/data_in_arena_decl.c
delete mode 100644 tools/testing/selftests/bpf/progs/data_in_arena_extern.c
delete mode 100644 tools/testing/selftests/bpf/progs/data_in_arena_fail.c
delete mode 100644 tools/testing/selftests/bpf/progs/data_in_arena_nomap.c
delete mode 100644 tools/testing/selftests/bpf/progs/data_in_arena_rust.rs
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index 1e4af62e8312..c99e0f623e10 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -423,16 +423,6 @@ $(LIBARENA_ASAN_SKEL): $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ) $(LIBARENA_BPF_DEPS)
+$(MAKE) -C libarena libarena_asan.skel.h $(LIBARENA_MAKE_ARGS)
endif
-# #![no_std] looks for compiler_builtins too. Nothing of it is used.
-ifneq ($(RUST_CORE),)
-$(RUST_CORE): $(RUST_CORE_SRC)
- $(call msg,RUSTC,,$@)
- $(Q)mkdir -p $(@D)
- +$(Q)$(RUSTC_BPF) -A warnings --edition 2024 --crate-name core --out-dir $(@D) $<
- +$(Q)echo '#![feature(compiler_builtins)] #![compiler_builtins] #![no_std]' | \
- $(RUSTC_BPF) -A warnings --crate-name compiler_builtins --out-dir $(@D) -
-endif
-
# Generated test list headers
define gen_tests_hdr
@@ -468,7 +458,7 @@ RUNNER_PREREQS := $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ) $(BPFTOOL) \
$(VERIFY_SIG_HDR) $(PRIVATE_KEY) $(VERIFICATION_CERT) \
$(LIBARENA_SKEL) $(LIBARENA_ASAN_SKEL) \
prog_tests/tests.h map_tests/tests.h \
- $(RUNNER_OBJS) $(RUST_CORE)
+ $(RUNNER_OBJS)
# Runtime fixtures for each test_progs flavor.
RUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read \
diff --git a/tools/testing/selftests/bpf/Makefile.buildvars b/tools/testing/selftests/bpf/Makefile.buildvars
index ff3476bc40a4..d2a0c0031b87 100644
--- a/tools/testing/selftests/bpf/Makefile.buildvars
+++ b/tools/testing/selftests/bpf/Makefile.buildvars
@@ -109,31 +109,6 @@ HOST_INCLUDE_DIR := $(INCLUDE_DIR)
endif
RESOLVE_BTFIDS := $(HOST_BUILD_DIR)/resolve_btfids/resolve_btfids
-# Programs in Rust are built by upstream rustc. It has no prebuilt core for
-# the bpf target, so it has to come with the source of core:
-# rustup component add rust-src
-# core is built as edition 2024, which it is since rustc 1.87.
-# rustc emits LLVM bitcode and clang makes the object of it, so clang has to be
-# 23 or newer and not older than LLVM of rustc.
-# Otherwise RUST_CORE is empty and the tests are skipped.
-RUSTC ?= rustc
-RUST_CORE_SRC := $(wildcard $(shell $(RUSTC) --print sysroot 2>/dev/null)$\
- /lib/rustlib/src/rust/library/core/src/lib.rs)
-ifneq ($(RUST_CORE_SRC),)
-ifeq ($(shell { clang=$$(echo __clang_major__ | $(CLANG) -E -P -x c -) && \
- llvm=$$($(srctree)/scripts/rustc-llvm-version.sh $(RUSTC)) && \
- [ $$($(srctree)/scripts/rustc-version.sh $(RUSTC)) -ge 108700 ] && \
- [ $$clang -ge 23 ] && [ $$clang -ge $$((llvm / 10000)) ]; } \
- 2>/dev/null && echo y),y)
-RUST_CORE := $(BUILD_DIR)/rust/libcore.rlib
-endif
-endif
-# RUSTC_BOOTSTRAP=1 is to build core with a stable rustc, like the kernel does.
-# panic=abort is a stop gap until panic=unwind is supported.
-RUSTC_BPF = RUSTC_BOOTSTRAP=1 $(RUSTC) -O -C panic=abort --crate-type rlib \
- --target $(if $(IS_LITTLE_ENDIAN),bpfel,bpfeb)-unknown-none \
- -L $(dir $(RUST_CORE))
-
DEFAULT_BPFTOOL := $(HOST_SCRATCH_DIR)/sbin/bpftool
ifneq ($(CROSS_COMPILE),)
CROSS_BPFTOOL := $(SCRATCH_DIR)/sbin/bpftool
diff --git a/tools/testing/selftests/bpf/Makefile.skel b/tools/testing/selftests/bpf/Makefile.skel
index 6570c48f92b0..e8699903feef 100644
--- a/tools/testing/selftests/bpf/Makefile.skel
+++ b/tools/testing/selftests/bpf/Makefile.skel
@@ -20,8 +20,7 @@ ifneq ($(BPF_CC),)
BPF_SRCS := $(notdir $(wildcard progs/*.c))
BPF_OBJS := $(patsubst %.c,$(RDIR)/%.bpf.o,$(BPF_SRCS))
-SKEL_BLACKLIST := btf__% test_pinning_invalid.c test_sk_assign.c \
- data_in_arena_extern.c data_in_arena_nomap.c
+SKEL_BLACKLIST := btf__% test_pinning_invalid.c test_sk_assign.c
LINKED_SKELS := test_static_linked.skel.h linked_funcs.skel.h \
linked_vars.skel.h linked_maps.skel.h linked_arena.skel.h \
@@ -138,21 +137,6 @@ $(LINKED_SKELS_H): $(RDIR)/%.skel.h: $$(addprefix $(RDIR)/,$$($$*.skel.h-deps))
$(BPFTOOL) $(GEN_SKEL) | $(RDIR)
$(Q)$(cmd_bpf_link_skel)
-# Programs in Rust, see Makefile.buildvars. No skeletons: the objects may be absent.
-ifneq ($(RUST_CORE),)
-ifeq ($(BPF_CC),$(CLANG))
-RUST_OBJS := $(patsubst progs/%.rs,$(RDIR)/%.bpf.o,$(wildcard progs/*.rs))
-BPF_OBJS += $(RUST_OBJS)
-
-$(RUST_OBJS): $(RDIR)/%.bpf.o: progs/%.rs $(RUST_CORE) | $(RDIR)
- $(call msg,RUSTC,$(BINARY),$@)
- +$(Q)$(RUSTC_BPF) --edition 2021 -C debuginfo=2 --emit=llvm-bc \
- $(patsubst -mcpu=%,-C target-cpu=%,$(filter -mcpu=%,$(BPF_CC_FLAGS))) \
- -o $(dir $(RUST_CORE))$(BINARY)-$*.bc $< && \
- $(BPF_CC) $(BPF_CC_FLAGS) -c $(dir $(RUST_CORE))$(BINARY)-$*.bc -o $@ $(call skip_on_fail,BPF)
-endif
-endif
-
# Programs in C++. No skeletons: gcc doesn't build them.
ifeq ($(BPF_CC),$(CLANG))
CXX_OBJS := $(patsubst progs/%.cpp,$(RDIR)/%.bpf.o,$(wildcard progs/*.cpp))
diff --git a/tools/testing/selftests/bpf/prog_tests/data_in_arena.c b/tools/testing/selftests/bpf/prog_tests/data_in_arena.c
deleted file mode 100644
index cb1023507c01..000000000000
--- a/tools/testing/selftests/bpf/prog_tests/data_in_arena.c
+++ /dev/null
@@ -1,216 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-#include <test_progs.h>
-#include "data_in_arena.skel.h"
-#include "data_in_arena_decl.skel.h"
-#include "data_in_arena_fail.skel.h"
-
-static int run_prog(struct bpf_program *prog)
-{
- LIBBPF_OPTS(bpf_test_run_opts, topts);
-
- if (!ASSERT_OK(bpf_prog_test_run_opts(bpf_program__fd(prog), &topts), "test_run"))
- return -1;
- return topts.retval;
-}
-
-static void run(struct data_in_arena *skel, int counter)
-{
- int i;
-
- ASSERT_EQ(run_prog(skel->progs.use_data), counter + 7 + 1 + 2, "retval");
- ASSERT_EQ(skel->bss->sum, counter + 7 + 1 + 2, "sum");
- ASSERT_EQ(skel->data->counter, counter + 1, "counter");
- ASSERT_EQ(skel->data->pair[1], 5, "pair[1]");
- for (i = 0; i < 4; i++)
- ASSERT_EQ(skel->bss->table[i], 10 * (i + 1) + i, "table");
-}
-
-static void test_in_arena(void)
-{
- struct bpf_map_info info = {};
- struct data_in_arena *skel;
- __u32 len = sizeof(info);
- struct bpf_map *arena;
- size_t sz;
-
- skel = data_in_arena__open();
- if (!ASSERT_OK_PTR(skel, "open"))
- return;
-
- arena = bpf_object__find_map_by_name(skel->obj, "arena");
- if (!ASSERT_OK_PTR(arena, "arena"))
- goto out;
- ASSERT_EQ(bpf_map__type(arena), BPF_MAP_TYPE_ARENA, "arena type");
- ASSERT_EQ(bpf_map__max_entries(arena), 1, "arena pages");
- ASSERT_OK(bpf_map__set_max_entries(arena, 8), "arena resize");
- ASSERT_FALSE(bpf_map__autocreate(skel->maps.data), "data autocreate");
- ASSERT_FALSE(bpf_map__autocreate(skel->maps.bss), "bss autocreate");
- ASSERT_FALSE(bpf_map__autocreate(skel->maps.rodata), "rodata autocreate");
- ASSERT_EQ(bpf_map__set_autocreate(skel->maps.data, true), -EOPNOTSUPP, "set_autocreate");
- ASSERT_EQ(bpf_map__set_value_size(skel->maps.bss, 4096), -EOPNOTSUPP, "set_value_size");
- ASSERT_EQ(bpf_map__initial_value(skel->maps.data, &sz), skel->data, "initial_value");
- ASSERT_EQ(sz, sizeof(*skel->data), "initial_value size");
-
- /* initial values are set the usual way */
- skel->data->counter = 100;
-
- if (!ASSERT_OK(data_in_arena__load(skel), "load"))
- goto out;
- /* there are no maps behind the sections */
- ASSERT_ERR(bpf_map_get_info_by_fd(bpf_map__fd(skel->maps.data), &info, &len), "data map");
- ASSERT_ERR(bpf_map_get_info_by_fd(bpf_map__fd(skel->maps.bss), &info, &len), "bss map");
- ASSERT_ERR(bpf_map_get_info_by_fd(bpf_map__fd(skel->maps.rodata), &info, &len),
- "rodata map");
- run(skel, 100);
-
- /* pointers to data next to pointers to functions */
- ASSERT_EQ(run_prog(skel->progs.use_ops), 42 + 1 + 'e', "use_ops");
- ASSERT_EQ(skel->data->counter, 102, "counter");
-
- /* alignment of sections and pointers to data in data */
- ASSERT_EQ((unsigned long)&skel->bss->aligned64 % 64, 0, "alignment");
- ASSERT_EQ(run_prog(skel->progs.use_ptrs), 0, "use_ptrs");
- ASSERT_EQ(skel->data->x, 43, "x");
- ASSERT_EQ(skel->bss->aligned64.v[7], 7, "aligned64");
- ASSERT_EQ(skel->data->px, &skel->data->x, "px");
-
- /* format strings of bpf_printk() and BPF_SNPRINTF() */
- ASSERT_EQ(run_prog(skel->progs.use_printk), sizeof("43-7"), "use_printk");
- ASSERT_STREQ(skel->bss->out, "43-7", "out");
-out:
- data_in_arena__destroy(skel);
-}
-
-/* The object has an arena map and __arena variables */
-static void test_declared_arena(void)
-{
- struct data_in_arena_decl *skel;
- struct bpf_map *map;
- int arenas = 0;
-
- skel = data_in_arena_decl__open();
- if (!ASSERT_OK_PTR(skel, "open"))
- return;
- bpf_object__for_each_map(map, skel->obj)
- arenas += bpf_map__type(map) == BPF_MAP_TYPE_ARENA;
- ASSERT_EQ(arenas, 1, "no second arena");
- skel->data->counter = 6;
- if (!ASSERT_OK(data_in_arena_decl__load(skel), "load"))
- goto out;
- ASSERT_EQ(run_prog(skel->progs.use_data), 6 + 7 + 11, "retval");
- ASSERT_EQ(run_prog(skel->progs.use_data), 7 + 7 + 12, "retval");
- ASSERT_EQ(skel->bss->sum, 7 + 7 + 12, "sum");
- ASSERT_EQ(skel->data->counter, 8, "counter");
-out:
- data_in_arena_decl__destroy(skel);
-}
-
-/* A pointer in data that can't be made an address of arena fails the load */
-static void test_ptr_to_map(void)
-{
- struct data_in_arena_fail *skel;
-
- skel = data_in_arena_fail__open();
- if (!ASSERT_OK_PTR(skel, "open"))
- return;
- ASSERT_ERR(data_in_arena_fail__load(skel), "load");
- data_in_arena_fail__destroy(skel);
-}
-
-/* So does a pointer to a variable of the kernel. There is no skeleton: the open fails. */
-static void test_ptr_to_extern(void)
-{
- struct bpf_object *obj;
-
- obj = bpf_object__open_file("./data_in_arena_extern.bpf.o", NULL);
- if (!ASSERT_ERR_PTR(obj, "open"))
- bpf_object__close(obj);
-}
-
-/* __arena variables and no arena map. There is no skeleton: the open fails. */
-static void test_arena_var_no_map(void)
-{
- struct bpf_object *obj;
-
- obj = bpf_object__open_file("./data_in_arena_nomap.bpf.o", NULL);
- if (!ASSERT_ERR_PTR(obj, "open"))
- bpf_object__close(obj);
-}
-
-/* The address of an arena that libbpf doesn't create is not known. No pointers in data then. */
-static void test_not_my_arena(bool pin)
-{
- LIBBPF_OPTS(bpf_map_create_opts, opts, .map_flags = BPF_F_MMAPABLE);
- struct data_in_arena *skel;
- struct bpf_map *arena;
- int fd = -1;
-
- skel = data_in_arena__open();
- if (!ASSERT_OK_PTR(skel, "open"))
- return;
- arena = bpf_object__find_map_by_name(skel->obj, "arena");
- if (!ASSERT_OK_PTR(arena, "arena"))
- goto out;
- if (pin) {
- ASSERT_OK(bpf_map__set_pin_path(arena, "/sys/fs/bpf/data_in_arena"), "pin_path");
- } else {
- fd = bpf_map_create(BPF_MAP_TYPE_ARENA, "arena", 0, 0, 1, &opts);
- if (!ASSERT_GE(fd, 0, "map_create"))
- goto out;
- ASSERT_OK(bpf_map__reuse_fd(arena, fd), "reuse_fd");
- }
- ASSERT_EQ(data_in_arena__load(skel), -ENOTSUP, "load");
-out:
- if (fd >= 0)
- close(fd);
- data_in_arena__destroy(skel);
-}
-
-/* The object is there if rustc and clang can build it, see Makefile.buildvars */
-static void test_rust(void)
-{
- const char *file = "./data_in_arena_rust.bpf.o";
- struct bpf_object *obj;
-
- if (access(file, R_OK)) {
- test__skip();
- return;
- }
- obj = bpf_object__open_file(file, NULL);
- if (!ASSERT_OK_PTR(obj, "open"))
- return;
- if (!ASSERT_OK(bpf_object__load(obj), "load"))
- goto out;
- /* libbpf goes on without BTF when the kernel doesn't take it */
- ASSERT_GE(bpf_object__btf_fd(obj), 0, "btf_fd");
- ASSERT_EQ(run_prog(bpf_object__find_program_by_name(obj, "list_in_data")),
- 100 + 20 + 3, "retval");
-out:
- bpf_object__close(obj);
-}
-
-void test_data_in_arena(void)
-{
-#if !defined(__x86_64__) && !defined(__aarch64__)
- /* other JITs don't take BPF_F_ARENA_SCALAR */
- test__skip();
- return;
-#endif
- if (test__start_subtest("arena"))
- test_in_arena();
- if (test__start_subtest("declared_arena"))
- test_declared_arena();
- if (test__start_subtest("ptr_to_map"))
- test_ptr_to_map();
- if (test__start_subtest("ptr_to_extern"))
- test_ptr_to_extern();
- if (test__start_subtest("arena_var_no_map"))
- test_arena_var_no_map();
- if (test__start_subtest("pinned_arena"))
- test_not_my_arena(true);
- if (test__start_subtest("reused_arena"))
- test_not_my_arena(false);
- if (test__start_subtest("rust"))
- test_rust();
-}
diff --git a/tools/testing/selftests/bpf/progs/data_in_arena.c b/tools/testing/selftests/bpf/progs/data_in_arena.c
deleted file mode 100644
index 18f38103c0fc..000000000000
--- a/tools/testing/selftests/bpf/progs/data_in_arena.c
+++ /dev/null
@@ -1,112 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-#include <linux/bpf.h>
-#include <bpf/bpf_helpers.h>
-
-/* global data of the object is in arena */
-char data_in_arena SEC(".arena.data");
-
-int counter = 5;
-long pair[2] = { 1, 2 };
-int x = 42;
-long sum;
-long table[4];
-struct {
- long v[8];
-} aligned64 __attribute__((aligned(64)));
-const volatile int ro = 7;
-const volatile long ro_table[4] = { 10, 20, 30, 40 };
-
-/* const strings stay in a map for helpers and kfuncs, a copy of them is in arena */
-const char hello[] SEC(".rodata.str.hello") = "hello";
-
-/* pointers to data that are stored in data */
-int *px = &x;
-const char *str = hello;
-int *const volatile cpx SEC(".data.rel.ro") = &x;
-
-SEC("syscall")
-int use_data(void *ctx)
-{
- int i;
-
- for (i = 0; i < 4; i++)
- table[i] = ro_table[i] + i;
- sum = counter + ro + pair[0] + pair[1];
- counter++;
- __sync_fetch_and_add(&pair[1], 3);
- return sum;
-}
-
-typedef int (*op_fn)(int);
-
-static __noinline int add1(int v)
-{
- return v + 1;
-}
-
-/*
- * Pointers to functions and to data in read-only data of a program with callx.
- * Volatile, so that the compiler doesn't replace the pointers with what
- * they point to.
- */
-static const volatile struct {
- op_fn fn;
- int *data;
- const char *name;
-} ops SEC(".data.rel.ro") = { add1, &x, hello };
-
-SEC("syscall")
-int use_ops(void *ctx)
-{
- /* a program has the arena when its code refers to it */
- counter++;
-#ifdef __clang__
- return ops.fn(*ops.data) + ops.name[1];
-#else
- /* gcc doesn't support indirect calls */
- return add1(*ops.data) + ops.name[1];
-#endif
-}
-
-SEC("syscall")
-int use_ptrs(void *ctx)
-{
- unsigned long addr = (unsigned long)&aligned64;
- char local[4] = "abc";
-
- /* hide the address from the compiler, it knows that '& 63' is 0 */
- asm volatile ("" : "+r"(addr));
- if (addr & 63)
- return 1;
- aligned64.v[7] = 7;
- if (*px != 42)
- return 2;
- *px = 43;
- if (x != 43 || *cpx != 43)
- return 3;
- if (str[0] != 'h' || str[4] != 'o' || str[5])
- return 4;
- /* the literal is in a map */
- if (bpf_strncmp(local, sizeof(local), "abc"))
- return 5;
- return 0;
-}
-
-char out[16];
-
-/* format strings are in a map */
-SEC("syscall")
-int use_printk(void *ctx)
-{
- char buf[sizeof(out)];
- int i, n;
-
- bpf_printk("counter %d", counter);
- n = BPF_SNPRINTF(buf, sizeof(buf), "%d-%d", x, ro);
- for (i = 0; i < sizeof(out); i++)
- out[i] = buf[i];
- return n;
-}
-
-char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/data_in_arena_decl.c b/tools/testing/selftests/bpf/progs/data_in_arena_decl.c
deleted file mode 100644
index 01bc4fea8f0c..000000000000
--- a/tools/testing/selftests/bpf/progs/data_in_arena_decl.c
+++ /dev/null
@@ -1,37 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-#define BPF_NO_KFUNC_PROTOTYPES
-#include <vmlinux.h>
-#include <bpf/bpf_helpers.h>
-#include "bpf_experimental.h"
-#include <bpf_arena_common.h>
-
-struct {
- __uint(type, BPF_MAP_TYPE_ARENA);
- __uint(map_flags, BPF_F_MMAPABLE);
- __uint(max_entries, 4);
-} arena SEC(".maps");
-
-/* global data of the object is in arena */
-char data_in_arena SEC(".arena.data");
-
-int counter = 5;
-long sum;
-const volatile int ro = 7;
-
-#if defined(__BPF_FEATURE_ADDR_SPACE_CAST)
-int __arena avar = 11;
-#else
-int avar = 11;
-#endif
-
-SEC("syscall")
-int use_data(void *ctx)
-{
- sum = counter + ro + avar;
- counter++;
- avar++;
- return sum;
-}
-
-char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/data_in_arena_extern.c b/tools/testing/selftests/bpf/progs/data_in_arena_extern.c
deleted file mode 100644
index dc1ad5ca3bdd..000000000000
--- a/tools/testing/selftests/bpf/progs/data_in_arena_extern.c
+++ /dev/null
@@ -1,20 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-#include <linux/bpf.h>
-#include <bpf/bpf_helpers.h>
-
-/* global data of the object is in arena */
-char data_in_arena SEC(".arena.data");
-
-extern const int bpf_prog_active __ksym;
-
-/* the variable of the kernel is not in arena */
-const void *kp = &bpf_prog_active;
-
-SEC("syscall")
-int ptr_to_extern(void *ctx)
-{
- return *(int *)kp;
-}
-
-char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/data_in_arena_fail.c b/tools/testing/selftests/bpf/progs/data_in_arena_fail.c
deleted file mode 100644
index ad8afe9afc96..000000000000
--- a/tools/testing/selftests/bpf/progs/data_in_arena_fail.c
+++ /dev/null
@@ -1,20 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-#include <linux/bpf.h>
-#include <bpf/bpf_helpers.h>
-
-/* global data of the object is in arena */
-char data_in_arena SEC(".arena.data");
-
-int x = 42;
-/* the table is read-only data with pointers. It's not in arena. */
-int *const volatile tbl[1] SEC(".data.rel.ro") = { &x };
-int *const volatile *pp = tbl;
-
-SEC("syscall")
-int ptr_to_map(void *ctx)
-{
- return **pp;
-}
-
-char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/data_in_arena_nomap.c b/tools/testing/selftests/bpf/progs/data_in_arena_nomap.c
deleted file mode 100644
index ccf66321eb36..000000000000
--- a/tools/testing/selftests/bpf/progs/data_in_arena_nomap.c
+++ /dev/null
@@ -1,20 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-#include <vmlinux.h>
-#include <bpf/bpf_helpers.h>
-#include "bpf_arena_common.h"
-
-/* global data of the object is in arena */
-char data_in_arena SEC(".arena.data");
-
-int counter = 5;
-/* needs an arena map that is declared. The one that libbpf creates won't do. */
-int __arena_global avar = 11;
-
-SEC("syscall")
-int arena_var(void *ctx)
-{
- return avar + counter;
-}
-
-char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/data_in_arena_rust.rs b/tools/testing/selftests/bpf/progs/data_in_arena_rust.rs
deleted file mode 100644
index 3dd9d4208232..000000000000
--- a/tools/testing/selftests/bpf/progs/data_in_arena_rust.rs
+++ /dev/null
@@ -1,73 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-// Why .data, .bss and .rodata of a program in Rust are in arena.
-//
-// A reference in Rust is an address. It doesn't say what it points to and it
-// can be stored in data. The list below has a node in each of the sections.
-// The nodes are linked by references that are in the data:
-// IN_BSS.next is stored by the program,
-// IN_DATA.next is a relocation in .data against .rodata.
-// sum() loads the references back and reads the three nodes with the same insn.
-//
-// When the sections are array maps libbpf skips the relocation in .data, and
-// what sum() loads from a node is a number that can't be dereferenced:
-// R1 invalid mem access 'scalar'
-// In arena the address of a node is a number to begin with.
-
-#![no_std]
-#![no_main]
-
-// Tell libbpf to keep .data, .bss and .rodata in arena.
-#[used]
-#[link_section = ".arena.data"]
-static DATA_IN_ARENA: u8 = 0;
-
-#[used]
-#[link_section = "license"]
-static LICENSE: [u8; 4] = *b"GPL\0";
-
-// panic=abort is a stop gap until panic=unwind is supported.
-// Nothing here panics, so the handler is not a part of the program.
-#[panic_handler]
-fn panic(_info: &core::panic::PanicInfo) -> ! {
- loop {}
-}
-
-pub struct Node {
- val: u32,
- next: Option<&'static Node>,
-}
-
-// no_mangle makes them visible outside, so LLVM can't fold the list into a constant.
-#[no_mangle]
-static IN_RODATA: Node = Node { val: 3, next: None };
-#[no_mangle]
-static mut IN_DATA: Node = Node {
- val: 20,
- next: Some(&IN_RODATA),
-};
-#[no_mangle]
-static mut IN_BSS: Node = Node { val: 0, next: None };
-
-#[inline(never)]
-fn sum(mut node: Option<&Node>) -> u32 {
- let mut sum = 0;
- // The verifier wants a bound.
- for _ in 0..8 {
- let Some(n) = node else { break };
- sum += n.val;
- node = n.next;
- }
- sum
-}
-
-#[no_mangle]
-#[link_section = "syscall"]
-pub extern "C" fn list_in_data(_ctx: *mut u8) -> u32 {
- unsafe {
- let head = &mut *&raw mut IN_BSS;
- head.val = 100;
- head.next = Some(&*&raw const IN_DATA);
- sum(Some(head))
- }
-}
--
2.55.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH v2 bpf 02/19] libbpf: Remove .arena.data
2026-10-27 19:02 [PATCH v2 bpf 00/19] bpf: Remove BPF_F_ARENA_SCALAR, fix scalar evolution and scalar forks Alexei Starovoitov
2026-10-27 19:02 ` [PATCH v2 bpf 01/19] selftests/bpf: Remove tests for global data in arena Alexei Starovoitov
@ 2026-10-27 19:02 ` Alexei Starovoitov
2026-10-27 19:02 ` [PATCH v2 bpf 03/19] libbpf: Keep format strings of bpf_printk() in .rodata Alexei Starovoitov
` (17 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: Alexei Starovoitov @ 2026-10-27 19:02 UTC (permalink / raw)
To: bpf; +Cc: daniel, andrii, eddyz87, memxor
From: Alexei Starovoitov <ast@kernel.org>
Rust has no address spaces. Thus a global variable cannot have the __arena
attribute. The .arena.data section was a workaround. It told libbpf to do
two things:
- Move .data, .bss and .rodata into arena.
- Load the progs with BPF_F_ARENA_SCALAR.
The compiler must select the location of a variable, not libbpf. libbpf
already puts the .addr_space.1 section in arena. The __arena variables of C
are in this section, and LLVM can use it for Rust too. Until LLVM does
that, the global data of Rust progs stays in array maps.
This reverts commit 8ab58ed42b08 ("libbpf: Keep global data in arena when
the object has .arena.data"). libbpf 1.8.0 does not have the section.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
tools/lib/bpf/libbpf.c | 433 ++---------------------------------------
1 file changed, 20 insertions(+), 413 deletions(-)
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 8fcb8cf39cc0..fb8a1f07c5b6 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -552,26 +552,6 @@ struct bpf_struct_ops {
#define STRUCT_OPS_SEC ".struct_ops"
#define STRUCT_OPS_LINK_SEC ".struct_ops.link"
#define ARENA_SEC ".addr_space.1"
-/*
- * An object with this section keeps its global data in arena instead of
- * array maps. What is in the section doesn't matter. .data, .bss and .rodata
- * sections become a part of the arena map of the object, like __arena global
- * variables are. If the object doesn't declare an arena map libbpf creates
- * one that is just large enough for the data. bpf_object__find_map_by_name()
- * finds it by the name "arena" and bpf_map__set_max_entries() changes its
- * size before the object is loaded.
- * Read-only sections with pointers to functions and sections with constant
- * strings stay frozen array maps. A constant that a helper or a kfunc takes
- * by pointer has to be in such section.
- * Pointers to data that are stored in data become addresses of arena.
- * The load fails if such pointer points to a section that is not in arena,
- * or if the arena is pinned or reused.
- *
- * It's for programs written in Rust. There are no address spaces in Rust,
- * the program accesses arena through plain numbers, so programs of the object
- * are loaded with BPF_F_ARENA_SCALAR.
- */
-#define ARENA_DATA_SEC ".arena.data"
enum libbpf_map_type {
LIBBPF_MAP_UNSPEC,
@@ -627,13 +607,6 @@ struct bpf_map {
/* pointers to functions in the data of an internal map, see obj->func_ptrs */
struct func_ptr *func_ptrs;
size_t func_ptr_cnt;
- /*
- * Data of the internal map is a part of arena data at arena_off.
- * The map is not created, unless it's a copy that is in arena.
- */
- bool in_arena;
- bool arena_copy;
- size_t arena_off;
};
enum extern_type {
@@ -808,17 +781,6 @@ struct bpf_object {
void *arena_data;
size_t arena_data_sz;
size_t arena_data_off;
- bool data_in_arena;
- bool arena_mapped;
- /* pointers to data in the data of internal maps, when data is in arena */
- struct data_ptr {
- int sec_idx;
- size_t sec_off;
- int targ_sec_idx;
- size_t targ_off;
- char *sym_name;
- } *data_ptrs;
- size_t data_ptr_cnt;
void *jumptables_data;
size_t jumptables_data_sz;
@@ -1644,7 +1606,6 @@ static struct bpf_object *bpf_object__new(const char *path,
obj->efile.obj_buf = obj_buf;
obj->efile.obj_buf_sz = obj_buf_sz;
obj->efile.btf_maps_shndx = -1;
- obj->efile.arena_data_shndx = -1;
obj->kconfig_map_idx = -1;
obj->arena_map_idx = -1;
@@ -3126,136 +3087,6 @@ static int init_arena_map_data(struct bpf_object *obj, struct bpf_map *map,
return 0;
}
-static bool map_is_const_str(const struct bpf_map *map)
-{
- return map->libbpf_type == LIBBPF_MAP_RODATA &&
- str_has_pfx(map->real_name, RODATA_SEC ".str");
-}
-
-static bool map_data_goes_to_arena(const struct bpf_object *obj, const struct bpf_map *map)
-{
- int i;
-
- switch (map->libbpf_type) {
- case LIBBPF_MAP_DATA:
- case LIBBPF_MAP_BSS:
- return true;
- case LIBBPF_MAP_RODATA:
- break;
- default:
- return false;
- }
-
- /*
- * Const strings that kfuncs and helpers take have to be in a frozen
- * map. Pointers to them that are stored in data point to a copy of
- * the strings that is in arena.
- */
- if (map_is_const_str(map))
- return true;
- /*
- * Read-only data with pointers stays a frozen map: the kernel
- * recognizes pointers to functions in it.
- */
- if (str_has_pfx(map->real_name, DATA_REL_RO_SEC))
- return false;
- for (i = 0; i < obj->efile.sec_cnt; i++) {
- const struct elf_sec_desc *sec = &obj->efile.secs[i];
-
- if (sec->sec_type == SEC_RELO && sec->shdr->sh_info == map->sec_idx)
- return false;
- }
- return true;
-}
-
-/* Returns the offset of the data of the map in arena data that is sz bytes so far */
-static size_t map_arena_off(const struct bpf_object *obj, const struct bpf_map *map, size_t sz)
-{
- size_t align = obj->efile.secs[map->sec_idx].shdr->sh_addralign;
-
- return roundup(sz, max(align, sizeof(__u64)));
-}
-
-/*
- * Make .data, .bss and .rodata a part of arena data. They follow __arena
- * variables of the object, if there are any.
- */
-static int bpf_object__init_arena_data(struct bpf_object *obj)
-{
- const size_t page_sz = sysconf(_SC_PAGE_SIZE);
- size_t sz = obj->arena_data_sz;
- struct bpf_map *map;
- void *data;
- int i;
-
- if (!obj->data_in_arena)
- return 0;
-
- for (i = 0; i < obj->nr_maps; i++) {
- map = &obj->maps[i];
- if (map_data_goes_to_arena(obj, map))
- sz = map_arena_off(obj, map, sz) + map->def.value_size;
- }
- if (sz == obj->arena_data_sz)
- return 0;
-
- if (obj->arena_map_idx < 0) {
- /* bpf_object__init_user_btf_maps() didn't look: there is no .maps section */
- if (obj->efile.arena_data) {
- pr_warn("elf: sec '%s': to use global __arena variables the ARENA map should be explicitly declared in SEC(\".maps\")\n",
- ARENA_SEC);
- return -ENOENT;
- }
-
- map = bpf_object__add_map(obj);
- if (IS_ERR(map))
- return PTR_ERR(map);
-
- map->real_name = strdup("arena");
- map->name = strdup("arena");
- if (!map->real_name || !map->name) {
- zfree(&map->real_name);
- zfree(&map->name);
- return -ENOMEM;
- }
- map->sec_idx = -1;
- map->def.type = BPF_MAP_TYPE_ARENA;
- map->def.max_entries = roundup(sz, page_sz) / page_sz;
- map->def.map_flags = BPF_F_MMAPABLE;
- obj->arena_map_idx = map - obj->maps;
- }
-
- data = realloc(obj->arena_data, sz);
- if (!data)
- return -ENOMEM;
- memset(data + obj->arena_data_sz, 0, sz - obj->arena_data_sz);
- sz = obj->arena_data_sz;
- obj->arena_data = data;
- obj->maps[obj->arena_map_idx].mmaped = data;
-
- for (i = 0; i < obj->nr_maps; i++) {
- map = &obj->maps[i];
- if (!map_data_goes_to_arena(obj, map))
- continue;
- sz = map_arena_off(obj, map, sz);
- map->arena_off = sz;
- memcpy(data + sz, map->mmaped, map->def.value_size);
- sz += map->def.value_size;
- pr_debug("map '%s': data is in arena at offset %zu\n", map->name, map->arena_off);
- if (map_is_const_str(map)) {
- map->arena_copy = true;
- continue;
- }
- munmap(map->mmaped, bpf_map_mmap_sz(map));
- /* make bpf_map__initial_value() and bpf_map__set_initial_value() work */
- map->mmaped = data + map->arena_off;
- map->autocreate = false;
- map->in_arena = true;
- }
- obj->arena_data_sz = sz;
- return 0;
-}
-
static int bpf_object__init_user_btf_maps(struct bpf_object *obj, bool strict,
const char *pin_root_path)
{
@@ -3348,7 +3179,6 @@ static int bpf_object__init_maps(struct bpf_object *obj,
err = err ?: bpf_object__init_global_data_maps(obj);
err = err ?: bpf_object__init_kconfig_map(obj);
err = err ?: bpf_object_init_struct_ops(obj);
- err = err ?: bpf_object__init_arena_data(obj);
return err;
}
@@ -4163,11 +3993,6 @@ static int bpf_object__elf_collect(struct bpf_object *obj)
if (!sh)
return -LIBBPF_ERRNO__FORMAT;
- /* the second pass has to know it when it sees relocations of data */
- name = elf_sec_str(obj, sh->sh_name);
- if (name && strcmp(name, ARENA_DATA_SEC) == 0)
- obj->data_in_arena = true;
-
if (sh->sh_type == SHT_SYMTAB) {
if (obj->efile.symbols) {
pr_warn("elf: multiple symbol tables in %s\n", obj->path);
@@ -4284,8 +4109,6 @@ static int bpf_object__elf_collect(struct bpf_object *obj)
} else if (strcmp(name, ARENA_SEC) == 0) {
obj->efile.arena_data = data;
obj->efile.arena_data_shndx = idx;
- } else if (strcmp(name, ARENA_DATA_SEC) == 0) {
- /* the marker, see the first pass */
} else if (strcmp(name, JUMPTABLES_SEC) == 0) {
obj->jumptables_data = malloc(data->d_size);
if (!obj->jumptables_data)
@@ -4310,9 +4133,6 @@ static int bpf_object__elf_collect(struct bpf_object *obj)
* have pointers to functions.
*/
if (!section_have_execinstr(obj, targ_sec_idx) &&
- !(obj->data_in_arena &&
- (!strcmp(name, ".rel" DATA_SEC) ||
- str_has_pfx(name, ".rel" DATA_SEC "."))) &&
strcmp(name, ".rel" RODATA_SEC) &&
!str_has_pfx(name, ".rel" RODATA_SEC ".") &&
strcmp(name, ".rel" DATA_REL_RO_SEC) &&
@@ -5064,10 +4884,6 @@ static int bpf_program__record_reloc(struct bpf_program *prog,
reloc_desc->insn_idx = insn_idx;
reloc_desc->map_idx = map_idx;
reloc_desc->sym_off = sym->st_value;
- if (map->in_arena) {
- reloc_desc->map_idx = obj->arena_map_idx;
- reloc_desc->sym_off += map->arena_off;
- }
return 0;
}
@@ -5286,9 +5102,6 @@ int bpf_map__set_autocreate(struct bpf_map *map, bool autocreate)
if (map_is_created(map))
return libbpf_err(-EBUSY);
- if (map->in_arena)
- return libbpf_err(-EOPNOTSUPP);
-
map->autocreate = autocreate;
return 0;
}
@@ -5569,45 +5382,6 @@ bpf_object__reuse_map(struct bpf_map *map)
return 0;
}
-static struct bpf_map *bpf_object__sec_map(struct bpf_object *obj, int sec_idx)
-{
- int i;
-
- for (i = 0; i < obj->nr_maps; i++)
- if (bpf_map__is_internal(&obj->maps[i]) && obj->maps[i].sec_idx == sec_idx)
- return &obj->maps[i];
- return NULL;
-}
-
-static int bpf_object__mmap_arena(struct bpf_object *obj, struct bpf_map *map)
-{
- int i, err;
-
- map->mmaped = mmap((void *)(long)map->map_extra,
- bpf_map_mmap_sz(map), PROT_READ | PROT_WRITE,
- map->map_extra ? MAP_SHARED | MAP_FIXED : MAP_SHARED,
- map->fd, 0);
- if (map->mmaped == MAP_FAILED) {
- err = -errno;
- map->mmaped = NULL;
- pr_warn("map '%s': failed to mmap arena: %s\n",
- map->name, errstr(err));
- return err;
- }
- if (obj->arena_data) {
- memcpy(map->mmaped + obj->arena_data_off, obj->arena_data,
- obj->arena_data_sz);
- zfree(&obj->arena_data);
- }
- for (i = 0; i < obj->nr_maps; i++) {
- struct bpf_map *m = &obj->maps[i];
-
- if (m->in_arena)
- m->mmaped = map->mmaped + obj->arena_data_off + m->arena_off;
- }
- return 0;
-}
-
static int
bpf_object__populate_internal_map(struct bpf_object *obj, struct bpf_map *map)
{
@@ -5968,10 +5742,6 @@ bpf_object__create_maps(struct bpf_object *obj)
continue;
}
- /* see bpf_object__relocate_data_ptrs() */
- if (map->def.type == BPF_MAP_TYPE_ARENA && obj->arena_mapped)
- continue;
-
err = map_set_def_max_entries(map);
if (err)
goto err_out;
@@ -6009,9 +5779,22 @@ bpf_object__create_maps(struct bpf_object *obj)
if (err < 0)
goto err_out;
} else if (map->def.type == BPF_MAP_TYPE_ARENA) {
- err = bpf_object__mmap_arena(obj, map);
- if (err)
+ map->mmaped = mmap((void *)(long)map->map_extra,
+ bpf_map_mmap_sz(map), PROT_READ | PROT_WRITE,
+ map->map_extra ? MAP_SHARED | MAP_FIXED : MAP_SHARED,
+ map->fd, 0);
+ if (map->mmaped == MAP_FAILED) {
+ err = -errno;
+ map->mmaped = NULL;
+ pr_warn("map '%s': failed to mmap arena: %s\n",
+ map->name, errstr(err));
return err;
+ }
+ if (obj->arena_data) {
+ memcpy(map->mmaped + obj->arena_data_off, obj->arena_data,
+ obj->arena_data_sz);
+ zfree(&obj->arena_data);
+ }
}
if (map->init_slots_sz && map->def.type != BPF_MAP_TYPE_PROG_ARRAY) {
err = init_map_in_map_slots(obj, map);
@@ -8026,81 +7809,6 @@ static int bpf_program_fixup_func_info(struct bpf_object *obj, struct bpf_progra
return err;
}
-/*
- * Turn pointers to data that are stored in data into addresses of arena.
- * The arena is created here, ahead of the other maps: the address has to be
- * known before the maps that hold the pointers are created.
- */
-static int bpf_object__relocate_data_ptrs(struct bpf_object *obj)
-{
- struct bpf_map *arena, *map, *targ;
- __u64 addr, val;
- size_t i;
- int err;
-
- if (!obj->data_ptr_cnt)
- return 0;
-
- for (i = 0; i < obj->data_ptr_cnt; i++) {
- struct data_ptr *p = &obj->data_ptrs[i];
-
- if (p->targ_sec_idx < 0)
- continue;
- map = bpf_object__sec_map(obj, p->sec_idx);
- targ = bpf_object__sec_map(obj, p->targ_sec_idx);
- if (map && (map->in_arena || map->autocreate) &&
- (!targ || (!targ->in_arena && !targ->arena_copy))) {
- pr_warn("sec '%s': pointer to '%s' at offset %zu can't be resolved: sec '%s' is not in arena\n",
- map->real_name, p->sym_name, p->sec_off,
- targ ? targ->real_name : "<?>");
- return -LIBBPF_ERRNO__RELOC;
- }
- }
-
- if (obj->gen_loader) {
- pr_warn("pointers to data in data are not supported by light skeleton\n");
- return -ENOTSUP;
- }
-
- if (obj->arena_map_idx < 0)
- return 0;
-
- arena = &obj->maps[obj->arena_map_idx];
- if (!arena->autocreate)
- return 0;
- /* libbpf doesn't mmap an arena that it didn't create, so the address is not known */
- if (arena->reused || arena->pin_path) {
- pr_warn("map '%s': pointers to data in data are not supported with pinned or reused arena\n",
- arena->name);
- return -ENOTSUP;
- }
-
- err = bpf_object__create_map(obj, arena, false);
- err = err ?: bpf_object__mmap_arena(obj, arena);
- if (err) {
- pr_warn("map '%s': failed to create: %s\n", arena->name, errstr(err));
- return err;
- }
- obj->arena_mapped = true;
-
- for (i = 0; i < obj->data_ptr_cnt; i++) {
- struct data_ptr *p = &obj->data_ptrs[i];
-
- map = bpf_object__sec_map(obj, p->sec_idx);
- if (!map || (!map->in_arena && !map->autocreate))
- continue;
-
- addr = (__u64)(unsigned long)arena->mmaped + obj->arena_data_off + p->targ_off;
- if (p->targ_sec_idx >= 0)
- addr += bpf_object__sec_map(obj, p->targ_sec_idx)->arena_off;
-
- memcpy(&val, map->mmaped + p->sec_off, sizeof(val));
- val += addr;
- memcpy(map->mmaped + p->sec_off, &val, sizeof(val));
- }
- return 0;
-}
-
static int bpf_object__relocate(struct bpf_object *obj, const char *targ_btf_path)
{
struct bpf_program *prog;
@@ -8123,10 +7831,8 @@ static int bpf_object__relocate(struct bpf_object *obj, const char *targ_btf_pat
size_t mmap_sz = bpf_map_mmap_sz(arena_map);
if (data_sz > mmap_sz) {
- pr_warn("map '%s': declared ARENA map size (%zu) is too small to hold global %s of size %zu\n",
- arena_map->name, mmap_sz,
- obj->data_in_arena ? "data" : "__arena variables",
- obj->arena_data_sz);
+ pr_warn("map '%s': declared ARENA map size (%zu) is too small to hold global __arena variables of size %zu\n",
+ arena_map->name, mmap_sz, obj->arena_data_sz);
return -E2BIG;
}
@@ -8135,10 +7841,6 @@ static int bpf_object__relocate(struct bpf_object *obj, const char *targ_btf_pat
obj->arena_data_off = mmap_sz - data_sz;
}
- err = bpf_object__relocate_data_ptrs(obj);
- if (err)
- return err;
-
/* Before relocating calls pre-process relocations and mark
* few ld_imm64 instructions that points to subprogs.
* Otherwise bpf_object__reloc_code() later would have to consider
@@ -8371,85 +8073,12 @@ static int bpf_object__collect_map_relos(struct bpf_object *obj,
return 0;
}
-/*
- * A pointer to data that is stored in data. When data is in arena
- * the program dereferences what it loads from data, so the pointer has
- * to be the address of the target in arena.
- */
-static int bpf_object__collect_data_ptr(struct bpf_object *obj, const char *relo_sec_name,
- int relo_idx, size_t sec_idx, const Elf64_Rel *rel,
- const Elf64_Sym *sym)
-{
- Elf_Data *scn_data = obj->efile.secs[sec_idx].data;
- const char *sym_name = elf_sym_str(obj, sym->st_name) ?: "<?>";
- struct data_ptr *ptrs;
-
- if (ELF64_ST_TYPE(sym->st_info) == STT_SECTION && sym->st_shndx < obj->efile.sec_cnt)
- sym_name = elf_sec_name(obj, elf_sec_by_idx(obj, sym->st_shndx)) ?: "<?>";
-
- if (ELF64_R_TYPE(rel->r_info) != R_BPF_64_ABS64 ||
- sym->st_shndx >= obj->efile.sec_cnt ||
- (sym->st_shndx != obj->efile.arena_data_shndx &&
- !bpf_object__shndx_is_data(obj, sym->st_shndx)) ||
- rel->r_offset >= scn_data->d_size ||
- scn_data->d_size - rel->r_offset < sizeof(__u64)) {
- pr_warn("sec '%s': relo #%d: can't resolve pointer to '%s' at offset %zu when data is in arena\n",
- relo_sec_name, relo_idx, sym_name, (size_t)rel->r_offset);
- return -LIBBPF_ERRNO__RELOC;
- }
-
- ptrs = libbpf_reallocarray(obj->data_ptrs, obj->data_ptr_cnt + 1, sizeof(*ptrs));
- if (!ptrs)
- return -ENOMEM;
- obj->data_ptrs = ptrs;
-
- ptrs[obj->data_ptr_cnt].sec_idx = sec_idx;
- ptrs[obj->data_ptr_cnt].sec_off = rel->r_offset;
- /* __arena variables are at the start of arena data */
- ptrs[obj->data_ptr_cnt].targ_sec_idx =
- sym->st_shndx == obj->efile.arena_data_shndx ? -1 : sym->st_shndx;
- ptrs[obj->data_ptr_cnt].targ_off = sym->st_value;
- ptrs[obj->data_ptr_cnt].sym_name = strdup(sym_name);
- if (!ptrs[obj->data_ptr_cnt].sym_name)
- return -ENOMEM;
- obj->data_ptr_cnt++;
-
- pr_debug("sec '%s': relo #%d: pointer at offset %zu to '%s'\n",
- relo_sec_name, relo_idx, (size_t)rel->r_offset, sym_name);
- return 0;
-}
-
-/* Collect pointers in a data section that went to arena */
-static int bpf_object__collect_data_relos(struct bpf_object *obj,
- Elf64_Shdr *shdr, Elf_Data *data)
-{
- int i, err, nrels = shdr->sh_size / shdr->sh_entsize;
- const char *relo_sec_name;
- Elf64_Sym *sym;
- Elf64_Rel *rel;
-
- relo_sec_name = elf_sec_str(obj, shdr->sh_name) ?: "<?>";
- for (i = 0; i < nrels; i++) {
- rel = elf_rel_by_idx(data, i);
- sym = rel ? elf_sym_by_idx(obj, ELF64_R_SYM(rel->r_info)) : NULL;
- if (!sym) {
- pr_warn("sec '%s': failed to get relo #%d\n", relo_sec_name, i);
- return -LIBBPF_ERRNO__FORMAT;
- }
- err = bpf_object__collect_data_ptr(obj, relo_sec_name, i, shdr->sh_info, rel, sym);
- if (err)
- return err;
- }
- return 0;
-}
-
/*
* Collect pointers to functions in a read-only data section. They are
* R_BPF_64_ABS64 relocations against .text section, where the offset of
* a static function in the section is stored in place. Relocations in data
* sections were ignored before pointers to functions were supported. Those
- * that are something else, e.g. pointers to data, still are, unless data
- * is in arena.
+ * that are something else, e.g. pointers to data, still are.
*/
static int bpf_object__collect_rodata_relos(struct bpf_object *obj,
Elf64_Shdr *shdr, Elf_Data *data)
@@ -8485,15 +8114,6 @@ static int bpf_object__collect_rodata_relos(struct bpf_object *obj,
if (ELF64_R_TYPE(rel->r_info) != R_BPF_64_ABS64 ||
!sym_is_subprog(sym, obj->efile.text_shndx)) {
- int err;
-
- if (obj->data_in_arena) {
- err = bpf_object__collect_data_ptr(obj, relo_sec_name, i,
- sec_idx, rel, sym);
- if (err)
- return err;
- continue;
- }
pr_debug("sec '%s': relo #%d: not a pointer to a function, skipping...\n",
relo_sec_name, i);
continue;
@@ -8569,8 +8189,6 @@ static int bpf_object__collect_relos(struct bpf_object *obj)
if (obj->efile.secs[idx].sec_type == SEC_RODATA)
err = bpf_object__collect_rodata_relos(obj, shdr, data);
- else if (obj->efile.secs[idx].sec_type == SEC_DATA)
- err = bpf_object__collect_data_relos(obj, shdr, data);
else if (obj->efile.secs[idx].sec_type == SEC_ST_OPS)
err = bpf_object__collect_st_ops_relos(obj, shdr, data);
else if (idx == obj->efile.btf_maps_shndx)
@@ -8914,9 +8532,6 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog
}
load_attr.log_level = log_level;
load_attr.prog_flags = prog->prog_flags;
- /* the program accesses its data in arena through plain numbers */
- if (obj->data_in_arena)
- load_attr.prog_flags |= BPF_F_ARENA_SCALAR;
load_attr.fd_array = obj->fd_array;
err = bpf_object__bind_arena(obj, prog, &load_attr);
if (err)
@@ -8991,7 +8606,7 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog
for (i = 0; i < obj->nr_maps; i++) {
map = &prog->obj->maps[i];
- if (map->libbpf_type != LIBBPF_MAP_RODATA || map->in_arena)
+ if (map->libbpf_type != LIBBPF_MAP_RODATA)
continue;
if (bpf_prog_bind_map(ret, map->fd, NULL)) {
@@ -10557,7 +10172,7 @@ static void bpf_map__destroy(struct bpf_map *map)
zfree(&map->init_slots);
map->init_slots_sz = 0;
- if (map->mmaped && map->mmaped != map->obj->arena_data && !map->in_arena)
+ if (map->mmaped && map->mmaped != map->obj->arena_data)
munmap(map->mmaped, bpf_map_mmap_sz(map));
map->mmaped = NULL;
@@ -10642,9 +10257,6 @@ void bpf_object__close(struct bpf_object *obj)
close(obj->func_ptr_maps[i].fd);
zfree(&obj->func_ptr_maps);
zfree(&obj->func_ptrs);
- for (i = 0; i < obj->data_ptr_cnt; i++)
- zfree(&obj->data_ptrs[i].sym_name);
- zfree(&obj->data_ptrs);
if (obj->btf_module_allowlist) {
for (i = 0; i < obj->btf_module_allowlist_cnt; i++)
@@ -11011,8 +10623,6 @@ int bpf_program__clone(struct bpf_program *prog, const struct bpf_prog_load_opts
attr.token_fd = OPTS_GET(opts, token_fd, 0) ?: obj->token_fd;
if (attr.token_fd)
attr.prog_flags |= BPF_F_TOKEN_FD;
- if (obj->data_in_arena)
- attr.prog_flags |= BPF_F_ARENA_SCALAR;
prog_btf_fd = OPTS_GET(opts, prog_btf_fd, 0);
if (!prog_btf_fd && obj->btf)
@@ -12019,9 +11629,6 @@ int bpf_map__set_value_size(struct bpf_map *map, __u32 size)
if (map_is_created(map))
return libbpf_err(-EBUSY);
- if (map->in_arena)
- return libbpf_err(-EOPNOTSUPP);
-
if (map->mmaped) {
size_t mmap_old_sz, mmap_new_sz;
int err;
--
2.55.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH v2 bpf 03/19] libbpf: Keep format strings of bpf_printk() in .rodata
2026-10-27 19:02 [PATCH v2 bpf 00/19] bpf: Remove BPF_F_ARENA_SCALAR, fix scalar evolution and scalar forks Alexei Starovoitov
2026-10-27 19:02 ` [PATCH v2 bpf 01/19] selftests/bpf: Remove tests for global data in arena Alexei Starovoitov
2026-10-27 19:02 ` [PATCH v2 bpf 02/19] libbpf: Remove .arena.data Alexei Starovoitov
@ 2026-10-27 19:02 ` Alexei Starovoitov
2026-10-27 19:02 ` [PATCH v2 bpf 04/19] selftests/bpf: Remove tests for arena access through numbers Alexei Starovoitov
` (16 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: Alexei Starovoitov @ 2026-10-27 19:02 UTC (permalink / raw)
To: bpf; +Cc: daniel, andrii, eddyz87, memxor
From: Alexei Starovoitov <ast@kernel.org>
The format strings of bpf_printk() moved from .rodata to .rodata.str for
one reason. .arena.data moved .rodata into arena, and helpers cannot read
strings from arena.
.arena.data is removed. Move the strings back to .rodata. As a result, an
object that has its own .rodata has one map less.
This reverts commit 373e1223ff00
("libbpf: Keep format strings of bpf_printk() in .rodata.str"). libbpf
1.8.0 has the strings in .rodata.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
tools/lib/bpf/bpf_helpers.h | 18 +++++-------------
1 file changed, 5 insertions(+), 13 deletions(-)
diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h
index b37b727ef216..9d160b5b9c0e 100644
--- a/tools/lib/bpf/bpf_helpers.h
+++ b/tools/lib/bpf/bpf_helpers.h
@@ -248,21 +248,13 @@ enum libbpf_tristate {
#define ___bpf_fill(arr, args...) \
___bpf_apply(___bpf_fill, ___bpf_narg(args))(arr, 0, args)
-/*
- * Format strings are in a section of their own. When global data of the object
- * is in arena (the object has .arena.data section) libbpf keeps .rodata.str*
- * sections in read-only maps, which is where helpers and kfuncs take strings
- * from.
- */
-#define ___bpf_fmt_sec __attribute__((section(".rodata.str")))
-
/*
* BPF_SEQ_PRINTF to wrap bpf_seq_printf to-be-printed values
* in a structure.
*/
#define BPF_SEQ_PRINTF(seq, fmt, args...) \
({ \
- static const char ___fmt[] ___bpf_fmt_sec = fmt; \
+ static const char ___fmt[] = fmt; \
unsigned long long ___param[___bpf_narg(args)]; \
\
_Pragma("GCC diagnostic push") \
@@ -280,7 +272,7 @@ enum libbpf_tristate {
*/
#define BPF_SNPRINTF(out, out_size, fmt, args...) \
({ \
- static const char ___fmt[] ___bpf_fmt_sec = fmt; \
+ static const char ___fmt[] = fmt; \
unsigned long long ___param[___bpf_narg(args)]; \
\
_Pragma("GCC diagnostic push") \
@@ -295,7 +287,7 @@ enum libbpf_tristate {
#ifdef BPF_NO_GLOBAL_DATA
#define BPF_PRINTK_FMT_MOD
#else
-#define BPF_PRINTK_FMT_MOD static const ___bpf_fmt_sec
+#define BPF_PRINTK_FMT_MOD static const
#endif
#define __bpf_printk(fmt, ...) \
@@ -311,7 +303,7 @@ enum libbpf_tristate {
*/
#define __bpf_vprintk(fmt, args...) \
({ \
- static const char ___fmt[] ___bpf_fmt_sec = fmt; \
+ static const char ___fmt[] = fmt; \
unsigned long long ___param[___bpf_narg(args)]; \
\
_Pragma("GCC diagnostic push") \
@@ -325,7 +317,7 @@ enum libbpf_tristate {
#define bpf_stream_printk(stream_id, fmt, args...) \
({ \
- static const char ___fmt[] ___bpf_fmt_sec = fmt; \
+ static const char ___fmt[] = fmt; \
unsigned long long ___param[___bpf_narg(args)]; \
\
_Pragma("GCC diagnostic push") \
--
2.55.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH v2 bpf 04/19] selftests/bpf: Remove tests for arena access through numbers
2026-10-27 19:02 [PATCH v2 bpf 00/19] bpf: Remove BPF_F_ARENA_SCALAR, fix scalar evolution and scalar forks Alexei Starovoitov
` (2 preceding siblings ...)
2026-10-27 19:02 ` [PATCH v2 bpf 03/19] libbpf: Keep format strings of bpf_printk() in .rodata Alexei Starovoitov
@ 2026-10-27 19:02 ` Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 05/19] bpf: Remove BPF_F_ARENA_SCALAR Alexei Starovoitov
` (15 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: Alexei Starovoitov @ 2026-10-27 19:02 UTC (permalink / raw)
To: bpf; +Cc: daniel, andrii, eddyz87, memxor
From: Alexei Starovoitov <ast@kernel.org>
With BPF_F_ARENA_SCALAR the verifier accepts a load or a store through a
scalar as an access to arena. The next patch removes the flag. Remove its
tests.
Keep one test. Without the flag the verifier must reject such a load, also
in a prog that has an arena. Move this test to verifier_arena.c.
The removed tests were the only tests of a special path in the x86 JIT for
the fetching AND/OR/XOR atomics. Only the flag used that path.
arena_atomics tests the usual path.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
.../bpf/prog_tests/arena_scalar_blinded.c | 21 -
.../selftests/bpf/prog_tests/verifier.c | 2 -
.../selftests/bpf/progs/arena_noarena.c | 15 -
.../selftests/bpf/progs/verifier_arena.c | 15 +
.../bpf/progs/verifier_arena_scalar.c | 1155 -----------------
tools/testing/selftests/bpf/test_loader.c | 2 -
6 files changed, 15 insertions(+), 1195 deletions(-)
delete mode 100644 tools/testing/selftests/bpf/prog_tests/arena_scalar_blinded.c
delete mode 100644 tools/testing/selftests/bpf/progs/verifier_arena_scalar.c
diff --git a/tools/testing/selftests/bpf/prog_tests/arena_scalar_blinded.c b/tools/testing/selftests/bpf/prog_tests/arena_scalar_blinded.c
deleted file mode 100644
index 2ac2e9a652fe..000000000000
--- a/tools/testing/selftests/bpf/prog_tests/arena_scalar_blinded.c
+++ /dev/null
@@ -1,21 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-#include <test_progs.h>
-#include "sysctl_helpers.h"
-#include "verifier_arena_scalar.skel.h"
-
-/* The same tests with constants of the programs blinded */
-void serial_test_arena_scalar_blinded(void)
-{
- const char *harden = "/proc/sys/net/core/bpf_jit_harden";
- char old[16] = {};
-
- if (!is_jit_enabled()) {
- test__skip();
- return;
- }
- if (sysctl_set_or_fail(harden, old, "2"))
- return;
- RUN_TESTS(verifier_arena_scalar);
- sysctl_set_or_fail(harden, NULL, old);
-}
diff --git a/tools/testing/selftests/bpf/prog_tests/verifier.c b/tools/testing/selftests/bpf/prog_tests/verifier.c
index 04740ba37e91..6a494c381266 100644
--- a/tools/testing/selftests/bpf/prog_tests/verifier.c
+++ b/tools/testing/selftests/bpf/prog_tests/verifier.c
@@ -13,7 +13,6 @@
#include "verifier_and.skel.h"
#include "verifier_arena.skel.h"
#include "verifier_arena_large.skel.h"
-#include "verifier_arena_scalar.skel.h"
#include "verifier_arena_globals1.skel.h"
#include "verifier_arena_globals2.skel.h"
#include "verifier_array_access.skel.h"
@@ -207,7 +206,6 @@ void test_verifier_align(void) { RUN(verifier_align); }
void test_verifier_and(void) { RUN(verifier_and); }
void test_verifier_arena(void) { RUN(verifier_arena); }
void test_verifier_arena_large(void) { RUN(verifier_arena_large); }
-void test_verifier_arena_scalar(void) { RUN(verifier_arena_scalar); }
void test_verifier_arena_globals1(void) { RUN(verifier_arena_globals1); }
void test_verifier_arena_globals2(void) { RUN(verifier_arena_globals2); }
void test_verifier_basic_stack(void) { RUN(verifier_basic_stack); }
diff --git a/tools/testing/selftests/bpf/progs/arena_noarena.c b/tools/testing/selftests/bpf/progs/arena_noarena.c
index edbd40cd5be8..1f46e5aa403b 100644
--- a/tools/testing/selftests/bpf/progs/arena_noarena.c
+++ b/tools/testing/selftests/bpf/progs/arena_noarena.c
@@ -21,19 +21,4 @@ int arena_arg_no_arena(void *ctx)
return 0;
}
-SEC("syscall")
-__flag(BPF_F_ARENA_SCALAR)
-__arch_x86_64
-__arch_arm64
-__description("arena_scalar: no arena, no access through a number")
-__failure __msg("R6 invalid mem access 'scalar'")
-__naked void no_arena(void)
-{
- asm volatile (" \
- r6 = 0x100000000000 ll; \
- r0 = *(u64 *)(r6 + 0); \
- exit; \
-" ::: __clobber_all);
-}
-
char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/verifier_arena.c b/tools/testing/selftests/bpf/progs/verifier_arena.c
index 1ce9c4b78481..b17a6dfc100d 100644
--- a/tools/testing/selftests/bpf/progs/verifier_arena.c
+++ b/tools/testing/selftests/bpf/progs/verifier_arena.c
@@ -610,6 +610,21 @@ int mixed_arena_scalar_alu64_arena_first(void *ctx)
return 0;
}
+SEC("syscall")
+__description("arena: a number is not an address in arena")
+__failure __msg("R6 invalid mem access 'scalar'")
+__naked void access_through_number(void)
+{
+ asm volatile (" \
+ r1 = %[arena] ll; \
+ r6 = 0x100000000000 ll; \
+ r0 = *(u64 *)(r6 + 0); \
+ exit; \
+" :
+ : __imm_addr(arena)
+ : __clobber_all);
+}
+
SEC("syscall")
__success __retval(0)
int scalar_xor_arena_ptr(void *ctx)
diff --git a/tools/testing/selftests/bpf/progs/verifier_arena_scalar.c b/tools/testing/selftests/bpf/progs/verifier_arena_scalar.c
deleted file mode 100644
index 6f0efd9202ca..000000000000
--- a/tools/testing/selftests/bpf/progs/verifier_arena_scalar.c
+++ /dev/null
@@ -1,1155 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-#include <linux/bpf.h>
-#include <bpf/bpf_helpers.h>
-#include "../../../include/linux/filter.h"
-#include "bpf_misc.h"
-
-void *bpf_arena_alloc_pages(void *map, void *addr, __u32 page_cnt, int node_id,
- __u64 flags) __ksym;
-void bpf_arena_free_pages(void *map, void *ptr, __u32 page_cnt) __ksym;
-
-#ifdef __TARGET_ARCH_arm64
-#define ARENA_VM_START (1ull << 32)
-#else
-#define ARENA_VM_START (1ull << 44)
-#endif
-
-struct {
- __uint(type, BPF_MAP_TYPE_ARENA);
- __uint(map_flags, BPF_F_MMAPABLE);
- __uint(max_entries, 4);
- __ulong(map_extra, ARENA_VM_START);
-} arena SEC(".maps");
-
-struct {
- __uint(type, BPF_MAP_TYPE_HASH);
- __uint(max_entries, 1);
- __type(key, int);
- __type(value, long long);
-} hash SEC(".maps");
-
-/* JITs that take BPF_F_ARENA_SCALAR */
-#define __arena_scalar __flag(BPF_F_ARENA_SCALAR) __arch_x86_64 __arch_arm64
-
-/* BTF FUNC records are not generated for kfuncs referenced from inline assembly */
-void __kfunc_btf_root(void)
-{
- bpf_arena_alloc_pages(0, 0, 0, 0, 0);
- bpf_arena_free_pages(0, 0, 0);
-}
-
-/* Tests start with r6 = address of a new page as the user space sees it, a number */
-
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: load and store of every size")
-__success __retval(0)
-__load_if_JITed()
-__naked void ld_st_sizes(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- r2 = 0; \
- r3 = 1; \
- r4 = -1; \
- r5 = 0; \
- call %[bpf_arena_alloc_pages]; \
- r6 = r0; \
- r0 = 100; \
- if r6 == 0 goto 9f; \
- r7 = r6; \
- r1 = 0x1122334455667788 ll; \
- *(u64 *)(r6 + 0) = r1; \
- *(u32 *)(r6 + 8) = r1; \
- *(u16 *)(r6 + 12) = r1; \
- *(u8 *)(r6 + 14) = r1; \
- *(u64 *)(r6 + 16) = 0x1234; \
- *(u32 *)(r6 + 24) = 0x5678; \
- *(u16 *)(r6 + 28) = 0x9a; \
- *(u8 *)(r6 + 30) = 0xbc; \
- r0 = 1; \
- r2 = *(u64 *)(r6 + 0); \
- if r2 != r1 goto 9f; \
- r0 = 2; \
- r2 = *(u32 *)(r6 + 8); \
- if r2 != 0x55667788 goto 9f; \
- r0 = 3; \
- r2 = *(u16 *)(r6 + 12); \
- if r2 != 0x7788 goto 9f; \
- r0 = 4; \
- r2 = *(u8 *)(r6 + 14); \
- if r2 != 0x88 goto 9f; \
- r0 = 5; \
- r2 = *(u64 *)(r6 + 16); \
- if r2 != 0x1234 goto 9f; \
- r0 = 6; \
- r2 = *(u32 *)(r6 + 24); \
- if r2 != 0x5678 goto 9f; \
- r0 = 7; \
- r2 = *(u16 *)(r6 + 28); \
- if r2 != 0x9a goto 9f; \
- r0 = 8; \
- r2 = *(u8 *)(r6 + 30); \
- if r2 != 0xbc goto 9f; \
- /* the address is what it was */ \
- r0 = 9; \
- if r6 != r7 goto 9f; \
- r0 = 10; \
- r7 >>= 32; \
- if r7 == 0 goto 9f; \
- r0 = 0; \
-9: exit; \
-" :
- : __imm_addr(arena),
- __imm(bpf_arena_alloc_pages)
- : __clobber_all);
-}
-
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: load into the register that holds the address")
-__success __retval(0)
-__load_if_JITed()
-__naked void ld_into_base(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- r2 = 0; \
- r3 = 1; \
- r4 = -1; \
- r5 = 0; \
- call %[bpf_arena_alloc_pages]; \
- r6 = r0; \
- r0 = 100; \
- if r6 == 0 goto 9f; \
- r1 = 77; \
- *(u64 *)(r6 + 0) = r1; \
- r6 = *(u64 *)(r6 + 0); \
- r0 = 1; \
- if r6 != 77 goto 9f; \
- r0 = 0; \
-9: exit; \
-" :
- : __imm_addr(arena),
- __imm(bpf_arena_alloc_pages)
- : __clobber_all);
-}
-
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: sign extending load")
-__success __retval(0)
-__load_if_JITed()
-__naked void ldsx(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- r2 = 0; \
- r3 = 1; \
- r4 = -1; \
- r5 = 0; \
- call %[bpf_arena_alloc_pages]; \
- r6 = r0; \
- r0 = 100; \
- if r6 == 0 goto 9f; \
- r7 = r6; \
- *(u64 *)(r6 + 0) = 0x80; \
- .8byte %[ldsx_insn]; /* r2 = *(s8 *)(r6 + 0) */ \
- r0 = 1; \
- if r2 != -128 goto 9f; \
- r0 = 2; \
- if r6 != r7 goto 9f; \
- r0 = 0; \
-9: exit; \
-" :
- : __imm_addr(arena),
- __imm(bpf_arena_alloc_pages),
- __imm_insn(ldsx_insn, BPF_RAW_INSN(BPF_LDX | BPF_MEMSX | BPF_B,
- BPF_REG_2, BPF_REG_6, 0, 0))
- : __clobber_all);
-}
-
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: address loaded from arena")
-__success __retval(0)
-__load_if_JITed()
-__naked void ptr_chase(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- r2 = 0; \
- r3 = 1; \
- r4 = -1; \
- r5 = 0; \
- call %[bpf_arena_alloc_pages]; \
- r6 = r0; \
- r0 = 100; \
- if r6 == 0 goto 9f; \
- /* page[0] = &page[64]; page[64] = 5; */ \
- r1 = r6; \
- r1 += 64; \
- *(u64 *)(r6 + 0) = r1; \
- *(u64 *)(r1 + 0) = 5; \
- r2 = *(u64 *)(r6 + 0); \
- r0 = 1; \
- if r2 != r1 goto 9f; \
- r3 = *(u64 *)(r2 + 0); \
- r0 = 2; \
- if r3 != 5 goto 9f; \
- r0 = 0; \
-9: exit; \
-" :
- : __imm_addr(arena),
- __imm(bpf_arena_alloc_pages)
- : __clobber_all);
-}
-
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: atomics")
-__success __retval(0)
-__load_if_JITed()
-__naked void atomics(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- r2 = 0; \
- r3 = 1; \
- r4 = -1; \
- r5 = 0; \
- call %[bpf_arena_alloc_pages]; \
- r6 = r0; \
- r0 = 100; \
- if r6 == 0 goto 9f; \
- r7 = r6; \
- *(u64 *)(r6 + 8) = 1; \
- r1 = 2; \
- lock *(u64 *)(r6 + 8) += r1; \
- r1 = 4; \
- .8byte %[fetch_add_insn]; /* r1 = atomic_fetch_add((u64 *)(r6 + 8), r1) */ \
- r0 = 1; \
- if r1 != 3 goto 9f; \
- r1 = 8; \
- .8byte %[xchg_insn]; /* r1 = xchg_64(r6 + 8, r1) */ \
- r0 = 2; \
- if r1 != 7 goto 9f; \
- r0 = 8; \
- r1 = 16; \
- .8byte %[cmpxchg_insn]; /* r0 = cmpxchg_64(r6 + 8, r0, r1) */ \
- r2 = r0; \
- r0 = 3; \
- if r2 != 8 goto 9f; \
- r2 = *(u64 *)(r6 + 8); \
- r0 = 4; \
- if r2 != 16 goto 9f; \
- r0 = 5; \
- if r6 != r7 goto 9f; \
- r0 = 0; \
-9: exit; \
-" :
- : __imm_addr(arena),
- __imm(bpf_arena_alloc_pages),
- __imm_insn(fetch_add_insn, BPF_ATOMIC_OP(BPF_DW, BPF_ADD | BPF_FETCH,
- BPF_REG_6, BPF_REG_1, 8)),
- __imm_insn(xchg_insn, BPF_ATOMIC_OP(BPF_DW, BPF_XCHG, BPF_REG_6, BPF_REG_1, 8)),
- __imm_insn(cmpxchg_insn, BPF_ATOMIC_OP(BPF_DW, BPF_CMPXCHG, BPF_REG_6, BPF_REG_1, 8))
- : __clobber_all);
-}
-
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: fetching and/or/xor")
-__success __retval(0)
-__load_if_JITed()
-__naked void fetch_and_or_xor(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- r2 = 0; \
- r3 = 1; \
- r4 = -1; \
- r5 = 0; \
- call %[bpf_arena_alloc_pages]; \
- r6 = r0; \
- r0 = 100; \
- if r6 == 0 goto 9f; \
- r7 = 0x11000000110 ll; \
- *(u64 *)(r6 + 8) = r7; \
- r0 = r6; \
- r0 ^= 0x55; \
- r1 = 0x1100000011 ll; \
- .8byte %[fetch_or_dw]; /* r1 = atomic_fetch_or((u64 *)(r6 + 8), r1) */ \
- r2 = r6; \
- r2 ^= 0x55; \
- if r0 == r2 goto 1f; \
- r0 = 99; \
- goto 9f; \
-1: r0 = 1; \
- if r1 != r7 goto 9f; \
- r7 = 0x11100000111 ll; \
- r1 = 0x11000000111 ll; \
- .8byte %[fetch_and_dw]; /* r1 = atomic_fetch_and((u64 *)(r6 + 8), r1) */ \
- r0 = 2; \
- if r1 != r7 goto 9f; \
- r7 = 0x11000000111 ll; \
- r1 = 0x101 ll; \
- .8byte %[fetch_xor_dw]; /* r1 = atomic_fetch_xor((u64 *)(r6 + 8), r1) */ \
- r0 = 3; \
- if r1 != r7 goto 9f; \
- r7 = 0x11000000010 ll; \
- r2 = *(u64 *)(r6 + 8); \
- r0 = 4; \
- if r2 != r7 goto 9f; \
- *(u32 *)(r6 + 16) = 0x110; \
- r1 = 0xdead000000000011 ll; \
- .8byte %[fetch_or_w]; /* r1 = atomic_fetch_or((u32 *)(r6 + 16), r1) */ \
- r0 = 5; \
- if r1 != 0x110 goto 9f; \
- r1 = 0xdead000000000101 ll; \
- .8byte %[fetch_and_w]; /* r1 = atomic_fetch_and((u32 *)(r6 + 16), r1) */ \
- r0 = 6; \
- if r1 != 0x111 goto 9f; \
- r1 = 0xdead000000000011 ll; \
- .8byte %[fetch_xor_w]; /* r1 = atomic_fetch_xor((u32 *)(r6 + 16), r1) */ \
- r0 = 7; \
- if r1 != 0x101 goto 9f; \
- r2 = *(u32 *)(r6 + 16); \
- r0 = 8; \
- if r2 != 0x110 goto 9f; \
- r0 = 0; \
-9: exit; \
-" :
- : __imm_addr(arena),
- __imm(bpf_arena_alloc_pages),
- __imm_insn(fetch_or_dw, BPF_ATOMIC_OP(BPF_DW, BPF_OR | BPF_FETCH,
- BPF_REG_6, BPF_REG_1, 8)),
- __imm_insn(fetch_and_dw, BPF_ATOMIC_OP(BPF_DW, BPF_AND | BPF_FETCH,
- BPF_REG_6, BPF_REG_1, 8)),
- __imm_insn(fetch_xor_dw, BPF_ATOMIC_OP(BPF_DW, BPF_XOR | BPF_FETCH,
- BPF_REG_6, BPF_REG_1, 8)),
- __imm_insn(fetch_or_w, BPF_ATOMIC_OP(BPF_W, BPF_OR | BPF_FETCH,
- BPF_REG_6, BPF_REG_1, 16)),
- __imm_insn(fetch_and_w, BPF_ATOMIC_OP(BPF_W, BPF_AND | BPF_FETCH,
- BPF_REG_6, BPF_REG_1, 16)),
- __imm_insn(fetch_xor_w, BPF_ATOMIC_OP(BPF_W, BPF_XOR | BPF_FETCH,
- BPF_REG_6, BPF_REG_1, 16))
- : __clobber_all);
-}
-
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: fetching and/or/xor with the operand in r0")
-__success __retval(0)
-__load_if_JITed()
-__naked void fetch_and_or_xor_r0(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- r2 = 0; \
- r3 = 1; \
- r4 = -1; \
- r5 = 0; \
- call %[bpf_arena_alloc_pages]; \
- r6 = r0; \
- r0 = 100; \
- if r6 == 0 goto 9f; \
- r7 = 0x11000000110 ll; \
- *(u64 *)(r6 + 8) = r7; \
- r0 = 0x1100000011 ll; \
- .8byte %[fetch_or_dw]; /* r0 = atomic_fetch_or((u64 *)(r6 + 8), r0) */ \
- r2 = r0; \
- r0 = 1; \
- if r2 != r7 goto 9f; \
- r7 = 0x11100000111 ll; \
- r0 = 0x11000000111 ll; \
- .8byte %[fetch_and_dw]; /* r0 = atomic_fetch_and((u64 *)(r6 + 8), r0) */ \
- r2 = r0; \
- r0 = 2; \
- if r2 != r7 goto 9f; \
- r7 = 0x11000000111 ll; \
- r0 = 0x101 ll; \
- .8byte %[fetch_xor_dw]; /* r0 = atomic_fetch_xor((u64 *)(r6 + 8), r0) */ \
- r2 = r0; \
- r0 = 3; \
- if r2 != r7 goto 9f; \
- r7 = 0x11000000010 ll; \
- r2 = *(u64 *)(r6 + 8); \
- r0 = 4; \
- if r2 != r7 goto 9f; \
- *(u32 *)(r6 + 16) = 0x110; \
- r0 = 0xdead000000000011 ll; \
- .8byte %[fetch_or_w]; /* r0 = atomic_fetch_or((u32 *)(r6 + 16), r0) */ \
- r2 = r0; \
- r0 = 5; \
- if r2 != 0x110 goto 9f; \
- r0 = 0xdead000000000101 ll; \
- .8byte %[fetch_and_w]; /* r0 = atomic_fetch_and((u32 *)(r6 + 16), r0) */ \
- r2 = r0; \
- r0 = 6; \
- if r2 != 0x111 goto 9f; \
- r0 = 0xdead000000000011 ll; \
- .8byte %[fetch_xor_w]; /* r0 = atomic_fetch_xor((u32 *)(r6 + 16), r0) */ \
- r2 = r0; \
- r0 = 7; \
- if r2 != 0x101 goto 9f; \
- r2 = *(u32 *)(r6 + 16); \
- r0 = 8; \
- if r2 != 0x110 goto 9f; \
- r0 = 0; \
-9: exit; \
-" :
- : __imm_addr(arena),
- __imm(bpf_arena_alloc_pages),
- __imm_insn(fetch_or_dw, BPF_ATOMIC_OP(BPF_DW, BPF_OR | BPF_FETCH,
- BPF_REG_6, BPF_REG_0, 8)),
- __imm_insn(fetch_and_dw, BPF_ATOMIC_OP(BPF_DW, BPF_AND | BPF_FETCH,
- BPF_REG_6, BPF_REG_0, 8)),
- __imm_insn(fetch_xor_dw, BPF_ATOMIC_OP(BPF_DW, BPF_XOR | BPF_FETCH,
- BPF_REG_6, BPF_REG_0, 8)),
- __imm_insn(fetch_or_w, BPF_ATOMIC_OP(BPF_W, BPF_OR | BPF_FETCH,
- BPF_REG_6, BPF_REG_0, 16)),
- __imm_insn(fetch_and_w, BPF_ATOMIC_OP(BPF_W, BPF_AND | BPF_FETCH,
- BPF_REG_6, BPF_REG_0, 16)),
- __imm_insn(fetch_xor_w, BPF_ATOMIC_OP(BPF_W, BPF_XOR | BPF_FETCH,
- BPF_REG_6, BPF_REG_0, 16))
- : __clobber_all);
-}
-
-/*
- * The page is gone, so the access faults. The fetched register has to come back
- * 0 and everything else has to be left alone, r0 included. Reading the address
- * again has to give 0 as well: had the fault been recovered by the arena mapping
- * its zeroed scratch page instead of by the JIT's exception table entry, the
- * access would have succeeded and left a 1 there.
- */
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: fetching or through a number to a freed page")
-__success __retval(0)
-__load_if_JITed()
-__naked void fetch_or_freed_page(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- r2 = 0; \
- r3 = 1; \
- r4 = -1; \
- r5 = 0; \
- call %[bpf_arena_alloc_pages]; \
- r6 = r0; \
- r0 = 100; \
- if r6 == 0 goto 9f; \
- *(u64 *)(r6 + 8) = 0x55; \
- r1 = %[arena] ll; \
- r2 = r6; \
- r3 = 1; \
- call %[bpf_arena_free_pages]; \
- r0 = r6; \
- r0 ^= 0x55; \
- r1 = 1; \
- .8byte %[fetch_or_dw]; /* faults: r1 = atomic_fetch_or((u64 *)(r6 + 8), r1) */ \
- r2 = r6; \
- r2 ^= 0x55; \
- if r0 == r2 goto 1f; \
- r0 = 1; \
- goto 9f; \
-1: r0 = 2; \
- if r1 != 0 goto 9f; \
- r3 = *(u64 *)(r6 + 8); \
- r0 = 3; \
- if r3 != 0 goto 9f; \
- r0 = 0; \
-9: exit; \
-" :
- : __imm_addr(arena),
- __imm(bpf_arena_alloc_pages),
- __imm(bpf_arena_free_pages),
- __imm_insn(fetch_or_dw, BPF_ATOMIC_OP(BPF_DW, BPF_OR | BPF_FETCH,
- BPF_REG_6, BPF_REG_1, 8))
- : __clobber_all);
-}
-
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: fetching or with the operand in r0 to a freed page")
-__success __retval(0)
-__load_if_JITed()
-__naked void fetch_or_r0_freed_page(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- r2 = 0; \
- r3 = 1; \
- r4 = -1; \
- r5 = 0; \
- call %[bpf_arena_alloc_pages]; \
- r6 = r0; \
- r0 = 100; \
- if r6 == 0 goto 9f; \
- *(u64 *)(r6 + 8) = 0x55; \
- r1 = %[arena] ll; \
- r2 = r6; \
- r3 = 1; \
- call %[bpf_arena_free_pages]; \
- r0 = 1; \
- .8byte %[fetch_or_dw]; /* faults: r0 = atomic_fetch_or((u64 *)(r6 + 8), r0) */ \
- r2 = r0; \
- r0 = 1; \
- if r2 != 0 goto 9f; \
- r3 = *(u64 *)(r6 + 8); \
- r0 = 2; \
- if r3 != 0 goto 9f; \
- r0 = 0; \
-9: exit; \
-" :
- : __imm_addr(arena),
- __imm(bpf_arena_alloc_pages),
- __imm(bpf_arena_free_pages),
- __imm_insn(fetch_or_dw, BPF_ATOMIC_OP(BPF_DW, BPF_OR | BPF_FETCH,
- BPF_REG_6, BPF_REG_0, 8))
- : __clobber_all);
-}
-
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: cmpxchg through r0")
-__success __retval(0)
-__load_if_JITed()
-__naked void cmpxchg_r0(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- r2 = 0; \
- r3 = 1; \
- r4 = -1; \
- r5 = 0; \
- call %[bpf_arena_alloc_pages]; \
- r6 = r0; \
- r0 = 100; \
- if r6 == 0 goto 9f; \
- /* The address is in r0. The value at the address is not equal to it. */ \
- *(u64 *)(r6 + 0) = 3; \
- r0 = r6; \
- r1 = 5; \
- .8byte %[cmpxchg_insn]; /* r0 = cmpxchg_64(r0 + 0, r0, r1) */ \
- r2 = r0; \
- r0 = 1; \
- if r2 != 3 goto 9f; \
- r2 = *(u64 *)(r6 + 0); \
- r0 = 2; \
- if r2 != 3 goto 9f; \
- r0 = 0; \
-9: exit; \
-" :
- : __imm_addr(arena),
- __imm(bpf_arena_alloc_pages),
- __imm_insn(cmpxchg_insn, BPF_ATOMIC_OP(BPF_DW, BPF_CMPXCHG, BPF_REG_0, BPF_REG_1, 0))
- : __clobber_all);
-}
-
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: xchg into the register that holds the address")
-__success __retval(0)
-__load_if_JITed()
-__naked void xchg_into_base(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- r2 = 0; \
- r3 = 1; \
- r4 = -1; \
- r5 = 0; \
- call %[bpf_arena_alloc_pages]; \
- r6 = r0; \
- r0 = 100; \
- if r6 == 0 goto 9f; \
- *(u64 *)(r6 + 0) = 3; \
- r1 = r6; \
- .8byte %[xchg_insn]; /* r1 = xchg_64(r1 + 0, r1) */ \
- r0 = 1; \
- if r1 != 3 goto 9f; \
- r2 = *(u64 *)(r6 + 0); \
- r0 = 2; \
- if r2 != r6 goto 9f; \
- r0 = 0; \
-9: exit; \
-" :
- : __imm_addr(arena),
- __imm(bpf_arena_alloc_pages),
- __imm_insn(xchg_insn, BPF_ATOMIC_OP(BPF_DW, BPF_XCHG, BPF_REG_1, BPF_REG_1, 0))
- : __clobber_all);
-}
-
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: xchg into the register that holds a pointer to stack")
-__failure __msg("misaligned access off (0x0; 0xffffffffffffffff)+0 size 8")
-__naked void xchg_into_stack_ptr(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- r1 = 0; \
- *(u64 *)(r10 - 8) = r1; \
- r1 = r10; \
- r1 += -8; \
- .8byte %[xchg_insn]; /* r1 = xchg_64(r1 + 0, r1) */ \
- r0 = 0; \
- exit; \
-" :
- : __imm_addr(arena),
- __imm_insn(xchg_insn, BPF_ATOMIC_OP(BPF_DW, BPF_XCHG, BPF_REG_1, BPF_REG_1, 0))
- : __clobber_all);
-}
-
-#ifdef CAN_USE_LOAD_ACQ_STORE_REL
-
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: load-acquire and store-release")
-__success __retval(0)
-__load_if_JITed()
-__naked void load_acq_store_rel(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- r2 = 0; \
- r3 = 1; \
- r4 = -1; \
- r5 = 0; \
- call %[bpf_arena_alloc_pages]; \
- r6 = r0; \
- r0 = 100; \
- if r6 == 0 goto 9f; \
- r7 = r6; \
- r1 = 0x1234; \
- .8byte %[store_release_insn]; /* store_release((u64 *)(r6 + 8), r1) */ \
- .8byte %[load_acquire_insn]; /* r2 = load_acquire((u64 *)(r6 + 8)) */ \
- r0 = 1; \
- if r2 != 0x1234 goto 9f; \
- .8byte %[load_acquire8_insn]; /* w2 = load_acquire((u8 *)(r6 + 8)) */ \
- r0 = 2; \
- if r2 != 0x34 goto 9f; \
- r0 = 3; \
- if r6 != r7 goto 9f; \
- r0 = 0; \
-9: exit; \
-" :
- : __imm_addr(arena),
- __imm(bpf_arena_alloc_pages),
- __imm_insn(store_release_insn,
- BPF_ATOMIC_OP(BPF_DW, BPF_STORE_REL, BPF_REG_6, BPF_REG_1, 8)),
- __imm_insn(load_acquire_insn,
- BPF_ATOMIC_OP(BPF_DW, BPF_LOAD_ACQ, BPF_REG_2, BPF_REG_6, 8)),
- __imm_insn(load_acquire8_insn,
- BPF_ATOMIC_OP(BPF_B, BPF_LOAD_ACQ, BPF_REG_2, BPF_REG_6, 8))
- : __clobber_all);
-}
-
-#endif /* CAN_USE_LOAD_ACQ_STORE_REL */
-
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: number that is not an address in arena")
-__success __retval(0)
-__load_if_JITed()
-__naked void not_in_arena(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- /* nothing is allocated: all loads read 0, stores are dropped */ \
- r6 = 0xdeadbeef00000000 ll; \
- r0 = 1; \
- r2 = *(u64 *)(r6 + 0); \
- if r2 != 0 goto 9f; \
- r0 = 2; \
- r2 = *(u8 *)(r6 - 32768); \
- if r2 != 0 goto 9f; \
- r6 = 0x12345678ffffffff ll; \
- r0 = 3; \
- r2 = *(u64 *)(r6 + 32760); \
- if r2 != 0 goto 9f; \
- *(u64 *)(r6 + 32760) = 1; \
- *(u8 *)(r6 + 32767) = r2; \
- r1 = 1; \
- lock *(u64 *)(r6 + 32760) += r1; \
- r0 = 0; \
-9: exit; \
-" :
- : __imm_addr(arena)
- : __clobber_all);
-}
-
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: store through the address of the stack as a number")
-__success __retval(0)
-__load_if_JITed()
-__naked void stack_addr_as_number(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- *(u64 *)(r10 - 8) = 5; \
- r6 = r10; \
- r6 |= 0; \
- /* r6 is a number now. The store goes to arena, not to the stack. */ \
- *(u64 *)(r6 - 8) = 7; \
- r1 = 9; \
- *(u64 *)(r6 - 8) = r1; \
- lock *(u64 *)(r6 - 8) += r1; \
- r2 = *(u64 *)(r10 - 8); \
- r0 = 1; \
- if r2 != 5 goto 9f; \
- r0 = 0; \
-9: exit; \
-" :
- : __imm_addr(arena)
- : __clobber_all);
-}
-
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: 64-bit math on the address stays 64-bit")
-__success __retval(0)
-__load_if_JITed()
-__naked void alu64_after_access(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- r2 = 0; \
- r3 = 1; \
- r4 = -1; \
- r5 = 0; \
- call %[bpf_arena_alloc_pages]; \
- r6 = r0; \
- r0 = 100; \
- if r6 == 0 goto 9f; \
- r2 = *(u64 *)(r6 + 0); \
- r7 = r6; \
- r7 += 8; \
- r7 -= r6; \
- r0 = 1; \
- if r7 != 8 goto 9f; \
- r7 = r6; \
- r7 >>= 32; \
- r0 = 2; \
- if r7 == 0 goto 9f; \
- r0 = 0; \
-9: exit; \
-" :
- : __imm_addr(arena),
- __imm(bpf_arena_alloc_pages)
- : __clobber_all);
-}
-
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: st of an immediate changes no register")
-__success __retval(0)
-__load_if_JITed()
-__naked void st_keeps_regs(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- r2 = 0; \
- r3 = 1; \
- r4 = -1; \
- r5 = 0; \
- call %[bpf_arena_alloc_pages]; \
- r6 = r0; \
- r0 = 100; \
- if r6 == 0 goto 9f; \
- r0 = r6; \
- r1 = 0x1111; \
- r2 = 0x2222; \
- *(u64 *)(r0 + 0) = 5; \
- r3 = r0; \
- r0 = 1; \
- if r3 != r6 goto 9f; \
- r0 = 2; \
- if r1 != 0x1111 goto 9f; \
- r0 = 3; \
- if r2 != 0x2222 goto 9f; \
- r0 = 0x3333; \
- r1 = r6; \
- *(u32 *)(r1 + 8) = -7; \
- r3 = r0; \
- r0 = 4; \
- if r3 != 0x3333 goto 9f; \
- r0 = 5; \
- if r1 != r6 goto 9f; \
- r0 = 6; \
- r3 = *(u64 *)(r6 + 0); \
- if r3 != 5 goto 9f; \
- r0 = 7; \
- r3 = *(u64 *)(r6 + 8); \
- r4 = 0xfffffff9 ll; \
- if r3 != r4 goto 9f; \
- r0 = 0; \
-9: exit; \
-" :
- : __imm_addr(arena),
- __imm(bpf_arena_alloc_pages)
- : __clobber_all);
-}
-
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: access through a number with garbage in the upper half")
-__success __retval(0)
-__load_if_JITed()
-__naked void st_value_garbage(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- r2 = 0; \
- r3 = 1; \
- r4 = -1; \
- r5 = 0; \
- call %[bpf_arena_alloc_pages]; \
- r6 = r0; \
- r0 = 100; \
- if r6 == 0 goto 9f; \
- r1 = 0xdeadbeef00001000 ll; \
- *(u64 *)(r6 + 2048) = r1; \
- r7 = *(u64 *)(r6 + 2048); \
- r8 = *(u64 *)(r6 + 2048); \
- r1 = 3; \
- *(u64 *)(r8 + 0) = 1; \
- r0 = 1; \
- if r8 != r7 goto 9f; \
- *(u8 *)(r8 + 1) = 1; \
- r0 = 2; \
- if r8 != r7 goto 9f; \
- *(u32 *)(r8 + 4) = r1; \
- r0 = 3; \
- if r8 != r7 goto 9f; \
- r2 = *(u16 *)(r8 + 2); \
- r0 = 4; \
- if r8 != r7 goto 9f; \
- r0 = 5; \
- if r1 != 3 goto 9f; \
- r0 = 0; \
-9: exit; \
-" :
- : __imm_addr(arena),
- __imm(bpf_arena_alloc_pages)
- : __clobber_all);
-}
-
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: access through a number without the upper half")
-__success __retval(0)
-__load_if_JITed()
-__naked void st_value_small(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- r2 = 0; \
- r3 = 1; \
- r4 = -1; \
- r5 = 0; \
- call %[bpf_arena_alloc_pages]; \
- r6 = r0; \
- r0 = 100; \
- if r6 == 0 goto 9f; \
- r1 = 0x1000 ll; \
- *(u64 *)(r6 + 2048) = r1; \
- r7 = *(u64 *)(r6 + 2048); \
- r8 = *(u64 *)(r6 + 2048); \
- r1 = 3; \
- *(u64 *)(r8 + 0) = 1; \
- r0 = 1; \
- if r8 != r7 goto 9f; \
- *(u8 *)(r8 + 1) = 1; \
- r0 = 2; \
- if r8 != r7 goto 9f; \
- *(u32 *)(r8 + 4) = r1; \
- r0 = 3; \
- if r8 != r7 goto 9f; \
- r2 = *(u16 *)(r8 + 2); \
- r0 = 4; \
- if r8 != r7 goto 9f; \
- r0 = 5; \
- if r1 != 3 goto 9f; \
- r0 = 0; \
-9: exit; \
-" :
- : __imm_addr(arena),
- __imm(bpf_arena_alloc_pages)
- : __clobber_all);
-}
-
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: atomics through a number that is not an address in arena")
-__success __retval(0)
-__load_if_JITed()
-__naked void atomic_value(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- r2 = 0; \
- r3 = 1; \
- r4 = -1; \
- r5 = 0; \
- call %[bpf_arena_alloc_pages]; \
- r6 = r0; \
- r0 = 100; \
- if r6 == 0 goto 9f; \
- r1 = 0xdeadbeef00001000 ll; \
- *(u64 *)(r6 + 2048) = r1; \
- r7 = *(u64 *)(r6 + 2048); \
- r8 = *(u64 *)(r6 + 2048); \
- r1 = 3; \
- lock *(u64 *)(r8 + 0) += r1; \
- r0 = 1; \
- if r8 != r7 goto 9f; \
- .8byte %[fetch_add_insn]; \
- r0 = 2; \
- if r8 != r7 goto 9f; \
- .8byte %[xchg_insn]; \
- r0 = 3; \
- if r8 != r7 goto 9f; \
- r0 = 0; \
- .8byte %[cmpxchg_insn]; \
- r0 = 4; \
- if r8 != r7 goto 9f; \
- r0 = 0; \
-9: exit; \
-" :
- : __imm_addr(arena),
- __imm(bpf_arena_alloc_pages),
- __imm_insn(fetch_add_insn, BPF_ATOMIC_OP(BPF_DW, BPF_ADD | BPF_FETCH,
- BPF_REG_8, BPF_REG_1, 0)),
- __imm_insn(xchg_insn, BPF_ATOMIC_OP(BPF_DW, BPF_XCHG, BPF_REG_8, BPF_REG_1, 0)),
- __imm_insn(cmpxchg_insn, BPF_ATOMIC_OP(BPF_DW, BPF_CMPXCHG, BPF_REG_8, BPF_REG_1, 0))
- : __clobber_all);
-}
-
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: number and pointer to arena at the same insn")
-__success __retval(0)
-__load_if_JITed()
-__naked void mixed_number_arena(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- r2 = 0; \
- r3 = 1; \
- r4 = -1; \
- r5 = 0; \
- call %[bpf_arena_alloc_pages]; \
- r6 = r0; \
- r0 = 100; \
- if r6 == 0 goto 9f; \
- *(u64 *)(r6 + 8) = 0x1234; \
- /* a number that the verifier does not know, 0 at run time */ \
- r7 = *(u64 *)(r6 + 16); \
- r8 = r6; \
- if r7 != 0 goto 1f; \
- .8byte %[cast_kern_insn]; \
-1: *(u8 *)(r8 + 0) = 1; \
- *(u8 *)(r8 + 1) = r7; \
- r2 = *(u8 *)(r8 + 1); \
- lock *(u64 *)(r8 + 24) += r7; \
- r0 = 0; \
- if r7 != 0 goto 9f; \
- /* pointer to arena only: JIT adds all 64 bits of r8 to the base */ \
- r0 = 2; \
- r2 = *(u64 *)(r8 + 8); \
- if r2 != 0x1234 goto 9f; \
- r0 = 3; \
- r2 = *(u8 *)(r6 + 0); \
- if r2 != 1 goto 9f; \
- r0 = 0; \
-9: exit; \
-" :
- : __imm_addr(arena),
- __imm(bpf_arena_alloc_pages),
- __imm_insn(cast_kern_insn, BPF_RAW_INSN(BPF_ALU64 | BPF_MOV | BPF_X,
- BPF_REG_8, BPF_REG_8, 1, 1))
- : __clobber_all);
-}
-
-SEC("syscall")
-__description("arena_scalar: no flag, no access through a number")
-__failure __msg("R6 invalid mem access 'scalar'")
-__naked void no_flag(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- r6 = 0x100000000000 ll; \
- r0 = *(u64 *)(r6 + 0); \
- exit; \
-" :
- : __imm_addr(arena)
- : __clobber_all);
-}
-
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: pointer that is NULL is an address in arena")
-__success __retval(0)
-__load_if_JITed()
-__naked void null_ptr(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- r1 = 0; \
- *(u32 *)(r10 - 4) = r1; \
- r2 = r10; \
- r2 += -4; \
- r1 = %[hash] ll; \
- call %[bpf_map_lookup_elem]; \
- r1 = r0; \
- r0 = 1; \
- if r1 != 0 goto 9f; \
- /* nothing is allocated: the load reads 0 */ \
- r0 = *(u64 *)(r1 + 0); \
-9: exit; \
-" :
- : __imm_addr(arena),
- __imm_addr(hash),
- __imm(bpf_map_lookup_elem)
- : __clobber_all);
-}
-
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: pointer that is NULL with an offset is an address in arena")
-__success __retval(0)
-__load_if_JITed()
-__naked void null_ptr_off(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- r1 = 0; \
- *(u32 *)(r10 - 4) = r1; \
- r2 = r10; \
- r2 += -4; \
- r1 = %[hash] ll; \
- call %[bpf_map_lookup_elem]; \
- r1 = r0; \
- r0 = 1; \
- if r1 != 0 goto 9f; \
- r1 += 8; \
- *(u64 *)(r1 + 0) = 5; \
- r0 = *(u64 *)(r1 + 0); \
-9: exit; \
-" :
- : __imm_addr(arena),
- __imm_addr(hash),
- __imm(bpf_map_lookup_elem)
- : __clobber_all);
-}
-
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: number that is less than a page is an address in arena")
-__success __retval(0)
-__load_if_JITed()
-__naked void small_number(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- call %[bpf_get_prandom_u32]; \
- r1 = r0; \
- r1 &= 0xfff; \
- r0 = *(u64 *)(r1 + 0); \
- exit; \
-" :
- : __imm_addr(arena),
- __imm(bpf_get_prandom_u32)
- : __clobber_all);
-}
-
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: number is not a pointer for a helper")
-__failure __msg("R2 type=scalar expected=")
-__naked void helper_arg(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- r2 = 0; \
- r3 = 1; \
- r4 = -1; \
- r5 = 0; \
- call %[bpf_arena_alloc_pages]; \
- r6 = r0; \
- r0 = 100; \
- if r6 == 0 goto 9f; \
- r1 = %[hash] ll; \
- r2 = r6; \
- call %[bpf_map_lookup_elem]; \
- r0 = 0; \
-9: exit; \
-" :
- : __imm_addr(arena),
- __imm_addr(hash),
- __imm(bpf_arena_alloc_pages),
- __imm(bpf_map_lookup_elem)
- : __clobber_all);
-}
-
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: number and pointer to stack at the same insn")
-__failure __msg("same insn cannot be used with different pointers")
-__load_if_JITed()
-__naked void mixed_number_stack(void)
-{
- asm volatile (" \
- r1 = %[arena] ll; \
- r2 = 0; \
- r3 = 1; \
- r4 = -1; \
- r5 = 0; \
- call %[bpf_arena_alloc_pages]; \
- r6 = r0; \
- r0 = 100; \
- if r6 == 0 goto 9f; \
- *(u64 *)(r10 - 8) = 0; \
- call %[bpf_get_prandom_u32]; \
- if w0 != 0 goto 1f; \
- r6 = r10; \
- r6 += -8; \
-1: r0 = *(u64 *)(r6 + 0); \
- r0 = 0; \
-9: exit; \
-" :
- : __imm_addr(arena),
- __imm(bpf_arena_alloc_pages),
- __imm(bpf_get_prandom_u32)
- : __clobber_all);
-}
-
-static int st_cb(__u64 idx, void *ctx)
-{
- volatile long *p = *(volatile long **)ctx;
-
- p[idx] = 7;
- return 0;
-}
-
-SEC("syscall")
-__arena_scalar
-__description("arena_scalar: store through a number in a callback")
-__success __retval(0)
-__load_if_JITed()
-int st_in_callback(void *unused)
-{
- volatile long *p = bpf_arena_alloc_pages(&arena, NULL, 1, -1, 0);
-
- if (!p)
- return 100;
- bpf_loop(4, st_cb, &p, 0);
- return p[0] + p[3] + p[4] - 14;
-}
-
-char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/test_loader.c b/tools/testing/selftests/bpf/test_loader.c
index 824e5912ebcb..d24e4fedae46 100644
--- a/tools/testing/selftests/bpf/test_loader.c
+++ b/tools/testing/selftests/bpf/test_loader.c
@@ -590,8 +590,6 @@ static int parse_test_spec(struct test_loader *tester,
update_flags(&spec->prog_flags, BPF_F_XDP_HAS_FRAGS, clear);
} else if (strcmp(val, "BPF_F_TEST_REG_INVARIANTS") == 0) {
update_flags(&spec->prog_flags, BPF_F_TEST_REG_INVARIANTS, clear);
- } else if (strcmp(val, "BPF_F_ARENA_SCALAR") == 0) {
- update_flags(&spec->prog_flags, BPF_F_ARENA_SCALAR, clear);
} else /* assume numeric value */ {
err = parse_int(val, &flags, "test prog flags");
if (err)
--
2.55.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH v2 bpf 05/19] bpf: Remove BPF_F_ARENA_SCALAR
2026-10-27 19:02 [PATCH v2 bpf 00/19] bpf: Remove BPF_F_ARENA_SCALAR, fix scalar evolution and scalar forks Alexei Starovoitov
` (3 preceding siblings ...)
2026-10-27 19:02 ` [PATCH v2 bpf 04/19] selftests/bpf: Remove tests for arena access through numbers Alexei Starovoitov
@ 2026-10-27 19:03 ` Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 06/19] bpf, x86: Remove r0_on_stack from fetching atomics in arena Alexei Starovoitov
` (14 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: Alexei Starovoitov @ 2026-10-27 19:03 UTC (permalink / raw)
To: bpf; +Cc: daniel, andrii, eddyz87, memxor
From: Alexei Starovoitov <ast@kernel.org>
Rust has no address spaces. Thus a prog in Rust accesses arena through
scalars. With BPF_F_ARENA_SCALAR the verifier accepts each load and each
store through a scalar as an access to arena.
The verifier rewrites one insn at a time. Thus it must add one insn to each
access. The compiler knows the source of each pointer. It can emit one
addr_space_cast for each pointer, as it does for the __arena pointers of C.
flag: one move for each access cast: one cast for each pointer
ax = (u32)r1 r1 = addr_space_cast(r1, 0, 1)
r2 = *(u64 *)(ax + 0) r2 = *(u64 *)(r1 + 0)
ax = (u32)r1 r3 = *(u64 *)(r1 + 8)
r3 = *(u64 *)(ax + 8)
20 progs of rust-bpf, with the flag and with the casts that a prototype of
an LLVM pass emits:
flag casts
moves or casts 1568 571
insns 14864 13938
jited bytes 72351 69111
No released kernel has the flag. LLVM does not emit the casts for Rust at
this time. Until it does, Rust progs that access arena do not load.
scx_mavd and scx_nitosis set the flag too. They are experimental schedulers
in C in sched-ext/scx, and they use the flag to omit __arena. They must use
__arena again.
This reverts commit 4c651a91bdfc
("bpf: Treat load and store through a number as arena access"). One part
stays: BPF_REG_AX in reg2pt_regs[] of the x86 JIT. The fetching AND/OR/XOR
atomics in arena use it now.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
arch/arm64/net/bpf_jit_comp.c | 5 ---
arch/x86/net/bpf_jit_comp.c | 5 ---
include/linux/bpf_verifier.h | 2 --
include/linux/filter.h | 1 -
include/uapi/linux/bpf.h | 6 ----
kernel/bpf/core.c | 9 +----
kernel/bpf/fixups.c | 39 ---------------------
kernel/bpf/syscall.c | 4 ---
kernel/bpf/verifier.c | 64 ++++++++--------------------------
tools/include/uapi/linux/bpf.h | 6 ----
10 files changed, 16 insertions(+), 125 deletions(-)
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index aaa34c739004..33632fb20997 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -3502,11 +3502,6 @@ bool bpf_jit_supports_arena(void)
return true;
}
-bool bpf_jit_supports_arena_scalar(void)
-{
- return true;
-}
-
bool bpf_jit_supports_insn(struct bpf_insn *insn, bool in_arena)
{
if (!in_arena)
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index fd65ac84cfc7..7bfb7210484e 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -5240,11 +5240,6 @@ bool bpf_jit_supports_arena(void)
return true;
}
-bool bpf_jit_supports_arena_scalar(void)
-{
- return true;
-}
-
bool bpf_jit_supports_insn(struct bpf_insn *insn, bool in_arena)
{
return true;
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index e51984cb3035..e3f7a0b3bad3 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -782,7 +782,6 @@ struct bpf_insn_aux_data {
u64 non_sleepable:1; /* helper/kfunc may be called from non-sleepable context */
u64 is_iter_next:1; /* bpf_iter_<type>_next() kfunc call */
u64 call_with_percpu_alloc_ptr:1; /* {this,per}_cpu_ptr() with prog percpu alloc */
- u64 arena_scalar:1; /* ldx/stx/st/atomic through a number, it's an address in arena */
u64 bb_end:1; /* last instruction of a basic block */
u64 need_scev:1;
u64 alu_state:5; /* used in combination with alu_limit */
@@ -1074,7 +1073,6 @@ struct bpf_verifier_env {
bool strict_alignment; /* perform strict pointer alignment checks */
bool test_state_freq; /* test verifier with different pruning frequency */
bool test_reg_invariants; /* fail verification on register invariants violations */
- bool arena_scalar; /* load and store through a number is arena access */
struct bpf_verifier_state *cur_state; /* current verifier state */
/* Search pruning optimization, array of list_heads for
* lists of struct bpf_verifier_state_list.
diff --git a/include/linux/filter.h b/include/linux/filter.h
index e0218e677a1d..98254742ae37 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -1287,7 +1287,6 @@ bool bpf_jit_supports_far_kfunc_call(void);
bool bpf_jit_supports_exceptions(void);
bool bpf_jit_supports_ptr_xchg(void);
bool bpf_jit_supports_arena(void);
-bool bpf_jit_supports_arena_scalar(void);
bool bpf_jit_supports_insn(struct bpf_insn *insn, bool in_arena);
bool bpf_jit_supports_private_stack(void);
bool bpf_jit_supports_large_stack(void);
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index f29b0aad440c..8d7228834305 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -1357,12 +1357,6 @@ enum bpf_perf_event_type {
/* The verifier internal test flag. Behavior is undefined */
#define BPF_F_TEST_REG_INVARIANTS (1U << 7)
-/*
- * Load and store through a number is an access to the arena of the program
- * at the low 32 bits of the number. It's for programs written in Rust.
- */
-#define BPF_F_ARENA_SCALAR (1U << 8)
-
/* link_create.kprobe_multi.flags used in LINK_CREATE command for
* BPF_TRACE_KPROBE_MULTI attach type to create return probe.
*/
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 0a8ba603f44b..7cf37690d41f 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -2639,8 +2639,7 @@ static bool bpf_prog_select_interpreter(struct bpf_prog *fp)
/*
* Whether a function has an insn that works with the arena base, which a JIT
* sets up in the prologue when bpf_prog_uses_arena_insns() says so. Decided on
- * the final insns of the function: an access through an arena pointer, or
- * through a number in a program loaded with BPF_F_ARENA_SCALAR, has been
+ * the final insns of the function: an access through an arena pointer has been
* rewritten to a BPF_PROBE_MEM32 form by then, a cast to the user address
* space keeps BPF_ADDR_SPACE_CAST, and a kfunc call with an __arena argument,
* which the JIT rebases, has a function model that says so. A load of an
@@ -3518,12 +3517,6 @@ bool __weak bpf_jit_supports_arena(void)
return false;
}
-/* Whether JIT takes BPF_REG_AX as the address of arena access */
-bool __weak bpf_jit_supports_arena_scalar(void)
-{
- return false;
-}
-
bool __weak bpf_jit_supports_insn(struct bpf_insn *insn, bool in_arena)
{
return false;
diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
index f062ef7dd943..41bc37e205e0 100644
--- a/kernel/bpf/fixups.c
+++ b/kernel/bpf/fixups.c
@@ -44,31 +44,6 @@ static bool is_addr_space_cast32(struct bpf_prog *prog, const struct bpf_insn *i
return false;
}
-/*
- * The insn accesses arena through a number. JITs add the base of arena
- * to the register as it is, so the access goes through the low 32 bits of
- * the number in BPF_REG_AX. Registers of the program are not changed: the
- * number may be compared or stored later, and on another path the register
- * may be PTR_TO_ARENA.
- *
- * Constant blinding leaves insns that use BPF_REG_AX alone, so the immediate
- * of st through a number is not blinded.
- */
-static int arena_scalar_access(const struct bpf_insn *insn, struct bpf_insn *buf)
-{
- bool load = BPF_CLASS(insn->code) == BPF_LDX || bpf_atomic_is_load_acq(insn);
- struct bpf_insn *patch = buf;
-
- *patch++ = BPF_MOV32_REG(BPF_REG_AX, load ? insn->src_reg : insn->dst_reg);
- *patch = *insn;
- if (load)
- patch->src_reg = BPF_REG_AX;
- else
- patch->dst_reg = BPF_REG_AX;
- patch++;
- return patch - buf;
-}
-
/* Return the regno defined by the insn, or -1. */
static int insn_def_regno(const struct bpf_insn *insn)
{
@@ -1880,20 +1855,6 @@ int bpf_do_misc_fixups(struct bpf_verifier_env *env)
/* Convert BPF_CLASS(insn->code) == BPF_ALU64 to 32-bit ALU */
insn->code = BPF_ALU | BPF_OP(insn->code) | BPF_SRC(insn->code);
- if (env->insn_aux_data[i + delta].arena_scalar) {
- cnt = arena_scalar_access(insn, insn_buf);
-
- new_prog = bpf_patch_insn_data(env, i + delta, insn_buf, cnt);
- if (!new_prog)
- return -ENOMEM;
-
- delta += cnt - 1;
- prog = new_prog;
- env->prog = prog;
- insn = prog->insnsi + i + delta;
- goto next_insn;
- }
-
/* Make sdiv/smod divide-by-minus-one exceptions impossible. */
if ((insn->code == (BPF_ALU64 | BPF_MOD | BPF_K) ||
insn->code == (BPF_ALU64 | BPF_DIV | BPF_K) ||
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index 90bc7230afa8..60c4d84b9aba 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -2971,13 +2971,9 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr, struct bpf_log_at
BPF_F_XDP_HAS_FRAGS |
BPF_F_XDP_DEV_BOUND_ONLY |
BPF_F_TEST_REG_INVARIANTS |
- BPF_F_ARENA_SCALAR |
BPF_F_TOKEN_FD))
return -EINVAL;
- if ((attr->prog_flags & BPF_F_ARENA_SCALAR) && !bpf_jit_supports_arena_scalar())
- return -EOPNOTSUPP;
-
bpf_prog_load_fixup_attach_type(attr);
if (attr->prog_flags & BPF_F_TOKEN_FD) {
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index a0a5f075d201..c1070aba621b 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -5500,19 +5500,6 @@ static bool is_arena_reg(struct bpf_verifier_env *env, int regno)
return reg->type == PTR_TO_ARENA;
}
-/*
- * There are no address spaces in Rust, addresses of arena are plain numbers.
- * When the program is loaded with BPF_F_ARENA_SCALAR and has an arena a load or
- * a store through a number is an access to arena at the low 32 bits of it,
- * like the access through PTR_TO_ARENA is. The register stays a number.
- */
-static bool is_arena_scalar(struct bpf_verifier_env *env, int regno)
-{
- const struct bpf_reg_state *reg = reg_state(env, regno);
-
- return reg->type == SCALAR_VALUE && env->arena_scalar && env->prog->aux->arena;
-}
-
static bool is_load_acq_unsafe(struct bpf_verifier_env *env, int regno,
struct bpf_insn *insn)
{
@@ -5543,7 +5530,7 @@ static bool atomic_ptr_type_ok(struct bpf_verifier_env *env, int regno,
return false;
if (is_sk_reg(env, regno))
return false;
- if (is_arena_reg(env, regno) || is_arena_scalar(env, regno))
+ if (is_arena_reg(env, regno))
return bpf_jit_supports_insn(insn, true);
if (is_load_acq_unsafe(env, regno, insn))
return false;
@@ -7524,22 +7511,6 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, struct b
static int save_aux_ptr_type(struct bpf_verifier_env *env, enum bpf_reg_type type,
bool allow_trust_mismatch);
-/*
- * Returns the register to check the access of the current insn with.
- * For a number in a program with an arena that is 'arena'.
- */
-static struct bpf_reg_state *mem_access_reg(struct bpf_verifier_env *env, int regno,
- struct bpf_reg_state *arena)
-{
- if (!is_arena_scalar(env, regno))
- return cur_regs(env) + regno;
-
- memset(arena, 0, sizeof(*arena));
- arena->type = PTR_TO_ARENA;
- env->insn_aux_data[env->insn_idx].arena_scalar = true;
- return arena;
-}
-
static int check_load_mem(struct bpf_verifier_env *env, struct bpf_insn *insn,
bool strict_alignment_once, bool is_ldsx,
bool allow_trust_mismatch, const char *ctx)
@@ -7547,7 +7518,6 @@ static int check_load_mem(struct bpf_verifier_env *env, struct bpf_insn *insn,
struct bpf_verifier_state *vstate = env->cur_state;
struct bpf_func_state *state = vstate->frame[vstate->curframe];
struct bpf_reg_state *regs = cur_regs(env);
- struct bpf_reg_state arena, *src_reg;
enum bpf_reg_type src_reg_type;
int err;
@@ -7569,16 +7539,16 @@ static int check_load_mem(struct bpf_verifier_env *env, struct bpf_insn *insn,
if (err)
return err;
- src_reg = mem_access_reg(env, insn->src_reg, &arena);
- src_reg_type = src_reg->type;
+ src_reg_type = regs[insn->src_reg].type;
/*
* check_stack_read_fixed_off() may refine the modification's origin to
* the source stack slot.
*/
bpf_diag_mod_begin(env, ®s[insn->dst_reg], NULL, BPF_DIAG_MOD_WRITE);
- err = check_mem_access(env, env->insn_idx, src_reg, argno_from_reg(insn->src_reg),
- insn->off, BPF_SIZE(insn->code), BPF_READ, insn->dst_reg,
+ err = check_mem_access(env, env->insn_idx, regs + insn->src_reg,
+ argno_from_reg(insn->src_reg), insn->off,
+ BPF_SIZE(insn->code), BPF_READ, insn->dst_reg,
strict_alignment_once, is_ldsx);
/*
* Paths through a kernel struct pointer and through other pointers
@@ -7606,7 +7576,6 @@ static int check_store_reg(struct bpf_verifier_env *env, struct bpf_insn *insn,
struct bpf_verifier_state *vstate = env->cur_state;
struct bpf_func_state *state = vstate->frame[vstate->curframe];
struct bpf_reg_state *regs = cur_regs(env);
- struct bpf_reg_state arena, *dst_reg;
enum bpf_reg_type dst_reg_type;
int err;
@@ -7628,12 +7597,12 @@ static int check_store_reg(struct bpf_verifier_env *env, struct bpf_insn *insn,
if (err)
return err;
- dst_reg = mem_access_reg(env, insn->dst_reg, &arena);
- dst_reg_type = dst_reg->type;
+ dst_reg_type = regs[insn->dst_reg].type;
/* Check if (dst_reg + off) is writeable. */
- err = check_mem_access(env, env->insn_idx, dst_reg, argno_from_reg(insn->dst_reg),
- insn->off, BPF_SIZE(insn->code), BPF_WRITE, insn->src_reg,
+ err = check_mem_access(env, env->insn_idx, regs + insn->dst_reg,
+ argno_from_reg(insn->dst_reg), insn->off,
+ BPF_SIZE(insn->code), BPF_WRITE, insn->src_reg,
strict_alignment_once, false);
err = err ?: save_aux_ptr_type(env, dst_reg_type, false);
@@ -7643,7 +7612,7 @@ static int check_store_reg(struct bpf_verifier_env *env, struct bpf_insn *insn,
static int check_atomic_rmw(struct bpf_verifier_env *env,
struct bpf_insn *insn)
{
- struct bpf_reg_state arena, *dst_reg;
+ struct bpf_reg_state *dst_reg;
int load_reg;
int err;
@@ -7688,9 +7657,6 @@ static int check_atomic_rmw(struct bpf_verifier_env *env,
return -EACCES;
}
- /* load_reg may be dst_reg. Look at dst_reg before it's marked as unknown. */
- dst_reg = mem_access_reg(env, insn->dst_reg, &arena);
-
load_reg = bpf_atomic_load_reg(insn);
if (load_reg >= 0) {
/* check and record load of old value */
@@ -7699,6 +7665,8 @@ static int check_atomic_rmw(struct bpf_verifier_env *env,
return err;
}
+ dst_reg = cur_regs(env) + insn->dst_reg;
+
/* Check whether we can read the memory, with second call for fetch
* case to simulate the register fill.
*/
@@ -20289,17 +20257,16 @@ static int do_check_insn(struct bpf_verifier_env *env, bool *do_print_state)
return check_stack_arg_write(env, state, insn->off, NULL);
}
- struct bpf_reg_state arena, *dst_reg;
enum bpf_reg_type dst_reg_type;
err = check_reg_arg(env, insn->dst_reg, SRC_OP);
if (err)
return err;
- dst_reg = mem_access_reg(env, insn->dst_reg, &arena);
- dst_reg_type = dst_reg->type;
+ dst_reg_type = cur_regs(env)[insn->dst_reg].type;
- err = check_mem_access(env, env->insn_idx, dst_reg, argno_from_reg(insn->dst_reg),
+ err = check_mem_access(env, env->insn_idx, cur_regs(env) + insn->dst_reg,
+ argno_from_reg(insn->dst_reg),
insn->off, BPF_SIZE(insn->code),
BPF_WRITE, -1, false, false);
if (err)
@@ -23935,7 +23902,6 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr,
if (is_priv)
env->test_state_freq = attr->prog_flags & BPF_F_TEST_STATE_FREQ;
env->test_reg_invariants = attr->prog_flags & BPF_F_TEST_REG_INVARIANTS;
- env->arena_scalar = attr->prog_flags & BPF_F_ARENA_SCALAR;
env->explored_states = kvzalloc_objs(struct list_head,
state_htab_size(env),
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index f29b0aad440c..8d7228834305 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -1357,12 +1357,6 @@ enum bpf_perf_event_type {
/* The verifier internal test flag. Behavior is undefined */
#define BPF_F_TEST_REG_INVARIANTS (1U << 7)
-/*
- * Load and store through a number is an access to the arena of the program
- * at the low 32 bits of the number. It's for programs written in Rust.
- */
-#define BPF_F_ARENA_SCALAR (1U << 8)
-
/* link_create.kprobe_multi.flags used in LINK_CREATE command for
* BPF_TRACE_KPROBE_MULTI attach type to create return probe.
*/
--
2.55.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH v2 bpf 06/19] bpf, x86: Remove r0_on_stack from fetching atomics in arena
2026-10-27 19:02 [PATCH v2 bpf 00/19] bpf: Remove BPF_F_ARENA_SCALAR, fix scalar evolution and scalar forks Alexei Starovoitov
` (4 preceding siblings ...)
2026-10-27 19:03 ` [PATCH v2 bpf 05/19] bpf: Remove BPF_F_ARENA_SCALAR Alexei Starovoitov
@ 2026-10-27 19:03 ` Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 07/19] bpf: Clear delta of a register that scalar evolution widens Alexei Starovoitov
` (13 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: Alexei Starovoitov @ 2026-10-27 19:03 UTC (permalink / raw)
To: bpf; +Cc: daniel, andrii, eddyz87, memxor
From: Alexei Starovoitov <ast@kernel.org>
x86 has no fetching AND/OR/XOR. Thus the JIT emits a loop with cmpxchg.
cmpxchg uses RAX, and RAX holds BPF R0. Thus the JIT saves R0 before the
loop.
prog the address is in the JIT saves R0 in
-------------------------- ------------------ -------------------
without the flag a BPF register BPF_REG_AX
with BPF_F_ARENA_SCALAR BPF_REG_AX the stack
The flag is removed. Remove the second variant.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
arch/x86/net/bpf_jit_comp.c | 58 +++++++++----------------------------
1 file changed, 14 insertions(+), 44 deletions(-)
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 7bfb7210484e..d90b6d238b3a 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -1745,26 +1745,20 @@ static int emit_atomic_ld_st_index(u8 **pprog, u32 atomic_op, u32 size,
/*
* A fetching AND/OR/XOR can't be implemented with a single x86 insn, so do a
* CMPXCHG loop. @index_reg is X86_REG_R12 for an arena access or -1 otherwise.
- * Unless @r0_on_stack, the caller has substituted BPF_REG_AX for R0 in
- * @dst_reg and @src_reg.
+ * The caller has saved R0 in BPF_REG_AX and has substituted BPF_REG_AX for R0
+ * in @dst_reg and @src_reg.
*
* Both the load and the CMPXCHG can fault when accessing the arena, so their
* addresses are handed back in @fault[0] and @fault[1]. A fault has to resume
* at @resume, which is past the loop and past the move that delivers the old
* value, but before R0 is restored. Both are NULL for the non-arena case,
* which needs no fixup.
- *
- * R0 is expected to be saved in BPF_REG_AX by the caller, or pushed on the
- * stack if @r0_on_stack. In that case an R0 operand stays in the stack slot,
- * RAX ends up holding the old value that R0 is to return, and the slot is
- * dropped rather than popped.
*/
static int emit_atomic_fetch_rmw(u8 **pprog, struct bpf_insn *insn, u32 dst_reg,
- u32 src_reg, int index_reg, bool r0_on_stack,
+ u32 src_reg, int index_reg,
u8 **fault, u8 **resume)
{
bool is64 = BPF_SIZE(insn->code) == BPF_DW;
- bool operand_on_stack = r0_on_stack && src_reg == BPF_REG_0;
u8 *branch_target, *prog = *pprog;
int err;
@@ -1783,15 +1777,7 @@ static int emit_atomic_fetch_rmw(u8 **pprog, struct bpf_insn *insn, u32 dst_reg,
* Perform the (commutative) operation locally, put the result in
* the AUX_REG.
*/
- if (operand_on_stack) {
- /* mov r11, [rsp]: the operand is the R0 saved there, RAX is the old value */
- if (is64)
- EMIT4(0x4C, 0x8B, 0x1C, 0x24);
- else
- EMIT4(0x44, 0x8B, 0x1C, 0x24);
- } else {
- emit_mov_reg(&prog, is64, AUX_REG, BPF_REG_0);
- }
+ emit_mov_reg(&prog, is64, AUX_REG, BPF_REG_0);
maybe_emit_mod(&prog, AUX_REG, src_reg, is64);
EMIT2(simple_alu_opcodes[BPF_OP(insn->imm)],
add_2reg(0xC0, AUX_REG, src_reg));
@@ -1810,20 +1796,14 @@ static int emit_atomic_fetch_rmw(u8 **pprog, struct bpf_insn *insn, u32 dst_reg,
/* ZF tells us whether we won the race. If it's cleared we need to try again. */
EMIT2(X86_JNE, -(prog - branch_target) - 2);
- /* Return the pre-modification value, which RAX already is for R0 */
- if (!operand_on_stack)
- emit_mov_reg(&prog, is64, src_reg, BPF_REG_0);
+ /* Return the pre-modification value */
+ emit_mov_reg(&prog, is64, src_reg, BPF_REG_0);
if (resume)
*resume = prog;
/* Restore R0 after clobbering RAX */
- if (!r0_on_stack)
- emit_mov_reg(&prog, true, BPF_REG_0, BPF_REG_AX);
- else if (operand_on_stack)
- EMIT2(0x41, 0x5B); /* pop r11, dropping the operand */
- else
- EMIT1(0x58); /* pop rax */
+ emit_mov_reg(&prog, true, BPF_REG_0, BPF_REG_AX);
*pprog = prog;
return 0;
@@ -3354,7 +3334,7 @@ static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *
}
if (is_atomic_fetch) {
err = emit_atomic_fetch_rmw(&prog, insn, real_dst_reg,
- real_src_reg, -1, false,
+ real_src_reg, -1,
NULL, NULL);
if (err)
return err;
@@ -3383,29 +3363,19 @@ static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *
case BPF_STX | BPF_PROBE_ATOMIC | BPF_DW:
if (is_atomic_fetch_op(insn)) {
u32 real_src_reg = src_reg, real_dst_reg = dst_reg;
- /*
- * An access through a number has its address in
- * BPF_REG_AX (see arena_scalar_access()), so R0 is
- * saved on the stack instead, as div and mod do.
- */
- bool r0_on_stack = dst_reg == BPF_REG_AX;
u8 *fault[2], *resume;
int j;
/* Will need RAX as a CMPXCHG operand so save R0 */
- if (r0_on_stack) {
- EMIT1(0x50); /* push rax */
- } else {
- emit_mov_reg(&prog, true, BPF_REG_AX, BPF_REG_0);
- if (src_reg == BPF_REG_0)
- real_src_reg = BPF_REG_AX;
- if (dst_reg == BPF_REG_0)
- real_dst_reg = BPF_REG_AX;
- }
+ emit_mov_reg(&prog, true, BPF_REG_AX, BPF_REG_0);
+ if (src_reg == BPF_REG_0)
+ real_src_reg = BPF_REG_AX;
+ if (dst_reg == BPF_REG_0)
+ real_dst_reg = BPF_REG_AX;
err = emit_atomic_fetch_rmw(&prog, insn, real_dst_reg,
real_src_reg, X86_REG_R12,
- r0_on_stack, fault, &resume);
+ fault, &resume);
if (err)
return err;
--
2.55.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH v2 bpf 07/19] bpf: Clear delta of a register that scalar evolution widens
2026-10-27 19:02 [PATCH v2 bpf 00/19] bpf: Remove BPF_F_ARENA_SCALAR, fix scalar evolution and scalar forks Alexei Starovoitov
` (5 preceding siblings ...)
2026-10-27 19:03 ` [PATCH v2 bpf 06/19] bpf, x86: Remove r0_on_stack from fetching atomics in arena Alexei Starovoitov
@ 2026-10-27 19:03 ` Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 08/19] selftests/bpf: Test that a widened register is not linked Alexei Starovoitov
` (12 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: Alexei Starovoitov @ 2026-10-27 19:03 UTC (permalink / raw)
To: bpf; +Cc: daniel, andrii, eddyz87, memxor
From: Alexei Starovoitov <ast@kernel.org>
After 'r3 = r8; r3 += 4' the verifier knows that r3 == r8 + 4. The two
registers have the same id, and r3 keeps the difference in reg->delta. When
the prog compares one of them with a constant, sync_linked_regs()
calculates the range of the other register from the difference of the two
deltas.
At a loop head scalar evolution widens a register: it replaces the register
with its range over all iterations. Then the register is not linked to
other registers. scratch_widened_reg_id() clears reg->id, but it does not
clear reg->delta. A subsequent copy of the register gets the stale delta.
Then the range of the copy is incorrect by this delta.
r3 = r8
r3 += 4 ; R3=scalar(id=1+4)
1: ; r3 is widened: no id, delta is still 4
r4 = r3 ; id=2 and delta 4 in both
r4 += 1 ; R4=scalar(id=2+1)
if r3 > 10 goto 3f ; R3 <= 10, R4 = R3 + 1 - 4 <= 7
if r4 <= 7 goto 2f
... ; not verified, r4 is up to 11 here
The verifier does not verify code that can run. Use clear_scalar_id(). It
clears id and delta together. Move it to bpf_verifier.h, so that scev.c can
call it.
Fixes: 6ccf49f79327 ("bpf: Use SCEV to widen bounded loops")
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
include/linux/bpf_verifier.h | 6 ++++++
kernel/bpf/scev.c | 2 +-
kernel/bpf/verifier.c | 6 ------
3 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index e3f7a0b3bad3..7baeb831281f 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -1313,6 +1313,12 @@ static inline bool bpf_is_force_checkpoint(struct bpf_verifier_env *env, int ins
return env->insn_aux_data[insn_idx].force_checkpoint;
}
+static inline void clear_scalar_id(struct bpf_reg_state *reg)
+{
+ reg->id = 0;
+ reg->delta = 0;
+}
+
static inline void mark_calls_callback(struct bpf_verifier_env *env, int idx)
{
env->insn_aux_data[idx].calls_callback = true;
diff --git a/kernel/bpf/scev.c b/kernel/bpf/scev.c
index 04d4962448e2..3f7534cda47a 100644
--- a/kernel/bpf/scev.c
+++ b/kernel/bpf/scev.c
@@ -2391,7 +2391,7 @@ static void scratch_widened_reg_id(struct bpf_verifier_env *env, struct bpf_reg_
{
switch (base_type(reg->type)) {
case SCALAR_VALUE:
- reg->id = 0;
+ clear_scalar_id(reg);
break;
case PTR_TO_PACKET:
case PTR_TO_PACKET_META:
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index c1070aba621b..8139e41ace00 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -3824,12 +3824,6 @@ static bool __is_pointer_value(bool allow_ptr_leaks,
return is_pointer_regtype(reg->type);
}
-static void clear_scalar_id(struct bpf_reg_state *reg)
-{
- reg->id = 0;
- reg->delta = 0;
-}
-
static void assign_scalar_id_before_mov(struct bpf_verifier_env *env,
struct bpf_reg_state *src_reg)
{
--
2.55.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH v2 bpf 08/19] selftests/bpf: Test that a widened register is not linked
2026-10-27 19:02 [PATCH v2 bpf 00/19] bpf: Remove BPF_F_ARENA_SCALAR, fix scalar evolution and scalar forks Alexei Starovoitov
` (6 preceding siblings ...)
2026-10-27 19:03 ` [PATCH v2 bpf 07/19] bpf: Clear delta of a register that scalar evolution widens Alexei Starovoitov
@ 2026-10-27 19:03 ` Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 09/19] selftests/bpf: Wait for the last maps of map_kptr to be freed Alexei Starovoitov
` (11 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: Alexei Starovoitov @ 2026-10-27 19:03 UTC (permalink / raw)
To: bpf; +Cc: daniel, andrii, eddyz87, memxor
From: Alexei Starovoitov <ast@kernel.org>
Scalar evolution widens a register that has delta 4. Then the prog copies
the register. With the stale delta the verifier finds that a read of
uninitialized r9 is dead code, and it loads the prog. See
"bpf: Clear delta of a register that scalar evolution widens".
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
.../selftests/bpf/progs/verifier_scev.c | 34 +++++++++++++++++++
1 file changed, 34 insertions(+)
diff --git a/tools/testing/selftests/bpf/progs/verifier_scev.c b/tools/testing/selftests/bpf/progs/verifier_scev.c
index 37434efd1490..ddc9667bb078 100644
--- a/tools/testing/selftests/bpf/progs/verifier_scev.c
+++ b/tools/testing/selftests/bpf/progs/verifier_scev.c
@@ -1377,6 +1377,40 @@ __naked __noinline __used static int probe_read_stack(void)
: __clobber_all);
}
+/*
+ * r3 = r8 + 4 is linked to r8 when it enters the loop. Widening drops the link. If
+ * it leaves the 4 behind then r4 = r3 + 1 is taken for r3 - 3 when r3 is compared:
+ * r4 <= 7 instead of r4 <= 11, and the read of r9 looks like dead code.
+ */
+SEC("socket")
+__failure __msg("R9 !read_ok")
+__naked void widened_reg_is_not_linked(void)
+{
+ asm volatile (
+ "call %[bpf_get_prandom_u32];"
+ "r8 = r0;"
+ "r8 &= 0xff;"
+ "r3 = r8;"
+ "r3 += 4;"
+ "r1 = 0;"
+"1:"
+ "r4 = r3;"
+ "r4 += 1;"
+ "if r3 > 10 goto 3f;"
+ "if r4 <= 7 goto 2f;"
+ "r0 = *(u64 *)(r9 + 0);"
+"2:"
+ "r3 += 1;"
+ "r1 += 1;"
+ "if r1 < 2 goto 1b;"
+"3:"
+ "r0 = r8;"
+ "exit;"
+ :
+ : __imm(bpf_get_prandom_u32)
+ : __clobber_all);
+}
+
/* The unknown SCEV of an unbound STACK_MISC slot must not prevent widening. */
SEC("tracepoint")
__success
--
2.55.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH v2 bpf 09/19] selftests/bpf: Wait for the last maps of map_kptr to be freed
2026-10-27 19:02 [PATCH v2 bpf 00/19] bpf: Remove BPF_F_ARENA_SCALAR, fix scalar evolution and scalar forks Alexei Starovoitov
` (7 preceding siblings ...)
2026-10-27 19:03 ` [PATCH v2 bpf 08/19] selftests/bpf: Test that a widened register is not linked Alexei Starovoitov
@ 2026-10-27 19:03 ` Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 10/19] selftests/bpf: Avoid a page fault in rcu_reentry while the handler is armed Alexei Starovoitov
` (10 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: Alexei Starovoitov @ 2026-10-27 19:03 UTC (permalink / raw)
To: bpf; +Cc: daniel, andrii, eddyz87, memxor
From: Alexei Starovoitov <ast@kernel.org>
map_kptr_race/htab_leak fails occasionally in a complete run of test_progs:
test_htab_leak:FAIL:htab refcount unexpected htab refcount:
actual 6 != expected 2
This is a race between two tests. It is not a leak in the kernel.
- map_kptr runs immediately before htab_leak. The two tests examine the
refcount of the same test object.
- The kernel frees maps asynchronously. Thus the maps drop their
references some time after close().
- map_kptr waits for that after each subtest, but not after the last one.
Wait after the last subtest too.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
tools/testing/selftests/bpf/prog_tests/map_kptr.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/tools/testing/selftests/bpf/prog_tests/map_kptr.c b/tools/testing/selftests/bpf/prog_tests/map_kptr.c
index 17e707dddda8..f103f8cd458b 100644
--- a/tools/testing/selftests/bpf/prog_tests/map_kptr.c
+++ b/tools/testing/selftests/bpf/prog_tests/map_kptr.c
@@ -225,6 +225,11 @@ void serial_test_map_kptr(void)
/* Observe refcount dropping to 1 on map release. */
test_map_kptr_success(true);
+
+ /* The next test counts the same references */
+ ASSERT_OK(kern_sync_rcu_tasks_trace(skel), "sync rcu_tasks_trace");
+ ASSERT_OK(kern_sync_rcu(), "sync rcu");
+ wait_for_map_release();
}
rcu_tasks_trace_gp__destroy(skel);
--
2.55.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH v2 bpf 10/19] selftests/bpf: Avoid a page fault in rcu_reentry while the handler is armed
2026-10-27 19:02 [PATCH v2 bpf 00/19] bpf: Remove BPF_F_ARENA_SCALAR, fix scalar evolution and scalar forks Alexei Starovoitov
` (8 preceding siblings ...)
2026-10-27 19:03 ` [PATCH v2 bpf 09/19] selftests/bpf: Wait for the last maps of map_kptr to be freed Alexei Starovoitov
@ 2026-10-27 19:03 ` Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 11/19] bpf: Don't widen a pointer to a buffer in scalar evolution Alexei Starovoitov
` (9 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: Alexei Starovoitov @ 2026-10-27 19:03 UTC (permalink / raw)
To: bpf; +Cc: daniel, andrii, eddyz87, memxor
From: Alexei Starovoitov <ast@kernel.org>
The test arms a handler at rcu_segcblist_enqueue() for its thread. Then it
deletes an element of a task storage map. The delete gets to
rcu_segcblist_enqueue(), and the handler does a nested delete there.
The handler fires at the first rcu_segcblist_enqueue() in the thread. A
page fault can be the first:
skel->bss->target_pid = tid the handler is armed
the next insn is in a page that is not in memory
asm_exc_page_fault
filemap_fault
v9fs_issue_read
p9_req_put
irq_work_run
rcu_defer_drain
rcu_segcblist_enqueue
the handler deletes the element
bpf_map_delete_elem() -ENOENT
test_rcu_reentry:FAIL:boot_delete unexpected error: -2 (errno 2)
The result depends on the layout of test_progs. In the binary that failed,
test_rcu_reentry() crosses the end of a page immediately after the store.
This binary fails each time when it is on 9p. A copy of the same file on
ext4 passes.
Move the store and the delete to a function. Call it one time without a pid
and with a bad fd. This call brings the text and .bss into memory, and it
does not arm the handler.
Fixes: 0464f587c61a ("selftests/bpf: Add a call_srcu() re-entry reproducer")
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
.../selftests/bpf/prog_tests/rcu_reentry.c | 19 +++++++++++++++----
1 file changed, 15 insertions(+), 4 deletions(-)
diff --git a/tools/testing/selftests/bpf/prog_tests/rcu_reentry.c b/tools/testing/selftests/bpf/prog_tests/rcu_reentry.c
index de23a14b3d40..10b4ea913921 100644
--- a/tools/testing/selftests/bpf/prog_tests/rcu_reentry.c
+++ b/tools/testing/selftests/bpf/prog_tests/rcu_reentry.c
@@ -23,10 +23,17 @@ static bool have_tree_srcu(void)
return kallsyms_find("srcu_expedite_current", &addr) == 0;
}
+/* Arm the handler for this thread, then trigger call_rcu_tasks_trace(). */
+static noinline int arm_and_delete(struct rcu_reentry *skel, int pid, int map_fd, int *pidfd)
+{
+ skel->bss->target_pid = pid;
+ return bpf_map_delete_elem(map_fd, pidfd);
+}
+
void test_rcu_reentry(void)
{
struct rcu_reentry *skel;
- int err, pidfd = -1, map_fd;
+ int err, pidfd = -1, bad_fd = -1, map_fd;
cpu_set_t set, old_set;
bool affinity_saved;
__u64 val = 1;
@@ -63,9 +70,13 @@ void test_rcu_reentry(void)
if (!ASSERT_OK(err, "boot_create"))
goto restore;
- /* Arm the handler for this thread, then trigger call_rcu_tasks_trace(). */
- skel->bss->target_pid = syscall(__NR_gettid);
- err = bpf_map_delete_elem(map_fd, &pidfd);
+ /*
+ * The handler fires at the first rcu_segcblist_enqueue() in this thread, and it deletes
+ * the element. A page fault can get there too, when the file system reads the page.
+ * Go through the same code first, without a pid, to bring its text and .bss in.
+ */
+ arm_and_delete(skel, 0, map_fd, &bad_fd);
+ err = arm_and_delete(skel, syscall(__NR_gettid), map_fd, &pidfd);
if (!ASSERT_OK(err, "boot_delete"))
goto restore;
--
2.55.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH v2 bpf 11/19] bpf: Don't widen a pointer to a buffer in scalar evolution
2026-10-27 19:02 [PATCH v2 bpf 00/19] bpf: Remove BPF_F_ARENA_SCALAR, fix scalar evolution and scalar forks Alexei Starovoitov
` (9 preceding siblings ...)
2026-10-27 19:03 ` [PATCH v2 bpf 10/19] selftests/bpf: Avoid a page fault in rcu_reentry while the handler is armed Alexei Starovoitov
@ 2026-10-27 19:03 ` Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 12/19] selftests/bpf: Test a loop that moves a pointer to a buffer Alexei Starovoitov
` (8 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: Alexei Starovoitov @ 2026-10-27 19:03 UTC (permalink / raw)
To: bpf; +Cc: daniel, andrii, eddyz87, memxor
From: Alexei Starovoitov <ast@kernel.org>
Scalar evolution widens counted loops. Since that change the verifier
rejects a loop over the per-cpu values of a map element:
SEC("iter/bpf_map_elem")
...
for (i = 0; i < n; i++) {
out[i] = *(__u32 *)pptr;
pptr += 8;
}
loop header at 12, widening r2 to 0..56 step 8
R2 invalid variable buffer offset: off=0, var_off=(0x0; 0x38)
pptr is a pointer to a buffer (PTR_TO_BUF). Scalar evolution replaces its
offset with the range over all iterations. But __check_buffer_access()
accepts only a constant offset. Thus a prog cannot use a widened
PTR_TO_BUF.
Do not widen it. Then the verifier verifies the loop one iteration at a
time, as it did before.
loop before SCEV with SCEV this patch
-------------------------------- ----------- ---------- ----------
counts in i, reads through pptr loads rejected loads
counts in p ('p < end'), no read 1M insns loads 1M insns
counts in p, reads through p 1M insns rejected 1M insns
The verifier cannot decide a comparison of two pointers to a buffer. Thus
it follows 'p < end' until the limit of 1M insns.
bpf_iter_bpf_percpu_array_map.c and bpf_iter_bpf_percpu_hash_map.c have the
first loop. They load today because num_cpus is volatile. The prog reads
the bound again in each iteration. Scalar evolution cannot count such a
loop, and it widens nothing.
Fixes: 6ccf49f79327 ("bpf: Use SCEV to widen bounded loops")
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
kernel/bpf/scev.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/kernel/bpf/scev.c b/kernel/bpf/scev.c
index 3f7534cda47a..6231d16aa311 100644
--- a/kernel/bpf/scev.c
+++ b/kernel/bpf/scev.c
@@ -2379,7 +2379,6 @@ static bool is_widenable_reg_type(const struct bpf_reg_state *reg)
case PTR_TO_PACKET:
case PTR_TO_PACKET_META:
case PTR_TO_MEM:
- case PTR_TO_BUF:
case PTR_TO_BTF_ID:
return true;
default:
--
2.55.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH v2 bpf 12/19] selftests/bpf: Test a loop that moves a pointer to a buffer
2026-10-27 19:02 [PATCH v2 bpf 00/19] bpf: Remove BPF_F_ARENA_SCALAR, fix scalar evolution and scalar forks Alexei Starovoitov
` (10 preceding siblings ...)
2026-10-27 19:03 ` [PATCH v2 bpf 11/19] bpf: Don't widen a pointer to a buffer in scalar evolution Alexei Starovoitov
@ 2026-10-27 19:03 ` Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 13/19] bpf: Don't clamp a register whose type changed in the loop Alexei Starovoitov
` (7 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: Alexei Starovoitov @ 2026-10-27 19:03 UTC (permalink / raw)
To: bpf; +Cc: daniel, andrii, eddyz87, memxor
From: Alexei Starovoitov <ast@kernel.org>
ptr_buf moves one pointer to a buffer until it gets to a second pointer. It
never reads through the pointer. It loads only if scalar evolution widens
the pointer, and that is no longer the case.
Change it to the loop that real progs have, for example
bpf_iter_bpf_percpu_array_map.c. It counts in a register and reads through
the pointer in each iteration. Without
"bpf: Don't widen a pointer to a buffer in scalar evolution" the verifier
rejects it with "invalid variable buffer offset".
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
tools/testing/selftests/bpf/progs/verifier_scev.c | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/tools/testing/selftests/bpf/progs/verifier_scev.c b/tools/testing/selftests/bpf/progs/verifier_scev.c
index ddc9667bb078..cf7916e1fc48 100644
--- a/tools/testing/selftests/bpf/progs/verifier_scev.c
+++ b/tools/testing/selftests/bpf/progs/verifier_scev.c
@@ -3517,21 +3517,23 @@ __naked void ptr_mem_other_reservation(void)
: __clobber_all);
}
+/* An access to a buffer at a variable offset is rejected: the pointer must not be widened. */
SEC("iter/bpf_map_elem")
__success
__log_level(2)
__flag(BPF_F_TEST_STATE_FREQ)
__msg("header_count is 4 ")
-__msg("widening r6 to 0..3 step 1")
+__msg("can't widen r6, expr is (+ r6 8)")
__naked void ptr_buf(void)
{
asm volatile (" \
r6 = *(u64 *)(r1 + %[value]); \
if r6 == 0 goto 2f; \
- r7 = r6; \
- r7 += 4; \
-1: r6 += 1; \
- if r6 < r7 goto 1b; \
+ r7 = 0; \
+1: r0 = *(u32 *)(r6 + 0); \
+ r6 += 8; \
+ r7 += 1; \
+ if r7 < 4 goto 1b; \
2: r0 = 0; \
exit; \
" :
--
2.55.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH v2 bpf 13/19] bpf: Don't clamp a register whose type changed in the loop
2026-10-27 19:02 [PATCH v2 bpf 00/19] bpf: Remove BPF_F_ARENA_SCALAR, fix scalar evolution and scalar forks Alexei Starovoitov
` (11 preceding siblings ...)
2026-10-27 19:03 ` [PATCH v2 bpf 12/19] selftests/bpf: Test a loop that moves a pointer to a buffer Alexei Starovoitov
@ 2026-10-27 19:03 ` Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 14/19] bpf: Reset the range of a packet pointer that scalar evolution clamps Alexei Starovoitov
` (6 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: Alexei Starovoitov @ 2026-10-27 19:03 UTC (permalink / raw)
To: bpf; +Cc: daniel, andrii, eddyz87, memxor
From: Alexei Starovoitov <ast@kernel.org>
When a state returns to the head of a counted loop, bpf_clamp_scev_regs()
replaces the range of each register that changes by a constant step. The
new range is the range over all iterations. The function calculates it from
the register in the state that entered the loop. For a pointer this range
is the range of its offsets.
A call in the loop can invalidate the pointer. bpf_skb_pull_data() does it
to packet pointers, bpf_ringbuf_discard() does it to ringbuf memory. Then
the register is an unknown scalar. Scalar evolution does not see that, and
the clamp gives the offsets of the pointer to the scalar.
r6 = bpf_ringbuf_reserve() ; R6=ringbuf_mem
r7 = r6
1: if first iteration goto 2f
r1 = map_value
r1 += r6 ; accepted: R6 is 0 or 1
r0 = *(u8 *)(r1 + 0)
goto 3f
2: bpf_ringbuf_discard(r7) ; R6=scalar()
3: r6 += 1
r9 += 1
if r9 < 2 goto 1b ; clamp: R6=scalar(umax=1)
At run time r6 is the address of the ringbuf memory plus 1. The prog adds
it to a pointer to a map value, and the kernel crashes in the access.
Do not clamp a register that has a different type than at the entry of the
loop. Such a state is not within the state at the head. Thus the verifier
continues with it, and r6 stays an unknown scalar.
Fixes: 6ccf49f79327 ("bpf: Use SCEV to widen bounded loops")
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
kernel/bpf/scev.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/kernel/bpf/scev.c b/kernel/bpf/scev.c
index 6231d16aa311..89f96682af3f 100644
--- a/kernel/bpf/scev.c
+++ b/kernel/bpf/scev.c
@@ -2897,6 +2897,14 @@ int bpf_clamp_scev_regs(struct bpf_verifier_env *env, struct bpf_func_state *cur
is_unbound_scev_reg(env, entry_st, r))
continue;
+ /*
+ * A call in the loop can change the type: it invalidates packet pointers or
+ * releases a reference. The bounds are the offsets of the pointer at the entry.
+ * They say nothing about the value that is in the register now.
+ */
+ if (reg->type != entry_reg->type)
+ continue;
+
linear_bounds(entry_reg, max_header_count, &coefs, &bounds);
if (log->level & BPF_LOG_LEVEL2) {
bpf_log(log, "loop header at %d, clamping ", insn_idx);
--
2.55.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH v2 bpf 14/19] bpf: Reset the range of a packet pointer that scalar evolution clamps
2026-10-27 19:02 [PATCH v2 bpf 00/19] bpf: Remove BPF_F_ARENA_SCALAR, fix scalar evolution and scalar forks Alexei Starovoitov
` (12 preceding siblings ...)
2026-10-27 19:03 ` [PATCH v2 bpf 13/19] bpf: Don't clamp a register whose type changed in the loop Alexei Starovoitov
@ 2026-10-27 19:03 ` Alexei Starovoitov
2026-10-27 19:30 ` sashiko-bot
2026-10-27 19:03 ` [PATCH v2 bpf 15/19] selftests/bpf: Test the clamp of pointers in scalar evolution Alexei Starovoitov
` (5 subsequent siblings)
19 siblings, 1 reply; 25+ messages in thread
From: Alexei Starovoitov @ 2026-10-27 19:03 UTC (permalink / raw)
To: bpf; +Cc: daniel, andrii, eddyz87, memxor
From: Alexei Starovoitov <ast@kernel.org>
A compare with pkt_end sets 'range' in all packet pointers that have the
same id. Then 'range - offset' bytes from the pointer are in the packet.
When a state returns to the head of a counted loop, bpf_clamp_scev_regs()
gives a new id to a packet pointer, and the offsets of all iterations. It
does not change the range. But the range is valid only together with the
old id and the old offsets.
1: ; R6=pkt(id=1,r=0,off=0..100)
r5 = r6
r5 += 100
if r5 > pkt_end goto 2f ; R6=pkt(id=1,r=200,off=0..100)
r6 += 100 ; R6=pkt(id=1,r=200,off=100..200)
r9 += 1
if r9 < 2 goto 1b ; clamp: R6=pkt(id=2,r=200,off=0..100)
after bytes from r6 that the verifier accepts
---------------------- ---------------------------------------
the compare 100, correct
r6 += 100 0, correct
the clamp 100, incorrect. Nothing compared this r6
Usually the verifier prunes the state at the head, and the range has no
effect. The verifier continues with the state when a different part of it
is not within the state at the head. A flag in a 32-bit spill is
sufficient. Then the prog reads and writes after pkt_end. A write to
skb_shared_info crashes the kernel.
Take the range that the pointer has at the entry of the loop. No compare in
the loop found this range, so it is valid in all iterations. The state at
the head has the same range, so the verifier prunes as before.
Fixes: 6ccf49f79327 ("bpf: Use SCEV to widen bounded loops")
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
kernel/bpf/scev.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/kernel/bpf/scev.c b/kernel/bpf/scev.c
index 89f96682af3f..7b82fd79f9cc 100644
--- a/kernel/bpf/scev.c
+++ b/kernel/bpf/scev.c
@@ -2914,6 +2914,13 @@ int bpf_clamp_scev_regs(struct bpf_verifier_env *env, struct bpf_func_state *cur
bpf_log(log, "\n");
}
scratch_widened_reg_id(env, reg);
+ /*
+ * The range of a packet pointer goes together with its id and with the offsets
+ * that it had in the compare with pkt_end. Both are replaced here. What holds for
+ * the new offsets is the range at the entry.
+ */
+ if (type_is_pkt_pointer(reg->type))
+ reg->range = max(entry_reg->range, 0);
err = bpf_set_reg_range(env, reg, bounds.r64, bounds.r32,
bounds.base, bounds.step);
if (err)
--
2.55.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH v2 bpf 15/19] selftests/bpf: Test the clamp of pointers in scalar evolution
2026-10-27 19:02 [PATCH v2 bpf 00/19] bpf: Remove BPF_F_ARENA_SCALAR, fix scalar evolution and scalar forks Alexei Starovoitov
` (13 preceding siblings ...)
2026-10-27 19:03 ` [PATCH v2 bpf 14/19] bpf: Reset the range of a packet pointer that scalar evolution clamps Alexei Starovoitov
@ 2026-10-27 19:03 ` Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 16/19] bpf: Finish the insn in the state that maybe_fork_scalars() splits off Alexei Starovoitov
` (4 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: Alexei Starovoitov @ 2026-10-27 19:03 UTC (permalink / raw)
To: bpf; +Cc: daniel, andrii, eddyz87, memxor
From: Alexei Starovoitov <ast@kernel.org>
test the loop result
-------------------------------------- ------------------------- ------
clamp_released_pointer releases ringbuf memory reject
clamp_invalidated_pkt_pointer calls bpf_skb_pull_data() reject
clamp_pkt_pointer_keeps_no_range compares with pkt_end reject
clamp_pkt_pointer_keeps_range_of_entry has no compare loads
In the first three tests a flag in a 32-bit spill separates the first
iteration from the second one. Scalar evolution does not widen the flag.
Thus the verifier does not prune the state that returns to the head of the
loop, and it verifies the second iteration with the clamped register.
Without the fixes the three progs load.
In the last test the compare with pkt_end is in front of the loop. Its
range must stay valid in the loop.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
.../selftests/bpf/progs/verifier_scev.c | 163 ++++++++++++++++++
1 file changed, 163 insertions(+)
diff --git a/tools/testing/selftests/bpf/progs/verifier_scev.c b/tools/testing/selftests/bpf/progs/verifier_scev.c
index cf7916e1fc48..fa01a616af9d 100644
--- a/tools/testing/selftests/bpf/progs/verifier_scev.c
+++ b/tools/testing/selftests/bpf/progs/verifier_scev.c
@@ -3773,4 +3773,167 @@ __naked void post_cond_jlt32_copy(void)
: __clobber_all);
}
+/*
+ * r6 is a pointer to ringbuf memory with a step of 1. bpf_ringbuf_discard() in the first iteration
+ * makes a number of it. The offsets 0..1 of the pointer are not the range of this number: it's an
+ * address. The flag in the 32-bit spill is not widened, so the second iteration is verified
+ * on its own.
+ */
+SEC("socket")
+__failure __msg("math between map_value pointer and register with unbounded min value")
+__naked void clamp_released_pointer(void)
+{
+ asm volatile (
+ "r1 = 0;"
+ "*(u32 *)(r10 - 12) = r1;"
+ "r2 = r10;"
+ "r2 += -12;"
+ "r1 = %[small_map] ll;"
+ "call %[bpf_map_lookup_elem];"
+ "if r0 == 0 goto 9f;"
+ "r8 = r0;"
+ "r1 = %[ringbuf] ll;"
+ "r2 = 16;"
+ "r3 = 0;"
+ "call %[bpf_ringbuf_reserve];"
+ "if r0 == 0 goto 9f;"
+ "r6 = r0;"
+ "r7 = r0;"
+ "r9 = 0;"
+ "w1 = 0;"
+ "*(u32 *)(r10 - 8) = r1;"
+"1:"
+ "w4 = *(u32 *)(r10 - 8);"
+ "if w4 == 0 goto 2f;"
+ "r1 = r8;"
+ "r1 += r6;"
+ "r0 = *(u8 *)(r1 + 0);"
+ "goto 3f;"
+"2:"
+ "r1 = r7;"
+ "r2 = 0;"
+ "call %[bpf_ringbuf_discard];"
+"3:"
+ "w4 = 1;"
+ "*(u32 *)(r10 - 8) = r4;"
+ "r6 += 1;"
+ "r9 += 1;"
+ "if r9 < 2 goto 1b;"
+"9:"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(bpf_map_lookup_elem), __imm(bpf_ringbuf_reserve), __imm(bpf_ringbuf_discard),
+ __imm_addr(small_map), __imm_addr(ringbuf)
+ : __clobber_all);
+}
+
+/* The same with a packet pointer and bpf_skb_pull_data() */
+SEC("tc")
+__failure __msg("math between map_value pointer and register with unbounded min value")
+__naked void clamp_invalidated_pkt_pointer(void)
+{
+ asm volatile (
+ "r7 = r1;"
+ "r1 = 0;"
+ "*(u32 *)(r10 - 12) = r1;"
+ "r2 = r10;"
+ "r2 += -12;"
+ "r1 = %[small_map] ll;"
+ "call %[bpf_map_lookup_elem];"
+ "if r0 == 0 goto 9f;"
+ "r8 = r0;"
+ "r6 = *(u32 *)(r7 + %[data]);"
+ "r9 = 0;"
+ "w1 = 0;"
+ "*(u32 *)(r10 - 8) = r1;"
+"1:"
+ "w4 = *(u32 *)(r10 - 8);"
+ "if w4 == 0 goto 2f;"
+ "r1 = r8;"
+ "r1 += r6;"
+ "r0 = *(u8 *)(r1 + 0);"
+ "goto 3f;"
+"2:"
+ "r1 = r7;"
+ "r2 = 0;"
+ "call %[bpf_skb_pull_data];"
+"3:"
+ "w4 = 1;"
+ "*(u32 *)(r10 - 8) = r4;"
+ "r6 += 1;"
+ "r9 += 1;"
+ "if r9 < 2 goto 1b;"
+"9:"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(bpf_map_lookup_elem), __imm(bpf_skb_pull_data), __imm_addr(small_map),
+ __imm_const(data, offsetof(struct __sk_buff, data))
+ : __clobber_all);
+}
+
+/*
+ * The compare with data_end is for r6 of this iteration: 100 bytes from r6 are in the packet. Then
+ * r6 moves by 100. The range that the compare found is not for the r6 that returns to the head.
+ */
+SEC("tc")
+__failure __msg("invalid access to packet")
+__naked void clamp_pkt_pointer_keeps_no_range(void)
+{
+ asm volatile (
+ "r7 = *(u32 *)(r1 + %[data_end]);"
+ "r6 = *(u32 *)(r1 + %[data]);"
+ "r9 = 0;"
+ "w1 = 0;"
+ "*(u32 *)(r10 - 8) = r1;"
+"1:"
+ "w4 = *(u32 *)(r10 - 8);"
+ "if w4 == 0 goto 2f;"
+ /* second iteration: nothing compared this r6 with data_end */
+ "r0 = *(u64 *)(r6 + 56);"
+"2:"
+ "r5 = r6;"
+ "r5 += 100;"
+ "if r5 > r7 goto 9f;"
+ "w4 = 1;"
+ "*(u32 *)(r10 - 8) = r4;"
+ "r6 += 100;"
+ "r9 += 1;"
+ "if r9 < 2 goto 1b;"
+"9:"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm_const(data, offsetof(struct __sk_buff, data)),
+ __imm_const(data_end, offsetof(struct __sk_buff, data_end))
+ : __clobber_all);
+}
+
+/* A compare in front of the loop is for all iterations: the range at the entry stays */
+SEC("tc")
+__success __retval(0)
+__naked void clamp_pkt_pointer_keeps_range_of_entry(void)
+{
+ asm volatile (
+ "r7 = *(u32 *)(r1 + %[data_end]);"
+ "r6 = *(u32 *)(r1 + %[data]);"
+ "r5 = r6;"
+ "r5 += 40;"
+ "if r5 > r7 goto 9f;"
+ "r9 = 0;"
+"1:"
+ "r0 = *(u64 *)(r6 + 0);"
+ "r6 += 8;"
+ "r9 += 1;"
+ "if r9 < 4 goto 1b;"
+"9:"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm_const(data, offsetof(struct __sk_buff, data)),
+ __imm_const(data_end, offsetof(struct __sk_buff, data_end))
+ : __clobber_all);
+}
+
char _license[] SEC("license") = "GPL";
--
2.55.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH v2 bpf 16/19] bpf: Finish the insn in the state that maybe_fork_scalars() splits off
2026-10-27 19:02 [PATCH v2 bpf 00/19] bpf: Remove BPF_F_ARENA_SCALAR, fix scalar evolution and scalar forks Alexei Starovoitov
` (14 preceding siblings ...)
2026-10-27 19:03 ` [PATCH v2 bpf 15/19] selftests/bpf: Test the clamp of pointers in scalar evolution Alexei Starovoitov
@ 2026-10-27 19:03 ` Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 17/19] selftests/bpf: Test maybe_fork_scalars() at the head of a counted loop Alexei Starovoitov
` (3 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: Alexei Starovoitov @ 2026-10-27 19:03 UTC (permalink / raw)
To: bpf; +Cc: daniel, andrii, eddyz87, memxor
From: Alexei Starovoitov <ast@kernel.org>
When a register is -1 or 0, maybe_fork_scalars() splits the state in two
before an 'and' or an 'or' with a constant. The result is more accurate for
each value alone. The verifier continues with -1. It pushes a copy of the
state with 0 and verifies the copy later, from the same insn.
today this patch
N: r1 &= 8 r1 is -1 or 0 N: r1 &= 8 r1 is -1 or 0
/ \ / \
r1 = -1 r1 = 0 r1 = -1 r1 = 0
r1 &= 8 push to N r1 &= 8 r1 &= 8
| | push to N+1
N+1: ... N+1: ...
At insn N the copy is a part of the state that it came from: 0 is in
[-1, 0]. If pruning compares the two states, it drops the copy, and no one
verifies the case r1 = 0.
The first state has branches that are not explored. Usually pruning does
not use such states. In a loop that scalar evolution counted it does, and
the head of such a loop always keeps its state. Thus a fork at the head of
a counted loop loses the case r1 = 0:
r9 = a map value of 8 bytes
r6 = -1
1: r6 &= 8 r6 is -1 or 0, only -1 is verified: 8
r6 ^= 8 0. It is 8 when r6 was 0.
r1 = r9
r1 += r6
r0 = *(u8 *)(r1 + 0)
r6 = -1 or 0
r7 += 1
if r7 < 100 goto 1b
from 11 to 11: safe
The verifier accepts this prog. The prog reads the byte that follows the
map value. With a constant of 64 Tbyte in a register it reads an address
that is not mapped, and the kernel crashes.
Do the insn in the copy too, and push the copy to the next insn. All other
code that pushes a state does the same.
No verdict changes in selftests, Cilium and scx. Each fork processes one
insn less, and the sequence in which the verifier explores states changes
slightly. 62 of 161 progs of Cilium process a different number of insns,
from -0.7% to +3.3%. The total is 986970 -> 987013. 2 of 563 progs of scx
process one insn less.
Fixes: 6ccf49f79327 ("bpf: Use SCEV to widen bounded loops")
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
kernel/bpf/verifier.c | 27 ++++++++++++++++++---------
1 file changed, 18 insertions(+), 9 deletions(-)
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 8139e41ace00..02b2915f0014 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -17289,11 +17289,14 @@ static bool is_safe_to_compute_dst_reg_range(struct bpf_insn *insn,
}
}
+static int adjust_scalar_min_max_vals(struct bpf_verifier_env *env, struct bpf_insn *insn,
+ struct bpf_reg_state *dst_reg, struct bpf_reg_state src_reg);
+
static int maybe_fork_scalars(struct bpf_verifier_env *env, struct bpf_insn *insn,
- struct bpf_reg_state *dst_reg)
+ struct bpf_reg_state *dst_reg, const struct bpf_reg_state *src_reg)
{
struct bpf_verifier_state *branch;
- struct bpf_reg_state *regs;
+ struct bpf_reg_state *reg;
bool alu32;
if (reg_smin(dst_reg) == -1 && reg_smax(dst_reg) == 0)
@@ -17303,19 +17306,25 @@ static int maybe_fork_scalars(struct bpf_verifier_env *env, struct bpf_insn *ins
else
return 0;
- branch = push_stack(env, env->insn_idx, env->insn_idx, false);
+ branch = push_stack(env, env->insn_idx + 1, env->insn_idx, false);
if (IS_ERR(branch))
return PTR_ERR(branch);
- regs = branch->frame[branch->curframe]->regs;
+ reg = &branch->frame[branch->curframe]->regs[insn->dst_reg];
if (alu32) {
- __mark_reg32_known(®s[insn->dst_reg], 0);
+ __mark_reg32_known(reg, 0);
__mark_reg32_known(dst_reg, -1ull);
} else {
- __mark_reg_known(®s[insn->dst_reg], 0);
+ __mark_reg_known(reg, 0);
__mark_reg_known(dst_reg, -1ull);
}
- return 0;
+ /*
+ * The insn is done with in the branch too, and what gets to the next insn is a state
+ * like any other. At this insn the branch is a part of the state that is looked at:
+ * it must not be compared with it.
+ */
+ clear_scalar_id(reg);
+ return adjust_scalar_min_max_vals(env, insn, reg, *src_reg);
}
/* WARNING: This function does calculations on 64-bit values, but the actual
@@ -17420,7 +17429,7 @@ static int adjust_scalar_min_max_vals(struct bpf_verifier_env *env,
break;
case BPF_AND:
if (tnum_is_const(src_reg.var_off)) {
- ret = maybe_fork_scalars(env, insn, dst_reg);
+ ret = maybe_fork_scalars(env, insn, dst_reg, &src_reg);
if (ret)
return ret;
}
@@ -17431,7 +17440,7 @@ static int adjust_scalar_min_max_vals(struct bpf_verifier_env *env,
break;
case BPF_OR:
if (tnum_is_const(src_reg.var_off)) {
- ret = maybe_fork_scalars(env, insn, dst_reg);
+ ret = maybe_fork_scalars(env, insn, dst_reg, &src_reg);
if (ret)
return ret;
}
--
2.55.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH v2 bpf 17/19] selftests/bpf: Test maybe_fork_scalars() at the head of a counted loop
2026-10-27 19:02 [PATCH v2 bpf 00/19] bpf: Remove BPF_F_ARENA_SCALAR, fix scalar evolution and scalar forks Alexei Starovoitov
` (15 preceding siblings ...)
2026-10-27 19:03 ` [PATCH v2 bpf 16/19] bpf: Finish the insn in the state that maybe_fork_scalars() splits off Alexei Starovoitov
@ 2026-10-27 19:03 ` Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 18/19] bpf: Keep stack arguments in arch_bpf_timed_may_goto() Alexei Starovoitov
` (2 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: Alexei Starovoitov @ 2026-10-27 19:03 UTC (permalink / raw)
To: bpf; +Cc: daniel, andrii, eddyz87, memxor
From: Alexei Starovoitov <ast@kernel.org>
The insn that forks is the head of a loop that scalar evolution counted.
The state with 0 that maybe_fork_scalars() splits off is within the state
that is kept at the head. The access is out of bounds only in the state
with 0.
Without the previous patch the verifier accepts the prog.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
.../selftests/bpf/progs/verifier_bounds.c | 41 +++++++++++++++++++
1 file changed, 41 insertions(+)
diff --git a/tools/testing/selftests/bpf/progs/verifier_bounds.c b/tools/testing/selftests/bpf/progs/verifier_bounds.c
index df8d5309657e..05efa9f4a522 100644
--- a/tools/testing/selftests/bpf/progs/verifier_bounds.c
+++ b/tools/testing/selftests/bpf/progs/verifier_bounds.c
@@ -2159,6 +2159,47 @@ l0_%=: r0 = 0; \
: __clobber_all);
}
+/*
+ * The insn that forks is the head of a loop that scalar evolution counted. The state with
+ * [-1, 0] is kept there, and the state with 0 that is split off is within it.
+ */
+SEC("socket")
+__description("maybe_fork_scalars: AND at the head of a counted loop rejects OOB")
+__failure __msg("invalid access to map value")
+__naked void and_scalar_fork_at_long_loop_head_rejects_oob(void)
+{
+ asm volatile (" \
+ r1 = 0; \
+ *(u64*)(r10 - 8) = r1; \
+ r2 = r10; \
+ r2 += -8; \
+ r1 = %[map_hash_8b] ll; \
+ call %[bpf_map_lookup_elem]; \
+ if r0 == 0 goto l0_%=; \
+ r9 = r0; \
+ r7 = 0; \
+ r6 = -1; \
+l1_%=: r6 &= 8; \
+ /* r6 is 8 (current) or 0 (pushed) */ \
+ r6 ^= 8; \
+ /* pushed path: r6 = 8, OOB for value_size=8 */ \
+ r1 = r9; \
+ r1 += r6; \
+ r0 = *(u8*)(r1 + 0); \
+ r1 = *(u64*)(r9 + 0); \
+ r6 = -1; \
+ if r1 == 0 goto l2_%=; \
+ r6 = 0; \
+l2_%=: r7 += 1; \
+ if r7 < 100 goto l1_%=; \
+l0_%=: r0 = 0; \
+ exit; \
+" :
+ : __imm(bpf_map_lookup_elem),
+ __imm_addr(map_hash_8b)
+ : __clobber_all);
+}
+
/*
* Last jump can be detected as always taken because the intersection of R5 and
* R7 32bit tnums produces a constant that isn't within R7's s32 bounds.
--
2.55.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH v2 bpf 18/19] bpf: Keep stack arguments in arch_bpf_timed_may_goto()
2026-10-27 19:02 [PATCH v2 bpf 00/19] bpf: Remove BPF_F_ARENA_SCALAR, fix scalar evolution and scalar forks Alexei Starovoitov
` (16 preceding siblings ...)
2026-10-27 19:03 ` [PATCH v2 bpf 17/19] selftests/bpf: Test maybe_fork_scalars() at the head of a counted loop Alexei Starovoitov
@ 2026-10-27 19:03 ` Alexei Starovoitov
2026-10-27 19:48 ` bot+bpf-ci
2026-10-27 19:03 ` [PATCH v2 bpf 19/19] selftests/bpf: Test stack arguments across may_goto Alexei Starovoitov
2026-10-28 4:10 ` [PATCH v2 bpf 00/19] bpf: Remove BPF_F_ARENA_SCALAR, fix scalar evolution and scalar forks patchwork-bot+netdevbpf
19 siblings, 1 reply; 25+ messages in thread
From: Alexei Starovoitov @ 2026-10-27 19:03 UTC (permalink / raw)
To: bpf; +Cc: daniel, andrii, eddyz87, memxor
From: Alexei Starovoitov <ast@kernel.org>
The JITs pass the first stack arguments of a call in registers:
x86-64 r9 argument 6
arm64 x5-x7 arguments 6-8
riscv64 a5-a7 arguments 6-8
'*(u64 *)(r11 - 8) = rX' writes such a register in the caller.
'rX = *(u64 *)(r11 + 8)' reads it in the callee. A call changes these
registers. Thus the verifier accepts no call between the write and the call
of the callee, and no call in the callee before the read.
may_goto is not a call for the verifier. But one time in 65535 iterations
it calls arch_bpf_timed_may_goto(), and this function calls C code. It
saves R0-R5 only.
*(u64 *)(r11 - 8) = 0 ; the verifier: argument 6 is 0
call sub
sub:
1: may_goto 2f ; C code changes r9
goto 1b
2: r2 = *(u64 *)(r11 + 8) ; r2 is a kernel address
r1 = map_value
r1 += r2
*(u64 *)(r1 + 0) = 0
On x86-64 the C code changes r9 when the time of may_goto is up. With the
hpet clocksource a look at the clock changes r9 too. The prog above crashes
the kernel. LLVM does not emit such code: it reads stack arguments at the
entry of a function, and it writes them immediately before the call.
Save and restore these registers too. The cost is 2 insns on x86-64, 4 on
arm64 and 6 on riscv64, one time in 65535 iterations.
Fixes: 324c3ca6eed6 ("bpf,x86: Implement JIT support for stack arguments")
Fixes: 235b2fe772f5 ("bpf, arm64: Add JIT support for stack arguments")
Fixes: ba754185f902 ("riscv, bpf: Add BPF stack arguments support for RV64 JIT")
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
arch/arm64/net/bpf_timed_may_goto.S | 10 ++++++++--
arch/riscv/net/bpf_timed_may_goto.S | 27 ++++++++++++++++++---------
arch/x86/net/bpf_timed_may_goto.S | 3 +++
3 files changed, 29 insertions(+), 11 deletions(-)
diff --git a/arch/arm64/net/bpf_timed_may_goto.S b/arch/arm64/net/bpf_timed_may_goto.S
index a9a802711a7f..1b4a4f91ffc9 100644
--- a/arch/arm64/net/bpf_timed_may_goto.S
+++ b/arch/arm64/net/bpf_timed_may_goto.S
@@ -5,13 +5,16 @@
SYM_FUNC_START(arch_bpf_timed_may_goto)
/* Allocate stack space and emit frame record */
- stp x29, x30, [sp, #-64]!
+ stp x29, x30, [sp, #-96]!
mov x29, sp
/* Save BPF registers R0 - R5 (x8, x0-x4)*/
stp x8, x0, [sp, #16]
stp x1, x2, [sp, #32]
stp x3, x4, [sp, #48]
+ /* Save stack arguments 6 - 8 (x5-x7) */
+ stp x5, x6, [sp, #64]
+ str x7, [sp, #80]
/*
* Stack depth was passed in BPF_REG_AX (x9), add it to the BPF_FP
@@ -32,9 +35,12 @@ SYM_FUNC_START(arch_bpf_timed_may_goto)
ldp x8, x0, [sp, #16]
ldp x1, x2, [sp, #32]
ldp x3, x4, [sp, #48]
+ /* Restore stack arguments 6 - 8 (x5-x7) */
+ ldp x5, x6, [sp, #64]
+ ldr x7, [sp, #80]
/* Restore FP and LR */
- ldp x29, x30, [sp], #64
+ ldp x29, x30, [sp], #96
ret
SYM_FUNC_END(arch_bpf_timed_may_goto)
diff --git a/arch/riscv/net/bpf_timed_may_goto.S b/arch/riscv/net/bpf_timed_may_goto.S
index fcf850f244c9..6ec37f634aff 100644
--- a/arch/riscv/net/bpf_timed_may_goto.S
+++ b/arch/riscv/net/bpf_timed_may_goto.S
@@ -10,15 +10,16 @@
* - output: updated count in BPF_REG_AX (t0)
*
* Calls bpf_check_timed_may_goto(ptr) with the standard RISC-V ABI, where
- * ptr = BPF_REG_FP (s5) + BPF_REG_AX (t0). BPF R0-R5 (t6, a0-a4) are saved
- * across the call; BPF_REG_FP (s5) is callee-saved and needs no saving.
+ * ptr = BPF_REG_FP (s5) + BPF_REG_AX (t0). BPF R0-R5 (t6, a0-a4) and stack
+ * arguments 6-8 (a5-a7) are saved across the call; BPF_REG_FP (s5) is
+ * callee-saved and needs no saving.
*/
SYM_FUNC_START(arch_bpf_timed_may_goto)
- addi sp, sp, -(8*SZREG)
- REG_S ra, 7*SZREG(sp)
- REG_S s0, 6*SZREG(sp)
- addi s0, sp, 8*SZREG
+ addi sp, sp, -(12*SZREG)
+ REG_S ra, 11*SZREG(sp)
+ REG_S s0, 10*SZREG(sp)
+ addi s0, sp, 12*SZREG
/* Save BPF registers R0-R5 (t6, a0-a4) */
REG_S t6, 5*SZREG(sp)
@@ -27,6 +28,10 @@ SYM_FUNC_START(arch_bpf_timed_may_goto)
REG_S a2, 2*SZREG(sp)
REG_S a3, 1*SZREG(sp)
REG_S a4, 0*SZREG(sp)
+ /* Save stack arguments 6-8 (a5-a7) */
+ REG_S a5, 6*SZREG(sp)
+ REG_S a6, 7*SZREG(sp)
+ REG_S a7, 8*SZREG(sp)
add a0, t0, s5
call bpf_check_timed_may_goto
@@ -39,9 +44,13 @@ SYM_FUNC_START(arch_bpf_timed_may_goto)
REG_L a1, 3*SZREG(sp)
REG_L a0, 4*SZREG(sp)
REG_L t6, 5*SZREG(sp)
+ /* Restore stack arguments 6-8 */
+ REG_L a5, 6*SZREG(sp)
+ REG_L a6, 7*SZREG(sp)
+ REG_L a7, 8*SZREG(sp)
- REG_L s0, 6*SZREG(sp)
- REG_L ra, 7*SZREG(sp)
- addi sp, sp, 8*SZREG
+ REG_L s0, 10*SZREG(sp)
+ REG_L ra, 11*SZREG(sp)
+ addi sp, sp, 12*SZREG
ret
SYM_FUNC_END(arch_bpf_timed_may_goto)
diff --git a/arch/x86/net/bpf_timed_may_goto.S b/arch/x86/net/bpf_timed_may_goto.S
index 6c6e03f725a3..dd2dde89836b 100644
--- a/arch/x86/net/bpf_timed_may_goto.S
+++ b/arch/x86/net/bpf_timed_may_goto.S
@@ -22,6 +22,8 @@ SYM_FUNC_START(arch_bpf_timed_may_goto)
pushq %rdx
pushq %rcx
pushq %r8
+ /* Save stack argument 6. */
+ pushq %r9
/*
* r10 has the pointer to count and timestamp. JIT added the frame
@@ -36,6 +38,7 @@ SYM_FUNC_START(arch_bpf_timed_may_goto)
/* BPF_REG_AX=r10 will be stored into count, so move return value to it. */
movq %rax, %r10
+ popq %r9
/* Restore r5-r0. */
popq %r8
popq %rcx
--
2.55.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH v2 bpf 19/19] selftests/bpf: Test stack arguments across may_goto
2026-10-27 19:02 [PATCH v2 bpf 00/19] bpf: Remove BPF_F_ARENA_SCALAR, fix scalar evolution and scalar forks Alexei Starovoitov
` (17 preceding siblings ...)
2026-10-27 19:03 ` [PATCH v2 bpf 18/19] bpf: Keep stack arguments in arch_bpf_timed_may_goto() Alexei Starovoitov
@ 2026-10-27 19:03 ` Alexei Starovoitov
2026-10-28 4:10 ` [PATCH v2 bpf 00/19] bpf: Remove BPF_F_ARENA_SCALAR, fix scalar evolution and scalar forks patchwork-bot+netdevbpf
19 siblings, 0 replies; 25+ messages in thread
From: Alexei Starovoitov @ 2026-10-27 19:03 UTC (permalink / raw)
To: bpf; +Cc: daniel, andrii, eddyz87, memxor
From: Alexei Starovoitov <ast@kernel.org>
Each test passes arguments 6-8 on the stack. Thus it uses all the registers
in which the JITs of x86-64, arm64 and riscv64 pass stack arguments.
may_goto then
------------------------------------------- ---------------------------
70000 iterations in the callee the callee reads the args
the time is up in the callee the callee reads the args
70000 iterations in the caller, after the the caller calls the callee
writes of the args
70000 iterations have one look at the clock. Without the fix on x86-64 the
second test fails, and the first test fails with the hpet clocksource. The
third test passes there. The result depends on the registers that the C
code uses.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
.../selftests/bpf/progs/verifier_stack_arg.c | 119 ++++++++++++++++++
1 file changed, 119 insertions(+)
diff --git a/tools/testing/selftests/bpf/progs/verifier_stack_arg.c b/tools/testing/selftests/bpf/progs/verifier_stack_arg.c
index b9f799f6e405..15a12497e074 100644
--- a/tools/testing/selftests/bpf/progs/verifier_stack_arg.c
+++ b/tools/testing/selftests/bpf/progs/verifier_stack_arg.c
@@ -433,6 +433,125 @@ __naked void stack_arg_sequential_calls(void)
);
}
+/*
+ * A JIT passes the first stack arguments in registers. may_goto is not a call: they must be
+ * the same after it, also when it looks at the clock. That is once in 65535 iterations.
+ */
+__naked __noinline __used
+static long loops_then_reads_8args(long a, long b, long c, long d, long e, long f, long g, long h)
+{
+ asm volatile (
+ "r1 = 0;"
+ "1:"
+ "may_goto 2f;"
+ "r1 += 1;"
+ "if r1 < 70000 goto 1b;"
+ "2:"
+ "r0 = *(u64 *)(r11 + 8);"
+ "r1 = *(u64 *)(r11 + 16);"
+ "r0 += r1;"
+ "r1 = *(u64 *)(r11 + 24);"
+ "r0 += r1;"
+ "exit;"
+ );
+}
+
+SEC("tc")
+__description("stack_arg: may_goto looks at the clock, then the callee reads its args")
+__success __retval(876)
+__naked void stack_arg_may_goto_in_callee(void)
+{
+ asm volatile (
+ "r1 = 1;"
+ "r2 = 2;"
+ "r3 = 3;"
+ "r4 = 4;"
+ "r5 = 5;"
+ "*(u64 *)(r11 - 24) = 800;"
+ "*(u64 *)(r11 - 16) = 70;"
+ "*(u64 *)(r11 - 8) = 6;"
+ "call loops_then_reads_8args;"
+ "exit;"
+ ::: __clobber_all
+ );
+}
+
+__naked __noinline __used
+static long spins_then_reads_8args(long a, long b, long c, long d, long e, long f, long g, long h)
+{
+ asm volatile (
+ "1:"
+ "may_goto 2f;"
+ "goto 1b;"
+ "2:"
+ "r0 = *(u64 *)(r11 + 8);"
+ "r1 = *(u64 *)(r11 + 16);"
+ "r0 += r1;"
+ "r1 = *(u64 *)(r11 + 24);"
+ "r0 += r1;"
+ "exit;"
+ );
+}
+
+SEC("tc")
+__description("stack_arg: the time of may_goto is up, then the callee reads its args")
+__success __retval(876)
+__naked void stack_arg_may_goto_timeout_in_callee(void)
+{
+ asm volatile (
+ "r1 = 1;"
+ "r2 = 2;"
+ "r3 = 3;"
+ "r4 = 4;"
+ "r5 = 5;"
+ "*(u64 *)(r11 - 24) = 800;"
+ "*(u64 *)(r11 - 16) = 70;"
+ "*(u64 *)(r11 - 8) = 6;"
+ "call spins_then_reads_8args;"
+ "exit;"
+ ::: __clobber_all
+ );
+}
+
+__naked __noinline __used
+static long reads_8args(long a, long b, long c, long d, long e, long f, long g, long h)
+{
+ asm volatile (
+ "r0 = *(u64 *)(r11 + 8);"
+ "r1 = *(u64 *)(r11 + 16);"
+ "r0 += r1;"
+ "r1 = *(u64 *)(r11 + 24);"
+ "r0 += r1;"
+ "exit;"
+ );
+}
+
+SEC("tc")
+__description("stack_arg: may_goto looks at the clock between the stores of the args and the call")
+__success __retval(876)
+__naked void stack_arg_may_goto_in_caller(void)
+{
+ asm volatile (
+ "*(u64 *)(r11 - 24) = 800;"
+ "*(u64 *)(r11 - 16) = 70;"
+ "*(u64 *)(r11 - 8) = 6;"
+ "r6 = 0;"
+ "1:"
+ "may_goto 2f;"
+ "r6 += 1;"
+ "if r6 < 70000 goto 1b;"
+ "2:"
+ "r1 = 1;"
+ "r2 = 2;"
+ "r3 = 3;"
+ "r4 = 4;"
+ "r5 = 5;"
+ "call reads_8args;"
+ "exit;"
+ ::: __clobber_all
+ );
+}
+
#else
SEC("socket")
--
2.55.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [PATCH v2 bpf 14/19] bpf: Reset the range of a packet pointer that scalar evolution clamps
2026-10-27 19:03 ` [PATCH v2 bpf 14/19] bpf: Reset the range of a packet pointer that scalar evolution clamps Alexei Starovoitov
@ 2026-10-27 19:30 ` sashiko-bot
2026-10-27 20:09 ` Alexei Starovoitov
0 siblings, 1 reply; 25+ messages in thread
From: sashiko-bot @ 2026-10-27 19:30 UTC (permalink / raw)
To: Alexei Starovoitov, bpf
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Critical] bpf: out-of-bounds packet access via step tightening in bpf_clamp_scev_regs()
- [Medium] bpf: bpf_widen_scev_regs() does not reset negative packet pointer range
--
commit a6747880bf7fa92bdac4e2fa9d3223b8203c86f7
Author: Alexei Starovoitov <ast@kernel.org>
bpf: Reset the range of a packet pointer that scalar evolution clamps
This patch resets a clamped packet pointer's range to max(entry_reg->range, 0)
in bpf_clamp_scev_regs() so that packet bounds gained from pkt_end comparisons
inside a counted loop do not remain attached to the widened offset range across
all iterations.
> diff --git a/kernel/bpf/scev.c b/kernel/bpf/scev.c
> index 89f96682af3ff..7b82fd79f9cc9 100644
> --- a/kernel/bpf/scev.c
> +++ b/kernel/bpf/scev.c
> @@ -2914,6 +2914,13 @@ int bpf_clamp_scev_regs(struct bpf_verifier_env *env, struct bpf_func_state *cur
> bpf_log(log, "\n");
> }
> scratch_widened_reg_id(env, reg);
> + /*
> + * The range of a packet pointer goes together with its id and with the offsets
> + * that it had in the compare with pkt_end. Both are replaced here. What holds for
> + * the new offsets is the range at the entry.
> + */
> + if (type_is_pkt_pointer(reg->type))
> + reg->range = max(entry_reg->range, 0);
[Severity: Medium]
Should bpf_widen_scev_regs() also clamp reg->range with max(reg->range, 0)
when widening a packet pointer at loop entry?
When a packet pointer fails a pkt_end comparison before a counted loop,
mark_pkt_end() sets entry_reg->range to AT_PKT_END (-1) or BEYOND_PKT_END (-2).
At loop entry, bpf_widen_scev_regs() replaces the packet pointer's id and
offsets across all iterations without clearing a negative reg->range:
kernel/bpf/scev.c:bpf_widen_scev_regs() {
...
scratch_widened_reg_id(env, reg);
err = bpf_set_reg_range(env, reg, bounds.r64, bounds.r32,
bounds.base, bounds.step);
...
}
If the loop has a negative slope that moves the pointer backwards into the
packet on subsequent iterations, is_pkt_ptr_branch_taken() in
kernel/bpf/verifier.c will still fold pkt_end comparisons on the widened
register using the stale negative range marker.
In addition, when the backedge reaches the loop header, bpf_clamp_scev_regs()
sets rcur->range to 0 via max(entry_reg->range, 0) while the widened header
state still has rold->range < 0, causing regsafe() in kernel/bpf/states.c to
reject state pruning at the loop header:
kernel/bpf/states.c:regsafe() {
...
if (rold->range < 0 || rcur->range < 0) {
/* special case for [BEYOND|AT]_PKT_END */
if (rold->range != rcur->range)
return false;
}
...
}
> err = bpf_set_reg_range(env, reg, bounds.r64, bounds.r32,
> bounds.base, bounds.step);
[Severity: Critical]
Can preserving entry_reg->range across bpf_set_reg_range() here (and in
bpf_widen_scev_regs()) allow out-of-bounds packet reads and writes past
pkt_end when bounds.step tightens reg_umax(reg)?
In find_good_pkt_pointers(), comparing a derived packet pointer against
pkt_end sets reg->range = reg_umax(dst_reg), and check_packet_access() ->
check_mem_region_access() allows accesses of size bytes when
reg_umax(reg) + off + size <= reg->range. As seen in commit b88822d2584f
("bpf: Fix packet range of pointers sharing an id"), tightening the umax of a
packet pointer via var_off alignment while keeping its range is unsafe.
Before a counted loop, a packet pointer can have an unaligned umax paired with
an aligned var_off and step == 1:
r4 &= 0x38
if r4 > 50 goto exit ; r4: r64=[0, 50], var_off=(0x0; 0x38), step=1
r6 = pkt_data
r6 += r4 ; r6: pkt(id=1, r=0, r64=[0, 50], var_off=(0x0; 0x38), step=1)
r5 = r6
r5 += 8 ; r5: pkt(id=1, r=0, r64=[8, 58], var_off=(0x0; 0x78), step=1)
if r5 > pkt_end goto exit ; r6: pkt(id=1, r=58, r64=[0, 50], var_off=(0x0; 0x38), step=1)
When entering or clamping a 2-iteration loop (max_header_count = 2) that
increments r6 by 8 (slope = 8), linear_bounds() in kernel/bpf/scev.c computes
bounds.r64 = [0, 58] and promotes tnum_alignment(entry_reg->var_off) into
bounds.step:
kernel/bpf/scev.c:linear_bounds() {
...
if (cnum64_is_const(reg->r64) && !overflow) {
out->step = slope_abs;
out->base = imod(reg_smin(reg), out->step);
} else {
out->step = 1u << min_t(u32, tnum_alignment(reg->var_off), __ffs(slope_abs));
out->base = 0;
}
}
Here bounds.step becomes 8. Calling bpf_set_reg_range() -> reg_bounds_sync()
-> __update_bounds_from_step() -> cnum64_intersect_linear() then rounds
reg_umax(reg) down from 58 to 56 while reg->range remains 58 (and similarly
rounds [0, 50] down to [0, 48] with range = 50 for a negative slope r6 -= 8).
At runtime, if r4 == 48 and pkt_len == 56, on the second iteration r6 reaches
pkt_data + 56 == pkt_end, but check_packet_access() will permit a 2-byte
access at r6 + 0 because reg_umax(reg) + 2 <= 58.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261027190314.1440199-1-alexei.starovoitov@gmail.com?part=14
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v2 bpf 18/19] bpf: Keep stack arguments in arch_bpf_timed_may_goto()
2026-10-27 19:03 ` [PATCH v2 bpf 18/19] bpf: Keep stack arguments in arch_bpf_timed_may_goto() Alexei Starovoitov
@ 2026-10-27 19:48 ` bot+bpf-ci
2026-10-28 4:53 ` Alexei Starovoitov
0 siblings, 1 reply; 25+ messages in thread
From: bot+bpf-ci @ 2026-10-27 19:48 UTC (permalink / raw)
To: alexei.starovoitov, bpf
Cc: daniel, andrii, eddyz87, memxor, ast, andrii, daniel, martin.lau,
eddyz87, yonghong.song, mason, ihor.solodrai
[-- Attachment #1: Type: text/plain, Size: 4644 bytes --]
> bpf: Keep stack arguments in arch_bpf_timed_may_goto()
TL;DR: The fix covers may_goto, but LD_ABS/LD_IND may have the same flaw: they
call C helpers that can clobber stack-argument registers while the verifier
trusts them. Should check_ld_abs() handle it too?
> bpf: Keep stack arguments in arch_bpf_timed_may_goto()
>
> The JITs pass the first stack arguments of a call in registers:
>
> x86-64 r9 argument 6
> arm64 x5-x7 arguments 6-8
> riscv64 a5-a7 arguments 6-8
>
> '*(u64 *)(r11 - 8) = rX' writes such a register in the caller.
> 'rX = *(u64 *)(r11 + 8)' reads it in the callee. A call changes these
> registers. Thus the verifier accepts no call between the write and the call
> of the callee, and no call in the callee before the read.
>
> may_goto is not a call for the verifier. But one time in 65535 iterations
> it calls arch_bpf_timed_may_goto(), and this function calls C code. It
> saves R0-R5 only.
>
> *(u64 *)(r11 - 8) = 0 ; the verifier: argument 6 is 0
> call sub
> sub:
> 1: may_goto 2f ; C code changes r9
> goto 1b
> 2: r2 = *(u64 *)(r11 + 8) ; r2 is a kernel address
> r1 = map_value
> r1 += r2
> *(u64 *)(r1 + 0) = 0
>
> On x86-64 the C code changes r9 when the time of may_goto is up. With the
> hpet clocksource a look at the clock changes r9 too. The prog above crashes
> the kernel. LLVM does not emit such code: it reads stack arguments at the
> entry of a function, and it writes them immediately before the call.
>
> Save and restore these registers too. The cost is 2 insns on x86-64, 4 on
> arm64 and 6 on riscv64, one time in 65535 iterations.
>
> Fixes: 324c3ca6eed6 ("bpf,x86: Implement JIT support for stack arguments")
> Fixes: 235b2fe772f5 ("bpf, arm64: Add JIT support for stack arguments")
> Fixes: ba754185f902 ("riscv, bpf: Add BPF stack arguments support for RV64 JIT")
> Signed-off-by: Alexei Starovoitov <ast@kernel.org>
The commit message says "the verifier accepts no call between the write and
the call of the callee, and no call in the callee before the read", and
treats may_goto as the instruction that is not a call for the verifier but
still reaches C code.
Is may_goto the only such case? LD_ABS/LD_IND look like they have the same
problem, and nothing in this series covers them.
The comment above check_ld_abs() in kernel/bpf/verifier.c calls them
"wrappers of function calls", and bpf_gen_ld_abs() in net/core/filter.c
rewrites them into a real call:
*insn++ = BPF_EMIT_CALL(bpf_skb_load_helper_8_no_cache); /* or _16 / _32 */
But check_ld_abs() only scratches R1-R5:
for (i = 0; i < CALLER_SAVED_REGS; i++) {
bpf_mark_reg_not_init(env, ®s[caller_saved[i]]);
check_reg_arg(env, caller_saved[i], DST_OP_NO_MARK);
}
It does not set cur_func(env)->no_stack_arg_load, which the call path in
do_check() sets so that check_stack_arg_read() refuses a later r11 load. It
also does not call invalidate_outgoing_stack_args(), which
check_helper_call(), check_kfunc_call() and check_func_call() do.
This looks reachable. may_access_skb() allows LD_ABS in SOCKET_FILTER,
SCHED_CLS and SCHED_ACT, and a static subprog with BTF can use it through
has_ld_abs. btf_check_arg_slots() rejects stack args only for global
functions. So a tc program could have:
sub(a1..a6): ; a6 is stack arg 6
r6 = r1
r0 = *(u8 *)skb[0] ; LD_ABS -> call bpf_skb_load_helper_8_no_cache()
r2 = *(u64 *)(r11 + 8) ; verifier still trusts the caller's value
The same applies on the caller side:
*(u64 *)(r11 - 8) = 0
LD_ABS
call sub
On x86-64, r9 is not saved around a helper call when a program has stack
args, because priv_frame_ptr is NULL in that case. arm64 does not save
x5-x7 and riscv64 does not save a5-a7 around a helper call either. The C
helper can change these registers, for example on the
skb_header_pointer()/skb_copy_bits() path. That is the same kind of
verifier/runtime mismatch as in the may_goto example above.
This patch does not introduce the gap, since it goes back to the same three
Fixes: commits, and the later commits in the series do not touch
check_ld_abs() or no_stack_arg_load.
Should LD_ABS/LD_IND be handled as well, for example by setting
no_stack_arg_load and calling invalidate_outgoing_stack_args() in
check_ld_abs()?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/40568443226
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v2 bpf 14/19] bpf: Reset the range of a packet pointer that scalar evolution clamps
2026-10-27 19:30 ` sashiko-bot
@ 2026-10-27 20:09 ` Alexei Starovoitov
0 siblings, 0 replies; 25+ messages in thread
From: Alexei Starovoitov @ 2026-10-27 20:09 UTC (permalink / raw)
To: sashiko-reviews, bpf
On Tue, Oct 27, 2026 at 12:30 PM sashiko-bot@kernel.org <sashiko-bot@kernel.org> wrote:
>> + if (type_is_pkt_pointer(reg->type))
>> + reg->range = max(entry_reg->range, 0);
>
> [Severity: Medium]
> Should bpf_widen_scev_regs() also clamp reg->range with max(reg->range, 0)
> when widening a packet pointer at loop entry?
[...]
> If the loop has a negative slope that moves the pointer backwards into the
> packet on subsequent iterations, is_pkt_ptr_branch_taken() in
> kernel/bpf/verifier.c will still fold pkt_end comparisons on the widened
> register using the stale negative range marker.
>
> In addition, when the backedge reaches the loop header, bpf_clamp_scev_regs()
> sets rcur->range to 0 via max(entry_reg->range, 0) while the widened header
> state still has rold->range < 0, causing regsafe() in kernel/bpf/states.c to
> reject state pruning at the loop header:
Not a bug.
The negative range is correct in the first iteration. The next iterations
come from the clamp with range 0. regsafe() doesn't prune 0 against -2,
so the verifier verifies the loop again with range 0 and takes both
branches of the compare.
The cost is one more pass over the loop. It's the same before this patch:
adjust_ptr_min_max_vals() clears a negative range in r6 -= 8.
>> err = bpf_set_reg_range(env, reg, bounds.r64, bounds.r32,
>> bounds.base, bounds.step);
>
> [Severity: Critical]
> Can preserving entry_reg->range across bpf_set_reg_range() here (and in
> bpf_widen_scev_regs()) allow out-of-bounds packet reads and writes past
> pkt_end when bounds.step tightens reg_umax(reg)?
[...]
> Here bounds.step becomes 8. Calling bpf_set_reg_range() -> reg_bounds_sync()
> -> __update_bounds_from_step() -> cnum64_intersect_linear() then rounds
> reg_umax(reg) down from 58 to 56 while reg->range remains 58 (and similarly
> rounds [0, 50] down to [0, 48] with range = 50 for a negative slope r6 -= 8).
yeah. It's an old bug, not from this patch.
bpf_widen_scev_regs() has it since commit 6ccf49f79327: it keeps the range
and lowers umax in the same way. The clamp is not necessary to hit it.
I have a fix ready and will send in follow up after this lands.
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v2 bpf 00/19] bpf: Remove BPF_F_ARENA_SCALAR, fix scalar evolution and scalar forks
2026-10-27 19:02 [PATCH v2 bpf 00/19] bpf: Remove BPF_F_ARENA_SCALAR, fix scalar evolution and scalar forks Alexei Starovoitov
` (18 preceding siblings ...)
2026-10-27 19:03 ` [PATCH v2 bpf 19/19] selftests/bpf: Test stack arguments across may_goto Alexei Starovoitov
@ 2026-10-28 4:10 ` patchwork-bot+netdevbpf
19 siblings, 0 replies; 25+ messages in thread
From: patchwork-bot+netdevbpf @ 2026-10-28 4:10 UTC (permalink / raw)
To: Alexei Starovoitov; +Cc: bpf, daniel, andrii, eddyz87, memxor
Hello:
This series was applied to bpf/bpf.git (master)
by Alexei Starovoitov <ast@kernel.org>:
On Tue, 27 Oct 2026 19:02:55 +0000 you wrote:
> From: Alexei Starovoitov <ast@kernel.org>
>
> This series removes BPF_F_ARENA_SCALAR and fixes bugs in scalar evolution,
> in maybe_fork_scalars() and in the trampoline of may_goto. The groups of
> patches do not depend on each other.
>
> v1 was Part 1 of "bpf: Stack in arena and a verifier that joins states"
> [1]. Parts 2 and 3 of that series are not here.
>
> [...]
Here is the summary with links:
- [v2,bpf,01/19] selftests/bpf: Remove tests for global data in arena
https://git.kernel.org/bpf/bpf/c/3c13b2e5a40e
- [v2,bpf,02/19] libbpf: Remove .arena.data
https://git.kernel.org/bpf/bpf/c/5b0678b2f9b9
- [v2,bpf,03/19] libbpf: Keep format strings of bpf_printk() in .rodata
https://git.kernel.org/bpf/bpf/c/58cbd31c399b
- [v2,bpf,04/19] selftests/bpf: Remove tests for arena access through numbers
https://git.kernel.org/bpf/bpf/c/fe8e06aa0575
- [v2,bpf,05/19] bpf: Remove BPF_F_ARENA_SCALAR
https://git.kernel.org/bpf/bpf/c/b62dee3a8bb5
- [v2,bpf,06/19] bpf, x86: Remove r0_on_stack from fetching atomics in arena
https://git.kernel.org/bpf/bpf/c/701edc66772b
- [v2,bpf,07/19] bpf: Clear delta of a register that scalar evolution widens
https://git.kernel.org/bpf/bpf/c/3f9924747c44
- [v2,bpf,08/19] selftests/bpf: Test that a widened register is not linked
https://git.kernel.org/bpf/bpf/c/165115760433
- [v2,bpf,09/19] selftests/bpf: Wait for the last maps of map_kptr to be freed
https://git.kernel.org/bpf/bpf/c/2ad395a38471
- [v2,bpf,10/19] selftests/bpf: Avoid a page fault in rcu_reentry while the handler is armed
https://git.kernel.org/bpf/bpf/c/3c0ca9f9a98e
- [v2,bpf,11/19] bpf: Don't widen a pointer to a buffer in scalar evolution
https://git.kernel.org/bpf/bpf/c/41750e5d682a
- [v2,bpf,12/19] selftests/bpf: Test a loop that moves a pointer to a buffer
https://git.kernel.org/bpf/bpf/c/1b4ed112beb8
- [v2,bpf,13/19] bpf: Don't clamp a register whose type changed in the loop
https://git.kernel.org/bpf/bpf/c/1536e9b4375f
- [v2,bpf,14/19] bpf: Reset the range of a packet pointer that scalar evolution clamps
https://git.kernel.org/bpf/bpf/c/41b06cfd1cea
- [v2,bpf,15/19] selftests/bpf: Test the clamp of pointers in scalar evolution
https://git.kernel.org/bpf/bpf/c/ed5036531482
- [v2,bpf,16/19] bpf: Finish the insn in the state that maybe_fork_scalars() splits off
https://git.kernel.org/bpf/bpf/c/19caa38daf9d
- [v2,bpf,17/19] selftests/bpf: Test maybe_fork_scalars() at the head of a counted loop
https://git.kernel.org/bpf/bpf/c/df9e4a87a8d3
- [v2,bpf,18/19] bpf: Keep stack arguments in arch_bpf_timed_may_goto()
https://git.kernel.org/bpf/bpf/c/4d0f560f4926
- [v2,bpf,19/19] selftests/bpf: Test stack arguments across may_goto
https://git.kernel.org/bpf/bpf/c/b433cdb1cc4f
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v2 bpf 18/19] bpf: Keep stack arguments in arch_bpf_timed_may_goto()
2026-10-27 19:48 ` bot+bpf-ci
@ 2026-10-28 4:53 ` Alexei Starovoitov
0 siblings, 0 replies; 25+ messages in thread
From: Alexei Starovoitov @ 2026-10-28 4:53 UTC (permalink / raw)
To: bot+bpf-ci, bpf
Cc: daniel, andrii, eddyz87, memxor, martin.lau, yonghong.song, mason,
ihor.solodrai
On Tue, Oct 27, 2026 at 12:48 PM bot+bpf-ci@kernel.org <bot+bpf-ci@kernel.org> wrote:
> Is may_goto the only such case? LD_ABS/LD_IND look like they have the same
> problem, and nothing in this series covers them.
[...]
> Should LD_ABS/LD_IND be handled as well, for example by setting
> no_stack_arg_load and calling invalidate_outgoing_stack_args() in
> check_ld_abs()?
yeah. It's an old bug, not from this patch.
check_ld_abs() has it since commit 0f6bd5e7a804 ("bpf: Support stack
arguments for bpf functions"). It takes hand written asm and CAP_BPF to
hit it. LLVM doesn't emit such code, same as with may_goto.
Low prio fix. Will get to it eventually.
^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2026-10-28 4:53 UTC | newest]
Thread overview: 25+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-10-27 19:02 [PATCH v2 bpf 00/19] bpf: Remove BPF_F_ARENA_SCALAR, fix scalar evolution and scalar forks Alexei Starovoitov
2026-10-27 19:02 ` [PATCH v2 bpf 01/19] selftests/bpf: Remove tests for global data in arena Alexei Starovoitov
2026-10-27 19:02 ` [PATCH v2 bpf 02/19] libbpf: Remove .arena.data Alexei Starovoitov
2026-10-27 19:02 ` [PATCH v2 bpf 03/19] libbpf: Keep format strings of bpf_printk() in .rodata Alexei Starovoitov
2026-10-27 19:02 ` [PATCH v2 bpf 04/19] selftests/bpf: Remove tests for arena access through numbers Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 05/19] bpf: Remove BPF_F_ARENA_SCALAR Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 06/19] bpf, x86: Remove r0_on_stack from fetching atomics in arena Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 07/19] bpf: Clear delta of a register that scalar evolution widens Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 08/19] selftests/bpf: Test that a widened register is not linked Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 09/19] selftests/bpf: Wait for the last maps of map_kptr to be freed Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 10/19] selftests/bpf: Avoid a page fault in rcu_reentry while the handler is armed Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 11/19] bpf: Don't widen a pointer to a buffer in scalar evolution Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 12/19] selftests/bpf: Test a loop that moves a pointer to a buffer Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 13/19] bpf: Don't clamp a register whose type changed in the loop Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 14/19] bpf: Reset the range of a packet pointer that scalar evolution clamps Alexei Starovoitov
2026-10-27 19:30 ` sashiko-bot
2026-10-27 20:09 ` Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 15/19] selftests/bpf: Test the clamp of pointers in scalar evolution Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 16/19] bpf: Finish the insn in the state that maybe_fork_scalars() splits off Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 17/19] selftests/bpf: Test maybe_fork_scalars() at the head of a counted loop Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 18/19] bpf: Keep stack arguments in arch_bpf_timed_may_goto() Alexei Starovoitov
2026-10-27 19:48 ` bot+bpf-ci
2026-10-28 4:53 ` Alexei Starovoitov
2026-10-27 19:03 ` [PATCH v2 bpf 19/19] selftests/bpf: Test stack arguments across may_goto Alexei Starovoitov
2026-10-28 4:10 ` [PATCH v2 bpf 00/19] bpf: Remove BPF_F_ARENA_SCALAR, fix scalar evolution and scalar forks patchwork-bot+netdevbpf
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.