* [PATCH bpf-next v4 1/9] bpf: Upgrade scalar to PTR_TO_ARENA on arena pointer addition
2026-04-07 4:57 [PATCH bpf-next v4 0/9] Introduce arena library and runtime Emil Tsalapatis
@ 2026-04-07 4:57 ` Emil Tsalapatis
2026-04-07 5:43 ` bot+bpf-ci
` (2 more replies)
2026-04-07 4:57 ` [PATCH bpf-next v4 2/9] selftests/bpf: Add test for scalar/arena " Emil Tsalapatis
` (7 subsequent siblings)
8 siblings, 3 replies; 19+ messages in thread
From: Emil Tsalapatis @ 2026-04-07 4:57 UTC (permalink / raw)
To: bpf; +Cc: ast, andrii, memxor, daniel, eddyz87, song, Emil Tsalapatis
The compiler sometimes stores the result of a PTR_TO_ARENA + SCALAR
addition into the scalar register rather than the pointer register.
Handle this case by upgrading the destination scalar register to
PTR_TO_ARENA, matching the existing handling when the destination is
already PTR_TO_ARENA.
Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
Acked-by: Song Liu <song@kernel.org>
---
kernel/bpf/verifier.c | 27 +++++++++++++++++++++++++--
1 file changed, 25 insertions(+), 2 deletions(-)
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 8c1cf2eb6cbb..30bb71f90477 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -16297,11 +16297,34 @@ static int adjust_reg_min_max_vals(struct bpf_verifier_env *env,
int err;
dst_reg = ®s[insn->dst_reg];
- src_reg = NULL;
+ if (BPF_SRC(insn->code) == BPF_X)
+ src_reg = ®s[insn->src_reg];
+ else
+ src_reg = NULL;
- if (dst_reg->type == PTR_TO_ARENA) {
+ /* Case where at least one operand is an arena. */
+ if (dst_reg->type == PTR_TO_ARENA || (src_reg && src_reg->type == PTR_TO_ARENA)) {
struct bpf_insn_aux_data *aux = cur_aux(env);
+ /* The compiler sometimes stores the result of a SCALAR/PTR_TO_ARENA
+ * operation into the scalar register. Properly mark the result as
+ * holding an arena pointer.
+ */
+ if (dst_reg->type != PTR_TO_ARENA) {
+ /* Can't do arena arithmetic with non-scalars. */
+ if (dst_reg->type != SCALAR_VALUE) {
+ verbose(env, "R%d pointer %s arena prohibited\n",
+ insn->dst_reg,
+ bpf_alu_string[opcode >> 4]);
+ return -EACCES;
+ }
+
+ /* We have a SCALAR_PTR_TO_ARENA operation,
+ * propagate the info to dst_reg. */
+ *dst_reg = *src_reg;
+ }
+
+
if (BPF_CLASS(insn->code) == BPF_ALU64)
/*
* 32-bit operations zero upper bits automatically.
--
2.53.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* Re: [PATCH bpf-next v4 1/9] bpf: Upgrade scalar to PTR_TO_ARENA on arena pointer addition
2026-04-07 4:57 ` [PATCH bpf-next v4 1/9] bpf: Upgrade scalar to PTR_TO_ARENA on arena pointer addition Emil Tsalapatis
@ 2026-04-07 5:43 ` bot+bpf-ci
2026-04-07 5:52 ` Leon Hwang
2026-04-07 16:10 ` Alexei Starovoitov
2 siblings, 0 replies; 19+ messages in thread
From: bot+bpf-ci @ 2026-04-07 5:43 UTC (permalink / raw)
To: emil, bpf
Cc: ast, andrii, memxor, daniel, eddyz87, song, emil, ast, andrii,
daniel, martin.lau, eddyz87, yonghong.song, clm, ihor.solodrai
[-- Attachment #1: Type: text/plain, Size: 2446 bytes --]
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index 1bebbdb3b693..6370290e72fd 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -16619,11 +16619,34 @@ static int adjust_reg_min_max_vals(struct bpf_verifier_env *env,
[ ... ]
> + if (dst_reg->type != PTR_TO_ARENA) {
> + /* Can't do arena arithmetic with non-scalars. */
> + if (dst_reg->type != SCALAR_VALUE) {
> + verbose(env, "R%d pointer %s arena prohibited\n",
> + insn->dst_reg,
> + bpf_alu_string[opcode >> 4]);
> + return -EACCES;
> + }
> +
> + /* We have a SCALAR_PTR_TO_ARENA operation,
> + * propagate the info to dst_reg. */
> + *dst_reg = *src_reg;
> + }
Should subreg_def be updated after the struct copy here?
Before adjust_reg_min_max_vals() is called, check_alu_op() calls
check_reg_arg(env, insn->dst_reg, DST_OP_NO_MARK) which sets
dst_reg->subreg_def for the current instruction. The struct copy
overwrites this with src_reg's subreg_def, which points to the
instruction that originally created the PTR_TO_ARENA (e.g. the
addr_space_cast).
Elsewhere in check_alu_op(), every MOV path that uses
copy_register_state() explicitly resets subreg_def afterwards:
check_alu_op():
copy_register_state(dst_reg, src_reg);
dst_reg->subreg_def = env->insn_idx + 1;
For ALU64 operations this is benign, because mark_insn_zext() clears
src_reg->subreg_def to DEF_NOT_SUBREG during the preceding SRC_OP
check. But for ALU32 operations (where rw64 is false and
mark_insn_zext is not called), the stale subreg_def propagates into
the destination register.
On architectures where bpf_jit_needs_zext() returns true,
opt_subreg_zext_lo32_rnd_hi32() relies on zext_dst (set via
mark_insn_zext using subreg_def) to insert explicit zero-extension.
With a stale subreg_def, the current ALU instruction would not get
zext_dst set, potentially leaving upper 32 bits uninitialized.
When dst_reg is already PTR_TO_ARENA (the pre-existing path), the
subreg_def from check_reg_arg is preserved. Adding something like:
*dst_reg = *src_reg;
dst_reg->subreg_def = env->insn_idx + 1;
would keep the new path consistent with both the old path and the
MOV handling convention.
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/24065579421
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PATCH bpf-next v4 1/9] bpf: Upgrade scalar to PTR_TO_ARENA on arena pointer addition
2026-04-07 4:57 ` [PATCH bpf-next v4 1/9] bpf: Upgrade scalar to PTR_TO_ARENA on arena pointer addition Emil Tsalapatis
2026-04-07 5:43 ` bot+bpf-ci
@ 2026-04-07 5:52 ` Leon Hwang
2026-04-07 16:10 ` Alexei Starovoitov
2 siblings, 0 replies; 19+ messages in thread
From: Leon Hwang @ 2026-04-07 5:52 UTC (permalink / raw)
To: Emil Tsalapatis, bpf; +Cc: ast, andrii, memxor, daniel, eddyz87, song
On 7/4/26 12:57, Emil Tsalapatis wrote:
[...]
> +
> + /* We have a SCALAR_PTR_TO_ARENA operation,
> + * propagate the info to dst_reg. */
NIT: should keep consistent comment style.
> + *dst_reg = *src_reg;
> + }
> +
> +
NIT: dup blank line.
Thanks,
Leon
> if (BPF_CLASS(insn->code) == BPF_ALU64)
> /*
> * 32-bit operations zero upper bits automatically.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH bpf-next v4 1/9] bpf: Upgrade scalar to PTR_TO_ARENA on arena pointer addition
2026-04-07 4:57 ` [PATCH bpf-next v4 1/9] bpf: Upgrade scalar to PTR_TO_ARENA on arena pointer addition Emil Tsalapatis
2026-04-07 5:43 ` bot+bpf-ci
2026-04-07 5:52 ` Leon Hwang
@ 2026-04-07 16:10 ` Alexei Starovoitov
2 siblings, 0 replies; 19+ messages in thread
From: Alexei Starovoitov @ 2026-04-07 16:10 UTC (permalink / raw)
To: Emil Tsalapatis
Cc: bpf, Alexei Starovoitov, Andrii Nakryiko, Kumar Kartikeya Dwivedi,
Daniel Borkmann, Eduard, Song Liu
On Mon, Apr 6, 2026 at 9:57 PM Emil Tsalapatis <emil@etsalapatis.com> wrote:
>
> The compiler sometimes stores the result of a PTR_TO_ARENA + SCALAR
> addition into the scalar register rather than the pointer register.
> Handle this case by upgrading the destination scalar register to
> PTR_TO_ARENA, matching the existing handling when the destination is
> already PTR_TO_ARENA.
>
> Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
> Acked-by: Song Liu <song@kernel.org>
> ---
> kernel/bpf/verifier.c | 27 +++++++++++++++++++++++++--
> 1 file changed, 25 insertions(+), 2 deletions(-)
>
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index 8c1cf2eb6cbb..30bb71f90477 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -16297,11 +16297,34 @@ static int adjust_reg_min_max_vals(struct bpf_verifier_env *env,
> int err;
>
> dst_reg = ®s[insn->dst_reg];
> - src_reg = NULL;
> + if (BPF_SRC(insn->code) == BPF_X)
> + src_reg = ®s[insn->src_reg];
> + else
> + src_reg = NULL;
>
> - if (dst_reg->type == PTR_TO_ARENA) {
> + /* Case where at least one operand is an arena. */
> + if (dst_reg->type == PTR_TO_ARENA || (src_reg && src_reg->type == PTR_TO_ARENA)) {
> struct bpf_insn_aux_data *aux = cur_aux(env);
>
> + /* The compiler sometimes stores the result of a SCALAR/PTR_TO_ARENA
> + * operation into the scalar register. Properly mark the result as
> + * holding an arena pointer.
> + */
No networking comment style in the new code. Please.
Can probably drop the whole comment.
> + if (dst_reg->type != PTR_TO_ARENA) {
> + /* Can't do arena arithmetic with non-scalars. */
> + if (dst_reg->type != SCALAR_VALUE) {
> + verbose(env, "R%d pointer %s arena prohibited\n",
How agents suppose to understand this message ?
The comment right above it is more meaningful than this verifier message.
> + insn->dst_reg,
> + bpf_alu_string[opcode >> 4]);
> + return -EACCES;
> + }
> +
> + /* We have a SCALAR_PTR_TO_ARENA operation,
> + * propagate the info to dst_reg. */
As Leon said...
Also SCALAR_PTR_TO_ARENA is a new enum invention.
I say, just remove the comment.
> + *dst_reg = *src_reg;
> + }
> +
> +
> if (BPF_CLASS(insn->code) == BPF_ALU64)
> /*
> * 32-bit operations zero upper bits automatically.
> --
> 2.53.0
>
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH bpf-next v4 2/9] selftests/bpf: Add test for scalar/arena pointer addition
2026-04-07 4:57 [PATCH bpf-next v4 0/9] Introduce arena library and runtime Emil Tsalapatis
2026-04-07 4:57 ` [PATCH bpf-next v4 1/9] bpf: Upgrade scalar to PTR_TO_ARENA on arena pointer addition Emil Tsalapatis
@ 2026-04-07 4:57 ` Emil Tsalapatis
2026-04-07 4:57 ` [PATCH bpf-next v4 3/9] selftests/bpf: Move bpf_arena_spin_lock.h to the top level Emil Tsalapatis
` (6 subsequent siblings)
8 siblings, 0 replies; 19+ messages in thread
From: Emil Tsalapatis @ 2026-04-07 4:57 UTC (permalink / raw)
To: bpf; +Cc: ast, andrii, memxor, daniel, eddyz87, song, Emil Tsalapatis
Add a selftest that ensures scalar += ptr_to_arena instructions
are accepted by the verifier.
Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
Acked-by: Song Liu <song@kernel.org>
---
.../selftests/bpf/progs/verifier_arena.c | 85 +++++++++++++++++++
1 file changed, 85 insertions(+)
diff --git a/tools/testing/selftests/bpf/progs/verifier_arena.c b/tools/testing/selftests/bpf/progs/verifier_arena.c
index c4b8daac4388..7ba60a7b9ac9 100644
--- a/tools/testing/selftests/bpf/progs/verifier_arena.c
+++ b/tools/testing/selftests/bpf/progs/verifier_arena.c
@@ -477,4 +477,89 @@ int arena_kfuncs_under_bpf_lock(void *ctx)
return 0;
}
+
+#if defined(__BPF_FEATURE_ADDR_SPACE_CAST)
+
+/*
+ * Test that scalar += PTR_TO_ARENA correctly upgrades the
+ * destination register to a PTR_TO_ARENA.
+ */
+SEC("syscall")
+__success __retval(0)
+int scalar_add_arena_ptr(void *ctx)
+{
+ int __arena *scalar, *arena_ptr;
+
+ volatile char __arena *base = arena_base(&arena);
+
+ asm volatile (
+ "%[arena_ptr] = 8192;"
+ "%[arena_ptr] = addr_space_cast(%[arena_ptr], 0x0, 0x1);"
+ "%[scalar] = 12;"
+ "%[scalar] += %[arena_ptr];"
+ : [scalar] "=r"(scalar),
+ [arena_ptr] "=&r"(arena_ptr)
+ : "r"(base)
+ :
+ );
+ return 0;
+}
+
+/*
+ * Test that PTR_TO_ARENA + PTR_TO_ARENA is allowed.
+ */
+SEC("syscall")
+__success __retval(0)
+int arena_ptr_add_arena_ptr(void *ctx)
+{
+ int __arena *arena_ptr2, *arena_ptr1;
+
+ /* Needed for the verifier to link the arena to the subprog. */
+ volatile char __arena *base = arena_base(&arena);
+
+ asm volatile (
+ "%[arena_ptr1] = 8192;"
+ "%[arena_ptr1] = addr_space_cast(%[arena_ptr1], 0x0, 0x1);"
+ "%[arena_ptr2] = 4096;"
+ "%[arena_ptr2] = addr_space_cast(%[arena_ptr2], 0x0, 0x1);"
+ "%[arena_ptr2] += %[arena_ptr1];"
+ : [arena_ptr2] "=r"(arena_ptr2),
+ [arena_ptr1] "=&r"(arena_ptr1)
+ : "r"(base)
+ :
+ );
+ return 0;
+}
+
+/*
+ * Test that PTR_TO_ARENA + non-arena pointer is rejected.
+ */
+SEC("syscall")
+__failure __msg("pointer += arena prohibited")
+int arena_ptr_add_to_non_arena_ptr(void *ctx)
+{
+ int __arena *arena_ptr;
+ void *dst;
+
+ /* Needed for the verifier to link the arena to the subprog. */
+ volatile char __arena *base = arena_base(&arena);
+
+ asm volatile (
+ "%[arena_ptr] = 8192;"
+ "%[arena_ptr] = addr_space_cast(%[arena_ptr], 0x0, 0x1);"
+ "%[dst] = %[ctx];"
+ "%[dst] += %[arena_ptr];"
+ : [arena_ptr] "=&r"(arena_ptr),
+ [dst] "=&r"(dst)
+ : [ctx] "r"(ctx), "r"(base)
+ :
+ );
+
+ (void)ctx;
+
+ return 0;
+}
+
+#endif
+
char _license[] SEC("license") = "GPL";
--
2.53.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH bpf-next v4 3/9] selftests/bpf: Move bpf_arena_spin_lock.h to the top level
2026-04-07 4:57 [PATCH bpf-next v4 0/9] Introduce arena library and runtime Emil Tsalapatis
2026-04-07 4:57 ` [PATCH bpf-next v4 1/9] bpf: Upgrade scalar to PTR_TO_ARENA on arena pointer addition Emil Tsalapatis
2026-04-07 4:57 ` [PATCH bpf-next v4 2/9] selftests/bpf: Add test for scalar/arena " Emil Tsalapatis
@ 2026-04-07 4:57 ` Emil Tsalapatis
2026-04-07 4:57 ` [PATCH bpf-next v4 4/9] selftests/bpf: Deduplicate WRITE_ONCE macro between headers Emil Tsalapatis
` (5 subsequent siblings)
8 siblings, 0 replies; 19+ messages in thread
From: Emil Tsalapatis @ 2026-04-07 4:57 UTC (permalink / raw)
To: bpf; +Cc: ast, andrii, memxor, daniel, eddyz87, song, Emil Tsalapatis
The bpf_arena_spin_lock.h header is useful for all programs and not
just the selftests. Move it to the top level of the BPF selftests
to make it more readily accessible.
Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
Acked-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Song Liu <song@kernel.org>
---
tools/testing/selftests/bpf/{progs => }/bpf_arena_spin_lock.h | 4 ++--
tools/testing/selftests/bpf/progs/arena_spin_lock.c | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
rename tools/testing/selftests/bpf/{progs => }/bpf_arena_spin_lock.h (99%)
diff --git a/tools/testing/selftests/bpf/progs/bpf_arena_spin_lock.h b/tools/testing/selftests/bpf/bpf_arena_spin_lock.h
similarity index 99%
rename from tools/testing/selftests/bpf/progs/bpf_arena_spin_lock.h
rename to tools/testing/selftests/bpf/bpf_arena_spin_lock.h
index f90531cf3ee5..680c9e6cb35d 100644
--- a/tools/testing/selftests/bpf/progs/bpf_arena_spin_lock.h
+++ b/tools/testing/selftests/bpf/bpf_arena_spin_lock.h
@@ -107,7 +107,7 @@ struct arena_qnode {
#define _Q_LOCKED_VAL (1U << _Q_LOCKED_OFFSET)
#define _Q_PENDING_VAL (1U << _Q_PENDING_OFFSET)
-struct arena_qnode __arena qnodes[_Q_MAX_CPUS][_Q_MAX_NODES];
+struct arena_qnode __weak __arena __hidden qnodes[_Q_MAX_CPUS][_Q_MAX_NODES];
static inline u32 encode_tail(int cpu, int idx)
{
@@ -240,7 +240,7 @@ static __always_inline int arena_spin_trylock(arena_spinlock_t __arena *lock)
return likely(atomic_try_cmpxchg_acquire(&lock->val, &val, _Q_LOCKED_VAL));
}
-__noinline
+__noinline __weak
int arena_spin_lock_slowpath(arena_spinlock_t __arena __arg_arena *lock, u32 val)
{
struct arena_mcs_spinlock __arena *prev, *next, *node0, *node;
diff --git a/tools/testing/selftests/bpf/progs/arena_spin_lock.c b/tools/testing/selftests/bpf/progs/arena_spin_lock.c
index 086b57a426cf..6c04e7707644 100644
--- a/tools/testing/selftests/bpf/progs/arena_spin_lock.c
+++ b/tools/testing/selftests/bpf/progs/arena_spin_lock.c
@@ -4,7 +4,7 @@
#include <bpf/bpf_tracing.h>
#include <bpf/bpf_helpers.h>
#include "bpf_misc.h"
-#include "bpf_arena_spin_lock.h"
+#include "../bpf_arena_spin_lock.h"
struct {
__uint(type, BPF_MAP_TYPE_ARENA);
--
2.53.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH bpf-next v4 4/9] selftests/bpf: Deduplicate WRITE_ONCE macro between headers
2026-04-07 4:57 [PATCH bpf-next v4 0/9] Introduce arena library and runtime Emil Tsalapatis
` (2 preceding siblings ...)
2026-04-07 4:57 ` [PATCH bpf-next v4 3/9] selftests/bpf: Move bpf_arena_spin_lock.h to the top level Emil Tsalapatis
@ 2026-04-07 4:57 ` Emil Tsalapatis
2026-04-07 4:57 ` [PATCH bpf-next v4 5/9] selftests/bpf: Add basic libarena scaffolding Emil Tsalapatis
` (4 subsequent siblings)
8 siblings, 0 replies; 19+ messages in thread
From: Emil Tsalapatis @ 2026-04-07 4:57 UTC (permalink / raw)
To: bpf; +Cc: ast, andrii, memxor, daniel, eddyz87, song, Emil Tsalapatis
The WRITE_ONCE macro is identically defined both in bpf_atomic.h
and in bpf_arena_common.h. Deduplicate the definitions by moving
the macro to the commonly used bpf_experimental.h header.
Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
Acked-by: Song Liu <song@kernel.org>
---
tools/testing/selftests/bpf/bpf_arena_common.h | 4 ----
tools/testing/selftests/bpf/bpf_atomic.h | 4 ----
tools/testing/selftests/bpf/bpf_experimental.h | 3 +++
3 files changed, 3 insertions(+), 8 deletions(-)
diff --git a/tools/testing/selftests/bpf/bpf_arena_common.h b/tools/testing/selftests/bpf/bpf_arena_common.h
index 16f8ce832004..a0503ecbc315 100644
--- a/tools/testing/selftests/bpf/bpf_arena_common.h
+++ b/tools/testing/selftests/bpf/bpf_arena_common.h
@@ -2,10 +2,6 @@
/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */
#pragma once
-#ifndef WRITE_ONCE
-#define WRITE_ONCE(x, val) ((*(volatile typeof(x) *) &(x)) = (val))
-#endif
-
#ifndef NUMA_NO_NODE
#define NUMA_NO_NODE (-1)
#endif
diff --git a/tools/testing/selftests/bpf/bpf_atomic.h b/tools/testing/selftests/bpf/bpf_atomic.h
index c550e5711967..d77338ac7e6b 100644
--- a/tools/testing/selftests/bpf/bpf_atomic.h
+++ b/tools/testing/selftests/bpf/bpf_atomic.h
@@ -40,10 +40,6 @@ extern bool CONFIG_X86_64 __kconfig __weak;
/* No-op for BPF */
#define cpu_relax() ({})
-#define READ_ONCE(x) (*(volatile typeof(x) *)&(x))
-
-#define WRITE_ONCE(x, val) ((*(volatile typeof(x) *)&(x)) = (val))
-
#define cmpxchg(p, old, new) __sync_val_compare_and_swap((p), old, new)
#define try_cmpxchg(p, pold, new) \
diff --git a/tools/testing/selftests/bpf/bpf_experimental.h b/tools/testing/selftests/bpf/bpf_experimental.h
index 2234bd6bc9d3..56214f7051c6 100644
--- a/tools/testing/selftests/bpf/bpf_experimental.h
+++ b/tools/testing/selftests/bpf/bpf_experimental.h
@@ -11,6 +11,9 @@
/* Convenience macro to wrap over bpf_obj_new */
#define bpf_obj_new(type) ((type *)bpf_obj_new(bpf_core_type_id_local(type)))
+#define READ_ONCE(x) (*(volatile typeof(x) *)&(x))
+#define WRITE_ONCE(x, val) ((*(volatile typeof(x) *)&(x)) = (val))
+
/* Convenience macro to wrap over bpf_percpu_obj_new */
#define bpf_percpu_obj_new(type) ((type __percpu_kptr *)bpf_percpu_obj_new(bpf_core_type_id_local(type)))
--
2.53.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH bpf-next v4 5/9] selftests/bpf: Add basic libarena scaffolding
2026-04-07 4:57 [PATCH bpf-next v4 0/9] Introduce arena library and runtime Emil Tsalapatis
` (3 preceding siblings ...)
2026-04-07 4:57 ` [PATCH bpf-next v4 4/9] selftests/bpf: Deduplicate WRITE_ONCE macro between headers Emil Tsalapatis
@ 2026-04-07 4:57 ` Emil Tsalapatis
2026-04-07 16:21 ` Alexei Starovoitov
2026-04-07 4:57 ` [PATCH bpf-next v4 6/9] selftests/bpf: Add arena ASAN runtime to libarena Emil Tsalapatis
` (3 subsequent siblings)
8 siblings, 1 reply; 19+ messages in thread
From: Emil Tsalapatis @ 2026-04-07 4:57 UTC (permalink / raw)
To: bpf; +Cc: ast, andrii, memxor, daniel, eddyz87, song, Emil Tsalapatis
Add initial code for an arena-based BPF library. The current commit
introduces a test runner and Makefile for the library. Library code
can be added just by including the source file in the library's src/
subdirectory. Future commits will introduce the library code itself.
The current commit also includes a standalone test runner. This is
to keep libarena self-contained and testable even when copied out
of the kernel tree. Subsequent commits add integration with test_progs.
Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
---
tools/testing/selftests/bpf/.gitignore | 1 +
tools/testing/selftests/bpf/Makefile | 15 ++
tools/testing/selftests/bpf/libarena/Makefile | 50 ++++++
.../selftests/bpf/libarena/include/common.h | 49 ++++++
.../bpf/libarena/include/selftest_helpers.h | 81 ++++++++++
.../selftests/bpf/libarena/include/userapi.h | 26 +++
.../bpf/libarena/selftests/selftest.c | 153 ++++++++++++++++++
.../bpf/libarena/selftests/selftest.h | 12 ++
.../selftests/bpf/libarena/src/common.bpf.c | 65 ++++++++
9 files changed, 452 insertions(+)
create mode 100644 tools/testing/selftests/bpf/libarena/Makefile
create mode 100644 tools/testing/selftests/bpf/libarena/include/common.h
create mode 100644 tools/testing/selftests/bpf/libarena/include/selftest_helpers.h
create mode 100644 tools/testing/selftests/bpf/libarena/include/userapi.h
create mode 100644 tools/testing/selftests/bpf/libarena/selftests/selftest.c
create mode 100644 tools/testing/selftests/bpf/libarena/selftests/selftest.h
create mode 100644 tools/testing/selftests/bpf/libarena/src/common.bpf.c
diff --git a/tools/testing/selftests/bpf/.gitignore b/tools/testing/selftests/bpf/.gitignore
index bfdc5518ecc8..7f1960d6b59e 100644
--- a/tools/testing/selftests/bpf/.gitignore
+++ b/tools/testing/selftests/bpf/.gitignore
@@ -49,3 +49,4 @@ verification_cert.h
*.BTF.base
usdt_1
usdt_2
+libarena/test_libarena
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index f75c4f52c028..0d332c991023 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -153,6 +153,7 @@ override define CLEAN
$(Q)$(RM) -r $(TEST_KMODS)
$(Q)$(RM) -r $(EXTRA_CLEAN)
$(Q)$(MAKE) -C test_kmods clean
+ $(Q)$(MAKE) -C libarena clean
$(Q)$(MAKE) docs-clean
endef
@@ -739,6 +740,17 @@ $(VERIFY_SIG_HDR): $(VERIFICATION_CERT)
echo "};"; \
echo "unsigned int test_progs_verification_cert_len = $$(wc -c < $<);") > $@
+LIBARENA_MAKE_ARGS = \
+ BPFTOOL="$(BPFTOOL)" \
+ INCLUDE_DIR="$(HOST_INCLUDE_DIR)" \
+ LIBBPF_INCLUDE="$(HOST_INCLUDE_DIR)" \
+ BPFOBJ="$(BPFOBJ)" \
+ LDLIBS="$(LDLIBS) -lzstd" \
+ CLANG="$(CLANG)" \
+ BPF_CFLAGS="$(BPF_CFLAGS) $(CLANG_CFLAGS)" \
+ BPF_TARGET_ENDIAN="$(BPF_TARGET_ENDIAN)" \
+ Q="$(Q)"
+
# Define test_progs test runner.
TRUNNER_TESTS_DIR := prog_tests
TRUNNER_BPF_PROGS_DIR := progs
@@ -931,3 +943,6 @@ override define INSTALL_RULE
rsync -a $(OUTPUT)/$$DIR/*.bpf.o $(INSTALL_PATH)/$$DIR;\
done
endef
+
+test_libarena: $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ)
+ +$(MAKE) -C libarena $@ $(LIBARENA_MAKE_ARGS)
diff --git a/tools/testing/selftests/bpf/libarena/Makefile b/tools/testing/selftests/bpf/libarena/Makefile
new file mode 100644
index 000000000000..151ab6d0509c
--- /dev/null
+++ b/tools/testing/selftests/bpf/libarena/Makefile
@@ -0,0 +1,50 @@
+# SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
+# Copyright (c) 2026 Meta Platforms, Inc. and affiliates.
+
+.PHONY: clean
+
+LIBARENA=$(abspath .)
+
+
+LIBARENA_SOURCES = $(wildcard $(LIBARENA)/src/*.bpf.c) $(wildcard $(LIBARENA)/selftests/*.bpf.c)
+LIBARENA_OBJECTS = $(notdir $(LIBARENA_SOURCES:.bpf.c=.bpf.o))
+
+INCLUDES = -I$(LIBARENA)/include -I$(LIBARENA)/..
+ifneq ($(INCLUDE_DIR),)
+INCLUDES += -I$(INCLUDE_DIR)
+endif
+ifneq ($(LIBBPF_INCLUDE),)
+INCLUDES += -I$(LIBBPF_INCLUDE)
+endif
+
+# ENABLE_ATOMICS_TESTS required because we use arena spinlocks
+override BPF_CFLAGS += -DENABLE_ATOMICS_TESTS
+override BPF_CFLAGS += -O2 -Wno-incompatible-pointer-types-discards-qualifiers
+override BPF_CFLAGS += $(INCLUDES)
+
+CFLAGS = -O2 -no-pie
+CFLAGS += $(INCLUDES)
+
+vpath %.bpf.c $(LIBARENA)/src $(LIBARENA)/selftests
+vpath %.c $(LIBARENA)/src $(LIBARENA)/selftests
+
+all: test_libarena
+
+test_libarena: selftest.c $(BPFOBJ) libarena.skel.h
+ $(call msg,BINARY,libarena,$@)
+ $(Q)$(CLANG) $(CFLAGS) $< $(BPFOBJ) $(LDLIBS) -o $@
+
+libarena.skel.h: main.bpf.o
+ $(call msg,GEN-SKEL,libarena,$@)
+ $(Q)$(BPFTOOL) gen skeleton $< name "libarena" > $@
+
+main.bpf.o: $(LIBARENA_OBJECTS)
+ $(call msg,GEN-OBJ,libarena,$@)
+ $(Q)$(BPFTOOL) gen object $@ $^
+
+%.bpf.o: %.bpf.c
+ $(call msg,CLNG-BPF,libarena,$@)
+ $(Q)$(CLANG) $(BPF_CFLAGS) $(BPF_TARGET_ENDIAN) -c $< -o $@
+
+clean:
+ $(Q)rm -f *.skel.h *.bpf.o test_libarena
diff --git a/tools/testing/selftests/bpf/libarena/include/common.h b/tools/testing/selftests/bpf/libarena/include/common.h
new file mode 100644
index 000000000000..544a398a0d1e
--- /dev/null
+++ b/tools/testing/selftests/bpf/libarena/include/common.h
@@ -0,0 +1,49 @@
+// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#pragma once
+
+#ifdef __BPF__
+
+#include <vmlinux.h>
+
+#include "bpf_experimental.h"
+#include "bpf_arena_common.h"
+#include "bpf_arena_spin_lock.h"
+
+#include <asm-generic/errno.h>
+
+#ifndef __BPF_FEATURE_ADDR_SPACE_CAST
+#error "Arena allocators require bpf_addr_space_cast feature"
+#endif
+
+#define arena_stdout(fmt, ...) bpf_stream_printk(1, (fmt), ##__VA_ARGS__)
+#define arena_stderr(fmt, ...) bpf_stream_printk(2, (fmt), ##__VA_ARGS__)
+
+#ifndef __maybe_unused
+#define __maybe_unused __attribute__((__unused__))
+#endif
+
+#define private(name) SEC(".data." #name) __hidden __attribute__((aligned(8)))
+
+#define ARENA_PAGES (1UL << (32 - __builtin_ffs(__PAGE_SIZE) + 1))
+
+struct {
+ __uint(type, BPF_MAP_TYPE_ARENA);
+ __uint(map_flags, BPF_F_MMAPABLE);
+ __uint(max_entries, ARENA_PAGES); /* number of pages */
+#if defined(__TARGET_ARCH_arm64) || defined(__aarch64__)
+ __ulong(map_extra, (1ull << 32)); /* start of mmap() region */
+#else
+ __ulong(map_extra, (1ull << 44)); /* start of mmap() region */
+#endif
+} arena __weak SEC(".maps");
+
+extern const volatile u32 zero;
+
+int arena_fls(__u64 word);
+
+#endif /* __BPF__ */
+
+struct arena_get_base_args {
+ void __arena *arena_base;
+};
diff --git a/tools/testing/selftests/bpf/libarena/include/selftest_helpers.h b/tools/testing/selftests/bpf/libarena/include/selftest_helpers.h
new file mode 100644
index 000000000000..ee445d8f5b26
--- /dev/null
+++ b/tools/testing/selftests/bpf/libarena/include/selftest_helpers.h
@@ -0,0 +1,81 @@
+// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#pragma once
+
+#include <errno.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <sys/mman.h>
+
+#include <bpf/libbpf.h>
+#include <bpf/bpf.h>
+
+static inline int libarena_run_prog(int prog_fd)
+{
+ LIBBPF_OPTS(bpf_test_run_opts, opts);
+ int ret;
+
+ ret = bpf_prog_test_run_opts(prog_fd, &opts);
+ if (ret)
+ return ret;
+
+ return opts.retval;
+}
+
+static inline int libarena_get_arena_base(int arena_get_base_fd,
+ void **arena_base)
+{
+ LIBBPF_OPTS(bpf_test_run_opts, opts);
+ struct arena_get_base_args args = { .arena_base = NULL };
+ int ret;
+
+ opts.ctx_in = &args;
+ opts.ctx_size_in = sizeof(args);
+
+ ret = bpf_prog_test_run_opts(arena_get_base_fd, &opts);
+ if (ret)
+ return ret;
+ if (opts.retval)
+ return opts.retval;
+
+ *arena_base = args.arena_base;
+ return 0;
+}
+
+static inline int libarena_get_globals_pages(int arena_get_base_fd,
+ size_t arena_all_pages,
+ u64 *globals_pages)
+{
+ size_t pgsize = sysconf(_SC_PAGESIZE);
+ void *arena_base;
+ ssize_t i;
+ u8 *vec;
+ int ret;
+
+ ret = libarena_get_arena_base(arena_get_base_fd, &arena_base);
+ if (ret)
+ return ret;
+
+ if (!arena_base)
+ return -EINVAL;
+
+ vec = calloc(arena_all_pages, sizeof(*vec));
+ if (!vec)
+ return -ENOMEM;
+
+ if (mincore(arena_base, arena_all_pages * pgsize, vec) < 0) {
+ ret = -errno;
+ free(vec);
+ return ret;
+ }
+
+ *globals_pages = 0;
+ for (i = arena_all_pages - 1; i >= 0; i--) {
+ if (!(vec[i] & 0x1))
+ break;
+ *globals_pages += 1;
+ }
+
+ free(vec);
+ return 0;
+}
diff --git a/tools/testing/selftests/bpf/libarena/include/userapi.h b/tools/testing/selftests/bpf/libarena/include/userapi.h
new file mode 100644
index 000000000000..b4ac6bc9bd79
--- /dev/null
+++ b/tools/testing/selftests/bpf/libarena/include/userapi.h
@@ -0,0 +1,26 @@
+// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#pragma once
+
+#include <stdint.h>
+
+/*
+ * Header for the userspace C programs that load
+ * and initialize the BPF code.
+ */
+
+#define __arena
+
+typedef uint8_t u8;
+typedef uint16_t u16;
+typedef uint32_t u32;
+typedef uint64_t u64;
+typedef int8_t s8;
+typedef int16_t s16;
+typedef int32_t s32;
+typedef int64_t s64;
+
+/* Dummy "definition" for userspace. */
+#define arena_spinlock_t u64
+
+#include "common.h"
diff --git a/tools/testing/selftests/bpf/libarena/selftests/selftest.c b/tools/testing/selftests/bpf/libarena/selftests/selftest.c
new file mode 100644
index 000000000000..b69adc7baaab
--- /dev/null
+++ b/tools/testing/selftests/bpf/libarena/selftests/selftest.c
@@ -0,0 +1,153 @@
+// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+
+#define _GNU_SOURCE
+#include <assert.h>
+#include <errno.h>
+#include <stdio.h>
+#include <string.h>
+#include <unistd.h>
+
+#include <bpf/libbpf.h>
+#include <bpf/bpf.h>
+
+#include <sys/mman.h>
+#include <sys/resource.h>
+#include <sys/sysinfo.h>
+
+#include <userapi.h>
+#include <selftest_helpers.h>
+
+#include "../libarena.skel.h"
+typedef struct libarena selftest;
+#define selftest__open libarena__open
+#define selftest__open_and_load libarena__open_and_load
+#define selftest__load libarena__load
+#define selftest__attach libarena__attach
+#define selftest__destroy libarena__destroy
+
+static bool verbose = false;
+static int testno = 1;
+
+static int
+run_prog_verbose(int prog_fd)
+{
+ char buf[1024];
+ int ret, err;
+
+ ret = libarena_run_prog(prog_fd);
+
+ if (ret)
+ fprintf(stderr, "error %d in %s\n", ret, __func__);
+
+ if (verbose) {
+ printf("BPF stdout:\n");
+ while ((err = bpf_prog_stream_read(prog_fd, 1, buf, 1024, NULL)) > 0)
+ printf("%.*s", err, buf);
+
+ if (err)
+ return err;
+
+ printf("BPF stderr:\n");
+ while ((err = bpf_prog_stream_read(prog_fd, 2, buf, 1024, NULL)) > 0)
+ printf("%.*s", err, buf);
+
+ if (err)
+ return err;
+ }
+
+ return ret;
+}
+
+static int libbpf_print_fn(enum libbpf_print_level level,
+ const char *format, va_list args)
+{
+ if (level == LIBBPF_DEBUG)
+ return 0;
+ return vfprintf(stderr, format, args);
+}
+
+int run_test(selftest *skel, const struct bpf_program *prog)
+{
+ int prog_fd;
+ int ret;
+
+ ret = libarena_run_prog(bpf_program__fd(skel->progs.arena_alloc_reserve));
+ if (ret)
+ return ret;
+
+ prog_fd = bpf_program__fd(prog);
+ if (prog_fd < 0)
+ return prog_fd;
+
+ return run_prog_verbose(prog_fd);
+}
+
+#define TEST(__test) \
+int run_##__test(void) \
+{ \
+ selftest *skel; \
+ int ret; \
+ \
+ skel = selftest__open_and_load(); \
+ if (!skel) { \
+ ret = -EINVAL; \
+ goto error_no_destroy; \
+ } \
+ \
+ ret = selftest__attach(skel); \
+ if (ret) \
+ goto error; \
+ \
+ ret = run_test(skel, skel->progs.__test); \
+ if (ret) \
+ goto error; \
+ \
+ selftest__destroy(skel); \
+ \
+ printf("ok %d - %s\n", testno++, #__test); \
+ return 0; \
+ \
+error: \
+ selftest__destroy(skel); \
+error_no_destroy: \
+ printf("not ok %d - %s\n", testno++, #__test); \
+ return ret; \
+}
+
+static void
+banner(const char *progpath)
+{
+ char *name = basename(progpath);
+
+ printf("%s\n", name);
+
+ printf("=== %s ===\n", "libarena selftests");
+}
+
+int main(int argc, char *argv[])
+{
+ int ret;
+
+ struct rlimit rlim = {
+ .rlim_cur = RLIM_INFINITY,
+ .rlim_max = RLIM_INFINITY,
+ };
+
+ banner(argv[0]);
+
+ for (int i = 1; i < argc; i++) {
+ if (strcmp(argv[i], "-v") == 0 || strcmp(argv[i], "--verbose") == 0)
+ verbose = true;
+ }
+
+ ret = setrlimit(RLIMIT_MEMLOCK, &rlim);
+ if (ret) {
+ perror("setrlimit");
+ return ret;
+ }
+
+ libbpf_set_print(libbpf_print_fn);
+
+ return 0;
+}
diff --git a/tools/testing/selftests/bpf/libarena/selftests/selftest.h b/tools/testing/selftests/bpf/libarena/selftests/selftest.h
new file mode 100644
index 000000000000..b1cbff4d343b
--- /dev/null
+++ b/tools/testing/selftests/bpf/libarena/selftests/selftest.h
@@ -0,0 +1,12 @@
+// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#pragma once
+
+#define ALLOC_SELFTEST(func, ...) \
+ do { \
+ int ret = func(__VA_ARGS__); \
+ if (ret) { \
+ arena_stderr("SELFTEST %s FAIL: %d", #func, ret); \
+ return ret; \
+ } \
+ } while (0)
diff --git a/tools/testing/selftests/bpf/libarena/src/common.bpf.c b/tools/testing/selftests/bpf/libarena/src/common.bpf.c
new file mode 100644
index 000000000000..cbb729290d3c
--- /dev/null
+++ b/tools/testing/selftests/bpf/libarena/src/common.bpf.c
@@ -0,0 +1,65 @@
+// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <common.h>
+
+const volatile u32 zero = 0;
+
+/* How many pages do we reserve at the beginning of the arena segment? */
+#define RESERVE_ALLOC (8)
+
+int arena_fls(__u64 word)
+{
+ unsigned int num = 0;
+
+ if (word & 0xffffffff00000000ULL) {
+ num += 32;
+ word >>= 32;
+ }
+
+ if (word & 0xffff0000) {
+ num += 16;
+ word >>= 16;
+ }
+
+ if (word & 0xff00) {
+ num += 8;
+ word >>= 8;
+ }
+
+ if (word & 0xf0) {
+ num += 4;
+ word >>= 4;
+ }
+
+ if (word & 0xc) {
+ num += 2;
+ word >>= 2;
+ }
+
+ if (word & 0x2) {
+ num += 1;
+ }
+
+ return num;
+}
+
+/*
+ * Userspace API, required for setting up the arena
+ * address space before starting the allocator.
+ */
+
+SEC("syscall") __weak
+int arena_get_base(struct arena_get_base_args *args)
+{
+ args->arena_base = arena_base(&arena);
+
+ return 0;
+}
+
+SEC("syscall") __weak
+int arena_alloc_reserve(void)
+{
+ return bpf_arena_reserve_pages(&arena, NULL, RESERVE_ALLOC);
+}
+
+char _license[] SEC("license") = "GPL";
--
2.53.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* Re: [PATCH bpf-next v4 5/9] selftests/bpf: Add basic libarena scaffolding
2026-04-07 4:57 ` [PATCH bpf-next v4 5/9] selftests/bpf: Add basic libarena scaffolding Emil Tsalapatis
@ 2026-04-07 16:21 ` Alexei Starovoitov
0 siblings, 0 replies; 19+ messages in thread
From: Alexei Starovoitov @ 2026-04-07 16:21 UTC (permalink / raw)
To: Emil Tsalapatis
Cc: bpf, Alexei Starovoitov, Andrii Nakryiko, Kumar Kartikeya Dwivedi,
Daniel Borkmann, Eduard, Song Liu
On Mon, Apr 6, 2026 at 9:57 PM Emil Tsalapatis <emil@etsalapatis.com> wrote:
>
> Add initial code for an arena-based BPF library. The current commit
> introduces a test runner and Makefile for the library. Library code
> can be added just by including the source file in the library's src/
> subdirectory. Future commits will introduce the library code itself.
>
> The current commit also includes a standalone test runner. This is
> to keep libarena self-contained and testable even when copied out
> of the kernel tree. Subsequent commits add integration with test_progs.
>
> Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
> ---
> tools/testing/selftests/bpf/.gitignore | 1 +
> tools/testing/selftests/bpf/Makefile | 15 ++
> tools/testing/selftests/bpf/libarena/Makefile | 50 ++++++
> .../selftests/bpf/libarena/include/common.h | 49 ++++++
> .../bpf/libarena/include/selftest_helpers.h | 81 ++++++++++
> .../selftests/bpf/libarena/include/userapi.h | 26 +++
> .../bpf/libarena/selftests/selftest.c | 153 ++++++++++++++++++
> .../bpf/libarena/selftests/selftest.h | 12 ++
> .../selftests/bpf/libarena/src/common.bpf.c | 65 ++++++++
> 9 files changed, 452 insertions(+)
> create mode 100644 tools/testing/selftests/bpf/libarena/Makefile
> create mode 100644 tools/testing/selftests/bpf/libarena/include/common.h
> create mode 100644 tools/testing/selftests/bpf/libarena/include/selftest_helpers.h
> create mode 100644 tools/testing/selftests/bpf/libarena/include/userapi.h
> create mode 100644 tools/testing/selftests/bpf/libarena/selftests/selftest.c
> create mode 100644 tools/testing/selftests/bpf/libarena/selftests/selftest.h
> create mode 100644 tools/testing/selftests/bpf/libarena/src/common.bpf.c
>
> diff --git a/tools/testing/selftests/bpf/.gitignore b/tools/testing/selftests/bpf/.gitignore
> index bfdc5518ecc8..7f1960d6b59e 100644
> --- a/tools/testing/selftests/bpf/.gitignore
> +++ b/tools/testing/selftests/bpf/.gitignore
> @@ -49,3 +49,4 @@ verification_cert.h
> *.BTF.base
> usdt_1
> usdt_2
> +libarena/test_libarena
> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> index f75c4f52c028..0d332c991023 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile
> @@ -153,6 +153,7 @@ override define CLEAN
> $(Q)$(RM) -r $(TEST_KMODS)
> $(Q)$(RM) -r $(EXTRA_CLEAN)
> $(Q)$(MAKE) -C test_kmods clean
> + $(Q)$(MAKE) -C libarena clean
> $(Q)$(MAKE) docs-clean
> endef
>
> @@ -739,6 +740,17 @@ $(VERIFY_SIG_HDR): $(VERIFICATION_CERT)
> echo "};"; \
> echo "unsigned int test_progs_verification_cert_len = $$(wc -c < $<);") > $@
>
> +LIBARENA_MAKE_ARGS = \
> + BPFTOOL="$(BPFTOOL)" \
> + INCLUDE_DIR="$(HOST_INCLUDE_DIR)" \
> + LIBBPF_INCLUDE="$(HOST_INCLUDE_DIR)" \
> + BPFOBJ="$(BPFOBJ)" \
> + LDLIBS="$(LDLIBS) -lzstd" \
> + CLANG="$(CLANG)" \
> + BPF_CFLAGS="$(BPF_CFLAGS) $(CLANG_CFLAGS)" \
> + BPF_TARGET_ENDIAN="$(BPF_TARGET_ENDIAN)" \
> + Q="$(Q)"
> +
> # Define test_progs test runner.
> TRUNNER_TESTS_DIR := prog_tests
> TRUNNER_BPF_PROGS_DIR := progs
> @@ -931,3 +943,6 @@ override define INSTALL_RULE
> rsync -a $(OUTPUT)/$$DIR/*.bpf.o $(INSTALL_PATH)/$$DIR;\
> done
> endef
> +
> +test_libarena: $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ)
> + +$(MAKE) -C libarena $@ $(LIBARENA_MAKE_ARGS)
> diff --git a/tools/testing/selftests/bpf/libarena/Makefile b/tools/testing/selftests/bpf/libarena/Makefile
> new file mode 100644
> index 000000000000..151ab6d0509c
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/libarena/Makefile
> @@ -0,0 +1,50 @@
> +# SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
> +# Copyright (c) 2026 Meta Platforms, Inc. and affiliates.
> +
> +.PHONY: clean
> +
> +LIBARENA=$(abspath .)
> +
> +
> +LIBARENA_SOURCES = $(wildcard $(LIBARENA)/src/*.bpf.c) $(wildcard $(LIBARENA)/selftests/*.bpf.c)
> +LIBARENA_OBJECTS = $(notdir $(LIBARENA_SOURCES:.bpf.c=.bpf.o))
> +
> +INCLUDES = -I$(LIBARENA)/include -I$(LIBARENA)/..
> +ifneq ($(INCLUDE_DIR),)
> +INCLUDES += -I$(INCLUDE_DIR)
> +endif
> +ifneq ($(LIBBPF_INCLUDE),)
> +INCLUDES += -I$(LIBBPF_INCLUDE)
> +endif
> +
> +# ENABLE_ATOMICS_TESTS required because we use arena spinlocks
> +override BPF_CFLAGS += -DENABLE_ATOMICS_TESTS
> +override BPF_CFLAGS += -O2 -Wno-incompatible-pointer-types-discards-qualifiers
> +override BPF_CFLAGS += $(INCLUDES)
> +
> +CFLAGS = -O2 -no-pie
> +CFLAGS += $(INCLUDES)
> +
> +vpath %.bpf.c $(LIBARENA)/src $(LIBARENA)/selftests
> +vpath %.c $(LIBARENA)/src $(LIBARENA)/selftests
> +
> +all: test_libarena
> +
> +test_libarena: selftest.c $(BPFOBJ) libarena.skel.h
> + $(call msg,BINARY,libarena,$@)
> + $(Q)$(CLANG) $(CFLAGS) $< $(BPFOBJ) $(LDLIBS) -o $@
> +
> +libarena.skel.h: main.bpf.o
> + $(call msg,GEN-SKEL,libarena,$@)
> + $(Q)$(BPFTOOL) gen skeleton $< name "libarena" > $@
> +
> +main.bpf.o: $(LIBARENA_OBJECTS)
> + $(call msg,GEN-OBJ,libarena,$@)
> + $(Q)$(BPFTOOL) gen object $@ $^
> +
> +%.bpf.o: %.bpf.c
> + $(call msg,CLNG-BPF,libarena,$@)
> + $(Q)$(CLANG) $(BPF_CFLAGS) $(BPF_TARGET_ENDIAN) -c $< -o $@
> +
> +clean:
> + $(Q)rm -f *.skel.h *.bpf.o test_libarena
> diff --git a/tools/testing/selftests/bpf/libarena/include/common.h b/tools/testing/selftests/bpf/libarena/include/common.h
> new file mode 100644
> index 000000000000..544a398a0d1e
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/libarena/include/common.h
> @@ -0,0 +1,49 @@
> +// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
> +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
> +#pragma once
> +
> +#ifdef __BPF__
Are you trying to re-use this common.h in user space code?
> +
> +#include <vmlinux.h>
> +
> +#include "bpf_experimental.h"
> +#include "bpf_arena_common.h"
> +#include "bpf_arena_spin_lock.h"
> +
> +#include <asm-generic/errno.h>
> +
> +#ifndef __BPF_FEATURE_ADDR_SPACE_CAST
> +#error "Arena allocators require bpf_addr_space_cast feature"
> +#endif
> +
> +#define arena_stdout(fmt, ...) bpf_stream_printk(1, (fmt), ##__VA_ARGS__)
> +#define arena_stderr(fmt, ...) bpf_stream_printk(2, (fmt), ##__VA_ARGS__)
> +
> +#ifndef __maybe_unused
> +#define __maybe_unused __attribute__((__unused__))
> +#endif
> +
> +#define private(name) SEC(".data." #name) __hidden __attribute__((aligned(8)))
> +
> +#define ARENA_PAGES (1UL << (32 - __builtin_ffs(__PAGE_SIZE) + 1))
> +
> +struct {
> + __uint(type, BPF_MAP_TYPE_ARENA);
> + __uint(map_flags, BPF_F_MMAPABLE);
> + __uint(max_entries, ARENA_PAGES); /* number of pages */
> +#if defined(__TARGET_ARCH_arm64) || defined(__aarch64__)
> + __ulong(map_extra, (1ull << 32)); /* start of mmap() region */
> +#else
> + __ulong(map_extra, (1ull << 44)); /* start of mmap() region */
> +#endif
> +} arena __weak SEC(".maps");
> +
> +extern const volatile u32 zero;
> +
> +int arena_fls(__u64 word);
> +
> +#endif /* __BPF__ */
> +
> +struct arena_get_base_args {
> + void __arena *arena_base;
> +};
If so, then how above suppose to compile in user space?
__arena is not defined yet.
> diff --git a/tools/testing/selftests/bpf/libarena/include/selftest_helpers.h b/tools/testing/selftests/bpf/libarena/include/selftest_helpers.h
> new file mode 100644
> index 000000000000..ee445d8f5b26
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/libarena/include/selftest_helpers.h
> @@ -0,0 +1,81 @@
> +// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
> +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
> +#pragma once
> +
> +#include <errno.h>
> +#include <stdlib.h>
> +#include <unistd.h>
> +#include <sys/mman.h>
> +
> +#include <bpf/libbpf.h>
> +#include <bpf/bpf.h>
> +
> +static inline int libarena_run_prog(int prog_fd)
> +{
> + LIBBPF_OPTS(bpf_test_run_opts, opts);
> + int ret;
> +
> + ret = bpf_prog_test_run_opts(prog_fd, &opts);
> + if (ret)
> + return ret;
> +
> + return opts.retval;
> +}
> +
> +static inline int libarena_get_arena_base(int arena_get_base_fd,
> + void **arena_base)
> +{
> + LIBBPF_OPTS(bpf_test_run_opts, opts);
> + struct arena_get_base_args args = { .arena_base = NULL };
> + int ret;
> +
> + opts.ctx_in = &args;
> + opts.ctx_size_in = sizeof(args);
> +
> + ret = bpf_prog_test_run_opts(arena_get_base_fd, &opts);
> + if (ret)
> + return ret;
> + if (opts.retval)
> + return opts.retval;
> +
> + *arena_base = args.arena_base;
> + return 0;
> +}
> +
> +static inline int libarena_get_globals_pages(int arena_get_base_fd,
> + size_t arena_all_pages,
> + u64 *globals_pages)
> +{
> + size_t pgsize = sysconf(_SC_PAGESIZE);
> + void *arena_base;
> + ssize_t i;
> + u8 *vec;
> + int ret;
> +
> + ret = libarena_get_arena_base(arena_get_base_fd, &arena_base);
> + if (ret)
> + return ret;
> +
> + if (!arena_base)
> + return -EINVAL;
> +
> + vec = calloc(arena_all_pages, sizeof(*vec));
> + if (!vec)
> + return -ENOMEM;
> +
> + if (mincore(arena_base, arena_all_pages * pgsize, vec) < 0) {
> + ret = -errno;
> + free(vec);
> + return ret;
> + }
> +
> + *globals_pages = 0;
> + for (i = arena_all_pages - 1; i >= 0; i--) {
> + if (!(vec[i] & 0x1))
> + break;
> + *globals_pages += 1;
> + }
> +
> + free(vec);
> + return 0;
> +}
> diff --git a/tools/testing/selftests/bpf/libarena/include/userapi.h b/tools/testing/selftests/bpf/libarena/include/userapi.h
> new file mode 100644
> index 000000000000..b4ac6bc9bd79
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/libarena/include/userapi.h
> @@ -0,0 +1,26 @@
> +// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
> +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
> +#pragma once
> +
> +#include <stdint.h>
> +
> +/*
> + * Header for the userspace C programs that load
> + * and initialize the BPF code.
> + */
> +
> +#define __arena
> +
> +typedef uint8_t u8;
> +typedef uint16_t u16;
> +typedef uint32_t u32;
> +typedef uint64_t u64;
> +typedef int8_t s8;
> +typedef int16_t s16;
> +typedef int32_t s32;
> +typedef int64_t s64;
> +
> +/* Dummy "definition" for userspace. */
> +#define arena_spinlock_t u64
> +
> +#include "common.h"
ohh. So user space _must_ include userapi.h first ?
Why is this not part of common.h ?
Why bother with extra userapi.h header?
> diff --git a/tools/testing/selftests/bpf/libarena/selftests/selftest.c b/tools/testing/selftests/bpf/libarena/selftests/selftest.c
> new file mode 100644
> index 000000000000..b69adc7baaab
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/libarena/selftests/selftest.c
> @@ -0,0 +1,153 @@
> +// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
> +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
> +
> +#define _GNU_SOURCE
> +#include <assert.h>
> +#include <errno.h>
> +#include <stdio.h>
> +#include <string.h>
> +#include <unistd.h>
> +
> +#include <bpf/libbpf.h>
> +#include <bpf/bpf.h>
> +
> +#include <sys/mman.h>
> +#include <sys/resource.h>
> +#include <sys/sysinfo.h>
> +
> +#include <userapi.h>
> +#include <selftest_helpers.h>
> +
> +#include "../libarena.skel.h"
> +typedef struct libarena selftest;
> +#define selftest__open libarena__open
> +#define selftest__open_and_load libarena__open_and_load
> +#define selftest__load libarena__load
> +#define selftest__attach libarena__attach
> +#define selftest__destroy libarena__destroy
I don't like this obfuscation.
What's the point of these #define-s?
> +
> +static bool verbose = false;
> +static int testno = 1;
> +
> +static int
> +run_prog_verbose(int prog_fd)
> +{
> + char buf[1024];
> + int ret, err;
> +
> + ret = libarena_run_prog(prog_fd);
> +
> + if (ret)
> + fprintf(stderr, "error %d in %s\n", ret, __func__);
> +
> + if (verbose) {
> + printf("BPF stdout:\n");
> + while ((err = bpf_prog_stream_read(prog_fd, 1, buf, 1024, NULL)) > 0)
> + printf("%.*s", err, buf);
> +
> + if (err)
> + return err;
> +
> + printf("BPF stderr:\n");
> + while ((err = bpf_prog_stream_read(prog_fd, 2, buf, 1024, NULL)) > 0)
> + printf("%.*s", err, buf);
> +
> + if (err)
> + return err;
> + }
> +
> + return ret;
> +}
> +
> +static int libbpf_print_fn(enum libbpf_print_level level,
> + const char *format, va_list args)
> +{
> + if (level == LIBBPF_DEBUG)
> + return 0;
> + return vfprintf(stderr, format, args);
> +}
> +
> +int run_test(selftest *skel, const struct bpf_program *prog)
> +{
> + int prog_fd;
> + int ret;
> +
> + ret = libarena_run_prog(bpf_program__fd(skel->progs.arena_alloc_reserve));
> + if (ret)
> + return ret;
> +
> + prog_fd = bpf_program__fd(prog);
> + if (prog_fd < 0)
> + return prog_fd;
> +
> + return run_prog_verbose(prog_fd);
> +}
> +
> +#define TEST(__test) \
> +int run_##__test(void) \
> +{ \
> + selftest *skel; \
> + int ret; \
> + \
> + skel = selftest__open_and_load(); \
> + if (!skel) { \
> + ret = -EINVAL; \
> + goto error_no_destroy; \
> + } \
> + \
> + ret = selftest__attach(skel); \
> + if (ret) \
> + goto error; \
> + \
> + ret = run_test(skel, skel->progs.__test); \
> + if (ret) \
> + goto error; \
> + \
> + selftest__destroy(skel); \
> + \
> + printf("ok %d - %s\n", testno++, #__test); \
> + return 0; \
> + \
> +error: \
> + selftest__destroy(skel); \
> +error_no_destroy: \
> + printf("not ok %d - %s\n", testno++, #__test); \
> + return ret; \
> +}
> +
> +static void
> +banner(const char *progpath)
> +{
> + char *name = basename(progpath);
> +
> + printf("%s\n", name);
> +
> + printf("=== %s ===\n", "libarena selftests");
> +}
> +
> +int main(int argc, char *argv[])
> +{
> + int ret;
> +
> + struct rlimit rlim = {
> + .rlim_cur = RLIM_INFINITY,
> + .rlim_max = RLIM_INFINITY,
> + };
> +
> + banner(argv[0]);
> +
> + for (int i = 1; i < argc; i++) {
> + if (strcmp(argv[i], "-v") == 0 || strcmp(argv[i], "--verbose") == 0)
> + verbose = true;
> + }
> +
> + ret = setrlimit(RLIMIT_MEMLOCK, &rlim);
where did you copy this from?
It wasn't necessary for very long time.
> + if (ret) {
> + perror("setrlimit");
> + return ret;
> + }
> +
> + libbpf_set_print(libbpf_print_fn);
> +
> + return 0;
> +}
> diff --git a/tools/testing/selftests/bpf/libarena/selftests/selftest.h b/tools/testing/selftests/bpf/libarena/selftests/selftest.h
> new file mode 100644
> index 000000000000..b1cbff4d343b
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/libarena/selftests/selftest.h
> @@ -0,0 +1,12 @@
> +// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
> +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
> +#pragma once
> +
> +#define ALLOC_SELFTEST(func, ...) \
> + do { \
> + int ret = func(__VA_ARGS__); \
> + if (ret) { \
> + arena_stderr("SELFTEST %s FAIL: %d", #func, ret); \
> + return ret; \
> + } \
> + } while (0)
I'm not excited about yet another special macro that hides control flow.
Pls stick to ASSERT*() family.
pw-bot: cr
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH bpf-next v4 6/9] selftests/bpf: Add arena ASAN runtime to libarena
2026-04-07 4:57 [PATCH bpf-next v4 0/9] Introduce arena library and runtime Emil Tsalapatis
` (4 preceding siblings ...)
2026-04-07 4:57 ` [PATCH bpf-next v4 5/9] selftests/bpf: Add basic libarena scaffolding Emil Tsalapatis
@ 2026-04-07 4:57 ` Emil Tsalapatis
2026-04-07 16:39 ` Alexei Starovoitov
2026-04-07 4:57 ` [PATCH bpf-next v4 7/9] selftests/bpf: Add ASAN support for libarena selftests Emil Tsalapatis
` (2 subsequent siblings)
8 siblings, 1 reply; 19+ messages in thread
From: Emil Tsalapatis @ 2026-04-07 4:57 UTC (permalink / raw)
To: bpf; +Cc: ast, andrii, memxor, daniel, eddyz87, song, Emil Tsalapatis
Add an address sanitizer (ASAN) runtime to the arena library. The
ASAN runtime implements the functions injected into BPF binaries
by LLVM sanitization when ASAN is enabled during compilation.
The runtime also includes functions called explicitly by memory
allocation code to mark memory as poisoned/unpoisoned to ASAN.
This code is a no-op when sanitization is turned off.
Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
---
.../selftests/bpf/libarena/include/asan.h | 124 ++++
.../selftests/bpf/libarena/include/common.h | 1 +
.../selftests/bpf/libarena/src/asan.bpf.c | 539 ++++++++++++++++++
.../selftests/bpf/libarena/src/common.bpf.c | 1 +
4 files changed, 665 insertions(+)
create mode 100644 tools/testing/selftests/bpf/libarena/include/asan.h
create mode 100644 tools/testing/selftests/bpf/libarena/src/asan.bpf.c
diff --git a/tools/testing/selftests/bpf/libarena/include/asan.h b/tools/testing/selftests/bpf/libarena/include/asan.h
new file mode 100644
index 000000000000..d2eeabba3ffc
--- /dev/null
+++ b/tools/testing/selftests/bpf/libarena/include/asan.h
@@ -0,0 +1,124 @@
+// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#pragma once
+
+struct asan_init_args {
+ u64 arena_all_pages;
+ u64 arena_globals_pages;
+};
+
+int asan_init(struct asan_init_args *args);
+
+/* Parameters usable by userspace. */
+extern volatile u64 __asan_shadow_memory_dynamic_address;
+extern volatile u32 asan_reported;
+extern volatile bool asan_inited;
+extern volatile bool asan_report_once;
+extern volatile bool asan_emit_stack;
+
+#ifdef __BPF__
+
+#define ASAN_SHADOW_SHIFT 3
+#define ASAN_SHADOW_SCALE (1ULL << ASAN_SHADOW_SHIFT)
+#define ASAN_GRANULE_MASK ((1ULL << ASAN_SHADOW_SHIFT) - 1)
+#define ASAN_GRANULE(addr) ((s8)((u32)(u64)((addr)) & ASAN_GRANULE_MASK))
+
+#define __noasan __attribute__((no_sanitize("address")))
+
+#ifdef BPF_ARENA_ASAN
+
+/*
+ * Defined as char * to get 1-byte granularity for pointer arithmetic.
+ */
+typedef s8 __arena s8a;
+
+/*
+ * Address to shadow map translation.
+ */
+static inline
+s8a *mem_to_shadow(void __arena __arg_arena *addr)
+{
+ return (s8a *)(((u32)(u64)addr >> ASAN_SHADOW_SHIFT) + __asan_shadow_memory_dynamic_address);
+}
+
+/*
+ * Helper for directly reading the shadow map.
+ */
+static inline __noasan
+s8 asan_shadow_value(void __arena __arg_arena *addr)
+{
+ return *(s8a *)mem_to_shadow(addr);
+}
+
+__weak __noasan
+bool asan_ready(void)
+{
+ return __asan_shadow_memory_dynamic_address;
+}
+
+/*
+ * Shadow map manipulation helpers.
+ */
+int asan_poison(void __arena *addr, s8 val, size_t size);
+int asan_unpoison(void __arena *addr, size_t size);
+bool asan_shadow_set(void __arena *addr);
+
+/*
+ * Dummy calls to ensure the ASAN runtime's BTF information is present
+ * in every object file when compiling the runtime and local BPF code
+ * separately. The runtime calls are injected into the LLVM IR file
+ */
+#define DECLARE_ASAN_LOAD_STORE_SIZE(size) \
+ void __asan_store##size(void *addr); \
+ void __asan_store##size##_noabort(void *addr); \
+ void __asan_load##size(void *addr); \
+ void __asan_load##size##_noabort(void *addr); \
+ void __asan_report_store##size(void *addr); \
+ void __asan_report_store##size##_noabort(void *addr); \
+ void __asan_report_load##size(void *addr); \
+ void __asan_report_load##size##_noabort(void *addr);
+
+DECLARE_ASAN_LOAD_STORE_SIZE(1);
+DECLARE_ASAN_LOAD_STORE_SIZE(2);
+DECLARE_ASAN_LOAD_STORE_SIZE(4);
+DECLARE_ASAN_LOAD_STORE_SIZE(8);
+
+#define ASAN_DUMMY_CALLS_SIZE(size, arg) \
+do { \
+ __asan_store##size((arg)); \
+ __asan_store##size##_noabort((arg)); \
+ __asan_load##size((arg)); \
+ __asan_load##size##_noabort((arg)); \
+ __asan_report_store##size((arg)); \
+ __asan_report_store##size##_noabort((arg)); \
+ __asan_report_load##size((arg)); \
+ __asan_report_load##size##_noabort((arg)); \
+} while (0)
+
+#define ASAN_DUMMY_CALLS_ALL(arg) \
+do { \
+ ASAN_DUMMY_CALLS_SIZE(1, (arg)); \
+ ASAN_DUMMY_CALLS_SIZE(2, (arg)); \
+ ASAN_DUMMY_CALLS_SIZE(4, (arg)); \
+ ASAN_DUMMY_CALLS_SIZE(8, (arg)); \
+} while (0)
+
+__weak __noasan
+int asan_dummy_call(void) {
+ /* Use the shadow map base to prevent it from being optimized out. */
+ if (__asan_shadow_memory_dynamic_address)
+ ASAN_DUMMY_CALLS_ALL(NULL);
+
+ return 0;
+}
+#else /* BPF_ARENA_ASAN */
+
+static inline int asan_poison(void __arena *addr, s8 val, size_t size) { return 0; }
+static inline int asan_unpoison(void __arena *addr, size_t size) { return 0; }
+static inline bool asan_shadow_set(void __arena *addr) { return 0; }
+static inline s8 asan_shadow_value(void __arena *addr) { return 0; }
+__weak bool asan_ready(void) { return true; }
+
+#endif /* BPF_ARENA_ASAN */
+
+#endif /* __BPF__ */
diff --git a/tools/testing/selftests/bpf/libarena/include/common.h b/tools/testing/selftests/bpf/libarena/include/common.h
index 544a398a0d1e..f48395358d1d 100644
--- a/tools/testing/selftests/bpf/libarena/include/common.h
+++ b/tools/testing/selftests/bpf/libarena/include/common.h
@@ -39,6 +39,7 @@ struct {
} arena __weak SEC(".maps");
extern const volatile u32 zero;
+extern volatile u64 asan_violated;
int arena_fls(__u64 word);
diff --git a/tools/testing/selftests/bpf/libarena/src/asan.bpf.c b/tools/testing/selftests/bpf/libarena/src/asan.bpf.c
new file mode 100644
index 000000000000..b3fae0ed020c
--- /dev/null
+++ b/tools/testing/selftests/bpf/libarena/src/asan.bpf.c
@@ -0,0 +1,539 @@
+// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <common.h>
+#include <asan.h>
+
+/*
+ * Address sanitizer (ASAN) for arena-based BPF programs. The
+ * sanitizer tracks valid arena memory and triggers an error
+ * when poisoned memory is read or written to. Starting point
+ * was the KASAN implementation in mm/.
+ *
+ * The API
+ * -------
+ *
+ * The implementation includes two kinds of components: Implementation
+ * of ASAN hooks injected by LLVM into the program, and API calls that
+ * allocators use to mark memory as valid or invalid. The full list is:
+ *
+ * LLVM stubs:
+ *
+ * void __asan_{load, store}<size>(void *addr)
+ * Checks whether an access is valid. All variations covered
+ * by check_region_inline().
+ *
+ * void __asan_{store, load}((void *addr, ssize_t size)
+ *
+ * void __asan_report_{load, store}<size>(void *addr)
+ * Report an access violation for the program. Used when LLVM
+ * uses direct code generation for shadow map checks.
+ *
+ * void *__asan_memcpy(void *d, const void *s, size_t n)
+ * void *__asan_memmove(void *d, const void *s, size_t n)
+ * void *__asan_memset(void *p, int c, size_t n)
+ * Hooks for ASAN instrumentation of the LLVM mem* builtins.
+ * Currently unimplemented just like the builtins themselves.
+ *
+ * API methods:
+ *
+ * asan_init()
+ * Initialize the ASAN map for the arena.
+ *
+ * asan_poison()
+ * Mark a region of memory as poisoned. Accessing poisoned memory
+ * causes asan_report() to fire. Invoked during free().
+ *
+ * asan_unpoison()
+ * Mark a region as unpoisoned after alloc().
+ *
+ * asan_shadow_set()
+ * Check a byte's validity directly.
+ *
+ * The Algorithm In Brief
+ * ----------------------
+ * Each group of 8 bytes is mapped to a "granule" in the shadow map. This
+ * granule is the size of the byte and describes which bytes are valid.
+ * Possible values are:
+ *
+ * 0: All bytes are valid. Makes checks in the middle of an allocated region
+ * (most of them) fast.
+ * (0, 7]: How many consecutive bytes are valid, starting from the lowest one.
+ * The tradeoff is that we can't poison individual bytes in the middle of a
+ * valid region.
+ * [0x80, 0xff]: Special poison values, can be used to denote specific error
+ * modes (e.g., recently freed vs uninitialized memory).
+ *
+ * The mapping between a memory location and its shadow is:
+ * shadow_addr = shadow_base + (addr >> 3). We retain the 8:1 data:shadow
+ * ratio of existing ASAN implementations as a compromise between tracking
+ * granularity and space usage/scan overhead.
+ */
+
+#ifdef BPF_ARENA_ASAN
+
+#pragma clang attribute push(__attribute__((no_sanitize("address"))), \
+ apply_to = function)
+
+#define SHADOW_ALL_ZEROES ((u64)-1)
+
+/*
+ * Canary variable for ASAN violations. Set to the offending address.
+ */
+volatile u64 asan_violated = 0;
+
+/*
+ * Shadow map occupancy map.
+ */
+volatile u64 __asan_shadow_memory_dynamic_address;
+
+volatile u32 asan_reported = false;
+volatile bool asan_inited = false;
+
+/*
+ * Set during program load.
+ */
+volatile bool asan_report_once = false;
+volatile bool asan_emit_stack = false;
+
+/*
+ * BPF does not currently support the memset/memcpy/memcmp intrinsics.
+ */
+__weak int asan_memset(s8a __arg_arena *dst, s8 val, size_t size)
+{
+ size_t i;
+
+ for (i = zero; i < size && can_loop; i++)
+ dst[i] = val;
+
+ return 0;
+}
+
+/* Validate a 1-byte access, always within a single byte. */
+static __always_inline bool memory_is_poisoned_1(s8a *addr)
+{
+ s8 shadow_value = asan_shadow_value(addr);
+
+ /* Byte is 0, access is valid. */
+ if (likely(!shadow_value))
+ return false;
+
+ /*
+ * Byte is non-zero. Access is valid if granule offset in [0, shadow_value),
+ * so the memory is poisoned if shadow_value is negative or smaller than
+ * the granule's value.
+ */
+
+ return ASAN_GRANULE(addr) >= shadow_value;
+}
+
+/* Validate a 2- 4-, 8-byte access, shadow spans up to 2 bytes. */
+static __always_inline bool memory_is_poisoned_2_4_8(s8a *addr, u64 size)
+{
+ u64 end = (u64)addr + size - 1;
+
+ /*
+ * Region fully within a single byte (addition didn't
+ * overflow above ASAN_GRANULE).
+ */
+ if (likely(ASAN_GRANULE(end) >= size - 1))
+ return memory_is_poisoned_1((s8a *)end);
+
+ /*
+ * Otherwise first byte must be fully unpoisoned, and second byte
+ * must be unpoisoned up to the end of the accessed region.
+ */
+
+ return asan_shadow_value(addr) || memory_is_poisoned_1((s8a *)end);
+}
+
+/*
+ * Explicit ASAN check.
+ */
+__weak bool asan_shadow_set(void __arena __arg_arena *addr)
+{
+ return memory_is_poisoned_1(addr);
+}
+
+static __always_inline u64 first_nonzero_byte(u64 addr, size_t size)
+{
+ while (size && can_loop) {
+ if (unlikely(*(s8a *)addr))
+ return addr;
+ addr += 1;
+ size -= 1;
+ }
+
+ return SHADOW_ALL_ZEROES;
+}
+
+static __always_inline bool memory_is_poisoned_n(s8a *addr, u64 size)
+{
+ u64 ret;
+ u64 start;
+ u64 end;
+
+ /* Size of [start, end] is end - start + 1. */
+ start = (u64)mem_to_shadow(addr);
+ end = (u64)mem_to_shadow(addr + size - 1);
+
+ ret = first_nonzero_byte(start, (end - start) + 1);
+ if (likely(ret == SHADOW_ALL_ZEROES))
+ return false;
+
+ return __builtin_expect(ret != end || ASAN_GRANULE(addr + size - 1) >=
+ *(s8a *)end, false);
+}
+
+__weak int asan_report(s8a __arg_arena *addr, size_t sz,
+ bool write)
+{
+ u32 reported = __sync_val_compare_and_swap(&asan_reported, false, true);
+
+ /* Only report the first ASAN violation. */
+ if (reported && asan_report_once)
+ return 0;
+
+ asan_violated = (u64)addr;
+
+ if (asan_emit_stack) {
+ arena_stderr("Memory violation for address %p (0x%lx) for %s of size %ld",
+ addr, (u64)addr, write ? "write" : "read", sz);
+ bpf_stream_print_stack(BPF_STDERR);
+ }
+
+ return 0;
+}
+
+static __always_inline bool check_asan_args(s8a *addr, size_t size,
+ bool *result)
+{
+ bool valid = true;
+
+ /* Size 0 accesses are valid even if the address is invalid. */
+ if (unlikely(size == 0))
+ goto confirmed_valid;
+
+ /*
+ * Wraparound is possible for values close to the the edge of the
+ * 4GiB boundary of the arena (last valid address is 1UL << 32 - 1).
+ *
+ *
+ * The wraparound detection below works for small sizes.check_asan_args is
+ * always called from the builtin ASAN checks, so 1 <= size <= 64. We do
+ * not implement storeN/loadN, so size is guaranteed to be in that range.
+ * Even if we did, sane storeN/loadN intrinsics are not expected to have
+ * a large enough size that
+ *
+ * - addr + size > MAX_U32
+ * - (u32)(addr + size) > (u32) addr
+ *
+ * which would defeat wraparound detection.
+ */
+ if (unlikely((u32)(u64)(addr + size) < (u32)(u64)addr))
+ goto confirmed_invalid;
+
+ return false;
+
+confirmed_invalid:
+ valid = false;
+
+ /* FALLTHROUGH */
+confirmed_valid:
+ *result = valid;
+
+ return true;
+}
+
+static __always_inline bool check_region_inline(void *ptr, size_t size,
+ bool write)
+{
+ s8a *addr = (s8a *)(u64)ptr;
+ bool is_poisoned, is_valid;
+
+ if (check_asan_args(addr, size, &is_valid)) {
+ if (!is_valid)
+ asan_report(addr, size, write);
+ return is_valid;
+ }
+
+ switch (size) {
+ case 1:
+ is_poisoned = memory_is_poisoned_1(addr);
+ break;
+ case 2:
+ case 4:
+ case 8:
+ is_poisoned = memory_is_poisoned_2_4_8(addr, size);
+ break;
+ default:
+ is_poisoned = memory_is_poisoned_n(addr, size);
+ }
+
+ if (is_poisoned) {
+ asan_report(addr, size, write);
+ return false;
+ }
+
+ return true;
+}
+
+/*
+ * __alias is not supported for BPF so define *__noabort() variants as wrappers.
+ */
+#define DEFINE_ASAN_LOAD_STORE(size) \
+ __hidden void __asan_store##size(void *addr) \
+ { \
+ check_region_inline(addr, size, true); \
+ } \
+ __hidden void __asan_store##size##_noabort(void *addr) \
+ { \
+ check_region_inline(addr, size, true); \
+ } \
+ __hidden void __asan_load##size(void *addr) \
+ { \
+ check_region_inline(addr, size, false); \
+ } \
+ __hidden void __asan_load##size##_noabort(void *addr) \
+ { \
+ check_region_inline(addr, size, false); \
+ } \
+ __hidden void __asan_report_store##size(void *addr) \
+ { \
+ asan_report((s8a *)addr, size, true); \
+ } \
+ __hidden void __asan_report_store##size##_noabort(void *addr) \
+ { \
+ asan_report((s8a *)addr, size, true); \
+ } \
+ __hidden void __asan_report_load##size(void *addr) \
+ { \
+ asan_report((s8a *)addr, size, false); \
+ } \
+ __hidden void __asan_report_load##size##_noabort(void *addr) \
+ { \
+ asan_report((s8a *)addr, size, false); \
+ }
+
+DEFINE_ASAN_LOAD_STORE(1);
+DEFINE_ASAN_LOAD_STORE(2);
+DEFINE_ASAN_LOAD_STORE(4);
+DEFINE_ASAN_LOAD_STORE(8);
+
+void __asan_storeN(void *addr, ssize_t size)
+{
+ check_region_inline(addr, size, true);
+}
+
+void __asan_loadN(void *addr, ssize_t size)
+{
+ check_region_inline(addr, size, false);
+}
+
+/*
+ * We currently do not sanitize globals.
+ */
+void __asan_register_globals(void *globals, size_t n)
+{
+}
+
+void __asan_unregister_globals(void *globals, size_t n)
+{
+}
+
+/*
+ * We do not currently have memcpy/memmove/memset intrinsics
+ * in LLVM. Do not implement sanitization.
+ */
+void *__asan_memcpy(void *d, const void *s, size_t n)
+{
+ arena_stderr("ASAN: Unexpected %s call", __func__);
+ return NULL;
+}
+
+void *__asan_memmove(void *d, const void *s, size_t n)
+{
+ arena_stderr("ASAN: Unexpected %s call", __func__);
+ return NULL;
+}
+
+void *__asan_memset(void *p, int c, size_t n)
+{
+ arena_stderr("ASAN: Unexpected %s call", __func__);
+ return NULL;
+}
+
+/*
+ * Poisoning code, used when we add more freed memory to the allocator by:
+ * a) pulling memory from the arena segment using bpf_arena_alloc_pages()
+ * b) freeing memory from application code
+ */
+__hidden __noasan int asan_poison(void __arena *addr, s8 val, size_t size)
+{
+ s8a *shadow;
+ size_t len;
+
+ /*
+ * Poisoning from a non-granule address makes no sense: We can only allocate
+ * memory to the application that has a granule-aligned starting address,
+ * and bpf_arena_alloc_pages returns page-aligned memory. A non-aligned
+ * addr then implies we're freeing a different address than the one we
+ * allocated.
+ */
+ if (unlikely((u64)addr & ASAN_GRANULE_MASK))
+ return -EINVAL;
+
+ /*
+ * We cannot free an unaligned region because it'd be possible that we
+ * cannot describe the resulting poisoning state of the granule in
+ * the ASAN encoding.
+ *
+ * Every granule represents a region of memory that looks like the
+ * following (P for poisoned bytes, C for clear):
+ *
+ * <Clear> <Poisoned>
+ * [ C C C ... P P ]
+ *
+ * The value of the granule's shadow map is the number of clear bytes in
+ * it. We cannot represent granules with the following state:
+ *
+ * [ P P ... C C ... P P ]
+ *
+ * That would be possible if we could free unaligned regions, so prevent that.
+ */
+ if (unlikely(size & ASAN_GRANULE_MASK))
+ return -EINVAL;
+
+ shadow = mem_to_shadow(addr);
+ len = size >> ASAN_SHADOW_SHIFT;
+
+ asan_memset(shadow, val, len);
+
+ return 0;
+}
+
+/*
+ * Unpoisoning code for marking memory as valid during allocation calls.
+ *
+ * Very similar to asan_poison, except we need to round up instead of
+ * down, then partially poison the last granule if necessary.
+ *
+ * Partial poisoning is useful for keeping the padding poisoned. Allocations
+ * are granule-aligned, so we we're reserving granule-aligned sizes for the
+ * allocation. However, we want to still treat accesses to the padding as
+ * invalid. Partial poisoning takes care of that. Freeing and poisoning the
+ * memory is still done in granule-aligned sizes and repoisons the already
+ * poisoned padding.
+ */
+__hidden __noasan int asan_unpoison(void __arena *addr, size_t size)
+{
+ size_t partial = size & ASAN_GRANULE_MASK;
+ s8a *shadow;
+ size_t len;
+
+ /*
+ * We cannot allocate in the middle of the granule. The ASAN shadow
+ * map encoding only describes regions of memory where every granule
+ * follows this format (P for poisoned, C for clear):
+ *
+ * <Clear> <Poisoned>
+ * [ C C C ... P P ]
+ *
+ * This is so we can use a single number in [0, ASAN_SHADOW_SCALE)
+ * to represent the poison state of the granule.
+ */
+ if (unlikely((u64)addr & ASAN_GRANULE_MASK))
+ return -EINVAL;
+
+ shadow = mem_to_shadow(addr);
+ len = size >> ASAN_SHADOW_SHIFT;
+
+ asan_memset(shadow, 0, len);
+
+ /*
+ * If we are allocating a non-granule aligned region, we need to adjust
+ * the last byte of the shadow map to list how many bytes in the granule
+ * are unpoisoned. If the region is aligned, then the memset call above
+ * was enough.
+ */
+ if (partial)
+ shadow[len] = partial;
+
+ return 0;
+}
+
+/*
+ * Initialize ASAN state when necessary. Triggered from userspace before
+ * allocator startup.
+ */
+SEC("syscall")
+__hidden __noasan int asan_init(struct asan_init_args *args)
+{
+ u64 globals_pages = args->arena_globals_pages;
+ u64 all_pages = args->arena_all_pages;
+ u64 shadowmap, shadow_pgoff;
+ u64 shadow_pages;
+
+ if (asan_inited)
+ return 0;
+
+ /*
+ * Round up the shadow map size to the nearest page.
+ */
+ shadow_pages = all_pages >> ASAN_SHADOW_SHIFT;
+ if ((all_pages & ((1 << ASAN_SHADOW_SHIFT) -1 )))
+ shadow_pages += 1;
+
+ /*
+ * Make sure the numbers provided by userspace are sane.
+ */
+ if (all_pages > (1ULL << 32) / __PAGE_SIZE) {
+ arena_stderr("error: arena size %lx too large", all_pages);
+ return -EINVAL;
+ }
+
+ if (globals_pages > all_pages) {
+ arena_stderr("error: globals %lx do not fit in arena %lx", globals_pages, all_pages);
+ return -EINVAL;
+ }
+
+ if (globals_pages + shadow_pages > all_pages) {
+ arena_stderr("error: globals %lx do not leave room for shadow map %lx (arena pages %lx)",
+ globals_pages, shadow_pages, all_pages);
+ return -EINVAL;
+ }
+
+ shadow_pgoff = all_pages - shadow_pages - globals_pages;
+ __asan_shadow_memory_dynamic_address = shadow_pgoff * __PAGE_SIZE;
+
+ /*
+ * Allocate the last (1/ASAN_SHADOW_SCALE)th of an arena's pages for the map
+ * We find the offset and size from the arena map.
+ *
+ * The allocated map pages are zeroed out, meaning all memory is marked as valid
+ * even if it's not allocated already. This is expected: Since the actual memory
+ * pages are not allocated, accesses to it will trigger page faults and will be
+ * reported through BPF streams. Any pages allocated through bpf_arena_alloc_pages
+ * should be poisoned by the allocator right after the call succeeds.
+ */
+ shadowmap = (u64)bpf_arena_alloc_pages(
+ &arena, (void __arena *)__asan_shadow_memory_dynamic_address,
+ shadow_pages, NUMA_NO_NODE, 0);
+ if (!shadowmap) {
+ arena_stderr("Could not allocate shadow map\n");
+
+ __asan_shadow_memory_dynamic_address = 0;
+
+ return -ENOMEM;
+ }
+
+ asan_inited = true;
+
+ return 0;
+}
+
+#pragma clang attribute pop
+
+#endif /* BPF_ARENA_ASAN */
+
+__weak char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/libarena/src/common.bpf.c b/tools/testing/selftests/bpf/libarena/src/common.bpf.c
index cbb729290d3c..31acb259ec5a 100644
--- a/tools/testing/selftests/bpf/libarena/src/common.bpf.c
+++ b/tools/testing/selftests/bpf/libarena/src/common.bpf.c
@@ -1,6 +1,7 @@
// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
#include <common.h>
+#include <asan.h>
const volatile u32 zero = 0;
--
2.53.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* Re: [PATCH bpf-next v4 6/9] selftests/bpf: Add arena ASAN runtime to libarena
2026-04-07 4:57 ` [PATCH bpf-next v4 6/9] selftests/bpf: Add arena ASAN runtime to libarena Emil Tsalapatis
@ 2026-04-07 16:39 ` Alexei Starovoitov
0 siblings, 0 replies; 19+ messages in thread
From: Alexei Starovoitov @ 2026-04-07 16:39 UTC (permalink / raw)
To: Emil Tsalapatis
Cc: bpf, Alexei Starovoitov, Andrii Nakryiko, Kumar Kartikeya Dwivedi,
Daniel Borkmann, Eduard, Song Liu
On Mon, Apr 6, 2026 at 9:57 PM Emil Tsalapatis <emil@etsalapatis.com> wrote:
>
> Add an address sanitizer (ASAN) runtime to the arena library. The
> ASAN runtime implements the functions injected into BPF binaries
> by LLVM sanitization when ASAN is enabled during compilation.
>
> The runtime also includes functions called explicitly by memory
> allocation code to mark memory as poisoned/unpoisoned to ASAN.
> This code is a no-op when sanitization is turned off.
>
> Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
> ---
> .../selftests/bpf/libarena/include/asan.h | 124 ++++
> .../selftests/bpf/libarena/include/common.h | 1 +
> .../selftests/bpf/libarena/src/asan.bpf.c | 539 ++++++++++++++++++
> .../selftests/bpf/libarena/src/common.bpf.c | 1 +
> 4 files changed, 665 insertions(+)
> create mode 100644 tools/testing/selftests/bpf/libarena/include/asan.h
> create mode 100644 tools/testing/selftests/bpf/libarena/src/asan.bpf.c
>
> diff --git a/tools/testing/selftests/bpf/libarena/include/asan.h b/tools/testing/selftests/bpf/libarena/include/asan.h
> new file mode 100644
> index 000000000000..d2eeabba3ffc
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/libarena/include/asan.h
> @@ -0,0 +1,124 @@
> +// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
> +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
> +#pragma once
> +
> +struct asan_init_args {
> + u64 arena_all_pages;
> + u64 arena_globals_pages;
> +};
> +
> +int asan_init(struct asan_init_args *args);
> +
> +/* Parameters usable by userspace. */
not a helpful comment.
> +extern volatile u64 __asan_shadow_memory_dynamic_address;
> +extern volatile u32 asan_reported;
> +extern volatile bool asan_inited;
> +extern volatile bool asan_report_once;
> +extern volatile bool asan_emit_stack;
> +
> +#ifdef __BPF__
> +
> +#define ASAN_SHADOW_SHIFT 3
> +#define ASAN_SHADOW_SCALE (1ULL << ASAN_SHADOW_SHIFT)
> +#define ASAN_GRANULE_MASK ((1ULL << ASAN_SHADOW_SHIFT) - 1)
> +#define ASAN_GRANULE(addr) ((s8)((u32)(u64)((addr)) & ASAN_GRANULE_MASK))
> +
> +#define __noasan __attribute__((no_sanitize("address")))
> +
> +#ifdef BPF_ARENA_ASAN
> +
> +/*
> + * Defined as char * to get 1-byte granularity for pointer arithmetic.
> + */
not a good comment either.
> +typedef s8 __arena s8a;
> +
> +/*
> + * Address to shadow map translation.
> + */
delete it too.
> +static inline
> +s8a *mem_to_shadow(void __arena __arg_arena *addr)
> +{
> + return (s8a *)(((u32)(u64)addr >> ASAN_SHADOW_SHIFT) + __asan_shadow_memory_dynamic_address);
> +}
> +
> +/*
> + * Helper for directly reading the shadow map.
> + */
> +static inline __noasan
> +s8 asan_shadow_value(void __arena __arg_arena *addr)
> +{
> + return *(s8a *)mem_to_shadow(addr);
> +}
what? Whole new helper just to avoid typing *(s8a *) ?
> +
> +__weak __noasan
> +bool asan_ready(void)
> +{
> + return __asan_shadow_memory_dynamic_address;
> +}
> +
> +/*
> + * Shadow map manipulation helpers.
> + */
delete the comment.
> +int asan_poison(void __arena *addr, s8 val, size_t size);
> +int asan_unpoison(void __arena *addr, size_t size);
> +bool asan_shadow_set(void __arena *addr);
> +
> +/*
> + * Dummy calls to ensure the ASAN runtime's BTF information is present
> + * in every object file when compiling the runtime and local BPF code
> + * separately. The runtime calls are injected into the LLVM IR file
> + */
> +#define DECLARE_ASAN_LOAD_STORE_SIZE(size) \
> + void __asan_store##size(void *addr); \
> + void __asan_store##size##_noabort(void *addr); \
> + void __asan_load##size(void *addr); \
> + void __asan_load##size##_noabort(void *addr); \
> + void __asan_report_store##size(void *addr); \
> + void __asan_report_store##size##_noabort(void *addr); \
> + void __asan_report_load##size(void *addr); \
> + void __asan_report_load##size##_noabort(void *addr);
> +
> +DECLARE_ASAN_LOAD_STORE_SIZE(1);
> +DECLARE_ASAN_LOAD_STORE_SIZE(2);
> +DECLARE_ASAN_LOAD_STORE_SIZE(4);
> +DECLARE_ASAN_LOAD_STORE_SIZE(8);
> +
> +#define ASAN_DUMMY_CALLS_SIZE(size, arg) \
> +do { \
> + __asan_store##size((arg)); \
> + __asan_store##size##_noabort((arg)); \
> + __asan_load##size((arg)); \
> + __asan_load##size##_noabort((arg)); \
> + __asan_report_store##size((arg)); \
> + __asan_report_store##size##_noabort((arg)); \
> + __asan_report_load##size((arg)); \
> + __asan_report_load##size##_noabort((arg)); \
> +} while (0)
> +
> +#define ASAN_DUMMY_CALLS_ALL(arg) \
> +do { \
> + ASAN_DUMMY_CALLS_SIZE(1, (arg)); \
> + ASAN_DUMMY_CALLS_SIZE(2, (arg)); \
> + ASAN_DUMMY_CALLS_SIZE(4, (arg)); \
> + ASAN_DUMMY_CALLS_SIZE(8, (arg)); \
> +} while (0)
> +
> +__weak __noasan
> +int asan_dummy_call(void) {
> + /* Use the shadow map base to prevent it from being optimized out. */
> + if (__asan_shadow_memory_dynamic_address)
> + ASAN_DUMMY_CALLS_ALL(NULL);
> +
> + return 0;
> +}
so every bpf prog with arena will have this asan_dummy_call() function
just for BTF info?
It doesn't smell right.
I think it's a bug on LLVM side. It should be preserving BTF
when it emits calls to __asan*.
> +#else /* BPF_ARENA_ASAN */
> +
> +static inline int asan_poison(void __arena *addr, s8 val, size_t size) { return 0; }
> +static inline int asan_unpoison(void __arena *addr, size_t size) { return 0; }
> +static inline bool asan_shadow_set(void __arena *addr) { return 0; }
> +static inline s8 asan_shadow_value(void __arena *addr) { return 0; }
> +__weak bool asan_ready(void) { return true; }
> +
> +#endif /* BPF_ARENA_ASAN */
> +
> +#endif /* __BPF__ */
> diff --git a/tools/testing/selftests/bpf/libarena/include/common.h b/tools/testing/selftests/bpf/libarena/include/common.h
> index 544a398a0d1e..f48395358d1d 100644
> --- a/tools/testing/selftests/bpf/libarena/include/common.h
> +++ b/tools/testing/selftests/bpf/libarena/include/common.h
> @@ -39,6 +39,7 @@ struct {
> } arena __weak SEC(".maps");
>
> extern const volatile u32 zero;
> +extern volatile u64 asan_violated;
>
> int arena_fls(__u64 word);
>
> diff --git a/tools/testing/selftests/bpf/libarena/src/asan.bpf.c b/tools/testing/selftests/bpf/libarena/src/asan.bpf.c
> new file mode 100644
> index 000000000000..b3fae0ed020c
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/libarena/src/asan.bpf.c
> @@ -0,0 +1,539 @@
> +// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
> +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
> +#include <vmlinux.h>
> +#include <common.h>
> +#include <asan.h>
> +
> +/*
> + * Address sanitizer (ASAN) for arena-based BPF programs. The
> + * sanitizer tracks valid arena memory and triggers an error
> + * when poisoned memory is read or written to. Starting point
> + * was the KASAN implementation in mm/.
KASAN part of the comment looks obsolete and not really helpful
in isolation. "Starting point was.." so?
How is it different ?
If it said "inspired by KASAN" that would be read differently.
Or just delete it.
> + *
> + * The API
> + * -------
> + *
> + * The implementation includes two kinds of components: Implementation
> + * of ASAN hooks injected by LLVM into the program, and API calls that
> + * allocators use to mark memory as valid or invalid. The full list is:
> + *
> + * LLVM stubs:
> + *
> + * void __asan_{load, store}<size>(void *addr)
> + * Checks whether an access is valid. All variations covered
> + * by check_region_inline().
> + *
> + * void __asan_{store, load}((void *addr, ssize_t size)
> + *
> + * void __asan_report_{load, store}<size>(void *addr)
> + * Report an access violation for the program. Used when LLVM
> + * uses direct code generation for shadow map checks.
> + *
> + * void *__asan_memcpy(void *d, const void *s, size_t n)
> + * void *__asan_memmove(void *d, const void *s, size_t n)
> + * void *__asan_memset(void *p, int c, size_t n)
> + * Hooks for ASAN instrumentation of the LLVM mem* builtins.
> + * Currently unimplemented just like the builtins themselves.
> + *
> + * API methods:
> + *
> + * asan_init()
> + * Initialize the ASAN map for the arena.
> + *
> + * asan_poison()
> + * Mark a region of memory as poisoned. Accessing poisoned memory
> + * causes asan_report() to fire. Invoked during free().
> + *
> + * asan_unpoison()
> + * Mark a region as unpoisoned after alloc().
> + *
> + * asan_shadow_set()
> + * Check a byte's validity directly.
> + *
> + * The Algorithm In Brief
> + * ----------------------
> + * Each group of 8 bytes is mapped to a "granule" in the shadow map. This
> + * granule is the size of the byte and describes which bytes are valid.
> + * Possible values are:
> + *
> + * 0: All bytes are valid. Makes checks in the middle of an allocated region
> + * (most of them) fast.
> + * (0, 7]: How many consecutive bytes are valid, starting from the lowest one.
> + * The tradeoff is that we can't poison individual bytes in the middle of a
> + * valid region.
> + * [0x80, 0xff]: Special poison values, can be used to denote specific error
> + * modes (e.g., recently freed vs uninitialized memory).
> + *
> + * The mapping between a memory location and its shadow is:
> + * shadow_addr = shadow_base + (addr >> 3). We retain the 8:1 data:shadow
> + * ratio of existing ASAN implementations as a compromise between tracking
> + * granularity and space usage/scan overhead.
> + */
> +
> +#ifdef BPF_ARENA_ASAN
> +
> +#pragma clang attribute push(__attribute__((no_sanitize("address"))), \
> + apply_to = function)
> +
> +#define SHADOW_ALL_ZEROES ((u64)-1)
> +
> +/*
> + * Canary variable for ASAN violations. Set to the offending address.
> + */
> +volatile u64 asan_violated = 0;
> +
> +/*
> + * Shadow map occupancy map.
> + */
> +volatile u64 __asan_shadow_memory_dynamic_address;
> +
> +volatile u32 asan_reported = false;
> +volatile bool asan_inited = false;
> +
> +/*
> + * Set during program load.
> + */
> +volatile bool asan_report_once = false;
> +volatile bool asan_emit_stack = false;
> +
> +/*
> + * BPF does not currently support the memset/memcpy/memcmp intrinsics.
and? I don't get what comment is trying to say.
> + */
> +__weak int asan_memset(s8a __arg_arena *dst, s8 val, size_t size)
right above the comment said __asan_memset is not implemented.
Yet this is an implementation. What is significance of double underscore?
> +{
> + size_t i;
> +
> + for (i = zero; i < size && can_loop; i++)
> + dst[i] = val;
> +
> + return 0;
> +}
> +
> +/* Validate a 1-byte access, always within a single byte. */
> +static __always_inline bool memory_is_poisoned_1(s8a *addr)
> +{
> + s8 shadow_value = asan_shadow_value(addr);
> +
> + /* Byte is 0, access is valid. */
> + if (likely(!shadow_value))
> + return false;
> +
> + /*
> + * Byte is non-zero. Access is valid if granule offset in [0, shadow_value),
> + * so the memory is poisoned if shadow_value is negative or smaller than
> + * the granule's value.
> + */
> +
> + return ASAN_GRANULE(addr) >= shadow_value;
> +}
> +
> +/* Validate a 2- 4-, 8-byte access, shadow spans up to 2 bytes. */
> +static __always_inline bool memory_is_poisoned_2_4_8(s8a *addr, u64 size)
> +{
> + u64 end = (u64)addr + size - 1;
> +
> + /*
> + * Region fully within a single byte (addition didn't
> + * overflow above ASAN_GRANULE).
> + */
> + if (likely(ASAN_GRANULE(end) >= size - 1))
> + return memory_is_poisoned_1((s8a *)end);
> +
> + /*
> + * Otherwise first byte must be fully unpoisoned, and second byte
> + * must be unpoisoned up to the end of the accessed region.
> + */
> +
> + return asan_shadow_value(addr) || memory_is_poisoned_1((s8a *)end);
> +}
> +
> +/*
> + * Explicit ASAN check.
the comment is too terse to be useful.
> + */
> +__weak bool asan_shadow_set(void __arena __arg_arena *addr)
> +{
> + return memory_is_poisoned_1(addr);
> +}
> +
> +static __always_inline u64 first_nonzero_byte(u64 addr, size_t size)
> +{
> + while (size && can_loop) {
> + if (unlikely(*(s8a *)addr))
> + return addr;
> + addr += 1;
> + size -= 1;
> + }
> +
> + return SHADOW_ALL_ZEROES;
> +}
> +
> +static __always_inline bool memory_is_poisoned_n(s8a *addr, u64 size)
> +{
> + u64 ret;
> + u64 start;
> + u64 end;
> +
> + /* Size of [start, end] is end - start + 1. */
> + start = (u64)mem_to_shadow(addr);
> + end = (u64)mem_to_shadow(addr + size - 1);
> +
> + ret = first_nonzero_byte(start, (end - start) + 1);
> + if (likely(ret == SHADOW_ALL_ZEROES))
> + return false;
> +
> + return __builtin_expect(ret != end || ASAN_GRANULE(addr + size - 1) >=
> + *(s8a *)end, false);
__builtin_expect ? Use unlikely().
> +}
> +
> +__weak int asan_report(s8a __arg_arena *addr, size_t sz,
> + bool write)
> +{
> + u32 reported = __sync_val_compare_and_swap(&asan_reported, false, true);
> +
> + /* Only report the first ASAN violation. */
> + if (reported && asan_report_once)
> + return 0;
> +
> + asan_violated = (u64)addr;
> +
> + if (asan_emit_stack) {
> + arena_stderr("Memory violation for address %p (0x%lx) for %s of size %ld",
> + addr, (u64)addr, write ? "write" : "read", sz);
> + bpf_stream_print_stack(BPF_STDERR);
> + }
> +
> + return 0;
> +}
> +
> +static __always_inline bool check_asan_args(s8a *addr, size_t size,
> + bool *result)
> +{
> + bool valid = true;
> +
> + /* Size 0 accesses are valid even if the address is invalid. */
> + if (unlikely(size == 0))
> + goto confirmed_valid;
> +
> + /*
> + * Wraparound is possible for values close to the the edge of the
> + * 4GiB boundary of the arena (last valid address is 1UL << 32 - 1).
> + *
> + *
> + * The wraparound detection below works for small sizes.check_asan_args is
> + * always called from the builtin ASAN checks, so 1 <= size <= 64. We do
> + * not implement storeN/loadN, so size is guaranteed to be in that range.
> + * Even if we did, sane storeN/loadN intrinsics are not expected to have
> + * a large enough size that
> + *
> + * - addr + size > MAX_U32
> + * - (u32)(addr + size) > (u32) addr
> + *
> + * which would defeat wraparound detection.
> + */
> + if (unlikely((u32)(u64)(addr + size) < (u32)(u64)addr))
> + goto confirmed_invalid;
> +
> + return false;
in this path the result will stay uninitialized. Bug or not?
> +
> +confirmed_invalid:
> + valid = false;
> +
> + /* FALLTHROUGH */
> +confirmed_valid:
> + *result = valid;
> +
> + return true;
> +}
> +
> +static __always_inline bool check_region_inline(void *ptr, size_t size,
> + bool write)
I have allergy to bool arguments.
Please use 'u32 flags' from the day one and human readable
enum, so the callsites are easy to read.
> +{
> + s8a *addr = (s8a *)(u64)ptr;
> + bool is_poisoned, is_valid;
> +
> + if (check_asan_args(addr, size, &is_valid)) {
> + if (!is_valid)
> + asan_report(addr, size, write);
> + return is_valid;
> + }
> +
> + switch (size) {
> + case 1:
> + is_poisoned = memory_is_poisoned_1(addr);
> + break;
> + case 2:
> + case 4:
> + case 8:
> + is_poisoned = memory_is_poisoned_2_4_8(addr, size);
> + break;
> + default:
> + is_poisoned = memory_is_poisoned_n(addr, size);
> + }
> +
> + if (is_poisoned) {
> + asan_report(addr, size, write);
> + return false;
> + }
> +
> + return true;
> +}
> +
> +/*
> + * __alias is not supported for BPF so define *__noabort() variants as wrappers.
> + */
> +#define DEFINE_ASAN_LOAD_STORE(size) \
> + __hidden void __asan_store##size(void *addr) \
> + { \
> + check_region_inline(addr, size, true); \
> + } \
> + __hidden void __asan_store##size##_noabort(void *addr) \
> + { \
> + check_region_inline(addr, size, true); \
> + } \
> + __hidden void __asan_load##size(void *addr) \
> + { \
> + check_region_inline(addr, size, false); \
> + } \
> + __hidden void __asan_load##size##_noabort(void *addr) \
> + { \
> + check_region_inline(addr, size, false); \
unlike here where false vs true is not meaningful without
reading the prototype.
> + } \
> + __hidden void __asan_report_store##size(void *addr) \
> + { \
> + asan_report((s8a *)addr, size, true); \
> + } \
> + __hidden void __asan_report_store##size##_noabort(void *addr) \
> + { \
> + asan_report((s8a *)addr, size, true); \
> + } \
> + __hidden void __asan_report_load##size(void *addr) \
> + { \
> + asan_report((s8a *)addr, size, false); \
> + } \
> + __hidden void __asan_report_load##size##_noabort(void *addr) \
> + { \
> + asan_report((s8a *)addr, size, false); \
> + }
> +
> +DEFINE_ASAN_LOAD_STORE(1);
> +DEFINE_ASAN_LOAD_STORE(2);
> +DEFINE_ASAN_LOAD_STORE(4);
> +DEFINE_ASAN_LOAD_STORE(8);
> +
> +void __asan_storeN(void *addr, ssize_t size)
> +{
> + check_region_inline(addr, size, true);
> +}
> +
> +void __asan_loadN(void *addr, ssize_t size)
> +{
> + check_region_inline(addr, size, false);
> +}
> +
> +/*
> + * We currently do not sanitize globals.
> + */
> +void __asan_register_globals(void *globals, size_t n)
> +{
> +}
> +
> +void __asan_unregister_globals(void *globals, size_t n)
> +{
> +}
> +
> +/*
> + * We do not currently have memcpy/memmove/memset intrinsics
> + * in LLVM. Do not implement sanitization.
> + */
> +void *__asan_memcpy(void *d, const void *s, size_t n)
> +{
> + arena_stderr("ASAN: Unexpected %s call", __func__);
> + return NULL;
> +}
> +
> +void *__asan_memmove(void *d, const void *s, size_t n)
> +{
> + arena_stderr("ASAN: Unexpected %s call", __func__);
> + return NULL;
> +}
> +
> +void *__asan_memset(void *p, int c, size_t n)
> +{
> + arena_stderr("ASAN: Unexpected %s call", __func__);
> + return NULL;
> +}
> +
> +/*
> + * Poisoning code, used when we add more freed memory to the allocator by:
> + * a) pulling memory from the arena segment using bpf_arena_alloc_pages()
> + * b) freeing memory from application code
> + */
> +__hidden __noasan int asan_poison(void __arena *addr, s8 val, size_t size)
> +{
> + s8a *shadow;
> + size_t len;
> +
> + /*
> + * Poisoning from a non-granule address makes no sense: We can only allocate
> + * memory to the application that has a granule-aligned starting address,
> + * and bpf_arena_alloc_pages returns page-aligned memory. A non-aligned
> + * addr then implies we're freeing a different address than the one we
> + * allocated.
> + */
> + if (unlikely((u64)addr & ASAN_GRANULE_MASK))
> + return -EINVAL;
> +
> + /*
> + * We cannot free an unaligned region because it'd be possible that we
> + * cannot describe the resulting poisoning state of the granule in
> + * the ASAN encoding.
> + *
> + * Every granule represents a region of memory that looks like the
> + * following (P for poisoned bytes, C for clear):
> + *
> + * <Clear> <Poisoned>
> + * [ C C C ... P P ]
> + *
> + * The value of the granule's shadow map is the number of clear bytes in
> + * it. We cannot represent granules with the following state:
> + *
> + * [ P P ... C C ... P P ]
> + *
> + * That would be possible if we could free unaligned regions, so prevent that.
> + */
> + if (unlikely(size & ASAN_GRANULE_MASK))
> + return -EINVAL;
> +
> + shadow = mem_to_shadow(addr);
> + len = size >> ASAN_SHADOW_SHIFT;
> +
> + asan_memset(shadow, val, len);
> +
> + return 0;
> +}
> +
> +/*
> + * Unpoisoning code for marking memory as valid during allocation calls.
> + *
> + * Very similar to asan_poison, except we need to round up instead of
> + * down, then partially poison the last granule if necessary.
> + *
> + * Partial poisoning is useful for keeping the padding poisoned. Allocations
> + * are granule-aligned, so we we're reserving granule-aligned sizes for the
> + * allocation. However, we want to still treat accesses to the padding as
> + * invalid. Partial poisoning takes care of that. Freeing and poisoning the
> + * memory is still done in granule-aligned sizes and repoisons the already
> + * poisoned padding.
> + */
> +__hidden __noasan int asan_unpoison(void __arena *addr, size_t size)
> +{
> + size_t partial = size & ASAN_GRANULE_MASK;
> + s8a *shadow;
> + size_t len;
> +
> + /*
> + * We cannot allocate in the middle of the granule. The ASAN shadow
> + * map encoding only describes regions of memory where every granule
> + * follows this format (P for poisoned, C for clear):
> + *
> + * <Clear> <Poisoned>
> + * [ C C C ... P P ]
> + *
> + * This is so we can use a single number in [0, ASAN_SHADOW_SCALE)
> + * to represent the poison state of the granule.
> + */
> + if (unlikely((u64)addr & ASAN_GRANULE_MASK))
> + return -EINVAL;
> +
> + shadow = mem_to_shadow(addr);
> + len = size >> ASAN_SHADOW_SHIFT;
> +
> + asan_memset(shadow, 0, len);
> +
> + /*
> + * If we are allocating a non-granule aligned region, we need to adjust
> + * the last byte of the shadow map to list how many bytes in the granule
> + * are unpoisoned. If the region is aligned, then the memset call above
> + * was enough.
> + */
> + if (partial)
> + shadow[len] = partial;
> +
> + return 0;
> +}
> +
> +/*
> + * Initialize ASAN state when necessary. Triggered from userspace before
> + * allocator startup.
> + */
> +SEC("syscall")
> +__hidden __noasan int asan_init(struct asan_init_args *args)
> +{
> + u64 globals_pages = args->arena_globals_pages;
> + u64 all_pages = args->arena_all_pages;
> + u64 shadowmap, shadow_pgoff;
shadowmap stands out. shadow_map ?
> + u64 shadow_pages;
> +
> + if (asan_inited)
> + return 0;
> +
> + /*
> + * Round up the shadow map size to the nearest page.
> + */
> + shadow_pages = all_pages >> ASAN_SHADOW_SHIFT;
> + if ((all_pages & ((1 << ASAN_SHADOW_SHIFT) -1 )))
> + shadow_pages += 1;
> +
> + /*
> + * Make sure the numbers provided by userspace are sane.
> + */
not a helpful comment.
> + if (all_pages > (1ULL << 32) / __PAGE_SIZE) {
> + arena_stderr("error: arena size %lx too large", all_pages);
> + return -EINVAL;
> + }
> +
> + if (globals_pages > all_pages) {
> + arena_stderr("error: globals %lx do not fit in arena %lx", globals_pages, all_pages);
> + return -EINVAL;
> + }
> +
> + if (globals_pages + shadow_pages > all_pages) {
> + arena_stderr("error: globals %lx do not leave room for shadow map %lx (arena pages %lx)",
> + globals_pages, shadow_pages, all_pages);
> + return -EINVAL;
> + }
> +
> + shadow_pgoff = all_pages - shadow_pages - globals_pages;
> + __asan_shadow_memory_dynamic_address = shadow_pgoff * __PAGE_SIZE;
> +
> + /*
> + * Allocate the last (1/ASAN_SHADOW_SCALE)th of an arena's pages for the map
> + * We find the offset and size from the arena map.
> + *
> + * The allocated map pages are zeroed out, meaning all memory is marked as valid
> + * even if it's not allocated already. This is expected: Since the actual memory
> + * pages are not allocated, accesses to it will trigger page faults and will be
> + * reported through BPF streams. Any pages allocated through bpf_arena_alloc_pages
> + * should be poisoned by the allocator right after the call succeeds.
> + */
> + shadowmap = (u64)bpf_arena_alloc_pages(
> + &arena, (void __arena *)__asan_shadow_memory_dynamic_address,
> + shadow_pages, NUMA_NO_NODE, 0);
> + if (!shadowmap) {
> + arena_stderr("Could not allocate shadow map\n");
> +
> + __asan_shadow_memory_dynamic_address = 0;
> +
> + return -ENOMEM;
> + }
> +
> + asan_inited = true;
> +
> + return 0;
> +}
> +
> +#pragma clang attribute pop
> +
> +#endif /* BPF_ARENA_ASAN */
> +
> +__weak char _license[] SEC("license") = "GPL";
> diff --git a/tools/testing/selftests/bpf/libarena/src/common.bpf.c b/tools/testing/selftests/bpf/libarena/src/common.bpf.c
> index cbb729290d3c..31acb259ec5a 100644
> --- a/tools/testing/selftests/bpf/libarena/src/common.bpf.c
> +++ b/tools/testing/selftests/bpf/libarena/src/common.bpf.c
> @@ -1,6 +1,7 @@
> // SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
> /* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
> #include <common.h>
> +#include <asan.h>
>
> const volatile u32 zero = 0;
>
> --
> 2.53.0
>
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH bpf-next v4 7/9] selftests/bpf: Add ASAN support for libarena selftests
2026-04-07 4:57 [PATCH bpf-next v4 0/9] Introduce arena library and runtime Emil Tsalapatis
` (5 preceding siblings ...)
2026-04-07 4:57 ` [PATCH bpf-next v4 6/9] selftests/bpf: Add arena ASAN runtime to libarena Emil Tsalapatis
@ 2026-04-07 4:57 ` Emil Tsalapatis
2026-04-07 17:12 ` Alexei Starovoitov
2026-04-07 4:57 ` [PATCH bpf-next v4 8/9] selftests/bpf: Add buddy allocator for libarena Emil Tsalapatis
2026-04-07 4:57 ` [PATCH bpf-next v4 9/9] selftests/bpf: Add selftests for libarena buddy allocator Emil Tsalapatis
8 siblings, 1 reply; 19+ messages in thread
From: Emil Tsalapatis @ 2026-04-07 4:57 UTC (permalink / raw)
To: bpf; +Cc: ast, andrii, memxor, daniel, eddyz87, song, Emil Tsalapatis
Expand the arena library selftest infrastructure to support
address sanitization. Add the compiler flags necessary to
compile the library under ASAN when supported.
Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
---
tools/testing/selftests/bpf/.gitignore | 1 +
tools/testing/selftests/bpf/Makefile | 13 +++++-
tools/testing/selftests/bpf/libarena/Makefile | 31 ++++++++++++-
.../bpf/libarena/include/selftest_helpers.h | 28 +++++++++++
.../selftests/bpf/libarena/include/userapi.h | 1 +
.../bpf/libarena/selftests/selftest.c | 27 ++++++++++-
.../bpf/libarena/selftests/st_asan_common.h | 46 +++++++++++++++++++
7 files changed, 143 insertions(+), 4 deletions(-)
create mode 100644 tools/testing/selftests/bpf/libarena/selftests/st_asan_common.h
diff --git a/tools/testing/selftests/bpf/.gitignore b/tools/testing/selftests/bpf/.gitignore
index 7f1960d6b59e..50fa79b2daac 100644
--- a/tools/testing/selftests/bpf/.gitignore
+++ b/tools/testing/selftests/bpf/.gitignore
@@ -50,3 +50,4 @@ verification_cert.h
usdt_1
usdt_2
libarena/test_libarena
+libarena/test_libarena_asan
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index 0d332c991023..f10e865338f6 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -751,6 +751,13 @@ LIBARENA_MAKE_ARGS = \
BPF_TARGET_ENDIAN="$(BPF_TARGET_ENDIAN)" \
Q="$(Q)"
+# libarena_asan.skel.h currently requires LLVM 22. For now,
+# only integrate the non-ASAN version with test_progs.
+LIBARENA_SKEL := libarena/libarena.skel.h
+
+$(LIBARENA_SKEL): $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ)
+ +$(MAKE) -C libarena libarena.skel.h $(LIBARENA_MAKE_ARGS)
+
# Define test_progs test runner.
TRUNNER_TESTS_DIR := prog_tests
TRUNNER_BPF_PROGS_DIR := progs
@@ -774,7 +781,8 @@ TRUNNER_EXTRA_SOURCES := test_progs.c \
flow_dissector_load.h \
ip_check_defrag_frags.h \
bpftool_helpers.c \
- usdt_1.c usdt_2.c
+ usdt_1.c usdt_2.c \
+ $(LIBARENA_SKEL)
TRUNNER_LIB_SOURCES := find_bit.c
TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read \
$(OUTPUT)/liburandom_read.so \
@@ -946,3 +954,6 @@ endef
test_libarena: $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ)
+$(MAKE) -C libarena $@ $(LIBARENA_MAKE_ARGS)
+
+test_libarena_asan: $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ)
+ +$(MAKE) -C libarena $@ $(LIBARENA_MAKE_ARGS)
diff --git a/tools/testing/selftests/bpf/libarena/Makefile b/tools/testing/selftests/bpf/libarena/Makefile
index 151ab6d0509c..c988da56c863 100644
--- a/tools/testing/selftests/bpf/libarena/Makefile
+++ b/tools/testing/selftests/bpf/libarena/Makefile
@@ -8,6 +8,7 @@ LIBARENA=$(abspath .)
LIBARENA_SOURCES = $(wildcard $(LIBARENA)/src/*.bpf.c) $(wildcard $(LIBARENA)/selftests/*.bpf.c)
LIBARENA_OBJECTS = $(notdir $(LIBARENA_SOURCES:.bpf.c=.bpf.o))
+LIBARENA_OBJECTS_ASAN = $(notdir $(LIBARENA_SOURCES:.bpf.c=_asan.bpf.o))
INCLUDES = -I$(LIBARENA)/include -I$(LIBARENA)/..
ifneq ($(INCLUDE_DIR),)
@@ -17,6 +18,13 @@ ifneq ($(LIBBPF_INCLUDE),)
INCLUDES += -I$(LIBBPF_INCLUDE)
endif
+ASAN_FLAGS = -fsanitize=kernel-address -fno-stack-protector -fno-builtin
+ASAN_FLAGS += -mllvm -asan-instrument-address-spaces=1 -mllvm -asan-shadow-addr-space=1
+ASAN_FLAGS += -mllvm -asan-use-stack-safety=0 -mllvm -asan-stack=0
+ASAN_FLAGS += -mllvm -asan-kernel=1
+ASAN_FLAGS += -mllvm -asan-constructor-kind=none
+ASAN_FLAGS += -mllvm -asan-destructor-kind=none
+
# ENABLE_ATOMICS_TESTS required because we use arena spinlocks
override BPF_CFLAGS += -DENABLE_ATOMICS_TESTS
override BPF_CFLAGS += -O2 -Wno-incompatible-pointer-types-discards-qualifiers
@@ -28,23 +36,42 @@ CFLAGS += $(INCLUDES)
vpath %.bpf.c $(LIBARENA)/src $(LIBARENA)/selftests
vpath %.c $(LIBARENA)/src $(LIBARENA)/selftests
-all: test_libarena
+all: test_libarena test_libarena_asan
+
+test_libarena_asan: selftest.c $(BPFOBJ) libarena_asan.skel.h
+ $(call msg,BINARY,libarena,$@)
+ $(Q)$(CLANG) $(CFLAGS) -DBPF_ARENA_ASAN $< $(BPFOBJ) $(LDLIBS) -o $@
test_libarena: selftest.c $(BPFOBJ) libarena.skel.h
$(call msg,BINARY,libarena,$@)
$(Q)$(CLANG) $(CFLAGS) $< $(BPFOBJ) $(LDLIBS) -o $@
+skeletons: libarena.skel.h libarena_asan.skel.h
+.PHONY: skeletons
+
+libarena_asan.skel.h: main_asan.bpf.o
+ $(call msg,GEN-SKEL,libarena,$@)
+ $(Q)$(BPFTOOL) gen skeleton $< name "libarena_asan" > $@
+
libarena.skel.h: main.bpf.o
$(call msg,GEN-SKEL,libarena,$@)
$(Q)$(BPFTOOL) gen skeleton $< name "libarena" > $@
+main_asan.bpf.o: $(LIBARENA_OBJECTS_ASAN)
+ $(call msg,GEN-OBJ,libarena,$@)
+ $(Q)$(BPFTOOL) gen object $@ $^
+
main.bpf.o: $(LIBARENA_OBJECTS)
$(call msg,GEN-OBJ,libarena,$@)
$(Q)$(BPFTOOL) gen object $@ $^
+%_asan.bpf.o: %.bpf.c
+ $(call msg,CLNG-BPF,libarena,$@)
+ $(Q)$(CLANG) $(BPF_CFLAGS) $(ASAN_FLAGS) -DBPF_ARENA_ASAN $(BPF_TARGET_ENDIAN) -c $< -o $@
+
%.bpf.o: %.bpf.c
$(call msg,CLNG-BPF,libarena,$@)
$(Q)$(CLANG) $(BPF_CFLAGS) $(BPF_TARGET_ENDIAN) -c $< -o $@
clean:
- $(Q)rm -f *.skel.h *.bpf.o test_libarena
+ $(Q)rm -f *.skel.h *.bpf.o *.linked*.o test_libarena test_libarena_asan
diff --git a/tools/testing/selftests/bpf/libarena/include/selftest_helpers.h b/tools/testing/selftests/bpf/libarena/include/selftest_helpers.h
index ee445d8f5b26..0cb3fdddd535 100644
--- a/tools/testing/selftests/bpf/libarena/include/selftest_helpers.h
+++ b/tools/testing/selftests/bpf/libarena/include/selftest_helpers.h
@@ -79,3 +79,31 @@ static inline int libarena_get_globals_pages(int arena_get_base_fd,
free(vec);
return 0;
}
+
+static inline int libarena_asan_init(int arena_get_base_fd,
+ int asan_init_fd,
+ size_t arena_all_pages)
+{
+ LIBBPF_OPTS(bpf_test_run_opts, opts);
+ struct asan_init_args args;
+ u64 globals_pages;
+ int ret;
+
+ ret = libarena_get_globals_pages(arena_get_base_fd,
+ arena_all_pages, &globals_pages);
+ if (ret)
+ return ret;
+
+ args = (struct asan_init_args){
+ .arena_all_pages = arena_all_pages,
+ .arena_globals_pages = globals_pages,
+ };
+
+ opts.ctx_in = &args;
+ opts.ctx_size_in = sizeof(args);
+
+ ret = bpf_prog_test_run_opts(asan_init_fd, &opts);
+ if (ret)
+ return ret;
+ return opts.retval;
+}
diff --git a/tools/testing/selftests/bpf/libarena/include/userapi.h b/tools/testing/selftests/bpf/libarena/include/userapi.h
index b4ac6bc9bd79..5d3a851fb65a 100644
--- a/tools/testing/selftests/bpf/libarena/include/userapi.h
+++ b/tools/testing/selftests/bpf/libarena/include/userapi.h
@@ -24,3 +24,4 @@ typedef int64_t s64;
#define arena_spinlock_t u64
#include "common.h"
+#include "asan.h"
diff --git a/tools/testing/selftests/bpf/libarena/selftests/selftest.c b/tools/testing/selftests/bpf/libarena/selftests/selftest.c
index b69adc7baaab..2bd8363ea614 100644
--- a/tools/testing/selftests/bpf/libarena/selftests/selftest.c
+++ b/tools/testing/selftests/bpf/libarena/selftests/selftest.c
@@ -18,6 +18,17 @@
#include <userapi.h>
#include <selftest_helpers.h>
+#include "selftest.h"
+
+#ifdef BPF_ARENA_ASAN
+#include "../libarena_asan.skel.h"
+typedef struct libarena_asan selftest;
+#define selftest__open libarena_asan__open
+#define selftest__open_and_load libarena_asan__open_and_load
+#define selftest__load libarena_asan__load
+#define selftest__attach libarena_asan__attach
+#define selftest__destroy libarena_asan__destroy
+#else
#include "../libarena.skel.h"
typedef struct libarena selftest;
#define selftest__open libarena__open
@@ -25,6 +36,7 @@ typedef struct libarena selftest;
#define selftest__load libarena__load
#define selftest__attach libarena__attach
#define selftest__destroy libarena__destroy
+#endif
static bool verbose = false;
static int testno = 1;
@@ -69,6 +81,7 @@ static int libbpf_print_fn(enum libbpf_print_level level,
int run_test(selftest *skel, const struct bpf_program *prog)
{
+ size_t arena_pages = (1UL << 32) / sysconf(_SC_PAGESIZE);
int prog_fd;
int ret;
@@ -76,6 +89,15 @@ int run_test(selftest *skel, const struct bpf_program *prog)
if (ret)
return ret;
+#ifdef BPF_ARENA_ASAN
+ ret = libarena_asan_init(
+ bpf_program__fd(skel->progs.arena_get_base),
+ bpf_program__fd(skel->progs.asan_init),
+ arena_pages);
+ if (ret)
+ return ret;
+#endif
+
prog_fd = bpf_program__fd(prog);
if (prog_fd < 0)
return prog_fd;
@@ -119,10 +141,13 @@ static void
banner(const char *progpath)
{
char *name = basename(progpath);
+ bool is_asan;
+ /* Check if our BPF programs are ASAN-capable using strstr on the prog name. */
printf("%s\n", name);
+ is_asan = strstr(name, "_asan");
- printf("=== %s ===\n", "libarena selftests");
+ printf("=== %s %s===\n", "libarena selftests", is_asan ? "(asan) " : "");
}
int main(int argc, char *argv[])
diff --git a/tools/testing/selftests/bpf/libarena/selftests/st_asan_common.h b/tools/testing/selftests/bpf/libarena/selftests/st_asan_common.h
new file mode 100644
index 000000000000..a907c9b45651
--- /dev/null
+++ b/tools/testing/selftests/bpf/libarena/selftests/st_asan_common.h
@@ -0,0 +1,46 @@
+// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+
+#pragma once
+
+#define ST_PAGES 64
+
+#define ASAN_MAP_STATE(addr) \
+ do { \
+ arena_stdout("%s:%d ASAN %lx -> (val: %x gran: %x set: [%s])", \
+ __func__, __LINE__, addr, \
+ asan_shadow_value((addr)), ASAN_GRANULE(addr), \
+ asan_shadow_set((addr)) ? "yes" : "no"); \
+ } while (0)
+
+/*
+ * Emit an error and force the current function to exit if the ASAN
+ * violation state is unexpected. Reset the violation state after.
+ */
+#define ASAN_VALIDATE_ADDR(cond, addr) \
+ do { \
+ asm volatile("" ::: "memory"); \
+ if ((asan_violated != 0) != (cond)) { \
+ arena_stdout("%s:%d ASAN asan_violated %lx", __func__, \
+ __LINE__, (u64)asan_violated); \
+ ASAN_MAP_STATE((addr)); \
+ return -EINVAL; \
+ } \
+ asan_violated = 0; \
+ } while (0)
+
+#define ASAN_VALIDATE() \
+ do { \
+ if ((asan_violated)) { \
+ arena_stdout("%s:%d Found ASAN violation at %lx", \
+ __func__, __LINE__, asan_violated); \
+ return -EINVAL; \
+ } \
+ } while (0)
+
+struct blob {
+ volatile u8 mem[59];
+ u8 oob;
+};
+
+
--
2.53.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* Re: [PATCH bpf-next v4 7/9] selftests/bpf: Add ASAN support for libarena selftests
2026-04-07 4:57 ` [PATCH bpf-next v4 7/9] selftests/bpf: Add ASAN support for libarena selftests Emil Tsalapatis
@ 2026-04-07 17:12 ` Alexei Starovoitov
0 siblings, 0 replies; 19+ messages in thread
From: Alexei Starovoitov @ 2026-04-07 17:12 UTC (permalink / raw)
To: Emil Tsalapatis
Cc: bpf, Alexei Starovoitov, Andrii Nakryiko, Kumar Kartikeya Dwivedi,
Daniel Borkmann, Eduard, Song Liu
On Mon, Apr 6, 2026 at 9:57 PM Emil Tsalapatis <emil@etsalapatis.com> wrote:
>
> Expand the arena library selftest infrastructure to support
> address sanitization. Add the compiler flags necessary to
> compile the library under ASAN when supported.
>
> Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
> ---
> tools/testing/selftests/bpf/.gitignore | 1 +
> tools/testing/selftests/bpf/Makefile | 13 +++++-
> tools/testing/selftests/bpf/libarena/Makefile | 31 ++++++++++++-
> .../bpf/libarena/include/selftest_helpers.h | 28 +++++++++++
> .../selftests/bpf/libarena/include/userapi.h | 1 +
> .../bpf/libarena/selftests/selftest.c | 27 ++++++++++-
> .../bpf/libarena/selftests/st_asan_common.h | 46 +++++++++++++++++++
> 7 files changed, 143 insertions(+), 4 deletions(-)
> create mode 100644 tools/testing/selftests/bpf/libarena/selftests/st_asan_common.h
>
> diff --git a/tools/testing/selftests/bpf/.gitignore b/tools/testing/selftests/bpf/.gitignore
> index 7f1960d6b59e..50fa79b2daac 100644
> --- a/tools/testing/selftests/bpf/.gitignore
> +++ b/tools/testing/selftests/bpf/.gitignore
> @@ -50,3 +50,4 @@ verification_cert.h
> usdt_1
> usdt_2
> libarena/test_libarena
> +libarena/test_libarena_asan
> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> index 0d332c991023..f10e865338f6 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile
> @@ -751,6 +751,13 @@ LIBARENA_MAKE_ARGS = \
> BPF_TARGET_ENDIAN="$(BPF_TARGET_ENDIAN)" \
> Q="$(Q)"
>
> +# libarena_asan.skel.h currently requires LLVM 22. For now,
> +# only integrate the non-ASAN version with test_progs.
> +LIBARENA_SKEL := libarena/libarena.skel.h
> +
> +$(LIBARENA_SKEL): $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ)
> + +$(MAKE) -C libarena libarena.skel.h $(LIBARENA_MAKE_ARGS)
> +
> # Define test_progs test runner.
> TRUNNER_TESTS_DIR := prog_tests
> TRUNNER_BPF_PROGS_DIR := progs
> @@ -774,7 +781,8 @@ TRUNNER_EXTRA_SOURCES := test_progs.c \
> flow_dissector_load.h \
> ip_check_defrag_frags.h \
> bpftool_helpers.c \
> - usdt_1.c usdt_2.c
> + usdt_1.c usdt_2.c \
> + $(LIBARENA_SKEL)
> TRUNNER_LIB_SOURCES := find_bit.c
> TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read \
> $(OUTPUT)/liburandom_read.so \
> @@ -946,3 +954,6 @@ endef
>
> test_libarena: $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ)
> +$(MAKE) -C libarena $@ $(LIBARENA_MAKE_ARGS)
> +
> +test_libarena_asan: $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ)
> + +$(MAKE) -C libarena $@ $(LIBARENA_MAKE_ARGS)
> diff --git a/tools/testing/selftests/bpf/libarena/Makefile b/tools/testing/selftests/bpf/libarena/Makefile
> index 151ab6d0509c..c988da56c863 100644
> --- a/tools/testing/selftests/bpf/libarena/Makefile
> +++ b/tools/testing/selftests/bpf/libarena/Makefile
> @@ -8,6 +8,7 @@ LIBARENA=$(abspath .)
>
> LIBARENA_SOURCES = $(wildcard $(LIBARENA)/src/*.bpf.c) $(wildcard $(LIBARENA)/selftests/*.bpf.c)
> LIBARENA_OBJECTS = $(notdir $(LIBARENA_SOURCES:.bpf.c=.bpf.o))
> +LIBARENA_OBJECTS_ASAN = $(notdir $(LIBARENA_SOURCES:.bpf.c=_asan.bpf.o))
>
> INCLUDES = -I$(LIBARENA)/include -I$(LIBARENA)/..
> ifneq ($(INCLUDE_DIR),)
> @@ -17,6 +18,13 @@ ifneq ($(LIBBPF_INCLUDE),)
> INCLUDES += -I$(LIBBPF_INCLUDE)
> endif
>
> +ASAN_FLAGS = -fsanitize=kernel-address -fno-stack-protector -fno-builtin
> +ASAN_FLAGS += -mllvm -asan-instrument-address-spaces=1 -mllvm -asan-shadow-addr-space=1
> +ASAN_FLAGS += -mllvm -asan-use-stack-safety=0 -mllvm -asan-stack=0
> +ASAN_FLAGS += -mllvm -asan-kernel=1
> +ASAN_FLAGS += -mllvm -asan-constructor-kind=none
> +ASAN_FLAGS += -mllvm -asan-destructor-kind=none
> +
> # ENABLE_ATOMICS_TESTS required because we use arena spinlocks
> override BPF_CFLAGS += -DENABLE_ATOMICS_TESTS
> override BPF_CFLAGS += -O2 -Wno-incompatible-pointer-types-discards-qualifiers
> @@ -28,23 +36,42 @@ CFLAGS += $(INCLUDES)
> vpath %.bpf.c $(LIBARENA)/src $(LIBARENA)/selftests
> vpath %.c $(LIBARENA)/src $(LIBARENA)/selftests
>
> -all: test_libarena
> +all: test_libarena test_libarena_asan
> +
> +test_libarena_asan: selftest.c $(BPFOBJ) libarena_asan.skel.h
> + $(call msg,BINARY,libarena,$@)
> + $(Q)$(CLANG) $(CFLAGS) -DBPF_ARENA_ASAN $< $(BPFOBJ) $(LDLIBS) -o $@
>
> test_libarena: selftest.c $(BPFOBJ) libarena.skel.h
> $(call msg,BINARY,libarena,$@)
> $(Q)$(CLANG) $(CFLAGS) $< $(BPFOBJ) $(LDLIBS) -o $@
>
> +skeletons: libarena.skel.h libarena_asan.skel.h
> +.PHONY: skeletons
> +
> +libarena_asan.skel.h: main_asan.bpf.o
> + $(call msg,GEN-SKEL,libarena,$@)
> + $(Q)$(BPFTOOL) gen skeleton $< name "libarena_asan" > $@
> +
> libarena.skel.h: main.bpf.o
> $(call msg,GEN-SKEL,libarena,$@)
> $(Q)$(BPFTOOL) gen skeleton $< name "libarena" > $@
>
> +main_asan.bpf.o: $(LIBARENA_OBJECTS_ASAN)
> + $(call msg,GEN-OBJ,libarena,$@)
> + $(Q)$(BPFTOOL) gen object $@ $^
> +
> main.bpf.o: $(LIBARENA_OBJECTS)
> $(call msg,GEN-OBJ,libarena,$@)
> $(Q)$(BPFTOOL) gen object $@ $^
>
> +%_asan.bpf.o: %.bpf.c
> + $(call msg,CLNG-BPF,libarena,$@)
> + $(Q)$(CLANG) $(BPF_CFLAGS) $(ASAN_FLAGS) -DBPF_ARENA_ASAN $(BPF_TARGET_ENDIAN) -c $< -o $@
> +
> %.bpf.o: %.bpf.c
> $(call msg,CLNG-BPF,libarena,$@)
> $(Q)$(CLANG) $(BPF_CFLAGS) $(BPF_TARGET_ENDIAN) -c $< -o $@
>
> clean:
> - $(Q)rm -f *.skel.h *.bpf.o test_libarena
> + $(Q)rm -f *.skel.h *.bpf.o *.linked*.o test_libarena test_libarena_asan
> diff --git a/tools/testing/selftests/bpf/libarena/include/selftest_helpers.h b/tools/testing/selftests/bpf/libarena/include/selftest_helpers.h
> index ee445d8f5b26..0cb3fdddd535 100644
> --- a/tools/testing/selftests/bpf/libarena/include/selftest_helpers.h
> +++ b/tools/testing/selftests/bpf/libarena/include/selftest_helpers.h
> @@ -79,3 +79,31 @@ static inline int libarena_get_globals_pages(int arena_get_base_fd,
> free(vec);
> return 0;
> }
> +
> +static inline int libarena_asan_init(int arena_get_base_fd,
> + int asan_init_fd,
> + size_t arena_all_pages)
> +{
> + LIBBPF_OPTS(bpf_test_run_opts, opts);
> + struct asan_init_args args;
> + u64 globals_pages;
> + int ret;
> +
> + ret = libarena_get_globals_pages(arena_get_base_fd,
> + arena_all_pages, &globals_pages);
> + if (ret)
> + return ret;
> +
> + args = (struct asan_init_args){
> + .arena_all_pages = arena_all_pages,
> + .arena_globals_pages = globals_pages,
> + };
> +
> + opts.ctx_in = &args;
> + opts.ctx_size_in = sizeof(args);
> +
> + ret = bpf_prog_test_run_opts(asan_init_fd, &opts);
> + if (ret)
> + return ret;
> + return opts.retval;
> +}
> diff --git a/tools/testing/selftests/bpf/libarena/include/userapi.h b/tools/testing/selftests/bpf/libarena/include/userapi.h
> index b4ac6bc9bd79..5d3a851fb65a 100644
> --- a/tools/testing/selftests/bpf/libarena/include/userapi.h
> +++ b/tools/testing/selftests/bpf/libarena/include/userapi.h
> @@ -24,3 +24,4 @@ typedef int64_t s64;
> #define arena_spinlock_t u64
>
> #include "common.h"
> +#include "asan.h"
> diff --git a/tools/testing/selftests/bpf/libarena/selftests/selftest.c b/tools/testing/selftests/bpf/libarena/selftests/selftest.c
> index b69adc7baaab..2bd8363ea614 100644
> --- a/tools/testing/selftests/bpf/libarena/selftests/selftest.c
> +++ b/tools/testing/selftests/bpf/libarena/selftests/selftest.c
> @@ -18,6 +18,17 @@
> #include <userapi.h>
> #include <selftest_helpers.h>
>
> +#include "selftest.h"
> +
> +#ifdef BPF_ARENA_ASAN
> +#include "../libarena_asan.skel.h"
> +typedef struct libarena_asan selftest;
> +#define selftest__open libarena_asan__open
> +#define selftest__open_and_load libarena_asan__open_and_load
> +#define selftest__load libarena_asan__load
> +#define selftest__attach libarena_asan__attach
> +#define selftest__destroy libarena_asan__destroy
ohh. I see. So you want to double compile the same bpf code
with and without asan and reuse the tests?
I guess #defines are fine then.
> +#else
> #include "../libarena.skel.h"
> typedef struct libarena selftest;
> #define selftest__open libarena__open
> @@ -25,6 +36,7 @@ typedef struct libarena selftest;
> #define selftest__load libarena__load
> #define selftest__attach libarena__attach
> #define selftest__destroy libarena__destroy
> +#endif
>
> static bool verbose = false;
> static int testno = 1;
> @@ -69,6 +81,7 @@ static int libbpf_print_fn(enum libbpf_print_level level,
>
> int run_test(selftest *skel, const struct bpf_program *prog)
> {
> + size_t arena_pages = (1UL << 32) / sysconf(_SC_PAGESIZE);
> int prog_fd;
> int ret;
>
> @@ -76,6 +89,15 @@ int run_test(selftest *skel, const struct bpf_program *prog)
> if (ret)
> return ret;
>
> +#ifdef BPF_ARENA_ASAN
> + ret = libarena_asan_init(
> + bpf_program__fd(skel->progs.arena_get_base),
> + bpf_program__fd(skel->progs.asan_init),
> + arena_pages);
> + if (ret)
> + return ret;
> +#endif
> +
> prog_fd = bpf_program__fd(prog);
> if (prog_fd < 0)
> return prog_fd;
> @@ -119,10 +141,13 @@ static void
> banner(const char *progpath)
> {
> char *name = basename(progpath);
> + bool is_asan;
>
> + /* Check if our BPF programs are ASAN-capable using strstr on the prog name. */
> printf("%s\n", name);
> + is_asan = strstr(name, "_asan");
>
> - printf("=== %s ===\n", "libarena selftests");
> + printf("=== %s %s===\n", "libarena selftests", is_asan ? "(asan) " : "");
> }
>
> int main(int argc, char *argv[])
> diff --git a/tools/testing/selftests/bpf/libarena/selftests/st_asan_common.h b/tools/testing/selftests/bpf/libarena/selftests/st_asan_common.h
> new file mode 100644
> index 000000000000..a907c9b45651
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/libarena/selftests/st_asan_common.h
> @@ -0,0 +1,46 @@
> +// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
> +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
> +
> +#pragma once
> +
> +#define ST_PAGES 64
> +
> +#define ASAN_MAP_STATE(addr) \
> + do { \
> + arena_stdout("%s:%d ASAN %lx -> (val: %x gran: %x set: [%s])", \
> + __func__, __LINE__, addr, \
> + asan_shadow_value((addr)), ASAN_GRANULE(addr), \
> + asan_shadow_set((addr)) ? "yes" : "no"); \
> + } while (0)
> +
> +/*
> + * Emit an error and force the current function to exit if the ASAN
> + * violation state is unexpected. Reset the violation state after.
> + */
> +#define ASAN_VALIDATE_ADDR(cond, addr) \
> + do { \
> + asm volatile("" ::: "memory"); \
> + if ((asan_violated != 0) != (cond)) { \
> + arena_stdout("%s:%d ASAN asan_violated %lx", __func__, \
> + __LINE__, (u64)asan_violated); \
> + ASAN_MAP_STATE((addr)); \
> + return -EINVAL; \
> + } \
> + asan_violated = 0; \
> + } while (0)
> +
> +#define ASAN_VALIDATE() \
> + do { \
> + if ((asan_violated)) { \
> + arena_stdout("%s:%d Found ASAN violation at %lx", \
> + __func__, __LINE__, asan_violated); \
> + return -EINVAL; \
> + } \
> + } while (0)
ASAN_VALIDATE_ADDR has to be a macro, but the other two can
be static inline functions.
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH bpf-next v4 8/9] selftests/bpf: Add buddy allocator for libarena
2026-04-07 4:57 [PATCH bpf-next v4 0/9] Introduce arena library and runtime Emil Tsalapatis
` (6 preceding siblings ...)
2026-04-07 4:57 ` [PATCH bpf-next v4 7/9] selftests/bpf: Add ASAN support for libarena selftests Emil Tsalapatis
@ 2026-04-07 4:57 ` Emil Tsalapatis
2026-04-07 5:43 ` bot+bpf-ci
2026-04-07 17:07 ` Alexei Starovoitov
2026-04-07 4:57 ` [PATCH bpf-next v4 9/9] selftests/bpf: Add selftests for libarena buddy allocator Emil Tsalapatis
8 siblings, 2 replies; 19+ messages in thread
From: Emil Tsalapatis @ 2026-04-07 4:57 UTC (permalink / raw)
To: bpf; +Cc: ast, andrii, memxor, daniel, eddyz87, song, Emil Tsalapatis
Add a byte-oriented buddy allocator for libarena. The buddy
allocator provides an alloc/free interface for small arena
allocations (down to 16 bytes).
Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
---
.../selftests/bpf/libarena/include/buddy.h | 64 ++
.../selftests/bpf/libarena/src/buddy.bpf.c | 879 ++++++++++++++++++
2 files changed, 943 insertions(+)
create mode 100644 tools/testing/selftests/bpf/libarena/include/buddy.h
create mode 100644 tools/testing/selftests/bpf/libarena/src/buddy.bpf.c
diff --git a/tools/testing/selftests/bpf/libarena/include/buddy.h b/tools/testing/selftests/bpf/libarena/include/buddy.h
new file mode 100644
index 000000000000..aa63752bc657
--- /dev/null
+++ b/tools/testing/selftests/bpf/libarena/include/buddy.h
@@ -0,0 +1,64 @@
+// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#pragma once
+
+/* Buddy allocator-related structs. */
+
+struct buddy_chunk;
+typedef struct buddy_chunk __arena buddy_chunk_t;
+
+struct buddy_header;
+typedef struct buddy_header __arena buddy_header_t;
+
+enum buddy_consts {
+ BUDDY_MIN_ALLOC_SHIFT = 4,
+ BUDDY_MIN_ALLOC_BYTES = 1 << BUDDY_MIN_ALLOC_SHIFT,
+ BUDDY_CHUNK_NUM_ORDERS = 1 << 4, /* 4 bits per order */
+ BUDDY_CHUNK_BYTES = BUDDY_MIN_ALLOC_BYTES << BUDDY_CHUNK_NUM_ORDERS,
+ BUDDY_HEADER_OFF = 8, /* header byte offset, see buddy.bpf.c for details */
+ BUDDY_CHUNK_PAGES = BUDDY_CHUNK_BYTES / __PAGE_SIZE,
+ BUDDY_CHUNK_ITEMS = 1 << BUDDY_CHUNK_NUM_ORDERS,
+ BUDDY_CHUNK_OFFSET_MASK = BUDDY_CHUNK_BYTES - 1,
+ BUDDY_VADDR_OFFSET = BUDDY_CHUNK_BYTES, /* Start aligning at chunk */
+ BUDDY_VADDR_SIZE = BUDDY_CHUNK_BYTES << 10 /* 1024 chunks maximum */
+};
+
+struct buddy_header {
+ u32 prev_index; /* "Pointer" to the previous available allocation of the same size. */
+ u32 next_index; /* Same for the next allocation. */
+};
+
+/*
+ * We bring memory into the allocator 1MiB at a time.
+ */
+struct buddy_chunk {
+ /* The order of the current allocation for a item. 4 bits per order. */
+ u8 orders[BUDDY_CHUNK_ITEMS / 2];
+ /*
+ * Bit to denote whether chunk is allocated. Size of the allocated/free
+ * chunk found from the orders array.
+ */
+ u8 allocated[BUDDY_CHUNK_ITEMS / 8];
+ /* Freelists for O(1) allocation. */
+ u64 freelists[BUDDY_CHUNK_NUM_ORDERS];
+ buddy_chunk_t *prev;
+ buddy_chunk_t *next;
+};
+
+struct buddy {
+ buddy_chunk_t *first_chunk; /* Pointer to the chunk linked list. */
+ arena_spinlock_t __arena *lock; /* Allocator lock */
+ u64 vaddr; /* Allocation into reserved vaddr */
+};
+
+#ifdef __BPF__
+
+int buddy_init(struct buddy *buddy, arena_spinlock_t __arena *lock);
+int buddy_destroy(struct buddy *buddy);
+int buddy_free_internal(struct buddy *buddy, u64 free);
+#define buddy_free(buddy, ptr) do { buddy_free_internal((buddy), (u64)(ptr)); } while (0)
+u64 buddy_alloc_internal(struct buddy *buddy, size_t size);
+#define buddy_alloc(alloc, size) ((void __arena *)buddy_alloc_internal((alloc), (size)))
+
+
+#endif /* __BPF__ */
diff --git a/tools/testing/selftests/bpf/libarena/src/buddy.bpf.c b/tools/testing/selftests/bpf/libarena/src/buddy.bpf.c
new file mode 100644
index 000000000000..4343e39760ab
--- /dev/null
+++ b/tools/testing/selftests/bpf/libarena/src/buddy.bpf.c
@@ -0,0 +1,879 @@
+// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+
+#include <common.h>
+#include <asan.h>
+#include <buddy.h>
+
+/*
+ * Buddy allocator arena-based implementation.
+ *
+ * Memory is organized into chunks. These chunks
+ * cannot be coalesced or split. Allocating
+ * chunks allocates their memory eagerly.
+ *
+ * Internally, each chunk is organized into blocks.
+ * Blocks _can_ be coalesced/split, but only inside
+ * the chunk. Each block can be allocated or
+ * unallocated. If allocated, the entire block holds
+ * user data. If unallocated, the block is mostly
+ * invalid memory, with the exception of a header
+ * used for freelist tracking.
+ *
+ * The header is placed at an offset inside the block
+ * to prevent off-by-one errors from the previous block
+ * from trivially overwriting the header. Such an error
+ * is also not catchable by ASAN, since the header remains
+ * valid memory even after the block is freed. It is still
+ * theoretically possible for the header to be corrupted
+ * without being caught by ASAN, but harder.
+ *
+ * Since the allocator needs to track order information for
+ * both allocated and free blocks, and allocated blocks cannot
+ * store a header, the allocator also stores per-chunk order
+ * information in a reserved region at the beginning of the
+ * chunk. The header includes a bitmap with the order of blocks
+ * and their allocation state. It also includes the freelist
+ * heads for the allocation itself.
+ */
+
+
+enum {
+ BUDDY_POISONED = (s8)0xef,
+};
+
+static inline int buddy_lock(struct buddy *buddy)
+{
+ return arena_spin_lock(buddy->lock);
+}
+
+static inline void buddy_unlock(struct buddy *buddy)
+{
+ arena_spin_unlock(buddy->lock);
+}
+
+/*
+ * Reserve part of the arena address space for the allocator. We use
+ * this to get aligned addresses for the chunks, since the arena
+ * page alloc kfuncs do not support alignment.
+ */
+static int buddy_reserve_arena_vaddr(struct buddy *buddy)
+{
+ buddy->vaddr = 0;
+
+ return bpf_arena_reserve_pages(&arena,
+ (void __arena *)BUDDY_VADDR_OFFSET,
+ BUDDY_VADDR_SIZE / __PAGE_SIZE);
+}
+
+/*
+ * Free up any unused address space. Used only during teardown.
+ */
+static void buddy_unreserve_arena_vaddr(struct buddy *buddy)
+{
+ bpf_arena_free_pages(
+ &arena, (void __arena *)(BUDDY_VADDR_OFFSET + buddy->vaddr),
+ (BUDDY_VADDR_SIZE - buddy->vaddr) / __PAGE_SIZE);
+
+ buddy->vaddr = 0;
+}
+
+/*
+ * Carve out part of the reserved address space and hand it over
+ * to the buddy allocator.
+ *
+ * We are assuming the buddy allocator is the only allocator in the
+ * system, so there is no race between this function reserving a
+ * page range and some other allocator actually making the BPF call
+ * to really create and reserve it.
+ */
+__weak int buddy_alloc_arena_vaddr(struct buddy *buddy, u64 *vaddrp)
+{
+ u64 vaddr, old, new;
+
+ if (!buddy || !vaddrp)
+ return -EINVAL;
+
+ do {
+ vaddr = buddy->vaddr;
+ new = vaddr + BUDDY_CHUNK_BYTES;
+
+ if (new > BUDDY_VADDR_SIZE)
+ return -EINVAL;
+
+ old = __sync_val_compare_and_swap(&buddy->vaddr, vaddr, new);
+ } while (old != vaddr && can_loop);
+
+ if (old != vaddr)
+ return -EINVAL;
+
+ *vaddrp = BUDDY_VADDR_OFFSET + vaddr;
+
+ return 0;
+}
+
+static u64 arena_next_pow2(__u64 n)
+{
+ n--;
+ n |= n >> 1;
+ n |= n >> 2;
+ n |= n >> 4;
+ n |= n >> 8;
+ n |= n >> 16;
+ n |= n >> 32;
+ n++;
+
+ return n;
+}
+
+__weak
+int idx_set_allocated(buddy_chunk_t __arg_arena *chunk, u64 idx, bool allocated)
+{
+ if (unlikely(idx >= BUDDY_CHUNK_ITEMS)) {
+ arena_stderr("setting state of of invalid idx (%d, max %d)\n", idx,
+ BUDDY_CHUNK_ITEMS);
+ return -EINVAL;
+ }
+
+ if (allocated)
+ chunk->allocated[idx / 8] |= 1 << (idx % 8);
+ else
+ chunk->allocated[idx / 8] &= ~(1 << (idx % 8));
+
+ return 0;
+}
+
+static int idx_is_allocated(buddy_chunk_t *chunk, u64 idx, bool *allocated)
+{
+ if (unlikely(idx >= BUDDY_CHUNK_ITEMS)) {
+ arena_stderr("getting state of invalid idx (%llu, max %d)\n", idx,
+ BUDDY_CHUNK_ITEMS);
+ return -EINVAL;
+ }
+
+ *allocated = chunk->allocated[idx / 8] & (1 << (idx % 8));
+ return 0;
+}
+
+__weak
+int idx_set_order(buddy_chunk_t __arg_arena *chunk, u64 idx, u8 order)
+{
+ u8 prev_order;
+
+ if (unlikely(order >= BUDDY_CHUNK_NUM_ORDERS)) {
+ arena_stderr("setting invalid order %u\n", order);
+ return -EINVAL;
+ }
+
+ if (unlikely(idx >= BUDDY_CHUNK_ITEMS)) {
+ arena_stderr("setting order of invalid idx (%d, max %d)\n", idx,
+ BUDDY_CHUNK_ITEMS);
+ return -EINVAL;
+ }
+
+ /*
+ * We store two order instances per byte, one per nibble.
+ * Retain the existing nibble.
+ */
+ prev_order = chunk->orders[idx / 2];
+ if (idx & 0x1) {
+ order &= 0xf;
+ order |= (prev_order & 0xf0);
+ } else {
+ order <<= 4;
+ order |= (prev_order & 0xf);
+ }
+
+ chunk->orders[idx / 2] = order;
+
+ return 0;
+}
+
+static u8 idx_get_order(buddy_chunk_t *chunk, u64 idx)
+{
+ u8 result;
+
+ _Static_assert(BUDDY_CHUNK_NUM_ORDERS <= 16,
+ "order must fit in 4 bits");
+
+ if (unlikely(idx >= BUDDY_CHUNK_ITEMS)) {
+ arena_stderr("setting order of invalid idx\n");
+ return BUDDY_CHUNK_NUM_ORDERS;
+ }
+
+ result = chunk->orders[idx / 2];
+
+ return (idx & 0x1) ? (result & 0xf) : (result >> 4);
+}
+
+static void __arena *idx_to_addr(buddy_chunk_t *chunk, size_t idx)
+{
+ u64 address;
+
+ if (unlikely(idx >= BUDDY_CHUNK_ITEMS)) {
+ arena_stderr("setting order of invalid idx\n");
+ return NULL;
+ }
+
+ /*
+ * The data blocks start in the chunk after the metadata block.
+ * We find the actual address by indexing into the region at an
+ * BUDDY_MIN_ALLOC_BYTES granularity, the minimum allowed.
+ * The index number already accounts for the fact that the first
+ * blocks in the chunk are occupied by the metadata, so we do
+ * not need to offset it.
+ */
+
+ address = (u64)chunk + (idx * BUDDY_MIN_ALLOC_BYTES);
+
+ return (void __arena *)address;
+}
+
+static buddy_header_t *idx_to_header(buddy_chunk_t *chunk, size_t idx)
+{
+ bool allocated;
+ u64 address;
+
+ if (unlikely(idx_is_allocated(chunk, idx, &allocated))) {
+ arena_stderr("accessing invalid idx 0x%lx\n", idx);
+ return NULL;
+ }
+
+ if (unlikely(allocated)) {
+ arena_stderr("accessing allocated idx 0x%lx as header\n", idx);
+ return NULL;
+ }
+
+ address = (u64)idx_to_addr(chunk, idx);
+
+ /*
+ * Offset the header within the block. This avoids accidental overwrites
+ * to the header because of off-by-one errors when using adjacent blocks.
+ *
+ * The offset has been chosen as a compromise between ASAN effectiveness
+ * and allocator granularity:
+ * 1) ASAN dictates valid data runs are 8-byte aligned.
+ * 2) We want to keep a low minimum allocation size (currently 16).
+ *
+ * As a result, we have only two possible positions for the header: Bytes
+ * 0 and 8. Keeping the header in byte 0 means off-by-ones from the previous
+ * block touch the header, and, since the header must be accessible, ASAN
+ * will not trigger. Keeping the header on byte 8 means off-by-one errors from
+ * the previous block are caught by ASAN. Negative offsets are rarer, so
+ * while accesses into the block from the next block are possible, they are
+ * less probable.
+ */
+
+ return (buddy_header_t *)(address + BUDDY_HEADER_OFF);
+}
+
+static void header_add_freelist(buddy_chunk_t *chunk,
+ buddy_header_t *header, u64 idx, u8 order)
+{
+ buddy_header_t *tmp_header;
+
+ idx_set_order(chunk, idx, order);
+
+ header->next_index = chunk->freelists[order];
+ header->prev_index = BUDDY_CHUNK_ITEMS;
+
+ if (header->next_index != BUDDY_CHUNK_ITEMS) {
+ tmp_header = idx_to_header(chunk, header->next_index);
+ tmp_header->prev_index = idx;
+ }
+
+ chunk->freelists[order] = idx;
+}
+
+static void header_remove_freelist(buddy_chunk_t *chunk,
+ buddy_header_t *header, u8 order)
+{
+ buddy_header_t *tmp_header;
+
+ if (header->prev_index != BUDDY_CHUNK_ITEMS) {
+ tmp_header = idx_to_header(chunk, header->prev_index);
+ tmp_header->next_index = header->next_index;
+ }
+
+ if (header->next_index != BUDDY_CHUNK_ITEMS) {
+ tmp_header = idx_to_header(chunk, header->next_index);
+ tmp_header->prev_index = header->prev_index;
+ }
+
+ /* Pop off the list head if necessary. */
+ if (idx_to_header(chunk, chunk->freelists[order]) == header)
+ chunk->freelists[order] = header->next_index;
+
+ header->prev_index = BUDDY_CHUNK_ITEMS;
+ header->next_index = BUDDY_CHUNK_ITEMS;
+}
+
+static u64 size_to_order(size_t size)
+{
+ u64 order;
+
+ /*
+ * Legal sizes are [1, 4GiB] (the biggest possible arena).
+ * Of course, sizes close to GiB are practically impossible
+ * to fulfill and allocation will fail, but that's taken care
+ * of by the caller.
+ */
+
+ if (unlikely(size == 0 || size > (1UL << 32))) {
+ arena_stderr("illegal size request %lu\n", size);
+ return 64;
+ }
+ /*
+ * To find the order of the allocation we find the first power of two
+ * >= the requested size, take the log2, then adjust it for the minimum
+ * allocation size by removing the minimum shift from it. Requests
+ * smaller than the minimum allocation size are rounded up.
+ */
+ order = arena_fls(arena_next_pow2(size));
+ if (order < BUDDY_MIN_ALLOC_SHIFT)
+ return 0;
+
+ return order - BUDDY_MIN_ALLOC_SHIFT;
+}
+
+__weak
+int add_leftovers_to_freelist(buddy_chunk_t __arg_arena *chunk, u32 cur_idx,
+ u64 min_order, u64 max_order)
+{
+ buddy_header_t *header;
+ u64 ord;
+ u32 idx;
+
+ for (ord = min_order; ord < max_order && can_loop; ord++) {
+ /* Mark the buddy as free and add it to the freelists. */
+ idx = cur_idx + (1 << ord);
+
+ header = idx_to_header(chunk, idx);
+ if (unlikely(!header)) {
+ arena_stderr("idx %u has no header", idx);
+ return -EINVAL;
+ }
+
+ asan_unpoison(header, sizeof(*header));
+
+ idx_set_allocated(chunk, idx, false);
+ header_add_freelist(chunk, header, idx, ord);
+ }
+
+ return 0;
+}
+
+static buddy_chunk_t *buddy_chunk_get(struct buddy *buddy)
+{
+ u64 order, ord, min_order, max_order;
+ buddy_chunk_t *chunk;
+ size_t left;
+ int power2;
+ u64 vaddr;
+ u32 idx;
+ int ret;
+
+ /*
+ * Step 1: Allocate a properly aligned chunk, and
+ * prep it for insertion into the buddy allocator.
+ * We don't need the allocator lock until step 2.
+ */
+
+ ret = buddy_alloc_arena_vaddr(buddy, &vaddr);
+ if (ret)
+ return NULL;
+
+ /* Addresses must be aligned to the chunk boundary. */
+ if (vaddr % BUDDY_CHUNK_BYTES)
+ return NULL;
+
+ /* Unreserve the address space. */
+ bpf_arena_free_pages(&arena, (void __arena *)vaddr,
+ BUDDY_CHUNK_PAGES);
+
+ chunk = bpf_arena_alloc_pages(&arena, (void __arena *)vaddr,
+ BUDDY_CHUNK_PAGES, NUMA_NO_NODE, 0);
+ if (!chunk) {
+ arena_stderr("[ALLOC FAILED]");
+ return NULL;
+ }
+
+ if (buddy_lock(buddy)) {
+ /*
+ * We cannot reclaim the vaddr space, but that is ok - this
+ * operation should always succeed. The error path is to catch
+ * accidental deadlocks that will cause -ENOMEMs to the program as
+ * the allocator fails to refill itself, in which case vaddr usage
+ * is the least of our worries.
+ */
+ bpf_arena_free_pages(&arena, (void __arena *)vaddr, BUDDY_CHUNK_PAGES);
+ return NULL;
+ }
+
+ asan_poison(chunk, BUDDY_POISONED, BUDDY_CHUNK_PAGES * __PAGE_SIZE);
+
+ /* Unpoison the chunk itself. */
+ asan_unpoison(chunk, sizeof(*chunk));
+
+ /* Mark all freelists as empty. */
+ for (ord = zero; ord < BUDDY_CHUNK_NUM_ORDERS && can_loop; ord++)
+ chunk->freelists[ord] = BUDDY_CHUNK_ITEMS;
+
+ /*
+ * Initialize the chunk by carving out a page range to hold the metadata
+ * struct above, then dumping the rest of the pages into the allocator.
+ */
+
+ _Static_assert(BUDDY_CHUNK_PAGES * __PAGE_SIZE >=
+ BUDDY_MIN_ALLOC_BYTES *
+ BUDDY_CHUNK_ITEMS,
+ "chunk must fit within the allocation");
+
+ /*
+ * Step 2: Reserve a chunk for the chunk metadata, then breaks
+ * the rest of the full allocation into the different buckets.
+ * We allocating the memory by grabbing blocks of progressively
+ * smaller sizes from the allocator, which are guaranteed to be
+ * continuous.
+ *
+ * This operation also populates the allocator.
+ *
+ * Algorithm:
+ *
+ * - max_order: The last order allocation we made
+ * - left: How many bytes are left to allocate
+ * - cur_index: Current index into the top-level block we are
+ * allocating from.
+ *
+ * Step 3:
+ * - Find the largest power-of-2 allocation still smaller than left (infimum)
+ * - Reserve a chunk of that size, along with its buddy
+ * - For every order from [infimum + 1, last order), carve out a block
+ * and put it into the allocator.
+ *
+ * Example: Chunk size 0b1010000 (80 bytes)
+ *
+ * Step 1:
+ *
+ * idx infimum 1 << max_order
+ * 0 64 128 1 << 20
+ * |________|_________|______________________|
+ *
+ * Blocks set aside:
+ * [0, 64) - Completely allocated
+ * [64, 128) - Will be further split in the next iteration
+ *
+ * Blocks added to the allocator:
+ * [128, 256)
+ * [256, 512)
+ * ...
+ * [1 << 18, 1 << 19)
+ * [1 << 19, 1 << 20)
+ *
+ * Step 2:
+ *
+ * idx infimum idx + 1 << max_order
+ * 64 80 96 64 + 1 << 6 = 128
+ * |________|_________|______________________|
+ *
+ * Blocks set aside:
+ * [64, 80) - Completely allocated
+ *
+ * Blocks added to the allocator:
+ * [80, 96) - left == 0 so the buddy is unused and marked as freed
+ * [96, 128)
+ */
+ max_order = BUDDY_CHUNK_NUM_ORDERS;
+ left = sizeof(*chunk);
+ idx = 0;
+ while (left && can_loop) {
+ power2 = arena_fls(left);
+ /*
+ * Note: The condition below only triggers to catch serious bugs
+ * early. There is no sane way to undo any block insertions from
+ * the allocated chunk, so just leak any leftover allocations,
+ * emit a diagnostic, unlock and exit.
+ *
+ */
+ if (unlikely(power2 >= BUDDY_CHUNK_NUM_ORDERS)) {
+ arena_stderr(
+ "buddy chunk metadata require allocation of order %d\n",
+ power2);
+ arena_stderr(
+ "chunk has size of 0x%lx bytes (left %lx bytes)\n",
+ sizeof(*chunk), left);
+ buddy_unlock(buddy);
+
+ return NULL;
+ }
+
+ /* Round up allocations that are too small. */
+
+ left -= (power2 >= BUDDY_MIN_ALLOC_SHIFT) ? 1 << power2 : left;
+ order = (power2 >= BUDDY_MIN_ALLOC_SHIFT) ? power2 - BUDDY_MIN_ALLOC_SHIFT : 0;
+
+ idx_set_allocated(chunk, idx, true);
+
+ /*
+ * Starting an order above the one we allocated, populate
+ * the allocator with free blocks. If this is the last
+ * allocation (left == 0), also mark the buddy as free.
+ *
+ * See comment above about error handling: The error path
+ * is only there as a way to mitigate deeply buggy allocator
+ * states by emitting a diagnostic in add_leftovers_to_freelist()
+ * and leaking any memory not added in the freelists.
+ */
+ min_order = left ? order + 1 : order;
+ if (add_leftovers_to_freelist(chunk, idx, min_order, max_order)) {
+ buddy_unlock(buddy);
+ return NULL;
+ }
+
+ /* Adjust the index. */
+ idx += 1 << order;
+ max_order = order;
+ }
+
+ buddy_unlock(buddy);
+
+ return chunk;
+}
+
+__hidden int buddy_init(struct buddy *buddy,
+ arena_spinlock_t __arg_arena __arena *lock)
+{
+ buddy_chunk_t *chunk;
+ int ret;
+
+ buddy->lock = lock;
+
+ if (!asan_ready())
+ return -EINVAL;
+
+ /*
+ * Reserve enough address space to ensure allocations are aligned.
+ */
+ if ((ret = buddy_reserve_arena_vaddr(buddy)))
+ return ret;
+
+ _Static_assert(BUDDY_CHUNK_PAGES > 0,
+ "chunk must use one or more pages");
+
+ chunk = buddy_chunk_get(buddy);
+
+ if (buddy_lock(buddy)) {
+ bpf_arena_free_pages(&arena, chunk, BUDDY_CHUNK_PAGES);
+ return -EINVAL;
+ }
+
+ /* Chunk is already properly unpoisoned if allocated. */
+ if (chunk) {
+ chunk->next = buddy->first_chunk;
+ chunk->prev = NULL;
+ }
+
+ /* Put the chunk at the beginning of the list. */
+ buddy->first_chunk = chunk;
+
+ buddy_unlock(buddy);
+
+ return chunk ? 0 : -ENOMEM;
+}
+
+/*
+ * Destroy the allocator. This does not check whether there are any allocations
+ * currently in use, so any pages being accessed will start taking arena faults.
+ * We do not take a lock because we are freeing arena pages, and nobody should
+ * be using the allocator at that point in the execution.
+ */
+__weak int buddy_destroy(struct buddy *buddy)
+{
+ buddy_chunk_t *chunk, *next;
+
+ if (!buddy)
+ return -EINVAL;
+
+ /*
+ * Traverse all buddy chunks and free them back to the arena
+ * with the same granularity they were allocated with.
+ */
+ for (chunk = buddy->first_chunk; chunk && can_loop; chunk = next) {
+ next = chunk->next;
+
+ /* Wholesale poison the entire block. */
+ asan_poison(chunk, BUDDY_POISONED,
+ BUDDY_CHUNK_PAGES * __PAGE_SIZE);
+ bpf_arena_free_pages(&arena, chunk, BUDDY_CHUNK_PAGES);
+ }
+
+ /* Free up any part of the address space that did not get used. */
+ buddy_unreserve_arena_vaddr(buddy);
+
+ /* Clear all fields. */
+ buddy->first_chunk = NULL;
+
+ return 0;
+}
+
+__weak u64 buddy_chunk_alloc(buddy_chunk_t __arg_arena *chunk,
+ int order_req)
+{
+ buddy_header_t *header, *tmp_header, *next_header;
+ u32 idx, tmpidx, retidx;
+ u64 address;
+ u64 order = 0;
+ u64 i;
+
+ for (order = order_req; order < BUDDY_CHUNK_NUM_ORDERS && can_loop; order++) {
+ if (chunk->freelists[order] != BUDDY_CHUNK_ITEMS)
+ break;
+ }
+
+ if (order >= BUDDY_CHUNK_NUM_ORDERS)
+ return (u64)NULL;
+
+ retidx = chunk->freelists[order];
+ header = idx_to_header(chunk, retidx);
+ chunk->freelists[order] = header->next_index;
+
+ if (header->next_index != BUDDY_CHUNK_ITEMS) {
+ next_header = idx_to_header(chunk, header->next_index);
+ next_header->prev_index = BUDDY_CHUNK_ITEMS;
+ }
+
+ header->prev_index = BUDDY_CHUNK_ITEMS;
+ header->next_index = BUDDY_CHUNK_ITEMS;
+ if (idx_set_order(chunk, retidx, order_req))
+ return (u64)NULL;
+
+ if (idx_set_allocated(chunk, retidx, true))
+ return (u64)NULL;
+
+ /*
+ * Do not unpoison the address yet, will be done by the caller
+ * because the caller has the exact allocation size requested.
+ */
+ address = (u64)idx_to_addr(chunk, retidx);
+
+ /* If we allocated from a larger-order chunk, split the buddies. */
+ for (i = order_req; i < order && can_loop; i++) {
+ /*
+ * Flip the bit for the current order (the bit is guaranteed
+ * to be 0, so just add 1 << i).
+ */
+ idx = retidx + (1 << i);
+
+ /* Add the buddy of the allocation to the free list. */
+ header = idx_to_header(chunk, idx);
+ /* Unpoison the buddy header */
+ asan_unpoison(header, sizeof(*header));
+ if (idx_set_allocated(chunk, idx, false))
+ return (u64)NULL;
+
+ if (idx_set_order(chunk, idx, i))
+ return (u64)NULL;
+
+ /* Push the header to the beginning of the freelists list. */
+ tmpidx = chunk->freelists[i];
+
+ header->prev_index = BUDDY_CHUNK_ITEMS;
+ header->next_index = tmpidx;
+
+ if (tmpidx != BUDDY_CHUNK_ITEMS) {
+ tmp_header = idx_to_header(chunk, tmpidx);
+ tmp_header->prev_index = idx;
+ }
+
+ chunk->freelists[i] = idx;
+ }
+
+ return address;
+}
+
+/* Scan the existing chunks for available memory. */
+static u64 buddy_alloc_from_existing_chunks(struct buddy *buddy, int order)
+{
+ buddy_chunk_t *chunk;
+ u64 address;
+
+ for (chunk = buddy->first_chunk; chunk != NULL && can_loop;
+ chunk = chunk->next) {
+ address = buddy_chunk_alloc(chunk, order);
+ if (address)
+ return address;
+ }
+
+ return (u64)NULL;
+}
+
+/*
+ * Try an allocation from a newly allocated chunk. Also
+ * incorporate the chunk into the linked list.
+ */
+static u64 buddy_alloc_from_new_chunk(struct buddy *buddy, buddy_chunk_t *chunk, int order)
+{
+ u64 address;
+
+ if (buddy_lock(buddy))
+ return (u64)NULL;
+
+
+ /*
+ * Add the chunk into the allocator and try
+ * to allocate specifically from that chunk.
+ */
+ chunk->next = buddy->first_chunk;
+ chunk->prev = NULL;
+ buddy->first_chunk = chunk;
+
+ address = buddy_chunk_alloc(buddy->first_chunk, order);
+
+ buddy_unlock(buddy);
+
+ return (u64)address;
+}
+__weak
+u64 buddy_alloc_internal(struct buddy *buddy, size_t size)
+{
+ buddy_chunk_t *chunk;
+ u64 address = (u64)NULL;
+ int order;
+
+ if (!buddy)
+ return (u64)NULL;
+
+ order = size_to_order(size);
+ if (order >= BUDDY_CHUNK_NUM_ORDERS || order < 0) {
+ arena_stderr("invalid order %d (sz %lu)\n", order, size);
+ return (u64)NULL;
+ }
+
+ if (buddy_lock(buddy))
+ return (u64)NULL;
+
+ address = buddy_alloc_from_existing_chunks(buddy, order);
+ buddy_unlock(buddy);
+ if (address)
+ goto done;
+
+ /* Get a new chunk. */
+ chunk = buddy_chunk_get(buddy);
+ if (chunk)
+ address = buddy_alloc_from_new_chunk(buddy, chunk, order);
+
+done:
+ /* If we failed to allocate memory, return NULL. */
+ if (!address)
+ return (u64)NULL;
+
+ /*
+ * Unpoison exactly the amount of bytes requested. If the
+ * data is smaller than the header, we must poison any
+ * unused bytes that were part of the header.
+ */
+ if (size < BUDDY_HEADER_OFF + sizeof(buddy_header_t))
+ asan_poison((u8 __arena *)address + BUDDY_HEADER_OFF,
+ BUDDY_POISONED, sizeof(buddy_header_t));
+
+ asan_unpoison((u8 __arena *)address, size);
+
+ return address;
+}
+
+static __always_inline int buddy_free_unlocked(struct buddy *buddy, u64 addr)
+{
+ buddy_header_t *header, *buddy_header;
+ u64 idx, buddy_idx, tmp_idx;
+ buddy_chunk_t *chunk;
+ bool allocated;
+ u8 order;
+
+ if (!buddy)
+ return -EINVAL;
+
+ if (addr & (BUDDY_MIN_ALLOC_BYTES - 1)) {
+ arena_stderr("Freeing unaligned address %llx\n", addr);
+ return -EINVAL;
+ }
+
+ /* Get (chunk, idx) out of the address. */
+ chunk = (void __arena *)(addr & ~BUDDY_CHUNK_OFFSET_MASK);
+ idx = (addr & BUDDY_CHUNK_OFFSET_MASK) / BUDDY_MIN_ALLOC_BYTES;
+
+ /* Mark the block as unallocated so we can access the header. */
+ idx_set_allocated(chunk, idx, false);
+
+ order = idx_get_order(chunk, idx);
+ header = idx_to_header(chunk, idx);
+
+ /* The header is in the block itself, keep it unpoisoned. */
+ asan_poison((u8 __arena *)addr, BUDDY_POISONED,
+ BUDDY_MIN_ALLOC_BYTES << order);
+ asan_unpoison(header, sizeof(*header));
+
+ /*
+ * Coalescing loop. Merge with free buddies of equal order.
+ * For every coalescing step, keep the left buddy and
+ * drop the right buddy's header.
+ */
+ for (; order < BUDDY_CHUNK_NUM_ORDERS && can_loop; order++) {
+ buddy_idx = idx ^ (1 << order);
+
+ /* Check if the buddy is actually free. */
+ idx_is_allocated(chunk, buddy_idx, &allocated);
+ if (allocated)
+ break;
+
+ /*
+ * If buddy is not the same order as the chunk
+ * being freed, then we're done coalescing.
+ */
+ if (idx_get_order(chunk, buddy_idx) != order)
+ break;
+
+ buddy_header = idx_to_header(chunk, buddy_idx);
+ header_remove_freelist(chunk, buddy_header, order);
+
+ /* Keep the left header out of the two buddies, drop the other one. */
+ if (buddy_idx < idx) {
+ tmp_idx = idx;
+ idx = buddy_idx;
+ buddy_idx = tmp_idx;
+ }
+
+ /* Remove the buddy from the freelists so that we can merge it. */
+ idx_set_order(chunk, buddy_idx, order);
+
+ buddy_header = idx_to_header(chunk, buddy_idx);
+ asan_poison(buddy_header, BUDDY_POISONED,
+ sizeof(*buddy_header));
+ }
+
+ /* Header properly freed but not in any freelists yet .*/
+ idx_set_order(chunk, idx, order);
+
+ header = idx_to_header(chunk, idx);
+ header_add_freelist(chunk, header, idx, order);
+
+ return 0;
+}
+
+__weak int buddy_free_internal(struct buddy *buddy, u64 addr)
+{
+ int ret;
+
+ if (!buddy)
+ return -EINVAL;
+
+ if ((ret = buddy_lock(buddy)))
+ return ret;
+
+ buddy_free_unlocked(buddy, addr);
+
+ buddy_unlock(buddy);
+
+ return 0;
+}
+
+__weak char _license[] SEC("license") = "GPL";
--
2.53.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* Re: [PATCH bpf-next v4 8/9] selftests/bpf: Add buddy allocator for libarena
2026-04-07 4:57 ` [PATCH bpf-next v4 8/9] selftests/bpf: Add buddy allocator for libarena Emil Tsalapatis
@ 2026-04-07 5:43 ` bot+bpf-ci
2026-04-07 17:07 ` Alexei Starovoitov
1 sibling, 0 replies; 19+ messages in thread
From: bot+bpf-ci @ 2026-04-07 5:43 UTC (permalink / raw)
To: emil, bpf
Cc: ast, andrii, memxor, daniel, eddyz87, song, emil, ast, andrii,
daniel, martin.lau, eddyz87, yonghong.song, clm, ihor.solodrai
[-- Attachment #1: Type: text/plain, Size: 2041 bytes --]
> diff --git a/tools/testing/selftests/bpf/libarena/src/buddy.bpf.c b/tools/testing/selftests/bpf/libarena/src/buddy.bpf.c
> new file mode 100644
> index 000000000000..4343e39760ab
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/libarena/src/buddy.bpf.c
[ ... ]
> +static u8 idx_get_order(buddy_chunk_t *chunk, u64 idx)
> +{
> + u8 result;
> +
> + _Static_assert(BUDDY_CHUNK_NUM_ORDERS <= 16,
> + "order must fit in 4 bits");
> +
> + if (unlikely(idx >= BUDDY_CHUNK_ITEMS)) {
> + arena_stderr("setting order of invalid idx\n");
> + return BUDDY_CHUNK_NUM_ORDERS;
> + }
This is a getter function, but the error message says "setting order".
Looks like it was copied from idx_set_order().
> +
> + result = chunk->orders[idx / 2];
> +
> + return (idx & 0x1) ? (result & 0xf) : (result >> 4);
> +}
> +
> +static void __arena *idx_to_addr(buddy_chunk_t *chunk, size_t idx)
> +{
> + u64 address;
> +
> + if (unlikely(idx >= BUDDY_CHUNK_ITEMS)) {
> + arena_stderr("setting order of invalid idx\n");
> + return NULL;
> + }
Same issue here -- idx_to_addr() converts an index to an address, but
the error message says "setting order of invalid idx".
[ ... ]
> +__weak int buddy_free_internal(struct buddy *buddy, u64 addr)
> +{
> + int ret;
> +
> + if (!buddy)
> + return -EINVAL;
> +
> + if ((ret = buddy_lock(buddy)))
> + return ret;
> +
> + buddy_free_unlocked(buddy, addr);
Should the return value of buddy_free_unlocked() be captured here?
It returns -EINVAL on error (e.g., unaligned address), but
buddy_free_internal() always returns 0 after a successful lock
acquisition regardless of whether the free itself succeeded.
Something like:
ret = buddy_free_unlocked(buddy, addr);
buddy_unlock(buddy);
return ret;
> +
> + buddy_unlock(buddy);
> +
> + return 0;
> +}
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/24065579421
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PATCH bpf-next v4 8/9] selftests/bpf: Add buddy allocator for libarena
2026-04-07 4:57 ` [PATCH bpf-next v4 8/9] selftests/bpf: Add buddy allocator for libarena Emil Tsalapatis
2026-04-07 5:43 ` bot+bpf-ci
@ 2026-04-07 17:07 ` Alexei Starovoitov
1 sibling, 0 replies; 19+ messages in thread
From: Alexei Starovoitov @ 2026-04-07 17:07 UTC (permalink / raw)
To: Emil Tsalapatis
Cc: bpf, Alexei Starovoitov, Andrii Nakryiko, Kumar Kartikeya Dwivedi,
Daniel Borkmann, Eduard, Song Liu
On Mon, Apr 6, 2026 at 9:57 PM Emil Tsalapatis <emil@etsalapatis.com> wrote:
>
> Add a byte-oriented buddy allocator for libarena. The buddy
> allocator provides an alloc/free interface for small arena
> allocations (down to 16 bytes).
what happens to allocation request of < 16 bytes?
and what is the max ?
> Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
> ---
> .../selftests/bpf/libarena/include/buddy.h | 64 ++
> .../selftests/bpf/libarena/src/buddy.bpf.c | 879 ++++++++++++++++++
> 2 files changed, 943 insertions(+)
> create mode 100644 tools/testing/selftests/bpf/libarena/include/buddy.h
> create mode 100644 tools/testing/selftests/bpf/libarena/src/buddy.bpf.c
>
> diff --git a/tools/testing/selftests/bpf/libarena/include/buddy.h b/tools/testing/selftests/bpf/libarena/include/buddy.h
> new file mode 100644
> index 000000000000..aa63752bc657
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/libarena/include/buddy.h
> @@ -0,0 +1,64 @@
> +// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
> +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
> +#pragma once
> +
> +/* Buddy allocator-related structs. */
not a useful comment.
> +
> +struct buddy_chunk;
> +typedef struct buddy_chunk __arena buddy_chunk_t;
> +
> +struct buddy_header;
> +typedef struct buddy_header __arena buddy_header_t;
> +
> +enum buddy_consts {
> + BUDDY_MIN_ALLOC_SHIFT = 4,
> + BUDDY_MIN_ALLOC_BYTES = 1 << BUDDY_MIN_ALLOC_SHIFT,
> + BUDDY_CHUNK_NUM_ORDERS = 1 << 4, /* 4 bits per order */
this 4 is the same as BUDDY_MIN_ALLOC_SHIFT or can be different?
> + BUDDY_CHUNK_BYTES = BUDDY_MIN_ALLOC_BYTES << BUDDY_CHUNK_NUM_ORDERS,
> + BUDDY_HEADER_OFF = 8, /* header byte offset, see buddy.bpf.c for details */
sizeof(struct buddy_header) instead?
> + BUDDY_CHUNK_PAGES = BUDDY_CHUNK_BYTES / __PAGE_SIZE,
> + BUDDY_CHUNK_ITEMS = 1 << BUDDY_CHUNK_NUM_ORDERS,
> + BUDDY_CHUNK_OFFSET_MASK = BUDDY_CHUNK_BYTES - 1,
> + BUDDY_VADDR_OFFSET = BUDDY_CHUNK_BYTES, /* Start aligning at chunk */
> + BUDDY_VADDR_SIZE = BUDDY_CHUNK_BYTES << 10 /* 1024 chunks maximum */
I tried to make sense of these constants in isolation and got
completely lost.
> +};
> +
> +struct buddy_header {
> + u32 prev_index; /* "Pointer" to the previous available allocation of the same size. */
> + u32 next_index; /* Same for the next allocation. */
> +};
> +
> +/*
> + * We bring memory into the allocator 1MiB at a time.
> + */
> +struct buddy_chunk {
> + /* The order of the current allocation for a item. 4 bits per order. */
> + u8 orders[BUDDY_CHUNK_ITEMS / 2];
> + /*
> + * Bit to denote whether chunk is allocated. Size of the allocated/free
> + * chunk found from the orders array.
> + */
> + u8 allocated[BUDDY_CHUNK_ITEMS / 8];
> + /* Freelists for O(1) allocation. */
> + u64 freelists[BUDDY_CHUNK_NUM_ORDERS];
> + buddy_chunk_t *prev;
> + buddy_chunk_t *next;
> +};
> +
> +struct buddy {
> + buddy_chunk_t *first_chunk; /* Pointer to the chunk linked list. */
> + arena_spinlock_t __arena *lock; /* Allocator lock */
> + u64 vaddr; /* Allocation into reserved vaddr */
> +};
ohh. so the main 'buddy' object is not inside arena and
that's why you need a pointer to a lock instead of directly
embedding the lock in struct buddy?
Can it move to global arena data instead?
> +
> +#ifdef __BPF__
> +
> +int buddy_init(struct buddy *buddy, arena_spinlock_t __arena *lock);
> +int buddy_destroy(struct buddy *buddy);
> +int buddy_free_internal(struct buddy *buddy, u64 free);
> +#define buddy_free(buddy, ptr) do { buddy_free_internal((buddy), (u64)(ptr)); } while (0)
> +u64 buddy_alloc_internal(struct buddy *buddy, size_t size);
> +#define buddy_alloc(alloc, size) ((void __arena *)buddy_alloc_internal((alloc), (size)))
> +
> +
> +#endif /* __BPF__ */
> diff --git a/tools/testing/selftests/bpf/libarena/src/buddy.bpf.c b/tools/testing/selftests/bpf/libarena/src/buddy.bpf.c
> new file mode 100644
> index 000000000000..4343e39760ab
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/libarena/src/buddy.bpf.c
> @@ -0,0 +1,879 @@
> +// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
> +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
> +
> +#include <common.h>
> +#include <asan.h>
> +#include <buddy.h>
> +
> +/*
> + * Buddy allocator arena-based implementation.
> + *
> + * Memory is organized into chunks. These chunks
> + * cannot be coalesced or split. Allocating
> + * chunks allocates their memory eagerly.
> + *
> + * Internally, each chunk is organized into blocks.
> + * Blocks _can_ be coalesced/split, but only inside
> + * the chunk. Each block can be allocated or
> + * unallocated. If allocated, the entire block holds
> + * user data. If unallocated, the block is mostly
> + * invalid memory, with the exception of a header
> + * used for freelist tracking.
> + *
> + * The header is placed at an offset inside the block
> + * to prevent off-by-one errors from the previous block
> + * from trivially overwriting the header. Such an error
> + * is also not catchable by ASAN, since the header remains
> + * valid memory even after the block is freed. It is still
> + * theoretically possible for the header to be corrupted
> + * without being caught by ASAN, but harder.
> + *
> + * Since the allocator needs to track order information for
> + * both allocated and free blocks, and allocated blocks cannot
> + * store a header, the allocator also stores per-chunk order
> + * information in a reserved region at the beginning of the
> + * chunk. The header includes a bitmap with the order of blocks
> + * and their allocation state. It also includes the freelist
> + * heads for the allocation itself.
> + */
> +
> +
> +enum {
> + BUDDY_POISONED = (s8)0xef,
> +};
> +
> +static inline int buddy_lock(struct buddy *buddy)
> +{
> + return arena_spin_lock(buddy->lock);
> +}
> +
> +static inline void buddy_unlock(struct buddy *buddy)
> +{
> + arena_spin_unlock(buddy->lock);
> +}
> +
> +/*
> + * Reserve part of the arena address space for the allocator. We use
> + * this to get aligned addresses for the chunks, since the arena
> + * page alloc kfuncs do not support alignment.
what kind of alignment is needed? Like align to 1Mbyte or something?
> + */
> +static int buddy_reserve_arena_vaddr(struct buddy *buddy)
> +{
> + buddy->vaddr = 0;
> +
> + return bpf_arena_reserve_pages(&arena,
> + (void __arena *)BUDDY_VADDR_OFFSET,
> + BUDDY_VADDR_SIZE / __PAGE_SIZE);
> +}
> +
> +/*
> + * Free up any unused address space. Used only during teardown.
> + */
> +static void buddy_unreserve_arena_vaddr(struct buddy *buddy)
> +{
> + bpf_arena_free_pages(
> + &arena, (void __arena *)(BUDDY_VADDR_OFFSET + buddy->vaddr),
> + (BUDDY_VADDR_SIZE - buddy->vaddr) / __PAGE_SIZE);
> +
> + buddy->vaddr = 0;
> +}
> +
> +/*
> + * Carve out part of the reserved address space and hand it over
> + * to the buddy allocator.
> + *
> + * We are assuming the buddy allocator is the only allocator in the
> + * system, so there is no race between this function reserving a
> + * page range and some other allocator actually making the BPF call
> + * to really create and reserve it.
> + */
> +__weak int buddy_alloc_arena_vaddr(struct buddy *buddy, u64 *vaddrp)
> +{
> + u64 vaddr, old, new;
> +
> + if (!buddy || !vaddrp)
> + return -EINVAL;
> +
> + do {
> + vaddr = buddy->vaddr;
> + new = vaddr + BUDDY_CHUNK_BYTES;
> +
> + if (new > BUDDY_VADDR_SIZE)
> + return -EINVAL;
> +
> + old = __sync_val_compare_and_swap(&buddy->vaddr, vaddr, new);
> + } while (old != vaddr && can_loop);
the comment says that there is no race, then why retry loop?
> +
> + if (old != vaddr)
> + return -EINVAL;
> +
> + *vaddrp = BUDDY_VADDR_OFFSET + vaddr;
> +
> + return 0;
> +}
> +
> +static u64 arena_next_pow2(__u64 n)
> +{
> + n--;
> + n |= n >> 1;
> + n |= n >> 2;
> + n |= n >> 4;
> + n |= n >> 8;
> + n |= n >> 16;
> + n |= n >> 32;
> + n++;
> +
> + return n;
> +}
> +
> +__weak
> +int idx_set_allocated(buddy_chunk_t __arg_arena *chunk, u64 idx, bool allocated)
> +{
> + if (unlikely(idx >= BUDDY_CHUNK_ITEMS)) {
> + arena_stderr("setting state of of invalid idx (%d, max %d)\n", idx,
> + BUDDY_CHUNK_ITEMS);
> + return -EINVAL;
> + }
> +
> + if (allocated)
> + chunk->allocated[idx / 8] |= 1 << (idx % 8);
> + else
> + chunk->allocated[idx / 8] &= ~(1 << (idx % 8));
> +
> + return 0;
> +}
> +
> +static int idx_is_allocated(buddy_chunk_t *chunk, u64 idx, bool *allocated)
> +{
> + if (unlikely(idx >= BUDDY_CHUNK_ITEMS)) {
> + arena_stderr("getting state of invalid idx (%llu, max %d)\n", idx,
> + BUDDY_CHUNK_ITEMS);
> + return -EINVAL;
> + }
> +
> + *allocated = chunk->allocated[idx / 8] & (1 << (idx % 8));
> + return 0;
> +}
> +
> +__weak
> +int idx_set_order(buddy_chunk_t __arg_arena *chunk, u64 idx, u8 order)
larger than 256 orders will be silently truncated by the compiler.
Use u32?
> +{
> + u8 prev_order;
> +
> + if (unlikely(order >= BUDDY_CHUNK_NUM_ORDERS)) {
> + arena_stderr("setting invalid order %u\n", order);
> + return -EINVAL;
> + }
> +
> + if (unlikely(idx >= BUDDY_CHUNK_ITEMS)) {
> + arena_stderr("setting order of invalid idx (%d, max %d)\n", idx,
> + BUDDY_CHUNK_ITEMS);
> + return -EINVAL;
> + }
> +
> + /*
> + * We store two order instances per byte, one per nibble.
> + * Retain the existing nibble.
> + */
> + prev_order = chunk->orders[idx / 2];
> + if (idx & 0x1) {
> + order &= 0xf;
> + order |= (prev_order & 0xf0);
> + } else {
> + order <<= 4;
> + order |= (prev_order & 0xf);
> + }
> +
> + chunk->orders[idx / 2] = order;
> +
> + return 0;
> +}
> +
> +static u8 idx_get_order(buddy_chunk_t *chunk, u64 idx)
> +{
> + u8 result;
> +
> + _Static_assert(BUDDY_CHUNK_NUM_ORDERS <= 16,
> + "order must fit in 4 bits");
> +
> + if (unlikely(idx >= BUDDY_CHUNK_ITEMS)) {
> + arena_stderr("setting order of invalid idx\n");
> + return BUDDY_CHUNK_NUM_ORDERS;
> + }
> +
> + result = chunk->orders[idx / 2];
> +
> + return (idx & 0x1) ? (result & 0xf) : (result >> 4);
> +}
> +
> +static void __arena *idx_to_addr(buddy_chunk_t *chunk, size_t idx)
> +{
> + u64 address;
> +
> + if (unlikely(idx >= BUDDY_CHUNK_ITEMS)) {
> + arena_stderr("setting order of invalid idx\n");
> + return NULL;
> + }
> +
> + /*
> + * The data blocks start in the chunk after the metadata block.
> + * We find the actual address by indexing into the region at an
> + * BUDDY_MIN_ALLOC_BYTES granularity, the minimum allowed.
> + * The index number already accounts for the fact that the first
> + * blocks in the chunk are occupied by the metadata, so we do
> + * not need to offset it.
> + */
> +
> + address = (u64)chunk + (idx * BUDDY_MIN_ALLOC_BYTES);
> +
> + return (void __arena *)address;
> +}
> +
> +static buddy_header_t *idx_to_header(buddy_chunk_t *chunk, size_t idx)
> +{
> + bool allocated;
> + u64 address;
> +
> + if (unlikely(idx_is_allocated(chunk, idx, &allocated))) {
> + arena_stderr("accessing invalid idx 0x%lx\n", idx);
> + return NULL;
> + }
> +
> + if (unlikely(allocated)) {
> + arena_stderr("accessing allocated idx 0x%lx as header\n", idx);
> + return NULL;
> + }
> +
> + address = (u64)idx_to_addr(chunk, idx);
> +
> + /*
> + * Offset the header within the block. This avoids accidental overwrites
> + * to the header because of off-by-one errors when using adjacent blocks.
> + *
> + * The offset has been chosen as a compromise between ASAN effectiveness
> + * and allocator granularity:
> + * 1) ASAN dictates valid data runs are 8-byte aligned.
> + * 2) We want to keep a low minimum allocation size (currently 16).
> + *
> + * As a result, we have only two possible positions for the header: Bytes
> + * 0 and 8. Keeping the header in byte 0 means off-by-ones from the previous
> + * block touch the header, and, since the header must be accessible, ASAN
> + * will not trigger. Keeping the header on byte 8 means off-by-one errors from
> + * the previous block are caught by ASAN. Negative offsets are rarer, so
> + * while accesses into the block from the next block are possible, they are
> + * less probable.
> + */
> +
> + return (buddy_header_t *)(address + BUDDY_HEADER_OFF);
> +}
> +
> +static void header_add_freelist(buddy_chunk_t *chunk,
double space
> + buddy_header_t *header, u64 idx, u8 order)
> +{
> + buddy_header_t *tmp_header;
> +
> + idx_set_order(chunk, idx, order);
> +
> + header->next_index = chunk->freelists[order];
> + header->prev_index = BUDDY_CHUNK_ITEMS;
BUDDY_CHUNK_ITEMS is used like NULL ?
The header is u32 + u32. Is it really worth saving 8 bytes?
Use two proper pointers?
How many headers are there?
> +
> + if (header->next_index != BUDDY_CHUNK_ITEMS) {
> + tmp_header = idx_to_header(chunk, header->next_index);
> + tmp_header->prev_index = idx;
> + }
> +
> + chunk->freelists[order] = idx;
> +}
> +
> +static void header_remove_freelist(buddy_chunk_t *chunk,
> + buddy_header_t *header, u8 order)
> +{
> + buddy_header_t *tmp_header;
> +
> + if (header->prev_index != BUDDY_CHUNK_ITEMS) {
> + tmp_header = idx_to_header(chunk, header->prev_index);
> + tmp_header->next_index = header->next_index;
> + }
> +
> + if (header->next_index != BUDDY_CHUNK_ITEMS) {
> + tmp_header = idx_to_header(chunk, header->next_index);
> + tmp_header->prev_index = header->prev_index;
> + }
> +
> + /* Pop off the list head if necessary. */
> + if (idx_to_header(chunk, chunk->freelists[order]) == header)
> + chunk->freelists[order] = header->next_index;
> +
> + header->prev_index = BUDDY_CHUNK_ITEMS;
> + header->next_index = BUDDY_CHUNK_ITEMS;
> +}
> +
> +static u64 size_to_order(size_t size)
> +{
> + u64 order;
> +
> + /*
> + * Legal sizes are [1, 4GiB] (the biggest possible arena).
> + * Of course, sizes close to GiB are practically impossible
> + * to fulfill and allocation will fail, but that's taken care
> + * of by the caller.
> + */
> +
> + if (unlikely(size == 0 || size > (1UL << 32))) {
> + arena_stderr("illegal size request %lu\n", size);
> + return 64;
> + }
> + /*
> + * To find the order of the allocation we find the first power of two
> + * >= the requested size, take the log2, then adjust it for the minimum
> + * allocation size by removing the minimum shift from it. Requests
> + * smaller than the minimum allocation size are rounded up.
> + */
> + order = arena_fls(arena_next_pow2(size));
> + if (order < BUDDY_MIN_ALLOC_SHIFT)
> + return 0;
> +
> + return order - BUDDY_MIN_ALLOC_SHIFT;
> +}
> +
> +__weak
> +int add_leftovers_to_freelist(buddy_chunk_t __arg_arena *chunk, u32 cur_idx,
> + u64 min_order, u64 max_order)
> +{
> + buddy_header_t *header;
> + u64 ord;
> + u32 idx;
> +
> + for (ord = min_order; ord < max_order && can_loop; ord++) {
> + /* Mark the buddy as free and add it to the freelists. */
> + idx = cur_idx + (1 << ord);
> +
> + header = idx_to_header(chunk, idx);
> + if (unlikely(!header)) {
> + arena_stderr("idx %u has no header", idx);
> + return -EINVAL;
> + }
> +
> + asan_unpoison(header, sizeof(*header));
> +
> + idx_set_allocated(chunk, idx, false);
> + header_add_freelist(chunk, header, idx, ord);
> + }
> +
> + return 0;
> +}
> +
> +static buddy_chunk_t *buddy_chunk_get(struct buddy *buddy)
> +{
> + u64 order, ord, min_order, max_order;
> + buddy_chunk_t *chunk;
> + size_t left;
> + int power2;
> + u64 vaddr;
> + u32 idx;
> + int ret;
> +
> + /*
> + * Step 1: Allocate a properly aligned chunk, and
> + * prep it for insertion into the buddy allocator.
> + * We don't need the allocator lock until step 2.
> + */
> +
> + ret = buddy_alloc_arena_vaddr(buddy, &vaddr);
> + if (ret)
> + return NULL;
> +
> + /* Addresses must be aligned to the chunk boundary. */
> + if (vaddr % BUDDY_CHUNK_BYTES)
> + return NULL;
> +
> + /* Unreserve the address space. */
> + bpf_arena_free_pages(&arena, (void __arena *)vaddr,
> + BUDDY_CHUNK_PAGES);
> +
> + chunk = bpf_arena_alloc_pages(&arena, (void __arena *)vaddr,
> + BUDDY_CHUNK_PAGES, NUMA_NO_NODE, 0);
> + if (!chunk) {
> + arena_stderr("[ALLOC FAILED]");
> + return NULL;
> + }
> +
> + if (buddy_lock(buddy)) {
> + /*
> + * We cannot reclaim the vaddr space, but that is ok - this
> + * operation should always succeed. The error path is to catch
> + * accidental deadlocks that will cause -ENOMEMs to the program as
> + * the allocator fails to refill itself, in which case vaddr usage
> + * is the least of our worries.
> + */
> + bpf_arena_free_pages(&arena, (void __arena *)vaddr, BUDDY_CHUNK_PAGES);
> + return NULL;
> + }
> +
> + asan_poison(chunk, BUDDY_POISONED, BUDDY_CHUNK_PAGES * __PAGE_SIZE);
> +
> + /* Unpoison the chunk itself. */
> + asan_unpoison(chunk, sizeof(*chunk));
> +
> + /* Mark all freelists as empty. */
> + for (ord = zero; ord < BUDDY_CHUNK_NUM_ORDERS && can_loop; ord++)
> + chunk->freelists[ord] = BUDDY_CHUNK_ITEMS;
BUDDY_CHUNK_ITEMS sentinel makes it harder to undestand.
Why not zero ? and normal pointers ?
> +
> + /*
> + * Initialize the chunk by carving out a page range to hold the metadata
> + * struct above, then dumping the rest of the pages into the allocator.
> + */
> +
> + _Static_assert(BUDDY_CHUNK_PAGES * __PAGE_SIZE >=
> + BUDDY_MIN_ALLOC_BYTES *
> + BUDDY_CHUNK_ITEMS,
> + "chunk must fit within the allocation");
> +
> + /*
> + * Step 2: Reserve a chunk for the chunk metadata, then breaks
> + * the rest of the full allocation into the different buckets.
> + * We allocating the memory by grabbing blocks of progressively
> + * smaller sizes from the allocator, which are guaranteed to be
> + * continuous.
> + *
> + * This operation also populates the allocator.
> + *
> + * Algorithm:
> + *
> + * - max_order: The last order allocation we made
> + * - left: How many bytes are left to allocate
> + * - cur_index: Current index into the top-level block we are
> + * allocating from.
> + *
> + * Step 3:
> + * - Find the largest power-of-2 allocation still smaller than left (infimum)
> + * - Reserve a chunk of that size, along with its buddy
> + * - For every order from [infimum + 1, last order), carve out a block
> + * and put it into the allocator.
> + *
> + * Example: Chunk size 0b1010000 (80 bytes)
> + *
> + * Step 1:
> + *
> + * idx infimum 1 << max_order
> + * 0 64 128 1 << 20
> + * |________|_________|______________________|
> + *
> + * Blocks set aside:
> + * [0, 64) - Completely allocated
> + * [64, 128) - Will be further split in the next iteration
> + *
> + * Blocks added to the allocator:
> + * [128, 256)
> + * [256, 512)
> + * ...
> + * [1 << 18, 1 << 19)
> + * [1 << 19, 1 << 20)
> + *
> + * Step 2:
> + *
> + * idx infimum idx + 1 << max_order
> + * 64 80 96 64 + 1 << 6 = 128
> + * |________|_________|______________________|
> + *
> + * Blocks set aside:
> + * [64, 80) - Completely allocated
> + *
> + * Blocks added to the allocator:
> + * [80, 96) - left == 0 so the buddy is unused and marked as freed
> + * [96, 128)
> + */
> + max_order = BUDDY_CHUNK_NUM_ORDERS;
> + left = sizeof(*chunk);
> + idx = 0;
> + while (left && can_loop) {
> + power2 = arena_fls(left);
> + /*
> + * Note: The condition below only triggers to catch serious bugs
> + * early. There is no sane way to undo any block insertions from
> + * the allocated chunk, so just leak any leftover allocations,
> + * emit a diagnostic, unlock and exit.
> + *
> + */
> + if (unlikely(power2 >= BUDDY_CHUNK_NUM_ORDERS)) {
> + arena_stderr(
> + "buddy chunk metadata require allocation of order %d\n",
> + power2);
> + arena_stderr(
> + "chunk has size of 0x%lx bytes (left %lx bytes)\n",
> + sizeof(*chunk), left);
> + buddy_unlock(buddy);
> +
> + return NULL;
> + }
> +
> + /* Round up allocations that are too small. */
> +
> + left -= (power2 >= BUDDY_MIN_ALLOC_SHIFT) ? 1 << power2 : left;
> + order = (power2 >= BUDDY_MIN_ALLOC_SHIFT) ? power2 - BUDDY_MIN_ALLOC_SHIFT : 0;
> +
> + idx_set_allocated(chunk, idx, true);
> +
> + /*
> + * Starting an order above the one we allocated, populate
> + * the allocator with free blocks. If this is the last
> + * allocation (left == 0), also mark the buddy as free.
> + *
> + * See comment above about error handling: The error path
> + * is only there as a way to mitigate deeply buggy allocator
> + * states by emitting a diagnostic in add_leftovers_to_freelist()
> + * and leaking any memory not added in the freelists.
> + */
> + min_order = left ? order + 1 : order;
> + if (add_leftovers_to_freelist(chunk, idx, min_order, max_order)) {
> + buddy_unlock(buddy);
> + return NULL;
> + }
> +
> + /* Adjust the index. */
> + idx += 1 << order;
> + max_order = order;
> + }
> +
> + buddy_unlock(buddy);
> +
> + return chunk;
> +}
> +
> +__hidden int buddy_init(struct buddy *buddy,
> + arena_spinlock_t __arg_arena __arena *lock)
> +{
> + buddy_chunk_t *chunk;
> + int ret;
> +
> + buddy->lock = lock;
> +
> + if (!asan_ready())
> + return -EINVAL;
> +
> + /*
> + * Reserve enough address space to ensure allocations are aligned.
> + */
> + if ((ret = buddy_reserve_arena_vaddr(buddy)))
> + return ret;
> +
> + _Static_assert(BUDDY_CHUNK_PAGES > 0,
> + "chunk must use one or more pages");
> +
> + chunk = buddy_chunk_get(buddy);
> +
> + if (buddy_lock(buddy)) {
> + bpf_arena_free_pages(&arena, chunk, BUDDY_CHUNK_PAGES);
> + return -EINVAL;
> + }
> +
> + /* Chunk is already properly unpoisoned if allocated. */
> + if (chunk) {
> + chunk->next = buddy->first_chunk;
> + chunk->prev = NULL;
> + }
> +
> + /* Put the chunk at the beginning of the list. */
> + buddy->first_chunk = chunk;
> +
> + buddy_unlock(buddy);
> +
> + return chunk ? 0 : -ENOMEM;
> +}
> +
> +/*
> + * Destroy the allocator. This does not check whether there are any allocations
> + * currently in use, so any pages being accessed will start taking arena faults.
> + * We do not take a lock because we are freeing arena pages, and nobody should
> + * be using the allocator at that point in the execution.
> + */
> +__weak int buddy_destroy(struct buddy *buddy)
> +{
> + buddy_chunk_t *chunk, *next;
> +
> + if (!buddy)
> + return -EINVAL;
> +
> + /*
> + * Traverse all buddy chunks and free them back to the arena
> + * with the same granularity they were allocated with.
> + */
> + for (chunk = buddy->first_chunk; chunk && can_loop; chunk = next) {
> + next = chunk->next;
> +
> + /* Wholesale poison the entire block. */
> + asan_poison(chunk, BUDDY_POISONED,
> + BUDDY_CHUNK_PAGES * __PAGE_SIZE);
> + bpf_arena_free_pages(&arena, chunk, BUDDY_CHUNK_PAGES);
> + }
> +
> + /* Free up any part of the address space that did not get used. */
> + buddy_unreserve_arena_vaddr(buddy);
> +
> + /* Clear all fields. */
> + buddy->first_chunk = NULL;
> +
> + return 0;
> +}
> +
> +__weak u64 buddy_chunk_alloc(buddy_chunk_t __arg_arena *chunk,
> + int order_req)
> +{
> + buddy_header_t *header, *tmp_header, *next_header;
> + u32 idx, tmpidx, retidx;
> + u64 address;
> + u64 order = 0;
> + u64 i;
> +
> + for (order = order_req; order < BUDDY_CHUNK_NUM_ORDERS && can_loop; order++) {
> + if (chunk->freelists[order] != BUDDY_CHUNK_ITEMS)
> + break;
> + }
> +
> + if (order >= BUDDY_CHUNK_NUM_ORDERS)
> + return (u64)NULL;
> +
> + retidx = chunk->freelists[order];
> + header = idx_to_header(chunk, retidx);
> + chunk->freelists[order] = header->next_index;
> +
> + if (header->next_index != BUDDY_CHUNK_ITEMS) {
> + next_header = idx_to_header(chunk, header->next_index);
> + next_header->prev_index = BUDDY_CHUNK_ITEMS;
> + }
if everything is a pointer the whole algorithm would be so much easier
to understand. Is it worth it to keep indices vs pointers?
Please do the math and explain the savings.
> +
> + header->prev_index = BUDDY_CHUNK_ITEMS;
> + header->next_index = BUDDY_CHUNK_ITEMS;
> + if (idx_set_order(chunk, retidx, order_req))
> + return (u64)NULL;
> +
> + if (idx_set_allocated(chunk, retidx, true))
> + return (u64)NULL;
> +
> + /*
> + * Do not unpoison the address yet, will be done by the caller
> + * because the caller has the exact allocation size requested.
> + */
> + address = (u64)idx_to_addr(chunk, retidx);
> +
> + /* If we allocated from a larger-order chunk, split the buddies. */
> + for (i = order_req; i < order && can_loop; i++) {
> + /*
> + * Flip the bit for the current order (the bit is guaranteed
> + * to be 0, so just add 1 << i).
> + */
> + idx = retidx + (1 << i);
> +
> + /* Add the buddy of the allocation to the free list. */
> + header = idx_to_header(chunk, idx);
> + /* Unpoison the buddy header */
> + asan_unpoison(header, sizeof(*header));
> + if (idx_set_allocated(chunk, idx, false))
> + return (u64)NULL;
> +
> + if (idx_set_order(chunk, idx, i))
> + return (u64)NULL;
> +
> + /* Push the header to the beginning of the freelists list. */
> + tmpidx = chunk->freelists[i];
> +
> + header->prev_index = BUDDY_CHUNK_ITEMS;
> + header->next_index = tmpidx;
> +
> + if (tmpidx != BUDDY_CHUNK_ITEMS) {
> + tmp_header = idx_to_header(chunk, tmpidx);
> + tmp_header->prev_index = idx;
> + }
> +
> + chunk->freelists[i] = idx;
> + }
> +
> + return address;
> +}
> +
> +/* Scan the existing chunks for available memory. */
> +static u64 buddy_alloc_from_existing_chunks(struct buddy *buddy, int order)
> +{
> + buddy_chunk_t *chunk;
> + u64 address;
> +
> + for (chunk = buddy->first_chunk; chunk != NULL && can_loop;
> + chunk = chunk->next) {
> + address = buddy_chunk_alloc(chunk, order);
> + if (address)
> + return address;
> + }
> +
> + return (u64)NULL;
> +}
> +
> +/*
> + * Try an allocation from a newly allocated chunk. Also
> + * incorporate the chunk into the linked list.
> + */
> +static u64 buddy_alloc_from_new_chunk(struct buddy *buddy, buddy_chunk_t *chunk, int order)
> +{
> + u64 address;
> +
> + if (buddy_lock(buddy))
> + return (u64)NULL;
> +
> +
> + /*
> + * Add the chunk into the allocator and try
> + * to allocate specifically from that chunk.
> + */
> + chunk->next = buddy->first_chunk;
> + chunk->prev = NULL;
> + buddy->first_chunk = chunk;
> +
> + address = buddy_chunk_alloc(buddy->first_chunk, order);
> +
> + buddy_unlock(buddy);
> +
> + return (u64)address;
> +}
> +__weak
> +u64 buddy_alloc_internal(struct buddy *buddy, size_t size)
> +{
> + buddy_chunk_t *chunk;
> + u64 address = (u64)NULL;
> + int order;
> +
> + if (!buddy)
> + return (u64)NULL;
> +
> + order = size_to_order(size);
> + if (order >= BUDDY_CHUNK_NUM_ORDERS || order < 0) {
> + arena_stderr("invalid order %d (sz %lu)\n", order, size);
> + return (u64)NULL;
> + }
> +
> + if (buddy_lock(buddy))
> + return (u64)NULL;
> +
> + address = buddy_alloc_from_existing_chunks(buddy, order);
> + buddy_unlock(buddy);
> + if (address)
> + goto done;
> +
> + /* Get a new chunk. */
> + chunk = buddy_chunk_get(buddy);
> + if (chunk)
> + address = buddy_alloc_from_new_chunk(buddy, chunk, order);
> +
> +done:
> + /* If we failed to allocate memory, return NULL. */
> + if (!address)
> + return (u64)NULL;
> +
> + /*
> + * Unpoison exactly the amount of bytes requested. If the
> + * data is smaller than the header, we must poison any
> + * unused bytes that were part of the header.
> + */
> + if (size < BUDDY_HEADER_OFF + sizeof(buddy_header_t))
> + asan_poison((u8 __arena *)address + BUDDY_HEADER_OFF,
> + BUDDY_POISONED, sizeof(buddy_header_t));
> +
> + asan_unpoison((u8 __arena *)address, size);
> +
> + return address;
> +}
> +
> +static __always_inline int buddy_free_unlocked(struct buddy *buddy, u64 addr)
> +{
> + buddy_header_t *header, *buddy_header;
> + u64 idx, buddy_idx, tmp_idx;
> + buddy_chunk_t *chunk;
> + bool allocated;
> + u8 order;
> +
> + if (!buddy)
> + return -EINVAL;
> +
> + if (addr & (BUDDY_MIN_ALLOC_BYTES - 1)) {
> + arena_stderr("Freeing unaligned address %llx\n", addr);
> + return -EINVAL;
> + }
> +
> + /* Get (chunk, idx) out of the address. */
> + chunk = (void __arena *)(addr & ~BUDDY_CHUNK_OFFSET_MASK);
> + idx = (addr & BUDDY_CHUNK_OFFSET_MASK) / BUDDY_MIN_ALLOC_BYTES;
> +
> + /* Mark the block as unallocated so we can access the header. */
> + idx_set_allocated(chunk, idx, false);
> +
> + order = idx_get_order(chunk, idx);
> + header = idx_to_header(chunk, idx);
> +
> + /* The header is in the block itself, keep it unpoisoned. */
> + asan_poison((u8 __arena *)addr, BUDDY_POISONED,
> + BUDDY_MIN_ALLOC_BYTES << order);
> + asan_unpoison(header, sizeof(*header));
> +
> + /*
> + * Coalescing loop. Merge with free buddies of equal order.
> + * For every coalescing step, keep the left buddy and
> + * drop the right buddy's header.
> + */
> + for (; order < BUDDY_CHUNK_NUM_ORDERS && can_loop; order++) {
> + buddy_idx = idx ^ (1 << order);
> +
> + /* Check if the buddy is actually free. */
> + idx_is_allocated(chunk, buddy_idx, &allocated);
> + if (allocated)
> + break;
> +
> + /*
> + * If buddy is not the same order as the chunk
> + * being freed, then we're done coalescing.
> + */
> + if (idx_get_order(chunk, buddy_idx) != order)
> + break;
> +
> + buddy_header = idx_to_header(chunk, buddy_idx);
> + header_remove_freelist(chunk, buddy_header, order);
> +
> + /* Keep the left header out of the two buddies, drop the other one. */
> + if (buddy_idx < idx) {
> + tmp_idx = idx;
> + idx = buddy_idx;
> + buddy_idx = tmp_idx;
> + }
> +
> + /* Remove the buddy from the freelists so that we can merge it. */
> + idx_set_order(chunk, buddy_idx, order);
> +
> + buddy_header = idx_to_header(chunk, buddy_idx);
> + asan_poison(buddy_header, BUDDY_POISONED,
> + sizeof(*buddy_header));
> + }
> +
> + /* Header properly freed but not in any freelists yet .*/
> + idx_set_order(chunk, idx, order);
> +
> + header = idx_to_header(chunk, idx);
> + header_add_freelist(chunk, header, idx, order);
> +
> + return 0;
> +}
> +
> +__weak int buddy_free_internal(struct buddy *buddy, u64 addr)
> +{
> + int ret;
> +
> + if (!buddy)
> + return -EINVAL;
> +
> + if ((ret = buddy_lock(buddy)))
> + return ret;
> +
> + buddy_free_unlocked(buddy, addr);
> +
> + buddy_unlock(buddy);
> +
> + return 0;
> +}
> +
> +__weak char _license[] SEC("license") = "GPL";
> --
> 2.53.0
>
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH bpf-next v4 9/9] selftests/bpf: Add selftests for libarena buddy allocator
2026-04-07 4:57 [PATCH bpf-next v4 0/9] Introduce arena library and runtime Emil Tsalapatis
` (7 preceding siblings ...)
2026-04-07 4:57 ` [PATCH bpf-next v4 8/9] selftests/bpf: Add buddy allocator for libarena Emil Tsalapatis
@ 2026-04-07 4:57 ` Emil Tsalapatis
2026-04-07 17:14 ` Alexei Starovoitov
8 siblings, 1 reply; 19+ messages in thread
From: Emil Tsalapatis @ 2026-04-07 4:57 UTC (permalink / raw)
To: bpf; +Cc: ast, andrii, memxor, daniel, eddyz87, song, Emil Tsalapatis
Introduce selftests for the buddy allocator without ASAN.
Add the libarena selftests both to the libarena test
runner and to test_progs, so that they are a) available when
libarena is pulled as a standalone library, and b) exercised
along with all other test programs in this directory.
Currently ASAN for libarena requires LLVM 22, which is not
available in the CI. Delay testing the ASAN case directly
through test_progs until that is the case. The ASAN version
is available for testing on demand through the library's own
harness.
Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
---
.../bpf/libarena/selftests/selftest.c | 12 +
.../libarena/selftests/st_asan_buddy.bpf.c | 251 ++++++++++++++++++
.../bpf/libarena/selftests/st_buddy.bpf.c | 230 ++++++++++++++++
.../selftests/bpf/prog_tests/libarena.c | 44 +++
4 files changed, 537 insertions(+)
create mode 100644 tools/testing/selftests/bpf/libarena/selftests/st_asan_buddy.bpf.c
create mode 100644 tools/testing/selftests/bpf/libarena/selftests/st_buddy.bpf.c
create mode 100644 tools/testing/selftests/bpf/prog_tests/libarena.c
diff --git a/tools/testing/selftests/bpf/libarena/selftests/selftest.c b/tools/testing/selftests/bpf/libarena/selftests/selftest.c
index 2bd8363ea614..ad7e00c2de0f 100644
--- a/tools/testing/selftests/bpf/libarena/selftests/selftest.c
+++ b/tools/testing/selftests/bpf/libarena/selftests/selftest.c
@@ -137,6 +137,12 @@ error_no_destroy: \
return ret; \
}
+TEST(test_buddy);
+
+#ifdef BPF_ARENA_ASAN
+TEST(asan_test_buddy);
+#endif
+
static void
banner(const char *progpath)
{
@@ -174,5 +180,11 @@ int main(int argc, char *argv[])
libbpf_set_print(libbpf_print_fn);
+ run_test_buddy();
+
+#ifdef BPF_ARENA_ASAN
+ run_asan_test_buddy();
+#endif
+
return 0;
}
diff --git a/tools/testing/selftests/bpf/libarena/selftests/st_asan_buddy.bpf.c b/tools/testing/selftests/bpf/libarena/selftests/st_asan_buddy.bpf.c
new file mode 100644
index 000000000000..61b551639f6a
--- /dev/null
+++ b/tools/testing/selftests/bpf/libarena/selftests/st_asan_buddy.bpf.c
@@ -0,0 +1,251 @@
+// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+
+#include <common.h>
+
+#include <asan.h>
+#include <buddy.h>
+
+#include "selftest.h"
+
+
+#ifdef BPF_ARENA_ASAN
+
+#include "st_asan_common.h"
+
+private(ST_BUDDY) struct buddy st_buddy_asan;
+
+u64 __arena st_asan_buddy_lock;
+
+static __always_inline int asan_test_buddy_oob_single(size_t alloc_size)
+{
+ u8 __arena *mem;
+ int i;
+
+ ASAN_VALIDATE();
+
+ mem = buddy_alloc(&st_buddy_asan, alloc_size);
+ if (!mem) {
+ arena_stdout("buddy_alloc failed for size %lu", alloc_size);
+ return -ENOMEM;
+ }
+
+ ASAN_VALIDATE();
+
+ for (i = zero; i < alloc_size && can_loop; i++) {
+ mem[i] = 0xba;
+ ASAN_VALIDATE_ADDR(false, &mem[i]);
+ }
+
+ mem[alloc_size] = 0xba;
+ ASAN_VALIDATE_ADDR(true, &mem[alloc_size]);
+
+ buddy_free(&st_buddy_asan, mem);
+
+ return 0;
+}
+
+/*
+ * Factored out because ASAN_VALIDATE_ADDR is complex enough to cause
+ * verification failures if verified with the rest of asan_test_buddy_uaf_single.
+ */
+__weak int asan_test_buddy_byte(u8 __arena __arg_arena *mem, int i, bool freed)
+{
+ /* The header in freed blocks doesn't get poisoned. */
+ if (freed && BUDDY_HEADER_OFF <= i &&
+ i < BUDDY_HEADER_OFF + sizeof(struct buddy_header))
+ return 0;
+
+ mem[i] = 0xba;
+ ASAN_VALIDATE_ADDR(freed, &mem[i]);
+
+ return 0;
+}
+
+__weak int asan_test_buddy_uaf_single(size_t alloc_size)
+{
+ u8 __arena *mem;
+ int ret;
+ int i;
+
+ mem = buddy_alloc(&st_buddy_asan, alloc_size);
+ if (!mem) {
+ arena_stdout("buddy_alloc failed for size %lu", alloc_size);
+ return -ENOMEM;
+ }
+
+ ASAN_VALIDATE();
+
+ for (i = zero; i < alloc_size && can_loop; i++) {
+ ret = asan_test_buddy_byte(mem, i, false);
+ if (ret)
+ return ret;
+ }
+
+ ASAN_VALIDATE();
+
+ buddy_free(&st_buddy_asan, mem);
+
+ for (i = zero; i < alloc_size && can_loop; i++) {
+ ret = asan_test_buddy_byte(mem, i, true);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+struct buddy_blob {
+ volatile u8 mem[48];
+ u8 oob;
+};
+
+static __always_inline int asan_test_buddy_blob_single(void)
+{
+ volatile struct buddy_blob __arena *blob;
+ const size_t alloc_size = sizeof(struct buddy_blob) - 1;
+
+ blob = buddy_alloc(&st_buddy_asan, alloc_size);
+ if (!blob)
+ return -ENOMEM;
+
+ blob->mem[0] = 0xba;
+ ASAN_VALIDATE_ADDR(false, &blob->mem[0]);
+
+ blob->mem[47] = 0xba;
+ ASAN_VALIDATE_ADDR(false, &blob->mem[47]);
+
+ blob->oob = 0;
+ ASAN_VALIDATE_ADDR(true, &blob->oob);
+
+ buddy_free(&st_buddy_asan, (void __arena *)blob);
+
+ return 0;
+}
+
+static __always_inline int asan_test_buddy_oob(void)
+{
+ size_t sizes[] = {
+ 7, 8, 17, 18, 64, 256, 317, 512, 1024,
+ };
+ int ret, i;
+
+ ret = buddy_init(&st_buddy_asan,
+ (arena_spinlock_t __arena *)&st_asan_buddy_lock);
+ if (ret) {
+ arena_stdout("buddy_init failed with %d", ret);
+ return ret;
+ }
+
+ for (i = zero; i < sizeof(sizes) / sizeof(sizes[0]) && can_loop; i++) {
+ ret = asan_test_buddy_oob_single(sizes[i]);
+ if (ret) {
+ arena_stdout("%s:%d Failed for size %lu", __func__,
+ __LINE__, sizes[i]);
+ buddy_destroy(&st_buddy_asan);
+ return ret;
+ }
+ }
+
+ buddy_destroy(&st_buddy_asan);
+
+ ASAN_VALIDATE();
+
+ return 0;
+}
+
+__weak int asan_test_buddy_uaf(void)
+{
+ size_t sizes[] = { 16, 32, 64, 128, 256, 512, 128, 1024, 16384 };
+ int ret, i;
+
+ ret = buddy_init(&st_buddy_asan,
+ (arena_spinlock_t __arena *)&st_asan_buddy_lock);
+ if (ret) {
+ arena_stdout("buddy_init failed with %d", ret);
+ return ret;
+ }
+
+ for (i = zero; i < sizeof(sizes) / sizeof(sizes[0]) && can_loop; i++) {
+ ret = asan_test_buddy_uaf_single(sizes[i]);
+ if (ret) {
+ arena_stdout("%s:%d Failed for size %lu", __func__,
+ __LINE__, sizes[i]);
+ buddy_destroy(&st_buddy_asan);
+ return ret;
+ }
+ }
+
+ buddy_destroy(&st_buddy_asan);
+
+ ASAN_VALIDATE();
+
+ return 0;
+}
+
+static __always_inline int asan_test_buddy_blob(void)
+{
+ const int iters = 10;
+ int ret, i;
+
+ ret = buddy_init(&st_buddy_asan,
+ (arena_spinlock_t __arena *)&st_asan_buddy_lock);
+ if (ret) {
+ arena_stdout("buddy_init failed with %d", ret);
+ return ret;
+ }
+
+ for (i = zero; i < iters && can_loop; i++) {
+ ret = asan_test_buddy_blob_single();
+ if (ret) {
+ arena_stdout("%s:%d Failed on iteration %d", __func__,
+ __LINE__, i);
+ buddy_destroy(&st_buddy_asan);
+ return ret;
+ }
+ }
+
+ buddy_destroy(&st_buddy_asan);
+
+ ASAN_VALIDATE();
+
+ return 0;
+}
+
+SEC("syscall")
+int asan_test_buddy(void)
+{
+ int ret;
+
+ ret = asan_test_buddy_oob();
+ if (ret) {
+ arena_stdout("%s:%d OOB test failed", __func__, __LINE__);
+ return ret;
+ }
+
+ ret = asan_test_buddy_uaf();
+ if (ret) {
+ arena_stdout("%s:%d UAF test failed", __func__, __LINE__);
+ return ret;
+ }
+
+ ret = asan_test_buddy_blob();
+ if (ret) {
+ arena_stdout("%s:%d blob test failed", __func__, __LINE__);
+ return ret;
+ }
+
+ return 0;
+}
+
+#else
+
+SEC("syscall")
+int asan_test_buddy(void)
+{
+ return -EOPNOTSUPP;
+}
+
+#endif /* BPF_ARENA_ASAN */
+
+__weak char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/libarena/selftests/st_buddy.bpf.c b/tools/testing/selftests/bpf/libarena/selftests/st_buddy.bpf.c
new file mode 100644
index 000000000000..d8354dd2d7fc
--- /dev/null
+++ b/tools/testing/selftests/bpf/libarena/selftests/st_buddy.bpf.c
@@ -0,0 +1,230 @@
+// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+
+#include <common.h>
+
+#include <asan.h>
+#include <buddy.h>
+
+#include "selftest.h"
+
+
+private(ST_BUDDY) struct buddy st_buddy;
+static u64 __arena st_buddy_lock;
+
+struct segarr_entry {
+ u8 __arena *block;
+ size_t sz;
+ u8 poison;
+};
+
+typedef struct segarr_entry __arena segarr_entry_t;
+
+#define SEGARRLEN (512)
+static struct segarr_entry __arena segarr[SEGARRLEN];
+static void __arena *ptrs[17];
+size_t __arena alloc_sizes[] = { 3, 17, 1025, 129, 16350, 333, 9, 517 };
+size_t __arena alloc_multiple_sizes[] = { 3, 17, 1025, 129, 16350, 333, 9, 517, 2099 };
+size_t __arena alloc_free_sizes[] = { 3, 17, 64, 129, 256, 333, 512, 517 };
+size_t __arena alignment_sizes[] = { 1, 3, 7, 8, 9, 15, 16, 17, 31,
+ 32, 64, 100, 128, 255, 256, 512, 1000 };
+
+static int test_buddy_create(void)
+{
+ const int iters = 10;
+ int ret, i;
+
+ for (i = zero; i < iters && can_loop; i++) {
+ ret = buddy_init(
+ &st_buddy, (arena_spinlock_t __arena *)&st_buddy_lock);
+ if (ret)
+ return ret;
+
+ ret = buddy_destroy(&st_buddy);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int test_buddy_alloc(void)
+{
+ void __arena *mem;
+ int ret, i;
+
+ for (i = zero; i < 8 && can_loop; i++) {
+ ret = buddy_init(
+ &st_buddy, (arena_spinlock_t __arena *)&st_buddy_lock);
+ if (ret)
+ return ret;
+
+ mem = buddy_alloc(&st_buddy, alloc_sizes[i]);
+ if (!mem) {
+ buddy_destroy(&st_buddy);
+ return -ENOMEM;
+ }
+
+ buddy_destroy(&st_buddy);
+ }
+
+ return 0;
+}
+
+static int test_buddy_alloc_free(void)
+{
+ const int iters = 800;
+ void __arena *mem;
+ int ret, i;
+
+ ret = buddy_init(&st_buddy,
+ (arena_spinlock_t __arena *)&st_buddy_lock);
+ if (ret)
+ return ret;
+
+ for (i = zero; i < iters && can_loop; i++) {
+ mem = buddy_alloc(&st_buddy, alloc_free_sizes[(i * 5) % 8]);
+ if (!mem) {
+ buddy_destroy(&st_buddy);
+ return -ENOMEM;
+ }
+
+ buddy_free(&st_buddy, mem);
+ }
+
+ buddy_destroy(&st_buddy);
+
+ return 0;
+}
+
+static int test_buddy_alloc_multiple(void)
+{
+ int ret, j;
+ u32 i, idx;
+ u8 __arena *mem;
+ size_t sz;
+ u8 poison;
+
+ ret = buddy_init(&st_buddy,
+ (arena_spinlock_t __arena *)&st_buddy_lock);
+ if (ret)
+ return ret;
+
+ /*
+ * Cycle through each size, allocating an entry in the
+ * segarr. Continue for SEGARRLEN iterations. For every
+ * allocation write down the size, use the current index
+ * as a poison value, and log it with the pointer in the
+ * segarr entry. Use the poison value to poison the entire
+ * allocated memory according to the size given.
+ */
+ idx = 0;
+ for (i = zero; i < SEGARRLEN && can_loop; i++) {
+ sz = alloc_multiple_sizes[i % 9];
+ poison = (u8)i;
+
+ mem = buddy_alloc(&st_buddy, sz);
+ if (!mem) {
+ buddy_destroy(&st_buddy);
+ arena_stdout("%s:%d", __func__, __LINE__);
+ return -ENOMEM;
+ }
+
+ segarr[i].block = mem;
+ segarr[i].sz = sz;
+ segarr[i].poison = poison;
+
+ for (j = zero; j < sz && can_loop; j++) {
+ mem[j] = poison;
+ if (mem[j] != poison) {
+ buddy_destroy(&st_buddy);
+ return -EINVAL;
+ }
+ }
+ }
+
+ /*
+ * Go to (i * 17) % SEGARRLEN, and free the block pointed to.
+ * Before freeing, check all bytes have the poisoned value
+ * corresponding to the element. If any values are unexpected,
+ * return an error. Skip some elements to test destroying the
+ * buddy allocator while data is still allocated.
+ */
+ for (i = 10; i < SEGARRLEN && can_loop; i++) {
+ idx = (i * 17) % SEGARRLEN;
+
+ mem = segarr[idx].block;
+ sz = segarr[idx].sz;
+ poison = segarr[idx].poison;
+
+ for (j = zero; j < sz && can_loop; j++) {
+ if (mem[j] != poison) {
+ buddy_destroy(&st_buddy);
+ arena_stdout("%s:%d %lx %u vs %u", __func__,
+ __LINE__, &mem[j], mem[j], poison);
+ return -EINVAL;
+ }
+ }
+
+ buddy_free(&st_buddy, mem);
+ }
+
+ buddy_destroy(&st_buddy);
+
+ return 0;
+}
+
+static int test_buddy_alignment(void)
+{
+ int ret, i;
+
+ ret = buddy_init(&st_buddy,
+ (arena_spinlock_t __arena *)&st_buddy_lock);
+ if (ret)
+ return ret;
+
+ /* Allocate various sizes and check alignment */
+ for (i = zero; i < 17 && can_loop; i++) {
+ ptrs[i] = buddy_alloc(&st_buddy, alignment_sizes[i]);
+ if (!ptrs[i]) {
+ arena_stdout("alignment test: alloc failed for size %lu",
+ alignment_sizes[i]);
+ buddy_destroy(&st_buddy);
+ return -ENOMEM;
+ }
+
+ /* Check 8-byte alignment */
+ if ((u64)ptrs[i] & 0x7) {
+ arena_stdout(
+ "alignment test: ptr %llx not 8-byte aligned (size %lu)",
+ (u64)ptrs[i], alignment_sizes[i]);
+ buddy_destroy(&st_buddy);
+ return -EINVAL;
+ }
+ }
+
+ /* Free all allocations */
+ for (i = zero; i < 17 && can_loop; i++) {
+ buddy_free(&st_buddy, ptrs[i]);
+ }
+
+ buddy_destroy(&st_buddy);
+
+ return 0;
+}
+
+#define BUDDY_ALLOC_SELFTEST(suffix) ALLOC_SELFTEST(test_buddy_##suffix)
+
+SEC("syscall")
+__weak int test_buddy(void)
+{
+ BUDDY_ALLOC_SELFTEST(create);
+ BUDDY_ALLOC_SELFTEST(alloc);
+ BUDDY_ALLOC_SELFTEST(alloc_free);
+ BUDDY_ALLOC_SELFTEST(alloc_multiple);
+ BUDDY_ALLOC_SELFTEST(alignment);
+
+ return 0;
+}
+
+__weak char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/prog_tests/libarena.c b/tools/testing/selftests/bpf/prog_tests/libarena.c
new file mode 100644
index 000000000000..ab0153f8baed
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/libarena.c
@@ -0,0 +1,44 @@
+// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <test_progs.h>
+#include <unistd.h>
+
+#define __arena
+typedef uint64_t u64;
+typedef uint8_t u8;
+
+#include "libarena/include/common.h"
+#include "libarena/include/asan.h"
+#include "libarena/include/selftest_helpers.h"
+
+#include "libarena/libarena.skel.h"
+
+static void test_libarena_buddy(void)
+{
+ struct libarena *skel;
+ int ret;
+
+ skel = libarena__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "open_and_load"))
+ return;
+
+ ret = libarena__attach(skel);
+ if (!ASSERT_OK(ret, "attach"))
+ goto out;
+
+ ret = libarena_run_prog(bpf_program__fd(skel->progs.arena_alloc_reserve));
+ if (!ASSERT_OK(ret, "arena_alloc_reserve"))
+ goto out;
+
+ ret = libarena_run_prog(bpf_program__fd(skel->progs.test_buddy));
+ ASSERT_OK(ret, "test_buddy");
+
+out:
+ libarena__destroy(skel);
+}
+
+void test_libarena(void)
+{
+ if (test__start_subtest("buddy"))
+ test_libarena_buddy();
+}
--
2.53.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* Re: [PATCH bpf-next v4 9/9] selftests/bpf: Add selftests for libarena buddy allocator
2026-04-07 4:57 ` [PATCH bpf-next v4 9/9] selftests/bpf: Add selftests for libarena buddy allocator Emil Tsalapatis
@ 2026-04-07 17:14 ` Alexei Starovoitov
0 siblings, 0 replies; 19+ messages in thread
From: Alexei Starovoitov @ 2026-04-07 17:14 UTC (permalink / raw)
To: Emil Tsalapatis
Cc: bpf, Alexei Starovoitov, Andrii Nakryiko, Kumar Kartikeya Dwivedi,
Daniel Borkmann, Eduard, Song Liu
On Mon, Apr 6, 2026 at 9:57 PM Emil Tsalapatis <emil@etsalapatis.com> wrote:
>
> Introduce selftests for the buddy allocator without ASAN.
> Add the libarena selftests both to the libarena test
> runner and to test_progs, so that they are a) available when
> libarena is pulled as a standalone library, and b) exercised
> along with all other test programs in this directory.
>
> Currently ASAN for libarena requires LLVM 22, which is not
> available in the CI. Delay testing the ASAN case directly
> through test_progs until that is the case. The ASAN version
> is available for testing on demand through the library's own
> harness.
>
> Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
> ---
> .../bpf/libarena/selftests/selftest.c | 12 +
> .../libarena/selftests/st_asan_buddy.bpf.c | 251 ++++++++++++++++++
> .../bpf/libarena/selftests/st_buddy.bpf.c | 230 ++++++++++++++++
> .../selftests/bpf/prog_tests/libarena.c | 44 +++
> 4 files changed, 537 insertions(+)
> create mode 100644 tools/testing/selftests/bpf/libarena/selftests/st_asan_buddy.bpf.c
> create mode 100644 tools/testing/selftests/bpf/libarena/selftests/st_buddy.bpf.c
> create mode 100644 tools/testing/selftests/bpf/prog_tests/libarena.c
>
> diff --git a/tools/testing/selftests/bpf/libarena/selftests/selftest.c b/tools/testing/selftests/bpf/libarena/selftests/selftest.c
> index 2bd8363ea614..ad7e00c2de0f 100644
> --- a/tools/testing/selftests/bpf/libarena/selftests/selftest.c
> +++ b/tools/testing/selftests/bpf/libarena/selftests/selftest.c
> @@ -137,6 +137,12 @@ error_no_destroy: \
> return ret; \
> }
>
> +TEST(test_buddy);
> +
> +#ifdef BPF_ARENA_ASAN
> +TEST(asan_test_buddy);
> +#endif
> +
> static void
> banner(const char *progpath)
> {
> @@ -174,5 +180,11 @@ int main(int argc, char *argv[])
>
> libbpf_set_print(libbpf_print_fn);
>
> + run_test_buddy();
> +
> +#ifdef BPF_ARENA_ASAN
> + run_asan_test_buddy();
> +#endif
> +
> return 0;
> }
> diff --git a/tools/testing/selftests/bpf/libarena/selftests/st_asan_buddy.bpf.c b/tools/testing/selftests/bpf/libarena/selftests/st_asan_buddy.bpf.c
> new file mode 100644
> index 000000000000..61b551639f6a
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/libarena/selftests/st_asan_buddy.bpf.c
> @@ -0,0 +1,251 @@
> +// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause
> +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
> +
> +#include <common.h>
> +
> +#include <asan.h>
> +#include <buddy.h>
> +
> +#include "selftest.h"
> +
> +
> +#ifdef BPF_ARENA_ASAN
> +
> +#include "st_asan_common.h"
> +
> +private(ST_BUDDY) struct buddy st_buddy_asan;
> +
> +u64 __arena st_asan_buddy_lock;
> +
> +static __always_inline int asan_test_buddy_oob_single(size_t alloc_size)
> +{
> + u8 __arena *mem;
> + int i;
> +
> + ASAN_VALIDATE();
> +
> + mem = buddy_alloc(&st_buddy_asan, alloc_size);
> + if (!mem) {
> + arena_stdout("buddy_alloc failed for size %lu", alloc_size);
> + return -ENOMEM;
> + }
> +
> + ASAN_VALIDATE();
> +
> + for (i = zero; i < alloc_size && can_loop; i++) {
> + mem[i] = 0xba;
> + ASAN_VALIDATE_ADDR(false, &mem[i]);
> + }
> +
> + mem[alloc_size] = 0xba;
> + ASAN_VALIDATE_ADDR(true, &mem[alloc_size]);
> +
> + buddy_free(&st_buddy_asan, mem);
> +
> + return 0;
> +}
> +
> +/*
> + * Factored out because ASAN_VALIDATE_ADDR is complex enough to cause
> + * verification failures if verified with the rest of asan_test_buddy_uaf_single.
> + */
> +__weak int asan_test_buddy_byte(u8 __arena __arg_arena *mem, int i, bool freed)
> +{
> + /* The header in freed blocks doesn't get poisoned. */
> + if (freed && BUDDY_HEADER_OFF <= i &&
> + i < BUDDY_HEADER_OFF + sizeof(struct buddy_header))
> + return 0;
> +
> + mem[i] = 0xba;
> + ASAN_VALIDATE_ADDR(freed, &mem[i]);
correcting my earlier comment...
ASAN_VALIDATE_ADDR() can be static inline function too.
So all 3 don't need to be macroses.
^ permalink raw reply [flat|nested] 19+ messages in thread