Re: [PATCHSET SLOP RFC 1/6] bpf: Support __arena suffix for kfunc arguments

["Kumar Kartikeya Dwivedi" <memxor@gmail.com>]

On Mon Jul 13, 2026 at 4:44 AM CEST, Tejun Heo wrote:
> Passing an arena pointer to a kfunc takes two steps today. There is no
> arena pointer argument type, so the pointer crosses the boundary as a bare
> scalar, and the kfunc then offsets it by the arena base and casts it
> before it can touch the memory. Every such kfunc open-codes the same
> translation.
>
> Add the __arena argument suffix and both steps go away. The kfunc declares
> the parameter by its real pointer type and dereferences it directly. The
> verifier rebases each one at the call site, rN = kern_vm_start + (u32)rN
> with NULL preserved. No bounds check is needed: the u32 offset stays within
> the guard-padded arena kernel mapping, and a fault on an unpopulated page
> recovers through the per-arena scratch page. A suffixed argument accepts a
> PTR_TO_ARENA or scalar register, matching global subprog arena arguments.
>
> Signed-off-by: Tejun Heo <tj@kernel.org>
> ---
>  Documentation/bpf/kfuncs.rst | 22 +++++++++
>  include/linux/bpf_verifier.h |  2 +
>  kernel/bpf/verifier.c        | 86 +++++++++++++++++++++++++++++++++---
>  3 files changed, 105 insertions(+), 5 deletions(-)
>
> diff --git a/Documentation/bpf/kfuncs.rst b/Documentation/bpf/kfuncs.rst
> index c801a330aece..bad2ac85b318 100644
> --- a/Documentation/bpf/kfuncs.rst
> +++ b/Documentation/bpf/kfuncs.rst
> @@ -250,6 +250,26 @@ This annotation is used to indicate that the argument is a constant string.
>                  ...
>          }
>
> +2.3.7 __arena Annotation
> +------------------------
> +
> +This annotation is used to indicate that the pointer argument points into
> +the program's arena. The verifier translates it to a directly
> +dereferenceable kernel address before the call, subject to the access
> +rules described in :ref:`BPF_kfunc_arena_access`. NULL is preserved and the
> +kfunc is responsible for checking for NULL before dereferencing.
> +

Here we can specify the maximum permitted size (GUARD_SZ/2 = 32kb).

> +An example is given below::
> +
> +        __bpf_kfunc int bpf_process_item(struct item *item__arena)
> +        {
> +        ...
> +        }
> +
> +Calling such a kfunc requires the program to use an arena map. The program
> +can pass any value without compromising the kernel. A value that does not
> +point into the arena is a program bug.
> +
>  .. _BPF_kfunc_nodef:
>
>  2.4 Using an existing kernel function
> @@ -487,6 +507,8 @@ In order to accommodate such requirements, the verifier will enforce strict
>  PTR_TO_BTF_ID type matching if two types have the exact same name, with one
>  being suffixed with ``___init``.
>
> +.. _BPF_kfunc_arena_access:
> +
>  2.8 Accessing arena memory through kfunc arguments
>  --------------------------------------------------
>
> diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
> index 317e99b9acc0..2153d546736a 100644
> --- a/include/linux/bpf_verifier.h
> +++ b/include/linux/bpf_verifier.h
> @@ -690,6 +690,8 @@ struct bpf_insn_aux_data {
>  	 */
>  	u8 fastcall_spills_num:3;
>  	u8 arg_prog:4;
> +	/* bitmask of R1-R5 kfunc args to rebase to arena kernel addresses */
> +	u8 arg_arena_regs;
>
>  	/* below fields are initialized once */
>  	unsigned int orig_idx; /* original instruction index */
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index 03e2202cca13..b2b00a612e58 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -10856,6 +10856,11 @@ static bool is_kfunc_arg_irq_flag(const struct btf *btf, const struct btf_param
>  	return btf_param_match_suffix(btf, arg, "__irq_flag");
>  }
>
> +static bool is_kfunc_arg_arena(const struct btf *btf, const struct btf_param *arg)
> +{
> +	return btf_param_match_suffix(btf, arg, "__arena");
> +}
> +
>  static bool is_kfunc_arg_scalar_with_name(const struct btf *btf,
>  					  const struct btf_param *arg,
>  					  const char *name)
> @@ -12060,6 +12065,32 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
>  			continue;
>  		}
>
> +		if (is_kfunc_arg_arena(btf, &args[i])) {
> +			t = btf_type_skip_modifiers(btf, args[i].type, NULL);
> +			if (verifier_bug_if(!btf_type_is_ptr(t), env,
> +					    "kfunc %s arg#%d has __arena tag on non-pointer",
> +					    func_name, i))
> +				return -EFAULT;
> +			if (!env->prog->aux->arena) {
> +				verbose(env,
> +					"%s arena pointer requires a program with an associated arena\n",
> +					reg_arg_name(env, argno));
> +				return -EINVAL;
> +			}
> +			if (regno < 0) {
> +				verbose(env, "%s arena pointer cannot be a stack argument\n",
> +					reg_arg_name(env, argno));
> +				return -EINVAL;
> +			}
> +			if (reg->type != PTR_TO_ARENA && reg->type != SCALAR_VALUE) {
> +				verbose(env, "%s is not a pointer to arena or scalar\n",
> +					reg_arg_name(env, argno));
> +				return -EINVAL;
> +			}
> +			cur_aux(env)->arg_arena_regs |= BIT(regno - BPF_REG_1);
> +			continue;
> +		}
> +
>  		if (is_kfunc_arg_ignore(btf, &args[i]) || is_kfunc_arg_implicit(meta, i))
>  			continue;
>
> @@ -19890,13 +19921,58 @@ int bpf_fixup_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
>  		*cnt = 6;
>  	}
>
> -	if (env->insn_aux_data[insn_idx].arg_prog) {
> -		u32 regno = env->insn_aux_data[insn_idx].arg_prog;
> -		struct bpf_insn ld_addrs[2] = { BPF_LD_IMM64(regno, (long)env->prog->aux) };
> +	if (env->insn_aux_data[insn_idx].arg_prog ||
> +	    env->insn_aux_data[insn_idx].arg_arena_regs) {
> +		unsigned long arena_regs = env->insn_aux_data[insn_idx].arg_arena_regs;
> +		u64 kern_vm_start = bpf_arena_get_kern_vm_start(env->prog->aux->arena);
> +		u32 prog_regno = env->insn_aux_data[insn_idx].arg_prog;
>  		int idx = *cnt;
> +		int bit;
>
> -		insn_buf[idx++] = ld_addrs[0];
> -		insn_buf[idx++] = ld_addrs[1];
> +		if (verifier_bug_if(idx, env, "kfunc call at %d already patched", insn_idx))
> +			return -EFAULT;
> +
> +		/*
> +		 * A kfunc may take both an __arena arg and the __prog aux. Emit
> +		 * each present fixup, then append the call once.
> +		 */
> +		if (arena_regs) {
> +			bool blinding = env->prog->blinding_requested;
> +			int base_reg = blinding ? BPF_REG_0 : BPF_REG_AX;
> +			struct bpf_insn ld[2] = { BPF_LD_IMM64(base_reg, kern_vm_start) };
> +
> +			/*
> +			 * rN = kern_vm_start + (u32)rN, NULL left as 0.
> +			 * Constant blinding rewrites immediate-carrying insns
> +			 * unless they use BPF_REG_AX. With hardening, the
> +			 * constant sits in R0 (an AX-destined LD_IMM64 would be
> +			 * corrupted) and NULL is tested against AX = 0. Without
> +			 * hardening, use the cheaper AX constant and immediate
> +			 * test.
> +			 */
> +			insn_buf[idx++] = ld[0];
> +			insn_buf[idx++] = ld[1];
> +			if (blinding)
> +				insn_buf[idx++] = BPF_MOV64_IMM(BPF_REG_AX, 0);
> +
> +			for_each_set_bit(bit, &arena_regs, MAX_BPF_FUNC_REG_ARGS) {
> +				int regno = BPF_REG_1 + bit;
> +
> +				insn_buf[idx++] = BPF_ZEXT_REG(regno);
> +				insn_buf[idx++] = blinding ?
> +					BPF_JMP_REG(BPF_JEQ, regno, BPF_REG_AX, 1) :
> +					BPF_JMP_IMM(BPF_JEQ, regno, 0, 1);
> +				insn_buf[idx++] = BPF_ALU64_REG(BPF_ADD, regno, base_reg);
> +			}
> +		}
> +		if (prog_regno) {
> +			struct bpf_insn ld_addrs[2] = {
> +				BPF_LD_IMM64(prog_regno, (long)env->prog->aux)
> +			};
> +
> +			insn_buf[idx++] = ld_addrs[0];
> +			insn_buf[idx++] = ld_addrs[1];
> +		}
>  		insn_buf[idx++] = *insn;
>  		*cnt = idx;
>  	}

I think overall this makes sense. One thing we can change is making __arena
imply non-NULL arena pointers, which will lead to a more optimized sequence, and
__arena_nullable when we want NULL to be preserved in either direction, to
represent optional data.

We likely need two bitmaps, one to represent nullness of arena regs and one for
arena regs.

Also, instead of handling them in fixup in verifier, we can let JITs handle them
because they already hold arena base in some fixed register.

E.g. we can add the following in kernel/bpf/core.c.

struct bpf_jit_arena_args {
	u8 regs;
	u8 nullable_regs;
};

struct bpf_jit_arena_args
bpf_kfunc_arena_args(const struct bpf_verifier_env *env,
		     const struct bpf_prog *prog, int insn_idx)
{
	struct bpf_jit_arena_args args = {};

	if (!env)
		return args;

	insn_idx += prog->aux->subprog_start;
	args.regs = env->insn_aux_data[insn_idx].arg_arena_regs;
	args.nullable_regs =
		env->insn_aux_data[insn_idx].arg_arena_nullable_regs;
	return args;
}

The accessor expects a zero-based instruction index local to prog.
x86’s JIT loop is one-based and must pass i - 1 in such case.
You can see bpf_insn_is_indirect_target()'s usage for inspiration.

For now I guess we can add support for x86 and arm64. I'll show pseudocode for
both x86, arm64 would be similar in spirit.

We can place a new emit_kfunc_arena_args() right before emit_call().

if (!imm32)
	return -EINVAL;

if (src_reg == BPF_PSEUDO_KFUNC_CALL)
	emit_kfunc_arena_args(env, bpf_prog, i - 1, &prog);

ip += x86_call_depth_emit_accounting(&prog, func, ip);
emit_call(&prog, func, ip);

For emit_kfunc_arena_args(), it would look something like this (pseudocode-ish):

static int
emit_kfunc_arena_args(struct bpf_verifier_env *env,
		      struct bpf_prog *prog, int insn_idx,
		      u8 **pprog)
{
	struct bpf_jit_arena_args args;
	u8 *native_prog = *pprog;
	int bit;

	args = bpf_kfunc_arena_args(env, prog, insn_idx);

	if (WARN_ON_ONCE(args.nullable_regs & ~args.regs))
		return -EINVAL;

	if (WARN_ON_ONCE(args.regs && !prog->aux->arena))
		return -EINVAL;

	for (bit = 0; bit < MAX_BPF_FUNC_REG_ARGS; bit++) {
		int reg;

		if (!(args.regs & BIT(bit)))
			continue;

		/*
		 * These logical JIT registers map as:
		 *
		 * BPF_REG_1 -> RDI
		 * BPF_REG_2 -> RSI
		 * BPF_REG_3 -> RDX
		 * BPF_REG_4 -> RCX
		 * BPF_REG_5 -> R8
		 */
		reg = BPF_REG_1 + bit;

		/*
		 * mov reg32, reg32
		 *
		 * Extract the low 32 bits and clear the native register's
		 * upper 32 bits.
		 */
		emit_mov_reg(&native_prog, false, reg, reg);

		if (args.nullable_regs & BIT(bit)) {
			/*
			 * test reg32, reg32
			 * jz after_add
			 */
			emit_test_reg32(&native_prog, reg);

			/*
			 * add reg64, r12 is three bytes for all argument
			 * registers, so a short jump can skip it directly.
			 */
			emit_jz_short(&native_prog, /* displacement */ 3);
		}

		/*
		 * add reg64, r12
		 *
		 * R12 contains kern_vm_start.
		 */
		emit_add_reg64(&native_prog, reg, X86_REG_R12);
	}

	*pprog = native_prog;
	return 0;
}

arm64 will be similar, but I will let AI figure out that one. One opportunity
for arm64 case is to use add xN, x28, wN, uxtw single-instruction sequence for
non-nullable case.

Lastly, we should add bpf_jit_supports_arena_args(), make it return false from a
__weak function in kernel/bpf/core.c, and then override from x86 and arm64 JIT.

Then return an error when arena kfunc args (nullable or not) are used when this
predicate is false in check_kfunc_call() logic.

Since we're using the JIT for emitting this, we should not need special blinding
related handling.

For tests, I think we cover a lot already, but I'd confirmation on whether
nullable/non-nullable cases work correctly, in particular, low32 bits as zero
should produce kern_vm_start with __arena, and 0 with __arena_nullable, etc.
We can probably throw in tests that test JIT emitted sequence as well, there are
several examples that do this in the selftests.  Since we'll be limited to x86
and arm64 we can guard the tests to only work on these two arches.

Finally, it would make sense to add documentation for both tags, and we should
be done.

--

Next, for the struct_ops side, the main idea would be to do translation before
we save args into the context for the program instead of reloading it from
context, translating, and then having to store back. Just like program->kernel,
we will need some per-JIT handling of kernel->program translation.

Thus, we can avoid injecting a BPF entry prologue or performing a second
load/modify/store pass over the saved context. Fold arena conversion into the
architecture trampoline’s existing save_args() path while it copies native
arguments into the BPF u64 ctx[].

Pseudocode:
/* __arena */
ctx_value = (u32)(kaddr - kern_vm_start);
/* __arena_nullable */
ctx_value = kaddr ? (u32)(kaddr - kern_vm_start) : 0;

For nullable arguments, test the complete 64-bit native pointer before
subtracting or truncating. Only an actual kernel NULL should take the NULL path.
Derive arena argument information from prog->aux->ctx_arg_info, keyed by
info->offset, which is the offset in the flattened trampoline context. Do not
use the logical argument number because preceding 16-byte arguments occupy two
u64 slots. This lookup happens while generating the trampoline; no runtime loop
or metadata lookup should be emitted.

Only enable this for the single-node struct_ops indirect trampoline. Obtain the
program from the struct_ops fentry node and pass optional arena metadata into
the initial save_args(..., for_call_origin=false) invocation. Generic
trampolines may contain multiple programs with different arenas and must not use
this conversion. We can probably make those return -ENOTSUPP if ther eis a p

On x86, translate through RAX and store the translated value directly into the ctx slot:
# __arena, native argument in RDI
movl    %edi, %eax
subl    $kern_vm_start_lo, %eax
movq    %rax, ctx_slot

# __arena_nullable
movq    %rdi, %rax
testq   %rax, %rax
jz      1f
subl    $kern_vm_start_lo, %eax
1:
movq    %rax, ctx_slot

Use the same operation after loading a stack-passed argument into RAX. A 32-bit
subtraction is sufficient and also clears the upper 32 bits:

(u32)(kaddr - base) == (u32)kaddr - (u32)base;

On arm64, materialize only lower_32_bits(kern_vm_start) in a temporary such as W11. For register arguments:
// __arena
sub     w10, w0, w11
str     x10, [sp, #ctx_slot]

// __arena_nullable
mov     x10, x0
cbz     x10, 1f
sub     w10, w10, w11
1:
str     x10, [sp, #ctx_slot]

For stack-passed arguments, first load the full pointer into X10, perform the
same optional CBZ and 32-bit subtraction, and then store X10 into the ctx slot.
Writing W10 automatically clears X10’s upper bits, so no separate UXTW is
needed.

Non-arena arguments should retain the existing direct save path. Since this is
native trampoline code generation, kern_vm_start is not inserted into the BPF
instruction stream and requires no constant-blinding special case.

--

Once you respin arm64 experts can help validate arm64 bits, I am no expert
there.