From: Yonghong Song <yonghong.song@linux.dev>
To: bot+bpf-ci@kernel.org, bpf@vger.kernel.org
Cc: ast@kernel.org, andrii@kernel.org, daniel@iogearbox.net,
eddyz87@gmail.com, kernel-team@fb.com, martin.lau@kernel.org,
mason@kernel.org, ihor.solodrai@linux.dev
Subject: Re: [PATCH bpf-next v6 07/21] bpf: Resume a covered call at its landing pad
Date: Sun, 27 Sep 2026 17:17:22 -0700 [thread overview]
Message-ID: <e114594f-e79e-49df-ac7c-39fcc114aa1e@linux.dev> (raw)
In-Reply-To: <e0a07e78708d4f576ee9a50c2d7c2137b6f399d9de52917a7e59e4602716c1d7@mail.kernel.org>
On 9/27/26 1:40 PM, bot+bpf-ci@kernel.org wrote:
>> diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
>> index 6d78c20e6507..0143688896b0 100644
>> --- a/include/linux/bpf_verifier.h
>> +++ b/include/linux/bpf_verifier.h
>> @@ -836,6 +836,7 @@ struct bpf_subprog_info {
>> s16 fastcall_stack_off;
>> bool has_tail_call: 1;
>> bool might_throw: 1;
>> + bool might_unwind: 1;
>> bool tail_call_reachable: 1;
>> bool has_ld_abs: 1;
>> bool is_cb: 1;
> [ ... ]
>
>> diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c
>> index 08aee86a155c..f291611fe578 100644
>> --- a/kernel/bpf/helpers.c
>> +++ b/kernel/bpf/helpers.c
>> @@ -5095,6 +5095,8 @@ BTF_ID_FLAGS(func, bpf_task_from_pid, KF_ACQUIRE | KF_RET_NULL)
>> BTF_ID_FLAGS(func, bpf_task_from_vpid, KF_ACQUIRE | KF_RET_NULL)
>> BTF_ID_FLAGS(func, bpf_throw)
>> +BTF_ID_FLAGS(func, bpf_unwind)
>> +BTF_ID_FLAGS(func, bpf_unwind_resume)
>> #ifdef CONFIG_BPF_EVENTS
>> BTF_ID_FLAGS(func, bpf_send_signal_task)
>> #endif
> Can a CAP_BPF program call bpf_unwind() at this commit? The kfuncs are
> registered in generic_btf_ids, which makes them resolvable, and the body
> at kernel/bpf/helpers.c is still __bpf_kfunc void bpf_unwind(void) { },
> so the call returns normally to the next instruction.
>
> However, the verifier assumes bpf_unwind() never falls through:
>
> kernel/bpf/verifier.c:process_bpf_unwind() ends the path with
> PROCESS_BPF_EXIT for an uncovered call, and jumps to the pad for a
> covered call. liveness.c keeps a fall-through edge, but do_check()
> never explores it. As a result, any instructions reachable only from
> after the call are skipped by the verifier and removed by
> bpf_opt_remove_dead_code().
>
> At run time the call returns and execution continues into whatever the
> JIT placed next, with r0-r5 clobbered, or off the end of the image.
>
> This is fixed later in the series by 7924e9e105cf "bpf: Dispatch cleanup
> pads by rewriting return addresses", which implements the real body via
> arch_bpf_stack_walk_ra()/bpf_unwind_rewrite(), and bpf_exc_keep_exits()
> puts 'r0 = 0; exit' after every bpf_unwind call.
As you suggested, I will move two BTF_ID_FLAGS's into commit
"bpf: Dispatch cleanup pads by rewriting return addresses".
>
> [ ... ]
>
>> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
>> index fc3df452de2e..77176250f866 100644
>> --- a/kernel/bpf/verifier.c
>> +++ b/kernel/bpf/verifier.c
>> @@ -19167,6 +19167,40 @@ enum {
>> INSN_IDX_UPDATED = 2,
>> };
>>
>> +static int push_cleanup_pad_branch(struct bpf_verifier_env *env, int insn_idx)
>> +{
>> + struct bpf_verifier_state *branch;
>> + struct bpf_func_state *frame;
>> + int pad = bpf_exc_pad_of_call(env, insn_idx);
>> +
>> + if (pad < 0)
>> + return 0;
>> + branch = push_stack(env, pad, insn_idx, false);
>> + if (IS_ERR(branch))
>> + return PTR_ERR(branch);
>> + frame = branch->frame[branch->curframe];
>> + /*
>> + * The state at that call with the caller-saved registers gone: the
>> + * callee's epilogue put r6-r9 and the stack back on the way out.
>> + */
>> + clear_caller_saved_regs(env, frame->regs);
>> + mark_reg_unknown(env, frame->regs, BPF_REG_0);
>> + return 0;
>> +}
>> +
>> +static int process_bpf_unwind(struct bpf_verifier_env *env, int *insn_idx)
>> +{
>> + struct bpf_func_state *frame = cur_func(env);
>> + int pad = bpf_exc_pad_of_call(env, *insn_idx);
>> +
>> + if (pad < 0)
>> + return PROCESS_BPF_EXIT;
>> + clear_caller_saved_regs(env, frame->regs);
>> + mark_reg_unknown(env, frame->regs, BPF_REG_0);
>> + *insn_idx = pad;
>> + return INSN_IDX_UPDATED;
>> +}
>> +
>> static int process_bpf_exit_full(struct bpf_verifier_env *env,
>> bool *do_print_state,
>> bool exception_exit)
>> @@ -19404,6 +19438,20 @@ static int do_check_insn(struct bpf_verifier_env *env, bool *do_print_state)
>>
>> env->jmps_processed++;
>> if (opcode == BPF_CALL) {
>> + if (bpf_is_unwind_kfunc(insn))
>> + return process_bpf_unwind(env, &env->insn_idx);
>> + if (bpf_is_unwind_resume_kfunc(insn)) {
>> + /*
>> + * Mark r0 a known zero -- unknown first, as
>> + * the known-zero helper keeps the type it
>> + * finds, which here is NOT_INIT. The fixups
>> + * lower this to 'r0 = 0; exit', so the frame
>> + * returns a real zero.
>> + */
>> + mark_reg_unknown(env, cur_regs(env), BPF_REG_0);
>> + mark_reg_known_zero(env, cur_regs(env), BPF_REG_0);
>> + return process_bpf_exit_full(env, do_print_state, false);
>> + }
>> if (env->cur_state->active_locks) {
>> /* similar to static subprog calls callx is allowed under a lock */
>> if (!bpf_is_callx(insn) &&
> Both new kfuncs are handled before check_kfunc_call(), so
> bpf_fetch_kfunc_arg_meta() never runs for them. That function rejects a
> kfunc with -EACCES "calling kernel function %s is not allowed" when
> btf_kfunc_is_allowed() fails for the program type or the kfunc has no
> flags. bpf_add_kfunc_call() only resolves the BTF id and does not apply
> that filter.
You are right, we need bpf_fetch_kfunc_arg_meta check. Will fix.
>
> As a result, bpf_unwind() and bpf_unwind_resume() are callable from any
> program type that can make kfunc calls, including types that
> generic_kfunc_set is not registered for (it is registered for TRACING,
> SCHED_CLS, XDP, STRUCT_OPS, SYSCALL and CGROUP_SKB).
>
> It also skips the usual sleepable, RCU, preempt and IRQ-context checks
> in check_kfunc_call().
>
> ---
>
> At this commit a bpf_unwind_resume() call is verified as a frame exit
> with r0 = 0, but the comment says "The fixups lower this to
> 'r0 = 0; exit'". Does that lowering exist yet? No fixup for it exists in
> kernel/bpf/fixups.c at this commit.
Comment is not right. The fixup is in later patch
"bpf: Dispatch cleanup pads by rewriting return addresses".
>
> This call is also accepted anywhere, not only inside a landing pad. So a
> CAP_BPF program can place 'call bpf_unwind_resume' in its main path. The
> verifier treats the path as ended and never looks at the following
> instructions, which the dead-code pass then removes or sanitizes.
>
> At run time the real kfunc body runs WARN_ONCE(1, "exception cleanup
> resume was not lowered to a return\n"). That is a user-triggerable
> warning, and a panic with panic_on_warn. The call then returns normally
> with r0-r5 clobbered and falls into instructions that were never
> verified for this state, or off the end of the JIT image.
>
> This is fixed later in the series by 7924e9e105cf "bpf: Dispatch cleanup
> pads by rewriting return addresses", which adds the bpf_do_misc_fixups
> lowering, and 2a1e4ed1e9ae "bpf: Refuse a landing pad that does not
> resume", which rejects "resume at insn %d is not in a landing pad".
>
>> @@ -19422,6 +19470,10 @@ static int do_check_insn(struct bpf_verifier_env *env, bool *do_print_state)
>> }
>> }
>> mark_reg_scratched(env, BPF_REG_0);
>> + /* An unwind out of this call resumes at the pad. */
>> + err = push_cleanup_pad_branch(env, env->insn_idx);
>> + if (err)
>> + return err;
>> if (bpf_in_stack_arg_cnt(&env->subprog_info[cur_func(env)->subprogno]))
>> cur_func(env)->no_stack_arg_load = true;
>> if (bpf_is_callx(insn))
> [ ... ]
>
>
> ---
> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
>
> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/36346422430
next prev parent reply other threads:[~2026-09-28 0:17 UTC|newest]
Thread overview: 56+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-26 5:00 [PATCH bpf-next v6 00/21] bpf: Run exception cleanup landing pads when bpf_unwind() unwinds Yonghong Song
2026-09-26 5:00 ` [PATCH bpf-next v6 01/21] bpf: Pack bpf_insn_aux_data flags into bit fields Yonghong Song
2026-09-26 5:00 ` [PATCH bpf-next v6 02/21] bpf: Accept the compiler's exception cleanup table at program load Yonghong Song
2026-09-26 5:00 ` [PATCH bpf-next v6 03/21] bpf: Add the bpf_unwind() and bpf_unwind_resume() kfuncs Yonghong Song
2026-09-26 5:00 ` [PATCH bpf-next v6 04/21] bpf: Add lookups for exception cleanup resumes and landing pads Yonghong Song
2026-09-26 5:00 ` [PATCH bpf-next v6 05/21] bpf: Prepare for an exception cleanup table before the CFG walk Yonghong Song
2026-09-26 5:16 ` sashiko-bot
2026-09-26 23:54 ` Yonghong Song
2026-09-27 20:39 ` bot+bpf-ci
2026-09-28 0:01 ` Yonghong Song
2026-09-26 5:00 ` [PATCH bpf-next v6 06/21] bpf: Make exception landing pads reachable in the CFG Yonghong Song
2026-09-26 5:21 ` sashiko-bot
2026-09-27 0:02 ` Yonghong Song
2026-09-27 20:40 ` bot+bpf-ci
2026-09-28 0:12 ` Yonghong Song
2026-09-26 5:00 ` [PATCH bpf-next v6 07/21] bpf: Resume a covered call at its landing pad Yonghong Song
2026-09-26 5:15 ` sashiko-bot
2026-09-26 8:21 ` Alexei Starovoitov
2026-09-27 0:04 ` Yonghong Song
2026-09-27 0:41 ` Yonghong Song
2026-09-27 20:40 ` bot+bpf-ci
2026-09-28 0:17 ` Yonghong Song [this message]
2026-09-26 5:00 ` [PATCH bpf-next v6 08/21] bpf: Refuse a landing pad that does not resume Yonghong Song
2026-09-26 5:17 ` sashiko-bot
2026-09-27 3:06 ` Yonghong Song
2026-09-27 20:40 ` bot+bpf-ci
2026-09-28 0:29 ` Yonghong Song
2026-09-26 5:00 ` [PATCH bpf-next v6 09/21] bpf: Refuse a private stack for a program with an exception cleanup table Yonghong Song
2026-09-26 5:00 ` [PATCH bpf-next v6 10/21] bpf: Dispatch cleanup pads by rewriting return addresses Yonghong Song
2026-09-27 20:40 ` bot+bpf-ci
2026-09-28 1:08 ` Yonghong Song
2026-09-26 5:01 ` [PATCH bpf-next v6 11/21] bpf, x86: Dispatch exception cleanup pads at run time Yonghong Song
2026-09-26 5:15 ` sashiko-bot
2026-09-27 4:35 ` Yonghong Song
2026-09-27 20:39 ` bot+bpf-ci
2026-09-28 3:10 ` Yonghong Song
2026-09-26 5:01 ` [PATCH bpf-next v6 12/21] bpf, arm64: " Yonghong Song
2026-09-26 5:14 ` sashiko-bot
2026-09-27 20:40 ` bot+bpf-ci
2026-09-26 5:01 ` [PATCH bpf-next v6 13/21] libbpf: Resolve the compiler's _Unwind_Resume to the kernel's kfunc Yonghong Song
2026-09-26 5:01 ` [PATCH bpf-next v6 14/21] libbpf: Add cleanup_info to bpf_prog_load_opts Yonghong Song
2026-09-26 5:01 ` [PATCH bpf-next v6 15/21] libbpf: Collect .bpf_cleanup records and pass them to the kernel Yonghong Song
2026-09-27 20:39 ` bot+bpf-ci
2026-09-28 3:28 ` Yonghong Song
2026-09-26 5:01 ` [PATCH bpf-next v6 16/21] libbpf: Carry the exception cleanup table through the light skeleton Yonghong Song
2026-09-26 5:01 ` [PATCH bpf-next v6 17/21] libbpf: Let the static linker carry .bpf_cleanup relocations Yonghong Song
2026-09-26 5:01 ` [PATCH bpf-next v6 18/21] selftests/bpf: Add an end-to-end .bpf_cleanup exception test Yonghong Song
2026-09-26 5:01 ` [PATCH bpf-next v6 19/21] selftests/bpf: Add __set_global() and __ret_global() test tags Yonghong Song
2026-09-26 5:18 ` sashiko-bot
2026-09-27 4:58 ` Yonghong Song
2026-09-27 20:24 ` bot+bpf-ci
2026-09-28 3:36 ` Yonghong Song
2026-09-26 5:01 ` [PATCH bpf-next v6 20/21] selftests/bpf: Cover the exception cleanup shapes the chain does not reach Yonghong Song
2026-09-27 20:40 ` bot+bpf-ci
2026-09-28 3:49 ` Yonghong Song
2026-09-26 5:01 ` [PATCH bpf-next v6 21/21] selftests/bpf: Load an exception cleanup program from a light skeleton Yonghong Song
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e114594f-e79e-49df-ac7c-39fcc114aa1e@linux.dev \
--to=yonghong.song@linux.dev \
--cc=andrii@kernel.org \
--cc=ast@kernel.org \
--cc=bot+bpf-ci@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=eddyz87@gmail.com \
--cc=ihor.solodrai@linux.dev \
--cc=kernel-team@fb.com \
--cc=martin.lau@kernel.org \
--cc=mason@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox