Re: [PATCH bpf-next] bpf: Fix suspicious RCU usage for sleepable bpf_get_stack()

[sashiko-bot@kernel.org]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] bpf: per-CPU bpf_raw_tp_regs corruption in bpf_get_stack_raw_tp_sleepable()
--

commit c66538ad7075b69839a1969f72a7ea7eff53e112
Author: Feng Yang <yangfeng@kylinos.cn>

bpf: Fix suspicious RCU usage for sleepable bpf_get_stack()

This patch routes sleepable tracepoint and raw tracepoint/tracing BPF
programs calling bpf_get_stack() to bpf_get_stack_sleepable() so that
classic RCU read lock is held when accessing callchain entries and the
helper is marked might_sleep.

> diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
> index 14adfb85fdb2b..7887d1c00be83 100644
> --- a/kernel/trace/bpf_trace.c
> +++ b/kernel/trace/bpf_trace.c
[ ... ]
> @@ -1705,6 +1727,33 @@ static const struct bpf_func_proto bpf_get_stack_proto_raw_tp = {
>  	.arg4_type	= ARG_ANYTHING,
>  };
>  
> +BPF_CALL_4(bpf_get_stack_raw_tp_sleepable, struct bpf_raw_tracepoint_args *, args,
> +	   void *, buf, u32, size, u64, flags)
> +{
> +	struct pt_regs *regs = get_bpf_raw_tp_regs();
> +	int ret;
> +
> +	if (IS_ERR(regs))
> +		return PTR_ERR(regs);
> +
> +	perf_fetch_caller_regs(regs);
> +	ret = bpf_get_stack_sleepable((unsigned long) regs, (unsigned long) buf,
> +				      (unsigned long) size, flags, 0);
> +	put_bpf_raw_tp_regs();
> +	return ret;
> +}

[Severity: High]
Can holding the per-CPU bpf_raw_tp_regs slot across bpf_get_stack_sleepable()
corrupt active register state or exhaust the per-CPU slots on the local CPU?

In kernel/trace/bpf_trace.c, get_bpf_raw_tp_regs() and put_bpf_raw_tp_regs()
manage a 3-slot per-CPU array indexed as a LIFO stack via
bpf_raw_tp_nest_level:

    static struct pt_regs *get_bpf_raw_tp_regs(void)
    {
        struct bpf_raw_tp_regs *tp_regs = this_cpu_ptr(&bpf_raw_tp_regs);
        int nest_level = this_cpu_inc_return(bpf_raw_tp_nest_level);

        if (nest_level > ARRAY_SIZE(tp_regs->regs)) {
            this_cpu_dec(bpf_raw_tp_nest_level);
            return ERR_PTR(-EBUSY);
        }

        return &tp_regs->regs[nest_level - 1];
    }

    static void put_bpf_raw_tp_regs(void)
    {
        this_cpu_dec(bpf_raw_tp_nest_level);
    }

Sleepable BPF programs using raw_tp_prog_func_proto() or
tracing_prog_func_proto() run under rcu_read_lock_trace() and
migrate_disable() with preemption enabled. When such a program calls
bpf_get_stack_raw_tp_sleepable() with BPF_F_USER_STACK | BPF_F_USER_BUILD_ID,
bpf_get_stack_sleepable() -> __bpf_get_stack() only disables preemption around
get_perf_callchain() and callchain_store(), and then re-enables preemption
before calling callchain_finalize():

    /* trace should not be dereferenced after this point */
    if (may_fault)
        rcu_read_unlock();
    preempt_enable();

    return callchain_finalize(buf, size, trace_nr, elem_size, flags, may_fault);

Inside callchain_finalize() -> stack_map_get_build_id_offset_sleepable() ->
build_id_parse_file(), the task can sleep on page faults or filesystem reads
while still holding the per-CPU bpf_raw_tp_regs slot.

If task A on a CPU acquires regs[0] (nest_level becomes 1) and sleeps or is
preempted inside callchain_finalize(), task B on the same CPU can call
get_bpf_raw_tp_regs(), acquire regs[1] (nest_level becomes 2), populate
regs[1] via perf_fetch_caller_regs(), and get preempted before
__bpf_get_stack() disables preemption. If task A then wakes up on the same
CPU and calls put_bpf_raw_tp_regs(), bpf_raw_tp_nest_level is decremented
from 2 to 1 while task B is still about to use regs[1]. Any subsequent caller
on that CPU (another task, or an IRQ/NMI raw tracepoint program calling
bpf_get_stack_raw_tp(), bpf_get_stackid_raw_tp(), or
bpf_perf_event_output_raw_tp()) will increment bpf_raw_tp_nest_level back to
2, receive the same &tp_regs->regs[1] pointer, and overwrite task B's saved
registers before task B runs get_perf_callchain().

In addition, if three sleepable tasks on the same CPU block concurrently in
build_id_parse_file(), bpf_raw_tp_nest_level stays at 3 for the duration of
the I/O, causing all get_bpf_raw_tp_regs() calls on that CPU across task,
IRQ, and NMI contexts to fail with -EBUSY.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261028093548.790977-1-yangfeng59949@163.com?part=1