Re: [PATCH bpf-next 03/10] bpf: Support stack arguments for bpf functions

Alexei Starovoitov
2026-04-06T17:24:24Z

On Mon, Apr 6, 2026 at 9:19 AM Yonghong Song <yonghong.song@linux.dev> wrote:
>
>
>
> On 4/6/26 8:17 AM, Alexei Starovoitov wrote:
> > On Sun, Apr 5, 2026 at 11:03 PM Yonghong Song <yonghong.song@linux.dev> wrote:
> >>
> >>
> >> On 4/5/26 9:51 PM, Alexei Starovoitov wrote:
> >>> On Sun, Apr 5, 2026 at 9:29 PM Yonghong Song <yonghong.song@linux.dev> wrote:
> >>>>> For 6th argument JIT emits these two stx/ldx as moves to/from x86's r9.
> >>>> For stx case, we should move the bottom stack value (6th argument)
> >>>> to r9 and pop the bottom stack slot (8 bytes).
> >>> That doesn't sound right.
> >>> Passing 6th argument should not involve stack manipulation at all.
> >>> No push and no pop.
> >> The following stack layout can avoid the above push/pop issue:
> >>
> >> incoming stack arg N -> 1
> >> return adderss
> >> saved rbp
> >> BPF program stack
> >> tail call cnt <== if tail call reachable
> > let's disallow mixing 6+ args and tailcalls.
>
> Okay.
>
> >
> >> callee-saved regs
> >> r9 <== if priv_frame_ptr is not null
> > I guess we can also disable private stack and 6+ args.
> > Looks like it's a bit in the way.
> > The compilers will emit
> > stx [r12 - N], src_reg // store of outgoing 6th arg
> > call
> >
> > For priv stack JIT will emit push_r9 while JITing call insn.
> > But it should push_r9 before stx converts to 'mov %r9 <- %src_reg'.
>
> I guess this is not enough. we can push_r9 in prologue. But for
> priv stack, r9 is still used in the normal bpf load/store codes.
> This will interfere with jitting 'mov %r9 <- %src_reg'.
>
> As you mentioned later, disable priv_stack would work.
>
> > Maybe we can reload r9 after the call instead of push/pop.
> >
> >> outgoing stack arg 1
> >> outgoing stack arg M -> 2
> >>
> >> After the above pushing (outgoing stack arg M -> 2),
> >> if bpf-to-bpf, push the outgoing stack arg 1.
> >> If kfunc, move outgoing stack arg 1 to r9.
> > Ideally JIT treats subprog calls and kfunc calls the same way.
> > Why should they be different ?
> > In the callee, in both cases, the first insn in the prologue
> > can be 'mov [rbp + ] <- %r9',
> >
> > so that later 'ldx [r12 + ' can be JITed as-is. Here priv stack
> > is not in the way.
> > Without priv stack we can avoid this first 'mov [rbp + ] <- %r9'
> > in the prologue and instead JIT 'ldx [r12 + ' as 'mov %dst_reg <- %r9'
> > since nothing will be overwriting %r9.
> >
> > It feels to me that it's easier to disallow priv stack for now.
>
> So if we disallow priv_stack, we can do
> stx[r12 - first_arg_off] = val/reg => mov %r9, val/reg

yes.

> other arg_off will be pushed to stack in reverse orders.

no. Not pushed. they will be JITed in place stx[r12 -off] -> mov[%rbp], val
Probably needs add %rsp somewhere to grow the stack.

> So for kfunc, 'mov %r9, val' already there, so we should be okay.
> For bpf-to-bpf, do 'push %r9'.

Don't follow. why should they be different?