Re: [PATCH bpf-next 03/10] bpf: Support stack arguments for bpf functions

[Yonghong Song <yonghong.song@linux.dev>]

On 4/6/26 8:17 AM, Alexei Starovoitov wrote:
> On Sun, Apr 5, 2026 at 11:03 PM Yonghong Song <yonghong.song@linux.dev> wrote:
>>
>>
>> On 4/5/26 9:51 PM, Alexei Starovoitov wrote:
>>> On Sun, Apr 5, 2026 at 9:29 PM Yonghong Song <yonghong.song@linux.dev> wrote:
>>>>> For 6th argument JIT emits these two stx/ldx as moves to/from x86's r9.
>>>> For stx case, we should move the bottom stack value (6th argument)
>>>> to r9 and pop the bottom stack slot (8 bytes).
>>> That doesn't sound right.
>>> Passing 6th argument should not involve stack manipulation at all.
>>> No push and no pop.
>> The following stack layout can avoid the above push/pop issue:
>>
>>      incoming stack arg N -> 1
>>      return adderss
>>      saved rbp
>>      BPF program stack
>>      tail call cnt <== if tail call reachable
> let's disallow mixing 6+ args and tailcalls.

Okay.

>
>>      callee-saved regs
>>      r9 <== if priv_frame_ptr is not null
> I guess we can also disable private stack and 6+ args.
> Looks like it's a bit in the way.
> The compilers will emit
> stx [r12 - N], src_reg // store of outgoing 6th arg
> call
>
> For priv stack JIT will emit push_r9 while JITing call insn.
> But it should push_r9 before stx converts to 'mov %r9 <- %src_reg'.

I guess this is not enough. we can push_r9 in prologue. But for
priv stack, r9 is still used in the normal bpf load/store codes.
This will interfere with jitting 'mov %r9 <- %src_reg'.

As you mentioned later, disable priv_stack would work.

> Maybe we can reload r9 after the call instead of push/pop.
>
>>      outgoing stack arg 1
>>      outgoing stack arg M -> 2
>>
>> After the above pushing (outgoing stack arg M -> 2),
>> if bpf-to-bpf, push the outgoing stack arg 1.
>> If kfunc, move outgoing stack arg 1 to r9.
> Ideally JIT treats subprog calls and kfunc calls the same way.
> Why should they be different ?
> In the callee, in both cases, the first insn in the prologue
> can be 'mov [rbp + ] <- %r9',
>
> so that later 'ldx [r12 + ' can be JITed as-is. Here priv stack
> is not in the way.
> Without priv stack we can avoid this first 'mov [rbp + ] <- %r9'
> in the prologue and instead JIT 'ldx [r12 + ' as 'mov %dst_reg <- %r9'
> since nothing will be overwriting %r9.
>
> It feels to me that it's easier to disallow priv stack for now.

So if we disallow priv_stack, we can do
    stx[r12 - first_arg_off] = val/reg => mov %r9, val/reg
other arg_off will be pushed to stack in reverse orders.

So for kfunc, 'mov %r9, val' already there, so we should be okay.
For bpf-to-bpf, do 'push %r9'.

Is this correct?