Re: [PATCH] bpf, x86: allow function arguments up to 12 for TRACING

[Eduard Zingerman <eddyz87@gmail.com>]

On Wed, 2023-05-31 at 17:03 +0800, Menglong Dong wrote:
> On Wed, May 31, 2023 at 4:01 PM Jiri Olsa <olsajiri@gmail.com> wrote:
> > 
> > On Tue, May 30, 2023 at 12:44:23PM +0800, menglong8.dong@gmail.com wrote:
> > > From: Menglong Dong <imagedong@tencent.com>
> > > 
> > > For now, the BPF program of type BPF_PROG_TYPE_TRACING can only be used
> > > on the kernel functions whose arguments count less than 6. This is not
> > > friendly at all, as too many functions have arguments count more than 6.
> > > 
> > > Therefore, let's enhance it by increasing the function arguments count
> > > allowed in arch_prepare_bpf_trampoline(), for now, only x86_64.
> > > 
> > > For the case that we don't need to call origin function, which means
> > > without BPF_TRAMP_F_CALL_ORIG, we need only copy the function arguments
> > > that stored in the frame of the caller to current frame. The arguments
> > > of arg6-argN are stored in "$rbp + 0x18", we need copy them to
> > > "$rbp - regs_off + (6 * 8)".
> > > 
> > > For the case with BPF_TRAMP_F_CALL_ORIG, we need prepare the arguments
> > > in stack before call origin function, which means we need alloc extra
> > > "8 * (arg_count - 6)" memory in the top of the stack. Note, there should
> > > not be any data be pushed to the stack before call the origin function.
> > > Then, we have to store rbx with 'mov' instead of 'push'.
> > > 
> > > It works well for the FENTRY and FEXIT, I'm not sure if there are other
> > > complicated cases.
> > > 
> > > Signed-off-by: Menglong Dong <imagedong@tencent.com>
> > > ---
> > >  arch/x86/net/bpf_jit_comp.c | 88 ++++++++++++++++++++++++++++++++-----
> > 
> > please add selftests for this.. I had to add one to be able to check
> > the generated trampoline
> > 
> 
> Okay!
> 
> BTW, I failed to compile the latest selftests/bpf with
> the following errors:
> 
> progs/verifier_and.c:58:16: error: invalid operand for instruction
>         asm volatile ("                                 \
> 

These tests were moved to use inline assembly recently (2 month ago).
Discussion at the time was whether to use \n\ or \ terminators at the
end of each line. People opted for \ as easier to read.
Replacing \ with \n\ and compiling this test using clang 14 shows
more informative error message:

$ make -j14 `pwd`/verifier_and.bpf.o
  CLNG-BPF [test_maps] verifier_and.bpf.o
progs/verifier_and.c:68:1: error: invalid operand for instruction
        w1 %%= 2;                                       \n\
^
<inline asm>:11:5: note: instantiated into assembly here
        w1 %= 2; 

My guess is that clang 14 does not know how to handle operations on
32-bit sub-registers w[0-9].

But using clang 14 I get some errors not related to inline assembly as well.
Also, I recall that there were runtime issues with clang 14 and
tests using enum64.

All-in-all, you need newer version of clang for tests nowadays,
sorry for inconvenience.

> The version of clang I used is:
> 
> clang --version
> Debian clang version 14.0.6
> Target: x86_64-pc-linux-gnu
> Thread model: posix
> InstalledDir: /usr/bin
> 
> Does anyone know the reason?
> 
> Thanks!
> Menglong Dong
> 
> > jirka
> > 
> > 
>