FYI: bpf selftest verif_scale_strobemeta_subprogs failed with latest llvm19

FYI: bpf selftest verif_scale_strobemeta_subprogs failed with latest llvm19

[Yonghong Song]

The selftest verif_scale_strobemeta_subprogs failed with latest llvm19 compiler.
For example,

   $ ./test_progs -n 498
   ...
   libbpf: prog 'on_event': BPF program load failed: Permission denied
   libbpf: prog 'on_event': -- BEGIN PROG LOAD LOG --
   combined stack size of 4 calls is 544. Too large
   verification time 1417195 usec
   stack depth 24+440+0+32
   processed 53561 insns (limit 1000000) max_states_per_insn 18 total_states 1457 peak_states 308 mark_read 146
   -- END PROG LOAD LOG --
   libbpf: prog 'on_event': failed to load: -13
   libbpf: failed to load object 'strobemeta_subprogs.bpf.o'
   scale_test:FAIL:expect_success unexpected error: -13 (errno 13)
   #498     verif_scale_strobemeta_subprogs:FAIL
   Summary: 0/0 PASSED, 0 SKIPPED, 1 FAILED

The maximum stack size exceeded 512 bytes and caused verification failure.

The following llvm patch caused the above regression:
   https://github.com/llvm/llvm-project/pull/68882

I will do some analysis and try to find a solution to resolve this failure.

Thanks,

Yonghong

Re: FYI: bpf selftest verif_scale_strobemeta_subprogs failed with latest llvm19

[Andrii Nakryiko]

On Sun, Feb 4, 2024 at 10:58 PM Yonghong Song <yonghong.song@linux.dev> wrote:
>
> The selftest verif_scale_strobemeta_subprogs failed with latest llvm19 compiler.
> For example,
>
>    $ ./test_progs -n 498
>    ...
>    libbpf: prog 'on_event': BPF program load failed: Permission denied
>    libbpf: prog 'on_event': -- BEGIN PROG LOAD LOG --
>    combined stack size of 4 calls is 544. Too large
>    verification time 1417195 usec
>    stack depth 24+440+0+32

Is it a `struct strobe_map_raw map` in read_map_var()? I think we
should move it to a per-cpu array anyways (not saying we shouldn't fix
Clang regression), in production we've done this already a while ago
:)

>    processed 53561 insns (limit 1000000) max_states_per_insn 18 total_states 1457 peak_states 308 mark_read 146
>    -- END PROG LOAD LOG --
>    libbpf: prog 'on_event': failed to load: -13
>    libbpf: failed to load object 'strobemeta_subprogs.bpf.o'
>    scale_test:FAIL:expect_success unexpected error: -13 (errno 13)
>    #498     verif_scale_strobemeta_subprogs:FAIL
>    Summary: 0/0 PASSED, 0 SKIPPED, 1 FAILED
>
> The maximum stack size exceeded 512 bytes and caused verification failure.
>
> The following llvm patch caused the above regression:
>    https://github.com/llvm/llvm-project/pull/68882
>
> I will do some analysis and try to find a solution to resolve this failure.
>
> Thanks,
>
> Yonghong
>
>

Re: FYI: bpf selftest verif_scale_strobemeta_subprogs failed with latest llvm19

[Yonghong Song]

On 2/5/24 11:30 AM, Andrii Nakryiko wrote:
> On Sun, Feb 4, 2024 at 10:58 PM Yonghong Song <yonghong.song@linux.dev> wrote:
>> The selftest verif_scale_strobemeta_subprogs failed with latest llvm19 compiler.
>> For example,
>>
>>     $ ./test_progs -n 498
>>     ...
>>     libbpf: prog 'on_event': BPF program load failed: Permission denied
>>     libbpf: prog 'on_event': -- BEGIN PROG LOAD LOG --
>>     combined stack size of 4 calls is 544. Too large
>>     verification time 1417195 usec
>>     stack depth 24+440+0+32
> Is it a `struct strobe_map_raw map` in read_map_var()? I think we
> should move it to a per-cpu array anyways (not saying we shouldn't fix
> Clang regression), in production we've done this already a while ago
> :)

Yes, moving 'struct strobe_map_raw map' to be a global variable
does work. I just sent a kernel patch to reduce the stack requirement
for jitted code, which fixed this regression.
   https://lore.kernel.org/bpf/20240206063010.1352503-1-yonghong.song@linux.dev/

But let us keep the code as is for now, so we could use it to capture future
potential clang regressions.

>
>>     processed 53561 insns (limit 1000000) max_states_per_insn 18 total_states 1457 peak_states 308 mark_read 146
>>     -- END PROG LOAD LOG --
>>     libbpf: prog 'on_event': failed to load: -13
>>     libbpf: failed to load object 'strobemeta_subprogs.bpf.o'
>>     scale_test:FAIL:expect_success unexpected error: -13 (errno 13)
>>     #498     verif_scale_strobemeta_subprogs:FAIL
>>     Summary: 0/0 PASSED, 0 SKIPPED, 1 FAILED
>>
>> The maximum stack size exceeded 512 bytes and caused verification failure.
>>
>> The following llvm patch caused the above regression:
>>     https://github.com/llvm/llvm-project/pull/68882
>>
>> I will do some analysis and try to find a solution to resolve this failure.
>>
>> Thanks,
>>
>> Yonghong
>>
>>