Re: bpf: Question about odd BPF verifier behaviour

[Eduard Zingerman <eddyz87@gmail.com>]

On Thu, 2023-02-23 at 09:37 +0000, Matt Bobrowski wrote:
[...]
> LMK whether you need any more information.
> 
> /M

Hi Matt,

Unfortunately I can't reproduce the issue.
Here are the versions of the tools/repos that I use:

- kernel (tried both):
  - https://github.com/torvalds/linux.git
    a5c95ca18a98 ("Merge tag 'drm-next-2023-02-23' of git://anongit.freedesktop.org/drm/drm")
  - https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git
    830b3c68c1fb ("Linux 6.1")
- config (tried both):
  - one obtained using your instructions
  - my small debug config for executing BPF tests ([1])
- LLVM:
  https://github.com/llvm/llvm-project.git
  bc85cf168743 ("[TextAPI] Add support for TBDv5 Files to nm & tapi-diff")
- pahole:
  git@github.com:acmel/dwarves.git
  ef68019 ("pahole: Update man page for options also")
- libbpf-bootstrap (just followed your instructions):
  https://github.com/libbpf/libbpf-bootstrap
  db4f7ad ("cmake: Fix btf header missing in legacy kernel env.")
- gcc (from my distro):
  gcc version 11.3.0 (Ubuntu 11.3.0-1ubuntu1~22.04)
- cat /etc/os-release 
  NAME="Linux Mint"
  VERSION="21.1 (Vera)"
  ID=linuxmint
  ID_LIKE="ubuntu debian"
  PRETTY_NAME="Linux Mint 21.1"
  VERSION_ID="21.1"
  VERSION_CODENAME=vera
  UBUNTU_CODENAME=jammy

Could you please copy-paste output of the `fentry` application, I'd
like to see the log output of the libbpf while it processes
relocations, e.g. here is what it prints for me:

    # /home/eddy/work/libbpf-bootstrap/examples/c/fentry
    libbpf: loading object 'fentry_bpf' from buffer
    libbpf: elf: section(3) lsm.s/bprm_committed_creds, size 136, link 0, flags 6, type=1
    libbpf: sec 'lsm.s/bprm_committed_creds': found program 'dbg' at insn offset 0 (0 bytes), code size 17 insns (136 bytes)
    libbpf: elf: section(4) license, size 13, link 0, flags 3, type=1
    libbpf: license of fentry_bpf is Dual BSD/GPL
    libbpf: elf: section(5) .BTF, size 5114, link 0, flags 0, type=1
    libbpf: elf: section(7) .BTF.ext, size 188, link 0, flags 0, type=1
    libbpf: elf: section(10) .symtab, size 96, link 1, flags 0, type=2
    libbpf: looking for externs among 4 symbols...
    libbpf: collected 0 externs total
    libbpf: loading kernel BTF '/sys/kernel/btf/vmlinux': 0
    libbpf: sec 'lsm.s/bprm_committed_creds': found 1 CO-RE relocations
    libbpf: CO-RE relocating [6] struct linux_binprm: found target candidate [7241] struct linux_binprm in [vmlinux]
    libbpf: prog 'dbg': relo #0: <byte_off> [6] struct linux_binprm.file (0:11 @ offset 64)
    libbpf: prog 'dbg': relo #0: matching candidate #0 <byte_off> [7241] struct linux_binprm.file (0:11 @ offset 64)
    libbpf: prog 'dbg': relo #0: patched insn #10 (LDX/ST/STX) off 64 -> 64
    Successfully started! Please run `sudo cat /sys/kernel/debug/tracing/trace_pipe` to see output of the BPF programs.

Also, could you please compile `veristat` tool as below:

    cd ${kernel}/tools/testing/selftests/bpf
    make -j16 veristat

And post the output of the following command (from within QEMU):

    ./veristat -l7 -v ${path-to-libbpf-bootstrap-within-vm}/examples/c/.output/fentry.bpf.o

It should produce the verification log as an output.

The reason I'm asking is that your verification log looks kinda strange:

>    ; bpf_ima_file_hash(bprm->file, buf, 64);
>    13: (b7) r3 = 64                      ; R3_w=64
>    14: (85) call bpf_ima_file_hash#193
>    cannot access ptr member next with moff 0 in struct llist_node with off 0 size 1
>    R1 is of type file but file is expected
>    processed 15 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0

I don't understand why it mentions `struct llist_node` here and don't
have such messages in my log ([2]).

Thanks,
Eduard

[1] My config for BPF testing
    https://gist.github.com/eddyz87/aca79692d7bf57cfdd01b283b4304fd8
[2] Veristat verification log
    https://gist.github.com/eddyz87/49b211740bf99c426a37a3555b4542a3