Re: kprobe not always triggering

[Daniel Wagner <dwagner@suse.de>]

On Thu, Jan 16, 2025 at 08:48:18AM +0900, Masami Hiramatsu wrote:
> Have you tried to use `perf probe`? Even if it is inlined, you can find them
> by
> 
>  perf probe -a nvme_change_ctrl_state new_state

I had to build my own perf, as the version from the distro didn't like
my kernel and just crashed. Thus I added a bunch libraries to my build
system. The resulting perf installed the probe and the missing events
show up:

          <idle>-0       [000] ..s1.   245.406139: nvme_complete_rq: nvme1: qid=0, cmdid=36865, res=0x0, retries=0, flags=0x0, status=0x371
          <idle>-0       [000] .Ns1.   245.407192: nvme_change_ctrl_state: (nvme_change_ctrl_state+0x0/0x180 [nvme_core]) new_state=0x2
   kworker/u33:0-65      [000] .N...   245.407258: nvme_change_ctrl_state: (nvme_change_ctrl_state+0x0/0x180 [nvme_core]) new_state=0x2
   kworker/u33:1-78      [000] .....   245.421290: nvme_change_ctrl_state: (nvme_change_ctrl_state+0x0/0x180 [nvme_core]) new_state=0x3

I double checked my previous setup with kprobes and now the missing
events show up as well. It looks like my setup changes makes clang to
not inline this function anymore, or there is no additional debug info
available?

> But perhaps, some of them are inlined with constant numbers (maybe you can
> find them as nvme_change_ctrl_state.constprop.XXX in kallsyms)

# cat /proc/kallsyms | grep nvme_change_ctrl_state
ffffffffc03dc04d r __kstrtab_nvme_change_ctrl_state     [nvme_core]
ffffffffc03dc064 r __kstrtabns_nvme_change_ctrl_state   [nvme_core]
ffffffffc03ca130 r __ksymtab_nvme_change_ctrl_state     [nvme_core]
ffffffffc0405de0 t __pfx_nvme_change_ctrl_state [nvme_core]
ffffffffc0405df0 t nvme_change_ctrl_state       [nvme_core]

This is with the new kernel, so after installing the libraries for the
perf build.

> If my guess is correct, it is not inlined but the constant propagation
> optimization[1] has been done by the compiler.
> 
> [1] https://en.wikipedia.org/wiki/Constant_folding#Constant_propagation
> 
> IIRC, commit 66f69b219716 ("perf probe: Support DW_AT_const_value constant value")
> should support this case.

Okay, this commit is already in (v5.4), so it was always in.

Anyway, I can't reproduce anymore.

Daniel