[GIT PULL] ftrace: Updates for 6.15

[GIT PULL] ftrace: Updates for 6.15

[Steven Rostedt]

Linus,

ftrace changes for v6.15:

- Record function parameters for function and function graph tracers

  An option has been added to function tracer (func-args) and the function
  graph tracer (funcgraph-args) that when set, the tracers will record the
  registers that hold the arguments into each function event. On reading of
  the trace, it will use BTF to print those arguments. Most archs support up
  to 6 arguments (depending on the complexity of the arguments) and those
  are printed. If a function has more arguments then what was recorded, the
  output will end with " ... )".

    Example of function graph tracer:

    6)              | dummy_xmit [dummy](skb = 0x8887c100, dev = 0x872ca000) {
    6)              |   consume_skb(skb = 0x8887c100) {
    6)              |     skb_release_head_state(skb = 0x8887c100) {
    6)  0.178 us    |       sock_wfree(skb = 0x8887c100)
    6)  0.627 us    |     }

- The rest of the changes are minor clean ups and fixes


Please pull the latest ftrace-v6.15 tree, which can be found at:


  git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace.git
ftrace-v6.15

Tag SHA1: d96c183fee82a7cb9e3875c6e65fae637768d592
Head SHA1: 391dda1bd7c56de62b96126214f040fe8965561b


Haiyue Wang (1):
      fgraph: Correct typo in ftrace_return_to_handler comment

Jiapeng Chong (1):
      function_graph: Remove the unused variable func

Sasha Levin (1):
      tracing: Use hashtable.h for event_hash

Steven Rostedt (2):
      ftrace: Have ftrace_free_filter() WARN and exit if ops is active
      ftrace: Have funcgraph-args take affect during tracing

Sven Schnelle (3):
      ftrace: Add print_function_args()
      ftrace: Add support for function argument to graph tracer
      ftrace: Add arguments to function tracer

Tengda Wu (1):
      tracing: Fix use-after-free in print_graph_function_flags during tracer switching

----
 include/linux/ftrace_regs.h          |   5 +
 kernel/trace/Kconfig                 |  12 +++
 kernel/trace/fgraph.c                |   2 +-
 kernel/trace/ftrace.c                |   2 +
 kernel/trace/trace.c                 |  14 ++-
 kernel/trace/trace.h                 |   5 +-
 kernel/trace/trace_entries.h         |  12 ++-
 kernel/trace/trace_functions.c       |  46 ++++++++-
 kernel/trace/trace_functions_graph.c | 176 ++++++++++++++++++++++++++++-------
 kernel/trace/trace_irqsoff.c         |  14 ++-
 kernel/trace/trace_output.c          | 122 +++++++++++++++++++++---
 kernel/trace/trace_output.h          |   9 ++
 kernel/trace/trace_sched_wakeup.c    |   6 +-
 13 files changed, 351 insertions(+), 74 deletions(-)
---------------------------

Re: [GIT PULL] ftrace: Updates for 6.15

[pr-tracker-bot]

The pull request you sent on Tue, 25 Mar 2025 19:39:35 -0400:

> git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace.git ftrace-v6.15

has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/31eb415bf6f06c90fdd9b635caf3a6c5110a38b6

Thank you!

-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html

Re: [GIT PULL] ftrace: Updates for 6.15

[Christian Loehle]

________________________________________
From: pr-tracker-bot@kernel.org <pr-tracker-bot@kernel.org>
Sent: 28 March 2025 00:09
To: Steven Rostedt <rostedt@goodmis.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>; LKML <linux-kernel@vger.kernel.org>; Masami Hiramatsu <mhiramat@kernel.org>; Mark Rutland <Mark.Rutland@arm.com>; Mathieu Desnoyers <mathieu.desnoyers@efficios.com>; Andrew Morton <akpm@linux-foundation.org>; Haiyue Wang <haiyuewa@163.com>; Jiapeng Chong <jiapeng.chong@linux.alibaba.com>; Sasha Levin <sashal@kernel.org>; Sven Schnelle <svens@linux.ibm.com>; Tengda Wu <wutengda@huaweicloud.com>
Subject: Re: [GIT PULL] ftrace: Updates for 6.15
 
 
>The pull request you sent on Tue, 25 Mar 2025 19:39:35 -0400:
>
>> git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace.git ftrace-v6.15
>
>has been merged into torvalds/linux.git:
>https://git.kernel.org/torvalds/c/31eb415bf6f06c90fdd9b635caf3a6c5110a38b6
>
>Thank you!
>
>--
>Deet-doot-dot, I am a bot.
>https://korg.docs.kernel.org/prtracker.html

Hi Steven,
I'm pretty sure this causes the build to fail on linus' tree and next:
4e82c87058f4 (HEAD -> master, origin/master, origin/HEAD) Merge tag 'rust-6.15' of git://git.kernel.org/pub/scm/linux/kernel/git/ojeda/linux

aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!
aarch64-linux-gnu-ld: kernel/trace/trace_output.o: in function `print_function_args':
/home/chrloe01/development/linux-mainline/kernel/trace/trace_output.c:712: undefined reference to `btf_find_func_proto'
aarch64-linux-gnu-ld: /home/chrloe01/development/linux-mainline/kernel/trace/trace_output.c:716: undefined reference to `btf_get_func_param'
make[2]: *** [scripts/Makefile.vmlinux:77: vmlinux] Error 1
make[1]: *** [/data_nvme1n1/chrloe01/development/linux-mainline/Makefile:1234: vmlinux] Error 2
make: *** [Makefile:251: __sub-make] Error 2

Sound familiar?
grep BTF .config
CONFIG_DEBUG_INFO_BTF=y
CONFIG_PAHOLE_HAS_SPLIT_BTF=y
CONFIG_DEBUG_INFO_BTF_MODULES=y
# CONFIG_MODULE_ALLOW_BTF_MISMATCH is not set

Enabling CONFIG_PROBE_EVENTS_BTF_ARGS passes the build.

Re: [GIT PULL] ftrace: Updates for 6.15

[Christian Loehle]

On 3/31/25 17:57, Christian Loehle wrote:
> 
> 
> ________________________________________
> From: pr-tracker-bot@kernel.org <pr-tracker-bot@kernel.org>
> Sent: 28 March 2025 00:09
> To: Steven Rostedt <rostedt@goodmis.org>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>; LKML <linux-kernel@vger.kernel.org>; Masami Hiramatsu <mhiramat@kernel.org>; Mark Rutland <Mark.Rutland@arm.com>; Mathieu Desnoyers <mathieu.desnoyers@efficios.com>; Andrew Morton <akpm@linux-foundation.org>; Haiyue Wang <haiyuewa@163.com>; Jiapeng Chong <jiapeng.chong@linux.alibaba.com>; Sasha Levin <sashal@kernel.org>; Sven Schnelle <svens@linux.ibm.com>; Tengda Wu <wutengda@huaweicloud.com>
> Subject: Re: [GIT PULL] ftrace: Updates for 6.15
> 
> 
>> The pull request you sent on Tue, 25 Mar 2025 19:39:35 -0400:
>>
>>> git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace.git ftrace-v6.15
>>
>> has been merged into torvalds/linux.git:
>> https://git.kernel.org/torvalds/c/31eb415bf6f06c90fdd9b635caf3a6c5110a38b6
>>
>> Thank you!
>>
>> --
>> Deet-doot-dot, I am a bot.
>> https://korg.docs.kernel.org/prtracker.html
> 
> Hi Steven,
> I'm pretty sure this causes the build to fail on linus' tree and next:
> 4e82c87058f4 (HEAD -> master, origin/master, origin/HEAD) Merge tag 'rust-6.15' of git://git.kernel.org/pub/scm/linux/kernel/git/ojeda/linux
> 
> aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
> aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!
> aarch64-linux-gnu-ld: kernel/trace/trace_output.o: in function `print_function_args':
> /home/chrloe01/development/linux-mainline/kernel/trace/trace_output.c:712: undefined reference to `btf_find_func_proto'
> aarch64-linux-gnu-ld: /home/chrloe01/development/linux-mainline/kernel/trace/trace_output.c:716: undefined reference to `btf_get_func_param'
> make[2]: *** [scripts/Makefile.vmlinux:77: vmlinux] Error 1
> make[1]: *** [/data_nvme1n1/chrloe01/development/linux-mainline/Makefile:1234: vmlinux] Error 2
> make: *** [Makefile:251: __sub-make] Error 2
> 
> Sound familiar?
> grep BTF .config
> CONFIG_DEBUG_INFO_BTF=y
> CONFIG_PAHOLE_HAS_SPLIT_BTF=y
> CONFIG_DEBUG_INFO_BTF_MODULES=y
> # CONFIG_MODULE_ALLOW_BTF_MISMATCH is not set
> 
> Enabling CONFIG_PROBE_EVENTS_BTF_ARGS passes the build.
> 

In linus' tree we have:
76fe0337c219 ftrace: Add arguments to function tracer
c7a60a733c37 ftrace: Have funcgraph-args take affect during tracing
ff5c9c576e75 ftrace: Add support for function argument to graph tracer
533c20b062d7 ftrace: Add print_function_args()
3d44e1d1575a md: switch personalities to use md_submodule_head

3d44e1d1575a md: switch personalities to use md_submodule_head
builds fine.

533c20b062d7 ftrace: Add print_function_args()
is the first one to fail but with:
kernel/trace/trace_output.c: In function ‘print_function_args’:
kernel/trace/trace_output.c:724:26: error: ‘FTRACE_REGS_MAX_ARGS’ undeclared (first use in this function)
  724 |                 if (a == FTRACE_REGS_MAX_ARGS) {
      |                          ^~~~~~~~~~~~~~~~~~~~
kernel/trace/trace_output.c:724:26: note: each undeclared identifier is reported only once for each function it appears in

then 
ff5c9c576e75 ftrace: Add support for function argument to graph tracer
shows

aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!
aarch64-linux-gnu-ld: kernel/trace/trace_output.o: in function `print_function_args':
/home/chrloe01/development/linux-mainline/kernel/trace/trace_output.c:711: undefined reference to `btf_find_func_proto'
aarch64-linux-gnu-ld: /home/chrloe01/development/linux-mainline/kernel/trace/trace_output.c:715: undefined reference to `btf_get_func_param'
make[2]: *** [scripts/Makefile.vmlinux:77: vmlinux] Error 1
make[1]: *** [/data_nvme1n1/chrloe01/development/linux-mainline/Makefile:1226: vmlinux] Error 2
make: *** [Makefile:251: __sub-make] Error 2
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j16 -s Image  5747.92s user 554.69s system 830% cpu 12:38.50 total

fails just like reported linus' tree.

Re: [GIT PULL] ftrace: Updates for 6.15

[Leon Romanovsky]

On Mon, Mar 31, 2025 at 04:57:53PM +0000, Christian Loehle wrote:
> 
> 
> ________________________________________
> From: pr-tracker-bot@kernel.org <pr-tracker-bot@kernel.org>
> Sent: 28 March 2025 00:09
> To: Steven Rostedt <rostedt@goodmis.org>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>; LKML <linux-kernel@vger.kernel.org>; Masami Hiramatsu <mhiramat@kernel.org>; Mark Rutland <Mark.Rutland@arm.com>; Mathieu Desnoyers <mathieu.desnoyers@efficios.com>; Andrew Morton <akpm@linux-foundation.org>; Haiyue Wang <haiyuewa@163.com>; Jiapeng Chong <jiapeng.chong@linux.alibaba.com>; Sasha Levin <sashal@kernel.org>; Sven Schnelle <svens@linux.ibm.com>; Tengda Wu <wutengda@huaweicloud.com>
> Subject: Re: [GIT PULL] ftrace: Updates for 6.15
>  
>  
> >The pull request you sent on Tue, 25 Mar 2025 19:39:35 -0400:
> >
> >> git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace.git ftrace-v6.15
> >
> >has been merged into torvalds/linux.git:
> >https://git.kernel.org/torvalds/c/31eb415bf6f06c90fdd9b635caf3a6c5110a38b6
> >
> >Thank you!
> >
> >--
> >Deet-doot-dot, I am a bot.
> >https://korg.docs.kernel.org/prtracker.html
> 
> Hi Steven,
> I'm pretty sure this causes the build to fail on linus' tree and next:
> 4e82c87058f4 (HEAD -> master, origin/master, origin/HEAD) Merge tag 'rust-6.15' of git://git.kernel.org/pub/scm/linux/kernel/git/ojeda/linux
> 
> aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
> aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!
> aarch64-linux-gnu-ld: kernel/trace/trace_output.o: in function `print_function_args':
> /home/chrloe01/development/linux-mainline/kernel/trace/trace_output.c:712: undefined reference to `btf_find_func_proto'
> aarch64-linux-gnu-ld: /home/chrloe01/development/linux-mainline/kernel/trace/trace_output.c:716: undefined reference to `btf_get_func_param'
> make[2]: *** [scripts/Makefile.vmlinux:77: vmlinux] Error 1
> make[1]: *** [/data_nvme1n1/chrloe01/development/linux-mainline/Makefile:1234: vmlinux] Error 2
> make: *** [Makefile:251: __sub-make] Error 2

We see same issue in our CI systems.

13:29:09  + make -s -j64 ARCH=x86
13:32:31  ld: kernel/trace/trace_output.o: in function `print_function_args':
13:32:31  /home/jenkins/agent/workspace/kernel_build/linux/kernel/trace/trace_output.c:713:(.text+0x20f1): undefined reference to `btf_find_func_proto'
13:32:31  ld: /home/jenkins/agent/workspace/kernel_build/linux/kernel/trace/trace_output.c:717:(.text+0x210d): undefined reference to `btf_get_func_param'
13:32:31  make[2]: *** [scripts/Makefile.vmlinux:77: vmlinux] Error 1
13:32:31  make[1]: *** [/home/jenkins/agent/workspace/kernel_build/linux/Makefile:1234: vmlinux] Error 2
13:32:31  make: *** [Makefile:251: __sub-make] Error 2

Thanks

> 
> Sound familiar?
> grep BTF .config
> CONFIG_DEBUG_INFO_BTF=y
> CONFIG_PAHOLE_HAS_SPLIT_BTF=y
> CONFIG_DEBUG_INFO_BTF_MODULES=y
> # CONFIG_MODULE_ALLOW_BTF_MISMATCH is not set
> 
> Enabling CONFIG_PROBE_EVENTS_BTF_ARGS passes the build.
> 
> 

Re: [GIT PULL] ftrace: Updates for 6.15

[Steven Rostedt]

On Tue, 1 Apr 2025 14:31:59 +0300
Leon Romanovsky <leon@kernel.org> wrote:

> We see same issue in our CI systems.
> 
> 13:29:09  + make -s -j64 ARCH=x86
> 13:32:31  ld: kernel/trace/trace_output.o: in function `print_function_args':
> 13:32:31  /home/jenkins/agent/workspace/kernel_build/linux/kernel/trace/trace_output.c:713:(.text+0x20f1): undefined reference to `btf_find_func_proto'
> 13:32:31  ld: /home/jenkins/agent/workspace/kernel_build/linux/kernel/trace/trace_output.c:717:(.text+0x210d): undefined reference to `btf_get_func_param'
> 13:32:31  make[2]: *** [scripts/Makefile.vmlinux:77: vmlinux] Error 1
> 13:32:31  make[1]: *** [/home/jenkins/agent/workspace/kernel_build/linux/Makefile:1234: vmlinux] Error 2
> 13:32:31  make: *** [Makefile:251: __sub-make] Error 2

Looks like a dependency is missing and/or an #ifdef block needs to be added.

Thanks for the report, I'll take a look into it.

-- Steve