Re: [PATCH 2/4] ftrace/instances: Clear function triggers when removing instances

Steven Rostedt
2017-05-16T02:20:07Z

On Sun, 14 May 2017 01:01:02 +0530
"Naveen N. Rao" <naveen.n.rao@linux.vnet.ibm.com> wrote:

> If instance directories are deleted while there are registered function
> triggers:
>
> # cd /sys/kernel/debug/tracing/instances
> # mkdir test
> # echo "schedule:enable_event:sched:sched_switch" > test/set_ftrace_filter
> # rmdir test
> Unable to handle kernel paging request for data at address 0x00000008
> Unable to handle kernel paging request for data at address 0x00000008
> Faulting instruction address: 0xc0000000021edde8
> Oops: Kernel access of bad area, sig: 11 [#1]
> SMP NR_CPUS=2048
> NUMA
> pSeries
> Modules linked in: iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 xt_tcpudp tun bridge stp llc kvm iptable_filter fuse binfmt_misc pseries_rng rng_core vmx_crypto ib_iser rdma_cm iw_cm ib_cm ib_core libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c multipath virtio_net virtio_blk virtio_pci crc32c_vpmsum virtio_ring virtio
> CPU: 8 PID: 8694 Comm: rmdir Not tainted 4.11.0-nnr+ #113
> task: c0000000bab52800 task.stack: c0000000baba0000
> NIP: c0000000021edde8 LR: c0000000021f0590 CTR: c000000002119620
> REGS: c0000000baba3870 TRAP: 0300 Not tainted (4.11.0-nnr+)
> MSR: 8000000000009033 <SF,EE,ME,IR,DR,RI,LE>
> CR: 22002422 XER: 20000000
> CFAR: 00007fffabb725a8 DAR: 0000000000000008 DSISR: 40000000 SOFTE: 0
> GPR00: c00000000220f750 c0000000baba3af0 c000000003157e00 0000000000000000
> GPR04: 0000000000000040 00000000000000eb 0000000000000040 0000000000000000
> GPR08: 0000000000000000 0000000000000113 0000000000000000 c00000000305db98
> GPR12: c000000002119620 c00000000fd42c00 0000000000000000 0000000000000000
> GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> GPR20: 0000000000000000 0000000000000000 c0000000bab52e90 0000000000000000
> GPR24: 0000000000000000 00000000000000eb 0000000000000040 c0000000baba3bb0
> GPR28: c00000009cb06eb0 c0000000bab52800 c00000009cb06eb0 c0000000baba3bb0
> NIP [c0000000021edde8] ring_buffer_lock_reserve+0x8/0x4e0
> LR [c0000000021f0590] trace_event_buffer_lock_reserve+0xe0/0x1a0
> Call Trace:
> [c0000000baba3af0] [c0000000021f96c8] trace_event_buffer_commit+0x1b8/0x280 (unreliable)
> [c0000000baba3b60] [c00000000220f750] trace_event_buffer_reserve+0x80/0xd0
> [c0000000baba3b90] [c0000000021196b8] trace_event_raw_event_sched_switch+0x98/0x180
> [c0000000baba3c10] [c0000000029d9980] __schedule+0x6e0/0xab0
> [c0000000baba3ce0] [c000000002122230] do_task_dead+0x70/0xc0
> [c0000000baba3d10] [c0000000020ea9c8] do_exit+0x828/0xd00
> [c0000000baba3dd0] [c0000000020eaf70] do_group_exit+0x60/0x100
> [c0000000baba3e10] [c0000000020eb034] SyS_exit_group+0x24/0x30
> [c0000000baba3e30] [c00000000200bcec] system_call+0x38/0x54
> Instruction dump:
> 60000000 60420000 7d244b78 7f63db78 4bffaa09 393efff8 793e0020 39200000
> 4bfffecc 60420000 3c4c00f7 3842a020 <81230008> 2f890000 409e02f0 a14d0008
> ---[ end trace b917b8985d0e650b ]---
> Unable to handle kernel paging request for data at address 0x00000008
> Faulting instruction address: 0xc0000000021edde8
> Unable to handle kernel paging request for data at address 0x00000008
> Faulting instruction address: 0xc0000000021edde8
> Faulting instruction address: 0xc0000000021edde8
>
> To address this, let's clear all registered function probes before
> deleting the ftrace instance.
>
> Reported-by: Michael Ellerman <mpe@ellerman.id.au>
> Signed-off-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
> ---
> kernel/trace/ftrace.c | 8 ++++++++
> kernel/trace/trace.c | 1 +
> kernel/trace/trace.h | 1 +
> 3 files changed, 10 insertions(+)
>
> diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
> index 28dc824ad072..1b96d927a082 100644
> --- a/kernel/trace/ftrace.c
> +++ b/kernel/trace/ftrace.c
> @@ -4256,6 +4256,14 @@ unregister_ftrace_function_probe_func(char *glob, struct trace_array *tr,
> return ret;
> }
>
> +void clear_ftrace_function_probes(struct trace_array *tr)
> +{
> + struct ftrace_func_probe *probe, *n;
> +
> + list_for_each_entry_safe(probe, n, &tr->func_probes, list)
> + unregister_ftrace_function_probe_func(NULL, tr, probe->probe_ops);
> +}
> +
> static LIST_HEAD(ftrace_commands);
> static DEFINE_MUTEX(ftrace_cmd_mutex);
>
> diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
> index c4536c449021..3f2aed4ad1ed 100644
> --- a/kernel/trace/trace.c
> +++ b/kernel/trace/trace.c
> @@ -7550,6 +7550,7 @@ static int instance_rmdir(const char *name)
> }
>
> tracing_set_nop(tr);
> + clear_ftrace_function_probes(tr);
> event_trace_del_tracer(tr);
> ftrace_clear_pids(tr);
> ftrace_destroy_function_files(tr);
> diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h
> index 291a1bca5748..98e0845f7235 100644
> --- a/kernel/trace/trace.h
> +++ b/kernel/trace/trace.h
> @@ -980,6 +980,7 @@ register_ftrace_function_probe(char *glob, struct trace_array *tr,
> extern int
> unregister_ftrace_function_probe_func(char *glob, struct trace_array *tr,
> struct ftrace_probe_ops *ops);
> +extern void clear_ftrace_function_probes(struct trace_array *tr);

This needs to have a stub function when CONFIG_DYNAMIC_FTRACE is not
defined. Otherwise we have:

kernel/trace/trace.c:7553:2: error: implicit declaration of function 'clear_ftrace_function_probes' [-Werror=implicit-function-declaration]
clear_ftrace_function_probes(tr);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~

-- Steve

>
> int register_ftrace_command(struct ftrace_func_command *cmd);
> int unregister_ftrace_command(struct ftrace_func_command *cmd);