Re: [PATCH 1/2 v2] ftrace: Be first to run code modification on modules

Masami Hiramatsu
2013-01-08T07:57:44Z

(2013/01/07 23:02), Steven Rostedt wrote:
> From: Steven Rostedt <srostedt@redhat.com>
>
> If some other kernel subsystem has a module notifier, and adds a kprobe
> to a ftrace mcount point (now that kprobes work on ftrace points),
> when the ftrace notifier runs it will fail and disable ftrace, as well
> as kprobes that are attached to ftrace points.
>
> Here's the error:
>
> WARNING: at kernel/trace/ftrace.c:1618 ftrace_bug+0x239/0x280()
> Hardware name: Bochs
> Modules linked in: fat(+) stap_56d28a51b3fe546293ca0700b10bcb29__8059(F) nfsv4 auth_rpcgss nfs dns_resolver fscache xt_nat iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack lockd sunrpc ppdev parport_pc parport microcode virtio_net i2c_piix4 drm_kms_helper ttm drm i2c_core [last unloaded: bid_shared]
> Pid: 8068, comm: modprobe Tainted: GF 3.7.0-0.rc8.git0.1.fc19.x86_64 #1
> Call Trace:
> [<ffffffff8105e70f>] warn_slowpath_common+0x7f/0xc0
> [<ffffffff81134106>] ? __probe_kernel_read+0x46/0x70
> [<ffffffffa0180000>] ? 0xffffffffa017ffff
> [<ffffffffa0180000>] ? 0xffffffffa017ffff
> [<ffffffff8105e76a>] warn_slowpath_null+0x1a/0x20
> [<ffffffff810fd189>] ftrace_bug+0x239/0x280
> [<ffffffff810fd626>] ftrace_process_locs+0x376/0x520
> [<ffffffff810fefb7>] ftrace_module_notify+0x47/0x50
> [<ffffffff8163912d>] notifier_call_chain+0x4d/0x70
> [<ffffffff810882f8>] __blocking_notifier_call_chain+0x58/0x80
> [<ffffffff81088336>] blocking_notifier_call_chain+0x16/0x20
> [<ffffffff810c2a23>] sys_init_module+0x73/0x220
> [<ffffffff8163d719>] system_call_fastpath+0x16/0x1b
> ---[ end trace 9ef46351e53bbf80 ]---
> ftrace failed to modify [<ffffffffa0180000>] init_once+0x0/0x20 [fat]
> actual: cc:bb:d2:4b:e1
>
> A kprobe was added to the init_once() function in the fat module on load.
> But this happened before ftrace could have touched the code. As ftrace
> didn't run yet, the kprobe system had no idea it was a ftrace point and
> simply added a breakpoint to the code (0xcc in the cc:bb:d2:4b:e1).
>
> Then when ftrace went to modify the location from a call to mcount/fentry
> into a nop, it didn't see a call op, but instead it saw the breakpoint op
> and not knowing what to do with it, ftrace shut itself down.
>
> The solution is to simply give the ftrace module notifier the max priority.
> This should have been done regardless, as the core code ftrace modification
> also happens very early on in boot up. This makes the module modification
> closer to core modification.

Correct! Thank you for fix that.

Acked-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>

> Reported-by: Frank Ch. Eigler <fche@redhat.com>
> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
> ---
> kernel/trace/ftrace.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
> index 51b7159..356bc2f 100644
> --- a/kernel/trace/ftrace.c
> +++ b/kernel/trace/ftrace.c
> @@ -3998,7 +3998,7 @@ static int ftrace_module_notify(struct notifier_block *self,
>
> struct notifier_block ftrace_module_nb = {
> .notifier_call = ftrace_module_notify,
> - .priority = 0,
> + .priority = INT_MAX, /* Run before anything that can use kprobes */
> };
>
> extern unsigned long __start_mcount_loc[];
>

--
Masami HIRAMATSU
IT Management Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com