From: zhouchengming <zhouchengming1@huawei.com>
To: Borislav Petkov <bp@suse.de>
Cc: <mhiramat@kernel.org>, <ananth@linux.vnet.ibm.com>,
<anil.s.keshavamurthy@intel.com>, <davem@davemloft.net>,
<hpa@zytor.com>, <tglx@linutronix.de>, <peterz@infradead.org>,
<jkosina@suse.cz>, <rostedt@goodmis.org>, <mjurczyk@google.com>,
<x86@kernel.org>, <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] kprobes, x86/alternatives: use text_mutex to protect smp_alt_modules
Date: Fri, 27 Oct 2017 19:42:45 +0800 [thread overview]
Message-ID: <59F31BB5.90905@huawei.com> (raw)
In-Reply-To: <20171027111527.GD1305@nazgul.tnic>
On 2017/10/27 19:15, Borislav Petkov wrote:
> On Fri, Oct 27, 2017 at 05:34:44PM +0800, Zhou Chengming wrote:
>> Fixes: 2cfa197 "ftrace/alternatives: Introducing *_text_reserved
>> functions"
>>
>> We use alternatives_text_reserved() to check if the address is in
>> the fixed pieces of alternative reserved, but the problem is that
>> we don't hold the smp_alt mutex when call this function. So the list
>> traversal may encounter a deleted list_head if another path is doing
>> alternatives_smp_module_del().
> Is this something you've triggered on a real machine or is this just
> from code staring?
>
Hi, thanks for your reply.
This is a real bug happened on one of our machines, below is the calltrace.
We can see the trigger is at alternatives_text_reserved+0x20/0x80, and
encounter a deleted (poisoned) list_head.
[ 14.016190] general protection fault: 0000 [#1] SMP
[ 14.016988] CPU 0
[ 14.017287] Modules linked in: mlx4_ib(O+) mlx4_en(O+) ib_sa(O) ib_mad(O) ib_core(O) ib_addr(O) ipv6 mlx4_core(O) compat(O) igb(O) rtc_cmos dca button ata_piix ahci libahci libata ext3 jbd mbcache usbhid hid uhci_hcd ehci_hcd processor thermal_sys hwmon usbcore usb_common sd_mod crc_t10dif virtio_console(O) virtio_pci(O) virtio_net(O) kvm_ivshmem(O) virtio_scsi(O) scsi_mod virtio_blk(O) virtio(O) virtio_ring(O) pv_channel(O)
[ 14.020005]
[ 14.020005] Pid: 1979, comm: modprobe Tainted: G O 3.4.24.19-0.11-default #1 QEMU Standard PC (i440FX + PIIX, 1996)
[ 14.020005] RIP: 0010:[<ffffffff81007eb0>] [<ffffffff81007eb0>] alternatives_text_reserved+0x20/0x80
[ 14.020005] RSP: 0018:ffff880ea355bcb8 EFLAGS: 00010283
[ 14.020005] RAX: dead000000000100 RBX: ffffffffa02af720 RCX: dead0000000000d0
[ 14.020005] RDX: ffffffffa02f0588 RSI: ffffffffa02d2fc0 RDI: ffffffffa02d2fc0
[ 14.020005] RBP: ffff880ea355bcb8 R08: ffffffffa02f3b68 R09: 00017f4ae12d2fc0
[ 14.020005] R10: 00000000000000e8 R11: ffffffffa02bb9d7 R12: 0000000000000000
[ 14.020005] R13: ffffffffa02af720 R14: ffffffffa0307140 R15: ffffffffa02af730
[ 14.020005] FS: 00007f26c6acc700(0000) GS:ffff880fff200000(0000) knlGS:0000000000000000
[ 14.020005] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 14.020005] CR2: 00007fd4adc3b000 CR3: 0000000ea40ea000 CR4: 00000000001407f0
[ 14.020005] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 14.020005] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 14.020005] Process modprobe (pid: 1979, threadinfo ffff880ea355a000, task ffff880e9eb9e600)
[ 14.020005] Stack:
[ 14.020005] ffff880ea355bcd8 ffffffff8145b148 ffffffffa02af720 ffffffffa02af720
[ 14.020005] ffff880ea355bd18 ffffffff8145ed38 0000000000000000 0000000000000000
[ 14.041015] ffff880ea355bd90 ffffffffa02af720 0000000000000001 ffff880ea355bd90
[ 14.041015] Call Trace:
[ 14.041015] [<ffffffff8145b148>] arch_prepare_kprobe+0x18/0x80
[ 14.042982] [<ffffffff8145ed38>] register_kprobe+0x338/0x4c0
[ 14.042982] [<ffffffff8145f658>] register_jprobes+0x98/0xc0
[ 14.042982] [<ffffffff8145f69a>] register_jprobe+0x1a/0x20
[ 14.042982] [<ffffffffa02a5f5d>] mlx4_stats_sysfs_create+0x2d/0x150 [mlx4_en]
[ 14.042982] [<ffffffffa02a4087>] mlx4_en_init_netdev+0xc77/0xe50 [mlx4_en]
[ 14.042982] [<ffffffffa029263a>] mlx4_en_add+0x44a/0x550 [mlx4_en]
[ 14.042982] [<ffffffffa02d2d1c>] mlx4_add_device+0x4c/0xe0 [mlx4_core]
[ 14.042982] [<ffffffffa02b6000>] ? 0xffffffffa02b5fff
[ 14.042982] [<ffffffffa02d2e23>] mlx4_register_interface+0x73/0xb0 [mlx4_core]
[ 14.042982] [<ffffffffa02b6000>] ? 0xffffffffa02b5fff
[ 14.042982] [<ffffffffa02b6031>] __init_backport+0x31/0x1000 [mlx4_en]
[ 14.042982] [<ffffffff810002b2>] do_one_initcall+0x122/0x180
[ 14.042982] [<ffffffff8109ed2d>] sys_init_module+0xbd/0x220
[ 14.042982] [<ffffffff81460c99>] system_call_fastpath+0x16/0x1b
[ 14.042982] Code: 66 66 2e 0f 1f 84 00 00 00 00 00 48 8b 05 49 e9 80 00 55 48 89 e5 48 3d e0 67 81 81 48 8d 48 d0 74 59 66 0f 1f 84 00 00 00 00 00 <48> 3b 71 20 72 3a 48 3b 79 28 77 34 48 8b 41 10 4c 8b 41 18 4c
[ 14.042982] RIP [<ffffffff81007eb0>] alternatives_text_reserved+0x20/0x80
[ 14.042982] RSP <ffff880ea355bcb8>
next prev parent reply other threads:[~2017-10-27 11:43 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-10-27 9:34 [PATCH] kprobes, x86/alternatives: use text_mutex to protect smp_alt_modules Zhou Chengming
2017-10-27 11:15 ` Borislav Petkov
2017-10-27 11:42 ` zhouchengming [this message]
2017-10-27 12:33 ` Borislav Petkov
2017-10-27 13:30 ` zhouchengming
2017-10-28 8:43 ` Masami Hiramatsu
2017-10-28 9:51 ` zhouchengming
2017-10-27 14:15 ` Peter Zijlstra
2017-10-28 1:26 ` zhouchengming
2017-10-28 8:44 ` Masami Hiramatsu
2017-10-30 8:03 ` Masami Hiramatsu
2017-10-31 21:59 ` Steven Rostedt
2017-11-01 1:48 ` zhouchengming
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=59F31BB5.90905@huawei.com \
--to=zhouchengming1@huawei.com \
--cc=ananth@linux.vnet.ibm.com \
--cc=anil.s.keshavamurthy@intel.com \
--cc=bp@suse.de \
--cc=davem@davemloft.net \
--cc=hpa@zytor.com \
--cc=jkosina@suse.cz \
--cc=linux-kernel@vger.kernel.org \
--cc=mhiramat@kernel.org \
--cc=mjurczyk@google.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox