From: zhouchengming <zhouchengming1@huawei.com>
To: Borislav Petkov <bp@suse.de>
Cc: <mhiramat@kernel.org>, <ananth@linux.vnet.ibm.com>,
<anil.s.keshavamurthy@intel.com>, <davem@davemloft.net>,
<hpa@zytor.com>, <tglx@linutronix.de>, <peterz@infradead.org>,
<jkosina@suse.cz>, <rostedt@goodmis.org>, <mjurczyk@google.com>,
<x86@kernel.org>, <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] kprobes, x86/alternatives: use text_mutex to protect smp_alt_modules
Date: Fri, 27 Oct 2017 19:42:45 +0800 [thread overview]
Message-ID: <59F31BB5.90905@huawei.com> (raw)
In-Reply-To: <20171027111527.GD1305@nazgul.tnic>
On 2017/10/27 19:15, Borislav Petkov wrote:
> On Fri, Oct 27, 2017 at 05:34:44PM +0800, Zhou Chengming wrote:
>> Fixes: 2cfa197 "ftrace/alternatives: Introducing *_text_reserved
>> functions"
>>
>> We use alternatives_text_reserved() to check if the address is in
>> the fixed pieces of alternative reserved, but the problem is that
>> we don't hold the smp_alt mutex when call this function. So the list
>> traversal may encounter a deleted list_head if another path is doing
>> alternatives_smp_module_del().
> Is this something you've triggered on a real machine or is this just
> from code staring?
>
Hi, thanks for your reply.
This is a real bug happened on one of our machines, below is the calltrace.
We can see the trigger is at alternatives_text_reserved+0x20/0x80, and
encounter a deleted (poisoned) list_head.
[ 14.016190] general protection fault: 0000 [#1] SMP
[ 14.016988] CPU 0
[ 14.017287] Modules linked in: mlx4_ib(O+) mlx4_en(O+) ib_sa(O) ib_mad(O) ib_core(O) ib_addr(O) ipv6 mlx4_core(O) compat(O) igb(O) rtc_cmos dca button ata_piix ahci libahci libata ext3 jbd mbcache usbhid hid uhci_hcd ehci_hcd processor thermal_sys hwmon usbcore usb_common sd_mod crc_t10dif virtio_console(O) virtio_pci(O) virtio_net(O) kvm_ivshmem(O) virtio_scsi(O) scsi_mod virtio_blk(O) virtio(O) virtio_ring(O) pv_channel(O)
[ 14.020005]
[ 14.020005] Pid: 1979, comm: modprobe Tainted: G O 3.4.24.19-0.11-default #1 QEMU Standard PC (i440FX + PIIX, 1996)
[ 14.020005] RIP: 0010:[<ffffffff81007eb0>] [<ffffffff81007eb0>] alternatives_text_reserved+0x20/0x80
[ 14.020005] RSP: 0018:ffff880ea355bcb8 EFLAGS: 00010283
[ 14.020005] RAX: dead000000000100 RBX: ffffffffa02af720 RCX: dead0000000000d0
[ 14.020005] RDX: ffffffffa02f0588 RSI: ffffffffa02d2fc0 RDI: ffffffffa02d2fc0
[ 14.020005] RBP: ffff880ea355bcb8 R08: ffffffffa02f3b68 R09: 00017f4ae12d2fc0
[ 14.020005] R10: 00000000000000e8 R11: ffffffffa02bb9d7 R12: 0000000000000000
[ 14.020005] R13: ffffffffa02af720 R14: ffffffffa0307140 R15: ffffffffa02af730
[ 14.020005] FS: 00007f26c6acc700(0000) GS:ffff880fff200000(0000) knlGS:0000000000000000
[ 14.020005] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 14.020005] CR2: 00007fd4adc3b000 CR3: 0000000ea40ea000 CR4: 00000000001407f0
[ 14.020005] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 14.020005] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 14.020005] Process modprobe (pid: 1979, threadinfo ffff880ea355a000, task ffff880e9eb9e600)
[ 14.020005] Stack:
[ 14.020005] ffff880ea355bcd8 ffffffff8145b148 ffffffffa02af720 ffffffffa02af720
[ 14.020005] ffff880ea355bd18 ffffffff8145ed38 0000000000000000 0000000000000000
[ 14.041015] ffff880ea355bd90 ffffffffa02af720 0000000000000001 ffff880ea355bd90
[ 14.041015] Call Trace:
[ 14.041015] [<ffffffff8145b148>] arch_prepare_kprobe+0x18/0x80
[ 14.042982] [<ffffffff8145ed38>] register_kprobe+0x338/0x4c0
[ 14.042982] [<ffffffff8145f658>] register_jprobes+0x98/0xc0
[ 14.042982] [<ffffffff8145f69a>] register_jprobe+0x1a/0x20
[ 14.042982] [<ffffffffa02a5f5d>] mlx4_stats_sysfs_create+0x2d/0x150 [mlx4_en]
[ 14.042982] [<ffffffffa02a4087>] mlx4_en_init_netdev+0xc77/0xe50 [mlx4_en]
[ 14.042982] [<ffffffffa029263a>] mlx4_en_add+0x44a/0x550 [mlx4_en]
[ 14.042982] [<ffffffffa02d2d1c>] mlx4_add_device+0x4c/0xe0 [mlx4_core]
[ 14.042982] [<ffffffffa02b6000>] ? 0xffffffffa02b5fff
[ 14.042982] [<ffffffffa02d2e23>] mlx4_register_interface+0x73/0xb0 [mlx4_core]
[ 14.042982] [<ffffffffa02b6000>] ? 0xffffffffa02b5fff
[ 14.042982] [<ffffffffa02b6031>] __init_backport+0x31/0x1000 [mlx4_en]
[ 14.042982] [<ffffffff810002b2>] do_one_initcall+0x122/0x180
[ 14.042982] [<ffffffff8109ed2d>] sys_init_module+0xbd/0x220
[ 14.042982] [<ffffffff81460c99>] system_call_fastpath+0x16/0x1b
[ 14.042982] Code: 66 66 2e 0f 1f 84 00 00 00 00 00 48 8b 05 49 e9 80 00 55 48 89 e5 48 3d e0 67 81 81 48 8d 48 d0 74 59 66 0f 1f 84 00 00 00 00 00 <48> 3b 71 20 72 3a 48 3b 79 28 77 34 48 8b 41 10 4c 8b 41 18 4c
[ 14.042982] RIP [<ffffffff81007eb0>] alternatives_text_reserved+0x20/0x80
[ 14.042982] RSP <ffff880ea355bcb8>
next prev parent reply other threads:[~2017-10-27 11:43 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-10-27 9:34 [PATCH] kprobes, x86/alternatives: use text_mutex to protect smp_alt_modules Zhou Chengming
2017-10-27 11:15 ` Borislav Petkov
2017-10-27 11:42 ` zhouchengming [this message]
2017-10-27 12:33 ` Borislav Petkov
2017-10-27 13:30 ` zhouchengming
2017-10-28 8:43 ` Masami Hiramatsu
2017-10-28 9:51 ` zhouchengming
2017-10-27 14:15 ` Peter Zijlstra
2017-10-28 1:26 ` zhouchengming
2017-10-28 8:44 ` Masami Hiramatsu
2017-10-30 8:03 ` Masami Hiramatsu
2017-10-31 21:59 ` Steven Rostedt
2017-11-01 1:48 ` zhouchengming
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=59F31BB5.90905@huawei.com \
--to=zhouchengming1@huawei.com \
--cc=ananth@linux.vnet.ibm.com \
--cc=anil.s.keshavamurthy@intel.com \
--cc=bp@suse.de \
--cc=davem@davemloft.net \
--cc=hpa@zytor.com \
--cc=jkosina@suse.cz \
--cc=linux-kernel@vger.kernel.org \
--cc=mhiramat@kernel.org \
--cc=mjurczyk@google.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.