* [PATCH] xen/qspinlock: Don't kick CPU if IRQ is not initialized
@ 2016-04-22 12:05 Ross Lagerwall
2016-04-22 14:34 ` Boris Ostrovsky
0 siblings, 1 reply; 2+ messages in thread
From: Ross Lagerwall @ 2016-04-22 12:05 UTC (permalink / raw)
To: xen-devel
Cc: Juergen Gross, x86, Ross Lagerwall, Ingo Molnar, David Vrabel,
H. Peter Anvin, Boris Ostrovsky, Thomas Gleixner
1fb3a8b2cfb2 ("xen/spinlock: Fix locking path engaging too soon under
PVHVM.") moved the initalization of the kicker interrupt until after
native_cpu_up() is called. However, when using qspinlocks, a CPU may try
to kick another CPU that is spinning (because it has not yet initialized
its kicker interrupt), resulting in the following bug during boot:
[ 0.384000] kernel BUG at /build/linux-Ay7j_C/linux-4.4.0/drivers/xen/events/events_base.c:1210!
[ 0.384000] invalid opcode: 0000 [#1] SMP
[ 0.384000] Modules linked in:
[ 0.384000] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.4.0-21-generic #37-Ubuntu
[ 0.384000] Hardware name: Xen HVM domU, BIOS 4.6.1-xs124820 04/20/2016
[ 0.384000] task: ffff88003d758000 ti: ffff88003d760000 task.ti: ffff88003d760000
[ 0.384000] RIP: 0010:[<ffffffff814c97c9>] [<ffffffff814c97c9>] xen_send_IPI_one+0x59/0x60
[ 0.384000] RSP: 0000:ffff88003d763d30 EFLAGS: 00010086
[ 0.384000] RAX: ffff88003da522fc RBX: 00000005528d31c0 RCX: 0000000000000001
[ 0.384000] RDX: ffff88003da57840 RSI: 0000000000000003 RDI: 00000000ffffffff
[ 0.384000] RBP: ffff88003d763d30 R08: 0000000000000100 R09: ffff88003f7c7900
[ 0.384000] R10: ffff88003da4a080 R11: ffff88003da4a060 R12: 000000000000a90d
[ 0.384000] R13: 000000055285a88a R14: 0000000000000001 R15: ffffffff820d3db0
[ 0.384000] FS: 0000000000000000(0000) GS:ffff88003da00000(0000) knlGS:0000000000000000
[ 0.384000] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 0.384000] CR2: ffff880002200000 CR3: 0000000001e0a000 CR4: 00000000000006f0
[ 0.384000] Stack:
[ 0.384000] ffff88003d763d40 ffffffff8102be9e ffff88003d763de0 ffffffff810cabc2
[ 0.384000] ffffffff810ca6d1 ffff88003da4a060 ffff88003da4a080 0000000000000001
[ 0.384000] ffff88003da4a080 ffffffff820d3db0 0000000000080000 0000000500000000
[ 0.384000] Call Trace:
[ 0.384000] [<ffffffff8102be9e>] xen_qlock_kick+0xe/0x10
[ 0.384000] [<ffffffff810cabc2>] __pv_queued_spin_unlock+0xb2/0xf0
[ 0.384000] [<ffffffff810ca6d1>] ? __raw_callee_save___pv_queued_spin_unlock+0x11/0x20
[ 0.384000] [<ffffffff81052936>] ? check_tsc_warp+0x76/0x150
[ 0.384000] [<ffffffff81052aa6>] check_tsc_sync_source+0x96/0x160
[ 0.384000] [<ffffffff81051e28>] native_cpu_up+0x3d8/0x9f0
[ 0.384000] [<ffffffff8102b315>] xen_hvm_cpu_up+0x35/0x80
[ 0.384000] [<ffffffff8108198c>] _cpu_up+0x13c/0x180
[ 0.384000] [<ffffffff81081a4a>] cpu_up+0x7a/0xa0
[ 0.384000] [<ffffffff81f80dfc>] smp_init+0x7f/0x81
[ 0.384000] [<ffffffff81f5a121>] kernel_init_freeable+0xef/0x212
[ 0.384000] [<ffffffff81817f30>] ? rest_init+0x80/0x80
[ 0.384000] [<ffffffff81817f3e>] kernel_init+0xe/0xe0
[ 0.384000] [<ffffffff8182488f>] ret_from_fork+0x3f/0x70
[ 0.384000] [<ffffffff81817f30>] ? rest_init+0x80/0x80
To fix this, only send the kick if the target CPU's interrupt has been
initialized. This check isn't racy, because the target is waiting for
the spinlock, so it won't have initialized the interrupt in the
meantime.
Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
---
arch/x86/xen/spinlock.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/arch/x86/xen/spinlock.c b/arch/x86/xen/spinlock.c
index 9e2ba5c..f42e78d 100644
--- a/arch/x86/xen/spinlock.c
+++ b/arch/x86/xen/spinlock.c
@@ -27,6 +27,12 @@ static bool xen_pvspin = true;
static void xen_qlock_kick(int cpu)
{
+ int irq = per_cpu(lock_kicker_irq, cpu);
+
+ /* Don't kick if the target's kicker interrupt is not initialized. */
+ if (irq == -1)
+ return;
+
xen_send_IPI_one(cpu, XEN_SPIN_UNLOCK_VECTOR);
}
--
2.4.3
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 2+ messages in thread* Re: [PATCH] xen/qspinlock: Don't kick CPU if IRQ is not initialized
2016-04-22 12:05 [PATCH] xen/qspinlock: Don't kick CPU if IRQ is not initialized Ross Lagerwall
@ 2016-04-22 14:34 ` Boris Ostrovsky
0 siblings, 0 replies; 2+ messages in thread
From: Boris Ostrovsky @ 2016-04-22 14:34 UTC (permalink / raw)
To: Ross Lagerwall, xen-devel
Cc: Juergen Gross, x86, Ingo Molnar, David Vrabel, H. Peter Anvin,
Thomas Gleixner
On 04/22/2016 08:05 AM, Ross Lagerwall wrote:
> 1fb3a8b2cfb2 ("xen/spinlock: Fix locking path engaging too soon under
> PVHVM.") moved the initalization of the kicker interrupt until after
> native_cpu_up() is called. However, when using qspinlocks, a CPU may try
> to kick another CPU that is spinning (because it has not yet initialized
> its kicker interrupt), resulting in the following bug during boot:
>
> [ 0.384000] kernel BUG at /build/linux-Ay7j_C/linux-4.4.0/drivers/xen/events/events_base.c:1210!
> [ 0.384000] invalid opcode: 0000 [#1] SMP
> [ 0.384000] Modules linked in:
> [ 0.384000] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.4.0-21-generic #37-Ubuntu
> [ 0.384000] Hardware name: Xen HVM domU, BIOS 4.6.1-xs124820 04/20/2016
> [ 0.384000] task: ffff88003d758000 ti: ffff88003d760000 task.ti: ffff88003d760000
> [ 0.384000] RIP: 0010:[<ffffffff814c97c9>] [<ffffffff814c97c9>] xen_send_IPI_one+0x59/0x60
> [ 0.384000] RSP: 0000:ffff88003d763d30 EFLAGS: 00010086
> [ 0.384000] RAX: ffff88003da522fc RBX: 00000005528d31c0 RCX: 0000000000000001
> [ 0.384000] RDX: ffff88003da57840 RSI: 0000000000000003 RDI: 00000000ffffffff
> [ 0.384000] RBP: ffff88003d763d30 R08: 0000000000000100 R09: ffff88003f7c7900
> [ 0.384000] R10: ffff88003da4a080 R11: ffff88003da4a060 R12: 000000000000a90d
> [ 0.384000] R13: 000000055285a88a R14: 0000000000000001 R15: ffffffff820d3db0
> [ 0.384000] FS: 0000000000000000(0000) GS:ffff88003da00000(0000) knlGS:0000000000000000
> [ 0.384000] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 0.384000] CR2: ffff880002200000 CR3: 0000000001e0a000 CR4: 00000000000006f0
> [ 0.384000] Stack:
> [ 0.384000] ffff88003d763d40 ffffffff8102be9e ffff88003d763de0 ffffffff810cabc2
> [ 0.384000] ffffffff810ca6d1 ffff88003da4a060 ffff88003da4a080 0000000000000001
> [ 0.384000] ffff88003da4a080 ffffffff820d3db0 0000000000080000 0000000500000000
> [ 0.384000] Call Trace:
> [ 0.384000] [<ffffffff8102be9e>] xen_qlock_kick+0xe/0x10
> [ 0.384000] [<ffffffff810cabc2>] __pv_queued_spin_unlock+0xb2/0xf0
> [ 0.384000] [<ffffffff810ca6d1>] ? __raw_callee_save___pv_queued_spin_unlock+0x11/0x20
> [ 0.384000] [<ffffffff81052936>] ? check_tsc_warp+0x76/0x150
> [ 0.384000] [<ffffffff81052aa6>] check_tsc_sync_source+0x96/0x160
> [ 0.384000] [<ffffffff81051e28>] native_cpu_up+0x3d8/0x9f0
> [ 0.384000] [<ffffffff8102b315>] xen_hvm_cpu_up+0x35/0x80
> [ 0.384000] [<ffffffff8108198c>] _cpu_up+0x13c/0x180
> [ 0.384000] [<ffffffff81081a4a>] cpu_up+0x7a/0xa0
> [ 0.384000] [<ffffffff81f80dfc>] smp_init+0x7f/0x81
> [ 0.384000] [<ffffffff81f5a121>] kernel_init_freeable+0xef/0x212
> [ 0.384000] [<ffffffff81817f30>] ? rest_init+0x80/0x80
> [ 0.384000] [<ffffffff81817f3e>] kernel_init+0xe/0xe0
> [ 0.384000] [<ffffffff8182488f>] ret_from_fork+0x3f/0x70
> [ 0.384000] [<ffffffff81817f30>] ? rest_init+0x80/0x80
>
> To fix this, only send the kick if the target CPU's interrupt has been
> initialized. This check isn't racy, because the target is waiting for
> the spinlock, so it won't have initialized the interrupt in the
> meantime.
>
> Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
> ---
> arch/x86/xen/spinlock.c | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/arch/x86/xen/spinlock.c b/arch/x86/xen/spinlock.c
> index 9e2ba5c..f42e78d 100644
> --- a/arch/x86/xen/spinlock.c
> +++ b/arch/x86/xen/spinlock.c
> @@ -27,6 +27,12 @@ static bool xen_pvspin = true;
>
> static void xen_qlock_kick(int cpu)
> {
> + int irq = per_cpu(lock_kicker_irq, cpu);
> +
> + /* Don't kick if the target's kicker interrupt is not initialized. */
> + if (irq == -1)
> + return;
> +
> xen_send_IPI_one(cpu, XEN_SPIN_UNLOCK_VECTOR);
> }
Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2016-04-22 14:35 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-04-22 12:05 [PATCH] xen/qspinlock: Don't kick CPU if IRQ is not initialized Ross Lagerwall
2016-04-22 14:34 ` Boris Ostrovsky
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).