Re: [PATCH] xen/qspinlock: Don't kick CPU if IRQ is not initialized

[Boris Ostrovsky <boris.ostrovsky@oracle.com>]

On 04/22/2016 08:05 AM, Ross Lagerwall wrote:
> 1fb3a8b2cfb2 ("xen/spinlock: Fix locking path engaging too soon under
> PVHVM.") moved the initalization of the kicker interrupt until after
> native_cpu_up() is called. However, when using qspinlocks, a CPU may try
> to kick another CPU that is spinning (because it has not yet initialized
> its kicker interrupt), resulting in the following bug during boot:
>
> [    0.384000] kernel BUG at /build/linux-Ay7j_C/linux-4.4.0/drivers/xen/events/events_base.c:1210!
> [    0.384000] invalid opcode: 0000 [#1] SMP
> [    0.384000] Modules linked in:
> [    0.384000] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.4.0-21-generic #37-Ubuntu
> [    0.384000] Hardware name: Xen HVM domU, BIOS 4.6.1-xs124820 04/20/2016
> [    0.384000] task: ffff88003d758000 ti: ffff88003d760000 task.ti: ffff88003d760000
> [    0.384000] RIP: 0010:[<ffffffff814c97c9>]  [<ffffffff814c97c9>] xen_send_IPI_one+0x59/0x60
> [    0.384000] RSP: 0000:ffff88003d763d30  EFLAGS: 00010086
> [    0.384000] RAX: ffff88003da522fc RBX: 00000005528d31c0 RCX: 0000000000000001
> [    0.384000] RDX: ffff88003da57840 RSI: 0000000000000003 RDI: 00000000ffffffff
> [    0.384000] RBP: ffff88003d763d30 R08: 0000000000000100 R09: ffff88003f7c7900
> [    0.384000] R10: ffff88003da4a080 R11: ffff88003da4a060 R12: 000000000000a90d
> [    0.384000] R13: 000000055285a88a R14: 0000000000000001 R15: ffffffff820d3db0
> [    0.384000] FS:  0000000000000000(0000) GS:ffff88003da00000(0000) knlGS:0000000000000000
> [    0.384000] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [    0.384000] CR2: ffff880002200000 CR3: 0000000001e0a000 CR4: 00000000000006f0
> [    0.384000] Stack:
> [    0.384000]  ffff88003d763d40 ffffffff8102be9e ffff88003d763de0 ffffffff810cabc2
> [    0.384000]  ffffffff810ca6d1 ffff88003da4a060 ffff88003da4a080 0000000000000001
> [    0.384000]  ffff88003da4a080 ffffffff820d3db0 0000000000080000 0000000500000000
> [    0.384000] Call Trace:
> [    0.384000]  [<ffffffff8102be9e>] xen_qlock_kick+0xe/0x10
> [    0.384000]  [<ffffffff810cabc2>] __pv_queued_spin_unlock+0xb2/0xf0
> [    0.384000]  [<ffffffff810ca6d1>] ? __raw_callee_save___pv_queued_spin_unlock+0x11/0x20
> [    0.384000]  [<ffffffff81052936>] ? check_tsc_warp+0x76/0x150
> [    0.384000]  [<ffffffff81052aa6>] check_tsc_sync_source+0x96/0x160
> [    0.384000]  [<ffffffff81051e28>] native_cpu_up+0x3d8/0x9f0
> [    0.384000]  [<ffffffff8102b315>] xen_hvm_cpu_up+0x35/0x80
> [    0.384000]  [<ffffffff8108198c>] _cpu_up+0x13c/0x180
> [    0.384000]  [<ffffffff81081a4a>] cpu_up+0x7a/0xa0
> [    0.384000]  [<ffffffff81f80dfc>] smp_init+0x7f/0x81
> [    0.384000]  [<ffffffff81f5a121>] kernel_init_freeable+0xef/0x212
> [    0.384000]  [<ffffffff81817f30>] ? rest_init+0x80/0x80
> [    0.384000]  [<ffffffff81817f3e>] kernel_init+0xe/0xe0
> [    0.384000]  [<ffffffff8182488f>] ret_from_fork+0x3f/0x70
> [    0.384000]  [<ffffffff81817f30>] ? rest_init+0x80/0x80
>
> To fix this, only send the kick if the target CPU's interrupt has been
> initialized. This check isn't racy, because the target is waiting for
> the spinlock, so it won't have initialized the interrupt in the
> meantime.
>
> Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
> ---
>   arch/x86/xen/spinlock.c | 6 ++++++
>   1 file changed, 6 insertions(+)
>
> diff --git a/arch/x86/xen/spinlock.c b/arch/x86/xen/spinlock.c
> index 9e2ba5c..f42e78d 100644
> --- a/arch/x86/xen/spinlock.c
> +++ b/arch/x86/xen/spinlock.c
> @@ -27,6 +27,12 @@ static bool xen_pvspin = true;
>   
>   static void xen_qlock_kick(int cpu)
>   {
> +	int irq = per_cpu(lock_kicker_irq, cpu);
> +
> +	/* Don't kick if the target's kicker interrupt is not initialized. */
> +	if (irq == -1)
> +		return;
> +
>   	xen_send_IPI_one(cpu, XEN_SPIN_UNLOCK_VECTOR);
>   }


Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel