From mboxrd@z Thu Jan  1 00:00:00 1970
From: "Yang, Xiaowei" <xiaowei.yang@intel.com>
Subject: One issue of pvops dom0's spinlock code
Date: Mon, 07 Sep 2009 15:49:18 +0800
Message-ID: <4AA4BAFE.50906@intel.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Return-path: <xen-devel-bounces@lists.xensource.com>
List-Unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xensource.com>
List-Help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-Subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
Sender: xen-devel-bounces@lists.xensource.com
Errors-To: xen-devel-bounces@lists.xensource.com
To: Jeremy Fitzhardinge <jeremy@goop.org>, "xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>
List-Id: xen-devel@lists.xenproject.org

As we tested pvops dom0, sometimes we met some vCPUs hung due to dead lock. 
After checking the dom0's stack (see below) and the code, we found it's caused 
by this commit: 1e696f638 (xen: allow interrupts to be enabled while doing a 
blocking spin). If we don't enable irq inside spinlock slow path the issue is gone.

Jeremy,
Can you have a check of this?:)

Thanks,
Xiaowei

[ 2631.686041] Call Trace:
[ 2631.689073]  <IRQ>  [<ffffffff811a8b5f>] ? xen_poll_irq+0x49/0x53
[ 2631.695338]  [<ffffffff8100ff74>] xen_spin_lock_slow+0x13f/0x204
[ 2631.703151]  [<ffffffff810100ef>] xen_spin_lock_flags+0xb6/0xe6
[ 2631.709405]  [<ffffffff81050d68>] ? delayed_work_timer_fn+0x0/0x33
[ 2631.715663]  [<ffffffff813283ba>] _spin_lock_irqsave+0x30/0x39
[ 2631.723479]  [<ffffffff81050d42>] __queue_work+0x18/0x3e
[ 2631.728171]  [<ffffffff81050d97>] delayed_work_timer_fn+0x2f/0x33
[ 2631.735988]  [<ffffffff81049f41>] run_timer_softirq+0x160/0x1f1
[ 2631.743804]  [<ffffffff811a7488>] ? unmask_evtchn+0x34/0xd6
[ 2631.748493]  [<ffffffff81045f6f>] __do_softirq+0xa2/0x13d
[ 2631.754756]  [<ffffffff81013e0c>] call_softirq+0x1c/0x30
[ 2631.761007]  [<ffffffff81015950>] do_softirq+0x42/0x88
[ 2631.767262]  [<ffffffff81045ecb>] irq_exit+0x3f/0x41
[ 2631.771955]  [<ffffffff811a82af>] xen_evtchn_do_upcall+0x13e/0x15a
[ 2631.779774]  [<ffffffff81013e5e>] xen_do_hypervisor_callback+0x1e/0x30
[ 2631.787584]  <EOI>  [<ffffffff8100ff5d>] ? xen_spin_lock_slow+0x128/0x204
[ 2631.795407]  [<ffffffff810100ef>] ? xen_spin_lock_flags+0xb6/0xe6
[ 2631.801696]  [<ffffffff810862a4>] ? lru_add_drain_per_cpu+0x0/0xb
[ 2631.809483]  [<ffffffff813283ba>] ? _spin_lock_irqsave+0x30/0x39
[ 2631.815743]  [<ffffffff81328520>] ? _spin_unlock_irqrestore+0x27/0x2a
[ 2631.823557]  [<ffffffff81053fb3>] ? finish_wait+0x3b/0x67
[ 2631.828251]  [<ffffffff81050f7b>] ? worker_thread+0xb6/0x1f9
[ 2631.836067]  [<ffffffff81053e8e>] ? autoremove_wake_function+0x0/0x38
[ 2631.842325]  [<ffffffff81050ec5>] ? worker_thread+0x0/0x1f9
[ 2631.848576]  [<ffffffff81053aeb>] ? kthread+0x8f/0x97
[ 2631.854829]  [<ffffffff81013d0a>] ? child_rip+0xa/0x20
[ 2631.861083]  [<ffffffff81012ee7>] ? int_ret_from_sys_call+0x7/0x1b
[ 2631.867343]  [<ffffffff8101369d>] ? retint_restore_args+0x5/0x6
[ 2631.875159]  [<ffffffff81013d00>] ? child_rip+0x0/0x20