From mboxrd@z Thu Jan  1 00:00:00 1970
From: Gleb Natapov <gleb@redhat.com>
Subject: Re: [nVMX with: v3.9-11789-ge0fd9af] Stack trace when L2 guest is
 rebooted.
Date: Mon, 13 May 2013 10:06:07 +0300
Message-ID: <20130513070607.GL10830@redhat.com>
References: <CAOaxAcbi5dRfMii4mvaqMrmJRjJ=1LSBAKGX4rCYS7Gnqs3jkg@mail.gmail.com>
 <OF5FC7AA03.FBAAF737-ONC2257B69.00467175-C2257B69.004761A3@il.ibm.com>
 <CAOaxAcZkKePsn8VXfdG-ySvvatpEQawwV3bJjdRwU0ufpteQWw@mail.gmail.com>
 <CAOaxAcbc52Yq1XnAPWGL5o9ygDQ2mQQOmm2C=rWyMKnWwJSOAg@mail.gmail.com>
 <20130512155217.GB26185@redhat.com>
 <CAOaxAcZHrPT+bmn2bCemSL5qppiAhCRY8kuwtfXELQdn9_iMHA@mail.gmail.com>
 <519088C5.8050703@siemens.com>
 <20130513063923.GC26185@redhat.com>
 <1B4B44D9196EFF41AE41FDA404FC0A100112CF2C@SHSMSX101.ccr.corp.intel.com>
 <51908EDC.3090701@siemens.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Cc: "Ren, Yongjie" <yongjie.ren@intel.com>,
	Kashyap Chamarthy <kashyap.cv@gmail.com>,
	Abel Gordon <ABELG@il.ibm.com>,
	"Nakajima, Jun" <jun.nakajima@intel.com>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	"kvm-owner@vger.kernel.org" <kvm-owner@vger.kernel.org>
To: Jan Kiszka <jan.kiszka@siemens.com>
Return-path: <kvm-owner@vger.kernel.org>
Received: from mx1.redhat.com ([209.132.183.28]:52936 "EHLO mx1.redhat.com"
	rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
	id S1754039Ab3EMHGO (ORCPT <rfc822;kvm@vger.kernel.org>);
	Mon, 13 May 2013 03:06:14 -0400
Content-Disposition: inline
In-Reply-To: <51908EDC.3090701@siemens.com>
Sender: kvm-owner@vger.kernel.org
List-ID: <kvm.vger.kernel.org>

On Mon, May 13, 2013 at 08:57:32AM +0200, Jan Kiszka wrote:
> On 2013-05-13 08:45, Ren, Yongjie wrote:
> >> -----Original Message-----
> >> From: kvm-owner@vger.kernel.org [mailto:kvm-owner@vger.kernel.org]
> >> On Behalf Of Gleb Natapov
> >> Sent: Monday, May 13, 2013 2:39 PM
> >> To: Jan Kiszka
> >> Cc: Kashyap Chamarthy; Abel Gordon; Nakajima, Jun;
> >> kvm@vger.kernel.org; kvm-owner@vger.kernel.org
> >> Subject: Re: [nVMX with: v3.9-11789-ge0fd9af] Stack trace when L2 guest
> >> is rebooted.
> >>
> >> On Mon, May 13, 2013 at 08:31:33AM +0200, Jan Kiszka wrote:
> >>> On 2013-05-12 18:52, Kashyap Chamarthy wrote:
> >>>>>> --------------------
> >>>>>> ....
> >>>>>> [  217.938034] Uhhuh. NMI received for unknown reason 30 on CPU
> >> 0.
> >>>>>> [  217.938034] Do you have a strange power saving mode enabled?
> >>>>>> .[  222.523373] Uhhuh. NMI received for unknown reason 20 on
> >> CPU 0.
> >>>>>> [  222.524073] Do you have a strange power saving mode enabled?
> >>>>>> [  222.524073] Dazed and confused, but trying to continue
> >>>>>> [  243.860319] Uhhuh. NMI received for unknown reason 30 on CPU
> >> 0
> >>>>>> .....
> >>>>>> --------------------
> >>>>>> At the moment, L2 guest creation stuck at the above message
> >>>>>>
> >>>>> Are those in L2 dmesg or L1?
> >>>>
> >>>> L2 dmesg.
> >>>>
> >>>>
> >>>>>> $ cat /etc/grub2.cfg  | egrep -i 'hpet|nmi'
> >>>>>>
> >>>>> IIRC watchdog is enabled by default.
> >>>>
> >>>> Indeed, you're right. I disabled NMI on L1, and rebooted the newly
> >>>> created L2 guest starts just fine.
> >>>
> >>> NMI watchdogs go via some perf counters theses days IIRC. Can anyone
> >>> tell me which of those may be used in Kashyap's setup? I'm probably
> >>> lacking them for my guests and therefore do not see the errors.
> >>>
> >> Try running with -cpu host for L1. Your CPU definition probably lacks
> >> PMU leaf.
> >>
> > I met the same NMI issue in L2, too.
> > L1: -cpu host  (or -cpu Haswell,+vmx)
> > L2: -cpu qemu64 by default
> > 
> > If I use '-cpu qemu64,+vmx' to create L1, I'll not meet NMI issue in L2.
> > 
> 
> That, and it looks like my guest kernel was lacking
> CONFIG_LOCKUP_DETECTOR. Will rebuild and retest later.
> 
It looks like NMI injected by L0 to L1 are mistakenly injected into L2.
Can you test this by injecting NMI into L1 via qemu monitor?

--
			Gleb.