From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: issues with PLE and/or scheduler. Date: Tue, 20 Dec 2011 16:41:51 -0400 Message-ID: <20111220204151.GB768@andromeda.dapyr.net> References: <20111220204107.GA768@andromeda.dapyr.net> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Content-Disposition: inline In-Reply-To: <20111220204107.GA768@andromeda.dapyr.net> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: xen-devel@lists.xensource.com, konrad.wilk@oracle.com, George.Dunlap@eu.citrix.com, keir@xen.org, andrew.thomas@oracle.com List-Id: xen-devel@lists.xenproject.org On Tue, Dec 20, 2011 at 04:41:07PM -0400, Konrad Rzeszutek Wilk wrote: > Hey folks, > > I am sending this on behalf of Andrew since our internal email system > is dropping all xen-devel mailing lists :-( And I forgot to CC andrew on it. Added here. > > Anyhow: > > This is with xen-4.1-testing cs 23201:1c89f7d29fbb > and using the default "credit" scheduler. > > I've run into an interesting issue with HVM guests which > make use of Pause Loop Exiting (ie. on westmere systems; > and also on romley systems): after yielding the cpu, guests > don't seem to receive timer interrupts correctly.. > > Some background: for historical reasons (ie old templates) we boot > OL/RHEL guests with the following settings: > > kernel parameters: clock=pit nohpet nopmtimer > vm.cfg: timer_mode = 2 > > With PLE enabled, 2.6.32 guests will crash early on with: > ..MP-BIOS bug: 8254 timer not connected to IO-APIC > # a few lines omitted.. > Kernel panic - not syncing: IO-APIC + timer doesn't work! Boot with > apic=debug > > While 2.6.18-238 (ie OL/RHEL5u6) will fail to find the timer, but > continue and lock up in the serial line initialization. > > ..MP-BIOS bug: 8254 timer not connected to IO-APIC > # continues until lock up here: > Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing enabled > > Instrumenting the 2.6.32 code (ie timer_irq_works()) shows that jiffies > isn't advancing (or only 1 or 2 ticks are being received, which is insufficient > for "working"). This is on a "quiet" system with no other activity. > So, even though the guest has voluntarily yielded the cpu (through PLE), > I would still expect it to receive every clock tick (even with timer_mode=2) > as there is no other work to do on the system. > > Disabling PLE allows both 2.6.18 and 2.6.32 guests to boot.. [As an > aside, so does setting ple_gap to 41 (ie prior to 21355:727ccaaa6cce) -- > the perf counters show no exits happening, so this is equivalent to disabling PLE.] > > I'm hoping someone who knows the scheduler well will be able to quickly > decide whether this is a bug or a feature... > > Andrew