Re: Xen 4 TSC problems

[Olivier Hanesse <olivier.hanesse@gmail.com>]

[-- Attachment #1.1: Type: text/plain, Size: 6811 bytes --]

Yes this is what I mean.
I am glad to hear that it isn't a bad sign :)
I thought of a bad sign, because on system with "reliable TSC", this counter
is always 0.

2011/2/28 Dan Magenheimer <dan.magenheimer@oracle.com>

> Hi Olivier –
>
>
>
> By “warp=3000+ in  debug message” do you mean the Xen boot message “TSC has
> constant rate..., warp = NNNN”?
>
>
>
> If so, this is a very different “warp” measured in cycles, not in seconds,
> so 3000 is more like a microsecond not an hour, ! and this is normal (not a
> bad sign).
>
>
>
> Dan
>
>
>
> *From:* Olivier Hanesse [mailto:olivier.hanesse@gmail.com]
> *Sent:* Monday, February 28, 2011 8:23 AM
> *To:* Dan Magenheimer
> *Cc:* Jeremy Fitzhardinge; Keir Fraser; Jan Beulich; Mark Adams;
> xen-devel@lists.xensource.com; Xen Users; Keir Fraser
>
> *Subject:* Re: [Xen-devel] Xen 4 TSC problems
>
>
>
> Keir :
>
>
>
> Yes, it is "under progress".
>
> To make this change, I had to reboot every server, so it is taking time
> (production server :()
>
> So i was hoping to find a quick method to mitigate this issue on domUs
> while rebooting servers.
>
>
>
> As this bug happens once or twice per server since October, I can't say
> that right now that changing platform timer to PIT fixed it. I have to wait
> (I hope forever!) this bug to happen again on a 'patched' server ...
>
>
>
> But even with clcoksource=pit, I am seeing some warp=3000+ in debug message
> ? I guess it is not a good sign, is it ?
>
>
>
> Jan : I was hoping to find a way to make the domU clocksource more
> "independent" like with xen3.2.
>
>
>
>
>
> 2011/2/28 Dan Magenheimer <dan.magenheimer@oracle.com>
>
> Hi Olivier –
>
>
>
> It is the Xen clocksource that you want to try to change, not the dom0
> clocksource.  To do this, you need to specify “clocksource=pit” on the Xen
> boot line (and reboot), not the dom0 boot line.
>
>
>
> I believe Mark Adams played with tsc_mode to see if it solved! his
> (similar? identical?) problem last year, and it didn’t make any difference.
>
>
> Please try booting Xen with “clocksource=pit” and ensu! re that “Platform
> timer is 1.19MHz PIT” appears in the Xen boot messages.  If the 50min jump
> does not appear again, it would point to a problem in the hpet, either
> hardware or software.
>
>
>
> Thanks,
>
> Dan
>
>
>
> *From:* Olivier Hanesse [mailto:olivier.hanesse@gmail.com]
> *Sent:* Monday, February 28, 2011 7:37 AM
> *To:* Jeremy Fitzhardinge
> *Cc:* Dan Magenheimer; Keir Fraser; Jan Beulich; Mark Adams;
> xen-devel@lists.xensource.com; Xen Users; Keir Fraser
>
>
> *Subject:* Re: [Xen-devel] Xen 4 TSC problems
>
>
>
> Hello,
>
>
>
> It happened again twice this weekend.
>
>
>
> What about setting "tsc_mode=2" for my vms ? Should this mode prevent this
> bug (coming from a bad emulated tsc due to firmware issue ? is it possible
> ?) from affecting time in domUs ?
>
>
>
> Setting clocksource=pit, make 'tsc' available in
> "/sys/devices/system/clocksource/clocksource0/available_clocksource"
> (otherwise only xen is available, is it norma! l ? ).
>
>
>
> Should I bypass xen clocksource and use tsc as a clocksource for dom0/domU
> ? or  will it be worsed ?
>
>
>
> Regards
>
>
>
> Olivier
>
>
>
> 2011/2/24 Jeremy Fitzhardinge <jeremy@goop.org>
>
> On 02/24/2011 09:43 AM, Dan Magenheimer wrote:
> > Just a wild guess, but this in Olivier's posted output:
> >
> > (XEN) Platform timer appears to have unexpectedly wrapped 10 or more
> times.
> >
> > and the fact that a 32-bit HPET wrap is ~300 seconds and, with the
> > "10 or more times", 10 * 300 seconds is 3000 seconds, might be a clue
> > (or a complete red herring, but I thought it worth mentioning).
> >
> > Mark and Olivier, it would be interesting to know if you are
> > using the same processor/system.
>
> It definitely seems like some kind of problem on the host system rather
> than anything in the guests themselves. !  If the platform timer is
> misbehaving, then Xen could be completely screwing up the pvclock
> calibration which it then passes to guests.
>
> Could it be one of those "platform clock stops in certain power states"
> problems?
>
>
>    J
>
> >> -----Original Message-----
> >> From: Keir Fraser [mailto:keir.xen@gmail.com]
> >> Sent: Thursday, February 24, 2011 7:52 AM
> >> To: Olivier Hanesse; Jan Beulich
> >> Cc: Mark Adams; Jeremy Fitzhardinge; xen-devel@lists.xensource.com; Xen
> >> Users; Dan Magenheimer; Keir Fraser
> >> Subject: Re: [Xen-devel] Xen 4 TSC problems
> >>
> >> On 24/02/2011 14:20, "Olivier Hanesse" <olivier.hanesse@gmail.com>
> >> wrote:
> >>
> >>> Both dom0 and domUs are affected by this" jump".
> >>>
> >>> I expect to see something like "TSC marked as reliable, warp = 0".
> >>> I got this on newer hardware with same config/distros.
> >> It depends on the CPU itself, older CPUs do not have the super-stable
> >> TSC
> >> features. But that should never cause a massive 3000s time jump.
> >>
> >>> Is there a way to measure if it is a TSC warp ? to point out a cpu
> >> tsc issue ?
> >>
> >> The TSC warps or out-of-sync issues that we could reasonably expect
> >> would be
> >> on the order of microseconds. A 3000s warp is something else entirely.
> >> Xen
> >> is very confused and/or some TSC or platform timer has jumped a long
> >> way
> >> (indicating a hardware/firmware issue).
> >>
> >>  -- Keir
> >>
>
> >&gt! ;> 2011/2/24 Jan Beulich <JBeulich@novell.com>
>
>
> >>>>>>> On 24.02.11 at 12:57, Olivier Hanesse <olivier.hanesse@gmail.com>
> >> wrote:
> >>>>> I tried to turn off cstates with max_cstate=0 without success
> >> (still "not
> >>>>> reliable").
> >>>>>
> >>>>> With cpuidle=0, I also got :
> >>>>>
> >>>>> (XEN) TSC has constant rate, deep Cstates possible, so not
> >> reliable,
> >>>>> warp=3022 (count=1)
> >>>> This message by itself isn't telling much I believe.
> >>>>
> >>>>> xm info | grep command
> >>>>> xen_commandline        : dom0_mem=512M cpuidle=0 loglvl=all
> >> guest_loglvl=all
>
> >>>>> dom0_max_vcpus=1 dom0_vcp! us_pin console=vga,com1 com1=19200,8n1
>
>
> >>>>>
> >>>>> Keir :
> >>>>>
> >>>>> Using clocksource=pit :
> >>>>>
> >>>>> (XEN) Platform timer is 1.193MHz PIT
> >>>>>
> >>>>> I also got :
> >>>>>
> >>>>> (XEN) TSC has constant rate, deep Cstates possible, so not
> >> reliable,
> >>>>> warp=3262 (count=2)
> >>>> The question is whether any of this eliminates the time jumps seen
> >>>> by your DomU-s (from your past mails I wasn't actually sure whether
> >>>> Dom0 also experienced this problem, albeit it would be odd if it
> >> didn't).
> >>>> Jan
> >>>>
> >>>> Jan
> >>>>
> >>>
> >>
>
>
>
>
>

[-- Attachment #1.2: Type: text/html, Size: 12227 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel