(XEN) Platform timer appears to have unexpectedly wrapped 10 or more times.

(XEN) Platform timer appears to have unexpectedly wrapped 10 or more times.

[Josip Rodin]

Hi,

Sometimes, seemingly randomly, long-running Xen domains, using clocksource
xen, have their clock shift by ~3000 or ~36000 seconds, often with the dom0
complaining about clocksource tsc. The number changes if the hypervisor is
explicitly told to use clocksource=pit, but it still happens. It doesn't
seem to be particularly hardware-specific.

See also http://bugs.debian.org/599161

I see from the archives that others have reported this problem here in
February last year, but I couldn't find a resolution. Can anyone help?

(Please Cc: responses, I'm not subscribed.)

-- 
     2. That which causes joy or happiness.

Re: (XEN) Platform timer appears to have unexpectedly wrapped 10 or more times.

[Philippe.Simonet]

Hi

I have the same problem with 4 machines (hp dl385, AMD), the problem appears one time per machine and per month ...
No chance to re-produce it faster. On some other machine with another AMD processor, I never had this problem. Olivier Hanesse
reported the problem on some Intel CPU and not on other processor.

I'm using debian squeeze, with 

xen-hypervisor-4.0-amd64                4.0.1-4
linux-image-2.6.32-5-xen-amd64          2.6.32-38 (dom0)

that last changes that I have done is to add cpuidle=0 to hypervisor command line. stable since 3-4 weeks (...)

I have e test system that runs 20 paravirt. domus, with wheezy, with same problematic HW, with
xen-hypervisor-4.1-amd64                4.1.2-2
linux-image-3.1.0-1-amd64               3.1.6-1 (dom0)
and this system never had the TSC jump problem. but test system are test system ....

I would be happy if you could reproduce the problem faster a me, so testing new xen / dom0 version could be
easier ...

thanks and regards

Philippe




> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-
> bounces@lists.xensource.com] On Behalf Of Josip Rodin
> Sent: Wednesday, January 04, 2012 6:24 PM
> To: xen-devel@lists.xensource.com
> Subject: [Xen-devel] (XEN) Platform timer appears to have unexpectedly
> wrapped 10 or more times.
> 
> Hi,
> 
> Sometimes, seemingly randomly, long-running Xen domains, using
> clocksource xen, have their clock shift by ~3000 or ~36000 seconds, often
> with the dom0 complaining about clocksource tsc. The number changes if the
> hypervisor is explicitly told to use clocksource=pit, but it still happens. It
> doesn't seem to be particularly hardware-specific.
> 
> See also http://bugs.debian.org/599161
> 
> I see from the archives that others have reported this problem here in
> February last year, but I couldn't find a resolution. Can anyone help?
> 
> (Please Cc: responses, I'm not subscribed.)
> 
> --
>      2. That which causes joy or happiness.
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: (XEN) Platform timer appears to have unexpectedly wrapped 10 or more times.

[Andreas Kinzler]

I have the same problem. Do you use ntpd? Which version? I suspect some 
link between the problem and ntpd.

Andreas

On 05.01.2012 10:39, Philippe.Simonet@swisscom.com wrote:
> Hi
>
> I have the same problem with 4 machines (hp dl385, AMD), the problem appears one time per machine and per month ...
> No chance to re-produce it faster. On some other machine with another AMD processor, I never had this problem. Olivier Hanesse
> reported the problem on some Intel CPU and not on other processor.
>
> I'm using debian squeeze, with
>
> xen-hypervisor-4.0-amd64                4.0.1-4
> linux-image-2.6.32-5-xen-amd64          2.6.32-38 (dom0)
>
> that last changes that I have done is to add cpuidle=0 to hypervisor command line. stable since 3-4 weeks (...)
>
> I have e test system that runs 20 paravirt. domus, with wheezy, with same problematic HW, with
> xen-hypervisor-4.1-amd64                4.1.2-2
> linux-image-3.1.0-1-amd64               3.1.6-1 (dom0)
> and this system never had the TSC jump problem. but test system are test system ....
>
> I would be happy if you could reproduce the problem faster a me, so testing new xen / dom0 version could be
> easier ...
>
> thanks and regards
>
> Philippe
>
>
>
>
>> -----Original Message-----
>> From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-
>> bounces@lists.xensource.com] On Behalf Of Josip Rodin
>> Sent: Wednesday, January 04, 2012 6:24 PM
>> To: xen-devel@lists.xensource.com
>> Subject: [Xen-devel] (XEN) Platform timer appears to have unexpectedly
>> wrapped 10 or more times.
>>
>> Hi,
>>
>> Sometimes, seemingly randomly, long-running Xen domains, using
>> clocksource xen, have their clock shift by ~3000 or ~36000 seconds, often
>> with the dom0 complaining about clocksource tsc. The number changes if the
>> hypervisor is explicitly told to use clocksource=pit, but it still happens. It
>> doesn't seem to be particularly hardware-specific.
>>
>> See also http://bugs.debian.org/599161
>>
>> I see from the archives that others have reported this problem here in
>> February last year, but I couldn't find a resolution. Can anyone help?
>>
>> (Please Cc: responses, I'm not subscribed.)
>>
>> --
>>       2. That which causes joy or happiness.
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: (XEN) Platform timer appears to have unexpectedly wrapped 10 or more times.

[Josip Rodin]

On Tue, Apr 10, 2012 at 02:33:18PM +0200, Andreas Kinzler wrote:
> I have the same problem. Do you use ntpd? Which version? I suspect
> some link between the problem and ntpd.

But ntpd is still a userland program, shouldn't the kernel exert the
ultimate control over who gets to screw with the system's clock source?

In any case, I did a quick grep of ntpd sources and found references to
RDTSC only in the Windows port, so the link doesn't seem obvious.

-- 
     2. That which causes joy or happiness.

Re: (XEN) Platform timer appears to have unexpectedly wrapped 10 or more times.

[Andreas Kinzler]

> On Tue, Apr 10, 2012 at 02:33:18PM +0200, Andreas Kinzler wrote:
>> I have the same problem. Do you use ntpd? Which version? I suspect
>> some link between the problem and ntpd.
> But ntpd is still a userland program, shouldn't the kernel exert the
> ultimate control over who gets to screw with the system's clock source?
> In any case, I did a quick grep of ntpd sources and found references to
> RDTSC only in the Windows port, so the link doesn't seem obvious.

Right, there is no obvious link (but which hard to reproduce bug is 
obvious?) but see 
"http://lists.xen.org/archives/html/xen-devel/2011-10/msg01269.html". 
There seems to be some connection.

Regards Andreas

Re: (XEN) Platform timer appears to have unexpectedly wrapped 10 or more times.

[Josip Rodin]

On Tue, Apr 10, 2012 at 03:59:42PM +0200, Andreas Kinzler wrote:
> >On Tue, Apr 10, 2012 at 02:33:18PM +0200, Andreas Kinzler wrote:
> >>I have the same problem. Do you use ntpd? Which version? I suspect
> >>some link between the problem and ntpd.
> >But ntpd is still a userland program, shouldn't the kernel exert the
> >ultimate control over who gets to screw with the system's clock source?
> >In any case, I did a quick grep of ntpd sources and found references to
> >RDTSC only in the Windows port, so the link doesn't seem obvious.
> 
> Right, there is no obvious link (but which hard to reproduce bug is
> obvious?) but see
> "http://lists.xen.org/archives/html/xen-devel/2011-10/msg01269.html".
> There seems to be some connection.

>From that (garbled) log, I don't see it - you didn't even post the matching
kernel log message with timestamp, so it's impossible to see if and when
it coincided when the ntpd messages.

Over here we're running ntpd, obviously, in dom0 and in domU, but the only
observed interaction of this problem with ntpd is that they become useless
with this kind of a major drift.

-- 
     2. That which causes joy or happiness.