[XEN-3.4] any changes to time/clock handling?

[XEN-3.4] any changes to time/clock handling?

[Daniel Schroeder]

hello *,

with 3.4 i have massive clock differences between dom0 and domu...
e.g. every domu is about two hours in the future...
...with < 3.4 everything was synced...

tia
daniel

Re: [XEN-3.4] any changes to time/clock handling?

[Keir Fraser]

On 22/05/2009 09:47, "Daniel Schroeder" <sec@dschroeder.info> wrote:

> hello *,
> 
> with 3.4 i have massive clock differences between dom0 and domu...
> e.g. every domu is about two hours in the future...
> ...with < 3.4 everything was synced...

Are the domUs HVM or PV? Could it be we are getting confused between GMT and
localtime?

 -- Keir

Re: [XEN-3.4] any changes to time/clock handling?

[Daniel Schroeder]

Keir Fraser wrote:
> On 22/05/2009 09:47, "Daniel Schroeder" <sec@dschroeder.info> wrote:
> 
>> hello *,
>>
>> with 3.4 i have massive clock differences between dom0 and domu...
>> e.g. every domu is about two hours in the future...
>> ...with < 3.4 everything was synced...
> 
> Are the domUs HVM or PV? Could it be we are getting confused between GMT and
> localtime?
> 
>  -- Keir
> 
> 
yepp, i think so...they are pv domUs and their configs were without a
localtime entry...i thought the default is 0...with releases before 3.4
the domUs behaved like localtime = 0.

Interesting is the fact, that only a couple domUs are two hours in the
future and the others are correct. With nearly identical configs...

I have added "localtime = 0" and it seems to be fine...so, probably solved..

cheers,
daniel

[XEN-3.4] pv_ops dom0 time/clock handling

[Daniel Schroeder]

hello *,

> 
> I have added "localtime = 0" and it seems to be fine...so, probably solved..
> 
this works for OpenSUSE 2.6.29.X XEN dom0 Kernel...
...i have tested latest git-next repo .30-rc6 and the pv domU is two
hours ahead.
I have checked the following:

dom0 ntpd working, correct time and /etc/localtime is correct
(Europe/Berlin)

domU no ntpd and correct timezone and /etc/localtime (Europe/Berlin)
"localtime = 0" in pv domU config file...

it seems to me, that the domU gets not UTC as basetime...

any help appreciated...
daniel

Re: [XEN-3.4] pv_ops dom0 time/clock handling

[Keir Fraser]

On 25/05/2009 10:11, "Daniel Schroeder" <sec@dschroeder.info> wrote:

>> I have added "localtime = 0" and it seems to be fine...so, probably solved..
>> 
> this works for OpenSUSE 2.6.29.X XEN dom0 Kernel...
> ...i have tested latest git-next repo .30-rc6 and the pv domU is two
> hours ahead.
> I have checked the following:
> 
> dom0 ntpd working, correct time and /etc/localtime is correct
> (Europe/Berlin)
> 
> domU no ntpd and correct timezone and /etc/localtime (Europe/Berlin)
> "localtime = 0" in pv domU config file...
> 
> it seems to me, that the domU gets not UTC as basetime...

This could plausibly be a pv_ops bug. I fthe only thing you change is the
kernel -- keeping all configuration the same -- then dom0 will be providing
exactly the same time info to both kernels.

 -- Keir

Re: [XEN-3.4] pv_ops dom0 time/clock handling

[Daniel Schroeder]

Keir Fraser wrote:
> On 25/05/2009 10:11, "Daniel Schroeder" <sec@dschroeder.info> wrote:
> 
>>> I have added "localtime = 0" and it seems to be fine...so, probably solved..
>>>
>> this works for OpenSUSE 2.6.29.X XEN dom0 Kernel...
>> ...i have tested latest git-next repo .30-rc6 and the pv domU is two
>> hours ahead.
>> I have checked the following:
>>
>> dom0 ntpd working, correct time and /etc/localtime is correct
>> (Europe/Berlin)
>>
>> domU no ntpd and correct timezone and /etc/localtime (Europe/Berlin)
>> "localtime = 0" in pv domU config file...
>>
>> it seems to me, that the domU gets not UTC as basetime...
> 
> This could plausibly be a pv_ops bug. I fthe only thing you change is the
> kernel -- keeping all configuration the same -- then dom0 will be providing
> exactly the same time info to both kernels.
> 
checked with OpenSUSE 2.6.29.X xenified dom0...pv domU has the right
time...booted pvops .30-rc6, pv domU gets probably localtime as
basetime, so domU is two hours in the future...

daniel

Re: [XEN-3.4] pv_ops dom0 time/clock handling

[Jeremy Fitzhardinge]

Daniel Schroeder wrote:
>>> this works for OpenSUSE 2.6.29.X XEN dom0 Kernel...
>>> ...i have tested latest git-next repo .30-rc6 and the pv domU is two
>>> hours ahead.
>>> I have checked the following:
>>>
>>> dom0 ntpd working, correct time and /etc/localtime is correct
>>> (Europe/Berlin)
>>>
>>> domU no ntpd and correct timezone and /etc/localtime (Europe/Berlin)
>>> "localtime = 0" in pv domU config file...
>>>
>>> it seems to me, that the domU gets not UTC as basetime...
>>>       
>> This could plausibly be a pv_ops bug. I fthe only thing you change is the
>> kernel -- keeping all configuration the same -- then dom0 will be providing
>> exactly the same time info to both kernels.
>>
>>     
> checked with OpenSUSE 2.6.29.X xenified dom0...pv domU has the right
> time...booted pvops .30-rc6, pv domU gets probably localtime as
> basetime, so domU is two hours in the future...
>   

Time is pretty straightforward, and there's no TZ calculation involved 
at any points.  Its hard to see how you'd get a two hour delta (is it 
exactly 2 hours?).

One difference between pvops time handling and the -xen kernels, is that 
they defaulted to slaving the domU time off the hypervisor at all times, 
so a system time change would propagate into guests.  I don't implement 
that in pvops kernels, so they'll maintain independent time unless you 
explicitly sync with some mechanism like ntp.

    J

Re: [XEN-3.4] pv_ops dom0 time/clock handling

[Daniel Schroeder]

Jeremy Fitzhardinge wrote:
> Daniel Schroeder wrote:
>>>> this works for OpenSUSE 2.6.29.X XEN dom0 Kernel...
>>>> ...i have tested latest git-next repo .30-rc6 and the pv domU is two
>>>> hours ahead.
>>>> I have checked the following:
>>>>
>>>> dom0 ntpd working, correct time and /etc/localtime is correct
>>>> (Europe/Berlin)
>>>>
>>>> domU no ntpd and correct timezone and /etc/localtime (Europe/Berlin)
>>>> "localtime = 0" in pv domU config file...
>>>>
>>>> it seems to me, that the domU gets not UTC as basetime...
>>>>       
>>> This could plausibly be a pv_ops bug. I fthe only thing you change is
>>> the
>>> kernel -- keeping all configuration the same -- then dom0 will be
>>> providing
>>> exactly the same time info to both kernels.
>>>
>>>     
>> checked with OpenSUSE 2.6.29.X xenified dom0...pv domU has the right
>> time...booted pvops .30-rc6, pv domU gets probably localtime as
>> basetime, so domU is two hours in the future...
>>   
> 
> Time is pretty straightforward, and there's no TZ calculation involved
> at any points.  Its hard to see how you'd get a two hour delta (is it
> exactly 2 hours?).

Wed May 27 09:33:12 CEST 2009
Mi Mai 27 11:33:13 CEST 2009

btw: i have checked with 32 bit pvops and 64 bit pvops dom0 and the time
 in the domU is correct with the 64bit pvops dom0 kernel...this wasnt
the same domU...so, next step for me, is to copy the domU to the 64bit
system and try again...to verify, that this only happens for me with
32bit pvops dom0...
> 
> One difference between pvops time handling and the -xen kernels, is that
> they defaulted to slaving the domU time off the hypervisor at all times,
> so a system time change would propagate into guests.  I don't implement
> that in pvops kernels, so they'll maintain independent time unless you
> explicitly sync with some mechanism like ntp.
hmm...does this mean, that i have to use ntp in domU if i use the pvops
kernel? Because time changes in dom0 doesnt propagate into domUs?

Daniel

Re: [XEN-3.4] pv_ops dom0 time/clock handling

[Jeremy Fitzhardinge]

Daniel Schroeder wrote:
> Wed May 27 09:33:12 CEST 2009
> Mi Mai 27 11:33:13 CEST 2009
>   

That does look like some kind of TZ issue, but I'm not sure where it 
would be.

> btw: i have checked with 32 bit pvops and 64 bit pvops dom0 and the time
>  in the domU is correct with the 64bit pvops dom0 kernel...this wasnt
> the same domU...so, next step for me, is to copy the domU to the 64bit
> system and try again...to verify, that this only happens for me with
> 32bit pvops dom0...
>   

How odd.

>> One difference between pvops time handling and the -xen kernels, is that
>> they defaulted to slaving the domU time off the hypervisor at all times,
>> so a system time change would propagate into guests.  I don't implement
>> that in pvops kernels, so they'll maintain independent time unless you
>> explicitly sync with some mechanism like ntp.
>>     
> hmm...does this mean, that i have to use ntp in domU if i use the pvops
> kernel? Because time changes in dom0 doesnt propagate into domUs?
>   

It will set its initial time-of-day clock from the hypervisor's at boot, 
but then proceed independently (but the timebase is derived from the 
hypervisor's clock).  If you want it to track a reference, you need to 
run ntp.

    J

Re: [XEN-3.4] pv_ops dom0 time/clock handling --solved

[Daniel Schroeder]

Jeremy Fitzhardinge wrote:
> Daniel Schroeder wrote:
>> Wed May 27 09:33:12 CEST 2009
>> Mi Mai 27 11:33:13 CEST 2009
>>   
> 
> That does look like some kind of TZ issue, but I'm not sure where it
> would be.
> 
>> btw: i have checked with 32 bit pvops and 64 bit pvops dom0 and the time
>>  in the domU is correct with the 64bit pvops dom0 kernel...this wasnt
>> the same domU...so, next step for me, is to copy the domU to the 64bit
>> system and try again...to verify, that this only happens for me with
>> 32bit pvops dom0...
>>   
> 
> How odd.
> 
>>> One difference between pvops time handling and the -xen kernels, is that
>>> they defaulted to slaving the domU time off the hypervisor at all times,
>>> so a system time change would propagate into guests.  I don't implement
>>> that in pvops kernels, so they'll maintain independent time unless you
>>> explicitly sync with some mechanism like ntp.
>>>     
>> hmm...does this mean, that i have to use ntp in domU if i use the pvops
>> kernel? Because time changes in dom0 doesnt propagate into domUs?
>>   
> 
> It will set its initial time-of-day clock from the hypervisor's at boot,

hmm...hwclock --show on affected system:

hwclock --show
Wed May 27 23:06:20 2009  -0.778274 seconds

hwclock was set to localtime...

on not affected system

hwclock --show
Cannot access the Hardware Clock via any known method.
Use the --debug option to see the details of our search for an access
method.

i have changed the settings in my distro from hwclock localtime to UTC,
reset the hwclock
hwclock --utc --systohc
rebooted
everything is fine now...

:)

thx!
daniel

Re: [XEN-3.4] pv_ops dom0 time/clock handling --solved

[Jeremy Fitzhardinge]

Daniel Schroeder wrote:
> Jeremy Fitzhardinge wrote:
>   
>> Daniel Schroeder wrote:
>>     
>>> Wed May 27 09:33:12 CEST 2009
>>> Mi Mai 27 11:33:13 CEST 2009
>>>   
>>>       
>> That does look like some kind of TZ issue, but I'm not sure where it
>> would be.
>>
>>     
>>> btw: i have checked with 32 bit pvops and 64 bit pvops dom0 and the time
>>>  in the domU is correct with the 64bit pvops dom0 kernel...this wasnt
>>> the same domU...so, next step for me, is to copy the domU to the 64bit
>>> system and try again...to verify, that this only happens for me with
>>> 32bit pvops dom0...
>>>   
>>>       
>> How odd.
>>
>>     
>>>> One difference between pvops time handling and the -xen kernels, is that
>>>> they defaulted to slaving the domU time off the hypervisor at all times,
>>>> so a system time change would propagate into guests.  I don't implement
>>>> that in pvops kernels, so they'll maintain independent time unless you
>>>> explicitly sync with some mechanism like ntp.
>>>>     
>>>>         
>>> hmm...does this mean, that i have to use ntp in domU if i use the pvops
>>> kernel? Because time changes in dom0 doesnt propagate into domUs?
>>>   
>>>       
>> It will set its initial time-of-day clock from the hypervisor's at boot,
>>     
>
> hmm...hwclock --show on affected system:
>
> hwclock --show
> Wed May 27 23:06:20 2009  -0.778274 seconds
>
> hwclock was set to localtime...
>
> on not affected system
>
> hwclock --show
> Cannot access the Hardware Clock via any known method.
> Use the --debug option to see the details of our search for an access
> method.
>
> i have changed the settings in my distro from hwclock localtime to UTC,
> reset the hwclock
> hwclock --utc --systohc
> rebooted
> everything is fine now...

Hm, that probably needs looking at.  hwclock directly pokes the hardware 
to set the system time, which isn't very nice; there's a proper 
hypercall to do that.  The fact that it has different behaviour on 32 
and 64 bit is particularly ugly...

Not sure what the right answer would be.  From a user perspective, I 
guess making hwclock do the right thing under Xen is the answer, but I'm 
not sure how/where it is maintained.

    J

Re: [XEN-3.4] pv_ops dom0 time/clock handling

[Daniel Schroeder]

Jeremy Fitzhardinge wrote:
> Daniel Schroeder wrote:
>> Jeremy Fitzhardinge wrote:
>>  
>>> Daniel Schroeder wrote:
>>>    
>>>> Wed May 27 09:33:12 CEST 2009
>>>> Mi Mai 27 11:33:13 CEST 2009
>>>>         
>>> That does look like some kind of TZ issue, but I'm not sure where it
>>> would be.
>>>
>>>    
>>>> btw: i have checked with 32 bit pvops and 64 bit pvops dom0 and the
>>>> time
>>>>  in the domU is correct with the 64bit pvops dom0 kernel...this wasnt
>>>> the same domU...so, next step for me, is to copy the domU to the 64bit
>>>> system and try again...to verify, that this only happens for me with
>>>> 32bit pvops dom0...
>>>>         
>>> How odd.
>>>
>>>    
>>>>> One difference between pvops time handling and the -xen kernels, is
>>>>> that
>>>>> they defaulted to slaving the domU time off the hypervisor at all
>>>>> times,
>>>>> so a system time change would propagate into guests.  I don't
>>>>> implement
>>>>> that in pvops kernels, so they'll maintain independent time unless you
>>>>> explicitly sync with some mechanism like ntp.
>>>>>             
>>>> hmm...does this mean, that i have to use ntp in domU if i use the pvops
>>>> kernel? Because time changes in dom0 doesnt propagate into domUs?
>>>>         
>>> It will set its initial time-of-day clock from the hypervisor's at boot,
>>>     
>>
>> hmm...hwclock --show on affected system:
>>
>> hwclock --show
>> Wed May 27 23:06:20 2009  -0.778274 seconds
>>
>> hwclock was set to localtime...
>>
>> on not affected system
>>
>> hwclock --show
>> Cannot access the Hardware Clock via any known method.
>> Use the --debug option to see the details of our search for an access
>> method.
>>
>> i have changed the settings in my distro from hwclock localtime to UTC,
>> reset the hwclock
>> hwclock --utc --systohc
>> rebooted
>> everything is fine now...
> 
> Hm, that probably needs looking at.  hwclock directly pokes the hardware
> to set the system time, which isn't very nice; there's a proper
> hypercall to do that.  The fact that it has different behaviour on 32
> and 64 bit is particularly ugly...

the issue with my 64bit kernel is a missing driver, so that hwclock
cannot find the clock and cannot update it(i think, that this is a
driver/kernel option issue).So the hw clock is still in factory default
mode (utc). My specific 32bit system gets updated by my settings of my
distribution to sync the hwclock with system clock and so runs the
hwclock in localtime.

> 
> Not sure what the right answer would be.  From a user perspective, I
> guess making hwclock do the right thing under Xen is the answer, but I'm
> not sure how/where it is maintained.

The interesting question for me is: what is the difference between
xenified kernel and pvops kernel? Xenified Kernel had no problem with
localtime hwclock...

You wrote, that the hypervisor provides domUs with hwclock at boot
time...the hypervisor alone cannot estimate if the time it gets from the
hwclock is utc or whatever...

daniel