how to keep time of windows pvhvm synchronized with host after resuming

how to keep time of windows pvhvm synchronized with host after resuming

[ANNIE LI]

Hi

When the virtual machine is resumed from suspend, the guest operating 
system's wall-clock time remains at the value it had at the time of the 
suspension. For linux pvm, we can avoid resolve issue by setting 
independent_wallclock=0. However, HVM(or PVHVM) does not support this 
feature.

I did some investigation on how to implement this in windows 
para-virtualization driver. In \include\xen\interface\xen.h, there are 
several variables about timer such as: wc_sec and wc_nsec, and 
vcpu_time_info struct. It is very strange, wc_sec and wc_nsec is always 
zero for my windows vm with para-virtualization driver. Vcpu_time_info 
only contains time info for specific vcpu. Should i use those variables 
to get accurate time for VM? Is there any corresponding source code or 
algorithms available now? Is it safe to simply update vm time after 
resuming?

Citrix 5.6 Windows para-virtualization driver will notify windows kernel 
time change after resuming, and resumed windows vm with Citrix pv driver 
can keep it's time synchronized with host. Would you like to give me 
some clues about how to implement this?

Any help is greatly appreciated.

Thanks
Annie

RE: how to keep time of windows pvhvm synchronized with host after resuming

[Paul Durrant]

[-- Attachment #1: Type: text/plain, Size: 1882 bytes --]

As I understand it wc_sec + wc_nsec should give the number of nanoseconds since the Unix epoch at which the system was booted.

  Paul

> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-
> bounces@lists.xensource.com] On Behalf Of ANNIE LI
> Sent: 16 September 2010 09:31
> To: xen-devel@lists.xensource.com
> Subject: [Xen-devel] how to keep time of windows pvhvm synchronized
> with host after resuming
> 
> Hi
> 
> When the virtual machine is resumed from suspend, the guest
> operating
> system's wall-clock time remains at the value it had at the time of
> the
> suspension. For linux pvm, we can avoid resolve issue by setting
> independent_wallclock=0. However, HVM(or PVHVM) does not support
> this
> feature.
> 
> I did some investigation on how to implement this in windows
> para-virtualization driver. In \include\xen\interface\xen.h, there
> are
> several variables about timer such as: wc_sec and wc_nsec, and
> vcpu_time_info struct. It is very strange, wc_sec and wc_nsec is
> always
> zero for my windows vm with para-virtualization driver.
> Vcpu_time_info
> only contains time info for specific vcpu. Should i use those
> variables
> to get accurate time for VM? Is there any corresponding source code
> or
> algorithms available now? Is it safe to simply update vm time after
> resuming?
> 
> Citrix 5.6 Windows para-virtualization driver will notify windows
> kernel
> time change after resuming, and resumed windows vm with Citrix pv
> driver
> can keep it's time synchronized with host. Would you like to give me
> some clues about how to implement this?
> 
> Any help is greatly appreciated.
> 
> Thanks
> Annie
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: how to keep time of windows pvhvm synchronized with host after resuming

[Tim Deegan]

At 09:59 +0100 on 16 Sep (1284631199), Paul Durrant wrote:
> As I understand it wc_sec + wc_nsec should give the number of
> nanoseconds since the Unix epoch at which the system was booted.

More specifically, they're what you should add to the current Xen system
time to get the wallclock time (they can change over time as the system
time drifts).  I'm surprised to hear that they're zero.  Even if dom0
isn't updating them (which IIRC pv-ops dom0 doesn't) they should have
been set from the RTC at boot time.  It's possible that your domain's
shared-info page is in the wrong word size - if you haven't set up a
hypercall page or set HVM_PARAM_CALLBACK_IRQ since the domain was
created (i.e. since restore) it might be wrong. 

One problem with using wc_[n]sec to get the wallclock time is that the
usual Xen way to get the system time is to interpolate it from the RDTSC
value and the per-vcpu time info, but in a HVM guest the OS may have
changed the TSC offset to something that the driver can't figure out.
The Citrix PV drivers use the HVMOP_get_time hypercall to get the system
time instead; that hypercall is in xen-unstable but not the 4.0 branch.

If you're using a linux guest with Stefano's pv-on-hvm patches then I
think you should be able to read the PV system time as normal.

Tim.

>   Paul
> 
> > -----Original Message-----
> > From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-
> > bounces@lists.xensource.com] On Behalf Of ANNIE LI
> > Sent: 16 September 2010 09:31
> > To: xen-devel@lists.xensource.com
> > Subject: [Xen-devel] how to keep time of windows pvhvm synchronized
> > with host after resuming
> > 
> > Hi
> > 
> > When the virtual machine is resumed from suspend, the guest
> > operating
> > system's wall-clock time remains at the value it had at the time of
> > the
> > suspension. For linux pvm, we can avoid resolve issue by setting
> > independent_wallclock=0. However, HVM(or PVHVM) does not support
> > this
> > feature.
> > 
> > I did some investigation on how to implement this in windows
> > para-virtualization driver. In \include\xen\interface\xen.h, there
> > are
> > several variables about timer such as: wc_sec and wc_nsec, and
> > vcpu_time_info struct. It is very strange, wc_sec and wc_nsec is
> > always
> > zero for my windows vm with para-virtualization driver.
> > Vcpu_time_info
> > only contains time info for specific vcpu. Should i use those
> > variables
> > to get accurate time for VM? Is there any corresponding source code
> > or
> > algorithms available now? Is it safe to simply update vm time after
> > resuming?
> > 
> > Citrix 5.6 Windows para-virtualization driver will notify windows
> > kernel
> > time change after resuming, and resumed windows vm with Citrix pv
> > driver
> > can keep it's time synchronized with host. Would you like to give me
> > some clues about how to implement this?
> > 
> > Any help is greatly appreciated.
> > 
> > Thanks
> > Annie
> > 
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel

Content-Description: ATT00001..txt
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel


-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, XenServer Engineering
Citrix Systems UK Ltd.  (Company #02937203, SL9 0BG)

Re: how to keep time of windows pvhvm synchronized with host after resuming

[ANNIE LI]

[-- Attachment #1.1: Type: text/plain, Size: 3648 bytes --]



Tim Deegan wrote:
> At 09:59 +0100 on 16 Sep (1284631199), Paul Durrant wrote:
>   
>> As I understand it wc_sec + wc_nsec should give the number of
>> nanoseconds since the Unix epoch at which the system was booted.
>>     
>
> More specifically, they're what you should add to the current Xen system
> time to get the wallclock time (they can change over time as the system
> time drifts).  I'm surprised to hear that they're zero.  Even if dom0
> isn't updating them (which IIRC pv-ops dom0 doesn't) they should have
> been set from the RTC at boot time.  It's possible that your domain's
> shared-info page is in the wrong word size - if you haven't set up a
> hypercall page or set HVM_PARAM_CALLBACK_IRQ since the domain was
> created (i.e. since restore) it might be wrong. 
>   
Thanks very much for the details.
I setup the hypercall page and HVM_PARAM_CALLBACK_IRQ, will do further 
check about this issue.
> One problem with using wc_[n]sec to get the wallclock time is that the
> usual Xen way to get the system time is to interpolate it from the RDTSC
> value and the per-vcpu time info, but in a HVM guest the OS may have
> changed the TSC offset to something that the driver can't figure out.
> The Citrix PV drivers use the HVMOP_get_time hypercall to get the system
> time instead; that hypercall is in xen-unstable but not the 4.0 branch.
>
> If you're using a linux guest with Stefano's pv-on-hvm patches then I
> think you should be able to read the PV system time as normal.
>   
Tons of Thanks. I will do more test and try to implement this as your 
explanation.

Thanks
Annie
>   
> Tim.
>
>   
>>   Paul
>>
>>     
>>> -----Original Message-----
>>> From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-
>>> bounces@lists.xensource.com] On Behalf Of ANNIE LI
>>> Sent: 16 September 2010 09:31
>>> To: xen-devel@lists.xensource.com
>>> Subject: [Xen-devel] how to keep time of windows pvhvm synchronized
>>> with host after resuming
>>>
>>> Hi
>>>
>>> When the virtual machine is resumed from suspend, the guest
>>> operating
>>> system's wall-clock time remains at the value it had at the time of
>>> the
>>> suspension. For linux pvm, we can avoid resolve issue by setting
>>> independent_wallclock=0. However, HVM(or PVHVM) does not support
>>> this
>>> feature.
>>>
>>> I did some investigation on how to implement this in windows
>>> para-virtualization driver. In \include\xen\interface\xen.h, there
>>> are
>>> several variables about timer such as: wc_sec and wc_nsec, and
>>> vcpu_time_info struct. It is very strange, wc_sec and wc_nsec is
>>> always
>>> zero for my windows vm with para-virtualization driver.
>>> Vcpu_time_info
>>> only contains time info for specific vcpu. Should i use those
>>> variables
>>> to get accurate time for VM? Is there any corresponding source code
>>> or
>>> algorithms available now? Is it safe to simply update vm time after
>>> resuming?
>>>
>>> Citrix 5.6 Windows para-virtualization driver will notify windows
>>> kernel
>>> time change after resuming, and resumed windows vm with Citrix pv
>>> driver
>>> can keep it's time synchronized with host. Would you like to give me
>>> some clues about how to implement this?
>>>
>>> Any help is greatly appreciated.
>>>
>>> Thanks
>>> Annie
>>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xensource.com
>>> http://lists.xensource.com/xen-devel
>>>       
>
> Content-Description: ATT00001..txt
>   
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>>     
>
>
>   

[-- Attachment #1.2: Type: text/html, Size: 5086 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: how to keep time of windows pvhvm synchronized with host after resuming

[ANNIE LI]

[-- Attachment #1.1: Type: text/plain, Size: 3614 bytes --]


> More specifically, they're what you should add to the current Xen system
> time to get the wallclock time (they can change over time as the system
> time drifts).  I'm surprised to hear that they're zero.  Even if dom0
> isn't updating them (which IIRC pv-ops dom0 doesn't) they should have
> been set from the RTC at boot time.  It's possible that your domain's
> shared-info page is in the wrong word size - if you haven't set up a
> hypercall page or set HVM_PARAM_CALLBACK_IRQ since the domain was
> created (i.e. since restore) it might be wrong. 
>   
Maybe this is not a problem of shared-info page.
The problem is wc_sec and wc_nsec does not change until i modify 
wallclock time of windows vm manually.
If i do not change the wallclock, wc_sec will keep the same value. For 
example: the initial value of wc_sec is zero,
then it changes to 0x4c64da92 if i change wallclock, 0x4c64da92 is about 
40 years since 1970.

I am using xen3.4 for test, any patch i missed?

Thanks
Annie
> One problem with using wc_[n]sec to get the wallclock time is that the
> usual Xen way to get the system time is to interpolate it from the RDTSC
> value and the per-vcpu time info, but in a HVM guest the OS may have
> changed the TSC offset to something that the driver can't figure out.
> The Citrix PV drivers use the HVMOP_get_time hypercall to get the system
> time instead; that hypercall is in xen-unstable but not the 4.0 branch.
>
> If you're using a linux guest with Stefano's pv-on-hvm patches then I
> think you should be able to read the PV system time as normal.
>
> Tim.
>
>   
>>   Paul
>>
>>     
>>> -----Original Message-----
>>> From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-
>>> bounces@lists.xensource.com] On Behalf Of ANNIE LI
>>> Sent: 16 September 2010 09:31
>>> To: xen-devel@lists.xensource.com
>>> Subject: [Xen-devel] how to keep time of windows pvhvm synchronized
>>> with host after resuming
>>>
>>> Hi
>>>
>>> When the virtual machine is resumed from suspend, the guest
>>> operating
>>> system's wall-clock time remains at the value it had at the time of
>>> the
>>> suspension. For linux pvm, we can avoid resolve issue by setting
>>> independent_wallclock=0. However, HVM(or PVHVM) does not support
>>> this
>>> feature.
>>>
>>> I did some investigation on how to implement this in windows
>>> para-virtualization driver. In \include\xen\interface\xen.h, there
>>> are
>>> several variables about timer such as: wc_sec and wc_nsec, and
>>> vcpu_time_info struct. It is very strange, wc_sec and wc_nsec is
>>> always
>>> zero for my windows vm with para-virtualization driver.
>>> Vcpu_time_info
>>> only contains time info for specific vcpu. Should i use those
>>> variables
>>> to get accurate time for VM? Is there any corresponding source code
>>> or
>>> algorithms available now? Is it safe to simply update vm time after
>>> resuming?
>>>
>>> Citrix 5.6 Windows para-virtualization driver will notify windows
>>> kernel
>>> time change after resuming, and resumed windows vm with Citrix pv
>>> driver
>>> can keep it's time synchronized with host. Would you like to give me
>>> some clues about how to implement this?
>>>
>>> Any help is greatly appreciated.
>>>
>>> Thanks
>>> Annie
>>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xensource.com
>>> http://lists.xensource.com/xen-devel
>>>       
>
> Content-Description: ATT00001..txt
>   
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>>     
>
>
>   

[-- Attachment #1.2: Type: text/html, Size: 4746 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: how to keep time of windows pvhvm synchronized with host after resuming

[Tim Deegan]

At 11:44 +0100 on 17 Sep (1284723898), ANNIE LI wrote:
> Maybe this is not a problem of shared-info page.

> The problem is wc_sec and wc_nsec does not change until i modify
> wallclock time of windows vm manually.
> If i do not change the wallclock, wc_sec will keep the same value. For
> example: the initial value of wc_sec is zero,
> then it changes to 0x4c64da92 if i change wallclock, 0x4c64da92 is
> about 40 years since 1970.
>
> I am using xen3.4 for test, any patch i missed?

Not that I can remember.  You could try scattering printks in 
hvm_latch_shinfo_size() to see if it's getting called at all,
and in arch_set_info_guest() to see if it's calling
update_domain_wallclock_time() like it should. 

Tim.
 
> Thanks
> Annie
> 
> 
> One problem with using wc_[n]sec to get the wallclock time is that the
> usual Xen way to get the system time is to interpolate it from the RDTSC
> value and the per-vcpu time info, but in a HVM guest the OS may have
> changed the TSC offset to something that the driver can't figure out.
> The Citrix PV drivers use the HVMOP_get_time hypercall to get the system
> time instead; that hypercall is in xen-unstable but not the 4.0 branch.
> 
> If you're using a linux guest with Stefano's pv-on-hvm patches then I
> think you should be able to read the PV system time as normal.
> 
> Tim.
> 
> 
> 
>   Paul
> 
> 
> 
> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com<mailto:xen-devel-bounces@lists.xensource.com> [mailto:xen-devel-
> bounces@lists.xensource.com<mailto:bounces@lists.xensource.com>] On Behalf Of ANNIE LI
> Sent: 16 September 2010 09:31
> To: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com>
> Subject: [Xen-devel] how to keep time of windows pvhvm synchronized
> with host after resuming
> 
> Hi
> 
> When the virtual machine is resumed from suspend, the guest
> operating
> system's wall-clock time remains at the value it had at the time of
> the
> suspension. For linux pvm, we can avoid resolve issue by setting
> independent_wallclock=0. However, HVM(or PVHVM) does not support
> this
> feature.
> 
> I did some investigation on how to implement this in windows
> para-virtualization driver. In \include\xen\interface\xen.h, there
> are
> several variables about timer such as: wc_sec and wc_nsec, and
> vcpu_time_info struct. It is very strange, wc_sec and wc_nsec is
> always
> zero for my windows vm with para-virtualization driver.
> Vcpu_time_info
> only contains time info for specific vcpu. Should i use those
> variables
> to get accurate time for VM? Is there any corresponding source code
> or
> algorithms available now? Is it safe to simply update vm time after
> resuming?
> 
> Citrix 5.6 Windows para-virtualization driver will notify windows
> kernel
> time change after resuming, and resumed windows vm with Citrix pv
> driver
> can keep it's time synchronized with host. Would you like to give me
> some clues about how to implement this?
> 
> Any help is greatly appreciated.
> 
> Thanks
> Annie
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com<mailto:Xen-devel@lists.xensource.com>
> http://lists.xensource.com/xen-devel
> 
> 
> 
> Content-Description: ATT00001..txt
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com<mailto:Xen-devel@lists.xensource.com>
> http://lists.xensource.com/xen-devel
> 
> 
> 
> 
> 

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, XenServer Engineering
Citrix Systems UK Ltd.  (Company #02937203, SL9 0BG)

Re: how to keep time of windows pvhvm synchronized with host after resuming

[ANNIE LI]

[-- Attachment #1.1: Type: text/plain, Size: 3160 bytes --]



Tim Deegan wrote:
>
> Not that I can remember.  You could try scattering printks in 
> hvm_latch_shinfo_size() to see if it's getting called at all,
> and in arch_set_info_guest() to see if it's calling
> update_domain_wallclock_time() like it should. 
>   
Thanks, I will add log into those functions to see...

Thanks
Annie
> Tim.
>  
>   
>> Thanks
>> Annie
>>
>>
>> One problem with using wc_[n]sec to get the wallclock time is that the
>> usual Xen way to get the system time is to interpolate it from the RDTSC
>> value and the per-vcpu time info, but in a HVM guest the OS may have
>> changed the TSC offset to something that the driver can't figure out.
>> The Citrix PV drivers use the HVMOP_get_time hypercall to get the system
>> time instead; that hypercall is in xen-unstable but not the 4.0 branch.
>>
>> If you're using a linux guest with Stefano's pv-on-hvm patches then I
>> think you should be able to read the PV system time as normal.
>>
>> Tim.
>>
>>
>>
>>   Paul
>>
>>
>>
>> -----Original Message-----
>> From: xen-devel-bounces@lists.xensource.com<mailto:xen-devel-bounces@lists.xensource.com> [mailto:xen-devel-
>> bounces@lists.xensource.com<mailto:bounces@lists.xensource.com>] On Behalf Of ANNIE LI
>> Sent: 16 September 2010 09:31
>> To: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com>
>> Subject: [Xen-devel] how to keep time of windows pvhvm synchronized
>> with host after resuming
>>
>> Hi
>>
>> When the virtual machine is resumed from suspend, the guest
>> operating
>> system's wall-clock time remains at the value it had at the time of
>> the
>> suspension. For linux pvm, we can avoid resolve issue by setting
>> independent_wallclock=0. However, HVM(or PVHVM) does not support
>> this
>> feature.
>>
>> I did some investigation on how to implement this in windows
>> para-virtualization driver. In \include\xen\interface\xen.h, there
>> are
>> several variables about timer such as: wc_sec and wc_nsec, and
>> vcpu_time_info struct. It is very strange, wc_sec and wc_nsec is
>> always
>> zero for my windows vm with para-virtualization driver.
>> Vcpu_time_info
>> only contains time info for specific vcpu. Should i use those
>> variables
>> to get accurate time for VM? Is there any corresponding source code
>> or
>> algorithms available now? Is it safe to simply update vm time after
>> resuming?
>>
>> Citrix 5.6 Windows para-virtualization driver will notify windows
>> kernel
>> time change after resuming, and resumed windows vm with Citrix pv
>> driver
>> can keep it's time synchronized with host. Would you like to give me
>> some clues about how to implement this?
>>
>> Any help is greatly appreciated.
>>
>> Thanks
>> Annie
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com<mailto:Xen-devel@lists.xensource.com>
>> http://lists.xensource.com/xen-devel
>>
>>
>>
>> Content-Description: ATT00001..txt
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com<mailto:Xen-devel@lists.xensource.com>
>> http://lists.xensource.com/xen-devel
>>
>>
>>
>>
>>
>>     
>
>   

[-- Attachment #1.2: Type: text/html, Size: 4569 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: how to keep time of windows pvhvm synchronized with host after resuming

[ANNIE LI]

> Not that I can remember.  You could try scattering printks in
> hvm_latch_shinfo_size() to see if it's getting called at all,
> and in arch_set_info_guest() to see if it's calling
> update_domain_wallclock_time() like it should.
hvm_latch_shinfo_size() is called and HVM_PARAM_CALLBACK_IRQ is sent to 
do_hvm_op in hvm.c.

Totally, 4 functions call update_domain_wallclock_time, they are 
rtc_set_time(), arch_set_info_guest(), construct_dom0() and 
do_settime().The result is:
rtc_set_time() is never called.
construct_dom0() and arch_set_info_guest() are called once. However, 
update_domain_wallclock_time() was not called in arch_set_info_guest() 
since v->vcpu_id is 1 instead of 0. Is it the expected result?

do_settime() is called regularly. The call route is 
do_platform_op()->XENPF_settime->do_settime()->update_domain_wallclock_time(). 

Is do_settime(...) the function to update wc_sec and wc_nsec? Parameters 
secs and nsecs are always variable. It seems wc_sec and wc_nsec are 
calculated from secs, nsecs and system_time_base, but wc_sec keep 
unchanged all the time. Anything else i missed?

Thanks
Annie

Re: how to keep time of windows pvhvm synchronized with host after resuming

[Tim Deegan]

At 09:51 +0100 on 22 Sep (1285149077), ANNIE LI wrote:
> > Not that I can remember.  You could try scattering printks in
> > hvm_latch_shinfo_size() to see if it's getting called at all,
> > and in arch_set_info_guest() to see if it's calling
> > update_domain_wallclock_time() like it should.
> hvm_latch_shinfo_size() is called and HVM_PARAM_CALLBACK_IRQ is sent to 
> do_hvm_op in hvm.c.
> 
> Totally, 4 functions call update_domain_wallclock_time, they are 
> rtc_set_time(), arch_set_info_guest(), construct_dom0() and 
> do_settime().The result is:
> rtc_set_time() is never called.
> construct_dom0() and arch_set_info_guest() are called once. However, 
> update_domain_wallclock_time() was not called in arch_set_info_guest() 
> since v->vcpu_id is 1 instead of 0. Is it the expected result?

I had expected arch_set_info_guest() to be called for every vcpu in the
HVM guest on restore, because boot_vcpu() is called from hvm_load_cpu_ctxt().

> do_settime() is called regularly. The call route is 
> do_platform_op()->XENPF_settime->do_settime()->update_domain_wallclock_time(). 
> 
> Is do_settime(...) the function to update wc_sec and wc_nsec? Parameters 
> secs and nsecs are always variable. It seems wc_sec and wc_nsec are 
> calculated from secs, nsecs and system_time_base, but wc_sec keep 
> unchanged all the time. Anything else i missed?

Yes, do_settime() updates the master copy of wc_sec and wc_nsec from its
inputs, and then copies them to all domains' private copies.  So the
question is:
 - is the _master_ copy of wc_sec always zero; or
 - is the master copy right and the HVM domain's copy wrong; or
 - is the HVM domain's copy right as seen from Xen but 
   wrong as seen from the tools inside the guest?

(i.e. is wallclock broken, is propagation broken, or are the tools and
Xen using different layouts for the shared_info page?)

Cheers,

Tim.

> Thanks
> Annie

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, XenServer Engineering
Citrix Systems UK Ltd.  (Company #02937203, SL9 0BG)

Re: how to keep time of windows pvhvm synchronized with host after resuming

[ANNIE LI]

Sorry for my slow response since vacation in China now. :-)
>
> I had expected arch_set_info_guest() to be called for every vcpu in the
> HVM guest on restore, because boot_vcpu() is called from hvm_load_cpu_ctxt().
arch_set_info_guest is called when vm is booting, but it is not called 
on HVM guest restore.
I will check boot_vcpu() and hvm_load_cpu_ctxt() further.
> Yes, do_settime() updates the master copy of wc_sec and wc_nsec from its
> inputs, and then copies them to all domains' private copies.  So the
> question is:
>   - is the _master_ copy of wc_sec always zero; or
 From my recent logs, wc_sec is not zero, but it always keeps as certain 
unchanged value. In do_settime(...),
Input parameters secs, nsecs and system_time_base change every time when 
calling this function. But wc_secs
keep unchanged although it is calculated from those 3 parameters by: 
(secs * 1000000000ULL) + (u64)nsecs - system_time_base;
Anything wrong here?
>   - is the master copy right and the HVM domain's copy wrong; or
Is update_domain_wallclock_time doing HVM domain's copy? From my log, 
wc_secs of HVM domain's copy is
same as that of master copy.
>   - is the HVM domain's copy right as seen from Xen but
>     wrong as seen from the tools inside the guest?
wc_secs value from tools inside the guest is same as HVM domain's copy.
> (i.e. is wallclock broken, is propagation broken, or are the tools and
> Xen using different layouts for the shared_info page?)
Maybe something wrong, it seems the log printed is not identical with 
what you expected.

Thanks
Annie.

Re: how to keep time of windows pvhvm synchronized with host after resuming

[Tim Deegan]

At 14:53 +0100 on 24 Sep (1285340029), ANNIE LI wrote:
> > I had expected arch_set_info_guest() to be called for every vcpu in the
> > HVM guest on restore, because boot_vcpu() is called from hvm_load_cpu_ctxt().
> arch_set_info_guest is called when vm is booting, but it is not called 
> on HVM guest restore.
> I will check boot_vcpu() and hvm_load_cpu_ctxt() further.
> > Yes, do_settime() updates the master copy of wc_sec and wc_nsec from its
> > inputs, and then copies them to all domains' private copies.  So the
> > question is:
> >   - is the _master_ copy of wc_sec always zero; or
>  From my recent logs, wc_sec is not zero, but it always keeps as certain 
> unchanged value. In do_settime(...),
> Input parameters secs, nsecs and system_time_base change every time when 
> calling this function. But wc_secs
> keep unchanged although it is calculated from those 3 parameters by: 
> (secs * 1000000000ULL) + (u64)nsecs - system_time_base;
> Anything wrong here?

No, that's fine.  wc_secs is what you add to NOW() to get wallclock to
it shouldn't change, or at least very rarely.  I thought from your
earlier email that it was always zero, which would be odd.

> >   - is the master copy right and the HVM domain's copy wrong; or
> Is update_domain_wallclock_time doing HVM domain's copy? From my log, 
> wc_secs of HVM domain's copy is
> same as that of master copy.

That's reassuring.

> >   - is the HVM domain's copy right as seen from Xen but
> >     wrong as seen from the tools inside the guest?
> wc_secs value from tools inside the guest is same as HVM domain's copy.

And is that equal to zero?  If not, what exactly is the problem?  I
think I've got confused about what's going wrong.

Cheers,

Tim.

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, XenServer Engineering
Citrix Systems UK Ltd.  (Company #02937203, SL9 0BG)

Re: how to keep time of windows pvhvm synchronized with host after resuming

[ANNIE LI]

On 2010-9-24 23:18, Tim Deegan wrote:
>
> No, that's fine.  wc_secs is what you add to NOW() to get wallclock to
> it shouldn't change, or at least very rarely.  I thought from your
> earlier email that it was always zero, which would be odd.

Thanks for your confirmation.
  I thought wc_secs is a changeable value, so i am wrong.
The NOW() is calling get_s_time() here, which has problem with RDTSC you 
mentioned.
So the next step is to get system time through hypercall HVMOP_get_time, 
and then add wc_secs
to get wallclock.
Is it the patch you mentioned in 
http://lists.xensource.com/archives/html/xen-devel/2010-07/msg00477.html ?
I will add this patch and get the wallclock time then.

>
> And is that equal to zero?  If not, what exactly is the problem?  I
> think I've got confused about what's going wrong.

Yes, I got zero value for wc_secs several days ago. But it became 
non-zero after i rebooted dom0,
and i failed to reproduce the issue now. I assumed wc_secs should change 
regularly.
Sorry for my misunderstanding.

Thanks
Annie

Re: how to keep time of windows pvhvm synchronized with host after resuming

[ANNIE LI]

  Thank you so much!

I tried the patch in 
http://lists.xensource.com/archives/html/xen-devel/2010-07/msg00477.html.
And i can get the right wallclock time through wc_sec, wc_nsec and 
hypercall HVMOP_get_time now.

BTW, I still do not know why I always hit issue several days ago: wc_sec 
kept zero value on boot,
and changed to non-zero value if i changed clock time manually in 
windows vm . But things went well after
i rebooted dom0. Maybe something wrong with the system at that time.

Thanks
Annie

Re: how to keep time of windows pvhvm synchronized with host after resuming

[Tim Deegan]

At 08:27 +0100 on 25 Sep (1285403255), ANNIE LI wrote:
> I tried the patch in 
> http://lists.xensource.com/archives/html/xen-devel/2010-07/msg00477.html.
> And i can get the right wallclock time through wc_sec, wc_nsec and 
> hypercall HVMOP_get_time now.

Great!

> BTW, I still do not know why I always hit issue several days ago: wc_sec 
> kept zero value on boot,
> and changed to non-zero value if i changed clock time manually in 
> windows vm . But things went well after
> i rebooted dom0. Maybe something wrong with the system at that time.

I can't think why that would have happened.  If it starts happening
again maybe we can figure it out.

Cheers,

Tim.

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, XenServer Engineering
Citrix Systems UK Ltd.  (Company #02937203, SL9 0BG)

Re: how to keep time of windows pvhvm synchronized with host after resuming

[ANNIE LI]

On 2010-9-27 17:53, Tim Deegan wrote:
> I can't think why that would have happened.  If it starts happening
> again maybe we can figure it out.

Yes, i will save my system for debug if this issue happens again.

BTW, do_settime() is not called immediately after 
save/restore/migration, and i must wait for several seconds to get 
wc_secs and wc_nsecs to update wallclock time of guest.

do_platform_op()->XENPF_settime->do_settime()->update_domain_wallclock_time() 
is called regularly, what is interval time of calling do_platform_op? 
Can I change internal time of calling do_stetime()?

Thanks
Annie
> Cheers,
>
> Tim.
>

Re: how to keep time of windows pvhvm synchronized with host after resuming

[Tim Deegan]

At 04:54 +0100 on 28 Sep (1285649648), ANNIE LI wrote:
> do_platform_op()->XENPF_settime->do_settime()->update_domain_wallclock_time() 
> is called regularly, what is interval time of calling do_platform_op? 

That's determined by the timekeeping code in the dom0 kernel.  It's
making hypercalls to tell Xen what time it is.

Tim.

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, XenServer Engineering
Citrix Systems UK Ltd.  (Company #02937203, SL9 0BG)