common time reference between domU and dom0

common time reference between domU and dom0

[James Harper]

I would like to be able to measure front/back ring message latency
between a Windows DomU and Dom0. To do this I need a common time
reference between the two, preferably with at least microsecond
precision (latency <1us is irrelevant for my purposes). Does such a
thing exist? A small amount of overhead in measuring this latency is
acceptable.

Thanks

James

Re: common time reference between domU and dom0

[Keir Fraser]

System time, as exposed to guests by Xen, should be good enough.

 -- Keir


On 10/12/2010 05:23, "James Harper" <james.harper@bendigoit.com.au> wrote:

> I would like to be able to measure front/back ring message latency
> between a Windows DomU and Dom0. To do this I need a common time
> reference between the two, preferably with at least microsecond
> precision (latency <1us is irrelevant for my purposes). Does such a
> thing exist? A small amount of overhead in measuring this latency is
> acceptable.
> 
> Thanks
> 
> James
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

RE: common time reference between domU and dom0

[James Harper]

> 
> System time, as exposed to guests by Xen, should be good enough.
> 

How is this exposed for HVM systems?

James

Re: common time reference between domU and dom0

[Keir Fraser]

On 10/12/2010 08:23, "James Harper" <james.harper@bendigoit.com.au> wrote:

>> 
>> System time, as exposed to guests by Xen, should be good enough.
>> 
> 
> How is this exposed for HVM systems?

Given all HVM guest time sources are trapped to the hypervisor, including
TSC, you may as well just use the HVMOP_get_time hypercall.

 -- Keir

> James

RE: common time reference between domU and dom0

[James Harper]

> On 10/12/2010 08:23, "James Harper" <james.harper@bendigoit.com.au>
wrote:
> 
> >>
> >> System time, as exposed to guests by Xen, should be good enough.
> >>
> >
> > How is this exposed for HVM systems?
> 
> Given all HVM guest time sources are trapped to the hypervisor,
including
> TSC, you may as well just use the HVMOP_get_time hypercall.
> 

Is that a new call? I don't see it mentioned in 4.0-testing anywhere.

Is there a fallback if HVMOP_get_time doesn't exist?

James

Re: common time reference between domU and dom0

[Paolo Bonzini]

On 12/10/2010 11:20 AM, James Harper wrote:
> Is that a new call? I don't see it mentioned in 4.0-testing anywhere.
>
> Is there a fallback if HVMOP_get_time doesn't exist?

Not really, but it is really an easy backport to your hypervisor.  It 
will even be in RHEL5.6/CentOS 5.6.

Paolo

Re: Re: common time reference between domU and dom0

[Keir Fraser]

On 10/12/2010 10:22, "Paolo Bonzini" <pbonzini@redhat.com> wrote:

> On 12/10/2010 11:20 AM, James Harper wrote:
>> Is that a new call? I don't see it mentioned in 4.0-testing anywhere.
>> 
>> Is there a fallback if HVMOP_get_time doesn't exist?
> 
> Not really, but it is really an easy backport to your hypervisor.  It
> will even be in RHEL5.6/CentOS 5.6.

It's probably worth backporting for 4.0.2 too.

 -- Keir

> Paolo
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

RE: Re: common time reference between domU and dom0

[Dan Magenheimer]

Urk.  Paolo, I hope Redhat isn't using HVMOP_get_time now as a basis
for pvops time?  That was one of the purposes of all the tsc work
in 4.0 so selecting TSC as the clocksource should have the same
result (and IIRC the trap is faster than the hypercall).

> -----Original Message-----
> From: Keir Fraser [mailto:keir@xen.org]
> Sent: Friday, December 10, 2010 3:46 AM
> To: Paolo Bonzini; James Harper
> Cc: xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] Re: common time reference between domU and
> dom0
> 
> On 10/12/2010 10:22, "Paolo Bonzini" <pbonzini@redhat.com> wrote:
> 
> > On 12/10/2010 11:20 AM, James Harper wrote:
> >> Is that a new call? I don't see it mentioned in 4.0-testing
> anywhere.
> >>
> >> Is there a fallback if HVMOP_get_time doesn't exist?
> >
> > Not really, but it is really an easy backport to your hypervisor.  It
> > will even be in RHEL5.6/CentOS 5.6.
> 
> It's probably worth backporting for 4.0.2 too.
> 
>  -- Keir
> 
> > Paolo
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: Re: common time reference between domU and dom0

[Paolo Bonzini]

On 12/10/2010 09:27 PM, Dan Magenheimer wrote:
> Urk.  Paolo, I hope Redhat isn't using HVMOP_get_time now as a basis
> for pvops time?

No.  I only plan to use it in Windows guests.  That said...

> That was one of the purposes of all the tsc work
> in 4.0 so selecting TSC as the clocksource should have the same
> result (and IIRC the trap is faster than the hypercall).

... this has not been backported to RHEL5 though.

Paolo

Re: common time reference between domU and dom0

[Keir Fraser]

On 10/12/2010 10:20, "James Harper" <james.harper@bendigoit.com.au> wrote:

>> On 10/12/2010 08:23, "James Harper" <james.harper@bendigoit.com.au>
> wrote:
>> 
>>>> 
>>>> System time, as exposed to guests by Xen, should be good enough.
>>>> 
>>> 
>>> How is this exposed for HVM systems?
>> 
>> Given all HVM guest time sources are trapped to the hypervisor,
> including
>> TSC, you may as well just use the HVMOP_get_time hypercall.
>> 
> 
> Is that a new call? I don't see it mentioned in 4.0-testing anywhere.

Yeah I think it's in 4.1 only.

> Is there a fallback if HVMOP_get_time doesn't exist?

Switch on vtsc (might be default) and just use RDTSC? Might be good enough.
I'm not sure that computing system time yourself from fields in shared info
works well for HVM guests. HVMOP_get_time I think had been in XenServer
patch queue a while and has just been pulled across to upstream.

 -- Keir

> James
> 

Re: common time reference between domU and dom0

[Tim Deegan]

At 10:45 +0000 on 10 Dec (1291977908), Keir Fraser wrote:
> On 10/12/2010 10:20, "James Harper" <james.harper@bendigoit.com.au> wrote:
> 
> >> On 10/12/2010 08:23, "James Harper" <james.harper@bendigoit.com.au>
> > wrote:
> >> 
> >>>> 
> >>>> System time, as exposed to guests by Xen, should be good enough.
> >>>> 
> >>> 
> >>> How is this exposed for HVM systems?
> >> 
> >> Given all HVM guest time sources are trapped to the hypervisor,
> > including
> >> TSC, you may as well just use the HVMOP_get_time hypercall.
> >> 
> > 
> > Is that a new call? I don't see it mentioned in 4.0-testing anywhere.
> 
> Yeah I think it's in 4.1 only.
> 
> > Is there a fallback if HVMOP_get_time doesn't exist?
> 
> Switch on vtsc (might be default) and just use RDTSC? Might be good enough.

IIRC HVMOP_get_time was added because from Windows drivers you can't
tell what the Windows kernel has done to the TSC so you can't use it to
get system time yourself.  It seems to me that exposing the TSC offset
alongside the rest of the shared info would fix that but maybe you'd be
as well off backporting HVMOP_get_time instead.

Tim.

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, Xen Platform Team
Citrix Systems UK Ltd.  (Company #02937203, SL9 0BG)

RE: common time reference between domU and dom0

[Dan Magenheimer]

> >>> How is this exposed for HVM systems?
> >>
> >> Given all HVM guest time sources are trapped to the hypervisor,
> > including
> >> TSC, you may as well just use the HVMOP_get_time hypercall.
> >>
> >
> > Is that a new call? I don't see it mentioned in 4.0-testing anywhere.
> 
> Yeah I think it's in 4.1 only.
> 
> > Is there a fallback if HVMOP_get_time doesn't exist?
> 
> Switch on vtsc (might be default) and just use RDTSC? Might be good
> enough.
> I'm not sure that computing system time yourself from fields in shared
> info
> works well for HVM guests. HVMOP_get_time I think had been in XenServer
> patch queue a while and has just been pulled across to upstream.

I believe vtsc will be different in different domains as I believe
Xen adjusts it so tsc in any guest effectively starts at zero at time
of domain launch.  Also, an HVM domain may "write" to tsc and
change its effective value.

Also, I'm not sure IIRC, but I think vtsc is never on for dom0.

And IIRC the default for vtsc (vtsc=0) is dependent on the underlying
hardware and events (such as migration/save/restore) that transpire.
It can be forced on in a vm config file (but again IIRC not for dom0).

See xen-XXX.hg/docs/misc/tscmode.txt for XXX >= 4.0

Dan

Re: common time reference between domU and dom0

[Henry Pepper]

I was under the impression that none of the domains has the correct time.
 Neither Dom0 nor DomU receives a timer interrupt, but only a timer
event. Events that could be missed.
So timing in the millisecond can have jitter and drift.
Is there actually a way to keep a precise timer in a domain?

On top of this I would think that both domains would have to have
exclusive access to their own CPU in order to avoid being switched off
of the CPU in periods of time(quantums of 30 ms using the default
scheduler).

  Henry

On Fri, Dec 10, 2010 at 8:19 AM, Keir Fraser <keir@xen.org> wrote:
> System time, as exposed to guests by Xen, should be good enough.
>
>  -- Keir
>
>
> On 10/12/2010 05:23, "James Harper" <james.harper@bendigoit.com.au> wrote:
>
>> I would like to be able to measure front/back ring message latency
>> between a Windows DomU and Dom0. To do this I need a common time
>> reference between the two, preferably with at least microsecond
>> precision (latency <1us is irrelevant for my purposes). Does such a
>> thing exist? A small amount of overhead in measuring this latency is
>> acceptable.
>>
>> Thanks
>>
>> James
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>

Re: common time reference between domU and dom0

[Keir Fraser]

All domains have access to Xen system time, either via hypercall or by
calculating it themselves directly given information Xen provides in shared
memory. Time interrupt/events can't really be missed - they are delivered
synchronously to the guest.

 -- Keir

On 10/12/2010 08:26, "Henry Pepper" <henrypepr@gmail.com> wrote:

> I was under the impression that none of the domains has the correct time.
>  Neither Dom0 nor DomU receives a timer interrupt, but only a timer
> event. Events that could be missed.
> So timing in the millisecond can have jitter and drift.
> Is there actually a way to keep a precise timer in a domain?
> 
> On top of this I would think that both domains would have to have
> exclusive access to their own CPU in order to avoid being switched off
> of the CPU in periods of time(quantums of 30 ms using the default
> scheduler).
> 
>   Henry
> 
> On Fri, Dec 10, 2010 at 8:19 AM, Keir Fraser <keir@xen.org> wrote:
>> System time, as exposed to guests by Xen, should be good enough.
>> 
>>  -- Keir
>> 
>> 
>> On 10/12/2010 05:23, "James Harper" <james.harper@bendigoit.com.au> wrote:
>> 
>>> I would like to be able to measure front/back ring message latency
>>> between a Windows DomU and Dom0. To do this I need a common time
>>> reference between the two, preferably with at least microsecond
>>> precision (latency <1us is irrelevant for my purposes). Does such a
>>> thing exist? A small amount of overhead in measuring this latency is
>>> acceptable.
>>> 
>>> Thanks
>>> 
>>> James
>>> 
>>> 
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xensource.com
>>> http://lists.xensource.com/xen-devel
>> 
>> 
>> 
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel