From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dan Magenheimer Subject: Re: Questions on the PIT and APIC timer in Xen Date: Thu, 29 Mar 2012 08:08:43 -0700 (PDT) Message-ID: References: <09b9e0f3-8548-447f-ba52-33992932f321@default> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0387058376339498618==" Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Bei Guan Cc: Xen Devel , "Andrei E. Warkentin" List-Id: xen-devel@lists.xenproject.org --===============0387058376339498618== Content-Type: multipart/alternative; boundary="__1333033731149157247abhmt112.oracle.com" --__1333033731149157247abhmt112.oracle.com Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable (sorry all for the top post... it's hard to reply to HTML properly in my ma= iler) Hi Ben - =20 On recent Linux kernels, setting "clocksource=3Dtsc tsc=3Dreliable" as gues= t kernel parameters should be faster than pvclock because pvclock reads the= tsc to drive its algorithm. CMOS timer is very very slow. If you are not familiar with virtual time is= sues, you should read the VMware paper. Xen virtual time management is a l= ittle different (because VMware can do binary translation) but most of the = concepts are the same and this paper also is a very good explanation of all= the different clock/timer hardware: =20 http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf =20 From: Bei Guan [mailto:gbtju85@gmail.com]=20 Sent: Wednesday, March 28, 2012 7:45 PM To: Dan Magenheimer Cc: Xen Devel; Andrei E. Warkentin Subject: Re: [Xen-devel] Questions on the PIT and APIC timer in Xen =20 =20 2012/3/28 Dan Magenheimer Both the PIT and APIC (as well as HPET) timers are emulated by Xen for an H= VM guest. However there have always been rare hard-to-reproduce problems w= hen guests use them. At one time, there were workarounds that required a s= pecial parameter "timer_mode" to be selected for HVM guests. If I recall c= orrectly, different timer modes overcame different limitations of different= timer emulations (or maybe mismatches between guest "bugs" exposed by virt= ual time and true hypervisor emulation). These timer_mode workarounds may = or may not work with more recent Linux and Windows guests. And there are s= till periodic bug fixes being applied for timers in the hypervisor emulatio= n code. =20 At 4.0, full TSC emulation was added in an attempt to provide a high-resolu= tion guaranteed monotonic timer. However, most legacy HVM guests cannot o= r refuse to use TSC as a timer. (And there may be lurking bugs there as we= ll.) For most PV guest OS's, the pvclock algorithm is used so PIT/APIC/HPET emul= ation should be unnecessary (and, if used, is likely buggy). And actually = pvclock should be superceded by using TSC, which is always faster, but for = most workloads the performance difference doesn't matter. =20 If you are interested in a good (but difficult) Xen project, "virtual time"= still needs work and documentation. I would be happy to provide lots of b= ackground documentation but can't volunteer to help much more than that rig= ht now. Hi Dan, =20 Thank you for your detailed reply and your kindness. So, you mean the pvclock algorithm is very faster than any Xen emulated tim= er, while a TSC timer is very faster than the pvclock for some workloads, r= ight? Is it possible that Xen initial its timer from the hardware now, such as CM= OS timer? Why PV guest OS doesn't use this timer instead of using the timer= from the pv clock algorithm? If there is any problem with my question, ple= ase correct me.=20 =20 Thank you very much.=20 Bei =20 =20 =20 =20 Hope that helps, Dan =20 From: Bei Guan [mailto:HYPERLINK "mailto:gbtju85@gmail.com" \ngbtju85@gmail= .com]=20 Sent: Wednesday, March 28, 2012 1:32 AM To: Xen Devel Cc: Andrei E. Warkentin Subject: [Xen-devel] Questions on the PIT and APIC timer in Xen =20 Hi, =20 I wonder whether the PIT/APIC timer for HVM Guest OS is emulated by Xen or = not. What's difference between setting the boot PIT/APIC timer for a PV Guest OS= and HVM Guest OS? =20 =20 I would appreciate any reply for my question. =20 --=20 Best Regards, Bei Guan =20 =20 --=20 Best Regards, Bei Guan =20 --__1333033731149157247abhmt112.oracle.com Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: quoted-printable

(sorry al= l for the top post... it’s hard to reply to HTML properly in my maile= r)


Hi Ben –=

 

On recent Linux kernels, setting “clocksour= ce=3Dtsc tsc=3Dreliable” as guest kernel parameters should be faster = than pvclock because pvclock reads the tsc to drive its algorithm.


CMOS timer is very very slow.&= nbsp; If you are not familiar with virtual time issues, you should read the= VMware paper.  Xen virtual time management is a little different (bec= ause VMware can do binary translation) but most of the concepts are the sam= e and this paper also is a very good explanation of all the different clock= /timer hardware:

&nbs= p;

http://www.vmware.com/files/pdf= /Timekeeping-In-VirtualMachines.pdf

 

From: Bei Guan [mailto:gbtju85@gmail.com]
Sent: We= dnesday, March 28, 2012 7:45 PM
To: Dan Magenheimer
Cc:= Xen Devel; Andrei E. Warkentin
Subject: Re: [Xen-devel] Question= s on the PIT and APIC timer in Xen

 

 

2012/3/28 Dan Ma= genheimer <dan.magenheimer= @oracle.com>

Both the PIT and = APIC (as well as HPET) timers are emulated by Xen for an HVM guest.  H= owever there have always been rare hard-to-reproduce problems when guests u= se them.  At one time, there were workarounds that required a special = parameter “timer_mode” to be selected for HVM guests.  If = I recall correctly, different timer modes overcame different limitations of= different timer emulations (or maybe mismatches between guest “bugs&= #8221; exposed by virtual time and true hypervisor emulation).  These = timer_mode workarounds may or may not work with more recent Linux and Windo= ws guests.  And there are still periodic bug fixes being applied for t= imers in the hypervisor emulation code.

=  

At 4.0, full TSC emulation was ad= ded in an attempt to provide a high-resolution guaranteed monotonic timer. =   However, most legacy HVM guests cannot or refuse to use TSC as = a timer.  (And there may be lurking bugs there as well.)


For most PV guest OS’s, the pvclock algorit= hm is used so PIT/APIC/HPET emulation should be unnecessary (and, if used, = is likely buggy).  And actually pvclock should be superceded by using = TSC, which is always faster, but for most workloads the performance differe= nce doesn’t matter.

 

If you are interested in a good (but difficul= t) Xen project, “virtual time” still needs work and documentati= on.  I would be happy to provide lots of background documentation but = can’t volunteer to help much more than that right now.

Hi Dan,

 

Thank you for your detailed reply and your kindness.=

So, you mean the pvclock algorithm is v= ery faster than any Xen emulated timer, while a TSC timer is very faster th= an the pvclock for some workloads, right?

Is it possible that Xen initial its timer from the hardware no= w, such as CMOS timer? Why PV guest OS doesn't use this timer instead of us= ing the timer from the pv clock algorithm? If there is any problem with my = question, please correct me. 

 

Thank you very m= uch. 

Bei

=

 

 

 <= /o:p>



--
Best Regard= s,

 

--__1333033731149157247abhmt112.oracle.com-- --===============0387058376339498618== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============0387058376339498618==--