kvm-clock clocksource efficiency versus tsc...

kvm-clock clocksource efficiency versus tsc...

[Erik Brakkee]

OS: Centos 6.2
KVM version: qemu-kvm-tools-0.12.1.2-2.209.el6_2.4.x86_64
             qemu-kvm-0.12.1.2-2.209.el6_2.4.x86_64
uname -a:
Linux myhost 2.6.32-220.7.1.el6.x86_64 #1 SMP Wed Mar 7 00:52:02 GMT 2012
x86_64 x86_64 x86_64 GNU/Linux

Hi,


I have been performance testing a time tracing utiliity for a Java
enterprise application at work. The idea is that we measure time for
different parts of our application and build time trees for this
information. The time tree can be viewed and analysed in case of problems.
It is also automatically output in case a certain operation is taking
longer than a threshold.

As part of these tests we found that it matters a lot what type of
clocksource is used. In particular, with hpet and acpi_pm the execution is
very slow (700ns per call (similar results using clock_gettime() in a C
program). In addition, hpet and acpi_pm sycnrhonize the application. This
is of course a disaster for server applications that tend to query the
current time quite a lot.

What works quite well is the tsc clocksource. In that case, the time drops
to about 38ns on one of our systems and we can prove that there is no
synchronization anymore which is good.

When running inside a KVM VM the default clock source is kvm-clock. This
clock takes about 160ns per call and also does not synchronise the
application. However, using the tsc clock source delivers similar
performance on the virtual machine as on the host.

I now have the following questions:
* can the performance of kvm-clock be optimized further to be (almost)
identical to that of the host's clock source?
* what are the consequences of using the tsc clock in combination with
NTPD? Will this result in system instability or larger than usual clock
skew?

Cheers
  Erik

Re: kvm-clock clocksource efficiency versus tsc...

[Marcelo Tosatti]

On Tue, Oct 16, 2012 at 10:54:59AM +0200, Erik Brakkee wrote:
> OS: Centos 6.2
> KVM version: qemu-kvm-tools-0.12.1.2-2.209.el6_2.4.x86_64
>              qemu-kvm-0.12.1.2-2.209.el6_2.4.x86_64
> uname -a:
> Linux myhost 2.6.32-220.7.1.el6.x86_64 #1 SMP Wed Mar 7 00:52:02 GMT 2012
> x86_64 x86_64 x86_64 GNU/Linux
> 
> Hi,
> 
> 
> I have been performance testing a time tracing utiliity for a Java
> enterprise application at work. The idea is that we measure time for
> different parts of our application and build time trees for this
> information. The time tree can be viewed and analysed in case of problems.
> It is also automatically output in case a certain operation is taking
> longer than a threshold.
> 
> As part of these tests we found that it matters a lot what type of
> clocksource is used. In particular, with hpet and acpi_pm the execution is
> very slow (700ns per call (similar results using clock_gettime() in a C
> program). In addition, hpet and acpi_pm sycnrhonize the application. This
> is of course a disaster for server applications that tend to query the
> current time quite a lot.
> 
> What works quite well is the tsc clocksource. In that case, the time drops
> to about 38ns on one of our systems and we can prove that there is no
> synchronization anymore which is good.
> 
> When running inside a KVM VM the default clock source is kvm-clock. This
> clock takes about 160ns per call and also does not synchronise the
> application. However, using the tsc clock source delivers similar
> performance on the virtual machine as on the host.
> 
> I now have the following questions:
> * can the performance of kvm-clock be optimized further to be (almost)
> identical to that of the host's clock source?

There is a patchset on the list "pvclock vsyscall support + KVM hypervisor
support (v2)", using percentages of my testbox it would reduce your case
from 160ns to 64ns. Testers welcome.

> * what are the consequences of using the tsc clock in combination with
> NTPD? Will this result in system instability or larger than usual clock
> skew?

- Can't migrate guest to host with different TSC frequency.
- No stability of system clock with guest vmsave/vmrestore, suspend/resume 
of the host.
- Requires host with stable TSC. 

Excluding the cases above, as long as NTP manages to synchronize the
system clock you're fine (ntpq -c rv "frequency" field below 100).