Re: kvm guest loops_per_jiffy miscalibration under host load

["Daniel P. Berrange" <berrange@redhat.com>]

On Wed, Jul 02, 2008 at 01:40:21PM -0300, Marcelo Tosatti wrote:
> I have been discussing with Glauber and Gerd the problem where KVM
> guests miscalibrate loops_per_jiffy if there's sufficient load on the
> host.
> 
> calibrate_delay_direct() failed to get a good estimate for
> loops_per_jiffy.
> Probably due to long platform interrupts. Consider using "lpj=" boot
> option.
> Calibrating delay loop... <3>107.00 BogoMIPS (lpj=214016)
> 
> While this particular host calculates lpj=1597041.

[snip]

> Possible solutions:
> 
> - Require the admin to preset "lpj=". Nasty, not user friendly.
> - Pass the proper lpj value via a paravirt interface. Won't cover
>   fullvirt guests.
> - Have the management app guarantee a minimum amount of CPU required
> for proper calibration during guest initialization.

I talked with Marcelo about this latter option - the idea being to have
libvirt do  set_priority(PRIO_PROC, $pid, -20) on the KVM binary for
a short time at startup. The problem with this is that there's no
easy way to determine when, or for how long, to provide this priority
boost.   During initial startup there is an arbitrary unknown delay
from the BIOS - 3 seconds to choose the boot device, a further 20-60
seconds if you choose PXE before the guest kernel  is actually booted,
arbitrary delay if booting from CDROM and it has not auto boot timeout,
arbitrary user configurable delay in GRUB to choose kernel if booting
from harddisk.

So to stand any reasonable chance of working libvirt would have to give
the -20 priority boost for a good 60 seconds. If we want to start 2 or
more guests at once this soon stops being a viable workaround. And we
can't detect reboots triggered by the admin inside the guest. So I 
don't see how we can reliably solve this from userspace management
tools, unless someone has better ideas than a priority boost for a short
while.

Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|