SMP/DRBVD issues ...

[Gareth Bult <gareth@bult.co.uk>]

Hi,

I'm new to the list so apologies if this is known / fixed, but I've not been able to find satisfactory answers in the archives.

I'm running a number of boxes with KVM on the stock Ubuntu 9.10 kernel.
Generally it works very well and I have live migration working on DRBD volumes - very impressed - good job!

However I have a number of issues that I note others have also reported, for which I've not seen fixes;

a. SMP, it appears on Ubuntu 9.10 at the very least - does not work. Whereas setting -smp 2 does actually
   start two kvm threads, the overall performance of the VM is slower than if you use -smp 1, AND the combined
   kvm threads use way more CPU on the host than they should.

   [note; this is using virt-manager to setup and maintain, CPU's are AMD Phenom II X4 @ 3.2G ]

   I think that some of the performance hit comes down to processes not being tagged to specific CPU's -
   I've noticed on Zen that if you run a 4 thread guess on a 4 core CPU with nothing else running, so it doesn't
   need to move threads between cores, you get quite a large performance boost.
   But, this doesn't really cover the huge impact on the host. The guest can be showing 15% CPU util when
   configured with 4 cores, while the host is showing 280%.

   I can supply more information if needed, but the problem seems to blatant I'm hoping people already know
   about it and that can someone can supply some details re; a way forward.

b. DRBD and migration, in order to make this work both hosts it appears must be configured for with the
    parameter 'allow-two-primaries'.  This makes me a little nervous, but it does seem to work. There is
    however one massive flaw, KVM does not seem to be DRBD aware and with two volumes on two machines,
    it's possible to start two instances of a given virtual machine. i.e. neither instance successfully locks the
    volume to prevent another instance also starting on it. As you will guess, this has a detrimental effect on
    the underlying volume.

    Is there some way to make KVM apply a lock to a DRBD device such that another instance of the VM cannot
    be started on another host?  (incidentally, XEN does this 'out of the box' for drbd volumes, so I'm guessing
    it is possible somehow...? )

    Note; it's nice to configure VM's to auto start on a given machine so in the event of a power failure the VM will
    boot with the host. However, if this machine fails and you need to start the VM on an alternative machine,
    when the original machine recovers / reboots, it will attempt (and succeed) in auto booting the same VM
    leaving you with two copies of the same VM and a screwed guest image.

Bit of slightly off topic feedback for anyone using Ubuntu 9.10 .. firstly virt-manager / libvirt leaves a migrated
volume in "paused" state following a migration. Clicking "pause" twice following a migration will resume the
guest on the new host. (I've seen this one reported a number of times)

What I've not see reported is that with an Ubuntu 9.10 guest, shortly after migration a vanilla Ubuntu 9.10 server
guest will hang, usually immediately it's resumed. The apparent fix (it's worked on all of the VM's I was having a
problem with) is "apt-get install acpid" (!)

Any help / suggestions re; SMP/DRBD would be much appreciated.

Many thanks.

Gareth.