Re: [Qemu-devel] Re: [libvirt] Libvirt debug API

[Anthony Liguori <anthony@codemonkey.ws>]

On 04/23/2010 07:48 AM, Avi Kivity wrote:
> On 04/22/2010 09:49 PM, Anthony Liguori wrote:
>>> real API. Say, adding a device libvirt doesn't know about or 
>>> stopping the VM
>>> while libvirt thinks it's still running or anything like that.
>>   Another problem is issuing Monitor commands that could confuse 
>> libvirt's
>>
>> We need to make libvirt and qemu smarter.
>>
>> We already face this problem today with multiple libvirt users.  This 
>> is why sophisticated management mechanisms (like LDAP) have 
>> mechanisms to do transactions or at least a series of atomic operations.
>
> And people said qmp/json was overengineered...
>
> But seriously, transactions won't help anything.  qemu maintains 
> state, and when you have two updaters touching a shared variable not 
> excepting each other to, things break, no matter how much locking 
> there is.

Let's consider some concrete examples.  I'm using libvirt and QMP and in 
QMP, I want to hot unplug a device.

Today, I do this by listing the pci devices, and issuing a pci_del that 
takes a PCI address.  This is intrinsically racy though because in the 
worst case scenario, in between when I enumerate pci devices and do the 
pci_del in QMP, in libvirt, I've done a pci_del and then a pci_add 
within libvirt of a completely different device.

There are a few ways to solve this, the simplest being that we give 
devices unique ids that are never reused and instead of pci_del taking a 
pci bus address, it takes a device id.  That would address this race.

You can get very far by just being clever about unique ids and 
notifications.  There are some cases where a true RMW may be required 
but I can't really think of one off hand.  The way LDAP addresses this 
is that it has a batched operation and a simple set of boolean 
comparison operations.  This lets you execute a batched operation that 
will do a RMW.

>   The only way that separate monitors could work is if they touch 
> completely separate state, which is difficult to ensure if you upgrade 
> your libvirt.
>

I don't think this is as difficult of a problem as you think it is.  If 
you look at Active Directory and the whole set of management tools based 
on it, they certainly allow concurrent management applications.  You can 
certainly get into trouble still but with just some careful 
considerations, you can make two management applications work together 
90% of the time without much fuss on the applications part.

Regards,

Anthony Liguori