Re: Elaboration of "Question about sharing spinlock_t among VMs in Xen"

[Juergen Gross <jgross@suse.com>]

On 27/06/16 17:24, Dagaen Golomb wrote:
>>>>> *** The question is as follows ***
>>>>> Suppose I have two Linux VMs sharing the same spinlock_t lock (through
>>>>> the sharing memory) on the same host. Suppose we have one process in
>>>>> each VM. Each process uses the linux function spin_lock(&lock) [1] to
>>>>> grab & release the lock.
>>>>> Will these two processes in the two VMs have race on the shared lock?
>>>
>>>> You can't do this: depending on which Linux version you use you will
>>>> find that kernel uses ticketlocks or qlocks locks which keep track of
>>>> who is holding the lock (obviously this information is internal to VM).
>>>> On top of this on Xen we use pvlocks which add another (internal)
>>>> control layer.
>>>
>>> I wanted to see if this can be done with the correct combination of
>>> versions and parameters. We are using 4.1.0 for all domains, which
>>> still has the CONFIG_PARAVIRT_SPINLOCK option. I've recompiled the
>>> guests with this option set to n, and have also added the boot

Just a paranoid question: how exactly does the .config line look like?
It should _not_ be

CONFIG_PARAVIRT_SPINLOCK=n

but rather:

# CONFIG_PARAVIRT_SPINLOCK is not set

>>> parameter xen_nopvspin to both domains and dom0 for good measure. A
>>> basic ticketlock holds all the information inside the struct itself to
>>> order the requests, and I believe this is the version I'm using.
>>
>> Hm, weird. B/c from arch/x86/include/asm/spinlock_types.h:
>>   6 #ifdef CONFIG_PARAVIRT_SPINLOCKS
>>   7 #define __TICKET_LOCK_INC       2
>>   8 #define TICKET_SLOWPATH_FLAG    ((__ticket_t)1)
>>   9 #else
>>  10 #define __TICKET_LOCK_INC       1
>>  11 #define TICKET_SLOWPATH_FLAG    ((__ticket_t)0)
>>  12 #endif
>>  13
>>
>> Which means that one of your guests is adding '2' while another is
>> adding '1'. Or one of them is putting the 'slowpath' flag
>> which means that the paravirt spinlock is enabled.
> 
> Interesting. I went back to check on one of my guests, and the .config
> from the source tree I used, as well as the one in /boot/ for the
> current build both have it "not set" which shows as unchecked in make
> menuconfig, where the option was disabled. So this domain appears to
> be correctly configured. The thing is, the other domain is literally a
> copy of this domain. Either both are wrong or neither are.

One other thing you should be aware of: as soon as one of your guests
has only one vcpu it will drop the "lock" prefixes for updates of the
lock word. So there will be a chance of races just because one or both
guests are thinking no other cpu can access the lock word concurrently.


Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel