Re: Linux spin lock enhancement on xen

[Mukesh Rathor <mukesh.rathor@oracle.com>]

On Tue, 17 Aug 2010 10:43:04 -0700
Jeremy Fitzhardinge <jeremy@goop.org> wrote:

>  On 08/16/2010 06:33 PM, Mukesh Rathor wrote:
> > In my worst case test scenario, I get about 20-36% improvement when
> > the system is two to three times over provisioned. 
> >
> > Please provide any feedback. I would like to submit official patch
> > for SCHEDOP_yield_to in xen.
> 
> This approach only works for old-style spinlocks.  Ticketlocks also
> have the problem of making sure the next vcpu gets scheduled on
> unlock.

Well, unfortunately, looks like old-style spinlocks are gonna be around 
for a very long time. I've heard there are customers still on EL3!


> Have you looked at the pv spinlocks I have upstream in the pvops
> kernels, which use the (existing) poll hypercall to block the waiting
> vcpu until the lock is free?
>     J

>How does this compare with Jeremy's existing paravirtualised spinlocks
>in pv_ops? They required no hypervsior changes. Cc'ing Jeremy.
> -- Keir

Yeah, I looked at it today. What pv-ops is doing is forcing a yield
via a fake irq/event channel poll, after storing the lock pointer in
a per cpu area. The unlock'er then IPIs the vcpus waiting. The lock
holder may not be running tho, and there is no hint to hypervisor
to run it. So you may have many waitor's come and leave for no
reason.

To me this is more of an overhead than needed in a guest. In my
approach, the hypervisor is hinted exactly which vcpu is the 
lock holder. Often many VCPUs are pinned to a set of physical cpus
due to licensing and other reasons. So this really helps a vcpu
that is holding a spin lock, wanting to do some possibly real
time work, get scheduled and move on. Moreover, number of vcpus is
going up pretty fast.

Thanks,
Mukesh