Re: Linux spin lock enhancement on xen

[George Dunlap <George.Dunlap@eu.citrix.com>]

On Tue, Aug 24, 2010 at 2:25 PM, Jan Beulich <JBeulich@novell.com> wrote:
> No, I understood it that way. What I was referring to is (as an
> example) the case where two vCPU-s on the sam pCPU's run queue
> both yield: They will each move after the other in the run queue in
> close succession, but neither will really make progress, and neither
> will really increase the likelihood of the respective lock holder to
> get a chance to run.

Ah, I see.  In order for this to be a waste, it needs to be the case that:
* Two vcpus from different domains grab a spinlock and are then preempted
* Two vcpus from different domains then fail to grab the spinlock
* The two vcpus holding the locks are kept from getting cpu by
{another vcpu, other vcpus} which uses a long time-slice
* The two waiting for the lock share a cpu with each other and no one else

Of course in this situation, it would be nice if Xen could migrate one
of the other vcpus to the cpu of the two yielding vcpus.  That
shouldn't be too hard to implement, at least to see if it has a
measurable impact on aggregate throughput.

> Immediately, or after a few (hundred) spin cycles?

It depends on the implementation.  The Citrix guest tools do binary
patching of spinlock routines for w2k3 and XP; I believe they spin for
1000 cycles or so.  The viridian enlightenments I believe would yield
immediately.  I think the pause instruction causes a yield immediately
as well.

Yielding immediately when the host is not overloaded is actually
probably not optimal: if the vcpu holding the lock is currently
running, it's likely that by the time the vcpu makes it to the
scheduler, the lock it's waiting for has already been released.
(Which is part of the reason it's a spinlock and not a semaphore.)

> And so I do with this. Apart from suspecting fairness issues with
> your yield_to proposal (as I wrote), my point just is - we won't
> know if a "complicated" solution outperforms a "simple" one if we
> don't try it.

Are you volunteering? :-)

 -George