Re: [PATCH] rwsem-spinlock: let rwsem write lock stealable

[Ingo Molnar <mingo@kernel.org>]

* Yuanhan Liu <yuanhan.liu@linux.intel.com> wrote:

> > > output with this patch:
> > > -----------------------
> > > cpu 00:   0   0   ...   1   1   2   1   1   1   2   1   1   1 .... 1   3
> > > cpu 01:   0   0   ...   1   1   1   1   1   1   2   1   1   1 .... 1   3
> > > cpu 02:   0   0   ...   2   2   3   2   0   2   1   2   1   1 .... 1   1
> > > cpu 03:   0   0   ...   2   2   3   2   1   2   1   2   1   1 .... 1   1
> > > cpu 04:   0   1   ...   2   0   0   1   0   1   3   1   1   1 .... 1   1
> > > cpu 05:   0   1   ...   2   0   1   1   0   1   2   1   1   1 .... 1   1
> > > cpu 06:   0   0   ...   2   1   1   2   0   1   2   1   1   1 .... 2   1
> > > cpu 07:   0   0   ...   2   1   1   2   0   1   2   1   1   1 .... 2   1
> > > cpu 08:   0   0   ...   1   1   1   1   1   1   1   1   1   1 .... 0   0
> > > cpu 09:   0   0   ...   1   1   1   1   1   1   1   1   1   1 .... 0   0
> > > cpu 10:   0   0   ...   1   1   1   0   0   1   1   1   1   1 .... 0   0
> > > cpu 11:   0   0   ...   1   1   1   0   0   1   1   1   1   2 .... 1   0
> > > cpu 12:   0   0   ...   1   1   1   0   1   1   0   0   0   1 .... 2   1
> > > cpu 13:   0   0   ...   1   1   1   0   1   1   1   0   1   2 .... 2   0
> > > cpu 14:   0   0   ...   2   0   0   0   0   1   1   1   1   1 .... 2   2
> > > cpu 15:   0   0   ...   2   0   0   1   0   1   1   1   1   1 .... 2   2
> > > ------------------------------------------------------------------------
> > > Where you can see that CPU is much busier with this patch.
> > 
> > That looks really good - quite similar to how it behaved 
> > with mutexes, right?
> 
> Yes :)
> 
> And the result is almost same with mutex lock when MUTEX_SPIN_ON_OWNER
> is disabled, and that's the reason you will see massive processes(about
> 100) queued on each CPU in my last report:
>     https://lkml.org/lkml/2013/1/29/84

Just curious: how does MUTEX_SPIN_ON_OWNER versus 
!MUTEX_SPIN_ON_OWNER compare, for this particular, 
massively-contended anon-vma locks benchmark?

> > Does this recover most of the performance regression?
> 
> Yes, there is only a 10% gap here then. I guess that's because 
> I used the general rwsem lock 
> implementation(lib/rwsem-spinlock.c), but not the XADD 
> one(lib/rwsem.c). I guess the gap may be a little smaller if 
> we do the same thing to lib/rwsem.c.

Is part of the gap due to MUTEX_SPIN_ON_OWNER perhaps?

I'm surprised that rwsem-spinlock versus rwsem.c would show a 
10% performance difference - assuming you have lock 
debugging/tracing disabled in the .config.

( Once the performance regression is fixed, another thing to 
  check would be to reduce anon-vma lock contention. )

Thanks,

	Ingo