Re: [POC][PATCH] xfs: reduce ilock contention on buffered randrw workload

[Davidlohr Bueso <dbueso@suse.com>]

On Thu, 2019-04-18 at 13:10 +1000, Dave Chinner wrote:
> Now the stuff I've been working on has the same interface as
> Davidlohr's patch, so I can swap and change them without thinking
> about it. It's still completely unoptimised, but:
> 
> 			IOPS read/write (direct IO)
> processes	rwsem		DB rangelock	XFS
> rangelock
>  1		78k / 78k	75k / 75k	72k / 72k
>  2		131k / 131k	123k / 123k	133k / 133k
>  4		267k / 267k	183k / 183k	237k / 237k
>  8		372k / 372k	177k / 177k	265k / 265k
>  16		315k / 315k	135k / 135k	228k / 228k
> 
> It's still substantially faster than the interval tree code.

In general another big difference between both rangelock vs rwsems
(when comparing them with full ranges) is that the latter will do
writer optimistic spinning, so saving a context switch under the right
scenarios provides mayor wins for rwsems -- I'm not sure if this
applies to your fio tests, though. And pretty soon readers will also do
this, hence rwsem will become a try-hard-not-to-sleep lock.

One of the reasons why I was hesitant with Btrees was the fact that
insertion requires memory allocation, something I wanted to avoid...
per your numbers, sacrificing tree depth was the wrong choice. Thanks
for sharing these numbers.

> 
> BTW, if I take away the rwsem serialisation altogether, this
> test tops out at just under 500k/500k at 8 threads, and at 16
> threads has started dropping off (~440k/440k). So the rwsem is
> a scalability limitation at just 8 threads....
> 
> /me goes off and thinks more about adding optimistic lock coupling
> to the XFS iext btree to get rid of the need for tree-wide
> locking altogether

I was not aware of this code.

Thanks,
Davidlohr