Re: [RFC PATCH RT] rwsem: The return of multi-reader PI rwsems

[Steven Rostedt <rostedt@goodmis.org>]

On Thu, 10 Apr 2014 17:03:36 +0200
Sebastian Andrzej Siewior <bigeasy@linutronix.de> wrote:

> On 04/10/2014 04:44 PM, Clark Williams wrote:
> > The means of each group of five test runs are:
> > 
> > vanilla.log:  1210117 rt.log:  17210953 (14.2 x slower than
> > vanilla) rt-fixes.log:  10062027 (8.3 x slower than vanilla) 
> > rt-multi.log:  3179582  (2.x x slower than vanilla)
> > 
> > 
> > As expected, vanilla kicked RT's butt when hammering on the 
> > mmap_sem. But somewhat unexpectedly, your fixups helped quite a
> > bit and the multi+fixups got RT back into being almost
> > respectable.
> > 
> > Obviously these are just preliminary results on one piece of h/w
> > but it looks promising.
> 
> Is it easy to look at the latency when you have multiple readers and
> and a high prio writer which has to boost all those readers away
> instead just one?
> Or is this something that should not happen for a high prio RT task
> because it has all memory already allocated?

Note, the patch includes a /proc/sys/kernel/rwsem_reader_limit that is
default value set at possible_cpus. The user can change it to any
number. A number of zero means unlimited, and a number of 1 makes it
work like it did without the patch.

This means that a writer will only have to wait for rwsem_reader_limit
readers, which may give a longer latency, but it is still bounded. Also
note that the rwsem is not fair respect to first come first served, but
priority driven. Same priority tasks may be first in first out, but if
there's a writer waiting and a higher priority reader comes along, it
can still get the lock, making the writer wait longer. But as the
reader is higher priority than the writer, that is expected. The same
priority reader will block if there's a writer waiting.

Clark, it would be interesting if you run the test again with my
patches but set rwsem_reader_limit to 1, and see what the impact of
that is.

-- Steve