Re: [PATCH linux-2.6 01/04] brsem: implement big reader semaphore

[Tejun Heo <htejun@gmail.com>]

  Hello, Nick.

Nick Piggin wrote:
> Tejun Heo wrote:
> 
>> 01_brsem_implement_brsem.patch
>>
>>     This patch implements big reader semaphore - a rwsem with very
>>     cheap reader-side operations and very expensive writer-side
>>     operations.  For details, please read comments at the top of
>>     kern/brsem.c.
>>
> 
> This thing looks pretty overengineered. It is difficult to
> read with all the little wrapper functions and weird naming
> schemes.

  As I've said in the other reply, I'll first rip off three stage init 
thing for cpucontrol.  That added pretty much complexity to it.  And 
with the weird naming scheme, please tell me how to fix it.  I have no 
problem renaming things.

> What would be wrong with an array of NR_CPUS rwsems? The only
> tiny trick you would have to do AFAIKS is have up_read remember
> what rwsem down_read took, but that could be returned from
> down_read as a token.

  Using array of rwsems means that every reader-side ops performs 
(unnecessary) real down and up operations on the semaphore which involve 
atomic memory op and probably memory barrier.  These are pretty 
expensive operations.

  What brsem tries to do is implementing rwsem semantics while 
performing only normal (as opposed to atomic/barrier) intstructions 
during reader-side operations.  That's why all the call_on_all_cpus 
stuff is needed while write-locking.  Do you think avoiding 
atomic/barrier stuff would be an overkill?

> I have been meaning to do something like this for mmap_sem to
> see what happens to page fault scalability (though the heavy
> write-side would make such a scheme unsuitable for mainline).

  I agree that brsem write-locking would be too heavy for mmap_sep for 
usual cases.

  Thanks.

-- 
tejun