Re: [PATCH linux-2.6 01/04] brsem: implement big reader semaphore

Tejun Heo
2005-09-25T08:12:15Z

Hello, Nick.

Nick Piggin wrote:
> Tejun Heo wrote:
>
>> 01_brsem_implement_brsem.patch
>>
>> This patch implements big reader semaphore - a rwsem with very
>> cheap reader-side operations and very expensive writer-side
>> operations. For details, please read comments at the top of
>> kern/brsem.c.
>>
>
> This thing looks pretty overengineered. It is difficult to
> read with all the little wrapper functions and weird naming
> schemes.

As I've said in the other reply, I'll first rip off three stage init
thing for cpucontrol. That added pretty much complexity to it. And
with the weird naming scheme, please tell me how to fix it. I have no
problem renaming things.

> What would be wrong with an array of NR_CPUS rwsems? The only
> tiny trick you would have to do AFAIKS is have up_read remember
> what rwsem down_read took, but that could be returned from
> down_read as a token.

Using array of rwsems means that every reader-side ops performs
(unnecessary) real down and up operations on the semaphore which involve
atomic memory op and probably memory barrier. These are pretty
expensive operations.

What brsem tries to do is implementing rwsem semantics while
performing only normal (as opposed to atomic/barrier) intstructions
during reader-side operations. That's why all the call_on_all_cpus
stuff is needed while write-locking. Do you think avoiding
atomic/barrier stuff would be an overkill?

> I have been meaning to do something like this for mmap_sem to
> see what happens to page fault scalability (though the heavy
> write-side would make such a scheme unsuitable for mainline).

I agree that brsem write-locking would be too heavy for mmap_sep for
usual cases.

Thanks.

--
tejun