Re: [PATCH v2 7/4] ipc: fine grained locking for semtimedop

[Rik van Riel <riel@redhat.com>]

On 03/06/2013 05:57 PM, Linus Torvalds wrote:
> On Wed, Mar 6, 2013 at 5:15 PM, Rik van Riel <riel@redhat.com> wrote:
>>
>> If the call is a semop manipulating just one semaphore in
>> an array with multiple semaphores, the read/write lock for
>> the semaphore array is taken in shared (read) mode, and the
>> individual semaphore's lock is taken.
>
> You know, we do something like this already elsewhere, and I think we
> do it slightly better. See the mm_take_all_locks() logic in mm/mmap.c.

That would work. If we are about to do one of the uncommon operations,or 
sma->complex_count is set, we need to take the outer lock and all of the
inner locks.

The only complication would be interactions with the non-semaphore code
in ipc/util.c, which manipulates the kern_ipc_perm structures, which are
part of the sem_array structure.

> That said, judging by your numbers, your read-write lock seems to work
> fine too, even though I'd worry about cacheline ping-pong (but not
> contention) on the readers. So it doesn't seem optimal, but it sure as
> hell seems better than what we do now ;)

I can take a stab at implementing the take_all_locks approach tomorrow.

If things turn out to be easier than I fear, I will send an updated
patch.  If the resulting changes to the rest of ipc/ turn out to be
too ugly to live, the rwsem performance is likely to be good enough
for a while, and I'll just send an email without a patch :)