Re: [RFC] Cancellable MCS spinlock rework

[Waiman Long <waiman.long@hp.com>]

On 07/03/2014 04:51 PM, Jason Low wrote:
> On Thu, 2014-07-03 at 16:35 -0400, Waiman Long wrote:
>> On 07/03/2014 02:34 PM, Jason Low wrote:
>>> On Thu, 2014-07-03 at 10:09 -0700, Davidlohr Bueso wrote:
>>>> On Thu, 2014-07-03 at 09:31 +0200, Peter Zijlstra wrote:
>>>>> On Wed, Jul 02, 2014 at 10:30:03AM -0700, Jason Low wrote:
>>>>>> Would potentially reducing the size of the rw semaphore structure by 32
>>>>>> bits (for all architectures using optimistic spinning) be a nice
>>>>>> benefit?
>>>>> Possibly, although I had a look at the mutex structure and we didn't
>>>>> have a hole to place it in, unlike what you found with the rwsem.
>>>> Yeah, and currently struct rw_semaphore is the largest lock we have in
>>>> the kernel. Shaving off space is definitely welcome.
>>> Right, especially if it could help things like xfs inode.
>>>
>> I do see a point in reducing the size of the rwsem structure. However, I
>> don't quite understand the point of converting pointers in the
>> optimistic_spin_queue structure to atomic_t.
> Converting the pointers in the optimistic_spin_queue to atomic_t would
> mean we're fully operating on atomic operations instead of using the
> potentially racy cmpxchg + ACCESS_ONCE stores on the pointers.

Yes, the ACCESS_ONCE macro for data store does have problem on some 
architectures. However, I prefer a more holistic solution to solve this 
problem rather than a workaround by changing the pointers to atomic_t's. 
It is because even if we make the change,  we are still not sure if that 
will work for those architectures as we have no machine to verify that. 
Why not let the champions of those architectures to propose changes 
instead of making some untested changes now and penalize commonly used 
architectures like x86.

> If we're in the process of using the CPU numbers in atomic_t, I thought
> we might as well fix that as well since it has actually been shown to
> result in lockups on some architectures. We can then avoid needing to
> implement the tricky architecture workarounds for optimistic spinning.
> Wouldn't that be a "nice-have"?
>
> Jason
>

I am not aware of any tricky architectural workarounds other than 
disabling optimistic spinning for those that don't support it.

-Longman