Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

["Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com>]

On 12/08/2012 12:01 AM, Tejun Heo wrote:
> Hello, Srivatsa.
> 
> On Fri, Dec 07, 2012 at 11:54:01PM +0530, Srivatsa S. Bhat wrote:
>>> lg_lock doesn't do local nesting and I'm not sure how big a deal that
>>> is as I don't know how many should be converted.  But if nesting is an
>>> absolute necessity, it would be much better to implement generic
>>> rwlock variant (say, lg_rwlock) rather than implementing unusual
>>> cpuhotplug-specific percpu synchronization construct.
>>
>> To be honest, at a certain point in time while designing this, I did
>> realize that this was getting kinda overly complicated ;-) ... but I
>> wanted to see how this would actually work out when finished and get
>> some feedback on the same, hence I posted it out. But this also proves
>> that we _can_ actually compete with the flexibility of preempt_disable()
>> and still be safe with respect to locking, if we really want to ;-)
> 
> I got confused by comparison to preempt_disable() but you're right
> that percpu rwlock shouldn't be able to introduce locking dependency
> which doesn't exist with non-percpu rwlock.  ie. write locking should
> be atomic w.r.t. to all readers.

Yep!

>  At the simplest, this can be
> implemented by writer backing out all the way if try-locking any CPU
> fails and retrying the whole thing.  That should be correct but has
> the potential of starving the writer.
> 

Exactly! This is what I mentioned yesterday in the link below, and said
that its not good because of writer starvation ("too wasteful"):
https://lkml.org/lkml/2012/12/6/290

> What we need here is a generic percpu-rwlock.  I don't know which
> exact implementation strategy we should choose.  Maybe your switching
> to global rwlock is the right solution.  But, at any rate, I think it
> would be best to implement proper percpu-rwlock and then apply it to
> CPU hotplug.  It's actually gonna be pretty fitting as
> get_online_cpus() is being converted to percpu-rwsem.  IIUC, Oleg has
> been working on this for a while now.  Oleg, what do you think?
> 

Hmm, that sounds good.

Regards,
Srivatsa S. Bhat