Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

[Tejun Heo <tj@kernel.org>]

Hello, Srivatsa.

On Fri, Dec 07, 2012 at 11:54:01PM +0530, Srivatsa S. Bhat wrote:
> > lg_lock doesn't do local nesting and I'm not sure how big a deal that
> > is as I don't know how many should be converted.  But if nesting is an
> > absolute necessity, it would be much better to implement generic
> > rwlock variant (say, lg_rwlock) rather than implementing unusual
> > cpuhotplug-specific percpu synchronization construct.
> 
> To be honest, at a certain point in time while designing this, I did
> realize that this was getting kinda overly complicated ;-) ... but I
> wanted to see how this would actually work out when finished and get
> some feedback on the same, hence I posted it out. But this also proves
> that we _can_ actually compete with the flexibility of preempt_disable()
> and still be safe with respect to locking, if we really want to ;-)

I got confused by comparison to preempt_disable() but you're right
that percpu rwlock shouldn't be able to introduce locking dependency
which doesn't exist with non-percpu rwlock.  ie. write locking should
be atomic w.r.t. to all readers.  At the simplest, this can be
implemented by writer backing out all the way if try-locking any CPU
fails and retrying the whole thing.  That should be correct but has
the potential of starving the writer.

What we need here is a generic percpu-rwlock.  I don't know which
exact implementation strategy we should choose.  Maybe your switching
to global rwlock is the right solution.  But, at any rate, I think it
would be best to implement proper percpu-rwlock and then apply it to
CPU hotplug.  It's actually gonna be pretty fitting as
get_online_cpus() is being converted to percpu-rwsem.  IIUC, Oleg has
been working on this for a while now.  Oleg, what do you think?

Thanks.

-- 
tejun