Re: [PATCH] VFS: br_write_lock locks on possible CPUs other than online CPUs

[mengcong <mc@linux.vnet.ibm.com>]

On Tue, 2011-12-20 at 06:27 +0000, Al Viro wrote:
> On Tue, Dec 20, 2011 at 10:26:05AM +0530, Srivatsa S. Bhat wrote:
> > Oh, right, that has to be handled as well...
> > 
> > Hmmm... How about registering a CPU hotplug notifier callback during lock init
> > time, and then for every cpu that gets onlined (after we took a copy of the
> > cpu_online_mask to work with), we see if that cpu is different from the ones
> > we have already locked, and if it is, we lock it in the callback handler and
> > update the locked_cpu_mask appropriately (so that we release the locks properly
> > during the unlock operation).
> > 
> > Handling the newly introduced race between the callback handler and lock-unlock
> > code must not be difficult, I believe..
> > 
> > Any loopholes in this approach? Or is the additional complexity just not worth
> > it here?
> 
> To summarize the modified variant of that approach hashed out on IRC:
> 
On which IRC do you discuss?

> 	* lglock grows three extra things: spinlock, cpu bitmap and cpu hotplug
> notifier.
> 	* foo_global_lock_online starts with grabbing that spinlock and
> loops over the cpus in that bitmap.
> 	* foo_global_unlock_online loops over the same bitmap and then drops
> that spinlock
> 	* callback of the notifier is going to do all bitmap updates.  Under
> that spinlock.  Events that need handling definitely include the things like
> "was going up but failed", since we need the bitmap to contain all online CPUs
> at all time, preferably without too much junk beyond that.  IOW, we need to add
> it there _before_ low-level __cpu_up() calls set_cpu_online().  Which means
> that we want to clean up on failed attempt to up it.  Taking a CPU down is
> probably less PITA; just clear bit on the final "the sucker's dead" event.
> 	* bitmap is initialized once, at the same time we set the notifier
> up.  Just grab the spinlock and do
> 	for_each_online_cpu(N)
> 		add N to bitmap
> then release the spinlock and let the callbacks handle all updates.
> 
> I think that'll work with relatively little pain, but I'm not familiar enough
> with the cpuhotplug notifiers, so I'd rather have the folks familiar with those
> to supply the set of events to watch for...
>