Re: [PATCH RFC tip/core/rcu] Eliminate deadlock between CPU hotplug and expedited grace periods

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Wed, Sep 03, 2014 at 05:28:50PM +0200, Peter Zijlstra wrote:
> On Wed, Sep 03, 2014 at 08:03:06AM -0700, Paul E. McKenney wrote:
> > > > Normal RCU grace periods avoid this by synchronizing on a lock acquired by
> > > > the RCU CPU-hotplug notifiers, but this does not work for the expedited
> > > > grace periods because the outgoing CPU can be running random tasks for
> > > > quite some time after RCU's notifier executes.  So the fix is just to
> > > > drop back to a normal grace period when there is a CPU-hotplug operation
> > > > in progress.
> > > 
> > > So why are we 'normally' doing an expedited call here anyhow? 
> > 
> > Presumably because they set either the boot parameter or
> > the sysfs variable that causes synchronize_sched() to so
> > synchronize_sched_expedited().
> 
> That's not a why but a how. Why does that option exist, why are we doing
> this?

As you say below, to reduce RCU grace-period latency on small systems.
And one level of abstraction's why is another level's how.  ;-)

> I cannot actually find a sysfs variable that controls this though; only
> the rcu_pm_notifier. It seems to favour doing an expedited call when
> suspending on 'small' machines.

See rcu_expedited_store() in kernel/ksysfs.c.  Or the rcu_expedited
module_param() in kernel/rcu/update.c.

> > > But those are not within hotplug bits. Also weren't we removing them? I
> > > thought we didn't appreciate spraying IPIs like they do?
> > 
> > I hadn't heard anything about removing them, but making the
> > expedited primitives a bit less IPI-happy is on my list.
> 
> I had some recollections of removing a fair number of expedited calls,
> but its was a long while ago so what do I know ;-)

If a given use case can tolerate the latency of normal calls, then the
normal calls certainly are preferable, no two ways about it.

> Making them less IPI happy would be good indeed.

Priority of this task duly increased.  ;-)

							Thanx, Paul