Re: [PATCH RFC tip/core/rcu] Parallelize and economize NOCB kthread wakeups

Mike Galbraith
2014-07-04T03:24:02Z

On Thu, 2014-07-03 at 09:29 -0700, Paul E. McKenney wrote:
> On Thu, Jul 03, 2014 at 07:48:40AM +0200, Mike Galbraith wrote:
> > On Wed, 2014-07-02 at 22:21 -0700, Paul E. McKenney wrote:
> > > On Thu, Jul 03, 2014 at 05:31:19AM +0200, Mike Galbraith wrote:
> >
> > > > NO_HZ_FULL is a property of a set of CPUs. isolcpus is supposed to go
> > > > away as being a redundant interface to manage a single property of a set
> > > > of CPUs, but it's perfectly fine for NO_HZ_FULL to add an interface to
> > > > manage a single property of a set of CPUs. What am I missing?
> > >
> > > Well, for now, it can only be specified at build time or at boot time.
> > > In theory, it is possible to change a CPU from being callback-offloaded
> > > to not at runtime, but there would need to be an extremely good reason
> > > for adding that level of complexity. Lots of "fun" races in there...
> >
> > Yeah, understood.
> >
> > (still it's a NO_HZ_FULL wart though IMHO, would be prettier and more
> > usable if it eventually became unified with cpuset and learned how to
> > tap-dance properly;)
>
> Agreed, it would in some sense be nice. What specifically do you need
> it for?

I personally have zero use for the thing (git/vi aren't particularly
perturbation sensitive;). I'm just doing occasional drive-by testing
from a distro perspective, how well does it work, what does it cost etc.

> Are you really running workloads that generate large numbers of
> callbacks spread across most of the CPUs? It was this sort of workload
> that caused Rik's system to show scary CPU-time accumulation, due to
> the high overhead of frequent one-to-many wakeups.
>
> If your systems aren't running that kind of high-callback-rate workload,
> just set CONFIG_RCU_NOCB_CPU_ALL=y and don't worry about it.
>
> If your systems -are- running that kind of high-callback-rate workload,
> but your system has fewer than 200 CPUs, ensure that you have enough
> housekeeping CPUs to allow the grace-period kthread sufficient CPU time,
> set CONFIG_RCU_NOCB_CPU_ALL=y and don't worry about it.
>
> If your systems -are- running that kind of high-callback-rate workload,
> and your system has more than 200 CPUs, apply the following patch,
> set CONFIG_RCU_NOCB_CPU_ALL=y and once again don't worry about it. ;-)

Turn it on and don't worry about it is exactly what distros want the
obscure feature with very few users to be. Last time I did a drive-by,
my boxen said I should continue to worry about it ;-)

-Mike