Re: [PATCH tip/core/rcu 02/12] rcu: Panic if RCU tree can not accommodate all CPUs

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Thu, Jul 30, 2015 at 05:34:24PM +0200, Peter Zijlstra wrote:
> On Thu, Jul 30, 2015 at 05:32:51PM +0200, Peter Zijlstra wrote:
> > On Thu, Jul 30, 2015 at 08:25:17AM -0700, Paul E. McKenney wrote:
> > > On Thu, Jul 30, 2015 at 02:28:35PM +0200, Peter Zijlstra wrote:
> > > > On Fri, Jul 17, 2015 at 03:30:53PM -0700, Paul E. McKenney wrote:
> > > > > +	if (n > rcu_capacity[MAX_RCU_LVLS])
> > > > > +		panic("rcu_init_geometry: rcu_capacity[] is too small");
> > > > 
> > > > How can this ever happen? We _know_ NR_CPUS at compile time, there's no
> > > > way we can get more CPUs than that -- even if the hardware has more,
> > > > we'll stop enumerating.
> > > 
> > > You can make this happen by building with CONFIG_RCU_FANOUT=2 and
> > > CONFIG_RCU_FANOUT_LEAF=2, then running on a system with more than 16 CPUs.
> > > The kernel boot parameter rcutree.rcu_fanout_leaf=2 can be substituted for
> > > CONFIG_RCU_FANOUT_LEAF=2, hence the need for a runtime test.  I do this
> > > sort of thing for my rcutorture testing in order to test a four-level
> > > rcu_node tree with only 16 CPUs.
> > 
> > How about we make the build fail if NR_CPUS exceeds that maximum fanout?
> 
> Or better yet, auto-select these numbers based on NR_CPUS and remove the
> configs.

The current strategy is to default CONFIG_RCU_FANOUT to 32 on 32-bit
systems and to 64 on 64-bit systems, and to default CONFIG_RCU_FANOUT_LEAF
to 16.  This accommodates up to 4,194,304 CPUs, which should be enough
for anyone (famous last words!).  At boot time, RCU automatically shrinks
its combining tree based on the value of nr_cpu_ids.  So the overall
effect is that very few people should need to mess with these RCU
parameters.

The purpose of the CONFIG_RCU_FANOUT_LEAF=16 default is to keep
lock contention down to a dull roar on systems that deliver the
scheduling-clock interrupts simultaneously across all CPUs, and you should
boot with skew_tick if you want to set CONFIG_RCU_FANOUT_LEAF larger.

							Thanx, Paul