From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752504AbbG3QNp (ORCPT ); Thu, 30 Jul 2015 12:13:45 -0400 Received: from e36.co.us.ibm.com ([32.97.110.154]:36455 "EHLO e36.co.us.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751235AbbG3QNn (ORCPT ); Thu, 30 Jul 2015 12:13:43 -0400 X-Helo: d03dlp03.boulder.ibm.com X-MailFrom: paulmck@linux.vnet.ibm.com X-RcptTo: linux-kernel@vger.kernel.org Date: Thu, 30 Jul 2015 09:01:41 -0700 From: "Paul E. McKenney" To: Peter Zijlstra Cc: linux-kernel@vger.kernel.org, mingo@kernel.org, jiangshanlai@gmail.com, dipankar@in.ibm.com, akpm@linux-foundation.org, mathieu.desnoyers@efficios.com, josh@joshtriplett.org, tglx@linutronix.de, rostedt@goodmis.org, dhowells@redhat.com, edumazet@google.com, dvhart@linux.intel.com, fweisbec@gmail.com, oleg@redhat.com, bobby.prani@gmail.com, Alexander Gordeev Subject: Re: [PATCH tip/core/rcu 02/12] rcu: Panic if RCU tree can not accommodate all CPUs Message-ID: <20150730160141.GI27280@linux.vnet.ibm.com> Reply-To: paulmck@linux.vnet.ibm.com References: <20150717223041.GA14464@linux.vnet.ibm.com> <1437172263-15466-1-git-send-email-paulmck@linux.vnet.ibm.com> <1437172263-15466-2-git-send-email-paulmck@linux.vnet.ibm.com> <20150730122835.GX19282@twins.programming.kicks-ass.net> <20150730152517.GE27280@linux.vnet.ibm.com> <20150730153251.GL25159@twins.programming.kicks-ass.net> <20150730153424.GR18673@twins.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150730153424.GR18673@twins.programming.kicks-ass.net> User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 15073016-0021-0000-0000-00000EDD58DA Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Jul 30, 2015 at 05:34:24PM +0200, Peter Zijlstra wrote: > On Thu, Jul 30, 2015 at 05:32:51PM +0200, Peter Zijlstra wrote: > > On Thu, Jul 30, 2015 at 08:25:17AM -0700, Paul E. McKenney wrote: > > > On Thu, Jul 30, 2015 at 02:28:35PM +0200, Peter Zijlstra wrote: > > > > On Fri, Jul 17, 2015 at 03:30:53PM -0700, Paul E. McKenney wrote: > > > > > + if (n > rcu_capacity[MAX_RCU_LVLS]) > > > > > + panic("rcu_init_geometry: rcu_capacity[] is too small"); > > > > > > > > How can this ever happen? We _know_ NR_CPUS at compile time, there's no > > > > way we can get more CPUs than that -- even if the hardware has more, > > > > we'll stop enumerating. > > > > > > You can make this happen by building with CONFIG_RCU_FANOUT=2 and > > > CONFIG_RCU_FANOUT_LEAF=2, then running on a system with more than 16 CPUs. > > > The kernel boot parameter rcutree.rcu_fanout_leaf=2 can be substituted for > > > CONFIG_RCU_FANOUT_LEAF=2, hence the need for a runtime test. I do this > > > sort of thing for my rcutorture testing in order to test a four-level > > > rcu_node tree with only 16 CPUs. > > > > How about we make the build fail if NR_CPUS exceeds that maximum fanout? > > Or better yet, auto-select these numbers based on NR_CPUS and remove the > configs. The current strategy is to default CONFIG_RCU_FANOUT to 32 on 32-bit systems and to 64 on 64-bit systems, and to default CONFIG_RCU_FANOUT_LEAF to 16. This accommodates up to 4,194,304 CPUs, which should be enough for anyone (famous last words!). At boot time, RCU automatically shrinks its combining tree based on the value of nr_cpu_ids. So the overall effect is that very few people should need to mess with these RCU parameters. The purpose of the CONFIG_RCU_FANOUT_LEAF=16 default is to keep lock contention down to a dull roar on systems that deliver the scheduling-clock interrupts simultaneously across all CPUs, and you should boot with skew_tick if you want to set CONFIG_RCU_FANOUT_LEAF larger. Thanx, Paul