From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752333AbbG3PzB (ORCPT ); Thu, 30 Jul 2015 11:55:01 -0400 Received: from e31.co.us.ibm.com ([32.97.110.149]:48311 "EHLO e31.co.us.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751417AbbG3PzA (ORCPT ); Thu, 30 Jul 2015 11:55:00 -0400 X-Helo: d03dlp01.boulder.ibm.com X-MailFrom: paulmck@linux.vnet.ibm.com X-RcptTo: linux-kernel@vger.kernel.org Date: Thu, 30 Jul 2015 08:54:54 -0700 From: "Paul E. McKenney" To: Peter Zijlstra Cc: linux-kernel@vger.kernel.org, mingo@kernel.org, jiangshanlai@gmail.com, dipankar@in.ibm.com, akpm@linux-foundation.org, mathieu.desnoyers@efficios.com, josh@joshtriplett.org, tglx@linutronix.de, rostedt@goodmis.org, dhowells@redhat.com, edumazet@google.com, dvhart@linux.intel.com, fweisbec@gmail.com, oleg@redhat.com, bobby.prani@gmail.com, Alexander Gordeev Subject: Re: [PATCH tip/core/rcu 02/12] rcu: Panic if RCU tree can not accommodate all CPUs Message-ID: <20150730155454.GH27280@linux.vnet.ibm.com> Reply-To: paulmck@linux.vnet.ibm.com References: <20150717223041.GA14464@linux.vnet.ibm.com> <1437172263-15466-1-git-send-email-paulmck@linux.vnet.ibm.com> <1437172263-15466-2-git-send-email-paulmck@linux.vnet.ibm.com> <20150730122835.GX19282@twins.programming.kicks-ass.net> <20150730152517.GE27280@linux.vnet.ibm.com> <20150730153251.GL25159@twins.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150730153251.GL25159@twins.programming.kicks-ass.net> User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 15073015-8236-0000-0000-00000D9981AF Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Jul 30, 2015 at 05:32:51PM +0200, Peter Zijlstra wrote: > On Thu, Jul 30, 2015 at 08:25:17AM -0700, Paul E. McKenney wrote: > > On Thu, Jul 30, 2015 at 02:28:35PM +0200, Peter Zijlstra wrote: > > > On Fri, Jul 17, 2015 at 03:30:53PM -0700, Paul E. McKenney wrote: > > > > + if (n > rcu_capacity[MAX_RCU_LVLS]) > > > > + panic("rcu_init_geometry: rcu_capacity[] is too small"); > > > > > > How can this ever happen? We _know_ NR_CPUS at compile time, there's no > > > way we can get more CPUs than that -- even if the hardware has more, > > > we'll stop enumerating. > > > > You can make this happen by building with CONFIG_RCU_FANOUT=2 and > > CONFIG_RCU_FANOUT_LEAF=2, then running on a system with more than 16 CPUs. > > The kernel boot parameter rcutree.rcu_fanout_leaf=2 can be substituted for > > CONFIG_RCU_FANOUT_LEAF=2, hence the need for a runtime test. I do this > > sort of thing for my rcutorture testing in order to test a four-level > > rcu_node tree with only 16 CPUs. > > How about we make the build fail if NR_CPUS exceeds that maximum fanout? Good point, and it already does, and I clearly was confused, apologies. So the real way to make this happen is (for example) to build with CONFIG_RCU_FANOUT=2 and CONFIG_RCU_FANOUT_LEAF=16 (the default), which could accommodate up to 128 CPUs. Then boot with rcutree.rcu_fanout_leaf=2 on a system with more than 16 CPUs, with rcutree.rcu_fanout_leaf=3 on a system with more than 24 CPUs, and so on. Of course, the truly macho way to get this error message is to build with CONFIG_RCU_FANOUT=64 and CONFIG_RCU_FANOUT_LEAF=64, then boot with rcutree.rcu_fanout_leaf=63 on a system with more than 16,515,072 CPUs. Of course, you get serious style points if the system manages to stay up for more than 24 hours without a hardware failure. ;-) Thanx, Paul