Re: 4.2.2: NR_CPUS effectively being 1 bug

[Mike Galbraith <umgwanakikbuti@gmail.com>]

On Sat, 2015-10-03 at 10:52 +0200, Ingo Molnar wrote:
> * Mike Galbraith <umgwanakikbuti@gmail.com> wrote:
> 
> > On Sat, 2015-10-03 at 10:18 +0200, Ingo Molnar wrote:
> > > * Mike Galbraith <umgwanakikbuti@gmail.com> wrote:
> > > 
> > > > On Sat, 2015-10-03 at 09:38 +0200, Ingo Molnar wrote:
> > > > > * Peter Zijlstra <peterz@infradead.org> wrote:
> > > > > 
> > > > > > On Fri, Oct 02, 2015 at 02:00:33PM -0700, Andrew Morton wrote:
> > > > > > > On Fri, 2 Oct 2015 23:46:59 +0300 Alexey Dobriyan <adobriyan@gmail.com> wrote:
> > > > > > > 
> > > > > > > > FYI, I've upgraded from 4.1.7 to 4.2.1 (and retested with 4.2.2) and
> > > > > > > > everything is scheduled on 1 CPU out of 4 (i5 760).
> > > > > > > > 
> > > > > > > > 	$ sudo cat /proc/1/status | grep cpu -i
> > > > > > > > 	Cpus_allowed:   1
> > > > > > > > 	Cpus_allowed_list:      0
> > > > > > > > 
> > > > > > > > Every process inherits this tiny cpumask.
> > > > > > > 
> > > > > > > Sell the other CPUs on ebay?
> > > > > > > 
> > > > > > > I haven't seen such a report before - maybe it rings a bell with Peter
> > > > > > > & Ingo?
> > > > > > 
> > > > > > I think this is related to some NO_HZ_FULL quackery. People seem to have
> > > > > > enabled stuff they've really no sane reason for.
> > > > > 
> > > > > So the question is, is CONFIG_NO_HZ_FULL_ALL=y enabled? If yes then please disable 
> > > > > it.
> > > > > 
> > > > > Frederic, is there a fix for that? The Kconfig help text for CONFIG_NO_HZ_FULL_ALL 
> > > > > says::
> > > > > 
> > > > >  CONFIG_NO_HZ_FULL_ALL:
> > > > > 
> > > > >  If the user doesn't pass the nohz_full boot option to
> > > > >  define the range of full dynticks CPUs, consider that all
> > > > >  CPUs in the system are full dynticks by default.
> > > > >  Note the boot CPU will still be kept outside the range to
> > > > >  handle the timekeeping duty.
> > > > > 
> > > > > I can see people enabling that. Why are all CPUs lost if it's done?
> > > > 
> > > > Simple.  Rik made it such that cpu_isolated_map is immune to cpusets,
> > > > and Chris made tick_nohz_full_mask automatically set cpu_isolated_map,
> > > > so now if nohz_full is ever turned on, that CPU is gone from the generic
> > > > pool forever, with obvious consequences for CONFIG_NO_HZ_FULL_ALL.
> > > > 
> > > > Lovely eh?
> > > 
> > > Which commit is that? Needs to be reverted...
> > 
> > 8cb9764f nohz: Set isolcpus when nohz_full is set
> > 
> > That's the one that makes them all automatically go away.
> 
> Ok, looks like a patch with good intentions but bad effects. Mind sending a revert 
> patch, changelogged, signed off?

No.  They know.

	-Mike