[PATCH] sched: Fix isolcpus boot option

[PATCH] sched: Fix isolcpus boot option

[Anton Blanchard]

We allocate and zero cpu_isolated_map after the isolcpus __setup option
has run. This means cpu_isolated_map always ends up empty and if
CPUMASK_OFFSTACK is enabled we write to a cpumask that hasn't been
allocated.

To keep the fix to a minimum this patch stores a pointer to the cmdline
option and parses it after we allocate and zero the cpumask.

Signed-off-by: Anton Blanchard <anton@samba.org>
---

Might be worth moving cpu_isolated_map into kernel/cpu.c and treat it
like the other cpu masks but I figured that change would have trouble
making 2.6.32. Thoughts?

Index: linux.trees.git/kernel/sched.c
===================================================================
--- linux.trees.git.orig/kernel/sched.c	2009-11-28 09:35:29.000000000 +1100
+++ linux.trees.git/kernel/sched.c	2009-11-28 21:50:44.000000000 +1100
@@ -8098,10 +8098,14 @@ cpu_attach_domain(struct sched_domain *s
 /* cpus with isolated domains */
 static cpumask_var_t cpu_isolated_map;
 
-/* Setup the mask of cpus configured for isolated domains */
+/*
+ * Since it's too early to allocate cpu_isolated_map we just store
+ * a pointer to the parameter and allocate and initialize it in sched_init
+ */
+static __initdata char *isolated_cpu_str;
 static int __init isolated_cpu_setup(char *str)
 {
-	cpulist_parse(str, cpu_isolated_map);
+	isolated_cpu_str = str;
 	return 1;
 }
 
@@ -9650,6 +9654,7 @@ void __init sched_init(void)
 	alloc_cpumask_var(&nohz.ilb_grp_nohz_mask, GFP_NOWAIT);
 #endif
 	zalloc_cpumask_var(&cpu_isolated_map, GFP_NOWAIT);
+	cpulist_parse(isolated_cpu_str, cpu_isolated_map);
 #endif /* SMP */
 
 	perf_event_init();

Re: [PATCH] sched: Fix isolcpus boot option

[Peter Zijlstra]

On Mon, 2009-11-30 at 11:05 +1100, Anton Blanchard wrote:
> Might be worth moving cpu_isolated_map into kernel/cpu.c and treat it
> like the other cpu masks but I figured that change would have trouble
> making 2.6.32. Thoughts? 

I'd love to kill off isolcpus in favour of exclusive cpusets, except
that we need a bit more work on the latter to really make that happen.

Re: [PATCH] sched: Fix isolcpus boot option

[Rusty Russell]

On Mon, 30 Nov 2009 10:35:32 am Anton Blanchard wrote:
> 
> We allocate and zero cpu_isolated_map after the isolcpus __setup option
> has run. This means cpu_isolated_map always ends up empty and if
> CPUMASK_OFFSTACK is enabled we write to a cpumask that hasn't been
> allocated.
> 
> To keep the fix to a minimum this patch stores a pointer to the cmdline
> option and parses it after we allocate and zero the cpumask.

I introduced this regression in:

	commit 49557e620339cb134127b5bfbcfecc06b77d0232
	Author: Rusty Russell <rusty@rustcorp.com.au>
	Date:   Mon Nov 2 20:37:20 2009 +1030

	    sched: Fix boot crash by zalloc()ing most of the cpu masks

But this change made isolcpus= never work, CPUMASK_OFFSTACK or no.

Your patch works and is minimal.

Acked-by: Rusty Russell <rusty@rustcorp.com.au>

Thanks,
Rusty.

PS. This would have also worked:

diff --git a/kernel/sched.c b/kernel/sched.c
--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -8045,6 +8045,7 @@ static cpumask_var_t cpu_isolated_map;
 /* Setup the mask of cpus configured for isolated domains */
 static int __init isolated_cpu_setup(char *str)
 {
+	alloc_bootmem_cpumask_var(&cpu_isolated_map);
 	cpulist_parse(str, cpu_isolated_map);
 	return 1;
 }
@@ -9571,7 +9572,9 @@ void __init sched_init(void)
 	zalloc_cpumask_var(&nohz.cpu_mask, GFP_NOWAIT);
 	alloc_cpumask_var(&nohz.ilb_grp_nohz_mask, GFP_NOWAIT);
 #endif
-	zalloc_cpumask_var(&cpu_isolated_map, GFP_NOWAIT);
+	/* May be allocated at isolcpus cmdline parse time */
+	if (cpu_isolated_map == NULL)
+		zalloc_cpumask_var(&cpu_isolated_map, GFP_NOWAIT);
 #endif /* SMP */
 
 	perf_event_init();

[PATCH] sched: Fix isolcpus boot option

[Rusty Russell]

On Mon, 30 Nov 2009 10:35:32 am Anton Blanchard wrote:
> We allocate and zero cpu_isolated_map after the isolcpus __setup option
> has run. This means cpu_isolated_map always ends up empty and if
> CPUMASK_OFFSTACK is enabled we write to a cpumask that hasn't been
> allocated.

I introduced this regression in 49557e620339cb13 (sched: Fix boot
crash by zalloc()ing most of the cpu masks).

Use the bootmem allocator if they set isolcpus=, otherwise allocate
and zero like normal.

Reported-by: Anton Blanchard <anton@samba.org>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Tested-by: Anton Blanchard <anton@samba.org>
Cc: stable@kernel.org

diff --git a/kernel/sched.c b/kernel/sched.c
--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -8045,6 +8045,7 @@ static cpumask_var_t cpu_isolated_map;
 /* Setup the mask of cpus configured for isolated domains */
 static int __init isolated_cpu_setup(char *str)
 {
+	alloc_bootmem_cpumask_var(&cpu_isolated_map);
 	cpulist_parse(str, cpu_isolated_map);
 	return 1;
 }
@@ -9571,7 +9572,9 @@ void __init sched_init(void)
 	zalloc_cpumask_var(&nohz.cpu_mask, GFP_NOWAIT);
 	alloc_cpumask_var(&nohz.ilb_grp_nohz_mask, GFP_NOWAIT);
 #endif
-	zalloc_cpumask_var(&cpu_isolated_map, GFP_NOWAIT);
+	/* May be allocated at isolcpus cmdline parse time */
+	if (cpu_isolated_map == NULL)
+		zalloc_cpumask_var(&cpu_isolated_map, GFP_NOWAIT);
 #endif /* SMP */
 
 	perf_event_init();

[tip:sched/core] sched: Fix isolcpus boot option

[tip-bot for Rusty Russell]

Commit-ID:  bdddd2963c0264c56f18043f6fa829d3c1d3d1c0
Gitweb:     http://git.kernel.org/tip/bdddd2963c0264c56f18043f6fa829d3c1d3d1c0
Author:     Rusty Russell <rusty@rustcorp.com.au>
AuthorDate: Wed, 2 Dec 2009 14:09:16 +1030
Committer:  Ingo Molnar <mingo@elte.hu>
CommitDate: Wed, 2 Dec 2009 10:27:16 +0100

sched: Fix isolcpus boot option

Anton Blanchard wrote:

> We allocate and zero cpu_isolated_map after the isolcpus
> __setup option has run. This means cpu_isolated_map always
> ends up empty and if CPUMASK_OFFSTACK is enabled we write to a
> cpumask that hasn't been allocated.

I introduced this regression in 49557e620339cb13 (sched: Fix
boot crash by zalloc()ing most of the cpu masks).

Use the bootmem allocator if they set isolcpus=, otherwise
allocate and zero like normal.

Reported-by: Anton Blanchard <anton@samba.org>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Cc: peterz@infradead.org
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: <stable@kernel.org>
LKML-Reference: <200912021409.17013.rusty@rustcorp.com.au>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Tested-by: Anton Blanchard <anton@samba.org>
---
 kernel/sched.c |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/kernel/sched.c b/kernel/sched.c
index 1031cae..4883fee 100644
--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -8061,6 +8061,7 @@ static cpumask_var_t cpu_isolated_map;
 /* Setup the mask of cpus configured for isolated domains */
 static int __init isolated_cpu_setup(char *str)
 {
+	alloc_bootmem_cpumask_var(&cpu_isolated_map);
 	cpulist_parse(str, cpu_isolated_map);
 	return 1;
 }
@@ -9609,7 +9610,9 @@ void __init sched_init(void)
 	zalloc_cpumask_var(&nohz.cpu_mask, GFP_NOWAIT);
 	alloc_cpumask_var(&nohz.ilb_grp_nohz_mask, GFP_NOWAIT);
 #endif
-	zalloc_cpumask_var(&cpu_isolated_map, GFP_NOWAIT);
+	/* May be allocated at isolcpus cmdline parse time */
+	if (cpu_isolated_map == NULL)
+		zalloc_cpumask_var(&cpu_isolated_map, GFP_NOWAIT);
 #endif /* SMP */
 
 	perf_event_init();