Build error due to "x86/fpu, sched: Dynamically allocate 'struct fpu'"

Build error due to "x86/fpu, sched: Dynamically allocate 'struct fpu'"

[Guenter]

Hi,

Commit 0c8c0f03e3a2 ("x86/fpu, sched: Dynamically allocate 'struct fpu'")
causes s390 builds in mainline to fail as follows.

arch/s390/kernel/traps.c: Assembler messages:
arch/s390/kernel/traps.c:262: Error: operand out of range
	(0x00000000000023e8 is not between 0x0000000000000000 and 0x0000000000000fff)
arch/s390/kernel/traps.c:300: Error: operand out of range
	(0x00000000000023e8 is not between 0x0000000000000000 and 0x0000000000000fff)

Bisect log is as follows.

# bad: [9d37e6679dfddbb5fa605fb2d7ff448f7cd6d038] Merge branch 'fixes' of git://ftp.arm.linux.org.uk/~rmk/linux-arm
# good: [bc0195aad0daa2ad5b0d76cce22b167bc3435590] Linux 4.2-rc2
git bisect start 'HEAD' 'v4.2-rc2'
# good: [a27507ca2d796cfa8d907de31ad730359c8a6d06] x86/nmi/64: Reorder nested NMI checks
git bisect good a27507ca2d796cfa8d907de31ad730359c8a6d06
# good: [eb254374a30cc53f976f2302f2198813a3b687ea] Merge tag 'staging-4.2-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging
git bisect good eb254374a30cc53f976f2302f2198813a3b687ea
# good: [3a26a5b1513cddfc018c8e264979bc6e28f8ec1f] Merge branch 'akpm' (patches from Andrew)
git bisect good 3a26a5b1513cddfc018c8e264979bc6e28f8ec1f
# good: [f79a17bf268cc043eecffb65033b2e58fc037eef] Merge branch 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect good f79a17bf268cc043eecffb65033b2e58fc037eef
# bad: [5aaeb5c01c5b6c0be7b7aadbf3ace9f3a4458c3d] x86/fpu, sched: Introduce CONFIG_ARCH_WANTS_DYNAMIC_TASK_STRUCT and use it on x86
git bisect bad 5aaeb5c01c5b6c0be7b7aadbf3ace9f3a4458c3d
# good: [36f1a77b3aa57c5c2eb1ae2d67d07c4350a78345] x86/nmi/64: Make the "NMI executing" variable more consistent
git bisect good 36f1a77b3aa57c5c2eb1ae2d67d07c4350a78345
# bad: [0c8c0f03e3a292e031596484275c14cf39c0ab7a] x86/fpu, sched: Dynamically allocate 'struct fpu'
git bisect bad 0c8c0f03e3a292e031596484275c14cf39c0ab7a
# good: [a97439aa1aec10387797b4abae3cf117de1c90d7] x86/entry/64, x86/nmi/64: Add CONFIG_DEBUG_ENTRY NMI testing code
git bisect good a97439aa1aec10387797b4abae3cf117de1c90d7
# first bad commit: [0c8c0f03e3a292e031596484275c14cf39c0ab7a] x86/fpu, sched: Dynamically allocate 'struct fpu'

Guenter

Re: Build error due to "x86/fpu, sched: Dynamically allocate 'struct fpu'"

[Guenter]

On Sat, Jul 18, 2015 at 04:27:17PM -0700, Guenter wrote:
> Hi,
> 
> Commit 0c8c0f03e3a2 ("x86/fpu, sched: Dynamically allocate 'struct fpu'")
> causes s390 builds in mainline to fail as follows.
> 
> arch/s390/kernel/traps.c: Assembler messages:
> arch/s390/kernel/traps.c:262: Error: operand out of range
> 	(0x00000000000023e8 is not between 0x0000000000000000 and 0x0000000000000fff)
> arch/s390/kernel/traps.c:300: Error: operand out of range
> 	(0x00000000000023e8 is not between 0x0000000000000000 and 0x0000000000000fff)
> 

Also:

arm64:allmodconfig:

arch/arm64/kernel/entry.S: Assembler messages:
arch/arm64/kernel/entry.S:588: Error: immediate out of range
arch/arm64/kernel/entry.S:597: Error: immediate out of range
make[1]: *** [arch/arm64/kernel/entry.o] Error 1

I didn't bisect that one, but it looks like the cause is the same.

Guenter

[PATCH] sched, arm64: Fix the fallout of increasing the offset of 'thread_struct' within 'task_struct'

[Ingo Molnar]

* Guenter <linux@roeck-us.net> wrote:

> On Sat, Jul 18, 2015 at 04:27:17PM -0700, Guenter wrote:
> > Hi,
> > 
> > Commit 0c8c0f03e3a2 ("x86/fpu, sched: Dynamically allocate 'struct fpu'")
> > causes s390 builds in mainline to fail as follows.
> > 
> > arch/s390/kernel/traps.c: Assembler messages:
> > arch/s390/kernel/traps.c:262: Error: operand out of range
> > 	(0x00000000000023e8 is not between 0x0000000000000000 and 0x0000000000000fff)
> > arch/s390/kernel/traps.c:300: Error: operand out of range
> > 	(0x00000000000023e8 is not between 0x0000000000000000 and 0x0000000000000fff)
> > 
> 
> Also:
> 
> arm64:allmodconfig:
> 
> arch/arm64/kernel/entry.S: Assembler messages:
> arch/arm64/kernel/entry.S:588: Error: immediate out of range
> arch/arm64/kernel/entry.S:597: Error: immediate out of range
> make[1]: *** [arch/arm64/kernel/entry.o] Error 1
> 
> I didn't bisect that one, but it looks like the cause is the same.

Hm, it looks like the new, increased offset of 'thread_struct' within 
'task_struct' goes over a limit that these instructions are able to support on 
arm64:

 arch/arm64/kernel/asm-offsets.c:  DEFINE(THREAD_CPU_CONTEXT,    offsetof(struct task_struct, thread.cpu_context));
 arch/arm64/kernel/entry.S:      add     x8, x0, #THREAD_CPU_CONTEXT
 arch/arm64/kernel/entry.S:      add     x8, x1, #THREAD_CPU_CONTEXT

If there's no instruction that can support such offset sizes then I suspect the 
straightforward fix would be to pass in thread_struct instead - like the patch 
below. That's a tiny bit cleaner for type encapsulation anyway.

Warning: it's not even build tested, but in case it works:

  Signed-off-by: Ingo Molnar <mingo@kernel.org>

Thanks,

	Ingo

================

 arch/arm64/include/asm/processor.h | 4 ++--
 arch/arm64/kernel/asm-offsets.c    | 2 +-
 arch/arm64/kernel/process.c        | 2 +-
 3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index e4c893e54f01..890f84bb3b8c 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -152,8 +152,8 @@ static inline void cpu_relax(void)
 #define cpu_relax_lowlatency()                cpu_relax()
 
 /* Thread switching */
-extern struct task_struct *cpu_switch_to(struct task_struct *prev,
-					 struct task_struct *next);
+extern struct task_struct *cpu_switch_to(struct thread_struct *prev,
+					 struct thread_struct *next);
 
 #define task_pt_regs(p) \
 	((struct pt_regs *)(THREAD_START_SP + task_stack_page(p)) - 1)
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index c99701a34d7b..3785373c2369 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -39,7 +39,7 @@ int main(void)
   DEFINE(TI_TASK,		offsetof(struct thread_info, task));
   DEFINE(TI_CPU,		offsetof(struct thread_info, cpu));
   BLANK();
-  DEFINE(THREAD_CPU_CONTEXT,	offsetof(struct task_struct, thread.cpu_context));
+  DEFINE(THREAD_CPU_CONTEXT,	offsetof(struct thread_struct, cpu_context));
   BLANK();
   DEFINE(S_X0,			offsetof(struct pt_regs, regs[0]));
   DEFINE(S_X1,			offsetof(struct pt_regs, regs[1]));
diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
index 223b093c9440..436e95bda1b2 100644
--- a/arch/arm64/kernel/process.c
+++ b/arch/arm64/kernel/process.c
@@ -325,7 +325,7 @@ struct task_struct *__switch_to(struct task_struct *prev,
 	dsb(ish);
 
 	/* the actual thread switch */
-	last = cpu_switch_to(prev, next);
+	last = cpu_switch_to(&prev.thread, &next.thread);
 
 	return last;
 }

sched, s390: Fix the fallout of increasing the offset of 'thread_struct' within 'task_struct'

[Ingo Molnar]

* Guenter <linux@roeck-us.net> wrote:

> Hi,
> 
> Commit 0c8c0f03e3a2 ("x86/fpu, sched: Dynamically allocate 'struct fpu'")
> causes s390 builds in mainline to fail as follows.
> 
> arch/s390/kernel/traps.c: Assembler messages:
> arch/s390/kernel/traps.c:262: Error: operand out of range
> 	(0x00000000000023e8 is not between 0x0000000000000000 and 0x0000000000000fff)
> arch/s390/kernel/traps.c:300: Error: operand out of range
> 	(0x00000000000023e8 is not between 0x0000000000000000 and 0x0000000000000fff)


Yeah, so I'm really out on a limb here as I know next to nothing about s390 
assembly, but the build failure appears to be analogous to the arm64 one: the 
offset of thread_struct fields within task_struct increased due to commit 
0c8c0f03e3a2 ("x86/fpu, sched: Dynamically allocate 'struct fpu'"), which 
increased assembly offsets beyond the limit this instruction can apparently 
encode.

Does the (untested!) patch below help?

It's an equivalent transformation on the C side, but it might cause GCC to 
generate different assembly code, because we now have a temporary variable with 
much smaller offsets.

The code is also a tiny bit cleaner this way, as the 'current->thread.fp_regs' 
pattern isn't repeated twice.

In case this works:

   Signed-off-by: Ingo Molnar <mingo@kernel.org>

Thanks,

	Ingo

================>

 arch/s390/kernel/traps.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/arch/s390/kernel/traps.c b/arch/s390/kernel/traps.c
index 4d96c9f53455..db6f0eec55b5 100644
--- a/arch/s390/kernel/traps.c
+++ b/arch/s390/kernel/traps.c
@@ -251,6 +251,7 @@ int alloc_vector_registers(struct task_struct *tsk)
 
 void vector_exception(struct pt_regs *regs)
 {
+	s390_fp_regs *fp_regs = &current->thread.fp_regs;
 	int si_code, vic;
 
 	if (!MACHINE_HAS_VX) {
@@ -259,8 +260,9 @@ void vector_exception(struct pt_regs *regs)
 	}
 
 	/* get vector interrupt code from fpc */
-	asm volatile("stfpc %0" : "=m" (current->thread.fp_regs.fpc));
-	vic = (current->thread.fp_regs.fpc & 0xf00) >> 8;
+	asm volatile("stfpc %0" : "=m" (fp_regs->fpc));
+	vic = (fp_regs->fpc & 0xf00) >> 8;
+
 	switch (vic) {
 	case 1: /* invalid vector operation */
 		si_code = FPE_FLTINV;

Re: [PATCH] sched, arm64: Fix the fallout of increasing the offset of 'thread_struct' within 'task_struct'

[Guenter Roeck]

On 07/20/2015 12:20 AM, Ingo Molnar wrote:
>
> * Guenter <linux@roeck-us.net> wrote:
>
>> On Sat, Jul 18, 2015 at 04:27:17PM -0700, Guenter wrote:
>>> Hi,
>>>
>>> Commit 0c8c0f03e3a2 ("x86/fpu, sched: Dynamically allocate 'struct fpu'")
>>> causes s390 builds in mainline to fail as follows.
>>>
>>> arch/s390/kernel/traps.c: Assembler messages:
>>> arch/s390/kernel/traps.c:262: Error: operand out of range
>>> 	(0x00000000000023e8 is not between 0x0000000000000000 and 0x0000000000000fff)
>>> arch/s390/kernel/traps.c:300: Error: operand out of range
>>> 	(0x00000000000023e8 is not between 0x0000000000000000 and 0x0000000000000fff)
>>>
>>
>> Also:
>>
>> arm64:allmodconfig:
>>
>> arch/arm64/kernel/entry.S: Assembler messages:
>> arch/arm64/kernel/entry.S:588: Error: immediate out of range
>> arch/arm64/kernel/entry.S:597: Error: immediate out of range
>> make[1]: *** [arch/arm64/kernel/entry.o] Error 1
>>
>> I didn't bisect that one, but it looks like the cause is the same.
>
> Hm, it looks like the new, increased offset of 'thread_struct' within
> 'task_struct' goes over a limit that these instructions are able to support on
> arm64:
>
>   arch/arm64/kernel/asm-offsets.c:  DEFINE(THREAD_CPU_CONTEXT,    offsetof(struct task_struct, thread.cpu_context));
>   arch/arm64/kernel/entry.S:      add     x8, x0, #THREAD_CPU_CONTEXT
>   arch/arm64/kernel/entry.S:      add     x8, x1, #THREAD_CPU_CONTEXT
>
> If there's no instruction that can support such offset sizes then I suspect the
> straightforward fix would be to pass in thread_struct instead - like the patch
> below. That's a tiny bit cleaner for type encapsulation anyway.
>

Olof submitted a different patch to solve the problem:
	http://www.spinics.net/lists/kernel/msg2036825.html

His patch is passing cpu_context instead of thread_context.

> Warning: it's not even build tested, but in case it works:
>
>    Signed-off-by: Ingo Molnar <mingo@kernel.org>
>
> Thanks,
>
> 	Ingo
>
> ================
>
>   arch/arm64/include/asm/processor.h | 4 ++--
>   arch/arm64/kernel/asm-offsets.c    | 2 +-
>   arch/arm64/kernel/process.c        | 2 +-
>   3 files changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
> index e4c893e54f01..890f84bb3b8c 100644
> --- a/arch/arm64/include/asm/processor.h
> +++ b/arch/arm64/include/asm/processor.h
> @@ -152,8 +152,8 @@ static inline void cpu_relax(void)
>   #define cpu_relax_lowlatency()                cpu_relax()
>
>   /* Thread switching */
> -extern struct task_struct *cpu_switch_to(struct task_struct *prev,
> -					 struct task_struct *next);
> +extern struct task_struct *cpu_switch_to(struct thread_struct *prev,
> +					 struct thread_struct *next);
>
>   #define task_pt_regs(p) \
>   	((struct pt_regs *)(THREAD_START_SP + task_stack_page(p)) - 1)
> diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
> index c99701a34d7b..3785373c2369 100644
> --- a/arch/arm64/kernel/asm-offsets.c
> +++ b/arch/arm64/kernel/asm-offsets.c
> @@ -39,7 +39,7 @@ int main(void)
>     DEFINE(TI_TASK,		offsetof(struct thread_info, task));
>     DEFINE(TI_CPU,		offsetof(struct thread_info, cpu));
>     BLANK();
> -  DEFINE(THREAD_CPU_CONTEXT,	offsetof(struct task_struct, thread.cpu_context));
> +  DEFINE(THREAD_CPU_CONTEXT,	offsetof(struct thread_struct, cpu_context));
>     BLANK();
>     DEFINE(S_X0,			offsetof(struct pt_regs, regs[0]));
>     DEFINE(S_X1,			offsetof(struct pt_regs, regs[1]));
> diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
> index 223b093c9440..436e95bda1b2 100644
> --- a/arch/arm64/kernel/process.c
> +++ b/arch/arm64/kernel/process.c
> @@ -325,7 +325,7 @@ struct task_struct *__switch_to(struct task_struct *prev,
>   	dsb(ish);
>
>   	/* the actual thread switch */
> -	last = cpu_switch_to(prev, next);
> +	last = cpu_switch_to(&prev.thread, &next.thread);

Doesn't compile.

arch/arm64/kernel/process.c: In function ‘__switch_to’:
arch/arm64/kernel/process.c:328:28: error: request for member ‘thread’ in something not a structure or union
   last = cpu_switch_to(&prev.thread, &next.thread);
                             ^
arch/arm64/kernel/process.c:328:42: error: request for member ‘thread’ in something not a structure or union
   last = cpu_switch_to(&prev.thread, &next.thread);

It would have to be
	last = cpu_switch_to(&prev->thread, &next->thread);

which does compile, but fails to run in qemu.

Guenter

Re: [PATCH] sched, arm64: Fix the fallout of increasing the offset of 'thread_struct' within 'task_struct'

[Catalin Marinas]

On Mon, Jul 20, 2015 at 06:52:28AM -0700, Guenter Roeck wrote:
> On 07/20/2015 12:20 AM, Ingo Molnar wrote:
> >* Guenter <linux@roeck-us.net> wrote:
> >>On Sat, Jul 18, 2015 at 04:27:17PM -0700, Guenter wrote:
> >>>Commit 0c8c0f03e3a2 ("x86/fpu, sched: Dynamically allocate 'struct fpu'")
> >>>causes s390 builds in mainline to fail as follows.
> >>>
> >>>arch/s390/kernel/traps.c: Assembler messages:
> >>>arch/s390/kernel/traps.c:262: Error: operand out of range
> >>>	(0x00000000000023e8 is not between 0x0000000000000000 and 0x0000000000000fff)
> >>>arch/s390/kernel/traps.c:300: Error: operand out of range
> >>>	(0x00000000000023e8 is not between 0x0000000000000000 and 0x0000000000000fff)
> >>>
> >>
> >>Also:
> >>
> >>arm64:allmodconfig:
> >>
> >>arch/arm64/kernel/entry.S: Assembler messages:
> >>arch/arm64/kernel/entry.S:588: Error: immediate out of range
> >>arch/arm64/kernel/entry.S:597: Error: immediate out of range
> >>make[1]: *** [arch/arm64/kernel/entry.o] Error 1
> >>
> >>I didn't bisect that one, but it looks like the cause is the same.
> >
> >Hm, it looks like the new, increased offset of 'thread_struct' within
> >'task_struct' goes over a limit that these instructions are able to support on
> >arm64:
> >
> >  arch/arm64/kernel/asm-offsets.c:  DEFINE(THREAD_CPU_CONTEXT,    offsetof(struct task_struct, thread.cpu_context));
> >  arch/arm64/kernel/entry.S:      add     x8, x0, #THREAD_CPU_CONTEXT
> >  arch/arm64/kernel/entry.S:      add     x8, x1, #THREAD_CPU_CONTEXT
> >
> >If there's no instruction that can support such offset sizes then I suspect the
> >straightforward fix would be to pass in thread_struct instead - like the patch
> >below. That's a tiny bit cleaner for type encapsulation anyway.
> 
> Olof submitted a different patch to solve the problem:
> 	http://www.spinics.net/lists/kernel/msg2036825.html
> 
> His patch is passing cpu_context instead of thread_context.

It's likely that we'll use this patch instead:

http://lkml.kernel.org/g/20150720105345.GC9908@arm.com

-- 
Catalin

Re: sched, s390: Fix the fallout of increasing the offset of 'thread_struct' within 'task_struct'

[Guenter Roeck]

On 07/20/2015 12:34 AM, Ingo Molnar wrote:
>
> * Guenter <linux@roeck-us.net> wrote:
>
>> Hi,
>>
>> Commit 0c8c0f03e3a2 ("x86/fpu, sched: Dynamically allocate 'struct fpu'")
>> causes s390 builds in mainline to fail as follows.
>>
>> arch/s390/kernel/traps.c: Assembler messages:
>> arch/s390/kernel/traps.c:262: Error: operand out of range
>> 	(0x00000000000023e8 is not between 0x0000000000000000 and 0x0000000000000fff)
>> arch/s390/kernel/traps.c:300: Error: operand out of range
>> 	(0x00000000000023e8 is not between 0x0000000000000000 and 0x0000000000000fff)
>
>
> Yeah, so I'm really out on a limb here as I know next to nothing about s390
> assembly, but the build failure appears to be analogous to the arm64 one: the
> offset of thread_struct fields within task_struct increased due to commit
> 0c8c0f03e3a2 ("x86/fpu, sched: Dynamically allocate 'struct fpu'"), which
> increased assembly offsets beyond the limit this instruction can apparently
> encode.
>
> Does the (untested!) patch below help?
>
> It's an equivalent transformation on the C side, but it might cause GCC to
> generate different assembly code, because we now have a temporary variable with
> much smaller offsets.
>
> The code is also a tiny bit cleaner this way, as the 'current->thread.fp_regs'
> pattern isn't repeated twice.
>
> In case this works:
>
>     Signed-off-by: Ingo Molnar <mingo@kernel.org>
>
> Thanks,
>
> 	Ingo
>
> ================>
>
>   arch/s390/kernel/traps.c | 6 ++++--
>   1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/arch/s390/kernel/traps.c b/arch/s390/kernel/traps.c
> index 4d96c9f53455..db6f0eec55b5 100644
> --- a/arch/s390/kernel/traps.c
> +++ b/arch/s390/kernel/traps.c
> @@ -251,6 +251,7 @@ int alloc_vector_registers(struct task_struct *tsk)
>
>   void vector_exception(struct pt_regs *regs)
>   {
> +	s390_fp_regs *fp_regs = &current->thread.fp_regs;
>   	int si_code, vic;
>
>   	if (!MACHINE_HAS_VX) {
> @@ -259,8 +260,9 @@ void vector_exception(struct pt_regs *regs)
>   	}
>
>   	/* get vector interrupt code from fpc */
> -	asm volatile("stfpc %0" : "=m" (current->thread.fp_regs.fpc));
> -	vic = (current->thread.fp_regs.fpc & 0xf00) >> 8;
> +	asm volatile("stfpc %0" : "=m" (fp_regs->fpc));
> +	vic = (fp_regs->fpc & 0xf00) >> 8;
> +

No idea why, but this still fails with the same error (I suspect the compiler
tries to optimize the fp_regs variable away). I can compile the code by using
a local variable '__u32 fpc', but obviously I don't know if that is correct.
I don't have a working qemu configuration for s390, so I can not run any tests.

Guenter