i387/FPU init issues...

i387/FPU init issues...

[jamal]

Peoplez,

Ive narrowed down a problem i am having with an old P2 to commit
61c4628b538608c1a85211ed8438136adfeb9a95 with subject "x86, fpu: split
FPU state from task struct - v5" (Authored by Suresh and committed by
Ingo on Apr/19).

In the process i learnt how painfully time consuming and boring a blind
git bisect feast could be (the last time a kernel worked on the P2 was
back in 2.6.23). I literally spent no less than 10 hours tracking this
(Ok, I was chewing tobbaco in between running git bisect bad/good,
compile, copy over kernel, spit here, reboot, test).
Also this patch is so huge that given my lack of knowledge in the area,
i couldnt do better bisecting to be more exact on what is causing this.
i.e the patch is not bisect-friendly. 
So the best i can do is have other people take it from here.

I am able to reproduce the issue consistently on my laptop using qemu
(which helped speed debugging a bit). I have also narrowed it down to
include/asm-x86/i387.h::__save_init_fpu in (32 bit version) - it dies
somewhere in calling the following line:

----
        alternative_input(
                "fnsave %[fx] ;fwait;" GENERIC_NOP8 GENERIC_NOP4,
                "fxsave %[fx]\n"
                "bt $7,%[fsw] ; jnc 1f ; fnclex\n1:",
                X86_FEATURE_FXSR,
                [fx] "m" (tsk->thread.xstate->fxsave),
                [fsw] "m" (tsk->thread.xstate->fxsave.swd) : "memory");
----------

The only thing that has changed there compared to good version is the
last two lines. But that looks sane to me given the struct naming has
changed. So i am suspecting the calling path perhaps not setting
something or other.

------------ boot output paste ----------------------
[....]
Compat vDSO mapped to ffffe000.
CPU: Intel Pentium II (Klamath) stepping 03
Checking 'hlt' instruction... OK.
Freeing SMP alternatives: 0k freed
invalid opcode: 0000 [#1]
Modules linked in:

Pid: 0, comm: swapper Not tainted (2.6.25-00000-g61c4628 #22)
EIP: 0060:[<c01012d0>] EFLAGS: 00000202 CPU: 0
EIP is at prepare_to_copy+0x20/0x50
EAX: c1101880 EBX: fffffff4 ECX: c04eff80 EDX: c04bb3e0
ESI: c04bb3e0 EDI: c04eff80 EBP: c04efeb0 ESP: c04efeb0
 DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068
Process swapper (pid: 0, ti=c04ee000 task=c04bb3e0 task.ti=c04ee000)
Stack: c04eff0c c01183a0 00000000 c0543566 00000000 c04eff84 00000296
c04effa4
       c04eff80 00000000 00800b00 00000001 c04eff5c 00000296 c0543565
c0543544
       00000026 c04effb4 00000296 c04effd4 00000000 00800b00 c04eff80
c04eff64
Call Trace:
 [<c01183a0>] ? copy_process+0x60/0x10d0
 [<c0119504>] ? do_fork+0x54/0x210
 [<c01355cc>] ? lock_release_holdtime+0x6c/0x70
 [<c04f0000>] ? __init_begin+0x0/0x69
 [<c010fe5d>] ? change_page_attr_set_clr+0xcd/0x1e0
 [<c0101996>] ? kernel_thread+0x86/0xa0
 [<c04f0710>] ? kernel_init+0x0/0x270
 [<c04f0710>] ? kernel_init+0x0/0x270
 [<c0103260>] ? kernel_thread_helper+0x0/0x10
 [<c03bb874>] ? rest_init+0x14/0x50
 [<c04f0b7a>] ? start_kernel+0x1fa/0x280
 [<c04f03f0>] ? unknown_bootoption+0x0/0x210
 [<c04f02b8>] ? i386_start_kernel+0x8/0x10
 =======================
Code: 8d 74 26 00 8d bc 27 00 00 00 00 55 89 c2 8b 40 04 89 e5 f6 40 0c
01 74 32 8b 82 60 02 00 00 0f ae 00 0f ba 60 02 07 73 02 db e2 <0f> 1f
00 90 8d b4 26 00 00 00 00 89 f6 8b 42 04 83 60 0c fe 0f
EIP: [<c01012d0>] prepare_to_copy+0x20/0x50 SS:ESP 0068:c04efeb0
---[ end trace ca143223eefdc828 ]---
Kernel panic - not syncing: Attempted to kill the idle task!
------------------------------------------------------------------------

Remedy:
I am able to get the system to boot fine if i passed an option to tell
it there is no i387 and compile in math emulation (but then a few of
standard programs start segfaulting on me and i dont wanna go chasing
that).

Let me know what you want me to try out since i can do this on my laptop
now.

cheers,
jamal

Re: i387/FPU init issues...

[James Courtier-Dutton]

jamal wrote:
> Peoplez,
>
> Ive narrowed down a problem i am having with an old P2 to commit
> 61c4628b538608c1a85211ed8438136adfeb9a95 with subject "x86, fpu: split
> FPU state from task struct - v5" (Authored by Suresh and committed by
> Ingo on Apr/19).
>
> In the process i learnt how painfully time consuming and boring a blind
> git bisect feast could be (the last time a kernel worked on the P2 was
> back in 2.6.23). I literally spent no less than 10 hours tracking this
> (Ok, I was chewing tobbaco in between running git bisect bad/good,
> compile, copy over kernel, spit here, reboot, test).
> Also this patch is so huge that given my lack of knowledge in the area,
> i couldnt do better bisecting to be more exact on what is causing this.
> i.e the patch is not bisect-friendly. 
> So the best i can do is have other people take it from here.
>
> I am able to reproduce the issue consistently on my laptop using qemu
> (which helped speed debugging a bit). I have also narrowed it down to
> include/asm-x86/i387.h::__save_init_fpu in (32 bit version) - it dies
> somewhere in calling the following line:
>
> ----
>         alternative_input(
>                 "fnsave %[fx] ;fwait;" GENERIC_NOP8 GENERIC_NOP4,
>                 "fxsave %[fx]\n"
>                 "bt $7,%[fsw] ; jnc 1f ; fnclex\n1:",
>                 X86_FEATURE_FXSR,
>                 [fx] "m" (tsk->thread.xstate->fxsave),
>                 [fsw] "m" (tsk->thread.xstate->fxsave.swd) : "memory");
> ----------
>
> The only thing that has changed there compared to good version is the
> last two lines. But that looks sane to me given the struct naming has
> changed. So i am suspecting the calling path perhaps not setting
> something or other.
>
>   
My guess would be that the jnc 1f is now wrong.

Re: i387/FPU init issues...

[jamal]

On Sat, 2008-03-05 at 11:57 +0100, James Courtier-Dutton wrote:

> > The only thing that has changed there compared to good version is the
> > last two lines. But that looks sane to me given the struct naming has
> > changed. So i am suspecting the calling path perhaps not setting
> > something or other.
> >
> >   
> My guess would be that the jnc 1f is now wrong.

Do elucidate. The instruction with jnc 1f has always been there.
Is there something as a result of the compiler or the codepath that
makes it now wrong?

cheers,
jamal

Re: i387/FPU init issues...

[Thomas Gleixner]

On Sat, 3 May 2008, jamal wrote:
>  =======================
> Code: 8d 74 26 00 8d bc 27 00 00 00 00 55 89 c2 8b 40 04 89 e5 f6 40 0c
> 01 74 32 8b 82 60 02 00 00 0f ae 00 0f ba 60 02 07 73 02 db e2 <0f> 1f
> 00 90 8d b4 26 00 00 00 00 89 f6 8b 42 04 83 60 0c fe 0f

This looks bad.

  0f ae 00		fxsave (%eax)
  0f ba 60 02 07       	btl    $0x7,0x2(%eax)
  73 02                	jae    (skip fnclex)
  db e2                	fnclex 
  0f 1f 00             	nopl   (%eax)

 ^^^^ This is a P4+ instruction. So it's not surprising that the P2
 chokes. The question is where this comes from.

we have:
#define P6_NOP3 ".byte 0x0f,0x1f,0x00\n"

So the alternatives code applies the wrong nop padding for your
CPU. This was probably introduced with commit
32c464f5d9701db45bc1673288594e664065388e. 

Jan, are you sure that P3 knows the P6 NOPs ? AFAICT its P4, but I
have to dig up the manuals.

Jamal, does the following patch solve your problem ? Please provide
also output of /proc/cpuinfo.

Thanks,
	tglx
---

--- linux-2.6.orig/arch/x86/kernel/alternative.c
+++ linux-2.6/arch/x86/kernel/alternative.c
@@ -158,7 +158,6 @@ static const struct nop {
 	{ X86_FEATURE_K8, k8_nops },
 	{ X86_FEATURE_K7, k7_nops },
 	{ X86_FEATURE_P4, p6_nops },
-	{ X86_FEATURE_P3, p6_nops },
 	{ -1, NULL }
 };
 

Re: i387/FPU init issues...

[jamal]

On Sat, 2008-03-05 at 17:31 +0200, Thomas Gleixner wrote:
> On Sat, 3 May 2008, jamal wrote:
> >  =======================
> > Code: 8d 74 26 00 8d bc 27 00 00 00 00 55 89 c2 8b 40 04 89 e5 f6 40 0c
> > 01 74 32 8b 82 60 02 00 00 0f ae 00 0f ba 60 02 07 73 02 db e2 <0f> 1f
> > 00 90 8d b4 26 00 00 00 00 89 f6 8b 42 04 83 60 0c fe 0f
> 
> This looks bad.
> 
>   0f ae 00		fxsave (%eax)
>   0f ba 60 02 07       	btl    $0x7,0x2(%eax)
>   73 02                	jae    (skip fnclex)
>   db e2                	fnclex 
>   0f 1f 00             	nopl   (%eax)
> 
>  ^^^^ This is a P4+ instruction. So it's not surprising that the P2
>  chokes. The question is where this comes from.
> 
> we have:
> #define P6_NOP3 ".byte 0x0f,0x1f,0x00\n"

Dang - I feel i should have saved myself all that git bisecting
and just posted the oops ;->

> So the alternatives code applies the wrong nop padding for your
> CPU. This was probably introduced with commit
> 32c464f5d9701db45bc1673288594e664065388e. 
> 
> Jan, are you sure that P3 knows the P6 NOPs ? AFAICT its P4, but I
> have to dig up the manuals.
> 
> Jamal, does the following patch solve your problem ? 


Indeed it does - thanks.

> Please provide
> also output of /proc/cpuinfo.

mambo:~# cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 3
model name      : Pentium II (Klamath)
stepping        : 3
cpu MHz         : 1063.771
cache size      : 128 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 2
wp              : yes
flags           : fpu de pse tsc msr pae mce cx8 sep pge cmov mmx fxsr
sse sse2
bogomips        : 2160.92
clflush size    : 32
power management:

cheers,
jamal

Re: i387/FPU init issues...

[Ingo Molnar]

* jamal <hadi@cyberus.ca> wrote:

> > we have:
> > #define P6_NOP3 ".byte 0x0f,0x1f,0x00\n"
> 
> Dang - I feel i should have saved myself all that git bisecting
> and just posted the oops ;->

your bisection was still very useful - it pinpointed the NOPs - the 
assembly code around the NOP changed so a different length NOP was 
patched in => which did not work on your CPU. So thanks for that! The 
fix is already in x86.git (we'll update the commit log).

> > So the alternatives code applies the wrong nop padding for your CPU. 
> > This was probably introduced with commit 
> > 32c464f5d9701db45bc1673288594e664065388e.
> > 
> > Jan, are you sure that P3 knows the P6 NOPs ? AFAICT its P4, but I 
> > have to dig up the manuals.
> > 
> > Jamal, does the following patch solve your problem ?
> 
> Indeed it does - thanks.

great. So this NOP is indeed not generally known to all "P6 and later" 
CPUs. (the PII)

	Ingo

Re: i387/FPU init issues...

[Thomas Gleixner]

On Sat, 3 May 2008, Ingo Molnar wrote:
> * jamal <hadi@cyberus.ca> wrote:
> 
> > > we have:
> > > #define P6_NOP3 ".byte 0x0f,0x1f,0x00\n"
> > 
> > Dang - I feel i should have saved myself all that git bisecting
> > and just posted the oops ;->
> 
> your bisection was still very useful - it pinpointed the NOPs - the 
> assembly code around the NOP changed so a different length NOP was 
> patched in => which did not work on your CPU. So thanks for that! The 
> fix is already in x86.git (we'll update the commit log).
> 
> > > So the alternatives code applies the wrong nop padding for your CPU. 
> > > This was probably introduced with commit 
> > > 32c464f5d9701db45bc1673288594e664065388e.
> > > 
> > > Jan, are you sure that P3 knows the P6 NOPs ? AFAICT its P4, but I 
> > > have to dig up the manuals.
> > > 
> > > Jamal, does the following patch solve your problem ?
> > 
> > Indeed it does - thanks.
> 
> great. So this NOP is indeed not generally known to all "P6 and later" 
> CPUs. (the PII)

Looks like. My analysis was wrong, as I got the P6 vs. PII/PIII
confused :) Damn unintutive numbering, I thought ARM is worse but I'm
not so sure anymore.

But the oops clearly identified that instruction sequence. So for now
we remove the X86_FEATURE_P3 -> P6_NOPS to be on the safe side.

Thanks,
	tglx

Re: i387/FPU init issues...

[Jan Engelhardt]

On Saturday 2008-05-03 19:39, Thomas Gleixner wrote:
>> great. So this NOP is indeed not generally known to all "P6 and later" 
>> CPUs. (the PII)
>
>Looks like. My analysis was wrong, as I got the P6 vs. PII/PIII
>confused :) Damn unintutive numbering, I thought ARM is worse but I'm
>not so sure anymore.

Guess that Intel named it Pentium II either because Hexium
("5"86:Pentium, "6"86:Hexium) would have been a strange name, or the
successor to the Pentium/586 was not that great an improvement.
Or something else? Always kept me wondering.

Re: i387/FPU init issues...

[H. Peter Anvin]

Jan Engelhardt wrote:
> On Saturday 2008-05-03 19:39, Thomas Gleixner wrote:
>>> great. So this NOP is indeed not generally known to all "P6 and later" 
>>> CPUs. (the PII)
>> Looks like. My analysis was wrong, as I got the P6 vs. PII/PIII
>> confused :) Damn unintutive numbering, I thought ARM is worse but I'm
>> not so sure anymore.
> 
> Guess that Intel named it Pentium II either because Hexium
> ("5"86:Pentium, "6"86:Hexium) would have been a strange name, or the
> successor to the Pentium/586 was not that great an improvement.
> Or something else? Always kept me wondering.

Yeah, "Hexium" didn't quite work, and they thought they'd already gotten 
a working brand with "Pentium".  That it clashed with their previous 
public prerelease naming scheme of P+number ("P", I believe, for 
"project" or "processor") didn't matter.

The Pentium 4 is properly called the P7, but almost noone calls it that.

"Pentium" is also a highly unstable isotope of hydrogen (Hydrogen-5), 
with a half-life under a zeptosecond.

	-hpa

Re: i387/FPU init issues...

[Lennart Sorensen]

On Sun, May 04, 2008 at 11:31:00PM +0200, Jan Engelhardt wrote:
> Guess that Intel named it Pentium II either because Hexium
> ("5"86:Pentium, "6"86:Hexium) would have been a strange name, or the
> successor to the Pentium/586 was not that great an improvement.
> Or something else? Always kept me wondering.

Well the brand was well recognized, so they called the P6 "Pentium Pro".
Then when they went to make a cheaper version for regular consumers,
they made the Pentium II based on the PPro, by making the L2 cache half
speed (the PPro ran L2 cache at full speed) and put the L2 cache on a
PCB with the core in a cartridge.  After a while they added SSE and
called it the Pentium !!! (check the logo, they really used exclamation
marks) and a while later they managed to shrink their process enough
that they could integrate the L2 cache on die, so they made the cache
full speed again but on the same die as the cpu core, and soon after
started offering plain CPUs again rather than cartridges.  The Pentium 4
is of course a totally unrelated design.  The Pentium M followed up on
the P3 but aiming for lower power consumption rather than maximum
performance, and later evolved into the Core and then Core 2 (where
again they focused on maximum performance).  I think the Pentium 4 may
have tarnished the Pentium brand enough that they needed a new name, and
hence "Core" came about.

-- 
Len Sorensen

Re: i387/FPU init issues...

[H. Peter Anvin]

[-- Attachment #1: Type: text/plain, Size: 970 bytes --]

jamal wrote:
> 
> Indeed it does - thanks.
> 
>> Please provide
>> also output of /proc/cpuinfo.
> 
> mambo:~# cat /proc/cpuinfo
> processor       : 0
> vendor_id       : GenuineIntel
> cpu family      : 6
> model           : 3
> model name      : Pentium II (Klamath)
> stepping        : 3
> cpu MHz         : 1063.771
> cache size      : 128 KB
> fdiv_bug        : no
> hlt_bug         : no
> f00f_bug        : no
> coma_bug        : no
> fpu             : yes
> fpu_exception   : yes
> cpuid level     : 2
> wp              : yes
> flags           : fpu de pse tsc msr pae mce cx8 sep pge cmov mmx fxsr
> sse sse2
> bogomips        : 2160.92
> clflush size    : 32
> power management:
> 

This is very odd.

Could you try running the attached C program on this processor and 
report the result?  (Binary included for convenience.)

Arjan: this seems to directly contradict the Intel documentation.  Do 
you have any way to find out what the deal is with this?

	-hpa

[-- Attachment #2: p6nops.c --]
[-- Type: text/x-csrc, Size: 1609 bytes --]

#include <setjmp.h>
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

#define P6_NOP1 ".byte 0x90\n"
#define P6_NOP2 ".byte 0x66,0x90\n"
#define P6_NOP3 ".byte 0x0f,0x1f,0x00\n"
#define P6_NOP4 ".byte 0x0f,0x1f,0x40,0\n"
#define P6_NOP5 ".byte 0x0f,0x1f,0x44,0x00,0\n"
#define P6_NOP6 ".byte 0x66,0x0f,0x1f,0x44,0x00,0\n"
#define P6_NOP7 ".byte 0x0f,0x1f,0x80,0,0,0,0\n"
#define P6_NOP8 ".byte 0x0f,0x1f,0x84,0x00,0,0,0,0\n"

static sigjmp_buf bail_buf;

static void sigill(int sig)
{
	(void)sig;
	siglongjmp(bail_buf, 1);
}

static int do_test(int n)
{
	struct sigaction old_sa, sa;
	int err;

	memset(&sa, 0, sizeof sa);
	sa.sa_handler = sigill;
	sigaction(SIGILL, &sa, &old_sa);

	err = sigsetjmp(bail_buf, 1);
	if (!err) {
		switch (n) {
		case -1:
			asm volatile("ud2a"); /* Test the mechanism */
			break;
		case 0:
			asm volatile("");
			break;
		case 1:
			asm volatile(P6_NOP1);
			break;
		case 2:
			asm volatile(P6_NOP2);
			break;
		case 3:
			asm volatile(P6_NOP3);
			break;
		case 4:
			asm volatile(P6_NOP4);
			break;
		case 5:
			asm volatile(P6_NOP5);
			break;
		case 6:
			asm volatile(P6_NOP6);
			break;
		case 7:
			asm volatile(P6_NOP7);
			break;
		case 8:
			asm volatile(P6_NOP8);
			break;
		default:
			abort();
		}
	}

	sigaction(SIGILL, &old_sa, NULL);
	return err;
}

int main(void)
{
	int i;
	int test, err = 0;

	if (!do_test(-1)) {
		printf("Trap mechanism broken!\n");
		return 2;
	}

	for (i = 0; i <= 8; i++) {
		test = do_test(i);
		err |= test;
		printf("Test %d: %s\n", i, test ? "err" : "ok");
	}

	return err;
}

[-- Attachment #3: p6nops --]
[-- Type: application/octet-stream, Size: 10555 bytes --]

Re: i387/FPU init issues...

[Mikael Pettersson]

H. Peter Anvin writes:
 > jamal wrote:
 > > 
 > > Indeed it does - thanks.
 > > 
 > >> Please provide
 > >> also output of /proc/cpuinfo.
 > > 
 > > mambo:~# cat /proc/cpuinfo
 > > processor       : 0
 > > vendor_id       : GenuineIntel
 > > cpu family      : 6
 > > model           : 3
 > > model name      : Pentium II (Klamath)
 > > stepping        : 3
 > > cpu MHz         : 1063.771
 > > cache size      : 128 KB
 > > fdiv_bug        : no
 > > hlt_bug         : no
 > > f00f_bug        : no
 > > coma_bug        : no
 > > fpu             : yes
 > > fpu_exception   : yes
 > > cpuid level     : 2
 > > wp              : yes
 > > flags           : fpu de pse tsc msr pae mce cx8 sep pge cmov mmx fxsr
 > > sse sse2
 > > bogomips        : 2160.92
 > > clflush size    : 32
 > > power management:
 > > 
 > 
 > This is very odd.
 > 
 > Could you try running the attached C program on this processor and 
 > report the result?  (Binary included for convenience.)
 > 
 > Arjan: this seems to directly contradict the Intel documentation.  Do 
 > you have any way to find out what the deal is with this?

hpa's p6nops test program works fine here on both a PII (family 6 model 3)
and a Pentium Pro (family 6 model 1 stepping 9).

I'm noticing another anomaly in jamal's /proc/cpuinfo above: since when
can a PII have sse and sse2? As far as I recall, it was the PIII that
added sse, and sse2 came with the P4.

/Mikael

Re: i387/FPU init issues...

[H. Peter Anvin]

jamal wrote:
> 
> mambo:~# cat /proc/cpuinfo
> processor       : 0
> vendor_id       : GenuineIntel
> cpu family      : 6
> model           : 3
> model name      : Pentium II (Klamath)
> stepping        : 3
> cpu MHz         : 1063.771
> cache size      : 128 KB
> fdiv_bug        : no
> hlt_bug         : no
> f00f_bug        : no
> coma_bug        : no
> fpu             : yes
> fpu_exception   : yes
> cpuid level     : 2
> wp              : yes
> flags           : fpu de pse tsc msr pae mce cx8 sep pge cmov mmx fxsr
> sse sse2
> bogomips        : 2160.92
> clflush size    : 32
> power management:
> 

Jamal, is this the /proc/cpuinfo from Qemu or from your hardware platform?

	-hpa

Re: i387/FPU init issues...

[H. Peter Anvin]

Thomas Gleixner wrote:
> 
> Jan, are you sure that P3 knows the P6 NOPs ? AFAICT its P4, but I
> have to dig up the manuals.
> 

Pentium III is the P6 core, so it will.

Intel explicitly documents "all processors with family 6 or F."

	-hpa

Re: i387/FPU init issues...

[James Courtier-Dutton]

H. Peter Anvin wrote:
> Thomas Gleixner wrote:
>>
>> Jan, are you sure that P3 knows the P6 NOPs ? AFAICT its P4, but I
>> have to dig up the manuals.
>>
>
> Pentium III is the P6 core, so it will.
>
> Intel explicitly documents "all processors with family 6 or F."
>
> -hpa

 From the intel manual

0F 1F /0 NOP
The multi-byte form of NOP is available on processors with model encoding:
• CPUID.01H.EAX[Bytes 11:8] = 0110B or 1111B
The multi-byte NOP instruction does not alter the content of a register 
and will not
issue a memory operation. The instruction’s operation is the same in 
non-64-bit
modes and 64-bit mode.

Re: i387/FPU init issues...

[H. Peter Anvin]

James Courtier-Dutton wrote:
> H. Peter Anvin wrote:
>> Thomas Gleixner wrote:
>>>
>>> Jan, are you sure that P3 knows the P6 NOPs ? AFAICT its P4, but I
>>> have to dig up the manuals.
>>>
>>
>> Pentium III is the P6 core, so it will.
>>
>> Intel explicitly documents "all processors with family 6 or F."
>>
>> -hpa
> 
>  From the intel manual
> 
> 0F 1F /0 NOP
> The multi-byte form of NOP is available on processors with model encoding:
> • CPUID.01H.EAX[Bytes 11:8] = 0110B or 1111B
> The multi-byte NOP instruction does not alter the content of a register 
> and will not
> issue a memory operation. The instruction’s operation is the same in 
> non-64-bit
> modes and 64-bit mode.
> 

I believe that's what I just said...

	-hpa

Re: i387/FPU init issues...

[Thomas Gleixner]

On Sat, 3 May 2008, H. Peter Anvin wrote:

> Thomas Gleixner wrote:
> > 
> > Jan, are you sure that P3 knows the P6 NOPs ? AFAICT its P4, but I
> > have to dig up the manuals.
> > 
> 
> Pentium III is the P6 core, so it will.
> 
> Intel explicitly documents "all processors with family 6 or F."

Yeah, I looked up myself and noticed that I confused those stupid
numbers again.

Nevertheless reality seems to tell a different story :)

Thanks,

	tglx

Re: i387/FPU init issues...

[Mikael Pettersson]

Thomas Gleixner writes:
 > On Sat, 3 May 2008, H. Peter Anvin wrote:
 > 
 > > Thomas Gleixner wrote:
 > > > 
 > > > Jan, are you sure that P3 knows the P6 NOPs ? AFAICT its P4, but I
 > > > have to dig up the manuals.
 > > > 
 > > 
 > > Pentium III is the P6 core, so it will.
 > > 
 > > Intel explicitly documents "all processors with family 6 or F."
 > 
 > Yeah, I looked up myself and noticed that I confused those stupid
 > numbers again.
 > 
 > Nevertheless reality seems to tell a different story :)

I've tested the "0f 1f 00" 3-byte NOP on two PIIs,
a Klamath (family 6 model 3 stepping 4) and a Deschutes
(family 6 model 5 stepping 0), and it worked fine on both.

Are you absolutely sure it's this 3-byte NOP that oopses?

I also noticed that jamal's /proc/cpuinfo contains
> cpu MHz         : 1063.771
which is either wrong or indicates serious overclocking.
The PIIs maxed out at about 400/450MHz.

/Mikael

Re: i387/FPU init issues...

[H. Peter Anvin]

Mikael Pettersson wrote:
> 
> Are you absolutely sure it's this 3-byte NOP that oopses?
> 
> I also noticed that jamal's /proc/cpuinfo contains
>> cpu MHz         : 1063.771
> which is either wrong or indicates serious overclocking.
> The PIIs maxed out at about 400/450MHz.
> 

Good spotting, and yes, the roof for Pentium II was 450 MHz.  Pentium II 
had a TSC, so it shouldn't be wrong unless there wasn't a bigger 
timekeeping error on this platform.

Jamal, can you clarify?

	-hpa

Re: i387/FPU init issues...

[H. Peter Anvin]

Mikael Pettersson wrote:
> 
> I also noticed that jamal's /proc/cpuinfo contains
>> cpu MHz         : 1063.771
> which is either wrong or indicates serious overclocking.
> The PIIs maxed out at about 400/450MHz.
> 

It's worse than that... it's claiming to be a *Klamath*, which topped 
out at 300 MHz.

	-hpa

Re: i387/FPU init issues...

[Thomas Gleixner]

On Sat, 3 May 2008, H. Peter Anvin wrote:

> Mikael Pettersson wrote:
> > 
> > I also noticed that jamal's /proc/cpuinfo contains
> > > cpu MHz         : 1063.771
> > which is either wrong or indicates serious overclocking.
> > The PIIs maxed out at about 400/450MHz.
> > 
> 
> It's worse than that... it's claiming to be a *Klamath*, which topped out at
> 300 MHz.

This might also be caused by a calibration error. We've seen enough of
them already.

Thanks,
	tglx

Re: i387/FPU init issues...

[H. Peter Anvin]

Thomas Gleixner wrote:
> 
> This might also be caused by a calibration error. We've seen enough of
> them already.
> 

No doubt.  However, there is *something* ugly going on.

	-hpa

Re: i387/FPU init issues...

[Ingo Molnar]

* H. Peter Anvin <hpa@zytor.com> wrote:

>> This might also be caused by a calibration error. We've seen enough 
>> of them already.
>
> No doubt.  However, there is *something* ugly going on.

i think a vital clue can be found in the original report:

|  I am able to reproduce the issue consistently on my laptop using qemu 
|  (which helped speed debugging a bit). I have also narrowed it down to 
|  include/asm-x86/i387.h::__save_init_fpu in (32 bit version) - it dies 
|  somewhere in calling the following line:

so it might just be incorrect Qemu emulation of a PII's NOP instruction?

(which btw. probably proves that Linux is the first OS to make 
intelligent use of those instructions?)

	Ingo

Re: i387/FPU init issues...

[H. Peter Anvin]

Ingo Molnar wrote:
> * H. Peter Anvin <hpa@zytor.com> wrote:
> 
>>> This might also be caused by a calibration error. We've seen enough 
>>> of them already.
>> No doubt.  However, there is *something* ugly going on.
> 
> i think a vital clue can be found in the original report:
> 
> |  I am able to reproduce the issue consistently on my laptop using qemu 
> |  (which helped speed debugging a bit). I have also narrowed it down to 
> |  include/asm-x86/i387.h::__save_init_fpu in (32 bit version) - it dies 
> |  somewhere in calling the following line:
> 
> so it might just be incorrect Qemu emulation of a PII's NOP instruction?
> 
> (which btw. probably proves that Linux is the first OS to make 
> intelligent use of those instructions?)
> 

Ah, yes, this is a Qemu bug, then.

And yes, it probably does mean exactly that :)

	-hpa

Re: i387/FPU init issues...

[Maciej W. Rozycki]

On Sat, 3 May 2008, H. Peter Anvin wrote:

> Intel explicitly documents "all processors with family 6 or F."

 It depends (as usually with Intel) on what document you are looking at.  
Based on my short research, the instruction has been retroactively added
to the list of supported opcodes.  Even my somewhat dated P4 manual does
not list it, never mind its predecessors.  It could have been accidentally
omitted or even buggy in some early members of the P6 family and this
could have been the reason for not documenting it from the beginning (the
case of FFREEP comes to mind).

  Maciej

Re: i387/FPU init issues...

[H. Peter Anvin]

Maciej W. Rozycki wrote:
> On Sat, 3 May 2008, H. Peter Anvin wrote:
> 
>> Intel explicitly documents "all processors with family 6 or F."
> 
>  It depends (as usually with Intel) on what document you are looking at.  
> Based on my short research, the instruction has been retroactively added
> to the list of supported opcodes.  Even my somewhat dated P4 manual does
> not list it, never mind its predecessors.  It could have been accidentally
> omitted or even buggy in some early members of the P6 family and this
> could have been the reason for not documenting it from the beginning (the
> case of FFREEP comes to mind).
> 

It has retroactively been added to the documented list for all P6 core 
chips - that should mean it works on all of them.  The most common 
reason for not documenting something (other than various Pure Evil NDA 
schemes) is that it hasn't been properly verified.  However, 
verification can be done a posteori.

	-hpa

Re: i387/FPU init issues...

[Maciej W. Rozycki]

On Sat, 3 May 2008, H. Peter Anvin wrote:

> It has retroactively been added to the documented list for all P6 core 
> chips - that should mean it works on all of them.  The most common 
> reason for not documenting something (other than various Pure Evil NDA 
> schemes) is that it hasn't been properly verified.  However, 
> verification can be done a posteori.

 True, but people do make mistakes from time to time. :)  If I had a
choice between a piece of silicon and a piece of documentation to trust, I
would choose the former.

 Obviously this specific case has turned out to be an issue with Qemu, so
it is somewhat irrelevant and given how the opcodes were added to the
architecture I would consider the emulator excused.

  Maciej

Re: i387/FPU init issues...

[jamal]

On Sat, 2008-03-05 at 22:17 +0100, Maciej W. Rozycki wrote:

>  True, but people do make mistakes from time to time. :)  If I had a
> choice between a piece of silicon and a piece of documentation to trust, I
> would choose the former.
> 
>  Obviously this specific case has turned out to be an issue with Qemu, so
> it is somewhat irrelevant and given how the opcodes were added to the
> architecture I would consider the emulator excused.

Sorry folks - had to run some errands. Wow, thanks for all the
responses.

Yes, the posting i just did (as pointed in my first email) was on
qemu (so was the /proc/cpuinfo). I moved to qemu because it was less
painful to do the git bisecting on my laptop; i used the same .config.
I should also note that qemu seems to have worked fine in the past.
In any case I will try to rerun on the older hardware which i can access
next week again. I am begining to doubt myself if it is the same issue;
i know even there it was pointing to FPU.
So is the correct fix then to go patch qemu then? I should point i am
running a slightly older version of qemu that has a few patches (nothing
to do with x86 emulation).

hpa, results from running on qemu (not the hardware) are:
-----
mambo:~# ./hpa
Test 0: ok
Test 1: ok
Test 2: ok
Test 3: err
Test 4: err
Test 5: err
Test 6: err
Test 7: err
Test 8: err
----------------

cheers,
jamal

Re: i387/FPU init issues...

[Sebastian Herbszt]

jamal,

> Sorry folks - had to run some errands. Wow, thanks for all the
> responses.
> 
> Yes, the posting i just did (as pointed in my first email) was on
> qemu (so was the /proc/cpuinfo). I moved to qemu because it was less
> painful to do the git bisecting on my laptop; i used the same .config.
> I should also note that qemu seems to have worked fine in the past.
> In any case I will try to rerun on the older hardware which i can access
> next week again. I am begining to doubt myself if it is the same issue;
> i know even there it was pointing to FPU.
> So is the correct fix then to go patch qemu then? I should point i am
> running a slightly older version of qemu that has a few patches (nothing
> to do with x86 emulation).
> 
> hpa, results from running on qemu (not the hardware) are:
> -----
> mambo:~# ./hpa
> Test 0: ok
> Test 1: ok
> Test 2: ok
> Test 3: err
> Test 4: err
> Test 5: err
> Test 6: err
> Test 7: err
> Test 8: err
> ----------------

this is the same result i get with qemu 0.8.2. With version 0.9.0 and
later it is:

debian:~# ./hpa
Test 0: ok
Test 1: ok
Test 2: ok
Test 3: ok
Test 4: ok
Test 5: ok
Test 6: ok
Test 7: ok
Test 8: ok

> cheers,
> jamal
 
-  Sebastian

Re: i387/FPU init issues...

[jamal]

On Sun, 2008-04-05 at 15:08 +0200, Sebastian Herbszt wrote:


> this is the same result i get with qemu 0.8.2. 

Thanks for validating. 
I am embarassed to say I am still running version 0.7.0 - problem is i
have patched it semi-substantially to add features which i didnt get
much luck in pushing in; so i cant upgrade overnight.

> With version 0.9.0 and
> later it is:

Ok, maybe it is time to invest time to forward port my patches. I dont
see anything in the release notes between 0.8.2 and 0.9.0 which says
this problem was known and fixed in some specific version. Any pointers?

Thanks again - you saved me a lot of time.

cheers,
jamal

Re: i387/FPU init issues...

[Sebastian Herbszt]

>> this is the same result i get with qemu 0.8.2. 
> 
> Thanks for validating. 
> I am embarassed to say I am still running version 0.7.0 - problem is i
> have patched it semi-substantially to add features which i didnt get
> much luck in pushing in; so i cant upgrade overnight.
> 
>> With version 0.9.0 and
>> later it is:
> 
> Ok, maybe it is time to invest time to forward port my patches. I dont
> see anything in the release notes between 0.8.2 and 0.9.0 which says
> this problem was known and fixed in some specific version. Any pointers?

>From http://fabrice.bellard.free.fr/qemu/changelog.html
version 0.9.0:
...
- several x86 and x86_64 emulation fixes

Revision 2145 introduced "multi byte nop support".
See http://svn.savannah.nongnu.org/viewvc/trunk/target-i386/translate.c

> Thanks again - you saved me a lot of time.
> 
> cheers,
> jamal

- Sebastian

Re: i387/FPU init issues...

[Arjan van de Ven]

H. Peter Anvin wrote:
> Maciej W. Rozycki wrote:
>> On Sat, 3 May 2008, H. Peter Anvin wrote:
>>
>>> Intel explicitly documents "all processors with family 6 or F."
>>
>>  It depends (as usually with Intel) on what document you are looking 
>> at.  Based on my short research, the instruction has been 
>> retroactively added
>> to the list of supported opcodes.  Even my somewhat dated P4 manual does
>> not list it, never mind its predecessors.  It could have been 
>> accidentally
>> omitted or even buggy in some early members of the P6 family and this
>> could have been the reason for not documenting it from the beginning (the
>> case of FFREEP comes to mind).
>>
> 
> It has retroactively been added to the documented list for all P6 core 
> chips - that should mean it works on all of them.  The most common 
> reason for not documenting something (other than various Pure Evil NDA 
> schemes) is that it hasn't been properly verified.  However, 
> verification can be done a posteori.

the other reason is that certain groups of "unknown" opcodes will act as NOP.
Which for this purpose is .. exactly the right thing retroactive

> 
>     -hpa

Re: i387/FPU init issues...

[H. Peter Anvin]

Arjan van de Ven wrote:
>>
>> It has retroactively been added to the documented list for all P6 core 
>> chips - that should mean it works on all of them.  The most common 
>> reason for not documenting something (other than various Pure Evil NDA 
>> schemes) is that it hasn't been properly verified.  However, 
>> verification can be done a posteori.
> 
> the other reason is that certain groups of "unknown" opcodes will act as 
> NOP.
> Which for this purpose is .. exactly the right thing retroactive
> 

Specifically, I believe the P6 added a whole class of instructions which 
would act as NOP's if used on older chips.  It has been used for 
prefetch instructions, etc.  At some point, the 0F 1F /0 group was 
declared to be NOP, and nothing else, for all future.

	-hpa