Fail to early boot with v2.6.27-rc2 to at least v2.6.29-rc2 due to dc1e35c

Fail to early boot with v2.6.27-rc2 to at least v2.6.29-rc2 due to dc1e35c

[Avuton Olrich]

Hello,

My computer fails to make it past 'Unpacking kernel' with anything
later than dc1e35, to at least v2.6.29-rc2 due to dc1e35c, at least so
git bisect told me. While writing this bug I discovered I was using
gcc-4.1.1 when I thought I was using gcc-4.3.2; I upgraded, recompiled
2.6.28.1 and same results so I assume the same results would come from
me doing the 4 hour bisect again.

ver_linux:
Linux rocket 2.6.27.12 #2 SMP PREEMPT Mon Jan 19 05:41:54 PST 2009
x86_64 Intel(R) Core(TM)2 Duo CPU E8600 @ 3.33GHz GenuineIntel
GNU/Linux

Gnu C                  4.3.2
Gnu make               3.81
binutils               2.19
util-linux             ./ver_linux: line 23: fdformat: command not found
mount                  support
module-init-tools      found
xfsprogs               2.10.2
Linux C Library        2.9
Dynamic linker (ldd)   2.9
Procps                 3.2.7
Kbd                    1.15
Sh-utils               6.12
Modules Loaded         snd_pcm_oss snd_mixer_oss snd_seq_dummy
snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device rtc_cmos nvidia
intel_agp

(util-linux version is 2.14.1)

/proc/version:
Linux version 2.6.27.12 (avuton@rocket) (gcc version 4.3.2 (Gentoo
4.3.2-r2 p1.5, pie-10.1.5) ) #2 SMP PREEMPT Mon Jan 19 05:41:54 PST
2009

/proc/iomem:
00000000-0000ffff : reserved
00010000-0009ebff : System RAM
0009ec00-0009ffff : reserved
000c0000-000cffff : pnp 00:0c
000e4000-000fffff : reserved
00100000-7ff8ffff : System RAM
  00200000-0064d311 : Kernel code
  0064d312-00813c57 : Kernel data
  00878000-009c39cf : Kernel bss
7ff90000-7ff9dfff : ACPI Tables
7ff9e000-7ffdffff : ACPI Non-volatile Storage
7ffe0000-7fffffff : reserved
88000000-880000ff : 0000:00:1f.3
bfe00000-dfdfffff : PCI Bus 0000:01
  c0000000-cfffffff : 0000:01:00.0
dfe00000-dfefffff : PCI Bus 0000:04
e0000000-efffffff : PCI MMCONFIG 0
  e0000000-efffffff : pnp 00:0b
f8800000-fe8fffff : PCI Bus 0000:01
  fa000000-fbffffff : 0000:01:00.0
  fd000000-fdffffff : 0000:01:00.0
    fd000000-fdffffff : nvidia
  fe8e0000-fe8fffff : 0000:01:00.0
fe900000-fe9fffff : PCI Bus 0000:02
  fe9e0000-fe9effff : 0000:02:00.0
  fe9fe000-fe9fffff : 0000:02:00.0
    fe9fe000-fe9fffff : ahci
fea00000-feafffff : PCI Bus 0000:03
  feaa0000-feabffff : 0000:03:00.0
  feac0000-feafffff : 0000:03:00.0
    feac0000-feafffff : atl1
febf8000-febfbfff : 0000:00:1b.0
  febf8000-febfbfff : ICH HD audio
febfe000-febfec00 : pnp 00:07
febff000-febff3ff : 0000:00:1d.7
  febff000-febff3ff : ehci_hcd
febff400-febff7ff : 0000:00:1a.7
  febff400-febff7ff : ehci_hcd
febff800-febfffff : 0000:00:1f.2
  febff800-febfffff : ahci
fec00000-fec00fff : IOAPIC 0
  fec00000-fec00fff : pnp 00:09
fed00000-fed003ff : HPET 0
fed14000-fed19fff : pnp 00:01
fed1c000-fed1ffff : pnp 00:07
fed20000-fed3ffff : pnp 00:07
fed45000-fed89fff : pnp 00:07
fee00000-fee00fff : Local APIC
  fee00000-fee00fff : reserved
ffb00000-ffffffff : reserved
100000000-17fffffff : System RAM

/proc/ioports:
0000-001f : dma1
0020-0021 : pic1
0040-0043 : timer0
0050-0053 : timer1
0060-0060 : keyboard
0064-0064 : keyboard
0070-0071 : rtc0
0080-008f : dma page reg
00a0-00a1 : pic2
00c0-00df : dma2
00f0-00ff : fpu
0290-0297 : pnp 00:06
03c0-03df : vga+
0400-041f : 0000:00:1f.3
  0400-041f : i801_smbus
0480-04bf : 0000:00:1f.0
  0480-04bf : pnp 00:07
04d0-04d1 : pnp 00:07
0800-087f : 0000:00:1f.0
  0800-087f : pnp 00:07
    0800-0803 : ACPI PM1a_EVT_BLK
    0804-0805 : ACPI PM1a_CNT_BLK
    0808-080b : ACPI PM_TMR
    0810-0815 : ACPI CPU throttle
    0828-082f : ACPI GPE0_BLK
    0860-087f : iTCO_wdt
0cf8-0cff : PCI conf1
a000-afff : PCI Bus 0000:01
  ac00-ac7f : 0000:01:00.0
b000-bfff : PCI Bus 0000:02
  b400-b40f : 0000:02:00.1
    b400-b40f : pata_jmicron
  b480-b483 : 0000:02:00.1
    b480-b483 : pata_jmicron
  b800-b807 : 0000:02:00.1
    b800-b807 : pata_jmicron
  b880-b883 : 0000:02:00.1
    b880-b883 : pata_jmicron
  bc00-bc07 : 0000:02:00.1
    bc00-bc07 : pata_jmicron
c000-cfff : PCI Bus 0000:05
  c880-c89f : 0000:05:02.0
    c880-c89f : EMU10K1
  cc00-cc07 : 0000:05:02.1
d800-d81f : 0000:00:1d.0
  d800-d81f : uhci_hcd
d880-d89f : 0000:00:1d.1
  d880-d89f : uhci_hcd
dc00-dc1f : 0000:00:1d.2
  dc00-dc1f : uhci_hcd
e000-e01f : 0000:00:1a.0
  e000-e01f : uhci_hcd
e080-e09f : 0000:00:1a.1
  e080-e09f : uhci_hcd
e400-e41f : 0000:00:1f.2
  e400-e41f : ahci
e480-e483 : 0000:00:1f.2
  e480-e483 : ahci
e800-e807 : 0000:00:1f.2
  e800-e807 : ahci
e880-e883 : 0000:00:1f.2
  e880-e883 : ahci
ec00-ec07 : 0000:00:1f.2
  ec00-ec07 : ahci

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 23
model name	: Intel(R) Core(TM)2 Duo CPU     E8600  @ 3.33GHz
stepping	: 10
cpu MHz		: 1998.000
cache size	: 6144 KB
physical id	: 0
siblings	: 2
core id		: 0
cpu cores	: 1
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 2
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
lm constant_tsc pebs bts rep_good nopl pni monitor ds_cpl vmx smx est
tm2 ssse3 cx16 xtpr sse4_1 lahf_lm
bogomips	: 6655.98
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 23
model name	: Intel(R) Core(TM)2 Duo CPU     E8600  @ 3.33GHz
stepping	: 10
cpu MHz		: 1998.000
cache size	: 6144 KB
physical id	: 0
siblings	: 2
core id		: 0
cpu cores	: 1
apicid		: 1
initial apicid	: 1
fpu		: yes
fpu_exception	: yes
cpuid level	: 2
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
lm constant_tsc pebs bts rep_good nopl pni monitor ds_cpl vmx smx est
tm2 ssse3 cx16 xtpr sse4_1 lahf_lm
bogomips	: 6655.71
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:

bisected to:
commit dc1e35c6e95e8923cf1d3510438b63c600fee1e2
Author: Suresh Siddha <suresh.b.siddha@intel.com>
Date:   Tue Jul 29 10:29:19 2008 -0700

    x86, xsave: enable xsave/xrstor on cpus with xsave support

    Enables xsave/xrstor by turning on cr4.osxsave on cpu's which have
    the xsave support. For now, features that OS supports/enabled are
    FP and SSE.

    Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com>
    Signed-off-by: H. Peter Anvin <hpa@zytor.com>
    Signed-off-by: Ingo Molnar <mingo@elte.hu>

http://avuton.googlepages.com/lspci-vvv
http://avuton.googlepages.com/config-2.6.28
-- 
avuton
--
|  (\_/)  This is Bunny. Copy and paste Bunny
| (='.'=) into your signature to help him gain
| (")_(") world domination.

Re: Fail to early boot with v2.6.27-rc2 to at least v2.6.29-rc2 due to dc1e35c

[Avuton Olrich]

On Mon, Jan 19, 2009 at 6:04 AM, Avuton Olrich <avuton@gmail.com> wrote:
> Hello,
>
> My computer fails to make it past 'Unpacking kernel' with anything
> later than dc1e35, to at least v2.6.29-rc2 due to dc1e35c, at least so
> git bisect told me. While writing this bug I discovered I was using
> gcc-4.1.1 when I thought I was using gcc-4.3.2; I upgraded, recompiled
> 2.6.28.1 and same results so I assume the same results would come from
> me doing the 4 hour bisect again.

I was inaccurate, It did not print 'Unpacking kernel', it was 'Booting
the kernel'.
-- 
avuton
--
|  (\_/)  This is Bunny. Copy and paste Bunny
| (='.'=) into your signature to help him gain
| (")_(") world domination.

Re: Fail to early boot with v2.6.27-rc2 to at least v2.6.29-rc2 due to dc1e35c

[H. Peter Anvin]

[-- Attachment #1: Type: text/plain, Size: 251 bytes --]

> 
> bisected to:
> commit dc1e35c6e95e8923cf1d3510438b63c600fee1e2
> Author: Suresh Siddha <suresh.b.siddha@intel.com>
> Date:   Tue Jul 29 10:29:19 2008 -0700
> 

Avuton, could you please compile the attached program and send us the 
output?

	-hpa

[-- Attachment #2: cpuid.c --]
[-- Type: text/x-csrc, Size: 1096 bytes --]

#include <stdio.h>
#include <ctype.h>
#include <stdint.h>

char *make_string(uint32_t val)
{
  static char string[5] = "xxxx";
  int i, ch;

  for ( i = 0 ; i < 4 ; i++ ) {
    ch = val & 0xff;
    string[i] = isprint(ch) ? ch : '.';
    val >>= 8;
  }

  return string;
}

void dump_cpuid_level(uint32_t level)
{
  uint32_t eax, ebx, ecx, edx;

  asm("cpuid" : "=a" (eax), "=b" (ebx), "=c" (ecx), "=d" (edx)
      : "a" (level));

  printf("%08X:  %08X %s  ", level, eax, make_string(eax));
  printf("%08X %s  ", ebx, make_string(ebx));
  printf("%08X %s  ", ecx, make_string(ecx));
  printf("%08X %s\n", edx, make_string(edx));
}

void dump_levels(uint32_t region)
{
  uint32_t max, n;

  asm("cpuid" : "=a" (max) : "a" (region) : "ebx","ecx","edx");
  if ( (max & 0xffff0000) == region ) {
    for ( n = region ; n <= max ; n++ ) {
      dump_cpuid_level(n);
    }
  }
}

int main(int argc, char *argv[])
{
  uint32_t n;

  printf("Level      EAX            EBX            ECX            EDX            \n");

  for ( n = 0 ; n <= 0xffff ; n++ ) {
    dump_levels(n << 16);
  }

  return 0;
}

Re: Fail to early boot with v2.6.27-rc2 to at least v2.6.29-rc2 due to dc1e35c

[Avuton Olrich]

2009/1/19 H. Peter Anvin <hpa@zytor.com>:
>>
>> bisected to:
>> commit dc1e35c6e95e8923cf1d3510438b63c600fee1e2
>> Author: Suresh Siddha <suresh.b.siddha@intel.com>
>> Date:   Tue Jul 29 10:29:19 2008 -0700
>>
>
> Avuton, could you please compile the attached program and send us the
> output?

Sure whichever flavor you prefer:
http://avuton.googlepages.com/cpuid-out (3M)
http://avuton.googlepages.com/cpuid-out.bz2 (40K)
-- 
avuton
--
|  (\_/)  This is Bunny. Copy and paste Bunny
| (='.'=) into your signature to help him gain
| (")_(") world domination.

Re: Fail to early boot with v2.6.27-rc2 to at least v2.6.29-rc2 due to dc1e35c

[H. Peter Anvin]

Avuton Olrich wrote:
> 2009/1/19 H. Peter Anvin <hpa@zytor.com>:
>>> bisected to:
>>> commit dc1e35c6e95e8923cf1d3510438b63c600fee1e2
>>> Author: Suresh Siddha <suresh.b.siddha@intel.com>
>>> Date:   Tue Jul 29 10:29:19 2008 -0700
>>>
>> Avuton, could you please compile the attached program and send us the
>> output?
> 
> Sure whichever flavor you prefer:
> http://avuton.googlepages.com/cpuid-out (3M)
> http://avuton.googlepages.com/cpuid-out.bz2 (40K)

OK, that's an XSAVE-capable CPU, and yet the XSAVE code causes a crash. 
  Not good...

	-hpa

Re: Fail to early boot with v2.6.27-rc2 to at least v2.6.29-rc2 due to dc1e35c

[Suresh Siddha]

On Mon, Jan 19, 2009 at 12:07:34PM -0800, H. Peter Anvin wrote:
> Avuton Olrich wrote:
> > 2009/1/19 H. Peter Anvin <hpa@zytor.com>:
> >>> bisected to:
> >>> commit dc1e35c6e95e8923cf1d3510438b63c600fee1e2
> >>> Author: Suresh Siddha <suresh.b.siddha@intel.com>
> >>> Date:   Tue Jul 29 10:29:19 2008 -0700
> >>>
> >> Avuton, could you please compile the attached program and send us the
> >> output?
> > 
> > Sure whichever flavor you prefer:
> > http://avuton.googlepages.com/cpuid-out (3M)
> > http://avuton.googlepages.com/cpuid-out.bz2 (40K)
> 
> OK, that's an XSAVE-capable CPU, and yet the XSAVE code causes a crash. 
>   Not good...

Avuton, Does your bios has an option which says limit the cpuid
vector limit to '2' or something. Can you disable that option and
re-check? It will typically be located under cpu configuration settings.

My system which has same stepping etc works just fine. But my bios doesn't
have the cpuid limit option. I need to check if this issue is because
of the cpuid limit option that is enabled in the bios.

thanks,
suresh

Re: Fail to early boot with v2.6.27-rc2 to at least v2.6.29-rc2 due to dc1e35c

[Avuton Olrich]

On Mon, Jan 19, 2009 at 12:11 PM, Suresh Siddha
<suresh.b.siddha@intel.com> wrote:
> Avuton, Does your bios has an option which says limit the cpuid
> vector limit to '2' or something. Can you disable that option and
> re-check? It will typically be located under cpu configuration settings.

You're the winner of the prize. The culprit in my bios was:

Max CPUID Value Limit: Enabled

I disabled it and 2.6.28.1 boots fine now.

Thanks!
-- 
avuton
--
|  (\_/)  This is Bunny. Copy and paste Bunny
| (='.'=) into your signature to help him gain
| (")_(") world domination.

Re: Fail to early boot with v2.6.27-rc2 to at least v2.6.29-rc2 due to dc1e35c

[Suresh Siddha]

On Mon, Jan 19, 2009 at 01:46:37PM -0800, Avuton Olrich wrote:
> On Mon, Jan 19, 2009 at 12:11 PM, Suresh Siddha
> <suresh.b.siddha@intel.com> wrote:
> > Avuton, Does your bios has an option which says limit the cpuid
> > vector limit to '2' or something. Can you disable that option and
> > re-check? It will typically be located under cpu configuration settings.
> 
> You're the winner of the prize. The culprit in my bios was:
>
> Max CPUID Value Limit: Enabled

Though the bios is the culprit and this option will severely limit
the cpu capabilities that OS can take advantage of, OS should fallback
to a safer mode. I will have a patch for it.

Also, I wonder, if we should complain/scream during boot if we find only
fewer cpuid levels on modern generation cpu's.

thanks,
suresh

Re: Fail to early boot with v2.6.27-rc2 to at least v2.6.29-rc2 due to dc1e35c

[H. Peter Anvin]

Suresh Siddha wrote:
> On Mon, Jan 19, 2009 at 01:46:37PM -0800, Avuton Olrich wrote:
>> On Mon, Jan 19, 2009 at 12:11 PM, Suresh Siddha
>> <suresh.b.siddha@intel.com> wrote:
>>> Avuton, Does your bios has an option which says limit the cpuid
>>> vector limit to '2' or something. Can you disable that option and
>>> re-check? It will typically be located under cpu configuration settings.
>> You're the winner of the prize. The culprit in my bios was:
>>
>> Max CPUID Value Limit: Enabled
> 
> Though the bios is the culprit and this option will severely limit
> the cpu capabilities that OS can take advantage of, OS should fallback
> to a safer mode. I will have a patch for it.
> 
> Also, I wonder, if we should complain/scream during boot if we find only
> fewer cpuid levels on modern generation cpu's.
> 

We should, or if this block is reversible, we should probably just undo 
it (the reason people put this block in places is because of, ahem, 
inferior operating systems having bugs.)

Do you know how this is managed?  Via an MSR?

	-hpa

Re: Fail to early boot with v2.6.27-rc2 to at least v2.6.29-rc2 due to dc1e35c

[Suresh Siddha]

On Mon, Jan 19, 2009 at 02:07:36PM -0800, H. Peter Anvin wrote:
> Suresh Siddha wrote:
> > 
> > Also, I wonder, if we should complain/scream during boot if we find only
> > fewer cpuid levels on modern generation cpu's.
> > 
> 
> We should, or if this block is reversible, we should probably just undo 
> it (the reason people put this block in places is because of, ahem, 
> inferior operating systems having bugs.)
> 
> Do you know how this is managed?  Via an MSR?

IA32_MISC_ENABLE MSR bit 22

Re: Fail to early boot with v2.6.27-rc2 to at least v2.6.29-rc2 due to dc1e35c

[H. Peter Anvin]

Suresh Siddha wrote:
> On Mon, Jan 19, 2009 at 02:07:36PM -0800, H. Peter Anvin wrote:
>> Suresh Siddha wrote:
>>> Also, I wonder, if we should complain/scream during boot if we find only
>>> fewer cpuid levels on modern generation cpu's.
>>>
>> We should, or if this block is reversible, we should probably just undo 
>> it (the reason people put this block in places is because of, ahem, 
>> inferior operating systems having bugs.)
>>
>> Do you know how this is managed?  Via an MSR?
> 
> IA32_MISC_ENABLE MSR bit 22

LOL, the official name of this bit is "IA32_MISC_ENABLES.BOOT_NT4"; kind 
of says it all.  In fact, I remember the problems we had with NT4 and 
CPUID back from the Transmeta days, and there, too, we ended up with a 
CPUID hack which Linux unconditionally disables.

I'll write up a patch.

	-hpa

Re: Fail to early boot with v2.6.27-rc2 to at least v2.6.29-rc2 due to dc1e35c

[Andi Kleen]

"H. Peter Anvin" <hpa@zytor.com> writes:
>
> We should, or if this block is reversible, we should probably just
> undo it (the reason people put this block in places is because of,
> ahem, inferior operating systems having bugs.)

Even if it's undone it would be still a good idea to make the cpuid
code bulletproof, just in case someone writes a broken emulator or similar.

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only.

Re: Fail to early boot with v2.6.27-rc2 to at least v2.6.29-rc2 due to dc1e35c

[Suresh Siddha]

On Tue, Jan 20, 2009 at 09:20:37PM -0800, Andi Kleen wrote:
> "H. Peter Anvin" <hpa@zytor.com> writes:
> >
> > We should, or if this block is reversible, we should probably just
> > undo it (the reason people put this block in places is because of,
> > ahem, inferior operating systems having bugs.)
> 
> Even if it's undone it would be still a good idea to make the cpuid
> code bulletproof, just in case someone writes a broken emulator or similar.

Ok. Here is the patch for this aswell. Thanks.

---
From: Suresh Siddha <suresh.b.siddha@intel.com>
Subject: [patch] x86: check for the presence of cpuid xsave leaf

Avuton Olrich reported early boot crashes with v2.6.28 and
bisected it down to dc1e35c6e95e8923cf1d3510438b63c600fee1e2
("x86, xsave: enable xsave/xrstor on cpus with xsave support").

Root cause of this showed that bios has enabled the "Max CPUID Value Limit:"
option which confuses the kernel by showing xsave capability without
the cpuid xsave leaf.

Peter fixed the impact of bios limiting the cpuid limit by checking
for the limit bit set in the MSR_IA32_MISC_ENABLE is set and clearing it.

Andi suggests to make xsave code also bulletproof, just incase if someone
writes a  broken simulator.

Check for the presence of CPUID_XSAVE_LEAF before enabling it.

Reported-and-bisected-by: Avuton Olrich <avuton@gmail.com>
Tested-by: Avuton Olrich <avuton@gmail.com>
Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com>
---

diff --git a/arch/x86/include/asm/xsave.h b/arch/x86/include/asm/xsave.h
index 08e9a1a..96bb62f 100644
--- a/arch/x86/include/asm/xsave.h
+++ b/arch/x86/include/asm/xsave.h
@@ -27,7 +27,7 @@ extern unsigned int xstate_size;
 extern u64 pcntxt_mask;
 extern struct xsave_struct *init_xstate_buf;
 
-extern void xsave_cntxt_init(void);
+extern int xsave_cntxt_init(void);
 extern void xsave_init(void);
 extern int init_fpu(struct task_struct *child);
 extern int check_for_xstate(struct i387_fxsave_struct __user *buf,
diff --git a/arch/x86/kernel/i387.c b/arch/x86/kernel/i387.c
index b0f61f0..2e2e8b1 100644
--- a/arch/x86/kernel/i387.c
+++ b/arch/x86/kernel/i387.c
@@ -65,10 +65,8 @@ void __cpuinit init_thread_xstate(void)
 		return;
 	}
 
-	if (cpu_has_xsave) {
-		xsave_cntxt_init();
+	if (!xsave_cntxt_init())
 		return;
-	}
 
 	if (cpu_has_fxsr)
 		xstate_size = sizeof(struct i387_fxsave_struct);
diff --git a/arch/x86/kernel/xsave.c b/arch/x86/kernel/xsave.c
index 2b54fe0..5384a3b 100644
--- a/arch/x86/kernel/xsave.c
+++ b/arch/x86/kernel/xsave.c
@@ -307,14 +307,25 @@ static void __init setup_xstate_init(void)
 	init_xstate_buf->i387.mxcsr = MXCSR_DEFAULT;
 }
 
+#define CPUID_XSAVE_LEAF	0xd
+
 /*
  * Enable and initialize the xsave feature.
  */
-void __ref xsave_cntxt_init(void)
+int __ref xsave_cntxt_init(void)
 {
 	unsigned int eax, ebx, ecx, edx;
 
-	cpuid_count(0xd, 0, &eax, &ebx, &ecx, &edx);
+	if (!cpu_has_xsave)
+		return -1;
+
+	if (boot_cpu_data.cpuid_level < CPUID_XSAVE_LEAF) {
+		printk(KERN_ERR "cpuid xsave leaf 0xd not supported\n");
+		setup_clear_cpu_cap(X86_FEATURE_XSAVE);
+		return -1;
+	}
+
+	cpuid_count(CPUID_XSAVE_LEAF, 0, &eax, &ebx, &ecx, &edx);
 	pcntxt_mask = eax + ((u64)edx << 32);
 
 	if ((pcntxt_mask & XSTATE_FPSSE) != XSTATE_FPSSE) {
@@ -342,4 +353,5 @@ void __ref xsave_cntxt_init(void)
 	printk(KERN_INFO "xsave/xrstor: enabled xstate_bv 0x%llx, "
 	       "cntxt size 0x%x\n",
 	       pcntxt_mask, xstate_size);
+	return 0;
 }

Re: Fail to early boot with v2.6.27-rc2 to at least v2.6.29-rc2 due to dc1e35c

[H. Peter Anvin]

Suresh Siddha wrote:
> Ok. Here is the patch for this aswell. Thanks.

I wonder if it wouldn't be better to do this in the CPUID code rather 
than in the xsave code...

	-hpa

Re: Fail to early boot with v2.6.27-rc2 to at least v2.6.29-rc2 due to dc1e35c

[Suresh Siddha]

On Thu, Jan 22, 2009 at 02:40:56PM -0800, H. Peter Anvin wrote:
> Suresh Siddha wrote:
> > Ok. Here is the patch for this aswell. Thanks.
> 
> I wonder if it wouldn't be better to do this in the CPUID code rather
> than in the xsave code...

We can do this in the cpuid code aswell, but the dependency information
varies from feature to feature and there are no architectural methods here.
So I think its better to keep it near the code enabling that feaure.

thanks,
suresh

Re: Fail to early boot with v2.6.27-rc2 to at least v2.6.29-rc2 due to dc1e35c

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 629 bytes --]

On Mon, 19 Jan 2009 13:57:36 PST, Suresh Siddha said:

> Though the bios is the culprit and this option will severely limit
> the cpu capabilities that OS can take advantage of, OS should fallback
> to a safer mode. I will have a patch for it.
> 
> Also, I wonder, if we should complain/scream during boot if we find only
> fewer cpuid levels on modern generation cpu's.

I think a KERN_INFO "Core2 E9700 expected 6 cpuid levels, got 4"
would possibly be a good idea.

Might be a good idea to check what happens under VMWare and similar though, it
looks like the type of thing a hypervisor is likely to do something odd to us...

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: Fail to early boot with v2.6.27-rc2 to at least v2.6.29-rc2 due to dc1e35c

[H. Peter Anvin]

[-- Attachment #1: Type: text/plain, Size: 930 bytes --]

Valdis.Kletnieks@vt.edu wrote:
> On Mon, 19 Jan 2009 13:57:36 PST, Suresh Siddha said:
> 
>> Though the bios is the culprit and this option will severely limit
>> the cpu capabilities that OS can take advantage of, OS should fallback
>> to a safer mode. I will have a patch for it.
>>
>> Also, I wonder, if we should complain/scream during boot if we find only
>> fewer cpuid levels on modern generation cpu's.
> 
> I think a KERN_INFO "Core2 E9700 expected 6 cpuid levels, got 4"
> would possibly be a good idea.
> 
> Might be a good idea to check what happens under VMWare and similar though, it
> looks like the type of thing a hypervisor is likely to do something odd to us...

I think a much better idea is to just clear the MSR bit.  Attached is a
patch to do exactly that, which I will commit after testing.

	-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.


[-- Attachment #2: cpuid.diff --]
[-- Type: text/x-patch, Size: 666 bytes --]

diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c
index 8ea6929..1107015 100644
--- a/arch/x86/kernel/cpu/intel.c
+++ b/arch/x86/kernel/cpu/intel.c
@@ -29,6 +29,16 @@
 
 static void __cpuinit early_init_intel(struct cpuinfo_x86 *c)
 {
+	u64 misc_enable;
+
+	/* If MSR_IA32_MISC_ENABLE exists, unmask CPUID levels if masked */
+	if (!rdmsrl_safe(MSR_IA32_MISC_ENABLE, &misc_enable)) {
+		if (misc_enable & (1 << 22)) {
+			misc_enable &= ~(1 << 22);
+			wrmsrl(MSR_IA32_MISC_ENABLE, misc_enable);
+		}
+	}
+
 	if ((c->x86 == 0xf && c->x86_model >= 0x03) ||
 		(c->x86 == 0x6 && c->x86_model >= 0x0e))
 		set_cpu_cap(c, X86_FEATURE_CONSTANT_TSC);

Re: Fail to early boot with v2.6.27-rc2 to at least v2.6.29-rc2 due to dc1e35c

[H. Peter Anvin]

[-- Attachment #1: Type: text/plain, Size: 559 bytes --]

Avuton Olrich wrote:
> Hello,
> 
> My computer fails to make it past 'Unpacking kernel' with anything
> later than dc1e35, to at least v2.6.29-rc2 due to dc1e35c, at least so
> git bisect told me. While writing this bug I discovered I was using
> gcc-4.1.1 when I thought I was using gcc-4.3.2; I upgraded, recompiled
> 2.6.28.1 and same results so I assume the same results would come from
> me doing the 4 hour bisect again.
> 

Hi Avuton,

Could you apply these two patches and verify that they work, even with 
the BIOS CPUID level limit enabled?

	-hpa


[-- Attachment #2: 0001-x86-add-MSR_IA32_MISC_ENABLE-bits-to-asm-msr-index.patch --]
[-- Type: text/x-patch, Size: 2439 bytes --]

>From 2afec5648181c201ea48c442e2b47d11a73d9f50 Mon Sep 17 00:00:00 2001
From: H. Peter Anvin <hpa@linux.intel.com>
Date: Wed, 21 Jan 2009 15:01:56 -0800
Subject: [PATCH] x86: add MSR_IA32_MISC_ENABLE bits to <asm/msr-index.h>

Impact: None (new bit definitions currently unused)

Add bit definitions for the MSR_IA32_MISC_ENABLE MSRs to
<asm/msr-index.h>.

Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
---
 arch/x86/include/asm/msr-index.h |   29 +++++++++++++++++++++++++++++
 1 files changed, 29 insertions(+), 0 deletions(-)

diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index cb58643..358acc5 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -202,6 +202,35 @@
 #define MSR_IA32_THERM_STATUS		0x0000019c
 #define MSR_IA32_MISC_ENABLE		0x000001a0
 
+/* MISC_ENABLE bits: architectural */
+#define MSR_IA32_MISC_ENABLE_FAST_STRING	(1ULL << 0)
+#define MSR_IA32_MISC_ENABLE_TCC		(1ULL << 1)
+#define MSR_IA32_MISC_ENABLE_EMON		(1ULL << 7)
+#define MSR_IA32_MISC_ENABLE_BTS_UNAVAIL	(1ULL << 11)
+#define MSR_IA32_MISC_ENABLE_PEBS_UNAVAIL	(1ULL << 12)
+#define MSR_IA32_MISC_ENABLE_ENHANCED_SPEEDSTEP	(1ULL << 16)
+#define MSR_IA32_MISC_ENABLE_MWAIT		(1ULL << 18)
+#define MSR_IA32_MISC_ENABLE_LIMIT_CPUID	(1ULL << 22)
+#define MSR_IA32_MISC_ENABLE_XTPR_DISABLE	(1ULL << 23)
+#define MSR_IA32_MISC_ENABLE_XD_DISABLE		(1ULL << 34)
+
+/* MISC_ENABLE bits: model-specific, meaning may vary from core to core */
+#define MSR_IA32_MISC_ENABLE_X87_COMPAT		(1ULL << 2)
+#define MSR_IA32_MISC_ENABLE_TM1		(1ULL << 3)
+#define MSR_IA32_MISC_ENABLE_SPLIT_LOCK_DISABLE	(1ULL << 4)
+#define MSR_IA32_MISC_ENABLE_L3CACHE_DISABLE	(1ULL << 6)
+#define MSR_IA32_MISC_ENABLE_SUPPRESS_LOCK	(1ULL << 8)
+#define MSR_IA32_MISC_ENABLE_PREFETCH_DISABLE	(1ULL << 9)
+#define MSR_IA32_MISC_ENABLE_FERR		(1ULL << 10)
+#define MSR_IA32_MISC_ENABLE_FERR_MULTIPLEX	(1ULL << 10)
+#define MSR_IA32_MISC_ENABLE_TM2		(1ULL << 13)
+#define MSR_IA32_MISC_ENABLE_ADJ_PREF_DISABLE	(1ULL << 19)
+#define MSR_IA32_MISC_ENABLE_SPEEDSTEP_LOCK	(1ULL << 20)
+#define MSR_IA32_MISC_ENABLE_L1D_CONTEXT	(1ULL << 24)
+#define MSR_IA32_MISC_ENABLE_DCU_PREF_DISABLE	(1ULL << 37)
+#define MSR_IA32_MISC_ENABLE_TURBO_DISABLE	(1ULL << 38)
+#define MSR_IA32_MISC_ENABLE_IP_PREF_DISABLE	(1ULL << 39)
+
 /* Intel Model 6 */
 #define MSR_P6_EVNTSEL0			0x00000186
 #define MSR_P6_EVNTSEL1			0x00000187
-- 
1.6.0.6


[-- Attachment #3: 0002-x86-unmask-CPUID-levels-on-Intel-CPUs.patch --]
[-- Type: text/x-patch, Size: 1325 bytes --]

>From 128b048be5309bb43641a3c91d599b07158edfd9 Mon Sep 17 00:00:00 2001
From: H. Peter Anvin <hpa@linux.intel.com>
Date: Wed, 21 Jan 2009 15:04:32 -0800
Subject: [PATCH] x86: unmask CPUID levels on Intel CPUs

Impact: Fixes crashes with misconfigured BIOSes on XSAVE hardware

If the CPUID limit bit in MSR_IA32_MISC_ENABLE is set, clear it to
make all CPUID information available.  This is required for some
features to work, in particular XSAVE.

Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
---
 arch/x86/kernel/cpu/intel.c |   10 ++++++++++
 1 files changed, 10 insertions(+), 0 deletions(-)

diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c
index 8ea6929..43c1dcf 100644
--- a/arch/x86/kernel/cpu/intel.c
+++ b/arch/x86/kernel/cpu/intel.c
@@ -29,6 +29,16 @@
 
 static void __cpuinit early_init_intel(struct cpuinfo_x86 *c)
 {
+	u64 misc_enable;
+
+	/* Unmask CPUID levels if masked */
+	if (!rdmsrl_safe(MSR_IA32_MISC_ENABLE, &misc_enable) &&
+	    (misc_enable & MSR_IA32_MISC_ENABLE_LIMIT_CPUID)) {
+		misc_enable &= ~MSR_IA32_MISC_ENABLE_LIMIT_CPUID;
+		wrmsrl(MSR_IA32_MISC_ENABLE, misc_enable);
+		c->cpuid_level = cpuid_eax(0);
+	}
+
 	if ((c->x86 == 0xf && c->x86_model >= 0x03) ||
 		(c->x86 == 0x6 && c->x86_model >= 0x0e))
 		set_cpu_cap(c, X86_FEATURE_CONSTANT_TSC);
-- 
1.6.0.6

Re: Fail to early boot with v2.6.27-rc2 to at least v2.6.29-rc2 due to dc1e35c

[Avuton Olrich]

On Wed, Jan 21, 2009 at 4:38 PM, H. Peter Anvin <hpa@zytor.com> wrote:
> Avuton Olrich wrote:
>>
>> Hello,
>>
>> My computer fails to make it past 'Unpacking kernel' with anything
>> later than dc1e35, to at least v2.6.29-rc2 due to dc1e35c, at least so
>> git bisect told me. While writing this bug I discovered I was using
>> gcc-4.1.1 when I thought I was using gcc-4.3.2; I upgraded, recompiled
>> 2.6.28.1 and same results so I assume the same results would come from
>> me doing the 4 hour bisect again.
>>
>
> Hi Avuton,
>
> Could you apply these two patches and verify that they work, even with the
> BIOS CPUID level limit enabled?

Worked perfect, again thanks!
-- 
avuton
--
|  (\_/)  This is Bunny. Copy and paste Bunny
| (='.'=) into your signature to help him gain
| (")_(") world domination.

Re: Fail to early boot with v2.6.27-rc2 to at least v2.6.29-rc2 due to dc1e35c

[Ingo Molnar]

* Avuton Olrich <avuton@gmail.com> wrote:

> On Wed, Jan 21, 2009 at 4:38 PM, H. Peter Anvin <hpa@zytor.com> wrote:
> > Avuton Olrich wrote:
> >>
> >> Hello,
> >>
> >> My computer fails to make it past 'Unpacking kernel' with anything
> >> later than dc1e35, to at least v2.6.29-rc2 due to dc1e35c, at least so
> >> git bisect told me. While writing this bug I discovered I was using
> >> gcc-4.1.1 when I thought I was using gcc-4.3.2; I upgraded, recompiled
> >> 2.6.28.1 and same results so I assume the same results would come from
> >> me doing the 4 hour bisect again.
> >>
> >
> > Hi Avuton,
> >
> > Could you apply these two patches and verify that they work, even with the
> > BIOS CPUID level limit enabled?
> 
> Worked perfect, again thanks!

Thanks Avuton - i've updated the commit with your Tested-by tag, see the 
final commit below.

	Ingo

--------------------->
>From 066941bd4eeb159307a5d7d795100d0887c00442 Mon Sep 17 00:00:00 2001
From: H. Peter Anvin <hpa@linux.intel.com>
Date: Wed, 21 Jan 2009 15:04:32 -0800
Subject: [PATCH] x86: unmask CPUID levels on Intel CPUs

Impact: Fixes crashes with misconfigured BIOSes on XSAVE hardware

Avuton Olrich reported early boot crashes with v2.6.28 and
bisected it down to dc1e35c6e95e8923cf1d3510438b63c600fee1e2
("x86, xsave: enable xsave/xrstor on cpus with xsave support").

If the CPUID limit bit in MSR_IA32_MISC_ENABLE is set, clear it to
make all CPUID information available.  This is required for some
features to work, in particular XSAVE.

Reported-and-bisected-by: Avuton Olrich <avuton@gmail.com>
Tested-by: Avuton Olrich <avuton@gmail.com>
Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
---
 arch/x86/kernel/cpu/intel.c |   10 ++++++++++
 1 files changed, 10 insertions(+), 0 deletions(-)

diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c
index 8ea6929..43c1dcf 100644
--- a/arch/x86/kernel/cpu/intel.c
+++ b/arch/x86/kernel/cpu/intel.c
@@ -29,6 +29,16 @@
 
 static void __cpuinit early_init_intel(struct cpuinfo_x86 *c)
 {
+	u64 misc_enable;
+
+	/* Unmask CPUID levels if masked */
+	if (!rdmsrl_safe(MSR_IA32_MISC_ENABLE, &misc_enable) &&
+	    (misc_enable & MSR_IA32_MISC_ENABLE_LIMIT_CPUID)) {
+		misc_enable &= ~MSR_IA32_MISC_ENABLE_LIMIT_CPUID;
+		wrmsrl(MSR_IA32_MISC_ENABLE, misc_enable);
+		c->cpuid_level = cpuid_eax(0);
+	}
+
 	if ((c->x86 == 0xf && c->x86_model >= 0x03) ||
 		(c->x86 == 0x6 && c->x86_model >= 0x0e))
 		set_cpu_cap(c, X86_FEATURE_CONSTANT_TSC);