xen/master head PVOPS kernel 2.6.31.1 fails to load under Xen Unstable on Intel's i7 (x) X58. config, console output attached.

xen/master head PVOPS kernel 2.6.31.1 fails to load under Xen Unstable on Intel's i7 (x) X58. config, console output attached.

[Bruce Edge]

Does the xen/master dom0 kernel just not currrently support the i7
with x58 chipset?

The data here is obtained from an HP DL380 G6 with 2 Intel nehalem
E5540 chips. The exact same boot time hang is obtained also with a
supermicro with a single CPU and the same X58 chipset.

Serial console output and xen debug dump here:

    http://pastebin.com/f36278be5

Kernel .config is here

    http://pastebin.com/f5ff4cf82

The current 2.6.31.1 pvops  xen/master kernel hangs when running under
Xen on a proliant the a supermicro, both X58 i7 boxes.
The Xen console is still functional but there is zero output from the
dom0 kernel.

I've tried booting with these xen versions (both of which work with a
2.6.30.3 dom0)
   http://xenbits.xensource.com/xen-unstable.hg
   http://xenbits.xensource.com/xen-3.4-testing.hg

And with the 2.6.31.1 source from both (is there a difference?)
   git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git linux-2.6-xen
   git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git xen/master

Here are the xen related settings (see above pastebin link for
complete details):
CONFIG_XEN=y
CONFIG_XEN_TIME_VSYSCALL=y
CONFIG_XEN_MAX_DOMAIN_MEMORY=32
CONFIG_XEN_SAVE_RESTORE=y
CONFIG_XEN_DEBUG_FS=y
CONFIG_XEN_DOM0=y
CONFIG_XEN_PRIVILEGED_GUEST=y
CONFIG_XEN_DOM0_PCI=y
CONFIG_MICROCODE_XEN=y
CONFIG_PCI_XEN=y
CONFIG_XEN_BLKDEV_FRONTEND=y
CONFIG_XEN_NETDEV_FRONTEND=y
CONFIG_XEN_KBDDEV_FRONTEND=m
CONFIG_HVC_XEN=y
CONFIG_XEN_FBDEV_FRONTEND=m
CONFIG_XEN_BALLOON=y
CONFIG_XEN_SCRUB_PAGES=y
CONFIG_XEN_DEV_EVTCHN=y
CONFIG_XEN_BACKEND=y
CONFIG_XEN_BLKDEV_BACKEND=y
CONFIG_XEN_NETDEV_BACKEND=y
CONFIG_XENFS=y
CONFIG_XEN_COMPAT_XENFS=y
CONFIG_XEN_SYS_HYPERVISOR=y
CONFIG_XEN_XENBUS_FRONTEND=y
CONFIG_XEN_S3=y
CONFIG_XEN_GNTDEV=y


0 #> lspci
00:00.0 Host bridge: Intel Corporation X58 I/O Hub to ESI Port (rev 13)
00:01.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root
Port 1 (rev 13)
00:02.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root
Port 2 (rev 13)
00:03.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root
Port 3 (rev 13)
00:04.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root
Port 4 (rev 13)
00:05.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root
Port 5 (rev 13)
00:06.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root
Port 6 (rev 13)
00:07.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root
Port 7 (rev 13)
00:08.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root
Port 8 (rev 13)
00:09.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root
Port 9 (rev 13)
00:0a.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root
Port 10 (rev 13)
00:0d.0 Host bridge: Intel Corporation Device 343a (rev 13)
00:0d.1 Host bridge: Intel Corporation Device 343b (rev 13)
00:0d.2 Host bridge: Intel Corporation Device 343c (rev 13)
00:0d.3 Host bridge: Intel Corporation Device 343d (rev 13)
00:0d.4 Host bridge: Intel Corporation X58 Physical Layer Port 0 (rev 13)
00:0d.5 Host bridge: Intel Corporation Quickpath Interconnect Physical
Layer Port 1 (rev 13)
00:0d.6 Host bridge: Intel Corporation Device 341a (rev 13)
00:0e.0 Host bridge: Intel Corporation Device 341c (rev 13)
00:0e.1 Host bridge: Intel Corporation Device 341d (rev 13)
00:0e.2 Host bridge: Intel Corporation Device 341e (rev 13)
00:0e.3 Host bridge: Intel Corporation Device 341f (rev 13)
00:0e.4 Host bridge: Intel Corporation Device 3439 (rev 13)
00:14.0 PIC: Intel Corporation X58 I/O Hub System Management Registers (rev 13)
00:14.1 PIC: Intel Corporation X58 I/O Hub GPIO and Scratch Pad
Registers (rev 13)
00:14.2 PIC: Intel Corporation X58 I/O Hub Control Status and RAS
Registers (rev 13)
00:1c.0 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Port 1
00:1c.2 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Port 3
00:1d.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB
UHCI Controller #1
00:1d.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB
UHCI Controller #2
00:1d.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB
UHCI Controller #3
00:1d.3 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB
UHCI Controller #6
00:1d.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2
EHCI Controller #1
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90)
00:1f.0 ISA bridge: Intel Corporation 82801JIB (ICH10) LPC Interface Controller
01:03.0 VGA compatible controller: ATI Technologies Inc ES1000 (rev 02)
01:04.0 System peripheral: Compaq Computer Corporation Integrated
Lights Out Controller (rev 03)
01:04.2 System peripheral: Compaq Computer Corporation Integrated
Lights Out  Processor (rev 03)
01:04.4 USB Controller: Hewlett-Packard Company Proliant iLO2 virtual
USB controller
01:04.6 IPMI SMIC interface: Hewlett-Packard Company Proliant iLO2 virtual UART
02:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709
Gigabit Ethernet (rev 20)
02:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709
Gigabit Ethernet (rev 20)
03:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709
Gigabit Ethernet (rev 20)
03:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709
Gigabit Ethernet (rev 20)
04:00.0 RAID bus controller: Hewlett-Packard Company Smart Array G6
controllers (rev 01)
07:00.0 Fibre Channel: PMC-Sierra Inc. Device 8032 (rev 05)
07:00.1 Fibre Channel: PMC-Sierra Inc. Device 8032 (rev 05)
07:00.2 Fibre Channel: PMC-Sierra Inc. Device 8032 (rev 05)
07:00.3 Fibre Channel: PMC-Sierra Inc. Device 8032 (rev 05)
14:00.0 Fibre Channel: PMC-Sierra Inc. Device 8032 (rev 05)
14:00.1 Fibre Channel: PMC-Sierra Inc. Device 8032 (rev 05)
17:00.0 Fibre Channel: PMC-Sierra Inc. Device 8032 (rev 05)
17:00.1 Fibre Channel: PMC-Sierra Inc. Device 8032 (rev 05)

#> cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 26
model name      : Intel(R) Xeon(R) CPU           E5540  @ 2.53GHz
stepping        : 5
cpu MHz         : 2533.462
cache size      : 8192 KB
physical id     : 0
siblings        : 1
core id         : 0
cpu cores       : 1
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov
pat clflush acpi mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc up
rep_good tsc_reliable nonstop_tsc pni est ssse3 cx16 sse4_1 sse4_2
hypervisor lahf_lm ida
bogomips        : 5067.97
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

What other information can I provide?

-Bruce

Re: xen/master head PVOPS kernel 2.6.31.1 fails to load under Xen Unstable on Intel's i7 (x) X58. config, console output attached.

[Konrad Rzeszutek Wilk]

On Sun, Oct 11, 2009 at 10:21:12PM -0700, Bruce Edge wrote:
> Does the xen/master dom0 kernel just not currrently support the i7
> with x58 chipset?

No idea.

But can you add some more verbose logging information?

Add this in the kernel /xen.gz line: "guest_loglvl=all"
and in the 'module /vmlinuz" add this:
"earlyprintk=xen irqpoll console=hvc0 loglevel=10  debug"

If 'quiet' shows in the line, remove it.

.. snip ..

> CONFIG_XEN_S3=y
> CONFIG_XEN_GNTDEV=y

What is the CONFIG_NR_CPUS set to? Try with CONFIG_NR_CPUS=64

Re: xen/master head PVOPS kernel 2.6.31.1 fails to load under Xen Unstable on Intel's i7 (x) X58. config, console output attached.

[Bruce Edge]

Works!
Thanks, this looks like it may have just been the hvc0 vs xvc0 console spec.

Although I'm not sure why with the xvc0 console the network was still
non-functional. I'll have to play with the boot options and find out
what specific item breaks it.

Thanks for the suggestion.

-Bruce

On Mon, Oct 12, 2009 at 7:10 AM, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
> On Sun, Oct 11, 2009 at 10:21:12PM -0700, Bruce Edge wrote:
>> Does the xen/master dom0 kernel just not currrently support the i7
>> with x58 chipset?
>
> No idea.
>
> But can you add some more verbose logging information?
>
> Add this in the kernel /xen.gz line: "guest_loglvl=all"
> and in the 'module /vmlinuz" add this:
> "earlyprintk=xen irqpoll console=hvc0 loglevel=10  debug"
>
> If 'quiet' shows in the line, remove it.
>
> .. snip ..
>
>> CONFIG_XEN_S3=y
>> CONFIG_XEN_GNTDEV=y
>
> What is the CONFIG_NR_CPUS set to? Try with CONFIG_NR_CPUS=64
>

Re: xen/master head PVOPS kernel 2.6.31.1 fails to load under Xen Unstable on Intel's i7 (x) X58. config, console output attached.

[Boris Derzhavets]

[-- Attachment #1.1: Type: text/plain, Size: 1589 bytes --]

Are you saying , that CONFIG_NR_CPUS=64 is not important ?
Just set up console to hvc0 resolves the problem.

Boris.

--- On Mon, 10/12/09, Bruce Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] xen/master head PVOPS kernel 2.6.31.1 fails to load  under Xen Unstable on Intel's i7 (x) X58. config, console output attached.
To: xen-devel@lists.xensource.com
Date: Monday, October 12, 2009, 12:03 PM

Works!
Thanks, this looks like it may have just been the hvc0 vs xvc0 console spec.

Although I'm not sure why with the xvc0 console the network was still
non-functional. I'll have to play with the boot options and find out
what specific item breaks it.

Thanks for the suggestion.

-Bruce

On Mon, Oct 12, 2009 at 7:10 AM, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
> On Sun, Oct 11, 2009 at 10:21:12PM -0700, Bruce Edge wrote:
>> Does the xen/master dom0 kernel just not currrently support the i7
>> with x58 chipset?
>
> No idea.
>
> But can you add some more verbose logging information?
>
> Add this in the kernel /xen.gz line: "guest_loglvl=all"
> and in the 'module /vmlinuz" add this:
> "earlyprintk=xen irqpoll console=hvc0 loglevel=10  debug"
>
> If 'quiet' shows in the line, remove it.
>
> .. snip ..
>
>> CONFIG_XEN_S3=y
>> CONFIG_XEN_GNTDEV=y
>
> What is the CONFIG_NR_CPUS set to? Try with CONFIG_NR_CPUS=64
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

[-- Attachment #1.2: Type: text/html, Size: 2327 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: xen/master head PVOPS kernel 2.6.31.1 fails to load under Xen Unstable on Intel's i7 (x) X58. config, console output attached.

[Bruce Edge]

[-- Attachment #1.1: Type: text/plain, Size: 2106 bytes --]

Yes. That is the case. I had CONFIG_NR_CPUS=16 I hever tried the =64 kernel
as the console=hvc0 fixed the problem:

console=xvc0 - nothing, dead, not even a ping, although xen console was
still alive.
console=hvc0 - dom0 boots all the way up, network functional, looks good so
far.

-Bruce

On Mon, Oct 12, 2009 at 9:54 AM, Boris Derzhavets <bderzhavets@yahoo.com>wrote:

> Are you saying , that CONFIG_NR_CPUS=64 is not important ?
> Just set up console to hvc0 resolves the problem.
>
> Boris.
>
> --- On *Mon, 10/12/09, Bruce Edge <bruce.edge@gmail.com>* wrote:
>
>
> From: Bruce Edge <bruce.edge@gmail.com>
> Subject: Re: [Xen-devel] xen/master head PVOPS kernel 2.6.31.1 fails to
> load under Xen Unstable on Intel's i7 (x) X58. config, console output
> attached.
> To: xen-devel@lists.xensource.com
> Date: Monday, October 12, 2009, 12:03 PM
>
> Works!
> Thanks, this looks like it may have just been the hvc0 vs xvc0 console
> spec.
>
> Although I'm not sure why with the xvc0 console the network was still
> non-functional. I'll have to play with the boot options and find out
> what specific item breaks it.
>
> Thanks for the suggestion.
>
> -Bruce
>
> On Mon, Oct 12, 2009 at 7:10 AM, Konrad Rzeszutek Wilk
> <konrad.wilk@oracle.com <http://mc/compose?to=konrad.wilk@oracle.com>>
> wrote:
> > On Sun, Oct 11, 2009 at 10:21:12PM -0700, Bruce Edge wrote:
> >> Does the xen/master dom0 kernel just not currrently support the i7
> >> with x58 chipset?
> >
> > No idea.
> >
> > But can you add some more verbose logging information?
> >
> > Add this in the kernel /xen.gz line: "guest_loglvl=all"
> > and in the 'module /vmlinuz" add this:
> > "earlyprintk=xen irqpoll console=hvc0 loglevel=10  debug"
> >
> > If 'quiet' shows in the line, remove it.
> >
> > .. snip ..
> >
> >> CONFIG_XEN_S3=y
> >> CONFIG_XEN_GNTDEV=y
> >
> > What is the CONFIG_NR_CPUS set to? Try with CONFIG_NR_CPUS=64
> >
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com<http://mc/compose?to=Xen-devel@lists.xensource.com>
> http://lists.xensource.com/xen-devel
>
>
>

[-- Attachment #1.2: Type: text/html, Size: 3383 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: xen/master head PVOPS kernel 2.6.31.1 fails to load under Xen Unstable on Intel's i7 (x) X58. config, console output attached.

[Konrad Rzeszutek Wilk]

On Mon, Oct 12, 2009 at 09:59:16AM -0700, Bruce Edge wrote:
> Yes. That is the case. I had CONFIG_NR_CPUS=16 I hever tried the =64 kernel
> as the console=hvc0 fixed the problem:
> 
> console=xvc0 - nothing, dead, not even a ping, although xen console was
> still alive.
> console=hvc0 - dom0 boots all the way up, network functional, looks good so
> far.

Oh yeah. It seems to change over major Xen-enabled Linux distros (It used to be ttyS8,
then went to xvc0, and now is at hvc0).

I am happy to have been able to fix this grievous looking failure so easy. Wish more of
these type showed up :-)

No idea about the network thought.