From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
To: Ed Swierk <eswierk@skyportsystems.com>,
boris.ostrovsky@oracle.com, david.vrabel@citrix.com,
jgross@suse.com
Cc: xen-devel@lists.xensource.com
Subject: Re: PAT-related crash booting Linux 4.4 + Xen 4.5 on VMware ESXi
Date: Mon, 23 May 2016 10:15:23 -0400 [thread overview]
Message-ID: <20160523141523.GB9487@char.us.oracle.com> (raw)
In-Reply-To: <1463788689-52520-1-git-send-email-eswierk@skyportsystems.com>
On Fri, May 20, 2016 at 04:58:09PM -0700, Ed Swierk wrote:
> I've encountered two problems booting a Linux 4.4 dom0 on recent
> stable xen 4.5 on VMware ESXi 5.5.0.
>
> One has the same "ata_piix: probe of 0000:00:07.1 failed with error
> -22" symptom discussed some time ago, and prevents the kernel from
> seeing any of the virtual IDE drives exposed by VMware. This problem
> is fixed by applying Stefano's patch
> (https://lkml.org/lkml/2016/4/20/345).
>
> Another problem occurs very early during boot:
>
> (XEN) Xen version 4.5.4-pre ( 4.5.4~pre-1skyport2) (eswierk@skyportsystems.com) (gcc (Debian 5.2.1-19.1skyport1) 5.2.1 20150930) debug=n Thu May 19 12:06:20 PDT 2016
> (XEN) Bootloader: SYSLINUX 4.05 20140113
> (XEN) Command line: xen console=com1,vga com1=115200 no-bootscrub dom0_mem=2048M,max:2048M ignore_loglevel
> (XEN) Video information:
> (XEN) VGA is text mode 80x25, font 8x16
> (XEN) Disc information:
> (XEN) Found 1 MBR signatures
> (XEN) Found 1 EDD information structures
> (XEN) Xen-e820 RAM map:
> (XEN) 0000000000000000 - 000000000009f800 (usable)
> (XEN) 000000000009f800 - 00000000000a0000 (reserved)
> (XEN) 00000000000dc000 - 0000000000100000 (reserved)
> (XEN) 0000000000100000 - 00000000bfef0000 (usable)
> (XEN) 00000000bfef0000 - 00000000bfeff000 (ACPI data)
> (XEN) 00000000bfeff000 - 00000000bff00000 (ACPI NVS)
> (XEN) 00000000bff00000 - 00000000c0000000 (usable)
> (XEN) 00000000f0000000 - 00000000f8000000 (reserved)
> (XEN) 00000000fec00000 - 00000000fec10000 (reserved)
> (XEN) 00000000fee00000 - 00000000fee01000 (reserved)
> (XEN) 00000000fffe0000 - 0000000100000000 (reserved)
> (XEN) 0000000100000000 - 00000001c0000000 (usable)
> (XEN) ACPI: RSDP 000F6B80, 0024 (r2 PTLTD )
> (XEN) ACPI: XSDT BFEF0F70, 0054 (r1 INTEL 440BX 6040000 VMW 1324272)
> (XEN) ACPI: FACP BFEFEE73, 00F4 (r4 INTEL 440BX 6040000 PTL F4240)
> (XEN) ACPI: DSDT BFEF1252, DC21 (r1 PTLTD Custom 6040000 MSFT 3000001)
> (XEN) ACPI: FACS BFEFFFC0, 0040
> (XEN) ACPI: BOOT BFEF122A, 0028 (r1 PTLTD $SBFTBL$ 6040000 LTP 1)
> (XEN) ACPI: APIC BFEF1194, 0096 (r1 PTLTD APIC 6040000 LTP 0)
> (XEN) ACPI: MCFG BFEF1158, 003C (r1 PTLTD $PCITBL$ 6040000 LTP 1)
> (XEN) ACPI: SRAT BFEF1028, 0130 (r2 VMWARE MEMPLUG 6040000 VMW 1)
> (XEN) ACPI: WAET BFEF1000, 0028 (r1 VMWARE VMW WAET 6040000 VMW 1)
> (XEN) System RAM: 6143MB (6291004kB)
> (XEN) Domain heap initialised
> (XEN) Processor #0 7:15 APIC version 21
> (XEN) Processor #2 7:15 APIC version 21
> (XEN) Processor #4 7:15 APIC version 21
> (XEN) Processor #6 7:15 APIC version 21
> (XEN) Processor #8 7:15 APIC version 21
> (XEN) Processor #10 7:15 APIC version 21
> (XEN) IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-23
> (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
> (XEN) Not enabling x2APIC: depends on iommu_supports_eim.
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) Detected 2299.474 MHz processor.
> (XEN) Initing memory sharing.
> (XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
> (XEN) I/O virtualisation disabled
> (XEN) ENABLING IO-APIC IRQs
> (XEN) -> Using new ACK method
> (XEN) Platform timer is 3.579MHz ACPI PM Timer
> (XEN) Allocated console ring of 16 KiB.
> (XEN) VMX: Supported advanced features:
> (XEN) - APIC TPR shadow
> (XEN) - Extended Page Tables (EPT)
> (XEN) - Virtual-Processor Identifiers (VPID)
> (XEN) - Virtual NMI
> (XEN) - MSR direct-access bitmap
> (XEN) - Unrestricted Guest
> (XEN) HVM: ASIDs enabled.
> (XEN) HVM: VMX enabled
> (XEN) HVM: Hardware Assisted Paging (HAP) not detected
> (XEN) Brought up 6 CPUs
> (XEN) Dom0 has maximum 600 PIRQs
> (XEN) *** LOADING DOMAIN 0 ***
> (XEN) Xen kernel: 64-bit, lsb, compat32
> (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x3000000
> (XEN) PHYSICAL MEMORY ARRANGEMENT:
> (XEN) Dom0 alloc.: 00000001b0000000->00000001b4000000 (504002 pages to be allocated)
> (XEN) Init. ramdisk: 00000001bf0c2000->00000001bffffe00
> (XEN) VIRTUAL MEMORY ARRANGEMENT:
> (XEN) Loaded kernel: ffffffff81000000->ffffffff83000000
> (XEN) Init. ramdisk: 0000000000000000->0000000000000000
> (XEN) Phys-Mach map: 0000008000000000->0000008000400000
> (XEN) Start info: ffffffff83000000->ffffffff830004b4
> (XEN) Page tables: ffffffff83001000->ffffffff8301e000
> (XEN) Boot stack: ffffffff8301e000->ffffffff8301f000
> (XEN) TOTAL: ffffffff80000000->ffffffff83400000
> (XEN) ENTRY ADDRESS: ffffffff8200d1f0
> (XEN) Dom0 has maximum 6 VCPUs
> (XEN) Std. Loglevel: Errors and warnings
> (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
> (XEN) Xen is relinquishing VGA console.
> (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
> (XEN) Freed 308kB init memory.
> mapping kernel into physical memory
> (XEN) traps.c:459:d0v0 Unhandled invalid opcode fault/trap [#6] on VCPU 0 [ec=0000]
> (XEN) domain_crash_sync called from entry.S: fault at ffff82d0802286c3 create_bounce_frame+0x12b/0x13a
> (XEN) Domain 0 (vcpu#0) crashed on cpu#0:
> (XEN) ----[ Xen-4.5.4-pre x86_64 debug=n Not tainted ]----
> (XEN) CPU: 0
> (XEN) RIP: e033:[<ffffffff81053cbd>]
> (XEN) RFLAGS: 0000000000000206 EM: 1 CONTEXT: pv guest (d0v0)
> (XEN) rax: 0000000000000022 rbx: 00000000ffffffff rcx: 0000000000000000
> (XEN) rdx: 0000000000000022 rsi: 0000000000000003 rdi: 0000000000000000
> (XEN) rbp: ffffffff81b67ea8 rsp: ffffffff81b67e68 r8: 0000000000000001
> (XEN) r9: 0000000000000001 r10: ffffffff81b67f20 r11: 6c61765f74617020
> (XEN) r12: 0000000000000000 r13: 0000000000000003 r14: 0000000000000000
> (XEN) r15: ffffffff81b67ebb cr0: 000000008005003b cr4: 00000000001526b0
> (XEN) cr3: 00000001b16eb000 cr2: 0000000000000000
> (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e02b cs: e033
> (XEN) Guest stack trace from rsp=ffffffff81b67e68:
> (XEN) 0000000000000000 6c61765f74617020 ffffffff81053cbd 000000010000e030
> (XEN) 0000000000010006 ffffffff81b67ea8 000000000000e02b ffffffff81b67f20
> (XEN) ffffffff81b67f10 ffffffff8105b339 55ffffff81b67f10 5520204355202043
> (XEN) 5520204355202043 5520204355202043 0020204355202043 0000000000000000
> (XEN) 0000000000000000 ffffffff81b67f38 0000000000000000 0000000000000000
> (XEN) 0000000000000000 ffffffff81b67ff0 ffffffff82010d0a 0000000000000000
> (XEN) 000306f200000000 fed8320300010800 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 ffffffff81b68008 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 00000000fffedb08
> (XEN) Domain 0 crashed: rebooting machine in 5 seconds.
> (XEN) Resetting with ACPI MEMORY or I/O RESET_REG.
>
> The crash occurs in pat_init_cache_modes(), called by
> xen_start_kernel(). The pat value from MSR_IA32_CR_PAT is 0.
> Strangely, the same kernel and Xen boot just fine on VMware Fusion
> 8.1.1, even though the MSR is 0 there as well.
>
> Anyway, guessing that it's pointless to call pat_init_cache_modes()
> when the CPU doesn't support PAT, I added a check for cpu_has_pat.
> This resolves the problem on ESXi and doesn't seem to break real
> hardware, though I'm not sure how to verify PAT functionality. So
> this is just an RFC.
Cc-ing maintainers.
>
> diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
> index 9a29803..209f680 100644
> --- a/arch/x86/xen/enlighten.c
> +++ b/arch/x86/xen/enlighten.c
> @@ -1633,8 +1633,12 @@ asmlinkage __visible void __init xen_start_kernel(void)
> * Modify the cache mode translation tables to match Xen's PAT
> * configuration.
> */
> - rdmsrl(MSR_IA32_CR_PAT, pat);
> - pat_init_cache_modes(pat);
> + if (cpu_has_pat) {
> + rdmsrl(MSR_IA32_CR_PAT, pat);
> + pat_init_cache_modes(pat);
> + } else {
> + xen_raw_console_write("CPU does not support PAT\n");
> + }
>
> /* keep using Xen gdt for now; no urgent need to change it */
>
>
> --Ed
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
next prev parent reply other threads:[~2016-05-23 14:15 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-05-20 23:58 PAT-related crash booting Linux 4.4 + Xen 4.5 on VMware ESXi Ed Swierk
2016-05-23 14:15 ` Konrad Rzeszutek Wilk [this message]
2016-05-23 20:13 ` Boris Ostrovsky
2016-05-23 22:52 ` Ed Swierk
2016-05-24 14:53 ` Kani, Toshimitsu
2016-05-24 15:25 ` Ed Swierk
2016-05-24 15:54 ` Boris Ostrovsky
2016-05-24 16:59 ` Kani, Toshimitsu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160523141523.GB9487@char.us.oracle.com \
--to=konrad.wilk@oracle.com \
--cc=boris.ostrovsky@oracle.com \
--cc=david.vrabel@citrix.com \
--cc=eswierk@skyportsystems.com \
--cc=jgross@suse.com \
--cc=xen-devel@lists.xensource.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).