Re: [BUG] (hvm)-virtualization doesn't work with xen 4.4

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 09/05/14 19:30, Michael Mair-Keimberger wrote:
> Hi Andrew,
> Hi List,
>
> Sorry, for this late reply!
> I didn't subscribe to the xen-devel mailing list and because of that i
> didn't got any reply. On [1] it's written the lists policy is to CC
> people so i though it's ok not to subscribe. It would be great if you
> could CC me next time.
>
> [1] http://wiki.xen.org/wiki/Reporting_Bugs_against_Xen
>
> Because of that i've just copied your answer into a new mail and send it
> directly to you + CC the list. Don't know if that is a proper way and i
> hope i don't mess up to much...
>
> About my bug:
>
>
>>> Hi List,
>>>
>>> This is a forwarded mail which i already sent to the xen-users mailing
>>> list, however it was suggested to open a BUG about my problem which
>>> seems to work via the xen-devel mailing list. :)
>>>
>>> Below is my description about my problem - at the end i've added some
>>> more information about my xen configuration. If you need some other
>>> information, please let me know :)
>>
>> Can you please post the complete xl dmesg from boot including one
>> attempted boot of the VM.  What you have pasted here is simply not
>> enough for us to help you with.  If you could boot xen with "loglvl=all
>> guest_loglvl=all" on the command line, that would help things.
>
> Hmm, i though i did include a complete xl dmesg in my last mail (though it
> was a bit far back at mail), however, below is another xl dmesg. I've
> also included my complete dmesg in case you need that too.
> xl dmesg was done after i tried to start the vm via "xl create
> /path/to/config"

Can you do that with "loglvl=all guest_loglvl=all" on the Xen command
line please.  It should produce a much larger log.

>
>
>> Xen 4.4.0
>> (XEN) Xen version 4.4.0 (@xonet.at) (x86_64-pc-linux-gnu-gcc (Gentoo
4.8.2 p1.3r1, pie-0.5.8r1) 4.8.2) debug=n Sat Apr 12 09:46:34 CEST 2014
>> (XEN) Latest ChangeSet:
>> (XEN) Bootloader: GRUB 2.02~beta1
>> (XEN) Command line: dom0_mem=8192M,max:8192M iommu=debug,verbose
xsave=1 dom0_max_vcpus=4 dom0_vcpus_pin ivrs_ioapic[8]=00:14.0
>> (XEN) Video information:
>> (XEN)  VGA is text mode 80x25, font 8x16
>> (XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
>> (XEN) Disc information:
>> (XEN)  Found 3 MBR signatures
>> (XEN)  Found 3 EDD information structures
>> (XEN) Xen-e820 RAM map:
>> (XEN)  0000000000000000 - 000000000009f800 (usable)
>> (XEN)  000000000009f800 - 00000000000a0000 (reserved)
>> (XEN)  00000000000f0000 - 0000000000100000 (reserved)
>> (XEN)  0000000000100000 - 00000000bfda0000 (usable)
>> (XEN)  00000000bfda0000 - 00000000bfdd1000 (ACPI NVS)
>> (XEN)  00000000bfdd1000 - 00000000bfe00000 (ACPI data)
>> (XEN)  00000000bfe00000 - 00000000bff00000 (reserved)
>> (XEN)  00000000e0000000 - 00000000f0000000 (reserved)
>> (XEN)  00000000fec00000 - 0000000100000000 (reserved)
>> (XEN)  0000000100000000 - 000000043f000000 (usable)
>> (XEN) ACPI: RSDP 000F6BF0, 0014 (r0 GBT   )
>> (XEN) ACPI: RSDT BFDD1000, 0050 (r1 GBT    GBTUACPI 42302E31 GBTU 
1010101)
>> (XEN) ACPI: FACP BFDD1080, 0074 (r1 GBT    GBTUACPI 42302E31 GBTU 
1010101)
>> (XEN) ACPI: DSDT BFDD1100, 7997 (r1 GBT    GBTUACPI     1000 MSFT 
3000000)
>> (XEN) ACPI: FACS BFDA0000, 0040
>> (XEN) ACPI: MSDM BFDD8B80, 0055 (r3 GBT    GBTUACPI 42302E31 GBTU 
1010101)
>> (XEN) ACPI: HPET BFDD8C00, 0038 (r1 GBT    GBTUACPI 42302E31
GBTU       98)
>> (XEN) ACPI: MCFG BFDD8C40, 003C (r1 GBT    GBTUACPI 42302E31 GBTU 
1010101)
>> (XEN) ACPI: EUDS BFDD8CC0, 0740 (r1 GBT                   
0             0)
>> (XEN) ACPI: MATS BFDD9400, 0034 (r1 GBT                   
0             0)
>> (XEN) ACPI: TAMG BFDD9470, 0182 (r1 GBT    GBT   B0 5455312E BG\x01\x01
53450101)
>> (XEN) ACPI: APIC BFDD8AC0, 00BC (r1 GBT    GBTUACPI 42302E31 GBTU 
1010101)
>> (XEN) ACPI: MATS BFDD9600, 66BB (r1        MATS RCM 80000001 INTL
20061109)
>> (XEN) ACPI: SSDT BFDDFD30, 1714 (r1 AMD    POWERNOW        1
AMD         1)
>> (XEN) ACPI: IVRS BFDE14C0, 0100 (r1  AMD     RD890S   202031
AMD         0)
>> (XEN) System RAM: 16365MB (16758012kB)
>> (XEN) Domain heap initialised
>> (XEN) Processor #0 5:2 APIC version 16
>> (XEN) Processor #1 5:2 APIC version 16
>> (XEN) Processor #2 5:2 APIC version 16
>> (XEN) Processor #3 5:2 APIC version 16
>> (XEN) Processor #4 5:2 APIC version 16
>> (XEN) Processor #5 5:2 APIC version 16
>> (XEN) Processor #6 5:2 APIC version 16
>> (XEN) Processor #7 5:2 APIC version 16
>> (XEN) IOAPIC[0]: apic_id 8, version 33, address 0xfec00000, GSI 0-23
>> (XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
>> (XEN) Using scheduler: SMP Credit Scheduler (credit)
>> (XEN) Detected 4026.919 MHz processor.
>> (XEN) Initing memory sharing.
>> (XEN) xstate_init: using cntxt_size: 0x3c0 and states: 0x4000000000000007
>> (XEN) AMD-Vi: IOMMU 0 Enabled.
>> (XEN) I/O virtualisation enabled
>> (XEN)  - Dom0 mode: Relaxed
>> (XEN) Interrupt remapping enabled
>> (XEN) ENABLING IO-APIC IRQs
>> (XEN)  -> Using new ACK method
>> (XEN) Platform timer is 14.318MHz HPET
>> (XEN) Allocated console ring of 16 KiB.
>> (XEN) HVM: ASIDs enabled.
>> (XEN) SVM: Supported advanced features:
>> (XEN)  - Nested Page Tables (NPT)
>> (XEN)  - Last Branch Record (LBR) Virtualisation
>> (XEN)  - Next-RIP Saved on #VMEXIT
>> (XEN)  - VMCB Clean Bits
>> (XEN)  - DecodeAssists
>> (XEN)  - Pause-Intercept Filter
>> (XEN)  - TSC Rate MSR
>> (XEN) HVM: SVM enabled
>> (XEN) HVM: Hardware Assisted Paging (HAP) detected
>> (XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
>> (XEN) spurious 8259A interrupt: IRQ7.
>> (XEN) CPU1: No irq handler for vector e7 (IRQ -2147483648)
>> (XEN) CPU2: No irq handler for vector e7 (IRQ -2147483648)
>> (XEN) CPU3: No irq handler for vector e7 (IRQ -2147483648)
>> (XEN) CPU4: No irq handler for vector e7 (IRQ -2147483648)
>> (XEN) CPU5: No irq handler for vector e7 (IRQ -2147483648)
>> (XEN) CPU6: No irq handler for vector e7 (IRQ -2147483648)
>> (XEN) Brought up 8 CPUs
>> (XEN) CPU7: No irq handler for vector e7 (IRQ -2147483648)

Certainly unrelated to your bug, but this would indicate some early
synchronisation issue between booting APs and inheriting global irq state.

>>
>> (XEN) *** LOADING DOMAIN 0 ***
>> (XEN)  Xen  kernel: 64-bit, lsb, compat32
>> (XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x1bd5000
>> (XEN) PHYSICAL MEMORY ARRANGEMENT:
>> (XEN)  Dom0 alloc.:   000000042c000000->0000000430000000 (2080768
pages to be allocated)
>> (XEN) VIRTUAL MEMORY ARRANGEMENT:
>> (XEN)  Loaded kernel: ffffffff81000000->ffffffff81bd5000
>> (XEN)  Init. ramdisk: ffffffff81bd5000->ffffffff81bd5000
>> (XEN)  Phys-Mach map: ffffffff81bd5000->ffffffff82bd5000
>> (XEN)  Start info:    ffffffff82bd5000->ffffffff82bd54b4
>> (XEN)  Page tables:   ffffffff82bd6000->ffffffff82bf1000
>> (XEN)  Boot stack:    ffffffff82bf1000->ffffffff82bf2000
>> (XEN)  TOTAL:         ffffffff80000000->ffffffff83000000
>> (XEN)  ENTRY ADDRESS: ffffffff816701f0
>> (XEN) Dom0 has maximum 4 VCPUs
>> (XEN) Scrubbing Free RAM:
................................................................................done.
>> (XEN) Initial low memory virq threshold set at 0x4000 pages.
>> (XEN) Std. Loglevel: Errors and warnings
>> (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
>> (XEN) Xen is relinquishing VGA console.
>> (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch
input to Xen)
>> (XEN) Freed 280kB init memory.
>> (XEN) traps.c:2514:d0 Domain attempted WRMSR 00000000c0010201 from
0x0000000000000000 to 0x000000000000ffff.
>> (XEN) mm.c:809: d0: Forcing read-only access to MFN e0002

This is curious, but probably unrelated to your bug.

>>
>> (XEN) io.c:204:d2 MMIO emulation failed @ 0008:ffff1f50: 32 00 00 00
00 00 00 00 36 99

Right, so this is a different instruction this time, and is 'xor
(%{r,e}ax),%al' this time, which could plausibly hit emulation if
%{r,e}ax is a bad pointer.

If you try to boot the guest several times, how does this particular
line change?


Can you set on_crash="preserve" for the vm, then also attach the result
of `xen-hvmctx $DOMID`

Thanks,

~Andrew