Report a BUG in libxl_dom.c

Report a BUG in libxl_dom.c

[rwxybh]

[-- Attachment #1.1: Type: text/plain, Size: 1167 bytes --]

Hi all!

I think there is a bug in liibxl_dom.c
for this function -> libxl__domain_resume_device_model

976 int libxl__domain_resume_device_model(libxl__gc *gc, uint32_t domid)
977 {
978 
979     switch (libxl__device_model_version_running(gc, domid)) {
980     case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN_TRADITIONAL: {
981         libxl__qemu_traditional_cmd(gc, domid, "continue");
982         libxl__wait_for_device_model(gc, domid, "running", NULL, NULL, NULL);
983         break;
984     }
985     case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN:
986         if (libxl__qmp_resume(gc, domid))
987             return ERROR_FAIL;
        break;
988     default:
989         return ERROR_INVAL;
990     }
991 
992     return 0;
993 }

we should add "break" between line 987 & 988

Another problem:

The guest hung after  cmd "xl save -c domid /fpath" with PV driver
if we remove/disable the pv driver, the function works well

Test ENV:
host: xen 4.3 + upstream qemu dm
dom0: Oracle UEK kernel
guest: win2k3 SP2 & centos 6.4

Additional
There is no valuable debug info in xl dmesg, /var/log/xen*

Any Suggestion?? 






rwxybh

[-- Attachment #1.2: Type: text/html, Size: 3544 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Report a BUG in libxl_dom.c

[Ian Campbell]

On Mon, 2013-07-29 at 09:25 +0800, rwxybh wrote:
> we should add "break" between line 987 & 988

Yes, it appears so. In cases where we deliberately fallthrough a switch
we should leave a comment. I don't think that is the case here though.

Please could you submit this as a patch.
http://wiki.xen.org/wiki/Submitting_Xen_Patches has some guidance on how
to do that.

>  
> Another problem:
>  
> The guest hung after  cmd "xl save -c domid /fpath" with PV driver
> if we remove/disable the pv driver, the function works well
>  
> Test ENV:
> host: xen 4.3 + upstream qemu dm
> dom0: Oracle UEK kernel
> guest: win2k3 SP2 & centos 6.4
>  
> Additional
> There is no valuable debug info in xl dmesg, /var/log/xen*

Perhaps something in the guest console? It seems odd that it should
impact both win2k3 and centos 6.4, they are very different beasts.

Which PV drivers do you use on win2k3?

Also please include the logs even if they don't look valuable to you --
someone might spot something you have missed.

Oh, wait. Why are you using "xl save -c"? That will do a checkpointed
save. I'm not sure many PVHVM drivers will support that. Does it work
with a plain "xl save"?

Ian.

Re: Report a BUG in libxl_dom.c

[rwxybh]

[-- Attachment #1.1: Type: text/plain, Size: 12520 bytes --]

Thanks for your reply!


Test ENV
xen 4.3
dom0 3.0
guest os centos6.2

execute command : xl save -c 14 ~/mem.img

The debug message is here!

xl dmesg
...
(XEN) HVM14: HVM Loader
(XEN) HVM14: Detected Xen v4.3.0
(XEN) HVM14: Xenbus rings @0xfeffc000, event channel 10
(XEN) HVM14: System requested SeaBIOS
(XEN) HVM14: CPU speed is 2133 MHz
(XEN) HVM14: Relocating guest memory for lowmem MMIO space disabled
(XEN) irq.c:270: Dom14 PCI link 0 changed 0 -> 5
(XEN) HVM14: PCI-ISA link 0 routed to IRQ5
(XEN) irq.c:270: Dom14 PCI link 1 changed 0 -> 10
(XEN) HVM14: PCI-ISA link 1 routed to IRQ10
(XEN) irq.c:270: Dom14 PCI link 2 changed 0 -> 11
(XEN) HVM14: PCI-ISA link 2 routed to IRQ11
(XEN) irq.c:270: Dom14 PCI link 3 changed 0 -> 5
(XEN) HVM14: PCI-ISA link 3 routed to IRQ5
(XEN) HVM14: pci dev 01:2 INTD->IRQ5
(XEN) HVM14: pci dev 01:3 INTA->IRQ10
(XEN) HVM14: pci dev 03:0 INTA->IRQ5
(XEN) HVM14: pci dev 04:0 INTA->IRQ5
(XEN) HVM14: pci dev 1d:0 INTA->IRQ10
(XEN) HVM14: pci dev 1d:1 INTB->IRQ11
(XEN) HVM14: pci dev 1d:2 INTC->IRQ5
(XEN) HVM14: pci dev 1d:7 INTD->IRQ5
(XEN) HVM14: No RAM in high memory; setting high_mem resource base to 100000000
(XEN) HVM14: pci dev 02:0 bar 10 size 001000000: 0f0000008
(XEN) HVM14: pci dev 03:0 bar 14 size 001000000: 0f1000008
(XEN) HVM14: pci dev 04:0 bar 10 size 000020000: 0f2000000
(XEN) HVM14: pci dev 04:0 bar 30 size 000020000: 0f2020000
(XEN) HVM14: pci dev 02:0 bar 30 size 000010000: 0f2040000
(XEN) HVM14: pci dev 02:0 bar 18 size 000001000: 0f2050000
(XEN) HVM14: pci dev 1d:7 bar 10 size 000001000: 0f2051000
(XEN) HVM14: pci dev 03:0 bar 10 size 000000100: 00000c001
(XEN) HVM14: pci dev 04:0 bar 14 size 000000040: 00000c101
(XEN) HVM14: pci dev 01:2 bar 20 size 000000020: 00000c141
(XEN) HVM14: pci dev 1d:0 bar 20 size 000000020: 00000c161
(XEN) HVM14: pci dev 1d:1 bar 20 size 000000020: 00000c181
(XEN) HVM14: pci dev 1d:2 bar 20 size 000000020: 00000c1a1
(XEN) HVM14: pci dev 01:1 bar 20 size 000000010: 00000c1c1
(XEN) HVM14: Multiprocessor initialisation:
(XEN) HVM14:  - CPU0 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM14:  - CPU1 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM14:  - CPU2 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM14:  - CPU3 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM14:  - CPU4 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM14:  - CPU5 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM14:  - CPU6 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM14:  - CPU7 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM14: Testing HVM environment:
(XEN) HVM14:  - REP INSB across page boundaries ... passed
(XEN) HVM14:  - GS base MSRs and SWAPGS ... passed
(XEN) HVM14: Passed 2 of 2 tests
(XEN) HVM14: Writing SMBIOS tables ...
(XEN) HVM14: Loading SeaBIOS ...
(XEN) HVM14: Creating MP tables ...
(XEN) HVM14: Loading ACPI ...
(XEN) HVM14: vm86 TSS at fc00a200
(XEN) HVM14: BIOS map:
(XEN) HVM14:  10000-100d3: Scratch space
(XEN) HVM14:  e0000-fffff: Main BIOS
(XEN) HVM14: E820 table:
(XEN) HVM14:  [00]: 00000000:00000000 - 00000000:000a0000: RAM
(XEN) HVM14:  HOLE: 00000000:000a0000 - 00000000:000e0000
(XEN) HVM14:  [01]: 00000000:000e0000 - 00000000:00100000: RESERVED
(XEN) HVM14:  [02]: 00000000:00100000 - 00000000:7c000000: RAM
(XEN) HVM14:  HOLE: 00000000:7c000000 - 00000000:fc000000
(XEN) HVM14:  [03]: 00000000:fc000000 - 00000001:00000000: RESERVED
(XEN) HVM14: Invoking SeaBIOS ...
(XEN) HVM14: SeaBIOS (version ?-20130725_191611-xentest)
(XEN) HVM14: 
(XEN) HVM14: Found Xen hypervisor signature at 40000000
(XEN) HVM14: xen: copy e820...
(XEN) HVM14: Ram Size=0x7c000000 (0x0000000000000000 high)
(XEN) HVM14: Relocating low data from 0x000e0db0 to 0x000ef790 (size 2156)
(XEN) HVM14: Relocating init from 0x000e161c to 0x7bfdfff0 (size 65252)
(XEN) HVM14: CPU Mhz=2134
(XEN) HVM14: Found 12 PCI devices (max PCI bus is 00)
(XEN) HVM14: Allocated Xen hypercall page at 7bfff000
(XEN) HVM14: Detected Xen v4.3.0
(XEN) HVM14: xen: copy BIOS tables...
(XEN) HVM14: Copying SMBIOS entry point from 0x00010010 to 0x000fdb10
(XEN) HVM14: Copying MPTABLE from 0xfc001240/fc001250 to 0x000fd9a0
(XEN) HVM14: Copying PIR from 0x00010030 to 0x000fd920
(XEN) HVM14: Copying ACPI RSDP from 0x000100b0 to 0x000fd8f0
(XEN) HVM14: Scan for VGA option rom
(XEN) HVM14: Running option rom at c000:0003
(XEN) stdvga.c:147:d14 entering stdvga and caching modes
(XEN) HVM14: Turning on vga text mode console
(XEN) HVM14: SeaBIOS (version ?-20130725_191611-xentest)
(XEN) HVM14: 
(XEN) HVM14: UHCI init on dev 00:01.2 (io=c140)
(XEN) HVM14: EHCI init on dev 00:1d.7 (regs=0xf2051020)
(XEN) HVM14: Found 1 lpt ports
(XEN) HVM14: Found 1 serial ports
(XEN) HVM14: ATA controller 1 at 1f0/3f4/0 (irq 14 dev 9)
(XEN) HVM14: ATA controller 2 at 170/374/0 (irq 15 dev 9)
(XEN) HVM14: ata0-0: QEMU HARDDISK ATA-7 Hard-Disk (10240 MiBytes)
(XEN) HVM14: Searching bootorder for: /pci@i0cf8/*@1,1/drive@0/disk@0
(XEN) HVM14: DVD/CD [ata1-0: QEMU DVD-ROM ATAPI-4 DVD/CD]
(XEN) HVM14: Searching bootorder for: /pci@i0cf8/*@1,1/drive@1/disk@0
(XEN) HVM14: PS2 keyboard initialized
(XEN) HVM14: All threads complete.
(XEN) HVM14: Scan for option roms
(XEN) HVM14: Running option rom at ca00:0003
(XEN) HVM14: pmm call arg1=1
(XEN) HVM14: pmm call arg1=0
(XEN) HVM14: pmm call arg1=1
(XEN) HVM14: pmm call arg1=0
(XEN) HVM14: Searching bootorder for: /pci@i0cf8/*@4
(XEN) HVM14: Press F12 for boot menu.
(XEN) HVM14: 
(XEN) HVM14: drive 0x000fd8a0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 s=20971520
(XEN) HVM14: 
(XEN) HVM14: Space available for UMB: 000cb000-000ee800
(XEN) HVM14: Returned 61440 bytes of ZoneHigh
(XEN) HVM14: e820 map has 6 items:
(XEN) HVM14:   0: 0000000000000000 - 000000000009fc00 = 1 RAM
(XEN) HVM14:   1: 000000000009fc00 - 00000000000a0000 = 2 RESERVED
(XEN) HVM14:   2: 00000000000f0000 - 0000000000100000 = 2 RESERVED
(XEN) HVM14:   3: 0000000000100000 - 000000007bfff000 = 1 RAM
(XEN) HVM14:   4: 000000007bfff000 - 000000007c000000 = 2 RESERVED
(XEN) HVM14:   5: 00000000fc000000 - 0000000100000000 = 2 RESERVED
(XEN) HVM14: enter handle_19:
(XEN) HVM14:   NULL
(XEN) HVM14: Booting from Hard Disk...
(XEN) HVM14: Booting from 0000:7c00
(XEN) irq.c:375: Dom14 callback via changed to Direct Vector 0xe9
(XEN) irq.c:270: Dom14 PCI link 0 changed 5 -> 0
(XEN) irq.c:270: Dom14 PCI link 1 changed 10 -> 0
(XEN) irq.c:270: Dom14 PCI link 2 changed 11 -> 0
(XEN) irq.c:270: Dom14 PCI link 3 changed 5 -> 0
(XEN) HVM14 save: CPU
(XEN) HVM14 save: PIC
(XEN) HVM14 save: IOAPIC
(XEN) HVM14 save: LAPIC
(XEN) HVM14 save: LAPIC_REGS
(XEN) HVM14 save: PCI_IRQ
(XEN) HVM14 save: ISA_IRQ
(XEN) HVM14 save: PCI_LINK
(XEN) HVM14 save: PIT
(XEN) HVM14 save: RTC
(XEN) HVM14 save: HPET
(XEN) HVM14 save: PMTIMER
(XEN) HVM14 save: MTRR
(XEN) HVM14 save: VIRIDIAN_DOMAIN
(XEN) HVM14 save: CPU_XSAVE
(XEN) HVM14 save: VIRIDIAN_VCPU
(XEN) HVM14 save: VMCE_VCPU
(XEN) HVM14 save: TSC_ADJUST
(XEN) irq.c:375: Dom14 callback via changed to Direct Vector 0xe9


qemu-dm-**.log
xc: error: linux_gnttab_set_max_grants: ioctl SET_MAX_GRANTS failed (22 = Invalid argument): Internal error
xen be: qdisk-51712: xc_gnttab_set_max_grants failed: Invalid argument
xc: error: linux_gnttab_set_max_grants: ioctl SET_MAX_GRANTS failed (22 = Invalid argument): Internal error
xen be: qdisk-51744: xc_gnttab_set_max_grants failed: Invalid argument
(/usr/sbin/xl:5152): Spice-Warning **: reds.c:2093:reds_handle_ticket: Invalid password
main_channel_link: add main channel client
main_channel_handle_parsed: net test: latency 4.814000 ms, bitrate 10334248 bps (9.855507 Mbps) LOW BANDWIDTH
qemu-system-i386: usbredirparser: Peer version: spice-gtk 0.19, using 64-bits ids
qemu-system-i386: usbredirparser: Peer version: spice-gtk 0.19, using 64-bits ids
qemu-system-i386: usbredirparser: Peer version: spice-gtk 0.19, using 64-bits ids
inputs_connect: inputs channel client create
red_dispatcher_set_cursor_peer: 
(/usr/sbin/xl:5152): Spice-Warning **: reds.c:2093:reds_handle_ticket: Invalid password
red_client_destroy: destroy client with #channels 7
red_dispatcher_disconnect_cursor_peer: 
red_channel_client_disconnect: 0x7f1f8829f490 (channel 0x7f1f8821d670 type 4 id 0)
red_channel_client_disconnect: 0x7f1f8829f490 (channel 0x7f1f8821d670 type 4 id 0)
red_channel_client_disconnect: 0x7f1fa71ec080 (channel 0x7f1fa6e48dc0 type 3 id 0)
red_channel_client_disconnect: 0x7f1fa71ec080 (channel 0x7f1fa6e48dc0 type 3 id 0)
red_channel_client_disconnect: 0x7f1fa71d7e40 (channel 0x7f1fa6ec3010 type 9 id 0)
red_channel_client_disconnect: 0x7f1fa71d7e40 (channel 0x7f1fa6ec3010 type 9 id 0)
red_channel_client_disconnect: 0x7f1fa71c3c00 (channel 0x7f1fa6eda3d0 type 9 id 1)
red_channel_client_disconnect: 0x7f1fa71c3c00 (channel 0x7f1fa6eda3d0 type 9 id 1)
red_dispatcher_disconnect_display_peer: 
red_channel_client_disconnect: 0x7f1f88243520 (channel 0x7f1f8821d0b0 type 2 id 0)
red_channel_client_disconnect: 0x7f1f88243520 (channel 0x7f1f8821d0b0 type 2 id 0)
red_channel_client_disconnect: 0x7f1fa71a9bd0 (channel 0x7f1fa6edce40 type 9 id 2)
red_channel_client_disconnect: 0x7f1fa71a9bd0 (channel 0x7f1fa6edce40 type 9 id 2)
red_channel_client_disconnect: 0x7f1fa71a5010 (channel 0x7f1fa6e3d880 type 1 id 0)
main_channel_client_on_disconnect: rcc=0x7f1fa71a5010
red_channel_client_disconnect: 0x7f1fa71a5010 (channel 0x7f1fa6e3d880 type 1 id 0)
main_channel_link: add main channel client
main_channel_handle_parsed: net test: latency 9.189000 ms, bitrate 19599399 bps (18.691443 Mbps)
qemu-system-i386: usbredirparser: Peer version: spice-gtk 0.19, using 64-bits ids
qemu-system-i386: usbredirparser: Peer version: spice-gtk 0.19, using 64-bits ids
inputs_connect: inputs channel client create
qemu-system-i386: usbredirparser: Peer version: spice-gtk 0.19, using 64-bits ids
red_dispatcher_set_cursor_peer: 


xl-**.log


Waiting for domain centos62 (domid 14) to die [pid 5282]
Domain 14 has shut down, reason code 2 0x2
Domain has suspended.
Done. Exiting now

Some useful message form guest os (centos 62): /var/log/message


...

Jul 29 17:57:06 localhost kernel: Xen HVM callback vector for event delivery is enabled
Jul 29 17:57:08 localhost kernel: irq 23: nobody cared (try booting with the "irqpoll" option)
Jul 29 17:57:08 localhost kernel: Pid: 13, comm: xenwatch Not tainted 2.6.32-220.el6.x86_64 #1
Jul 29 17:57:08 localhost kernel: Call Trace:
Jul 29 17:57:08 localhost kernel: <IRQ>  [<ffffffff810db47b>] ? __report_bad_irq+0x2b/0xa0
Jul 29 17:57:08 localhost kernel: [<ffffffff810db67c>] ? note_interrupt+0x18c/0x1d0
Jul 29 17:57:08 localhost kernel: [<ffffffff8102da59>] ? ack_apic_level+0x79/0x1b0
...





rwxybh

From: Ian Campbell
Date: 2013-07-29 18:18
To: rwxybh
CC: xen-devel
Subject: Re: [Xen-devel] Report a BUG in libxl_dom.c
On Mon, 2013-07-29 at 09:25 +0800, rwxybh wrote:
> we should add "break" between line 987 & 988

Yes, it appears so. In cases where we deliberately fallthrough a switch
we should leave a comment. I don't think that is the case here though.

Please could you submit this as a patch.
http://wiki.xen.org/wiki/Submitting_Xen_Patches has some guidance on how
to do that.

>  
> Another problem:
>  
> The guest hung after  cmd "xl save -c domid /fpath" with PV driver
> if we remove/disable the pv driver, the function works well
>  
> Test ENV:
> host: xen 4.3 + upstream qemu dm
> dom0: Oracle UEK kernel
> guest: win2k3 SP2 & centos 6.4
>  
> Additional
> There is no valuable debug info in xl dmesg, /var/log/xen*

Perhaps something in the guest console? It seems odd that it should
impact both win2k3 and centos 6.4, they are very different beasts.

Which PV drivers do you use on win2k3?

Also please include the logs even if they don't look valuable to you --
someone might spot something you have missed.

Oh, wait. Why are you using "xl save -c"? That will do a checkpointed
save. I'm not sure many PVHVM drivers will support that. Does it work
with a plain "xl save"?

Ian.



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[-- Attachment #1.2: Type: text/html, Size: 17519 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Report a BUG in libxl_dom.c

[Ian Campbell]

On Mon, 2013-07-29 at 18:41 +0800, rwxybh wrote:
> Thanks for your reply!
>  
>  
> Test ENV
> xen 4.3
> dom0 3.0
> guest os centos6.2
>  
> execute command : xl save -c 14 ~/mem.img
>  [...]
> 
> Some useful message form guest os (centos 62): /var/log/message

Thanks. It appears that PVHVM kernels are simply not able to deal with
checkpointed suspend. I've not noticed any work going on in order to
implement this so if you are interested in this functionality it is
likely to be a case of getting stuck into the kernel side code.

Ian.
>  
>  
> ...
>  
> Jul 29 17:57:06 localhost kernel: Xen HVM callback vector for event
> delivery is enabled
> Jul 29 17:57:08 localhost kernel: irq 23: nobody cared (try booting
> with the "irqpoll" option)
> Jul 29 17:57:08 localhost kernel: Pid: 13, comm: xenwatch Not tainted
> 2.6.32-220.el6.x86_64 #1
> Jul 29 17:57:08 localhost kernel: Call Trace:
> Jul 29 17:57:08 localhost kernel: <IRQ>  [<ffffffff810db47b>] ?
> __report_bad_irq+0x2b/0xa0
> Jul 29 17:57:08 localhost kernel: [<ffffffff810db67c>] ?
> note_interrupt+0x18c/0x1d0
> Jul 29 17:57:08 localhost kernel: [<ffffffff8102da59>] ?
> ack_apic_level+0x79/0x1b0
> ...
>