xen 3.2 various issues.

xen 3.2 various issues.

[Guillaume Rousse]

Hello again.

I'm currently testing xen 3.2 package released with mandriva 2008.1. 
However, I can't reuse my existing xen 3.1 HVM or PVM guest, nor install 
new HVM guests...

First, I can't launch any of my ISCSI-stored linux HVM guests. xm create 
return successfully, xm list show a guest without any state, and qemu-dm 
dies immediatly. Of course, vnc server doesn't run neither.

I can't find anything suspicious in the logs: no error message in 
xend.log, and qemu-dm only has:
xs_read(): vncpasswd get error. 
/vm/2a8782f0-638b-41d0-a3bf-1cb2b133735b/vncpasswd.
Could not parse VNC address

Setting an explicit VNC password remove the first message, not the second.

Second, I tried installing a new HVM guest through netboot, but the 
symptoms are the same as previously: qemu-dm fdying immediatly.

Third, I can launch an existing file-stored PVM guest, it boots, but 
halt before init is launched:
EXT2-fs warning (device hda1): ext2_fill_super: mounting ext3 filesystem 
as ext2
VFS: Mounted root (ext2 filesystem) readonly.
Freeing unused kernel memory: 172k freed

It feels like something is broken in the package, but I can't find what :/
-- 
Guillaume Rousse
Moyens Informatiques - INRIA Futurs
Tel: 01 69 35 69 62

Re: xen 3.2 various issues.

[Guillaume Rousse]

[-- Attachment #1: Type: text/plain, Size: 1352 bytes --]

Guillaume Rousse a écrit :
> Hello again.
> 
> I'm currently testing xen 3.2 package released with mandriva 2008.1. 
> However, I can't reuse my existing xen 3.1 HVM or PVM guest, nor install 
> new HVM guests...
I just updated to 3.2.1, the problem persists.

I'm joining console output, and xend log, when launching a PV host. I'm 
using the same kernel (linux-hg-2.6.18), for dom0 and domU.

xm list show the host is in b state:
[root@baobab xen]# xm list
Name                                        ID   Mem VCPUs      State 
Time(s)
Domain-0                                     0  7514     4     r----- 
  128.3
test                                         1   512     1     -b---- 
    2.9

According to the process table, xenconsole is apparently the parent of a 
defunt python process.

root      8442  0.0  0.0  20552   636 pts/0    Sl+  12:10   0:00  | 
   \_ /usr/lib64/xen/bin/xenconsole 1
root      8443  0.0  0.0      0     0 pts/0    Z+   12:10   0:00  | 
       \_ [python] <defunct>

And strace shows process 8442 hanging on read call:
[root@baobab xen]# strace -f -p 8442
Process 8548 attached with 2 threads - interrupt to quit
[pid  8442] select(7, [0 4 6], NULL, NULL, NULL <unfinished ...>
[pid  8548] read(3,

Any hint or other trace I could attempt ?
-- 
Guillaume Rousse
Moyens Informatiques - INRIA Futurs
Tel: 01 69 35 69 62

[-- Attachment #2: console --]
[-- Type: text/plain, Size: 3421 bytes --]

[root@baobab ~]# xm create -c test
Using config file "/etc/xen/test".
Started domain test
                   Bootdata ok (command line is root=/dev/hda1 ro 3)
Linux version 2.6.18.8-xen-3.2.1-2mdv (mandrake@deborah.mandriva.com) (gcc version 4.2.3 (4.2.3-6mnb1)) #1 SMP Fri May 23 12:36:08 EDT 2008
BIOS-provided physical RAM map:
 Xen: 0000000000000000 - 0000000020800000 (usable)
No mptable found.
Built 1 zonelists.  Total pages: 129820
Kernel command line: root=/dev/hda1 ro 3
Initializing CPU#0
PID hash table entries: 2048 (order: 11, 16384 bytes)
Xen reported: 2992.532 MHz processor.
Console: colour dummy device 80x25
Dentry cache hash table entries: 65536 (order: 7, 524288 bytes)
Inode-cache hash table entries: 32768 (order: 6, 262144 bytes)
Software IO TLB disabled
Memory: 510156k/532480k available (2019k kernel code, 14088k reserved, 876k data, 172k init)
Calibrating delay using timer specific routine.. 5987.88 BogoMIPS (lpj=29939420)
Security Framework v1.0.0 initialized
Capability LSM initialized
Mount-cache hash table entries: 256
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 4096K
CPU: Physical Processor ID: 3
CPU: Processor Core ID: 1
SMP alternatives: switching to UP code
Freeing SMP alternatives: 28k freed
Brought up 1 CPUs
migration_cost=0
NET: Registered protocol family 16
Brought up 1 CPUs
PCI: setting up Xen PCI frontend stub
ACPI: Interpreter disabled.
Linux Plug and Play Support v0.97 (c) Adam Belay
pnp: PnP ACPI: disabled
xen_mem: Initialising balloon driver.
PCI: System does not support PCI
PCI: System does not support PCI
NET: Registered protocol family 2
IP route cache hash table entries: 16384 (order: 5, 131072 bytes)
TCP established hash table entries: 65536 (order: 8, 1048576 bytes)
TCP bind hash table entries: 32768 (order: 7, 524288 bytes)
TCP: Hash tables configured (established 65536 bind 32768)
TCP reno registered
IA-32 Microcode Update Driver: v1.14a-xen <tigran@veritas.com>
audit: initializing netlink socket (disabled)
audit(1211969438.567:1): initialized
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
Initializing Cryptographic API
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered (default)
rtc: IRQ 8 is not free.
Non-volatile memory driver v1.2
RAMDISK driver initialized: 16 RAM disks of 16384K size 1024 blocksize
loop: loaded (max 8 devices)
Xen virtual console successfully installed as xvc0
Event-channel device installed.
netfront: Initialising virtual ethernet driver.
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 50MHz system bus speed for PIO modes; override with idebus=xx
PNP: No PS/2 controller found. Probing ports directly.
i8042.c: No controller found.
mice: PS/2 mouse device common for all mice
md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: bitmap version 4.39
NET: Registered protocol family 1
NET: Registered protocol family 17
xen-vbd: registered block device major 3
blkfront: hda1: barriers enabled
XENBUS: Device with no driver: device/console/0
BIOS EDD facility v0.16 2004-Jun-25, 0 devices found
EDD information not available.
md: Autodetecting RAID arrays.
md: autorun ...
md: ... autorun DONE.
EXT2-fs warning (device hda1): ext2_fill_super: mounting ext3 filesystem as ext2
VFS: Mounted root (ext2 filesystem) readonly.
Freeing unused kernel memory: 172k freed

[-- Attachment #3: xend.log --]
[-- Type: text/plain, Size: 8057 bytes --]

[2008-05-28 12:10:36 7985] DEBUG (XendDomainInfo:84) XendDomainInfo.create(['vm', ['name', 'test'], ['memory', 512], ['vcpus', 1], ['on_xend_start', 'ignore'], ['on_xend_stop', 'ignore'], ['image', ['linux', ['kernel', '/boot/vmlinuz-xen'], ['root', '/dev/hda1 ro'], ['args', '3']]], ['device', ['vbd', ['uname', 'phy:/dev/vg0/test'], ['dev', 'hda1'], ['mode', 'w']]], ['device', ['vif', ['bridge', 'services'], ['mac', 'aa:cc:dd:00:00:01']]]])
[2008-05-28 12:10:36 7985] DEBUG (XendDomainInfo:1618) XendDomainInfo.constructDomain
[2008-05-28 12:10:36 7985] DEBUG (balloon:132) Balloon: 131072 KiB free; need 2048; done.
[2008-05-28 12:10:36 7985] DEBUG (XendDomain:443) Adding Domain: 1
[2008-05-28 12:10:36 7985] DEBUG (XendDomainInfo:1703) XendDomainInfo.initDomain: 1 256
[2008-05-28 12:10:36 7985] DEBUG (XendDomainInfo:1738) _initDomain:shadow_memory=0x0, memory_static_max=0x20000000, memory_static_min=0x0.
[2008-05-28 12:10:36 7985] DEBUG (balloon:138) Balloon: 131064 KiB free; 0 to scrub; need 524288; retries: 20.
[2008-05-28 12:10:36 7985] DEBUG (balloon:153) Balloon: setting dom0 target to 7542 MiB.
[2008-05-28 12:10:36 7985] DEBUG (XendDomainInfo:693) Setting memory target of domain Domain-0 (0) to 7542 MiB.
[2008-05-28 12:10:37 7985] DEBUG (balloon:138) Balloon: 354296 KiB free; 0 to scrub; need 524288; retries: 20.
[2008-05-28 12:10:37 7985] DEBUG (balloon:153) Balloon: setting dom0 target to 7514 MiB.
[2008-05-28 12:10:37 7985] DEBUG (XendDomainInfo:693) Setting memory target of domain Domain-0 (0) to 7514 MiB.
[2008-05-28 12:10:37 7985] DEBUG (balloon:132) Balloon: 553700 KiB free; need 524288; done.
[2008-05-28 12:10:37 7985] INFO (image:139) buildDomain os=linux dom=1 vcpus=1
[2008-05-28 12:10:37 7985] DEBUG (image:363) domid          = 1
[2008-05-28 12:10:37 7985] DEBUG (image:364) memsize        = 512
[2008-05-28 12:10:37 7985] DEBUG (image:365) image          = /boot/vmlinuz-xen
[2008-05-28 12:10:37 7985] DEBUG (image:366) store_evtchn   = 1
[2008-05-28 12:10:37 7985] DEBUG (image:367) console_evtchn = 2
[2008-05-28 12:10:37 7985] DEBUG (image:368) cmdline        = root=/dev/hda1 ro 3
[2008-05-28 12:10:37 7985] DEBUG (image:369) ramdisk        = 
[2008-05-28 12:10:37 7985] DEBUG (image:370) vcpus          = 1
[2008-05-28 12:10:37 7985] DEBUG (image:371) features       = 
[2008-05-28 12:10:37 7985] INFO (XendDomainInfo:1514) createDevice: vbd : {'uuid': '04b8b601-dac1-dd62-fdbd-8b63eb479958', 'bootable': 1, 'driver': 'paravirtualised', 'dev': 'hda1', 'uname': 'phy:/dev/vg0/test', 'mode': 'w'}
[2008-05-28 12:10:37 7985] DEBUG (DevController:118) DevController: writing {'virtual-device': '769', 'device-type': 'disk', 'protocol': 'x86_64-abi', 'backend-id': '0', 'state': '1', 'backend': '/local/domain/0/backend/vbd/1/769'} to /local/domain/1/device/vbd/769.
[2008-05-28 12:10:37 7985] DEBUG (DevController:120) DevController: writing {'domain': 'test', 'frontend': '/local/domain/1/device/vbd/769', 'uuid': '04b8b601-dac1-dd62-fdbd-8b63eb479958', 'dev': 'hda1', 'state': '1', 'params': '/dev/vg0/test', 'mode': 'w', 'online': '1', 'frontend-id': '1', 'type': 'phy'} to /local/domain/0/backend/vbd/1/769.
[2008-05-28 12:10:37 7985] INFO (XendDomainInfo:1514) createDevice: vif : {'bridge': 'services', 'mac': 'aa:cc:dd:00:00:01', 'uuid': 'bba6726a-f00c-ac0b-c3df-d2e7789cf751'}
[2008-05-28 12:10:37 7985] DEBUG (DevController:118) DevController: writing {'mac': 'aa:cc:dd:00:00:01', 'handle': '0', 'protocol': 'x86_64-abi', 'backend-id': '0', 'state': '1', 'backend': '/local/domain/0/backend/vif/1/0'} to /local/domain/1/device/vif/0.
[2008-05-28 12:10:37 7985] DEBUG (DevController:120) DevController: writing {'bridge': 'services', 'domain': 'test', 'handle': '0', 'uuid': 'bba6726a-f00c-ac0b-c3df-d2e7789cf751', 'script': '/etc/xen/scripts/vif-bridge', 'mac': 'aa:cc:dd:00:00:01', 'frontend-id': '1', 'state': '1', 'online': '1', 'frontend': '/local/domain/1/device/vif/0'} to /local/domain/0/backend/vif/1/0.
[2008-05-28 12:10:37 7985] DEBUG (XendDomainInfo:2195) Storing VM details: {'on_xend_stop': 'ignore', 'shadow_memory': '0', 'uuid': '025dd05c-d006-f1f8-0a0f-8e5de6b3be03', 'on_reboot': 'restart', 'start_time': '1211969437.78', 'on_poweroff': 'destroy', 'on_xend_start': 'ignore', 'on_crash': 'restart', 'xend/restart_count': '0', 'vcpus': '1', 'vcpu_avail': '1', 'image': "(linux (kernel /boot/vmlinuz-xen) (args 'root=/dev/hda1 ro 3') (notes (FEATURES 'writable_page_tables|writable_descriptor_tables|auto_translated_physmap|pae_pgdir_above_4gb|supervisor_mode_kernel') (VIRT_BASE 18446744071562067968) (GUEST_VERSION 2.6) (PADDR_OFFSET 18446744071562067968) (GUEST_OS linux) (HYPERCALL_PAGE 18446744071564189696) (LOADER generic) (SUSPEND_CANCEL 1) (ENTRY 18446744071564165120) (XEN_VERSION xen-3.0)))", 'name': 'test'}
[2008-05-28 12:10:37 7985] DEBUG (XendDomainInfo:1008) Storing domain details: {'console/ring-ref': '2207622', 'image/entry': '18446744071564165120', 'console/port': '2', 'store/ring-ref': '2207623', 'image/loader': 'generic', 'vm': '/vm/025dd05c-d006-f1f8-0a0f-8e5de6b3be03', 'control/platform-feature-multiprocessor-suspend': '1', 'image/guest-os': 'linux', 'image/features/writable-descriptor-tables': '1', 'image/virt-base': '18446744071562067968', 'memory/target': '524288', 'image/guest-version': '2.6', 'image/features/supervisor-mode-kernel': '1', 'console/limit': '1048576', 'image/paddr-offset': '18446744071562067968', 'image/hypercall-page': '18446744071564189696', 'image/suspend-cancel': '1', 'cpu/0/availability': 'online', 'image/features/pae-pgdir-above-4gb': '1', 'image/features/writable-page-tables': '1', 'console/type': 'xenconsoled', 'image/features/auto-translated-physmap': '1', 'name': 'test', 'domid': '1', 'image/xen-version': 'xen-3.0', 'store/port': '1'}
[2008-05-28 12:10:37 7985] DEBUG (DevController:118) DevController: writing {'protocol': 'x86_64-abi', 'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/console/1/0'} to /local/domain/1/device/console/0.
[2008-05-28 12:10:37 7985] DEBUG (DevController:120) DevController: writing {'domain': 'test', 'frontend': '/local/domain/1/device/console/0', 'uuid': 'cbbaee82-bda0-621d-671f-a94174f40b3b', 'frontend-id': '1', 'state': '1', 'location': '2', 'online': '1', 'protocol': 'vt100'} to /local/domain/0/backend/console/1/0.
[2008-05-28 12:10:37 7985] DEBUG (XendDomainInfo:1092) XendDomainInfo.handleShutdownWatch
[2008-05-28 12:10:37 7985] DEBUG (DevController:151) Waiting for devices vif.
[2008-05-28 12:10:37 7985] DEBUG (DevController:156) Waiting for 0.
[2008-05-28 12:10:37 7985] DEBUG (DevController:595) hotplugStatusCallback /local/domain/0/backend/vif/1/0/hotplug-status.
[2008-05-28 12:10:38 7985] DEBUG (DevController:595) hotplugStatusCallback /local/domain/0/backend/vif/1/0/hotplug-status.
[2008-05-28 12:10:38 7985] DEBUG (DevController:609) hotplugStatusCallback 1.
[2008-05-28 12:10:38 7985] DEBUG (DevController:151) Waiting for devices vbd.
[2008-05-28 12:10:38 7985] DEBUG (DevController:156) Waiting for 769.
[2008-05-28 12:10:38 7985] DEBUG (DevController:595) hotplugStatusCallback /local/domain/0/backend/vbd/1/769/hotplug-status.
[2008-05-28 12:10:38 7985] DEBUG (DevController:609) hotplugStatusCallback 1.
[2008-05-28 12:10:38 7985] DEBUG (DevController:151) Waiting for devices irq.
[2008-05-28 12:10:38 7985] DEBUG (DevController:151) Waiting for devices vkbd.
[2008-05-28 12:10:38 7985] DEBUG (DevController:151) Waiting for devices vfb.
[2008-05-28 12:10:38 7985] DEBUG (DevController:151) Waiting for devices console.
[2008-05-28 12:10:38 7985] DEBUG (DevController:156) Waiting for 0.
[2008-05-28 12:10:38 7985] DEBUG (DevController:151) Waiting for devices pci.
[2008-05-28 12:10:38 7985] DEBUG (DevController:151) Waiting for devices ioports.
[2008-05-28 12:10:38 7985] DEBUG (DevController:151) Waiting for devices tap.
[2008-05-28 12:10:38 7985] DEBUG (DevController:151) Waiting for devices vtpm.
[2008-05-28 12:10:38 7985] INFO (XendDomain:1165) Domain test (1) unpaused.

[-- Attachment #4: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: xen 3.2 various issues.

[Deuce]

Are you building Xen as 32bit?  I was having issues with qemu-dm dying after similar VNC password messages logged.  Switching to 32bit PAE builds resolved the qemu-dm crashes.

This was with an xen-unstable post 3.2.  I don't know if 3.2 itself is affected by the same problem.

Jason

----- Original Message ----
> From: Guillaume Rousse <Guillaume.Rousse@inria.fr>
> To: xen-devel <xen-devel@lists.xensource.com>
> Sent: Wednesday, May 21, 2008 10:19:47 AM
> Subject: [Xen-devel] xen 3.2 various issues.
> 
> Hello again.
> 
> I'm currently testing xen 3.2 package released with mandriva 2008.1. 
> However, I can't reuse my existing xen 3.1 HVM or PVM guest, nor install 
> new HVM guests...
> 
> First, I can't launch any of my ISCSI-stored linux HVM guests. xm create 
> return successfully, xm list show a guest without any state, and qemu-dm 
> dies immediatly. Of course, vnc server doesn't run neither.
> 
> I can't find anything suspicious in the logs: no error message in 
> xend.log, and qemu-dm only has:
> xs_read(): vncpasswd get error. 
> /vm/2a8782f0-638b-41d0-a3bf-1cb2b133735b/vncpasswd.
> Could not parse VNC address
> 
> Setting an explicit VNC password remove the first message, not the second.
> 
> Second, I tried installing a new HVM guest through netboot, but the 
> symptoms are the same as previously: qemu-dm fdying immediatly.
> 
> Third, I can launch an existing file-stored PVM guest, it boots, but 
> halt before init is launched:
> EXT2-fs warning (device hda1): ext2_fill_super: mounting ext3 filesystem 
> as ext2
> VFS: Mounted root (ext2 filesystem) readonly.
> Freeing unused kernel memory: 172k freed
> 
> It feels like something is broken in the package, but I can't find what :/
> -- 
> Guillaume Rousse
> Moyens Informatiques - INRIA Futurs
> Tel: 01 69 35 69 62
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: xen 3.2 various issues.

[Guillaume Rousse]

Deuce a écrit :
> Are you building Xen as 32bit?  I was having issues with qemu-dm dying after similar VNC password messages logged.  Switching to 32bit PAE builds resolved the qemu-dm crashes.
This happen on a 64 bit host. I don't have any available 32bit host to test.

Strictly speaking, I'm not building xen, rather using official 
distribution packages.
-- 
Guillaume Rousse
Moyens Informatiques - INRIA Futurs
Tel: 01 69 35 69 62