Re: [PATCH] qemu-xen-3.4-testing: Fix read-only image file handling

[Michal Novotny <minovotn@redhat.com>]

Well,
it doesn't seem to be that easy because I can't boot any of the guests. 
No matter whether HVM or PV.

For PV it's connected with the hotplugging:

# xm create rhel5-32pv
Using config file "./rhel5-32pv".
Error: Device 0 (vif) could not be connected. Hotplug scripts not working.

fragment from xend.log:

[2010-06-07 17:30:30 6690] DEBUG (XendDomainInfo:103) 
XendDomainInfo.create(['vm', ['name', 'rhel5-32pv'], ['memory', 512], 
['maxmem', 512], ['on_poweroff', 'destroy'], ['on_reboot', 'restart'], 
['on_crash', 'restart'], ['on_xend_start', 'ignore'], ['on_xend_stop', 
'ignore'], ['vcpu_avail', 3], ['vcpus', 4], ['uuid', 
'731497a6-2c62-8cfb-0afb-4e1ff7bde24d'], ['oos', 1], ['bootloader', 
'/usr/bin/pygrub'], ['bootloader_args', '-q'], ['image', ['linux', 
['videoram', 4], ['tsc_mode', 0], ['nomigrate', 0]]], ['s3_integrity', 
1], ['device', ['tap', ['uname', 
'tap:aio:/var/lib/xen/images/colossus/rhel5-32pv.img'], ['dev', 'xvda'], 
['mode', 'w']]], ['device', ['vif', ['bridge', 'xenbr0'], ['mac', 
'00:16:3e:30:d0:e2']]]])
[2010-06-07 17:30:30 6690] DEBUG (XendDomainInfo:2510) 
XendDomainInfo.constructDomain
[2010-06-07 17:30:30 6690] DEBUG (balloon:187) Balloon: 638692 KiB free; 
need 16384; done.
[2010-06-07 17:30:30 6690] DEBUG (XendDomain:464) Adding Domain: 5
[2010-06-07 17:30:30 6690] DEBUG (XendDomainInfo:2848) 
XendDomainInfo.initDomain: 5 256
[2010-06-07 17:30:30 6852] DEBUG (XendBootloader:113) Launching 
bootloader as ['/usr/bin/pygrub', 
'--output=/var/run/xend/boot/xenbl.23510', '-q', 
'/var/lib/xen/images/colossus/rhel5-32pv.img'].
[2010-06-07 17:30:31 6690] DEBUG (XendDomainInfo:2875) 
_initDomain:shadow_memory=0x0, memory_static_max=0x20000000, 
memory_static_min=0x0.
[2010-06-07 17:30:31 6690] INFO (image:182) buildDomain os=linux dom=5 
vcpus=4
[2010-06-07 17:30:31 6690] DEBUG (image:721) domid          = 5
[2010-06-07 17:30:31 6690] DEBUG (image:722) memsize        = 512
[2010-06-07 17:30:31 6690] DEBUG (image:723) image          = 
/var/run/xend/boot/boot_kernel.K5aHnG
[2010-06-07 17:30:31 6690] DEBUG (image:724) store_evtchn   = 1
[2010-06-07 17:30:31 6690] DEBUG (image:725) console_evtchn = 2
[2010-06-07 17:30:31 6690] DEBUG (image:726) cmdline        = ro 
root=/dev/VolGroup00/LogVol00 rhgb quiet
[2010-06-07 17:30:31 6690] DEBUG (image:727) ramdisk        = 
/var/run/xend/boot/boot_ramdisk.q7eD9J
[2010-06-07 17:30:31 6690] DEBUG (image:728) vcpus          = 4
[2010-06-07 17:30:31 6690] DEBUG (image:729) features       =
[2010-06-07 17:30:31 6690] DEBUG (image:730) flags          = 0
[2010-06-07 17:30:31 6690] DEBUG (image:731) superpages     = 0
[2010-06-07 17:30:31 6690] INFO (XendDomainInfo:2369) createDevice: tap 
: {'bootable': 1, 'uname': 
'tap:aio:/var/lib/xen/images/colossus/rhel5-32pv.img', 'mode': 'w', 
'dev': 'xvda', 'uuid': 'ec14b93e-99df-8d7e-eac0-2aaa9332d6f6'}
[2010-06-07 17:30:31 6690] DEBUG (DevController:95) DevController: 
writing {'virtual-device': '51712', 'device-type': 'disk', 'protocol': 
'x86_32-abi', 'backend-id': '0', 'state': '1', 'backend': 
'/local/domain/0/backend/tap/5/51712'} to /local/domain/5/device/vbd/51712.
[2010-06-07 17:30:31 6690] DEBUG (DevController:97) DevController: 
writing {'domain': 'rhel5-32pv', 'frontend': 
'/local/domain/5/device/vbd/51712', 'uuid': 
'ec14b93e-99df-8d7e-eac0-2aaa9332d6f6', 'bootable': '1', 'dev': 'xvda', 
'state': '1', 'params': 
'aio:/var/lib/xen/images/colossus/rhel5-32pv.img', 'mode': 'w', 
'online': '1', 'frontend-id': '5', 'type': 'tap'} to 
/local/domain/0/backend/tap/5/51712.
[2010-06-07 17:30:31 6690] INFO (XendDomainInfo:2369) createDevice: vif 
: {'bridge': 'xenbr0', 'mac': '00:16:3e:30:d0:e2', 'uuid': 
'ca69ce95-45d3-8b63-9e42-1ee256014264'}
[2010-06-07 17:30:31 6690] DEBUG (DevController:95) DevController: 
writing {'mac': '00:16:3e:30:d0:e2', 'handle': '0', 'protocol': 
'x86_32-abi', 'backend-id': '0', 'state': '1', 'backend': 
'/local/domain/0/backend/vif/5/0'} to /local/domain/5/device/vif/0.
[2010-06-07 17:30:31 6690] DEBUG (DevController:97) DevController: 
writing {'bridge': 'xenbr0', 'domain': 'rhel5-32pv', 'handle': '0', 
'uuid': 'ca69ce95-45d3-8b63-9e42-1ee256014264', 'script': 
'/etc/xen/scripts/vif-bridge', 'mac': '00:16:3e:30:d0:e2', 
'frontend-id': '5', 'state': '1', 'online': '1', 'frontend': 
'/local/domain/5/device/vif/0'} to /local/domain/0/backend/vif/5/0.
[2010-06-07 17:30:31 6690] DEBUG (XendDomainInfo:3430) Storing VM 
details: {'on_xend_stop': 'ignore', 'pool_name': 'Pool-0', 
'shadow_memory': '0', 'uuid': '731497a6-2c62-8cfb-0afb-4e1ff7bde24d', 
'on_reboot': 'restart', 'start_time': '1275924631.91', 'on_poweroff': 
'destroy', 'bootloader_args': '-q', 'on_xend_start': 'ignore', 
'on_crash': 'restart', 'xend/restart_count': '0', 'vcpus': '4', 
'vcpu_avail': '3', 'bootloader': '/usr/bin/pygrub', 'image': "(linux 
(kernel ) (superpages 0) (tsc_mode 0) (videoram 4) (pci ()) (nomigrate 
0) (notes (FEATURES 
'writable_page_tables|writable_descriptor_tables|auto_translated_physmap|pae_pgdir_above_4gb|supervisor_mode_kernel') 
(VIRT_BASE 3221225472) (GUEST_VERSION 2.6) (PADDR_OFFSET 3221225472) 
(GUEST_OS linux) (HYPERCALL_PAGE 3225423872) (LOADER generic) (PAE_MODE 
yes) (ENTRY 3225419776) (XEN_VERSION xen-3.0)))", 'name': 'rhel5-32pv'}
[2010-06-07 17:30:31 6690] DEBUG (XendDomainInfo:1806) Storing domain 
details: {'console/ring-ref': '1066161', 'image/entry': '3225419776', 
'console/port': '2', 'cpu/3/availability': 'offline', 'store/ring-ref': 
'1066162', 'image/loader': 'generic', 'vm': 
'/vm/731497a6-2c62-8cfb-0afb-4e1ff7bde24d', 
'control/platform-feature-multiprocessor-suspend': '1', 'description': 
'', 'cpu/2/availability': 'offline', 'cpu/1/availability': 'online', 
'image/features/writable-descriptor-tables': '1', 'image/virt-base': 
'3221225472', 'memory/target': '524288', 'image/guest-version': '2.6', 
'image/features/supervisor-mode-kernel': '1', 'image/pae-mode': 'yes', 
'image/guest-os': 'linux', 'console/limit': '1048576', 
'image/paddr-offset': '3221225472', 'image/hypercall-page': 
'3225423872', 'cpu/0/availability': 'online', 
'image/features/pae-pgdir-above-4gb': '1', 
'image/features/writable-page-tables': '1', 'console/type': 
'xenconsoled', 'image/features/auto-translated-physmap': '1', 'name': 
'rhel5-32pv', 'domid': '5', 'image/xen-version': 'xen-3.0', 
'store/port': '1'}
[2010-06-07 17:30:32 6690] DEBUG (DevController:95) DevController: 
writing {'protocol': 'x86_32-abi', 'state': '1', 'backend-id': '0', 
'backend': '/local/domain/0/backend/console/5/0'} to 
/local/domain/5/device/console/0.
[2010-06-07 17:30:32 6690] DEBUG (DevController:97) DevController: 
writing {'domain': 'rhel5-32pv', 'frontend': 
'/local/domain/5/device/console/0', 'uuid': 
'2621cc44-62ab-8520-102d-cb34c2cc8cb7', 'frontend-id': '5', 'state': 
'1', 'location': '2', 'online': '1', 'protocol': 'vt100'} to 
/local/domain/0/backend/console/5/0.
[2010-06-07 17:30:32 6690] DEBUG (XendDomainInfo:1893) 
XendDomainInfo.handleShutdownWatch
[2010-06-07 17:30:32 6690] DEBUG (DevController:139) Waiting for devices 
vif2.
[2010-06-07 17:30:32 6690] DEBUG (DevController:139) Waiting for devices 
vif.
[2010-06-07 17:30:32 6690] DEBUG (DevController:144) Waiting for 0.
[2010-06-07 17:30:32 6690] DEBUG (DevController:628) 
hotplugStatusCallback /local/domain/0/backend/vif/5/0/hotplug-status.


[2010-06-07 17:32:12 6690] DEBUG (XendDomainInfo:3083) 
XendDomainInfo.destroy: domid=5
[2010-06-07 17:32:12 6690] DEBUG (XendDomainInfo:2413) Destroying device 
model
[2010-06-07 17:32:12 6690] DEBUG (XendDomainInfo:2420) Releasing devices
[2010-06-07 17:32:12 6690] DEBUG (XendDomainInfo:2426) Removing vif/0
[2010-06-07 17:32:12 6690] DEBUG (XendDomainInfo:1288) 
XendDomainInfo.destroyDevice: deviceClass = vif, device = vif/0
[2010-06-07 17:32:12 6690] DEBUG (XendDomainInfo:2426) Removing console/0
[2010-06-07 17:32:12 6690] DEBUG (XendDomainInfo:1288) 
XendDomainInfo.destroyDevice: deviceClass = console, device = console/0
[2010-06-07 17:32:12 6690] DEBUG (XendDomainInfo:2426) Removing tap/51712
[2010-06-07 17:32:12 6690] DEBUG (XendDomainInfo:1288) 
XendDomainInfo.destroyDevice: deviceClass = tap, device = tap/51712

For HVM guests it's:

# xm create rhel5-32fv
Using config file "./rhel5-32fv".
Error: Not enough memory is available, and dom0 cannot be shrunk any further

and also the xend.log fragment relevant to the issue:

[2010-06-07 17:34:30 6690] DEBUG (XendDomainInfo:103) 
XendDomainInfo.create(['vm', ['name', 'rhel5-32fv'], ['memory', 1024], 
['maxmem', 1024], ['on_poweroff', 'destroy'], ['on_reboot', 'restart'], 
['on_crash', 'restart'], ['on_xend_start', 'ignore'], ['on_xend_stop', 
'ignore'], ['vcpus', 1], ['uuid', 
'a21e0a64-8fbe-48c2-f28e-9c1cfc7427c7'], ['oos', 1], ['image', ['hvm', 
['kernel', '/usr/lib/xen/boot/hvmloader'], ['videoram', 4], ['parallel', 
'none'], ['serial', 'pty'], ['acpi', 1], ['apic', 1], ['boot', 'c'], 
['cpuid', []], ['cpuid_check', []], ['device_model', 
'/usr/lib64/xen/bin/qemu-dm'], ['fda', ''], ['fdb', ''], 
['guest_os_type', 'default'], ['hap', 1], ['hpet', 0], ['isa', 0], 
['keymap', 'en-us'], ['localtime', 0], ['nographic', 0], ['oos', 1], 
['pae', 1], ['pci', []], ['pci_msitranslate', 1], ['pci_power_mgmt', 0], 
['rtc_timeoffset', 0], ['sdl', 0], ['soundhw', ''], ['stdvga', 0], 
['timer_mode', 1], ['usb', 0], ['usbdevice', ''], ['vcpus', 1], ['vnc', 
1], ['vncdisplay', 5], ['vncunused', 0], ['viridian', 0], ['vpt_align', 
1], ['xauthority', '/root/.Xauthority'], ['xen_platform_pci', 1], 
['memory_sharing', 0], ['tsc_mode', 0], ['nomigrate', 0]]], 
['s3_integrity', 1], ['device', ['vbd', ['uname', 
'file:/var/lib/xen/images/colossus/rhel5-32fv.img'], ['dev', 'hda'], 
['mode', 'w']]], ['device', ['vbd', ['uname', 'file:/home2/test.img'], 
['dev', 'sda'], ['mode', 'w']]], ['device', ['vif', ['bridge', 'eth0'], 
['mac', '00:16:3e:6b:d5:c0'], ['type', 'netfront']]]])
[2010-06-07 17:34:30 6690] DEBUG (XendDomainInfo:2510) 
XendDomainInfo.constructDomain
[2010-06-07 17:34:30 6690] DEBUG (balloon:187) Balloon: 638692 KiB free; 
need 16384; done.
[2010-06-07 17:34:30 6690] DEBUG (XendDomain:464) Adding Domain: 6
[2010-06-07 17:34:30 6690] DEBUG (XendDomainInfo:2848) 
XendDomainInfo.initDomain: 6 256
[2010-06-07 17:34:30 6690] DEBUG (image:339) No VNC passwd configured 
for vfb access
[2010-06-07 17:34:31 6690] DEBUG (image:889) args: boot, val: c
[2010-06-07 17:34:31 6690] DEBUG (image:889) args: fda, val: None
[2010-06-07 17:34:31 6690] DEBUG (image:889) args: fdb, val: None
[2010-06-07 17:34:31 6690] DEBUG (image:889) args: soundhw, val: None
[2010-06-07 17:34:31 6690] DEBUG (image:889) args: localtime, val: 0
[2010-06-07 17:34:31 6690] DEBUG (image:889) args: serial, val: ['pty']
[2010-06-07 17:34:31 6690] DEBUG (image:889) args: std-vga, val: 0
[2010-06-07 17:34:31 6690] DEBUG (image:889) args: isa, val: 0
[2010-06-07 17:34:31 6690] DEBUG (image:889) args: acpi, val: 1
[2010-06-07 17:34:31 6690] DEBUG (image:889) args: usb, val: 0
[2010-06-07 17:34:31 6690] DEBUG (image:889) args: usbdevice, val: None
[2010-06-07 17:34:31 6690] DEBUG (image:889) args: gfx_passthru, val: None
[2010-06-07 17:34:31 6690] INFO (image:822) Need to create platform 
device.[domid:6]
[2010-06-07 17:34:31 6690] DEBUG (XendDomainInfo:2875) 
_initDomain:shadow_memory=0x0, memory_static_max=0x40000000, 
memory_static_min=0x0.
[2010-06-07 17:34:31 6690] DEBUG (balloon:172) Balloon: tmem 
relinquished 0 KiB of 431872 KiB requested.
[2010-06-07 17:34:31 6690] DEBUG (balloon:193) Balloon: 634112 KiB free; 
0 to scrub; need 1065984; retries: 20.
[2010-06-07 17:34:31 6690] DEBUG (balloon:207) Balloon: setting dom0 
target to 7524 MiB.
[2010-06-07 17:34:31 6690] DEBUG (XendDomainInfo:1479) Setting memory 
target of domain Domain-0 (0) to 7524 MiB.
[2010-06-07 17:34:31 6690] DEBUG (balloon:193) Balloon: 634112 KiB free; 
0 to scrub; need 1065984; retries: 20.




[2010-06-07 17:34:54 6690] DEBUG (XendDomainInfo:1479) Setting memory 
target of domain Domain-0 (0) to 7945 MiB.
[2010-06-07 17:34:54 6690] DEBUG (balloon:187) Balloon: 634112 KiB free; 
need 431104; done.
[2010-06-07 17:34:54 6690] ERROR (XendDomainInfo:2934) 
XendDomainInfo.initDomain: exception occurred
Traceback (most recent call last):
   File "/usr/lib64/python2.4/site-packages/xen/xend/XendDomainInfo.py", 
line 2893, in _initDomain
     balloon.free(memory + shadow + vtd_mem, self)
   File "/usr/lib64/python2.4/site-packages/xen/xend/balloon.py", line 
237, in free
     raise VmError(
VmError: Not enough memory is available, and dom0 cannot be shrunk any 
further
[2010-06-07 17:34:54 6690] ERROR (XendDomainInfo:485) VM start failed
Traceback (most recent call last):
   File "/usr/lib64/python2.4/site-packages/xen/xend/XendDomainInfo.py", 
line 471, in start
     XendTask.log_progress(31, 60, self._initDomain)
   File "/usr/lib64/python2.4/site-packages/xen/xend/XendTask.py", line 
209, in log_progress
     retval = func(*args, **kwds)
   File "/usr/lib64/python2.4/site-packages/xen/xend/XendDomainInfo.py", 
line 2937, in _initDomain
     raise exn
VmError: Not enough memory is available, and dom0 cannot be shrunk any 
further
[2010-06-07 17:34:54 6690] DEBUG (XendDomainInfo:3083) 
XendDomainInfo.destroy: domid=6
[2010-06-07 17:34:54 6690] DEBUG (XendDomainInfo:2413) Destroying device 
model
[2010-06-07 17:34:54 6690] ERROR (XendDomainInfo:2416) Device model 
destroy failed X86_HVM_ImageHandler instance has no attribute 
'sentinel_lock'
Traceback (most recent call last):
   File "/usr/lib64/python2.4/site-packages/xen/xend/XendDomainInfo.py", 
line 2414, in _releaseDevices
     self.image.destroyDeviceModel()
   File "/usr/lib64/python2.4/site-packages/xen/xend/image.py", line 
621, in destroyDeviceModel
     self.sentinel_lock.acquire()
AttributeError: X86_HVM_ImageHandler instance has no attribute 
'sentinel_lock'
[2010-06-07 17:34:54 6690] DEBUG (XendDomainInfo:2420) Releasing devices
[2010-06-07 17:34:54 6690] ERROR (XendDomainInfo:108) Domain 
construction failed
Traceback (most recent call last):
   File "/usr/lib64/python2.4/site-packages/xen/xend/XendDomainInfo.py", 
line 106, in create
     vm.start()
   File "/usr/lib64/python2.4/site-packages/xen/xend/XendDomainInfo.py", 
line 471, in start
     XendTask.log_progress(31, 60, self._initDomain)
   File "/usr/lib64/python2.4/site-packages/xen/xend/XendTask.py", line 
209, in log_progress
     retval = func(*args, **kwds)
   File "/usr/lib64/python2.4/site-packages/xen/xend/XendDomainInfo.py", 
line 2937, in _initDomain
     raise exn
VmError: Not enough memory is available, and dom0 cannot be shrunk any 
further

For PV guests there was a change of blktap to blktap2. Couldn't this be 
related? How could I check I'm using the blktap2 ? Also, my test machine 
is Intel Xeon X5460 @ 3.16 Ghz, quad-core with 8 GiB RAM running on 
RHEL-5 with upstream 4.1 xen.

Michal


-- 
Michal Novotny<minovotn@redhat.com>, RHCE
Virtualization Team (xen userspace), Red Hat