Re: [PATCH] qemu-xen-3.4-testing: Fix read-only image file handling

[Michal Novotny <minovotn@redhat.com>]

On 06/07/2010 07:56 PM, Keir Fraser wrote:
> ...
> I suggest add some tracing to function dom0_init() in xenstored and see
> where that function is failing. The two most obvious places were the opening
> of those two files in /proc. If they exist then I'm not sure what else could
> be going wrong.
>
> Oh, you should be able to cat /proc/xen/ksd_port and get an integer value.
> /proc/xen/ksd_kva by contrast gets mmap()ed by xenstored.
>    

Are you sure it's ksd_*? Isn't it xsd_*?

This is what I'm having:

# ls /proc/xen
capabilities  privcmd  xenbus  xsd_kva  xsd_port
# cat /proc/xen/xsd_port
21
#cat /proc/xen/xsd_kva
0xffff8801e8414000
#

I was finally able to make it work. The main problem was absence of 
xen-evtchn module in memory which is why the xenstore daemon was unable 
to start so I created /etc/sysconfig/modules/xen.modules with the 
following contents:
#!/bin/sh
MODULES="xenfs xen-evtchn xen_evtchn"
for i in $MODULES ; do
         modprobe $i >/dev/null 2>&1
done

and after that when I rebooted the dom0 there was xen_evtchn in the 
`lsmod` output. Unfortunately the xenstore daemon was not running but 
when I run `xend start` it came online and `xm list` was showing at 
least dom0 (I had no other domain here so this is good) but after reboot 
I have to do following to make `xm list` show good data:

# lsmod | grep xen
xen_evtchn              8344  0
xenfs                  15624  1
# xend start
Nothing to flush.
# xm list
Name                                        ID   Mem VCPUs      State   
Time(s)
Domain-0                                     0  7453     4     
r-----     54.1
#

The daemon doesn't start on boot although it's trying to start. I think 
that the problem is that event channel module does get loaded into the 
memory after the xend is trying to start xenstore daemon (which is bad 
since xenstore daemon can't start before event channel module) and 
itself which results into error in connection:

# xm list
Error: Unable to connect to xend: No such file or directory. Is xend 
running?
#

Now, when I'm having the working PVops kernel and tools running, I'm 
trying to get it start up a domain using `xm create HVMguest` and it 
returns:

[2010-06-08 13:58:24 5137] DEBUG (XendDomainInfo:103) 
XendDomainInfo.create(['vm', ['name', 'rhel5-32fv'], ['memory', 1024], 
['maxmem', 1024], ['on_poweroff', 'destroy'], ['on_reboot', 'restart'], 
['on_crash', 'restart'], ['on_xend_start', 'ignore'], ['on_xend_stop', 
'ignore'], ['vcpus', 1], ['uuid', 
'a21e0a64-8fbe-48c2-f28e-9c1cfc7427c7'], ['oos', 1], ['image', ['hvm', 
['kernel', '/usr/lib/xen/boot/hvmloader'], ['videoram', 4], ['parallel', 
'none'], ['serial', 'pty'], ['acpi', 1], ['apic', 1], ['boot', 'c'], 
['cpuid', []], ['cpuid_check', []], ['device_model', 
'/usr/lib64/xen/bin/qemu-dm'], ['fda', ''], ['fdb', ''], 
['guest_os_type', 'default'], ['hap', 1], ['hpet', 0], ['isa', 0], 
['keymap', 'en-us'], ['localtime', 0], ['nographic', 0], ['oos', 1], 
['pae', 1], ['pci', []], ['pci_msitranslate', 1], ['pci_power_mgmt', 0], 
['rtc_timeoffset', 0], ['sdl', 0], ['soundhw', ''], ['stdvga', 0], 
['timer_mode', 1], ['usb', 0], ['usbdevice', ''], ['vcpus', 1], ['vnc', 
1], ['vncdisplay', 5], ['vncunused', 0], ['viridian', 0], ['vpt_align', 
1], ['xauthority', '/root/.Xauthority'], ['xen_platform_pci', 1], 
['memory_sharing', 0], ['tsc_mode', 0], ['nomigrate', 0]]], 
['s3_integrity', 1], ['device', ['vbd', ['uname', 
'file:/var/lib/xen/images/colossus/rhel5-32fv.img'], ['dev', 'hda'], 
['mode', 'w']]], ['device', ['vbd', ['uname', 'file:/home2/test.img'], 
['dev', 'sda'], ['mode', 'w']]], ['device', ['vif', ['bridge', 'eth0'], 
['mac', '00:16:3e:6b:d5:c0'], ['type', 'netfront']]]])
[2010-06-08 13:58:24 5137] DEBUG (XendDomainInfo:2510) 
XendDomainInfo.constructDomain
[2010-06-08 13:58:24 5137] DEBUG (balloon:187) Balloon: 638692 KiB free; 
need 16384; done.
[2010-06-08 13:58:24 5137] DEBUG (XendDomain:464) Adding Domain: 4
[2010-06-08 13:58:24 5137] DEBUG (XendDomainInfo:2848) 
XendDomainInfo.initDomain: 4 256
[2010-06-08 13:58:24 5137] DEBUG (image:339) No VNC passwd configured 
for vfb access
[2010-06-08 13:58:24 5137] DEBUG (image:889) args: boot, val: c
[2010-06-08 13:58:24 5137] DEBUG (image:889) args: fda, val: None
[2010-06-08 13:58:24 5137] DEBUG (image:889) args: fdb, val: None
[2010-06-08 13:58:24 5137] DEBUG (image:889) args: soundhw, val: None
[2010-06-08 13:58:24 5137] DEBUG (image:889) args: localtime, val: 0
[2010-06-08 13:58:24 5137] DEBUG (image:889) args: serial, val: ['pty']
[2010-06-08 13:58:24 5137] DEBUG (image:889) args: std-vga, val: 0
[2010-06-08 13:58:24 5137] DEBUG (image:889) args: isa, val: 0
[2010-06-08 13:58:24 5137] DEBUG (image:889) args: acpi, val: 1
[2010-06-08 13:58:24 5137] DEBUG (image:889) args: usb, val: 0
[2010-06-08 13:58:24 5137] DEBUG (image:889) args: usbdevice, val: None
[2010-06-08 13:58:24 5137] DEBUG (image:889) args: gfx_passthru, val: None
[2010-06-08 13:58:24 5137] INFO (image:822) Need to create platform 
device.[domid:4]
[2010-06-08 13:58:24 5137] DEBUG (XendDomainInfo:2875) 
_initDomain:shadow_memory=0x0, memory_static_max=0x40000000, 
memory_static_min=0x0.
[2010-06-08 13:58:24 5137] DEBUG (balloon:172) Balloon: tmem 
relinquished 0 KiB of 431872 KiB requested.
[2010-06-08 13:58:24 5137] DEBUG (balloon:193) Balloon: 634112 KiB free; 
0 to scrub; need 1065984; retries: 20.
[2010-06-08 13:58:24 5137] DEBUG (balloon:207) Balloon: setting dom0 
target to 7524 MiB.
[2010-06-08 13:58:24 5137] DEBUG (XendDomainInfo:1479) Setting memory 
target of domain Domain-0 (0) to 7524 MiB.
[2010-06-08 13:58:24 5137] DEBUG (balloon:187) Balloon: 634112 KiB free; 
need 72704; done.
[2010-06-08 13:58:24 5137] DEBUG (balloon:193) Balloon: 634112 KiB free; 
0 to scrub; need 1065984; retries: 20.
[2010-06-08 13:58:47 5137] DEBUG (XendDomainInfo:1479) Setting memory 
target of domain Domain-0 (0) to 7945 MiB.
[2010-06-08 13:58:47 5137] DEBUG (balloon:187) Balloon: 634112 KiB free; 
need 431104; done.
[2010-06-08 13:58:47 5137] ERROR (XendDomainInfo:2934) 
XendDomainInfo.initDomain: exception occurred
Traceback (most recent call last):
   File "/usr/lib64/python2.4/site-packages/xen/xend/XendDomainInfo.py", 
line 2893, in _initDomain
     balloon.free(memory + shadow + vtd_mem, self)
   File "/usr/lib64/python2.4/site-packages/xen/xend/balloon.py", line 
237, in free
     raise VmError(
VmError: Not enough memory is available, and dom0 cannot be shrunk any 
further
[2010-06-08 13:58:47 5137] ERROR (XendDomainInfo:485) VM start failed
Traceback (most recent call last):
   File "/usr/lib64/python2.4/site-packages/xen/xend/XendDomainInfo.py", 
line 471, in start
     XendTask.log_progress(31, 60, self._initDomain)
   File "/usr/lib64/python2.4/site-packages/xen/xend/XendTask.py", line 
209, in log_progress
     retval = func(*args, **kwds)
   File "/usr/lib64/python2.4/site-packages/xen/xend/XendDomainInfo.py", 
line 2937, in _initDomain
     raise exn
VmError: Not enough memory is available, and dom0 cannot be shrunk any 
further
[2010-06-08 13:58:47 5137] DEBUG (XendDomainInfo:3083) 
XendDomainInfo.destroy: domid=4
[2010-06-08 13:58:47 5137] DEBUG (XendDomainInfo:2413) Destroying device 
model
[2010-06-08 13:58:47 5137] ERROR (XendDomainInfo:2416) Device model 
destroy failed X86_HVM_ImageHandler instance has no attribute 
'sentinel_lock'
Traceback (most recent call last):
   File "/usr/lib64/python2.4/site-packages/xen/xend/XendDomainInfo.py", 
line 2414, in _releaseDevices
     self.image.destroyDeviceModel()
   File "/usr/lib64/python2.4/site-packages/xen/xend/image.py", line 
621, in destroyDeviceModel
     self.sentinel_lock.acquire()
AttributeError: X86_HVM_ImageHandler instance has no attribute 
'sentinel_lock'
[2010-06-08 13:58:47 5137] DEBUG (XendDomainInfo:2420) Releasing devices
[2010-06-08 13:58:47 5137] ERROR (XendDomainInfo:108) Domain 
construction failed
Traceback (most recent call last):
   File "/usr/lib64/python2.4/site-packages/xen/xend/XendDomainInfo.py", 
line 106, in create
     vm.start()
   File "/usr/lib64/python2.4/site-packages/xen/xend/XendDomainInfo.py", 
line 471, in start
     XendTask.log_progress(31, 60, self._initDomain)
   File "/usr/lib64/python2.4/site-packages/xen/xend/XendTask.py", line 
209, in log_progress
     retval = func(*args, **kwds)
   File "/usr/lib64/python2.4/site-packages/xen/xend/XendDomainInfo.py", 
line 2937, in _initDomain
     raise exn
VmError: Not enough memory is available, and dom0 cannot be shrunk any 
further

I also tried to decrease the memory for dom0 as advised on the list 
(using the `xm mem-set 0 2048`) but still the same result but I saw 
something in the qemu-dm log file about -vcpu-avail option is invalid:

# cat /var/log/xen/qemu-dm-rhel5-32fv.log
domid: 1
qemu: the number of cpus is 1
/usr/lib64/xen/bin/qemu-dm: invalid option -- '-vcpu_avail'
#

Now I'm thinking whether there wasn't a change of qemu-dm to stubdom-dm. 
Where is it located and could you provide me some example configuration 
file for xen-4.1-unstable?

Thanks,
Michal
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>    


-- 
Michal Novotny<minovotn@redhat.com>, RHCE
Virtualization Team (xen userspace), Red Hat