Reuse QEMU image for HVM?

Reuse QEMU image for HVM?

[Jun Koi]

Hi,

I have an Windows VM in QEMU format (qcow2: btw, how can I confirm
that is qcow2, but not other formats?). How can I reuse that to run as
HVM in Xen?

Thanks,
Jun

Re: Reuse QEMU image for HVM?

[Jun Koi]

On Fri, Dec 5, 2008 at 3:52 PM, Jun Koi <junkoi2004@gmail.com> wrote:
> Hi,
>
> I have an Windows VM in QEMU format (qcow2: btw, how can I confirm
> that is qcow2, but not other formats?). How can I reuse that to run as
> HVM in Xen?
>

It seems we just run QEMU image normally?

My notes:
- It seems that 3.3.0 (or -unstable tree) no longer sets bridge name
as "xenbr0", but renames it to "vnet0"?
- It took me a long time to notice that hvmloader failed to compile,
as bcc is missing. But the error message that mentions "dev86 package
is needed" was very misleading, and it took me a while to figure it
out that what I really need is "bcc". So I think it is better to fix
this error message so it is more clear to users.

I tried to create HVM from my QEMU image, but have some problems.
Below is my configuration file for my QEMU image (winxp). I started it
with "xm create xp3.cfg", but only saw an VNC window appeared for 1
sec, then crashed when starting to boot the image (BIOS loading seems
to be done at the crashed time). Attached is also qemu-dm.log file.
Please anybody tell me what is the problem?

Thanks,
Jun


xp3.cfg
---------
kernel = '/usr/lib/xen/boot/hvmloader'
builder = 'hvm'
memory = '800'
device_model='/usr/lib/xen/bin/qemu-dm'
disk = ['file:/home/xen/img/img.xp3,ioemu:hda,w']
name = "xp3"
# vif = ['type=ioemu, bridge=xenbr0']
vif = ['type=ioemu, bridge=vnet0']
boot="d"
vnc=1
vncviewer=1
sdl=0



/var/log/xen/qemu-dm-xp3.log
--------------------------------------
domid: 7
qemu: the number of cpus is 1
config qemu network with xen bridge for  tap7.0 vnet0
Watching /local/domain/0/device-model/7/logdirty/next-active
Watching /local/domain/0/device-model/7/command
xs_read(): vncpasswd get error.
/vm/bb7c12a8-25e0-d91b-8ab5-0298eb5ae815/vncpasswd.
qemu_map_cache_init nr_buckets = 4000 size 196608
shared page at pfn 31ffe
buffered io page at pfn 31ffc
Time offset set 0
Register xen platform.
Done register platform.
xs_read(/local/domain/0/device-model/7/xen_extended_power_mgmt): read error
I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0
stolen video RAM at 317f8000
mapping video RAM from 317f8000
mapping vram to f0000000 - f0800000

Re: Re: Reuse QEMU image for HVM?

[Stefano Stabellini]

Jun Koi wrote:

> xp3.cfg
> ---------
> kernel = '/usr/lib/xen/boot/hvmloader'
> builder = 'hvm'
> memory = '800'
> device_model='/usr/lib/xen/bin/qemu-dm'
> disk = ['file:/home/xen/img/img.xp3,ioemu:hda,w']
> name = "xp3"
> # vif = ['type=ioemu, bridge=xenbr0']
> vif = ['type=ioemu, bridge=vnet0']
> boot="d"
> vnc=1
> vncviewer=1
> sdl=0

boot="d" means boot the cdrom and you didn't configure any cdrom;
try boot="cda".

Re: Re: Reuse QEMU image for HVM?

[Jun Koi]

On Fri, Dec 5, 2008 at 7:57 PM, Stefano Stabellini
<stefano.stabellini@eu.citrix.com> wrote:
> Jun Koi wrote:
>
>> xp3.cfg
>> ---------
>> kernel = '/usr/lib/xen/boot/hvmloader'
>> builder = 'hvm'
>> memory = '800'
>> device_model='/usr/lib/xen/bin/qemu-dm'
>> disk = ['file:/home/xen/img/img.xp3,ioemu:hda,w']
>> name = "xp3"
>> # vif = ['type=ioemu, bridge=xenbr0']
>> vif = ['type=ioemu, bridge=vnet0']
>> boot="d"
>> vnc=1
>> vncviewer=1
>> sdl=0
>
> boot="d" means boot the cdrom and you didn't configure any cdrom;
> try boot="cda".

I did that, and also with "boot = 'c'", butstill have the same
problem: VM crashed immediately when start to boot OS. The screen
shows that only BIOS was successfully initialized.

Do you have any idea on how to debug this problem?

Thanks,
Jun

Re: Re: Reuse QEMU image for HVM?

[Jun Koi]

On Fri, Dec 5, 2008 at 11:11 PM, Jun Koi <junkoi2004@gmail.com> wrote:
> On Fri, Dec 5, 2008 at 7:57 PM, Stefano Stabellini
> <stefano.stabellini@eu.citrix.com> wrote:
>> Jun Koi wrote:
>>
>>> xp3.cfg
>>> ---------
>>> kernel = '/usr/lib/xen/boot/hvmloader'
>>> builder = 'hvm'
>>> memory = '800'
>>> device_model='/usr/lib/xen/bin/qemu-dm'
>>> disk = ['file:/home/xen/img/img.xp3,ioemu:hda,w']
>>> name = "xp3"
>>> # vif = ['type=ioemu, bridge=xenbr0']
>>> vif = ['type=ioemu, bridge=vnet0']
>>> boot="d"
>>> vnc=1
>>> vncviewer=1
>>> sdl=0
>>
>> boot="d" means boot the cdrom and you didn't configure any cdrom;
>> try boot="cda".
>
> I did that, and also with "boot = 'c'", butstill have the same
> problem: VM crashed immediately when start to boot OS. The screen
> shows that only BIOS was successfully initialized.
>
> Do you have any idea on how to debug this problem?

It seems that the reason is qcow2 file created by QEMU tool is not
supported by Xen??

Thanks,
Jun

Re: Re: Reuse QEMU image for HVM?

[Stefano Stabellini]

Jun Koi wrote:

> I did that, and also with "boot = 'c'", butstill have the same
> problem: VM crashed immediately when start to boot OS. The screen
> shows that only BIOS was successfully initialized.
> 
> Do you have any idea on how to debug this problem?
> 

I think it is probably due to a version change in a device or in the
firmware, maybe it is just the version string that has changed.

It would be worth to test with a linux image too and see what happens:
if it crashes linux will be much easier to debug, if it doesn't it means
it is probably just a version string that changed somewhere.

Re: Re: Reuse QEMU image for HVM?

[Grant McWilliams]

[-- Attachment #1.1: Type: text/plain, Size: 622 bytes --]

>
>
>
> It seems that the reason is qcow2 file created by QEMU tool is not
> supported by Xen??
>
> Thanks,
> Jun
>
> _______________________________________________
>

>From what I know this is true. It is probably the biggest blunder of the
virtualization world! I'd like to
know who at the xen team decided to adopt qcow but not make it compatible
with QEMU. You can
use qcow but you need to use xen's tool to create it. From what I understand
there's a difference in
endianness of the file.


Grant McWilliams

Some people, when confronted with a problem, think "I know, I'll use
Windows."
Now they have two problems.

[-- Attachment #1.2: Type: text/html, Size: 964 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Re: Reuse QEMU image for HVM?

[Jun Koi]

On Mon, Dec 8, 2008 at 7:28 PM, Stefano Stabellini
<stefano.stabellini@eu.citrix.com> wrote:
> Jun Koi wrote:
>
>> I did that, and also with "boot = 'c'", butstill have the same
>> problem: VM crashed immediately when start to boot OS. The screen
>> shows that only BIOS was successfully initialized.
>>
>> Do you have any idea on how to debug this problem?
>>
>
> I think it is probably due to a version change in a device or in the
> firmware, maybe it is just the version string that has changed.
>
> It would be worth to test with a linux image too and see what happens:
> if it crashes linux will be much easier to debug, if it doesn't it means
> it is probably just a version string that changed somewhere.

I have the same problem with Linux image. And the same image works well in QEMU.

What do you mean by "version string changed"?

Thanks,
Jun

Re: Re: Reuse QEMU image for HVM?

[Jun Koi]

On Tue, Dec 9, 2008 at 1:34 AM, Grant McWilliams
<grantmasterflash@gmail.com> wrote:
>>
>>
>> It seems that the reason is qcow2 file created by QEMU tool is not
>> supported by Xen??
>>
>> Thanks,
>> Jun
>>
>> _______________________________________________
>
> From what I know this is true. It is probably the biggest blunder of the
> virtualization world! I'd like to
> know who at the xen team decided to adopt qcow but not make it compatible
> with QEMU. You can
> use qcow but you need to use xen's tool to create it. From what I understand
> there's a difference in
> endianness of the file.

Too sad if that is true. Could anybody confirm this?

I tried to see if that is true or not, by creating 2 images with
qemu-img (from QEMU) and qemu-img-xen (from Xen). The command is like
below:

qemu-img create -f qcow2 img.test1 10MB
qemu-img-xen create -f qcow2 img.test2 10MB

And then I tried to diff these 2 files (diff img.test1 img.test2), but
the result was: 2 files are completely same.

So it doesnt seem right that qemu-img-xen does something difference
with qcow2 format. I tried to compare qcow format, too, and didnt see
any difference.

Any idea??
Jun


>
> Grant McWilliams
>
> Some people, when confronted with a problem, think "I know, I'll use
> Windows."
> Now they have two problems.
>

Re: Re: Reuse QEMU image for HVM?

[Grant McWilliams]

[-- Attachment #1.1: Type: text/plain, Size: 1725 bytes --]

On Mon, Dec 8, 2008 at 7:50 PM, Jun Koi <junkoi2004@gmail.com> wrote:

> On Tue, Dec 9, 2008 at 1:34 AM, Grant McWilliams
> <grantmasterflash@gmail.com> wrote:
> >>
> >>
> >> It seems that the reason is qcow2 file created by QEMU tool is not
> >> supported by Xen??
> >>
> >> Thanks,
> >> Jun
> >>
> >> _______________________________________________
> >
> > From what I know this is true. It is probably the biggest blunder of the
> > virtualization world! I'd like to
> > know who at the xen team decided to adopt qcow but not make it compatible
> > with QEMU. You can
> > use qcow but you need to use xen's tool to create it. From what I
> understand
> > there's a difference in
> > endianness of the file.
>
> Too sad if that is true. Could anybody confirm this?
>
> I tried to see if that is true or not, by creating 2 images with
> qemu-img (from QEMU) and qemu-img-xen (from Xen). The command is like
> below:
>
> qemu-img create -f qcow2 img.test1 10MB
> qemu-img-xen create -f qcow2 img.test2 10MB
>
> And then I tried to diff these 2 files (diff img.test1 img.test2), but
> the result was: 2 files are completely same.
>
> So it doesnt seem right that qemu-img-xen does something difference
> with qcow2 format. I tried to compare qcow format, too, and didnt see
> any difference.
>
> Any idea??
> Jun
>



I googled it and the source I got it from said to use only qcow-create and
img2qcow utilities and not to use qemu-img because of the incompatibilities.
Later I saw posts on this mailing list outlining the endian difference
between qemu qcow and xen qcow.

Here's a thread that I followed, it's as new as March 08...
http://lists.xensource.com/archives/html/xen-devel/2008-03/msg00728.html


Grant McWilliams

[-- Attachment #1.2: Type: text/html, Size: 2450 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Re: Reuse QEMU image for HVM?

[Stefano Stabellini]

Jun Koi wrote:

> 
> I have the same problem with Linux image. And the same image works well in QEMU.
> 
> What do you mean by "version string changed"?
> 

I have just done few tests myself: I think it is just a matter of
correctly specifing the image format in the configuration file.
Please try:

disk = ['tap:qcow2:/home/xen/img/img.xp3,ioemu:hda,w']

Re: Re: Reuse QEMU image for HVM?

[Jun Koi]

On Tue, Dec 9, 2008 at 8:47 PM, Stefano Stabellini
<stefano.stabellini@eu.citrix.com> wrote:
> Jun Koi wrote:
>
>>
>> I have the same problem with Linux image. And the same image works well in QEMU.
>>
>> What do you mean by "version string changed"?
>>
>
> I have just done few tests myself: I think it is just a matter of
> correctly specifing the image format in the configuration file.
> Please try:
>
> disk = ['tap:qcow2:/home/xen/img/img.xp3,ioemu:hda,w']
>

Excellent! I can confirm that works for me.

The only problem is that QEMU provide different hardware from Xen, so
when I start the QEMU image on Xen, Windows complains and asks me to
"Reactivate Windows". Anyway to avoid that??

Thanks,
Jun

Re: Re: Reuse QEMU image for HVM?

[Stefano Stabellini]

Jun Koi wrote:

> On Tue, Dec 9, 2008 at 8:47 PM, Stefano Stabellini
> <stefano.stabellini@eu.citrix.com> wrote:
>> Jun Koi wrote:
>>
>>> I have the same problem with Linux image. And the same image works well in QEMU.
>>>
>>> What do you mean by "version string changed"?
>>>
>> I have just done few tests myself: I think it is just a matter of
>> correctly specifing the image format in the configuration file.
>> Please try:
>>
>> disk = ['tap:qcow2:/home/xen/img/img.xp3,ioemu:hda,w']
>>
> 
> Excellent! I can confirm that works for me.
> 
> The only problem is that QEMU provide different hardware from Xen, so
> when I start the QEMU image on Xen, Windows complains and asks me to
> "Reactivate Windows". Anyway to avoid that??
> 

There are not many differences in the device emulation between qemu
mainstream and xen now.
I think Windows would complain even if you try with a brand new qemu
compiled from svn, because your image is probably too old (created with
a too old qemu).

If you want to be sure which device caused it you can try booting linux
on your qemu and on xen and do an lspci -vvv on both.

Re: Re: Reuse QEMU image for HVM?

[Ian Jackson]

Stefano Stabellini writes ("Re: [Xen-devel] Re: Reuse QEMU image for HVM?"):
> There are not many differences in the device emulation between qemu
> mainstream and xen now.
> I think Windows would complain even if you try with a brand new qemu
> compiled from svn, because your image is probably too old (created with
> a too old qemu).

The real problem is probably that Xen's qemu reports various different
ids in its PCI devices: the `subsystem vendor' is set to a value
indicating Xensource.  In theory it would be possible to make a qemu
which did the same, but at the moment it's not that easy because the
changes to do this aren't easily separated out from the other changes
in our tree.

On the qemu upstream list it has been proposed that there should be a
single place where the subsystem vendor is controlled, and if that
patch were accepted it would make the difference between us and
upstream smaller and of course make it easier to make an upstream qemu
simulate more like Xen's.

The difference in subsystem vendors between the trees needs to remain,
though, because it's used for example by PV drivers to identify which
`native' devices need to be masked.  So for example if the system has
a pci ne2k ethernet card reporting subsystem vendor Xensource, and the
Xen PV driver in the guest has successfully discovered the
corresponding PV vif, it needs to prevent the ordinary guest's ne2k
driver from binding to the `ne2k' or the same interface will turn up
twice.  In the case of disks seeing the same disk via different paths
can cause even more serious problems.

Ian.