Problems KVM-84

Problems KVM-84

[Jay Mann]

Hi,

I just downloaded built and installed kvm-84 on ubuntu Hardy x64  2.6.24-23-
server and I’m getting the following 2 problems that did not exists in kvm-83.

1.	The qemu emulater (bios screen) takes a long time to start (~10 
seconds), and subsequently Libvirt times out when I try to start a guest VM 
from virsh.  

2.	In my windows xp guest, a new PCI RAM (Memory) Controller is found, but 
windows doesn’t know what driver to install (and neither do i)

Is anyone else experiencing these issues or are there any workarounds?  

Thanks,

-J

Re: Problems KVM-84

[Thomas Mueller]

> 
> I just downloaded built and installed kvm-84 on ubuntu Hardy x64 
> 2.6.24-23- server and I’m getting the following 2 problems that did not
> exists in kvm-83.
> 
> 1.	The qemu emulater (bios screen) takes a long time to start (~10
> seconds), and subsequently Libvirt times out when I try to start a guest
> VM from virsh.

also seen on debian lenny with kvm-84 and kernel 2.6.26. starting a vm 
takes about 10s (with kvm <= 83 ~1s) until the bios screen shows up.

- Thomas

Re: Problems KVM-84

[Avi Kivity]

Jay Mann wrote:
> Hi,
>
> I just downloaded built and installed kvm-84 on ubuntu Hardy x64  2.6.24-23-
> server and I’m getting the following 2 problems that did not exists in kvm-83.
>
> 1.	The qemu emulater (bios screen) takes a long time to start (~10 
> seconds), and subsequently Libvirt times out when I try to start a guest VM 
> from virsh.  
>   

Are you using a qcow image?  How large is it?

> 2.	In my windows xp guest, a new PCI RAM (Memory) Controller is found, but 
> windows doesn’t know what driver to install (and neither do i)
>   

That's the virtio-console device.  I guess we should disable it by default.

-- 
error compiling committee.c: too many arguments to function

RE: Problems KVM-84

[jmandawg]

> Are you using a qcow image?  How large is it?

Yes, it's 33GB.  But with kvm-83 it starts right up with no delay.

Re: Problems KVM-84

[Avi Kivity]

jmandawg wrote:
>> Are you using a qcow image?  How large is it?
>>     
>
> Yes, it's 33GB.  But with kvm-83 it starts right up with no delay.
>   

Can you run qemu like this:

   strace -c -fF qemu ...

And kill qemu immediately when the screen shows up?  Then post the results.

-- 
error compiling committee.c: too many arguments to function

Re: Problems KVM-84

[Avi Kivity]

J- MAN wrote:
> Also the second time i run the command, it only takes around 2-3 
> seconds for the window to open.
> ------------------------------------------------------------------------

I don't think this is a regression from kvm-83.  Large qcow2 files have 
lots of metadata which needs to be loaded up front.  If the kernel 
doesn't have this cached, it will take a long time.

-- 
error compiling committee.c: too many arguments to function

Re: Problems KVM-84

[Avi Kivity]

J- MAN wrote:
>
> > I don't think this is a regression from kvm-83. Large qcow2 files have 
> > lots of metadata which needs to be loaded up front. If the kernel 
> > doesn't have this cached, it will take a long time.
>
> I'm just wondering what happend between 83 and 84, because there is no 
> startup delay in 83.
> My real problem is that libvirt times out when waiting for my virtual 
> machine to start using 84, unless i manually start the machine first 
> to get it cached.
>
> This makes using libvirt & virt-manager useless with 84 unless i 
> pre-start all my VMs manually to get them cached.
>
> I'm suprised nobody else is having this issue.  If there is anything 
> else i can do to help please let me know.


Try (with kvm-83):

  echo 255 > /proc/sys/vm/drop-caches
  qemu ...

-- 
error compiling committee.c: too many arguments to function

Re: Problems KVM-84

[Avi Kivity]

Jay Mann wrote:
> Hi,
>
> I just downloaded built and installed kvm-84 on ubuntu Hardy x64  2.6.24-23-
> server and I’m getting the following 2 problems that did not exists in kvm-83.
>
> 1.	The qemu emulater (bios screen) takes a long time to start (~10 
> seconds), and subsequently Libvirt times out when I try to start a guest VM 
> from virsh.  
>   

This is caused by qemu r6404:

commit 5d4cbd78aa33f6d034a62207c99ad0b64af44621
Author: aliguori <aliguori@c046a42c-6fe2-441c-8c8c-71466251a162>
Date:   Thu Jan 22 18:57:22 2009 +0000

    block-qcow2: keep highest allocated byte (Uri Lublin)
   
    We want to know the highest written offset for qcow2 images.
    This gives a pretty good (and easy to calculate) estimation to how
    much more allocation can be done for the block device.
   
    It can be usefull for allocating more diskspace for that image
    (if possible, e.g. lvm) before we run out-of-disk-space
   
    Signed-off-by: Uri Lublin <uril@redhat.com>
    Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>

Scanning the file at startup is slow.  We need to find a better way.

-- 
error compiling committee.c: too many arguments to function

Re: Problems KVM-84

[Anthony Liguori]

Avi Kivity wrote:
> Jay Mann wrote:
>> Hi,
>>
>> I just downloaded built and installed kvm-84 on ubuntu Hardy x64  
>> 2.6.24-23-
>> server and I’m getting the following 2 problems that did not exists 
>> in kvm-83.
>>
>> 1.    The qemu emulater (bios screen) takes a long time to start (~10 
>> seconds), and subsequently Libvirt times out when I try to start a 
>> guest VM from virsh.    
>
> This is caused by qemu r6404:
>
> commit 5d4cbd78aa33f6d034a62207c99ad0b64af44621
> Author: aliguori <aliguori@c046a42c-6fe2-441c-8c8c-71466251a162>
> Date:   Thu Jan 22 18:57:22 2009 +0000
>
>    block-qcow2: keep highest allocated byte (Uri Lublin)
>      We want to know the highest written offset for qcow2 images.
>    This gives a pretty good (and easy to calculate) estimation to how
>    much more allocation can be done for the block device.
>      It can be usefull for allocating more diskspace for that image
>    (if possible, e.g. lvm) before we run out-of-disk-space
>      Signed-off-by: Uri Lublin <uril@redhat.com>
>    Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
>
> Scanning the file at startup is slow.  We need to find a better way.

Any quick ideas?  Seems like this is broken by design.  Unless we can 
find a quick fix, I'm going to revert this in the stable tree.

Regards,

Anthony Liguori

Re: [Qemu-devel] Re: Problems KVM-84

[Avi Kivity]

Anthony Liguori wrote:
>>
>> Scanning the file at startup is slow.  We need to find a better way.
>
> Any quick ideas?  Seems like this is broken by design.  

I agree.

> Unless we can find a quick fix, I'm going to revert this in the stable 
> tree.

We could cache the value in some unused area.  Set it up in qemu-img.  
Don't support it on older images.

But that's not a quick fix; I'd suggest reverting it in both stable and 
mainline, and re-applying in mainline when there's a fix.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Re: Problems KVM-84

[Anthony Liguori]

Avi Kivity wrote:
> Anthony Liguori wrote:
>>>
>>> Scanning the file at startup is slow.  We need to find a better way.
>>
>> Any quick ideas?  Seems like this is broken by design.  
>
> I agree.
>
>> Unless we can find a quick fix, I'm going to revert this in the 
>> stable tree.
>
> We could cache the value in some unused area.  Set it up in qemu-img.  
> Don't support it on older images.
>
> But that's not a quick fix; I'd suggest reverting it in both stable 
> and mainline, and re-applying in mainline when there's a fix.

Done.  Uri, please submit the series again when you've found a solution 
to these problems.

Regards,

Anthony Liguori

Re: [Qemu-devel] Re: Problems KVM-84

[Jamie Lokier]

Anthony Liguori wrote:
> >   block-qcow2: keep highest allocated byte (Uri Lublin)
> >     We want to know the highest written offset for qcow2 images.
> >   This gives a pretty good (and easy to calculate) estimation to how
> >   much more allocation can be done for the block device.
> >     It can be usefull for allocating more diskspace for that image
> >   (if possible, e.g. lvm) before we run out-of-disk-space
> >     Signed-off-by: Uri Lublin <uril@redhat.com>
> >   Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
> >
> >Scanning the file at startup is slow.  We need to find a better way.
> 
> Any quick ideas?  Seems like this is broken by design.  Unless we can 
> find a quick fix, I'm going to revert this in the stable tree.

I still don't understand the point in that feature.

You have the size of the qcow2 file (how much data is used), and the
size of the image it's representing (how much it could expand to).

What does the highest written offset help you anticipate?

Many guest filesystems write data all over the block device, not
concentrated at the start.  (E.g. ext2/3/4 and it's block groups).

-- Jamie

Re: Re: Problems KVM-84

[Uri Lublin]

Jamie Lokier wrote:
> Anthony Liguori wrote:
>>>   block-qcow2: keep highest allocated byte (Uri Lublin)
>>>     We want to know the highest written offset for qcow2 images.
>>>   This gives a pretty good (and easy to calculate) estimation to how
>>>   much more allocation can be done for the block device.
>>>     It can be usefull for allocating more diskspace for that image
>>>   (if possible, e.g. lvm) before we run out-of-disk-space
>>>     Signed-off-by: Uri Lublin <uril@redhat.com>
>>>   Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
>>>
>>> Scanning the file at startup is slow.  We need to find a better way.
>> Any quick ideas?  Seems like this is broken by design.  Unless we can 
>> find a quick fix, I'm going to revert this in the stable tree.
> 
> I still don't understand the point in that feature.
> 
> You have the size of the qcow2 file (how much data is used), and the
> size of the image it's representing (how much it could expand to).
> 
> What does the highest written offset help you anticipate?
You assume there is a file system installed. If I want to use e.g. LVM's logical 
volume, I can not tell the size of the qcow2 image. I want to be able to set a 
watermark and handle out-of-disk-space before it actually happens.

> 
> Many guest filesystems write data all over the block device, not
> concentrated at the start.  (E.g. ext2/3/4 and it's block groups).
qcow2 allocates disk space regardless of the guest write offset.
And if it happen that the image is sparse (or fragmented), num_free_bytes can 
tell us that.

Regards,
     Uri.

Re: Problems KVM-84

[Uri Lublin]

[-- Attachment #1: Type: text/plain, Size: 2041 bytes --]

Anthony Liguori wrote:
> Avi Kivity wrote:
>> Jay Mann wrote:
>>> Hi,
>>>
>>> I just downloaded built and installed kvm-84 on ubuntu Hardy x64  
>>> 2.6.24-23-
>>> server and I’m getting the following 2 problems that did not exists 
>>> in kvm-83.
>>>
>>> 1.    The qemu emulater (bios screen) takes a long time to start (~10 
>>> seconds), and subsequently Libvirt times out when I try to start a 
>>> guest VM from virsh.    
>>
>> This is caused by qemu r6404:
>>
>> commit 5d4cbd78aa33f6d034a62207c99ad0b64af44621
>> Author: aliguori <aliguori@c046a42c-6fe2-441c-8c8c-71466251a162>
>> Date:   Thu Jan 22 18:57:22 2009 +0000
>>
>>    block-qcow2: keep highest allocated byte (Uri Lublin)
>>      We want to know the highest written offset for qcow2 images.
>>    This gives a pretty good (and easy to calculate) estimation to how
>>    much more allocation can be done for the block device.
>>      It can be usefull for allocating more diskspace for that image
>>    (if possible, e.g. lvm) before we run out-of-disk-space
>>      Signed-off-by: Uri Lublin <uril@redhat.com>
>>    Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
>>
>> Scanning the file at startup is slow.  We need to find a better way.
> 
> Any quick ideas?  Seems like this is broken by design.  Unless we can 
> find a quick fix, I'm going to revert this in the stable tree.
> 
> Regards,
> 
> Anthony Liguori
> 

We only need to scan the given filename (no backing files). We can pass a flag 
to qcow_open (bdrv_open of bs->backing_hd) specifying it's a backing file.
(attached 2 patches). That makes things better for small images with big backing 
files. It does not fix the problem for very large images.

A better solution may we to allocate a qcow2-extension to keep highest-alloc and 
num-free-bytes so we don't have to scan refcount table of the image upon open.
With that solution qcow_create initializes them, qcow_open reads them and 
bdrv_close updates them. We can also add a qemu-img command to scan and update 
those values.

Regards,
     Uri.

[-- Attachment #2: 0001-block-pass-BDRV_BACKING-flag-to-backing-file-open.patch --]
[-- Type: text/x-patch, Size: 1782 bytes --]

>From 1ccf1940e0b45a9001b916bc6160c03a098a5d1d Mon Sep 17 00:00:00 2001
From: Uri Lublin <uril@redhat.com>
Date: Wed, 11 Mar 2009 20:16:23 +0200
Subject: [PATCH] block: pass BDRV_BACKING flag to backing-file open

With this information, open can behave differently for the image
filename and its backing files.

Signed-off-by: Uri Lublin <uril@redhat.com>
---
 qemu/block.c |    3 ++-
 qemu/block.h |    2 ++
 2 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/qemu/block.c b/qemu/block.c
index 4c556ec..45fa5f4 100644
--- a/qemu/block.c
+++ b/qemu/block.c
@@ -397,7 +397,7 @@ int bdrv_open2(BlockDriverState *bs, const char *filename, int flags,
     /* Note: for compatibility, we open disk image files as RDWR, and
        RDONLY as fallback */
     if (!(flags & BDRV_O_FILE))
-        open_flags = BDRV_O_RDWR | (flags & BDRV_O_CACHE_MASK);
+        open_flags = BDRV_O_RDWR | (flags & (BDRV_O_CACHE_MASK | BDRV_O_BACKING));
     else
         open_flags = flags & ~(BDRV_O_FILE | BDRV_O_SNAPSHOT);
     ret = drv->bdrv_open(bs, filename, open_flags);
@@ -429,6 +429,7 @@ int bdrv_open2(BlockDriverState *bs, const char *filename, int flags,
         }
         path_combine(backing_filename, sizeof(backing_filename),
                      filename, bs->backing_file);
+        open_flags |= BDRV_O_BACKING;
         if (bdrv_open(bs->backing_hd, backing_filename, open_flags) < 0)
             goto fail;
     }
diff --git a/qemu/block.h b/qemu/block.h
index e1927dd..ff70d64 100644
--- a/qemu/block.h
+++ b/qemu/block.h
@@ -56,6 +56,8 @@ typedef struct QEMUSnapshotInfo {
 
 #define BDRV_O_CACHE_MASK  (BDRV_O_NOCACHE | BDRV_O_CACHE_WB | BDRV_O_CACHE_DEF)
 
+#define BDRV_O_BACKING     0x0100
+
 void bdrv_info(void);
 void bdrv_info_stats(void);
 
-- 
1.6.0.6


[-- Attachment #3: 0002-block-qcow2-do-not-scan-refcounts-when-opening-a-ba.patch --]
[-- Type: text/x-patch, Size: 930 bytes --]

>From c618b293a64d1690105505d7cb28ba1ca8dd33c7 Mon Sep 17 00:00:00 2001
From: Uri Lublin <uril@redhat.com>
Date: Wed, 11 Mar 2009 20:18:23 +0200
Subject: [PATCH] block-qcow2: do not scan refcounts when opening a backing file

It takes too long and is not needed.

Signed-off-by: Uri Lublin <uril@redhat.com>
---
 qemu/block-qcow2.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/qemu/block-qcow2.c b/qemu/block-qcow2.c
index 465dcd6..41cdbe9 100644
--- a/qemu/block-qcow2.c
+++ b/qemu/block-qcow2.c
@@ -276,7 +276,8 @@ static int qcow_open(BlockDriverState *bs, const char *filename, int flags)
     if (refcount_init(bs) < 0)
         goto fail;
 
-    scan_refcount(bs, &s->highest_alloc, &s->nc_free);
+    if ((flags & BDRV_O_BACKING) == 0)
+        scan_refcount(bs, &s->highest_alloc, &s->nc_free);
 
     /* read the backing file name */
     if (header.backing_file_offset != 0) {
-- 
1.6.0.6