[Qemu-devel] qemu-img commit -- is there a limit on file sizes?

[Qemu-devel] qemu-img commit -- is there a limit on file sizes?

[walt]

Some background for my question:  I've been trying to install
and then update Windows Vista using kvm.  Everything works great
until I use 'qemu-img commit' to apply all the Windows Updates
to my original base install of Vista.

After doing the qemu-img commit step, the backing file is now
corrupt, 100% reproducibly.  I don't have the same problem with
Windows XP, however, and I wondered if the problem is caused by
the sheer size of the commit that Vista requires.

When I install XP, then windows-update, and then qemu-img commit
the updates, I'm committing about 1GB of updates to a 3GB backing
file.

When I install Vista and then later commit the Vista updates, I'm
committing a 3GB file to a 6GB backing file, and that's when the
corruption happens every time.

So I tried an experiment with Vista -- I deliberately limit the
number of windows updates I allow at any one time, and then use
qemu-img commit after each small update.  Voila, everything now
works perfectly -- no file corruption!

And that's why I suspect there is a functional limit to the size
of each commit I can do with qemu-img.

Any thoughts or possible diagnostic maneuvers to be tried?

Thanks!

(BTW, I get the same results using 32-bit linux and 64-bit linux
on the same amd64 machine, using both gcc3 and gcc4.)

Re: [Qemu-devel] qemu-img commit -- is there a limit on file sizes?

[Anthony Liguori]

walt wrote:
> Some background for my question:  I've been trying to install
> and then update Windows Vista using kvm.  Everything works great
> until I use 'qemu-img commit' to apply all the Windows Updates
> to my original base install of Vista.
>
> After doing the qemu-img commit step, the backing file is now
> corrupt, 100% reproducibly.  I don't have the same problem with
> Windows XP, however, and I wondered if the problem is caused by
> the sheer size of the commit that Vista requires.
>
> When I install XP, then windows-update, and then qemu-img commit
> the updates, I'm committing about 1GB of updates to a 3GB backing
> file.
>
> When I install Vista and then later commit the Vista updates, I'm
> committing a 3GB file to a 6GB backing file, and that's when the
> corruption happens every time.
>
> So I tried an experiment with Vista -- I deliberately limit the
> number of windows updates I allow at any one time, and then use
> qemu-img commit after each small update.  Voila, everything now
> works perfectly -- no file corruption!

We've started getting some reports of corruption on "commit" in KVM.  
There is a long standing disk corruption issue too that is very 
difficult to reproduce.  The thinking is that there is a bug somewhere 
in the qcow2 code.

Is anyone actively looking into this?

Regards,

Anthony Liguori

> And that's why I suspect there is a functional limit to the size
> of each commit I can do with qemu-img.
>
> Any thoughts or possible diagnostic maneuvers to be tried?
>
> Thanks!
>
> (BTW, I get the same results using 32-bit linux and 64-bit linux
> on the same amd64 machine, using both gcc3 and gcc4.)
>
>
>

Re: [Qemu-devel] qemu-img commit -- is there a limit on file sizes?

[Avi Kivity]

Anthony Liguori wrote:
>
> We've started getting some reports of corruption on "commit" in KVM.  
> There is a long standing disk corruption issue too that is very 
> difficult to reproduce.  The thinking is that there is a bug somewhere 
> in the qcow2 code.
>
> Is anyone actively looking into this?
>

I am, though my actively is a lot less than could be desired.  
Additional eyes would be welcome.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: [Qemu-devel] qemu-img commit -- is there a limit on file sizes?

[walt]

On Mon, 1 Dec 2008, Avi Kivity wrote:

> Anthony Liguori wrote:
> >
> > We've started getting some reports of corruption on "commit" in KVM.  There
> > is a long standing disk corruption issue too that is very difficult to
> > reproduce.  The thinking is that there is a bug somewhere in the qcow2 code.
> >
> > Is anyone actively looking into this?
> >
>
> I am, though my actively is a lot less than could be desired.  Additional eyes
> would be welcome.

FWIW, I must apologize for giving you incorrect data.  I'm seeing problems
now that have nothing to do with the size of the commit, and I'm beginning
to suspect that the commit step has nothing to do with the problem.  I'll
summarize my evidence because it seems potentially very important:

I installed WinXP on qcow2, which went perfectly.  I rebooted multiple times
with no problems and changed settings for my desktop and taskbar, rebooted
again with no problems.

Now, I make a new, fresh [what's the opposite of a backing file?] like this:
$qemu-img create -f qcow2 -b kvmXP kvmXP.delta

All I do is boot XP again like this:
$qemu-system-x86_64 -m 1000 kvmXP.delta

My shock is that one of the taskbar settings I changed has disappeared!
I can boot the original kvmXP qcow2 image and verify that my changes are
still there in the original, but not when I boot kvmXP.delta.

In case someone wants to try to reproduce it, the specific change I made
to the task bar is to display the animated network activity icon in the
system tray.  That icon never appears when I boot from kvmXP.delta.

In other words, from the instant I boot kvmXP.delta, the image of XP that
gets loaded into memory is not an accurate reflection of what's in the
backing file.  If that's true, then it's not surprising that the commit
step causes trouble.

Does my reasoning seem reasonable? :o)