[Qemu-devel] qemu-img convert corruption (raw to raw)

[Qemu-devel] qemu-img convert corruption (raw to raw)

[Dominik Csapak]

Hi,

since i got no answer on the qemu-discuss mailing list, i post it here
maybe someone can help me or point me in the right direction
(copy-pasted from my previous mail)

Hi,

I have a strange issue with qemu-img convert which corrupts my images, 
and found nothing relevant online.
Maybe someone here can help me.

My problem:

I have a Windows 10 VM with a hd image file of type RAW with cache mode 
writeback.
I boot it up, write a bunch of data (a few gigabytes with crystal disk 
mark) and shutdown.
As soon as the qemu/kvm process is gone i do a
"qemu-img convert -p -f raw -O raw vmdisk.raw vmdisk2.raw"
When it finishes i do a compare:
"cmp vmdisk.raw vmidsk2.raw"
The output is:
"vmdisk.raw vmdisk2.raw differ: byte yyyyy, line zzzzz"
this it tells me that the files are different.
this manifests by the fact that sometimes in windows on the new disk i 
get filesystem errors or damaged files.
(depending on where the writes were)

now the strange part:
it does not happen with linux guests
it does not happen when i do a "echo 3 > /proc/sys/vm/drop_caches" 
before the qemu-img command
it does not happen when i have the "-t none" or "-t writethrough" flags 
on my qemu-img command
it DOES happen when i do a sync before the qemu-img command
the "-T" parameter makes no difference
it does also not happen when i cp the file
also when i wait a few minutes before the qemu-img command it works 
correctly

now the real questions:
is writeback with windows not recommended (i would assume after a guest 
shutdown it would be safe)
what do the -t and -T flags on qemu-img convert and why does it bypass 
the host page cache (apparently?)
why does it work when i specify no caching on the target
(this baffles me, since it would seem that the source cache is the 
culprit, not the target)

i apologize for the long message
and hope someone can explain this to me

with kind regards
Dominik Csapak

Re: [Qemu-devel] qemu-img convert corruption (raw to raw)

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 2140 bytes --]

On Wed, Mar 23, 2016 at 09:20:06AM +0100, Dominik Csapak wrote:
> I have a strange issue with qemu-img convert which corrupts my images, and
> found nothing relevant online.
> Maybe someone here can help me.

Please post your QEMU command-line (ps aux | grep qemu while the guest
is running).

Please also post your host distro and kernel version (uname -r).

> My problem:
> 
> I have a Windows 10 VM with a hd image file of type RAW with cache mode
> writeback.
> I boot it up, write a bunch of data (a few gigabytes with crystal disk mark)
> and shutdown.
> As soon as the qemu/kvm process is gone i do a
> "qemu-img convert -p -f raw -O raw vmdisk.raw vmdisk2.raw"

Have you tried replacing qemu-img convert with cp?

> When it finishes i do a compare:
> "cmp vmdisk.raw vmidsk2.raw"
> The output is:
> "vmdisk.raw vmdisk2.raw differ: byte yyyyy, line zzzzz"
> this it tells me that the files are different.
> this manifests by the fact that sometimes in windows on the new disk i get
> filesystem errors or damaged files.
> (depending on where the writes were)
> 
> now the strange part:
> it does not happen with linux guests
> it does not happen when i do a "echo 3 > /proc/sys/vm/drop_caches" before
> the qemu-img command
> it does not happen when i have the "-t none" or "-t writethrough" flags on
> my qemu-img command
> it DOES happen when i do a sync before the qemu-img command
> the "-T" parameter makes no difference
> it does also not happen when i cp the file
> also when i wait a few minutes before the qemu-img command it works
> correctly
> 
> now the real questions:
> is writeback with windows not recommended (i would assume after a guest
> shutdown it would be safe)
> what do the -t and -T flags on qemu-img convert and why does it bypass the
> host page cache (apparently?)
> why does it work when i specify no caching on the target
> (this baffles me, since it would seem that the source cache is the culprit,
> not the target)
> 
> i apologize for the long message
> and hope someone can explain this to me
> 
> with kind regards
> Dominik Csapak
> 
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]