[Qemu-devel] qemu-img create doesn't always replace the existing file

[Qemu-devel] qemu-img create doesn't always replace the existing file

[Richard W.M. Jones]

When using 'qemu-img create', if the file being created already
exists, then qemu-img tries to read it first.  This has some
unexpected effects:


$ rm test.qcow2 
$ qemu-img create -f qcow2 -o compat=1.1,preallocation=off test.qcow2 1G
Formatting 'test.qcow2', fmt=qcow2 size=1073741824 compat=1.1 encryption=off cluster_size=65536 preallocation=off lazy_refcounts=off refcount_bits=16
$ du -sh test.qcow2 
196K test.qcow2


$ rm test.qcow2 
$ qemu-img create -f qcow2 -o compat=1.1,preallocation=falloc test.qcow2 1G
Formatting 'test.qcow2', fmt=qcow2 size=1073741824 compat=1.1 encryption=off cluster_size=65536 preallocation=falloc lazy_refcounts=off refcount_bits=16
$ qemu-img create -f qcow2 -o compat=1.1,preallocation=off test.qcow2 1G
Formatting 'test.qcow2', fmt=qcow2 size=1073741824 compat=1.1 encryption=off cluster_size=65536 preallocation=off lazy_refcounts=off refcount_bits=16
$ du -sh test.qcow2 
256K test.qcow2            # would expect this to be the same as above


$ rm test.qcow2 
$ mkfifo test.qcow2
$ qemu-img create -f qcow2 -o compat=1.1 test.qcow2 64M
Formatting 'test.qcow2', fmt=qcow2 size=67108864 compat=1.1 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16
qemu-img: test.qcow2: Could not resize file: Invalid argument


Is this a bug or is my understanding of 'qemu-img create' wrong?

Currently libguestfs uses 'qemu-img create' to create empty files.
However we don't unlink the target file first.  Should we be doing that?


Tested with:

qemu-img-2.6.0-3.fc24.x86_64
qemu-img-2.7.0-1.fc25.x86_64

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/

Re: [Qemu-devel] qemu-img create doesn't always replace the existing file

[Kevin Wolf]

[ Cc: qemu-block ]

Am 08.11.2016 um 11:58 hat Richard W.M. Jones geschrieben:
> When using 'qemu-img create', if the file being created already
> exists, then qemu-img tries to read it first.  This has some
> unexpected effects:
> 
> 
> $ rm test.qcow2 
> $ qemu-img create -f qcow2 -o compat=1.1,preallocation=off test.qcow2 1G
> Formatting 'test.qcow2', fmt=qcow2 size=1073741824 compat=1.1 encryption=off cluster_size=65536 preallocation=off lazy_refcounts=off refcount_bits=16
> $ du -sh test.qcow2 
> 196K test.qcow2
> 
> 
> $ rm test.qcow2 
> $ qemu-img create -f qcow2 -o compat=1.1,preallocation=falloc test.qcow2 1G
> Formatting 'test.qcow2', fmt=qcow2 size=1073741824 compat=1.1 encryption=off cluster_size=65536 preallocation=falloc lazy_refcounts=off refcount_bits=16
> $ qemu-img create -f qcow2 -o compat=1.1,preallocation=off test.qcow2 1G
> Formatting 'test.qcow2', fmt=qcow2 size=1073741824 compat=1.1 encryption=off cluster_size=65536 preallocation=off lazy_refcounts=off refcount_bits=16
> $ du -sh test.qcow2 
> 256K test.qcow2            # would expect this to be the same as above

For me it's actually even more:

$ du -h /tmp/test.qcow2 
448K    /tmp/test.qcow2

However...

$ ls -lh /tmp/test.qcow2 
-rw-r--r--. 1 kwolf kwolf 193K  8. Nov 15:00 /tmp/test.qcow2

So qemu-img can't be at fault, the file has the same size as always.

Are you using XFS? In my case I would have guessed that it's probably
some preallocation thing that XFS does internally. We've seen this
before that 'du' shows (sometimes by far) larger values than the file
size on XFS. That space is reclaimed later, though.

Kevin

Re: [Qemu-devel] qemu-img create doesn't always replace the existing file

[Richard W.M. Jones]

On Tue, Nov 08, 2016 at 03:05:24PM +0100, Kevin Wolf wrote:
> [ Cc: qemu-block ]
> 
> Am 08.11.2016 um 11:58 hat Richard W.M. Jones geschrieben:
> > When using 'qemu-img create', if the file being created already
> > exists, then qemu-img tries to read it first.  This has some
> > unexpected effects:
> > 
> > 
> > $ rm test.qcow2 
> > $ qemu-img create -f qcow2 -o compat=1.1,preallocation=off test.qcow2 1G
> > Formatting 'test.qcow2', fmt=qcow2 size=1073741824 compat=1.1 encryption=off cluster_size=65536 preallocation=off lazy_refcounts=off refcount_bits=16
> > $ du -sh test.qcow2 
> > 196K test.qcow2
> > 
> > 
> > $ rm test.qcow2 
> > $ qemu-img create -f qcow2 -o compat=1.1,preallocation=falloc test.qcow2 1G
> > Formatting 'test.qcow2', fmt=qcow2 size=1073741824 compat=1.1 encryption=off cluster_size=65536 preallocation=falloc lazy_refcounts=off refcount_bits=16
> > $ qemu-img create -f qcow2 -o compat=1.1,preallocation=off test.qcow2 1G
> > Formatting 'test.qcow2', fmt=qcow2 size=1073741824 compat=1.1 encryption=off cluster_size=65536 preallocation=off lazy_refcounts=off refcount_bits=16
> > $ du -sh test.qcow2 
> > 256K test.qcow2            # would expect this to be the same as above
> 
> For me it's actually even more:
> 
> $ du -h /tmp/test.qcow2 
> 448K    /tmp/test.qcow2
> 
> However...
> 
> $ ls -lh /tmp/test.qcow2 
> -rw-r--r--. 1 kwolf kwolf 193K  8. Nov 15:00 /tmp/test.qcow2
> 
> So qemu-img can't be at fault, the file has the same size as always.
> 
> Are you using XFS? In my case I would have guessed that it's probably
> some preallocation thing that XFS does internally. We've seen this
> before that 'du' shows (sometimes by far) larger values than the file
> size on XFS. That space is reclaimed later, though.

Yes I am, and indeed this looks like a filesystem artifact and not
a problem with qemu-img.

Thanks,

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org