[Qemu-devel] need to resurrect no-lock option?

[Qemu-devel] need to resurrect no-lock option?

[Christian Ehrhardt]

Hi,
this might have been discussed in the wake of the lock changes that took
place in 2.10 but I can't find anything clear enough to follow in the
current case.
There was an old submission [1] by Fam to make it possible to no-lock
qemu-img and others if needed. But it seems nothing like it made it along
to the locking we have in 2.10.

One (maybe more) case where missing this causes pain is e.g. running an
info check on a running guest.
In general info shouldn't need a write lock anyway, but without --no-lock
that use case is broken.

Repro of the case is rather simple:
$ qemu-img create -f qcow2 /tmp/test.qcow2 1M
$ qemu-system-x86_64 -hda /tmp/test.qcow2 -enable-kvm -nodefaults
-nographic &
$ qemu-img info /tmp/test.qcow2
qemu-img: Could not open '/tmp/test.qcow2': Failed to get shared "write"
lock
Is another process using the image?

[1]: https://lists.gnu.org/archive/html/qemu-block/2016-04/msg00349.html

-- 
Christian Ehrhardt
Software Engineer, Ubuntu Server
Canonical Ltd

Re: [Qemu-devel] need to resurrect no-lock option?

[Fam Zheng]

On Wed, 09/20 11:26, Christian Ehrhardt wrote:
> Hi,
> this might have been discussed in the wake of the lock changes that took
> place in 2.10 but I can't find anything clear enough to follow in the
> current case.
> There was an old submission [1] by Fam to make it possible to no-lock
> qemu-img and others if needed. But it seems nothing like it made it along
> to the locking we have in 2.10.
> 
> One (maybe more) case where missing this causes pain is e.g. running an
> info check on a running guest.
> In general info shouldn't need a write lock anyway, but without --no-lock
> that use case is broken.

The --no-lock option was later revised into --force-share, so...

> 
> Repro of the case is rather simple:
> $ qemu-img create -f qcow2 /tmp/test.qcow2 1M
> $ qemu-system-x86_64 -hda /tmp/test.qcow2 -enable-kvm -nodefaults
> -nographic &
> $ qemu-img info /tmp/test.qcow2
> qemu-img: Could not open '/tmp/test.qcow2': Failed to get shared "write"
> lock
> Is another process using the image?

... you can do this:

$ qemu-img info --force-share /tmp/test.qcow2
image: /tmp/test.qcow2
file format: qcow2
virtual size: 1.0M (1048576 bytes)
disk size: 196K
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

Fam

Re: [Qemu-devel] need to resurrect no-lock option?

[Christian Ehrhardt]

On Wed, Sep 20, 2017 at 11:51 AM, Fam Zheng <famz@redhat.com> wrote:

> On Wed, 09/20 11:26, Christian Ehrhardt wrote:
> > Hi,
> > this might have been discussed in the wake of the lock changes that took
> > place in 2.10 but I can't find anything clear enough to follow in the
> > current case.
> > There was an old submission [1] by Fam to make it possible to no-lock
> > qemu-img and others if needed. But it seems nothing like it made it along
> > to the locking we have in 2.10.
> >
> > One (maybe more) case where missing this causes pain is e.g. running an
> > info check on a running guest.
> > In general info shouldn't need a write lock anyway, but without --no-lock
> > that use case is broken.
>
> The --no-lock option was later revised into --force-share, so...
>

Thanks for the remapping of this old patch to what it ended up!
Test works for me, need to check with other affected cases.

>
> > Repro of the case is rather simple:
> > $ qemu-img create -f qcow2 /tmp/test.qcow2 1M
> > $ qemu-system-x86_64 -hda /tmp/test.qcow2 -enable-kvm -nodefaults
> > -nographic &
> > $ qemu-img info /tmp/test.qcow2
> > qemu-img: Could not open '/tmp/test.qcow2': Failed to get shared "write"
> > lock
> > Is another process using the image?
>
> ... you can do this:
>
> $ qemu-img info --force-share /tmp/test.qcow2
> image: /tmp/test.qcow2
> file format: qcow2
> virtual size: 1.0M (1048576 bytes)
> disk size: 196K
> cluster_size: 65536
> Format specific information:
>     compat: 1.1
>     lazy refcounts: false
>     refcount bits: 16
>     corrupt: false
>
> Fam
>



-- 
Christian Ehrhardt
Software Engineer, Ubuntu Server
Canonical Ltd

Re: [Qemu-devel] need to resurrect no-lock option?

[Stefan Hajnoczi]

On Wed, Sep 20, 2017 at 11:26:11AM +0200, Christian Ehrhardt wrote:
> Hi,
> this might have been discussed in the wake of the lock changes that took
> place in 2.10 but I can't find anything clear enough to follow in the
> current case.
> There was an old submission [1] by Fam to make it possible to no-lock
> qemu-img and others if needed. But it seems nothing like it made it along
> to the locking we have in 2.10.
> 
> One (maybe more) case where missing this causes pain is e.g. running an
> info check on a running guest.
> In general info shouldn't need a write lock anyway, but without --no-lock
> that use case is broken.

It's still an invalid use case.  There is no guarantee that qemu-img
will see a consistent version of the image file.  Metadata could change
underneath qemu-img because QEMU may still write to it.  QEMU may also
have some metadata cached so there's no guarantee that qemu-img sees an
up-to-date image.

Why do you need to run qemu-img on a disk image that is in use?

> Repro of the case is rather simple:
> $ qemu-img create -f qcow2 /tmp/test.qcow2 1M
> $ qemu-system-x86_64 -hda /tmp/test.qcow2 -enable-kvm -nodefaults
> -nographic &
> $ qemu-img info /tmp/test.qcow2
> qemu-img: Could not open '/tmp/test.qcow2': Failed to get shared "write"
> lock
> Is another process using the image?
> 
> [1]: https://lists.gnu.org/archive/html/qemu-block/2016-04/msg00349.html
> 
> -- 
> Christian Ehrhardt
> Software Engineer, Ubuntu Server
> Canonical Ltd

Re: [Qemu-devel] need to resurrect no-lock option?

[Daniel P. Berrange]

On Thu, Sep 21, 2017 at 01:33:20PM +0100, Stefan Hajnoczi wrote:
> On Wed, Sep 20, 2017 at 11:26:11AM +0200, Christian Ehrhardt wrote:
> > Hi,
> > this might have been discussed in the wake of the lock changes that took
> > place in 2.10 but I can't find anything clear enough to follow in the
> > current case.
> > There was an old submission [1] by Fam to make it possible to no-lock
> > qemu-img and others if needed. But it seems nothing like it made it along
> > to the locking we have in 2.10.
> > 
> > One (maybe more) case where missing this causes pain is e.g. running an
> > info check on a running guest.
> > In general info shouldn't need a write lock anyway, but without --no-lock
> > that use case is broken.
> 
> It's still an invalid use case.  There is no guarantee that qemu-img
> will see a consistent version of the image file.  Metadata could change
> underneath qemu-img because QEMU may still write to it.  QEMU may also
> have some metadata cached so there's no guarantee that qemu-img sees an
> up-to-date image.
> 
> Why do you need to run qemu-img on a disk image that is in use?

You have a directory full of images and you want to understand current usage
vs potential future usage. For this you need to get the virtual size, which
rquires 'qemu-img info' for non-raw files. Actually it would be even better
served by the new 'measure' command recently added.

The job analyzing this directory of images may not have any context as to
whether each file is in use by a running QEMU, so would just blindly call
'qemu-img info' on each file it finds.  There's of course potential that
opening the image in 'qemu-img info' could hit problems if the running QEMU
changed qcow2 metadata, but generally that would not have serious negative
impact and would be self-correcting next time the job analysed the directory.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] need to resurrect no-lock option?

[Fam Zheng]

On Thu, 09/21 13:43, Daniel P. Berrange wrote:
> On Thu, Sep 21, 2017 at 01:33:20PM +0100, Stefan Hajnoczi wrote:
> > On Wed, Sep 20, 2017 at 11:26:11AM +0200, Christian Ehrhardt wrote:
> > > Hi,
> > > this might have been discussed in the wake of the lock changes that took
> > > place in 2.10 but I can't find anything clear enough to follow in the
> > > current case.
> > > There was an old submission [1] by Fam to make it possible to no-lock
> > > qemu-img and others if needed. But it seems nothing like it made it along
> > > to the locking we have in 2.10.
> > > 
> > > One (maybe more) case where missing this causes pain is e.g. running an
> > > info check on a running guest.
> > > In general info shouldn't need a write lock anyway, but without --no-lock
> > > that use case is broken.
> > 
> > It's still an invalid use case.  There is no guarantee that qemu-img
> > will see a consistent version of the image file.  Metadata could change
> > underneath qemu-img because QEMU may still write to it.  QEMU may also
> > have some metadata cached so there's no guarantee that qemu-img sees an
> > up-to-date image.
> > 
> > Why do you need to run qemu-img on a disk image that is in use?
> 
> You have a directory full of images and you want to understand current usage
> vs potential future usage. For this you need to get the virtual size, which
> rquires 'qemu-img info' for non-raw files. Actually it would be even better
> served by the new 'measure' command recently added.
> 
> The job analyzing this directory of images may not have any context as to
> whether each file is in use by a running QEMU, so would just blindly call
> 'qemu-img info' on each file it finds.  There's of course potential that
> opening the image in 'qemu-img info' could hit problems if the running QEMU
> changed qcow2 metadata, but generally that would not have serious negative
> impact and would be self-correcting next time the job analysed the directory.

Probably it's helpful to add a hint about "--force-share" the in the error
message of "qemu-img info" when hitting an image lock failure?

Fam

Re: [Qemu-devel] need to resurrect no-lock option?

[Daniel P. Berrange]

On Thu, Sep 21, 2017 at 01:43:17PM +0100, Daniel P. Berrange wrote:
> On Thu, Sep 21, 2017 at 01:33:20PM +0100, Stefan Hajnoczi wrote:
> > On Wed, Sep 20, 2017 at 11:26:11AM +0200, Christian Ehrhardt wrote:
> > > Hi,
> > > this might have been discussed in the wake of the lock changes that took
> > > place in 2.10 but I can't find anything clear enough to follow in the
> > > current case.
> > > There was an old submission [1] by Fam to make it possible to no-lock
> > > qemu-img and others if needed. But it seems nothing like it made it along
> > > to the locking we have in 2.10.
> > > 
> > > One (maybe more) case where missing this causes pain is e.g. running an
> > > info check on a running guest.
> > > In general info shouldn't need a write lock anyway, but without --no-lock
> > > that use case is broken.
> > 
> > It's still an invalid use case.  There is no guarantee that qemu-img
> > will see a consistent version of the image file.  Metadata could change
> > underneath qemu-img because QEMU may still write to it.  QEMU may also
> > have some metadata cached so there's no guarantee that qemu-img sees an
> > up-to-date image.
> > 
> > Why do you need to run qemu-img on a disk image that is in use?
> 
> You have a directory full of images and you want to understand current usage
> vs potential future usage. For this you need to get the virtual size, which
> rquires 'qemu-img info' for non-raw files. Actually it would be even better
> served by the new 'measure' command recently added.
> 
> The job analyzing this directory of images may not have any context as to
> whether each file is in use by a running QEMU, so would just blindly call
> 'qemu-img info' on each file it finds.  There's of course potential that
> opening the image in 'qemu-img info' could hit problems if the running QEMU
> changed qcow2 metadata, but generally that would not have serious negative
> impact and would be self-correcting next time the job analysed the directory.

FYI this scenario has already hit OpenStack with breakage on new QMEU

  https://bugs.launchpad.net/nova/+bug/1718295

and thus they're just going to add '--force-shared' for all usage of
qemu-img info.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] need to resurrect no-lock option?

[Kevin Wolf]

Am 21.09.2017 um 15:18 hat Daniel P. Berrange geschrieben:
> On Thu, Sep 21, 2017 at 01:43:17PM +0100, Daniel P. Berrange wrote:
> > On Thu, Sep 21, 2017 at 01:33:20PM +0100, Stefan Hajnoczi wrote:
> > > On Wed, Sep 20, 2017 at 11:26:11AM +0200, Christian Ehrhardt wrote:
> > > > Hi,
> > > > this might have been discussed in the wake of the lock changes that took
> > > > place in 2.10 but I can't find anything clear enough to follow in the
> > > > current case.
> > > > There was an old submission [1] by Fam to make it possible to no-lock
> > > > qemu-img and others if needed. But it seems nothing like it made it along
> > > > to the locking we have in 2.10.
> > > > 
> > > > One (maybe more) case where missing this causes pain is e.g. running an
> > > > info check on a running guest.
> > > > In general info shouldn't need a write lock anyway, but without --no-lock
> > > > that use case is broken.
> > > 
> > > It's still an invalid use case.  There is no guarantee that qemu-img
> > > will see a consistent version of the image file.  Metadata could change
> > > underneath qemu-img because QEMU may still write to it.  QEMU may also
> > > have some metadata cached so there's no guarantee that qemu-img sees an
> > > up-to-date image.
> > > 
> > > Why do you need to run qemu-img on a disk image that is in use?
> > 
> > You have a directory full of images and you want to understand current usage
> > vs potential future usage. For this you need to get the virtual size, which
> > rquires 'qemu-img info' for non-raw files. Actually it would be even better
> > served by the new 'measure' command recently added.
> > 
> > The job analyzing this directory of images may not have any context as to
> > whether each file is in use by a running QEMU, so would just blindly call
> > 'qemu-img info' on each file it finds.  There's of course potential that
> > opening the image in 'qemu-img info' could hit problems if the running QEMU
> > changed qcow2 metadata, but generally that would not have serious negative
> > impact and would be self-correcting next time the job analysed the directory.
> 
> FYI this scenario has already hit OpenStack with breakage on new QMEU
> 
>   https://bugs.launchpad.net/nova/+bug/1718295
> 
> and thus they're just going to add '--force-shared' for all usage of
> qemu-img info.

As you know, the information returned by qemu-img with --force-shared is
not reliable if a VM is writing to the image. If you're okay with that,
you can use it, but I think it's appropriate to require an option for
this.

If you need something that works reliably with images that are in use,
we should be talking about QMP commands instead and libvirt should
decide which method needs to be used for each call.

Kevin