[Qemu-devel] [PATCH] monitor: fix object_del for command-line-created objects

[Qemu-devel] [PATCH] monitor: fix object_del for command-line-created objects

[Michael Roth]

Currently objects specified on the command-line are only partially
cleaned up when 'object_del' is issued in either HMP or QMP: the
object itself is fully finalized, but the QemuOpts are not removed.
This results in the following behavior:

  x86_64-softmmu/qemu-system-x86_64 -monitor stdio \
    -object memory-backend-ram,id=ram1,size=256M

  QEMU 2.7.91 monitor - type 'help' for more information
  (qemu) object_del ram1
  (qemu) object_del ram1
  object 'ram1' not found
  (qemu) object_add memory-backend-ram,id=ram1,size=256M
  Duplicate ID 'ram1' for object
  Try "help object_add" for more information

which can be an issue for use-cases like memory hotplug.

This happens on the HMP side because hmp_object_add() attempts to
create a temporary QemuOpts entry with ID 'ram1', which ends up
conflicting with the command-line-created entry, since it was never
cleaned up during the previous hmp_object_del() call.

We address this by adding checks in {qmp,hmp}_object_del to determine
whether an option group entry matching the object's ID is present,
and removing it if it is.

Note that qmp_object_add() never attempts to create a temporary
QemuOpts entry, so it does not encounter the duplicate ID error,
which is why this isn't generally visible in libvirt.

Cc: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
Cc: Markus Armbruster <armbru@redhat.com>
Cc: Eric Blake <eblake@redhat.com>
Cc: Daniel Berrange <berrange@redhat.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
 hmp.c | 10 ++++++++++
 qmp.c | 10 ++++++++++
 2 files changed, 20 insertions(+)

diff --git a/hmp.c b/hmp.c
index b869617..99b8bf3 100644
--- a/hmp.c
+++ b/hmp.c
@@ -2072,6 +2072,16 @@ void hmp_object_del(Monitor *mon, const QDict *qdict)
 
     user_creatable_del(id, &err);
     hmp_handle_error(mon, &err);
+
+    /* if object was defined on the command-line, remove its corresponding
+     * option group entry
+     */
+    if (err == NULL) {
+        QemuOptsList *opt_group = qemu_find_opts_err("object", &err);
+        if (opt_group) {
+            qemu_opts_del(qemu_opts_find(opt_group, id));
+        }
+    }
 }
 
 void hmp_info_memdev(Monitor *mon, const QDict *qdict)
diff --git a/qmp.c b/qmp.c
index 0028f0b..87c545d 100644
--- a/qmp.c
+++ b/qmp.c
@@ -686,6 +686,16 @@ void qmp_object_add(const char *type, const char *id,
 void qmp_object_del(const char *id, Error **errp)
 {
     user_creatable_del(id, errp);
+
+    /* if object was defined on the command-line, remove its corresponding
+     * option group entry
+     */
+    if (!(errp && *errp)) {
+        QemuOptsList *opt_group = qemu_find_opts_err("object", errp);
+        if (opt_group) {
+            qemu_opts_del(qemu_opts_find(opt_group, id));
+        }
+    }
 }
 
 MemoryDeviceInfoList *qmp_query_memory_devices(Error **errp)
-- 
1.9.1

Re: [Qemu-devel] [PATCH for-2.8] monitor: fix object_del for command-line-created objects

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 3238 bytes --]

On 11/30/2016 05:06 PM, Michael Roth wrote:
> Currently objects specified on the command-line are only partially
> cleaned up when 'object_del' is issued in either HMP or QMP: the
> object itself is fully finalized, but the QemuOpts are not removed.
> This results in the following behavior:
> 
>   x86_64-softmmu/qemu-system-x86_64 -monitor stdio \
>     -object memory-backend-ram,id=ram1,size=256M
> 
>   QEMU 2.7.91 monitor - type 'help' for more information
>   (qemu) object_del ram1
>   (qemu) object_del ram1
>   object 'ram1' not found
>   (qemu) object_add memory-backend-ram,id=ram1,size=256M
>   Duplicate ID 'ram1' for object
>   Try "help object_add" for more information
> 
> which can be an issue for use-cases like memory hotplug.

Nice analysis.

> 
> This happens on the HMP side because hmp_object_add() attempts to
> create a temporary QemuOpts entry with ID 'ram1', which ends up
> conflicting with the command-line-created entry, since it was never
> cleaned up during the previous hmp_object_del() call.
> 
> We address this by adding checks in {qmp,hmp}_object_del to determine
> whether an option group entry matching the object's ID is present,
> and removing it if it is.
> 
> Note that qmp_object_add() never attempts to create a temporary
> QemuOpts entry, so it does not encounter the duplicate ID error,
> which is why this isn't generally visible in libvirt.

Phew. Yeah, libvirt avoiding HMP where possible saves a lot of hassles.

> +++ b/hmp.c
> @@ -2072,6 +2072,16 @@ void hmp_object_del(Monitor *mon, const QDict *qdict)
>  
>      user_creatable_del(id, &err);
>      hmp_handle_error(mon, &err);
> +
> +    /* if object was defined on the command-line, remove its corresponding
> +     * option group entry
> +     */
> +    if (err == NULL) {
> +        QemuOptsList *opt_group = qemu_find_opts_err("object", &err);
> +        if (opt_group) {
> +            qemu_opts_del(qemu_opts_find(opt_group, id));
> +        }
> +    }

This one looks okay.

>  }
>  
>  void hmp_info_memdev(Monitor *mon, const QDict *qdict)
> diff --git a/qmp.c b/qmp.c
> index 0028f0b..87c545d 100644
> --- a/qmp.c
> +++ b/qmp.c
> @@ -686,6 +686,16 @@ void qmp_object_add(const char *type, const char *id,
>  void qmp_object_del(const char *id, Error **errp)
>  {
>      user_creatable_del(id, errp);
> +
> +    /* if object was defined on the command-line, remove its corresponding
> +     * option group entry
> +     */
> +    if (!(errp && *errp)) {

But this is wrong. Please spin a v2 that uses a local Error object, then
use error_propagate(err, errp). Making anything conditional on whether
an error occurred requires a local object, since the caller can pass in
NULL, but you want your cleanup to happen even when the caller doesn't
care about errors.

> +        QemuOptsList *opt_group = qemu_find_opts_err("object", errp);
> +        if (opt_group) {
> +            qemu_opts_del(qemu_opts_find(opt_group, id));
> +        }
> +    }
>  }
>  
>  MemoryDeviceInfoList *qmp_query_memory_devices(Error **errp)
> 

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH] monitor: fix object_del for command-line-created objects

[Daniel P. Berrange]

On Wed, Nov 30, 2016 at 05:06:16PM -0600, Michael Roth wrote:
> Currently objects specified on the command-line are only partially
> cleaned up when 'object_del' is issued in either HMP or QMP: the
> object itself is fully finalized, but the QemuOpts are not removed.
> This results in the following behavior:
> 
>   x86_64-softmmu/qemu-system-x86_64 -monitor stdio \
>     -object memory-backend-ram,id=ram1,size=256M
> 
>   QEMU 2.7.91 monitor - type 'help' for more information
>   (qemu) object_del ram1
>   (qemu) object_del ram1
>   object 'ram1' not found
>   (qemu) object_add memory-backend-ram,id=ram1,size=256M
>   Duplicate ID 'ram1' for object
>   Try "help object_add" for more information
> 
> which can be an issue for use-cases like memory hotplug.
> 
> This happens on the HMP side because hmp_object_add() attempts to
> create a temporary QemuOpts entry with ID 'ram1', which ends up
> conflicting with the command-line-created entry, since it was never
> cleaned up during the previous hmp_object_del() call.
> 
> We address this by adding checks in {qmp,hmp}_object_del to determine
> whether an option group entry matching the object's ID is present,
> and removing it if it is.
> 
> Note that qmp_object_add() never attempts to create a temporary
> QemuOpts entry, so it does not encounter the duplicate ID error,
> which is why this isn't generally visible in libvirt.
> 
> Cc: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> Cc: Markus Armbruster <armbru@redhat.com>
> Cc: Eric Blake <eblake@redhat.com>
> Cc: Daniel Berrange <berrange@redhat.com>
> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> ---
>  hmp.c | 10 ++++++++++
>  qmp.c | 10 ++++++++++
>  2 files changed, 20 insertions(+)
> 
> diff --git a/hmp.c b/hmp.c
> index b869617..99b8bf3 100644
> --- a/hmp.c
> +++ b/hmp.c
> @@ -2072,6 +2072,16 @@ void hmp_object_del(Monitor *mon, const QDict *qdict)
>  
>      user_creatable_del(id, &err);
>      hmp_handle_error(mon, &err);
> +
> +    /* if object was defined on the command-line, remove its corresponding
> +     * option group entry
> +     */
> +    if (err == NULL) {
> +        QemuOptsList *opt_group = qemu_find_opts_err("object", &err);
> +        if (opt_group) {
> +            qemu_opts_del(qemu_opts_find(opt_group, id));
> +        }
> +    }
>  }
>  
>  void hmp_info_memdev(Monitor *mon, const QDict *qdict)
> diff --git a/qmp.c b/qmp.c
> index 0028f0b..87c545d 100644
> --- a/qmp.c
> +++ b/qmp.c
> @@ -686,6 +686,16 @@ void qmp_object_add(const char *type, const char *id,
>  void qmp_object_del(const char *id, Error **errp)
>  {
>      user_creatable_del(id, errp);
> +
> +    /* if object was defined on the command-line, remove its corresponding
> +     * option group entry
> +     */
> +    if (!(errp && *errp)) {
> +        QemuOptsList *opt_group = qemu_find_opts_err("object", errp);
> +        if (opt_group) {
> +            qemu_opts_del(qemu_opts_find(opt_group, id));
> +        }
> +    }
>  }

I rather feel like this code ought to be in user_creatable_del(), since
all code paths which call user_creatable_del() need to do deal with
this scenario.

Also I'd like to see a unit test added that exposes this problem and
demonstrates the fix - could att it to tests/check-qom-proplist.c

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://entangle-photo.org       -o-    http://search.cpan.org/~danberr/ :|

Re: [Qemu-devel] [PATCH] monitor: fix object_del for command-line-created objects

[Dr. David Alan Gilbert]

* Daniel P. Berrange (berrange@redhat.com) wrote:
> On Wed, Nov 30, 2016 at 05:06:16PM -0600, Michael Roth wrote:
> > Currently objects specified on the command-line are only partially
> > cleaned up when 'object_del' is issued in either HMP or QMP: the
> > object itself is fully finalized, but the QemuOpts are not removed.
> > This results in the following behavior:
> > 
> >   x86_64-softmmu/qemu-system-x86_64 -monitor stdio \
> >     -object memory-backend-ram,id=ram1,size=256M
> > 
> >   QEMU 2.7.91 monitor - type 'help' for more information
> >   (qemu) object_del ram1
> >   (qemu) object_del ram1
> >   object 'ram1' not found
> >   (qemu) object_add memory-backend-ram,id=ram1,size=256M
> >   Duplicate ID 'ram1' for object
> >   Try "help object_add" for more information
> > 
> > which can be an issue for use-cases like memory hotplug.
> > 
> > This happens on the HMP side because hmp_object_add() attempts to
> > create a temporary QemuOpts entry with ID 'ram1', which ends up
> > conflicting with the command-line-created entry, since it was never
> > cleaned up during the previous hmp_object_del() call.
> > 
> > We address this by adding checks in {qmp,hmp}_object_del to determine
> > whether an option group entry matching the object's ID is present,
> > and removing it if it is.
> > 
> > Note that qmp_object_add() never attempts to create a temporary
> > QemuOpts entry, so it does not encounter the duplicate ID error,
> > which is why this isn't generally visible in libvirt.
> > 
> > Cc: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> > Cc: Markus Armbruster <armbru@redhat.com>
> > Cc: Eric Blake <eblake@redhat.com>
> > Cc: Daniel Berrange <berrange@redhat.com>
> > Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> > ---
> >  hmp.c | 10 ++++++++++
> >  qmp.c | 10 ++++++++++
> >  2 files changed, 20 insertions(+)
> > 
> > diff --git a/hmp.c b/hmp.c
> > index b869617..99b8bf3 100644
> > --- a/hmp.c
> > +++ b/hmp.c
> > @@ -2072,6 +2072,16 @@ void hmp_object_del(Monitor *mon, const QDict *qdict)
> >  
> >      user_creatable_del(id, &err);
> >      hmp_handle_error(mon, &err);
> > +
> > +    /* if object was defined on the command-line, remove its corresponding
> > +     * option group entry
> > +     */
> > +    if (err == NULL) {
> > +        QemuOptsList *opt_group = qemu_find_opts_err("object", &err);
> > +        if (opt_group) {
> > +            qemu_opts_del(qemu_opts_find(opt_group, id));
> > +        }
> > +    }
> >  }
> >  
> >  void hmp_info_memdev(Monitor *mon, const QDict *qdict)
> > diff --git a/qmp.c b/qmp.c
> > index 0028f0b..87c545d 100644
> > --- a/qmp.c
> > +++ b/qmp.c
> > @@ -686,6 +686,16 @@ void qmp_object_add(const char *type, const char *id,
> >  void qmp_object_del(const char *id, Error **errp)
> >  {
> >      user_creatable_del(id, errp);
> > +
> > +    /* if object was defined on the command-line, remove its corresponding
> > +     * option group entry
> > +     */
> > +    if (!(errp && *errp)) {
> > +        QemuOptsList *opt_group = qemu_find_opts_err("object", errp);
> > +        if (opt_group) {
> > +            qemu_opts_del(qemu_opts_find(opt_group, id));
> > +        }
> > +    }
> >  }
> 
> I rather feel like this code ought to be in user_creatable_del(), since
> all code paths which call user_creatable_del() need to do deal with
> this scenario.

Yes that seems cleaner;  the cleanup that's being done is not the result
of either of the monitor's actions.

Dave

> Also I'd like to see a unit test added that exposes this problem and
> demonstrates the fix - could att it to tests/check-qom-proplist.c
> 
> Regards,
> Daniel
> -- 
> |: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
> |: http://libvirt.org              -o-             http://virt-manager.org :|
> |: http://entangle-photo.org       -o-    http://search.cpan.org/~danberr/ :|
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH for-2.8] monitor: fix object_del for command-line-created objects

[Michael Roth]

Quoting Eric Blake (2016-11-30 20:33:56)
> On 11/30/2016 05:06 PM, Michael Roth wrote:
> > Currently objects specified on the command-line are only partially
> > cleaned up when 'object_del' is issued in either HMP or QMP: the
> > object itself is fully finalized, but the QemuOpts are not removed.
> > This results in the following behavior:
> > 
> >   x86_64-softmmu/qemu-system-x86_64 -monitor stdio \
> >     -object memory-backend-ram,id=ram1,size=256M
> > 
> >   QEMU 2.7.91 monitor - type 'help' for more information
> >   (qemu) object_del ram1
> >   (qemu) object_del ram1
> >   object 'ram1' not found
> >   (qemu) object_add memory-backend-ram,id=ram1,size=256M
> >   Duplicate ID 'ram1' for object
> >   Try "help object_add" for more information
> > 
> > which can be an issue for use-cases like memory hotplug.
> 
> Nice analysis.
> 
> > 
> > This happens on the HMP side because hmp_object_add() attempts to
> > create a temporary QemuOpts entry with ID 'ram1', which ends up
> > conflicting with the command-line-created entry, since it was never
> > cleaned up during the previous hmp_object_del() call.
> > 
> > We address this by adding checks in {qmp,hmp}_object_del to determine
> > whether an option group entry matching the object's ID is present,
> > and removing it if it is.
> > 
> > Note that qmp_object_add() never attempts to create a temporary
> > QemuOpts entry, so it does not encounter the duplicate ID error,
> > which is why this isn't generally visible in libvirt.
> 
> Phew. Yeah, libvirt avoiding HMP where possible saves a lot of hassles.
> 
> > +++ b/hmp.c
> > @@ -2072,6 +2072,16 @@ void hmp_object_del(Monitor *mon, const QDict *qdict)
> >  
> >      user_creatable_del(id, &err);
> >      hmp_handle_error(mon, &err);
> > +
> > +    /* if object was defined on the command-line, remove its corresponding
> > +     * option group entry
> > +     */
> > +    if (err == NULL) {
> > +        QemuOptsList *opt_group = qemu_find_opts_err("object", &err);
> > +        if (opt_group) {
> > +            qemu_opts_del(qemu_opts_find(opt_group, id));
> > +        }
> > +    }
> 
> This one looks okay.
> 
> >  }
> >  
> >  void hmp_info_memdev(Monitor *mon, const QDict *qdict)
> > diff --git a/qmp.c b/qmp.c
> > index 0028f0b..87c545d 100644
> > --- a/qmp.c
> > +++ b/qmp.c
> > @@ -686,6 +686,16 @@ void qmp_object_add(const char *type, const char *id,
> >  void qmp_object_del(const char *id, Error **errp)
> >  {
> >      user_creatable_del(id, errp);
> > +
> > +    /* if object was defined on the command-line, remove its corresponding
> > +     * option group entry
> > +     */
> > +    if (!(errp && *errp)) {
> 
> But this is wrong. Please spin a v2 that uses a local Error object, then
> use error_propagate(err, errp). Making anything conditional on whether
> an error occurred requires a local object, since the caller can pass in
> NULL, but you want your cleanup to happen even when the caller doesn't
> care about errors.

Ugh, I should know better by now. Thanks for the catch, will make sure
this is done properly in v2.

> 
> > +        QemuOptsList *opt_group = qemu_find_opts_err("object", errp);
> > +        if (opt_group) {
> > +            qemu_opts_del(qemu_opts_find(opt_group, id));
> > +        }
> > +    }
> >  }
> >  
> >  MemoryDeviceInfoList *qmp_query_memory_devices(Error **errp)
> > 
> 
> -- 
> Eric Blake   eblake redhat com    +1-919-301-3266
> Libvirt virtualization library http://libvirt.org
> 

Re: [Qemu-devel] [PATCH] monitor: fix object_del for command-line-created objects

[Michael Roth]

Quoting Dr. David Alan Gilbert (2016-12-01 06:34:10)
> * Daniel P. Berrange (berrange@redhat.com) wrote:
> > On Wed, Nov 30, 2016 at 05:06:16PM -0600, Michael Roth wrote:
> > > Currently objects specified on the command-line are only partially
> > > cleaned up when 'object_del' is issued in either HMP or QMP: the
> > > object itself is fully finalized, but the QemuOpts are not removed.
> > > This results in the following behavior:
> > > 
> > >   x86_64-softmmu/qemu-system-x86_64 -monitor stdio \
> > >     -object memory-backend-ram,id=ram1,size=256M
> > > 
> > >   QEMU 2.7.91 monitor - type 'help' for more information
> > >   (qemu) object_del ram1
> > >   (qemu) object_del ram1
> > >   object 'ram1' not found
> > >   (qemu) object_add memory-backend-ram,id=ram1,size=256M
> > >   Duplicate ID 'ram1' for object
> > >   Try "help object_add" for more information
> > > 
> > > which can be an issue for use-cases like memory hotplug.
> > > 
> > > This happens on the HMP side because hmp_object_add() attempts to
> > > create a temporary QemuOpts entry with ID 'ram1', which ends up
> > > conflicting with the command-line-created entry, since it was never
> > > cleaned up during the previous hmp_object_del() call.
> > > 
> > > We address this by adding checks in {qmp,hmp}_object_del to determine
> > > whether an option group entry matching the object's ID is present,
> > > and removing it if it is.
> > > 
> > > Note that qmp_object_add() never attempts to create a temporary
> > > QemuOpts entry, so it does not encounter the duplicate ID error,
> > > which is why this isn't generally visible in libvirt.
> > > 
> > > Cc: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> > > Cc: Markus Armbruster <armbru@redhat.com>
> > > Cc: Eric Blake <eblake@redhat.com>
> > > Cc: Daniel Berrange <berrange@redhat.com>
> > > Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> > > ---
> > >  hmp.c | 10 ++++++++++
> > >  qmp.c | 10 ++++++++++
> > >  2 files changed, 20 insertions(+)
> > > 
> > > diff --git a/hmp.c b/hmp.c
> > > index b869617..99b8bf3 100644
> > > --- a/hmp.c
> > > +++ b/hmp.c
> > > @@ -2072,6 +2072,16 @@ void hmp_object_del(Monitor *mon, const QDict *qdict)
> > >  
> > >      user_creatable_del(id, &err);
> > >      hmp_handle_error(mon, &err);
> > > +
> > > +    /* if object was defined on the command-line, remove its corresponding
> > > +     * option group entry
> > > +     */
> > > +    if (err == NULL) {
> > > +        QemuOptsList *opt_group = qemu_find_opts_err("object", &err);
> > > +        if (opt_group) {
> > > +            qemu_opts_del(qemu_opts_find(opt_group, id));
> > > +        }
> > > +    }
> > >  }
> > >  
> > >  void hmp_info_memdev(Monitor *mon, const QDict *qdict)
> > > diff --git a/qmp.c b/qmp.c
> > > index 0028f0b..87c545d 100644
> > > --- a/qmp.c
> > > +++ b/qmp.c
> > > @@ -686,6 +686,16 @@ void qmp_object_add(const char *type, const char *id,
> > >  void qmp_object_del(const char *id, Error **errp)
> > >  {
> > >      user_creatable_del(id, errp);
> > > +
> > > +    /* if object was defined on the command-line, remove its corresponding
> > > +     * option group entry
> > > +     */
> > > +    if (!(errp && *errp)) {
> > > +        QemuOptsList *opt_group = qemu_find_opts_err("object", errp);
> > > +        if (opt_group) {
> > > +            qemu_opts_del(qemu_opts_find(opt_group, id));
> > > +        }
> > > +    }
> > >  }
> > 
> > I rather feel like this code ought to be in user_creatable_del(), since
> > all code paths which call user_creatable_del() need to do deal with
> > this scenario.
> 
> Yes that seems cleaner;  the cleanup that's being done is not the result
> of either of the monitor's actions.

It was also for similar reasons that I was hesitant to put it in
user_creatable_del() :) Putting it higher up in the stack somehow felt
less icky, but given that they both violate encapsulation somewhat I
suppose it does makes sense to take the more practical/safer route and
put it in a common path. Will take this approach for v2.

> 
> Dave
> 
> > Also I'd like to see a unit test added that exposes this problem and
> > demonstrates the fix - could att it to tests/check-qom-proplist.c
> > 
> > Regards,
> > Daniel
> > -- 
> > |: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
> > |: http://libvirt.org              -o-             http://virt-manager.org :|
> > |: http://entangle-photo.org       -o-    http://search.cpan.org/~danberr/ :|
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
>