Re: [PATCH RFC V6 22/24] monitor,qdev: Introduce 'device_set' to change admin state of existing devices

[Markus Armbruster <armbru@redhat.com>]

salil.mehta@opnsrc.net writes:

> From: Salil Mehta <salil.mehta@huawei.com>
>
> This patch adds a "device_set" interface for modifying properties of devices
> that already exist in the guest topology. Unlike 'device_add'/'device_del'
> (hot-plug), 'device_set' does not create or destroy devices. It is intended
> for guest-visible hot-add semantics where hardware is provisioned at boot but
> logically enabled/disabled later via administrative policy.
>
> Compared to the existing 'qom-set' command, which is less intuitive and works
> only with object IDs, device_set provides a more device-oriented interface.
> It can be invoked at the QEMU prompt using natural device arguments, and the
> new '-deviceset' CLI option allows properties to be set at boot time, similar
> to how '-device' specifies device creation.

Why can't we use -device?

> While the initial implementation focuses on "admin-state" changes (e.g.,
> enable/disable a CPU already described by ACPI/DT), the interface is designed
> to be generic. In future, it could be used for other per-device set/unset
> style controls — beyond administrative power-states — provided the target
> device explicitly allows such changes. This enables fine-grained runtime
> control of device properties.

Beware, designing a generic interface can be harder, sometimes much
harder, than designing a specialized one.

device_add and qom-set are generic, and they have issues:

* device_add effectively bypasses QAPI by using 'gen': false.

  This bypasses QAPI's enforcement of documentation.  Property
  documentation is separate and poor.

  It also defeats introspection with query-qmp-schema.  You need to
  resort to other means instead, say QOM introspection (which is a bag
  of design flaws on its own), then map from QOM to qdev.

* device_add lets you specify any qdev property, even properties that
  are intended only for use by C code.

  This results in accidental external interfaces.

  We tend to name properties like "x-prop" to discourage external use,
  but I wouldn't bet my own money on us getting that always right.
  Moreover, there's beauties like "x-origin".

* qom-set & friends effectively bypass QAPI by using type 'any'.

  Again, the bypass results in poor documentation and a defeat of
  query-qmp-schema.

* qom-set lets you mess with any QOM property with a setter callback.

  Again, accidental external interfaces: most of these properties are
  not meant for use with qom-set.  For some, qom-set works, for some it
  silently does nothing, and for some it crashes.  A lot more dangerous
  than device_add.

  The "x-" convention can't help here: some properties are intended for
  external use with object-add, but not with qom-set.

We should avoid such issues in new interfaces.

We'll examine how this applies to device_set when I review the QAPI
schema.

> Key pieces:
>   * QMP: qmp_device_set() to update an existing device. The device can be
>     located by "id" or via driver+property match using a DeviceListener
>     callback (qdev_find_device()).
>   * HMP: "device_set" command with tab-completion. Errors are surfaced via
>     hmp_handle_error().
>   * CLI: "-deviceset" option for setting startup/admin properties at boot,
>     including a JSON form. Options are parsed into qemu_deviceset_opts and
>     applied after device creation.
>   * Docs/help: HMP help text and qemu-options.hx additions explain usage and
>     explicitly note that no hot-plug occurs.
>   * Safety: disallowed during live migration (migration_is_idle() check).
>
> Semantics:
>   * Operates on an existing DeviceState; no enumeration/new device appears.
>   * Complements device_add/device_del by providing state mutation only.
>   * Backward compatible: no behavior change unless "device_set"/"-deviceset"
>     is used.
>
> Examples:
>   HMP:
>     (qemu) device_set host-arm-cpu,core-id=3,admin-state=enable
>
>   CLI (at boot):
>     -smp cpus=4,maxcpus=4 \
>     -deviceset host-arm-cpu,core-id=2,admin-state=disable
>
>   QMP (JSON form):
>     { "execute": "device_set",
>       "arguments": {
>         "driver": "host-arm-cpu",
>         "core-id": 1,
>         "admin-state": "disable"
>       }
>     }

{"error": {"class": "CommandNotFound", "desc": "The command device_set has not been found"}}

Clue below.

> NOTE: The qdev_enable()/qdev_disable() hooks for acting on admin-state will be
> added in subsequent patches. Device classes must explicitly support any
> property they want to expose through device_set.
>
> Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
> ---
>  hmp-commands.hx         |  30 +++++++++
>  hw/arm/virt.c           |  86 +++++++++++++++++++++++++
>  hw/core/cpu-common.c    |  12 ++++
>  hw/core/qdev.c          |  21 ++++++
>  include/hw/arm/virt.h   |   1 +
>  include/hw/core/cpu.h   |  11 ++++
>  include/hw/qdev-core.h  |  22 +++++++
>  include/monitor/hmp.h   |   2 +
>  include/monitor/qdev.h  |  30 +++++++++
>  include/system/system.h |   1 +
>  qemu-options.hx         |  51 +++++++++++++--
>  system/qdev-monitor.c   | 139 +++++++++++++++++++++++++++++++++++++++-
>  system/vl.c             |  39 +++++++++++
>  13 files changed, 440 insertions(+), 5 deletions(-)

Clue: no update to the QAPI schema, i.e. the QMP command does not exist.

>
> diff --git a/hmp-commands.hx b/hmp-commands.hx
> index d0e4f35a30..18056cf21d 100644
> --- a/hmp-commands.hx
> +++ b/hmp-commands.hx
> @@ -707,6 +707,36 @@ SRST
>    or a QOM object path.
>  ERST
>  
> +{
> +    .name       = "device_set",
> +    .args_type  = "device:O",
> +    .params     = "driver[,prop=value][,...]",
> +    .help       = "set/unset existing device property",
> +    .cmd        = hmp_device_set,
> +    .command_completion = device_set_completion,
> +},
> +
> +SRST
> +``device_set`` *driver[,prop=value][,...]*
> +  Change the administrative power state of an existing device.
> +
> +  This command enables or disables a known device (e.g., CPU) using the
> +  "device_set" interface. It does not hotplug or add a new device.
> +
> +  Depending on platform support (e.g., PSCI or ACPI), this may trigger
> +  corresponding operational changes — such as powering down a CPU or
> +  transitioning it to active use.
> +
> +  Administrative state:
> +    * *enabled*  — Allows the guest to use the device (e.g., CPU_ON)
> +    * *disabled* — Prevents guest use; device is powered off (e.g., CPU_OFF)
> +
> +  Note: The device must already exist (be declared during machine creation).
> +
> +  Example:
> +      (qemu) device_set host-arm-cpu,core-id=3,admin-state=disabled
> +ERST

How exactly is the device selected?  You provide a clue above: 'can be
located by "id" or via driver+property match'.

I assume by "id" is just like device_del, i.e. by qdev ID or QOM path.

By "driver+property match" is not obvious.  Which of the arguments are
for matching, and which are for setting?

If "id" is specified, is there any matching?

The matching feature complicates this interface quite a bit.  I doubt
it's worth the complexity.  If you think it is, please split it off into
a separate patch.

Next question.  Is there a way for management applications to detect
whether a certain device supports device_set for a certain property?

Without that, what are management application supposed to do?  Hard-code
what works?  Run the command and see whether it fails?

I understand right now the command supports just "admin-state" for a
certain set of devices, so hard-coding would be possible.  But every new
(device, property) pair then requires management application updates,
and the hard-coded information becomes version specific.  This will
become unworkable real quick.  Not good enough for a command designed to
be generic.

> +
>      {
>          .name       = "cpu",
>          .args_type  = "index:i",

[...]

> diff --git a/qemu-options.hx b/qemu-options.hx
> index 83ccde341b..f517b91042 100644
> --- a/qemu-options.hx
> +++ b/qemu-options.hx
> @@ -375,7 +375,10 @@ SRST
>      This is different from CPU hotplug where additional CPUs are not even
>      present in the system description. Administratively disabled CPUs appear in
>      ACPI tables i.e. are provisioned, but cannot be used until explicitly
> -    enabled via QMP/HMP or the deviceset API.
> +    enabled via QMP/HMP or the deviceset API. On ACPI guests, each vCPU counted
> +    by 'disabledcpus=' is provisioned with '\ ``_STA``\ ' reporting Present=1
> +    and Enabled=0 (present-offline) at boot; it becomes Enabled=1 when brought
> +    online via 'device_set ... admin-state=enable'.
>  
>      On boards supporting CPU hotplug, the optional '\ ``maxcpus``\ ' parameter
>      can be set to enable further CPUs to be added at runtime. When both
> @@ -455,6 +458,15 @@ SRST
>  
>          -smp 2
>  
> +    Note: The cluster topology will only be generated in ACPI and exposed
> +    to guest if it's explicitly specified in -smp.
> +
> +    Note: Administratively disabled CPUs (specified via 'disabledcpus=' and
> +    '-deviceset' at CLI during boot) are especially useful for platforms like
> +    ARM that lack native CPU hotplug support. These CPUs will appear to the
> +    guest as unavailable, and any attempt to bring them online must go through
> +    QMP/HMP commands like 'device_set'.
> +
>      Examples using 'disabledcpus':
>  
>      For a board without CPU hotplug, enable 4 CPUs at boot and provision
> @@ -472,9 +484,6 @@ SRST
>      ::
>  
>          -smp cpus=4,disabledcpus=2,maxcpus=8
> -
> -    Note: The cluster topology will only be generated in ACPI and exposed
> -    to guest if it's explicitly specified in -smp.
>  ERST
>  
>  DEF("numa", HAS_ARG, QEMU_OPTION_numa,
> @@ -1281,6 +1290,40 @@ SRST
>  
>  ERST
>  
> +DEF("deviceset", HAS_ARG, QEMU_OPTION_deviceset,
> +    "-deviceset driver[,prop[=value]][,...]\n"
> +    "                Set administrative power state of an existing device.\n"
> +    "                Does not hotplug a new device. Can disable or enable\n"
> +    "                devices (such as CPUs) at boot based on policy.\n"
> +    "                Example:\n"
> +    "                    -deviceset host-arm-cpu,core-id=2,admin-state=disabled\n"
> +    "                Use '-deviceset help' for supported drivers\n"
> +    "                Use '-deviceset driver,help' for driver-specific properties\n",
> +    QEMU_ARCH_ALL)
> +SRST
> +``-deviceset driver[,prop[=value]][,...]``
> +    Configure an existing device's administrative power state or properties.
> +
> +    Unlike ``-device``, this option does not create a new device. Instead,
> +    it sets startup properties (such as administrative power state) for
> +    a device already declared via -smp or other machine configuration.
> +
> +    Example:
> +        -smp cpus=4
> +        -deviceset host-arm-cpu,core-id=2,admin-state=disabled
> +
> +    The above disables CPU core 2 at boot using administrative offlining.
> +    The guest may later re-enable the core (if permitted by platform policy).
> +
> +    ``state=enabled|disabled``
> +        Sets the administrative state of the device:
> +        - ``enabled``: device is made available at boot
> +        - ``disabled``: device is administratively disabled and powered off
> +
> +    Use ``-deviceset help`` to view all supported drivers.
> +    Use ``-deviceset driver,help`` for property-specific help.
> +ERST
> +
>  DEF("name", HAS_ARG, QEMU_OPTION_name,
>      "-name string1[,process=string2][,debug-threads=on|off]\n"
>      "                set the name of the guest\n"
> diff --git a/system/qdev-monitor.c b/system/qdev-monitor.c
> index 2ac92d0a07..1099b1237d 100644
> --- a/system/qdev-monitor.c
> +++ b/system/qdev-monitor.c
> @@ -263,12 +263,20 @@ static DeviceClass *qdev_get_device_class(const char **driver, Error **errp)
>      }
>  
>      dc = DEVICE_CLASS(oc);
> -    if (!dc->user_creatable) {
> +    if (!dc->user_creatable && !dc->admin_power_state_supported) {
>          error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "driver",
>                     "a pluggable device type");
>          return NULL;
>      }
>  
> +    if (phase_check(PHASE_MACHINE_READY) &&
> +        (!dc->hotpluggable || !dc->admin_power_state_supported)) {
> +        error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "driver",
> +                   "a pluggable device type or which supports changing power-"
> +                   "state administratively");
> +        return NULL;
> +    }
> +
>      if (object_class_dynamic_cast(oc, TYPE_SYS_BUS_DEVICE)) {
>          /* sysbus devices need to be allowed by the machine */
>          MachineClass *mc = MACHINE_CLASS(object_get_class(qdev_get_machine()));
> @@ -939,6 +947,76 @@ void qmp_device_del(const char *id, Error **errp)
>      }
>  }
>  
> +void qmp_device_set(const QDict *qdict, Error **errp)
> +{
> +    const char *state;
> +    const char *driver;
> +    DeviceState *dev;
> +    DeviceClass *dc;
> +    const char *id;
> +
> +    driver = qdict_get_try_str(qdict, "driver");
> +    if (!driver) {
> +        error_setg(errp, "Parameter 'driver' is missing");
> +        return;
> +    }
> +
> +    /* check driver exists and we are at the right phase of machine init */
> +    dc = qdev_get_device_class(&driver, errp);
> +    if (!dc) {

Since qdev_get_device_class() sets an error when it fails, *errp is not
null here, ...

> +        error_setg(errp, "driver '%s' not supported", driver);

... which makes this wrong.  Caught by error_setv()'s assertion.

Please test your error paths.

> +        return;
> +    }
> +
> +    if (migration_is_running()) {
> +        error_setg(errp, "device_set not allowed while migrating");
> +        return;
> +    }
> +
> +    id = qdict_get_try_str(qdict, "id");
> +
> +    if (id) {
> +        /* Lookup by ID */
> +        dev = find_device_state(id, false, errp);
> +        if (errp && *errp) {
> +            error_prepend(errp, "Device lookup failed for ID '%s': ", id);
> +            return;
> +        }
> +    } else {
> +        /* Lookup using driver and properties */
> +        dev = qdev_find_device(qdict, errp);
> +        if (errp && *errp) {
> +            error_prepend(errp, "Device lookup for %s failed: ", driver);
> +            return;
> +        }
> +    }
> +    if (!dev) {
> +        error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND,
> +                  "No device found for driver '%s'", driver);
> +        return;
> +    }
> +
> +    state = qdict_get_try_str(qdict, "admin-state");
> +    if (!state) {
> +        error_setg(errp, "no device state change specified for device %s ",
> +                   dev->id);
> +        return;
> +    } else if (!strcmp(state, "enable")) {
> +
> +        if (!qdev_enable(dev, qdev_get_parent_bus(DEVICE(dev)), errp)) {
> +            return;
> +        }
> +    } else if (!strcmp(state, "disable")) {
> +        if (!qdev_disable(dev, qdev_get_parent_bus(DEVICE(dev)), errp)) {
> +            return;
> +        }
> +    } else {
> +        error_setg(errp, "unrecognized specified state *%s* for device %s",
> +                   state, dev->id);
> +        return;
> +    }
> +}
> +
>  int qdev_sync_config(DeviceState *dev, Error **errp)
>  {
>      DeviceClass *dc = DEVICE_GET_CLASS(dev);

[...]