Re: [PATCH V1 0/6] fast qom tree get

[Markus Armbruster <armbru@redhat.com>]

Steven Sistare <steven.sistare@oracle.com> writes:

> On 4/9/2025 9:34 AM, Markus Armbruster wrote:
>> Steven Sistare <steven.sistare@oracle.com> writes:
>>> On 4/9/2025 3:39 AM, Markus Armbruster wrote:
>>>> Hi Steve, I apologize for the slow response.
>>>>
>>>> Steve Sistare <steven.sistare@oracle.com> writes:
>>>>
>>>>> Using qom-list and qom-get to get all the nodes and property values in a
>>>>> QOM tree can take multiple seconds because it requires 1000's of individual
>>>>> QOM requests.  Some managers fetch the entire tree or a large subset
>>>>> of it when starting a new VM, and this cost is a substantial fraction of
>>>>> start up time.
>>>>
>>>> "Some managers"... could you name one?
>>>
>>> My personal experience is with Oracle's OCI, but likely others could benefit.
>> 
>> Peter Krempa tells us libvirt would benefit.
>> 
>>>>> To reduce this cost, consider QAPI calls that fetch more information in
>>>>> each call:
>>>>>     * qom-list-get: given a path, return a list of properties and values.
>>>>>     * qom-list-getv: given a list of paths, return a list of properties and
>>>>>       values for each path.
>>>>>     * qom-tree-get: given a path, return all descendant nodes rooted at that
>>>>>       path, with properties and values for each.
>>>>
>>>> Libvirt developers, would you be interested in any of these?
>>>>
>>>>> In all cases, a returned property is represented by ObjectPropertyValue,
>>>>> with fields name, type, value, and error.  If an error occurs when reading
>>>>> a value, the value field is omitted, and the error message is returned in the
>>>>> the error field.  Thus an error for one property will not cause a bulk fetch
>>>>> operation to fail.
>>>>
>>>> Returning errors this way is highly unusual.  Observation; I'm not
>>>> rejecting this out of hand.  Can you elaborate a bit on why it's useful?
>>>
>>> It is considered an error to read some properties if they are not valid for
>>> the configuration.  And some properties are write-only and return an error
>>> if they are read.  Examples:
>>>
>>>     legacy-i8042: <EXCEPTION: Property 'vmmouse.legacy-i8042' is not readable> (str)
>>>     legacy-memory: <EXCEPTION: Property 'qemu64-x86_64-cpu.legacy-memory' is not readable> (str)
>>>     crash-information: <EXCEPTION: No crash occurred> (GuestPanicInformation)
>>>
>>> With conventional error handling, if any of these poison pills falls in the
>>> scope of a bulk get operation, the entire operation fails.
>> 
>> I suspect many of these poison pills are design mistakes.
>> 
>> If a property is not valid for the configuration, why does it exist?
>> QOM is by design dynamic.  I wish it wasn't, but as long as it is
>> dynamic, I can't see why we should create properties we know to be
>> unusable.
>> 
>> Why is reading crash-information an error when no crash occured?  This
>> is the *normal* case.  Errors are for the abnormal.
>> 
>> Anyway, asking you to fix design mistakes all over the place wouldn't be
>> fair.  So I'm asking you something else instead: do you actually need
>> the error information?
>
> I don't need the specific error message.
>
> I could return a boolean meaning "property not available" instead of returning
> the exact error message, as long as folks are OK with the output of the qom-tree
> script changing for these properties.

Let's put aside the qom-tree script for a moment.

In your patches, the queries return an object's properties as a list of
ObjectPropertyValue, defined as

    { 'struct': 'ObjectPropertyValue',
      'data': { 'name': 'str',
                'type': 'str',
                '*value': 'any',
                '*error': 'str' } }

As far as I understand, exactly one of @value and @error are present.

The list has no duplicates, i.e. no two elements have the same value of
"name".

Say we're interested in property "foo".  Three cases:

* The list has an element with "name": "foo", and the element has member
  "value": the property exists and "value" has its value.

* The list has an element with "name": "foo", and the element does not
  have member "value": the property exists, but its value cannot be
  gotten; member "error" has the error message.

* The list has no element with "name": "foo": the property does not
  exist.

If we simply drop ObjectPropertyValue member @error, we lose 'member
"error" has the error message'.  That's all.

If a need for more error information should arise later, we could add
member @error.  Or something else entirely.  Or tell people to qom-get
any properties qom-tree-get couldn't get for error information.  My
point is: dropping @error now does not tie our hands as far as I can
tell.

Back to qom-tree.  I believe this script is a development aid that
exists because qom-get is painful to use for humans.  Your qom-tree
command would completely obsolete it.  I wouldn't worry about it.
If you think I'm wrong there, please speak up!