Re: [PATCH V1 0/6] fast qom tree get

[Markus Armbruster <armbru@redhat.com>]

Hi Steve, I apologize for the slow response.

Steve Sistare <steven.sistare@oracle.com> writes:

> Using qom-list and qom-get to get all the nodes and property values in a
> QOM tree can take multiple seconds because it requires 1000's of individual
> QOM requests.  Some managers fetch the entire tree or a large subset
> of it when starting a new VM, and this cost is a substantial fraction of
> start up time.

"Some managers"... could you name one?

> To reduce this cost, consider QAPI calls that fetch more information in
> each call:
>   * qom-list-get: given a path, return a list of properties and values.
>   * qom-list-getv: given a list of paths, return a list of properties and
>     values for each path.
>   * qom-tree-get: given a path, return all descendant nodes rooted at that
>     path, with properties and values for each.

Libvirt developers, would you be interested in any of these?

> In all cases, a returned property is represented by ObjectPropertyValue,
> with fields name, type, value, and error.  If an error occurs when reading
> a value, the value field is omitted, and the error message is returned in the
> the error field.  Thus an error for one property will not cause a bulk fetch
> operation to fail.

Returning errors this way is highly unusual.  Observation; I'm not
rejecting this out of hand.  Can you elaborate a bit on why it's useful?

> To evaluate each method, I modified scripts/qmp/qom-tree to use the method,
> verified all methods produce the same output, and timed each using:
>
>   qemu-system-x86_64 -display none \
>     -chardev socket,id=monitor0,path=/tmp/vm1.sock,server=on,wait=off \
>     -mon monitor0,mode=control &
>
>   time qom-tree -s /tmp/vm1.sock > /dev/null

Cool!

> I only measured once per method, but the variation is low after a warm up run.
> The 'real - user - sys' column is a proxy for QEMU CPU time.
>
> method               real(s)   user(s)   sys(s)  (real - user - sys)(s)
> qom-list / qom-get   2.048     0.932     0.057   1.059
> qom-list-get         0.402     0.230     0.029   0.143
> qom-list-getv        0.200     0.132     0.015   0.053
> qom-tree-get         0.143     0.123     0.012   0.008
>
> qom-tree-get is the clear winner, reducing elapsed time by a factor of 14X,
> and reducing QEMU CPU time by 132X.
>
> qom-list-getv is slower when fetching the entire tree, but can beat
> qom-tree-get when only a subset of the tree needs to be fetched (not shown).
>
> qom-list-get is shown for comparison only, and is not included in this series.

If we have qom-list-getv, then qom-list-get is not worth having.