Sample 'qapi-schema-diff' output for v9.0.0 vs origin/master

Sample 'qapi-schema-diff' output for v9.0.0 vs origin/master

[John Snow]

[-- Attachment #1: Type: text/plain, Size: 6856 bytes --]

Hiya, here's some draft output of a script I'm working on to summarize QMP
wire format differences between QEMU versions.

This script works in terms of the QMP *wire format*, not the QAPI
*specification*. As a consequence, *almost* all type names are stripped
from this output and all nested structures are fully inlined - so changes
to shared data structures, enums, etc will manifest as many independent
changes. Similarly, changes to type names and type factorings that do not
change the wire format will not appear in this report at all.

This is still a WIP: if conditionals and features may not be fully
represented in this summary report.

Here's today's diff output, see if you think this format is "intuitive" or
makes sense to some extent; or, if you're feeling bored, if you believe
it's fully accurate:

jsnow@scv ~/s/qemu (master)> qapi-schema-diff qapi-compiled-v9.0.0.json
qapi-compiled-v9.0.0-1388-g80e8f060216.json
###################################
v9.0.0 ==> v9.0.0-1388-g80e8f060216
###################################

********
commands
********

Removed
=======
x-query-rdma

Modified
========
blockdev-backup (arguments)
    ++ arguments.discard-source: Optional<boolean>
drive-backup (arguments)
    ++ arguments.discard-source: Optional<boolean>
migrate (arguments)
    -- arguments.blk: Optional<boolean>
    -- arguments.inc: Optional<boolean>
migrate-incoming (arguments)
    ++ arguments.exit-on-error: Optional<boolean>
migrate-set-capabilities (arguments)
    ·· arguments.capabilities[].capability: enum
    --     'block'
    --     'compress'
migrate-set-parameters (arguments)
    -- arguments.block-incremental: Optional<boolean>
    -- arguments.compress-level: Optional<integer>
    -- arguments.compress-threads: Optional<integer>
    -- arguments.compress-wait-thread: Optional<boolean>
    -- arguments.decompress-threads: Optional<integer>
object-add (arguments)
    ·· arguments.qom-type: enum
    ++     'sev-snp-guest'
    ++ arguments<qom-type=sev-guest>.legacy-vm-type: Optional<boolean>
    ++ arguments<qom-type=sev-snp-guest>.author-key-enabled:
Optional<boolean>
    ++ arguments<qom-type=sev-snp-guest>.cbitpos: Optional<integer>
    ++ arguments<qom-type=sev-snp-guest>.guest-visible-workarounds:
Optional<string>
    ++ arguments<qom-type=sev-snp-guest>.host-data: Optional<string>
    ++ arguments<qom-type=sev-snp-guest>.id-auth: Optional<string>
    ++ arguments<qom-type=sev-snp-guest>.id-block: Optional<string>
    ++ arguments<qom-type=sev-snp-guest>.kernel-hashes: Optional<boolean>
    ++ arguments<qom-type=sev-snp-guest>.policy: Optional<integer>
    ++ arguments<qom-type=sev-snp-guest>.reduced-phys-bits: integer
    ++ arguments<qom-type=sev-snp-guest>.sev-device: Optional<string>
    ++ arguments<qom-type=sev-snp-guest>.vcek-disabled: Optional<boolean>
query-cpu-model-baseline (returns, arguments)
    ++ arguments.modela.deprecated-props: Optional<array>
    ++ arguments.modela.deprecated-props[]: string
    ++ arguments.modelb.deprecated-props: Optional<array>
    ++ arguments.modelb.deprecated-props[]: string
    ++ returns.model.deprecated-props: Optional<array>
    ++ returns.model.deprecated-props[]: string
query-cpu-model-comparison (arguments)
    ++ arguments.modela.deprecated-props: Optional<array>
    ++ arguments.modela.deprecated-props[]: string
    ++ arguments.modelb.deprecated-props: Optional<array>
    ++ arguments.modelb.deprecated-props[]: string
query-cpu-model-expansion (returns, arguments)
    ++ arguments.model.deprecated-props: Optional<array>
    ++ arguments.model.deprecated-props[]: string
    ++ returns.model.deprecated-props: Optional<array>
    ++ returns.model.deprecated-props[]: string
query-cpus-fast (returns)
    ++ returns[].props.module-id: Optional<integer>
    ·· returns[].target: enum
    --     'nios2'
query-hotpluggable-cpus (returns)
    ++ returns[].props.module-id: Optional<integer>
query-machines (returns, arguments)
    ++ arguments.compat-props: Optional<boolean>
    ++ returns[].compat-props: Optional<array>
    ++ returns[].compat-props[]: object
    ++ returns[].compat-props[].property: string
    ++ returns[].compat-props[].qom-type: string
    ++ returns[].compat-props[].value: string
query-migrate (returns)
    -- returns.compression: Optional<object>
    -- returns.compression.busy: integer
    -- returns.compression.busy-rate: number
    -- returns.compression.compressed-size: integer
    -- returns.compression.compression-rate: number
    -- returns.compression.pages: integer
    -- returns.disk: Optional<object>
    -- returns.disk.dirty-pages-rate: integer
    -- returns.disk.dirty-sync-count: integer
    -- returns.disk.dirty-sync-missed-zero-copy: integer
    -- returns.disk.downtime-bytes: integer
    -- returns.disk.duplicate: integer
    -- returns.disk.mbps: number
    -- returns.disk.multifd-bytes: integer
    -- returns.disk.normal: integer
    -- returns.disk.normal-bytes: integer
    -- returns.disk.page-size: integer
    -- returns.disk.pages-per-second: integer
    -- returns.disk.postcopy-bytes: integer
    -- returns.disk.postcopy-requests: integer
    -- returns.disk.precopy-bytes: integer
    -- returns.disk.remaining: integer
    -- returns.disk.skipped: integer
    -- returns.disk.total: integer
    -- returns.disk.transferred: integer
    -- returns.ram.skipped: integer
query-migrate-capabilities (returns)
    ·· returns[].capability: enum
    --     'block'
    --     'compress'
query-migrate-parameters (returns)
    -- returns.block-incremental: Optional<boolean>
    -- returns.compress-level: Optional<integer>
    -- returns.compress-threads: Optional<integer>
    -- returns.compress-wait-thread: Optional<boolean>
    -- returns.decompress-threads: Optional<integer>
query-sev (returns)
    -- returns.handle: integer
    -- returns.policy: integer
    ++ returns.sev-type: enum
    ++ returns<sev-type=sev-snp>.snp-policy: integer
    ++ returns<sev-type=sev>.handle: integer
    ++ returns<sev-type=sev>.policy: integer
query-target (returns)
    ·· returns.arch: enum
    --     'nios2'
set-numa-node (arguments)
    ++ arguments<type=cpu>.module-id: Optional<integer>
trace-event-get-state (returns, arguments)
    -- arguments.vcpu: Optional<integer>
    -- returns[].vcpu: boolean
trace-event-set-state (arguments)
    -- arguments.vcpu: Optional<integer>
transaction (arguments)
    ++ arguments.actions[]<type=blockdev-backup>.data.discard-source:
Optional<boolean>
    ++ arguments.actions[]<type=drive-backup>.data.discard-source:
Optional<boolean>

******
events
******

Added
=====
VFIO_MIGRATION

Removed
=======
MEM_UNPLUG_ERROR
RDMA_GID_STATUS_CHANGED

[-- Attachment #2: Type: text/html, Size: 7960 bytes --]

Re: Sample 'qapi-schema-diff' output for v9.0.0 vs origin/master

[Philippe Mathieu-Daudé]

Hi John,

On 12/6/24 20:40, John Snow wrote:
> Hiya, here's some draft output of a script I'm working on to summarize 
> QMP wire format differences between QEMU versions.
> 
> This script works in terms of the QMP *wire format*, not the QAPI 
> *specification*. As a consequence, *almost* all type names are stripped 
> from this output and all nested structures are fully inlined - so 
> changes to shared data structures, enums, etc will manifest as many 
> independent changes. Similarly, changes to type names and type 
> factorings that do not change the wire format will not appear in this 
> report at all.

Do you include changes on unstable features or are you skipping them?

Possible feature request: skip them by default until a --unstable
CLI option is used.

> 
> This is still a WIP: if conditionals and features may not be fully 
> represented in this summary report.

Regards,

Phil.

Re: Sample 'qapi-schema-diff' output for v9.0.0 vs origin/master

[John Snow]

[-- Attachment #1: Type: text/plain, Size: 2447 bytes --]

On Thu, Jun 13, 2024, 4:50 AM Philippe Mathieu-Daudé <philmd@linaro.org>
wrote:

> Hi John,
>

Hi Phil!


> On 12/6/24 20:40, John Snow wrote:
> > Hiya, here's some draft output of a script I'm working on to summarize
> > QMP wire format differences between QEMU versions.
> >
> > This script works in terms of the QMP *wire format*, not the QAPI
> > *specification*. As a consequence, *almost* all type names are stripped
> > from this output and all nested structures are fully inlined - so
> > changes to shared data structures, enums, etc will manifest as many
> > independent changes. Similarly, changes to type names and type
> > factorings that do not change the wire format will not appear in this
> > report at all.
>
> Do you include changes on unstable features or are you skipping them?
>
> Possible feature request: skip them by default until a --unstable
> CLI option is used.
>

In this early draft, I do include such changes. I might want to /opt-in/ to
hiding them (maybe: --hide-unstable), but it can be done.

(A summary mode that hides itemized changes may also be useful for
observing broad-changes-at-a-glance and might be worth adding, too.
--no-detail or --brief or some such.)

I assume we want to hide anything that has the unstable feature flag at the
command *root* and any command or event named with the x- prefix. (unstable
fields on stable commands ... tbd.)

I still need to implement full support for if/features, so this idea can
come along for the ride when I flesh out those details.

(features/if for top-level definitions are recorded, but are likely missing
for various nested fields. The diff output will remark on what changed on
the summary line where it says "(arguments)" or "(arguments, returns)" with
the tokens "if" or "features", but it doesn't break it out into discrete
elements like everything else does, yet - and it still likely only catches
those differences when they occur at the definition root.)


> >
> > This is still a WIP: if conditionals and features may not be fully
> > represented in this summary report.
>
> Regards,
>
> Phil.
>

Thanks for the suggestion!

BTW, I sent a new mail out to the list with a link to gitlab for source and
sample output for this tool and other goodies; the URL is
https://gitlab.com/jsnow/externalized-qapi - see the other mail for details
on what's in here, if curious.

--js

>

[-- Attachment #2: Type: text/html, Size: 3910 bytes --]

Re: Sample 'qapi-schema-diff' output for v9.0.0 vs origin/master

[Daniel P. Berrangé]

On Wed, Jun 12, 2024 at 02:40:40PM -0400, John Snow wrote:
> Hiya, here's some draft output of a script I'm working on to summarize QMP
> wire format differences between QEMU versions.
> 
> This script works in terms of the QMP *wire format*, not the QAPI
> *specification*. As a consequence, *almost* all type names are stripped
> from this output and all nested structures are fully inlined - so changes
> to shared data structures, enums, etc will manifest as many independent
> changes. Similarly, changes to type names and type factorings that do not
> change the wire format will not appear in this report at all.
> 
> This is still a WIP: if conditionals and features may not be fully
> represented in this summary report.
> 
> Here's today's diff output, see if you think this format is "intuitive" or
> makes sense to some extent; or, if you're feeling bored, if you believe
> it's fully accurate:
> 
> jsnow@scv ~/s/qemu (master)> qapi-schema-diff qapi-compiled-v9.0.0.json
> qapi-compiled-v9.0.0-1388-g80e8f060216.json
> ###################################
> v9.0.0 ==> v9.0.0-1388-g80e8f060216
> ###################################
> 
> ********
> commands
> ********
> 
> Removed
> =======
> x-query-rdma

I'd probably suggest that added/removed commands and
events liste the full set of properties too, as you
could conceptually say we're adding/removing each
property, plus the command itself.


> 
> Modified
> ========
> blockdev-backup (arguments)
>     ++ arguments.discard-source: Optional<boolean>
> drive-backup (arguments)
>     ++ arguments.discard-source: Optional<boolean>
> migrate (arguments)
>     -- arguments.blk: Optional<boolean>
>     -- arguments.inc: Optional<boolean>

With the headings groupings and indentation this feels like
we're targetting humans with this file, but at the same time
it is also kind of a raw machine feeling. The '(arguments)'
suffix feels redundant given the lines that follow.

I'm inclined to say this is mostly considered machine
and/or patch reviewer, targetted plain text. Consumer
human targetted would be something added to the QMP
ref, as an appendix with more structured RST format.

Does it make sense to group into added/removed/modified
for commands, if we then just mix added/removed properties
in the same place ?

We could just show it all together, by prefixing '+' or '-'
or '.' on the command names too. eg.

 - x-query-rdma
 -     returns.human-readable-text: str
 . blockdev-backup
 +     arguments.discard-source: Optional<boolean>
 . migrate
 -    arguments.blk: Optional<boolean>
 -    arguments.inc: Optional<boolean>
 . object-add
 .    arguments.qom-type: enum
 +        'sev-snp-guest'
 +    arguments[sev-guest].legacy-vm-type: Optional<boolean>
 +    arguments[sev-snp-guest].author-key-enabled: Optional<boolean>
 +    arguments[sev-snp-guest].cbitpos: Optional<integer>

> migrate-incoming (arguments)
>     ++ arguments.exit-on-error: Optional<boolean>
> migrate-set-capabilities (arguments)
>     ·· arguments.capabilities[].capability: enum
>     --     'block'
>     --     'compress'
> migrate-set-parameters (arguments)
>     -- arguments.block-incremental: Optional<boolean>
>     -- arguments.compress-level: Optional<integer>
>     -- arguments.compress-threads: Optional<integer>
>     -- arguments.compress-wait-thread: Optional<boolean>
>     -- arguments.decompress-threads: Optional<integer>
> object-add (arguments)
>     ·· arguments.qom-type: enum
>     ++     'sev-snp-guest'
>     ++ arguments<qom-type=sev-guest>.legacy-vm-type: Optional<boolean>
>     ++ arguments<qom-type=sev-snp-guest>.author-key-enabled: Optional<boolean>
>     ++ arguments<qom-type=sev-snp-guest>.cbitpos: Optional<integer>
>     ++ arguments<qom-type=sev-snp-guest>.guest-visible-workarounds: Optional<string>
>     ++ arguments<qom-type=sev-snp-guest>.host-data: Optional<string>
>     ++ arguments<qom-type=sev-snp-guest>.id-auth: Optional<string>
>     ++ arguments<qom-type=sev-snp-guest>.id-block: Optional<string>
>     ++ arguments<qom-type=sev-snp-guest>.kernel-hashes: Optional<boolean>
>     ++ arguments<qom-type=sev-snp-guest>.policy: Optional<integer>
>     ++ arguments<qom-type=sev-snp-guest>.reduced-phys-bits: integer
>     ++ arguments<qom-type=sev-snp-guest>.sev-device: Optional<string>
>     ++ arguments<qom-type=sev-snp-guest>.vcek-disabled: Optional<boolean>
> query-cpu-model-baseline (returns, arguments)
>     ++ arguments.modela.deprecated-props: Optional<array>
>     ++ arguments.modela.deprecated-props[]: string
>     ++ arguments.modelb.deprecated-props: Optional<array>
>     ++ arguments.modelb.deprecated-props[]: string
>     ++ returns.model.deprecated-props: Optional<array>
>     ++ returns.model.deprecated-props[]: string
> query-cpu-model-comparison (arguments)
>     ++ arguments.modela.deprecated-props: Optional<array>
>     ++ arguments.modela.deprecated-props[]: string
>     ++ arguments.modelb.deprecated-props: Optional<array>
>     ++ arguments.modelb.deprecated-props[]: string
> query-cpu-model-expansion (returns, arguments)
>     ++ arguments.model.deprecated-props: Optional<array>
>     ++ arguments.model.deprecated-props[]: string
>     ++ returns.model.deprecated-props: Optional<array>
>     ++ returns.model.deprecated-props[]: string
> query-cpus-fast (returns)
>     ++ returns[].props.module-id: Optional<integer>
>     ·· returns[].target: enum
>     --     'nios2'
> query-hotpluggable-cpus (returns)
>     ++ returns[].props.module-id: Optional<integer>
> query-machines (returns, arguments)
>     ++ arguments.compat-props: Optional<boolean>
>     ++ returns[].compat-props: Optional<array>
>     ++ returns[].compat-props[]: object
>     ++ returns[].compat-props[].property: string
>     ++ returns[].compat-props[].qom-type: string
>     ++ returns[].compat-props[].value: string
> query-migrate (returns)
>     -- returns.compression: Optional<object>
>     -- returns.compression.busy: integer
>     -- returns.compression.busy-rate: number
>     -- returns.compression.compressed-size: integer
>     -- returns.compression.compression-rate: number
>     -- returns.compression.pages: integer
>     -- returns.disk: Optional<object>
>     -- returns.disk.dirty-pages-rate: integer
>     -- returns.disk.dirty-sync-count: integer
>     -- returns.disk.dirty-sync-missed-zero-copy: integer
>     -- returns.disk.downtime-bytes: integer
>     -- returns.disk.duplicate: integer
>     -- returns.disk.mbps: number
>     -- returns.disk.multifd-bytes: integer
>     -- returns.disk.normal: integer
>     -- returns.disk.normal-bytes: integer
>     -- returns.disk.page-size: integer
>     -- returns.disk.pages-per-second: integer
>     -- returns.disk.postcopy-bytes: integer
>     -- returns.disk.postcopy-requests: integer
>     -- returns.disk.precopy-bytes: integer
>     -- returns.disk.remaining: integer
>     -- returns.disk.skipped: integer
>     -- returns.disk.total: integer
>     -- returns.disk.transferred: integer
>     -- returns.ram.skipped: integer
> query-migrate-capabilities (returns)
>     ·· returns[].capability: enum
>     --     'block'
>     --     'compress'
> query-migrate-parameters (returns)
>     -- returns.block-incremental: Optional<boolean>
>     -- returns.compress-level: Optional<integer>
>     -- returns.compress-threads: Optional<integer>
>     -- returns.compress-wait-thread: Optional<boolean>
>     -- returns.decompress-threads: Optional<integer>
> query-sev (returns)
>     -- returns.handle: integer
>     -- returns.policy: integer
>     ++ returns.sev-type: enum
>     ++ returns<sev-type=sev-snp>.snp-policy: integer
>     ++ returns<sev-type=sev>.handle: integer
>     ++ returns<sev-type=sev>.policy: integer
> query-target (returns)
>     ·· returns.arch: enum
>     --     'nios2'
> set-numa-node (arguments)
>     ++ arguments<type=cpu>.module-id: Optional<integer>
> trace-event-get-state (returns, arguments)
>     -- arguments.vcpu: Optional<integer>
>     -- returns[].vcpu: boolean
> trace-event-set-state (arguments)
>     -- arguments.vcpu: Optional<integer>
> transaction (arguments)
>     ++ arguments.actions[]<type=blockdev-backup>.data.discard-source:
> Optional<boolean>
>     ++ arguments.actions[]<type=drive-backup>.data.discard-source:
> Optional<boolean>
> 
> ******
> events
> ******
> 
> Added
> =====
> VFIO_MIGRATION
> 
> Removed
> =======
> MEM_UNPLUG_ERROR
> RDMA_GID_STATUS_CHANGED



With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|