Re: [Qemu-devel] [PATCH v4 1/5] iotests: add qmp recursive sorting function

[Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>]

19.12.2018 4:52, John Snow wrote:
> Python before 3.6 does not sort kwargs by default.
> If we want to print out pretty-printed QMP objects while
> preserving the "exec" > "arguments" ordering, we need a custom sort.
> 
> We can accomplish this by sorting **kwargs into an OrderedDict,
> which does preserve addition order.
> ---
>   tests/qemu-iotests/iotests.py | 21 +++++++++++++++++----
>   1 file changed, 17 insertions(+), 4 deletions(-)
> 
> diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
> index 9595429fea..9aec03c7a3 100644
> --- a/tests/qemu-iotests/iotests.py
> +++ b/tests/qemu-iotests/iotests.py
> @@ -30,6 +30,7 @@ import signal
>   import logging
>   import atexit
>   import io
> +from collections import OrderedDict
>   
>   sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'scripts'))
>   import qtest
> @@ -75,6 +76,16 @@ def qemu_img(*args):
>           sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' '.join(qemu_img_args + list(args))))
>       return exitcode
>   
> +def ordered_kwargs(kwargs):
> +    # kwargs prior to 3.6 are not ordered, so:
> +    od = OrderedDict()
> +    for k in sorted(kwargs.keys()):

you can use for k, v in sorted(kwargs.items()):
and use then v instead of kwargs[k]

> +        if isinstance(kwargs[k], dict):
> +            od[k] = ordered_kwargs(kwargs[k])
> +        else:
> +            od[k] = kwargs[k]
> +    return od
> +
>   def qemu_img_create(*args):
>       args = list(args)
>   
> @@ -257,8 +268,9 @@ def filter_img_info(output, filename):
>   def log(msg, filters=[]):
>       for flt in filters:
>           msg = flt(msg)

I think that trying to apply text filters to object should be fixed first.

> -    if type(msg) is dict or type(msg) is list:
> -        print(json.dumps(msg, sort_keys=True))
> +    if isinstance(msg, dict) or isinstance(msg, list):
> +        sort_keys = not isinstance(msg, OrderedDict)
> +        print(json.dumps(msg, sort_keys=sort_keys))
>       else:
>           print(msg)
>   
> @@ -448,8 +460,9 @@ class VM(qtest.QEMUQtestMachine):
>           return result
>   
>       def qmp_log(self, cmd, filters=[filter_testfiles], **kwargs):
> -        logmsg = '{"execute": "%s", "arguments": %s}' % \
> -            (cmd, json.dumps(kwargs, sort_keys=True))
> +        full_cmd = OrderedDict({"execute": cmd,
> +                                "arguments": ordered_kwargs(kwargs)})

no, you can't use dict as a parameter to constructor, as dict is not ordered,
use tuple of tuples, like OrderedDict((('execute': cmd), ('execute': ...)))



> +        logmsg = json.dumps(full_cmd)
>           log(logmsg, filters)

and I prefere fixing the thing, that we do json.dumps both in log() and qmp_log() before
this patch.

Also: so, we move all qmp_log callers to new logic (through sorting by hand with ordered_kwargs),
and it works? Then, maybe, move all log callers to new logic, and get rid of json.dumps at all,
to have one path instead of two?

>           result = self.qmp(cmd, **kwargs)
>           log(json.dumps(result, sort_keys=True), filters)
> 


-- 
Best regards,
Vladimir