Re: [PATCH] qmp-shell: Suppress banner and prompt when stdin is not a TTY

[Dov Murik <dovmurik@linux.vnet.ibm.com>]

On 20/01/2021 17:46, John Snow wrote:
> On 1/20/21 4:45 AM, Daniel P. Berrangé wrote:
>> On Wed, Jan 20, 2021 at 10:25:25AM +0200, Dov Murik wrote:
>>> Hi John,
>>>
>>> On 19/01/2021 22:02, John Snow wrote:
>>>> On 1/17/21 2:27 AM, Dov Murik wrote:
>>>>> Detect whether qmp-shell's standard input is not a TTY; in such case,
>>>>> assume a non-interactive mode, which suppresses the welcome banner and
>>>>> the "(QEMU)" prompt.  This allows for easier consumption of 
>>>>> qmp-shell's
>>>>> output in scripts.
>>>>>
>>>>> Example usage before this change:
>>>>>
>>>>>       $ printf "query-status\nquery-kvm\n" | sudo
>>>>> scripts/qmp/qmp-shell qmp-unix-sock
>>>>>       Welcome to the QMP low-level shell!
>>>>>       Connected to QEMU 5.1.50
>>>>>
>>>>>       (QEMU) {"return": {"status": "running", "singlestep": false,
>>>>> "running": true}}
>>>>>       (QEMU) {"return": {"enabled": true, "present": true}}
>>>>>       (QEMU)
>>>>>
>>>>> Example usage after this change:
>>>>>
>>>>>       $ printf "query-status\nquery-kvm\n" | sudo
>>>>> scripts/qmp/qmp-shell qmp-unix-sock
>>>>>       {"return": {"status": "running", "singlestep": false,
>>>>> "running": true}}
>>>>>       {"return": {"enabled": true, "present": true}}
>>>>>
>>>>> Signed-off-by: Dov Murik <dovmurik@linux.vnet.ibm.com>
>>>>
>>>> Hiya! I've been taking lead on modernizing a lot of our python
>>>> infrastructure, including our QMP library and qmp-shell.
>>>>
>>>> (Sorry, not in MAINTAINERS yet; but I am in the process of moving these
>>>> scripts and tools over to ./python/qemu/qmp.)
>>>
>>> Thanks for this effort.
>>>
>>>>
>>>> This change makes me nervous, because qmp-shell is not traditionally a
>>>> component we've thought of as needing to preserve backwards-compatible
>>>> behavior. Using it as a script meant to be consumed in a headless
>>>> fashion runs a bit counter to that assumption.
>>>>
>>>> I'd be less nervous if the syntax of qmp-shell was something that was
>>>> well thought-out and rigorously tested, but it's a hodge-podge of
>>>> whatever people needed at the moment. I am *very* reluctant to cement
>>>> it.
>>>
>>> Yes, I understand your choice.
>>>
>>>
>>>>
>>>> Are you trying to leverage the qmp.py library from bash?
>>>
>>> Yes, I want to send a few QMP commands and record their output.  If I 
>>> use
>>> socat to the unix-socket I need to serialize the JSON request myself, so
>>> using qmp-shell saves me that; also not sure if there's any 
>>> negotiation done
>>> at the beginning by qmp-shell.
>>
>> There is a handshake, but it is just a single json message.
>>
>> See docs/interop/qmp-intro.txt and qmp-spec.txt for guidance.
>>
>>> Is there an easier way to script qmp commands, short of writing my own
>>> python program which uses the qmp.py library?
>>
>> Yes, writing your own python program is probably best. Doing anything
>> complex is shell is almost always a mistake, as it is a very crude
>> and poor language compared to something like managing QEMU/QMP.
>>
>> Note that I don't believe that we've declared qmp.py to be a long
>> term stable interface for users outside of QEMU either. An alternative
>> is to just use the python sockets APIs directly to speak to QEMU/QMP
>>
> 
> Right. qmp.py is technically not stable either, but it at least doesn't 
> use an invented syntax we don't have a spec for ... and it is used by 
> quite a few other things in the tree, so I trust it /slightly/ more. I 
> cannot promise compatibility for scripts that aren't in the tree at this 
> time, though.
> 
> (I am working on an asyncio variant of the QMP library that I do hope to 
> promise stability for, but that's probably not something you can hope to 
> see in the short term. It will likely have an API that is at least 
> somewhat similar to the existing library, but will use asyncio 
> coroutines instead of blocking calls.)
> 
> You can look at ./tests/qemu-iotests/ for some examples of using the QMP 
> library that we have today; grep for '.qmp(' to find examples.
> 
> The connection for these tests is established in python/qemu/machine.py, 
> look at the 'self._qmp_connection' field. This connection is exposed via 
> the qmp(...) method, which the tests use. The library handles the (very 
> small) handshake.

Thanks John and Daniel for these pointers.  If we ever add documentation 
for qmp-shell itself (man page or similar), maybe we should add a 
warning there (not to consume its output in scripts).



> 
> There are also bash tests in ./tests/qemu-iotests/ that handle some QMP 
> by themselves, and might be up your alley for very simple cases. Test 
> 060 sets up its own QMP connection and just echoes JSON into the pipe.

Constructing and echoing JSON in bash is ugly.  Look at the code for 
_filter_qmp in common.filter -- that looks fragile.  That's why I tried 
to script qmp-shell to begin with and started messing with its welcome 
banner.  Instead I should switch to Python (or anything with a proper 
JSON parse/unparse).

-Dov