Re: [Qemu-devel] [QAPI+QGA 3/3] QEMU Guest Agent (virtagent) v6

[Michael Roth <mdroth@linux.vnet.ibm.com>]

On 07/14/2011 08:53 AM, Zhi Yong Wu wrote:
> On Thu, Jul 14, 2011 at 8:55 PM, Luiz Capitulino<lcapitulino@redhat.com>  wrote:
>> On Thu, 14 Jul 2011 10:53:51 +0800
>> Zhi Yong Wu<zwu.kernel@gmail.com>  wrote:
>>
>>> HI, Michael,
>>>
>>> On Tue, Jul 5, 2011 at 9:21 PM, Michael Roth<mdroth@linux.vnet.ibm.com>  wrote:
>>>> This is Set 3/3 of the QAPI+QGA patchsets.
>>>>
>>>> These patches apply on top of qapi-backport-set2-v5, and can also be obtained from:
>>>> git://repo.or.cz/qemu/mdroth.git qapi-backport-set3-v6
>>>>
>>>> (Set1+2 are a backport of some of the QAPI-related work from Anthony's
>>>> glib tree. The main goal is to get the basic code generation infrastructure in
>>>> place so that it can be used by the guest agent to implement a QMP-like guest
>>>> interface, and so that future work regarding the QMP conversion to QAPI can be
>>>> decoupled from the infrastructure bits. Set3 is the Qemu Guest Agent
>>>> (virtagent), rebased on the new code QAPI code generation infrastructure. This
>>>> is the first user of QAPI, QMP will follow.)
>>>> ___
>>>>
>>>> CHANGES SINCE V5:
>>>>   - switched to using qemu malloc/list functions where possible
>>>>   - removed unused proxy_path field in struct GAState
>>>>   - pid file now opened write-only, removed lockf() in favor of O_EXCL, added SIGINT/SIGTERM signal handlers to handle cleanup
>>>>   - cleaned up error-handling, switched to asserts where appropriate, removed unecessary gotos and NULL checks for qemu_free()/qobject_decref()
>>>>   - refactored send_payload() using helper functions
>>>>   - fixed improper handling of pidfile fd==0
>>>>   - changed guest-shutdown's "shutdown_mode" param to "mode"
>>>>   - switched to using kernel-generated FDs for guest-file-open rather than an autoincrement value
>>>>   - add maximum chunk size of guest-file-read/guest-file-write
>>>>   - added checks to avoid guest-file-write from writing data beyond the provided data buffer
>>>>   - made logging best-effort, removed handling of failures to log as errors
>>>>   - guest-shutdown exec errors now logged to guest syslog, clarified shutdown's asynchronous, no gauruntee nature in schema.
>>>>
>>>> CHANGES SINCE V4:
>>>>   - Removed timeout mechanism via worker thread/pthread_cancel due to potential memory leak. Will re-introduce guest-side timeout support in future version.
>>>>   - Fixed up fsfreeze code to use enums specified within the guest agent's qapi schema.
>>>>   - Fixed memory leak due to a log statement, and added missing cleanup functions for heap-allocated g_error objects.
>>>>   - Made "mode" param to guest-file-open optional, defaults to "r" (read-only)
>>>>
>>>> CHANGES SINCE V3:
>>>>   - Fixed error-handling issues in fsfreeze commands leading to certain mounted directories causing freeze/thaw operations to fail
>>>>   - Added cleanup hook to thaw filesystems on graceful guest agent exit
>>>>   - Removed unused enum values and added additional details to schema documentation
>>>>   - Fixed build issue that was missed due to deprecated files in source tree, removed unused includes
>>>>
>>>> CHANGES SINCE V2:
>>>>   - Rebased on new QAPI code generation framework
>>>>   - Dropped ability for QMP to act as a proxy for the guest agent, will be added when new QMP server is backported from Anthony's glib tree
>>>>   - Replaced negotiation/control events with a simple 2-way handshake implemented by a standard RPC (guest-sync)
>>>>   - Removed enforcement of "pristine" sessions, state is now global/persistant across multiple clients/connections
>>>>   - Fixed segfault in logging code
>>>>   - Added Jes' filesystem freeze patches
>>>>   - General cleanups
>>>>
>>>> CHANGES SINCE V1:
>>>>   - Added guest agent worker thread to execute RPCs in the guest. With this in place we have a reliable timeout mechanism for hung commands, currently set at 30 seconds.
>>>>   - Add framework for registering init/cleanup routines for stateful RPCs to clean up after themselves after a timeout.
>>>>   - Added the following RPCs: guest-file-{open,close,read,write,seek}, guest-shutdown, guest-info, and removed stubs for guest-view-file (now deprecated)
>>>>   - Added GUEST_AGENT_UP/GUEST_AGENT_DOWN QMP events
>>>>   - Switched to a TCP-style host-initiated 3-way handshake for channel negotiation, this simplifies client negotiation/interaction over the wire
>>>>   - Added configurable log level/log file/pid file options for guest agent
>>>>   - Various fixes for bugs/memory leaks and checkpatch.pl fixups
>>>>
>>>> ISSUES/TODOS:
>>>>   - Add unit tests for guest agent wire protocol
>>>>
>>>> OVERVIEW
>>>>
>>>> For a better overview of what these patches are meant to accomplish, please reference the RFC for virtagent:
>>>>
>>>> http://comments.gmane.org/gmane.comp.emulators.qemu/96096
>>>>
>>>> These patches integrate the previous virtagent guest agent work directly in QAPI/QMP to leverage it's auto-generated marshalling code. This has numerous benefits:
>>>>
>>>>   - addresses previous concerns over relying on external libraries to handle data encapsulation
>>>>   - reduces the need for manual unmarshalling of requests/responses, which makes adding new RPCs much safer/less error-prone, as well as cutting down on redundant code
>>>>   - QAPI documentation aligns completely with guest-side RPC implementation
>>>>   - is Just Better (TM)
>>>>
>>>> BUILD/USAGE
>>>>
>>>> build:
>>>>   ./configure --target-list=x86_64-softmmu
>>>>   make
>>>>   make qemu-ga #should be built on|for target guest
>>>>
>>>> start guest:
>>>>   qemu \
>>>>   -drive file=/home/mdroth/vm/rhel6_64_base.raw,snapshot=off,if=virtio \
>>>>   -net nic,model=virtio,macaddr=52:54:00:12:34:00 \
>>>>   -net tap,script=/etc/qemu-ifup \
>>>>   -vnc :1 -m 1024 --enable-kvm \
>>>>   -chardev socket,path=/tmp/qga.sock,server,nowait,id=qga \
>>>>   -device virtio-serial \
>>>>   -device virtserialport,chardev=qga,name=qga"
>>>>
>>>> use guest agent:
>>>>   ./qemu-ga -h
>>>>   ./qemu-ga -c virtio-serial -p /dev/virtio-ports/qga
>>> Is the above command "./qemu-ga" issued on guest teminal or hypervisor teminal?
>>
>> In the guest.
>
> Can we make it work similiar as a linux service? It may is more
> convenient to use.
>
> [root@f12 ~]# service qemu-ga start/stop/status/restart
>
> [root@f12 ~]# chkconfig qemu-ga on/off
>

Definitely, we need the agent to act as a service to be able to rely on 
it remaining after restarts and whatnot. We have some init scripts and 
whatnot for in-house testing/deployment, but this is somewhat 
distro-dependent and is more of a packaging concern than something for 
qemu-devel.

Although, if we do deployment via the guest tools ISO that I sent an RFC 
out for a while back, it may be worthwhile to cover this as part of that 
discussion since ideally we'd come up with something that Just Works on 
most distros. I hope to have a prototype of ISO generation scripts out 
within the next week or so.

>>
>>>
>>>>
>>>> start/use qmp:
>>>>   mdroth@illuin:~$ sudo socat unix-connect:/tmp/qga.sock readline
>>>>   {"execute":"guest-sync", "arguments":{"id":1234}}
>>>>   {"return": 1234}
>>>>
>>>>   {"execute":"guest-info"}
>>>>   {"return": {}}
>>>>
>>>>   {"execute": "guest-info"}
>>>>   {"return": {"version": "1.0"}}
>>>>
>>>>   {"execute":"guest-file-open", "arguments":{"filepath":"/tmp/testqga","mode":"w+"}}
>>>>   {"return": 0}
>>>>   {"execute":"guest-file-write", "arguments":{"filehandle":0,"data_b64":"aGVsbG8gd29ybGQhCg==","count":13}} // writes "hello world!\n"
>>>>   {"return": {"count": 13, "eof": false}}
>>>>
>>>>   {"execute":"guest-file-open", "arguments":{"filepath":"/tmp/testqga","mode":"r"}}
>>>>   {"return": 1}
>>>>   {"execute":"guest-file-read", "arguments":{"filehandle":1,"count":1024}}
>>>>   {"return": {"buf": "aGVsbG8gd29ybGQhCg==", "count": 13, "eof": true}}
>>>>   {"execute":"guest-file-close","arguments":{"filehandle":1}}
>>>>   {"return": {}}
>>>>
>>>>   Makefile                        |   22 +-
>>>>   configure                       |    1 +
>>>>   qapi-schema-guest.json          |  202 +++++++++++++
>>>>   qemu-ga.c                       |  631 +++++++++++++++++++++++++++++++++++++++
>>>>   qerror.c                        |    4 +
>>>>   qerror.h                        |    3 +
>>>>   qga/guest-agent-command-state.c |   73 +++++
>>>>   qga/guest-agent-commands.c      |  522 ++++++++++++++++++++++++++++++++
>>>>   qga/guest-agent-core.h          |   30 ++
>>>>   9 files changed, 1483 insertions(+), 5 deletions(-)
>>>>
>>>>
>>>>
>>
>>
>
>
>