[Qemu-devel] [RFC][PATCH 00/15] virtproxy: host/guest communication layer

[Michael Roth <mdroth@linux.vnet.ibm.com>]

OVERVIEW:

Virtproxy proxies and multiplexes socket streams over a data channel between a host and a guest (currently network connections, emulated serial, or virtio-serial channels are supported). This allows for services such as guest data collection agents, host/guest file transfer, and event generation/handling to be implemented/deployed as basic socket-based daemons, independently of the actual data channel.

This code is intended to provide a channel-independent abstraction layer for communicating with a QEMU-specific guest agent (in particular, the virtagent RPC guest agent which will follow this in a seperate patchset), but may have general utility beyond this (for instance: ssh/sftp/other guest agents/etc over isa/virtio serial), and so is submitted here as a seperate patchset.

Currently this communication involves 2 daemons (common code): 1 in the guest, and 1 in the host. Each end multiplexes/demultiplexes/proxies connections from the other end. In the future we hope to integrate the host component directly into qemu as a chardev.

BUILD/USAGE INFO:
  make qemu-vp
  ./qemu-vp -h

EXAMPLE USAGE:

 - Proxy http and ssh connections from a host to a guest over a virtio-serial connection:
    # start guest with virtio-serial. for example (RHEL6s13):
    qemu \
    -device virtio-serial \
    -chardev socket,path=/tmp/test0-virtioconsole.sock,server,nowait,id=test0 \
    -device virtconsole,chardev=test0,name=test0 \
    -chardev socket,path=/tmp/test1-virtio-serial.sock,server,nowait,id=test1 \
    -device virtserialport,chardev=test1,name=test1 \
    -chardev socket,path=/tmp/test2-virtio-serial.sock,server,nowait,id=test2 \
    -device virtserialport,chardev=test2,name=test2 \
    ...
    # in the host:
    ./qemu-vp -c unix-connect:/tmp/test2-virtio-serial.sock:- -o http:127.0.0.1:9080 \
              -o ssh:127.0.0.1:9022
    # in the guest:
    ./qemu-vp -c virtserial-open:/dev/virtio-ports/test2:- -i http:127.0.0.1:80 \
              -i ssh:127.0.0.1:22

    # from host, access guest http server
    wget http://locahost:9080
    # from host, access guest ssh server
    ssh localhost -p 9022

 - Proxy http and ssh connections from a host to a guest over a network connection:
    # start guest with network connectivity to host
    # in the guest:
    ./qemu-vp -c tcp-listen:<guest_ip>:9000 -i http:127.0.0.1:80 \
              -i ssh:127.0.0.1:22
    # in the host:
    ./qemu-vp -c tcp-connect:<guest_ip>:9000 -o http:127.0.0.1:9080 \
              -o ssh:127.0.0.1:9022
    ...

By specifying -i and -o options in the host and guest, respectively, the channel can also be used to establish connections from a guest to a host.

KNOWN ISSUES:

 - Deadlocking the guest: In tests over isa-serial ports I've hit cases where the chardev (socket) on the host-side seem to fill up the buffer, likely due to qemu rate-limiting data in accordance with the port's baud rate (which may explain why i hadn't seen this with network-based or virtio-serial data channels. When qemu-vp reads data from client connections it puts it into a VPPacket and tries to send the packet in it's entirety back over the channel. In this particular case that write() blocks (or vp_send_all() spins if we set O_NONBLOCK on the client FD). In the meantime qemu fills up the other end of the socket buffer and ends up spinning in qemu-char:send_all(), basically causing a deadlock between qemu and qemu-vp, and causing the guest to freeze.

   Currently I'm planning on replacing vp_send_all() with a function that simply buffers write()'s, which would allow the use of non-blocking write()'s out to the channel/chardev socket while still retaining wholeness/fifo-ordering of the VPPackets.

 - Sync issues with virtio-serial: This may or may not be related to the issue above, but I noticed some cases where proxied ssh sessions from the guest to the host would "lag" by a few bytes. For instance typing "top" would result in "to" being displayed, and the "p" wouldn't show up till I hit another key. This could be related to how I'm handling the buffering, but I haven't been able to reproduce using a network-based channel.

TODO:

 - Rework vp_send_all() to use buffering to avoid above-mentioned deadlock scenario 
 - Integrate qemu-vp directly into qemu by adding a virtproxy chardev device. For example:
     ./qemu-vp -c unix-connect:/tmp/vp1-virtio-serial.sock:- -o ssh:127.0.0.1:9022
   in the host, would be analogous to:
     qemu \
     -device virtio-serial \
     -chardev virtproxy,oforward=ssh:127.0.0.1:9022,id=vp1 \
     -device virtserialport,chardev=vp1,name=vp1
 - Better channel negotiation to gracefully handle guest reboots/disconnects/etc
 - Add monitor commands to add/remove virtproxy channels/oforwards/iforwards on the fly

 .gitignore  |    1 +
 Makefile    |    4 +-
 configure   |    1 +
 qemu-vp.c   |  618 +++++++++++++++++++++++++++++++++++++++++++++
 virtproxy.c |  799 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 virtproxy.h |   40 +++
 6 files changed, 1462 insertions(+), 1 deletions(-)