From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=58316 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1P9Mb4-0006zu-BK for qemu-devel@nongnu.org; Fri, 22 Oct 2010 14:44:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1P9Maz-0001ni-JP for qemu-devel@nongnu.org; Fri, 22 Oct 2010 14:44:10 -0400 Received: from e7.ny.us.ibm.com ([32.97.182.137]:35387) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1P9Maz-0001jy-GV for qemu-devel@nongnu.org; Fri, 22 Oct 2010 14:44:05 -0400 Received: from d01relay01.pok.ibm.com (d01relay01.pok.ibm.com [9.56.227.233]) by e7.ny.us.ibm.com (8.14.4/8.13.1) with ESMTP id o9MIRjsn022930 for ; Fri, 22 Oct 2010 14:27:45 -0400 Received: from d01av01.pok.ibm.com (d01av01.pok.ibm.com [9.56.224.215]) by d01relay01.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id o9MIhfj5383252 for ; Fri, 22 Oct 2010 14:43:41 -0400 Received: from d01av01.pok.ibm.com (loopback [127.0.0.1]) by d01av01.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id o9MIhdqo013799 for ; Fri, 22 Oct 2010 14:43:40 -0400 From: Michael Roth Date: Fri, 22 Oct 2010 13:43:16 -0500 Message-Id: <1287773011-24726-1-git-send-email-mdroth@linux.vnet.ibm.com> Subject: [Qemu-devel] [RFC][PATCH 00/15] virtproxy: host/guest communication layer List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: aliguori@linux.vnet.ibm.com, ryanh@us.ibm.com, agl@linux.vnet.ibm.com, mdroth@linux.vnet.ibm.com, abeekhof@redhat.com OVERVIEW: Virtproxy proxies and multiplexes socket streams over a data channel between a host and a guest (currently network connections, emulated serial, or virtio-serial channels are supported). This allows for services such as guest data collection agents, host/guest file transfer, and event generation/handling to be implemented/deployed as basic socket-based daemons, independently of the actual data channel. This code is intended to provide a channel-independent abstraction layer for communicating with a QEMU-specific guest agent (in particular, the virtagent RPC guest agent which will follow this in a seperate patchset), but may have general utility beyond this (for instance: ssh/sftp/other guest agents/etc over isa/virtio serial), and so is submitted here as a seperate patchset. Currently this communication involves 2 daemons (common code): 1 in the guest, and 1 in the host. Each end multiplexes/demultiplexes/proxies connections from the other end. In the future we hope to integrate the host component directly into qemu as a chardev. BUILD/USAGE INFO: make qemu-vp ./qemu-vp -h EXAMPLE USAGE: - Proxy http and ssh connections from a host to a guest over a virtio-serial connection: # start guest with virtio-serial. for example (RHEL6s13): qemu \ -device virtio-serial \ -chardev socket,path=/tmp/test0-virtioconsole.sock,server,nowait,id=test0 \ -device virtconsole,chardev=test0,name=test0 \ -chardev socket,path=/tmp/test1-virtio-serial.sock,server,nowait,id=test1 \ -device virtserialport,chardev=test1,name=test1 \ -chardev socket,path=/tmp/test2-virtio-serial.sock,server,nowait,id=test2 \ -device virtserialport,chardev=test2,name=test2 \ ... # in the host: ./qemu-vp -c unix-connect:/tmp/test2-virtio-serial.sock:- -o http:127.0.0.1:9080 \ -o ssh:127.0.0.1:9022 # in the guest: ./qemu-vp -c virtserial-open:/dev/virtio-ports/test2:- -i http:127.0.0.1:80 \ -i ssh:127.0.0.1:22 # from host, access guest http server wget http://locahost:9080 # from host, access guest ssh server ssh localhost -p 9022 - Proxy http and ssh connections from a host to a guest over a network connection: # start guest with network connectivity to host # in the guest: ./qemu-vp -c tcp-listen::9000 -i http:127.0.0.1:80 \ -i ssh:127.0.0.1:22 # in the host: ./qemu-vp -c tcp-connect::9000 -o http:127.0.0.1:9080 \ -o ssh:127.0.0.1:9022 ... By specifying -i and -o options in the host and guest, respectively, the channel can also be used to establish connections from a guest to a host. KNOWN ISSUES: - Deadlocking the guest: In tests over isa-serial ports I've hit cases where the chardev (socket) on the host-side seem to fill up the buffer, likely due to qemu rate-limiting data in accordance with the port's baud rate (which may explain why i hadn't seen this with network-based or virtio-serial data channels. When qemu-vp reads data from client connections it puts it into a VPPacket and tries to send the packet in it's entirety back over the channel. In this particular case that write() blocks (or vp_send_all() spins if we set O_NONBLOCK on the client FD). In the meantime qemu fills up the other end of the socket buffer and ends up spinning in qemu-char:send_all(), basically causing a deadlock between qemu and qemu-vp, and causing the guest to freeze. Currently I'm planning on replacing vp_send_all() with a function that simply buffers write()'s, which would allow the use of non-blocking write()'s out to the channel/chardev socket while still retaining wholeness/fifo-ordering of the VPPackets. - Sync issues with virtio-serial: This may or may not be related to the issue above, but I noticed some cases where proxied ssh sessions from the guest to the host would "lag" by a few bytes. For instance typing "top" would result in "to" being displayed, and the "p" wouldn't show up till I hit another key. This could be related to how I'm handling the buffering, but I haven't been able to reproduce using a network-based channel. TODO: - Rework vp_send_all() to use buffering to avoid above-mentioned deadlock scenario - Integrate qemu-vp directly into qemu by adding a virtproxy chardev device. For example: ./qemu-vp -c unix-connect:/tmp/vp1-virtio-serial.sock:- -o ssh:127.0.0.1:9022 in the host, would be analogous to: qemu \ -device virtio-serial \ -chardev virtproxy,oforward=ssh:127.0.0.1:9022,id=vp1 \ -device virtserialport,chardev=vp1,name=vp1 - Better channel negotiation to gracefully handle guest reboots/disconnects/etc - Add monitor commands to add/remove virtproxy channels/oforwards/iforwards on the fly .gitignore | 1 + Makefile | 4 +- configure | 1 + qemu-vp.c | 618 +++++++++++++++++++++++++++++++++++++++++++++ virtproxy.c | 799 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ virtproxy.h | 40 +++ 6 files changed, 1462 insertions(+), 1 deletions(-)