qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
* [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework
@ 2015-09-18 13:18 Daniel P. Berrange
  2015-09-18 13:18 ` [Qemu-devel] [PATCH v1 01/16] sockets: add helpers for creating SocketAddress from a socket Daniel P. Berrange
                   ` (16 more replies)
  0 siblings, 17 replies; 32+ messages in thread
From: Daniel P. Berrange @ 2015-09-18 13:18 UTC (permalink / raw)
  To: qemu-devel; +Cc: Paolo Bonzini, Dr. David Alan Gilbert, Gerd Hoffmann

This series of patches is a first formal submission for review
of another chunk of my large series supporting TLS across chardevs,
VNC and migration, previously shown here:

  https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg00829.html

In this series, I have provided only the general I/O channels
framework, which will ultimately be used by VNC, chardev and
migration code, whicih currently work as follows:

 - VNC: uses raw sockets APIs directly, layering in TLS, SASL
   and websockets where needed. This has resulted in what should
   be fairly generic code for TLS and websockets being entangled
   with the rest of the VNC server code.

 - Chardevs: uses GLib's GIOChannel framework. This only provides
   implementations for sockets and files. Its API lacks support for
   using struct iovec for read/writes, file descriptor passing,
   misc sockets ioctls/fcntls. While you can work around these
   problems by accessing the underling file descriptor directly,
   this breaks the encapsulation, requiring callers to know about
   specific implementations. It also does not have integration
   with QEMU Error reporting framework. So while the GIOChannel
   is extensible, extending it would result in throwing away
   pretty much the entire of the existing implementations

 - Migration: uses QEMUFile framework. The provides an abstract
   interface for I/O channels used during migration. It has
   impls for sockets, files, memory buffers & commands. While
   it has struct iovec support for writes, it does not have
   the same for reads. It also doesn't support file descriptor
   passing or control of the sockets ioctls/fcntls. It also
   does not have any explicit event loop integration, expecting
   the callers to directly access the underling file desctriptor
   and setup events on that. This limits its utility in cases
   where you need channels which are not backed by file
   descriptors. It has no integration with QEMU Error object
   for error reporting, has fixed semantics wrt writes
   ie must always do a full write, no partial writes, and
   most strangely forces either input or output mode, but
   refuses to allow both, so no bi-directional channels!

Out of all of these, the migration code is probably closest
to what is needed, but is still a good way off from being a
generic framework that be can reused outside of the migration
code.

There is also the GLib GIO library which provides a generic
framework, but we can't depend on that due to the minimum
GLib requirement. It also has various missing pieces such as
file descriptor passing, and no support for struct iovec
either.

Hence, this series was born, which tries to take the best of
the designs for the GIOChannel, QIO and QEMUFile code to
provide QIOChannel. Right from the start this new framework
is explicitly isolated from any other QEMU subsystem, so its
implementation will not get mixed up with specific use cases.

The QIOChannel abstract base class defines the overall API
contract

 - qio_channel_{write,writev,write_full} for writing data. The
   underling impl always uses struct iovec, but non-iov
   APIs are provided as simple wrappers for convenience

 - qio_channel_{read,readv,read_full} for reading data. The
   underling impl always uses struct iovec, but non-iov
   APIs are provided as simple wrappers for convenience

 - qio_channel_has_feature to determine which optional
   features the channel supports - eg file descriptor
   passing, nagle, etc

 - qio_channel_set_{blocking,delay,cork} for various fcntl
   and ioctl controls

 - qio_channel_{close,shutdown} for closing the I/O stream
   or individual channels

 - qio_channel_seek for random access channels

 - qio_channel_{add,create}_watch for integration into the
   main loop for event notifications

 - qio_channel_wait for blocking of execution pending an
   event notification

 - qio_channel_yield for switching coroutine until an
   event notification

All the APIs support Error ** object arguments where needed.
The object is built using QOM, in order to get reference
counting and sub-classing with type checking. They are *not*
user creatable/visible objects though - these are internal
infrastructure, so we will be free to adapt APIs/objects at
will over time.

In this series there are a variety of implementations. Some
of them provide an actual base layer data transport, while
others provide a data translation layer:

 - QIOChannelSocket - for socket based I/O, IPv4, IPV6 & UNIX,
                      both stream and datagram.
 - QIOChannelFile - any non-socket file, plain file, char dev,
                    fifo, pipe, etc
 - QIOChannelTLS - data translation layer to apply TLS protocol
 - QIOChannelWebsock - data translation layer to apply the
                       websockets server protocol
 - QIOChannelCommand - spawn & communicate with a command via
                       pipes
 - QIOChannelBuffer - a simple in-memory buffer

These 6 implementations are sufficient to support the current
needs of the VNC server, chardev network backends, and all
the various migration protocols, except RDMA (which is part
of the larger RFC series I've posted previously).

The sockets implementation in particular solves a long standing
limitation of the qemu-sockets.c API by providing a truely
asynchronous API for establishing listener sockets / connections.
The current code is only async for the socket establishment,
but still blocks the caller for DNS resolution.

I have not attempted to make the socket implementation support
multiple listener sockets in a single object. I looked at this
but it would significantly complicate the QIOChannelSocket
implementation. I intend to suggest an alternative approach
for that problem at a later date, whereby we define a
QIONetworkService object, which provides the infrastructure
for managing multiple QIOChannelSocket listeners, and clients,
which was resuable for any QEMU network service.

There are unit tests for the socket, file, tls, and command
channel implementations. I didn't create unit test for the
websock impl, as it needs a websock client to usefully test
it which doesn't exist in QEMU yet. The memory buffer impl
also lacks unit tests for now, simply due to not getting
around to it yet.

This series merely introduces all the new framework. To keep
it an acceptable length for review, it doesn't include any of
the patches which actually use the new APIs. If reviewers want
to see real world usage of these APIs, they can refer to my
previous RFC series.

  https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg00829.html

In particular

 - VNC conversion to QIOChannelSocket
    https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg00851.html

 - VNC conversion to QIOChannelTLS
    https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg00845.html

 - VNC conversion to QIOChannelWebsock
    https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg00842.html

 - Chardev conversion to QIOChannelSocket
    https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg00849.html

 - Chardev addition of TLS support
    https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg00848.html

 - All the migration ones, but most interesting addition of TLS
    https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg00871.html

David Gilbert had some feedback on my previous RFC series which
I have incorporated into this new posting. I did forget his
comments about using #defines instead of magic constants in
the websock patch, so that's still on my todo list.

Finally I've listed myself as maintainer for the new io/ and
include/io/ directory prefixes.

Daniel P. Berrange (16):
  sockets: add helpers for creating SocketAddress from a socket
  sockets: move qapi_copy_SocketAddress into qemu-sockets.c
  sockets: allow port to be NULL when listening on IP address
  ui: convert VNC startup code to use SocketAddress
  osdep: add qemu_fork() wrapper for safely handling signals
  coroutine: move into libqemuutil.a library
  io: add abstract QIOChannel classes
  io: add helper module for creating watches on FDs
  io: pull Buffer code out of VNC module
  io: add QIOTask class for async operations
  io: add QIOChannelSocket class
  io: add QIOChannelFile class
  io: add QIOChannelTLS class
  io: add QIOChannelWebsock class
  io: add QIOChannelCommand class
  io: add QIOChannelBuffer class

 MAINTAINERS                                        |   7 +
 Makefile                                           |   2 +
 Makefile.objs                                      |   9 +-
 Makefile.target                                    |   2 +
 block.c                                            |   2 +-
 block/qcow2.h                                      |   2 +-
 block/vdi.c                                        |   2 +-
 block/write-threshold.c                            |   2 +-
 blockjob.c                                         |   2 +-
 configure                                          |  11 +
 hw/9pfs/codir.c                                    |   2 +-
 hw/9pfs/cofile.c                                   |   2 +-
 hw/9pfs/cofs.c                                     |   2 +-
 hw/9pfs/coxattr.c                                  |   2 +-
 hw/9pfs/virtio-9p-coth.c                           |   2 +-
 hw/9pfs/virtio-9p-coth.h                           |   2 +-
 hw/9pfs/virtio-9p.h                                |   2 +-
 include/block/block.h                              |   2 +-
 include/block/block_int.h                          |   2 +-
 include/io/buffer.h                                | 118 +++
 include/io/channel-buffer.h                        |  59 ++
 include/io/channel-command.h                       |  91 ++
 include/io/channel-file.h                          |  93 ++
 include/io/channel-socket.h                        | 244 ++++++
 include/io/channel-tls.h                           | 142 +++
 include/io/channel-watch.h                         |  72 ++
 include/io/channel-websock.h                       | 108 +++
 include/io/channel.h                               | 523 ++++++++++++
 include/io/task.h                                  | 256 ++++++
 include/{block => qemu}/coroutine.h                |   0
 include/{block => qemu}/coroutine_int.h            |   2 +-
 include/qemu/osdep.h                               |  16 +
 include/qemu/sockets.h                             |  34 +
 io/Makefile.objs                                   |  10 +
 io/buffer.c                                        |  65 ++
 io/channel-buffer.c                                | 255 ++++++
 io/channel-command.c                               | 369 ++++++++
 io/channel-file.c                                  | 237 ++++++
 io/channel-socket.c                                | 763 +++++++++++++++++
 io/channel-tls.c                                   | 419 +++++++++
 io/channel-watch.c                                 | 200 +++++
 io/channel-websock.c                               | 948 +++++++++++++++++++++
 io/channel.c                                       | 290 +++++++
 io/task.c                                          | 159 ++++
 migration/qemu-file-buf.c                          |   2 +-
 migration/qemu-file-stdio.c                        |   2 +-
 migration/qemu-file-unix.c                         |   2 +-
 migration/qemu-file.c                              |   2 +-
 migration/rdma.c                                   |   2 +-
 nbd.c                                              |   2 +-
 qemu-char.c                                        |  25 -
 scripts/create_config                              |   9 +
 tests/.gitignore                                   |   7 +
 tests/Makefile                                     |  16 +
 tests/io-channel-helpers.c                         | 247 ++++++
 tests/io-channel-helpers.h                         |  33 +
 tests/test-coroutine.c                             |   4 +-
 tests/test-io-channel-command.c                    | 122 +++
 tests/test-io-channel-file.c                       |  91 ++
 tests/test-io-channel-socket.c                     | 367 ++++++++
 tests/test-io-channel-tls.c                        | 335 ++++++++
 tests/test-io-task.c                               | 276 ++++++
 tests/test-vmstate.c                               |   2 +-
 thread-pool.c                                      |   2 +-
 trace-events                                       |  56 ++
 ui/vnc-auth-sasl.c                                 |   4 +-
 ui/vnc-enc-tight.c                                 |  38 +-
 ui/vnc-enc-zlib.c                                  |   6 +-
 ui/vnc-enc-zrle.c                                  |  18 +-
 ui/vnc-jobs.c                                      |  15 +-
 ui/vnc-ws.c                                        |  36 +-
 ui/vnc-ws.h                                        |   6 +-
 ui/vnc.c                                           | 228 +++--
 ui/vnc.h                                           |  50 +-
 util/Makefile.objs                                 |   3 +
 coroutine-gthread.c => util/coroutine-gthread.c    |   2 +-
 .../coroutine-sigaltstack.c                        |   2 +-
 coroutine-ucontext.c => util/coroutine-ucontext.c  |   2 +-
 coroutine-win32.c => util/coroutine-win32.c        |   2 +-
 util/oslib-posix.c                                 |  71 ++
 util/oslib-win32.c                                 |   9 +
 qemu-coroutine-io.c => util/qemu-coroutine-io.c    |   2 +-
 .../qemu-coroutine-lock.c                          |   4 +-
 .../qemu-coroutine-sleep.c                         |   2 +-
 qemu-coroutine.c => util/qemu-coroutine.c          |   4 +-
 util/qemu-sockets.c                                | 158 +++-
 86 files changed, 7511 insertions(+), 287 deletions(-)
 create mode 100644 include/io/buffer.h
 create mode 100644 include/io/channel-buffer.h
 create mode 100644 include/io/channel-command.h
 create mode 100644 include/io/channel-file.h
 create mode 100644 include/io/channel-socket.h
 create mode 100644 include/io/channel-tls.h
 create mode 100644 include/io/channel-watch.h
 create mode 100644 include/io/channel-websock.h
 create mode 100644 include/io/channel.h
 create mode 100644 include/io/task.h
 rename include/{block => qemu}/coroutine.h (100%)
 rename include/{block => qemu}/coroutine_int.h (98%)
 create mode 100644 io/Makefile.objs
 create mode 100644 io/buffer.c
 create mode 100644 io/channel-buffer.c
 create mode 100644 io/channel-command.c
 create mode 100644 io/channel-file.c
 create mode 100644 io/channel-socket.c
 create mode 100644 io/channel-tls.c
 create mode 100644 io/channel-watch.c
 create mode 100644 io/channel-websock.c
 create mode 100644 io/channel.c
 create mode 100644 io/task.c
 create mode 100644 tests/io-channel-helpers.c
 create mode 100644 tests/io-channel-helpers.h
 create mode 100644 tests/test-io-channel-command.c
 create mode 100644 tests/test-io-channel-file.c
 create mode 100644 tests/test-io-channel-socket.c
 create mode 100644 tests/test-io-channel-tls.c
 create mode 100644 tests/test-io-task.c
 rename coroutine-gthread.c => util/coroutine-gthread.c (99%)
 rename coroutine-sigaltstack.c => util/coroutine-sigaltstack.c (99%)
 rename coroutine-ucontext.c => util/coroutine-ucontext.c (99%)
 rename coroutine-win32.c => util/coroutine-win32.c (98%)
 rename qemu-coroutine-io.c => util/qemu-coroutine-io.c (99%)
 rename qemu-coroutine-lock.c => util/qemu-coroutine-lock.c (98%)
 rename qemu-coroutine-sleep.c => util/qemu-coroutine-sleep.c (96%)
 rename qemu-coroutine.c => util/qemu-coroutine.c (98%)

-- 
2.4.3

^ permalink raw reply	[flat|nested] 32+ messages in thread

* [Qemu-devel] [PATCH v1 01/16] sockets: add helpers for creating SocketAddress from a socket
  2015-09-18 13:18 [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework Daniel P. Berrange
@ 2015-09-18 13:18 ` Daniel P. Berrange
  2015-09-18 13:18 ` [Qemu-devel] [PATCH v1 02/16] sockets: move qapi_copy_SocketAddress into qemu-sockets.c Daniel P. Berrange
                   ` (15 subsequent siblings)
  16 siblings, 0 replies; 32+ messages in thread
From: Daniel P. Berrange @ 2015-09-18 13:18 UTC (permalink / raw)
  To: qemu-devel; +Cc: Paolo Bonzini, Dr. David Alan Gilbert, Gerd Hoffmann

Add two helper methods that, given a socket file descriptor,
can return a populated SocketAddress struct containing either
the local or remote address information.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
 include/qemu/sockets.h |  30 ++++++++++++++
 util/qemu-sockets.c    | 110 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 140 insertions(+)

diff --git a/include/qemu/sockets.h b/include/qemu/sockets.h
index c174b5c..3ea7cc9 100644
--- a/include/qemu/sockets.h
+++ b/include/qemu/sockets.h
@@ -88,4 +88,34 @@ int socket_dgram(SocketAddress *remote, SocketAddress *local, Error **errp);
 int parse_host_port(struct sockaddr_in *saddr, const char *str);
 int socket_init(void);
 
+/**
+ * socket_local_address:
+ * @fd: the socket file handle
+ * @errp: pointer to uninitialized error object
+ *
+ * Get the string representation of the local socket
+ * address. A pointer to the allocated address information
+ * struct will be returned, which the caller is required to
+ * release with a call qapi_free_SocketAddress when no
+ * longer required.
+ *
+ * Returns: the socket address struct, or NULL on error
+ */
+SocketAddress *socket_local_address(int fd, Error **errp);
+
+/**
+ * socket_remote_address:
+ * @fd: the socket file handle
+ * @errp: pointer to uninitialized error object
+ *
+ * Get the string representation of the remote socket
+ * address. A pointer to the allocated address information
+ * struct will be returned, which the caller is required to
+ * release with a call qapi_free_SocketAddress when no
+ * longer required.
+ *
+ * Returns: the socket address struct, or NULL on error
+ */
+SocketAddress *socket_remote_address(int fd, Error **errp);
+
 #endif /* QEMU_SOCKET_H */
diff --git a/util/qemu-sockets.c b/util/qemu-sockets.c
index 2add83a..2bdfacf 100644
--- a/util/qemu-sockets.c
+++ b/util/qemu-sockets.c
@@ -1015,3 +1015,113 @@ int socket_dgram(SocketAddress *remote, SocketAddress *local, Error **errp)
     qemu_opts_del(opts);
     return fd;
 }
+
+
+static SocketAddress *
+socket_sockaddr_to_address_inet(struct sockaddr_storage *sa,
+                                socklen_t salen,
+                                Error **errp)
+{
+    char host[NI_MAXHOST];
+    char serv[NI_MAXSERV];
+    SocketAddress *addr;
+    int ret;
+
+    ret = getnameinfo((struct sockaddr *)sa, salen,
+                      host, sizeof(host),
+                      serv, sizeof(serv),
+                      NI_NUMERICHOST | NI_NUMERICSERV);
+    if (ret != 0) {
+        error_setg(errp, "Cannot format numeric socket address: %s\n",
+                   gai_strerror(ret));
+        return NULL;
+    }
+
+    addr = g_new0(SocketAddress, 1);
+    addr->kind = SOCKET_ADDRESS_KIND_INET;
+    addr->inet = g_new0(InetSocketAddress, 1);
+    addr->inet->host = g_strdup(host);
+    addr->inet->port = g_strdup(serv);
+    if (sa->ss_family == AF_INET) {
+        addr->inet->ipv4 = true;
+    } else {
+        addr->inet->ipv6 = true;
+    }
+
+    return addr;
+}
+
+
+#ifndef WIN32
+static SocketAddress *
+socket_sockaddr_to_address_unix(struct sockaddr_storage *sa,
+                                socklen_t salen,
+                                Error **errp)
+{
+    SocketAddress *addr;
+    struct sockaddr_un *su = (struct sockaddr_un *)sa;
+
+    addr = g_new0(SocketAddress, 1);
+    addr->kind = SOCKET_ADDRESS_KIND_UNIX;
+    addr->q_unix = g_new0(UnixSocketAddress, 1);
+    if (su->sun_path[0]) {
+        addr->q_unix->path = g_strndup(su->sun_path,
+                                       sizeof(su->sun_path));
+    }
+
+    return addr;
+}
+#endif /* WIN32 */
+
+static SocketAddress *
+socket_sockaddr_to_address(struct sockaddr_storage *sa,
+                           socklen_t salen,
+                           Error **errp)
+{
+    switch (sa->ss_family) {
+    case AF_INET:
+    case AF_INET6:
+        return socket_sockaddr_to_address_inet(sa, salen, errp);
+
+#ifndef WIN32
+    case AF_UNIX:
+        return socket_sockaddr_to_address_unix(sa, salen, errp);
+#endif /* WIN32 */
+
+    default:
+        error_setg(errp, "socket family %d unsupported",
+                   sa->ss_family);
+        return NULL;
+    }
+    return 0;
+}
+
+
+SocketAddress *socket_local_address(int fd, Error **errp)
+{
+    struct sockaddr_storage ss;
+    socklen_t sslen = sizeof(ss);
+
+    if (getsockname(fd, (struct sockaddr *)&ss, &sslen) < 0) {
+        error_setg_errno(errp, socket_error(), "%s",
+                         "Unable to query local socket address");
+        return NULL;
+    }
+
+    return socket_sockaddr_to_address(&ss, sslen, errp);
+}
+
+
+SocketAddress *socket_remote_address(int fd, Error **errp)
+{
+    struct sockaddr_storage ss;
+    socklen_t sslen = sizeof(ss);
+
+    if (getpeername(fd, (struct sockaddr *)&ss, &sslen) < 0) {
+        error_setg_errno(errp, socket_error(), "%s",
+                         "Unable to query remote socket address");
+        return NULL;
+    }
+
+    return socket_sockaddr_to_address(&ss, sslen, errp);
+}
-- 
2.4.3

^ permalink raw reply related	[flat|nested] 32+ messages in thread

* [Qemu-devel] [PATCH v1 02/16] sockets: move qapi_copy_SocketAddress into qemu-sockets.c
  2015-09-18 13:18 [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework Daniel P. Berrange
  2015-09-18 13:18 ` [Qemu-devel] [PATCH v1 01/16] sockets: add helpers for creating SocketAddress from a socket Daniel P. Berrange
@ 2015-09-18 13:18 ` Daniel P. Berrange
  2015-09-18 13:18 ` [Qemu-devel] [PATCH v1 03/16] sockets: allow port to be NULL when listening on IP address Daniel P. Berrange
                   ` (14 subsequent siblings)
  16 siblings, 0 replies; 32+ messages in thread
From: Daniel P. Berrange @ 2015-09-18 13:18 UTC (permalink / raw)
  To: qemu-devel; +Cc: Paolo Bonzini, Dr. David Alan Gilbert, Gerd Hoffmann

The qapi_copy_SocketAddress method is going to be useful
in more places than just qemu-char.c, so move it into
the qemu-sockets.c file to allow its reuse.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
 include/qemu/sockets.h |  4 ++++
 qemu-char.c            | 25 -------------------------
 util/qemu-sockets.c    | 30 ++++++++++++++++++++++++++++++
 3 files changed, 34 insertions(+), 25 deletions(-)

diff --git a/include/qemu/sockets.h b/include/qemu/sockets.h
index 3ea7cc9..5a183c5 100644
--- a/include/qemu/sockets.h
+++ b/include/qemu/sockets.h
@@ -118,4 +118,8 @@ SocketAddress *socket_local_address(int fd, Error **errp);
  */
 SocketAddress *socket_remote_address(int fd, Error **errp);
 
+
+void qapi_copy_SocketAddress(SocketAddress **p_dest,
+                             SocketAddress *src);
+
 #endif /* QEMU_SOCKET_H */
diff --git a/qemu-char.c b/qemu-char.c
index 653ea10..50ea4f7 100644
--- a/qemu-char.c
+++ b/qemu-char.c
@@ -92,31 +92,6 @@
 
 /***********************************************************/
 /* Socket address helpers */
-static void qapi_copy_SocketAddress(SocketAddress **p_dest,
-                                    SocketAddress *src)
-{
-    QmpOutputVisitor *qov;
-    QmpInputVisitor *qiv;
-    Visitor *ov, *iv;
-    QObject *obj;
-
-    *p_dest = NULL;
-
-    qov = qmp_output_visitor_new();
-    ov = qmp_output_get_visitor(qov);
-    visit_type_SocketAddress(ov, &src, NULL, &error_abort);
-    obj = qmp_output_get_qobject(qov);
-    qmp_output_visitor_cleanup(qov);
-    if (!obj) {
-        return;
-    }
-
-    qiv = qmp_input_visitor_new(obj);
-    iv = qmp_input_get_visitor(qiv);
-    visit_type_SocketAddress(iv, p_dest, NULL, &error_abort);
-    qmp_input_visitor_cleanup(qiv);
-    qobject_decref(obj);
-}
 
 static int SocketAddress_to_str(char *dest, int max_len,
                                 const char *prefix, SocketAddress *addr,
diff --git a/util/qemu-sockets.c b/util/qemu-sockets.c
index 2bdfacf..9cc5bee 100644
--- a/util/qemu-sockets.c
+++ b/util/qemu-sockets.c
@@ -25,6 +25,9 @@
 #include "monitor/monitor.h"
 #include "qemu/sockets.h"
 #include "qemu/main-loop.h"
+#include "qapi/qmp-input-visitor.h"
+#include "qapi/qmp-output-visitor.h"
+#include "qapi-visit.h"
 
 #ifndef AI_ADDRCONFIG
 # define AI_ADDRCONFIG 0
@@ -1125,3 +1128,30 @@ SocketAddress *socket_remote_address(int fd, Error **errp)
 
     return socket_sockaddr_to_address(&ss, sslen, errp);
 }
+
+
+void qapi_copy_SocketAddress(SocketAddress **p_dest,
+                             SocketAddress *src)
+{
+    QmpOutputVisitor *qov;
+    QmpInputVisitor *qiv;
+    Visitor *ov, *iv;
+    QObject *obj;
+
+    *p_dest = NULL;
+
+    qov = qmp_output_visitor_new();
+    ov = qmp_output_get_visitor(qov);
+    visit_type_SocketAddress(ov, &src, NULL, &error_abort);
+    obj = qmp_output_get_qobject(qov);
+    qmp_output_visitor_cleanup(qov);
+    if (!obj) {
+        return;
+    }
+
+    qiv = qmp_input_visitor_new(obj);
+    iv = qmp_input_get_visitor(qiv);
+    visit_type_SocketAddress(iv, p_dest, NULL, &error_abort);
+    qmp_input_visitor_cleanup(qiv);
+    qobject_decref(obj);
+}
-- 
2.4.3

^ permalink raw reply related	[flat|nested] 32+ messages in thread

* [Qemu-devel] [PATCH v1 03/16] sockets: allow port to be NULL when listening on IP address
  2015-09-18 13:18 [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework Daniel P. Berrange
  2015-09-18 13:18 ` [Qemu-devel] [PATCH v1 01/16] sockets: add helpers for creating SocketAddress from a socket Daniel P. Berrange
  2015-09-18 13:18 ` [Qemu-devel] [PATCH v1 02/16] sockets: move qapi_copy_SocketAddress into qemu-sockets.c Daniel P. Berrange
@ 2015-09-18 13:18 ` Daniel P. Berrange
  2015-09-18 13:18 ` [Qemu-devel] [PATCH v1 04/16] ui: convert VNC startup code to use SocketAddress Daniel P. Berrange
                   ` (13 subsequent siblings)
  16 siblings, 0 replies; 32+ messages in thread
From: Daniel P. Berrange @ 2015-09-18 13:18 UTC (permalink / raw)
  To: qemu-devel; +Cc: Paolo Bonzini, Dr. David Alan Gilbert, Gerd Hoffmann

If the port in the SocketAddress struct is NULL, it can allow
the kernel to automatically select a free port. This is useful
in particular in unit tests to avoid a race trying to find a
free port to run a test case on.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
 util/qemu-sockets.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/util/qemu-sockets.c b/util/qemu-sockets.c
index 9cc5bee..f9d2f6e 100644
--- a/util/qemu-sockets.c
+++ b/util/qemu-sockets.c
@@ -128,12 +128,15 @@ int inet_listen_opts(QemuOpts *opts, int port_offset, Error **errp)
     ai.ai_family = PF_UNSPEC;
     ai.ai_socktype = SOCK_STREAM;
 
-    if ((qemu_opt_get(opts, "host") == NULL) ||
-        (qemu_opt_get(opts, "port") == NULL)) {
-        error_setg(errp, "host and/or port not specified");
+    if ((qemu_opt_get(opts, "host") == NULL)) {
+        error_setg(errp, "host not specified");
         return -1;
     }
-    pstrcpy(port, sizeof(port), qemu_opt_get(opts, "port"));
+    if (qemu_opt_get(opts, "port") != NULL) {
+        pstrcpy(port, sizeof(port), qemu_opt_get(opts, "port"));
+    } else {
+        port[0] = '\0';
+    }
     addr = qemu_opt_get(opts, "host");
 
     to = qemu_opt_get_number(opts, "to", 0);
@@ -145,6 +148,10 @@ int inet_listen_opts(QemuOpts *opts, int port_offset, Error **errp)
     /* lookup */
     if (port_offset) {
         unsigned long long baseport;
+        if (strlen(port) == 0) {
+            error_setg(errp, "port not specified");
+            return -1;
+        }
         if (parse_uint_full(port, &baseport, 10) < 0) {
             error_setg(errp, "can't convert to a number: %s", port);
             return -1;
@@ -156,7 +163,8 @@ int inet_listen_opts(QemuOpts *opts, int port_offset, Error **errp)
         }
         snprintf(port, sizeof(port), "%d", (int)baseport + port_offset);
     }
-    rc = getaddrinfo(strlen(addr) ? addr : NULL, port, &ai, &res);
+    rc = getaddrinfo(strlen(addr) ? addr : NULL,
+                     strlen(port) ? port : NULL, &ai, &res);
     if (rc != 0) {
         error_setg(errp, "address resolution failed for %s:%s: %s", addr, port,
                    gai_strerror(rc));
-- 
2.4.3

^ permalink raw reply related	[flat|nested] 32+ messages in thread

* [Qemu-devel] [PATCH v1 04/16] ui: convert VNC startup code to use SocketAddress
  2015-09-18 13:18 [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework Daniel P. Berrange
                   ` (2 preceding siblings ...)
  2015-09-18 13:18 ` [Qemu-devel] [PATCH v1 03/16] sockets: allow port to be NULL when listening on IP address Daniel P. Berrange
@ 2015-09-18 13:18 ` Daniel P. Berrange
  2015-09-18 13:18 ` [Qemu-devel] [PATCH v1 05/16] osdep: add qemu_fork() wrapper for safely handling signals Daniel P. Berrange
                   ` (12 subsequent siblings)
  16 siblings, 0 replies; 32+ messages in thread
From: Daniel P. Berrange @ 2015-09-18 13:18 UTC (permalink / raw)
  To: qemu-devel; +Cc: Paolo Bonzini, Dr. David Alan Gilbert, Gerd Hoffmann

The VNC code is currently using QemuOpts to configure the
sockets connections / listeners it needs. Convert it to
use SocketAddress to bring it in line with modern QAPI
based code elsewhere in QEMU.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
 ui/vnc.c | 161 ++++++++++++++++++++++++++++++++++++---------------------------
 1 file changed, 91 insertions(+), 70 deletions(-)

diff --git a/ui/vnc.c b/ui/vnc.c
index d73966a..952e551 100644
--- a/ui/vnc.c
+++ b/ui/vnc.c
@@ -3506,18 +3506,14 @@ void vnc_display_open(const char *id, Error **errp)
 {
     VncDisplay *vs = vnc_display_find(id);
     QemuOpts *opts = qemu_opts_find(&qemu_vnc_opts, id);
-    QemuOpts *sopts, *wsopts;
+    SocketAddress *saddr = NULL, *wsaddr = NULL;
     const char *share, *device_id;
     QemuConsole *con;
     bool password = false;
     bool reverse = false;
     const char *vnc;
-    const char *has_to;
     char *h;
-    bool has_ipv4 = false;
-    bool has_ipv6 = false;
     const char *credid;
-    const char *websocket;
     bool sasl = false;
 #ifdef CONFIG_VNC_SASL
     int saslErr;
@@ -3539,44 +3535,83 @@ void vnc_display_open(const char *id, Error **errp)
         return;
     }
 
-    sopts = qemu_opts_create(&socket_optslist, NULL, 0, &error_abort);
-    wsopts = qemu_opts_create(&socket_optslist, NULL, 0, &error_abort);
-
     h = strrchr(vnc, ':');
     if (h) {
-        char *host;
         size_t hlen = h - vnc;
 
-        if (vnc[0] == '[' && vnc[hlen - 1] == ']') {
-            host = g_strndup(vnc + 1, hlen - 2);
+        const char *websocket = qemu_opt_get(opts, "websocket");
+        int to = qemu_opt_get_number(opts, "to", 0);
+        bool has_ipv4 = qemu_opt_get_bool(opts, "ipv4", false);
+        bool has_ipv6 = qemu_opt_get_bool(opts, "ipv6", false);
+
+        saddr = g_new0(SocketAddress, 1);
+        if (websocket) {
+            if (!qcrypto_hash_supports(QCRYPTO_HASH_ALG_SHA1)) {
+                error_setg(errp,
+                           "SHA1 hash support is required for websockets");
+                goto fail;
+            }
+
+            wsaddr = g_new0(SocketAddress, 1);
+            vs->ws_enabled = true;
+        }
+
+        if (strncmp(vnc, "unix:", 5) == 0) {
+            saddr->kind = SOCKET_ADDRESS_KIND_UNIX;
+            saddr->q_unix = g_new0(UnixSocketAddress, 1);
+            saddr->q_unix->path = g_strdup(vnc + 5);
+
+            if (vs->ws_enabled) {
+                error_setg(errp, "UNIX sockets not supported with websock");
+                goto fail;
+            }
         } else {
-            host = g_strndup(vnc, hlen);
+            unsigned long long baseport;
+            saddr->kind = SOCKET_ADDRESS_KIND_INET;
+            saddr->inet = g_new0(InetSocketAddress, 1);
+            if (vnc[0] == '[' && vnc[hlen - 1] == ']') {
+                saddr->inet->host = g_strndup(vnc + 1, hlen - 2);
+            } else {
+                saddr->inet->host = g_strndup(vnc, hlen);
+            }
+            if (parse_uint_full(h + 1, &baseport, 10) < 0) {
+                error_setg(errp, "can't convert to a number: %s", h + 1);
+                goto fail;
+            }
+            if (baseport > 65535 ||
+                baseport + 5900 > 65535) {
+                error_setg(errp, "port %s out of range", h + 1);
+                goto fail;
+            }
+            saddr->inet->port = g_strdup_printf(
+                "%d", (int)baseport + 5900);
+
+            if (to) {
+                saddr->inet->has_to = true;
+                saddr->inet->to = to;
+            }
+            saddr->inet->ipv4 = saddr->inet->has_ipv4 = has_ipv4;
+            saddr->inet->ipv6 = saddr->inet->has_ipv6 = has_ipv6;
+
+            if (vs->ws_enabled) {
+                wsaddr->kind = SOCKET_ADDRESS_KIND_INET;
+                wsaddr->inet = g_new0(InetSocketAddress, 1);
+                wsaddr->inet->host = g_strdup(saddr->inet->host);
+                wsaddr->inet->port = g_strdup(websocket);
+
+                if (to) {
+                    wsaddr->inet->has_to = true;
+                    wsaddr->inet->to = to;
+                }
+                wsaddr->inet->ipv4 = wsaddr->inet->has_ipv4 = has_ipv4;
+                wsaddr->inet->ipv6 = wsaddr->inet->has_ipv6 = has_ipv6;
+            }
         }
-        qemu_opt_set(sopts, "host", host, &error_abort);
-        qemu_opt_set(wsopts, "host", host, &error_abort);
-        qemu_opt_set(sopts, "port", h+1, &error_abort);
-        g_free(host);
     } else {
         error_setg(errp, "no vnc port specified");
         goto fail;
     }
 
-    has_to = qemu_opt_get(opts, "to");
-    has_ipv4 = qemu_opt_get_bool(opts, "ipv4", false);
-    has_ipv6 = qemu_opt_get_bool(opts, "ipv6", false);
-    if (has_to) {
-        qemu_opt_set(sopts, "to", has_to, &error_abort);
-        qemu_opt_set(wsopts, "to", has_to, &error_abort);
-    }
-    if (has_ipv4) {
-        qemu_opt_set(sopts, "ipv4", "on", &error_abort);
-        qemu_opt_set(wsopts, "ipv4", "on", &error_abort);
-    }
-    if (has_ipv6) {
-        qemu_opt_set(sopts, "ipv6", "on", &error_abort);
-        qemu_opt_set(wsopts, "ipv6", "on", &error_abort);
-    }
-
     password = qemu_opt_get_bool(opts, "password", false);
     if (password) {
         if (fips_get_state()) {
@@ -3682,16 +3717,6 @@ void vnc_display_open(const char *id, Error **errp)
     }
     vs->connections_limit = qemu_opt_get_number(opts, "connections", 32);
 
-    websocket = qemu_opt_get(opts, "websocket");
-    if (websocket) {
-        vs->ws_enabled = true;
-        qemu_opt_set(wsopts, "port", websocket, &error_abort);
-        if (!qcrypto_hash_supports(QCRYPTO_HASH_ALG_SHA1)) {
-            error_setg(errp, "SHA1 hash support is required for websockets");
-            goto fail;
-        }
-    }
-
 #ifdef CONFIG_VNC_JPEG
     vs->lossy = qemu_opt_get_bool(opts, "lossy", false);
 #endif
@@ -3725,7 +3750,7 @@ void vnc_display_open(const char *id, Error **errp)
     }
 #endif
 
-    if (vnc_display_setup_auth(vs, password, sasl, websocket, errp) < 0) {
+    if (vnc_display_setup_auth(vs, password, sasl, vs->ws_enabled, errp) < 0) {
         goto fail;
     }
 
@@ -3770,37 +3795,32 @@ void vnc_display_open(const char *id, Error **errp)
         int csock;
         vs->lsock = -1;
         vs->lwebsock = -1;
-        if (strncmp(vnc, "unix:", 5) == 0) {
-            csock = unix_connect(vnc+5, errp);
-        } else {
-            csock = inet_connect(vnc, errp);
+        if (vs->ws_enabled) {
+            error_setg(errp, "Cannot use websockets in reverse mode");
+            goto fail;
         }
+        csock = socket_connect(saddr, errp,
+                               NULL, NULL);
         if (csock < 0) {
             goto fail;
         }
+        vs->is_unix = saddr->kind == SOCKET_ADDRESS_KIND_UNIX;
         vnc_connect(vs, csock, false, false);
     } else {
         /* listen for connects */
-        if (strncmp(vnc, "unix:", 5) == 0) {
-            vs->lsock = unix_listen(vnc+5, NULL, 0, errp);
-            if (vs->lsock < 0) {
-                goto fail;
-            }
-            vs->is_unix = true;
-        } else {
-            vs->lsock = inet_listen_opts(sopts, 5900, errp);
-            if (vs->lsock < 0) {
-                goto fail;
-            }
-            if (vs->ws_enabled) {
-                vs->lwebsock = inet_listen_opts(wsopts, 0, errp);
-                if (vs->lwebsock < 0) {
-                    if (vs->lsock != -1) {
-                        close(vs->lsock);
-                        vs->lsock = -1;
-                    }
-                    goto fail;
+        vs->lsock = socket_listen(saddr, errp);
+        if (vs->lsock < 0) {
+            goto fail;
+        }
+        vs->is_unix = saddr->kind == SOCKET_ADDRESS_KIND_UNIX;
+        if (vs->ws_enabled) {
+            vs->lwebsock = socket_listen(wsaddr, errp);
+            if (vs->lwebsock < 0) {
+                if (vs->lsock != -1) {
+                    close(vs->lsock);
+                    vs->lsock = -1;
                 }
+                goto fail;
             }
         }
         vs->enabled = true;
@@ -3810,13 +3830,14 @@ void vnc_display_open(const char *id, Error **errp)
                                 NULL, vs);
         }
     }
-    qemu_opts_del(sopts);
-    qemu_opts_del(wsopts);
+
+    qapi_free_SocketAddress(saddr);
+    qapi_free_SocketAddress(wsaddr);
     return;
 
 fail:
-    qemu_opts_del(sopts);
-    qemu_opts_del(wsopts);
+    qapi_free_SocketAddress(saddr);
+    qapi_free_SocketAddress(wsaddr);
     vs->enabled = false;
     vs->ws_enabled = false;
 }
-- 
2.4.3

^ permalink raw reply related	[flat|nested] 32+ messages in thread

* [Qemu-devel] [PATCH v1 05/16] osdep: add qemu_fork() wrapper for safely handling signals
  2015-09-18 13:18 [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework Daniel P. Berrange
                   ` (3 preceding siblings ...)
  2015-09-18 13:18 ` [Qemu-devel] [PATCH v1 04/16] ui: convert VNC startup code to use SocketAddress Daniel P. Berrange
@ 2015-09-18 13:18 ` Daniel P. Berrange
  2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 06/16] coroutine: move into libqemuutil.a library Daniel P. Berrange
                   ` (11 subsequent siblings)
  16 siblings, 0 replies; 32+ messages in thread
From: Daniel P. Berrange @ 2015-09-18 13:18 UTC (permalink / raw)
  To: qemu-devel; +Cc: Paolo Bonzini, Dr. David Alan Gilbert, Gerd Hoffmann

When using regular fork() the child process of course inherits
all the parents' signal handlers. If the child then proceeds
to close() any open file descriptors, it may break some of those
registered signal handlers. The child generally does not want to
ever run any of the signal handlers tha parent may have installed
in the short time before it exec's. The parent may also have blocked
various signals which the child process will want enabled.

This introduces a wrapper qemu_fork() that takes care to sanitize
signal handling across fork. Before forking it blocks all signals
in the parent thread. After fork returns, the parent unblocks the
signals and carries on as usual. The child, however, resets all the
signal handlers back to their defaults before it unblocks signals.
The child process can now exec the binary in a "clean" signal
environment.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
 include/qemu/osdep.h | 16 ++++++++++++
 util/oslib-posix.c   | 71 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 util/oslib-win32.c   |  9 +++++++
 3 files changed, 96 insertions(+)

diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
index ab3c876..4f6af8d 100644
--- a/include/qemu/osdep.h
+++ b/include/qemu/osdep.h
@@ -67,6 +67,8 @@
 #include "sysemu/os-posix.h"
 #endif
 
+#include "qapi/error.h"
+
 #if defined(CONFIG_SOLARIS) && CONFIG_SOLARIS_VERSION < 10
 /* [u]int_fast*_t not in <sys/int_types.h> */
 typedef unsigned char           uint_fast8_t;
@@ -284,4 +286,18 @@ void os_mem_prealloc(int fd, char *area, size_t sz);
 
 int qemu_read_password(char *buf, int buf_size);
 
+/**
+ * qemu_fork:
+ *
+ * A version of fork that avoids signal handler race
+ * conditions that can lead to child process getting
+ * signals that are otherwise only expected by the
+ * parent. It also resets all signal handlers to the
+ * default settings.
+ *
+ * Returns 0 to child process, pid number to parent
+ * or -1 on failure.
+ */
+pid_t qemu_fork(Error **errp);
+
 #endif
diff --git a/util/oslib-posix.c b/util/oslib-posix.c
index 3ae4987..e7b8245 100644
--- a/util/oslib-posix.c
+++ b/util/oslib-posix.c
@@ -482,3 +482,74 @@ int qemu_read_password(char *buf, int buf_size)
     printf("\n");
     return ret;
 }
+
+
+pid_t qemu_fork(Error **errp)
+{
+    sigset_t oldmask, newmask;
+    struct sigaction sig_action;
+    int saved_errno;
+    pid_t pid;
+
+    /*
+     * Need to block signals now, so that child process can safely
+     * kill off caller's signal handlers without a race.
+     */
+    sigfillset(&newmask);
+    if (pthread_sigmask(SIG_SETMASK, &newmask, &oldmask) != 0) {
+        error_setg_errno(errp, errno,
+                         "cannot block signals");
+        return -1;
+    }
+
+    pid = fork();
+    saved_errno = errno;
+
+    if (pid < 0) {
+        /* attempt to restore signal mask, but ignore failure, to
+         * avoid obscuring the fork failure */
+        (void)pthread_sigmask(SIG_SETMASK, &oldmask, NULL);
+        error_setg_errno(errp, saved_errno,
+                         "cannot fork child process");
+        errno = saved_errno;
+        return -1;
+    } else if (pid) {
+        /* parent process */
+
+        /* Restore our original signal mask now that the child is
+         * safely running. Only documented failures are EFAULT (not
+         * possible, since we are using just-grabbed mask) or EINVAL
+         * (not possible, since we are using correct arguments).  */
+        (void)pthread_sigmask(SIG_SETMASK, &oldmask, NULL);
+    } else {
+        /* child process */
+        size_t i;
+
+        /* Clear out all signal handlers from parent so nothing
+         * unexpected can happen in our child once we unblock
+         * signals */
+        sig_action.sa_handler = SIG_DFL;
+        sig_action.sa_flags = 0;
+        sigemptyset(&sig_action.sa_mask);
+
+        for (i = 1; i < NSIG; i++) {
+            /* Only possible errors are EFAULT or EINVAL The former
+             * won't happen, the latter we expect, so no need to check
+             * return value */
+            (void)sigaction(i, &sig_action, NULL);
+        }
+
+        /* Unmask all signals in child, since we've no idea what the
+         * caller's done with their signal mask and don't want to
+         * propagate that to children */
+        sigemptyset(&newmask);
+        if (pthread_sigmask(SIG_SETMASK, &newmask, NULL) != 0) {
+            Error *local_err = NULL;
+            error_setg_errno(&local_err, errno,
+                             "cannot unblock signals");
+            error_report_err(local_err);
+            _exit(1);
+        }
+    }
+    return pid;
+}
diff --git a/util/oslib-win32.c b/util/oslib-win32.c
index 730a670..73dc35f 100644
--- a/util/oslib-win32.c
+++ b/util/oslib-win32.c
@@ -494,3 +494,12 @@ int qemu_read_password(char *buf, int buf_size)
     buf[i] = '\0';
     return 0;
 }
+
+
+pid_t qemu_fork(Error **errp)
+{
+    errno = ENOSYS;
+    error_setg_errno(errp, errno,
+                     "cannot fork child process");
+    return -1;
+}
-- 
2.4.3

^ permalink raw reply related	[flat|nested] 32+ messages in thread

* [Qemu-devel] [PATCH v1 06/16] coroutine: move into libqemuutil.a library
  2015-09-18 13:18 [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework Daniel P. Berrange
                   ` (4 preceding siblings ...)
  2015-09-18 13:18 ` [Qemu-devel] [PATCH v1 05/16] osdep: add qemu_fork() wrapper for safely handling signals Daniel P. Berrange
@ 2015-09-18 13:19 ` Daniel P. Berrange
  2015-09-22 12:07   ` Paolo Bonzini
  2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 07/16] io: add abstract QIOChannel classes Daniel P. Berrange
                   ` (10 subsequent siblings)
  16 siblings, 1 reply; 32+ messages in thread
From: Daniel P. Berrange @ 2015-09-18 13:19 UTC (permalink / raw)
  To: qemu-devel; +Cc: Paolo Bonzini, Dr. David Alan Gilbert, Gerd Hoffmann

The coroutine files are currently referenced by the block-obj-y
variable. The coroutine functionality though is already used by
more than just the block code. eg migration code uses coroutine
yield. In the future the I/O channel code will also use the
coroutine yield functionality. Since the coroutine code is nicely
self-contained it can be easily built as part of the libqemuutil.a
library, making it widely available.

The headers are also moved into include/qemu, instead of the
include/block directory, since they are now part of the util
codebase, and the impl was never in the block/ directory
either.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
 Makefile.objs                                           | 4 ----
 block.c                                                 | 2 +-
 block/qcow2.h                                           | 2 +-
 block/vdi.c                                             | 2 +-
 block/write-threshold.c                                 | 2 +-
 blockjob.c                                              | 2 +-
 hw/9pfs/codir.c                                         | 2 +-
 hw/9pfs/cofile.c                                        | 2 +-
 hw/9pfs/cofs.c                                          | 2 +-
 hw/9pfs/coxattr.c                                       | 2 +-
 hw/9pfs/virtio-9p-coth.c                                | 2 +-
 hw/9pfs/virtio-9p-coth.h                                | 2 +-
 hw/9pfs/virtio-9p.h                                     | 2 +-
 include/block/block.h                                   | 2 +-
 include/block/block_int.h                               | 2 +-
 include/{block => qemu}/coroutine.h                     | 0
 include/{block => qemu}/coroutine_int.h                 | 2 +-
 migration/qemu-file-buf.c                               | 2 +-
 migration/qemu-file-stdio.c                             | 2 +-
 migration/qemu-file-unix.c                              | 2 +-
 migration/qemu-file.c                                   | 2 +-
 migration/rdma.c                                        | 2 +-
 nbd.c                                                   | 2 +-
 tests/test-coroutine.c                                  | 4 ++--
 tests/test-vmstate.c                                    | 2 +-
 thread-pool.c                                           | 2 +-
 util/Makefile.objs                                      | 3 +++
 coroutine-gthread.c => util/coroutine-gthread.c         | 2 +-
 coroutine-sigaltstack.c => util/coroutine-sigaltstack.c | 2 +-
 coroutine-ucontext.c => util/coroutine-ucontext.c       | 2 +-
 coroutine-win32.c => util/coroutine-win32.c             | 2 +-
 qemu-coroutine-io.c => util/qemu-coroutine-io.c         | 2 +-
 qemu-coroutine-lock.c => util/qemu-coroutine-lock.c     | 4 ++--
 qemu-coroutine-sleep.c => util/qemu-coroutine-sleep.c   | 2 +-
 qemu-coroutine.c => util/qemu-coroutine.c               | 4 ++--
 35 files changed, 38 insertions(+), 39 deletions(-)
 rename include/{block => qemu}/coroutine.h (100%)
 rename include/{block => qemu}/coroutine_int.h (98%)
 rename coroutine-gthread.c => util/coroutine-gthread.c (99%)
 rename coroutine-sigaltstack.c => util/coroutine-sigaltstack.c (99%)
 rename coroutine-ucontext.c => util/coroutine-ucontext.c (99%)
 rename coroutine-win32.c => util/coroutine-win32.c (98%)
 rename qemu-coroutine-io.c => util/qemu-coroutine-io.c (99%)
 rename qemu-coroutine-lock.c => util/qemu-coroutine-lock.c (98%)
 rename qemu-coroutine-sleep.c => util/qemu-coroutine-sleep.c (96%)
 rename qemu-coroutine.c => util/qemu-coroutine.c (98%)

diff --git a/Makefile.objs b/Makefile.objs
index 3df2efc..2e94eaf 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -14,10 +14,6 @@ block-obj-$(CONFIG_WIN32) += aio-win32.o
 block-obj-y += block/
 block-obj-y += qemu-io-cmds.o
 
-block-obj-y += qemu-coroutine.o qemu-coroutine-lock.o qemu-coroutine-io.o
-block-obj-y += qemu-coroutine-sleep.o
-block-obj-y += coroutine-$(CONFIG_COROUTINE_BACKEND).o
-
 block-obj-m = block/
 
 #######################################################################
diff --git a/block.c b/block.c
index 6268e37..cfc9a13 100644
--- a/block.c
+++ b/block.c
@@ -33,7 +33,7 @@
 #include "sysemu/block-backend.h"
 #include "sysemu/sysemu.h"
 #include "qemu/notify.h"
-#include "block/coroutine.h"
+#include "qemu/coroutine.h"
 #include "block/qapi.h"
 #include "qmp-commands.h"
 #include "qemu/timer.h"
diff --git a/block/qcow2.h b/block/qcow2.h
index d700bf1..27b919c 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -26,7 +26,7 @@
 #define BLOCK_QCOW2_H
 
 #include "crypto/cipher.h"
-#include "block/coroutine.h"
+#include "qemu/coroutine.h"
 
 //#define DEBUG_ALLOC
 //#define DEBUG_ALLOC2
diff --git a/block/vdi.c b/block/vdi.c
index 062a654..d662f41 100644
--- a/block/vdi.c
+++ b/block/vdi.c
@@ -53,7 +53,7 @@
 #include "block/block_int.h"
 #include "qemu/module.h"
 #include "migration/migration.h"
-#include "block/coroutine.h"
+#include "qemu/coroutine.h"
 
 #if defined(CONFIG_UUID)
 #include <uuid/uuid.h>
diff --git a/block/write-threshold.c b/block/write-threshold.c
index a53c1f5..0fe3891 100644
--- a/block/write-threshold.c
+++ b/block/write-threshold.c
@@ -11,7 +11,7 @@
  */
 
 #include "block/block_int.h"
-#include "block/coroutine.h"
+#include "qemu/coroutine.h"
 #include "block/write-threshold.h"
 #include "qemu/notify.h"
 #include "qapi-event.h"
diff --git a/blockjob.c b/blockjob.c
index 62bb906..dc42d15 100644
--- a/blockjob.c
+++ b/blockjob.c
@@ -31,7 +31,7 @@
 #include "block/block_int.h"
 #include "qapi/qmp/qerror.h"
 #include "qapi/qmp/qjson.h"
-#include "block/coroutine.h"
+#include "qemu/coroutine.h"
 #include "qmp-commands.h"
 #include "qemu/timer.h"
 #include "qapi-event.h"
diff --git a/hw/9pfs/codir.c b/hw/9pfs/codir.c
index 65ad329..ec9cc7f 100644
--- a/hw/9pfs/codir.c
+++ b/hw/9pfs/codir.c
@@ -14,7 +14,7 @@
 
 #include "fsdev/qemu-fsdev.h"
 #include "qemu/thread.h"
-#include "block/coroutine.h"
+#include "qemu/coroutine.h"
 #include "virtio-9p-coth.h"
 
 int v9fs_co_readdir_r(V9fsPDU *pdu, V9fsFidState *fidp, struct dirent *dent,
diff --git a/hw/9pfs/cofile.c b/hw/9pfs/cofile.c
index 2efebf3..7cb55ee 100644
--- a/hw/9pfs/cofile.c
+++ b/hw/9pfs/cofile.c
@@ -14,7 +14,7 @@
 
 #include "fsdev/qemu-fsdev.h"
 #include "qemu/thread.h"
-#include "block/coroutine.h"
+#include "qemu/coroutine.h"
 #include "virtio-9p-coth.h"
 
 int v9fs_co_st_gen(V9fsPDU *pdu, V9fsPath *path, mode_t st_mode,
diff --git a/hw/9pfs/cofs.c b/hw/9pfs/cofs.c
index 42ee614..e1953a9 100644
--- a/hw/9pfs/cofs.c
+++ b/hw/9pfs/cofs.c
@@ -14,7 +14,7 @@
 
 #include "fsdev/qemu-fsdev.h"
 #include "qemu/thread.h"
-#include "block/coroutine.h"
+#include "qemu/coroutine.h"
 #include "virtio-9p-coth.h"
 
 static ssize_t __readlink(V9fsState *s, V9fsPath *path, V9fsString *buf)
diff --git a/hw/9pfs/coxattr.c b/hw/9pfs/coxattr.c
index 18ee08d..55c0d23 100644
--- a/hw/9pfs/coxattr.c
+++ b/hw/9pfs/coxattr.c
@@ -14,7 +14,7 @@
 
 #include "fsdev/qemu-fsdev.h"
 #include "qemu/thread.h"
-#include "block/coroutine.h"
+#include "qemu/coroutine.h"
 #include "virtio-9p-coth.h"
 
 int v9fs_co_llistxattr(V9fsPDU *pdu, V9fsPath *path, void *value, size_t size)
diff --git a/hw/9pfs/virtio-9p-coth.c b/hw/9pfs/virtio-9p-coth.c
index 8185c53..5057f8d 100644
--- a/hw/9pfs/virtio-9p-coth.c
+++ b/hw/9pfs/virtio-9p-coth.c
@@ -15,7 +15,7 @@
 #include "fsdev/qemu-fsdev.h"
 #include "qemu/thread.h"
 #include "qemu/event_notifier.h"
-#include "block/coroutine.h"
+#include "qemu/coroutine.h"
 #include "virtio-9p-coth.h"
 
 /* v9fs glib thread pool */
diff --git a/hw/9pfs/virtio-9p-coth.h b/hw/9pfs/virtio-9p-coth.h
index 4f51b25..0fbe49a 100644
--- a/hw/9pfs/virtio-9p-coth.h
+++ b/hw/9pfs/virtio-9p-coth.h
@@ -16,7 +16,7 @@
 #define _QEMU_VIRTIO_9P_COTH_H
 
 #include "qemu/thread.h"
-#include "block/coroutine.h"
+#include "qemu/coroutine.h"
 #include "virtio-9p.h"
 #include <glib.h>
 
diff --git a/hw/9pfs/virtio-9p.h b/hw/9pfs/virtio-9p.h
index 2e7d488..d7a4dc1 100644
--- a/hw/9pfs/virtio-9p.h
+++ b/hw/9pfs/virtio-9p.h
@@ -13,7 +13,7 @@
 #include "fsdev/file-op-9p.h"
 #include "fsdev/virtio-9p-marshal.h"
 #include "qemu/thread.h"
-#include "block/coroutine.h"
+#include "qemu/coroutine.h"
 
 enum {
     P9_TLERROR = 6,
diff --git a/include/block/block.h b/include/block/block.h
index ef67353..4d77291 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -4,7 +4,7 @@
 #include "block/aio.h"
 #include "qemu-common.h"
 #include "qemu/option.h"
-#include "block/coroutine.h"
+#include "qemu/coroutine.h"
 #include "block/accounting.h"
 #include "qapi/qmp/qobject.h"
 #include "qapi-types.h"
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 14ad4c3..c37ed77 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -28,7 +28,7 @@
 #include "block/block.h"
 #include "qemu/option.h"
 #include "qemu/queue.h"
-#include "block/coroutine.h"
+#include "qemu/coroutine.h"
 #include "qemu/timer.h"
 #include "qapi-types.h"
 #include "qemu/hbitmap.h"
diff --git a/include/block/coroutine.h b/include/qemu/coroutine.h
similarity index 100%
rename from include/block/coroutine.h
rename to include/qemu/coroutine.h
diff --git a/include/block/coroutine_int.h b/include/qemu/coroutine_int.h
similarity index 98%
rename from include/block/coroutine_int.h
rename to include/qemu/coroutine_int.h
index 9aa1aae..42d6838 100644
--- a/include/block/coroutine_int.h
+++ b/include/qemu/coroutine_int.h
@@ -26,7 +26,7 @@
 #define QEMU_COROUTINE_INT_H
 
 #include "qemu/queue.h"
-#include "block/coroutine.h"
+#include "qemu/coroutine.h"
 
 typedef enum {
     COROUTINE_YIELD = 1,
diff --git a/migration/qemu-file-buf.c b/migration/qemu-file-buf.c
index 2de9330..556f5dc 100644
--- a/migration/qemu-file-buf.c
+++ b/migration/qemu-file-buf.c
@@ -29,7 +29,7 @@
 #include "qemu/error-report.h"
 #include "qemu/iov.h"
 #include "qemu/sockets.h"
-#include "block/coroutine.h"
+#include "qemu/coroutine.h"
 #include "migration/migration.h"
 #include "migration/qemu-file.h"
 #include "migration/qemu-file-internal.h"
diff --git a/migration/qemu-file-stdio.c b/migration/qemu-file-stdio.c
index 285068b..002dc5d 100644
--- a/migration/qemu-file-stdio.c
+++ b/migration/qemu-file-stdio.c
@@ -22,7 +22,7 @@
  * THE SOFTWARE.
  */
 #include "qemu-common.h"
-#include "block/coroutine.h"
+#include "qemu/coroutine.h"
 #include "migration/qemu-file.h"
 
 typedef struct QEMUFileStdio {
diff --git a/migration/qemu-file-unix.c b/migration/qemu-file-unix.c
index bfbc086..e4f195a 100644
--- a/migration/qemu-file-unix.c
+++ b/migration/qemu-file-unix.c
@@ -24,7 +24,7 @@
 #include "qemu-common.h"
 #include "qemu/iov.h"
 #include "qemu/sockets.h"
-#include "block/coroutine.h"
+#include "qemu/coroutine.h"
 #include "migration/qemu-file.h"
 #include "migration/qemu-file-internal.h"
 
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index 6bb3dc1..d2359c4 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -26,7 +26,7 @@
 #include "qemu/error-report.h"
 #include "qemu/iov.h"
 #include "qemu/sockets.h"
-#include "block/coroutine.h"
+#include "qemu/coroutine.h"
 #include "migration/migration.h"
 #include "migration/qemu-file.h"
 #include "migration/qemu-file-internal.h"
diff --git a/migration/rdma.c b/migration/rdma.c
index 9424834..854fde4 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -19,7 +19,7 @@
 #include "qemu/main-loop.h"
 #include "qemu/sockets.h"
 #include "qemu/bitmap.h"
-#include "block/coroutine.h"
+#include "qemu/coroutine.h"
 #include <stdio.h>
 #include <sys/types.h>
 #include <sys/socket.h>
diff --git a/nbd.c b/nbd.c
index 07240bd..1724a1f 100644
--- a/nbd.c
+++ b/nbd.c
@@ -19,7 +19,7 @@
 #include "block/nbd.h"
 #include "sysemu/block-backend.h"
 
-#include "block/coroutine.h"
+#include "qemu/coroutine.h"
 
 #include <errno.h>
 #include <string.h>
diff --git a/tests/test-coroutine.c b/tests/test-coroutine.c
index b552d9f..f5951cb 100644
--- a/tests/test-coroutine.c
+++ b/tests/test-coroutine.c
@@ -12,8 +12,8 @@
  */
 
 #include <glib.h>
-#include "block/coroutine.h"
-#include "block/coroutine_int.h"
+#include "qemu/coroutine.h"
+#include "qemu/coroutine_int.h"
 
 /*
  * Check that qemu_in_coroutine() works
diff --git a/tests/test-vmstate.c b/tests/test-vmstate.c
index 1d620e0..4d13bd0 100644
--- a/tests/test-vmstate.c
+++ b/tests/test-vmstate.c
@@ -27,7 +27,7 @@
 #include "qemu-common.h"
 #include "migration/migration.h"
 #include "migration/vmstate.h"
-#include "block/coroutine.h"
+#include "qemu/coroutine.h"
 
 static char temp_file[] = "/tmp/vmst.test.XXXXXX";
 static int temp_fd;
diff --git a/thread-pool.c b/thread-pool.c
index ac909f4..402c778 100644
--- a/thread-pool.c
+++ b/thread-pool.c
@@ -18,7 +18,7 @@
 #include "qemu/queue.h"
 #include "qemu/thread.h"
 #include "qemu/osdep.h"
-#include "block/coroutine.h"
+#include "qemu/coroutine.h"
 #include "trace.h"
 #include "block/thread-pool.h"
 #include "qemu/main-loop.h"
diff --git a/util/Makefile.objs b/util/Makefile.objs
index 114d657..d8d7e7a 100644
--- a/util/Makefile.objs
+++ b/util/Makefile.objs
@@ -18,3 +18,6 @@ util-obj-y += getauxval.o
 util-obj-y += readline.o
 util-obj-y += rfifolock.o
 util-obj-y += rcu.o
+util-obj-y += qemu-coroutine.o qemu-coroutine-lock.o qemu-coroutine-io.o
+util-obj-y += qemu-coroutine-sleep.o
+util-obj-y += coroutine-$(CONFIG_COROUTINE_BACKEND).o
diff --git a/coroutine-gthread.c b/util/coroutine-gthread.c
similarity index 99%
rename from coroutine-gthread.c
rename to util/coroutine-gthread.c
index 6bd6d6b..0bcd778 100644
--- a/coroutine-gthread.c
+++ b/util/coroutine-gthread.c
@@ -20,7 +20,7 @@
 
 #include <glib.h>
 #include "qemu-common.h"
-#include "block/coroutine_int.h"
+#include "qemu/coroutine_int.h"
 
 typedef struct {
     Coroutine base;
diff --git a/coroutine-sigaltstack.c b/util/coroutine-sigaltstack.c
similarity index 99%
rename from coroutine-sigaltstack.c
rename to util/coroutine-sigaltstack.c
index 63519ff..39842a4 100644
--- a/coroutine-sigaltstack.c
+++ b/util/coroutine-sigaltstack.c
@@ -31,7 +31,7 @@
 #include <pthread.h>
 #include <signal.h>
 #include "qemu-common.h"
-#include "block/coroutine_int.h"
+#include "qemu/coroutine_int.h"
 
 typedef struct {
     Coroutine base;
diff --git a/coroutine-ucontext.c b/util/coroutine-ucontext.c
similarity index 99%
rename from coroutine-ucontext.c
rename to util/coroutine-ucontext.c
index 259fcb4..26cbebb 100644
--- a/coroutine-ucontext.c
+++ b/util/coroutine-ucontext.c
@@ -27,7 +27,7 @@
 #include <stdint.h>
 #include <ucontext.h>
 #include "qemu-common.h"
-#include "block/coroutine_int.h"
+#include "qemu/coroutine_int.h"
 
 #ifdef CONFIG_VALGRIND_H
 #include <valgrind/valgrind.h>
diff --git a/coroutine-win32.c b/util/coroutine-win32.c
similarity index 98%
rename from coroutine-win32.c
rename to util/coroutine-win32.c
index 17ace37..4f922c5 100644
--- a/coroutine-win32.c
+++ b/util/coroutine-win32.c
@@ -23,7 +23,7 @@
  */
 
 #include "qemu-common.h"
-#include "block/coroutine_int.h"
+#include "qemu/coroutine_int.h"
 
 typedef struct
 {
diff --git a/qemu-coroutine-io.c b/util/qemu-coroutine-io.c
similarity index 99%
rename from qemu-coroutine-io.c
rename to util/qemu-coroutine-io.c
index 28dc735..e1eae73 100644
--- a/qemu-coroutine-io.c
+++ b/util/qemu-coroutine-io.c
@@ -24,7 +24,7 @@
  */
 #include "qemu-common.h"
 #include "qemu/sockets.h"
-#include "block/coroutine.h"
+#include "qemu/coroutine.h"
 #include "qemu/iov.h"
 #include "qemu/main-loop.h"
 
diff --git a/qemu-coroutine-lock.c b/util/qemu-coroutine-lock.c
similarity index 98%
rename from qemu-coroutine-lock.c
rename to util/qemu-coroutine-lock.c
index 6b49033..130ee19 100644
--- a/qemu-coroutine-lock.c
+++ b/util/qemu-coroutine-lock.c
@@ -23,8 +23,8 @@
  */
 
 #include "qemu-common.h"
-#include "block/coroutine.h"
-#include "block/coroutine_int.h"
+#include "qemu/coroutine.h"
+#include "qemu/coroutine_int.h"
 #include "qemu/queue.h"
 #include "trace.h"
 
diff --git a/qemu-coroutine-sleep.c b/util/qemu-coroutine-sleep.c
similarity index 96%
rename from qemu-coroutine-sleep.c
rename to util/qemu-coroutine-sleep.c
index 9abb7fd..b35db56 100644
--- a/qemu-coroutine-sleep.c
+++ b/util/qemu-coroutine-sleep.c
@@ -11,7 +11,7 @@
  *
  */
 
-#include "block/coroutine.h"
+#include "qemu/coroutine.h"
 #include "qemu/timer.h"
 #include "block/aio.h"
 
diff --git a/qemu-coroutine.c b/util/qemu-coroutine.c
similarity index 98%
rename from qemu-coroutine.c
rename to util/qemu-coroutine.c
index c17a92b..8953560 100644
--- a/qemu-coroutine.c
+++ b/util/qemu-coroutine.c
@@ -16,8 +16,8 @@
 #include "qemu-common.h"
 #include "qemu/thread.h"
 #include "qemu/atomic.h"
-#include "block/coroutine.h"
-#include "block/coroutine_int.h"
+#include "qemu/coroutine.h"
+#include "qemu/coroutine_int.h"
 
 enum {
     POOL_BATCH_SIZE = 64,
-- 
2.4.3

^ permalink raw reply related	[flat|nested] 32+ messages in thread

* [Qemu-devel] [PATCH v1 07/16] io: add abstract QIOChannel classes
  2015-09-18 13:18 [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework Daniel P. Berrange
                   ` (5 preceding siblings ...)
  2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 06/16] coroutine: move into libqemuutil.a library Daniel P. Berrange
@ 2015-09-18 13:19 ` Daniel P. Berrange
  2015-09-22 12:19   ` Paolo Bonzini
  2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 08/16] io: add helper module for creating watches on FDs Daniel P. Berrange
                   ` (9 subsequent siblings)
  16 siblings, 1 reply; 32+ messages in thread
From: Daniel P. Berrange @ 2015-09-18 13:19 UTC (permalink / raw)
  To: qemu-devel; +Cc: Paolo Bonzini, Dr. David Alan Gilbert, Gerd Hoffmann

Start the new generic I/O channel framework by defining a
QIOChannel abstract base class. This is designed to feel
similar to GLib's GIOChannel, but with the addition of
support for using iovecs, qemu error reporting, file
descriptor passing, coroutine integration and use of
the QOM framework for easier sub-classing.

The intention is that anywhere in QEMU that almost
anywhere that deals with sockets will use this new I/O
infrastructure, so that it becomes trivial to then layer
in support for TLS encryption. This will at least include
the VNC server, char device backend and migration code.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
 MAINTAINERS          |   7 +
 Makefile             |   2 +
 Makefile.objs        |   5 +
 Makefile.target      |   2 +
 include/io/channel.h | 523 +++++++++++++++++++++++++++++++++++++++++++++++++++
 io/Makefile.objs     |   1 +
 io/channel.c         | 290 ++++++++++++++++++++++++++++
 7 files changed, 830 insertions(+)
 create mode 100644 include/io/channel.h
 create mode 100644 io/Makefile.objs
 create mode 100644 io/channel.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 2f4e8cf..348282a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1078,6 +1078,13 @@ F: crypto/
 F: include/crypto/
 F: tests/test-crypto-*
 
+I/O Channels
+M: Daniel P. Berrange <berrange@redhat.com>
+S: Maintained
+F: io/
+F: include/io/
+F: tests/test-io-*
+
 Usermode Emulation
 ------------------
 Overall
diff --git a/Makefile b/Makefile
index 291fb54..521cc38 100644
--- a/Makefile
+++ b/Makefile
@@ -155,6 +155,7 @@ dummy := $(call unnest-vars,, \
                 crypto-obj-y \
                 crypto-aes-obj-y \
                 qom-obj-y \
+                io-obj-y \
                 common-obj-y \
                 common-obj-m)
 
@@ -178,6 +179,7 @@ SOFTMMU_SUBDIR_RULES=$(filter %-softmmu,$(SUBDIR_RULES))
 $(SOFTMMU_SUBDIR_RULES): $(block-obj-y)
 $(SOFTMMU_SUBDIR_RULES): $(crypto-obj-y)
 $(SOFTMMU_SUBDIR_RULES): $(qom-obj-y)
+$(SOFTMMU_SUBDIR_RULES): $(io-obj-y)
 $(SOFTMMU_SUBDIR_RULES): config-all-devices.mak
 
 subdir-%:
diff --git a/Makefile.objs b/Makefile.objs
index 2e94eaf..9c7186f 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -27,6 +27,11 @@ crypto-aes-obj-y = crypto/
 
 qom-obj-y = qom/
 
+#######################################################################
+# io-obj-y is code used by both qemu system emulation and qemu-img
+
+io-obj-y = io/
+
 ######################################################################
 # smartcard
 
diff --git a/Makefile.target b/Makefile.target
index 962d004..34ddb7e 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -176,6 +176,7 @@ dummy := $(call unnest-vars,.., \
                crypto-obj-y \
                crypto-aes-obj-y \
                qom-obj-y \
+               io-obj-y \
                common-obj-y \
                common-obj-m)
 target-obj-y := $(target-obj-y-save)
@@ -185,6 +186,7 @@ all-obj-y += $(qom-obj-y)
 all-obj-$(CONFIG_SOFTMMU) += $(block-obj-y)
 all-obj-$(CONFIG_USER_ONLY) += $(crypto-aes-obj-y)
 all-obj-$(CONFIG_SOFTMMU) += $(crypto-obj-y)
+all-obj-$(CONFIG_SOFTMMU) += $(io-obj-y)
 
 $(QEMU_PROG_BUILD): config-devices.mak
 
diff --git a/include/io/channel.h b/include/io/channel.h
new file mode 100644
index 0000000..17a0c55
--- /dev/null
+++ b/include/io/channel.h
@@ -0,0 +1,523 @@
+/*
+ * QEMU I/O channels
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef QIO_CHANNEL_H__
+#define QIO_CHANNEL_H__
+
+#include "qemu-common.h"
+#include "qapi/error.h"
+#include "qom/object.h"
+
+#define TYPE_QIO_CHANNEL "qio-channel"
+#define QIO_CHANNEL(obj)                                    \
+    OBJECT_CHECK(QIOChannel, (obj), TYPE_QIO_CHANNEL)
+#define QIO_CHANNEL_CLASS(klass)                                    \
+    OBJECT_CLASS_CHECK(QIOChannelClass, klass, TYPE_QIO_CHANNEL)
+#define QIO_CHANNEL_GET_CLASS(obj)                                  \
+    OBJECT_GET_CLASS(QIOChannelClass, obj, TYPE_QIO_CHANNEL)
+
+typedef struct QIOChannel QIOChannel;
+typedef struct QIOChannelClass QIOChannelClass;
+
+#define QIO_CHANNEL_ERR_BLOCK -2
+
+typedef enum QIOChannelFeature QIOChannelFeature;
+
+enum QIOChannelFeature {
+    QIO_CHANNEL_FEATURE_FD_PASS  = (1 << 0),
+    QIO_CHANNEL_FEATURE_SHUTDOWN = (1 << 1),
+    QIO_CHANNEL_FEATURE_DELAY    = (1 << 2),
+    QIO_CHANNEL_FEATURE_CORK     = (1 << 3),
+};
+
+
+typedef enum QIOChannelShutdown QIOChannelShutdown;
+
+enum QIOChannelShutdown {
+    QIO_CHANNEL_SHUTDOWN_BOTH,
+    QIO_CHANNEL_SHUTDOWN_READ,
+    QIO_CHANNEL_SHUTDOWN_WRITE,
+};
+
+typedef gboolean (*QIOChannelFunc)(QIOChannel *ioc,
+                                   GIOCondition condition,
+                                   gpointer data);
+
+/**
+ * QIOChannel:
+ *
+ * The QIOChannel defines the core API for a generic I/O channel
+ * class hierarchy. It is inspired by GIOChannel, but has the
+ * following differences
+ *
+ *  - Use QOM to properly support arbitrary subclassing
+ *  - Support use of iovecs for efficient I/O with multiple blocks
+ *  - None of the character set translation, binary data exclusively
+ *  - Direct support for QEMU Error object reporting
+ *  - File descriptor passing
+ *  - Peeking at incoming data without reading it
+ *
+ * This base class is abstract so cannot be instantiated. There
+ * will be subclasses for dealing with sockets, files, and higher
+ * level protocols such as TLS, WebSocket, etc.
+ */
+
+struct QIOChannel {
+    Object parent;
+};
+
+/**
+ * QIOChannelClass:
+ *
+ * This class defines the contract that all subclasses
+ * must follow to provide specific channel implementations.
+ * All the callbacks are mandatory with the exception of
+ * io_has_feature, which defaults to returning false.
+ *
+ * Consult the corresponding public API docs for a description
+ * of the semantics of each callback
+ */
+struct QIOChannelClass {
+    ObjectClass parent;
+
+    /* Mandatory callbacks */
+    ssize_t (*io_writev)(QIOChannel *ioc,
+                         const struct iovec *iov,
+                         size_t niov,
+                         int *fds,
+                         size_t nfds,
+                         Error **errp);
+    ssize_t (*io_readv)(QIOChannel *ioc,
+                        const struct iovec *iov,
+                        size_t niov,
+                        int **fds,
+                        size_t *nfds,
+                        Error **errp);
+    int (*io_close)(QIOChannel *ioc,
+                    Error **errp);
+    GSource * (*io_create_watch)(QIOChannel *ioc,
+                                 GIOCondition condition);
+    int (*io_set_blocking)(QIOChannel *ioc,
+                           bool enabled,
+                           Error **errp);
+
+    /* Optional callbacks */
+    bool (*io_has_feature)(QIOChannel *ioc,
+                           QIOChannelFeature feature);
+    int (*io_shutdown)(QIOChannel *ioc,
+                       QIOChannelShutdown how,
+                       Error **errp);
+    int (*io_set_cork)(QIOChannel *ioc,
+                       bool enabled,
+                       Error **errp);
+    int (*io_set_delay)(QIOChannel *ioc,
+                        bool enabled,
+                        Error **errp);
+    off64_t (*io_seek)(QIOChannel *ioc,
+                       off64_t offset,
+                       int whence,
+                       Error **errp);
+};
+
+/* General I/O handling functions */
+
+/**
+ * qio_channel_has_feature:
+ * @ioc: the channel object
+ * @feature: the feature to check support of
+ *
+ * Determine whether the channel implementation supports
+ * the optional feature named in @feature.
+ *
+ * Returns: true if supported, false otherwise.
+ */
+bool qio_channel_has_feature(QIOChannel *ioc,
+                             QIOChannelFeature feature);
+
+/**
+ * qio_channel_readv_full:
+ * @ioc: the channel object
+ * @iov: the array of memory regions to read data into
+ * @niov: the length of the @iov array
+ * @fds: pointer to an array that will received file handles
+ * @nfds: pointer filled with number of elements in @fds on return
+ * @errp: pointer to an uninitialized error object
+ *
+ * Read data from the IO channel, storing it in the
+ * memory regions referenced by @iov. Each element
+ * in the @iov will be fully populated with data
+ * before the next one is used. The @niov parameter
+ * specifies the total number of elements in @iov.
+ *
+ * It is not required for all @iov to be filled with
+ * data. If the channel is in blocking mode, at least
+ * one byte of data will be read, but no more is
+ * guaranteed. If the channel is non-blocking and no
+ * data is available, it will return QIO_CHANNEL_ERR_BLOCK
+ *
+ * If the channel has passed any file descriptors,
+ * the @fds array pointer will be allocated and
+ * the elements filled with the received file
+ * descriptors. The @nfds pointer will be updated
+ * to indicate the size of the @fds array that
+ * was allocated. It is the callers responsibility
+ * to call close() on each file descriptor and to
+ * call g_free() on the array pointer in @fds.
+ *
+ * It is an error to pass a non-NULL @fds parameter
+ * unless qio_channel_has_feature() returns a true
+ * value for the QIO_CHANNEL_FEATURE_FD_PASS constant.
+ *
+ * Returns: the number of bytes read, or -1 on error,
+ * or QIO_CHANNEL_ERR_BLOCK if no data is available
+ * and the channel is non-blocking
+ */
+ssize_t qio_channel_readv_full(QIOChannel *ioc,
+                               const struct iovec *iov,
+                               size_t niov,
+                               int **fds,
+                               size_t *nfds,
+                               Error **errp);
+
+
+/**
+ * qio_channel_writev_full:
+ * @ioc: the channel object
+ * @iov: the array of memory regions to write data from
+ * @niov: the length of the @iov array
+ * @fds: an array of file handles to send
+ * @nfds: number of file handles in @fds
+ * @errp: pointer to an uninitialized error object
+ *
+ * Write data to the IO channel, reading it from the
+ * memory regions referenced by @iov. Each element
+ * in the @iov will be fully sent, before the next
+ * one is used. The @niov parameter specifies the
+ * total number of elements in @iov.
+ *
+ * It is not required for all @iov data to be fully
+ * sent. If the channel is in blocking mode, at least
+ * one byte of data will be sent, but no more is
+ * guaranteed. If the channel is non-blocking and no
+ * data can be sent, it will return QIO_CHANNEL_ERR_BLOCK
+ *
+ * If there are file descriptors to send, the @fds
+ * array should be non-NULL and provide the handles.
+ * All file descriptors will be sent if at least one
+ * byte of data was sent.
+ *
+ * It is an error to pass a non-NULL @fds parameter
+ * unless qio_channel_has_feature() returns a true
+ * value for the QIO_CHANNEL_FEATURE_FD_PASS constant.
+ *
+ * Returns: the number of bytes sent, or -1 on error,
+ * or QIO_CHANNEL_ERR_BLOCK if no data is can be sent
+ * and the channel is non-blocking
+ */
+ssize_t qio_channel_writev_full(QIOChannel *ioc,
+                                const struct iovec *iov,
+                                size_t niov,
+                                int *fds,
+                                size_t nfds,
+                                Error **errp);
+
+/**
+ * qio_channel_readv:
+ * @ioc: the channel object
+ * @iov: the array of memory regions to read data into
+ * @niov: the length of the @iov array
+ * @errp: pointer to an uninitialized error object
+ *
+ * Behaves as qio_channel_readv_full() but does not support
+ * receiving of file handles.
+ */
+ssize_t qio_channel_readv(QIOChannel *ioc,
+                          const struct iovec *iov,
+                          size_t niov,
+                          Error **errp);
+
+/**
+ * qio_channel_writev:
+ * @ioc: the channel object
+ * @iov: the array of memory regions to write data from
+ * @niov: the length of the @iov array
+ * @errp: pointer to an uninitialized error object
+ *
+ * Behaves as qio_channel_writev_full() but does not support
+ * sending of file handles.
+ */
+ssize_t qio_channel_writev(QIOChannel *ioc,
+                           const struct iovec *iov,
+                           size_t niov,
+                           Error **errp);
+
+/**
+ * qio_channel_readv:
+ * @ioc: the channel object
+ * @buf: the memory region to read data into
+ * @buflen: the length of @buf
+ * @errp: pointer to an uninitialized error object
+ *
+ * Behaves as qio_channel_readv_full() but does not support
+ * receiving of file handles, and only supports reading into
+ * a single memory region.
+ */
+ssize_t qio_channel_read(QIOChannel *ioc,
+                         char *buf,
+                         size_t buflen,
+                         Error **errp);
+
+/**
+ * qio_channel_writev:
+ * @ioc: the channel object
+ * @buf: the memory regions to send data from
+ * @buflen: the length of @buf
+ * @errp: pointer to an uninitialized error object
+ *
+ * Behaves as qio_channel_writev_full() but does not support
+ * sending of file handles, and only supports writing from a
+ * single memory region.
+ */
+ssize_t qio_channel_write(QIOChannel *ioc,
+                          const char *buf,
+                          size_t buflen,
+                          Error **errp);
+
+/**
+ * qio_channel_set_blocking:
+ * @ioc: the channel object
+ * @enabled: the blocking flag state
+ * @errp: pointer to an uninitialized error object
+ *
+ * If @enabled is true, then the channel is put into
+ * blocking mode, otherwise it will be non-blocking.
+ *
+ * In non-blocking mode, read/write operations may
+ * return QIO_CHANNEL_ERR_BLOCK if they would otherwise
+ * block on I/O
+ */
+int qio_channel_set_blocking(QIOChannel *ioc,
+                             bool enabled,
+                             Error **errp);
+
+/**
+ * qio_channel_close:
+ * @ioc: the channel object
+ * @errp: pointer to an uninitialized error object
+ *
+ * Close the channel, flushing any pending I/O
+ *
+ * Returns: 0 on success, -1 on error
+ */
+int qio_channel_close(QIOChannel *ioc,
+                      Error **errp);
+
+/**
+ * qio_channel_shutdown:
+ * @ioc: the channel object
+ * @how: the direction to shutdown
+ * @errp: pointer to an uninitialized error object
+ *
+ * Shutdowns transmission and/or receiving of data
+ * without closing the underlying transport.
+ *
+ * Not all implementations will support this facility,
+ * so may report an error. To avoid errors, the
+ * caller may check for the feature flag
+ * QIO_CHANNEL_FEATURE_SHUTDOWN prior to calling
+ * this method.
+ *
+ * Returns: 0 on success, -1 on error
+ */
+int qio_channel_shutdown(QIOChannel *ioc,
+                         QIOChannelShutdown how,
+                         Error **errp);
+
+/**
+ * qio_channel_set_delay:
+ * @ioc: the channel object
+ * @enabled: the new flag state
+ * @errp: pointer to an uninitialized error object
+ *
+ * Controls whether the underlying transport is
+ * permitted to delay writes in order to merge
+ * small packets. If @enabled is true, then the
+ * writes may be delayed in order to opportunistically
+ * merge small packets into larger ones. If @enabled
+ * is false, writes are dispatched immediately with
+ * no delay.
+ *
+ * When @enabled is false, applications may wish to
+ * use the qio_channel_set_cork() method to explicitly
+ * control write merging.
+ *
+ * On channels which are backed by a socket, this
+ * API corresponds to the inverse of TCP_NODELAY flag,
+ * controlling whether the Nagle algorithm is active.
+ *
+ * Not all implementations will support this facility,
+ * so may report an error. To avoid errors, the
+ * caller may check for the feature flag
+ * QIO_CHANNEL_FEATURE_DELAY prior to calling
+ * this method.
+ *
+ * Returns: 0 on success, -1 on error
+ */
+int qio_channel_set_delay(QIOChannel *ioc,
+                          bool enabled,
+                          Error **errp);
+
+/**
+ * qio_channel_set_cork:
+ * @ioc: the channel object
+ * @enabled: the new flag state
+ * @errp: pointer to an uninitialized error object
+ *
+ * Controls whether the underlying transport is
+ * permitted to dispatch data that is written.
+ * If @enabled is true, then any data written will
+ * be queued in local buffers until @enabled is
+ * set to false once again.
+ *
+ * This feature is typically used when the automatic
+ * write coalescing facility is disabled via the
+ * qio_channel_set_delay() method.
+ *
+ * On channels which are backed by a socket, this
+ * API corresponds to the TCP_CORK flag.
+ *
+ * Not all implementations will support this facility,
+ * so may report an error. To avoid errors, the
+ * caller may check for the feature flag
+ * QIO_CHANNEL_FEATURE_CORK prior to calling
+ * this method.
+ *
+ * Returns: 0 on success, -1 on error
+ */
+int qio_channel_set_cork(QIOChannel *ioc,
+                         bool enabled,
+                         Error **errp);
+
+
+/**
+ * qio_channel_seek:
+ * @ioc: the channel object
+ * @offset: the position to seek to, relative to @whence
+ * @whence: one of the POSIX SEEK_* constants
+ * @errp: pointer to an uninitialized error object
+ *
+ * Moves the current I/O position within the channel
+ * @ioc, to be @offset. The value of @offset is
+ * interpreted relative to @whence:
+ *
+ * SEEK_SET - the position is set to @offset bytes
+ * SEEK_CUR - the position is moved by @offset bytes
+ * SEEK_END - the position is set to end of the file plus @offset bytes
+ *
+ * Not all implementations will support this facility,
+ * so may report an error. To avoid errors, the
+ * caller may check for the feature flag
+ * QIO_CHANNEL_FEATURE_SEEKABLE prior to calling
+ * this method.
+ *
+ * Returns: the new position on success, (off64_t)-1 on failure
+ */
+off64_t qio_channel_io_seek(QIOChannel *ioc,
+                            off64_t offset,
+                            int whence,
+                            Error **errp);
+
+
+/**
+ * qio_channel_create_watch:
+ * @ioc: the channel object
+ * @condition: the I/O condition to monitor
+ *
+ * Create a new main loop source that is used to watch
+ * for the I/O condition @condition. Typically the
+ * qio_channel_add_watch() method would be used instead
+ * of this, since it directly attaches a callback to
+ * the source
+ *
+ * Returns: the new main loop source.
+ */
+GSource *qio_channel_create_watch(QIOChannel *ioc,
+                                  GIOCondition condition);
+
+/**
+ * qio_channel_add_watch:
+ * @ioc: the channel object
+ * @condition: the I/O condition to monitor
+ * @func: callback to invoke when the source becomes ready
+ * @user_data: opaque data to pass to @func
+ * @notify: callback to free @user_data
+ *
+ * Create a new main loop source that is used to watch
+ * for the I/O condition @condition. The callback @func
+ * will be registered against the source, to be invoked
+ * when the source becomes ready. The optional @user_data
+ * will be passed to @func when it is invoked. The @notify
+ * callback will be used to free @user_data when the
+ * watch is deleted
+ *
+ * The returned source ID can be used with g_source_remove()
+ * to remove and free the source when no longer required.
+ * Alternatively the @func callback can return a FALSE
+ * value.
+ *
+ * Returns: the source ID
+ */
+guint qio_channel_add_watch(QIOChannel *ioc,
+                            GIOCondition condition,
+                            QIOChannelFunc func,
+                            gpointer user_data,
+                            GDestroyNotify notify);
+
+
+/**
+ * qio_channel_yield:
+ * @ioc: the channel object
+ * @condition: the I/O condition to wait for
+ *
+ * Yields execution from the current coroutine until
+ * the condition indicated by @condition becomes
+ * available.
+ *
+ * This must only be called from coroutine context
+ */
+void qio_channel_yield(QIOChannel *ioc,
+                       GIOCondition condition);
+
+/**
+ * qio_channel_wait:
+ * @ioc: the channel object
+ * @condition: the I/O condition to wait for
+ *
+ * Block execution from the current thread until
+ * the condition indicated by @condition becomes
+ * available.
+ *
+ * This will enter a nested event loop to perform
+ * the wait.
+ */
+void qio_channel_wait(QIOChannel *ioc,
+                      GIOCondition condition);
+
+#endif /* QIO_CHANNEL_H__ */
diff --git a/io/Makefile.objs b/io/Makefile.objs
new file mode 100644
index 0000000..a6ed361
--- /dev/null
+++ b/io/Makefile.objs
@@ -0,0 +1 @@
+io-obj-y = channel.o
diff --git a/io/channel.c b/io/channel.c
new file mode 100644
index 0000000..297d05d
--- /dev/null
+++ b/io/channel.c
@@ -0,0 +1,290 @@
+/*
+ * QEMU I/O channels
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "io/channel.h"
+#include "qemu/coroutine.h"
+
+bool qio_channel_has_feature(QIOChannel *ioc,
+                             QIOChannelFeature feature)
+{
+    QIOChannelClass *klass = QIO_CHANNEL_GET_CLASS(ioc);
+
+    if (!klass->io_has_feature) {
+        return false;
+    }
+
+    return klass->io_has_feature(ioc, feature);
+}
+
+
+ssize_t qio_channel_readv_full(QIOChannel *ioc,
+                               const struct iovec *iov,
+                               size_t niov,
+                               int **fds,
+                               size_t *nfds,
+                               Error **errp)
+{
+    QIOChannelClass *klass = QIO_CHANNEL_GET_CLASS(ioc);
+
+    return klass->io_readv(ioc, iov, niov, fds, nfds, errp);
+}
+
+
+ssize_t qio_channel_writev_full(QIOChannel *ioc,
+                                const struct iovec *iov,
+                                size_t niov,
+                                int *fds,
+                                size_t nfds,
+                                Error **errp)
+{
+    QIOChannelClass *klass = QIO_CHANNEL_GET_CLASS(ioc);
+    return klass->io_writev(ioc, iov, niov, fds, nfds, errp);
+}
+
+
+ssize_t qio_channel_readv(QIOChannel *ioc,
+                          const struct iovec *iov,
+                          size_t niov,
+                          Error **errp)
+{
+    return qio_channel_readv_full(ioc, iov, niov, NULL, NULL, errp);
+}
+
+
+ssize_t qio_channel_writev(QIOChannel *ioc,
+                           const struct iovec *iov,
+                           size_t niov,
+                           Error **errp)
+{
+    return qio_channel_writev_full(ioc, iov, niov, NULL, 0, errp);
+}
+
+
+ssize_t qio_channel_read(QIOChannel *ioc,
+                         char *buf,
+                         size_t buflen,
+                         Error **errp)
+{
+    struct iovec iov = { .iov_base = buf, .iov_len = buflen };
+    return qio_channel_readv_full(ioc, &iov, 1, NULL, NULL, errp);
+}
+
+
+ssize_t qio_channel_write(QIOChannel *ioc,
+                          const char *buf,
+                          size_t buflen,
+                          Error **errp)
+{
+    struct iovec iov = { .iov_base = (char *)buf, .iov_len = buflen };
+    return qio_channel_writev_full(ioc, &iov, 1, NULL, 0, errp);
+}
+
+
+int qio_channel_set_blocking(QIOChannel *ioc,
+                              bool enabled,
+                              Error **errp)
+{
+    QIOChannelClass *klass = QIO_CHANNEL_GET_CLASS(ioc);
+    return klass->io_set_blocking(ioc, enabled, errp);
+}
+
+
+int qio_channel_close(QIOChannel *ioc,
+                      Error **errp)
+{
+    QIOChannelClass *klass = QIO_CHANNEL_GET_CLASS(ioc);
+    return klass->io_close(ioc, errp);
+}
+
+
+GSource *qio_channel_create_watch(QIOChannel *ioc,
+                                  GIOCondition condition)
+{
+    QIOChannelClass *klass = QIO_CHANNEL_GET_CLASS(ioc);
+    return klass->io_create_watch(ioc, condition);
+}
+
+
+guint qio_channel_add_watch(QIOChannel *ioc,
+                            GIOCondition condition,
+                            QIOChannelFunc func,
+                            gpointer user_data,
+                            GDestroyNotify notify)
+{
+    GSource *source;
+    guint id;
+
+    source = qio_channel_create_watch(ioc, condition);
+
+    g_source_set_callback(source, (GSourceFunc)func, user_data, notify);
+
+    id = g_source_attach(source, NULL);
+    g_source_unref(source);
+
+    return id;
+}
+
+
+int qio_channel_shutdown(QIOChannel *ioc,
+                         QIOChannelShutdown how,
+                         Error **errp)
+{
+    QIOChannelClass *klass = QIO_CHANNEL_GET_CLASS(ioc);
+
+    if (!klass->io_shutdown) {
+        error_setg(errp, "Data path shutdown not supported");
+        return -1;
+    }
+
+    return klass->io_shutdown(ioc, how, errp);
+}
+
+
+int qio_channel_set_delay(QIOChannel *ioc,
+                          bool enabled,
+                          Error **errp)
+{
+    QIOChannelClass *klass = QIO_CHANNEL_GET_CLASS(ioc);
+
+    if (!klass->io_set_delay) {
+        error_setg(errp, "Control of I/O delay not supported");
+        return -1;
+    }
+
+    return klass->io_set_delay(ioc, enabled, errp);
+}
+
+
+int qio_channel_set_cork(QIOChannel *ioc,
+                         bool enabled,
+                         Error **errp)
+{
+    QIOChannelClass *klass = QIO_CHANNEL_GET_CLASS(ioc);
+
+    if (!klass->io_set_cork) {
+        error_setg(errp, "Control of I/O cork not supported");
+        return -1;
+    }
+
+    return klass->io_set_cork(ioc, enabled, errp);
+}
+
+
+off64_t qio_channel_io_seek(QIOChannel *ioc,
+                            off64_t offset,
+                            int whence,
+                            Error **errp)
+{
+    QIOChannelClass *klass = QIO_CHANNEL_GET_CLASS(ioc);
+
+    if (!klass->io_seek) {
+        error_setg(errp, "Channel does not support random access");
+        return -1;
+    }
+
+    return klass->io_seek(ioc, offset, whence, errp);
+}
+
+
+typedef struct QIOChannelYieldData QIOChannelYieldData;
+struct QIOChannelYieldData {
+    QIOChannel *ioc;
+    Coroutine *co;
+};
+
+
+static gboolean qio_channel_yield_enter(QIOChannel *ioc,
+                                        GIOCondition condition,
+                                        gpointer opaque)
+{
+    QIOChannelYieldData *data = opaque;
+    qemu_coroutine_enter(data->co, NULL);
+    return FALSE;
+}
+
+
+void coroutine_fn qio_channel_yield(QIOChannel *ioc,
+                                    GIOCondition condition)
+{
+    QIOChannelYieldData data;
+
+    assert(qemu_in_coroutine());
+    data.ioc = ioc;
+    data.co = qemu_coroutine_self();
+    qio_channel_add_watch(ioc,
+                          condition,
+                          qio_channel_yield_enter,
+                          &data,
+                          NULL);
+    qemu_coroutine_yield();
+}
+
+
+static gboolean qio_channel_wait_complete(QIOChannel *ioc,
+                                          GIOCondition condition,
+                                          gpointer opaque)
+{
+    GMainLoop *loop = opaque;
+
+    g_main_loop_quit(loop);
+    return FALSE;
+}
+
+
+void qio_channel_wait(QIOChannel *ioc,
+                      GIOCondition condition)
+{
+    GMainContext *ctxt = g_main_context_new();
+    GMainLoop *loop = g_main_loop_new(ctxt, TRUE);
+    GSource *source;
+
+    source = qio_channel_create_watch(ioc, condition);
+
+    g_source_set_callback(source,
+                          (GSourceFunc)qio_channel_wait_complete,
+                          loop,
+                          NULL);
+
+    g_source_attach(source, ctxt);
+
+    g_main_loop_run(loop);
+
+    g_source_unref(source);
+    g_main_loop_unref(loop);
+    g_main_context_unref(ctxt);
+}
+
+
+static const TypeInfo qio_channel_info = {
+    .parent = TYPE_OBJECT,
+    .name = TYPE_QIO_CHANNEL,
+    .instance_size = sizeof(QIOChannel),
+    .abstract = true,
+    .class_size = sizeof(QIOChannelClass),
+};
+
+
+static void qio_channel_register_types(void)
+{
+    type_register_static(&qio_channel_info);
+}
+
+
+type_init(qio_channel_register_types);
-- 
2.4.3

^ permalink raw reply related	[flat|nested] 32+ messages in thread

* [Qemu-devel] [PATCH v1 08/16] io: add helper module for creating watches on FDs
  2015-09-18 13:18 [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework Daniel P. Berrange
                   ` (6 preceding siblings ...)
  2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 07/16] io: add abstract QIOChannel classes Daniel P. Berrange
@ 2015-09-18 13:19 ` Daniel P. Berrange
  2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 09/16] io: pull Buffer code out of VNC module Daniel P. Berrange
                   ` (8 subsequent siblings)
  16 siblings, 0 replies; 32+ messages in thread
From: Daniel P. Berrange @ 2015-09-18 13:19 UTC (permalink / raw)
  To: qemu-devel; +Cc: Paolo Bonzini, Dr. David Alan Gilbert, Gerd Hoffmann

A number of the channel implementations will require the
ability to create watches on file descriptors / sockets.
To avoid duplicating this code in each channel, provide a
helper API for dealing with file descriptor watches.

There are two watch implementations provided. The first
is useful for bi-directional file descriptors such as
sockets, regular files, character devices, etc. The
second works with a pair of unidirectional file descriptors
such as pipes.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
 include/io/channel-watch.h |  72 ++++++++++++++++
 io/Makefile.objs           |   1 +
 io/channel-watch.c         | 200 +++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 273 insertions(+)
 create mode 100644 include/io/channel-watch.h
 create mode 100644 io/channel-watch.c

diff --git a/include/io/channel-watch.h b/include/io/channel-watch.h
new file mode 100644
index 0000000..656358a
--- /dev/null
+++ b/include/io/channel-watch.h
@@ -0,0 +1,72 @@
+/*
+ * QEMU I/O channels watch helper APIs
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef QIO_CHANNEL_WATCH_H__
+#define QIO_CHANNEL_WATCH_H__
+
+#include "io/channel.h"
+
+/*
+ * This module provides helper functions that will be needed by
+ * the various QIOChannel implementations, for creating watches
+ * on file descriptors / sockets
+ */
+
+/**
+ * qio_channel_create_fd_watch:
+ * @ioc: the channel object
+ * @fd: the file descriptor
+ * @condition: the I/O condition
+ *
+ * Create a new main loop source that is able to
+ * monitor the file descriptor @fd for the
+ * I/O conditions in @condition. This is able
+ * monitor block devices, character devices,
+ * sockets, pipes but not plain files.
+ *
+ * Returns: the new main loop source
+ */
+GSource *qio_channel_create_fd_watch(QIOChannel *ioc,
+                                     int fd,
+                                     GIOCondition condition);
+
+/**
+ * qio_channel_create_fd_pair_watch:
+ * @ioc: the channel object
+ * @fdread: the file descriptor for reading
+ * @fdwrite: the file descriptor for writing
+ * @condition: the I/O condition
+ *
+ * Create a new main loop source that is able to
+ * monitor the pair of file descriptors @fdread
+ * and @fdwrite for the I/O conditions in @condition.
+ * This is intended for monitoring unidirectional
+ * file descriptors such as pipes, where a pair
+ * of descriptors is required for bidirectional
+ * I/O
+ *
+ * Returns: the new main loop source
+ */
+GSource *qio_channel_create_fd_pair_watch(QIOChannel *ioc,
+                                          int fdread,
+                                          int fdwrite,
+                                          GIOCondition condition);
+
+#endif /* QIO_CHANNEL_WATCH_H__ */
diff --git a/io/Makefile.objs b/io/Makefile.objs
index a6ed361..b02ea90 100644
--- a/io/Makefile.objs
+++ b/io/Makefile.objs
@@ -1 +1,2 @@
 io-obj-y = channel.o
+io-obj-y += channel-watch.o
diff --git a/io/channel-watch.c b/io/channel-watch.c
new file mode 100644
index 0000000..9564605
--- /dev/null
+++ b/io/channel-watch.c
@@ -0,0 +1,200 @@
+/*
+ * QEMU I/O channels watch helper APIs
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "io/channel-watch.h"
+
+typedef struct QIOChannelFDSource QIOChannelFDSource;
+struct QIOChannelFDSource {
+    GSource parent;
+    GPollFD fd;
+    QIOChannel *ioc;
+    GIOCondition condition;
+};
+
+
+typedef struct QIOChannelFDPairSource QIOChannelFDPairSource;
+struct QIOChannelFDPairSource {
+    GSource parent;
+    GPollFD fdread;
+    GPollFD fdwrite;
+    QIOChannel *ioc;
+    GIOCondition condition;
+};
+
+
+static gboolean
+qio_channel_fd_source_prepare(GSource *source G_GNUC_UNUSED,
+                              gint *timeout)
+{
+    *timeout = -1;
+
+    return FALSE;
+}
+
+
+static gboolean
+qio_channel_fd_source_check(GSource *source)
+{
+    QIOChannelFDSource *ssource = (QIOChannelFDSource *)source;
+
+    return ssource->fd.revents & ssource->condition;
+}
+
+
+static gboolean
+qio_channel_fd_source_dispatch(GSource *source,
+                               GSourceFunc callback,
+                               gpointer user_data)
+{
+    QIOChannelFunc func = (QIOChannelFunc)callback;
+    QIOChannelFDSource *ssource = (QIOChannelFDSource *)source;
+
+    return (*func)(ssource->ioc,
+                   ssource->fd.revents & ssource->condition,
+                   user_data);
+}
+
+
+static void
+qio_channel_fd_source_finalize(GSource *source)
+{
+    QIOChannelFDSource *ssource = (QIOChannelFDSource *)source;
+
+    object_unref(OBJECT(ssource->ioc));
+}
+
+
+static gboolean
+qio_channel_fd_pair_source_prepare(GSource *source G_GNUC_UNUSED,
+                                   gint *timeout)
+{
+    *timeout = -1;
+
+    return FALSE;
+}
+
+
+static gboolean
+qio_channel_fd_pair_source_check(GSource *source)
+{
+    QIOChannelFDPairSource *ssource = (QIOChannelFDPairSource *)source;
+    GIOCondition poll_condition = ssource->fdread.revents |
+        ssource->fdwrite.revents;
+
+    return poll_condition & ssource->condition;
+}
+
+
+static gboolean
+qio_channel_fd_pair_source_dispatch(GSource *source,
+                                    GSourceFunc callback,
+                                    gpointer user_data)
+{
+    QIOChannelFunc func = (QIOChannelFunc)callback;
+    QIOChannelFDPairSource *ssource = (QIOChannelFDPairSource *)source;
+    GIOCondition poll_condition = ssource->fdread.revents |
+        ssource->fdwrite.revents;
+
+    return (*func)(ssource->ioc,
+                   poll_condition & ssource->condition,
+                   user_data);
+}
+
+
+static void
+qio_channel_fd_pair_source_finalize(GSource *source)
+{
+    QIOChannelFDPairSource *ssource = (QIOChannelFDPairSource *)source;
+
+    object_unref(OBJECT(ssource->ioc));
+}
+
+
+GSourceFuncs qio_channel_fd_source_funcs = {
+    qio_channel_fd_source_prepare,
+    qio_channel_fd_source_check,
+    qio_channel_fd_source_dispatch,
+    qio_channel_fd_source_finalize
+};
+
+
+GSourceFuncs qio_channel_fd_pair_source_funcs = {
+    qio_channel_fd_pair_source_prepare,
+    qio_channel_fd_pair_source_check,
+    qio_channel_fd_pair_source_dispatch,
+    qio_channel_fd_pair_source_finalize
+};
+
+
+GSource *qio_channel_create_fd_watch(QIOChannel *ioc,
+                                     int fd,
+                                     GIOCondition condition)
+{
+    GSource *source;
+    QIOChannelFDSource *ssource;
+
+    source = g_source_new(&qio_channel_fd_source_funcs,
+                          sizeof(QIOChannelFDSource));
+    g_source_set_name(source, "QIOChannelFD");
+    ssource = (QIOChannelFDSource *)source;
+
+    ssource->ioc = ioc;
+    object_ref(OBJECT(ioc));
+
+    ssource->condition = condition;
+
+    ssource->fd.fd = fd;
+    ssource->fd.events = condition;
+
+    g_source_add_poll(source, &ssource->fd);
+
+    return source;
+}
+
+
+GSource *qio_channel_create_fd_pair_watch(QIOChannel *ioc,
+                                          int fdread,
+                                          int fdwrite,
+                                          GIOCondition condition)
+{
+    GSource *source;
+    QIOChannelFDPairSource *ssource;
+
+    source = g_source_new(&qio_channel_fd_pair_source_funcs,
+                          sizeof(QIOChannelFDPairSource));
+    g_source_set_name(source, "QIOChannelFDPair");
+    ssource = (QIOChannelFDPairSource *)source;
+
+    ssource->ioc = ioc;
+    object_ref(OBJECT(ioc));
+
+    ssource->condition = condition;
+
+    ssource->fdread.fd = fdread;
+    ssource->fdread.events = condition & G_IO_IN;
+
+    ssource->fdwrite.fd = fdwrite;
+    ssource->fdwrite.events = condition & G_IO_OUT;
+
+    g_source_add_poll(source, &ssource->fdread);
+    g_source_add_poll(source, &ssource->fdwrite);
+
+    return source;
+}
-- 
2.4.3

^ permalink raw reply related	[flat|nested] 32+ messages in thread

* [Qemu-devel] [PATCH v1 09/16] io: pull Buffer code out of VNC module
  2015-09-18 13:18 [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework Daniel P. Berrange
                   ` (7 preceding siblings ...)
  2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 08/16] io: add helper module for creating watches on FDs Daniel P. Berrange
@ 2015-09-18 13:19 ` Daniel P. Berrange
  2015-09-22 12:04   ` Paolo Bonzini
  2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 10/16] io: add QIOTask class for async operations Daniel P. Berrange
                   ` (7 subsequent siblings)
  16 siblings, 1 reply; 32+ messages in thread
From: Daniel P. Berrange @ 2015-09-18 13:19 UTC (permalink / raw)
  To: qemu-devel; +Cc: Paolo Bonzini, Dr. David Alan Gilbert, Gerd Hoffmann

The Buffer code in the VNC server is useful for the IO channel
code, so pull it out into a shared module, QIOBuffer.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
 include/io/buffer.h | 118 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 io/Makefile.objs    |   3 +-
 io/buffer.c         |  65 +++++++++++++++++++++++++++++
 ui/vnc-auth-sasl.c  |   4 +-
 ui/vnc-enc-tight.c  |  38 ++++++++---------
 ui/vnc-enc-zlib.c   |   6 +--
 ui/vnc-enc-zrle.c   |  18 ++++----
 ui/vnc-jobs.c       |  15 +++----
 ui/vnc-ws.c         |  36 ++++++++--------
 ui/vnc-ws.h         |   6 +--
 ui/vnc.c            |  67 ++++++-----------------------
 ui/vnc.h            |  50 ++++++++--------------
 12 files changed, 277 insertions(+), 149 deletions(-)
 create mode 100644 include/io/buffer.h
 create mode 100644 io/buffer.c

diff --git a/include/io/buffer.h b/include/io/buffer.h
new file mode 100644
index 0000000..2b1b261
--- /dev/null
+++ b/include/io/buffer.h
@@ -0,0 +1,118 @@
+/*
+ * QEMU I/O buffers
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef QIO_BUFFER_H__
+#define QIO_BUFFER_H__
+
+#include "qemu-common.h"
+
+typedef struct QIOBuffer QIOBuffer;
+
+/**
+ * QIOBuffer:
+ *
+ * The QIOBuffer object provides a simple dynamically resizing
+ * array, with separate tracking of capacity and usage. This
+ * is typically useful when buffering I/O data.
+ */
+
+struct QIOBuffer {
+    size_t capacity;
+    size_t offset;
+    uint8_t *buffer;
+};
+
+/**
+ * qio_buffer_reserve:
+ * @buffer: the buffer object
+ * @len: the minimum required free space
+ *
+ * Ensure that the buffer has space allocated for at least
+ * @len bytes. If the current buffer is too small, it will
+ * be reallocated, possibly to a larger size than requested.
+ */
+void qio_buffer_reserve(QIOBuffer *buffer, size_t len);
+
+/**
+ * qio_buffer_reset:
+ * @buffer: the buffer object
+ *
+ * Reset the length of the stored data to zero, but do
+ * not free / reallocate the memory buffer
+ */
+void qio_buffer_reset(QIOBuffer *buffer);
+
+/**
+ * qio_buffer_free:
+ * @buffer: the buffer object
+ *
+ * Reset the length of the stored data to zero and also
+ * free the internal memory buffer
+ */
+void qio_buffer_free(QIOBuffer *buffer);
+
+/**
+ * qio_buffer_append:
+ * @buffer: the buffer object
+ * @data: the data block to append
+ * @len: the length of @data in bytes
+ *
+ * Append the contents of @data to the end of the buffer.
+ * The caller must ensure that the buffer has sufficient
+ * free space for @len bytes, typically by calling the
+ * qio_buffer_reserve() method prior to appending.
+ */
+void qio_buffer_append(QIOBuffer *buffer, const void *data, size_t len);
+
+/**
+ * qio_buffer_advance:
+ * @buffer: the buffer object
+ * @len: the number of bytes to skip
+ *
+ * Remove @len bytes of data from the head of the buffer.
+ * The internal buffer will not be reallocated, so will
+ * have at least @len bytes of free space after this
+ * call completes
+ */
+void qio_buffer_advance(QIOBuffer *buffer, size_t len);
+
+/**
+ * qio_buffer_end:
+ * @buffer: the buffer object
+ *
+ * Get a pointer to the tail end of the internal buffer
+ * The returned pointer is only valid until the next
+ * call to qio_buffer_reserve().
+ *
+ * Returns: the tail of the buffer
+ */
+uint8_t *qio_buffer_end(QIOBuffer *buffer);
+
+/**
+ * qio_buffer_empty:
+ * @buffer: the buffer object
+ *
+ * Determine if the buffer contains any current data
+ *
+ * Returns: true if the buffer holds data, false otherwise
+ */
+gboolean qio_buffer_empty(QIOBuffer *buffer);
+
+#endif /* QIO_BUFFER_H__ */
diff --git a/io/Makefile.objs b/io/Makefile.objs
index b02ea90..593ed9e 100644
--- a/io/Makefile.objs
+++ b/io/Makefile.objs
@@ -1,2 +1,3 @@
-io-obj-y = channel.o
+io-obj-y = buffer.o
+io-obj-y += channel.o
 io-obj-y += channel-watch.o
diff --git a/io/buffer.c b/io/buffer.c
new file mode 100644
index 0000000..68ae68d
--- /dev/null
+++ b/io/buffer.c
@@ -0,0 +1,65 @@
+/*
+ * QEMU I/O buffers
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "io/buffer.h"
+
+void qio_buffer_reserve(QIOBuffer *buffer, size_t len)
+{
+    if ((buffer->capacity - buffer->offset) < len) {
+        buffer->capacity += (len + 1024);
+        buffer->buffer = g_realloc(buffer->buffer, buffer->capacity);
+    }
+}
+
+gboolean qio_buffer_empty(QIOBuffer *buffer)
+{
+    return buffer->offset == 0;
+}
+
+uint8_t *qio_buffer_end(QIOBuffer *buffer)
+{
+    return buffer->buffer + buffer->offset;
+}
+
+void qio_buffer_reset(QIOBuffer *buffer)
+{
+    buffer->offset = 0;
+}
+
+void qio_buffer_free(QIOBuffer *buffer)
+{
+    g_free(buffer->buffer);
+    buffer->offset = 0;
+    buffer->capacity = 0;
+    buffer->buffer = NULL;
+}
+
+void qio_buffer_append(QIOBuffer *buffer, const void *data, size_t len)
+{
+    memcpy(buffer->buffer + buffer->offset, data, len);
+    buffer->offset += len;
+}
+
+void qio_buffer_advance(QIOBuffer *buffer, size_t len)
+{
+    memmove(buffer->buffer, buffer->buffer + len,
+            (buffer->offset - len));
+    buffer->offset -= len;
+}
diff --git a/ui/vnc-auth-sasl.c b/ui/vnc-auth-sasl.c
index fc732bd..d118266 100644
--- a/ui/vnc-auth-sasl.c
+++ b/ui/vnc-auth-sasl.c
@@ -113,8 +113,8 @@ long vnc_client_read_sasl(VncState *vs)
         return vnc_client_io_error(vs, -1, -EIO);
     VNC_DEBUG("Read SASL Encoded %p size %ld Decoded %p size %d\n",
               encoded, ret, decoded, decodedLen);
-    buffer_reserve(&vs->input, decodedLen);
-    buffer_append(&vs->input, decoded, decodedLen);
+    qio_buffer_reserve(&vs->input, decodedLen);
+    qio_buffer_append(&vs->input, decoded, decodedLen);
     return decodedLen;
 }
 
diff --git a/ui/vnc-enc-tight.c b/ui/vnc-enc-tight.c
index 9a9ddf2..772ec79 100644
--- a/ui/vnc-enc-tight.c
+++ b/ui/vnc-enc-tight.c
@@ -856,7 +856,7 @@ static int tight_compress_data(VncState *vs, int stream_id, size_t bytes,
     }
 
     /* reserve memory in output buffer */
-    buffer_reserve(&vs->tight.zlib, bytes + 64);
+    qio_buffer_reserve(&vs->tight.zlib, bytes + 64);
 
     /* set pointers */
     zstream->next_in = vs->tight.tight.buffer;
@@ -879,7 +879,7 @@ static int tight_compress_data(VncState *vs, int stream_id, size_t bytes,
     tight_send_compact_size(vs, bytes);
     vnc_write(vs, vs->tight.zlib.buffer, bytes);
 
-    buffer_reset(&vs->tight.zlib);
+    qio_buffer_reset(&vs->tight.zlib);
 
     return bytes;
 }
@@ -1053,7 +1053,7 @@ static bool send_gradient_rect(VncState *vs, int x, int y, int w, int h)
     vnc_write_u8(vs, (stream | VNC_TIGHT_EXPLICIT_FILTER) << 4);
     vnc_write_u8(vs, VNC_TIGHT_FILTER_GRADIENT);
 
-    buffer_reserve(&vs->tight.gradient, w * 3 * sizeof (int));
+    qio_buffer_reserve(&vs->tight.gradient, w * 3 * sizeof(int));
 
     if (vs->tight.pixel24) {
         tight_filter_gradient24(vs, vs->tight.tight.buffer, w, h);
@@ -1066,7 +1066,7 @@ static bool send_gradient_rect(VncState *vs, int x, int y, int w, int h)
         bytes = 2;
     }
 
-    buffer_reset(&vs->tight.gradient);
+    qio_buffer_reset(&vs->tight.gradient);
 
     bytes = w * h * bytes;
     vs->tight.tight.offset = bytes;
@@ -1149,7 +1149,7 @@ static int send_palette_rect(VncState *vs, int x, int y,
 static void jpeg_init_destination(j_compress_ptr cinfo)
 {
     VncState *vs = cinfo->client_data;
-    Buffer *buffer = &vs->tight.jpeg;
+    QIOBuffer *buffer = &vs->tight.jpeg;
 
     cinfo->dest->next_output_byte = (JOCTET *)buffer->buffer + buffer->offset;
     cinfo->dest->free_in_buffer = (size_t)(buffer->capacity - buffer->offset);
@@ -1159,10 +1159,10 @@ static void jpeg_init_destination(j_compress_ptr cinfo)
 static boolean jpeg_empty_output_buffer(j_compress_ptr cinfo)
 {
     VncState *vs = cinfo->client_data;
-    Buffer *buffer = &vs->tight.jpeg;
+    QIOBuffer *buffer = &vs->tight.jpeg;
 
     buffer->offset = buffer->capacity;
-    buffer_reserve(buffer, 2048);
+    qio_buffer_reserve(buffer, 2048);
     jpeg_init_destination(cinfo);
     return TRUE;
 }
@@ -1171,7 +1171,7 @@ static boolean jpeg_empty_output_buffer(j_compress_ptr cinfo)
 static void jpeg_term_destination(j_compress_ptr cinfo)
 {
     VncState *vs = cinfo->client_data;
-    Buffer *buffer = &vs->tight.jpeg;
+    QIOBuffer *buffer = &vs->tight.jpeg;
 
     buffer->offset = buffer->capacity - cinfo->dest->free_in_buffer;
 }
@@ -1190,7 +1190,7 @@ static int send_jpeg_rect(VncState *vs, int x, int y, int w, int h, int quality)
         return send_full_color_rect(vs, x, y, w, h);
     }
 
-    buffer_reserve(&vs->tight.jpeg, 2048);
+    qio_buffer_reserve(&vs->tight.jpeg, 2048);
 
     cinfo.err = jpeg_std_error(&jerr);
     jpeg_create_compress(&cinfo);
@@ -1227,7 +1227,7 @@ static int send_jpeg_rect(VncState *vs, int x, int y, int w, int h, int quality)
 
     tight_send_compact_size(vs, vs->tight.jpeg.offset);
     vnc_write(vs, vs->tight.jpeg.buffer, vs->tight.jpeg.offset);
-    buffer_reset(&vs->tight.jpeg);
+    qio_buffer_reset(&vs->tight.jpeg);
 
     return 1;
 }
@@ -1270,7 +1270,7 @@ static void png_write_data(png_structp png_ptr, png_bytep data,
 {
     VncState *vs = png_get_io_ptr(png_ptr);
 
-    buffer_reserve(&vs->tight.png, vs->tight.png.offset + length);
+    qio_buffer_reserve(&vs->tight.png, vs->tight.png.offset + length);
     memcpy(vs->tight.png.buffer + vs->tight.png.offset, data, length);
 
     vs->tight.png.offset += length;
@@ -1351,7 +1351,7 @@ static int send_png_rect(VncState *vs, int x, int y, int w, int h,
 
     png_write_info(png_ptr, info_ptr);
 
-    buffer_reserve(&vs->tight.png, 2048);
+    qio_buffer_reserve(&vs->tight.png, 2048);
     linebuf = qemu_pixman_linebuf_create(PIXMAN_BE_r8g8b8, w);
     buf = (uint8_t *)pixman_image_get_data(linebuf);
     for (dy = 0; dy < h; dy++)
@@ -1377,14 +1377,14 @@ static int send_png_rect(VncState *vs, int x, int y, int w, int h,
 
     tight_send_compact_size(vs, vs->tight.png.offset);
     vnc_write(vs, vs->tight.png.buffer, vs->tight.png.offset);
-    buffer_reset(&vs->tight.png);
+    qio_buffer_reset(&vs->tight.png);
     return 1;
 }
 #endif /* CONFIG_VNC_PNG */
 
 static void vnc_tight_start(VncState *vs)
 {
-    buffer_reset(&vs->tight.tight);
+    qio_buffer_reset(&vs->tight.tight);
 
     // make the output buffer be the zlib buffer, so we can compress it later
     vs->tight.tmp = vs->output;
@@ -1686,13 +1686,13 @@ void vnc_tight_clear(VncState *vs)
         }
     }
 
-    buffer_free(&vs->tight.tight);
-    buffer_free(&vs->tight.zlib);
-    buffer_free(&vs->tight.gradient);
+    qio_buffer_free(&vs->tight.tight);
+    qio_buffer_free(&vs->tight.zlib);
+    qio_buffer_free(&vs->tight.gradient);
 #ifdef CONFIG_VNC_JPEG
-    buffer_free(&vs->tight.jpeg);
+    qio_buffer_free(&vs->tight.jpeg);
 #endif
 #ifdef CONFIG_VNC_PNG
-    buffer_free(&vs->tight.png);
+    qio_buffer_free(&vs->tight.png);
 #endif
 }
diff --git a/ui/vnc-enc-zlib.c b/ui/vnc-enc-zlib.c
index d1b97f2..47ba146 100644
--- a/ui/vnc-enc-zlib.c
+++ b/ui/vnc-enc-zlib.c
@@ -47,7 +47,7 @@ void vnc_zlib_zfree(void *x, void *addr)
 
 static void vnc_zlib_start(VncState *vs)
 {
-    buffer_reset(&vs->zlib.zlib);
+    qio_buffer_reset(&vs->zlib.zlib);
 
     // make the output buffer be the zlib buffer, so we can compress it later
     vs->zlib.tmp = vs->output;
@@ -96,7 +96,7 @@ static int vnc_zlib_stop(VncState *vs)
     }
 
     // reserve memory in output buffer
-    buffer_reserve(&vs->output, vs->zlib.zlib.offset + 64);
+    qio_buffer_reserve(&vs->output, vs->zlib.zlib.offset + 64);
 
     // set pointers
     zstream->next_in = vs->zlib.zlib.buffer;
@@ -148,5 +148,5 @@ void vnc_zlib_clear(VncState *vs)
     if (vs->zlib.stream.opaque) {
         deflateEnd(&vs->zlib.stream);
     }
-    buffer_free(&vs->zlib.zlib);
+    qio_buffer_free(&vs->zlib.zlib);
 }
diff --git a/ui/vnc-enc-zrle.c b/ui/vnc-enc-zrle.c
index ed3b484..bd1e320 100644
--- a/ui/vnc-enc-zrle.c
+++ b/ui/vnc-enc-zrle.c
@@ -36,7 +36,7 @@ static const int bits_per_packed_pixel[] = {
 
 static void vnc_zrle_start(VncState *vs)
 {
-    buffer_reset(&vs->zrle.zrle);
+    qio_buffer_reset(&vs->zrle.zrle);
 
     /* make the output buffer be the zlib buffer, so we can compress it later */
     vs->zrle.tmp = vs->output;
@@ -53,10 +53,10 @@ static void vnc_zrle_stop(VncState *vs)
 static void *zrle_convert_fb(VncState *vs, int x, int y, int w, int h,
                              int bpp)
 {
-    Buffer tmp;
+    QIOBuffer tmp;
 
-    buffer_reset(&vs->zrle.fb);
-    buffer_reserve(&vs->zrle.fb, w * h * bpp + bpp);
+    qio_buffer_reset(&vs->zrle.fb);
+    qio_buffer_reserve(&vs->zrle.fb, w * h * bpp + bpp);
 
     tmp = vs->output;
     vs->output = vs->zrle.fb;
@@ -72,7 +72,7 @@ static int zrle_compress_data(VncState *vs, int level)
 {
     z_streamp zstream = &vs->zrle.stream;
 
-    buffer_reset(&vs->zrle.zlib);
+    qio_buffer_reset(&vs->zrle.zlib);
 
     if (zstream->opaque != vs) {
         int err;
@@ -92,7 +92,7 @@ static int zrle_compress_data(VncState *vs, int level)
     }
 
     /* reserve memory in output buffer */
-    buffer_reserve(&vs->zrle.zlib, vs->zrle.zrle.offset + 64);
+    qio_buffer_reserve(&vs->zrle.zlib, vs->zrle.zrle.offset + 64);
 
     /* set pointers */
     zstream->next_in = vs->zrle.zrle.buffer;
@@ -360,7 +360,7 @@ void vnc_zrle_clear(VncState *vs)
     if (vs->zrle.stream.opaque) {
         deflateEnd(&vs->zrle.stream);
     }
-    buffer_free(&vs->zrle.zrle);
-    buffer_free(&vs->zrle.fb);
-    buffer_free(&vs->zrle.zlib);
+    qio_buffer_free(&vs->zrle.zrle);
+    qio_buffer_free(&vs->zrle.fb);
+    qio_buffer_free(&vs->zrle.zlib);
 }
diff --git a/ui/vnc-jobs.c b/ui/vnc-jobs.c
index 22c9abc..9824c34 100644
--- a/ui/vnc-jobs.c
+++ b/ui/vnc-jobs.c
@@ -54,7 +54,7 @@ struct VncJobQueue {
     QemuCond cond;
     QemuMutex mutex;
     QemuThread thread;
-    Buffer buffer;
+    QIOBuffer buffer;
     bool exit;
     QTAILQ_HEAD(, VncJob) jobs;
 };
@@ -167,7 +167,7 @@ void vnc_jobs_consume_buffer(VncState *vs)
     vnc_lock_output(vs);
     if (vs->jobs_buffer.offset) {
         vnc_write(vs, vs->jobs_buffer.buffer, vs->jobs_buffer.offset);
-        buffer_reset(&vs->jobs_buffer);
+        qio_buffer_reset(&vs->jobs_buffer);
     }
     flush = vs->csock != -1 && vs->abort != true;
     vnc_unlock_output(vs);
@@ -196,7 +196,7 @@ static void vnc_async_encoding_start(VncState *orig, VncState *local)
     local->output =  queue->buffer;
     local->csock = -1; /* Don't do any network work on this thread */
 
-    buffer_reset(&local->output);
+    qio_buffer_reset(&local->output);
 }
 
 static void vnc_async_encoding_end(VncState *orig, VncState *local)
@@ -273,10 +273,11 @@ static int vnc_worker_thread_loop(VncJobQueue *queue)
     vs.output.buffer[saved_offset + 1] = n_rectangles & 0xFF;
 
     vnc_lock_output(job->vs);
+
     if (job->vs->csock != -1) {
-        buffer_reserve(&job->vs->jobs_buffer, vs.output.offset);
-        buffer_append(&job->vs->jobs_buffer, vs.output.buffer,
-                      vs.output.offset);
+        qio_buffer_reserve(&job->vs->jobs_buffer, vs.output.offset);
+        qio_buffer_append(&job->vs->jobs_buffer, vs.output.buffer,
+                          vs.output.offset);
         /* Copy persistent encoding data */
         vnc_async_encoding_end(job->vs, &vs);
 
@@ -310,7 +311,7 @@ static void vnc_queue_clear(VncJobQueue *q)
 {
     qemu_cond_destroy(&queue->cond);
     qemu_mutex_destroy(&queue->mutex);
-    buffer_free(&queue->buffer);
+    qio_buffer_free(&queue->buffer);
     g_free(q);
     queue = NULL; /* Unset global queue */
 }
diff --git a/ui/vnc-ws.c b/ui/vnc-ws.c
index 175ea50..2fe4476 100644
--- a/ui/vnc-ws.c
+++ b/ui/vnc-ws.c
@@ -95,8 +95,8 @@ void vncws_handshake_read(void *opaque)
     /* Typical HTTP headers from novnc are 512 bytes, so limiting
      * total header size to 4096 is easily enough. */
     size_t want = 4096 - vs->ws_input.offset;
-    buffer_reserve(&vs->ws_input, want);
-    ret = vnc_client_read_buf(vs, buffer_end(&vs->ws_input), want);
+    qio_buffer_reserve(&vs->ws_input, want);
+    ret = vnc_client_read_buf(vs, qio_buffer_end(&vs->ws_input), want);
 
     if (!ret) {
         if (vs->csock == -1) {
@@ -111,7 +111,7 @@ void vncws_handshake_read(void *opaque)
     if (handshake_end) {
         qemu_set_fd_handler(vs->csock, vnc_client_read, NULL, vs);
         vncws_process_handshake(vs, vs->ws_input.buffer, vs->ws_input.offset);
-        buffer_advance(&vs->ws_input, handshake_end - vs->ws_input.buffer +
+        qio_buffer_advance(&vs->ws_input, handshake_end - vs->ws_input.buffer +
                 strlen(WS_HANDSHAKE_END));
     } else if (vs->ws_input.offset >= 4096) {
         VNC_DEBUG("End of headers not found in first 4096 bytes\n");
@@ -127,8 +127,8 @@ long vnc_client_read_ws(VncState *vs)
     size_t payload_size, header_size;
     VNC_DEBUG("Read websocket %p size %zd offset %zd\n", vs->ws_input.buffer,
             vs->ws_input.capacity, vs->ws_input.offset);
-    buffer_reserve(&vs->ws_input, 4096);
-    ret = vnc_client_read_buf(vs, buffer_end(&vs->ws_input), 4096);
+    qio_buffer_reserve(&vs->ws_input, 4096);
+    ret = vnc_client_read_buf(vs, qio_buffer_end(&vs->ws_input), 4096);
     if (!ret) {
         return 0;
     }
@@ -146,7 +146,7 @@ long vnc_client_read_ws(VncState *vs)
                 return err;
             }
 
-            buffer_advance(&vs->ws_input, header_size);
+            qio_buffer_advance(&vs->ws_input, header_size);
         }
         if (vs->ws_payload_remain != 0) {
             err = vncws_decode_frame_payload(&vs->ws_input,
@@ -162,10 +162,10 @@ long vnc_client_read_ws(VncState *vs)
             }
             ret += err;
 
-            buffer_reserve(&vs->input, payload_size);
-            buffer_append(&vs->input, payload, payload_size);
+            qio_buffer_reserve(&vs->input, payload_size);
+            qio_buffer_append(&vs->input, payload, payload_size);
 
-            buffer_advance(&vs->ws_input, payload_size);
+            qio_buffer_advance(&vs->ws_input, payload_size);
         }
     } while (vs->ws_input.offset > 0);
 
@@ -178,13 +178,13 @@ long vnc_client_write_ws(VncState *vs)
     VNC_DEBUG("Write WS: Pending output %p size %zd offset %zd\n",
               vs->output.buffer, vs->output.capacity, vs->output.offset);
     vncws_encode_frame(&vs->ws_output, vs->output.buffer, vs->output.offset);
-    buffer_reset(&vs->output);
+    qio_buffer_reset(&vs->output);
     ret = vnc_client_write_buf(vs, vs->ws_output.buffer, vs->ws_output.offset);
     if (!ret) {
         return 0;
     }
 
-    buffer_advance(&vs->ws_output, ret);
+    qio_buffer_advance(&vs->ws_output, ret);
 
     if (vs->ws_output.offset == 0) {
         qemu_set_fd_handler(vs->csock, vnc_client_read, NULL, vs);
@@ -267,8 +267,8 @@ void vncws_process_handshake(VncState *vs, uint8_t *line, size_t size)
     g_free(key);
 }
 
-void vncws_encode_frame(Buffer *output, const void *payload,
-        const size_t payload_size)
+void vncws_encode_frame(QIOBuffer *output, const void *payload,
+                        const size_t payload_size)
 {
     size_t header_size = 0;
     unsigned char opcode = WS_OPCODE_BINARY_FRAME;
@@ -295,12 +295,12 @@ void vncws_encode_frame(Buffer *output, const void *payload,
         header_size = 10;
     }
 
-    buffer_reserve(output, header_size + payload_size);
-    buffer_append(output, header.buf, header_size);
-    buffer_append(output, payload, payload_size);
+    qio_buffer_reserve(output, header_size + payload_size);
+    qio_buffer_append(output, header.buf, header_size);
+    qio_buffer_append(output, payload, payload_size);
 }
 
-int vncws_decode_frame_header(Buffer *input,
+int vncws_decode_frame_header(QIOBuffer *input,
                               size_t *header_size,
                               size_t *payload_remain,
                               WsMask *payload_mask)
@@ -354,7 +354,7 @@ int vncws_decode_frame_header(Buffer *input,
     return 1;
 }
 
-int vncws_decode_frame_payload(Buffer *input,
+int vncws_decode_frame_payload(QIOBuffer *input,
                                size_t *payload_remain, WsMask *payload_mask,
                                uint8_t **payload, size_t *payload_size)
 {
diff --git a/ui/vnc-ws.h b/ui/vnc-ws.h
index 4ab0a8c..2a222a8 100644
--- a/ui/vnc-ws.h
+++ b/ui/vnc-ws.h
@@ -77,13 +77,13 @@ void vncws_handshake_read(void *opaque);
 long vnc_client_write_ws(VncState *vs);
 long vnc_client_read_ws(VncState *vs);
 void vncws_process_handshake(VncState *vs, uint8_t *line, size_t size);
-void vncws_encode_frame(Buffer *output, const void *payload,
+void vncws_encode_frame(QIOBuffer *output, const void *payload,
             const size_t payload_size);
-int vncws_decode_frame_header(Buffer *input,
+int vncws_decode_frame_header(QIOBuffer *input,
                               size_t *header_size,
                               size_t *payload_remain,
                               WsMask *payload_mask);
-int vncws_decode_frame_payload(Buffer *input,
+int vncws_decode_frame_payload(QIOBuffer *input,
                                size_t *payload_remain, WsMask *payload_mask,
                                uint8_t **payload, size_t *payload_size);
 
diff --git a/ui/vnc.c b/ui/vnc.c
index 952e551..c4dec90 100644
--- a/ui/vnc.c
+++ b/ui/vnc.c
@@ -647,49 +647,6 @@ void vnc_framebuffer_update(VncState *vs, int x, int y, int w, int h,
     vnc_write_s32(vs, encoding);
 }
 
-void buffer_reserve(Buffer *buffer, size_t len)
-{
-    if ((buffer->capacity - buffer->offset) < len) {
-        buffer->capacity += (len + 1024);
-        buffer->buffer = g_realloc(buffer->buffer, buffer->capacity);
-    }
-}
-
-static int buffer_empty(Buffer *buffer)
-{
-    return buffer->offset == 0;
-}
-
-uint8_t *buffer_end(Buffer *buffer)
-{
-    return buffer->buffer + buffer->offset;
-}
-
-void buffer_reset(Buffer *buffer)
-{
-        buffer->offset = 0;
-}
-
-void buffer_free(Buffer *buffer)
-{
-    g_free(buffer->buffer);
-    buffer->offset = 0;
-    buffer->capacity = 0;
-    buffer->buffer = NULL;
-}
-
-void buffer_append(Buffer *buffer, const void *data, size_t len)
-{
-    memcpy(buffer->buffer + buffer->offset, data, len);
-    buffer->offset += len;
-}
-
-void buffer_advance(Buffer *buf, size_t len)
-{
-    memmove(buf->buffer, buf->buffer + len,
-            (buf->offset - len));
-    buf->offset -= len;
-}
 
 static void vnc_desktop_resize(VncState *vs)
 {
@@ -1220,10 +1177,10 @@ void vnc_disconnect_finish(VncState *vs)
     vnc_lock_output(vs);
     vnc_qmp_event(vs, QAPI_EVENT_VNC_DISCONNECTED);
 
-    buffer_free(&vs->input);
-    buffer_free(&vs->output);
-    buffer_free(&vs->ws_input);
-    buffer_free(&vs->ws_output);
+    qio_buffer_free(&vs->input);
+    qio_buffer_free(&vs->output);
+    qio_buffer_free(&vs->ws_input);
+    qio_buffer_free(&vs->ws_output);
 
     qapi_free_VncClientInfo(vs->info);
 
@@ -1251,7 +1208,7 @@ void vnc_disconnect_finish(VncState *vs)
     if (vs->bh != NULL) {
         qemu_bh_delete(vs->bh);
     }
-    buffer_free(&vs->jobs_buffer);
+    qio_buffer_free(&vs->jobs_buffer);
 
     for (i = 0; i < VNC_STAT_ROWS; ++i) {
         g_free(vs->lossy_rect[i]);
@@ -1393,7 +1350,7 @@ static ssize_t vnc_client_write_plain(VncState *vs)
     if (!ret)
         return 0;
 
-    buffer_advance(&vs->output, ret);
+    qio_buffer_advance(&vs->output, ret);
 
     if (vs->output.offset == 0) {
         qemu_set_fd_handler(vs->csock, vnc_client_read, NULL, vs);
@@ -1496,8 +1453,8 @@ static ssize_t vnc_client_read_plain(VncState *vs)
     ssize_t ret;
     VNC_DEBUG("Read plain %p size %zd offset %zd\n",
               vs->input.buffer, vs->input.capacity, vs->input.offset);
-    buffer_reserve(&vs->input, 4096);
-    ret = vnc_client_read_buf(vs, buffer_end(&vs->input), 4096);
+    qio_buffer_reserve(&vs->input, 4096);
+    ret = vnc_client_read_buf(vs, qio_buffer_end(&vs->input), 4096);
     if (!ret)
         return 0;
     vs->input.offset += ret;
@@ -1555,7 +1512,7 @@ void vnc_client_read(void *opaque)
         }
 
         if (!ret) {
-            buffer_advance(&vs->input, len);
+            qio_buffer_advance(&vs->input, len);
         } else {
             vs->read_handler_expect = ret;
         }
@@ -1564,13 +1521,13 @@ void vnc_client_read(void *opaque)
 
 void vnc_write(VncState *vs, const void *data, size_t len)
 {
-    buffer_reserve(&vs->output, len);
+    qio_buffer_reserve(&vs->output, len);
 
-    if (vs->csock != -1 && buffer_empty(&vs->output)) {
+    if (vs->csock != -1 && qio_buffer_empty(&vs->output)) {
         qemu_set_fd_handler(vs->csock, vnc_client_read, vnc_client_write, vs);
     }
 
-    buffer_append(&vs->output, data, len);
+    qio_buffer_append(&vs->output, data, len);
 }
 
 void vnc_write_s32(VncState *vs, int32_t value)
diff --git a/ui/vnc.h b/ui/vnc.h
index 4dd769c..339a1bf 100644
--- a/ui/vnc.h
+++ b/ui/vnc.h
@@ -34,6 +34,7 @@
 #include "audio/audio.h"
 #include "qemu/bitmap.h"
 #include "crypto/tlssession.h"
+#include "io/buffer.h"
 #include <zlib.h>
 #include <stdbool.h>
 
@@ -56,13 +57,6 @@
  *
  *****************************************************************************/
 
-typedef struct Buffer
-{
-    size_t capacity;
-    size_t offset;
-    uint8_t *buffer;
-} Buffer;
-
 typedef struct VncState VncState;
 typedef struct VncJob VncJob;
 typedef struct VncRect VncRect;
@@ -191,15 +185,15 @@ typedef struct VncTight {
     uint8_t quality;
     uint8_t compression;
     uint8_t pixel24;
-    Buffer tight;
-    Buffer tmp;
-    Buffer zlib;
-    Buffer gradient;
+    QIOBuffer tight;
+    QIOBuffer tmp;
+    QIOBuffer zlib;
+    QIOBuffer gradient;
 #ifdef CONFIG_VNC_JPEG
-    Buffer jpeg;
+    QIOBuffer jpeg;
 #endif
 #ifdef CONFIG_VNC_PNG
-    Buffer png;
+    QIOBuffer png;
 #endif
     int levels[4];
     z_stream stream[4];
@@ -210,18 +204,18 @@ typedef struct VncHextile {
 } VncHextile;
 
 typedef struct VncZlib {
-    Buffer zlib;
-    Buffer tmp;
+    QIOBuffer zlib;
+    QIOBuffer tmp;
     z_stream stream;
     int level;
 } VncZlib;
 
 typedef struct VncZrle {
     int type;
-    Buffer fb;
-    Buffer zrle;
-    Buffer tmp;
-    Buffer zlib;
+    QIOBuffer fb;
+    QIOBuffer zrle;
+    QIOBuffer tmp;
+    QIOBuffer zlib;
     z_stream stream;
     VncPalette palette;
 } VncZrle;
@@ -290,10 +284,10 @@ struct VncState
 
     VncClientInfo *info;
 
-    Buffer output;
-    Buffer input;
-    Buffer ws_input;
-    Buffer ws_output;
+    QIOBuffer output;
+    QIOBuffer input;
+    QIOBuffer ws_input;
+    QIOBuffer ws_output;
     size_t ws_payload_remain;
     WsMask ws_payload_mask;
     /* current output mode information */
@@ -315,7 +309,7 @@ struct VncState
     bool initialized;
     QemuMutex output_mutex;
     QEMUBH *bh;
-    Buffer jobs_buffer;
+    QIOBuffer jobs_buffer;
 
     /* Encoding specific, if you add something here, don't forget to
      *  update vnc_async_encoding_start()
@@ -535,14 +529,6 @@ ssize_t vnc_client_io_error(VncState *vs, ssize_t ret, int last_errno);
 void start_client_init(VncState *vs);
 void start_auth_vnc(VncState *vs);
 
-/* Buffer management */
-void buffer_reserve(Buffer *buffer, size_t len);
-void buffer_reset(Buffer *buffer);
-void buffer_free(Buffer *buffer);
-void buffer_append(Buffer *buffer, const void *data, size_t len);
-void buffer_advance(Buffer *buf, size_t len);
-uint8_t *buffer_end(Buffer *buffer);
-
 
 /* Misc helpers */
 
-- 
2.4.3

^ permalink raw reply related	[flat|nested] 32+ messages in thread

* [Qemu-devel] [PATCH v1 10/16] io: add QIOTask class for async operations
  2015-09-18 13:18 [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework Daniel P. Berrange
                   ` (8 preceding siblings ...)
  2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 09/16] io: pull Buffer code out of VNC module Daniel P. Berrange
@ 2015-09-18 13:19 ` Daniel P. Berrange
  2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 11/16] io: add QIOChannelSocket class Daniel P. Berrange
                   ` (6 subsequent siblings)
  16 siblings, 0 replies; 32+ messages in thread
From: Daniel P. Berrange @ 2015-09-18 13:19 UTC (permalink / raw)
  To: qemu-devel; +Cc: Paolo Bonzini, Dr. David Alan Gilbert, Gerd Hoffmann

A number of I/O operations need to be performed asynchronously
to avoid blocking the main loop. The caller of such APIs need
to provide a callback to be invoked on completion/error and
need access to the error, if any. The small QIOTask provides
a simple framework for dealing with such probes. The API
docs inline provide an outline of how this is to be used.

Some functions don't have the ability to run asynchronously
(eg getaddrinfo always blocks), so to facilitate their use,
the task class provides a mechanism to run a blocking
function in a thread, while triggering the completion
callback in the main event loop thread. This easily allows
any synchronous function to be made asynchronous, albeit
at the cost of spawning a thread.

In this series, the QIOTask class will be used for things like
the TLS handshake, the websockets handshake and TCP connect()
progress.

The concept of QIOTask is inspired by the GAsyncResult
interface / GTask class in the GIO libraries. The min
version requirements on glib don't allow those to be
used from QEMU, so QIOTask provides a facsimilie which
can be easily switched to GTask in the future if the
min version is increased.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
 include/io/task.h    | 256 +++++++++++++++++++++++++++++++++++++++++++++++
 io/Makefile.objs     |   1 +
 io/task.c            | 159 +++++++++++++++++++++++++++++
 tests/.gitignore     |   1 +
 tests/Makefile       |   3 +
 tests/test-io-task.c | 276 +++++++++++++++++++++++++++++++++++++++++++++++++++
 trace-events         |   9 ++
 7 files changed, 705 insertions(+)
 create mode 100644 include/io/task.h
 create mode 100644 io/task.c
 create mode 100644 tests/test-io-task.c

diff --git a/include/io/task.h b/include/io/task.h
new file mode 100644
index 0000000..2418714
--- /dev/null
+++ b/include/io/task.h
@@ -0,0 +1,256 @@
+/*
+ * QEMU I/O task
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef QIO_TASK_H__
+#define QIO_TASK_H__
+
+#include "qemu-common.h"
+#include "qapi/error.h"
+#include "qom/object.h"
+
+typedef struct QIOTask QIOTask;
+
+typedef void (*QIOTaskFunc)(Object *source,
+                            Error *err,
+                            gpointer opaque);
+
+typedef int (*QIOTaskWorker)(QIOTask *task,
+                             Error **errp,
+                             gpointer opaque);
+
+/**
+ * QIOTask:
+ *
+ * The QIOTask object provides a simple mechanism for reporting
+ * success / failure of long running background operations.
+ *
+ * A object on which the operation is to be performed could have
+ * a public API which accepts a task callback:
+ *
+ * <example>
+ *   <title>Task callback function signature</title>
+ *   <programlisting>
+ *  void myobject_operation(QMyObject *obj,
+ *                          QIOTaskFunc *func,
+ *                          gpointer opaque,
+ *                          GDestroyNotify *notify);
+ *   </programlisting>
+ * </example>
+ *
+ * The 'func' parameter is the callback to be invoked, and 'opaque'
+ * is data to pass to it. The optional 'notify' function is used
+ * to free 'opaque' when no longer needed.
+ *
+ * Now, lets say the implementation of this method wants to set
+ * a timer to run once a second checking for completion of some
+ * activity. It would do something like
+ *
+ * <example>
+ *   <title>Task callback function implementation</title>
+ *   <programlisting>
+ *    void myobject_operation(QMyObject *obj,
+ *                            QIOTaskFunc *func,
+ *                            gpointer opaque,
+ *                            GDestroyNotify *notify)
+ *    {
+ *      QIOTask *task;
+ *
+ *      task = qio_task_new(OBJECT(obj), func, opaque, notify);
+ *
+ *      g_timeout_add_full(G_PRIORITY_DEFAULT,
+ *                         1000,
+ *                         myobject_operation_timer,
+ *                         task,
+ *                         NULL);
+ *    }
+ *   </programlisting>
+ * </example>
+ *
+ * It could equally have setup a watch on a file descriptor or
+ * created a background thread, or something else entirely.
+ * Notice that the source object is passed to the task, and
+ * QIOTask will hold a reference on that. This ensure that
+ * the QMyObject instance cannot be garbage collected while
+ * the async task is still in progress.
+ *
+ * In this case, myobject_operation_timer will fire after
+ * 3 secs and do
+ *
+ * <example>
+ *   <title>Task timer function</title>
+ *   <programlisting>
+ *   gboolean myobject_operation_timer(gpointer opaque)
+ *   {
+ *      QIOTask *task = QIO_TASK(opaque);
+ *      Error *err;*
+ *
+ *      ...check something important...
+ *       if (err) {
+ *           qio_task_abort(task, err);
+ *           error_free(task);
+ *           return FALSE;
+ *       } else if (...work is completed ...) {
+ *           qio_task_complete(task);
+ *           return FALSE;
+ *       }
+ *       ...carry on polling ...
+ *       return TRUE;
+ *   }
+ *   </programlisting>
+ * </example>
+ *
+ * Once this function returns false, object_unref will be called
+ * automatically on the task causing it to be released and the
+ * ref on QMyObject dropped too.
+ *
+ * The QIOTask module can also be used to perform operations
+ * in a background thread context, while still reporting the
+ * results in the main event thread. This allows code which
+ * cannot easily be rewritten to be asychronous (such as DNS
+ * lookups) to be easily run non-blocking. Reporting the
+ * results in the main thread context means that the caller
+ * typically does not need to be concerned about thread
+ * safety wrt the QEMU global mutex.
+ *
+ * For example, the socket_listen() method will block the caller
+ * while DNS lookups take place if given a name, instead of IP
+ * address. The C library often do not provide a practical async
+ * DNS API, so the to get non-blocking DNS lookups in a portable
+ * manner requires use of a thread. So achieve a non-blocking
+ * socket listen using QIOTask would require:
+ *
+ * <example>
+ *    static int myobject_listen_worker(QIOTask *task,
+ *                                      Error **errp,
+ *                                      gpointer opaque)
+ *    {
+ *       QMyObject obj = QMY_OBJECT(qio_task_get_source(task));
+ *       SocketAddress *addr = opaque;
+ *
+ *       obj->fd = socket_listen(addr, errp);
+ *       if (obj->fd < 0) {
+ *          return -1;
+ *       }
+ *       return 0;
+ *    }
+ *
+ *    void myobject_listen_async(QMyObject *obj,
+ *                               SocketAddress *addr,
+ *                               QIOTaskFunc *func,
+ *                               gpointer opaque,
+ *                               GDestroyNotify *notify)
+ *    {
+ *      QIOTask *task;
+ *      SocketAddress *addrCopy;
+ *
+ *      qapi_copy_SocketAddress(&addrCopy, addr);
+ *      task = qio_task_new(OBJECT(obj), func, opaque, notify);
+ *
+ *      qio_task_run_in_thread(task, myobject_listen_worker,
+ *                             addrCopy,
+ *                             qapi_free_SocketAddress);
+ *    }
+ * </example>
+ *
+ * NB, The 'func' callback passed into myobject_listen_async
+ * will be invoked from the main event thread, despite the
+ * actual operation being performed in a different thread.
+ */
+
+/**
+ * qio_task_new:
+ * @source: the object on which the operation is invoked
+ * @func: the callback to invoke when the task completes
+ * @opaque: opaque data to pass to @func when invoked
+ * @destroy: optional callback to free @opaque
+ *
+ * Creates a new task struct to track completion of a
+ * background operation running on the object @source.
+ * When the operation completes or fails, the callback
+ * @func will be invoked. The callback can access the
+ * 'err' attribute in the task object to determine if
+ * the operation was successful or not.
+ *
+ * The returned task will be released when one of
+ * qio_task_abort() or qio_task_complete() are invoked.
+ *
+ * Returns: the task struct
+ */
+QIOTask *qio_task_new(Object *source,
+                      QIOTaskFunc func,
+                      gpointer opaque,
+                      GDestroyNotify destroy);
+
+/**
+ * qio_task_run_in_thread:
+ * @task: the task struct
+ * @worker: the function to invoke in a thread
+ * @opaque: opaque data to pass to @worker
+ * @destroy: function to free @opaque
+ *
+ * Run a task in a background thread. If @worker
+ * returns 0 it will call qio_task_complete() in
+ * the main event thread context. If @worker
+ * returns -1 it will call qio_task_abort() in
+ * the main event thread context.
+ */
+void qio_task_run_in_thread(QIOTask *task,
+                            QIOTaskWorker worker,
+                            gpointer opaque,
+                            GDestroyNotify destroy);
+
+/**
+ * qio_task_complete:
+ * @task: the task struct
+ *
+ * Mark the operation as succesfully completed
+ * and free the memory for @task.
+ */
+void qio_task_complete(QIOTask *task);
+
+/**
+ * qio_task_abort:
+ * @task: the task struct
+ * @err: the error to record for the operation
+ *
+ * Mark the operation as failed, with @err providing
+ * details about the failure. The @err may be freed
+ * afer the function returns, as the notification
+ * callback is invoked synchronously. The @task will
+ * be freed when this call completes.
+ */
+void qio_task_abort(QIOTask *task,
+                    Error *err);
+
+
+/**
+ * qio_task_get_source:
+ * @task: the task struct
+ *
+ * Get the source object associated with the background
+ * task. This returns a new reference to the object,
+ * which the caller must released with object_unref()
+ * when no longer required.
+ *
+ * Returns: the source object
+ */
+Object *qio_task_get_source(QIOTask *task);
+
+#endif /* QIO_TASK_H__ */
diff --git a/io/Makefile.objs b/io/Makefile.objs
index 593ed9e..bef2ca1 100644
--- a/io/Makefile.objs
+++ b/io/Makefile.objs
@@ -1,3 +1,4 @@
 io-obj-y = buffer.o
+io-obj-y += task.o
 io-obj-y += channel.o
 io-obj-y += channel-watch.o
diff --git a/io/task.c b/io/task.c
new file mode 100644
index 0000000..3127fca
--- /dev/null
+++ b/io/task.c
@@ -0,0 +1,159 @@
+/*
+ * QEMU I/O task
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "io/task.h"
+#include "qemu/thread.h"
+#include "trace.h"
+
+struct QIOTask {
+    Object *source;
+    QIOTaskFunc func;
+    gpointer opaque;
+    GDestroyNotify destroy;
+};
+
+
+QIOTask *qio_task_new(Object *source,
+                      QIOTaskFunc func,
+                      gpointer opaque,
+                      GDestroyNotify destroy)
+{
+    QIOTask *task;
+
+    task = g_new0(QIOTask, 1);
+
+    task->source = source;
+    object_ref(source);
+    task->func = func;
+    task->opaque = opaque;
+    task->destroy = destroy;
+
+    trace_qio_task_new(task, source, func, opaque);
+
+    return task;
+}
+
+static void qio_task_free(QIOTask *task)
+{
+    if (task->destroy) {
+        task->destroy(task->opaque);
+    }
+    object_unref(task->source);
+
+    g_free(task);
+}
+
+
+struct QIOTaskThreadData {
+    QIOTask *task;
+    QIOTaskWorker worker;
+    gpointer opaque;
+    GDestroyNotify destroy;
+    Error *err;
+    int ret;
+};
+
+
+static gboolean gio_task_thread_result(gpointer opaque)
+{
+    struct QIOTaskThreadData *data = opaque;
+
+    trace_qio_task_thread_result(data->task);
+    if (data->ret == 0) {
+        qio_task_complete(data->task);
+    } else {
+        qio_task_abort(data->task, data->err);
+    }
+
+    error_free(data->err);
+    if (data->destroy) {
+        data->destroy(data->opaque);
+    }
+
+    g_free(data);
+
+    return FALSE;
+}
+
+
+static gpointer qio_task_thread_worker(gpointer opaque)
+{
+    struct QIOTaskThreadData *data = opaque;
+
+    trace_qio_task_thread_run(data->task);
+    data->ret = data->worker(data->task, &data->err, data->opaque);
+    if (data->ret < 0 && data->err == NULL) {
+        error_setg(&data->err, "Task worker failed but did not set an error");
+    }
+
+    /* We're running in the background thread, and must only
+     * ever report the task results in the main event loop
+     * thread. So we schedule an idle callback to report
+     * the worker results
+     */
+    trace_qio_task_thread_exit(data->task);
+    g_idle_add(gio_task_thread_result, data);
+    return NULL;
+}
+
+
+void qio_task_run_in_thread(QIOTask *task,
+                            QIOTaskWorker worker,
+                            gpointer opaque,
+                            GDestroyNotify destroy)
+{
+    struct QIOTaskThreadData *data = g_new0(struct QIOTaskThreadData, 1);
+    QemuThread thread;
+
+    data->task = task;
+    data->worker = worker;
+    data->opaque = opaque;
+    data->destroy = destroy;
+
+    trace_qio_task_thread_start(task, worker, opaque);
+    qemu_thread_create(&thread,
+                       "io-task-worker",
+                       qio_task_thread_worker,
+                       data,
+                       QEMU_THREAD_DETACHED);
+}
+
+
+void qio_task_complete(QIOTask *task)
+{
+    task->func(task->source, NULL, task->opaque);
+    trace_qio_task_complete(task);
+    qio_task_free(task);
+}
+
+void qio_task_abort(QIOTask *task,
+                    Error *err)
+{
+    task->func(task->source, err, task->opaque);
+    trace_qio_task_abort(task);
+    qio_task_free(task);
+}
+
+
+Object *qio_task_get_source(QIOTask *task)
+{
+    object_ref(task->source);
+    return task->source;
+}
diff --git a/tests/.gitignore b/tests/.gitignore
index 2c5e2c3..6269480 100644
--- a/tests/.gitignore
+++ b/tests/.gitignore
@@ -23,6 +23,7 @@ test-cutils
 test-hbitmap
 test-int128
 test-iov
+test-io-task
 test-mul64
 test-opts-visitor
 test-qapi-event.[ch]
diff --git a/tests/Makefile b/tests/Makefile
index 7c6025a..4e2731c 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -80,6 +80,7 @@ check-unit-$(CONFIG_GNUTLS_HASH) += tests/test-crypto-hash$(EXESUF)
 check-unit-y += tests/test-crypto-cipher$(EXESUF)
 check-unit-$(CONFIG_GNUTLS) += tests/test-crypto-tlscredsx509$(EXESUF)
 check-unit-$(CONFIG_GNUTLS) += tests/test-crypto-tlssession$(EXESUF)
+check-unit-y += tests/test-io-task$(EXESUF)
 
 check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
 
@@ -292,6 +293,7 @@ test-qapi-obj-y = tests/test-qapi-visit.o tests/test-qapi-types.o \
 	$(test-qom-obj-y)
 test-crypto-obj-y = $(crypto-obj-y) $(test-qom-obj-y)
 test-block-obj-y = $(block-obj-y) $(test-crypto-obj-y)
+test-io-obj-y = $(io-obj-y) $(test-crypto-obj-y)
 
 tests/check-qint$(EXESUF): tests/check-qint.o $(test-util-obj-y)
 tests/check-qstring$(EXESUF): tests/check-qstring.o $(test-util-obj-y)
@@ -364,6 +366,7 @@ tests/test-crypto-tlscredsx509$(EXESUF): tests/test-crypto-tlscredsx509.o \
 	tests/crypto-tls-x509-helpers.o tests/pkix_asn1_tab.o $(test-crypto-obj-y)
 tests/test-crypto-tlssession$(EXESUF): tests/test-crypto-tlssession.o \
 	tests/crypto-tls-x509-helpers.o tests/pkix_asn1_tab.o $(test-crypto-obj-y)
+tests/test-io-task$(EXESUF): tests/test-io-task.o $(test-io-obj-y)
 
 libqos-obj-y = tests/libqos/pci.o tests/libqos/fw_cfg.o tests/libqos/malloc.o
 libqos-obj-y += tests/libqos/i2c.o tests/libqos/libqos.o
diff --git a/tests/test-io-task.c b/tests/test-io-task.c
new file mode 100644
index 0000000..dc2f5c2
--- /dev/null
+++ b/tests/test-io-task.c
@@ -0,0 +1,276 @@
+/*
+ * QEMU Crypto hash algorithms
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include <glib.h>
+
+#include "io/task.h"
+
+#define TYPE_DUMMY "qemu:dummy"
+
+typedef struct DummyObject DummyObject;
+typedef struct DummyObjectClass DummyObjectClass;
+
+struct DummyObject {
+    Object parent;
+};
+
+struct DummyObjectClass {
+    ObjectClass parent;
+};
+
+static const TypeInfo dummy_info = {
+    .parent = TYPE_OBJECT,
+    .name = TYPE_DUMMY,
+    .instance_size = sizeof(DummyObject),
+    .class_size = sizeof(DummyObjectClass),
+};
+
+struct TestTaskData {
+    Object *source;
+    Error *err;
+    bool freed;
+};
+
+
+static void task_callback(Object *source,
+                          Error *err,
+                          gpointer opaque)
+{
+    struct TestTaskData *data = opaque;
+
+    data->source = source;
+    data->err = err;
+}
+
+
+static void test_task_complete(void)
+{
+    QIOTask *task;
+    Object *obj = object_new(TYPE_DUMMY);
+    Object *src;
+    struct TestTaskData data = { NULL, NULL, false };
+
+    task = qio_task_new(obj, task_callback, &data, NULL);
+    src = qio_task_get_source(task);
+
+    qio_task_complete(task);
+
+    g_assert(obj == src);
+
+    object_unref(obj);
+    object_unref(src);
+
+    g_assert(data.source == obj);
+    g_assert(data.err == NULL);
+    g_assert(data.freed == false);
+}
+
+
+static void task_data_free(gpointer opaque)
+{
+    struct TestTaskData *data = opaque;
+
+    data->freed = true;
+}
+
+
+static void test_task_data_free(void)
+{
+    QIOTask *task;
+    Object *obj = object_new(TYPE_DUMMY);
+    struct TestTaskData data = { NULL, NULL, false };
+
+    task = qio_task_new(obj, task_callback, &data, task_data_free);
+
+    qio_task_complete(task);
+
+    object_unref(obj);
+
+    g_assert(data.source == obj);
+    g_assert(data.err == NULL);
+    g_assert(data.freed == true);
+}
+
+
+static void test_task_error(void)
+{
+    QIOTask *task;
+    Object *obj = object_new(TYPE_DUMMY);
+    struct TestTaskData data = { NULL, NULL, false };
+    Error *err = NULL;
+
+    task = qio_task_new(obj, task_callback, &data, NULL);
+
+    error_setg(&err, "Some error");
+
+    qio_task_abort(task, err);
+
+    error_free(err);
+    object_unref(obj);
+
+    g_assert(data.source == obj);
+    g_assert(data.err == err);
+    g_assert(data.freed == false);
+
+}
+
+
+struct TestThreadWorkerData {
+    Object *source;
+    Error *err;
+    bool fail;
+    GThread *worker;
+    GThread *complete;
+    GMainLoop *loop;
+};
+
+static int test_task_thread_worker(QIOTask *task,
+                                   Error **errp,
+                                   gpointer opaque)
+{
+    struct TestThreadWorkerData *data = opaque;
+
+    data->worker = g_thread_self();
+    g_thread_ref(data->worker);
+
+    if (data->fail) {
+        error_setg(errp, "Testing fail");
+        return -1;
+    }
+
+    return 0;
+}
+
+
+static void test_task_thread_callback(Object *source,
+                                      Error *err,
+                                      gpointer opaque)
+{
+    struct TestThreadWorkerData *data = opaque;
+
+    data->source = source;
+    data->err = err;
+
+    data->complete = g_thread_self();
+    g_thread_ref(data->complete);
+
+    g_main_loop_quit(data->loop);
+}
+
+
+static void test_task_thread_complete(void)
+{
+    QIOTask *task;
+    Object *obj = object_new(TYPE_DUMMY);
+    struct TestThreadWorkerData data = { 0 };
+    GThread *self;
+
+    data.loop = g_main_loop_new(g_main_context_default(),
+                                TRUE);
+
+    task = qio_task_new(obj,
+                        test_task_thread_callback,
+                        &data,
+                        NULL);
+
+    qio_task_run_in_thread(task,
+                           test_task_thread_worker,
+                           &data,
+                           NULL);
+
+    g_main_loop_run(data.loop);
+
+    g_main_loop_unref(data.loop);
+    object_unref(obj);
+
+    g_assert(data.source == obj);
+    g_assert(data.err == NULL);
+
+    self = g_thread_self();
+
+    /* Make sure the test_task_thread_worker actually got
+     * run in a different thread */
+    g_assert(data.worker != self);
+
+    /* And that the test_task_thread_callback got rnu in
+     * the main loop thread (ie this one) */
+    g_assert(data.complete == self);
+
+    g_thread_unref(data.worker);
+    g_thread_unref(data.complete);
+}
+
+
+static void test_task_thread_error(void)
+{
+    QIOTask *task;
+    Object *obj = object_new(TYPE_DUMMY);
+    struct TestThreadWorkerData data = { 0 };
+    GThread *self;
+
+    data.loop = g_main_loop_new(g_main_context_default(),
+                                TRUE);
+    data.fail = true;
+
+    task = qio_task_new(obj,
+                        test_task_thread_callback,
+                        &data,
+                        NULL);
+
+    qio_task_run_in_thread(task,
+                           test_task_thread_worker,
+                           &data,
+                           NULL);
+
+    g_main_loop_run(data.loop);
+
+    g_main_loop_unref(data.loop);
+    object_unref(obj);
+
+    g_assert(data.source == obj);
+    g_assert(data.err != NULL);
+
+    self = g_thread_self();
+
+    /* Make sure the test_task_thread_worker actually got
+     * run in a different thread */
+    g_assert(data.worker != self);
+
+    /* And that the test_task_thread_callback got rnu in
+     * the main loop thread (ie this one) */
+    g_assert(data.complete == self);
+
+    g_thread_unref(data.worker);
+    g_thread_unref(data.complete);
+}
+
+
+int main(int argc, char **argv)
+{
+    g_test_init(&argc, &argv, NULL);
+    module_call_init(MODULE_INIT_QOM);
+    type_register_static(&dummy_info);
+    g_test_add_func("/crypto/task/complete", test_task_complete);
+    g_test_add_func("/crypto/task/datafree", test_task_data_free);
+    g_test_add_func("/crypto/task/error", test_task_error);
+    g_test_add_func("/crypto/task/thread_complete", test_task_thread_complete);
+    g_test_add_func("/crypto/task/thread_error", test_task_thread_error);
+    return g_test_run();
+}
diff --git a/trace-events b/trace-events
index 88a2f14..987313a 100644
--- a/trace-events
+++ b/trace-events
@@ -1685,3 +1685,12 @@ qcrypto_tls_creds_x509_load_cert_list(void *creds, const char *file) "TLS creds
 
 # crypto/tlssession.c
 qcrypto_tls_session_new(void *session, void *creds, const char *hostname, const char *aclname, int endpoint) "TLS session new session=%p creds=%p hostname=%s aclname=%s endpoint=%d"
+
+# io/task.c
+qio_task_new(void *task, void *source, void *func, void *opaque) "Task new task=%p source=%p func=%p opaque=%p"
+qio_task_complete(void *task) "Task complete task=%p"
+qio_task_abort(void *task) "Task abort task=%p"
+qio_task_thread_start(void *task, void *worker, void *opaque) "Task thread start task=%p worker=%p opaque=%p"
+qio_task_thread_run(void *task) "Task thread run task=%p"
+qio_task_thread_exit(void *task) "Task thread exit task=%p"
+qio_task_thread_result(void *task) "Task thread result task=%p"
-- 
2.4.3

^ permalink raw reply related	[flat|nested] 32+ messages in thread

* [Qemu-devel] [PATCH v1 11/16] io: add QIOChannelSocket class
  2015-09-18 13:18 [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework Daniel P. Berrange
                   ` (9 preceding siblings ...)
  2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 10/16] io: add QIOTask class for async operations Daniel P. Berrange
@ 2015-09-18 13:19 ` Daniel P. Berrange
  2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 12/16] io: add QIOChannelFile class Daniel P. Berrange
                   ` (5 subsequent siblings)
  16 siblings, 0 replies; 32+ messages in thread
From: Daniel P. Berrange @ 2015-09-18 13:19 UTC (permalink / raw)
  To: qemu-devel; +Cc: Paolo Bonzini, Dr. David Alan Gilbert, Gerd Hoffmann

Implement a QIOChannel subclass that supports sockets I/O.
The implementation is able to manage a single socket file
descriptor, whether a TCP/UNIX listener, TCP/UNIX connection,
or a UDP datagram. It provides APIs which can listen and
connect either asynchronously or synchronously. Since there
is no asynchronous DNS lookup API available, it uses the
QIOTask helper for spawning a background thread to ensure
non-blocking operation.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
 configure                      |  11 +
 include/io/channel-socket.h    | 244 +++++++++++++
 io/Makefile.objs               |   1 +
 io/channel-socket.c            | 763 +++++++++++++++++++++++++++++++++++++++++
 scripts/create_config          |   9 +
 tests/.gitignore               |   1 +
 tests/Makefile                 |   3 +
 tests/io-channel-helpers.c     | 222 ++++++++++++
 tests/io-channel-helpers.h     |  30 ++
 tests/test-io-channel-socket.c | 367 ++++++++++++++++++++
 trace-events                   |  19 +
 11 files changed, 1670 insertions(+)
 create mode 100644 include/io/channel-socket.h
 create mode 100644 io/channel-socket.c
 create mode 100644 tests/io-channel-helpers.c
 create mode 100644 tests/io-channel-helpers.h
 create mode 100644 tests/test-io-channel-socket.c

diff --git a/configure b/configure
index 52f5b79..145b893 100755
--- a/configure
+++ b/configure
@@ -2262,6 +2262,14 @@ fi
 
 
 ##########################################
+# getifaddrs (for tests/test-io-channel-socket )
+
+have_ifaddrs_h=yes
+if ! check_include "ifaddrs.h" ; then
+  have_ifaddrs_h=no
+fi
+
+##########################################
 # VTE probe
 
 if test "$vte" != "no"; then
@@ -4934,6 +4942,9 @@ fi
 if test "$tasn1" = "yes" ; then
   echo "CONFIG_TASN1=y" >> $config_host_mak
 fi
+if test "$have_ifaddrs_h" = "yes" ; then
+    echo "HAVE_IFADDRS_H=y" >> $config_host_mak
+fi
 if test "$vte" = "yes" ; then
   echo "CONFIG_VTE=y" >> $config_host_mak
   echo "VTE_CFLAGS=$vte_cflags" >> $config_host_mak
diff --git a/include/io/channel-socket.h b/include/io/channel-socket.h
new file mode 100644
index 0000000..0719757
--- /dev/null
+++ b/include/io/channel-socket.h
@@ -0,0 +1,244 @@
+/*
+ * QEMU I/O channels sockets driver
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef QIO_CHANNEL_SOCKET_H__
+#define QIO_CHANNEL_SOCKET_H__
+
+#include "io/channel.h"
+#include "io/task.h"
+#include "qemu/sockets.h"
+
+#define TYPE_QIO_CHANNEL_SOCKET "qio-channel-socket"
+#define QIO_CHANNEL_SOCKET(obj)                                     \
+    OBJECT_CHECK(QIOChannelSocket, (obj), TYPE_QIO_CHANNEL_SOCKET)
+
+typedef struct QIOChannelSocket QIOChannelSocket;
+
+/**
+ * QIOChannelSocket:
+ *
+ * The QIOChannelSocket class provides a channel implementation
+ * that can transport data over a UNIX socket or TCP socket.
+ * Beyond the core channel API, it also provides functionality
+ * for accepting client connections, tuning some socket
+ * parameters and getting socket address strings.
+ */
+
+struct QIOChannelSocket {
+    QIOChannel parent;
+    int fd;
+    struct sockaddr_storage localAddr;
+    socklen_t localAddrLen;
+    struct sockaddr_storage remoteAddr;
+    socklen_t remoteAddrLen;
+};
+
+
+/**
+ * qio_channel_socket_new:
+ *
+ * Create a channel for performing I/O on a socket
+ * connection, that is initially closed. After
+ * creating the socket, it must be setup as a client
+ * connection or server.
+ *
+ * Returns: the socket channel object
+ */
+QIOChannelSocket *
+qio_channel_socket_new(void);
+
+/**
+ * qio_channel_socket_new_fd:
+ * @fd: the socket file descriptor
+ * @errp: pointer to an uninitialized error object
+ *
+ * Create a channel for performing I/O on the socket
+ * connection represented by the file descriptor @fd.
+ *
+ * Returns: the socket channel object, or NULL on error
+ */
+QIOChannelSocket *
+qio_channel_socket_new_fd(int fd,
+                          Error **errp);
+
+
+/**
+ * qio_channel_socket_connect_sync:
+ * @ioc: the socket channel object
+ * @addr: the address to connect to
+ * @errp: pointer to an uninitialized error object
+ *
+ * Attempt to connect to the address @addr. This method
+ * will run in the foreground so the caller will not regain
+ * execution control until the connection is established or
+ * an error occurs.
+ */
+int qio_channel_socket_connect_sync(QIOChannelSocket *ioc,
+                                    SocketAddress *addr,
+                                    Error **errp);
+
+/**
+ * qio_channel_socket_connect_async:
+ * @ioc: the socket channel object
+ * @addr: the address to connect to
+ * @callback: the function to invoke on completion
+ * @opaque: user data to pass to @callback
+ * @destroy: the function to free @opaque
+ *
+ * Attempt to connect to the address @addr. This method
+ * will run in the background so the caller will regain
+ * execution control immediately. The function @callback
+ * will be invoked on completion or failure.
+ */
+void qio_channel_socket_connect_async(QIOChannelSocket *ioc,
+                                      SocketAddress *addr,
+                                      QIOTaskFunc callback,
+                                      gpointer opaque,
+                                      GDestroyNotify destroy);
+
+
+/**
+ * qio_channel_socket_listen_sync:
+ * @ioc: the socket channel object
+ * @addr: the address to listen to
+ * @errp: pointer to an uninitialized error object
+ *
+ * Attempt to listen to the address @addr. This method
+ * will run in the foreground so the caller will not regain
+ * execution control until the connection is established or
+ * an error occurs.
+ */
+int qio_channel_socket_listen_sync(QIOChannelSocket *ioc,
+                                   SocketAddress *addr,
+                                   Error **errp);
+
+/**
+ * qio_channel_socket_listen_async:
+ * @ioc: the socket channel object
+ * @addr: the address to listen to
+ * @callback: the function to invoke on completion
+ * @opaque: user data to pass to @callback
+ * @destroy: the function to free @opaque
+ *
+ * Attempt to listen to the address @addr. This method
+ * will run in the background so the caller will regain
+ * execution control immediately. The function @callback
+ * will be invoked on completion or failure.
+ */
+void qio_channel_socket_listen_async(QIOChannelSocket *ioc,
+                                     SocketAddress *addr,
+                                     QIOTaskFunc callback,
+                                     gpointer opaque,
+                                     GDestroyNotify destroy);
+
+
+/**
+ * qio_channel_socket_dgram_sync:
+ * @ioc: the socket channel object
+ * @localAddr: the address to local bind address
+ * @remoteAddr: the address to remote peer address
+ * @errp: pointer to an uninitialized error object
+ *
+ * Attempt to initialize a datagram socket bound to
+ * @localAddr and communicating with peer @remoteAddr.
+ * This method will run in the foreground so the caller
+ * will not regain execution control until the socket
+ * is established or an error occurs.
+ */
+int qio_channel_socket_dgram_sync(QIOChannelSocket *ioc,
+                                  SocketAddress *localAddr,
+                                  SocketAddress *remoteAddr,
+                                  Error **errp);
+
+/**
+ * qio_channel_socket_dgram_async:
+ * @ioc: the socket channel object
+ * @localAddr: the address to local bind address
+ * @remoteAddr: the address to remote peer address
+ * @callback: the function to invoke on completion
+ * @opaque: user data to pass to @callback
+ * @destroy: the function to free @opaque
+ *
+ * Attempt to initialize a datagram socket bound to
+ * @localAddr and communicating with peer @remoteAddr.
+ * This method will run in the background so the caller
+ * will regain execution control immediately. The function
+ * @callback will be invoked on completion or failure.
+ */
+void qio_channel_socket_dgram_async(QIOChannelSocket *ioc,
+                                    SocketAddress *localAddr,
+                                    SocketAddress *remoteAddr,
+                                    QIOTaskFunc callback,
+                                    gpointer opaque,
+                                    GDestroyNotify destroy);
+
+
+/**
+ * qio_channel_socket_get_local_address:
+ * @ioc: the socket channel object
+ * @errp: pointer to an uninitialized error object
+ *
+ * Get the string representation of the local socket
+ * address. A pointer to the allocated address information
+ * struct will be returned, which the caller is required to
+ * release with a call qapi_free_SocketAddress when no
+ * longer required.
+ *
+ * Returns: 0 on success, -1 on error
+ */
+SocketAddress *
+qio_channel_socket_get_local_address(QIOChannelSocket *ioc,
+                                     Error **errp);
+
+/**
+ * qio_channel_socket_get_remote_address:
+ * @ioc: the socket channel object
+ * @errp: pointer to an uninitialized error object
+ *
+ * Get the string representation of the local socket
+ * address. A pointer to the allocated address information
+ * struct will be returned, which the caller is required to
+ * release with a call qapi_free_SocketAddress when no
+ * longer required.
+ *
+ * Returns: the socket address struct, or NULL on error
+ */
+SocketAddress *
+qio_channel_socket_get_remote_address(QIOChannelSocket *ioc,
+                                      Error **errp);
+
+
+/**
+ * qio_channel_socket_accept:
+ * @ioc: the socket channel object
+ * @errp: pointer to an uninitialized error object
+ *
+ * If the socket represents a server, then this accepts
+ * a new client connection. The returned channel will
+ * represent the connected client socket.
+ *
+ * Returns: the new client channel, or NULL on error
+ */
+QIOChannelSocket *
+qio_channel_socket_accept(QIOChannelSocket *ioc,
+                          Error **errp);
+
+
+#endif /* QIO_CHANNEL_SOCKET_H__ */
diff --git a/io/Makefile.objs b/io/Makefile.objs
index bef2ca1..f12ec26 100644
--- a/io/Makefile.objs
+++ b/io/Makefile.objs
@@ -2,3 +2,4 @@ io-obj-y = buffer.o
 io-obj-y += task.o
 io-obj-y += channel.o
 io-obj-y += channel-watch.o
+io-obj-y += channel-socket.o
diff --git a/io/channel-socket.c b/io/channel-socket.c
new file mode 100644
index 0000000..b2dbfe5
--- /dev/null
+++ b/io/channel-socket.c
@@ -0,0 +1,763 @@
+/*
+ * QEMU I/O channels sockets driver
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "io/channel-socket.h"
+#include "io/channel-watch.h"
+#include "trace.h"
+
+#define SOCKET_MAX_FDS 16
+
+SocketAddress *
+qio_channel_socket_get_local_address(QIOChannelSocket *ioc,
+                                     Error **errp)
+{
+    return socket_local_address(ioc->fd, errp);
+}
+
+SocketAddress *
+qio_channel_socket_get_remote_address(QIOChannelSocket *ioc,
+                                      Error **errp)
+{
+    return socket_remote_address(ioc->fd, errp);
+}
+
+QIOChannelSocket *
+qio_channel_socket_new(void)
+{
+    QIOChannelSocket *ioc;
+
+    ioc = QIO_CHANNEL_SOCKET(object_new(TYPE_QIO_CHANNEL_SOCKET));
+    ioc->fd = -1;
+
+    trace_qio_channel_socket_new(ioc);
+
+    return ioc;
+}
+
+
+static int
+qio_channel_socket_set_fd(QIOChannelSocket *ioc,
+                          int fd,
+                          Error **errp)
+{
+    if (ioc->fd != -1) {
+        error_setg(errp, "Socket is already open");
+        return -1;
+    }
+
+    ioc->fd = fd;
+    ioc->remoteAddrLen = sizeof(ioc->remoteAddr);
+    ioc->localAddrLen = sizeof(ioc->localAddr);
+
+    if (getpeername(fd, (struct sockaddr *)&ioc->remoteAddr,
+                    &ioc->remoteAddrLen) < 0) {
+        if (socket_error() == ENOTCONN) {
+            memset(&ioc->remoteAddr, 0, sizeof(ioc->remoteAddr));
+            ioc->remoteAddrLen = sizeof(ioc->remoteAddr);
+        } else {
+            error_setg_errno(errp, socket_error(),
+                             "Unable to query remote socket address");
+            goto error;
+        }
+    }
+
+    if (getsockname(fd, (struct sockaddr *)&ioc->localAddr,
+                    &ioc->localAddrLen) < 0) {
+        error_setg_errno(errp, socket_error(),
+                         "Unable to query local socket address");
+        goto error;
+    }
+
+    return 0;
+
+ error:
+    ioc->fd = -1; /* Let the caller close FD on failure */
+    return -1;
+}
+
+QIOChannelSocket *
+qio_channel_socket_new_fd(int fd,
+                          Error **errp)
+{
+    QIOChannelSocket *ioc;
+
+    ioc = qio_channel_socket_new();
+    if (qio_channel_socket_set_fd(ioc, fd, errp) < 0) {
+        object_unref(OBJECT(ioc));
+        return NULL;
+    }
+
+    trace_qio_channel_socket_new_fd(ioc, fd);
+
+    return ioc;
+}
+
+
+int qio_channel_socket_connect_sync(QIOChannelSocket *ioc,
+                                    SocketAddress *addr,
+                                    Error **errp)
+{
+    int fd;
+
+    trace_qio_channel_socket_connect_sync(ioc, addr);
+    fd = socket_connect(addr, errp, NULL, NULL);
+    if (fd < 0) {
+        trace_qio_channel_socket_connect_fail(ioc);
+        return -1;
+    }
+
+    trace_qio_channel_socket_connect_complete(ioc, fd);
+    if (qio_channel_socket_set_fd(ioc, fd, errp) < 0) {
+        close(fd);
+        return -1;
+    }
+
+    return 0;
+}
+
+
+static int qio_channel_socket_connect_worker(QIOTask *task,
+                                             Error **errp,
+                                             gpointer opaque)
+{
+    QIOChannelSocket *ioc = QIO_CHANNEL_SOCKET(qio_task_get_source(task));
+    SocketAddress *addr = opaque;
+
+    return qio_channel_socket_connect_sync(ioc,
+                                           addr,
+                                           errp);
+}
+
+
+void qio_channel_socket_connect_async(QIOChannelSocket *ioc,
+                                      SocketAddress *addr,
+                                      QIOTaskFunc callback,
+                                      gpointer opaque,
+                                      GDestroyNotify destroy)
+{
+    QIOTask *task = qio_task_new(
+        OBJECT(ioc), callback, opaque, destroy);
+    SocketAddress *addrCopy;
+
+    qapi_copy_SocketAddress(&addrCopy, addr);
+
+    /* socket_connect() does a non-blocking connect(), but it
+     * still blocks in DNS lookups, so we must use a thread */
+    trace_qio_channel_socket_connect_async(ioc, addr);
+    qio_task_run_in_thread(task,
+                           qio_channel_socket_connect_worker,
+                           addrCopy,
+                           (GDestroyNotify)qapi_free_SocketAddress);
+}
+
+
+int qio_channel_socket_listen_sync(QIOChannelSocket *ioc,
+                                   SocketAddress *addr,
+                                   Error **errp)
+{
+    int fd;
+
+    trace_qio_channel_socket_listen_sync(ioc, addr);
+    fd = socket_listen(addr, errp);
+    if (fd < 0) {
+        trace_qio_channel_socket_listen_fail(ioc);
+        return -1;
+    }
+
+    trace_qio_channel_socket_listen_complete(ioc, fd);
+    if (qio_channel_socket_set_fd(ioc, fd, errp) < 0) {
+        close(fd);
+        return -1;
+    }
+
+    return 0;
+}
+
+
+static int qio_channel_socket_listen_worker(QIOTask *task,
+                                            Error **errp,
+                                            gpointer opaque)
+{
+    QIOChannelSocket *ioc = QIO_CHANNEL_SOCKET(qio_task_get_source(task));
+    SocketAddress *addr = opaque;
+
+    return qio_channel_socket_listen_sync(ioc,
+                                          addr,
+                                          errp);
+}
+
+
+void qio_channel_socket_listen_async(QIOChannelSocket *ioc,
+                                     SocketAddress *addr,
+                                     QIOTaskFunc callback,
+                                     gpointer opaque,
+                                     GDestroyNotify destroy)
+{
+    QIOTask *task = qio_task_new(
+        OBJECT(ioc), callback, opaque, destroy);
+    SocketAddress *addrCopy;
+
+    qapi_copy_SocketAddress(&addrCopy, addr);
+
+    /* socket_listen() blocks in DNS lookups, so we must use a thread */
+    trace_qio_channel_socket_listen_async(ioc, addr);
+    qio_task_run_in_thread(task,
+                           qio_channel_socket_listen_worker,
+                           addrCopy,
+                           (GDestroyNotify)qapi_free_SocketAddress);
+}
+
+
+int qio_channel_socket_dgram_sync(QIOChannelSocket *ioc,
+                                  SocketAddress *localAddr,
+                                  SocketAddress *remoteAddr,
+                                  Error **errp)
+{
+    int fd;
+
+    trace_qio_channel_socket_dgram_sync(ioc, localAddr, remoteAddr);
+    fd = socket_dgram(localAddr, remoteAddr, errp);
+    if (fd < 0) {
+        trace_qio_channel_socket_dgram_fail(ioc);
+        return -1;
+    }
+
+    trace_qio_channel_socket_dgram_complete(ioc, fd);
+    if (qio_channel_socket_set_fd(ioc, fd, errp) < 0) {
+        close(fd);
+        return -1;
+    }
+
+    return 0;
+}
+
+
+struct QIOChannelSocketDGramWorkerData {
+    SocketAddress *localAddr;
+    SocketAddress *remoteAddr;
+};
+
+
+static void qio_channel_socket_dgram_worker_free(gpointer opaque)
+{
+    struct QIOChannelSocketDGramWorkerData *data = opaque;
+    qapi_free_SocketAddress(data->localAddr);
+    qapi_free_SocketAddress(data->remoteAddr);
+    g_free(data);
+}
+
+static int qio_channel_socket_dgram_worker(QIOTask *task,
+                                           Error **errp,
+                                           gpointer opaque)
+{
+    QIOChannelSocket *ioc = QIO_CHANNEL_SOCKET(qio_task_get_source(task));
+    struct QIOChannelSocketDGramWorkerData *data = opaque;
+
+    /* socket_dgram() blocks in DNS lookups, so we must use a thread */
+    return qio_channel_socket_dgram_sync(ioc,
+                                         data->localAddr,
+                                         data->remoteAddr,
+                                         errp);
+}
+
+
+void qio_channel_socket_dgram_async(QIOChannelSocket *ioc,
+                                    SocketAddress *localAddr,
+                                    SocketAddress *remoteAddr,
+                                    QIOTaskFunc callback,
+                                    gpointer opaque,
+                                    GDestroyNotify destroy)
+{
+    QIOTask *task = qio_task_new(
+        OBJECT(ioc), callback, opaque, destroy);
+    struct QIOChannelSocketDGramWorkerData *data = g_new0(
+        struct QIOChannelSocketDGramWorkerData, 1);
+
+    qapi_copy_SocketAddress(&data->localAddr, localAddr);
+    qapi_copy_SocketAddress(&data->remoteAddr, remoteAddr);
+
+    trace_qio_channel_socket_dgram_async(ioc, localAddr, remoteAddr);
+    qio_task_run_in_thread(task,
+                           qio_channel_socket_dgram_worker,
+                           data,
+                           qio_channel_socket_dgram_worker_free);
+}
+
+
+QIOChannelSocket *
+qio_channel_socket_accept(QIOChannelSocket *ioc,
+                          Error **errp)
+{
+    QIOChannelSocket *cioc;
+
+    cioc = QIO_CHANNEL_SOCKET(object_new(TYPE_QIO_CHANNEL_SOCKET));
+    cioc->fd = -1;
+    cioc->remoteAddrLen = sizeof(ioc->remoteAddr);
+    cioc->localAddrLen = sizeof(ioc->localAddr);
+
+ retry:
+    trace_qio_channel_socket_accept(ioc);
+    cioc->fd = accept(ioc->fd, (struct sockaddr *)&cioc->remoteAddr,
+                      &cioc->remoteAddrLen);
+    if (cioc->fd < 0) {
+        trace_qio_channel_socket_accept_fail(ioc);
+        if (socket_error() == EINTR) {
+            goto retry;
+        }
+        goto error;
+    }
+
+    if (getsockname(cioc->fd, (struct sockaddr *)&ioc->localAddr,
+                    &ioc->localAddrLen) < 0) {
+        error_setg_errno(errp, socket_error(),
+                         "Unable to query local socket address");
+        goto error;
+    }
+
+    trace_qio_channel_socket_accept_complete(ioc, cioc, cioc->fd);
+    return cioc;
+
+ error:
+    object_unref(OBJECT(cioc));
+    return NULL;
+}
+
+static void qio_channel_socket_init(Object *obj)
+{
+    QIOChannelSocket *ioc = QIO_CHANNEL_SOCKET(obj);
+    ioc->fd = -1;
+}
+
+static void qio_channel_socket_finalize(Object *obj)
+{
+    QIOChannelSocket *ioc = QIO_CHANNEL_SOCKET(obj);
+    if (ioc->fd != -1) {
+        close(ioc->fd);
+        ioc->fd = -1;
+    }
+}
+
+static bool qio_channel_socket_has_feature(QIOChannel *ioc,
+                                           QIOChannelFeature feature)
+{
+    QIOChannelSocket *sioc = QIO_CHANNEL_SOCKET(ioc);
+
+    switch (feature) {
+    case QIO_CHANNEL_FEATURE_FD_PASS:
+#ifndef WIN32
+        if (sioc->localAddr.ss_family == AF_UNIX) {
+            return true;
+        } else {
+            return false;
+        }
+#else
+        return false;
+#endif
+    case QIO_CHANNEL_FEATURE_SHUTDOWN:
+    case QIO_CHANNEL_FEATURE_CORK:
+    case QIO_CHANNEL_FEATURE_DELAY:
+        return true;
+    default:
+        return false;
+    }
+}
+
+
+#ifndef WIN32
+static void qio_channel_socket_copy_fds(struct msghdr *msg,
+                                        int **fds, size_t *nfds)
+{
+    struct cmsghdr *cmsg;
+
+    *nfds = 0;
+    *fds = NULL;
+
+    for (cmsg = CMSG_FIRSTHDR(msg); cmsg; cmsg = CMSG_NXTHDR(msg, cmsg)) {
+        int fd_size, i;
+        int gotfds;
+
+        if (cmsg->cmsg_len < CMSG_LEN(sizeof(int)) ||
+            cmsg->cmsg_level != SOL_SOCKET ||
+            cmsg->cmsg_type != SCM_RIGHTS) {
+            continue;
+        }
+
+        fd_size = cmsg->cmsg_len - CMSG_LEN(0);
+
+        if (!fd_size) {
+            continue;
+        }
+
+        gotfds = fd_size / sizeof(int);
+        *fds = g_renew(int, *fds, *nfds + gotfds);
+        memcpy(*fds + *nfds, CMSG_DATA(cmsg), fd_size);
+
+        for (i = 0; i < gotfds; i++) {
+            int fd = (*fds)[*nfds + i];
+            if (fd < 0) {
+                continue;
+            }
+
+            /* O_NONBLOCK is preserved across SCM_RIGHTS so reset it */
+            qemu_set_block(fd);
+
+#ifndef MSG_CMSG_CLOEXEC
+            qemu_set_cloexec(fd);
+#endif
+        }
+        *nfds += gotfds;
+    }
+}
+
+
+static ssize_t qio_channel_socket_readv(QIOChannel *ioc,
+                                        const struct iovec *iov,
+                                        size_t niov,
+                                        int **fds,
+                                        size_t *nfds,
+                                        Error **errp)
+{
+    QIOChannelSocket *sioc = QIO_CHANNEL_SOCKET(ioc);
+    ssize_t ret;
+    struct msghdr msg = { NULL, };
+    char control[CMSG_SPACE(sizeof(int) * SOCKET_MAX_FDS)];
+    int sflags = 0;
+
+#ifdef MSG_CMSG_CLOEXEC
+    sflags |= MSG_CMSG_CLOEXEC;
+#endif
+
+    msg.msg_iov = (struct iovec *)iov;
+    msg.msg_iovlen = niov;
+    if (fds && nfds) {
+        msg.msg_control = control;
+        msg.msg_controllen = sizeof(control);
+    }
+
+ retry:
+    ret = recvmsg(sioc->fd, &msg, sflags);
+    if (ret < 0) {
+        if (socket_error() == EAGAIN ||
+            socket_error() == EWOULDBLOCK) {
+            return QIO_CHANNEL_ERR_BLOCK;
+        }
+        if (socket_error() == EINTR) {
+            goto retry;
+        }
+
+        error_setg_errno(errp, socket_error(),
+                         "Unable to read from socket");
+        return -1;
+    }
+
+    if (fds && nfds) {
+        qio_channel_socket_copy_fds(&msg, fds, nfds);
+    }
+
+    return ret;
+}
+
+static ssize_t qio_channel_socket_writev(QIOChannel *ioc,
+                                         const struct iovec *iov,
+                                         size_t niov,
+                                         int *fds,
+                                         size_t nfds,
+                                         Error **errp)
+{
+    QIOChannelSocket *sioc = QIO_CHANNEL_SOCKET(ioc);
+    ssize_t ret;
+    struct msghdr msg = { NULL, };
+
+    msg.msg_iov = (struct iovec *)iov;
+    msg.msg_iovlen = niov;
+
+    if (nfds) {
+        char control[CMSG_SPACE(sizeof(int) * SOCKET_MAX_FDS)];
+        size_t fdsize = sizeof(int) * nfds;
+        struct cmsghdr *cmsg;
+
+        if (nfds > SOCKET_MAX_FDS) {
+            error_setg_errno(errp, -EINVAL,
+                             "Only %d FDs can be sent, got %zu",
+                             SOCKET_MAX_FDS, nfds);
+            return -1;
+        }
+
+        msg.msg_control = control;
+        msg.msg_controllen = CMSG_SPACE(sizeof(int) * nfds);
+
+        cmsg = CMSG_FIRSTHDR(&msg);
+        cmsg->cmsg_len = CMSG_LEN(fdsize);
+        cmsg->cmsg_level = SOL_SOCKET;
+        cmsg->cmsg_type = SCM_RIGHTS;
+        memcpy(CMSG_DATA(cmsg), fds, fdsize);
+    }
+
+ retry:
+    ret = sendmsg(sioc->fd, &msg, 0);
+    if (ret <= 0) {
+        if (socket_error() == EAGAIN ||
+            socket_error() == EWOULDBLOCK) {
+            return QIO_CHANNEL_ERR_BLOCK;
+        }
+        if (socket_error() == EINTR) {
+            goto retry;
+        }
+        error_setg_errno(errp, socket_error(),
+                         "Unable to write to socket");
+        return -1;
+    }
+    return ret;
+}
+#else /* WIN32 */
+static ssize_t qio_channel_socket_readv(QIOChannel *ioc,
+                                        const struct iovec *iov,
+                                        size_t niov,
+                                        int **fds,
+                                        size_t *nfds,
+                                        Error **errp)
+{
+    QIOChannelSocket *sioc = QIO_CHANNEL_SOCKET(ioc);
+    ssize_t done = 0;
+    ssize_t i;
+
+    if (fds || nfds) {
+        error_setg_errno(errp, EINVAL,
+                         "Channel does not support file descriptor passing");
+        return -1;
+    }
+
+    for (i = 0; i < niov; i++) {
+        ssize_t ret;
+    retry:
+        ret = recv(sioc->fd,
+                   iov[i].iov_base,
+                   iov[i].iov_len,
+                   0);
+        if (ret < 0) {
+            if (socket_error() == EAGAIN) {
+                if (done) {
+                    return done;
+                } else {
+                    return QIO_CHANNEL_ERR_BLOCK;
+                }
+            } else if (socket_error() == EINTR) {
+                goto retry;
+            } else {
+                error_setg_errno(errp, socket_error(),
+                                 "Unable to write to socket");
+                return -1;
+            }
+        }
+        done += ret;
+        if (ret < iov[i].iov_len) {
+            return done;
+        }
+    }
+
+    return done;
+}
+
+static ssize_t qio_channel_socket_writev(QIOChannel *ioc,
+                                         const struct iovec *iov,
+                                         size_t niov,
+                                         int *fds,
+                                         size_t nfds,
+                                         Error **errp)
+{
+    QIOChannelSocket *sioc = QIO_CHANNEL_SOCKET(ioc);
+    ssize_t done = 0;
+    ssize_t i;
+
+    if (fds || nfds) {
+        error_setg_errno(errp, EINVAL,
+                         "Channel does not support file descriptor passing");
+        return -1;
+    }
+
+    for (i = 0; i < niov; i++) {
+        ssize_t ret;
+    retry:
+        ret = send(sioc->fd,
+                   iov[i].iov_base,
+                   iov[i].iov_len,
+                   0);
+        if (ret < 0) {
+            if (socket_error() == EAGAIN) {
+                if (done) {
+                    return done;
+                } else {
+                    return QIO_CHANNEL_ERR_BLOCK;
+                }
+            } else if (socket_error() == EINTR) {
+                goto retry;
+            } else {
+                error_setg_errno(errp, socket_error(),
+                                 "Unable to write to socket");
+                return -1;
+            }
+        }
+        done += ret;
+        if (ret < iov[i].iov_len) {
+            return done;
+        }
+    }
+
+    return done;
+}
+#endif /* WIN32 */
+
+static int
+qio_channel_socket_set_blocking(QIOChannel *ioc,
+                                bool enabled,
+                                Error **errp)
+{
+    QIOChannelSocket *sioc = QIO_CHANNEL_SOCKET(ioc);
+
+    if (enabled) {
+        qemu_set_block(sioc->fd);
+    } else {
+        qemu_set_nonblock(sioc->fd);
+    }
+    return 0;
+}
+
+
+static int
+qio_channel_socket_set_delay(QIOChannel *ioc,
+                             bool enabled,
+                             Error **errp)
+{
+    QIOChannelSocket *sioc = QIO_CHANNEL_SOCKET(ioc);
+    int v = enabled ? 0 : 1;
+
+    if (qemu_setsockopt(sioc->fd,
+                        IPPROTO_TCP, TCP_NODELAY,
+                        &v, sizeof(v)) < 0) {
+        error_setg_errno(errp, socket_error(),
+                         "Unable to set TCP_NODELAY flag");
+        return -1;
+    }
+    return 0;
+}
+
+
+static int
+qio_channel_socket_set_cork(QIOChannel *ioc,
+                            bool enabled,
+                            Error **errp)
+{
+    QIOChannelSocket *sioc = QIO_CHANNEL_SOCKET(ioc);
+    int v = enabled ? 1 : 0;
+
+    if (socket_set_cork(sioc->fd, v) < 0) {
+        error_setg_errno(errp, socket_error(),
+                         "Unable to set TCP_CORK flag");
+        return -1;
+    }
+    return 0;
+}
+
+
+static int
+qio_channel_socket_close(QIOChannel *ioc,
+                         Error **errp)
+{
+    QIOChannelSocket *sioc = QIO_CHANNEL_SOCKET(ioc);
+
+    if (closesocket(sioc->fd) < 0) {
+        sioc->fd = -1;
+        error_setg_errno(errp, socket_error(),
+                         "Unable to close socket");
+        return -1;
+    }
+    sioc->fd = -1;
+    return 0;
+}
+
+static int
+qio_channel_socket_shutdown(QIOChannel *ioc,
+                            QIOChannelShutdown how,
+                            Error **errp)
+{
+    QIOChannelSocket *sioc = QIO_CHANNEL_SOCKET(ioc);
+    int sockhow;
+
+    switch (how) {
+    case QIO_CHANNEL_SHUTDOWN_READ:
+        sockhow = SHUT_RD;
+        break;
+    case QIO_CHANNEL_SHUTDOWN_WRITE:
+        sockhow = SHUT_WR;
+        break;
+    case QIO_CHANNEL_SHUTDOWN_BOTH:
+    default:
+        sockhow = SHUT_RDWR;
+        break;
+    }
+
+    if (shutdown(sioc->fd, sockhow) < 0) {
+        error_setg_errno(errp, socket_error(),
+                         "Unable to shutdown socket");
+        return -1;
+    }
+    return 0;
+}
+
+static GSource *qio_channel_socket_create_watch(QIOChannel *ioc,
+                                                GIOCondition condition)
+{
+    QIOChannelSocket *sioc = QIO_CHANNEL_SOCKET(ioc);
+    return qio_channel_create_fd_watch(ioc,
+                                       sioc->fd,
+                                       condition);
+}
+
+static void qio_channel_socket_class_init(ObjectClass *klass,
+                                          void *class_data G_GNUC_UNUSED)
+{
+    QIOChannelClass *ioc_klass = QIO_CHANNEL_CLASS(klass);
+
+    ioc_klass->io_has_feature = qio_channel_socket_has_feature;
+    ioc_klass->io_writev = qio_channel_socket_writev;
+    ioc_klass->io_readv = qio_channel_socket_readv;
+    ioc_klass->io_set_blocking = qio_channel_socket_set_blocking;
+    ioc_klass->io_close = qio_channel_socket_close;
+    ioc_klass->io_shutdown = qio_channel_socket_shutdown;
+    ioc_klass->io_set_cork = qio_channel_socket_set_cork;
+    ioc_klass->io_set_delay = qio_channel_socket_set_delay;
+    ioc_klass->io_create_watch = qio_channel_socket_create_watch;
+}
+
+static const TypeInfo qio_channel_socket_info = {
+    .parent = TYPE_QIO_CHANNEL,
+    .name = TYPE_QIO_CHANNEL_SOCKET,
+    .instance_size = sizeof(QIOChannelSocket),
+    .instance_init = qio_channel_socket_init,
+    .instance_finalize = qio_channel_socket_finalize,
+    .class_init = qio_channel_socket_class_init,
+};
+
+static void qio_channel_socket_register_types(void)
+{
+    type_register_static(&qio_channel_socket_info);
+}
+
+type_init(qio_channel_socket_register_types);
diff --git a/scripts/create_config b/scripts/create_config
index 546f889..9cb176f 100755
--- a/scripts/create_config
+++ b/scripts/create_config
@@ -61,6 +61,15 @@ case $line in
     value=${line#*=}
     echo "#define $name $value"
     ;;
+ HAVE_*=y) # configuration
+    name=${line%=*}
+    echo "#define $name 1"
+    ;;
+ HAVE_*=*) # configuration
+    name=${line%=*}
+    value=${line#*=}
+    echo "#define $name $value"
+    ;;
  ARCH=*) # configuration
     arch=${line#*=}
     arch_name=`echo $arch | LC_ALL=C tr '[a-z]' '[A-Z]'`
diff --git a/tests/.gitignore b/tests/.gitignore
index 6269480..ac891c6 100644
--- a/tests/.gitignore
+++ b/tests/.gitignore
@@ -23,6 +23,7 @@ test-cutils
 test-hbitmap
 test-int128
 test-iov
+test-io-channel-socket
 test-io-task
 test-mul64
 test-opts-visitor
diff --git a/tests/Makefile b/tests/Makefile
index 4e2731c..ca1846c 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -81,6 +81,7 @@ check-unit-y += tests/test-crypto-cipher$(EXESUF)
 check-unit-$(CONFIG_GNUTLS) += tests/test-crypto-tlscredsx509$(EXESUF)
 check-unit-$(CONFIG_GNUTLS) += tests/test-crypto-tlssession$(EXESUF)
 check-unit-y += tests/test-io-task$(EXESUF)
+check-unit-y += tests/test-io-channel-socket$(EXESUF)
 
 check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
 
@@ -367,6 +368,8 @@ tests/test-crypto-tlscredsx509$(EXESUF): tests/test-crypto-tlscredsx509.o \
 tests/test-crypto-tlssession$(EXESUF): tests/test-crypto-tlssession.o \
 	tests/crypto-tls-x509-helpers.o tests/pkix_asn1_tab.o $(test-crypto-obj-y)
 tests/test-io-task$(EXESUF): tests/test-io-task.o $(test-io-obj-y)
+tests/test-io-channel-socket$(EXESUF): tests/test-io-channel-socket.o \
+        tests/io-channel-helpers.o $(test-io-obj-y)
 
 libqos-obj-y = tests/libqos/pci.o tests/libqos/fw_cfg.o tests/libqos/malloc.o
 libqos-obj-y += tests/libqos/i2c.o tests/libqos/libqos.o
diff --git a/tests/io-channel-helpers.c b/tests/io-channel-helpers.c
new file mode 100644
index 0000000..c84b99f
--- /dev/null
+++ b/tests/io-channel-helpers.c
@@ -0,0 +1,222 @@
+/*
+ * QEMU I/O channel test helpers
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "io-channel-helpers.h"
+
+struct TestIOData {
+    QIOChannel *src;
+    QIOChannel *dst;
+    bool blocking;
+    size_t len;
+    size_t niov;
+    char *input;
+    struct iovec *inputv;
+    char *output;
+    struct iovec *outputv;
+    Error *writeerr;
+    Error *readerr;
+};
+
+
+static void test_skip_iovec(struct iovec **iov,
+                            size_t *niov,
+                            size_t skip,
+                            struct iovec *old)
+{
+    size_t offset = 0;
+    size_t i;
+
+    for (i = 0; i < *niov; i++) {
+        if (skip < (*iov)[i].iov_len) {
+            old->iov_len = (*iov)[i].iov_len;
+            old->iov_base = (*iov)[i].iov_base;
+
+            (*iov)[i].iov_len -= skip;
+            (*iov)[i].iov_base += skip;
+            break;
+        } else {
+            skip -= (*iov)[i].iov_len;
+
+            if (i == 0 && old->iov_base) {
+                (*iov)[i].iov_len = old->iov_len;
+                (*iov)[i].iov_base = old->iov_base;
+                old->iov_len = 0;
+                old->iov_base = NULL;
+            }
+
+            offset++;
+        }
+    }
+
+    *iov = *iov + offset;
+    *niov -= offset;
+}
+
+
+/* This thread sends all data using iovecs */
+static gpointer test_io_thread_writer(gpointer opaque)
+{
+    struct TestIOData *data = opaque;
+    struct iovec *iov = data->inputv;
+    size_t niov = data->niov;
+    struct iovec old = { 0 };
+
+    qio_channel_set_blocking(data->src, data->blocking, NULL);
+
+    while (niov) {
+        ssize_t ret;
+        ret = qio_channel_writev(data->src,
+                                 iov,
+                                 niov,
+                                 &data->writeerr);
+        if (ret == QIO_CHANNEL_ERR_BLOCK) {
+            if (data->blocking) {
+                error_setg(&data->writeerr,
+                           "Unexpected I/O blocking");
+                break;
+            } else {
+                qio_channel_wait(data->src,
+                                 G_IO_OUT);
+                continue;
+            }
+        } else if (ret < 0) {
+            break;
+        } else if (ret == 0) {
+            error_setg(&data->writeerr,
+                       "Unexpected zero length write");
+            break;
+        }
+
+        test_skip_iovec(&iov, &niov, ret, &old);
+    }
+
+    return NULL;
+}
+
+
+/* This thread receives all data using iovecs */
+static gpointer test_io_thread_reader(gpointer opaque)
+{
+    struct TestIOData *data = opaque;
+    struct iovec *iov = data->outputv;
+    size_t niov = data->niov;
+    struct iovec old = { 0 };
+
+    qio_channel_set_blocking(data->dst, data->blocking, NULL);
+
+    while (niov) {
+        ssize_t ret;
+
+        ret = qio_channel_readv(data->dst,
+                                iov,
+                                niov,
+                                &data->readerr);
+
+        if (ret == QIO_CHANNEL_ERR_BLOCK) {
+            if (data->blocking) {
+                error_setg(&data->writeerr,
+                           "Unexpected I/O blocking");
+                break;
+            } else {
+                qio_channel_wait(data->dst,
+                                 G_IO_IN);
+                continue;
+            }
+        } else if (ret < 0) {
+            break;
+        } else if (ret == 0) {
+            break;
+        }
+
+        test_skip_iovec(&iov, &niov, ret, &old);
+    }
+
+    return NULL;
+}
+
+
+static struct TestIOData *test_load_io_data(const char *filename)
+{
+    struct TestIOData *data = g_new0(struct TestIOData, 1);
+    GError *err = NULL;
+    size_t offset;
+    size_t i;
+
+#define IOVEC_LEN (64 * 1024)
+
+    if (!g_file_get_contents(filename, &data->input, &data->len, &err)) {
+        g_printerr("Unable to load file: %s\n", err->message);
+        abort();
+    }
+    data->len++; /* So that we send the trailing \0 too */
+    data->output = g_new0(gchar, data->len);
+
+    data->niov = (data->len / IOVEC_LEN) + 1;
+    data->inputv = g_new0(struct iovec, data->niov);
+    data->outputv = g_new0(struct iovec, data->niov);
+
+    for (i = 0, offset = 0; i < data->niov; i++, offset += IOVEC_LEN) {
+        data->inputv[i].iov_base = data->input + offset;
+        data->outputv[i].iov_base = data->output + offset;
+        if ((data->len - offset) > IOVEC_LEN) {
+            data->inputv[i].iov_len = IOVEC_LEN;
+            data->outputv[i].iov_len = IOVEC_LEN;
+        } else {
+            data->inputv[i].iov_len = data->len - offset;
+            data->outputv[i].iov_len = data->len - offset;
+        }
+    }
+
+    return data;
+}
+
+
+void test_io_channel_comms(bool blocking,
+                           QIOChannel *src,
+                           QIOChannel *dst)
+{
+    struct TestIOData *data;
+    GThread *reader, *writer;
+
+    data = test_load_io_data("libqemuutil.a");
+    data->src = src;
+    data->dst = dst;
+    data->blocking = blocking;
+
+    reader = g_thread_new("reader",
+                          test_io_thread_reader,
+                          data);
+    writer = g_thread_new("writer",
+                          test_io_thread_writer,
+                          data);
+
+    g_thread_join(reader);
+    g_thread_join(writer);
+
+    g_assert_cmpint(memcmp(data->input,
+                           data->output,
+                           data->len), ==, 0);
+    g_assert(data->readerr == NULL);
+    g_assert(data->writeerr == NULL);
+
+    g_free(data->input);
+    g_free(data->output);
+    g_free(data);
+}
diff --git a/tests/io-channel-helpers.h b/tests/io-channel-helpers.h
new file mode 100644
index 0000000..d0085a2
--- /dev/null
+++ b/tests/io-channel-helpers.h
@@ -0,0 +1,30 @@
+/*
+ * QEMU I/O channel test helpers
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "io/channel.h"
+
+#ifndef TEST_IO_CHANNEL_HELPERS
+#define TEST_IO_CHANNEL_HELPERS
+
+void test_io_channel_comms(bool blocking,
+                           QIOChannel *src,
+                           QIOChannel *dst);
+
+#endif /* TEST_IO_CHANNEL_HELPERS */
diff --git a/tests/test-io-channel-socket.c b/tests/test-io-channel-socket.c
new file mode 100644
index 0000000..5845a1e
--- /dev/null
+++ b/tests/test-io-channel-socket.c
@@ -0,0 +1,367 @@
+/*
+ * QEMU I/O channel sockets test
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "io/channel-socket.h"
+#include "io-channel-helpers.h"
+#ifdef HAVE_IFADDRS_H
+#include <ifaddrs.h>
+#endif
+
+static int check_protocol_support(bool *has_ipv4, bool *has_ipv6)
+{
+#ifdef HAVE_IFADDRS_H
+    struct ifaddrs *ifaddr = NULL, *ifa;
+    struct addrinfo hints = { 0 };
+    struct addrinfo *ai = NULL;
+    int gaierr;
+
+    *has_ipv4 = *has_ipv6 = false;
+
+    if (getifaddrs(&ifaddr) < 0) {
+        g_printerr("Failed to lookup interface addresses: %s\n",
+                   strerror(errno));
+        return -1;
+    }
+
+    for (ifa = ifaddr; ifa != NULL; ifa = ifa->ifa_next) {
+        if (!ifa->ifa_addr) {
+            continue;
+        }
+
+        if (ifa->ifa_addr->sa_family == AF_INET) {
+            *has_ipv4 = true;
+        }
+        if (ifa->ifa_addr->sa_family == AF_INET6) {
+            *has_ipv6 = true;
+        }
+    }
+
+    freeifaddrs(ifaddr);
+
+    hints.ai_flags = AI_PASSIVE | AI_ADDRCONFIG;
+    hints.ai_family = AF_INET6;
+    hints.ai_socktype = SOCK_STREAM;
+
+    gaierr = getaddrinfo("::1", NULL, &hints, &ai);
+    if (gaierr != 0) {
+        if (gaierr == EAI_ADDRFAMILY ||
+            gaierr == EAI_FAMILY ||
+            gaierr == EAI_NONAME) {
+            *has_ipv6 = false;
+        } else {
+            g_printerr("Failed to resolve ::1 address: %s\n",
+                       gai_strerror(gaierr));
+            return -1;
+        }
+    }
+
+    freeaddrinfo(ai);
+
+    return 0;
+#else
+    *has_ipv4 = *has_ipv6 = false;
+
+    return -1;
+#endif
+}
+
+
+static void test_io_channel_set_socket_bufs(QIOChannel *src,
+                                            QIOChannel *dst)
+{
+    int buflen = 64 * 1024;
+
+    /*
+     * Make the socket buffers small so that we see
+     * the effects of partial reads/writes
+     */
+    setsockopt(((QIOChannelSocket *)src)->fd,
+               SOL_SOCKET, SO_SNDBUF,
+               (char *)&buflen,
+               sizeof(buflen));
+
+    setsockopt(((QIOChannelSocket *)dst)->fd,
+               SOL_SOCKET, SO_SNDBUF,
+               (char *)&buflen,
+               sizeof(buflen));
+}
+
+
+static void test_io_channel_setup_sync(SocketAddress *listen_addr,
+                                       SocketAddress *connect_addr,
+                                       QIOChannel **src,
+                                       QIOChannel **dst)
+{
+    QIOChannelSocket *lioc;
+
+    lioc = qio_channel_socket_new();
+    qio_channel_socket_listen_sync(lioc, listen_addr, &error_abort);
+
+    if (listen_addr->kind == SOCKET_ADDRESS_KIND_INET) {
+        SocketAddress *laddr = qio_channel_socket_get_local_address(
+            lioc, &error_abort);
+
+        connect_addr->inet->port = g_strdup(laddr->inet->port);
+
+        qapi_free_SocketAddress(laddr);
+    }
+
+    *src = QIO_CHANNEL(qio_channel_socket_new());
+    qio_channel_socket_connect_sync(
+        QIO_CHANNEL_SOCKET(*src), connect_addr, &error_abort);
+    qio_channel_set_delay(*src, false, NULL);
+
+    *dst = QIO_CHANNEL(qio_channel_socket_accept(lioc, &error_abort));
+    g_assert(*dst);
+
+    test_io_channel_set_socket_bufs(*src, *dst);
+
+    object_unref(OBJECT(lioc));
+}
+
+
+struct TestIOChannelData {
+    bool err;
+    GMainLoop *loop;
+};
+
+
+static void test_io_channel_complete(Object *src,
+                                     Error *err,
+                                     gpointer opaque)
+{
+    struct TestIOChannelData *data = opaque;
+    data->err = err != NULL;
+    g_main_loop_quit(data->loop);
+}
+
+
+static void test_io_channel_setup_async(SocketAddress *listen_addr,
+                                        SocketAddress *connect_addr,
+                                        QIOChannel **src,
+                                        QIOChannel **dst)
+{
+    QIOChannelSocket *lioc;
+    struct TestIOChannelData data;
+
+    data.loop = g_main_loop_new(g_main_context_default(),
+                                TRUE);
+
+    lioc = qio_channel_socket_new();
+    qio_channel_socket_listen_async(
+        lioc, listen_addr,
+        test_io_channel_complete, &data, NULL);
+
+    g_main_loop_run(data.loop);
+
+    g_assert(!data.err);
+
+    if (listen_addr->kind == SOCKET_ADDRESS_KIND_INET) {
+        SocketAddress *laddr = qio_channel_socket_get_local_address(
+            lioc, &error_abort);
+
+        connect_addr->inet->port = g_strdup(laddr->inet->port);
+
+        qapi_free_SocketAddress(laddr);
+    }
+
+    *src = QIO_CHANNEL(qio_channel_socket_new());
+    qio_channel_socket_connect_async(
+        QIO_CHANNEL_SOCKET(*src), connect_addr,
+        test_io_channel_complete, &data, NULL);
+
+    g_main_loop_run(data.loop);
+
+    g_assert(!data.err);
+
+    *dst = QIO_CHANNEL(qio_channel_socket_accept(lioc, &error_abort));
+    g_assert(*dst);
+
+    qio_channel_set_delay(*src, false, NULL);
+    test_io_channel_set_socket_bufs(*src, *dst);
+
+    object_unref(OBJECT(lioc));
+}
+
+
+static void test_io_channel(bool async,
+                            SocketAddress *listen_addr,
+                            SocketAddress *connect_addr)
+{
+    QIOChannel *src, *dst;
+    if (async) {
+        test_io_channel_setup_async(listen_addr, connect_addr, &src, &dst);
+        test_io_channel_comms(true, src, dst);
+        object_unref(OBJECT(src));
+        object_unref(OBJECT(dst));
+
+        test_io_channel_setup_async(listen_addr, connect_addr, &src, &dst);
+        test_io_channel_comms(false, src, dst);
+        object_unref(OBJECT(src));
+        object_unref(OBJECT(dst));
+    } else {
+        test_io_channel_setup_sync(listen_addr, connect_addr, &src, &dst);
+        test_io_channel_comms(true, src, dst);
+        object_unref(OBJECT(src));
+        object_unref(OBJECT(dst));
+
+        test_io_channel_setup_sync(listen_addr, connect_addr, &src, &dst);
+        test_io_channel_comms(false, src, dst);
+        object_unref(OBJECT(src));
+        object_unref(OBJECT(dst));
+    }
+}
+
+
+static void test_io_channel_ipv4(bool async)
+{
+    SocketAddress *listen_addr = g_new0(SocketAddress, 1);
+    SocketAddress *connect_addr = g_new0(SocketAddress, 1);
+
+    listen_addr->kind = SOCKET_ADDRESS_KIND_INET;
+    listen_addr->inet = g_new0(InetSocketAddress, 1);
+    listen_addr->inet->host = g_strdup("0.0.0.0");
+    listen_addr->inet->port = NULL; /* Auto-select */
+
+    connect_addr->kind = SOCKET_ADDRESS_KIND_INET;
+    connect_addr->inet = g_new0(InetSocketAddress, 1);
+    connect_addr->inet->host = g_strdup("127.0.0.1");
+    connect_addr->inet->port = NULL; /* Filled in later */
+
+    test_io_channel(async, listen_addr, connect_addr);
+}
+
+
+static void test_io_channel_ipv4_sync(void)
+{
+    return test_io_channel_ipv4(false);
+}
+
+
+static void test_io_channel_ipv4_async(void)
+{
+    return test_io_channel_ipv4(true);
+}
+
+
+static void test_io_channel_ipv6(bool async)
+{
+    SocketAddress *listen_addr = g_new0(SocketAddress, 1);
+    SocketAddress *connect_addr = g_new0(SocketAddress, 1);
+
+    listen_addr->kind = SOCKET_ADDRESS_KIND_INET;
+    listen_addr->inet = g_new0(InetSocketAddress, 1);
+    listen_addr->inet->host = g_strdup("::");
+    listen_addr->inet->port = NULL; /* Auto-select */
+
+    connect_addr->kind = SOCKET_ADDRESS_KIND_INET;
+    connect_addr->inet = g_new0(InetSocketAddress, 1);
+    connect_addr->inet->host = g_strdup("::1");
+    connect_addr->inet->port = NULL; /* Filled in later */
+
+    test_io_channel(async, listen_addr, connect_addr);
+}
+
+
+static void test_io_channel_ipv6_sync(void)
+{
+    return test_io_channel_ipv6(false);
+}
+
+
+static void test_io_channel_ipv6_async(void)
+{
+    return test_io_channel_ipv6(true);
+}
+
+
+#ifndef _WIN32
+static void test_io_channel_unix(bool async)
+{
+    SocketAddress *listen_addr = g_new0(SocketAddress, 1);
+    SocketAddress *connect_addr = g_new0(SocketAddress, 1);
+
+#define TEST_SOCKET "test-io-channel-socket.sock"
+    listen_addr->kind = SOCKET_ADDRESS_KIND_UNIX;
+    listen_addr->q_unix = g_new0(UnixSocketAddress, 1);
+    listen_addr->q_unix->path = g_strdup(TEST_SOCKET);
+
+    connect_addr->kind = SOCKET_ADDRESS_KIND_UNIX;
+    connect_addr->q_unix = g_new0(UnixSocketAddress, 1);
+    connect_addr->q_unix->path = g_strdup(TEST_SOCKET);
+
+    test_io_channel(async, listen_addr, connect_addr);
+
+    unlink(TEST_SOCKET);
+}
+
+
+static void test_io_channel_unix_sync(void)
+{
+    return test_io_channel_unix(false);
+}
+
+
+static void test_io_channel_unix_async(void)
+{
+    return test_io_channel_unix(true);
+}
+#endif /* _WIN32 */
+
+
+int main(int argc, char **argv)
+{
+    bool has_ipv4, has_ipv6;
+
+    module_call_init(MODULE_INIT_QOM);
+
+    g_test_init(&argc, &argv, NULL);
+
+    /* We're creating actual IPv4/6 sockets, so we should
+     * check if the host running tests actually supports
+     * each protocol to avoid breaking tests on machines
+     * with either IPv4 or IPv6 disabled.
+     */
+    if (check_protocol_support(&has_ipv4, &has_ipv6) < 0) {
+        return 1;
+    }
+
+    if (has_ipv4) {
+        g_test_add_func("/io/channel/socket/ipv4-sync",
+                        test_io_channel_ipv4_sync);
+        g_test_add_func("/io/channel/socket/ipv4-async",
+                        test_io_channel_ipv4_async);
+    }
+    if (has_ipv6) {
+        g_test_add_func("/io/channel/socket/ipv6-sync",
+                        test_io_channel_ipv6_sync);
+        g_test_add_func("/io/channel/socket/ipv6-async",
+                        test_io_channel_ipv6_async);
+    }
+
+#ifndef _WIN32
+    g_test_add_func("/io/channel/socket/unix-sync",
+                    test_io_channel_unix_sync);
+    g_test_add_func("/io/channel/socket/unix-async",
+                    test_io_channel_unix_async);
+#endif /* _WIN32 */
+
+    return g_test_run();
+}
diff --git a/trace-events b/trace-events
index 987313a..609dc61 100644
--- a/trace-events
+++ b/trace-events
@@ -1694,3 +1694,22 @@ qio_task_thread_start(void *task, void *worker, void *opaque) "Task thread start
 qio_task_thread_run(void *task) "Task thread run task=%p"
 qio_task_thread_exit(void *task) "Task thread exit task=%p"
 qio_task_thread_result(void *task) "Task thread result task=%p"
+
+# io/channel-socket.c
+qio_channel_socket_new(void *ioc) "Socket new ioc=%p"
+qio_channel_socket_new_fd(void *ioc, int fd) "Socket new ioc=%p fd=%d"
+qio_channel_socket_connect_sync(void *ioc, void *addr) "Socket connect sync ioc=%p addr=%p"
+qio_channel_socket_connect_async(void *ioc, void *addr) "Socket connect async ioc=%p addr=%p"
+qio_channel_socket_connect_fail(void *ioc) "Socket connect fail ioc=%p"
+qio_channel_socket_connect_complete(void *ioc, int fd) "Socket connect complete ioc=%p fd=%d"
+qio_channel_socket_listen_sync(void *ioc, void *addr) "Socket listen sync ioc=%p addr=%p"
+qio_channel_socket_listen_async(void *ioc, void *addr) "Socket listen async ioc=%p addr=%p"
+qio_channel_socket_listen_fail(void *ioc) "Socket listen fail ioc=%p"
+qio_channel_socket_listen_complete(void *ioc, int fd) "Socket listen complete ioc=%p fd=%d"
+qio_channel_socket_dgram_sync(void *ioc, void *localAddr, void *remoteAddr) "Socket dgram sync ioc=%p localAddr=%p remoteAddr=%p"
+qio_channel_socket_dgram_async(void *ioc, void *localAddr, void *remoteAddr) "Socket dgram async ioc=%p localAddr=%p remoteAddr=%p"
+qio_channel_socket_dgram_fail(void *ioc) "Socket dgram fail ioc=%p"
+qio_channel_socket_dgram_complete(void *ioc, int fd) "Socket dgram complete ioc=%p fd=%d"
+qio_channel_socket_accept(void *ioc) "Socket accept start ioc=%p"
+qio_channel_socket_accept_fail(void *ioc) "Socket accept fail ioc=%p"
+qio_channel_socket_accept_complete(void *ioc, void *cioc, int fd) "Socket accept complete ioc=%p cioc=%p fd=%d"
-- 
2.4.3

^ permalink raw reply related	[flat|nested] 32+ messages in thread

* [Qemu-devel] [PATCH v1 12/16] io: add QIOChannelFile class
  2015-09-18 13:18 [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework Daniel P. Berrange
                   ` (10 preceding siblings ...)
  2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 11/16] io: add QIOChannelSocket class Daniel P. Berrange
@ 2015-09-18 13:19 ` Daniel P. Berrange
  2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 13/16] io: add QIOChannelTLS class Daniel P. Berrange
                   ` (4 subsequent siblings)
  16 siblings, 0 replies; 32+ messages in thread
From: Daniel P. Berrange @ 2015-09-18 13:19 UTC (permalink / raw)
  To: qemu-devel; +Cc: Paolo Bonzini, Dr. David Alan Gilbert, Gerd Hoffmann

Add a QIOChannel subclass that is capable of operating on things
that are files, such as plain files, pipes, character/block
devices, but notably not sockets.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
 include/io/channel-file.h    |  93 +++++++++++++++++
 io/Makefile.objs             |   1 +
 io/channel-file.c            | 237 +++++++++++++++++++++++++++++++++++++++++++
 tests/.gitignore             |   2 +
 tests/Makefile               |   3 +
 tests/io-channel-helpers.c   |  25 +++++
 tests/io-channel-helpers.h   |   3 +
 tests/test-io-channel-file.c |  91 +++++++++++++++++
 trace-events                 |   4 +
 9 files changed, 459 insertions(+)
 create mode 100644 include/io/channel-file.h
 create mode 100644 io/channel-file.c
 create mode 100644 tests/test-io-channel-file.c

diff --git a/include/io/channel-file.h b/include/io/channel-file.h
new file mode 100644
index 0000000..308e6d4
--- /dev/null
+++ b/include/io/channel-file.h
@@ -0,0 +1,93 @@
+/*
+ * QEMU I/O channels files driver
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef QIO_CHANNEL_FILE_H__
+#define QIO_CHANNEL_FILE_H__
+
+#include "io/channel.h"
+
+#define TYPE_QIO_CHANNEL_FILE "qio-channel-file"
+#define QIO_CHANNEL_FILE(obj)                                     \
+    OBJECT_CHECK(QIOChannelFile, (obj), TYPE_QIO_CHANNEL_FILE)
+
+typedef struct QIOChannelFile QIOChannelFile;
+
+/**
+ * QIOChannelFile:
+ *
+ * The QIOChannelFile object provides a channel implementation
+ * that is able to perform I/O on block devices, character
+ * devices, FIFOs, pipes and plain files. While it is technically
+ * able to work on sockets too on the UNIX platform, this is not
+ * portable to Windows and lacks some extra sockets specific
+ * functionality. So the QIOChannelSocket object is recommended
+ * for that use case.
+ *
+ */
+
+struct QIOChannelFile {
+    QIOChannel parent;
+    int fd;
+};
+
+
+/**
+ * qio_channel_file_new_fd:
+ * @fd: the file descriptor
+ *
+ * Create a new IO channel object for a file represented
+ * by the @fd parameter. @fd can be associated with a
+ * block device, character device, fifo, pipe, or a
+ * regular file. For sockets, the QIOChannelSocket class
+ * should be used instead, as this provides greater
+ * functionality and cross platform portability.
+ *
+ * The channel will own the passed in file descriptor
+ * and will take responsibility for closing it, so the
+ * caller must not close it. If appropriate the caller
+ * should dup() its FD before opening the channel.
+ *
+ * Returns: the new channel object
+ */
+QIOChannelFile *
+qio_channel_file_new_fd(int fd);
+
+/**
+ * qio_channel_file_new_path:
+ * @fd: the file descriptor
+ * @flags: the open flags (O_RDONLY|O_WRONLY|O_RDWR, etc)
+ * @mode: the file creation mode if O_WRONLY is set in @flags
+ * @errp: pointer to initialized error object
+ *
+ * Create a new IO channel object for a file represented
+ * by the @path parameter. @path can point to any
+ * type of file on which sequential I/O can be
+ * performed, whether it be a plain file, character
+ * device or block device.
+ *
+ * Returns: the new channel object
+ */
+QIOChannelFile *
+qio_channel_file_new_path(const char *path,
+                          int flags,
+                          mode_t mode,
+                          Error **errp);
+
+#endif /* QIO_CHANNEL_FILE_H__ */
diff --git a/io/Makefile.objs b/io/Makefile.objs
index f12ec26..9eb0fd9 100644
--- a/io/Makefile.objs
+++ b/io/Makefile.objs
@@ -3,3 +3,4 @@ io-obj-y += task.o
 io-obj-y += channel.o
 io-obj-y += channel-watch.o
 io-obj-y += channel-socket.o
+io-obj-y += channel-file.o
diff --git a/io/channel-file.c b/io/channel-file.c
new file mode 100644
index 0000000..3e67b22
--- /dev/null
+++ b/io/channel-file.c
@@ -0,0 +1,237 @@
+/*
+ * QEMU I/O channels files driver
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "io/channel-file.h"
+#include "io/channel-watch.h"
+#include "qemu/sockets.h"
+#include "trace.h"
+
+QIOChannelFile *
+qio_channel_file_new_fd(int fd)
+{
+    QIOChannelFile *ioc;
+
+    ioc = QIO_CHANNEL_FILE(object_new(TYPE_QIO_CHANNEL_FILE));
+
+    ioc->fd = fd;
+
+    trace_qio_channel_file_new_fd(ioc, fd);
+
+    return ioc;
+}
+
+
+QIOChannelFile *
+qio_channel_file_new_path(const char *path,
+                          int flags,
+                          mode_t mode,
+                          Error **errp)
+{
+    QIOChannelFile *ioc;
+
+    ioc = QIO_CHANNEL_FILE(object_new(TYPE_QIO_CHANNEL_FILE));
+
+    if (flags & O_WRONLY) {
+        ioc->fd = open(path, flags, mode);
+    } else {
+        ioc->fd = open(path, flags);
+    }
+    if (ioc->fd < 0) {
+        object_unref(OBJECT(ioc));
+        error_setg_errno(errp, errno,
+                         "Unable to open %s", path);
+        return NULL;
+    }
+
+    trace_qio_channel_file_new_path(ioc, path, flags, mode, ioc->fd);
+
+    return ioc;
+}
+
+
+static void qio_channel_file_init(Object *obj)
+{
+    QIOChannelFile *ioc = QIO_CHANNEL_FILE(obj);
+    ioc->fd = -1;
+}
+
+static void qio_channel_file_finalize(Object *obj)
+{
+    QIOChannelFile *ioc = QIO_CHANNEL_FILE(obj);
+    if (ioc->fd != -1) {
+        close(ioc->fd);
+        ioc->fd = -1;
+    }
+}
+
+
+static ssize_t qio_channel_file_readv(QIOChannel *ioc,
+                                      const struct iovec *iov,
+                                      size_t niov,
+                                      int **fds,
+                                      size_t *nfds,
+                                      Error **errp)
+{
+    QIOChannelFile *fioc = QIO_CHANNEL_FILE(ioc);
+    ssize_t ret;
+
+    if (fds || nfds) {
+        error_setg_errno(errp, EINVAL,
+                         "Channel does not support file descriptor passing");
+        return -1;
+    }
+
+ retry:
+    ret = readv(fioc->fd, iov, niov);
+    if (ret < 0) {
+        if (errno == EAGAIN ||
+            errno == EWOULDBLOCK) {
+            return QIO_CHANNEL_ERR_BLOCK;
+        }
+        if (errno == EINTR) {
+            goto retry;
+        }
+
+        error_setg_errno(errp, errno,
+                         "Unable to read from file");
+        return -1;
+    }
+
+    return ret;
+}
+
+static ssize_t qio_channel_file_writev(QIOChannel *ioc,
+                                       const struct iovec *iov,
+                                       size_t niov,
+                                       int *fds,
+                                       size_t nfds,
+                                       Error **errp)
+{
+    QIOChannelFile *fioc = QIO_CHANNEL_FILE(ioc);
+    ssize_t ret;
+
+    if (fds || nfds) {
+        error_setg_errno(errp, EINVAL,
+                         "Channel does not support file descriptor passing");
+        return -1;
+    }
+
+ retry:
+    ret = writev(fioc->fd, iov, niov);
+    if (ret <= 0) {
+        if (errno == EAGAIN ||
+            errno == EWOULDBLOCK) {
+            return QIO_CHANNEL_ERR_BLOCK;
+        }
+        if (errno == EINTR) {
+            goto retry;
+        }
+        error_setg_errno(errp, errno,
+                         "Unable to write to file");
+        return -1;
+    }
+    return ret;
+}
+
+static int qio_channel_file_set_blocking(QIOChannel *ioc,
+                                         bool enabled,
+                                         Error **errp)
+{
+    QIOChannelFile *fioc = QIO_CHANNEL_FILE(ioc);
+
+    if (enabled) {
+        qemu_set_block(fioc->fd);
+    } else {
+        qemu_set_nonblock(fioc->fd);
+    }
+    return 0;
+}
+
+
+static off64_t qio_channel_file_seek(QIOChannel *ioc,
+                                     off64_t offset,
+                                     int whence,
+                                     Error **errp)
+{
+    QIOChannelFile *fioc = QIO_CHANNEL_FILE(ioc);
+    off64_t ret;
+
+    ret = lseek64(fioc->fd, offset, whence);
+    if (ret == (off64_t)-1) {
+        error_setg_errno(errp, errno,
+                         "Unable to seek to offset %lld whence %d in file",
+                         (long long int)offset, whence);
+        return -1;
+    }
+    return ret;
+}
+
+
+static int qio_channel_file_close(QIOChannel *ioc,
+                                  Error **errp)
+{
+    QIOChannelFile *fioc = QIO_CHANNEL_FILE(ioc);
+
+    if (close(fioc->fd) < 0) {
+        error_setg_errno(errp, errno,
+                         "Unable to close file");
+        return -1;
+    }
+    return 0;
+}
+
+
+static GSource *qio_channel_file_create_watch(QIOChannel *ioc,
+                                              GIOCondition condition)
+{
+    QIOChannelFile *fioc = QIO_CHANNEL_FILE(ioc);
+    return qio_channel_create_fd_watch(ioc,
+                                       fioc->fd,
+                                       condition);
+}
+
+static void qio_channel_file_class_init(ObjectClass *klass,
+                                        void *class_data G_GNUC_UNUSED)
+{
+    QIOChannelClass *ioc_klass = QIO_CHANNEL_CLASS(klass);
+
+    ioc_klass->io_writev = qio_channel_file_writev;
+    ioc_klass->io_readv = qio_channel_file_readv;
+    ioc_klass->io_set_blocking = qio_channel_file_set_blocking;
+    ioc_klass->io_seek = qio_channel_file_seek;
+    ioc_klass->io_close = qio_channel_file_close;
+    ioc_klass->io_create_watch = qio_channel_file_create_watch;
+}
+
+static const TypeInfo qio_channel_file_info = {
+    .parent = TYPE_QIO_CHANNEL,
+    .name = TYPE_QIO_CHANNEL_FILE,
+    .instance_size = sizeof(QIOChannelFile),
+    .instance_init = qio_channel_file_init,
+    .instance_finalize = qio_channel_file_finalize,
+    .class_init = qio_channel_file_class_init,
+};
+
+static void qio_channel_file_register_types(void)
+{
+    type_register_static(&qio_channel_file_info);
+}
+
+type_init(qio_channel_file_register_types);
diff --git a/tests/.gitignore b/tests/.gitignore
index ac891c6..bb66d94 100644
--- a/tests/.gitignore
+++ b/tests/.gitignore
@@ -23,6 +23,8 @@ test-cutils
 test-hbitmap
 test-int128
 test-iov
+test-io-channel-file
+test-io-channel-file.txt
 test-io-channel-socket
 test-io-task
 test-mul64
diff --git a/tests/Makefile b/tests/Makefile
index ca1846c..3b0b57b 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -82,6 +82,7 @@ check-unit-$(CONFIG_GNUTLS) += tests/test-crypto-tlscredsx509$(EXESUF)
 check-unit-$(CONFIG_GNUTLS) += tests/test-crypto-tlssession$(EXESUF)
 check-unit-y += tests/test-io-task$(EXESUF)
 check-unit-y += tests/test-io-channel-socket$(EXESUF)
+check-unit-y += tests/test-io-channel-file$(EXESUF)
 
 check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
 
@@ -370,6 +371,8 @@ tests/test-crypto-tlssession$(EXESUF): tests/test-crypto-tlssession.o \
 tests/test-io-task$(EXESUF): tests/test-io-task.o $(test-io-obj-y)
 tests/test-io-channel-socket$(EXESUF): tests/test-io-channel-socket.o \
         tests/io-channel-helpers.o $(test-io-obj-y)
+tests/test-io-channel-file$(EXESUF): tests/test-io-channel-file.o \
+        tests/io-channel-helpers.o $(test-io-obj-y)
 
 libqos-obj-y = tests/libqos/pci.o tests/libqos/fw_cfg.o tests/libqos/malloc.o
 libqos-obj-y += tests/libqos/i2c.o tests/libqos/libqos.o
diff --git a/tests/io-channel-helpers.c b/tests/io-channel-helpers.c
index c84b99f..91c44b4 100644
--- a/tests/io-channel-helpers.c
+++ b/tests/io-channel-helpers.c
@@ -220,3 +220,28 @@ void test_io_channel_comms(bool blocking,
     g_free(data->output);
     g_free(data);
 }
+
+
+void test_io_channel_comms_serial(QIOChannel *src,
+                                  QIOChannel *dst)
+{
+    struct TestIOData *data;
+
+    data = test_load_io_data("libqemuutil.a");
+    data->src = src;
+    data->dst = dst;
+    data->blocking = true;
+
+    test_io_thread_writer(data);
+    test_io_thread_reader(data);
+
+    g_assert_cmpint(memcmp(data->input,
+                           data->output,
+                           data->len), ==, 0);
+    g_assert(data->readerr == NULL);
+    g_assert(data->writeerr == NULL);
+
+    g_free(data->input);
+    g_free(data->output);
+    g_free(data);
+}
diff --git a/tests/io-channel-helpers.h b/tests/io-channel-helpers.h
index d0085a2..7edbaad 100644
--- a/tests/io-channel-helpers.h
+++ b/tests/io-channel-helpers.h
@@ -27,4 +27,7 @@ void test_io_channel_comms(bool blocking,
                            QIOChannel *src,
                            QIOChannel *dst);
 
+void test_io_channel_comms_serial(QIOChannel *src,
+                                  QIOChannel *dst);
+
 #endif /* TEST_IO_CHANNEL_HELPERS */
diff --git a/tests/test-io-channel-file.c b/tests/test-io-channel-file.c
new file mode 100644
index 0000000..97394f9
--- /dev/null
+++ b/tests/test-io-channel-file.c
@@ -0,0 +1,91 @@
+/*
+ * QEMU I/O channel sockets test
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "io/channel-file.h"
+#include "io-channel-helpers.h"
+
+
+static void test_io_channel_file(void)
+{
+    QIOChannel *src, *dst;
+
+#define TEST_FILE "tests/test-io-channel-file.txt"
+    unlink(TEST_FILE);
+    src = QIO_CHANNEL(qio_channel_file_new_path(
+                          TEST_FILE,
+                          O_WRONLY | O_CREAT | O_TRUNC | O_BINARY, 0600,
+                          &error_abort));
+    dst = QIO_CHANNEL(qio_channel_file_new_path(
+                          TEST_FILE,
+                          O_RDONLY | O_BINARY, 0,
+                          &error_abort));
+    test_io_channel_comms_serial(src, dst);
+
+    unlink(TEST_FILE);
+    object_unref(OBJECT(src));
+    object_unref(OBJECT(dst));
+}
+
+
+#ifndef _WIN32
+static void test_io_channel_pipe(bool async)
+{
+    QIOChannel *src, *dst;
+    int fd[2];
+
+    if (pipe(fd) < 0) {
+        perror("pipe");
+        abort();
+    }
+
+    src = QIO_CHANNEL(qio_channel_file_new_fd(fd[1]));
+    dst = QIO_CHANNEL(qio_channel_file_new_fd(fd[0]));
+    test_io_channel_comms(async, src, dst);
+
+    object_unref(OBJECT(src));
+    object_unref(OBJECT(dst));
+}
+
+
+static void test_io_channel_pipe_async(void)
+{
+    test_io_channel_pipe(true);
+}
+
+static void test_io_channel_pipe_sync(void)
+{
+    test_io_channel_pipe(false);
+}
+#endif /* ! _WIN32 */
+
+
+int main(int argc, char **argv)
+{
+    module_call_init(MODULE_INIT_QOM);
+
+    g_test_init(&argc, &argv, NULL);
+
+    g_test_add_func("/io/channel/file", test_io_channel_file);
+#ifndef _WIN32
+    g_test_add_func("/io/channel/pipe/sync", test_io_channel_pipe_sync);
+    g_test_add_func("/io/channel/pipe/async", test_io_channel_pipe_async);
+#endif
+    return g_test_run();
+}
diff --git a/trace-events b/trace-events
index 609dc61..cff3261 100644
--- a/trace-events
+++ b/trace-events
@@ -1713,3 +1713,7 @@ qio_channel_socket_dgram_complete(void *ioc, int fd) "Socket dgram complete ioc=
 qio_channel_socket_accept(void *ioc) "Socket accept start ioc=%p"
 qio_channel_socket_accept_fail(void *ioc) "Socket accept fail ioc=%p"
 qio_channel_socket_accept_complete(void *ioc, void *cioc, int fd) "Socket accept complete ioc=%p cioc=%p fd=%d"
+
+# io/channel-file.c
+qio_channel_file_new_fd(void *ioc, int fd) "File new fd ioc=%p fd=%d"
+qio_channel_file_new_path(void *ioc, const char *path, int flags, int mode, int fd) "File new fd ioc=%p path=%s flags=%d mode=%d fd=%d"
-- 
2.4.3

^ permalink raw reply related	[flat|nested] 32+ messages in thread

* [Qemu-devel] [PATCH v1 13/16] io: add QIOChannelTLS class
  2015-09-18 13:18 [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework Daniel P. Berrange
                   ` (11 preceding siblings ...)
  2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 12/16] io: add QIOChannelFile class Daniel P. Berrange
@ 2015-09-18 13:19 ` Daniel P. Berrange
  2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 14/16] io: add QIOChannelWebsock class Daniel P. Berrange
                   ` (3 subsequent siblings)
  16 siblings, 0 replies; 32+ messages in thread
From: Daniel P. Berrange @ 2015-09-18 13:19 UTC (permalink / raw)
  To: qemu-devel; +Cc: Paolo Bonzini, Dr. David Alan Gilbert, Gerd Hoffmann

Add a QIOChannel subclass that can run the TLS protocol over
the top of another QIOChannel instance. The object provides a
simplified API to perform the handshake when starting the TLS
session. The layering of TLS over the underlying channel does
not have to be setup immediately. It is possible to take an
existing QIOChannel that has done some handshake and then swap
in the QIOChannelTLS layer. This allows for use with protocols
which start TLS right away, and those which start plain text
and then negotiate TLS.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
 include/io/channel-tls.h    | 142 +++++++++++++++
 io/Makefile.objs            |   1 +
 io/channel-tls.c            | 419 ++++++++++++++++++++++++++++++++++++++++++++
 tests/.gitignore            |   1 +
 tests/Makefile              |   4 +
 tests/test-io-channel-tls.c | 335 +++++++++++++++++++++++++++++++++++
 trace-events                |  10 ++
 7 files changed, 912 insertions(+)
 create mode 100644 include/io/channel-tls.h
 create mode 100644 io/channel-tls.c
 create mode 100644 tests/test-io-channel-tls.c

diff --git a/include/io/channel-tls.h b/include/io/channel-tls.h
new file mode 100644
index 0000000..0298b17
--- /dev/null
+++ b/include/io/channel-tls.h
@@ -0,0 +1,142 @@
+/*
+ * QEMU I/O channels TLS driver
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef QIO_CHANNEL_TLS_H__
+#define QIO_CHANNEL_TLS_H__
+
+#include "io/channel.h"
+#include "io/task.h"
+#include "crypto/tlssession.h"
+
+#define TYPE_QIO_CHANNEL_TLS "qio-channel-tls"
+#define QIO_CHANNEL_TLS(obj)                                     \
+    OBJECT_CHECK(QIOChannelTLS, (obj), TYPE_QIO_CHANNEL_TLS)
+
+typedef struct QIOChannelTLS QIOChannelTLS;
+
+/**
+ * QIOChannelTLS
+ *
+ * The QIOChannelTLS class provides a channel wrapper which
+ * can transparently run the TLS encryption protocol. It is
+ * usually used over a TCP socket, but there is actually no
+ * technical restriction on which type of master channel is
+ * used as the transport.
+ *
+ * This channel object is capable of running as either a
+ * TLS server or TLS client.
+ */
+
+struct QIOChannelTLS {
+    QIOChannel parent;
+    QIOChannel *master;
+    QCryptoTLSSession *session;
+};
+
+/**
+ * qio_channel_tls_new_server:
+ * @master: the underlying channel object
+ * @creds: the credentials to use for TLS handshake
+ * @aclname: the access control list for validating clients
+ * @errp: pointer to an uninitialized error object
+ *
+ * Create a new TLS channel that runs the server side of
+ * a TLS session. The TLS session handshake will use the
+ * credentials provided in @creds. If the @aclname parameter
+ * is non-NULL, then the client will have to provide
+ * credentials (ie a x509 client certificate) which will
+ * then be validated against the ACL.
+ *
+ * After creating the channel, it is mandatory to call
+ * the qio_channel_tls_handshake() method before attempting
+ * todo any I/O on the channel.
+ *
+ * Once the handshake has completed, all I/O should be done
+ * via the new TLS channel object and not the original
+ * master channel
+ *
+ * Returns: the new TLS channel object, or NULL
+ */
+QIOChannelTLS *
+qio_channel_tls_new_server(QIOChannel *master,
+                           QCryptoTLSCreds *creds,
+                           const char *aclname,
+                           Error **errp);
+
+/**
+ * qio_channel_tls_new_client:
+ * @master: the underlying channel object
+ * @creds: the credentials to use for TLS handshake
+ * @hostname: the user specified server hostname
+ * @errp: pointer to an uninitialized error object
+ *
+ * Create a new TLS channel that runs the client side of
+ * a TLS session. The TLS session handshake will use the
+ * credentials provided in @creds. The @hostname parameter
+ * should provide the user specified hostname of the server
+ * and will be validated against the server's credentials
+ * (ie CommonName of the x509 certificate)
+ *
+ * After creating the channel, it is mandatory to call
+ * the qio_channel_tls_handshake() method before attempting
+ * todo any I/O on the channel.
+ *
+ * Once the handshake has completed, all I/O should be done
+ * via the new TLS channel object and not the original
+ * master channel
+ *
+ * Returns: the new TLS channel object, or NULL
+ */
+QIOChannelTLS *
+qio_channel_tls_new_client(QIOChannel *master,
+                           QCryptoTLSCreds *creds,
+                           const char *hostname,
+                           Error **errp);
+
+/**
+ * qio_channel_tls_handshake:
+ * @ioc: the TLS channel object
+ * @func: the callback to invoke when completed
+ * @opaque: opaque data to pass to @func
+ * @destroy: optional callback to free @opaque
+ *
+ * Perform the TLS session handshake. This method
+ * will return immediately and the handshake will
+ * continue in the background, provided the main
+ * loop is running. When the handshake is complete,
+ * or fails, the @func callback will be invoked.
+ */
+void qio_channel_tls_handshake(QIOChannelTLS *ioc,
+                               QIOTaskFunc func,
+                               gpointer opaque,
+                               GDestroyNotify destroy);
+
+/**
+ * qio_channel_tls_get_session:
+ * @ioc: the TLS channel object
+ *
+ * Get the TLS session used by the channel.
+ *
+ * Returns: the TLS session
+ */
+QCryptoTLSSession *
+qio_channel_tls_get_session(QIOChannelTLS *ioc);
+
+#endif /* QIO_CHANNEL_TLS_H__ */
diff --git a/io/Makefile.objs b/io/Makefile.objs
index 9eb0fd9..2b33d3c 100644
--- a/io/Makefile.objs
+++ b/io/Makefile.objs
@@ -4,3 +4,4 @@ io-obj-y += channel.o
 io-obj-y += channel-watch.o
 io-obj-y += channel-socket.o
 io-obj-y += channel-file.o
+io-obj-y += channel-tls.o
diff --git a/io/channel-tls.c b/io/channel-tls.c
new file mode 100644
index 0000000..4dfeabe
--- /dev/null
+++ b/io/channel-tls.c
@@ -0,0 +1,419 @@
+/*
+ * QEMU I/O channels TLS driver
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "io/channel-tls.h"
+#include "trace.h"
+
+
+static ssize_t qio_channel_tls_write_handler(const char *buf,
+                                             size_t len,
+                                             void *opaque)
+{
+    QIOChannelTLS *tioc = QIO_CHANNEL_TLS(opaque);
+    ssize_t ret;
+
+    ret = qio_channel_write(tioc->master, buf, len, NULL);
+    if (ret == QIO_CHANNEL_ERR_BLOCK) {
+        errno = EAGAIN;
+        return -1;
+    } else if (ret < 0) {
+        errno = EIO;
+        return -1;
+    }
+    return ret;
+}
+
+static ssize_t qio_channel_tls_read_handler(char *buf,
+                                            size_t len,
+                                            void *opaque)
+{
+    QIOChannelTLS *tioc = QIO_CHANNEL_TLS(opaque);
+    ssize_t ret;
+
+    ret = qio_channel_read(tioc->master, buf, len, NULL);
+    if (ret == QIO_CHANNEL_ERR_BLOCK) {
+        errno = EAGAIN;
+        return -1;
+    } else if (ret < 0) {
+        errno = EIO;
+        return -1;
+    }
+    return ret;
+}
+
+
+QIOChannelTLS *
+qio_channel_tls_new_server(QIOChannel *master,
+                           QCryptoTLSCreds *creds,
+                           const char *aclname,
+                           Error **errp)
+{
+    QIOChannelTLS *ioc;
+
+    ioc = QIO_CHANNEL_TLS(object_new(TYPE_QIO_CHANNEL_TLS));
+
+    ioc->master = master;
+    object_ref(OBJECT(master));
+
+    ioc->session = qcrypto_tls_session_new(
+        creds,
+        NULL,
+        aclname,
+        QCRYPTO_TLS_CREDS_ENDPOINT_SERVER,
+        errp);
+    if (!ioc->session) {
+        goto error;
+    }
+
+    qcrypto_tls_session_set_callbacks(
+        ioc->session,
+        qio_channel_tls_write_handler,
+        qio_channel_tls_read_handler,
+        ioc);
+
+    trace_qio_channel_tls_new_server(ioc, master, creds, aclname);
+    return ioc;
+
+ error:
+    object_unref(OBJECT(ioc));
+    return NULL;
+}
+
+QIOChannelTLS *
+qio_channel_tls_new_client(QIOChannel *master,
+                           QCryptoTLSCreds *creds,
+                           const char *hostname,
+                           Error **errp)
+{
+    QIOChannelTLS *ioc;
+
+    ioc = QIO_CHANNEL_TLS(object_new(TYPE_QIO_CHANNEL_TLS));
+
+    ioc->master = master;
+    object_ref(OBJECT(master));
+
+    ioc->session = qcrypto_tls_session_new(
+        creds,
+        hostname,
+        NULL,
+        QCRYPTO_TLS_CREDS_ENDPOINT_CLIENT,
+        errp);
+    if (!ioc->session) {
+        goto error;
+    }
+
+    qcrypto_tls_session_set_callbacks(
+        ioc->session,
+        qio_channel_tls_write_handler,
+        qio_channel_tls_read_handler,
+        ioc);
+
+    trace_qio_channel_tls_new_client(ioc, master, creds, hostname);
+    return ioc;
+
+ error:
+    object_unref(OBJECT(ioc));
+    return NULL;
+}
+
+
+static gboolean qio_channel_tls_handshake_io(QIOChannel *ioc,
+                                             GIOCondition condition,
+                                             gpointer user_data);
+
+static void qio_channel_tls_handshake_task(QIOChannelTLS *ioc,
+                                           QIOTask *task)
+{
+    Error *err = NULL;
+    QCryptoTLSSessionHandshakeStatus status;
+
+    if (qcrypto_tls_session_handshake(ioc->session, &err) < 0) {
+        trace_qio_channel_tls_handshake_fail(ioc);
+        qio_task_abort(task, err);
+        goto cleanup;
+    }
+
+    status = qcrypto_tls_session_get_handshake_status(ioc->session);
+    if (status == QCRYPTO_TLS_HANDSHAKE_COMPLETE) {
+        trace_qio_channel_tls_handshake_complete(ioc);
+        if (qcrypto_tls_session_check_credentials(ioc->session,
+                                                  &err) < 0) {
+            trace_qio_channel_tls_credentials_deny(ioc);
+            qio_task_abort(task, err);
+            goto cleanup;
+        }
+        trace_qio_channel_tls_credentials_allow(ioc);
+        qio_task_complete(task);
+    } else {
+        GIOCondition condition;
+        if (status == QCRYPTO_TLS_HANDSHAKE_SENDING) {
+            condition = G_IO_OUT;
+        } else {
+            condition = G_IO_IN;
+        }
+
+        trace_qio_channel_tls_handshake_pending(ioc, status);
+        qio_channel_add_watch(ioc->master,
+                              condition,
+                              qio_channel_tls_handshake_io,
+                              task,
+                              NULL);
+    }
+
+ cleanup:
+    error_free(err);
+}
+
+
+static gboolean qio_channel_tls_handshake_io(QIOChannel *ioc,
+                                             GIOCondition condition,
+                                             gpointer user_data)
+{
+    QIOTask *task = user_data;
+    QIOChannelTLS *tioc = QIO_CHANNEL_TLS(
+        qio_task_get_source(task));
+
+    qio_channel_tls_handshake_task(
+       tioc, task);
+
+    object_unref(OBJECT(tioc));
+
+    return FALSE;
+}
+
+void qio_channel_tls_handshake(QIOChannelTLS *ioc,
+                               QIOTaskFunc func,
+                               gpointer opaque,
+                               GDestroyNotify destroy)
+{
+    QIOTask *task;
+
+    task = qio_task_new(OBJECT(ioc),
+                        func, opaque, destroy);
+
+    trace_qio_channel_tls_handshake_start(ioc);
+    qio_channel_tls_handshake_task(ioc, task);
+}
+
+
+static void qio_channel_tls_init(Object *obj G_GNUC_UNUSED)
+{
+}
+
+
+static void qio_channel_tls_finalize(Object *obj)
+{
+    QIOChannelTLS *ioc = QIO_CHANNEL_TLS(obj);
+
+    object_unref(OBJECT(ioc->master));
+    qcrypto_tls_session_free(ioc->session);
+}
+
+
+static bool qio_channel_tls_has_feature(QIOChannel *ioc,
+                                        QIOChannelFeature feature)
+{
+    QIOChannelTLS *tioc = QIO_CHANNEL_TLS(ioc);
+
+    switch (feature) {
+    case QIO_CHANNEL_FEATURE_SHUTDOWN:
+    case QIO_CHANNEL_FEATURE_CORK:
+    case QIO_CHANNEL_FEATURE_DELAY:
+        return qio_channel_has_feature(tioc->master, feature);
+    default:
+        return false;
+    }
+}
+
+
+static ssize_t qio_channel_tls_readv(QIOChannel *ioc,
+                                     const struct iovec *iov,
+                                     size_t niov,
+                                     int **fds,
+                                     size_t *nfds,
+                                     Error **errp)
+{
+    QIOChannelTLS *tioc = QIO_CHANNEL_TLS(ioc);
+    size_t i;
+    ssize_t got = 0;
+
+    if (fds || nfds) {
+        error_setg(errp,
+                   "Cannot receive file descriptors over TLS channel");
+        return -1;
+    }
+
+    for (i = 0 ; i < niov ; i++) {
+        ssize_t ret = qcrypto_tls_session_read(tioc->session,
+                                               iov[i].iov_base,
+                                               iov[i].iov_len);
+        if (ret < 0) {
+            if (errno == EAGAIN) {
+                if (got) {
+                    return got;
+                } else {
+                    return QIO_CHANNEL_ERR_BLOCK;
+                }
+            }
+
+            error_setg_errno(errp, errno,
+                             "Cannot read from TLS channel");
+            return -1;
+        }
+        got += ret;
+        if (ret < iov[i].iov_len) {
+            break;
+        }
+    }
+    return got;
+}
+
+
+static ssize_t qio_channel_tls_writev(QIOChannel *ioc,
+                                      const struct iovec *iov,
+                                      size_t niov,
+                                      int *fds,
+                                      size_t nfds,
+                                      Error **errp)
+{
+    QIOChannelTLS *tioc = QIO_CHANNEL_TLS(ioc);
+    size_t i;
+    ssize_t done = 0;
+
+    if (fds || nfds) {
+        error_setg(errp,
+                   "Cannot send file descriptors over TLS channel");
+        return -1;
+    }
+
+    for (i = 0 ; i < niov ; i++) {
+        ssize_t ret = qcrypto_tls_session_write(tioc->session,
+                                                iov[i].iov_base,
+                                                iov[i].iov_len);
+        if (ret <= 0) {
+            if (errno == EAGAIN) {
+                if (done) {
+                    return done;
+                } else {
+                    return QIO_CHANNEL_ERR_BLOCK;
+                }
+            }
+
+            error_setg_errno(errp, errno,
+                             "Cannot write to TLS channel");
+            return -1;
+        }
+        done += ret;
+        if (ret < iov[i].iov_len) {
+            break;
+        }
+    }
+    return done;
+}
+
+static int qio_channel_tls_set_blocking(QIOChannel *ioc,
+                                        bool enabled,
+                                        Error **errp)
+{
+    QIOChannelTLS *tioc = QIO_CHANNEL_TLS(ioc);
+
+    return qio_channel_set_blocking(tioc->master, enabled, errp);
+}
+
+static int qio_channel_tls_set_delay(QIOChannel *ioc,
+                                     bool enabled,
+                                     Error **errp)
+{
+    QIOChannelTLS *tioc = QIO_CHANNEL_TLS(ioc);
+
+    return qio_channel_set_delay(tioc->master, enabled, errp);
+}
+
+static int qio_channel_tls_set_cork(QIOChannel *ioc,
+                                    bool enabled,
+                                    Error **errp)
+{
+    QIOChannelTLS *tioc = QIO_CHANNEL_TLS(ioc);
+
+    return qio_channel_set_cork(tioc->master, enabled, errp);
+}
+
+static int qio_channel_tls_shutdown(QIOChannel *ioc,
+                                    QIOChannelShutdown how,
+                                    Error **errp)
+{
+    QIOChannelTLS *tioc = QIO_CHANNEL_TLS(ioc);
+
+    return qio_channel_shutdown(tioc->master, how, errp);
+}
+
+static int qio_channel_tls_close(QIOChannel *ioc,
+                                 Error **errp)
+{
+    QIOChannelTLS *tioc = QIO_CHANNEL_TLS(ioc);
+
+    return qio_channel_close(tioc->master, errp);
+}
+
+static GSource *qio_channel_tls_create_watch(QIOChannel *ioc,
+                                             GIOCondition condition)
+{
+    QIOChannelTLS *tioc = QIO_CHANNEL_TLS(ioc);
+
+    return qio_channel_create_watch(tioc->master, condition);
+}
+
+QCryptoTLSSession *
+qio_channel_tls_get_session(QIOChannelTLS *ioc)
+{
+    return ioc->session;
+}
+
+static void qio_channel_tls_class_init(ObjectClass *klass,
+                                       void *class_data G_GNUC_UNUSED)
+{
+    QIOChannelClass *ioc_klass = QIO_CHANNEL_CLASS(klass);
+
+    ioc_klass->io_has_feature = qio_channel_tls_has_feature;
+    ioc_klass->io_writev = qio_channel_tls_writev;
+    ioc_klass->io_readv = qio_channel_tls_readv;
+    ioc_klass->io_set_blocking = qio_channel_tls_set_blocking;
+    ioc_klass->io_set_delay = qio_channel_tls_set_delay;
+    ioc_klass->io_set_cork = qio_channel_tls_set_cork;
+    ioc_klass->io_close = qio_channel_tls_close;
+    ioc_klass->io_shutdown = qio_channel_tls_shutdown;
+    ioc_klass->io_create_watch = qio_channel_tls_create_watch;
+}
+
+static const TypeInfo qio_channel_tls_info = {
+    .parent = TYPE_QIO_CHANNEL,
+    .name = TYPE_QIO_CHANNEL_TLS,
+    .instance_size = sizeof(QIOChannelTLS),
+    .instance_init = qio_channel_tls_init,
+    .instance_finalize = qio_channel_tls_finalize,
+    .class_init = qio_channel_tls_class_init,
+};
+
+static void qio_channel_tls_register_types(void)
+{
+    type_register_static(&qio_channel_tls_info);
+}
+
+type_init(qio_channel_tls_register_types);
diff --git a/tests/.gitignore b/tests/.gitignore
index bb66d94..aa90bb2 100644
--- a/tests/.gitignore
+++ b/tests/.gitignore
@@ -26,6 +26,7 @@ test-iov
 test-io-channel-file
 test-io-channel-file.txt
 test-io-channel-socket
+test-io-channel-tls
 test-io-task
 test-mul64
 test-opts-visitor
diff --git a/tests/Makefile b/tests/Makefile
index 3b0b57b..9c60e28 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -83,6 +83,7 @@ check-unit-$(CONFIG_GNUTLS) += tests/test-crypto-tlssession$(EXESUF)
 check-unit-y += tests/test-io-task$(EXESUF)
 check-unit-y += tests/test-io-channel-socket$(EXESUF)
 check-unit-y += tests/test-io-channel-file$(EXESUF)
+check-unit-$(CONFIG_GNUTLS) += tests/test-io-channel-tls$(EXESUF)
 
 check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
 
@@ -373,6 +374,9 @@ tests/test-io-channel-socket$(EXESUF): tests/test-io-channel-socket.o \
         tests/io-channel-helpers.o $(test-io-obj-y)
 tests/test-io-channel-file$(EXESUF): tests/test-io-channel-file.o \
         tests/io-channel-helpers.o $(test-io-obj-y)
+tests/test-io-channel-tls$(EXESUF): tests/test-io-channel-tls.o \
+	tests/crypto-tls-x509-helpers.o tests/pkix_asn1_tab.o \
+	tests/io-channel-helpers.o $(test-io-obj-y)
 
 libqos-obj-y = tests/libqos/pci.o tests/libqos/fw_cfg.o tests/libqos/malloc.o
 libqos-obj-y += tests/libqos/i2c.o tests/libqos/libqos.o
diff --git a/tests/test-io-channel-tls.c b/tests/test-io-channel-tls.c
new file mode 100644
index 0000000..afab679
--- /dev/null
+++ b/tests/test-io-channel-tls.c
@@ -0,0 +1,335 @@
+/*
+ * Copyright (C) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library.  If not, see
+ * <http://www.gnu.org/licenses/>.
+ *
+ * Author: Daniel P. Berrange <berrange@redhat.com>
+ */
+
+
+#include <stdlib.h>
+#include <fcntl.h>
+
+#include "config-host.h"
+#include "crypto-tls-x509-helpers.h"
+#include "io/channel-tls.h"
+#include "io/channel-socket.h"
+#include "io-channel-helpers.h"
+#include "crypto/tlscredsx509.h"
+#include "qemu/acl.h"
+#include "qom/object_interfaces.h"
+
+#ifdef QCRYPTO_HAVE_TLS_TEST_SUPPORT
+
+#define WORKDIR "tests/test-io-channel-tls-work/"
+#define KEYFILE WORKDIR "key-ctx.pem"
+
+struct QIOChannelTLSTestData {
+    const char *servercacrt;
+    const char *clientcacrt;
+    const char *servercrt;
+    const char *clientcrt;
+    bool expectServerFail;
+    bool expectClientFail;
+    const char *hostname;
+    const char *const *wildcards;
+};
+
+struct QIOChannelTLSHandshakeData {
+    bool finished;
+    bool failed;
+};
+
+static void test_tls_handshake_done(Object *source,
+                                    Error *err,
+                                    gpointer opaque)
+{
+    struct QIOChannelTLSHandshakeData *data = opaque;
+
+    data->finished = true;
+    data->failed = err != NULL;
+}
+
+
+static QCryptoTLSCreds *test_tls_creds_create(QCryptoTLSCredsEndpoint endpoint,
+                                              const char *certdir,
+                                              Error **errp)
+{
+    Object *parent = object_get_objects_root();
+    Object *creds = object_new_with_props(
+        TYPE_QCRYPTO_TLS_CREDS_X509,
+        parent,
+        (endpoint == QCRYPTO_TLS_CREDS_ENDPOINT_SERVER ?
+         "testtlscredsserver" : "testtlscredsclient"),
+        errp,
+        "endpoint", (endpoint == QCRYPTO_TLS_CREDS_ENDPOINT_SERVER ?
+                     "server" : "client"),
+        "dir", certdir,
+        "verify-peer", "yes",
+        /* We skip initial sanity checks here because we
+         * want to make sure that problems are being
+         * detected at the TLS session validation stage,
+         * and the test-crypto-tlscreds test already
+         * validate the sanity check code.
+         */
+        "sanity-check", "no",
+        NULL
+        );
+
+    if (*errp) {
+        return NULL;
+    }
+    return QCRYPTO_TLS_CREDS(creds);
+}
+
+
+/*
+ * This tests validation checking of peer certificates
+ *
+ * This is replicating the checks that are done for an
+ * active TLS session after handshake completes. To
+ * simulate that we create our TLS contexts, skipping
+ * sanity checks. When then get a socketpair, and
+ * initiate a TLS session across them. Finally do
+ * do actual cert validation tests
+ */
+static void test_io_channel_tls(const void *opaque)
+{
+    struct QIOChannelTLSTestData *data =
+        (struct QIOChannelTLSTestData *)opaque;
+    QCryptoTLSCreds *clientCreds;
+    QCryptoTLSCreds *serverCreds;
+    QIOChannelTLS *clientChanTLS;
+    QIOChannelTLS *serverChanTLS;
+    QIOChannelSocket *clientChanSock;
+    QIOChannelSocket *serverChanSock;
+    qemu_acl *acl;
+    const char * const *wildcards;
+    int channel[2];
+    struct QIOChannelTLSHandshakeData clientHandshake = { false, false };
+    struct QIOChannelTLSHandshakeData serverHandshake = { false, false };
+    Error *err = NULL;
+    GMainContext *mainloop;
+
+    /* We'll use this for our fake client-server connection */
+    g_assert(socketpair(AF_UNIX, SOCK_STREAM, 0, channel) == 0);
+
+#define CLIENT_CERT_DIR "tests/test-crypto-tlssession-client/"
+#define SERVER_CERT_DIR "tests/test-crypto-tlssession-server/"
+    mkdir(CLIENT_CERT_DIR, 0700);
+    mkdir(SERVER_CERT_DIR, 0700);
+
+    unlink(SERVER_CERT_DIR QCRYPTO_TLS_CREDS_X509_CA_CERT);
+    unlink(SERVER_CERT_DIR QCRYPTO_TLS_CREDS_X509_SERVER_CERT);
+    unlink(SERVER_CERT_DIR QCRYPTO_TLS_CREDS_X509_SERVER_KEY);
+
+    unlink(CLIENT_CERT_DIR QCRYPTO_TLS_CREDS_X509_CA_CERT);
+    unlink(CLIENT_CERT_DIR QCRYPTO_TLS_CREDS_X509_CLIENT_CERT);
+    unlink(CLIENT_CERT_DIR QCRYPTO_TLS_CREDS_X509_CLIENT_KEY);
+
+    g_assert(link(data->servercacrt,
+                  SERVER_CERT_DIR QCRYPTO_TLS_CREDS_X509_CA_CERT) == 0);
+    g_assert(link(data->servercrt,
+                  SERVER_CERT_DIR QCRYPTO_TLS_CREDS_X509_SERVER_CERT) == 0);
+    g_assert(link(KEYFILE,
+                  SERVER_CERT_DIR QCRYPTO_TLS_CREDS_X509_SERVER_KEY) == 0);
+
+    g_assert(link(data->clientcacrt,
+                  CLIENT_CERT_DIR QCRYPTO_TLS_CREDS_X509_CA_CERT) == 0);
+    g_assert(link(data->clientcrt,
+                  CLIENT_CERT_DIR QCRYPTO_TLS_CREDS_X509_CLIENT_CERT) == 0);
+    g_assert(link(KEYFILE,
+                  CLIENT_CERT_DIR QCRYPTO_TLS_CREDS_X509_CLIENT_KEY) == 0);
+
+    clientCreds = test_tls_creds_create(
+        QCRYPTO_TLS_CREDS_ENDPOINT_CLIENT,
+        CLIENT_CERT_DIR,
+        &err);
+    g_assert(clientCreds != NULL);
+
+    serverCreds = test_tls_creds_create(
+        QCRYPTO_TLS_CREDS_ENDPOINT_SERVER,
+        SERVER_CERT_DIR,
+        &err);
+    g_assert(serverCreds != NULL);
+
+    acl = qemu_acl_init("channeltlsacl");
+    qemu_acl_reset(acl);
+    wildcards = data->wildcards;
+    while (wildcards && *wildcards) {
+        qemu_acl_append(acl, 0, *wildcards);
+        wildcards++;
+    }
+
+    clientChanSock = qio_channel_socket_new_fd(
+        channel[0], &err);
+    g_assert(clientChanSock != NULL);
+    serverChanSock = qio_channel_socket_new_fd(
+        channel[1], &err);
+    g_assert(serverChanSock != NULL);
+
+    /*
+     * We have an evil loop to do the handshake in a single
+     * thread, so we need these non-blocking to avoid deadlock
+     * of ourselves
+     */
+    qio_channel_set_blocking(QIO_CHANNEL(clientChanSock), false, NULL);
+    qio_channel_set_blocking(QIO_CHANNEL(serverChanSock), false, NULL);
+
+    /* Now the real part of the test, setup the sessions */
+    clientChanTLS = qio_channel_tls_new_client(
+        QIO_CHANNEL(clientChanSock), clientCreds,
+        data->hostname, &err);
+    g_assert(clientChanTLS != NULL);
+
+    serverChanTLS = qio_channel_tls_new_server(
+        QIO_CHANNEL(serverChanSock), serverCreds,
+        "channeltlsacl", &err);
+    g_assert(serverChanTLS != NULL);
+
+    qio_channel_tls_handshake(clientChanTLS,
+                              test_tls_handshake_done,
+                              &clientHandshake,
+                              NULL);
+    qio_channel_tls_handshake(serverChanTLS,
+                              test_tls_handshake_done,
+                              &serverHandshake,
+                              NULL);
+
+    /*
+     * Finally we loop around & around doing handshake on each
+     * session until we get an error, or the handshake completes.
+     * This relies on the socketpair being nonblocking to avoid
+     * deadlocking ourselves upon handshake
+     */
+    mainloop = g_main_context_default();
+    do {
+        g_main_context_iteration(mainloop, TRUE);
+    } while (!clientHandshake.finished &&
+             !serverHandshake.finished);
+
+    g_assert(clientHandshake.failed == data->expectClientFail);
+    g_assert(serverHandshake.failed == data->expectServerFail);
+
+    test_io_channel_comms(false,
+                          QIO_CHANNEL(clientChanTLS),
+                          QIO_CHANNEL(serverChanTLS));
+
+    test_io_channel_comms(true,
+                          QIO_CHANNEL(clientChanTLS),
+                          QIO_CHANNEL(serverChanTLS));
+
+    unlink(SERVER_CERT_DIR QCRYPTO_TLS_CREDS_X509_CA_CERT);
+    unlink(SERVER_CERT_DIR QCRYPTO_TLS_CREDS_X509_SERVER_CERT);
+    unlink(SERVER_CERT_DIR QCRYPTO_TLS_CREDS_X509_SERVER_KEY);
+
+    unlink(CLIENT_CERT_DIR QCRYPTO_TLS_CREDS_X509_CA_CERT);
+    unlink(CLIENT_CERT_DIR QCRYPTO_TLS_CREDS_X509_CLIENT_CERT);
+    unlink(CLIENT_CERT_DIR QCRYPTO_TLS_CREDS_X509_CLIENT_KEY);
+
+    rmdir(CLIENT_CERT_DIR);
+    rmdir(SERVER_CERT_DIR);
+
+    object_unparent(OBJECT(serverCreds));
+    object_unparent(OBJECT(clientCreds));
+
+    object_unref(OBJECT(serverChanTLS));
+    object_unref(OBJECT(clientChanTLS));
+
+    object_unref(OBJECT(serverChanSock));
+    object_unref(OBJECT(clientChanSock));
+
+    close(channel[0]);
+    close(channel[1]);
+}
+
+
+int main(int argc, char **argv)
+{
+    int ret;
+
+    module_call_init(MODULE_INIT_QOM);
+    g_test_init(&argc, &argv, NULL);
+    setenv("GNUTLS_FORCE_FIPS_MODE", "2", 1);
+
+    mkdir(WORKDIR, 0700);
+
+    test_tls_init(KEYFILE);
+
+# define TEST_CHANNEL(name, caCrt,                                      \
+                      serverCrt, clientCrt,                             \
+                      expectServerFail, expectClientFail,               \
+                      hostname, wildcards)                              \
+    struct QIOChannelTLSTestData name = {                               \
+        caCrt, caCrt, serverCrt, clientCrt,                             \
+        expectServerFail, expectClientFail,                             \
+        hostname, wildcards                                             \
+    };                                                                  \
+    g_test_add_data_func("/qio/channel/tls/" # name,                    \
+                         &name, test_io_channel_tls);
+
+    /* A perfect CA, perfect client & perfect server */
+
+    /* Basic:CA:critical */
+    TLS_ROOT_REQ(cacertreq,
+                 "UK", "qemu CA", NULL, NULL, NULL, NULL,
+                 true, true, true,
+                 true, true, GNUTLS_KEY_KEY_CERT_SIGN,
+                 false, false, NULL, NULL,
+                 0, 0);
+    TLS_CERT_REQ(servercertreq, cacertreq,
+                 "UK", "qemu.org", NULL, NULL, NULL, NULL,
+                 true, true, false,
+                 true, true,
+                 GNUTLS_KEY_DIGITAL_SIGNATURE | GNUTLS_KEY_KEY_ENCIPHERMENT,
+                 true, true, GNUTLS_KP_TLS_WWW_SERVER, NULL,
+                 0, 0);
+    TLS_CERT_REQ(clientcertreq, cacertreq,
+                 "UK", "qemu", NULL, NULL, NULL, NULL,
+                 true, true, false,
+                 true, true,
+                 GNUTLS_KEY_DIGITAL_SIGNATURE | GNUTLS_KEY_KEY_ENCIPHERMENT,
+                 true, true, GNUTLS_KP_TLS_WWW_CLIENT, NULL,
+                 0, 0);
+
+    const char *const wildcards[] = {
+        "C=UK,CN=qemu*",
+        NULL,
+    };
+    TEST_CHANNEL(basic, cacertreq.filename, servercertreq.filename,
+                 clientcertreq.filename, false, false,
+                 "qemu.org", wildcards);
+
+    ret = g_test_run();
+
+    test_tls_discard_cert(&clientcertreq);
+    test_tls_discard_cert(&servercertreq);
+    test_tls_discard_cert(&cacertreq);
+
+    test_tls_cleanup(KEYFILE);
+    rmdir(WORKDIR);
+
+    return ret == 0 ? EXIT_SUCCESS : EXIT_FAILURE;
+}
+
+#else /* ! QCRYPTO_HAVE_TLS_TEST_SUPPORT */
+
+int
+main(void)
+{
+    return EXIT_SUCCESS;
+}
+
+#endif /* ! QCRYPTO_HAVE_TLS_TEST_SUPPORT */
diff --git a/trace-events b/trace-events
index cff3261..5a4fad7 100644
--- a/trace-events
+++ b/trace-events
@@ -1717,3 +1717,13 @@ qio_channel_socket_accept_complete(void *ioc, void *cioc, int fd) "Socket accept
 # io/channel-file.c
 qio_channel_file_new_fd(void *ioc, int fd) "File new fd ioc=%p fd=%d"
 qio_channel_file_new_path(void *ioc, const char *path, int flags, int mode, int fd) "File new fd ioc=%p path=%s flags=%d mode=%d fd=%d"
+
+# io/channel-tls.c
+qio_channel_tls_new_client(void *ioc, void *master, void *creds, const char *hostname) "TLS new client ioc=%p master=%p creds=%p hostname=%s"
+qio_channel_tls_new_server(void *ioc, void *master, void *creds, const char *aclname) "TLS new client ioc=%p master=%p creds=%p acltname=%s"
+qio_channel_tls_handshake_start(void *ioc) "TLS handshake start ioc=%p"
+qio_channel_tls_handshake_pending(void *ioc, int status) "TLS handshake pending ioc=%p status=%d"
+qio_channel_tls_handshake_fail(void *ioc) "TLS handshake fail ioc=%p"
+qio_channel_tls_handshake_complete(void *ioc) "TLS handshake complete ioc=%p"
+qio_channel_tls_credentials_allow(void *ioc) "TLS credentials allow ioc=%p"
+qio_channel_tls_credentials_deny(void *ioc) "TLS credentials deny ioc=%p"
-- 
2.4.3

^ permalink raw reply related	[flat|nested] 32+ messages in thread

* [Qemu-devel] [PATCH v1 14/16] io: add QIOChannelWebsock class
  2015-09-18 13:18 [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework Daniel P. Berrange
                   ` (12 preceding siblings ...)
  2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 13/16] io: add QIOChannelTLS class Daniel P. Berrange
@ 2015-09-18 13:19 ` Daniel P. Berrange
  2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 15/16] io: add QIOChannelCommand class Daniel P. Berrange
                   ` (2 subsequent siblings)
  16 siblings, 0 replies; 32+ messages in thread
From: Daniel P. Berrange @ 2015-09-18 13:19 UTC (permalink / raw)
  To: qemu-devel; +Cc: Paolo Bonzini, Dr. David Alan Gilbert, Gerd Hoffmann

Add a QIOChannel subclass that can run the websocket protocol over
the top of another QIOChannel instance. This initial implementation
is only capable of acting as a websockets server. There is no support
for acting as a websockets client yet.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
 include/io/channel-websock.h | 108 +++++
 io/Makefile.objs             |   1 +
 io/channel-websock.c         | 948 +++++++++++++++++++++++++++++++++++++++++++
 trace-events                 |   8 +
 4 files changed, 1065 insertions(+)
 create mode 100644 include/io/channel-websock.h
 create mode 100644 io/channel-websock.c

diff --git a/include/io/channel-websock.h b/include/io/channel-websock.h
new file mode 100644
index 0000000..8e69d86
--- /dev/null
+++ b/include/io/channel-websock.h
@@ -0,0 +1,108 @@
+/*
+ * QEMU I/O channels driver websockets
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef QIO_CHANNEL_WEBSOCK_H__
+#define QIO_CHANNEL_WEBSOCK_H__
+
+#include "io/channel.h"
+#include "io/buffer.h"
+#include "io/task.h"
+
+#define TYPE_QIO_CHANNEL_WEBSOCK "qio-channel-websock"
+#define QIO_CHANNEL_WEBSOCK(obj)                                     \
+    OBJECT_CHECK(QIOChannelWebsock, (obj), TYPE_QIO_CHANNEL_WEBSOCK)
+
+typedef struct QIOChannelWebsock QIOChannelWebsock;
+typedef union QIOChannelWebsockMask QIOChannelWebsockMask;
+
+union QIOChannelWebsockMask {
+    char c[4];
+    uint32_t u;
+};
+
+/**
+ * QIOChannelWebsock
+ *
+ * The QIOChannelWebsock class provides a channel wrapper which
+ * can transparently run the HTTP websockets protocol. This is
+ * usually used over a TCP socket, but there is actually no
+ * technical restriction on which type of master channel is
+ * used as the transport.
+ *
+ * This channel object is currently only capable of running as
+ * a websocket server and is a pretty crude implementation
+ * of it, not supporting the full websockets protocol feature
+ * set. It is sufficient to use with a simple websockets
+ * client for encapsulating VNC for noVNC in-browser client.
+ */
+
+struct QIOChannelWebsock {
+    QIOChannel parent;
+    QIOChannel *master;
+    QIOBuffer encinput;
+    QIOBuffer encoutput;
+    QIOBuffer rawinput;
+    QIOBuffer rawoutput;
+    size_t payload_remain;
+    QIOChannelWebsockMask mask;
+    guint io_tag;
+    Error *io_err;
+    gboolean io_eof;
+};
+
+/**
+ * qio_channel_websock_new_server:
+ * @master: the underlying channel object
+ *
+ * Create a new websockets channel that runs the server
+ * side of the protocol.
+ *
+ * After creating the channel, it is mandatory to call
+ * the qio_channel_websock_handshake() method before attempting
+ * todo any I/O on the channel.
+ *
+ * Once the handshake has completed, all I/O should be done
+ * via the new websocket channel object and not the original
+ * master channel
+ *
+ * Returns: the new websockets channel object
+ */
+QIOChannelWebsock *
+qio_channel_websock_new_server(QIOChannel *master);
+
+/**
+ * qio_channel_websock_handshake:
+ * @ioc: the websocket channel object
+ * @func: the callback to invoke when completed
+ * @opaque: opaque data to pass to @func
+ * @destroy: optional callback to free @opaque
+ *
+ * Perform the websocket handshake. This method
+ * will return immediately and the handshake will
+ * continue in the background, provided the main
+ * loop is running. When the handshake is complete,
+ * or fails, the @func callback will be invoked.
+ */
+void qio_channel_websock_handshake(QIOChannelWebsock *ioc,
+                                   QIOTaskFunc func,
+                                   gpointer opaque,
+                                   GDestroyNotify destroy);
+
+#endif /* QIO_CHANNEL_WEBSOCK_H__ */
diff --git a/io/Makefile.objs b/io/Makefile.objs
index 2b33d3c..9f93087 100644
--- a/io/Makefile.objs
+++ b/io/Makefile.objs
@@ -5,3 +5,4 @@ io-obj-y += channel-watch.o
 io-obj-y += channel-socket.o
 io-obj-y += channel-file.o
 io-obj-y += channel-tls.o
+io-obj-y += channel-websock.o
diff --git a/io/channel-websock.c b/io/channel-websock.c
new file mode 100644
index 0000000..7031819
--- /dev/null
+++ b/io/channel-websock.c
@@ -0,0 +1,948 @@
+/*
+ * QEMU I/O channels driver websockets
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "io/channel-websock.h"
+#include "crypto/hash.h"
+#include "trace.h"
+
+
+/* Max amount to allow in rawinput/rawoutput buffers */
+#define QIO_CHANNEL_WEBSOCK_MAX_BUFFER 8192
+
+#define QIO_CHANNEL_WEBSOCK_CLIENT_KEY_LEN 24
+#define QIO_CHANNEL_WEBSOCK_GUID "258EAFA5-E914-47DA-95CA-C5AB0DC85B11"
+#define QIO_CHANNEL_WEBSOCK_GUID_LEN strlen(QIO_CHANNEL_WEBSOCK_GUID)
+
+#define QIO_CHANNEL_WEBSOCK_HANDSHAKE_RESPONSE  \
+    "HTTP/1.1 101 Switching Protocols\r\n"      \
+    "Upgrade: websocket\r\n"                    \
+    "Connection: Upgrade\r\n"                   \
+    "Sec-WebSocket-Accept: %s\r\n"              \
+    "Sec-WebSocket-Protocol: binary\r\n"        \
+    "\r\n"
+#define QIO_CHANNEL_WEBSOCK_HANDSHAKE_DELIM "\r\n"
+#define QIO_CHANNEL_WEBSOCK_HANDSHAKE_END "\r\n\r\n"
+#define QIO_CHANNEL_WEBSOCK_SUPPORTED_VERSION "13"
+
+#define QIO_CHANNEL_WEBSOCK_HEAD_MIN_LEN sizeof(uint16_t)
+#define QIO_CHANNEL_WEBSOCK_HEAD_MAX_LEN \
+    (QIO_CHANNEL_WEBSOCK_HEAD_MIN_LEN + sizeof(uint64_t) + sizeof(uint32_t))
+
+typedef struct QIOChannelWebsockHeader QIOChannelWebsockHeader;
+
+struct QEMU_PACKED QIOChannelWebsockHeader {
+    unsigned char b0;
+    unsigned char b1;
+    union {
+        struct QEMU_PACKED {
+            uint16_t l16;
+            QIOChannelWebsockMask m16;
+        } s16;
+        struct QEMU_PACKED {
+            uint64_t l64;
+            QIOChannelWebsockMask m64;
+        } s64;
+        QIOChannelWebsockMask m;
+    } u;
+};
+
+enum {
+    QIO_CHANNEL_WEBSOCK_OPCODE_CONTINUATION = 0x0,
+    QIO_CHANNEL_WEBSOCK_OPCODE_TEXT_FRAME = 0x1,
+    QIO_CHANNEL_WEBSOCK_OPCODE_BINARY_FRAME = 0x2,
+    QIO_CHANNEL_WEBSOCK_OPCODE_CLOSE = 0x8,
+    QIO_CHANNEL_WEBSOCK_OPCODE_PING = 0x9,
+    QIO_CHANNEL_WEBSOCK_OPCODE_PONG = 0xA
+};
+
+static char *qio_channel_websock_handshake_entry(const char *handshake,
+                                                 size_t handshake_len,
+                                                 const char *name)
+{
+    char *begin, *end, *ret = NULL;
+    char *line = g_strdup_printf("%s%s: ",
+                                 QIO_CHANNEL_WEBSOCK_HANDSHAKE_DELIM,
+                                 name);
+    begin = g_strstr_len(handshake, handshake_len, line);
+    if (begin != NULL) {
+        begin += strlen(line);
+        end = g_strstr_len(begin, handshake_len - (begin - handshake),
+                QIO_CHANNEL_WEBSOCK_HANDSHAKE_DELIM);
+        if (end != NULL) {
+            ret = g_strndup(begin, end - begin);
+        }
+    }
+    g_free(line);
+    return ret;
+}
+
+
+static int qio_channel_websock_handshake_send_response(QIOChannelWebsock *ioc,
+                                                       const char *key,
+                                                       Error **errp)
+{
+    char combined_key[QIO_CHANNEL_WEBSOCK_CLIENT_KEY_LEN +
+                      QIO_CHANNEL_WEBSOCK_GUID_LEN + 1];
+    char *accept = NULL, *response = NULL;
+    size_t responselen;
+
+    g_strlcpy(combined_key, key, QIO_CHANNEL_WEBSOCK_CLIENT_KEY_LEN + 1);
+    g_strlcat(combined_key, QIO_CHANNEL_WEBSOCK_GUID,
+              QIO_CHANNEL_WEBSOCK_CLIENT_KEY_LEN +
+              QIO_CHANNEL_WEBSOCK_GUID_LEN + 1);
+
+    /* hash and encode it */
+    if (qcrypto_hash_base64(QCRYPTO_HASH_ALG_SHA1,
+                            combined_key,
+                            QIO_CHANNEL_WEBSOCK_CLIENT_KEY_LEN +
+                            QIO_CHANNEL_WEBSOCK_GUID_LEN,
+                            &accept,
+                            errp) < 0) {
+        return -1;
+    }
+
+    response = g_strdup_printf(QIO_CHANNEL_WEBSOCK_HANDSHAKE_RESPONSE, accept);
+    responselen = strlen(response);
+    qio_buffer_reserve(&ioc->encoutput, responselen);
+    qio_buffer_append(&ioc->encoutput, response, responselen);
+
+    g_free(accept);
+    g_free(response);
+
+    return 0;
+}
+
+static int qio_channel_websock_handshake_process(QIOChannelWebsock *ioc,
+                                                 const char *line,
+                                                 size_t size,
+                                                 Error **errp)
+{
+    int ret = -1;
+    char *protocols = qio_channel_websock_handshake_entry(line, size,
+            "Sec-WebSocket-Protocol");
+    char *version = qio_channel_websock_handshake_entry(line, size,
+            "Sec-WebSocket-Version");
+    char *key = qio_channel_websock_handshake_entry(line, size,
+            "Sec-WebSocket-Key");
+
+    if (!protocols) {
+        error_setg(errp, "Missing websocket protocol header data");
+        goto cleanup;
+    }
+
+    if (!version) {
+        error_setg(errp, "Missing websocket version header data");
+        goto cleanup;
+    }
+
+    if (!key) {
+        error_setg(errp, "Missing websocket key header data");
+        goto cleanup;
+    }
+
+    if (!g_strrstr(protocols, "binary")) {
+        error_setg(errp, "No 'binary' protocol is supported by client '%s'",
+                   protocols);
+        goto cleanup;
+    }
+
+    if (!g_str_equal(version, QIO_CHANNEL_WEBSOCK_SUPPORTED_VERSION)) {
+        error_setg(errp, "Version '%s' is not supported by client '%s'",
+                   QIO_CHANNEL_WEBSOCK_SUPPORTED_VERSION, version);
+        goto cleanup;
+    }
+
+    if (strlen(key) != QIO_CHANNEL_WEBSOCK_CLIENT_KEY_LEN) {
+        error_setg(errp, "Key length '%zu' was not as expected '%d'",
+                   strlen(key), QIO_CHANNEL_WEBSOCK_CLIENT_KEY_LEN);
+        goto cleanup;
+    }
+
+    ret = qio_channel_websock_handshake_send_response(ioc, key, errp);
+
+ cleanup:
+    g_free(protocols);
+    g_free(version);
+    g_free(key);
+    return ret;
+}
+
+static int qio_channel_websock_handshake_read(QIOChannelWebsock *ioc,
+                                              Error **errp)
+{
+    char *handshake_end;
+    ssize_t ret;
+    /* Typical HTTP headers from novnc are 512 bytes, so limiting
+     * total header size to 4096 is easily enough. */
+    size_t want = 4096 - ioc->encinput.offset;
+    qio_buffer_reserve(&ioc->encinput, want);
+    ret = qio_channel_read(ioc->master,
+                           (char *)qio_buffer_end(&ioc->encinput), want, errp);
+    if (ret < 0) {
+        return -1;
+    }
+    ioc->encinput.offset += ret;
+
+    handshake_end = g_strstr_len((char *)ioc->encinput.buffer,
+                                 ioc->encinput.offset,
+                                 QIO_CHANNEL_WEBSOCK_HANDSHAKE_END);
+    if (!handshake_end) {
+        if (ioc->encinput.offset >= 4096) {
+            error_setg(errp,
+                       "End of headers not found in first 4096 bytes");
+            return -1;
+        } else {
+            return 0;
+        }
+    }
+
+    if (qio_channel_websock_handshake_process(ioc,
+                                              (char *)ioc->encinput.buffer,
+                                              ioc->encinput.offset,
+                                              errp) < 0) {
+        return -1;
+    }
+
+    qio_buffer_advance(&ioc->encinput,
+                       handshake_end - (char *)ioc->encinput.buffer +
+                       strlen(QIO_CHANNEL_WEBSOCK_HANDSHAKE_END));
+    return 1;
+}
+
+static gboolean qio_channel_websock_handshake_send(QIOChannel *ioc,
+                                                   GIOCondition condition,
+                                                   gpointer user_data)
+{
+    QIOTask *task = user_data;
+    QIOChannelWebsock *wioc = QIO_CHANNEL_WEBSOCK(
+        qio_task_get_source(task));
+    Error *err = NULL;
+    ssize_t ret;
+
+    ret = qio_channel_write(wioc->master,
+                            (char *)wioc->encoutput.buffer,
+                            wioc->encoutput.offset,
+                            &err);
+
+    if (ret < 0) {
+        trace_qio_channel_websock_handshake_fail(ioc);
+        qio_task_abort(task, err);
+        error_free(err);
+        return FALSE;
+    }
+
+    qio_buffer_advance(&wioc->encoutput, ret);
+    if (wioc->encoutput.offset == 0) {
+        trace_qio_channel_websock_handshake_complete(ioc);
+        qio_task_complete(task);
+        return FALSE;
+    }
+    trace_qio_channel_websock_handshake_pending(ioc, G_IO_OUT);
+    return TRUE;
+}
+
+static gboolean qio_channel_websock_handshake_io(QIOChannel *ioc,
+                                                 GIOCondition condition,
+                                                 gpointer user_data)
+{
+    QIOTask *task = user_data;
+    QIOChannelWebsock *wioc = QIO_CHANNEL_WEBSOCK(
+        qio_task_get_source(task));
+    Error *err = NULL;
+    int ret;
+
+    ret = qio_channel_websock_handshake_read(wioc, &err);
+    if (ret < 0) {
+        trace_qio_channel_websock_handshake_fail(ioc);
+        qio_task_abort(task, err);
+        error_free(err);
+        return FALSE;
+    }
+    if (ret == 0) {
+        trace_qio_channel_websock_handshake_pending(ioc, G_IO_IN);
+        /* need more data still */
+        return TRUE;
+    }
+
+    object_ref(OBJECT(task));
+    trace_qio_channel_websock_handshake_reply(ioc);
+    qio_channel_add_watch(
+        wioc->master,
+        G_IO_OUT,
+        qio_channel_websock_handshake_send,
+        task,
+        (GDestroyNotify)object_unref);
+    return FALSE;
+}
+
+
+static void qio_channel_websock_encode(QIOChannelWebsock *ioc)
+{
+    size_t header_size = 0;
+    unsigned char opcode = QIO_CHANNEL_WEBSOCK_OPCODE_BINARY_FRAME;
+    union {
+        char buf[QIO_CHANNEL_WEBSOCK_HEAD_MAX_LEN];
+        QIOChannelWebsockHeader ws;
+    } header;
+
+    if (!ioc->rawoutput.offset) {
+        return;
+    }
+
+    header.ws.b0 = 0x80 | (opcode & 0x0f);
+    if (ioc->rawoutput.offset <= 125) {
+        header.ws.b1 = (uint8_t)ioc->rawoutput.offset;
+        header_size = 2;
+    } else if (ioc->rawoutput.offset < 65536) {
+        header.ws.b1 = 0x7e;
+        header.ws.u.s16.l16 = cpu_to_be16((uint16_t)ioc->rawoutput.offset);
+        header_size = 4;
+    } else {
+        header.ws.b1 = 0x7f;
+        header.ws.u.s64.l64 = cpu_to_be64(ioc->rawoutput.offset);
+        header_size = 10;
+    }
+
+    qio_buffer_reserve(&ioc->encoutput, header_size + ioc->rawoutput.offset);
+    qio_buffer_append(&ioc->encoutput, header.buf, header_size);
+    qio_buffer_append(&ioc->encoutput, ioc->rawoutput.buffer,
+                      ioc->rawoutput.offset);
+    qio_buffer_reset(&ioc->rawoutput);
+}
+
+
+static ssize_t qio_channel_websock_decode_header(QIOChannelWebsock *ioc,
+                                                 Error **errp)
+{
+    unsigned char opcode = 0, fin = 0, has_mask = 0;
+    size_t header_size = 0;
+    size_t payload_len;
+    QIOChannelWebsockHeader *header =
+        (QIOChannelWebsockHeader *)ioc->encinput.buffer;
+
+    if (ioc->payload_remain) {
+        error_setg(errp,
+                   "Decoding header but %zu bytes of payload remain",
+                   ioc->payload_remain);
+        return -1;
+    }
+    if (ioc->encinput.offset < QIO_CHANNEL_WEBSOCK_HEAD_MIN_LEN + 4) {
+        /* header not complete */
+        return QIO_CHANNEL_ERR_BLOCK;
+    }
+
+    fin = (header->b0 & 0x80) >> 7;
+    opcode = header->b0 & 0x0f;
+    has_mask = (header->b1 & 0x80) >> 7;
+    payload_len = header->b1 & 0x7f;
+
+    if (opcode == QIO_CHANNEL_WEBSOCK_OPCODE_CLOSE) {
+        /* disconnect */
+        return 0;
+    }
+
+    /* Websocket frame sanity check:
+     * * Websocket fragmentation is not supported.
+     * * All  websockets frames sent by a client have to be masked.
+     * * Only binary encoding is supported.
+     */
+    if (!fin) {
+        error_setg(errp, "websocket fragmentation is not supported");
+        return -1;
+    }
+    if (!has_mask) {
+        error_setg(errp, "websocket frames must be masked");
+        return -1;
+    }
+    if (opcode != QIO_CHANNEL_WEBSOCK_OPCODE_BINARY_FRAME) {
+        error_setg(errp, "only binary websocket frames are supported");
+        return -1;
+    }
+
+    if (payload_len < 126) {
+        ioc->payload_remain = payload_len;
+        header_size = 6;
+        ioc->mask = header->u.m;
+    } else if (payload_len == 126 && ioc->encinput.offset >= 8) {
+        ioc->payload_remain = be16_to_cpu(header->u.s16.l16);
+        header_size = 8;
+        ioc->mask = header->u.s16.m16;
+    } else if (payload_len == 127 && ioc->encinput.offset >= 14) {
+        ioc->payload_remain = be64_to_cpu(header->u.s64.l64);
+        header_size = 14;
+        ioc->mask = header->u.s64.m64;
+    } else {
+        /* header not complete */
+        return QIO_CHANNEL_ERR_BLOCK;
+    }
+
+    qio_buffer_advance(&ioc->encinput, header_size);
+    return 1;
+}
+
+
+static ssize_t qio_channel_websock_decode_payload(QIOChannelWebsock *ioc,
+                                                  Error **errp)
+{
+    size_t i;
+    size_t payload_len;
+    uint32_t *payload32;
+
+    if (!ioc->payload_remain) {
+        error_setg(errp,
+                   "Decoding payload but no bytes of payload remain");
+        return -1;
+    }
+
+    /* If we aren't at the end of the payload, then drop
+     * off the last bytes, so we're always multiple of 4
+     * for purpose of unmasking, except at end of payload
+     */
+    if (ioc->encinput.offset < ioc->payload_remain) {
+        payload_len = ioc->encinput.offset - (ioc->encinput.offset % 4);
+    } else {
+        payload_len = ioc->payload_remain;
+    }
+    if (payload_len == 0) {
+        return QIO_CHANNEL_ERR_BLOCK;
+    }
+
+    ioc->payload_remain -= payload_len;
+
+    /* unmask frame */
+    /* process 1 frame (32 bit op) */
+    payload32 = (uint32_t *)ioc->encinput.buffer;
+    for (i = 0; i < payload_len / 4; i++) {
+        payload32[i] ^= ioc->mask.u;
+    }
+    /* process the remaining bytes (if any) */
+    for (i *= 4; i < payload_len; i++) {
+        ioc->encinput.buffer[i] ^= ioc->mask.c[i % 4];
+    }
+
+    qio_buffer_reserve(&ioc->rawinput, payload_len);
+    qio_buffer_append(&ioc->rawinput, ioc->encinput.buffer, payload_len);
+    qio_buffer_advance(&ioc->encinput, payload_len);
+    return payload_len;
+}
+
+
+QIOChannelWebsock *
+qio_channel_websock_new_server(QIOChannel *master)
+{
+    QIOChannelWebsock *wioc;
+
+    wioc = QIO_CHANNEL_WEBSOCK(object_new(TYPE_QIO_CHANNEL_WEBSOCK));
+
+    wioc->master = master;
+    object_ref(OBJECT(master));
+
+    trace_qio_channel_websock_new_server(wioc, master);
+    return wioc;
+}
+
+void qio_channel_websock_handshake(QIOChannelWebsock *ioc,
+                                   QIOTaskFunc func,
+                                   gpointer opaque,
+                                   GDestroyNotify destroy)
+{
+    QIOTask *task;
+
+    task = qio_task_new(OBJECT(ioc),
+                        func,
+                        opaque,
+                        destroy);
+
+    trace_qio_channel_websock_handshake_start(ioc);
+    trace_qio_channel_websock_handshake_pending(ioc, G_IO_IN);
+    qio_channel_add_watch(ioc->master,
+                          G_IO_IN,
+                          qio_channel_websock_handshake_io,
+                          task,
+                          NULL);
+}
+
+
+static void qio_channel_websock_finalize(Object *obj)
+{
+    QIOChannelWebsock *ioc = QIO_CHANNEL_WEBSOCK(obj);
+
+    qio_buffer_free(&ioc->encinput);
+    qio_buffer_free(&ioc->encoutput);
+    qio_buffer_free(&ioc->rawinput);
+    qio_buffer_free(&ioc->rawoutput);
+    object_unref(OBJECT(ioc->master));
+    if (ioc->io_tag) {
+        g_source_remove(ioc->io_tag);
+    }
+    if (ioc->io_err) {
+        error_free(ioc->io_err);
+    }
+}
+
+
+static bool qio_channel_websock_has_feature(QIOChannel *ioc,
+                                            QIOChannelFeature feature)
+{
+    QIOChannelWebsock *wioc = QIO_CHANNEL_WEBSOCK(ioc);
+
+    switch (feature) {
+    case QIO_CHANNEL_FEATURE_SHUTDOWN:
+    case QIO_CHANNEL_FEATURE_CORK:
+    case QIO_CHANNEL_FEATURE_DELAY:
+        return qio_channel_has_feature(wioc->master, feature);
+    default:
+        return false;
+    }
+}
+
+
+static ssize_t qio_channel_websock_read_wire(QIOChannelWebsock *ioc,
+                                             Error **errp)
+{
+    ssize_t ret;
+
+    if (ioc->encinput.offset < 4096) {
+        size_t want = 4096 - ioc->encinput.offset;
+
+        qio_buffer_reserve(&ioc->encinput, want);
+        ret = qio_channel_read(ioc->master,
+                               (char *)ioc->encinput.buffer +
+                               ioc->encinput.offset,
+                               want,
+                               errp);
+        if (ret < 0) {
+            return ret;
+        }
+        if (ret == 0 &&
+            ioc->encinput.offset == 0) {
+            return 0;
+        }
+        ioc->encinput.offset += ret;
+    }
+
+    if (ioc->payload_remain == 0) {
+        ret = qio_channel_websock_decode_header(ioc, errp);
+        if (ret < 0) {
+            return ret;
+        }
+        if (ret == 0) {
+            return 0;
+        }
+    }
+
+    ret = qio_channel_websock_decode_payload(ioc, errp);
+    if (ret < 0) {
+        return ret;
+    }
+    return ret;
+}
+
+
+static ssize_t qio_channel_websock_write_wire(QIOChannelWebsock *ioc,
+                                              Error **errp)
+{
+    ssize_t ret;
+    ssize_t done = 0;
+    qio_channel_websock_encode(ioc);
+
+    while (ioc->encoutput.offset > 0) {
+        ret = qio_channel_write(ioc->master,
+                                (char *)ioc->encoutput.buffer,
+                                ioc->encoutput.offset,
+                                errp);
+        if (ret < 0) {
+            if (ret == QIO_CHANNEL_ERR_BLOCK &&
+                done > 0) {
+                return done;
+            } else {
+                return ret;
+            }
+        }
+        qio_buffer_advance(&ioc->encoutput, ret);
+        done += ret;
+    }
+    return done;
+}
+
+
+static void qio_channel_websock_flush_free(gpointer user_data)
+{
+    QIOChannelWebsock *wioc = QIO_CHANNEL_WEBSOCK(user_data);
+    object_unref(OBJECT(wioc));
+}
+
+static void qio_channel_websock_set_watch(QIOChannelWebsock *ioc);
+
+static gboolean qio_channel_websock_flush(QIOChannel *ioc,
+                                          GIOCondition condition,
+                                          gpointer user_data)
+{
+    QIOChannelWebsock *wioc = QIO_CHANNEL_WEBSOCK(user_data);
+    ssize_t ret;
+
+    if (condition & G_IO_OUT) {
+        ret = qio_channel_websock_write_wire(wioc, &wioc->io_err);
+        if (ret < 0) {
+            goto cleanup;
+        }
+    }
+
+    if (condition & G_IO_IN) {
+        ret = qio_channel_websock_read_wire(wioc, &wioc->io_err);
+        if (ret < 0) {
+            goto cleanup;
+        }
+        if (ret == 0) {
+            wioc->io_eof = TRUE;
+        }
+    }
+
+ cleanup:
+    qio_channel_websock_set_watch(wioc);
+    return FALSE;
+}
+
+
+static void qio_channel_websock_unset_watch(QIOChannelWebsock *ioc)
+{
+    if (ioc->io_tag) {
+        g_source_remove(ioc->io_tag);
+        ioc->io_tag = 0;
+    }
+}
+
+static void qio_channel_websock_set_watch(QIOChannelWebsock *ioc)
+{
+    GIOCondition cond = 0;
+
+    qio_channel_websock_unset_watch(ioc);
+
+    if (ioc->io_err) {
+        return;
+    }
+
+    if (ioc->encoutput.offset) {
+        cond |= G_IO_OUT;
+    }
+    if (ioc->encinput.offset < QIO_CHANNEL_WEBSOCK_MAX_BUFFER &&
+        !ioc->io_eof) {
+        cond |= G_IO_IN;
+    }
+
+    if (cond) {
+        object_ref(OBJECT(ioc));
+        ioc->io_tag =
+            qio_channel_add_watch(ioc->master,
+                                  cond,
+                                  qio_channel_websock_flush,
+                                  ioc,
+                                  qio_channel_websock_flush_free);
+    }
+}
+
+
+static ssize_t qio_channel_websock_readv(QIOChannel *ioc,
+                                         const struct iovec *iov,
+                                         size_t niov,
+                                         int **fds,
+                                         size_t *nfds,
+                                         Error **errp)
+{
+    QIOChannelWebsock *wioc = QIO_CHANNEL_WEBSOCK(ioc);
+    size_t i;
+    ssize_t got = 0;
+    ssize_t ret;
+
+    if (fds || nfds) {
+        error_setg(errp,
+                   "Cannot receive file descriptors over websocket channel");
+        return -1;
+    }
+
+    if (wioc->io_err) {
+        *errp = error_copy(wioc->io_err);
+        return -1;
+    }
+
+    if (!wioc->rawinput.offset) {
+        ret = qio_channel_websock_read_wire(QIO_CHANNEL_WEBSOCK(ioc), errp);
+        if (ret < 0) {
+            return ret;
+        }
+    }
+
+    for (i = 0 ; i < niov ; i++) {
+        size_t want = iov[i].iov_len;
+        if (want > (wioc->rawinput.offset - got)) {
+            want = (wioc->rawinput.offset - got);
+        }
+
+        memcpy(iov[i].iov_base,
+               wioc->rawinput.buffer + got,
+               want);
+        got += want;
+
+        if (want < iov[i].iov_len) {
+            break;
+        }
+    }
+
+    qio_buffer_advance(&wioc->rawinput, got);
+    qio_channel_websock_set_watch(wioc);
+    return got;
+}
+
+
+static ssize_t qio_channel_websock_writev(QIOChannel *ioc,
+                                          const struct iovec *iov,
+                                          size_t niov,
+                                          int *fds,
+                                          size_t nfds,
+                                          Error **errp)
+{
+    QIOChannelWebsock *wioc = QIO_CHANNEL_WEBSOCK(ioc);
+    size_t i;
+    ssize_t done = 0;
+    ssize_t ret;
+
+    if (fds || nfds) {
+        error_setg(errp,
+                   "Cannot send file descriptors over websocket channel");
+        return -1;
+    }
+
+    if (wioc->io_err) {
+        *errp = error_copy(wioc->io_err);
+        return -1;
+    }
+
+    if (wioc->io_eof) {
+        error_setg(errp, "%s", "Broken pipe");
+        return -1;
+    }
+
+    for (i = 0; i < niov; i++) {
+        size_t want = iov[i].iov_len;
+        if ((want + wioc->rawoutput.offset) > QIO_CHANNEL_WEBSOCK_MAX_BUFFER) {
+            want = (QIO_CHANNEL_WEBSOCK_MAX_BUFFER - wioc->rawoutput.offset);
+        }
+        if (want == 0) {
+            goto done;
+        }
+
+        qio_buffer_reserve(&wioc->rawoutput, want);
+        qio_buffer_append(&wioc->rawoutput, iov[i].iov_base, want);
+        done += want;
+        if (want < iov[i].iov_len) {
+            break;
+        }
+    }
+
+ done:
+    ret = qio_channel_websock_write_wire(wioc, errp);
+    if (ret < 0 &&
+        ret != QIO_CHANNEL_ERR_BLOCK) {
+        qio_channel_websock_unset_watch(wioc);
+        return -1;
+    }
+
+    qio_channel_websock_set_watch(wioc);
+
+    if (done == 0) {
+        return QIO_CHANNEL_ERR_BLOCK;
+    }
+
+    return done;
+}
+
+static int qio_channel_websock_set_blocking(QIOChannel *ioc,
+                                            bool enabled,
+                                            Error **errp)
+{
+    QIOChannelWebsock *wioc = QIO_CHANNEL_WEBSOCK(ioc);
+
+    qio_channel_set_blocking(wioc->master, enabled, errp);
+    return 0;
+}
+
+static int qio_channel_websock_set_delay(QIOChannel *ioc,
+                                         bool enabled,
+                                         Error **errp)
+{
+    QIOChannelWebsock *tioc = QIO_CHANNEL_WEBSOCK(ioc);
+
+    return qio_channel_set_delay(tioc->master, enabled, errp);
+}
+
+static int qio_channel_websock_set_cork(QIOChannel *ioc,
+                                        bool enabled,
+                                        Error **errp)
+{
+    QIOChannelWebsock *tioc = QIO_CHANNEL_WEBSOCK(ioc);
+
+    return qio_channel_set_cork(tioc->master, enabled, errp);
+}
+
+static int qio_channel_websock_shutdown(QIOChannel *ioc,
+                                        QIOChannelShutdown how,
+                                        Error **errp)
+{
+    QIOChannelWebsock *tioc = QIO_CHANNEL_WEBSOCK(ioc);
+
+    return qio_channel_shutdown(tioc->master, how, errp);
+}
+
+static int qio_channel_websock_close(QIOChannel *ioc,
+                                     Error **errp)
+{
+    QIOChannelWebsock *wioc = QIO_CHANNEL_WEBSOCK(ioc);
+
+    return qio_channel_close(wioc->master, errp);
+}
+
+typedef struct QIOChannelWebsockSource QIOChannelWebsockSource;
+struct QIOChannelWebsockSource {
+    GSource parent;
+    QIOChannelWebsock *wioc;
+    GIOCondition condition;
+};
+
+static gboolean
+qio_channel_websock_source_prepare(GSource *source,
+                                   gint *timeout)
+{
+    QIOChannelWebsockSource *wsource = (QIOChannelWebsockSource *)source;
+    GIOCondition cond = 0;
+    *timeout = -1;
+
+    if (wsource->wioc->rawinput.offset) {
+        cond |= G_IO_IN;
+    }
+    if (wsource->wioc->rawoutput.offset < QIO_CHANNEL_WEBSOCK_MAX_BUFFER) {
+        cond |= G_IO_OUT;
+    }
+
+    return cond & wsource->condition;
+}
+
+static gboolean
+qio_channel_websock_source_check(GSource *source)
+{
+    QIOChannelWebsockSource *wsource = (QIOChannelWebsockSource *)source;
+    GIOCondition cond = 0;
+
+    if (wsource->wioc->rawinput.offset) {
+        cond |= G_IO_IN;
+    }
+    if (wsource->wioc->rawoutput.offset < QIO_CHANNEL_WEBSOCK_MAX_BUFFER) {
+        cond |= G_IO_OUT;
+    }
+
+    return cond & wsource->condition;
+}
+
+static gboolean
+qio_channel_websock_source_dispatch(GSource *source,
+                                    GSourceFunc callback,
+                                    gpointer user_data)
+{
+    QIOChannelFunc func = (QIOChannelFunc)callback;
+    QIOChannelWebsockSource *wsource = (QIOChannelWebsockSource *)source;
+    GIOCondition cond = 0;
+
+    if (wsource->wioc->rawinput.offset) {
+        cond |= G_IO_IN;
+    }
+    if (wsource->wioc->rawoutput.offset < QIO_CHANNEL_WEBSOCK_MAX_BUFFER) {
+        cond |= G_IO_OUT;
+    }
+
+    return (*func)(QIO_CHANNEL(wsource->wioc),
+                   (cond & wsource->condition),
+                   user_data);
+}
+
+static void
+qio_channel_websock_source_finalize(GSource *source)
+{
+    QIOChannelWebsockSource *ssource = (QIOChannelWebsockSource *)source;
+
+    object_unref(OBJECT(ssource->wioc));
+}
+
+GSourceFuncs qio_channel_websock_source_funcs = {
+    qio_channel_websock_source_prepare,
+    qio_channel_websock_source_check,
+    qio_channel_websock_source_dispatch,
+    qio_channel_websock_source_finalize
+};
+
+static GSource *qio_channel_websock_create_watch(QIOChannel *ioc,
+                                                 GIOCondition condition)
+{
+    QIOChannelWebsock *wioc = QIO_CHANNEL_WEBSOCK(ioc);
+    QIOChannelWebsockSource *ssource;
+    GSource *source;
+
+    source = g_source_new(&qio_channel_websock_source_funcs,
+                          sizeof(QIOChannelWebsockSource));
+    g_source_set_name(source, "QIOChannelWebsock");
+    ssource = (QIOChannelWebsockSource *)source;
+
+    ssource->wioc = wioc;
+    object_ref(OBJECT(wioc));
+
+    ssource->condition = condition;
+
+    qio_channel_websock_set_watch(wioc);
+    return source;
+}
+
+static void qio_channel_websock_class_init(ObjectClass *klass,
+                                           void *class_data G_GNUC_UNUSED)
+{
+    QIOChannelClass *ioc_klass = QIO_CHANNEL_CLASS(klass);
+
+    ioc_klass->io_has_feature = qio_channel_websock_has_feature;
+    ioc_klass->io_writev = qio_channel_websock_writev;
+    ioc_klass->io_readv = qio_channel_websock_readv;
+    ioc_klass->io_set_blocking = qio_channel_websock_set_blocking;
+    ioc_klass->io_set_cork = qio_channel_websock_set_cork;
+    ioc_klass->io_set_delay = qio_channel_websock_set_delay;
+    ioc_klass->io_close = qio_channel_websock_close;
+    ioc_klass->io_shutdown = qio_channel_websock_shutdown;
+    ioc_klass->io_create_watch = qio_channel_websock_create_watch;
+}
+
+static const TypeInfo qio_channel_websock_info = {
+    .parent = TYPE_QIO_CHANNEL,
+    .name = TYPE_QIO_CHANNEL_WEBSOCK,
+    .instance_size = sizeof(QIOChannelWebsock),
+    .instance_finalize = qio_channel_websock_finalize,
+    .class_init = qio_channel_websock_class_init,
+};
+
+static void qio_channel_websock_register_types(void)
+{
+    type_register_static(&qio_channel_websock_info);
+}
+
+type_init(qio_channel_websock_register_types);
diff --git a/trace-events b/trace-events
index 5a4fad7..fad6a27 100644
--- a/trace-events
+++ b/trace-events
@@ -1727,3 +1727,11 @@ qio_channel_tls_handshake_fail(void *ioc) "TLS handshake fail ioc=%p"
 qio_channel_tls_handshake_complete(void *ioc) "TLS handshake complete ioc=%p"
 qio_channel_tls_credentials_allow(void *ioc) "TLS credentials allow ioc=%p"
 qio_channel_tls_credentials_deny(void *ioc) "TLS credentials deny ioc=%p"
+
+# io/channel-websock.c
+qio_channel_websock_new_server(void *ioc, void *master) "Websock new client ioc=%p master=%p"
+qio_channel_websock_handshake_start(void *ioc) "Websock handshake start ioc=%p"
+qio_channel_websock_handshake_pending(void *ioc, int status) "Websock handshake pending ioc=%p status=%d"
+qio_channel_websock_handshake_reply(void *ioc) "Websock handshake reply ioc=%p"
+qio_channel_websock_handshake_fail(void *ioc) "Websock handshake fail ioc=%p"
+qio_channel_websock_handshake_complete(void *ioc) "Websock handshake complete ioc=%p"
-- 
2.4.3

^ permalink raw reply related	[flat|nested] 32+ messages in thread

* [Qemu-devel] [PATCH v1 15/16] io: add QIOChannelCommand class
  2015-09-18 13:18 [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework Daniel P. Berrange
                   ` (13 preceding siblings ...)
  2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 14/16] io: add QIOChannelWebsock class Daniel P. Berrange
@ 2015-09-18 13:19 ` Daniel P. Berrange
  2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 16/16] io: add QIOChannelBuffer class Daniel P. Berrange
  2015-09-22 12:23 ` [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework Paolo Bonzini
  16 siblings, 0 replies; 32+ messages in thread
From: Daniel P. Berrange @ 2015-09-18 13:19 UTC (permalink / raw)
  To: qemu-devel; +Cc: Paolo Bonzini, Dr. David Alan Gilbert, Gerd Hoffmann

Add a QIOChannel subclass that is capable of performing I/O
to/from a separate process, via a pair of pipes. The command
can be used for unidirectional or bi-directional I/O.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
 include/io/channel-command.h    |  91 ++++++++++
 io/Makefile.objs                |   1 +
 io/channel-command.c            | 369 ++++++++++++++++++++++++++++++++++++++++
 tests/.gitignore                |   2 +
 tests/Makefile                  |   3 +
 tests/test-io-channel-command.c | 122 +++++++++++++
 trace-events                    |   6 +
 7 files changed, 594 insertions(+)
 create mode 100644 include/io/channel-command.h
 create mode 100644 io/channel-command.c
 create mode 100644 tests/test-io-channel-command.c

diff --git a/include/io/channel-command.h b/include/io/channel-command.h
new file mode 100644
index 0000000..bd3c599
--- /dev/null
+++ b/include/io/channel-command.h
@@ -0,0 +1,91 @@
+/*
+ * QEMU I/O channels external command driver
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef QIO_CHANNEL_COMMAND_H__
+#define QIO_CHANNEL_COMMAND_H__
+
+#include "io/channel.h"
+
+#define TYPE_QIO_CHANNEL_COMMAND "qio-channel-command"
+#define QIO_CHANNEL_COMMAND(obj)                                     \
+    OBJECT_CHECK(QIOChannelCommand, (obj), TYPE_QIO_CHANNEL_COMMAND)
+
+typedef struct QIOChannelCommand QIOChannelCommand;
+
+
+/**
+ * QIOChannelCommand:
+ *
+ * The QIOChannelCommand class provides a channel implementation
+ * that can transport data with an externally running command
+ * via its stdio streams.
+ */
+
+struct QIOChannelCommand {
+    QIOChannel parent;
+    int writefd;
+    int readfd;
+    pid_t pid;
+};
+
+
+/**
+ * qio_channel_command_new_pid:
+ * @writefd: the FD connected to the command's stdin
+ * @readfd: the FD connected to the command's stdout
+ * @pid: the PID of the running child command
+ * @errp: pointer to an uninitialized error object
+ *
+ * Create a channel for performing I/O with the
+ * previously spawned command identified by @pid.
+ * The two file descriptors provide the connection
+ * to command's stdio streams, either one or which
+ * may be -1 to indicate that stream is not open.
+ *
+ * The channel will take ownership of the process
+ * @pid and will kill it when closing the channel.
+ * Similarly it will take responsibility for
+ * closing the file descriptors @writefd and @readfd.
+ *
+ * Returns: the command channel object, or NULL on error
+ */
+QIOChannelCommand *
+qio_channel_command_new_pid(int writefd,
+                            int readfd,
+                            pid_t pid);
+
+/**
+ * qio_channel_command_new_spawn:
+ * @argv: the NULL terminated list of command arguments
+ * @flags: the I/O mode, one of O_RDONLY, O_WRONLY, O_RDWR
+ * @errp: pointer to an uninitialized error object
+ *
+ * Create a channel for performing I/O with the
+ * command to be spawned with arguments @argv.
+ *
+ * Returns: the command channel object, or NULL on error
+ */
+QIOChannelCommand *
+qio_channel_command_new_spawn(const char *const argv[],
+                              int flags,
+                              Error **errp);
+
+
+#endif /* QIO_CHANNEL_COMMAND_H__ */
diff --git a/io/Makefile.objs b/io/Makefile.objs
index 9f93087..9e28ee5 100644
--- a/io/Makefile.objs
+++ b/io/Makefile.objs
@@ -6,3 +6,4 @@ io-obj-y += channel-socket.o
 io-obj-y += channel-file.o
 io-obj-y += channel-tls.o
 io-obj-y += channel-websock.o
+io-obj-y += channel-command.o
diff --git a/io/channel-command.c b/io/channel-command.c
new file mode 100644
index 0000000..d735efb
--- /dev/null
+++ b/io/channel-command.c
@@ -0,0 +1,369 @@
+/*
+ * QEMU I/O channels external command driver
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "io/channel-command.h"
+#include "io/channel-watch.h"
+#include "qemu/sockets.h"
+#include "trace.h"
+
+
+QIOChannelCommand *
+qio_channel_command_new_pid(int writefd,
+                            int readfd,
+                            pid_t pid)
+{
+    QIOChannelCommand *ioc;
+
+    ioc = QIO_CHANNEL_COMMAND(object_new(TYPE_QIO_CHANNEL_COMMAND));
+
+    ioc->readfd = readfd;
+    ioc->writefd = writefd;
+    ioc->pid = pid;
+
+    trace_qio_channel_command_new_pid(ioc, writefd, readfd, pid);
+    return ioc;
+}
+
+
+#ifndef WIN32
+QIOChannelCommand *
+qio_channel_command_new_spawn(const char *const argv[],
+                              int flags,
+                              Error **errp)
+{
+    pid_t pid = -1;
+    int stdinfd[2] = { -1, -1 };
+    int stdoutfd[2] = { -1, -1 };
+    int devnull = -1;
+    bool stdinnull = false, stdoutnull = false;
+    QIOChannelCommand *ioc;
+
+    flags = flags & O_ACCMODE;
+
+    if (flags == O_RDONLY) {
+        stdinnull = true;
+    }
+    if (flags == O_WRONLY) {
+        stdoutnull = true;
+    }
+
+    if (stdinnull || stdoutnull) {
+        devnull = open("/dev/null", O_RDWR);
+        if (!devnull) {
+            error_setg_errno(errp, errno,
+                             "Unable to open /dev/null");
+            goto error;
+        }
+    }
+
+    if ((!stdinnull && pipe(stdinfd) < 0) ||
+        (!stdoutnull && pipe(stdoutfd) < 0)) {
+        error_setg_errno(errp, errno,
+                         "Unable to open pipe");
+        goto error;
+    }
+
+    pid = qemu_fork(errp);
+    if (pid < 0) {
+        goto error;
+    }
+
+    if (pid == 0) { /* child */
+        dup2(stdinnull ? devnull : stdinfd[0], STDIN_FILENO);
+        dup2(stdoutnull ? devnull : stdoutfd[1], STDOUT_FILENO);
+        /* Leave stderr connected to qemu's stderr */
+
+        if (!stdinnull) {
+            close(stdinfd[0]);
+            close(stdinfd[1]);
+        }
+        if (!stdoutnull) {
+            close(stdoutfd[0]);
+            close(stdoutfd[1]);
+        }
+
+        execv(argv[0], (char * const *)argv);
+        _exit(1);
+    }
+
+    if (!stdinnull) {
+        close(stdinfd[0]);
+    }
+    if (!stdoutnull) {
+        close(stdoutfd[1]);
+    }
+
+    ioc = qio_channel_command_new_pid(stdinnull ? devnull : stdinfd[1],
+                                      stdoutnull ? devnull : stdoutfd[0],
+                                      pid);
+    trace_qio_channel_command_new_spawn(ioc, argv[0], flags);
+    return ioc;
+
+ error:
+    if (stdinfd[0] != -1) {
+        close(stdinfd[0]);
+    }
+    if (stdinfd[1] != -1) {
+        close(stdinfd[1]);
+    }
+    if (stdoutfd[0] != -1) {
+        close(stdoutfd[0]);
+    }
+    if (stdoutfd[1] != -1) {
+        close(stdoutfd[1]);
+    }
+    return NULL;
+}
+
+#else /* WIN32 */
+QIOChannelCommand *
+qio_channel_command_new_spawn(const char *const argv[],
+                              int flags,
+                              Error **errp)
+{
+    error_setg_errno(errp, ENOSYS,
+                     "Command spawn not supported on this platform");
+    return NULL;
+}
+#endif /* WIN32 */
+
+#ifndef WIN32
+static int qio_channel_command_abort(QIOChannelCommand *ioc,
+                                     Error **errp)
+{
+    pid_t ret;
+    int status;
+    int step = 0;
+
+    /* See if intermediate process has exited; if not, try a nice
+     * SIGTERM followed by a more severe SIGKILL.
+     */
+ rewait:
+    trace_qio_channel_command_abort(ioc, ioc->pid);
+    ret = waitpid(ioc->pid, &status, WNOHANG);
+    trace_qio_channel_command_wait(ioc, ioc->pid, ret, status);
+    if (ret == (pid_t)-1) {
+        if (errno == EINTR) {
+            goto rewait;
+        } else {
+            error_setg_errno(errp, errno,
+                             "Cannot wait on pid %llu",
+                             (unsigned long long)ioc->pid);
+            return -1;
+        }
+    } else if (ret == 0) {
+        if (step == 0) {
+            kill(ioc->pid, SIGTERM);
+        } else if (step == 1) {
+            kill(ioc->pid, SIGKILL);
+        } else {
+            error_setg(errp,
+                       "Process %llu refused to die",
+                       (unsigned long long)ioc->pid);
+            return -1;
+        }
+        usleep(10 * 1000);
+        goto rewait;
+    }
+
+    return 0;
+}
+#endif /* ! WIN32 */
+
+
+static void qio_channel_command_init(Object *obj)
+{
+    QIOChannelCommand *ioc = QIO_CHANNEL_COMMAND(obj);
+    ioc->readfd = -1;
+    ioc->writefd = -1;
+    ioc->pid = -1;
+}
+
+static void qio_channel_command_finalize(Object *obj)
+{
+    QIOChannelCommand *ioc = QIO_CHANNEL_COMMAND(obj);
+    if (ioc->readfd != -1) {
+        close(ioc->readfd);
+        ioc->readfd = -1;
+    }
+    if (ioc->writefd != -1) {
+        close(ioc->writefd);
+        ioc->writefd = -1;
+    }
+    if (ioc->pid > 0) {
+#ifndef WIN32
+        qio_channel_command_abort(ioc, NULL);
+#endif
+    }
+}
+
+
+static ssize_t qio_channel_command_readv(QIOChannel *ioc,
+                                         const struct iovec *iov,
+                                         size_t niov,
+                                         int **fds,
+                                         size_t *nfds,
+                                         Error **errp)
+{
+    QIOChannelCommand *cioc = QIO_CHANNEL_COMMAND(ioc);
+    ssize_t ret;
+
+    if (fds || nfds) {
+        error_setg_errno(errp, EINVAL,
+                         "Channel does not support file descriptor passing");
+        return -1;
+    }
+
+ retry:
+    ret = readv(cioc->readfd, iov, niov);
+    if (ret < 0) {
+        if (errno == EAGAIN ||
+            errno == EWOULDBLOCK) {
+            return QIO_CHANNEL_ERR_BLOCK;
+        }
+        if (errno == EINTR) {
+            goto retry;
+        }
+
+        error_setg_errno(errp, errno,
+                         "Unable to read from command");
+        return -1;
+    }
+
+    return ret;
+}
+
+static ssize_t qio_channel_command_writev(QIOChannel *ioc,
+                                          const struct iovec *iov,
+                                          size_t niov,
+                                          int *fds,
+                                          size_t nfds,
+                                          Error **errp)
+{
+    QIOChannelCommand *cioc = QIO_CHANNEL_COMMAND(ioc);
+    ssize_t ret;
+
+    if (fds || nfds) {
+        error_setg_errno(errp, EINVAL,
+                         "Channel does not support file descriptor passing");
+        return -1;
+    }
+
+ retry:
+    ret = writev(cioc->writefd, iov, niov);
+    if (ret <= 0) {
+        if (errno == EAGAIN ||
+            errno == EWOULDBLOCK) {
+            return QIO_CHANNEL_ERR_BLOCK;
+        }
+        if (errno == EINTR) {
+            goto retry;
+        }
+        error_setg_errno(errp, errno, "%s",
+                         "Unable to write to command");
+        return -1;
+    }
+    return ret;
+}
+
+static int qio_channel_command_set_blocking(QIOChannel *ioc,
+                                            bool enabled,
+                                            Error **errp)
+{
+    QIOChannelCommand *cioc = QIO_CHANNEL_COMMAND(ioc);
+
+    if (enabled) {
+        qemu_set_block(cioc->writefd);
+        qemu_set_block(cioc->readfd);
+    } else {
+        qemu_set_nonblock(cioc->writefd);
+        qemu_set_nonblock(cioc->readfd);
+    }
+
+    return 0;
+}
+
+
+static int qio_channel_command_close(QIOChannel *ioc,
+                                     Error **errp)
+{
+    QIOChannelCommand *cioc = QIO_CHANNEL_COMMAND(ioc);
+    int rv = 0;
+
+    /* We close FDs before killing, because that
+     * gives a better chance of clean shutdown
+     */
+    if (close(cioc->writefd) < 0) {
+        rv = -1;
+    }
+    if (close(cioc->readfd) < 0) {
+        rv = -1;
+    }
+#ifndef WIN32
+    if (qio_channel_command_abort(cioc, errp) < 0) {
+        return -1;
+    }
+#endif
+    if (rv < 0) {
+        error_setg_errno(errp, errno, "%s",
+                         "Unable to close command");
+    }
+    return rv;
+}
+
+
+static GSource *qio_channel_command_create_watch(QIOChannel *ioc,
+                                                 GIOCondition condition)
+{
+    QIOChannelCommand *cioc = QIO_CHANNEL_COMMAND(ioc);
+    return qio_channel_create_fd_pair_watch(ioc,
+                                            cioc->readfd,
+                                            cioc->writefd,
+                                            condition);
+}
+
+
+static void qio_channel_command_class_init(ObjectClass *klass,
+                                           void *class_data G_GNUC_UNUSED)
+{
+    QIOChannelClass *ioc_klass = QIO_CHANNEL_CLASS(klass);
+
+    ioc_klass->io_writev = qio_channel_command_writev;
+    ioc_klass->io_readv = qio_channel_command_readv;
+    ioc_klass->io_set_blocking = qio_channel_command_set_blocking;
+    ioc_klass->io_close = qio_channel_command_close;
+    ioc_klass->io_create_watch = qio_channel_command_create_watch;
+}
+
+static const TypeInfo qio_channel_command_info = {
+    .parent = TYPE_QIO_CHANNEL,
+    .name = TYPE_QIO_CHANNEL_COMMAND,
+    .instance_size = sizeof(QIOChannelCommand),
+    .instance_init = qio_channel_command_init,
+    .instance_finalize = qio_channel_command_finalize,
+    .class_init = qio_channel_command_class_init,
+};
+
+static void qio_channel_command_register_types(void)
+{
+    type_register_static(&qio_channel_command_info);
+}
+
+type_init(qio_channel_command_register_types);
diff --git a/tests/.gitignore b/tests/.gitignore
index aa90bb2..a02272d 100644
--- a/tests/.gitignore
+++ b/tests/.gitignore
@@ -23,6 +23,8 @@ test-cutils
 test-hbitmap
 test-int128
 test-iov
+test-io-channel-command
+test-io-channel-command.fifo
 test-io-channel-file
 test-io-channel-file.txt
 test-io-channel-socket
diff --git a/tests/Makefile b/tests/Makefile
index 9c60e28..bc5d24d 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -84,6 +84,7 @@ check-unit-y += tests/test-io-task$(EXESUF)
 check-unit-y += tests/test-io-channel-socket$(EXESUF)
 check-unit-y += tests/test-io-channel-file$(EXESUF)
 check-unit-$(CONFIG_GNUTLS) += tests/test-io-channel-tls$(EXESUF)
+check-unit-y += tests/test-io-channel-command$(EXESUF)
 
 check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
 
@@ -377,6 +378,8 @@ tests/test-io-channel-file$(EXESUF): tests/test-io-channel-file.o \
 tests/test-io-channel-tls$(EXESUF): tests/test-io-channel-tls.o \
 	tests/crypto-tls-x509-helpers.o tests/pkix_asn1_tab.o \
 	tests/io-channel-helpers.o $(test-io-obj-y)
+tests/test-io-channel-command$(EXESUF): tests/test-io-channel-command.o \
+        tests/io-channel-helpers.o $(test-io-obj-y)
 
 libqos-obj-y = tests/libqos/pci.o tests/libqos/fw_cfg.o tests/libqos/malloc.o
 libqos-obj-y += tests/libqos/i2c.o tests/libqos/libqos.o
diff --git a/tests/test-io-channel-command.c b/tests/test-io-channel-command.c
new file mode 100644
index 0000000..269b25b
--- /dev/null
+++ b/tests/test-io-channel-command.c
@@ -0,0 +1,122 @@
+/*
+ * QEMU I/O channel command test
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "io/channel-command.h"
+#include "io-channel-helpers.h"
+
+#ifndef WIN32
+static void test_io_channel_command_fifo(bool async)
+{
+#define TEST_FIFO "tests/test-io-channel-command.fifo"
+    QIOChannel *src, *dst;
+    char *srcfifo = g_strdup_printf("PIPE:%s,wronly", TEST_FIFO);
+    char *dstfifo = g_strdup_printf("PIPE:%s,rdonly", TEST_FIFO);
+    const char *srcargv[] = {
+        "/bin/socat", "-", srcfifo, NULL,
+    };
+    const char *dstargv[] = {
+        "/bin/socat", dstfifo, "-", NULL,
+    };
+
+    unlink(TEST_FIFO);
+    if (access("/bin/socat", X_OK) < 0) {
+        return; /* Pretend success if socat is not present */
+    }
+    if (mkfifo(TEST_FIFO, 0600) < 0) {
+        abort();
+    }
+    src = QIO_CHANNEL(qio_channel_command_new_spawn(srcargv,
+                                                    O_WRONLY,
+                                                    &error_abort));
+    dst = QIO_CHANNEL(qio_channel_command_new_spawn(dstargv,
+                                                    O_RDONLY,
+                                                    &error_abort));
+    test_io_channel_comms(async, src, dst);
+
+    object_unref(OBJECT(src));
+    object_unref(OBJECT(dst));
+
+    g_free(srcfifo);
+    g_free(dstfifo);
+    unlink(TEST_FIFO);
+}
+
+
+static void test_io_channel_command_fifo_async(void)
+{
+    test_io_channel_command_fifo(true);
+}
+
+static void test_io_channel_command_fifo_sync(void)
+{
+    test_io_channel_command_fifo(false);
+}
+
+
+static void test_io_channel_command_echo(bool async)
+{
+    QIOChannel *ioc;
+    const char *socatargv[] = {
+        "/bin/socat", "-", "-", NULL,
+    };
+
+    if (access("/bin/socat", X_OK) < 0) {
+        return; /* Pretend success if socat is not present */
+    }
+
+    ioc = QIO_CHANNEL(qio_channel_command_new_spawn(socatargv,
+                                                    O_RDWR,
+                                                    &error_abort));
+    test_io_channel_comms(async, ioc, ioc);
+
+    object_unref(OBJECT(ioc));
+}
+
+
+static void test_io_channel_command_echo_async(void)
+{
+    test_io_channel_command_echo(true);
+}
+
+static void test_io_channel_command_echo_sync(void)
+{
+    test_io_channel_command_echo(false);
+}
+#endif
+
+int main(int argc, char **argv)
+{
+    module_call_init(MODULE_INIT_QOM);
+
+    g_test_init(&argc, &argv, NULL);
+
+#ifndef WIN32
+    g_test_add_func("/io/channel/command/fifo/sync",
+                    test_io_channel_command_fifo_sync);
+    g_test_add_func("/io/channel/command/fifo/async",
+                    test_io_channel_command_fifo_async);
+    g_test_add_func("/io/channel/command/echo/sync",
+                    test_io_channel_command_echo_sync);
+    g_test_add_func("/io/channel/command/echo/async",
+                    test_io_channel_command_echo_async);
+#endif
+
+    return g_test_run();
+}
diff --git a/trace-events b/trace-events
index fad6a27..725be4b 100644
--- a/trace-events
+++ b/trace-events
@@ -1735,3 +1735,9 @@ qio_channel_websock_handshake_pending(void *ioc, int status) "Websock handshake
 qio_channel_websock_handshake_reply(void *ioc) "Websock handshake reply ioc=%p"
 qio_channel_websock_handshake_fail(void *ioc) "Websock handshake fail ioc=%p"
 qio_channel_websock_handshake_complete(void *ioc) "Websock handshake complete ioc=%p"
+
+# io/channel-command.c
+qio_channel_command_new_pid(void *ioc, int writefd, int readfd, int pid) "Command new pid ioc=%p writefd=%d readfd=%d pid=%d"
+qio_channel_command_new_spawn(void *ioc, const char *binary, int flags) "Command new spawn ioc=%p binary=%s flags=%d"
+qio_channel_command_abort(void *ioc, int pid) "Command abort ioc=%p pid=%d"
+qio_channel_command_wait(void *ioc, int pid, int ret, int status) "Command abort ioc=%p pid=%d ret=%d status=%d"
-- 
2.4.3

^ permalink raw reply related	[flat|nested] 32+ messages in thread

* [Qemu-devel] [PATCH v1 16/16] io: add QIOChannelBuffer class
  2015-09-18 13:18 [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework Daniel P. Berrange
                   ` (14 preceding siblings ...)
  2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 15/16] io: add QIOChannelCommand class Daniel P. Berrange
@ 2015-09-18 13:19 ` Daniel P. Berrange
  2015-09-25 10:15   ` Paolo Bonzini
  2015-09-22 12:23 ` [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework Paolo Bonzini
  16 siblings, 1 reply; 32+ messages in thread
From: Daniel P. Berrange @ 2015-09-18 13:19 UTC (permalink / raw)
  To: qemu-devel; +Cc: Paolo Bonzini, Dr. David Alan Gilbert, Gerd Hoffmann

Add a QIOChannel subclass that is capable of performing I/O
to/from a memory buffer. This implementation does not attempt
to support concurrent readers & writers. It is designed for
serialized access where by a single thread at a time may write
data, seek and then read data back out.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
 include/io/channel-buffer.h |  59 ++++++++++
 io/Makefile.objs            |   1 +
 io/channel-buffer.c         | 255 ++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 315 insertions(+)
 create mode 100644 include/io/channel-buffer.h
 create mode 100644 io/channel-buffer.c

diff --git a/include/io/channel-buffer.h b/include/io/channel-buffer.h
new file mode 100644
index 0000000..1e0f6c2
--- /dev/null
+++ b/include/io/channel-buffer.h
@@ -0,0 +1,59 @@
+/*
+ * QEMU I/O channels memory buffer driver
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef QIO_CHANNEL_BUFFER_H__
+#define QIO_CHANNEL_BUFFER_H__
+
+#include "io/channel.h"
+
+#define TYPE_QIO_CHANNEL_BUFFER "qio-channel-buffer"
+#define QIO_CHANNEL_BUFFER(obj)                                     \
+    OBJECT_CHECK(QIOChannelBuffer, (obj), TYPE_QIO_CHANNEL_BUFFER)
+
+typedef struct QIOChannelBuffer QIOChannelBuffer;
+
+/**
+ * QIOChannelBuffer:
+ *
+ * The QIOChannelBuffer object provides a channel implementation
+ * that is able to perform I/O to/from a memory buffer.
+ *
+ */
+
+struct QIOChannelBuffer {
+    QIOChannel parent;
+    size_t capacity; /* Total allocated memory */
+    size_t usage;    /* Current size of data */
+    size_t offset;   /* Offset for future I/O ops */
+    char *data;
+};
+
+
+/**
+ * qio_channel_buffer_new:
+ *
+ * Allocate a new buffer which is initially empty
+ *
+ * Returns: the new channel object
+ */
+QIOChannelBuffer *
+qio_channel_buffer_new(void);
+
+#endif /* QIO_CHANNEL_BUFFER_H__ */
diff --git a/io/Makefile.objs b/io/Makefile.objs
index 9e28ee5..5e47e6c 100644
--- a/io/Makefile.objs
+++ b/io/Makefile.objs
@@ -7,3 +7,4 @@ io-obj-y += channel-file.o
 io-obj-y += channel-tls.o
 io-obj-y += channel-websock.o
 io-obj-y += channel-command.o
+io-obj-y += channel-buffer.o
diff --git a/io/channel-buffer.c b/io/channel-buffer.c
new file mode 100644
index 0000000..8d85032
--- /dev/null
+++ b/io/channel-buffer.c
@@ -0,0 +1,255 @@
+/*
+ * QEMU I/O channels memory buffer driver
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "io/channel-buffer.h"
+#include "io/channel-watch.h"
+#include "qemu/sockets.h"
+#include "trace.h"
+
+QIOChannelBuffer *
+qio_channel_buffer_new(void)
+{
+    QIOChannelBuffer *ioc;
+
+    ioc = QIO_CHANNEL_BUFFER(object_new(TYPE_QIO_CHANNEL_BUFFER));
+
+    return ioc;
+}
+
+
+static void qio_channel_buffer_finalize(Object *obj)
+{
+    QIOChannelBuffer *ioc = QIO_CHANNEL_BUFFER(obj);
+    g_free(ioc->data);
+    ioc->capacity = ioc->usage = ioc->offset = 0;
+}
+
+
+static ssize_t qio_channel_buffer_readv(QIOChannel *ioc,
+                                        const struct iovec *iov,
+                                        size_t niov,
+                                        int **fds,
+                                        size_t *nfds,
+                                        Error **errp)
+{
+    QIOChannelBuffer *bioc = QIO_CHANNEL_BUFFER(ioc);
+    ssize_t ret = 0;
+    size_t i;
+
+    if (fds || nfds) {
+        error_setg_errno(errp, EINVAL,
+                         "Channel does not support buffer descriptor passing");
+        return -1;
+    }
+
+    for (i = 0; i < niov; i++) {
+        size_t want = iov[i].iov_len;
+        if (bioc->offset >= bioc->usage) {
+            break;
+        }
+        if ((bioc->offset + want) > bioc->usage)  {
+            want = bioc->usage - bioc->offset;
+        }
+        memcpy(iov[i].iov_base, bioc->data + bioc->offset, want);
+        ret += want;
+    }
+
+    return ret;
+}
+
+static ssize_t qio_channel_buffer_writev(QIOChannel *ioc,
+                                         const struct iovec *iov,
+                                         size_t niov,
+                                         int *fds,
+                                         size_t nfds,
+                                         Error **errp)
+{
+    QIOChannelBuffer *bioc = QIO_CHANNEL_BUFFER(ioc);
+    ssize_t ret = 0;
+    size_t i;
+    size_t towrite = 0;
+
+    if (fds || nfds) {
+        error_setg_errno(errp, EINVAL,
+                         "Channel does not support buffer descriptor passing");
+        return -1;
+    }
+
+    for (i = 0; i < niov; i++) {
+        towrite += iov[i].iov_len;
+    }
+
+    if ((bioc->offset + towrite) > bioc->capacity) {
+        bioc->capacity = bioc->offset + towrite;
+        bioc->data = g_realloc(bioc->data, bioc->capacity);
+    }
+
+    if (bioc->offset > bioc->usage) {
+        memset(bioc->data, 0, bioc->offset - bioc->usage);
+        bioc->usage = bioc->offset;
+    }
+
+    for (i = 0; i < niov; i++) {
+        memcpy(bioc->data + bioc->usage,
+               iov[i].iov_base,
+               iov[i].iov_len);
+        bioc->usage += iov[i].iov_len;
+        bioc->offset += iov[i].iov_len;
+        ret += iov[i].iov_len;
+    }
+
+    return ret;
+}
+
+static int qio_channel_buffer_set_blocking(QIOChannel *ioc G_GNUC_UNUSED,
+                                           bool enabled G_GNUC_UNUSED,
+                                           Error **errp G_GNUC_UNUSED)
+{
+    return 0;
+}
+
+
+static off64_t qio_channel_buffer_seek(QIOChannel *ioc,
+                                       off64_t offset,
+                                       int whence,
+                                       Error **errp)
+{
+    QIOChannelBuffer *bioc = QIO_CHANNEL_BUFFER(ioc);
+
+    bioc->offset = offset;
+
+    return offset;
+}
+
+
+static int qio_channel_buffer_close(QIOChannel *ioc,
+                                    Error **errp)
+{
+    QIOChannelBuffer *bioc = QIO_CHANNEL_BUFFER(ioc);
+
+    g_free(bioc->data);
+    bioc->capacity = bioc->usage = bioc->offset = 0;
+
+    return 0;
+}
+
+
+typedef struct QIOChannelBufferSource QIOChannelBufferSource;
+struct QIOChannelBufferSource {
+    GSource parent;
+    QIOChannelBuffer *bioc;
+    GIOCondition condition;
+};
+
+static gboolean
+qio_channel_buffer_source_prepare(GSource *source,
+                                  gint *timeout)
+{
+    QIOChannelBufferSource *bsource = (QIOChannelBufferSource *)source;
+
+    *timeout = -1;
+
+    return (G_IO_IN | G_IO_OUT) & bsource->condition;
+}
+
+static gboolean
+qio_channel_buffer_source_check(GSource *source)
+{
+    QIOChannelBufferSource *bsource = (QIOChannelBufferSource *)source;
+
+    return (G_IO_IN | G_IO_OUT) & bsource->condition;
+}
+
+static gboolean
+qio_channel_buffer_source_dispatch(GSource *source,
+                                   GSourceFunc callback,
+                                   gpointer user_data)
+{
+    QIOChannelFunc func = (QIOChannelFunc)callback;
+    QIOChannelBufferSource *bsource = (QIOChannelBufferSource *)source;
+
+    return (*func)(QIO_CHANNEL(bsource->bioc),
+                   ((G_IO_IN | G_IO_OUT) & bsource->condition),
+                   user_data);
+}
+
+static void
+qio_channel_buffer_source_finalize(GSource *source)
+{
+    QIOChannelBufferSource *ssource = (QIOChannelBufferSource *)source;
+
+    object_unref(OBJECT(ssource->bioc));
+}
+
+GSourceFuncs qio_channel_buffer_source_funcs = {
+    qio_channel_buffer_source_prepare,
+    qio_channel_buffer_source_check,
+    qio_channel_buffer_source_dispatch,
+    qio_channel_buffer_source_finalize
+};
+
+static GSource *qio_channel_buffer_create_watch(QIOChannel *ioc,
+                                                GIOCondition condition)
+{
+    QIOChannelBuffer *bioc = QIO_CHANNEL_BUFFER(ioc);
+    QIOChannelBufferSource *ssource;
+    GSource *source;
+
+    source = g_source_new(&qio_channel_buffer_source_funcs,
+                          sizeof(QIOChannelBufferSource));
+    g_source_set_name(source, "QIOChannelBuffer");
+    ssource = (QIOChannelBufferSource *)source;
+
+    ssource->bioc = bioc;
+    object_ref(OBJECT(bioc));
+
+    ssource->condition = condition;
+
+    return source;
+}
+
+
+static void qio_channel_buffer_class_init(ObjectClass *klass,
+                                          void *class_data G_GNUC_UNUSED)
+{
+    QIOChannelClass *ioc_klass = QIO_CHANNEL_CLASS(klass);
+
+    ioc_klass->io_writev = qio_channel_buffer_writev;
+    ioc_klass->io_readv = qio_channel_buffer_readv;
+    ioc_klass->io_set_blocking = qio_channel_buffer_set_blocking;
+    ioc_klass->io_seek = qio_channel_buffer_seek;
+    ioc_klass->io_close = qio_channel_buffer_close;
+    ioc_klass->io_create_watch = qio_channel_buffer_create_watch;
+}
+
+static const TypeInfo qio_channel_buffer_info = {
+    .parent = TYPE_QIO_CHANNEL,
+    .name = TYPE_QIO_CHANNEL_BUFFER,
+    .instance_size = sizeof(QIOChannelBuffer),
+    .instance_finalize = qio_channel_buffer_finalize,
+    .class_init = qio_channel_buffer_class_init,
+};
+
+static void qio_channel_buffer_register_types(void)
+{
+    type_register_static(&qio_channel_buffer_info);
+}
+
+type_init(qio_channel_buffer_register_types);
-- 
2.4.3

^ permalink raw reply related	[flat|nested] 32+ messages in thread

* Re: [Qemu-devel] [PATCH v1 09/16] io: pull Buffer code out of VNC module
  2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 09/16] io: pull Buffer code out of VNC module Daniel P. Berrange
@ 2015-09-22 12:04   ` Paolo Bonzini
  2015-09-22 12:20     ` Daniel P. Berrange
  0 siblings, 1 reply; 32+ messages in thread
From: Paolo Bonzini @ 2015-09-22 12:04 UTC (permalink / raw)
  To: Daniel P. Berrange, qemu-devel; +Cc: Dr. David Alan Gilbert, Gerd Hoffmann



On 18/09/2015 15:19, Daniel P. Berrange wrote:
> The Buffer code in the VNC server is useful for the IO channel
> code, so pull it out into a shared module, QIOBuffer.

Do all traces disappear from VNC once you're done?  If not, can you
instead move it to util/?

Paolo

> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
>  include/io/buffer.h | 118 ++++++++++++++++++++++++++++++++++++++++++++++++++++
>  io/Makefile.objs    |   3 +-
>  io/buffer.c         |  65 +++++++++++++++++++++++++++++
>  ui/vnc-auth-sasl.c  |   4 +-
>  ui/vnc-enc-tight.c  |  38 ++++++++---------
>  ui/vnc-enc-zlib.c   |   6 +--
>  ui/vnc-enc-zrle.c   |  18 ++++----
>  ui/vnc-jobs.c       |  15 +++----
>  ui/vnc-ws.c         |  36 ++++++++--------
>  ui/vnc-ws.h         |   6 +--
>  ui/vnc.c            |  67 ++++++-----------------------
>  ui/vnc.h            |  50 ++++++++--------------
>  12 files changed, 277 insertions(+), 149 deletions(-)
>  create mode 100644 include/io/buffer.h
>  create mode 100644 io/buffer.c
> 
> diff --git a/include/io/buffer.h b/include/io/buffer.h
> new file mode 100644
> index 0000000..2b1b261
> --- /dev/null
> +++ b/include/io/buffer.h
> @@ -0,0 +1,118 @@
> +/*
> + * QEMU I/O buffers
> + *
> + * Copyright (c) 2015 Red Hat, Inc.
> + *
> + * This library is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2 of the License, or (at your option) any later version.
> + *
> + * This library is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with this library; if not, see <http://www.gnu.org/licenses/>.
> + *
> + */
> +
> +#ifndef QIO_BUFFER_H__
> +#define QIO_BUFFER_H__
> +
> +#include "qemu-common.h"
> +
> +typedef struct QIOBuffer QIOBuffer;
> +
> +/**
> + * QIOBuffer:
> + *
> + * The QIOBuffer object provides a simple dynamically resizing
> + * array, with separate tracking of capacity and usage. This
> + * is typically useful when buffering I/O data.
> + */
> +
> +struct QIOBuffer {
> +    size_t capacity;
> +    size_t offset;
> +    uint8_t *buffer;
> +};
> +
> +/**
> + * qio_buffer_reserve:
> + * @buffer: the buffer object
> + * @len: the minimum required free space
> + *
> + * Ensure that the buffer has space allocated for at least
> + * @len bytes. If the current buffer is too small, it will
> + * be reallocated, possibly to a larger size than requested.
> + */
> +void qio_buffer_reserve(QIOBuffer *buffer, size_t len);
> +
> +/**
> + * qio_buffer_reset:
> + * @buffer: the buffer object
> + *
> + * Reset the length of the stored data to zero, but do
> + * not free / reallocate the memory buffer
> + */
> +void qio_buffer_reset(QIOBuffer *buffer);
> +
> +/**
> + * qio_buffer_free:
> + * @buffer: the buffer object
> + *
> + * Reset the length of the stored data to zero and also
> + * free the internal memory buffer
> + */
> +void qio_buffer_free(QIOBuffer *buffer);
> +
> +/**
> + * qio_buffer_append:
> + * @buffer: the buffer object
> + * @data: the data block to append
> + * @len: the length of @data in bytes
> + *
> + * Append the contents of @data to the end of the buffer.
> + * The caller must ensure that the buffer has sufficient
> + * free space for @len bytes, typically by calling the
> + * qio_buffer_reserve() method prior to appending.
> + */
> +void qio_buffer_append(QIOBuffer *buffer, const void *data, size_t len);
> +
> +/**
> + * qio_buffer_advance:
> + * @buffer: the buffer object
> + * @len: the number of bytes to skip
> + *
> + * Remove @len bytes of data from the head of the buffer.
> + * The internal buffer will not be reallocated, so will
> + * have at least @len bytes of free space after this
> + * call completes
> + */
> +void qio_buffer_advance(QIOBuffer *buffer, size_t len);
> +
> +/**
> + * qio_buffer_end:
> + * @buffer: the buffer object
> + *
> + * Get a pointer to the tail end of the internal buffer
> + * The returned pointer is only valid until the next
> + * call to qio_buffer_reserve().
> + *
> + * Returns: the tail of the buffer
> + */
> +uint8_t *qio_buffer_end(QIOBuffer *buffer);
> +
> +/**
> + * qio_buffer_empty:
> + * @buffer: the buffer object
> + *
> + * Determine if the buffer contains any current data
> + *
> + * Returns: true if the buffer holds data, false otherwise
> + */
> +gboolean qio_buffer_empty(QIOBuffer *buffer);
> +
> +#endif /* QIO_BUFFER_H__ */
> diff --git a/io/Makefile.objs b/io/Makefile.objs
> index b02ea90..593ed9e 100644
> --- a/io/Makefile.objs
> +++ b/io/Makefile.objs
> @@ -1,2 +1,3 @@
> -io-obj-y = channel.o
> +io-obj-y = buffer.o
> +io-obj-y += channel.o
>  io-obj-y += channel-watch.o
> diff --git a/io/buffer.c b/io/buffer.c
> new file mode 100644
> index 0000000..68ae68d
> --- /dev/null
> +++ b/io/buffer.c
> @@ -0,0 +1,65 @@
> +/*
> + * QEMU I/O buffers
> + *
> + * Copyright (c) 2015 Red Hat, Inc.
> + *
> + * This library is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2 of the License, or (at your option) any later version.
> + *
> + * This library is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with this library; if not, see <http://www.gnu.org/licenses/>.
> + *
> + */
> +
> +#include "io/buffer.h"
> +
> +void qio_buffer_reserve(QIOBuffer *buffer, size_t len)
> +{
> +    if ((buffer->capacity - buffer->offset) < len) {
> +        buffer->capacity += (len + 1024);
> +        buffer->buffer = g_realloc(buffer->buffer, buffer->capacity);
> +    }
> +}
> +
> +gboolean qio_buffer_empty(QIOBuffer *buffer)
> +{
> +    return buffer->offset == 0;
> +}
> +
> +uint8_t *qio_buffer_end(QIOBuffer *buffer)
> +{
> +    return buffer->buffer + buffer->offset;
> +}
> +
> +void qio_buffer_reset(QIOBuffer *buffer)
> +{
> +    buffer->offset = 0;
> +}
> +
> +void qio_buffer_free(QIOBuffer *buffer)
> +{
> +    g_free(buffer->buffer);
> +    buffer->offset = 0;
> +    buffer->capacity = 0;
> +    buffer->buffer = NULL;
> +}
> +
> +void qio_buffer_append(QIOBuffer *buffer, const void *data, size_t len)
> +{
> +    memcpy(buffer->buffer + buffer->offset, data, len);
> +    buffer->offset += len;
> +}
> +
> +void qio_buffer_advance(QIOBuffer *buffer, size_t len)
> +{
> +    memmove(buffer->buffer, buffer->buffer + len,
> +            (buffer->offset - len));
> +    buffer->offset -= len;
> +}
> diff --git a/ui/vnc-auth-sasl.c b/ui/vnc-auth-sasl.c
> index fc732bd..d118266 100644
> --- a/ui/vnc-auth-sasl.c
> +++ b/ui/vnc-auth-sasl.c
> @@ -113,8 +113,8 @@ long vnc_client_read_sasl(VncState *vs)
>          return vnc_client_io_error(vs, -1, -EIO);
>      VNC_DEBUG("Read SASL Encoded %p size %ld Decoded %p size %d\n",
>                encoded, ret, decoded, decodedLen);
> -    buffer_reserve(&vs->input, decodedLen);
> -    buffer_append(&vs->input, decoded, decodedLen);
> +    qio_buffer_reserve(&vs->input, decodedLen);
> +    qio_buffer_append(&vs->input, decoded, decodedLen);
>      return decodedLen;
>  }
>  
> diff --git a/ui/vnc-enc-tight.c b/ui/vnc-enc-tight.c
> index 9a9ddf2..772ec79 100644
> --- a/ui/vnc-enc-tight.c
> +++ b/ui/vnc-enc-tight.c
> @@ -856,7 +856,7 @@ static int tight_compress_data(VncState *vs, int stream_id, size_t bytes,
>      }
>  
>      /* reserve memory in output buffer */
> -    buffer_reserve(&vs->tight.zlib, bytes + 64);
> +    qio_buffer_reserve(&vs->tight.zlib, bytes + 64);
>  
>      /* set pointers */
>      zstream->next_in = vs->tight.tight.buffer;
> @@ -879,7 +879,7 @@ static int tight_compress_data(VncState *vs, int stream_id, size_t bytes,
>      tight_send_compact_size(vs, bytes);
>      vnc_write(vs, vs->tight.zlib.buffer, bytes);
>  
> -    buffer_reset(&vs->tight.zlib);
> +    qio_buffer_reset(&vs->tight.zlib);
>  
>      return bytes;
>  }
> @@ -1053,7 +1053,7 @@ static bool send_gradient_rect(VncState *vs, int x, int y, int w, int h)
>      vnc_write_u8(vs, (stream | VNC_TIGHT_EXPLICIT_FILTER) << 4);
>      vnc_write_u8(vs, VNC_TIGHT_FILTER_GRADIENT);
>  
> -    buffer_reserve(&vs->tight.gradient, w * 3 * sizeof (int));
> +    qio_buffer_reserve(&vs->tight.gradient, w * 3 * sizeof(int));
>  
>      if (vs->tight.pixel24) {
>          tight_filter_gradient24(vs, vs->tight.tight.buffer, w, h);
> @@ -1066,7 +1066,7 @@ static bool send_gradient_rect(VncState *vs, int x, int y, int w, int h)
>          bytes = 2;
>      }
>  
> -    buffer_reset(&vs->tight.gradient);
> +    qio_buffer_reset(&vs->tight.gradient);
>  
>      bytes = w * h * bytes;
>      vs->tight.tight.offset = bytes;
> @@ -1149,7 +1149,7 @@ static int send_palette_rect(VncState *vs, int x, int y,
>  static void jpeg_init_destination(j_compress_ptr cinfo)
>  {
>      VncState *vs = cinfo->client_data;
> -    Buffer *buffer = &vs->tight.jpeg;
> +    QIOBuffer *buffer = &vs->tight.jpeg;
>  
>      cinfo->dest->next_output_byte = (JOCTET *)buffer->buffer + buffer->offset;
>      cinfo->dest->free_in_buffer = (size_t)(buffer->capacity - buffer->offset);
> @@ -1159,10 +1159,10 @@ static void jpeg_init_destination(j_compress_ptr cinfo)
>  static boolean jpeg_empty_output_buffer(j_compress_ptr cinfo)
>  {
>      VncState *vs = cinfo->client_data;
> -    Buffer *buffer = &vs->tight.jpeg;
> +    QIOBuffer *buffer = &vs->tight.jpeg;
>  
>      buffer->offset = buffer->capacity;
> -    buffer_reserve(buffer, 2048);
> +    qio_buffer_reserve(buffer, 2048);
>      jpeg_init_destination(cinfo);
>      return TRUE;
>  }
> @@ -1171,7 +1171,7 @@ static boolean jpeg_empty_output_buffer(j_compress_ptr cinfo)
>  static void jpeg_term_destination(j_compress_ptr cinfo)
>  {
>      VncState *vs = cinfo->client_data;
> -    Buffer *buffer = &vs->tight.jpeg;
> +    QIOBuffer *buffer = &vs->tight.jpeg;
>  
>      buffer->offset = buffer->capacity - cinfo->dest->free_in_buffer;
>  }
> @@ -1190,7 +1190,7 @@ static int send_jpeg_rect(VncState *vs, int x, int y, int w, int h, int quality)
>          return send_full_color_rect(vs, x, y, w, h);
>      }
>  
> -    buffer_reserve(&vs->tight.jpeg, 2048);
> +    qio_buffer_reserve(&vs->tight.jpeg, 2048);
>  
>      cinfo.err = jpeg_std_error(&jerr);
>      jpeg_create_compress(&cinfo);
> @@ -1227,7 +1227,7 @@ static int send_jpeg_rect(VncState *vs, int x, int y, int w, int h, int quality)
>  
>      tight_send_compact_size(vs, vs->tight.jpeg.offset);
>      vnc_write(vs, vs->tight.jpeg.buffer, vs->tight.jpeg.offset);
> -    buffer_reset(&vs->tight.jpeg);
> +    qio_buffer_reset(&vs->tight.jpeg);
>  
>      return 1;
>  }
> @@ -1270,7 +1270,7 @@ static void png_write_data(png_structp png_ptr, png_bytep data,
>  {
>      VncState *vs = png_get_io_ptr(png_ptr);
>  
> -    buffer_reserve(&vs->tight.png, vs->tight.png.offset + length);
> +    qio_buffer_reserve(&vs->tight.png, vs->tight.png.offset + length);
>      memcpy(vs->tight.png.buffer + vs->tight.png.offset, data, length);
>  
>      vs->tight.png.offset += length;
> @@ -1351,7 +1351,7 @@ static int send_png_rect(VncState *vs, int x, int y, int w, int h,
>  
>      png_write_info(png_ptr, info_ptr);
>  
> -    buffer_reserve(&vs->tight.png, 2048);
> +    qio_buffer_reserve(&vs->tight.png, 2048);
>      linebuf = qemu_pixman_linebuf_create(PIXMAN_BE_r8g8b8, w);
>      buf = (uint8_t *)pixman_image_get_data(linebuf);
>      for (dy = 0; dy < h; dy++)
> @@ -1377,14 +1377,14 @@ static int send_png_rect(VncState *vs, int x, int y, int w, int h,
>  
>      tight_send_compact_size(vs, vs->tight.png.offset);
>      vnc_write(vs, vs->tight.png.buffer, vs->tight.png.offset);
> -    buffer_reset(&vs->tight.png);
> +    qio_buffer_reset(&vs->tight.png);
>      return 1;
>  }
>  #endif /* CONFIG_VNC_PNG */
>  
>  static void vnc_tight_start(VncState *vs)
>  {
> -    buffer_reset(&vs->tight.tight);
> +    qio_buffer_reset(&vs->tight.tight);
>  
>      // make the output buffer be the zlib buffer, so we can compress it later
>      vs->tight.tmp = vs->output;
> @@ -1686,13 +1686,13 @@ void vnc_tight_clear(VncState *vs)
>          }
>      }
>  
> -    buffer_free(&vs->tight.tight);
> -    buffer_free(&vs->tight.zlib);
> -    buffer_free(&vs->tight.gradient);
> +    qio_buffer_free(&vs->tight.tight);
> +    qio_buffer_free(&vs->tight.zlib);
> +    qio_buffer_free(&vs->tight.gradient);
>  #ifdef CONFIG_VNC_JPEG
> -    buffer_free(&vs->tight.jpeg);
> +    qio_buffer_free(&vs->tight.jpeg);
>  #endif
>  #ifdef CONFIG_VNC_PNG
> -    buffer_free(&vs->tight.png);
> +    qio_buffer_free(&vs->tight.png);
>  #endif
>  }
> diff --git a/ui/vnc-enc-zlib.c b/ui/vnc-enc-zlib.c
> index d1b97f2..47ba146 100644
> --- a/ui/vnc-enc-zlib.c
> +++ b/ui/vnc-enc-zlib.c
> @@ -47,7 +47,7 @@ void vnc_zlib_zfree(void *x, void *addr)
>  
>  static void vnc_zlib_start(VncState *vs)
>  {
> -    buffer_reset(&vs->zlib.zlib);
> +    qio_buffer_reset(&vs->zlib.zlib);
>  
>      // make the output buffer be the zlib buffer, so we can compress it later
>      vs->zlib.tmp = vs->output;
> @@ -96,7 +96,7 @@ static int vnc_zlib_stop(VncState *vs)
>      }
>  
>      // reserve memory in output buffer
> -    buffer_reserve(&vs->output, vs->zlib.zlib.offset + 64);
> +    qio_buffer_reserve(&vs->output, vs->zlib.zlib.offset + 64);
>  
>      // set pointers
>      zstream->next_in = vs->zlib.zlib.buffer;
> @@ -148,5 +148,5 @@ void vnc_zlib_clear(VncState *vs)
>      if (vs->zlib.stream.opaque) {
>          deflateEnd(&vs->zlib.stream);
>      }
> -    buffer_free(&vs->zlib.zlib);
> +    qio_buffer_free(&vs->zlib.zlib);
>  }
> diff --git a/ui/vnc-enc-zrle.c b/ui/vnc-enc-zrle.c
> index ed3b484..bd1e320 100644
> --- a/ui/vnc-enc-zrle.c
> +++ b/ui/vnc-enc-zrle.c
> @@ -36,7 +36,7 @@ static const int bits_per_packed_pixel[] = {
>  
>  static void vnc_zrle_start(VncState *vs)
>  {
> -    buffer_reset(&vs->zrle.zrle);
> +    qio_buffer_reset(&vs->zrle.zrle);
>  
>      /* make the output buffer be the zlib buffer, so we can compress it later */
>      vs->zrle.tmp = vs->output;
> @@ -53,10 +53,10 @@ static void vnc_zrle_stop(VncState *vs)
>  static void *zrle_convert_fb(VncState *vs, int x, int y, int w, int h,
>                               int bpp)
>  {
> -    Buffer tmp;
> +    QIOBuffer tmp;
>  
> -    buffer_reset(&vs->zrle.fb);
> -    buffer_reserve(&vs->zrle.fb, w * h * bpp + bpp);
> +    qio_buffer_reset(&vs->zrle.fb);
> +    qio_buffer_reserve(&vs->zrle.fb, w * h * bpp + bpp);
>  
>      tmp = vs->output;
>      vs->output = vs->zrle.fb;
> @@ -72,7 +72,7 @@ static int zrle_compress_data(VncState *vs, int level)
>  {
>      z_streamp zstream = &vs->zrle.stream;
>  
> -    buffer_reset(&vs->zrle.zlib);
> +    qio_buffer_reset(&vs->zrle.zlib);
>  
>      if (zstream->opaque != vs) {
>          int err;
> @@ -92,7 +92,7 @@ static int zrle_compress_data(VncState *vs, int level)
>      }
>  
>      /* reserve memory in output buffer */
> -    buffer_reserve(&vs->zrle.zlib, vs->zrle.zrle.offset + 64);
> +    qio_buffer_reserve(&vs->zrle.zlib, vs->zrle.zrle.offset + 64);
>  
>      /* set pointers */
>      zstream->next_in = vs->zrle.zrle.buffer;
> @@ -360,7 +360,7 @@ void vnc_zrle_clear(VncState *vs)
>      if (vs->zrle.stream.opaque) {
>          deflateEnd(&vs->zrle.stream);
>      }
> -    buffer_free(&vs->zrle.zrle);
> -    buffer_free(&vs->zrle.fb);
> -    buffer_free(&vs->zrle.zlib);
> +    qio_buffer_free(&vs->zrle.zrle);
> +    qio_buffer_free(&vs->zrle.fb);
> +    qio_buffer_free(&vs->zrle.zlib);
>  }
> diff --git a/ui/vnc-jobs.c b/ui/vnc-jobs.c
> index 22c9abc..9824c34 100644
> --- a/ui/vnc-jobs.c
> +++ b/ui/vnc-jobs.c
> @@ -54,7 +54,7 @@ struct VncJobQueue {
>      QemuCond cond;
>      QemuMutex mutex;
>      QemuThread thread;
> -    Buffer buffer;
> +    QIOBuffer buffer;
>      bool exit;
>      QTAILQ_HEAD(, VncJob) jobs;
>  };
> @@ -167,7 +167,7 @@ void vnc_jobs_consume_buffer(VncState *vs)
>      vnc_lock_output(vs);
>      if (vs->jobs_buffer.offset) {
>          vnc_write(vs, vs->jobs_buffer.buffer, vs->jobs_buffer.offset);
> -        buffer_reset(&vs->jobs_buffer);
> +        qio_buffer_reset(&vs->jobs_buffer);
>      }
>      flush = vs->csock != -1 && vs->abort != true;
>      vnc_unlock_output(vs);
> @@ -196,7 +196,7 @@ static void vnc_async_encoding_start(VncState *orig, VncState *local)
>      local->output =  queue->buffer;
>      local->csock = -1; /* Don't do any network work on this thread */
>  
> -    buffer_reset(&local->output);
> +    qio_buffer_reset(&local->output);
>  }
>  
>  static void vnc_async_encoding_end(VncState *orig, VncState *local)
> @@ -273,10 +273,11 @@ static int vnc_worker_thread_loop(VncJobQueue *queue)
>      vs.output.buffer[saved_offset + 1] = n_rectangles & 0xFF;
>  
>      vnc_lock_output(job->vs);
> +
>      if (job->vs->csock != -1) {
> -        buffer_reserve(&job->vs->jobs_buffer, vs.output.offset);
> -        buffer_append(&job->vs->jobs_buffer, vs.output.buffer,
> -                      vs.output.offset);
> +        qio_buffer_reserve(&job->vs->jobs_buffer, vs.output.offset);
> +        qio_buffer_append(&job->vs->jobs_buffer, vs.output.buffer,
> +                          vs.output.offset);
>          /* Copy persistent encoding data */
>          vnc_async_encoding_end(job->vs, &vs);
>  
> @@ -310,7 +311,7 @@ static void vnc_queue_clear(VncJobQueue *q)
>  {
>      qemu_cond_destroy(&queue->cond);
>      qemu_mutex_destroy(&queue->mutex);
> -    buffer_free(&queue->buffer);
> +    qio_buffer_free(&queue->buffer);
>      g_free(q);
>      queue = NULL; /* Unset global queue */
>  }
> diff --git a/ui/vnc-ws.c b/ui/vnc-ws.c
> index 175ea50..2fe4476 100644
> --- a/ui/vnc-ws.c
> +++ b/ui/vnc-ws.c
> @@ -95,8 +95,8 @@ void vncws_handshake_read(void *opaque)
>      /* Typical HTTP headers from novnc are 512 bytes, so limiting
>       * total header size to 4096 is easily enough. */
>      size_t want = 4096 - vs->ws_input.offset;
> -    buffer_reserve(&vs->ws_input, want);
> -    ret = vnc_client_read_buf(vs, buffer_end(&vs->ws_input), want);
> +    qio_buffer_reserve(&vs->ws_input, want);
> +    ret = vnc_client_read_buf(vs, qio_buffer_end(&vs->ws_input), want);
>  
>      if (!ret) {
>          if (vs->csock == -1) {
> @@ -111,7 +111,7 @@ void vncws_handshake_read(void *opaque)
>      if (handshake_end) {
>          qemu_set_fd_handler(vs->csock, vnc_client_read, NULL, vs);
>          vncws_process_handshake(vs, vs->ws_input.buffer, vs->ws_input.offset);
> -        buffer_advance(&vs->ws_input, handshake_end - vs->ws_input.buffer +
> +        qio_buffer_advance(&vs->ws_input, handshake_end - vs->ws_input.buffer +
>                  strlen(WS_HANDSHAKE_END));
>      } else if (vs->ws_input.offset >= 4096) {
>          VNC_DEBUG("End of headers not found in first 4096 bytes\n");
> @@ -127,8 +127,8 @@ long vnc_client_read_ws(VncState *vs)
>      size_t payload_size, header_size;
>      VNC_DEBUG("Read websocket %p size %zd offset %zd\n", vs->ws_input.buffer,
>              vs->ws_input.capacity, vs->ws_input.offset);
> -    buffer_reserve(&vs->ws_input, 4096);
> -    ret = vnc_client_read_buf(vs, buffer_end(&vs->ws_input), 4096);
> +    qio_buffer_reserve(&vs->ws_input, 4096);
> +    ret = vnc_client_read_buf(vs, qio_buffer_end(&vs->ws_input), 4096);
>      if (!ret) {
>          return 0;
>      }
> @@ -146,7 +146,7 @@ long vnc_client_read_ws(VncState *vs)
>                  return err;
>              }
>  
> -            buffer_advance(&vs->ws_input, header_size);
> +            qio_buffer_advance(&vs->ws_input, header_size);
>          }
>          if (vs->ws_payload_remain != 0) {
>              err = vncws_decode_frame_payload(&vs->ws_input,
> @@ -162,10 +162,10 @@ long vnc_client_read_ws(VncState *vs)
>              }
>              ret += err;
>  
> -            buffer_reserve(&vs->input, payload_size);
> -            buffer_append(&vs->input, payload, payload_size);
> +            qio_buffer_reserve(&vs->input, payload_size);
> +            qio_buffer_append(&vs->input, payload, payload_size);
>  
> -            buffer_advance(&vs->ws_input, payload_size);
> +            qio_buffer_advance(&vs->ws_input, payload_size);
>          }
>      } while (vs->ws_input.offset > 0);
>  
> @@ -178,13 +178,13 @@ long vnc_client_write_ws(VncState *vs)
>      VNC_DEBUG("Write WS: Pending output %p size %zd offset %zd\n",
>                vs->output.buffer, vs->output.capacity, vs->output.offset);
>      vncws_encode_frame(&vs->ws_output, vs->output.buffer, vs->output.offset);
> -    buffer_reset(&vs->output);
> +    qio_buffer_reset(&vs->output);
>      ret = vnc_client_write_buf(vs, vs->ws_output.buffer, vs->ws_output.offset);
>      if (!ret) {
>          return 0;
>      }
>  
> -    buffer_advance(&vs->ws_output, ret);
> +    qio_buffer_advance(&vs->ws_output, ret);
>  
>      if (vs->ws_output.offset == 0) {
>          qemu_set_fd_handler(vs->csock, vnc_client_read, NULL, vs);
> @@ -267,8 +267,8 @@ void vncws_process_handshake(VncState *vs, uint8_t *line, size_t size)
>      g_free(key);
>  }
>  
> -void vncws_encode_frame(Buffer *output, const void *payload,
> -        const size_t payload_size)
> +void vncws_encode_frame(QIOBuffer *output, const void *payload,
> +                        const size_t payload_size)
>  {
>      size_t header_size = 0;
>      unsigned char opcode = WS_OPCODE_BINARY_FRAME;
> @@ -295,12 +295,12 @@ void vncws_encode_frame(Buffer *output, const void *payload,
>          header_size = 10;
>      }
>  
> -    buffer_reserve(output, header_size + payload_size);
> -    buffer_append(output, header.buf, header_size);
> -    buffer_append(output, payload, payload_size);
> +    qio_buffer_reserve(output, header_size + payload_size);
> +    qio_buffer_append(output, header.buf, header_size);
> +    qio_buffer_append(output, payload, payload_size);
>  }
>  
> -int vncws_decode_frame_header(Buffer *input,
> +int vncws_decode_frame_header(QIOBuffer *input,
>                                size_t *header_size,
>                                size_t *payload_remain,
>                                WsMask *payload_mask)
> @@ -354,7 +354,7 @@ int vncws_decode_frame_header(Buffer *input,
>      return 1;
>  }
>  
> -int vncws_decode_frame_payload(Buffer *input,
> +int vncws_decode_frame_payload(QIOBuffer *input,
>                                 size_t *payload_remain, WsMask *payload_mask,
>                                 uint8_t **payload, size_t *payload_size)
>  {
> diff --git a/ui/vnc-ws.h b/ui/vnc-ws.h
> index 4ab0a8c..2a222a8 100644
> --- a/ui/vnc-ws.h
> +++ b/ui/vnc-ws.h
> @@ -77,13 +77,13 @@ void vncws_handshake_read(void *opaque);
>  long vnc_client_write_ws(VncState *vs);
>  long vnc_client_read_ws(VncState *vs);
>  void vncws_process_handshake(VncState *vs, uint8_t *line, size_t size);
> -void vncws_encode_frame(Buffer *output, const void *payload,
> +void vncws_encode_frame(QIOBuffer *output, const void *payload,
>              const size_t payload_size);
> -int vncws_decode_frame_header(Buffer *input,
> +int vncws_decode_frame_header(QIOBuffer *input,
>                                size_t *header_size,
>                                size_t *payload_remain,
>                                WsMask *payload_mask);
> -int vncws_decode_frame_payload(Buffer *input,
> +int vncws_decode_frame_payload(QIOBuffer *input,
>                                 size_t *payload_remain, WsMask *payload_mask,
>                                 uint8_t **payload, size_t *payload_size);
>  
> diff --git a/ui/vnc.c b/ui/vnc.c
> index 952e551..c4dec90 100644
> --- a/ui/vnc.c
> +++ b/ui/vnc.c
> @@ -647,49 +647,6 @@ void vnc_framebuffer_update(VncState *vs, int x, int y, int w, int h,
>      vnc_write_s32(vs, encoding);
>  }
>  
> -void buffer_reserve(Buffer *buffer, size_t len)
> -{
> -    if ((buffer->capacity - buffer->offset) < len) {
> -        buffer->capacity += (len + 1024);
> -        buffer->buffer = g_realloc(buffer->buffer, buffer->capacity);
> -    }
> -}
> -
> -static int buffer_empty(Buffer *buffer)
> -{
> -    return buffer->offset == 0;
> -}
> -
> -uint8_t *buffer_end(Buffer *buffer)
> -{
> -    return buffer->buffer + buffer->offset;
> -}
> -
> -void buffer_reset(Buffer *buffer)
> -{
> -        buffer->offset = 0;
> -}
> -
> -void buffer_free(Buffer *buffer)
> -{
> -    g_free(buffer->buffer);
> -    buffer->offset = 0;
> -    buffer->capacity = 0;
> -    buffer->buffer = NULL;
> -}
> -
> -void buffer_append(Buffer *buffer, const void *data, size_t len)
> -{
> -    memcpy(buffer->buffer + buffer->offset, data, len);
> -    buffer->offset += len;
> -}
> -
> -void buffer_advance(Buffer *buf, size_t len)
> -{
> -    memmove(buf->buffer, buf->buffer + len,
> -            (buf->offset - len));
> -    buf->offset -= len;
> -}
>  
>  static void vnc_desktop_resize(VncState *vs)
>  {
> @@ -1220,10 +1177,10 @@ void vnc_disconnect_finish(VncState *vs)
>      vnc_lock_output(vs);
>      vnc_qmp_event(vs, QAPI_EVENT_VNC_DISCONNECTED);
>  
> -    buffer_free(&vs->input);
> -    buffer_free(&vs->output);
> -    buffer_free(&vs->ws_input);
> -    buffer_free(&vs->ws_output);
> +    qio_buffer_free(&vs->input);
> +    qio_buffer_free(&vs->output);
> +    qio_buffer_free(&vs->ws_input);
> +    qio_buffer_free(&vs->ws_output);
>  
>      qapi_free_VncClientInfo(vs->info);
>  
> @@ -1251,7 +1208,7 @@ void vnc_disconnect_finish(VncState *vs)
>      if (vs->bh != NULL) {
>          qemu_bh_delete(vs->bh);
>      }
> -    buffer_free(&vs->jobs_buffer);
> +    qio_buffer_free(&vs->jobs_buffer);
>  
>      for (i = 0; i < VNC_STAT_ROWS; ++i) {
>          g_free(vs->lossy_rect[i]);
> @@ -1393,7 +1350,7 @@ static ssize_t vnc_client_write_plain(VncState *vs)
>      if (!ret)
>          return 0;
>  
> -    buffer_advance(&vs->output, ret);
> +    qio_buffer_advance(&vs->output, ret);
>  
>      if (vs->output.offset == 0) {
>          qemu_set_fd_handler(vs->csock, vnc_client_read, NULL, vs);
> @@ -1496,8 +1453,8 @@ static ssize_t vnc_client_read_plain(VncState *vs)
>      ssize_t ret;
>      VNC_DEBUG("Read plain %p size %zd offset %zd\n",
>                vs->input.buffer, vs->input.capacity, vs->input.offset);
> -    buffer_reserve(&vs->input, 4096);
> -    ret = vnc_client_read_buf(vs, buffer_end(&vs->input), 4096);
> +    qio_buffer_reserve(&vs->input, 4096);
> +    ret = vnc_client_read_buf(vs, qio_buffer_end(&vs->input), 4096);
>      if (!ret)
>          return 0;
>      vs->input.offset += ret;
> @@ -1555,7 +1512,7 @@ void vnc_client_read(void *opaque)
>          }
>  
>          if (!ret) {
> -            buffer_advance(&vs->input, len);
> +            qio_buffer_advance(&vs->input, len);
>          } else {
>              vs->read_handler_expect = ret;
>          }
> @@ -1564,13 +1521,13 @@ void vnc_client_read(void *opaque)
>  
>  void vnc_write(VncState *vs, const void *data, size_t len)
>  {
> -    buffer_reserve(&vs->output, len);
> +    qio_buffer_reserve(&vs->output, len);
>  
> -    if (vs->csock != -1 && buffer_empty(&vs->output)) {
> +    if (vs->csock != -1 && qio_buffer_empty(&vs->output)) {
>          qemu_set_fd_handler(vs->csock, vnc_client_read, vnc_client_write, vs);
>      }
>  
> -    buffer_append(&vs->output, data, len);
> +    qio_buffer_append(&vs->output, data, len);
>  }
>  
>  void vnc_write_s32(VncState *vs, int32_t value)
> diff --git a/ui/vnc.h b/ui/vnc.h
> index 4dd769c..339a1bf 100644
> --- a/ui/vnc.h
> +++ b/ui/vnc.h
> @@ -34,6 +34,7 @@
>  #include "audio/audio.h"
>  #include "qemu/bitmap.h"
>  #include "crypto/tlssession.h"
> +#include "io/buffer.h"
>  #include <zlib.h>
>  #include <stdbool.h>
>  
> @@ -56,13 +57,6 @@
>   *
>   *****************************************************************************/
>  
> -typedef struct Buffer
> -{
> -    size_t capacity;
> -    size_t offset;
> -    uint8_t *buffer;
> -} Buffer;
> -
>  typedef struct VncState VncState;
>  typedef struct VncJob VncJob;
>  typedef struct VncRect VncRect;
> @@ -191,15 +185,15 @@ typedef struct VncTight {
>      uint8_t quality;
>      uint8_t compression;
>      uint8_t pixel24;
> -    Buffer tight;
> -    Buffer tmp;
> -    Buffer zlib;
> -    Buffer gradient;
> +    QIOBuffer tight;
> +    QIOBuffer tmp;
> +    QIOBuffer zlib;
> +    QIOBuffer gradient;
>  #ifdef CONFIG_VNC_JPEG
> -    Buffer jpeg;
> +    QIOBuffer jpeg;
>  #endif
>  #ifdef CONFIG_VNC_PNG
> -    Buffer png;
> +    QIOBuffer png;
>  #endif
>      int levels[4];
>      z_stream stream[4];
> @@ -210,18 +204,18 @@ typedef struct VncHextile {
>  } VncHextile;
>  
>  typedef struct VncZlib {
> -    Buffer zlib;
> -    Buffer tmp;
> +    QIOBuffer zlib;
> +    QIOBuffer tmp;
>      z_stream stream;
>      int level;
>  } VncZlib;
>  
>  typedef struct VncZrle {
>      int type;
> -    Buffer fb;
> -    Buffer zrle;
> -    Buffer tmp;
> -    Buffer zlib;
> +    QIOBuffer fb;
> +    QIOBuffer zrle;
> +    QIOBuffer tmp;
> +    QIOBuffer zlib;
>      z_stream stream;
>      VncPalette palette;
>  } VncZrle;
> @@ -290,10 +284,10 @@ struct VncState
>  
>      VncClientInfo *info;
>  
> -    Buffer output;
> -    Buffer input;
> -    Buffer ws_input;
> -    Buffer ws_output;
> +    QIOBuffer output;
> +    QIOBuffer input;
> +    QIOBuffer ws_input;
> +    QIOBuffer ws_output;
>      size_t ws_payload_remain;
>      WsMask ws_payload_mask;
>      /* current output mode information */
> @@ -315,7 +309,7 @@ struct VncState
>      bool initialized;
>      QemuMutex output_mutex;
>      QEMUBH *bh;
> -    Buffer jobs_buffer;
> +    QIOBuffer jobs_buffer;
>  
>      /* Encoding specific, if you add something here, don't forget to
>       *  update vnc_async_encoding_start()
> @@ -535,14 +529,6 @@ ssize_t vnc_client_io_error(VncState *vs, ssize_t ret, int last_errno);
>  void start_client_init(VncState *vs);
>  void start_auth_vnc(VncState *vs);
>  
> -/* Buffer management */
> -void buffer_reserve(Buffer *buffer, size_t len);
> -void buffer_reset(Buffer *buffer);
> -void buffer_free(Buffer *buffer);
> -void buffer_append(Buffer *buffer, const void *data, size_t len);
> -void buffer_advance(Buffer *buf, size_t len);
> -uint8_t *buffer_end(Buffer *buffer);
> -
>  
>  /* Misc helpers */
>  
> 

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [Qemu-devel] [PATCH v1 06/16] coroutine: move into libqemuutil.a library
  2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 06/16] coroutine: move into libqemuutil.a library Daniel P. Berrange
@ 2015-09-22 12:07   ` Paolo Bonzini
  2015-09-24 14:46     ` Daniel P. Berrange
  0 siblings, 1 reply; 32+ messages in thread
From: Paolo Bonzini @ 2015-09-22 12:07 UTC (permalink / raw)
  To: Daniel P. Berrange, qemu-devel
  Cc: Kevin Wolf, Dr. David Alan Gilbert, Stefan Hajnoczi,
	Gerd Hoffmann



On 18/09/2015 15:19, Daniel P. Berrange wrote:
> The coroutine files are currently referenced by the block-obj-y
> variable. The coroutine functionality though is already used by
> more than just the block code. eg migration code uses coroutine
> yield. In the future the I/O channel code will also use the
> coroutine yield functionality. Since the coroutine code is nicely
> self-contained it can be easily built as part of the libqemuutil.a
> library, making it widely available.
> 
> The headers are also moved into include/qemu, instead of the
> include/block directory, since they are now part of the util
> codebase, and the impl was never in the block/ directory
> either.
> 
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>

Since you are at it, can you add an entry to the MAINTAINERS file,
assigning it to Stefan and Kevin?

Paolo

> ---
>  Makefile.objs                                           | 4 ----
>  block.c                                                 | 2 +-
>  block/qcow2.h                                           | 2 +-
>  block/vdi.c                                             | 2 +-
>  block/write-threshold.c                                 | 2 +-
>  blockjob.c                                              | 2 +-
>  hw/9pfs/codir.c                                         | 2 +-
>  hw/9pfs/cofile.c                                        | 2 +-
>  hw/9pfs/cofs.c                                          | 2 +-
>  hw/9pfs/coxattr.c                                       | 2 +-
>  hw/9pfs/virtio-9p-coth.c                                | 2 +-
>  hw/9pfs/virtio-9p-coth.h                                | 2 +-
>  hw/9pfs/virtio-9p.h                                     | 2 +-
>  include/block/block.h                                   | 2 +-
>  include/block/block_int.h                               | 2 +-
>  include/{block => qemu}/coroutine.h                     | 0
>  include/{block => qemu}/coroutine_int.h                 | 2 +-
>  migration/qemu-file-buf.c                               | 2 +-
>  migration/qemu-file-stdio.c                             | 2 +-
>  migration/qemu-file-unix.c                              | 2 +-
>  migration/qemu-file.c                                   | 2 +-
>  migration/rdma.c                                        | 2 +-
>  nbd.c                                                   | 2 +-
>  tests/test-coroutine.c                                  | 4 ++--
>  tests/test-vmstate.c                                    | 2 +-
>  thread-pool.c                                           | 2 +-
>  util/Makefile.objs                                      | 3 +++
>  coroutine-gthread.c => util/coroutine-gthread.c         | 2 +-
>  coroutine-sigaltstack.c => util/coroutine-sigaltstack.c | 2 +-
>  coroutine-ucontext.c => util/coroutine-ucontext.c       | 2 +-
>  coroutine-win32.c => util/coroutine-win32.c             | 2 +-
>  qemu-coroutine-io.c => util/qemu-coroutine-io.c         | 2 +-
>  qemu-coroutine-lock.c => util/qemu-coroutine-lock.c     | 4 ++--
>  qemu-coroutine-sleep.c => util/qemu-coroutine-sleep.c   | 2 +-
>  qemu-coroutine.c => util/qemu-coroutine.c               | 4 ++--
>  35 files changed, 38 insertions(+), 39 deletions(-)
>  rename include/{block => qemu}/coroutine.h (100%)
>  rename include/{block => qemu}/coroutine_int.h (98%)
>  rename coroutine-gthread.c => util/coroutine-gthread.c (99%)
>  rename coroutine-sigaltstack.c => util/coroutine-sigaltstack.c (99%)
>  rename coroutine-ucontext.c => util/coroutine-ucontext.c (99%)
>  rename coroutine-win32.c => util/coroutine-win32.c (98%)
>  rename qemu-coroutine-io.c => util/qemu-coroutine-io.c (99%)
>  rename qemu-coroutine-lock.c => util/qemu-coroutine-lock.c (98%)
>  rename qemu-coroutine-sleep.c => util/qemu-coroutine-sleep.c (96%)
>  rename qemu-coroutine.c => util/qemu-coroutine.c (98%)
> 
> diff --git a/Makefile.objs b/Makefile.objs
> index 3df2efc..2e94eaf 100644
> --- a/Makefile.objs
> +++ b/Makefile.objs
> @@ -14,10 +14,6 @@ block-obj-$(CONFIG_WIN32) += aio-win32.o
>  block-obj-y += block/
>  block-obj-y += qemu-io-cmds.o
>  
> -block-obj-y += qemu-coroutine.o qemu-coroutine-lock.o qemu-coroutine-io.o
> -block-obj-y += qemu-coroutine-sleep.o
> -block-obj-y += coroutine-$(CONFIG_COROUTINE_BACKEND).o
> -
>  block-obj-m = block/
>  
>  #######################################################################
> diff --git a/block.c b/block.c
> index 6268e37..cfc9a13 100644
> --- a/block.c
> +++ b/block.c
> @@ -33,7 +33,7 @@
>  #include "sysemu/block-backend.h"
>  #include "sysemu/sysemu.h"
>  #include "qemu/notify.h"
> -#include "block/coroutine.h"
> +#include "qemu/coroutine.h"
>  #include "block/qapi.h"
>  #include "qmp-commands.h"
>  #include "qemu/timer.h"
> diff --git a/block/qcow2.h b/block/qcow2.h
> index d700bf1..27b919c 100644
> --- a/block/qcow2.h
> +++ b/block/qcow2.h
> @@ -26,7 +26,7 @@
>  #define BLOCK_QCOW2_H
>  
>  #include "crypto/cipher.h"
> -#include "block/coroutine.h"
> +#include "qemu/coroutine.h"
>  
>  //#define DEBUG_ALLOC
>  //#define DEBUG_ALLOC2
> diff --git a/block/vdi.c b/block/vdi.c
> index 062a654..d662f41 100644
> --- a/block/vdi.c
> +++ b/block/vdi.c
> @@ -53,7 +53,7 @@
>  #include "block/block_int.h"
>  #include "qemu/module.h"
>  #include "migration/migration.h"
> -#include "block/coroutine.h"
> +#include "qemu/coroutine.h"
>  
>  #if defined(CONFIG_UUID)
>  #include <uuid/uuid.h>
> diff --git a/block/write-threshold.c b/block/write-threshold.c
> index a53c1f5..0fe3891 100644
> --- a/block/write-threshold.c
> +++ b/block/write-threshold.c
> @@ -11,7 +11,7 @@
>   */
>  
>  #include "block/block_int.h"
> -#include "block/coroutine.h"
> +#include "qemu/coroutine.h"
>  #include "block/write-threshold.h"
>  #include "qemu/notify.h"
>  #include "qapi-event.h"
> diff --git a/blockjob.c b/blockjob.c
> index 62bb906..dc42d15 100644
> --- a/blockjob.c
> +++ b/blockjob.c
> @@ -31,7 +31,7 @@
>  #include "block/block_int.h"
>  #include "qapi/qmp/qerror.h"
>  #include "qapi/qmp/qjson.h"
> -#include "block/coroutine.h"
> +#include "qemu/coroutine.h"
>  #include "qmp-commands.h"
>  #include "qemu/timer.h"
>  #include "qapi-event.h"
> diff --git a/hw/9pfs/codir.c b/hw/9pfs/codir.c
> index 65ad329..ec9cc7f 100644
> --- a/hw/9pfs/codir.c
> +++ b/hw/9pfs/codir.c
> @@ -14,7 +14,7 @@
>  
>  #include "fsdev/qemu-fsdev.h"
>  #include "qemu/thread.h"
> -#include "block/coroutine.h"
> +#include "qemu/coroutine.h"
>  #include "virtio-9p-coth.h"
>  
>  int v9fs_co_readdir_r(V9fsPDU *pdu, V9fsFidState *fidp, struct dirent *dent,
> diff --git a/hw/9pfs/cofile.c b/hw/9pfs/cofile.c
> index 2efebf3..7cb55ee 100644
> --- a/hw/9pfs/cofile.c
> +++ b/hw/9pfs/cofile.c
> @@ -14,7 +14,7 @@
>  
>  #include "fsdev/qemu-fsdev.h"
>  #include "qemu/thread.h"
> -#include "block/coroutine.h"
> +#include "qemu/coroutine.h"
>  #include "virtio-9p-coth.h"
>  
>  int v9fs_co_st_gen(V9fsPDU *pdu, V9fsPath *path, mode_t st_mode,
> diff --git a/hw/9pfs/cofs.c b/hw/9pfs/cofs.c
> index 42ee614..e1953a9 100644
> --- a/hw/9pfs/cofs.c
> +++ b/hw/9pfs/cofs.c
> @@ -14,7 +14,7 @@
>  
>  #include "fsdev/qemu-fsdev.h"
>  #include "qemu/thread.h"
> -#include "block/coroutine.h"
> +#include "qemu/coroutine.h"
>  #include "virtio-9p-coth.h"
>  
>  static ssize_t __readlink(V9fsState *s, V9fsPath *path, V9fsString *buf)
> diff --git a/hw/9pfs/coxattr.c b/hw/9pfs/coxattr.c
> index 18ee08d..55c0d23 100644
> --- a/hw/9pfs/coxattr.c
> +++ b/hw/9pfs/coxattr.c
> @@ -14,7 +14,7 @@
>  
>  #include "fsdev/qemu-fsdev.h"
>  #include "qemu/thread.h"
> -#include "block/coroutine.h"
> +#include "qemu/coroutine.h"
>  #include "virtio-9p-coth.h"
>  
>  int v9fs_co_llistxattr(V9fsPDU *pdu, V9fsPath *path, void *value, size_t size)
> diff --git a/hw/9pfs/virtio-9p-coth.c b/hw/9pfs/virtio-9p-coth.c
> index 8185c53..5057f8d 100644
> --- a/hw/9pfs/virtio-9p-coth.c
> +++ b/hw/9pfs/virtio-9p-coth.c
> @@ -15,7 +15,7 @@
>  #include "fsdev/qemu-fsdev.h"
>  #include "qemu/thread.h"
>  #include "qemu/event_notifier.h"
> -#include "block/coroutine.h"
> +#include "qemu/coroutine.h"
>  #include "virtio-9p-coth.h"
>  
>  /* v9fs glib thread pool */
> diff --git a/hw/9pfs/virtio-9p-coth.h b/hw/9pfs/virtio-9p-coth.h
> index 4f51b25..0fbe49a 100644
> --- a/hw/9pfs/virtio-9p-coth.h
> +++ b/hw/9pfs/virtio-9p-coth.h
> @@ -16,7 +16,7 @@
>  #define _QEMU_VIRTIO_9P_COTH_H
>  
>  #include "qemu/thread.h"
> -#include "block/coroutine.h"
> +#include "qemu/coroutine.h"
>  #include "virtio-9p.h"
>  #include <glib.h>
>  
> diff --git a/hw/9pfs/virtio-9p.h b/hw/9pfs/virtio-9p.h
> index 2e7d488..d7a4dc1 100644
> --- a/hw/9pfs/virtio-9p.h
> +++ b/hw/9pfs/virtio-9p.h
> @@ -13,7 +13,7 @@
>  #include "fsdev/file-op-9p.h"
>  #include "fsdev/virtio-9p-marshal.h"
>  #include "qemu/thread.h"
> -#include "block/coroutine.h"
> +#include "qemu/coroutine.h"
>  
>  enum {
>      P9_TLERROR = 6,
> diff --git a/include/block/block.h b/include/block/block.h
> index ef67353..4d77291 100644
> --- a/include/block/block.h
> +++ b/include/block/block.h
> @@ -4,7 +4,7 @@
>  #include "block/aio.h"
>  #include "qemu-common.h"
>  #include "qemu/option.h"
> -#include "block/coroutine.h"
> +#include "qemu/coroutine.h"
>  #include "block/accounting.h"
>  #include "qapi/qmp/qobject.h"
>  #include "qapi-types.h"
> diff --git a/include/block/block_int.h b/include/block/block_int.h
> index 14ad4c3..c37ed77 100644
> --- a/include/block/block_int.h
> +++ b/include/block/block_int.h
> @@ -28,7 +28,7 @@
>  #include "block/block.h"
>  #include "qemu/option.h"
>  #include "qemu/queue.h"
> -#include "block/coroutine.h"
> +#include "qemu/coroutine.h"
>  #include "qemu/timer.h"
>  #include "qapi-types.h"
>  #include "qemu/hbitmap.h"
> diff --git a/include/block/coroutine.h b/include/qemu/coroutine.h
> similarity index 100%
> rename from include/block/coroutine.h
> rename to include/qemu/coroutine.h
> diff --git a/include/block/coroutine_int.h b/include/qemu/coroutine_int.h
> similarity index 98%
> rename from include/block/coroutine_int.h
> rename to include/qemu/coroutine_int.h
> index 9aa1aae..42d6838 100644
> --- a/include/block/coroutine_int.h
> +++ b/include/qemu/coroutine_int.h
> @@ -26,7 +26,7 @@
>  #define QEMU_COROUTINE_INT_H
>  
>  #include "qemu/queue.h"
> -#include "block/coroutine.h"
> +#include "qemu/coroutine.h"
>  
>  typedef enum {
>      COROUTINE_YIELD = 1,
> diff --git a/migration/qemu-file-buf.c b/migration/qemu-file-buf.c
> index 2de9330..556f5dc 100644
> --- a/migration/qemu-file-buf.c
> +++ b/migration/qemu-file-buf.c
> @@ -29,7 +29,7 @@
>  #include "qemu/error-report.h"
>  #include "qemu/iov.h"
>  #include "qemu/sockets.h"
> -#include "block/coroutine.h"
> +#include "qemu/coroutine.h"
>  #include "migration/migration.h"
>  #include "migration/qemu-file.h"
>  #include "migration/qemu-file-internal.h"
> diff --git a/migration/qemu-file-stdio.c b/migration/qemu-file-stdio.c
> index 285068b..002dc5d 100644
> --- a/migration/qemu-file-stdio.c
> +++ b/migration/qemu-file-stdio.c
> @@ -22,7 +22,7 @@
>   * THE SOFTWARE.
>   */
>  #include "qemu-common.h"
> -#include "block/coroutine.h"
> +#include "qemu/coroutine.h"
>  #include "migration/qemu-file.h"
>  
>  typedef struct QEMUFileStdio {
> diff --git a/migration/qemu-file-unix.c b/migration/qemu-file-unix.c
> index bfbc086..e4f195a 100644
> --- a/migration/qemu-file-unix.c
> +++ b/migration/qemu-file-unix.c
> @@ -24,7 +24,7 @@
>  #include "qemu-common.h"
>  #include "qemu/iov.h"
>  #include "qemu/sockets.h"
> -#include "block/coroutine.h"
> +#include "qemu/coroutine.h"
>  #include "migration/qemu-file.h"
>  #include "migration/qemu-file-internal.h"
>  
> diff --git a/migration/qemu-file.c b/migration/qemu-file.c
> index 6bb3dc1..d2359c4 100644
> --- a/migration/qemu-file.c
> +++ b/migration/qemu-file.c
> @@ -26,7 +26,7 @@
>  #include "qemu/error-report.h"
>  #include "qemu/iov.h"
>  #include "qemu/sockets.h"
> -#include "block/coroutine.h"
> +#include "qemu/coroutine.h"
>  #include "migration/migration.h"
>  #include "migration/qemu-file.h"
>  #include "migration/qemu-file-internal.h"
> diff --git a/migration/rdma.c b/migration/rdma.c
> index 9424834..854fde4 100644
> --- a/migration/rdma.c
> +++ b/migration/rdma.c
> @@ -19,7 +19,7 @@
>  #include "qemu/main-loop.h"
>  #include "qemu/sockets.h"
>  #include "qemu/bitmap.h"
> -#include "block/coroutine.h"
> +#include "qemu/coroutine.h"
>  #include <stdio.h>
>  #include <sys/types.h>
>  #include <sys/socket.h>
> diff --git a/nbd.c b/nbd.c
> index 07240bd..1724a1f 100644
> --- a/nbd.c
> +++ b/nbd.c
> @@ -19,7 +19,7 @@
>  #include "block/nbd.h"
>  #include "sysemu/block-backend.h"
>  
> -#include "block/coroutine.h"
> +#include "qemu/coroutine.h"
>  
>  #include <errno.h>
>  #include <string.h>
> diff --git a/tests/test-coroutine.c b/tests/test-coroutine.c
> index b552d9f..f5951cb 100644
> --- a/tests/test-coroutine.c
> +++ b/tests/test-coroutine.c
> @@ -12,8 +12,8 @@
>   */
>  
>  #include <glib.h>
> -#include "block/coroutine.h"
> -#include "block/coroutine_int.h"
> +#include "qemu/coroutine.h"
> +#include "qemu/coroutine_int.h"
>  
>  /*
>   * Check that qemu_in_coroutine() works
> diff --git a/tests/test-vmstate.c b/tests/test-vmstate.c
> index 1d620e0..4d13bd0 100644
> --- a/tests/test-vmstate.c
> +++ b/tests/test-vmstate.c
> @@ -27,7 +27,7 @@
>  #include "qemu-common.h"
>  #include "migration/migration.h"
>  #include "migration/vmstate.h"
> -#include "block/coroutine.h"
> +#include "qemu/coroutine.h"
>  
>  static char temp_file[] = "/tmp/vmst.test.XXXXXX";
>  static int temp_fd;
> diff --git a/thread-pool.c b/thread-pool.c
> index ac909f4..402c778 100644
> --- a/thread-pool.c
> +++ b/thread-pool.c
> @@ -18,7 +18,7 @@
>  #include "qemu/queue.h"
>  #include "qemu/thread.h"
>  #include "qemu/osdep.h"
> -#include "block/coroutine.h"
> +#include "qemu/coroutine.h"
>  #include "trace.h"
>  #include "block/thread-pool.h"
>  #include "qemu/main-loop.h"
> diff --git a/util/Makefile.objs b/util/Makefile.objs
> index 114d657..d8d7e7a 100644
> --- a/util/Makefile.objs
> +++ b/util/Makefile.objs
> @@ -18,3 +18,6 @@ util-obj-y += getauxval.o
>  util-obj-y += readline.o
>  util-obj-y += rfifolock.o
>  util-obj-y += rcu.o
> +util-obj-y += qemu-coroutine.o qemu-coroutine-lock.o qemu-coroutine-io.o
> +util-obj-y += qemu-coroutine-sleep.o
> +util-obj-y += coroutine-$(CONFIG_COROUTINE_BACKEND).o
> diff --git a/coroutine-gthread.c b/util/coroutine-gthread.c
> similarity index 99%
> rename from coroutine-gthread.c
> rename to util/coroutine-gthread.c
> index 6bd6d6b..0bcd778 100644
> --- a/coroutine-gthread.c
> +++ b/util/coroutine-gthread.c
> @@ -20,7 +20,7 @@
>  
>  #include <glib.h>
>  #include "qemu-common.h"
> -#include "block/coroutine_int.h"
> +#include "qemu/coroutine_int.h"
>  
>  typedef struct {
>      Coroutine base;
> diff --git a/coroutine-sigaltstack.c b/util/coroutine-sigaltstack.c
> similarity index 99%
> rename from coroutine-sigaltstack.c
> rename to util/coroutine-sigaltstack.c
> index 63519ff..39842a4 100644
> --- a/coroutine-sigaltstack.c
> +++ b/util/coroutine-sigaltstack.c
> @@ -31,7 +31,7 @@
>  #include <pthread.h>
>  #include <signal.h>
>  #include "qemu-common.h"
> -#include "block/coroutine_int.h"
> +#include "qemu/coroutine_int.h"
>  
>  typedef struct {
>      Coroutine base;
> diff --git a/coroutine-ucontext.c b/util/coroutine-ucontext.c
> similarity index 99%
> rename from coroutine-ucontext.c
> rename to util/coroutine-ucontext.c
> index 259fcb4..26cbebb 100644
> --- a/coroutine-ucontext.c
> +++ b/util/coroutine-ucontext.c
> @@ -27,7 +27,7 @@
>  #include <stdint.h>
>  #include <ucontext.h>
>  #include "qemu-common.h"
> -#include "block/coroutine_int.h"
> +#include "qemu/coroutine_int.h"
>  
>  #ifdef CONFIG_VALGRIND_H
>  #include <valgrind/valgrind.h>
> diff --git a/coroutine-win32.c b/util/coroutine-win32.c
> similarity index 98%
> rename from coroutine-win32.c
> rename to util/coroutine-win32.c
> index 17ace37..4f922c5 100644
> --- a/coroutine-win32.c
> +++ b/util/coroutine-win32.c
> @@ -23,7 +23,7 @@
>   */
>  
>  #include "qemu-common.h"
> -#include "block/coroutine_int.h"
> +#include "qemu/coroutine_int.h"
>  
>  typedef struct
>  {
> diff --git a/qemu-coroutine-io.c b/util/qemu-coroutine-io.c
> similarity index 99%
> rename from qemu-coroutine-io.c
> rename to util/qemu-coroutine-io.c
> index 28dc735..e1eae73 100644
> --- a/qemu-coroutine-io.c
> +++ b/util/qemu-coroutine-io.c
> @@ -24,7 +24,7 @@
>   */
>  #include "qemu-common.h"
>  #include "qemu/sockets.h"
> -#include "block/coroutine.h"
> +#include "qemu/coroutine.h"
>  #include "qemu/iov.h"
>  #include "qemu/main-loop.h"
>  
> diff --git a/qemu-coroutine-lock.c b/util/qemu-coroutine-lock.c
> similarity index 98%
> rename from qemu-coroutine-lock.c
> rename to util/qemu-coroutine-lock.c
> index 6b49033..130ee19 100644
> --- a/qemu-coroutine-lock.c
> +++ b/util/qemu-coroutine-lock.c
> @@ -23,8 +23,8 @@
>   */
>  
>  #include "qemu-common.h"
> -#include "block/coroutine.h"
> -#include "block/coroutine_int.h"
> +#include "qemu/coroutine.h"
> +#include "qemu/coroutine_int.h"
>  #include "qemu/queue.h"
>  #include "trace.h"
>  
> diff --git a/qemu-coroutine-sleep.c b/util/qemu-coroutine-sleep.c
> similarity index 96%
> rename from qemu-coroutine-sleep.c
> rename to util/qemu-coroutine-sleep.c
> index 9abb7fd..b35db56 100644
> --- a/qemu-coroutine-sleep.c
> +++ b/util/qemu-coroutine-sleep.c
> @@ -11,7 +11,7 @@
>   *
>   */
>  
> -#include "block/coroutine.h"
> +#include "qemu/coroutine.h"
>  #include "qemu/timer.h"
>  #include "block/aio.h"
>  
> diff --git a/qemu-coroutine.c b/util/qemu-coroutine.c
> similarity index 98%
> rename from qemu-coroutine.c
> rename to util/qemu-coroutine.c
> index c17a92b..8953560 100644
> --- a/qemu-coroutine.c
> +++ b/util/qemu-coroutine.c
> @@ -16,8 +16,8 @@
>  #include "qemu-common.h"
>  #include "qemu/thread.h"
>  #include "qemu/atomic.h"
> -#include "block/coroutine.h"
> -#include "block/coroutine_int.h"
> +#include "qemu/coroutine.h"
> +#include "qemu/coroutine_int.h"
>  
>  enum {
>      POOL_BATCH_SIZE = 64,
> 

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [Qemu-devel] [PATCH v1 07/16] io: add abstract QIOChannel classes
  2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 07/16] io: add abstract QIOChannel classes Daniel P. Berrange
@ 2015-09-22 12:19   ` Paolo Bonzini
  2015-09-22 12:24     ` Daniel P. Berrange
  0 siblings, 1 reply; 32+ messages in thread
From: Paolo Bonzini @ 2015-09-22 12:19 UTC (permalink / raw)
  To: Daniel P. Berrange, qemu-devel; +Cc: Dr. David Alan Gilbert, Gerd Hoffmann



On 18/09/2015 15:19, Daniel P. Berrange wrote:
> +    QIO_CHANNEL_FEATURE_FD_PASS  = (1 << 0),
> +    QIO_CHANNEL_FEATURE_SHUTDOWN = (1 << 1),
> +    QIO_CHANNEL_FEATURE_DELAY    = (1 << 2),
> +    QIO_CHANNEL_FEATURE_CORK     = (1 << 3),

TCP_NODELAY and TCP_CORK are just hints; I think it is okay to just
ignore them if not supported.  You obviously disagree, so the question
is why? :)

Paolo

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [Qemu-devel] [PATCH v1 09/16] io: pull Buffer code out of VNC module
  2015-09-22 12:04   ` Paolo Bonzini
@ 2015-09-22 12:20     ` Daniel P. Berrange
  2015-09-22 12:21       ` Paolo Bonzini
  0 siblings, 1 reply; 32+ messages in thread
From: Daniel P. Berrange @ 2015-09-22 12:20 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: Gerd Hoffmann, qemu-devel, Dr. David Alan Gilbert

On Tue, Sep 22, 2015 at 02:04:35PM +0200, Paolo Bonzini wrote:
> 
> 
> On 18/09/2015 15:19, Daniel P. Berrange wrote:
> > The Buffer code in the VNC server is useful for the IO channel
> > code, so pull it out into a shared module, QIOBuffer.
> 
> Do all traces disappear from VNC once you're done?  If not, can you
> instead move it to util/?

It is still used in VNC code for decoding of various framebuffer
encodings. Only the websock usage gets eliminated. I'm happy to
move it to util/ directory.  Any preference on a name for the
APIs. I used QIOBuffer since i placed it under io/ directory.
Could keep that name when moved to util/, or give it a different
one.

> > Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> > ---
> >  include/io/buffer.h | 118 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> >  io/Makefile.objs    |   3 +-
> >  io/buffer.c         |  65 +++++++++++++++++++++++++++++
> >  ui/vnc-auth-sasl.c  |   4 +-
> >  ui/vnc-enc-tight.c  |  38 ++++++++---------
> >  ui/vnc-enc-zlib.c   |   6 +--
> >  ui/vnc-enc-zrle.c   |  18 ++++----
> >  ui/vnc-jobs.c       |  15 +++----
> >  ui/vnc-ws.c         |  36 ++++++++--------
> >  ui/vnc-ws.h         |   6 +--
> >  ui/vnc.c            |  67 ++++++-----------------------
> >  ui/vnc.h            |  50 ++++++++--------------
> >  12 files changed, 277 insertions(+), 149 deletions(-)
> >  create mode 100644 include/io/buffer.h
> >  create mode 100644 io/buffer.c

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [Qemu-devel] [PATCH v1 09/16] io: pull Buffer code out of VNC module
  2015-09-22 12:20     ` Daniel P. Berrange
@ 2015-09-22 12:21       ` Paolo Bonzini
  0 siblings, 0 replies; 32+ messages in thread
From: Paolo Bonzini @ 2015-09-22 12:21 UTC (permalink / raw)
  To: Daniel P. Berrange; +Cc: Gerd Hoffmann, qemu-devel, Dr. David Alan Gilbert



On 22/09/2015 14:20, Daniel P. Berrange wrote:
> It is still used in VNC code for decoding of various framebuffer
> encodings. Only the websock usage gets eliminated. I'm happy to
> move it to util/ directory.  Any preference on a name for the
> APIs. I used QIOBuffer since i placed it under io/ directory.
> Could keep that name when moved to util/, or give it a different
> one.

Just keeping Buffer would be okay with me.

Paolo

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework
  2015-09-18 13:18 [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework Daniel P. Berrange
                   ` (15 preceding siblings ...)
  2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 16/16] io: add QIOChannelBuffer class Daniel P. Berrange
@ 2015-09-22 12:23 ` Paolo Bonzini
  2015-09-24 14:51   ` Daniel P. Berrange
  16 siblings, 1 reply; 32+ messages in thread
From: Paolo Bonzini @ 2015-09-22 12:23 UTC (permalink / raw)
  To: Daniel P. Berrange, qemu-devel; +Cc: Dr. David Alan Gilbert, Gerd Hoffmann



On 18/09/2015 15:18, Daniel P. Berrange wrote:
> David Gilbert had some feedback on my previous RFC series which
> I have incorporated into this new posting. I did forget his
> comments about using #defines instead of magic constants in
> the websock patch, so that's still on my todo list.
> 
> Finally I've listed myself as maintainer for the new io/ and
> include/io/ directory prefixes.

I only made a few very small remarks.  The API is reasonable and you've
proved its utility by using it in the rest of your project.

I remember I had some remarks about the rest of the conversion and
especially the integration with coroutines, but it's not part of this
series.

Paolo

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [Qemu-devel] [PATCH v1 07/16] io: add abstract QIOChannel classes
  2015-09-22 12:19   ` Paolo Bonzini
@ 2015-09-22 12:24     ` Daniel P. Berrange
  2015-09-22 12:28       ` Paolo Bonzini
  0 siblings, 1 reply; 32+ messages in thread
From: Daniel P. Berrange @ 2015-09-22 12:24 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: Gerd Hoffmann, qemu-devel, Dr. David Alan Gilbert

On Tue, Sep 22, 2015 at 02:19:27PM +0200, Paolo Bonzini wrote:
> 
> 
> On 18/09/2015 15:19, Daniel P. Berrange wrote:
> > +    QIO_CHANNEL_FEATURE_FD_PASS  = (1 << 0),
> > +    QIO_CHANNEL_FEATURE_SHUTDOWN = (1 << 1),
> > +    QIO_CHANNEL_FEATURE_DELAY    = (1 << 2),
> > +    QIO_CHANNEL_FEATURE_CORK     = (1 << 3),
> 
> TCP_NODELAY and TCP_CORK are just hints; I think it is okay to just
> ignore them if not supported.  You obviously disagree, so the question
> is why? :)

Well I was just trying not to second guess what future uses we might
have of the QIOChannel API, so I went for the approach of providing a
way to probe any optional features upfront. Code doesn't have to use
this if it doesn't want to - it can just ignore errors from the API
call later.


Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [Qemu-devel] [PATCH v1 07/16] io: add abstract QIOChannel classes
  2015-09-22 12:24     ` Daniel P. Berrange
@ 2015-09-22 12:28       ` Paolo Bonzini
  2015-09-24 14:47         ` Daniel P. Berrange
  0 siblings, 1 reply; 32+ messages in thread
From: Paolo Bonzini @ 2015-09-22 12:28 UTC (permalink / raw)
  To: Daniel P. Berrange; +Cc: Gerd Hoffmann, qemu-devel, Dr. David Alan Gilbert



On 22/09/2015 14:24, Daniel P. Berrange wrote:
> On Tue, Sep 22, 2015 at 02:19:27PM +0200, Paolo Bonzini wrote:
>>
>>
>> On 18/09/2015 15:19, Daniel P. Berrange wrote:
>>> +    QIO_CHANNEL_FEATURE_FD_PASS  = (1 << 0),
>>> +    QIO_CHANNEL_FEATURE_SHUTDOWN = (1 << 1),
>>> +    QIO_CHANNEL_FEATURE_DELAY    = (1 << 2),
>>> +    QIO_CHANNEL_FEATURE_CORK     = (1 << 3),
>>
>> TCP_NODELAY and TCP_CORK are just hints; I think it is okay to just
>> ignore them if not supported.  You obviously disagree, so the question
>> is why? :)
> 
> Well I was just trying not to second guess what future uses we might
> have of the QIOChannel API, so I went for the approach of providing a
> way to probe any optional features upfront. Code doesn't have to use
> this if it doesn't want to - it can just ignore errors from the API
> call later.

Perhaps we can get to a middle ground where you can probe them but no
errors are reported?

Reporting errors is relatively heavyweight (g_new, possibly in a hot
path if the corresponding feature is not supported) and in general QEMU
never checked for the return value of socket_set_cork and
socket_set_nodelay.

The probe, however, should also test if SOL_TCP and TCP_CORK are
defined.  I think corking is a Linux-ism.

Paolo

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [Qemu-devel] [PATCH v1 06/16] coroutine: move into libqemuutil.a library
  2015-09-22 12:07   ` Paolo Bonzini
@ 2015-09-24 14:46     ` Daniel P. Berrange
  0 siblings, 0 replies; 32+ messages in thread
From: Daniel P. Berrange @ 2015-09-24 14:46 UTC (permalink / raw)
  To: Paolo Bonzini
  Cc: Kevin Wolf, Gerd Hoffmann, qemu-devel, Stefan Hajnoczi,
	Dr. David Alan Gilbert

On Tue, Sep 22, 2015 at 02:07:27PM +0200, Paolo Bonzini wrote:
> 
> 
> On 18/09/2015 15:19, Daniel P. Berrange wrote:
> > The coroutine files are currently referenced by the block-obj-y
> > variable. The coroutine functionality though is already used by
> > more than just the block code. eg migration code uses coroutine
> > yield. In the future the I/O channel code will also use the
> > coroutine yield functionality. Since the coroutine code is nicely
> > self-contained it can be easily built as part of the libqemuutil.a
> > library, making it widely available.
> > 
> > The headers are also moved into include/qemu, instead of the
> > include/block directory, since they are now part of the util
> > codebase, and the impl was never in the block/ directory
> > either.
> > 
> > Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> 
> Since you are at it, can you add an entry to the MAINTAINERS file,
> assigning it to Stefan and Kevin?

Yep, will do.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [Qemu-devel] [PATCH v1 07/16] io: add abstract QIOChannel classes
  2015-09-22 12:28       ` Paolo Bonzini
@ 2015-09-24 14:47         ` Daniel P. Berrange
  0 siblings, 0 replies; 32+ messages in thread
From: Daniel P. Berrange @ 2015-09-24 14:47 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: Gerd Hoffmann, qemu-devel, Dr. David Alan Gilbert

On Tue, Sep 22, 2015 at 02:28:25PM +0200, Paolo Bonzini wrote:
> 
> 
> On 22/09/2015 14:24, Daniel P. Berrange wrote:
> > On Tue, Sep 22, 2015 at 02:19:27PM +0200, Paolo Bonzini wrote:
> >>
> >>
> >> On 18/09/2015 15:19, Daniel P. Berrange wrote:
> >>> +    QIO_CHANNEL_FEATURE_FD_PASS  = (1 << 0),
> >>> +    QIO_CHANNEL_FEATURE_SHUTDOWN = (1 << 1),
> >>> +    QIO_CHANNEL_FEATURE_DELAY    = (1 << 2),
> >>> +    QIO_CHANNEL_FEATURE_CORK     = (1 << 3),
> >>
> >> TCP_NODELAY and TCP_CORK are just hints; I think it is okay to just
> >> ignore them if not supported.  You obviously disagree, so the question
> >> is why? :)
> > 
> > Well I was just trying not to second guess what future uses we might
> > have of the QIOChannel API, so I went for the approach of providing a
> > way to probe any optional features upfront. Code doesn't have to use
> > this if it doesn't want to - it can just ignore errors from the API
> > call later.
> 
> Perhaps we can get to a middle ground where you can probe them but no
> errors are reported?
> 
> Reporting errors is relatively heavyweight (g_new, possibly in a hot
> path if the corresponding feature is not supported) and in general QEMU
> never checked for the return value of socket_set_cork and
> socket_set_nodelay.
> 
> The probe, however, should also test if SOL_TCP and TCP_CORK are
> defined.  I think corking is a Linux-ism.

Having slept on it now, I think I'll just remove these two feature
flags and make the methods void with no error reporting. We can
always add feature probing back at a later date if it becomes
clearly required.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework
  2015-09-22 12:23 ` [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework Paolo Bonzini
@ 2015-09-24 14:51   ` Daniel P. Berrange
  2015-09-25 10:18     ` Paolo Bonzini
  0 siblings, 1 reply; 32+ messages in thread
From: Daniel P. Berrange @ 2015-09-24 14:51 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: Gerd Hoffmann, qemu-devel, Dr. David Alan Gilbert

On Tue, Sep 22, 2015 at 02:23:20PM +0200, Paolo Bonzini wrote:
> 
> 
> On 18/09/2015 15:18, Daniel P. Berrange wrote:
> > David Gilbert had some feedback on my previous RFC series which
> > I have incorporated into this new posting. I did forget his
> > comments about using #defines instead of magic constants in
> > the websock patch, so that's still on my todo list.
> > 
> > Finally I've listed myself as maintainer for the new io/ and
> > include/io/ directory prefixes.
> 
> I only made a few very small remarks.  The API is reasonable and you've
> proved its utility by using it in the rest of your project.

Thanks, I'd still love one or more people to do a detailed review of
the I/O channel implementations before I think about sending a PULL
request, since there's a fair amount of code in there....

> I remember I had some remarks about the rest of the conversion and
> especially the integration with coroutines, but it's not part of this
> series.

I'm curious now as to which bit you might be refering to here :-) IIRC
the only place I had any interaction with coroutines was the migration
code QEMUFile implementation, where I need to yield when reads block.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [Qemu-devel] [PATCH v1 16/16] io: add QIOChannelBuffer class
  2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 16/16] io: add QIOChannelBuffer class Daniel P. Berrange
@ 2015-09-25 10:15   ` Paolo Bonzini
  2015-09-25 10:21     ` Daniel P. Berrange
  0 siblings, 1 reply; 32+ messages in thread
From: Paolo Bonzini @ 2015-09-25 10:15 UTC (permalink / raw)
  To: Daniel P. Berrange, qemu-devel; +Cc: Dr. David Alan Gilbert, Gerd Hoffmann



On 18/09/2015 15:19, Daniel P. Berrange wrote:
> +    if (fds || nfds) {
> +        error_setg_errno(errp, EINVAL,
> +                         "Channel does not support buffer descriptor passing");
> +        return -1;
> +    }

Typo (file descriptor passing) and the check could be hoisted to
io/channel.c by using features.

In fact capabilities could perhaps be stored statically in struct
QIOChannel as a bitmap?

Paolo

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework
  2015-09-24 14:51   ` Daniel P. Berrange
@ 2015-09-25 10:18     ` Paolo Bonzini
  2015-09-25 10:20       ` Daniel P. Berrange
  0 siblings, 1 reply; 32+ messages in thread
From: Paolo Bonzini @ 2015-09-25 10:18 UTC (permalink / raw)
  To: Daniel P. Berrange; +Cc: Gerd Hoffmann, Dr. David Alan Gilbert, qemu-devel



On 24/09/2015 16:51, Daniel P. Berrange wrote:
> Thanks, I'd still love one or more people to do a detailed review of
> the I/O channel implementations before I think about sending a PULL
> request, since there's a fair amount of code in there....

I did a review of the file, socket, command and buffer code.  I made
another small suggestion regarding error checking.

The command code could add support for shutdown by closing the parent's
end of the pipe.  It can be useful to avoid deadlocks in bidirectional
pipes.

> I'm curious now as to which bit you might be refering to here :-) IIRC
> the only place I had any interaction with coroutines was the migration
> code QEMUFile implementation, where I need to yield when reads block.

I think it was NBD.

Paolo

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework
  2015-09-25 10:18     ` Paolo Bonzini
@ 2015-09-25 10:20       ` Daniel P. Berrange
  0 siblings, 0 replies; 32+ messages in thread
From: Daniel P. Berrange @ 2015-09-25 10:20 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: Gerd Hoffmann, Dr. David Alan Gilbert, qemu-devel

On Fri, Sep 25, 2015 at 12:18:05PM +0200, Paolo Bonzini wrote:
> 
> 
> On 24/09/2015 16:51, Daniel P. Berrange wrote:
> > Thanks, I'd still love one or more people to do a detailed review of
> > the I/O channel implementations before I think about sending a PULL
> > request, since there's a fair amount of code in there....
> 
> I did a review of the file, socket, command and buffer code.  I made
> another small suggestion regarding error checking.
> 
> The command code could add support for shutdown by closing the parent's
> end of the pipe.  It can be useful to avoid deadlocks in bidirectional
> pipes.

Hmm, yes, good idea.

> > I'm curious now as to which bit you might be refering to here :-) IIRC
> > the only place I had any interaction with coroutines was the migration
> > code QEMUFile implementation, where I need to yield when reads block.
> 
> I think it was NBD.

Ah ok. I did start looking at NBD but got scared by all the AIO context
handling & event loop integration, which is somewhat different from just
being able to register with gmainloop directly. So that's still something
I need to explore....

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

^ permalink raw reply	[flat|nested] 32+ messages in thread

* Re: [Qemu-devel] [PATCH v1 16/16] io: add QIOChannelBuffer class
  2015-09-25 10:15   ` Paolo Bonzini
@ 2015-09-25 10:21     ` Daniel P. Berrange
  0 siblings, 0 replies; 32+ messages in thread
From: Daniel P. Berrange @ 2015-09-25 10:21 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: Gerd Hoffmann, qemu-devel, Dr. David Alan Gilbert

On Fri, Sep 25, 2015 at 12:15:46PM +0200, Paolo Bonzini wrote:
> 
> 
> On 18/09/2015 15:19, Daniel P. Berrange wrote:
> > +    if (fds || nfds) {
> > +        error_setg_errno(errp, EINVAL,
> > +                         "Channel does not support buffer descriptor passing");
> > +        return -1;
> > +    }
> 
> Typo (file descriptor passing) and the check could be hoisted to
> io/channel.c by using features.
> 
> In fact capabilities could perhaps be stored statically in struct
> QIOChannel as a bitmap?

In general capabilities are static per class, but the socket code is an
exception, because whether you can do FD passing is a runtime check
based on the type of socket you opened. I might still be able to hoist
that up into the base class though, so will investigate..

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

^ permalink raw reply	[flat|nested] 32+ messages in thread

end of thread, other threads:[~2015-09-25 10:21 UTC | newest]

Thread overview: 32+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-09-18 13:18 [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework Daniel P. Berrange
2015-09-18 13:18 ` [Qemu-devel] [PATCH v1 01/16] sockets: add helpers for creating SocketAddress from a socket Daniel P. Berrange
2015-09-18 13:18 ` [Qemu-devel] [PATCH v1 02/16] sockets: move qapi_copy_SocketAddress into qemu-sockets.c Daniel P. Berrange
2015-09-18 13:18 ` [Qemu-devel] [PATCH v1 03/16] sockets: allow port to be NULL when listening on IP address Daniel P. Berrange
2015-09-18 13:18 ` [Qemu-devel] [PATCH v1 04/16] ui: convert VNC startup code to use SocketAddress Daniel P. Berrange
2015-09-18 13:18 ` [Qemu-devel] [PATCH v1 05/16] osdep: add qemu_fork() wrapper for safely handling signals Daniel P. Berrange
2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 06/16] coroutine: move into libqemuutil.a library Daniel P. Berrange
2015-09-22 12:07   ` Paolo Bonzini
2015-09-24 14:46     ` Daniel P. Berrange
2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 07/16] io: add abstract QIOChannel classes Daniel P. Berrange
2015-09-22 12:19   ` Paolo Bonzini
2015-09-22 12:24     ` Daniel P. Berrange
2015-09-22 12:28       ` Paolo Bonzini
2015-09-24 14:47         ` Daniel P. Berrange
2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 08/16] io: add helper module for creating watches on FDs Daniel P. Berrange
2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 09/16] io: pull Buffer code out of VNC module Daniel P. Berrange
2015-09-22 12:04   ` Paolo Bonzini
2015-09-22 12:20     ` Daniel P. Berrange
2015-09-22 12:21       ` Paolo Bonzini
2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 10/16] io: add QIOTask class for async operations Daniel P. Berrange
2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 11/16] io: add QIOChannelSocket class Daniel P. Berrange
2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 12/16] io: add QIOChannelFile class Daniel P. Berrange
2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 13/16] io: add QIOChannelTLS class Daniel P. Berrange
2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 14/16] io: add QIOChannelWebsock class Daniel P. Berrange
2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 15/16] io: add QIOChannelCommand class Daniel P. Berrange
2015-09-18 13:19 ` [Qemu-devel] [PATCH v1 16/16] io: add QIOChannelBuffer class Daniel P. Berrange
2015-09-25 10:15   ` Paolo Bonzini
2015-09-25 10:21     ` Daniel P. Berrange
2015-09-22 12:23 ` [Qemu-devel] [PATCH v1 00/16] Introduce I/O channels framework Paolo Bonzini
2015-09-24 14:51   ` Daniel P. Berrange
2015-09-25 10:18     ` Paolo Bonzini
2015-09-25 10:20       ` Daniel P. Berrange

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).