qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
* [Qemu-devel] [PATCH v1 RFC 00/34] Generic support for TLS protocol & I/O channels
@ 2015-04-17 14:22 Daniel P. Berrange
  2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 01/34] ui: remove check for failure of qemu_acl_init() Daniel P. Berrange
                   ` (34 more replies)
  0 siblings, 35 replies; 71+ messages in thread
From: Daniel P. Berrange @ 2015-04-17 14:22 UTC (permalink / raw)
  To: qemu-devel; +Cc: Paolo Bonzini, Gerd Hoffmann, Stefan Hajnoczi

The following series is a work-in-progress of my effort to bring TLS
encryption support to all the QEMU socket based backends. The current
patches just illustrate changes to the chardev backends, but future
patches will cover NBD client/server and the migration client/server
code. The original discussion was here:

https://lists.gnu.org/archive/html/qemu-devel/2015-02/msg00529.html

The bulk of the interesting work here has focused on pulling our the
TLS code from the VNC server and turning it into a self-contained,
reusable API. This has been accompanied with a general consolidation
of all cryptographic related code. So we have one place in the code
base for dealing with cryptographic hash functions (md5, sha, etc),
ciphers (DES, AES, etc), and protocols (TLS). The benefit of this
is that the rest of the code base doesn't have to be littered with
#ifdef HAVE_GNUTLS conditionals - all the integration code for gnutls
is in one place.

Building on that, the next step has been to define a generic I/O
channels API (inspired by GIOChannel, but improved to better suit
QEMU's requirements). This provides a higher level API for dealing
with POSIX sockets, and running protocols such as TLS, SASL, and
WebSockets over them. This again allowed the VNC code to be further
untangled, so it doesn't have to directly know about TLS or WebSockets
for the most part. This will allow us to add WebSockets support to
the chardev backends too at some point.

The patches series is sequenced into a number of logical groupings,
with a view to allowing the patches to be incrementally merged,
rather than having to take the entire series in one go.

 - QOM - a handful of minor enhancements & fixes to QOM, in
   particular some work to make handling of enum properties
   clearer, and to make instantiation of objects with properties
   simpler.

 - Crypto - introduce crypto/ and include/crypto/ directories
   that contain APIs for hash functions, cipher functions and
   the TLS credential (x509 cert) handling and TLS protocol
   itself. This will be further enhanced down the line as &
   when I get time to integrate support for LUKS.

 - Hash conversion - the quorum blockdrv and VNC server are
   converted to use the new crypto hash functions. This removes
   some of the #ifdef CONFIG_* conditionals from their codepaths

 - Cipher conversion - the qcow(2) blockdrv and VNC server are
   converted to use the new crypto cipher functions. This removes
   their direct dependency on QEMU's in-tree AES and DES impls. If
   QEMU is linked to GNUTLS, this will now transparently use either
   libgcrypt or nettle for the AES & DES algorithms. These impls are
   more actively maintained than QEMU's built-in impls and also
   benefit from FIPS certification in some distros. The built-in
   impls of course remain for those building without GNUTLS.

 - I/O Channels - introduce io/ and include/io/ directories that
   contain APIs for dealing with arbitrary bidirectional I/O channels.
   These can be POSIX sockets, fifos, pipes, or higher level layers
   such as TLS or WebSockets. Having a common API for all these
   concepts greatly facilitates the integration of TLS/WebSockets into
   the various areas of code in QEMU that are traditionally hardcoded
   to directly use the POSIX sockets API.

 - VNC conversion - convert the VNC server to use the new I/O channels
   APIs. This enables nearly all the remaining #ifdef CONFIG_*
   conditionals to be removed from the VNC codebase, greatly
   clarifying its code.

 - Chardev conversion - convert the chardev backend to use the new I/O
   channels APIs instead of GLib's GIOChannel. This also includes
   support for enabling TLS on the TCP chardev backend, nicely
   illustrating how the I/O channels APIs simplify support for such
   protocols.

I've attempted to get fairly complete API documentation coverage for
all the new code files I've created here, with example code usage too.
In addition, the crypto modules gain a number of new test suites to
validate correctness of the implementations, since this is critical
code to get right.

What I see still to be done (high == merge block, medium == nice to
have for merge, low == do it later):

 - The code is not fully ported to the Windows platform yet. In
   particular the QIOChannelSocket and QIOChannelFile classes are
   almost certainly broken & if they compile it is just luck. High
   priority.

 - Need to validate the reference counting / lifecycle of the
   QIOChannel objects in chardev & vnc servers to ensure I've not
   introduced any race conditions in their usage vs client disconnects.
   High priority.

 - Unit test coverage of all the QIOChannel subclasses to validate
   their correct operation. Medium priority

 - A QIOChannelMemory implementation that provides a memory buffer
   backed I/O channel. Mostly this will be used for the unit test
   suite, but might find other uses at some point. Medium priority.

 - APIs for establishing socket connections. Currently the
   QIOChannelSocket class is instantiated from a pre-connected socket
   file descriptor. It is desirable to have a constructor that just
   accepts a hostname/service/family and then performs the name
   resolution & connection code. This will make the API more consistent
   to use. Medium priority.

 - A QIOChannelSASL implementation that provides integration for the
   SASL authentication protocol. This will allow the last custom I/O
   layer to be removed from the VNC server code. Low priority, since
   we don't immediately want/need SASL support in chardev/migraton/nbd
   code.

 - A QIOChannelTelnet implementation that runs the telnet protocol, to
   replace the hacky telnet support that is hardwired into the chardev
   backends. Low priority.

 - The crypto API could usefully gain a cipher backend that uses the
   Linux kernel crypto API as an alternative to nettle or libgcrypt.
   Low priority, nice to have.

While I will probably start work on it, I'm not intending to submit
the update of the NBD/migration code, until this series has been
positively reviewed and looks like it is close to accepted for merge,
as there are already enough patches in this series as it is :-) The
aim though is to convert the NBD code to use QIOChannel instead of
direct sockets usage & add the TLS protocol extension previously
discussed with the NBD spec maintainer. The migration code will
either be adapted to use QIOChannel, or the QEMUFile code will be
adapted to use QIOChannel. Undecided which is the best approach there
at this time. Probably depends whether we can do a QIOChannelRDMA
impl that has performance on parity with what exists today.

For those interesting in testing I have made the series available
on github too

  https://github.com/berrange/qemu/tree/qemu-io-channel-7

The diffstat may look alarming but a good portion is in the test
suite and there's some quite verbose comments inline too which
bulk it up:

Daniel P. Berrange (34):
  ui: remove check for failure of qemu_acl_init()
  qom: document user creatable object types in help text
  qom: create objects in two phases
  qom: add object_new_propv / object_new_proplist constructors
  qom: make enum string tables const-correct
  qom: add a object_property_add_enum helper method
  qom: don't pass string table to object_get_enum method
  crypto: introduce new module for computing hash digests
  crypto: move built-in AES implementation into crypto/
  crypto: move built-in D3DES implementation into crypto/
  crypto: introduce generic cipher API & built-in implementation
  crypto: add a gcrypt cipher implementation
  crypto: add a nettle cipher implementation
  crypto: introduce new module for handling TLS credentials
  crypto: add sanity checking of TLS credentials
  crypto: introduce new module for handling TLS sessions
  block: convert quorum blockdrv to use crypto APIs
  ui: convert VNC websockets to use crypto APIs
  block: convert qcow/qcow2 to use generic cipher API
  ui: convert VNC to use generic cipher API
  io: add abstract QIOChannel classes
  io: add helper module for creating watches on UNIX FDs
  io: add QIOChannelSocket class
  io: add QIOChannelFile class
  io: add QIOTask class for async operations
  io: add QIOChannelTLS class
  io: pull Buffer code out of VNC module
  io: add QIOChannelWebsock class
  ui: convert VNC server to use QEMUIOChannelSocket classes
  ui: convert VNC server to use QIOChannelTLS
  ui: convert VNC server to use QIOChannelWebsock
  char: convert from GIOChannel to QIOChannel
  char: don't assume telnet initialization will not block
  char: introduce support for TLS encrypted TCP chardev backend

 Makefile.objs                         |    1 +
 backends/hostmem.c                    |   22 +-
 block/Makefile.objs                   |    2 +-
 block/qcow.c                          |  100 ++-
 block/qcow2-cluster.c                 |   46 +-
 block/qcow2.c                         |   95 +--
 block/qcow2.h                         |   13 +-
 block/quorum.c                        |   38 +-
 configure                             |  213 ++++---
 crypto/Makefile.objs                  |    7 +
 {util => crypto}/aes.c                |    2 +-
 crypto/cipher-builtin.c               |  391 ++++++++++++
 crypto/cipher-gcrypt.c                |  204 ++++++
 crypto/cipher-nettle.c                |  226 +++++++
 crypto/cipher.c                       |   31 +
 ui/d3des.c => crypto/desrfb.c         |    2 +-
 crypto/hash.c                         |  202 ++++++
 crypto/init.c                         |  160 +++++
 crypto/tlscreds.c                     | 1093 ++++++++++++++++++++++++++++++++
 crypto/tlssession.c                   |  546 ++++++++++++++++
 include/{qemu => crypto}/aes.h        |    0
 include/crypto/cipher.h               |  205 ++++++
 ui/d3des.h => include/crypto/desrfb.h |    0
 include/crypto/hash.h                 |  189 ++++++
 include/crypto/init.h                 |   29 +
 include/crypto/tlscreds.h             |  135 ++++
 include/crypto/tlssession.h           |  313 ++++++++++
 include/hw/qdev-core.h                |    2 +-
 include/io/buffer.h                   |  118 ++++
 include/io/channel-file.h             |   67 ++
 include/io/channel-socket.h           |  168 +++++
 include/io/channel-tls.h              |  142 +++++
 include/io/channel-unix.h             |   50 ++
 include/io/channel-websock.h          |  108 ++++
 include/io/channel.h                  |  388 ++++++++++++
 include/io/task.h                     |  168 +++++
 include/qapi/util.h                   |    2 +-
 include/qapi/visitor-impl.h           |    6 +-
 include/qapi/visitor.h                |    2 +-
 include/qom/object.h                  |   78 ++-
 io/Makefile.objs                      |    8 +
 io/buffer.c                           |   65 ++
 io/channel-file.c                     |  198 ++++++
 io/channel-socket.c                   |  572 +++++++++++++++++
 io/channel-tls.c                      |  393 ++++++++++++
 io/channel-unix.c                     |  100 +++
 io/channel-websock.c                  |  976 +++++++++++++++++++++++++++++
 io/channel.c                          |  178 ++++++
 io/task.c                             |   84 +++
 numa.c                                |    1 -
 qapi-schema.json                      |    2 +
 qapi/qapi-dealloc-visitor.c           |    3 +-
 qapi/qapi-util.c                      |    2 +-
 qapi/qapi-visit-core.c                |    6 +-
 qemu-char.c                           |  798 ++++++++++++------------
 qemu-options.hx                       |  137 +++-
 qom/object.c                          |  141 ++++-
 scripts/qapi-types.py                 |    4 +-
 target-arm/crypto_helper.c            |    2 +-
 target-i386/fpu_helper.c              |    1 -
 target-i386/ops_sse.h                 |    2 +-
 target-ppc/int_helper.c               |    2 +-
 tests/.gitignore                      |    9 +
 tests/Makefile                        |   16 +-
 tests/crypto-tls-helpers.c            |  485 +++++++++++++++
 tests/crypto-tls-helpers.h            |  133 ++++
 tests/pkix_asn1_tab.c                 | 1103 +++++++++++++++++++++++++++++++++
 tests/test-crypto-cipher.c            |  290 +++++++++
 tests/test-crypto-hash.c              |  209 +++++++
 tests/test-crypto-tlscreds.c          |  727 ++++++++++++++++++++++
 tests/test-crypto-tlssession.c        |  540 ++++++++++++++++
 ui/Makefile.objs                      |    6 +-
 ui/vnc-auth-sasl.c                    |   81 +--
 ui/vnc-auth-vencrypt.c                |   90 ++-
 ui/vnc-enc-tight.c                    |   38 +-
 ui/vnc-enc-zlib.c                     |    6 +-
 ui/vnc-enc-zrle.c                     |   18 +-
 ui/vnc-jobs.c                         |   25 +-
 ui/vnc-tls.c                          |   24 +-
 ui/vnc-tls.h                          |   69 ---
 ui/vnc-ws.c                           |  393 ++----------
 ui/vnc-ws.h                           |   75 +--
 ui/vnc.c                              |  997 ++++++++++++++---------------
 ui/vnc.h                              |  104 ++--
 util/Makefile.objs                    |    2 +-
 vl.c                                  |   37 +-
 86 files changed, 12868 insertions(+), 1848 deletions(-)
 create mode 100644 crypto/Makefile.objs
 rename {util => crypto}/aes.c (99%)
 create mode 100644 crypto/cipher-builtin.c
 create mode 100644 crypto/cipher-gcrypt.c
 create mode 100644 crypto/cipher-nettle.c
 create mode 100644 crypto/cipher.c
 rename ui/d3des.c => crypto/desrfb.c (99%)
 create mode 100644 crypto/hash.c
 create mode 100644 crypto/init.c
 create mode 100644 crypto/tlscreds.c
 create mode 100644 crypto/tlssession.c
 rename include/{qemu => crypto}/aes.h (100%)
 create mode 100644 include/crypto/cipher.h
 rename ui/d3des.h => include/crypto/desrfb.h (100%)
 create mode 100644 include/crypto/hash.h
 create mode 100644 include/crypto/init.h
 create mode 100644 include/crypto/tlscreds.h
 create mode 100644 include/crypto/tlssession.h
 create mode 100644 include/io/buffer.h
 create mode 100644 include/io/channel-file.h
 create mode 100644 include/io/channel-socket.h
 create mode 100644 include/io/channel-tls.h
 create mode 100644 include/io/channel-unix.h
 create mode 100644 include/io/channel-websock.h
 create mode 100644 include/io/channel.h
 create mode 100644 include/io/task.h
 create mode 100644 io/Makefile.objs
 create mode 100644 io/buffer.c
 create mode 100644 io/channel-file.c
 create mode 100644 io/channel-socket.c
 create mode 100644 io/channel-tls.c
 create mode 100644 io/channel-unix.c
 create mode 100644 io/channel-websock.c
 create mode 100644 io/channel.c
 create mode 100644 io/task.c
 create mode 100644 tests/crypto-tls-helpers.c
 create mode 100644 tests/crypto-tls-helpers.h
 create mode 100644 tests/pkix_asn1_tab.c
 create mode 100644 tests/test-crypto-cipher.c
 create mode 100644 tests/test-crypto-hash.c
 create mode 100644 tests/test-crypto-tlscreds.c
 create mode 100644 tests/test-crypto-tlssession.c
 delete mode 100644 ui/vnc-tls.h

-- 
2.1.0

^ permalink raw reply	[flat|nested] 71+ messages in thread

end of thread, other threads:[~2015-05-13 17:21 UTC | newest]

Thread overview: 71+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-04-17 14:22 [Qemu-devel] [PATCH v1 RFC 00/34] Generic support for TLS protocol & I/O channels Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 01/34] ui: remove check for failure of qemu_acl_init() Daniel P. Berrange
2015-04-17 15:56   ` Eric Blake
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 02/34] qom: document user creatable object types in help text Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 03/34] qom: create objects in two phases Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 04/34] qom: add object_new_propv / object_new_proplist constructors Daniel P. Berrange
2015-04-17 14:55   ` Paolo Bonzini
2015-04-17 15:16     ` Daniel P. Berrange
2015-04-17 16:11   ` Eric Blake
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 05/34] qom: make enum string tables const-correct Daniel P. Berrange
2015-04-17 14:56   ` Paolo Bonzini
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 06/34] qom: add a object_property_add_enum helper method Daniel P. Berrange
2015-04-17 14:56   ` Paolo Bonzini
2015-04-17 15:01     ` Paolo Bonzini
2015-04-17 15:11       ` Daniel P. Berrange
2015-04-17 15:19         ` Paolo Bonzini
2015-04-17 15:22           ` Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 07/34] qom: don't pass string table to object_get_enum method Daniel P. Berrange
2015-04-17 15:05   ` Paolo Bonzini
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 08/34] crypto: introduce new module for computing hash digests Daniel P. Berrange
2015-05-13 17:04   ` Daniel P. Berrange
2015-05-13 17:12     ` Paolo Bonzini
2015-05-13 17:21       ` Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 09/34] crypto: move built-in AES implementation into crypto/ Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 10/34] crypto: move built-in D3DES " Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 11/34] crypto: introduce generic cipher API & built-in implementation Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 12/34] crypto: add a gcrypt cipher implementation Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 13/34] crypto: add a nettle " Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 14/34] crypto: introduce new module for handling TLS credentials Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 15/34] crypto: add sanity checking of " Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 16/34] crypto: introduce new module for handling TLS sessions Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 17/34] block: convert quorum blockdrv to use crypto APIs Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 18/34] ui: convert VNC websockets " Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 19/34] block: convert qcow/qcow2 to use generic cipher API Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 20/34] ui: convert VNC " Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 21/34] io: add abstract QIOChannel classes Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 22/34] io: add helper module for creating watches on UNIX FDs Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 23/34] io: add QIOChannelSocket class Daniel P. Berrange
2015-04-17 15:28   ` Paolo Bonzini
2015-04-17 15:52     ` Daniel P. Berrange
2015-04-17 16:00       ` Paolo Bonzini
2015-04-20  7:18   ` Gerd Hoffmann
2015-04-23 12:31     ` Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 24/34] io: add QIOChannelFile class Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 25/34] io: add QIOTask class for async operations Daniel P. Berrange
2015-04-17 15:16   ` Paolo Bonzini
2015-04-17 15:49     ` Daniel P. Berrange
2015-04-17 15:57       ` Paolo Bonzini
2015-04-17 16:11         ` Daniel P. Berrange
2015-04-17 17:06           ` Paolo Bonzini
2015-04-17 17:38             ` Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 26/34] io: add QIOChannelTLS class Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 27/34] io: pull Buffer code out of VNC module Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 28/34] io: add QIOChannelWebsock class Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 29/34] ui: convert VNC server to use QEMUIOChannelSocket classes Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 30/34] ui: convert VNC server to use QIOChannelTLS Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 31/34] ui: convert VNC server to use QIOChannelWebsock Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 32/34] char: convert from GIOChannel to QIOChannel Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 33/34] char: don't assume telnet initialization will not block Daniel P. Berrange
2015-04-17 14:22 ` [Qemu-devel] [PATCH v1 RFC 34/34] char: introduce support for TLS encrypted TCP chardev backend Daniel P. Berrange
2015-04-17 18:27   ` Eric Blake
2015-04-23 12:32     ` Daniel P. Berrange
2015-05-04 20:07   ` Kashyap Chamarthy
2015-05-05 13:49     ` Daniel P. Berrange
2015-05-05 13:53       ` Paolo Bonzini
2015-05-05 13:56         ` Daniel P. Berrange
2015-05-05 14:54       ` Kashyap Chamarthy
2015-05-06  8:34         ` Kashyap Chamarthy
2015-05-06 10:18           ` Daniel P. Berrange
2015-05-06 11:38             ` Kashyap Chamarthy
2015-04-23 12:28 ` [Qemu-devel] [PATCH v1 RFC 00/34] Generic support for TLS protocol & I/O channels Stefan Hajnoczi

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).