Re: [PATCH] add ssl/tls support to relocation

[Zhigang Wang <zhigang.x.wang@oracle.com>]

[-- Attachment #1: Type: text/plain, Size: 3609 bytes --]

After further investigation, I find that I didn't get relocation using
ssl/tls: the read/write to the pyOpenSSL socket.fileno() will communicate
without data encrypted.

I have add a wrapper to the read/write with the following patch.

This patch also makes pyOpenSSL an optional package.

Note on changing:
    raise XendError("can't connect: %s" % err[1])
to:
    raise XendError("can't connect: %s" % err)
in tools/python/xen/xend/XendDomain.py: 
it will avoid the following error:

[2008-05-08 14:17:28 2678] ERROR (xmlrpclib2:178) Internal error handling
xend.domain.migrate
Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/xen/util/xmlrpclib2.py", line 131, in
_marshaled_dispatch
    response = self._dispatch(method, params)
  File "/usr/lib/python2.4/SimpleXMLRPCServer.py", line 406, in _dispatch
    return func(*params)
  File "/usr/lib/python2.4/site-packages/xen/xend/XendDomain.py", line 1335,
in domain_migrate
    raise XendError("can't connect: %s" % err[1])
IndexError: tuple index out of range

Implement reference: http://twistedmatrix.com/trac/browser/trunk/twisted/internet/tcp.py

Sorry for the careless.

thanks,

zhigang

Carb, Brian A wrote:
> This works on SLES10SP1 if we download a compatible python openssl rpm
> (python-openssl-0.6-17.x86_64.rpm) from opensuse.
> 
> brian carb
> unisys corporation - malvern, pa
> 
> -----Original Message-----
> From: Zhigang Wang [mailto:zhigang.x.wang@oracle.com] 
> Sent: Friday, May 02, 2008 2:11 PM
> To: Carb, Brian A
> Cc: xen-devel
> Subject: Re: [Xen-devel] [PATCH] add ssl/tls support to relocation
> 
> Yes. I add a dependency.
> 
> maybe we can separate SSLTCPListener to a new file like (tcp-ssl.py),
> and import it when needed, just like
> tools/python/xen/xend/server/SSLXMLRPCServer.py did.
> 
> will try it later, or you can help.
> 
> thanks,
> 
> zhigang
> 
> Carb, Brian A wrote:
>> With this patch included, xend start errors: 
>>
>> # xend start
>> Traceback (most recent call last):
>>   File "/usr/sbin/xend", line 44, in ?
>>     from xen.xend.server import SrvDaemon
>>   File
>> "/home/unisys/xen-unstable.hg/dist/install/usr/lib64/python/xen/xend/s
>> er
>> ver/SrvDaemon.py", line 26, in ?
>>     import relocate
>>   File
>> "/home/unisys/xen-unstable.hg/dist/install/usr/lib64/python/xen/xend/s
>> er
>> ver/relocate.py", line 23, in ?
>>     from xen.web import protocol, tcp, unix
>>   File
>>
> "/home/unisys/xen-unstable.hg/dist/install/usr/lib64/python/xen/web/tcp.
>> py", line 25, in ?
>>     from OpenSSL import SSL
>> ImportError: No module named OpenSSL
>>
>> I guess this produces a dependency on python-openssl, but this rpm is 
>> not included in the standard SLES10 distro.
>>
>> brian carb
>> unisys corporation - malvern, pa
>>
>>
>> -----Original Message-----
>> From: xen-devel-bounces@lists.xensource.com
>> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Zhigang 
>> Wang
>> Sent: Sunday, April 27, 2008 10:50 PM
>> To: xen-devel
>> Subject: [Xen-devel] [PATCH] add ssl/tls support to relocation
>>
>> hi, this patch add ssl/tls support to relocation:
>>
>>  * SSL/TLS support is disabled by default, as other server did.
>>
>>  * If "xend-relocation-server-ssl-key-file" and
>>    "xend-relocation-server-ssl-cert-file" exist, SSL/TLS is enabled
>>    automatically.
>>
>>  * "xend-relocation-tls" is used by relocation client only.
>>
>> Signed-off-by: Zhigang Wang <zhigang.x.wang@oracle.com>
>>
>>
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

[-- Attachment #2: xen-unstable-new.patch --]
[-- Type: text/x-patch, Size: 18713 bytes --]

Fix relocation ssl/tls support

 * Make a wrapper of read/write sock.fileno().

 * Makes pyOpenSSL an optional package.

 * Implement reference:
   http://twistedmatrix.com/trac/browser/trunk/twisted/internet/tcp.py

Signed-off-by: Zhigang Wang <zhigang.x.wang@oracle.com>

diff -Nura xen-unstable.orig/tools/python/xen/web/connectionssl.py xen-unstable/tools/python/xen/web/connectionssl.py
--- xen-unstable.orig/tools/python/xen/web/connectionssl.py	1970-01-01 08:00:00.000000000 +0800
+++ xen-unstable/tools/python/xen/web/connectionssl.py	2008-05-08 15:39:44.000000000 +0800
@@ -0,0 +1,188 @@
+#============================================================================
+# This library is free software; you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License as published by
+# the Free Software Foundation; either version 2.1 of the License, or
+# (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this library; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+#============================================================================
+# Copyright (C) 2005 Mike Wray <mike.wray@hp.com>
+# Copyright (C) 2005 XenSource Ltd.
+# Copyright (C) 2008 Oracle
+#============================================================================
+import os
+import socket
+
+from errno import EAGAIN, EINTR, EWOULDBLOCK
+from OpenSSL import SSL
+
+from connection import SocketServerConnection
+from xen.xend.XendLogging import log
+
+"""General classes to support server and client sockets, without
+specifying what kind of socket they are. There are subclasses
+for TCP and unix-domain sockets (see tcp.py and unix.py).
+
+As pyOpenSSL SSL.Connection fileno() method just retrieve the file descriptor
+number for the underlying socket, direct read/write to the file descriptor
+will result no data encrypted.
+
+recv2fd() and fd2send() are simple wrappers for functions who need direct
+read/write to a file descriptor rather than a socket like object.
+
+To use recv2fd(), you can create a pipe and start a thread to transfer all
+received data to one end of the pipe, then read from the other end:
+
+p2cread, p2cwrite = os.pipe()
+threading.Thread(target=connectionssl.recv2fd, args=(sock, p2cwrite)).start()
+os.read(p2cread, 1024)
+
+To use fd2send():
+
+p2cread, p2cwrite = os.pipe()
+threading.Thread(target=connectionssl.fd2send, args=(sock, p2cread)).start()
+os.write(p2cwrite, "data")
+
+"""
+
+BUFFER_SIZE = 1024
+
+
+class SSLSocketServerConnection(SocketServerConnection):
+    """An SSL aware accepted connection to a server.
+    """
+
+    def __init__(self, sock, protocol_class):
+        SocketServerConnection.__init__(self, sock, protocol_class)
+
+    def main(self):
+        try:
+            while True:
+                try:
+                    data = self.sock.recv(BUFFER_SIZE)
+                    if data == "":
+                        break
+                    if self.protocol.dataReceived(data):
+                        break
+                except socket.error, ex:
+                    if ex.args[0] not in (EWOULDBLOCK, EAGAIN, EINTR):
+                        break
+                except (SSL.WantReadError, SSL.WantWriteError, \
+                        SSL.WantX509LookupError):
+                    # The operation did not complete; the same I/O method
+                    # should be called again.
+                    continue
+                except SSL.ZeroReturnError:
+                    # The SSL Connection has been closed.
+                    break
+                except SSL.SysCallError, (retval, desc):
+                    if ((retval == -1 and desc == "Unexpected EOF")
+                        or retval > 0):
+                        # The SSL Connection is lost.
+                        break
+                    log.debug("SSL SysCallError:%d:%s" % (retval, desc))
+                    break
+                except SSL.Error, e:
+                    # other SSL errors
+                    log.debug("SSL Error:%s" % e)
+                    break
+        finally:
+            try:
+                self.sock.close()
+            except:
+                pass
+
+def recv2fd(sock, fd):
+    try:
+        while True:
+            try:
+                data = sock.recv(BUFFER_SIZE)
+                if data == "":
+                    break
+                count = 0
+                while count < len(data):
+                    try:
+                        nbytes = os.write(fd, data[count:])
+                        count += nbytes
+                    except os.error, ex:
+                        if ex.args[0] not in (EWOULDBLOCK, EAGAIN, EINTR):
+                            raise
+            except socket.error, ex:
+                if ex.args[0] not in (EWOULDBLOCK, EAGAIN, EINTR):
+                    break
+            except (SSL.WantReadError, SSL.WantWriteError, \
+                    SSL.WantX509LookupError):
+                # The operation did not complete; the same I/O method
+                # should be called again.
+                continue
+            except SSL.ZeroReturnError:
+                # The SSL Connection has been closed.
+                break
+            except SSL.SysCallError, (retval, desc):
+                if ((retval == -1 and desc == "Unexpected EOF")
+                    or retval > 0):
+                    # The SSL Connection is lost.
+                    break
+                log.debug("SSL SysCallError:%d:%s" % (retval, desc))
+                break
+            except SSL.Error, e:
+                # other SSL errors
+                log.debug("SSL Error:%s" % e)
+                break
+    finally:
+        try:
+            sock.close()
+            os.close(fd)
+        except:
+            pass
+
+def fd2send(sock, fd):
+    try:
+        while True:
+            try:
+                data = os.read(fd, BUFFER_SIZE)
+                if data == "":
+                    break
+                count = 0
+                while count < len(data):
+                    try:
+                        nbytes = sock.send(data[count:])
+                        count += nbytes
+                    except socket.error, ex:
+                        if ex.args[0] not in (EWOULDBLOCK, EAGAIN, EINTR):
+                            raise
+                    except (SSL.WantReadError, SSL.WantWriteError, \
+                            SSL.WantX509LookupError):
+                        # The operation did not complete; the same I/O method
+                        # should be called again.
+                        continue
+                    except SSL.ZeroReturnError:
+                        # The SSL Connection has been closed.
+                        raise
+                    except SSL.SysCallError, (retval, desc):
+                        if not (e[0] == -1 and data == ""):
+                            # errors when writing empty strings are expected
+                            # and can be ignored
+                            log.debug("SSL SysCallError:%d:%s" % (retval, desc))
+                            raise
+                    except SSL.Error, e:
+                        # other SSL errors
+                        log.debug("SSL Error:%s" % e)
+                        raise
+            except os.error, ex:
+                if ex.args[0] not in (EWOULDBLOCK, EAGAIN, EINTR):
+                    break
+    finally:
+        try:
+            sock.close()
+            os.close(fd)
+        except:
+            pass
+
diff -Nura xen-unstable.orig/tools/python/xen/web/tcp.py xen-unstable/tools/python/xen/web/tcp.py
--- xen-unstable.orig/tools/python/xen/web/tcp.py	2008-05-05 14:17:45.000000000 +0800
+++ xen-unstable/tools/python/xen/web/tcp.py	2008-05-05 14:32:16.000000000 +0800
@@ -22,8 +22,6 @@
 import socket
 import time
 
-from OpenSSL import SSL
-
 import connection
 
 from xen.xend.XendLogging import log
@@ -67,41 +65,3 @@
             except:
                 pass
 
-class SSLTCPListener(TCPListener):
-
-    def __init__(self, protocol_class, port, interface, hosts_allow,
-                 ssl_key_file = None, ssl_cert_file = None):
-        if not ssl_key_file or not ssl_cert_file:
-            raise ValueError("SSLXMLRPCServer requires ssl_key_file "
-                             "and ssl_cert_file to be set.")
-
-        self.ssl_key_file = ssl_key_file
-        self.ssl_cert_file = ssl_cert_file
-
-        TCPListener.__init__(self, protocol_class, port, interface, hosts_allow)
-
-
-    def createSocket(self):
-        # make a SSL socket
-        ctx = SSL.Context(SSL.SSLv23_METHOD)
-        ctx.set_options(SSL.OP_NO_SSLv2)
-        ctx.use_privatekey_file (self.ssl_key_file)
-        ctx.use_certificate_file(self.ssl_cert_file)
-        sock = SSL.Connection(ctx,
-                              socket.socket(socket.AF_INET, socket.SOCK_STREAM))
-        sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
-
-        # SO_REUSEADDR does not always ensure that we do not get an address
-        # in use error when restarted quickly
-        # we implement a timeout to try and avoid failing unnecessarily
-        timeout = time.time() + 30
-        while True:
-            try:
-                sock.bind((self.interface, self.port))
-                return sock
-            except socket.error, (_errno, strerrno):
-                if _errno == errno.EADDRINUSE and time.time() < timeout:
-                    time.sleep(0.5)
-                else:
-                    raise
-
diff -Nura xen-unstable.orig/tools/python/xen/web/tcpssl.py xen-unstable/tools/python/xen/web/tcpssl.py
--- xen-unstable.orig/tools/python/xen/web/tcpssl.py	1970-01-01 08:00:00.000000000 +0800
+++ xen-unstable/tools/python/xen/web/tcpssl.py	2008-05-08 15:38:54.000000000 +0800
@@ -0,0 +1,82 @@
+#============================================================================
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of version 2.1 of the GNU Lesser General Public
+# License as published by the Free Software Foundation.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+#============================================================================
+# Copyright (C) 2005 Mike Wray <mike.wray@hp.com>
+# Copyright (C) 2005 XenSource Ltd.
+# Copyright (C) 2008 Oracle
+#============================================================================
+
+
+import errno
+import socket
+import time
+
+import connection
+import connectionssl
+
+from OpenSSL import SSL
+from tcp import TCPListener
+from xen.xend.XendLogging import log
+
+
+class SSLTCPListener(TCPListener):
+
+    def __init__(self, protocol_class, port, interface, hosts_allow,
+                 ssl_key_file = None, ssl_cert_file = None):
+        if not ssl_key_file or not ssl_cert_file:
+            raise ValueError("SSLXMLRPCServer requires ssl_key_file "
+                             "and ssl_cert_file to be set.")
+
+        self.ssl_key_file = ssl_key_file
+        self.ssl_cert_file = ssl_cert_file
+
+        TCPListener.__init__(self, protocol_class, port, interface, hosts_allow)
+
+
+    def createSocket(self):
+        # make a SSL socket
+        ctx = SSL.Context(SSL.SSLv23_METHOD)
+        ctx.set_options(SSL.OP_NO_SSLv2)
+        ctx.use_privatekey_file (self.ssl_key_file)
+        ctx.use_certificate_file(self.ssl_cert_file)
+        sock = SSL.Connection(ctx,
+                              socket.socket(socket.AF_INET, socket.SOCK_STREAM))
+        sock.set_accept_state()
+        sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+
+        # SO_REUSEADDR does not always ensure that we do not get an address
+        # in use error when restarted quickly
+        # we implement a timeout to try and avoid failing unnecessarily
+        timeout = time.time() + 30
+        while True:
+            try:
+                sock.bind((self.interface, self.port))
+                return sock
+            except socket.error, (_errno, strerrno):
+                if _errno == errno.EADDRINUSE and time.time() < timeout:
+                    time.sleep(0.5)
+                else:
+                    raise
+
+
+    def acceptConnection(self, sock, addrport):
+        addr = addrport[0]
+        if connection.hostAllowed(addrport, self.hosts_allow):
+            connectionssl.SSLSocketServerConnection(sock, self.protocol_class)
+        else:
+            try:
+                sock.close()
+            except:
+                pass
+
diff -Nura xen-unstable.orig/tools/python/xen/xend/server/relocate.py xen-unstable/tools/python/xen/xend/server/relocate.py
--- xen-unstable.orig/tools/python/xen/xend/server/relocate.py	2008-05-05 14:17:46.000000000 +0800
+++ xen-unstable/tools/python/xen/xend/server/relocate.py	2008-05-07 16:39:00.000000000 +0800
@@ -17,10 +17,18 @@
 #============================================================================
 
 import re
+import os
 import sys
 import StringIO
+import threading
 
 from xen.web import protocol, tcp, unix
+try:
+    from xen.web import tcpssl
+    from xen.web import connectionssl
+    ssl_enabled = True
+except ImportError:
+    ssl_enabled = False
 
 from xen.xend import sxp
 from xen.xend import XendDomain
@@ -116,6 +124,27 @@
             log.error(name + ": no transport")
             raise XendError(name + ": no transport")
 
+    def op_sslreceive(self, name, _):
+        if self.transport:
+            self.send_reply(["ready", name])
+            p2cread, p2cwrite = os.pipe()
+            if not ssl_enabled:
+                raise ValueError("pyOpenSSL not installed. "
+                                 "Unable to start relocation with ssl")
+            threading.Thread(target=connectionssl.recv2fd,
+                             args=(self.transport.sock, p2cwrite)).start()
+            try:
+                XendDomain.instance().domain_restore_fd(p2cread,
+                                                        relocating=True)
+            except:
+                os.close(p2cread)
+                os.close(p2cwrite)
+                self.send_error()
+                self.close()
+        else:
+            log.error(name + ": no transport")
+            raise XendError(name + ": no transport")
+
 
 def listenRelocation():
     xoptions = XendOptions.instance()
@@ -136,10 +165,13 @@
         ssl_cert_file = xoptions.get_xend_relocation_server_ssl_cert_file()
 
         if ssl_key_file and ssl_cert_file:
-            tcp.SSLTCPListener(RelocationProtocol, port, interface = interface,
-                               hosts_allow = hosts_allow,
-                               ssl_key_file = ssl_key_file,
-                               ssl_cert_file = ssl_cert_file)
+            if not ssl_enabled:
+                raise ValueError("pyOpenSSL not installed. "
+                                 "Unable to start relocation server with ssl")
+            tcpssl.SSLTCPListener(RelocationProtocol, port, interface = interface,
+                                  hosts_allow = hosts_allow,
+                                  ssl_key_file = ssl_key_file,
+                                  ssl_cert_file = ssl_cert_file)
         else:
             tcp.TCPListener(RelocationProtocol, port, interface = interface,
                             hosts_allow = hosts_allow)
diff -Nura xen-unstable.orig/tools/python/xen/xend/XendDomain.py xen-unstable/tools/python/xen/xend/XendDomain.py
--- xen-unstable.orig/tools/python/xen/xend/XendDomain.py	2008-05-05 14:17:45.000000000 +0800
+++ xen-unstable/tools/python/xen/xend/XendDomain.py	2008-05-08 14:56:54.000000000 +0800
@@ -1297,19 +1297,39 @@
             tls = xoptions.get_xend_relocation_tls()
             if tls:
                 from OpenSSL import SSL
+                from xen.web import connectionssl
                 ctx = SSL.Context(SSL.SSLv23_METHOD)
-                sock = SSL.Connection(ctx, socket.socket(socket.AF_INET, socket.SOCK_STREAM))
+                sock = SSL.Connection(ctx,
+                           socket.socket(socket.AF_INET, socket.SOCK_STREAM))
                 sock.set_connect_state()
+                sock.connect((dst, port))
+                sock.send("sslreceive\n")
+                sock.recv(80)
+                p2cread, p2cwrite = os.pipe()
+                threading.Thread(target=connectionssl.fd2send,
+                                 args=(sock, p2cread)).start()
+                XendCheckpoint.save(p2cwrite, dominfo, True, live, dst,
+                                    node=node)
+                os.close(p2cread)
+                os.close(p2cwrite)
+                sock.shutdown()
+                sock.close()
             else:
                 sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-            sock.connect((dst, port))
+                # When connecting our ssl enabled relocation server using a
+                # plain socket, send will success but recv will block. Add a
+                # 30 seconds timeout to raise a socket.timeout exception to
+                # inform the client.
+                sock.settimeout(30.0)
+                sock.connect((dst, port))
+                sock.send("receive\n")
+                sock.recv(80)
+                sock.settimeout(None)
+                XendCheckpoint.save(sock.fileno(), dominfo, True, live, dst,
+                                    node=node)
+                sock.close()
         except socket.error, err:
-            raise XendError("can't connect: %s" % err[1])
-
-        sock.send("receive\n")
-        sock.recv(80)
-        XendCheckpoint.save(sock.fileno(), dominfo, True, live, dst, node=node)
-        sock.close()
+            raise XendError("can't connect: %s" % err)
 
     def domain_save(self, domid, dst, checkpoint=False):
         """Start saving a domain to file.

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel