[RFC][PATCH] Secure XML-RPC for Xend

[RFC][PATCH] Secure XML-RPC for Xend

[Anthony Liguori]

[-- Attachment #1: Type: text/plain, Size: 804 bytes --]

Hi,

The following patch implements a secure XML-RPC protocol for Xend.  
Instead of using HTTPS with basic authentication and dealing with all 
that nasty OpenSSL/PAM integration, it just uses SSH.  This gives you 
all the properties you want (great security and PAM integration) with 
very little code.

There are some minor issues so I'd rather it not be applied 
immediately.  I'd like to get some feedback from people as to whether 
this approach is reasonable.  A user-facing change is that now you can 
use the XM_SERVER environmental variable to specific an XML-RPC URI.

For instance:

XM_SERVER='ssh://root@rhesis.austin.ibm.com/RPC2' xm list

Runs xm list on a local machine but does all of the RPCs over a secure 
connection (prompting for passwords).

Thoughts?

Regards,

Anthony Liguori



[-- Attachment #2: xend-secure-xmlrpc.diff --]
[-- Type: text/plain, Size: 9293 bytes --]

# HG changeset patch
# User anthony@rhesis.austin.ibm.com
# Node ID 4de241a7e91a1e59b6db965f5d2ef10014f764e5
# Parent  4f1e39ec05d6ec711f9ba4a66a3653ed3e168311
Add support secure XML-RPC.  This is done by multiplexing multiple SSH
sessions over a single session (to avoid multiple password entries).  Here are
the changes:

1) Add support to xmlrpclib2.ServerProxy for ssh:// protocol
2) Add an xm serve command which proxies XML-RPC over stdio
3) Make xm look at the XM_SERVER variable to determine which XML-RPC protocol
   to use

There are some issues that need to be addressed before inclusion.  Namely:

1) Python moans about tempnam().  I don't think there's a better solution
   though.
2) A command *must* be executed to cleanup the ssh session on exit.  I
   currently use __del__() which doesn't seem to make Python happy in certain
   cases.
3) I have done basic testing but not regression testing with xm-test

diff -r 4f1e39ec05d6 -r 4de241a7e91a tools/python/xen/util/xmlrpclib2.py
--- a/tools/python/xen/util/xmlrpclib2.py	Thu Jun  8 15:51:39 2006
+++ b/tools/python/xen/util/xmlrpclib2.py	Fri Jun  9 01:59:02 2006
@@ -24,14 +24,117 @@
 import types
 
 from httplib import HTTPConnection, HTTP
-from xmlrpclib import Transport
+from xmlrpclib import Transport, getparser
+
 from SimpleXMLRPCServer import SimpleXMLRPCServer, SimpleXMLRPCRequestHandler
 import xmlrpclib, socket, os, stat
 import SocketServer
 
-import xen.xend.XendClient
 from xen.xend.XendLogging import log
 
+import os, commands, getpass, termios, fcntl
+
+class SSH:
+    def __init__(self, host, user=None, askpass=None):
+        """Constructor for SSH object.
+
+        This constructor allows you to specify a hostname, a username,
+        and an askpass program.  username will default to the current
+        user and if askpass is not specified, the password will be
+        read (if necessary) from the controlling terminal."""
+
+        self.sock = os.tempnam()
+        self.host = host
+        sock = self.sock
+        if user == None: user = getpass.getuser()
+        self.user = user
+
+        pid = os.fork()
+        if pid == 0:
+            if askpass:
+                f = open('/dev/tty', 'w')
+                os.environ['SSH_ASKPASS'] = askpass
+                fcntl.ioctl(f.fileno(), termios.TIOCNOTTY)
+                f.close()
+
+            os.execvp('/usr/bin/ssh', ['ssh', '-f', '-2', '-N', '-M', '-S',
+                      sock, '-a', '-x', '-l', user, host])
+
+        p,s = os.waitpid(pid, 0)
+        if s != 0:
+            raise OSError(s, 'Failed to start ssh server')
+
+    def close(self):
+        """Closes an SSH object destroying the SSH server.
+
+        This command must be called when the object is no longer
+        needed to ensure that the SSH server is destroyed properly."""
+
+        if self.sock:
+            commands.getstatusoutput('ssh -q -t -S %s -l %s -O exit %s' %
+                                     (self.sock, self.user, self.host))
+
+    # is this the best way to deal with this??
+    def __del__(self):
+        self.close()
+
+    def getcmd(self, cmd):
+        """Returns a command expanded to include SSH options.
+
+        This function will add the appropriate ssh command and
+        options to the passed in command string.  This is useful
+        if commands.getstatusoutput is not an appropriate exec
+        interface or if you want to handle error conditions in
+        your own way."""
+
+        return 'ssh -q -t -S %s -l %s %s %s' % (self.sock, self.user,
+                                                self.host, cmd)
+
+    def runcmd(self, cmd, data=None):
+        """Runs a command using an existing SSH connection.
+
+        This function will run the passed in command on a remote
+        machine and either return the output or raise an OSError
+        if the command exits with a non-zero status (or some
+        other failure occurs)."""
+
+        cmdline = self.getcmd(cmd)
+        if data:
+            f = open("/tmp/stuff.txt", "w")
+            f.write(data)
+            f.close()
+            cmdline = "cat /tmp/stuff.txt | %s" % cmdline
+        else:
+            cmdline = cmdline
+            
+        o,s = commands.getstatusoutput(cmdline)
+        if o != 0:
+            raise OSError(o,s)
+        return s
+
+class SSHTransport(object):
+    def __init__(self, ssh):
+        self.ssh = ssh
+        
+    def request(self, host, handler, request_body, verbose=0):
+        p, u = getparser()
+        s = self.ssh.runcmd("xm serve",
+                            """POST /%s HTTP/1.0
+User-Agent: Xen
+Host: %s
+Content-Type: text/xml
+Content-Length: %d
+
+%s""" % (handler, host, len(request_body), request_body))
+        lines = s.split('\n')
+        for i in range(len(lines)):
+            if lines[i] == '' or lines[i] == '\r':
+                s = '\n'.join(lines[(i + 1):])
+                break
+        
+        p.feed(s)
+        p.close()
+        return u.close()
 
 # A new ServerProxy that also supports httpu urls.  An http URL comes in the
 # form:
@@ -68,13 +171,32 @@
 
 
 class ServerProxy(xmlrpclib.ServerProxy):
+    def process_ssh_uri(self, uri, askpass):
+        uri = uri[6:]
+        parts = uri.split('@')
+        if len(parts) > 1:
+            user = parts[0]
+            uri = '@'.join(parts[1:])
+        else:
+            user = None
+        parts = uri.split('/')
+        if len(parts) < 2:
+            raise ValueError("Invalid ssh:// URL '%s'" % uri)
+        host = parts[0]
+        path = '/'.join(parts[1:])
+        ssh = SSH(host, user, askpass=askpass)
+        transport = SSHTransport(ssh)
+        return transport, 'http://%s/%s' % (host, path)
+        
     def __init__(self, uri, transport=None, encoding=None, verbose=0,
-                 allow_none=1):
+                 allow_none=1, askpass=None):
         if transport == None:
             (protocol, rest) = uri.split(':', 1)
             if protocol == 'httpu':
                 uri = 'http:' + rest
                 transport = UnixTransport()
+            elif protocol == 'ssh':
+                transport, uri = self.process_ssh_uri(uri, askpass)
         xmlrpclib.ServerProxy.__init__(self, uri, transport, encoding,
                                        verbose, allow_none)
 
@@ -121,6 +243,7 @@
         except xmlrpclib.Fault, fault:
             response = xmlrpclib.dumps(fault)
         except Exception, exn:
+            import xen.xend.XendClient
             log.exception(exn)
             response = xmlrpclib.dumps(
                 xmlrpclib.Fault(xen.xend.XendClient.ERROR_INTERNAL, str(exn)))
diff -r 4f1e39ec05d6 -r 4de241a7e91a tools/python/xen/xend/XendClient.py
--- a/tools/python/xen/xend/XendClient.py	Thu Jun  8 15:51:39 2006
+++ b/tools/python/xen/xend/XendClient.py	Fri Jun  9 01:59:02 2006
@@ -18,6 +18,7 @@
 #============================================================================
 
 from xen.util.xmlrpclib2 import ServerProxy
+import os
 
 XML_RPC_SOCKET = "/var/run/xend/xmlrpc.sock"
 
@@ -25,4 +26,7 @@
 ERROR_GENERIC = 2
 ERROR_INVALID_DOMAIN = 3
 
-server = ServerProxy('httpu:///var/run/xend/xmlrpc.sock')
+if os.environ.has_key('XM_SERVER'):
+    server = ServerProxy(os.environ['XM_SERVER'])
+else:
+    server = ServerProxy('httpu:///var/run/xend/xmlrpc.sock')
diff -r 4f1e39ec05d6 -r 4de241a7e91a tools/python/xen/xm/main.py
--- a/tools/python/xen/xm/main.py	Thu Jun  8 15:51:39 2006
+++ b/tools/python/xen/xm/main.py	Fri Jun  9 01:59:02 2006
@@ -124,6 +124,7 @@
 loadpolicy_help = "loadpolicy <policy>              Load binary policy into hypervisor"
 makepolicy_help = "makepolicy <policy>              Build policy and create .bin/.map files"
 labels_help     = "labels [policy] [type=DOM|..]    List <type> labels for (active) policy."
+serve_help      = "serve                            Proxy Xend XML-RPC over stdio"
 
 short_command_list = [
     "console",
@@ -171,7 +172,8 @@
 host_commands = [
     "dmesg",
     "info",
-    "log"
+    "log",
+    "serve",
     ]
 
 scheduler_commands = [
@@ -833,6 +835,36 @@
     arg_check(args, "log", 0)
     
     print server.xend.node.log()
+
+def xm_serve(args):
+    arg_check(args, "serve", 0)
+
+    try:
+        s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
+        s.connect(xen.xend.XendClient.XML_RPC_SOCKET)
+        f = sys.stdin
+        
+        content_length = 0
+        line = f.readline()
+        headers = ''
+        while len(line) != 0:
+            if line.lower().startswith('content-length: '):
+                content_length = int(line[16:])
+            headers += line
+            if line in ['\r\n', '\n']:
+                break
+            line = f.readline()
+
+        buf = f.read(content_length)
+        s.sendall(headers + buf)
+
+        buf = s.recv(4096)
+        while len(buf) != 0:
+            sys.stdout.write(buf)
+            buf = s.recv(4096)
+        s.close()
+    except Exception, ex:
+        print ex
 
 def parse_dev_info(info):
     def get_info(n, t, d):
@@ -1072,6 +1104,7 @@
     "dmesg": xm_dmesg,
     "info": xm_info,
     "log": xm_log,
+    "serve": xm_serve,
     # scheduler
     "sched-bvt": xm_sched_bvt,
     "sched-bvt-ctxallow": xm_sched_bvt_ctxallow,

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [RFC][PATCH] Secure XML-RPC for Xend

[Matthew Palmer]

On Thu, Jun 08, 2006 at 09:13:17PM -0500, Anthony Liguori wrote:
> The following patch implements a secure XML-RPC protocol for Xend.  
> Instead of using HTTPS with basic authentication and dealing with all 
> that nasty OpenSSL/PAM integration, it just uses SSH.  This gives you 
> all the properties you want (great security and PAM integration) with 
> very little code.

Are there any plans to make the XML-RPC interface easily accessable for
things other than xm?  Although HTTPS (I'd use client certs rather than
basic auth, but that's even worse from a PAM integration PoV) is more
overhead, it's at least platform-independent.  I've been doing a bit of
poking into using XML-RPC to control Xend from Ruby, and it's a hassle. 
Adding an SSH tunnel layer over the top is going to be even more of a
nightmare.

My workaround at the moment is to create a higher-level interface to control
Xend -- it's SOAP over HTTPS (with client certs to perform authentication
and, eventually, authorization).  I'm using SOAP simply because it's got
WSDL.  The interface is a lot simpler (very, very few S-expressions to deal
with), so it'll probably still exist even if Xend gets a more advanced
XML-RPC layer, but it'd save me some fiddling if I could poke Xend's XML-RPC
securely without needing to grub around for a Unix socket.

- Matt

RE: [RFC][PATCH] Secure XML-RPC for Xend

[Ian Pratt]

> The following patch implements a secure XML-RPC protocol for Xend.
> Instead of using HTTPS with basic authentication and dealing with all
> that nasty OpenSSL/PAM integration, it just uses SSH.  This gives you
> all the properties you want (great security and PAM integration) with
> very little code.

I think we just have to bite the bullet on this one. OpenSSL/PAM
integration isn't that hard, and it makes things much cleaner from a
client point of view, which is what really matters.

We can always use "stunnel" to make life easier.

Ian

 
> There are some minor issues so I'd rather it not be applied
> immediately.  I'd like to get some feedback from people as to whether
> this approach is reasonable.  A user-facing change is that now you can
> use the XM_SERVER environmental variable to specific an XML-RPC URI.
> 
> For instance:
> 
> XM_SERVER='ssh://root@rhesis.austin.ibm.com/RPC2' xm list
> 
> Runs xm list on a local machine but does all of the RPCs over a secure
> connection (prompting for passwords).
> 
> Thoughts?
> 
> Regards,
> 
> Anthony Liguori
> 

Re: [RFC][PATCH] Secure XML-RPC for Xend

[Anil Madhavapeddy]

On Thu, Jun 08, 2006 at 09:13:17PM -0500, Anthony Liguori wrote:
> Add support secure XML-RPC.  This is done by multiplexing multiple SSH
> sessions over a single session (to avoid multiple password entries).  Here are
> the changes:

I like the general idea, comments inline.

> 
> 1) Add support to xmlrpclib2.ServerProxy for ssh:// protocol
> 2) Add an xm serve command which proxies XML-RPC over stdio
> 3) Make xm look at the XM_SERVER variable to determine which XML-RPC protocol
>    to use
> 
> There are some issues that need to be addressed before inclusion.  Namely:
> 
> 1) Python moans about tempnam().  I don't think there's a better solution
>    though.

I don't like the dependency on directly calling ssh multiplexing,
as it requires a relatively modern OpenSSH (>3.9) and the above
race condition is introduced.  A newer feature in OpenSSH is to let
the ControlPath consist of "%h,%p,%r" wildcards which fill in the
host/user/port being connected to in a socket pathname, which solves
that particular race.

Why not just do the SSH every time, and let the user either set up
connection multiplexing or ssh agent in their local environment
instead?  That way it will work for old OpenSSH versions and you
don't have to deal with all the quirks.

> 2) A command *must* be executed to cleanup the ssh session on exit.  I
>    currently use __del__() which doesn't seem to make Python happy in certain
>    cases.
> 3) I have done basic testing but not regression testing with xm-test
> 
> diff -r 4f1e39ec05d6 -r 4de241a7e91a tools/python/xen/util/xmlrpclib2.py
...
> +    def runcmd(self, cmd, data=None):
> +        """Runs a command using an existing SSH connection.
> +
> +        This function will run the passed in command on a remote
> +        machine and either return the output or raise an OSError
> +        if the command exits with a non-zero status (or some
> +        other failure occurs)."""
> +
> +        cmdline = self.getcmd(cmd)
> +        if data:
> +            f = open("/tmp/stuff.txt", "w")
> +            f.write(data)
> +            f.close()
> +            cmdline = "cat /tmp/stuff.txt | %s" % cmdline

Ouch, this bit definitely needs to be fixed at least :)

-- 
Anil Madhavapeddy                                 http://anil.recoil.org
University of Cambridge                          http://www.cl.cam.ac.uk

Re: [RFC][PATCH] Secure XML-RPC for Xend

[Daniel Veillard]

On Fri, Jun 09, 2006 at 09:34:35AM +0100, Anil Madhavapeddy wrote:
> > 1) Python moans about tempnam().  I don't think there's a better solution
> >    though.
> 
> I don't like the dependency on directly calling ssh multiplexing,
> as it requires a relatively modern OpenSSH (>3.9) and the above
> race condition is introduced.  A newer feature in OpenSSH is to let
> the ControlPath consist of "%h,%p,%r" wildcards which fill in the
> host/user/port being connected to in a socket pathname, which solves
> that particular race.
> 
> Why not just do the SSH every time, and let the user either set up
> connection multiplexing or ssh agent in their local environment
> instead? 

   SSH authentication is really expensive especially when you compare to
other cost in the XML-RPC. I would really like some persistency
of the connection if possible, especially for operations like monitoring,
it's okay to reopen from time to time, but without reuse it would just not
work.

Daniel

-- 
Daniel Veillard      | Red Hat http://redhat.com/
veillard@redhat.com  | libxml GNOME XML XSLT toolkit  http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/

Re: [RFC][PATCH] Secure XML-RPC for Xend

[Anil Madhavapeddy]

On Fri, Jun 09, 2006 at 04:41:48AM -0400, Daniel Veillard wrote:
> 
>    SSH authentication is really expensive especially when you compare to
> other cost in the XML-RPC. I would really like some persistency
> of the connection if possible, especially for operations like monitoring,
> it's okay to reopen from time to time, but without reuse it would just not
> work.

Yes, but the right place to do it is not in Xend.  The auth caching
can be set up outside of Xend much more robustly depending on your
version of OpenSSH.  If done in Xend, then it definitely needs to
use the wildcard support in ControlPath to avoid the authentication
race condition, and an OpenSSH version check.

As Ian says, stunnel/SSL is probably easier from the client's point
of view (although I do like the easier SSH key management this patch
allows).

Anil

Re: [RFC][PATCH] Secure XML-RPC for Xend

[Anthony Liguori]

> Why not just do the SSH every time, and let the user either set up
> connection multiplexing or ssh agent in their local environment
> instead?  That way it will work for old OpenSSH versions and you
> don't have to deal with all the quirks.
>   

Another option is just to make sure that the XML-RPC supports keep-alive 
and reuse the same xm serve session.  I think that's probably the most 
compatible approach.

>> 2) A command *must* be executed to cleanup the ssh session on exit.  I
>>    currently use __del__() which doesn't seem to make Python happy in certain
>>    cases.
>> 3) I have done basic testing but not regression testing with xm-test
>>
>> diff -r 4f1e39ec05d6 -r 4de241a7e91a tools/python/xen/util/xmlrpclib2.py
>>     
> ...
>   
>> +    def runcmd(self, cmd, data=None):
>> +        """Runs a command using an existing SSH connection.
>> +
>> +        This function will run the passed in command on a remote
>> +        machine and either return the output or raise an OSError
>> +        if the command exits with a non-zero status (or some
>> +        other failure occurs)."""
>> +
>> +        cmdline = self.getcmd(cmd)
>> +        if data:
>> +            f = open("/tmp/stuff.txt", "w")
>> +            f.write(data)
>> +            f.close()
>> +            cmdline = "cat /tmp/stuff.txt | %s" % cmdline
>>     
>
> Ouch, this bit definitely needs to be fixed at least :)
>   

Yikes, didn't know that was still there :-)

Regards,

Anthony Liguori

Re: [RFC][PATCH] Secure XML-RPC for Xend

[Anthony Liguori]

Ian Pratt wrote:
>> The following patch implements a secure XML-RPC protocol for Xend.
>> Instead of using HTTPS with basic authentication and dealing with all
>> that nasty OpenSSL/PAM integration, it just uses SSH.  This gives you
>> all the properties you want (great security and PAM integration) with
>> very little code.
>>     
>
> I think we just have to bite the bullet on this one. OpenSSL/PAM
> integration isn't that hard, and it makes things much cleaner from a
> client point of view, which is what really matters.
>   

It's tempting to use https/basic auth since it seems like it ought to 
just work with existing clients.  However, that doesn't appear to be the 
case.

Python doesn't seem to provide any real support for authentication 
out-of-the-box.  It wouldn't be that hard to add but neither was an SSH 
transport.

The other problem is that Python doesn't provide support for certificate 
verification.  That's okay if you're just using Python to screen scrap 
but if you're in an enterprise environment it's not a very good thing.

The other problem I'm concerned about is certificate management on our 
end.  The average user is going have to end up using snake oil certs and 
I've always found configuring these things to be a real pain.

Another advantage to the SSH approach is that it makes firewall rules 
easier to manage.  There's no additional open port.  This is a minor 
benefit of course but I thought I'd mention it anyway :-)

Writing a client from scratch, I've found supporting the SSH method is 
far easy than https.  Best of all, there's no additional server 
configuration.

I'm clearly biased though, I'm interested to know what others think :-)

Regards,

Anthony Liguori

> We can always use "stunnel" to make life easier.
>
> Ian
>
>  
>   
>> There are some minor issues so I'd rather it not be applied
>> immediately.  I'd like to get some feedback from people as to whether
>> this approach is reasonable.  A user-facing change is that now you can
>> use the XM_SERVER environmental variable to specific an XML-RPC URI.
>>
>> For instance:
>>
>> XM_SERVER='ssh://root@rhesis.austin.ibm.com/RPC2' xm list
>>
>> Runs xm list on a local machine but does all of the RPCs over a secure
>> connection (prompting for passwords).
>>
>> Thoughts?
>>
>> Regards,
>>
>> Anthony Liguori
>>
>>     
>
>   

Re: [RFC][PATCH] Secure XML-RPC for Xend

[Anthony Liguori]

On Fri, 09 Jun 2006 09:54:44 +0100, Anil Madhavapeddy wrote:

> On Fri, Jun 09, 2006 at 04:41:48AM -0400, Daniel Veillard wrote:
>> 
>>    SSH authentication is really expensive especially when you compare to
>> other cost in the XML-RPC. I would really like some persistency
>> of the connection if possible, especially for operations like monitoring,
>> it's okay to reopen from time to time, but without reuse it would just not
>> work.
> 
> Yes, but the right place to do it is not in Xend.  The auth caching
> can be set up outside of Xend much more robustly depending on your
> version of OpenSSH.  If done in Xend, then it definitely needs to
> use the wildcard support in ControlPath to avoid the authentication
> race condition, and an OpenSSH version check.

I think Daniel is suggesting that we use HTTP Keep-Alive which I also
think is a really good idea.  I think this will come in handy regardless
of whether we use SSH.

This makes my patch a lot nicer though.  I just would make sure the
client uses Keep-Alive and then you get the same 1-time auth without
any of the SSH trickery.

I'm investigating this right now.  I seem to recall the HTTP server in
python providing support for Keep-Alive...

> 
> As Ian says, stunnel/SSL is probably easier from the client's point
> of view (although I do like the easier SSH key management this patch
> allows).

There doesn't have to be one solution.  The only real code that's needed
here is xm serve which is not more than 100 lines.  The client code is
more of an example.  I see no reason why we couldn't support all of these
protocols (httpu, http, https, ssh).

Regards,

Anthony Liguori

> Anil

Re: [RFC][PATCH] Secure XML-RPC for Xend

[Ewan Mellor]

On Thu, Jun 08, 2006 at 09:13:17PM -0500, Anthony Liguori wrote:

> Hi,
> 
> The following patch implements a secure XML-RPC protocol for Xend.  
> Instead of using HTTPS with basic authentication and dealing with all 
> that nasty OpenSSL/PAM integration, it just uses SSH.  This gives you 
> all the properties you want (great security and PAM integration) with 
> very little code.
> 
> There are some minor issues so I'd rather it not be applied 
> immediately.  I'd like to get some feedback from people as to whether 
> this approach is reasonable.  A user-facing change is that now you can 
> use the XM_SERVER environmental variable to specific an XML-RPC URI.

I'm with Ian -- I'd rather see the SSL/PAM solution done properly than this.
That said, I don't see why we can't have this transport as well -- it's not a
big patch.

What happens if SSH isn't installed?  I don't see any nice diagnostic of that,
so I'm guessing that it just splats out an "execv failed" exception (unless
I've missed something).

Ewan.

Re: [RFC][PATCH] Secure XML-RPC for Xend

[Ewan Mellor]

On Fri, Jun 09, 2006 at 07:10:23AM -0500, Anthony Liguori wrote:

> Ian Pratt wrote:
> >>The following patch implements a secure XML-RPC protocol for Xend.
> >>Instead of using HTTPS with basic authentication and dealing with all
> >>that nasty OpenSSL/PAM integration, it just uses SSH.  This gives you
> >>all the properties you want (great security and PAM integration) with
> >>very little code.
> >>    
> >
> >I think we just have to bite the bullet on this one. OpenSSL/PAM
> >integration isn't that hard, and it makes things much cleaner from a
> >client point of view, which is what really matters.
> >  
> 
> It's tempting to use https/basic auth since it seems like it ought to 
> just work with existing clients.  However, that doesn't appear to be the 
> case.
> 
> Python doesn't seem to provide any real support for authentication 
> out-of-the-box.  It wouldn't be that hard to add but neither was an SSH 
> transport.

Personally, I'd use SSL to secure the connection and authenticate the server
to the client, but I'd not use HTTP's basic auth -- I'd add a "login" message
that checked the username/password using PAM, in other words, have the
authentication of the user handled at Xend's level, rather than relying on the
transport/session layer to do it.  Like you say, HTTP's authentication stuff
doesn't seem to be well supported.

> The other problem is that Python doesn't provide support for certificate 
> verification.  That's okay if you're just using Python to screen scrap 
> but if you're in an enterprise environment it's not a very good thing.
> 
> The other problem I'm concerned about is certificate management on our 
> end.  The average user is going have to end up using snake oil certs and 
> I've always found configuring these things to be a real pain.

It's only not a pain with SSH because your distro has set it up for you to
generate a key at install time.  Hopefully, we could arrange or rely upon the
distros to arrange a similar thing for Xend.

Ewan.

Re: [RFC][PATCH] Secure XML-RPC for Xend

[Anthony Liguori]

Ewan Mellor wrote:
> On Thu, Jun 08, 2006 at 09:13:17PM -0500, Anthony Liguori wrote:
>
>   
>> Hi,
>>
>> The following patch implements a secure XML-RPC protocol for Xend.  
>> Instead of using HTTPS with basic authentication and dealing with all 
>> that nasty OpenSSL/PAM integration, it just uses SSH.  This gives you 
>> all the properties you want (great security and PAM integration) with 
>> very little code.
>>
>> There are some minor issues so I'd rather it not be applied 
>> immediately.  I'd like to get some feedback from people as to whether 
>> this approach is reasonable.  A user-facing change is that now you can 
>> use the XM_SERVER environmental variable to specific an XML-RPC URI.
>>     
>
> I'm with Ian -- I'd rather see the SSL/PAM solution done properly than this.
> That said, I don't see why we can't have this transport as well -- it's not a
> big patch.
>
> What happens if SSH isn't installed?  I don't see any nice diagnostic of that,
> so I'm guessing that it just splats out an "execv failed" exception (unless
> I've missed something).
>   

In the current code, Popen throws an OSError.

I really don't like catching exceptions and doing an sys.exit within the 
command handler.  I'd rather introduce a new exception type for use in 
xm and rethrow the OSError with a friendly message.  This will make 
localization quite a bit easier.

What do you think of this?

Regards,

Anthony Liguori

> Ewan.
>   

Re: [RFC][PATCH] Secure XML-RPC for Xend

[Anthony Liguori]

Ewan Mellor wrote:
> On Fri, Jun 09, 2006 at 07:10:23AM -0500, Anthony Liguori wrote:
>   
>> It's tempting to use https/basic auth since it seems like it ought to 
>> just work with existing clients.  However, that doesn't appear to be the 
>> case.
>>
>> Python doesn't seem to provide any real support for authentication 
>> out-of-the-box.  It wouldn't be that hard to add but neither was an SSH 
>> transport.
>>     
>
> Personally, I'd use SSL to secure the connection and authenticate the server
> to the client, but I'd not use HTTP's basic auth -- I'd add a "login" message
> that checked the username/password using PAM, in other words, have the
> authentication of the user handled at Xend's level, rather than relying on the
> transport/session layer to do it.  Like you say, HTTP's authentication stuff
> doesn't seem to be well supported.
>   

I see three ways of making this work.

1) Have a login message that returns some sort of random key.  This key 
would have to be passed to all other RPCs.  The key would have to come 
from a good random source and be sufficiently large that a brute-force 
attack wasn't likely.  We would have to introduce some sort of way to 
expire these keys though (to prevent a user from reusing a key from a 
previous authentication after the root user password has changed).

2) Pass the credentials to every RPC (this is essentially what HTTP 
Basic auth does).  The good thing about this is that it solves the above 
key expiration problem but PAM authentication can be pretty heavy weight 
depending on how your PAM is configured.  This could prove rather nasty.

3) Make sure the client uses Keep-Alive and have the login RPC 
authentication the rest of the session.  I like this approach the most 
since it avoids the problems with both of the above.  Of course, it 
assumes your client library is capable of doing XML-RPC over a 
Keep-Alive session which probably isn't a good assumption.

What do you think?  Is there another way of doing this that I'm missing?

The plus side of using the login RPC is that with stunnel it will be 
pretty easy to implement on the server end.

>> The other problem is that Python doesn't provide support for certificate 
>> verification.  That's okay if you're just using Python to screen scrap 
>> but if you're in an enterprise environment it's not a very good thing.
>>
>> The other problem I'm concerned about is certificate management on our 
>> end.  The average user is going have to end up using snake oil certs and 
>> I've always found configuring these things to be a real pain.
>>     
>
> It's only not a pain with SSH because your distro has set it up for you to
> generate a key at install time.  Hopefully, we could arrange or rely upon the
> distros to arrange a similar thing for Xend.
>   

It's worse because SSL assumes that there is a single root-of-trust 
(namely Verisign).  Properly signed certs are expensive.  Managing snake 
oils certs becomes a PITA.

I still think HTTPS is worth doing because it will be useful to an awful 
lot of clients.

Regards,

Anthony Liguori

> Ewan.
>   

Re: [RFC][PATCH] Secure XML-RPC for Xend

[Ewan Mellor]

On Wed, Jun 14, 2006 at 12:26:18PM -0500, Anthony Liguori wrote:

> Ewan Mellor wrote:
> >On Thu, Jun 08, 2006 at 09:13:17PM -0500, Anthony Liguori wrote:
> >
> >  
> >>Hi,
> >>
> >>The following patch implements a secure XML-RPC protocol for Xend.  
> >>Instead of using HTTPS with basic authentication and dealing with all 
> >>that nasty OpenSSL/PAM integration, it just uses SSH.  This gives you 
> >>all the properties you want (great security and PAM integration) with 
> >>very little code.
> >>
> >>There are some minor issues so I'd rather it not be applied 
> >>immediately.  I'd like to get some feedback from people as to whether 
> >>this approach is reasonable.  A user-facing change is that now you can 
> >>use the XM_SERVER environmental variable to specific an XML-RPC URI.
> >>    
> >
> >I'm with Ian -- I'd rather see the SSL/PAM solution done properly than 
> >this.
> >That said, I don't see why we can't have this transport as well -- it's 
> >not a
> >big patch.
> >
> >What happens if SSH isn't installed?  I don't see any nice diagnostic of 
> >that,
> >so I'm guessing that it just splats out an "execv failed" exception (unless
> >I've missed something).
> >  
> 
> In the current code, Popen throws an OSError.
> 
> I really don't like catching exceptions and doing an sys.exit within the 
> command handler.  I'd rather introduce a new exception type for use in 
> xm and rethrow the OSError with a friendly message.  This will make 
> localization quite a bit easier.
> 
> What do you think of this?

Sure, diagnose and rethrow all the way to the top level -- that's what main.py
does now for most things, and it makes it easier to integrate main.py into
larger applications too.

Ewan.