Re: [Qemu-devel] [RFC PATCH 0/2] GlusterFS support in QEMU - v2

[Stefan Hajnoczi <stefanha@gmail.com>]

On Sat, Jul 21, 2012 at 9:29 AM, Bharata B Rao
<bharata@linux.vnet.ibm.com> wrote:
> -drive file=gluster:server@port:volname:image
>
> - Here 'gluster' is the protocol.
> - 'server@port' specifies the server where the volume file specification for
>   the given volume resides. 'port' is the port number on which gluster
>   management daemon (glusterd) is listening. This is optional and if not
>   specified, QEMU will send 0 which will make libgfapi to use the default
>   port.

'server@port' is weird notation.  Normally it is 'server:port' (e.g.
URLs).  Can you change it?

What about the other transports supported by libgfapi: UNIX domain
sockets and RDMA?  My reading of glfs.h is that there are 3 connection
options:
1. 'transport': 'socket' (default), 'unix', 'rdma'
2. 'host': server hostname for 'socket', path to UNIX domain socket
for 'unix', or something else for 'rdma'
3. 'port': TCP port when 'socket' is used.  Ignored otherwise.

Unfortunately QEMU block drivers cannot take custom options yet.  That
would make it possible to cleanly map these connection options and
save you from inventing syntax which doesn't expose all options.

In the meantime it would be nice if the syntax exposed all options.

> Note that we are no longer using volfiles directly and use volume names
> instead. For this to work, gluster management daemon (glusterd) needs to
> be running on the QEMU node. This limits the QEMU user to access the volumes by
> the default volfiles that are generated by gluster CLI. This should be
> fine as long as gluster CLI provides the capability to generate or regenerate
> volume files for a given volume with the xlator set that QEMU user is
> interested in. GlusterFS developers tell me that this can be provided with
> some enhancements to Gluster CLI/glusterd. Note that the custom volume files
> is typically needed when GlusterFS server is co-located with QEMU in
> which case it would  be beneficial to get rid of client-server overhead and
> RPC communication overhead.

My knowledge of GlusterFS is limited.  Here is what I am thinking:

1. The user cannot specify a local configuration file, you require
that there is a glusterd running which provides configuration
information.
2. It is currently not possible to bypass RPC because the glusterd
managed configuration file doesn't support that.

I'm not sure if these statements are true?

Would you support local volfiles in the future again?  Why force users
to run glusterd?

> - As mentioned above, the VM image on gluster volume can be specified like
>   this:
>         -drive file=gluster:localhost:testvol:/F17,format=gluster
>
>   Note that format=gluster is not needed ideally and its a work around I have
>   until libgfapi provides a working connection cleanup routine (glfs_fini()).
>   When the format isn't specified, QEMU figures out the format by doing
>   find_image_format that results in one open and close before opening the
>   image file long term for standard read and write. Gluster connection
>   initialization is done from open and connection termination is done from
>   close. But since glfs_fini() isn't working yet, I am bypassing
>   find_image_format by specifying format=gluster directly which results in
>   just one open and hence I am not limited by glfs_fini().

Has libgfapi been released yet?  Does it have versioning which will
allow the QEMU GlusterFS block driver to build against different
versions?  I'm just wondering how the pieces will fit together once
distros start shipping them.

Stefan