[Qemu-devel] [PATCHv9 0/5] block: add native support for NFS

[Qemu-devel] [PATCHv9 0/5] block: add native support for NFS

[Peter Lieven]

This adds v9 of the NFS protocol driver + qemu-iotest adjustments.

v8->v9:
 - change _supported_proto for qemu-iotests 071 & 072 from generic to file

v7->v8:
 - bump libnfs requirement to 1.9.2
 - renamed NFSRPC->status to NFSRPC->ret [Benoit]
 - avoid double invocation of nfs_client_close in nfs_file_create [Benoit]
 - fixed error propagation in nfs_file_open [Benoit]

v6->v7:
 - removed unsused and partially non-existing libnfs header file includes [Stefan]
 - adjusted usage of qemu_opts_create_nofail [Stefan]
 - removed patch to fix output of qemu-iotest 067
 - only blacklist qemu-iotest 020 for NFS do not change proto from generic to file [Fam]

v5->v6:
 - use internal qemu function to parse the NFS url [Kevin]
 - zero pad short reads [Kevin, Paolo]
 - added qemu-iotests patches for basic nfs protocol support

v4->v5:
 - disussed with Ronnie and decided to move URL + Paramter parsing to LibNFS.
   This allows for URL parameter processing directly in LibNFS without altering
   the qemu NFS block driver. This bumps the version requirement for LibNFS
   to 1.9.0 though.
 - added a pointer to the LibNFS readme where additional information about
   ROOT privilidge requirements can be found as this raised a few concerns.
 - removed a trailing dot in an error statement [Fam].

v3->v4:
 - finally added full implementation of bdrv_get_allocated_file_size [Stefan]
 - removed trailing \n from error statements [Stefan]

v2->v3:
 - rebased the stefanha/block
 - use pkg_config to check for libnfs (ignoring cflags which are broken in 1.8.0) [Stefan]
 - fixed NFSClient declaration [Stefan]
 - renamed Task variables to task [Stefan]
 - renamed NFSTask to NFSRPC [Ronnie]
 - do not update bs->total_sectors in nfs_co_writev [Stefan]
 - return -ENOMEM on all async call failures [Stefan,Ronnie]
 - fully implement ftruncate
 - use util/uri.c for URL parsing [Stefan]
 - reworked nfs_file_open_common to nfs_client_open which works on NFSClient [Stefan]
 - added a comment ot the connect message that libnfs support NFSv3 only at the moment.
 - DID NOT add full implementation of bdrv_get_allocated_file_size because
   we are not in a coroutine context and I cannot do an async call here.
   I could do a sync call if there would be a guarantee that no requests
   are in flight. [Stefan]

v1->v2:
 - fixed block/Makefile.objs [Ronnie]
 - do not always register a read handler [Ronnie]
 - add support for reading beyond EOF [Fam]
 - fixed struct and paramter naming [Fam]
 - fixed overlong lines and whitespace errors [Fam]
 - return return status from libnfs whereever possible [Fam]
 - added comment why we set allocated_file_size to -ENOTSUP after write [Fam]
 - avoid segfault when parsing filname [Fam]
 - remove unused close_bh from NFSClient [Fam]
 - avoid dividing and mutliplying total_size by BDRV_SECTOR_SIZE in nfs_file_create [Fam] 

Peter Lieven (5):
  block: add native support for NFS
  qemu-iotests: change _supported_proto to file for various tests
  qemu-iotests: blacklist test 020 for NFS protocol
  qemu-iotests: enable test 016 and 025 to work with NFS protocol
  qemu-iotests: enable support for NFS protocol

 MAINTAINERS                  |    5 +
 block/Makefile.objs          |    1 +
 block/nfs.c                  |  439 ++++++++++++++++++++++++++++++++++++++++++
 configure                    |   26 +++
 qapi-schema.json             |    1 +
 tests/qemu-iotests/013       |    2 +-
 tests/qemu-iotests/014       |    2 +-
 tests/qemu-iotests/016       |    2 +-
 tests/qemu-iotests/018       |    2 +-
 tests/qemu-iotests/019       |    2 +-
 tests/qemu-iotests/020       |    5 +
 tests/qemu-iotests/023       |    2 +-
 tests/qemu-iotests/024       |    2 +-
 tests/qemu-iotests/025       |    2 +-
 tests/qemu-iotests/026       |    2 +-
 tests/qemu-iotests/028       |    2 +-
 tests/qemu-iotests/031       |    2 +-
 tests/qemu-iotests/034       |    2 +-
 tests/qemu-iotests/036       |    2 +-
 tests/qemu-iotests/037       |    2 +-
 tests/qemu-iotests/038       |    2 +-
 tests/qemu-iotests/039       |    2 +-
 tests/qemu-iotests/043       |    2 +-
 tests/qemu-iotests/046       |    2 +-
 tests/qemu-iotests/052       |    2 +-
 tests/qemu-iotests/054       |    2 +-
 tests/qemu-iotests/059       |    2 +-
 tests/qemu-iotests/060       |    2 +-
 tests/qemu-iotests/061       |    2 +-
 tests/qemu-iotests/063       |    2 +-
 tests/qemu-iotests/069       |    2 +-
 tests/qemu-iotests/071       |    2 +-
 tests/qemu-iotests/072       |    2 +-
 tests/qemu-iotests/common    |   22 ++-
 tests/qemu-iotests/common.rc |    3 +
 35 files changed, 526 insertions(+), 30 deletions(-)
 create mode 100644 block/nfs.c

-- 
1.7.9.5

[Qemu-devel] [PATCHv9 1/5] block: add native support for NFS

[Peter Lieven]

This patch adds native support for accessing images on NFS
shares without the requirement to actually mount the entire
NFS share on the host.

NFS Images can simply be specified by an url of the form:
nfs://<host>/<export>/<filename>[?param=value[&param2=value2[&...]]]

For example:
qemu-img create -f qcow2 nfs://10.0.0.1/qemu-images/test.qcow2

You need LibNFS from Ronnie Sahlberg available at:
   git://github.com/sahlberg/libnfs.git
for this to work.

During configure it is automatically probed for libnfs and support
is enabled on-the-fly. You can forbid or enforce libnfs support
with --disable-libnfs or --enable-libnfs respectively.

Due to NFS restrictions you might need to execute your binaries
as root, allow them to open priviledged ports (<1024) or specify
insecure option on the NFS server.

For additional information on ROOT vs. non-ROOT operation and URL
format + parameters see:
   https://raw.github.com/sahlberg/libnfs/master/README

Supported by qemu are the uid, gid and tcp-syncnt URL parameters.

LibNFS currently support NFS version 3 only.

Signed-off-by: Peter Lieven <pl@kamp.de>
---
 MAINTAINERS         |    5 +
 block/Makefile.objs |    1 +
 block/nfs.c         |  439 +++++++++++++++++++++++++++++++++++++++++++++++++++
 configure           |   26 +++
 qapi-schema.json    |    1 +
 5 files changed, 472 insertions(+)
 create mode 100644 block/nfs.c

diff --git a/MAINTAINERS b/MAINTAINERS
index fb53242..f8411f9 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -936,6 +936,11 @@ M: Peter Lieven <pl@kamp.de>
 S: Supported
 F: block/iscsi.c
 
+NFS
+M: Peter Lieven <pl@kamp.de>
+S: Maintained
+F: block/nfs.c
+
 SSH
 M: Richard W.M. Jones <rjones@redhat.com>
 S: Supported
diff --git a/block/Makefile.objs b/block/Makefile.objs
index 4e8c91e..e254a21 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -12,6 +12,7 @@ block-obj-$(CONFIG_LINUX_AIO) += linux-aio.o
 ifeq ($(CONFIG_POSIX),y)
 block-obj-y += nbd.o nbd-client.o sheepdog.o
 block-obj-$(CONFIG_LIBISCSI) += iscsi.o
+block-obj-$(CONFIG_LIBNFS) += nfs.o
 block-obj-$(CONFIG_CURL) += curl.o
 block-obj-$(CONFIG_RBD) += rbd.o
 block-obj-$(CONFIG_GLUSTERFS) += gluster.o
diff --git a/block/nfs.c b/block/nfs.c
new file mode 100644
index 0000000..ef731f0
--- /dev/null
+++ b/block/nfs.c
@@ -0,0 +1,439 @@
+/*
+ * QEMU Block driver for native access to files on NFS shares
+ *
+ * Copyright (c) 2014 Peter Lieven <pl@kamp.de>
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "config-host.h"
+
+#include <poll.h>
+#include "qemu-common.h"
+#include "qemu/config-file.h"
+#include "qemu/error-report.h"
+#include "block/block_int.h"
+#include "trace.h"
+#include "qemu/iov.h"
+#include "qemu/uri.h"
+#include "sysemu/sysemu.h"
+#include <nfsc/libnfs.h>
+
+typedef struct NFSClient {
+    struct nfs_context *context;
+    struct nfsfh *fh;
+    int events;
+    bool has_zero_init;
+} NFSClient;
+
+typedef struct NFSRPC {
+    int ret;
+    int complete;
+    QEMUIOVector *iov;
+    struct stat *st;
+    Coroutine *co;
+    QEMUBH *bh;
+} NFSRPC;
+
+static void nfs_process_read(void *arg);
+static void nfs_process_write(void *arg);
+
+static void nfs_set_events(NFSClient *client)
+{
+    int ev = nfs_which_events(client->context);
+    if (ev != client->events) {
+        qemu_aio_set_fd_handler(nfs_get_fd(client->context),
+                      (ev & POLLIN) ? nfs_process_read : NULL,
+                      (ev & POLLOUT) ? nfs_process_write : NULL,
+                      client);
+
+    }
+    client->events = ev;
+}
+
+static void nfs_process_read(void *arg)
+{
+    NFSClient *client = arg;
+    nfs_service(client->context, POLLIN);
+    nfs_set_events(client);
+}
+
+static void nfs_process_write(void *arg)
+{
+    NFSClient *client = arg;
+    nfs_service(client->context, POLLOUT);
+    nfs_set_events(client);
+}
+
+static void nfs_co_init_task(NFSClient *client, NFSRPC *task)
+{
+    *task = (NFSRPC) {
+        .co         = qemu_coroutine_self(),
+    };
+}
+
+static void nfs_co_generic_bh_cb(void *opaque)
+{
+    NFSRPC *task = opaque;
+    qemu_bh_delete(task->bh);
+    qemu_coroutine_enter(task->co, NULL);
+}
+
+static void
+nfs_co_generic_cb(int ret, struct nfs_context *nfs, void *data,
+                  void *private_data)
+{
+    NFSRPC *task = private_data;
+    task->complete = 1;
+    task->ret = ret;
+    if (task->ret > 0 && task->iov) {
+        if (task->ret <= task->iov->size) {
+            qemu_iovec_from_buf(task->iov, 0, data, task->ret);
+        } else {
+            task->ret = -EIO;
+        }
+    }
+    if (task->ret == 0 && task->st) {
+        memcpy(task->st, data, sizeof(struct stat));
+    }
+    if (task->co) {
+        task->bh = qemu_bh_new(nfs_co_generic_bh_cb, task);
+        qemu_bh_schedule(task->bh);
+    }
+}
+
+static int coroutine_fn nfs_co_readv(BlockDriverState *bs,
+                                     int64_t sector_num, int nb_sectors,
+                                     QEMUIOVector *iov)
+{
+    NFSClient *client = bs->opaque;
+    NFSRPC task;
+
+    nfs_co_init_task(client, &task);
+    task.iov = iov;
+
+    if (nfs_pread_async(client->context, client->fh,
+                        sector_num * BDRV_SECTOR_SIZE,
+                        nb_sectors * BDRV_SECTOR_SIZE,
+                        nfs_co_generic_cb, &task) != 0) {
+        return -ENOMEM;
+    }
+
+    while (!task.complete) {
+        nfs_set_events(client);
+        qemu_coroutine_yield();
+    }
+
+    if (task.ret < 0) {
+        return task.ret;
+    }
+
+    /* zero pad short reads */
+    if (task.ret < iov->size) {
+        qemu_iovec_memset(iov, task.ret, 0, iov->size - task.ret);
+    }
+
+    return 0;
+}
+
+static int coroutine_fn nfs_co_writev(BlockDriverState *bs,
+                                        int64_t sector_num, int nb_sectors,
+                                        QEMUIOVector *iov)
+{
+    NFSClient *client = bs->opaque;
+    NFSRPC task;
+    char *buf = NULL;
+
+    nfs_co_init_task(client, &task);
+
+    buf = g_malloc(nb_sectors * BDRV_SECTOR_SIZE);
+    qemu_iovec_to_buf(iov, 0, buf, nb_sectors * BDRV_SECTOR_SIZE);
+
+    if (nfs_pwrite_async(client->context, client->fh,
+                         sector_num * BDRV_SECTOR_SIZE,
+                         nb_sectors * BDRV_SECTOR_SIZE,
+                         buf, nfs_co_generic_cb, &task) != 0) {
+        g_free(buf);
+        return -ENOMEM;
+    }
+
+    while (!task.complete) {
+        nfs_set_events(client);
+        qemu_coroutine_yield();
+    }
+
+    g_free(buf);
+
+    if (task.ret != nb_sectors * BDRV_SECTOR_SIZE) {
+        return task.ret < 0 ? task.ret : -EIO;
+    }
+
+    return 0;
+}
+
+static int coroutine_fn nfs_co_flush(BlockDriverState *bs)
+{
+    NFSClient *client = bs->opaque;
+    NFSRPC task;
+
+    nfs_co_init_task(client, &task);
+
+    if (nfs_fsync_async(client->context, client->fh, nfs_co_generic_cb,
+                        &task) != 0) {
+        return -ENOMEM;
+    }
+
+    while (!task.complete) {
+        nfs_set_events(client);
+        qemu_coroutine_yield();
+    }
+
+    return task.ret;
+}
+
+/* TODO Convert to fine grained options */
+static QemuOptsList runtime_opts = {
+    .name = "nfs",
+    .head = QTAILQ_HEAD_INITIALIZER(runtime_opts.head),
+    .desc = {
+        {
+            .name = "filename",
+            .type = QEMU_OPT_STRING,
+            .help = "URL to the NFS file",
+        },
+        { /* end of list */ }
+    },
+};
+
+static void nfs_client_close(NFSClient *client)
+{
+    if (client->context) {
+        if (client->fh) {
+            nfs_close(client->context, client->fh);
+        }
+        qemu_aio_set_fd_handler(nfs_get_fd(client->context), NULL, NULL, NULL);
+        nfs_destroy_context(client->context);
+    }
+    memset(client, 0, sizeof(NFSClient));
+}
+
+static void nfs_file_close(BlockDriverState *bs)
+{
+    NFSClient *client = bs->opaque;
+    nfs_client_close(client);
+}
+
+static int64_t nfs_client_open(NFSClient *client, const char *filename,
+                               int flags, Error **errp)
+{
+    int ret = -EINVAL, i;
+    struct stat st;
+    URI *uri;
+    QueryParams *qp = NULL;
+    char *file = NULL, *strp = NULL;
+
+    uri = uri_parse(filename);
+    if (!uri) {
+        error_setg(errp, "Invalid URL specified");
+        goto fail;
+    }
+    strp = strrchr(uri->path, '/');
+    if (strp == NULL) {
+        error_setg(errp, "Invalid URL specified");
+        goto fail;
+    }
+    file = g_strdup(strp);
+    *strp = 0;
+
+    client->context = nfs_init_context();
+    if (client->context == NULL) {
+        error_setg(errp, "Failed to init NFS context");
+        goto fail;
+    }
+
+    qp = query_params_parse(uri->query);
+    for (i = 0; i < qp->n; i++) {
+        if (!qp->p[i].value) {
+            error_setg(errp, "Value for NFS parameter expected: %s",
+                       qp->p[i].name);
+            goto fail;
+        }
+        if (!strncmp(qp->p[i].name, "uid", 3)) {
+            nfs_set_uid(client->context, atoi(qp->p[i].value));
+        } else if (!strncmp(qp->p[i].name, "gid", 3)) {
+            nfs_set_gid(client->context, atoi(qp->p[i].value));
+        } else if (!strncmp(qp->p[i].name, "tcp-syncnt", 10)) {
+            nfs_set_tcp_syncnt(client->context, atoi(qp->p[i].value));
+        } else {
+            error_setg(errp, "Unknown NFS parameter name: %s",
+                       qp->p[i].name);
+            goto fail;
+        }
+    }
+
+    ret = nfs_mount(client->context, uri->server, uri->path);
+    if (ret < 0) {
+        error_setg(errp, "Failed to mount nfs share: %s",
+                   nfs_get_error(client->context));
+        goto fail;
+    }
+
+    if (flags & O_CREAT) {
+        ret = nfs_creat(client->context, file, 0600, &client->fh);
+        if (ret < 0) {
+            error_setg(errp, "Failed to create file: %s",
+                       nfs_get_error(client->context));
+            goto fail;
+        }
+    } else {
+        ret = nfs_open(client->context, file, flags, &client->fh);
+        if (ret < 0) {
+            error_setg(errp, "Failed to open file : %s",
+                       nfs_get_error(client->context));
+            goto fail;
+        }
+    }
+
+    ret = nfs_fstat(client->context, client->fh, &st);
+    if (ret < 0) {
+        error_setg(errp, "Failed to fstat file: %s",
+                   nfs_get_error(client->context));
+        goto fail;
+    }
+
+    ret = DIV_ROUND_UP(st.st_size, BDRV_SECTOR_SIZE);
+    client->has_zero_init = S_ISREG(st.st_mode);
+    goto out;
+fail:
+    nfs_client_close(client);
+out:
+    if (qp) {
+        query_params_free(qp);
+    }
+    uri_free(uri);
+    g_free(file);
+    return ret;
+}
+
+static int nfs_file_open(BlockDriverState *bs, QDict *options, int flags,
+                         Error **errp) {
+    NFSClient *client = bs->opaque;
+    int64_t ret;
+    QemuOpts *opts;
+    Error *local_err = NULL;
+
+    opts = qemu_opts_create(&runtime_opts, NULL, 0, &error_abort);
+    qemu_opts_absorb_qdict(opts, options, &local_err);
+    if (error_is_set(&local_err)) {
+        error_propagate(errp, local_err);
+        return -EINVAL;
+    }
+    ret = nfs_client_open(client, qemu_opt_get(opts, "filename"),
+                          (flags & BDRV_O_RDWR) ? O_RDWR : O_RDONLY,
+                          errp);
+    if (ret < 0) {
+        return ret;
+    }
+    bs->total_sectors = ret;
+    return 0;
+}
+
+static int nfs_file_create(const char *url, QEMUOptionParameter *options,
+                           Error **errp)
+{
+    int ret = 0;
+    int64_t total_size = 0;
+    NFSClient *client = g_malloc0(sizeof(NFSClient));
+
+    /* Read out options */
+    while (options && options->name) {
+        if (!strcmp(options->name, "size")) {
+            total_size = options->value.n;
+        }
+        options++;
+    }
+
+    ret = nfs_client_open(client, url, O_CREAT, errp);
+    if (ret < 0) {
+        goto out;
+    }
+    ret = nfs_ftruncate(client->context, client->fh, total_size);
+    nfs_client_close(client);
+out:
+    g_free(client);
+    return ret;
+}
+
+static int nfs_has_zero_init(BlockDriverState *bs)
+{
+    NFSClient *client = bs->opaque;
+    return client->has_zero_init;
+}
+
+static int64_t nfs_get_allocated_file_size(BlockDriverState *bs)
+{
+    NFSClient *client = bs->opaque;
+    NFSRPC task = {0};
+    struct stat st;
+
+    task.st = &st;
+    if (nfs_fstat_async(client->context, client->fh, nfs_co_generic_cb,
+                        &task) != 0) {
+        return -ENOMEM;
+    }
+
+    while (!task.complete) {
+        nfs_set_events(client);
+        qemu_aio_wait();
+    }
+
+    return (task.ret < 0 ? task.ret : st.st_blocks * st.st_blksize);
+}
+
+static int nfs_file_truncate(BlockDriverState *bs, int64_t offset)
+{
+    NFSClient *client = bs->opaque;
+    return nfs_ftruncate(client->context, client->fh, offset);
+}
+
+static BlockDriver bdrv_nfs = {
+    .format_name     = "nfs",
+    .protocol_name   = "nfs",
+
+    .instance_size   = sizeof(NFSClient),
+    .bdrv_needs_filename = true,
+    .bdrv_has_zero_init = nfs_has_zero_init,
+    .bdrv_get_allocated_file_size = nfs_get_allocated_file_size,
+    .bdrv_truncate = nfs_file_truncate,
+
+    .bdrv_file_open  = nfs_file_open,
+    .bdrv_close      = nfs_file_close,
+    .bdrv_create     = nfs_file_create,
+
+    .bdrv_co_readv         = nfs_co_readv,
+    .bdrv_co_writev        = nfs_co_writev,
+    .bdrv_co_flush_to_disk = nfs_co_flush,
+};
+
+static void nfs_block_init(void)
+{
+    bdrv_register(&bdrv_nfs);
+}
+
+block_init(nfs_block_init);
diff --git a/configure b/configure
index b472694..364bcd5 100755
--- a/configure
+++ b/configure
@@ -251,6 +251,7 @@ vss_win32_sdk=""
 win_sdk="no"
 want_tools="yes"
 libiscsi=""
+libnfs=""
 coroutine=""
 coroutine_pool=""
 seccomp=""
@@ -840,6 +841,10 @@ for opt do
   ;;
   --enable-libiscsi) libiscsi="yes"
   ;;
+  --disable-libnfs) libnfs="no"
+  ;;
+  --enable-libnfs) libnfs="yes"
+  ;;
   --enable-profiler) profiler="yes"
   ;;
   --disable-cocoa) cocoa="no"
@@ -1229,6 +1234,8 @@ Advanced options (experts only):
   --enable-rbd             enable building the rados block device (rbd)
   --disable-libiscsi       disable iscsi support
   --enable-libiscsi        enable iscsi support
+  --disable-libnfs         disable nfs support
+  --enable-libnfs          enable nfs support
   --disable-smartcard-nss  disable smartcard nss support
   --enable-smartcard-nss   enable smartcard nss support
   --disable-libusb         disable libusb (for usb passthrough)
@@ -3600,6 +3607,20 @@ elif test "$debug" = "no" ; then
   CFLAGS="-O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 $CFLAGS"
 fi
 
+##########################################
+# Do we have libnfs
+if test "$libnfs" != "no" ; then
+  if $pkg_config --atleast-version=1.9.2 libnfs; then
+    libnfs="yes"
+    libnfs_libs=$($pkg_config --libs libnfs)
+    LIBS="$LIBS $libnfs_libs"
+  else
+    if test "$libnfs" = "yes" ; then
+      feature_not_found "libnfs"
+    fi
+    libnfs="no"
+  fi
+fi
 
 # Disable zero malloc errors for official releases unless explicitly told to
 # enable/disable
@@ -3829,6 +3850,7 @@ echo "libiscsi support  $libiscsi (1.4.0)"
 else
 echo "libiscsi support  $libiscsi"
 fi
+echo "libnfs support    $libnfs"
 echo "build guest agent $guest_agent"
 echo "QGA VSS support   $guest_agent_with_vss"
 echo "seccomp support   $seccomp"
@@ -4165,6 +4187,10 @@ if test "$libiscsi" = "yes" ; then
   fi
 fi
 
+if test "$libnfs" = "yes" ; then
+  echo "CONFIG_LIBNFS=y" >> $config_host_mak
+fi
+
 if test "$seccomp" = "yes"; then
   echo "CONFIG_SECCOMP=y" >> $config_host_mak
 fi
diff --git a/qapi-schema.json b/qapi-schema.json
index 05ced9d..7cfb5e5 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -4371,6 +4371,7 @@
 # TODO gluster: Wait for structured options
 # TODO iscsi: Wait for structured options
 # TODO nbd: Should take InetSocketAddress for 'host'?
+# TODO nfs: Wait for structured options
 # TODO rbd: Wait for structured options
 # TODO sheepdog: Wait for structured options
 # TODO ssh: Should take InetSocketAddress for 'host'?
-- 
1.7.9.5

[Qemu-devel] [PATCHv9 2/5] qemu-iotests: change _supported_proto to file for various tests

[Peter Lieven]

all these tests do anything of the following and thus fail with any
protocol other than file:
 - the tests use rm, cp or mv shell commands which only work on file
 - the tests use qcow2.py
 - the images construct new filenames (e.g. backing file names) and
   the logic is broken for anything else than file

Signed-off-by: Peter Lieven <pl@kamp.de>
---
 tests/qemu-iotests/013 |    2 +-
 tests/qemu-iotests/014 |    2 +-
 tests/qemu-iotests/018 |    2 +-
 tests/qemu-iotests/019 |    2 +-
 tests/qemu-iotests/023 |    2 +-
 tests/qemu-iotests/024 |    2 +-
 tests/qemu-iotests/026 |    2 +-
 tests/qemu-iotests/028 |    2 +-
 tests/qemu-iotests/031 |    2 +-
 tests/qemu-iotests/034 |    2 +-
 tests/qemu-iotests/036 |    2 +-
 tests/qemu-iotests/037 |    2 +-
 tests/qemu-iotests/038 |    2 +-
 tests/qemu-iotests/039 |    2 +-
 tests/qemu-iotests/043 |    2 +-
 tests/qemu-iotests/046 |    2 +-
 tests/qemu-iotests/052 |    2 +-
 tests/qemu-iotests/054 |    2 +-
 tests/qemu-iotests/059 |    2 +-
 tests/qemu-iotests/060 |    2 +-
 tests/qemu-iotests/061 |    2 +-
 tests/qemu-iotests/063 |    2 +-
 tests/qemu-iotests/069 |    2 +-
 tests/qemu-iotests/071 |    2 +-
 tests/qemu-iotests/072 |    2 +-
 25 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/tests/qemu-iotests/013 b/tests/qemu-iotests/013
index 389f4b8..ea3cab9 100755
--- a/tests/qemu-iotests/013
+++ b/tests/qemu-iotests/013
@@ -41,7 +41,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 # much of this could be generic for any format supporting compression.
 _supported_fmt qcow qcow2
-_supported_proto generic
+_supported_proto file
 _supported_os Linux
 
 TEST_OFFSETS="0 4294967296"
diff --git a/tests/qemu-iotests/014 b/tests/qemu-iotests/014
index 0edeb4b..b23c2db 100755
--- a/tests/qemu-iotests/014
+++ b/tests/qemu-iotests/014
@@ -43,7 +43,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 # much of this could be generic for any format supporting snapshots
 _supported_fmt qcow2
-_supported_proto generic
+_supported_proto file
 _supported_os Linux
 
 TEST_OFFSETS="0 4294967296"
diff --git a/tests/qemu-iotests/018 b/tests/qemu-iotests/018
index 6f7f054..d8a7d43 100755
--- a/tests/qemu-iotests/018
+++ b/tests/qemu-iotests/018
@@ -41,7 +41,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 # Any format supporting backing files
 _supported_fmt qcow qcow2 vmdk qed
-_supported_proto generic
+_supported_proto file
 _supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
 
diff --git a/tests/qemu-iotests/019 b/tests/qemu-iotests/019
index b43e70f..e67445c 100755
--- a/tests/qemu-iotests/019
+++ b/tests/qemu-iotests/019
@@ -45,7 +45,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 # Any format supporting backing files
 _supported_fmt qcow qcow2 vmdk qed
-_supported_proto generic
+_supported_proto file
 _supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
diff --git a/tests/qemu-iotests/023 b/tests/qemu-iotests/023
index 090ed23..9ad06b9 100755
--- a/tests/qemu-iotests/023
+++ b/tests/qemu-iotests/023
@@ -41,7 +41,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 # much of this could be generic for any format supporting compression.
 _supported_fmt qcow qcow2
-_supported_proto generic
+_supported_proto file
 _supported_os Linux
 
 TEST_OFFSETS="0 4294967296"
diff --git a/tests/qemu-iotests/024 b/tests/qemu-iotests/024
index be974f0..9bf99e1 100755
--- a/tests/qemu-iotests/024
+++ b/tests/qemu-iotests/024
@@ -43,7 +43,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 # Currently only qcow2 and qed support rebasing
 _supported_fmt qcow2 qed
-_supported_proto generic
+_supported_proto file
 _supported_os Linux
 
 CLUSTER_SIZE=65536
diff --git a/tests/qemu-iotests/026 b/tests/qemu-iotests/026
index c9c5f83..df2884b 100755
--- a/tests/qemu-iotests/026
+++ b/tests/qemu-iotests/026
@@ -42,7 +42,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 # Currently only qcow2 supports rebasing
 _supported_fmt qcow2
-_supported_proto generic
+_supported_proto file
 _supported_os Linux
 _default_cache_mode "writethrough"
 _supported_cache_modes "writethrough" "none"
diff --git a/tests/qemu-iotests/028 b/tests/qemu-iotests/028
index 93a9fa6..a99e4fa 100755
--- a/tests/qemu-iotests/028
+++ b/tests/qemu-iotests/028
@@ -45,7 +45,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # Any format supporting backing files except vmdk and qcow which do not support
 # smaller backing files.
 _supported_fmt qcow2 qed
-_supported_proto generic
+_supported_proto file
 _supported_os Linux
 
 # Choose a size that is not necessarily a cluster size multiple for image
diff --git a/tests/qemu-iotests/031 b/tests/qemu-iotests/031
index c9070b0..1d920ea 100755
--- a/tests/qemu-iotests/031
+++ b/tests/qemu-iotests/031
@@ -41,7 +41,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 # This tests qcow2-specific low-level functionality
 _supported_fmt qcow2
-_supported_proto generic
+_supported_proto file
 _supported_os Linux
 
 CLUSTER_SIZE=65536
diff --git a/tests/qemu-iotests/034 b/tests/qemu-iotests/034
index 7349789..69c7858 100755
--- a/tests/qemu-iotests/034
+++ b/tests/qemu-iotests/034
@@ -39,7 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 . ./common.filter
 
 _supported_fmt qcow qcow2 vmdk qed
-_supported_proto generic
+_supported_proto file
 _supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
diff --git a/tests/qemu-iotests/036 b/tests/qemu-iotests/036
index e049a64..03b6aa9 100755
--- a/tests/qemu-iotests/036
+++ b/tests/qemu-iotests/036
@@ -44,7 +44,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 # This tests qcow2-specific low-level functionality
 _supported_fmt qcow2
-_supported_proto generic
+_supported_proto file
 _supported_os Linux
 
 # Only qcow2v3 and later supports feature bits
diff --git a/tests/qemu-iotests/037 b/tests/qemu-iotests/037
index e444349..9171d8c 100755
--- a/tests/qemu-iotests/037
+++ b/tests/qemu-iotests/037
@@ -39,7 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 . ./common.filter
 
 _supported_fmt qcow qcow2 vmdk qed
-_supported_proto generic
+_supported_proto file
 _supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
diff --git a/tests/qemu-iotests/038 b/tests/qemu-iotests/038
index 7bb7906..cfaf00a 100755
--- a/tests/qemu-iotests/038
+++ b/tests/qemu-iotests/038
@@ -39,7 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 . ./common.filter
 
 _supported_fmt qcow2 qed
-_supported_proto generic
+_supported_proto file
 _supported_os Linux
 
 CLUSTER_SIZE=2M
diff --git a/tests/qemu-iotests/039 b/tests/qemu-iotests/039
index 6abf472..9b355c0 100755
--- a/tests/qemu-iotests/039
+++ b/tests/qemu-iotests/039
@@ -42,7 +42,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 . ./common.filter
 
 _supported_fmt qcow2
-_supported_proto generic
+_supported_proto file
 _supported_os Linux
 _default_cache_mode "writethrough"
 _supported_cache_modes "writethrough"
diff --git a/tests/qemu-iotests/043 b/tests/qemu-iotests/043
index d7f1231..b316b97 100755
--- a/tests/qemu-iotests/043
+++ b/tests/qemu-iotests/043
@@ -41,7 +41,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 # Any format supporting backing files
 _supported_fmt qcow2 qed
-_supported_proto generic
+_supported_proto file
 _supported_os Linux
 
 
diff --git a/tests/qemu-iotests/046 b/tests/qemu-iotests/046
index 3f17ceb..2d44bbb 100755
--- a/tests/qemu-iotests/046
+++ b/tests/qemu-iotests/046
@@ -39,7 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 . ./common.filter
 
 _supported_fmt qcow2
-_supported_proto generic
+_supported_proto file
 _supported_os Linux
 
 CLUSTER_SIZE=64k
diff --git a/tests/qemu-iotests/052 b/tests/qemu-iotests/052
index 4d4e411..6bdae92 100755
--- a/tests/qemu-iotests/052
+++ b/tests/qemu-iotests/052
@@ -39,7 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 . ./common.filter
 
 _supported_fmt generic
-_supported_proto generic
+_supported_proto file
 _supported_os Linux
 _default_cache_mode "writethrough"
 _supported_cache_modes "writethrough"
diff --git a/tests/qemu-iotests/054 b/tests/qemu-iotests/054
index 5a0d1b1..c8b7082 100755
--- a/tests/qemu-iotests/054
+++ b/tests/qemu-iotests/054
@@ -39,7 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 . ./common.filter
 
 _supported_fmt qcow2
-_supported_proto generic
+_supported_proto file
 _supported_os Linux
 
 echo
diff --git a/tests/qemu-iotests/059 b/tests/qemu-iotests/059
index 2d604d3..ca5aa16 100755
--- a/tests/qemu-iotests/059
+++ b/tests/qemu-iotests/059
@@ -40,7 +40,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 # This tests vmdk-specific low-level functionality
 _supported_fmt vmdk
-_supported_proto generic
+_supported_proto file
 _supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
diff --git a/tests/qemu-iotests/060 b/tests/qemu-iotests/060
index bbb1909..af8ed9f 100755
--- a/tests/qemu-iotests/060
+++ b/tests/qemu-iotests/060
@@ -40,7 +40,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 # This tests qocw2-specific low-level functionality
 _supported_fmt qcow2
-_supported_proto generic
+_supported_proto file
 _supported_os Linux
 
 rt_offset=65536  # 0x10000 (XXX: just an assumption)
diff --git a/tests/qemu-iotests/061 b/tests/qemu-iotests/061
index e42f9bd..d3a6b38 100755
--- a/tests/qemu-iotests/061
+++ b/tests/qemu-iotests/061
@@ -40,7 +40,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 # This tests qocw2-specific low-level functionality
 _supported_fmt qcow2
-_supported_proto generic
+_supported_proto file
 _supported_os Linux
 
 echo
diff --git a/tests/qemu-iotests/063 b/tests/qemu-iotests/063
index 77503a2..a47493a 100755
--- a/tests/qemu-iotests/063
+++ b/tests/qemu-iotests/063
@@ -42,7 +42,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 . ./common.pattern
 
 _supported_fmt qcow qcow2 vmdk qed raw
-_supported_proto generic
+_supported_proto file
 _supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
diff --git a/tests/qemu-iotests/069 b/tests/qemu-iotests/069
index 50347d9..e661598 100755
--- a/tests/qemu-iotests/069
+++ b/tests/qemu-iotests/069
@@ -39,7 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 . ./common.filter
 
 _supported_fmt cow qed qcow qcow2 vmdk
-_supported_proto generic
+_supported_proto file
 _supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
 
diff --git a/tests/qemu-iotests/071 b/tests/qemu-iotests/071
index 2a22546..51edac1 100755
--- a/tests/qemu-iotests/071
+++ b/tests/qemu-iotests/071
@@ -39,7 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 . ./common.filter
 
 _supported_fmt generic
-_supported_proto generic
+_supported_proto file
 _supported_os Linux
 
 function do_run_qemu()
diff --git a/tests/qemu-iotests/072 b/tests/qemu-iotests/072
index a3876c2..58faa8b 100755
--- a/tests/qemu-iotests/072
+++ b/tests/qemu-iotests/072
@@ -39,7 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 . ./common.filter
 
 _supported_fmt vpc vmdk vhdx vdi qed qcow2 qcow cow
-_supported_proto generic
+_supported_proto file
 _supported_os Linux
 
 IMG_SIZE=64M
-- 
1.7.9.5

[Qemu-devel] [PATCHv9 3/5] qemu-iotests: blacklist test 020 for NFS protocol

[Peter Lieven]

reopening is currently not supported.

Signed-off-by: Peter Lieven <pl@kamp.de>
---
 tests/qemu-iotests/020 |    5 +++++
 1 file changed, 5 insertions(+)

diff --git a/tests/qemu-iotests/020 b/tests/qemu-iotests/020
index 73a0429..2f258dc 100755
--- a/tests/qemu-iotests/020
+++ b/tests/qemu-iotests/020
@@ -49,6 +49,11 @@ _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
                      "subformat=twoGbMaxExtentSparse"
 
+# NFS does not support bdrv_reopen_prepare thus qemu-img commit fails.
+if [ "$IMGPROTO" = "nfs" ]; then
+    _notrun "image protocol $IMGPROTO does not support bdrv_commit"
+fi
+
 TEST_OFFSETS="0 4294967296"
 
 _make_test_img 6G
-- 
1.7.9.5

[Qemu-devel] [PATCHv9 4/5] qemu-iotests: enable test 016 and 025 to work with NFS protocol

[Peter Lieven]

Signed-off-by: Peter Lieven <pl@kamp.de>
---
 tests/qemu-iotests/016 |    2 +-
 tests/qemu-iotests/025 |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/qemu-iotests/016 b/tests/qemu-iotests/016
index b87a32b..7ea9e94 100755
--- a/tests/qemu-iotests/016
+++ b/tests/qemu-iotests/016
@@ -39,7 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 . ./common.filter
 
 _supported_fmt raw
-_supported_proto file sheepdog
+_supported_proto file sheepdog nfs
 _supported_os Linux
 
 
diff --git a/tests/qemu-iotests/025 b/tests/qemu-iotests/025
index 9426c93..a5f45b4 100755
--- a/tests/qemu-iotests/025
+++ b/tests/qemu-iotests/025
@@ -40,7 +40,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 . ./common.pattern
 
 _supported_fmt raw qcow2 qed
-_supported_proto file sheepdog rbd
+_supported_proto file sheepdog rbd nfs
 _supported_os Linux
 
 echo "=== Creating image"
-- 
1.7.9.5

[Qemu-devel] [PATCHv9 5/5] qemu-iotests: enable support for NFS protocol

[Peter Lieven]

Signed-off-by: Peter Lieven <pl@kamp.de>
---
 tests/qemu-iotests/common    |   22 +++++++++++++++++++---
 tests/qemu-iotests/common.rc |    3 +++
 2 files changed, 22 insertions(+), 3 deletions(-)

diff --git a/tests/qemu-iotests/common b/tests/qemu-iotests/common
index 8b4e22c..5795358 100644
--- a/tests/qemu-iotests/common
+++ b/tests/qemu-iotests/common
@@ -144,10 +144,12 @@ check options
     -vpc                test vpc
     -vhdx               test vhdx
     -vmdk               test vmdk
+    -file               test file (default)
     -rbd                test rbd
     -sheepdog           test sheepdog
     -nbd                test nbd
     -ssh                test ssh
+    -nfs                test nfs
     -xdiff              graphical mode diff
     -nocache            use O_DIRECT on backing file
     -misalign           misalign memory allocations
@@ -211,22 +213,36 @@ testlist options
             xpand=false
             ;;
 
+        -file)
+            IMGPROTO=file
+            xpand=false
+            ;;
+
         -rbd)
             IMGPROTO=rbd
             xpand=false
             ;;
+
         -sheepdog)
             IMGPROTO=sheepdog
             xpand=false
             ;;
+
         -nbd)
             IMGPROTO=nbd
             xpand=false
             ;;
+
         -ssh)
             IMGPROTO=ssh
             xpand=false
             ;;
+
+        -nfs)
+            IMGPROTO=nfs
+            xpand=false
+            ;;
+
         -nocache)
             CACHEMODE="none"
             CACHEMODE_IS_DEFAULT=false
@@ -238,10 +254,10 @@ testlist options
             xpand=false
             ;;
 
-    -valgrind)
-        valgrind=true
+        -valgrind)
+            valgrind=true
             xpand=false
-        ;;
+            ;;
 
         -g)        # -g group ... pick from group file
             group=true
diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
index 0f68156..71e9a74 100644
--- a/tests/qemu-iotests/common.rc
+++ b/tests/qemu-iotests/common.rc
@@ -61,6 +61,9 @@ elif [ "$IMGPROTO" = "nbd" ]; then
 elif [ "$IMGPROTO" = "ssh" ]; then
     TEST_IMG_FILE=$TEST_DIR/t.$IMGFMT
     TEST_IMG="ssh://127.0.0.1$TEST_IMG_FILE"
+elif [ "$IMGPROTO" = "nfs" ]; then
+    TEST_DIR="nfs://127.0.0.1/$TEST_DIR"
+    TEST_IMG=$TEST_DIR/t.$IMGFMT
 else
     TEST_IMG=$IMGPROTO:$TEST_DIR/t.$IMGFMT
 fi
-- 
1.7.9.5

Re: [Qemu-devel] [PATCHv9 1/5] block: add native support for NFS

[Kevin Wolf]

Am 03.02.2014 um 10:26 hat Peter Lieven geschrieben:
> This patch adds native support for accessing images on NFS
> shares without the requirement to actually mount the entire
> NFS share on the host.
> 
> NFS Images can simply be specified by an url of the form:
> nfs://<host>/<export>/<filename>[?param=value[&param2=value2[&...]]]
> 
> For example:
> qemu-img create -f qcow2 nfs://10.0.0.1/qemu-images/test.qcow2
> 
> You need LibNFS from Ronnie Sahlberg available at:
>    git://github.com/sahlberg/libnfs.git
> for this to work.
> 
> During configure it is automatically probed for libnfs and support
> is enabled on-the-fly. You can forbid or enforce libnfs support
> with --disable-libnfs or --enable-libnfs respectively.
> 
> Due to NFS restrictions you might need to execute your binaries
> as root, allow them to open priviledged ports (<1024) or specify
> insecure option on the NFS server.
> 
> For additional information on ROOT vs. non-ROOT operation and URL
> format + parameters see:
>    https://raw.github.com/sahlberg/libnfs/master/README
> 
> Supported by qemu are the uid, gid and tcp-syncnt URL parameters.
> 
> LibNFS currently support NFS version 3 only.
> 
> Signed-off-by: Peter Lieven <pl@kamp.de>

Some minor comments in case you need to respin for another reason:

>  MAINTAINERS         |    5 +
>  block/Makefile.objs |    1 +
>  block/nfs.c         |  439 +++++++++++++++++++++++++++++++++++++++++++++++++++
>  configure           |   26 +++
>  qapi-schema.json    |    1 +
>  5 files changed, 472 insertions(+)
>  create mode 100644 block/nfs.c

> +typedef struct NFSRPC {
> +    int ret;
> +    int complete;

Should be bool.

> +    QEMUIOVector *iov;
> +    struct stat *st;
> +    Coroutine *co;
> +    QEMUBH *bh;
> +} NFSRPC;

> +static BlockDriver bdrv_nfs = {
> +    .format_name     = "nfs",
> +    .protocol_name   = "nfs",
> +
> +    .instance_size   = sizeof(NFSClient),
> +    .bdrv_needs_filename = true,
> +    .bdrv_has_zero_init = nfs_has_zero_init,
> +    .bdrv_get_allocated_file_size = nfs_get_allocated_file_size,
> +    .bdrv_truncate = nfs_file_truncate,

This group of options could have used alignment for = as well. :-)

> +    .bdrv_file_open  = nfs_file_open,
> +    .bdrv_close      = nfs_file_close,
> +    .bdrv_create     = nfs_file_create,
> +
> +    .bdrv_co_readv         = nfs_co_readv,
> +    .bdrv_co_writev        = nfs_co_writev,
> +    .bdrv_co_flush_to_disk = nfs_co_flush,
> +};

Kevin

Re: [Qemu-devel] [PATCHv9 0/5] block: add native support for NFS

[Kevin Wolf]

Am 03.02.2014 um 10:26 hat Peter Lieven geschrieben:
> This adds v9 of the NFS protocol driver + qemu-iotest adjustments.

Thanks, applied to the block branch.

Kevin