[Qemu-devel] [PULL 0/8] NBD patches through 26 Jan

[Qemu-devel] [PULL 0/8] NBD patches through 26 Jan

[Eric Blake]

The following changes since commit e607bbee553cfe73072870cef458cfa4e78133e2:

  Merge remote-tracking branch 'remotes/edgar/tags/edgar/xilinx-next-2018-01-26.for-upstream' into staging (2018-01-26 14:24:25 +0000)

are available in the Git repository at:

  git://repo.or.cz/qemu/ericb.git tags/pull-nbd-2018-01-26

for you to fetch changes up to 9776f0db6a19a0510e89b7aae38190b4811c95ba:

  nbd: implement bdrv_get_info callback (2018-01-26 09:58:46 -0600)

----------------------------------------------------------------
nbd patches for 2018-01-26

- Vladimir Sementsov-Ogievskiy - nbd export qmp interface
- Eric Blake - hmp: Add nbd_server_remove to mirror QMP command
- Edgar Kaziakhmedov - nbd: implement bdrv_get_info callback

----------------------------------------------------------------
Edgar Kaziakhmedov (1):
      nbd: implement bdrv_get_info callback

Eric Blake (2):
      hmp: Add name parameter to nbd_server_add
      hmp: Add nbd_server_remove to mirror QMP command

Vladimir Sementsov-Ogievskiy (5):
      qapi: add name parameter to nbd-server-add
      qapi: add nbd-server-remove
      iotest 147: add cases to test new @name parameter of nbd-server-add
      iotests: implement QemuIoInteractive class
      iotest 205: new test for qmp nbd-server-remove

 qapi/block.json               |  50 +++++++++++++-
 hmp.h                         |   1 +
 include/block/nbd.h           |   1 +
 block/nbd.c                   |  11 +++
 blockdev-nbd.c                |  38 ++++++++--
 hmp.c                         |  20 ++++--
 nbd/server.c                  |  13 ++++
 hmp-commands.hx               |  26 +++++--
 tests/qemu-iotests/147        |  68 ++++++++++++++----
 tests/qemu-iotests/147.out    |   4 +-
 tests/qemu-iotests/205        | 156 ++++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/205.out    |   5 ++
 tests/qemu-iotests/group      |   1 +
 tests/qemu-iotests/iotests.py |  38 ++++++++++
 14 files changed, 401 insertions(+), 31 deletions(-)
 create mode 100644 tests/qemu-iotests/205
 create mode 100644 tests/qemu-iotests/205.out

-- 
2.14.3

[Qemu-devel] [PULL 1/8] qapi: add name parameter to nbd-server-add

[Eric Blake]

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Allow user to specify name for new export, to not reuse internal
node name and to not show it to clients.

This also allows creating several exports per device.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180119135719.24745-2-vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 qapi/block.json |  9 +++++++--
 blockdev-nbd.c  | 14 +++++++++-----
 hmp.c           |  5 +++--
 3 files changed, 19 insertions(+), 9 deletions(-)

diff --git a/qapi/block.json b/qapi/block.json
index f093fa3f276..353e3a45bdf 100644
--- a/qapi/block.json
+++ b/qapi/block.json
@@ -213,14 +213,19 @@
 #
 # @device: The device name or node name of the node to be exported
 #
+# @name: Export name. If unspecified, the @device parameter is used as the
+#        export name. (Since 2.12)
+#
 # @writable: Whether clients should be able to write to the device via the
 #     NBD connection (default false).
 #
-# Returns: error if the device is already marked for export.
+# Returns: error if the server is not running, or export with the same name
+#          already exists.
 #
 # Since: 1.3.0
 ##
-{ 'command': 'nbd-server-add', 'data': {'device': 'str', '*writable': 'bool'} }
+{ 'command': 'nbd-server-add',
+  'data': {'device': 'str', '*name': 'str', '*writable': 'bool'} }

 ##
 # @nbd-server-stop:
diff --git a/blockdev-nbd.c b/blockdev-nbd.c
index 9e3c22109c6..104789e521c 100644
--- a/blockdev-nbd.c
+++ b/blockdev-nbd.c
@@ -140,8 +140,8 @@ void qmp_nbd_server_start(SocketAddressLegacy *addr,
     qapi_free_SocketAddress(addr_flat);
 }

-void qmp_nbd_server_add(const char *device, bool has_writable, bool writable,
-                        Error **errp)
+void qmp_nbd_server_add(const char *device, bool has_name, const char *name,
+                        bool has_writable, bool writable, Error **errp)
 {
     BlockDriverState *bs = NULL;
     BlockBackend *on_eject_blk;
@@ -152,8 +152,12 @@ void qmp_nbd_server_add(const char *device, bool has_writable, bool writable,
         return;
     }

-    if (nbd_export_find(device)) {
-        error_setg(errp, "NBD server already exporting device '%s'", device);
+    if (!has_name) {
+        name = device;
+    }
+
+    if (nbd_export_find(name)) {
+        error_setg(errp, "NBD server already has export named '%s'", name);
         return;
     }

@@ -177,7 +181,7 @@ void qmp_nbd_server_add(const char *device, bool has_writable, bool writable,
         return;
     }

-    nbd_export_set_name(exp, device);
+    nbd_export_set_name(exp, name);

     /* The list of named exports has a strong reference to this export now and
      * our only way of accessing it is through nbd_export_find(), so we can drop
diff --git a/hmp.c b/hmp.c
index 056bf70cf1e..5bcfc36de10 100644
--- a/hmp.c
+++ b/hmp.c
@@ -2203,7 +2203,8 @@ void hmp_nbd_server_start(Monitor *mon, const QDict *qdict)
             continue;
         }

-        qmp_nbd_server_add(info->value->device, true, writable, &local_err);
+        qmp_nbd_server_add(info->value->device, false, NULL,
+                           true, writable, &local_err);

         if (local_err != NULL) {
             qmp_nbd_server_stop(NULL);
@@ -2223,7 +2224,7 @@ void hmp_nbd_server_add(Monitor *mon, const QDict *qdict)
     bool writable = qdict_get_try_bool(qdict, "writable", false);
     Error *local_err = NULL;

-    qmp_nbd_server_add(device, true, writable, &local_err);
+    qmp_nbd_server_add(device, false, NULL, true, writable, &local_err);

     if (local_err != NULL) {
         hmp_handle_error(mon, &local_err);
-- 
2.14.3

[Qemu-devel] [PULL 2/8] hmp: Add name parameter to nbd_server_add

[Eric Blake]

Extend the flexibility of the previous QMP patch to also work
in HMP.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20180109192802.17167-1-eblake@redhat.com>
---
 hmp.c           | 3 ++-
 hmp-commands.hx | 9 +++++----
 2 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/hmp.c b/hmp.c
index 5bcfc36de10..7a64dd59c5c 100644
--- a/hmp.c
+++ b/hmp.c
@@ -2221,10 +2221,11 @@ exit:
 void hmp_nbd_server_add(Monitor *mon, const QDict *qdict)
 {
     const char *device = qdict_get_str(qdict, "device");
+    const char *name = qdict_get_try_str(qdict, "name");
     bool writable = qdict_get_try_bool(qdict, "writable", false);
     Error *local_err = NULL;

-    qmp_nbd_server_add(device, false, NULL, true, writable, &local_err);
+    qmp_nbd_server_add(device, !!name, name, true, writable, &local_err);

     if (local_err != NULL) {
         hmp_handle_error(mon, &local_err);
diff --git a/hmp-commands.hx b/hmp-commands.hx
index 6d5ebdf6ab1..b8b6fb91848 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -1553,17 +1553,18 @@ ETEXI

     {
         .name       = "nbd_server_add",
-        .args_type  = "writable:-w,device:B",
-        .params     = "nbd_server_add [-w] device",
+        .args_type  = "writable:-w,device:B,name:s?",
+        .params     = "nbd_server_add [-w] device [name]",
         .help       = "export a block device via NBD",
         .cmd        = hmp_nbd_server_add,
     },
 STEXI
-@item nbd_server_add @var{device}
+@item nbd_server_add @var{device} [ @var{name} ]
 @findex nbd_server_add
 Export a block device through QEMU's NBD server, which must be started
 beforehand with @command{nbd_server_start}.  The @option{-w} option makes the
-exported device writable too.
+exported device writable too.  The export name is controlled by @var{name},
+defaulting to @var{device}.
 ETEXI

     {
-- 
2.14.3

[Qemu-devel] [PULL 3/8] qapi: add nbd-server-remove

[Eric Blake]

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Add command for removing an export. It is needed for cases when we
don't want to keep the export after the operation on it was completed.
The other example is a temporary node, created with blockdev-add.
If we want to delete it we should firstly remove any corresponding
NBD export.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20180119135719.24745-3-vsementsov@virtuozzo.com>
[eblake: drop dead nb_clients code]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 qapi/block.json     | 41 +++++++++++++++++++++++++++++++++++++++++
 include/block/nbd.h |  1 +
 blockdev-nbd.c      | 24 ++++++++++++++++++++++++
 nbd/server.c        | 13 +++++++++++++
 4 files changed, 79 insertions(+)

diff --git a/qapi/block.json b/qapi/block.json
index 353e3a45bdf..c6945240029 100644
--- a/qapi/block.json
+++ b/qapi/block.json
@@ -227,6 +227,47 @@
 { 'command': 'nbd-server-add',
   'data': {'device': 'str', '*name': 'str', '*writable': 'bool'} }

+##
+# @NbdServerRemoveMode:
+#
+# Mode for removing an NBD export.
+#
+# @safe: Remove export if there are no existing connections, fail otherwise.
+#
+# @hard: Drop all connections immediately and remove export.
+#
+# Potential additional modes to be added in the future:
+#
+# hide: Just hide export from new clients, leave existing connections as is.
+#       Remove export after all clients are disconnected.
+#
+# soft: Hide export from new clients, answer with ESHUTDOWN for all further
+#       requests from existing clients.
+#
+# Since: 2.12
+##
+{'enum': 'NbdServerRemoveMode', 'data': ['safe', 'hard']}
+
+##
+# @nbd-server-remove:
+#
+# Remove NBD export by name.
+#
+# @name: Export name.
+#
+# @mode: Mode of command operation. See @NbdServerRemoveMode description.
+#        Default is 'safe'.
+#
+# Returns: error if
+#            - the server is not running
+#            - export is not found
+#            - mode is 'safe' and there are existing connections
+#
+# Since: 2.12
+##
+{ 'command': 'nbd-server-remove',
+  'data': {'name': 'str', '*mode': 'NbdServerRemoveMode'} }
+
 ##
 # @nbd-server-stop:
 #
diff --git a/include/block/nbd.h b/include/block/nbd.h
index 978e4433664..ee74ec391a8 100644
--- a/include/block/nbd.h
+++ b/include/block/nbd.h
@@ -261,6 +261,7 @@ NBDExport *nbd_export_new(BlockDriverState *bs, off_t dev_offset, off_t size,
                           bool writethrough, BlockBackend *on_eject_blk,
                           Error **errp);
 void nbd_export_close(NBDExport *exp);
+void nbd_export_remove(NBDExport *exp, NbdServerRemoveMode mode, Error **errp);
 void nbd_export_get(NBDExport *exp);
 void nbd_export_put(NBDExport *exp);

diff --git a/blockdev-nbd.c b/blockdev-nbd.c
index 104789e521c..a9f79c6778e 100644
--- a/blockdev-nbd.c
+++ b/blockdev-nbd.c
@@ -189,6 +189,30 @@ void qmp_nbd_server_add(const char *device, bool has_name, const char *name,
     nbd_export_put(exp);
 }

+void qmp_nbd_server_remove(const char *name,
+                           bool has_mode, NbdServerRemoveMode mode,
+                           Error **errp)
+{
+    NBDExport *exp;
+
+    if (!nbd_server) {
+        error_setg(errp, "NBD server not running");
+        return;
+    }
+
+    exp = nbd_export_find(name);
+    if (exp == NULL) {
+        error_setg(errp, "Export '%s' is not found", name);
+        return;
+    }
+
+    if (!has_mode) {
+        mode = NBD_SERVER_REMOVE_MODE_SAFE;
+    }
+
+    nbd_export_remove(exp, mode, errp);
+}
+
 void qmp_nbd_server_stop(Error **errp)
 {
     nbd_export_close_all();
diff --git a/nbd/server.c b/nbd/server.c
index 6caa8d17be7..112e3f69dff 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -1177,6 +1177,19 @@ void nbd_export_close(NBDExport *exp)
     nbd_export_put(exp);
 }

+void nbd_export_remove(NBDExport *exp, NbdServerRemoveMode mode, Error **errp)
+{
+    if (mode == NBD_SERVER_REMOVE_MODE_HARD || QTAILQ_EMPTY(&exp->clients)) {
+        nbd_export_close(exp);
+        return;
+    }
+
+    assert(mode == NBD_SERVER_REMOVE_MODE_SAFE);
+
+    error_setg(errp, "export '%s' still in use", exp->name);
+    error_append_hint(errp, "Use mode='hard' to force client disconnect\n");
+}
+
 void nbd_export_get(NBDExport *exp)
 {
     assert(exp->refcount > 0);
-- 
2.14.3

[Qemu-devel] [PULL 4/8] iotest 147: add cases to test new @name parameter of nbd-server-add

[Eric Blake]

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180119135719.24745-4-vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 tests/qemu-iotests/147     | 68 +++++++++++++++++++++++++++++++++++++---------
 tests/qemu-iotests/147.out |  4 +--
 2 files changed, 57 insertions(+), 15 deletions(-)

diff --git a/tests/qemu-iotests/147 b/tests/qemu-iotests/147
index 90f40ed2457..d2081df84b0 100755
--- a/tests/qemu-iotests/147
+++ b/tests/qemu-iotests/147
@@ -38,8 +38,8 @@ def flatten_sock_addr(crumpled_address):


 class NBDBlockdevAddBase(iotests.QMPTestCase):
-    def blockdev_add_options(self, address, export=None):
-        options = { 'node-name': 'nbd-blockdev',
+    def blockdev_add_options(self, address, export, node_name):
+        options = { 'node-name': node_name,
                     'driver': 'raw',
                     'file': {
                         'driver': 'nbd',
@@ -50,23 +50,28 @@ class NBDBlockdevAddBase(iotests.QMPTestCase):
             options['file']['export'] = export
         return options

-    def client_test(self, filename, address, export=None):
-        bao = self.blockdev_add_options(address, export)
+    def client_test(self, filename, address, export=None,
+                    node_name='nbd-blockdev', delete=True):
+        bao = self.blockdev_add_options(address, export, node_name)
         result = self.vm.qmp('blockdev-add', **bao)
         self.assert_qmp(result, 'return', {})

+        found = False
         result = self.vm.qmp('query-named-block-nodes')
         for node in result['return']:
-            if node['node-name'] == 'nbd-blockdev':
+            if node['node-name'] == node_name:
+                found = True
                 if isinstance(filename, str):
                     self.assert_qmp(node, 'image/filename', filename)
                 else:
                     self.assert_json_filename_equal(node['image']['filename'],
                                                     filename)
                 break
+        self.assertTrue(found)

-        result = self.vm.qmp('blockdev-del', node_name='nbd-blockdev')
-        self.assert_qmp(result, 'return', {})
+        if delete:
+            result = self.vm.qmp('blockdev-del', node_name=node_name)
+            self.assert_qmp(result, 'return', {})


 class QemuNBD(NBDBlockdevAddBase):
@@ -125,26 +130,63 @@ class BuiltinNBD(NBDBlockdevAddBase):
         except OSError:
             pass

-    def _server_up(self, address):
+    def _server_up(self, address, export_name=None, export_name2=None):
         result = self.server.qmp('nbd-server-start', addr=address)
         self.assert_qmp(result, 'return', {})

-        result = self.server.qmp('nbd-server-add', device='nbd-export')
+        if export_name is None:
+            result = self.server.qmp('nbd-server-add', device='nbd-export')
+        else:
+            result = self.server.qmp('nbd-server-add', device='nbd-export',
+                                     name=export_name)
         self.assert_qmp(result, 'return', {})

+        if export_name2 is not None:
+            result = self.server.qmp('nbd-server-add', device='nbd-export',
+                                     name=export_name2)
+            self.assert_qmp(result, 'return', {})
+
+
     def _server_down(self):
         result = self.server.qmp('nbd-server-stop')
         self.assert_qmp(result, 'return', {})

-    def test_inet(self):
+    def do_test_inet(self, export_name=None):
         address = { 'type': 'inet',
                     'data': {
                         'host': 'localhost',
                         'port': str(NBD_PORT)
                     } }
-        self._server_up(address)
-        self.client_test('nbd://localhost:%i/nbd-export' % NBD_PORT,
-                         flatten_sock_addr(address), 'nbd-export')
+        self._server_up(address, export_name)
+        export_name = export_name or 'nbd-export'
+        self.client_test('nbd://localhost:%i/%s' % (NBD_PORT, export_name),
+                         flatten_sock_addr(address), export_name)
+        self._server_down()
+
+    def test_inet_default_export_name(self):
+        self.do_test_inet()
+
+    def test_inet_same_export_name(self):
+        self.do_test_inet('nbd-export')
+
+    def test_inet_different_export_name(self):
+        self.do_test_inet('shadow')
+
+    def test_inet_two_exports(self):
+        address = { 'type': 'inet',
+                    'data': {
+                        'host': 'localhost',
+                        'port': str(NBD_PORT)
+                    } }
+        self._server_up(address, 'exp1', 'exp2')
+        self.client_test('nbd://localhost:%i/%s' % (NBD_PORT, 'exp1'),
+                         flatten_sock_addr(address), 'exp1', 'node1', False)
+        self.client_test('nbd://localhost:%i/%s' % (NBD_PORT, 'exp2'),
+                         flatten_sock_addr(address), 'exp2', 'node2', False)
+        result = self.vm.qmp('blockdev-del', node_name='node1')
+        self.assert_qmp(result, 'return', {})
+        result = self.vm.qmp('blockdev-del', node_name='node2')
+        self.assert_qmp(result, 'return', {})
         self._server_down()

     def test_inet6(self):
diff --git a/tests/qemu-iotests/147.out b/tests/qemu-iotests/147.out
index 3f8a935a082..dae404e2786 100644
--- a/tests/qemu-iotests/147.out
+++ b/tests/qemu-iotests/147.out
@@ -1,5 +1,5 @@
-......
+.........
 ----------------------------------------------------------------------
-Ran 6 tests
+Ran 9 tests

 OK
-- 
2.14.3

[Qemu-devel] [PULL 5/8] iotests: implement QemuIoInteractive class

[Eric Blake]

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Implement QemuIoInteractive to test nbd-server-remove command when
there are active connections.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20180119135719.24745-5-vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 tests/qemu-iotests/iotests.py | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 44477e92958..5a10b2d5345 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -93,6 +93,44 @@ def qemu_io(*args):
         sys.stderr.write('qemu-io received signal %i: %s\n' % (-exitcode, ' '.join(args)))
     return subp.communicate()[0]

+
+class QemuIoInteractive:
+    def __init__(self, *args):
+        self.args = qemu_io_args + list(args)
+        self._p = subprocess.Popen(self.args, stdin=subprocess.PIPE,
+                                   stdout=subprocess.PIPE,
+                                   stderr=subprocess.STDOUT)
+        assert self._p.stdout.read(9) == 'qemu-io> '
+
+    def close(self):
+        self._p.communicate('q\n')
+
+    def _read_output(self):
+        pattern = 'qemu-io> '
+        n = len(pattern)
+        pos = 0
+        s = []
+        while pos != n:
+            c = self._p.stdout.read(1)
+            # check unexpected EOF
+            assert c != ''
+            s.append(c)
+            if c == pattern[pos]:
+                pos += 1
+            else:
+                pos = 0
+
+        return ''.join(s[:-n])
+
+    def cmd(self, cmd):
+        # quit command is in close(), '\n' is added automatically
+        assert '\n' not in cmd
+        cmd = cmd.strip()
+        assert cmd != 'q' and cmd != 'quit'
+        self._p.stdin.write(cmd + '\n')
+        return self._read_output()
+
+
 def qemu_nbd(*args):
     '''Run qemu-nbd in daemon mode and return the parent's exit code'''
     return subprocess.call(qemu_nbd_args + ['--fork'] + list(args))
-- 
2.14.3

[Qemu-devel] [PULL 6/8] iotest 205: new test for qmp nbd-server-remove

[Eric Blake]

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20180119135719.24745-6-vsementsov@virtuozzo.com>
[eblake: adjust to next available test number]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 tests/qemu-iotests/205     | 156 +++++++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/205.out |   5 ++
 tests/qemu-iotests/group   |   1 +
 3 files changed, 162 insertions(+)
 create mode 100644 tests/qemu-iotests/205
 create mode 100644 tests/qemu-iotests/205.out

diff --git a/tests/qemu-iotests/205 b/tests/qemu-iotests/205
new file mode 100644
index 00000000000..10388920dc3
--- /dev/null
+++ b/tests/qemu-iotests/205
@@ -0,0 +1,156 @@
+#!/usr/bin/env python
+#
+# Tests for qmp command nbd-server-remove.
+#
+# Copyright (c) 2017 Virtuozzo International GmbH
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import os
+import sys
+import iotests
+import time
+from iotests import qemu_img, qemu_io, filter_qemu_io, QemuIoInteractive
+
+nbd_sock = 'nbd_sock'
+nbd_uri = 'nbd+unix:///exp?socket=' + nbd_sock
+disk = os.path.join(iotests.test_dir, 'disk')
+
+
+class TestNbdServerRemove(iotests.QMPTestCase):
+    def setUp(self):
+        qemu_img('create', '-f', iotests.imgfmt, disk, '1M')
+
+        self.vm = iotests.VM().add_drive(disk)
+        self.vm.launch()
+
+        address = {
+            'type': 'unix',
+            'data': {
+                'path': nbd_sock
+            }
+        }
+
+        result = self.vm.qmp('nbd-server-start', addr=address)
+        self.assert_qmp(result, 'return', {})
+        result = self.vm.qmp('nbd-server-add', device='drive0', name='exp')
+        self.assert_qmp(result, 'return', {})
+
+    def tearDown(self):
+        self.vm.shutdown()
+        os.remove(nbd_sock)
+        os.remove(disk)
+
+    def remove_export(self, name, mode=None):
+        if mode is None:
+            return self.vm.qmp('nbd-server-remove', name=name)
+        else:
+            return self.vm.qmp('nbd-server-remove', name=name, mode=mode)
+
+    def assertExportNotFound(self, name):
+        result = self.vm.qmp('nbd-server-remove', name=name)
+        self.assert_qmp(result, 'error/desc', "Export 'exp' is not found")
+
+    def assertExistingClients(self, result):
+        self.assert_qmp(result, 'error/desc', "export 'exp' still in use")
+
+    def assertReadOk(self, qemu_io_output):
+        self.assertEqual(
+                filter_qemu_io(qemu_io_output).strip(),
+                'read 512/512 bytes at offset 0\n' +
+                '512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)')
+
+    def assertReadFailed(self, qemu_io_output):
+        self.assertEqual(filter_qemu_io(qemu_io_output).strip(),
+                         'read failed: Input/output error')
+
+    def assertConnectFailed(self, qemu_io_output):
+        self.assertEqual(filter_qemu_io(qemu_io_output).strip(),
+                         "can't open device " + nbd_uri +
+                         ": Requested export not available\n"
+                         "server reported: export 'exp' not present")
+
+    def do_test_connect_after_remove(self, mode=None):
+        args = ('-r', '-f', 'raw', '-c', 'read 0 512', nbd_uri)
+        self.assertReadOk(qemu_io(*args))
+
+        result = self.remove_export('exp', mode)
+        self.assert_qmp(result, 'return', {})
+
+        self.assertExportNotFound('exp')
+        self.assertConnectFailed(qemu_io(*args))
+
+    def test_connect_after_remove_default(self):
+        self.do_test_connect_after_remove()
+
+    def test_connect_after_remove_safe(self):
+        self.do_test_connect_after_remove('safe')
+
+    def test_connect_after_remove_force(self):
+        self.do_test_connect_after_remove('hard')
+
+    def do_test_remove_during_connect_safe(self, mode=None):
+        qio = QemuIoInteractive('-r', '-f', 'raw', nbd_uri)
+        self.assertReadOk(qio.cmd('read 0 512'))
+
+        result = self.remove_export('exp', mode)
+        self.assertExistingClients(result)
+
+        self.assertReadOk(qio.cmd('read 0 512'))
+
+        qio.close()
+
+        result = self.remove_export('exp', mode)
+        self.assert_qmp(result, 'return', {})
+
+        self.assertExportNotFound('exp')
+
+    def test_remove_during_connect_default(self):
+        self.do_test_remove_during_connect_safe()
+
+    def test_remove_during_connect_safe(self):
+        self.do_test_remove_during_connect_safe('safe')
+
+    def test_remove_during_connect_hard(self):
+        qio = QemuIoInteractive('-r', '-f', 'raw', nbd_uri)
+        self.assertReadOk(qio.cmd('read 0 512'))
+
+        result = self.remove_export('exp', 'hard')
+        self.assert_qmp(result, 'return', {})
+
+        self.assertReadFailed(qio.cmd('read 0 512'))
+        self.assertExportNotFound('exp')
+
+        qio.close()
+
+    def test_remove_during_connect_safe_hard(self):
+        qio = QemuIoInteractive('-r', '-f', 'raw', nbd_uri)
+        self.assertReadOk(qio.cmd('read 0 512'))
+
+        result = self.remove_export('exp', 'safe')
+        self.assertExistingClients(result)
+
+        self.assertReadOk(qio.cmd('read 0 512'))
+
+        result = self.remove_export('exp', 'hard')
+        self.assert_qmp(result, 'return', {})
+
+        self.assertExportNotFound('exp')
+        self.assertReadFailed(qio.cmd('read 0 512'))
+        qio.close()
+
+
+if __name__ == '__main__':
+    iotests.main()
diff --git a/tests/qemu-iotests/205.out b/tests/qemu-iotests/205.out
new file mode 100644
index 00000000000..2f7d3902f23
--- /dev/null
+++ b/tests/qemu-iotests/205.out
@@ -0,0 +1,5 @@
+.......
+----------------------------------------------------------------------
+Ran 7 tests
+
+OK
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 8fc4f62cca6..a2dfe79d862 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -201,3 +201,4 @@
 202 rw auto quick
 203 rw auto
 204 rw auto quick
+205 rw auto quick
-- 
2.14.3

[Qemu-devel] [PULL 7/8] hmp: Add nbd_server_remove to mirror QMP command

[Eric Blake]

Since everything else about the nbd-server-* QMP commands is
accessible from HMP, we might as well make removing an export
available as well.  For now, I went with a bool flag rather
than a mode string for choosing between safe (default) and
hard modes.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180125144557.25502-1-eblake@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 hmp.h           |  1 +
 hmp.c           | 14 +++++++++++---
 hmp-commands.hx | 17 +++++++++++++++++
 3 files changed, 29 insertions(+), 3 deletions(-)

diff --git a/hmp.h b/hmp.h
index a6f56b1f29e..536cb91caa4 100644
--- a/hmp.h
+++ b/hmp.h
@@ -101,6 +101,7 @@ void hmp_sendkey(Monitor *mon, const QDict *qdict);
 void hmp_screendump(Monitor *mon, const QDict *qdict);
 void hmp_nbd_server_start(Monitor *mon, const QDict *qdict);
 void hmp_nbd_server_add(Monitor *mon, const QDict *qdict);
+void hmp_nbd_server_remove(Monitor *mon, const QDict *qdict);
 void hmp_nbd_server_stop(Monitor *mon, const QDict *qdict);
 void hmp_chardev_add(Monitor *mon, const QDict *qdict);
 void hmp_chardev_change(Monitor *mon, const QDict *qdict);
diff --git a/hmp.c b/hmp.c
index 7a64dd59c5c..b3de32d219b 100644
--- a/hmp.c
+++ b/hmp.c
@@ -2226,10 +2226,18 @@ void hmp_nbd_server_add(Monitor *mon, const QDict *qdict)
     Error *local_err = NULL;

     qmp_nbd_server_add(device, !!name, name, true, writable, &local_err);
+    hmp_handle_error(mon, &local_err);
+}

-    if (local_err != NULL) {
-        hmp_handle_error(mon, &local_err);
-    }
+void hmp_nbd_server_remove(Monitor *mon, const QDict *qdict)
+{
+    const char *name = qdict_get_str(qdict, "name");
+    bool force = qdict_get_try_bool(qdict, "force", false);
+    Error *err = NULL;
+
+    /* Rely on NBD_SERVER_REMOVE_MODE_SAFE being the default */
+    qmp_nbd_server_remove(name, force, NBD_SERVER_REMOVE_MODE_HARD, &err);
+    hmp_handle_error(mon, &err);
 }

 void hmp_nbd_server_stop(Monitor *mon, const QDict *qdict)
diff --git a/hmp-commands.hx b/hmp-commands.hx
index b8b6fb91848..c36a9ec4657 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -1565,6 +1565,23 @@ Export a block device through QEMU's NBD server, which must be started
 beforehand with @command{nbd_server_start}.  The @option{-w} option makes the
 exported device writable too.  The export name is controlled by @var{name},
 defaulting to @var{device}.
+ETEXI
+
+    {
+        .name       = "nbd_server_remove",
+        .args_type  = "force:-f,name:s",
+        .params     = "nbd_server_remove [-f] name",
+        .help       = "remove an export previously exposed via NBD",
+        .cmd        = hmp_nbd_server_remove,
+    },
+STEXI
+@item nbd_server_remove [-f] @var{name}
+@findex nbd_server_remove
+Stop exporting a block device through QEMU's NBD server, which was
+previously started with @command{nbd_server_add}.  The @option{-f}
+option forces the server to drop the export immediately even if
+clients are connected; otherwise the command fails unless there are no
+clients.
 ETEXI

     {
-- 
2.14.3

[Qemu-devel] [PULL 8/8] nbd: implement bdrv_get_info callback

[Eric Blake]

From: Edgar Kaziakhmedov <edgar.kaziakhmedov@virtuozzo.com>

Since mirror job supports efficient zero out target mechanism (see
in mirror_dirty_init()), implement bdrv_get_info to make it work
over NBD. Such improvement will allow using the largest chunk possible
and will decrease the number of NBD_CMD_WRITE_ZEROES requests on the wire.

Signed-off-by: Edgar Kaziakhmedov <edgar.kaziakhmedov@virtuozzo.com>
Message-Id: <20180118115158.17219-1-edgar.kaziakhmedov@virtuozzo.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 block/nbd.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/block/nbd.c b/block/nbd.c
index 8b8ba56cdd0..94220f6d143 100644
--- a/block/nbd.c
+++ b/block/nbd.c
@@ -566,6 +566,14 @@ static void nbd_refresh_filename(BlockDriverState *bs, QDict *options)
     bs->full_open_options = opts;
 }

+static int nbd_get_info(BlockDriverState *bs, BlockDriverInfo *bdi)
+{
+    if (bs->supported_zero_flags & BDRV_REQ_MAY_UNMAP) {
+        bdi->can_write_zeroes_with_unmap = true;
+    }
+    return 0;
+}
+
 static BlockDriver bdrv_nbd = {
     .format_name                = "nbd",
     .protocol_name              = "nbd",
@@ -583,6 +591,7 @@ static BlockDriver bdrv_nbd = {
     .bdrv_detach_aio_context    = nbd_detach_aio_context,
     .bdrv_attach_aio_context    = nbd_attach_aio_context,
     .bdrv_refresh_filename      = nbd_refresh_filename,
+    .bdrv_get_info              = nbd_get_info,
 };

 static BlockDriver bdrv_nbd_tcp = {
@@ -602,6 +611,7 @@ static BlockDriver bdrv_nbd_tcp = {
     .bdrv_detach_aio_context    = nbd_detach_aio_context,
     .bdrv_attach_aio_context    = nbd_attach_aio_context,
     .bdrv_refresh_filename      = nbd_refresh_filename,
+    .bdrv_get_info              = nbd_get_info,
 };

 static BlockDriver bdrv_nbd_unix = {
@@ -621,6 +631,7 @@ static BlockDriver bdrv_nbd_unix = {
     .bdrv_detach_aio_context    = nbd_detach_aio_context,
     .bdrv_attach_aio_context    = nbd_attach_aio_context,
     .bdrv_refresh_filename      = nbd_refresh_filename,
+    .bdrv_get_info              = nbd_get_info,
 };

 static void bdrv_nbd_init(void)
-- 
2.14.3

Re: [Qemu-devel] [PULL 0/8] NBD patches through 26 Jan

[Peter Maydell]

On 26 January 2018 at 16:04, Eric Blake <eblake@redhat.com> wrote:
> The following changes since commit e607bbee553cfe73072870cef458cfa4e78133e2:
>
>   Merge remote-tracking branch 'remotes/edgar/tags/edgar/xilinx-next-2018-01-26.for-upstream' into staging (2018-01-26 14:24:25 +0000)
>
> are available in the Git repository at:
>
>   git://repo.or.cz/qemu/ericb.git tags/pull-nbd-2018-01-26
>
> for you to fetch changes up to 9776f0db6a19a0510e89b7aae38190b4811c95ba:
>
>   nbd: implement bdrv_get_info callback (2018-01-26 09:58:46 -0600)
>
> ----------------------------------------------------------------
> nbd patches for 2018-01-26
>
> - Vladimir Sementsov-Ogievskiy - nbd export qmp interface
> - Eric Blake - hmp: Add nbd_server_remove to mirror QMP command
> - Edgar Kaziakhmedov - nbd: implement bdrv_get_info callback

Applied, thanks.

-- PMM

Re: [Qemu-devel] [PULL 6/8] iotest 205: new test for qmp nbd-server-remove

[Kevin Wolf]

Am 26.01.2018 um 17:04 hat Eric Blake geschrieben:
> From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> Message-Id: <20180119135719.24745-6-vsementsov@virtuozzo.com>
> [eblake: adjust to next available test number]
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
>  tests/qemu-iotests/205     | 156 +++++++++++++++++++++++++++++++++++++++++++++
>  tests/qemu-iotests/205.out |   5 ++
>  tests/qemu-iotests/group   |   1 +
>  3 files changed, 162 insertions(+)
>  create mode 100644 tests/qemu-iotests/205
>  create mode 100644 tests/qemu-iotests/205.out

This test case fails for luks. Maybe it's easy enough to enable luks, or
maybe we have to disable it. Not sure how other Python test cases handle
this. Can you have a look, please?

Kevin

Re: [Qemu-devel] [PULL 6/8] iotest 205: new test for qmp nbd-server-remove

[Vladimir Sementsov-Ogievskiy]

05.02.2018 12:24, Kevin Wolf wrote:
> Am 26.01.2018 um 17:04 hat Eric Blake geschrieben:
>> From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> Message-Id: <20180119135719.24745-6-vsementsov@virtuozzo.com>
>> [eblake: adjust to next available test number]
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> ---
>>   tests/qemu-iotests/205     | 156 +++++++++++++++++++++++++++++++++++++++++++++
>>   tests/qemu-iotests/205.out |   5 ++
>>   tests/qemu-iotests/group   |   1 +
>>   3 files changed, 162 insertions(+)
>>   create mode 100644 tests/qemu-iotests/205
>>   create mode 100644 tests/qemu-iotests/205.out
> This test case fails for luks. Maybe it's easy enough to enable luks, or
> maybe we have to disable it. Not sure how other Python test cases handle
> this. Can you have a look, please?
>
> Kevin

Hmm.

check -luks 205
...
+qemu-img: /work/src/qemu/master/tests/qemu-iotests/scratch/disk: 
Parameter 'key-secret' is required for cipher


I've looked through (grep-ped through) tests, and found only one python 
test with key-secret: 149. And it has own qemu_img_create() function.

On the other hand, all python tests, which have "qemu_img('create'" 
pattern, do not support luks. So, it should be easier to disable luks here.
I'll send a patch.

Also, why check-block do not check luks?

-- 
Best regards,
Vladimir