[PATCH 0/2] block: bdrv_reopen() with backing file in different AioContext

[PATCH 0/2] block: bdrv_reopen() with backing file in different AioContext

[Kevin Wolf]

Kevin Wolf (2):
  iotests: Refactor blockdev-reopen test for iothreads
  block: bdrv_reopen() with backing file in different AioContext

 block.c                    | 36 +++++++++++++++++++++++++---------
 tests/qemu-iotests/245     | 40 ++++++++++++++++++++++++++++----------
 tests/qemu-iotests/245.out |  4 ++--
 3 files changed, 59 insertions(+), 21 deletions(-)

-- 
2.20.1

[PATCH 1/2] iotests: Refactor blockdev-reopen test for iothreads

[Kevin Wolf]

We'll want to test more than one successful case in the future, so
prepare the test for that by a refactoring that runs each scenario in a
separate VM.

test_iothreads_switch_{backing,overlay} currently produce errors, but
these are cases that should actually work, by switching either the
backing file node or the overlay node to the AioContext of the other
node.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 tests/qemu-iotests/245     | 42 +++++++++++++++++++++++++++++---------
 tests/qemu-iotests/245.out |  4 ++--
 2 files changed, 34 insertions(+), 12 deletions(-)

diff --git a/tests/qemu-iotests/245 b/tests/qemu-iotests/245
index 489bf78bd0..5a2cd5ed0e 100755
--- a/tests/qemu-iotests/245
+++ b/tests/qemu-iotests/245
@@ -970,8 +970,7 @@ class TestBlockdevReopen(iotests.QMPTestCase):
         self.assertEqual(self.get_node('hd1'), None)
         self.assert_qmp(self.get_node('hd2'), 'ro', True)
 
-    # We don't allow setting a backing file that uses a different AioContext
-    def test_iothreads(self):
+    def run_test_iothreads(self, iothread_a, iothread_b, errmsg = None):
         opts = hd_opts(0)
         result = self.vm.qmp('blockdev-add', conv_keys = False, **opts)
         self.assert_qmp(result, 'return', {})
@@ -986,20 +985,43 @@ class TestBlockdevReopen(iotests.QMPTestCase):
         result = self.vm.qmp('object-add', qom_type='iothread', id='iothread1')
         self.assert_qmp(result, 'return', {})
 
-        result = self.vm.qmp('x-blockdev-set-iothread', node_name='hd0', iothread='iothread0')
+        result = self.vm.qmp('device_add', driver='virtio-scsi', id='scsi0',
+                             iothread=iothread_a)
         self.assert_qmp(result, 'return', {})
 
-        self.reopen(opts, {'backing': 'hd2'}, "Cannot use a new backing file with a different AioContext")
-
-        result = self.vm.qmp('x-blockdev-set-iothread', node_name='hd2', iothread='iothread1')
+        result = self.vm.qmp('device_add', driver='virtio-scsi', id='scsi1',
+                             iothread=iothread_b)
         self.assert_qmp(result, 'return', {})
 
-        self.reopen(opts, {'backing': 'hd2'}, "Cannot use a new backing file with a different AioContext")
+        if iothread_a:
+            result = self.vm.qmp('device_add', driver='scsi-hd', drive='hd0',
+                                 share_rw=True, bus="scsi0.0")
+            self.assert_qmp(result, 'return', {})
 
-        result = self.vm.qmp('x-blockdev-set-iothread', node_name='hd2', iothread='iothread0')
-        self.assert_qmp(result, 'return', {})
+        if iothread_b:
+            result = self.vm.qmp('device_add', driver='scsi-hd', drive='hd2',
+                                 share_rw=True, bus="scsi1.0")
+            self.assert_qmp(result, 'return', {})
 
-        self.reopen(opts, {'backing': 'hd2'})
+        self.reopen(opts, {'backing': 'hd2'}, errmsg)
+        self.vm.shutdown()
+
+    # We don't allow setting a backing file that uses a different AioContext if
+    # neither of them can switch to the other AioContext
+    def test_iothreads_error(self):
+        self.run_test_iothreads('iothread0', 'iothread1',
+                                "Cannot use a new backing file with a different AioContext")
+
+    def test_iothreads_compatible_users(self):
+        self.run_test_iothreads('iothread0', 'iothread0')
+
+    def test_iothreads_switch_backing(self):
+        self.run_test_iothreads('iothread0', None,
+                                "Cannot use a new backing file with a different AioContext")
+
+    def test_iothreads_switch_overlay(self):
+        self.run_test_iothreads(None, 'iothread0',
+                                "Cannot use a new backing file with a different AioContext")
 
 if __name__ == '__main__':
     iotests.main(supported_fmts=["qcow2"],
diff --git a/tests/qemu-iotests/245.out b/tests/qemu-iotests/245.out
index a19de5214d..682b93394d 100644
--- a/tests/qemu-iotests/245.out
+++ b/tests/qemu-iotests/245.out
@@ -1,6 +1,6 @@
-..................
+.....................
 ----------------------------------------------------------------------
-Ran 18 tests
+Ran 21 tests
 
 OK
 {"execute": "job-finalize", "arguments": {"id": "commit0"}}
-- 
2.20.1

[PATCH 2/2] block: bdrv_reopen() with backing file in different AioContext

[Kevin Wolf]

This patch allows bdrv_reopen() (and therefore the x-blockdev-reopen QMP
command) to attach a node as the new backing file even if the node is in
a different AioContext than the parent if one of both nodes can be moved
to the AioContext of the other node.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 block.c                | 36 +++++++++++++++++++++++++++---------
 tests/qemu-iotests/245 |  8 +++-----
 2 files changed, 30 insertions(+), 14 deletions(-)

diff --git a/block.c b/block.c
index 202c67e1e8..5dbba6cf31 100644
--- a/block.c
+++ b/block.c
@@ -3781,6 +3781,29 @@ static void bdrv_reopen_perm(BlockReopenQueue *q, BlockDriverState *bs,
     *shared = cumulative_shared_perms;
 }
 
+static bool bdrv_reopen_can_attach(BdrvChild *child,
+                                   BlockDriverState *new_child,
+                                   BlockDriverState *parent,
+                                   Error **errp)
+{
+    AioContext *parent_ctx = bdrv_get_aio_context(parent);
+    AioContext *child_ctx = bdrv_get_aio_context(new_child);
+    GSList *ignore;
+    bool ret;
+
+    ignore = g_slist_prepend(NULL, child);
+    ret = bdrv_can_set_aio_context(new_child, parent_ctx, &ignore, NULL);
+    g_slist_free(ignore);
+    if (ret) {
+        return ret;
+    }
+
+    ignore = g_slist_prepend(NULL, child);
+    ret = bdrv_can_set_aio_context(parent, child_ctx, &ignore, errp);
+    g_slist_free(ignore);
+    return ret;
+}
+
 /*
  * Take a BDRVReopenState and check if the value of 'backing' in the
  * reopen_state->options QDict is valid or not.
@@ -3832,16 +3855,11 @@ static int bdrv_reopen_parse_backing(BDRVReopenState *reopen_state,
     }
 
     /*
-     * TODO: before removing the x- prefix from x-blockdev-reopen we
-     * should move the new backing file into the right AioContext
-     * instead of returning an error.
+     * Check AioContext compatibility so that the bdrv_set_backing_hd() call in
+     * bdrv_reopen_commit() won't fail.
      */
-    if (new_backing_bs) {
-        if (bdrv_get_aio_context(new_backing_bs) != bdrv_get_aio_context(bs)) {
-            error_setg(errp, "Cannot use a new backing file "
-                       "with a different AioContext");
-            return -EINVAL;
-        }
+    if (!bdrv_reopen_can_attach(bs->backing, bs, new_backing_bs, errp)) {
+        return -EINVAL;
     }
 
     /*
diff --git a/tests/qemu-iotests/245 b/tests/qemu-iotests/245
index 5a2cd5ed0e..d6135ec14d 100755
--- a/tests/qemu-iotests/245
+++ b/tests/qemu-iotests/245
@@ -1010,18 +1010,16 @@ class TestBlockdevReopen(iotests.QMPTestCase):
     # neither of them can switch to the other AioContext
     def test_iothreads_error(self):
         self.run_test_iothreads('iothread0', 'iothread1',
-                                "Cannot use a new backing file with a different AioContext")
+                                "Cannot change iothread of active block backend")
 
     def test_iothreads_compatible_users(self):
         self.run_test_iothreads('iothread0', 'iothread0')
 
     def test_iothreads_switch_backing(self):
-        self.run_test_iothreads('iothread0', None,
-                                "Cannot use a new backing file with a different AioContext")
+        self.run_test_iothreads('iothread0', None)
 
     def test_iothreads_switch_overlay(self):
-        self.run_test_iothreads(None, 'iothread0',
-                                "Cannot use a new backing file with a different AioContext")
+        self.run_test_iothreads(None, 'iothread0')
 
 if __name__ == '__main__':
     iotests.main(supported_fmts=["qcow2"],
-- 
2.20.1

Re: [PATCH 0/2] block: bdrv_reopen() with backing file in different AioContext

[Peter Krempa]

On Thu, Feb 27, 2020 at 19:18:02 +0100, Kevin Wolf wrote:
> Kevin Wolf (2):
>   iotests: Refactor blockdev-reopen test for iothreads
>   block: bdrv_reopen() with backing file in different AioContext

Thanks for the patches. It fixes the problem we've talked about:

Tested-by: Peter Krempa <pkrempa@redhat.com>

Re: [PATCH 0/2] block: bdrv_reopen() with backing file in different AioContext

[Kevin Wolf]

Am 04.03.2020 um 10:29 hat Peter Krempa geschrieben:
> On Thu, Feb 27, 2020 at 19:18:02 +0100, Kevin Wolf wrote:
> > Kevin Wolf (2):
> >   iotests: Refactor blockdev-reopen test for iothreads
> >   block: bdrv_reopen() with backing file in different AioContext
> 
> Thanks for the patches. It fixes the problem we've talked about:
> 
> Tested-by: Peter Krempa <pkrempa@redhat.com>

Thanks for testing, applied to the block branch.

Kevin

Re: [PATCH 2/2] block: bdrv_reopen() with backing file in different AioContext

[Alberto Garcia]

On Thu 27 Feb 2020 07:18:04 PM CET, Kevin Wolf wrote:
>      /*
> -     * TODO: before removing the x- prefix from x-blockdev-reopen we
> -     * should move the new backing file into the right AioContext
> -     * instead of returning an error.
> +     * Check AioContext compatibility so that the bdrv_set_backing_hd() call in
> +     * bdrv_reopen_commit() won't fail.
>       */
> -    if (new_backing_bs) {
> -        if (bdrv_get_aio_context(new_backing_bs) != bdrv_get_aio_context(bs)) {
> -            error_setg(errp, "Cannot use a new backing file "
> -                       "with a different AioContext");
> -            return -EINVAL;
> -        }
> +    if (!bdrv_reopen_can_attach(bs->backing, bs, new_backing_bs, errp)) {
> +        return -EINVAL;
>      }

What happens here now if 'new_backing_bs' is NULL ?

It seems that you would be calling bdrv_can_set_aio_context(NULL, ...),
and it looks like that would crash.

Berto

Re: [PATCH 2/2] block: bdrv_reopen() with backing file in different AioContext

[Kevin Wolf]

Am 05.03.2020 um 16:54 hat Alberto Garcia geschrieben:
> On Thu 27 Feb 2020 07:18:04 PM CET, Kevin Wolf wrote:
> >      /*
> > -     * TODO: before removing the x- prefix from x-blockdev-reopen we
> > -     * should move the new backing file into the right AioContext
> > -     * instead of returning an error.
> > +     * Check AioContext compatibility so that the bdrv_set_backing_hd() call in
> > +     * bdrv_reopen_commit() won't fail.
> >       */
> > -    if (new_backing_bs) {
> > -        if (bdrv_get_aio_context(new_backing_bs) != bdrv_get_aio_context(bs)) {
> > -            error_setg(errp, "Cannot use a new backing file "
> > -                       "with a different AioContext");
> > -            return -EINVAL;
> > -        }
> > +    if (!bdrv_reopen_can_attach(bs->backing, bs, new_backing_bs, errp)) {
> > +        return -EINVAL;
> >      }
> 
> What happens here now if 'new_backing_bs' is NULL ?
> 
> It seems that you would be calling bdrv_can_set_aio_context(NULL, ...),
> and it looks like that would crash.

Not sure why I thought that this check isn't needed any more...

It actually works as long as everything runs in the main loop context
(because bdrv_get_aio_context(NULL) return the main context, so there is
nothing to do), which is why the test cases didn't fail. But as soon as
you move things to a different AioContext, they will fail.

Maybe even worse, the argument order for bdrv_reopen_can_attach() is
wrong.

Thanks for catching this, I'll send a v2.

Kevin