[Qemu-devel] [PATCH 0/6] block: Fix op blockers for inactive images

[Qemu-devel] [PATCH 0/6] block: Fix op blockers for inactive images

[Kevin Wolf]

Fam's image locking series introduced some special-casing in the file-posix
driver that avoids taking locks when the image is inactive. While this works,
it really isn't the job of the file-posix driver, but the core block layer
should consider that inactive nodes require a lot less permissions.

This series integrates op blockers with bdrv_inactivate/invalidate_cache() to
solve this problem gennerically, and removes the workaround in file-posix.

Kevin Wolf (6):
  migration: Unify block node activation error handling
  block: New BdrvChildRole.activate() for blk_resume_after_migration()
  block: Drop permissions when migration completes
  block: Inactivate parents before children
  block: Fix write/resize permissions for inactive images
  file-posix: Remove .bdrv_inactivate/invalidate_cache

 block.c                   | 76 ++++++++++++++++++++++++++++++++++++++++----
 block/block-backend.c     | 81 +++++++++++++++++++++++++++++++----------------
 block/file-posix.c        | 33 -------------------
 include/block/block.h     |  3 +-
 include/block/block_int.h |  6 ++++
 migration/migration.c     | 13 ++------
 migration/savevm.c        |  9 ++----
 qmp.c                     | 18 ++++-------
 8 files changed, 139 insertions(+), 100 deletions(-)

-- 
1.8.3.1

[Qemu-devel] [PATCH 1/6] migration: Unify block node activation error handling

[Kevin Wolf]

Migration code activates all block driver nodes on the destination when
the migration completes. It does so by calling
bdrv_invalidate_cache_all() and blk_resume_after_migration(). There is
one code path for precopy and one for postcopy migration, resulting in
four function calls, which used to have three different failure modes.

This patch unifies the behaviour so that failure to activate all block
nodes is non-fatal, but the error message is logged and the VM isn't
automatically started. 'cont' will retry activating the block nodes.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 migration/migration.c | 16 +++++-----------
 migration/savevm.c    | 12 +++++-------
 qmp.c                 | 18 +++++++++---------
 3 files changed, 19 insertions(+), 27 deletions(-)

diff --git a/migration/migration.c b/migration/migration.c
index 353f272..7cb0af2 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -338,20 +338,14 @@ static void process_incoming_migration_bh(void *opaque)
     Error *local_err = NULL;
     MigrationIncomingState *mis = opaque;
 
-    /* Make sure all file formats flush their mutable metadata */
+    /* Make sure all file formats flush their mutable metadata.
+     * If we get an error here, just don't restart the VM yet. */
     bdrv_invalidate_cache_all(&local_err);
-    if (local_err) {
-        migrate_set_state(&mis->state, MIGRATION_STATUS_ACTIVE,
-                          MIGRATION_STATUS_FAILED);
-        error_report_err(local_err);
-        migrate_decompress_threads_join();
-        exit(EXIT_FAILURE);
+    if (!local_err) {
+        blk_resume_after_migration(&local_err);
     }
-
-    /* If we get an error here, just don't restart the VM yet. */
-    blk_resume_after_migration(&local_err);
     if (local_err) {
-        error_free(local_err);
+        error_report_err(local_err);
         local_err = NULL;
         autostart = false;
     }
diff --git a/migration/savevm.c b/migration/savevm.c
index a00c1ab..0004f43 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -1616,16 +1616,14 @@ static void loadvm_postcopy_handle_run_bh(void *opaque)
 
     qemu_announce_self();
 
-    /* Make sure all file formats flush their mutable metadata */
+    /* Make sure all file formats flush their mutable metadata.
+     * If we get an error here, just don't restart the VM yet. */
     bdrv_invalidate_cache_all(&local_err);
-    if (local_err) {
-        error_report_err(local_err);
+    if (!local_err) {
+        blk_resume_after_migration(&local_err);
     }
-
-    /* If we get an error here, just don't restart the VM yet. */
-    blk_resume_after_migration(&local_err);
     if (local_err) {
-        error_free(local_err);
+        error_report_err(local_err);
         local_err = NULL;
         autostart = false;
     }
diff --git a/qmp.c b/qmp.c
index ab74cd7..25b5050 100644
--- a/qmp.c
+++ b/qmp.c
@@ -196,15 +196,15 @@ void qmp_cont(Error **errp)
     }
 
     /* Continuing after completed migration. Images have been inactivated to
-     * allow the destination to take control. Need to get control back now. */
-    if (runstate_check(RUN_STATE_FINISH_MIGRATE) ||
-        runstate_check(RUN_STATE_POSTMIGRATE))
-    {
-        bdrv_invalidate_cache_all(&local_err);
-        if (local_err) {
-            error_propagate(errp, local_err);
-            return;
-        }
+     * allow the destination to take control. Need to get control back now.
+     *
+     * If there are no inactive block nodes (e.g. because the VM was just
+     * paused rather than completing a migration), bdrv_inactivate_all() simply
+     * doesn't do anything. */
+    bdrv_invalidate_cache_all(&local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
+        return;
     }
 
     blk_resume_after_migration(&local_err);
-- 
1.8.3.1

Re: [Qemu-devel] [PATCH 1/6] migration: Unify block node activation error handling

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1049 bytes --]

On 05/04/2017 11:52 AM, Kevin Wolf wrote:
> Migration code activates all block driver nodes on the destination when
> the migration completes. It does so by calling
> bdrv_invalidate_cache_all() and blk_resume_after_migration(). There is
> one code path for precopy and one for postcopy migration, resulting in
> four function calls, which used to have three different failure modes.
> 
> This patch unifies the behaviour so that failure to activate all block
> nodes is non-fatal, but the error message is logged and the VM isn't
> automatically started. 'cont' will retry activating the block nodes.
> 
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
>  migration/migration.c | 16 +++++-----------
>  migration/savevm.c    | 12 +++++-------
>  qmp.c                 | 18 +++++++++---------
>  3 files changed, 19 insertions(+), 27 deletions(-)
> 
Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

[Qemu-devel] [PATCH 2/6] block: New BdrvChildRole.activate() for blk_resume_after_migration()

[Kevin Wolf]

Instead of manually calling blk_resume_after_migration() in migration
code after doing bdrv_invalidate_cache_all(), integrate the BlockBackend
activation with cache invalidation into a single function. This is
achieved with a new callback in BdrvChildRole that is called by
bdrv_invalidate_cache_all().

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 block.c                   | 12 +++++++++-
 block/block-backend.c     | 56 +++++++++++++++++++++++------------------------
 include/block/block.h     |  2 --
 include/block/block_int.h |  5 +++++
 migration/migration.c     |  3 ---
 migration/savevm.c        |  3 ---
 qmp.c                     |  6 -----
 7 files changed, 44 insertions(+), 43 deletions(-)

diff --git a/block.c b/block.c
index 70ca7b4..3e7f124 100644
--- a/block.c
+++ b/block.c
@@ -3958,7 +3958,7 @@ void bdrv_init_with_whitelist(void)
 
 void bdrv_invalidate_cache(BlockDriverState *bs, Error **errp)
 {
-    BdrvChild *child;
+    BdrvChild *child, *parent;
     Error *local_err = NULL;
     int ret;
 
@@ -3994,6 +3994,16 @@ void bdrv_invalidate_cache(BlockDriverState *bs, Error **errp)
         error_setg_errno(errp, -ret, "Could not refresh total sector count");
         return;
     }
+
+    QLIST_FOREACH(parent, &bs->parents, next_parent) {
+        if (parent->role->activate) {
+            parent->role->activate(parent, &local_err);
+            if (local_err) {
+                error_propagate(errp, local_err);
+                return;
+            }
+        }
+    }
 }
 
 void bdrv_invalidate_cache_all(Error **errp)
diff --git a/block/block-backend.c b/block/block-backend.c
index f5bf13e..a7ce72b 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -130,6 +130,32 @@ static const char *blk_root_get_name(BdrvChild *child)
     return blk_name(child->opaque);
 }
 
+/*
+ * Notifies the user of the BlockBackend that migration has completed. qdev
+ * devices can tighten their permissions in response (specifically revoke
+ * shared write permissions that we needed for storage migration).
+ *
+ * If an error is returned, the VM cannot be allowed to be resumed.
+ */
+static void blk_root_activate(BdrvChild *child, Error **errp)
+{
+    BlockBackend *blk = child->opaque;
+    Error *local_err = NULL;
+
+    if (!blk->disable_perm) {
+        return;
+    }
+
+    blk->disable_perm = false;
+
+    blk_set_perm(blk, blk->perm, blk->shared_perm, &local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
+        blk->disable_perm = true;
+        return;
+    }
+}
+
 static const BdrvChildRole child_root = {
     .inherit_options    = blk_root_inherit_options,
 
@@ -140,6 +166,8 @@ static const BdrvChildRole child_root = {
 
     .drained_begin      = blk_root_drained_begin,
     .drained_end        = blk_root_drained_end,
+
+    .activate           = blk_root_activate,
 };
 
 /*
@@ -601,34 +629,6 @@ void blk_get_perm(BlockBackend *blk, uint64_t *perm, uint64_t *shared_perm)
     *shared_perm = blk->shared_perm;
 }
 
-/*
- * Notifies the user of all BlockBackends that migration has completed. qdev
- * devices can tighten their permissions in response (specifically revoke
- * shared write permissions that we needed for storage migration).
- *
- * If an error is returned, the VM cannot be allowed to be resumed.
- */
-void blk_resume_after_migration(Error **errp)
-{
-    BlockBackend *blk;
-    Error *local_err = NULL;
-
-    for (blk = blk_all_next(NULL); blk; blk = blk_all_next(blk)) {
-        if (!blk->disable_perm) {
-            continue;
-        }
-
-        blk->disable_perm = false;
-
-        blk_set_perm(blk, blk->perm, blk->shared_perm, &local_err);
-        if (local_err) {
-            error_propagate(errp, local_err);
-            blk->disable_perm = true;
-            return;
-        }
-    }
-}
-
 static int blk_do_attach_dev(BlockBackend *blk, void *dev)
 {
     if (blk->dev) {
diff --git a/include/block/block.h b/include/block/block.h
index 877fbb0..80d51d8 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -369,8 +369,6 @@ void bdrv_invalidate_cache(BlockDriverState *bs, Error **errp);
 void bdrv_invalidate_cache_all(Error **errp);
 int bdrv_inactivate_all(void);
 
-void blk_resume_after_migration(Error **errp);
-
 /* Ensure contents are flushed to disk.  */
 int bdrv_flush(BlockDriverState *bs);
 int coroutine_fn bdrv_co_flush(BlockDriverState *bs);
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 1b4d08e..5637925 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -473,6 +473,11 @@ struct BdrvChildRole {
     void (*drained_begin)(BdrvChild *child);
     void (*drained_end)(BdrvChild *child);
 
+    /* Notifies the parent that the child has been activated (e.g. when
+     * migration is completing) and it can start requesting permissions and
+     * doing I/O on it. */
+    void (*activate)(BdrvChild *child, Error **errp);
+
     void (*attach)(BdrvChild *child);
     void (*detach)(BdrvChild *child);
 };
diff --git a/migration/migration.c b/migration/migration.c
index 7cb0af2..0d230f4 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -341,9 +341,6 @@ static void process_incoming_migration_bh(void *opaque)
     /* Make sure all file formats flush their mutable metadata.
      * If we get an error here, just don't restart the VM yet. */
     bdrv_invalidate_cache_all(&local_err);
-    if (!local_err) {
-        blk_resume_after_migration(&local_err);
-    }
     if (local_err) {
         error_report_err(local_err);
         local_err = NULL;
diff --git a/migration/savevm.c b/migration/savevm.c
index 0004f43..5689615 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -1619,9 +1619,6 @@ static void loadvm_postcopy_handle_run_bh(void *opaque)
     /* Make sure all file formats flush their mutable metadata.
      * If we get an error here, just don't restart the VM yet. */
     bdrv_invalidate_cache_all(&local_err);
-    if (!local_err) {
-        blk_resume_after_migration(&local_err);
-    }
     if (local_err) {
         error_report_err(local_err);
         local_err = NULL;
diff --git a/qmp.c b/qmp.c
index 25b5050..f656940 100644
--- a/qmp.c
+++ b/qmp.c
@@ -207,12 +207,6 @@ void qmp_cont(Error **errp)
         return;
     }
 
-    blk_resume_after_migration(&local_err);
-    if (local_err) {
-        error_propagate(errp, local_err);
-        return;
-    }
-
     if (runstate_check(RUN_STATE_INMIGRATE)) {
         autostart = 1;
     } else {
-- 
1.8.3.1

Re: [Qemu-devel] [PATCH 2/6] block: New BdrvChildRole.activate() for blk_resume_after_migration()

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 599 bytes --]

On 05/04/2017 11:52 AM, Kevin Wolf wrote:
> Instead of manually calling blk_resume_after_migration() in migration
> code after doing bdrv_invalidate_cache_all(), integrate the BlockBackend
> activation with cache invalidation into a single function. This is
> achieved with a new callback in BdrvChildRole that is called by
> bdrv_invalidate_cache_all().
> 
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

[Qemu-devel] [PATCH 3/6] block: Drop permissions when migration completes

[Kevin Wolf]

With image locking, permissions affect other qemu processes as well. We
want to be sure that the destination can run, so let's drop permissions
on the source when migration completes.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 block.c                   | 12 +++++++++++-
 block/block-backend.c     | 25 +++++++++++++++++++++++++
 include/block/block_int.h |  7 ++++---
 3 files changed, 40 insertions(+), 4 deletions(-)

diff --git a/block.c b/block.c
index 3e7f124..c3e7ebd 100644
--- a/block.c
+++ b/block.c
@@ -4028,7 +4028,7 @@ void bdrv_invalidate_cache_all(Error **errp)
 static int bdrv_inactivate_recurse(BlockDriverState *bs,
                                    bool setting_flag)
 {
-    BdrvChild *child;
+    BdrvChild *child, *parent;
     int ret;
 
     if (!setting_flag && bs->drv->bdrv_inactivate) {
@@ -4047,6 +4047,16 @@ static int bdrv_inactivate_recurse(BlockDriverState *bs,
 
     if (setting_flag) {
         bs->open_flags |= BDRV_O_INACTIVE;
+
+        QLIST_FOREACH(parent, &bs->parents, next_parent) {
+            if (parent->role->inactivate) {
+                ret = parent->role->inactivate(parent);
+                if (ret < 0) {
+                    bs->open_flags &= ~BDRV_O_INACTIVE;
+                    return ret;
+                }
+            }
+        }
     }
     return 0;
 }
diff --git a/block/block-backend.c b/block/block-backend.c
index a7ce72b..f3a6008 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -156,6 +156,30 @@ static void blk_root_activate(BdrvChild *child, Error **errp)
     }
 }
 
+static int blk_root_inactivate(BdrvChild *child)
+{
+    BlockBackend *blk = child->opaque;
+
+    if (blk->disable_perm) {
+        return 0;
+    }
+
+    /* Only inactivate BlockBackends for guest devices (which are inactive at
+     * this point because the VM is stopped) and unattached monitor-owned
+     * BlockBackends. If there is still any other user like a block job, then
+     * we simply can't inactivate the image. */
+    if (!blk->dev && !blk->name[0]) {
+        return -EPERM;
+    }
+
+    blk->disable_perm = true;
+    if (blk->root) {
+        bdrv_child_try_set_perm(blk->root, 0, BLK_PERM_ALL, &error_abort);
+    }
+
+    return 0;
+}
+
 static const BdrvChildRole child_root = {
     .inherit_options    = blk_root_inherit_options,
 
@@ -168,6 +192,7 @@ static const BdrvChildRole child_root = {
     .drained_end        = blk_root_drained_end,
 
     .activate           = blk_root_activate,
+    .inactivate         = blk_root_inactivate,
 };
 
 /*
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 5637925..5750a44 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -473,10 +473,11 @@ struct BdrvChildRole {
     void (*drained_begin)(BdrvChild *child);
     void (*drained_end)(BdrvChild *child);
 
-    /* Notifies the parent that the child has been activated (e.g. when
-     * migration is completing) and it can start requesting permissions and
-     * doing I/O on it. */
+    /* Notifies the parent that the child has been activated/inactivated (e.g.
+     * when migration is completing) and it can start/stop requesting
+     * permissions and doing I/O on it. */
     void (*activate)(BdrvChild *child, Error **errp);
+    int (*inactivate)(BdrvChild *child);
 
     void (*attach)(BdrvChild *child);
     void (*detach)(BdrvChild *child);
-- 
1.8.3.1

Re: [Qemu-devel] [PATCH 3/6] block: Drop permissions when migration completes

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 684 bytes --]

On 05/04/2017 11:52 AM, Kevin Wolf wrote:
> With image locking, permissions affect other qemu processes as well. We
> want to be sure that the destination can run, so let's drop permissions
> on the source when migration completes.
> 
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
>  block.c                   | 12 +++++++++++-
>  block/block-backend.c     | 25 +++++++++++++++++++++++++
>  include/block/block_int.h |  7 ++++---
>  3 files changed, 40 insertions(+), 4 deletions(-)
> 

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

[Qemu-devel] [PATCH 4/6] block: Inactivate parents before children

[Kevin Wolf]

The proper order for inactivating block nodes is that first the parents
get inactivated and then the children. If we do things in this order, we
can assert that we didn't accidentally leave a parent activated when one
of its child nodes is inactive.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 block.c | 25 ++++++++++++++++++-------
 1 file changed, 18 insertions(+), 7 deletions(-)

diff --git a/block.c b/block.c
index c3e7ebd..773bd64 100644
--- a/block.c
+++ b/block.c
@@ -762,6 +762,13 @@ static void bdrv_child_cb_drained_end(BdrvChild *child)
     bdrv_drained_end(bs);
 }
 
+static int bdrv_child_cb_inactivate(BdrvChild *child)
+{
+    BlockDriverState *bs = child->opaque;
+    assert(bs->open_flags & BDRV_O_INACTIVE);
+    return 0;
+}
+
 /*
  * Returns the options and flags that a temporary snapshot should get, based on
  * the originally requested flags (the originally requested image will have
@@ -822,6 +829,7 @@ const BdrvChildRole child_file = {
     .inherit_options = bdrv_inherited_options,
     .drained_begin   = bdrv_child_cb_drained_begin,
     .drained_end     = bdrv_child_cb_drained_end,
+    .inactivate      = bdrv_child_cb_inactivate,
 };
 
 /*
@@ -843,6 +851,7 @@ const BdrvChildRole child_format = {
     .inherit_options = bdrv_inherited_fmt_options,
     .drained_begin   = bdrv_child_cb_drained_begin,
     .drained_end     = bdrv_child_cb_drained_end,
+    .inactivate      = bdrv_child_cb_inactivate,
 };
 
 static void bdrv_backing_attach(BdrvChild *c)
@@ -928,6 +937,7 @@ const BdrvChildRole child_backing = {
     .inherit_options = bdrv_backing_options,
     .drained_begin   = bdrv_child_cb_drained_begin,
     .drained_end     = bdrv_child_cb_drained_end,
+    .inactivate      = bdrv_child_cb_inactivate,
 };
 
 static int bdrv_open_flags(BlockDriverState *bs, int flags)
@@ -4038,13 +4048,6 @@ static int bdrv_inactivate_recurse(BlockDriverState *bs,
         }
     }
 
-    QLIST_FOREACH(child, &bs->children, next) {
-        ret = bdrv_inactivate_recurse(child->bs, setting_flag);
-        if (ret < 0) {
-            return ret;
-        }
-    }
-
     if (setting_flag) {
         bs->open_flags |= BDRV_O_INACTIVE;
 
@@ -4058,6 +4061,14 @@ static int bdrv_inactivate_recurse(BlockDriverState *bs,
             }
         }
     }
+
+    QLIST_FOREACH(child, &bs->children, next) {
+        ret = bdrv_inactivate_recurse(child->bs, setting_flag);
+        if (ret < 0) {
+            return ret;
+        }
+    }
+
     return 0;
 }
 
-- 
1.8.3.1

Re: [Qemu-devel] [PATCH 4/6] block: Inactivate parents before children

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 643 bytes --]

On 05/04/2017 11:52 AM, Kevin Wolf wrote:
> The proper order for inactivating block nodes is that first the parents
> get inactivated and then the children. If we do things in this order, we
> can assert that we didn't accidentally leave a parent activated when one
> of its child nodes is inactive.
> 
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
>  block.c | 25 ++++++++++++++++++-------
>  1 file changed, 18 insertions(+), 7 deletions(-)
> 

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

[Qemu-devel] [PATCH 5/6] block: Fix write/resize permissions for inactive images

[Kevin Wolf]

Format drivers for inactive nodes don't need write/resize permissions on
their bs->file and can share write/resize with another VM (in fact, this
is the whole point of keeping images inactive). Represent this fact in
the op blocker system, so that image locking does the right thing
without special-casing inactive images.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 block.c               | 35 +++++++++++++++++++++++++++++++++--
 include/block/block.h |  1 +
 2 files changed, 34 insertions(+), 2 deletions(-)

diff --git a/block.c b/block.c
index 773bd64..cd89467 100644
--- a/block.c
+++ b/block.c
@@ -192,11 +192,20 @@ void path_combine(char *dest, int dest_size,
     }
 }
 
+/* Returns whether the image file is opened as read-only. Note that this can
+ * return false and writing to the image file is still not possible because the
+ * image is inactivated. */
 bool bdrv_is_read_only(BlockDriverState *bs)
 {
     return bs->read_only;
 }
 
+/* Returns whether the image file can be written to right now */
+bool bdrv_is_writable(BlockDriverState *bs)
+{
+    return !bdrv_is_read_only(bs) && !(bs->open_flags & BDRV_O_INACTIVE);
+}
+
 int bdrv_can_set_read_only(BlockDriverState *bs, bool read_only, Error **errp)
 {
     /* Do not set read_only if copy_on_read is enabled */
@@ -1512,7 +1521,7 @@ static int bdrv_check_perm(BlockDriverState *bs, uint64_t cumulative_perms,
 
     /* Write permissions never work with read-only images */
     if ((cumulative_perms & (BLK_PERM_WRITE | BLK_PERM_WRITE_UNCHANGED)) &&
-        bdrv_is_read_only(bs))
+        !bdrv_is_writable(bs))
     {
         error_setg(errp, "Block node is read-only");
         return -EPERM;
@@ -1797,7 +1806,7 @@ void bdrv_format_default_perms(BlockDriverState *bs, BdrvChild *c,
         bdrv_filter_default_perms(bs, c, role, perm, shared, &perm, &shared);
 
         /* Format drivers may touch metadata even if the guest doesn't write */
-        if (!bdrv_is_read_only(bs)) {
+        if (bdrv_is_writable(bs)) {
             perm |= BLK_PERM_WRITE | BLK_PERM_RESIZE;
         }
 
@@ -1823,6 +1832,10 @@ void bdrv_format_default_perms(BlockDriverState *bs, BdrvChild *c,
                   BLK_PERM_WRITE_UNCHANGED;
     }
 
+    if (bs->open_flags & BDRV_O_INACTIVE) {
+        shared |= BLK_PERM_WRITE | BLK_PERM_RESIZE;
+    }
+
     *nperm = perm;
     *nshared = shared;
 }
@@ -3969,6 +3982,7 @@ void bdrv_init_with_whitelist(void)
 void bdrv_invalidate_cache(BlockDriverState *bs, Error **errp)
 {
     BdrvChild *child, *parent;
+    uint64_t perm, shared_perm;
     Error *local_err = NULL;
     int ret;
 
@@ -4005,6 +4019,16 @@ void bdrv_invalidate_cache(BlockDriverState *bs, Error **errp)
         return;
     }
 
+    /* Update permissions, they may differ for inactive nodes */
+    bdrv_get_cumulative_perm(bs, &perm, &shared_perm);
+    ret = bdrv_check_perm(bs, perm, shared_perm, NULL, &local_err);
+    if (ret < 0) {
+        bs->open_flags |= BDRV_O_INACTIVE;
+        error_propagate(errp, local_err);
+        return;
+    }
+    bdrv_set_perm(bs, perm, shared_perm);
+
     QLIST_FOREACH(parent, &bs->parents, next_parent) {
         if (parent->role->activate) {
             parent->role->activate(parent, &local_err);
@@ -4049,6 +4073,8 @@ static int bdrv_inactivate_recurse(BlockDriverState *bs,
     }
 
     if (setting_flag) {
+        uint64_t perm, shared_perm;
+
         bs->open_flags |= BDRV_O_INACTIVE;
 
         QLIST_FOREACH(parent, &bs->parents, next_parent) {
@@ -4060,6 +4086,11 @@ static int bdrv_inactivate_recurse(BlockDriverState *bs,
                 }
             }
         }
+
+        /* Update permissions, they may differ for inactive nodes */
+        bdrv_get_cumulative_perm(bs, &perm, &shared_perm);
+        bdrv_check_perm(bs, perm, shared_perm, NULL, &error_abort);
+        bdrv_set_perm(bs, perm, shared_perm);
     }
 
     QLIST_FOREACH(child, &bs->children, next) {
diff --git a/include/block/block.h b/include/block/block.h
index 80d51d8..90932b4 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -435,6 +435,7 @@ int bdrv_is_allocated_above(BlockDriverState *top, BlockDriverState *base,
                             int64_t sector_num, int nb_sectors, int *pnum);
 
 bool bdrv_is_read_only(BlockDriverState *bs);
+bool bdrv_is_writable(BlockDriverState *bs);
 int bdrv_can_set_read_only(BlockDriverState *bs, bool read_only, Error **errp);
 int bdrv_set_read_only(BlockDriverState *bs, bool read_only, Error **errp);
 bool bdrv_is_sg(BlockDriverState *bs);
-- 
1.8.3.1

Re: [Qemu-devel] [PATCH 5/6] block: Fix write/resize permissions for inactive images

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1590 bytes --]

On 05/04/2017 11:52 AM, Kevin Wolf wrote:
> Format drivers for inactive nodes don't need write/resize permissions on
> their bs->file and can share write/resize with another VM (in fact, this
> is the whole point of keeping images inactive). Represent this fact in
> the op blocker system, so that image locking does the right thing
> without special-casing inactive images.
> 
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
>  block.c               | 35 +++++++++++++++++++++++++++++++++--
>  include/block/block.h |  1 +
>  2 files changed, 34 insertions(+), 2 deletions(-)
> 
> diff --git a/block.c b/block.c
> index 773bd64..cd89467 100644
> --- a/block.c
> +++ b/block.c
> @@ -192,11 +192,20 @@ void path_combine(char *dest, int dest_size,
>      }
>  }
>  
> +/* Returns whether the image file is opened as read-only. Note that this can
> + * return false and writing to the image file is still not possible because the

s/false and/false but/
s/is still not/still not be/

> + * image is inactivated. */
>  bool bdrv_is_read_only(BlockDriverState *bs)
>  {
>      return bs->read_only;
>  }
>  
> +/* Returns whether the image file can be written to right now */
> +bool bdrv_is_writable(BlockDriverState *bs)
> +{
> +    return !bdrv_is_read_only(bs) && !(bs->open_flags & BDRV_O_INACTIVE);
> +}

Nice.

Up to you if you think the grammar suggestion helps.
Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 5/6] block: Fix write/resize permissions for inactive images

[Xie Changlong]

在 5/5/2017 12:52 AM, Kevin Wolf 写道:
>   
> +/* Returns whether the image file can be written to right now */
> +bool bdrv_is_writable(BlockDriverState *bs)
> +{
> +    return !bdrv_is_read_only(bs) && !(bs->open_flags & BDRV_O_INACTIVE);
> +}
> +

This commit use BDRV_O_INACTIVE to judge whether the image file can be 
written or not. But it blocks replication driver on the secondary node. 
For replication in secondary, we must ensure that the whole chain are 
writable:


   ||
   ||                            .----------
   ||                            | Secondary
   ||                            '----------
   ||
   ||

                                                         virtio-blk
                                                              ^
------>  3 NBD                                               |
   ||     server                                          2 filter
   ||        ^                                                ^
   ||        |                                                |
   ||  Secondary disk <--------- hidden-disk 5 <--------- active-disk 4
   ||        |          backing        ^       backing
   ||        |                         |
   ||        |                         |
   ||        '-------------------------'
   ||           drive-backup sync=none 6

The root casue is when we run replication in secondary, vmstate changes 
to RUN_STATE_INMIGRATE, then blockdev_init() sets bdrv_flags |= 
BDRV_O_INACTIVE. So the whole chain become readonly. I've tried on my 
side, but it seems not easy to fix it. I wonder if there is any way to 
bypass this? Any suggestion would be appreciated.

It's very easy to reproduce this scenario:
(gdb) r
Starting program: /root/.xie/qemu-colo/x86_64-softmmu/qemu-system-x86_64 
-boot c -m 2048 -smp 2 -qmp stdio -vnc :0 -name secondary -enable-kvm 
-cpu qemu64,+kvmclock -device piix3-usb-uhci -device usb-tablet -drive 
if=none,id=colo-disk,file.filename=/root/.xie/suse.qcow2.orgin,file.node-name=secondary_disk,driver=qcow2,node-name=sec-qcow2-driver-for-nbd 
-drive 
if=ide,id=active-disk0,node-name=active-disk111,throttling.bps-total=70000000,driver=replication,node-name=secondary-replication-driver,mode=secondary,top-id=active-disk0,file.driver=qcow2,file.node-name=active-qcow2-driver,file.file.filename=/mnt/ramfs/active_disk.img,file.file.node-name=active_disk,file.backing.driver=qcow2,file.backing.file.filename=/mnt/ramfs/hidden_disk.img,file.backing.node-name=hidden-qcow2-driver,file.backing.file.node-name=hidden_disk,file.backing.backing=colo-disk 
-incoming tcp:0:8888
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[New Thread 0x7ffff4801700 (LWP 25252)]
[New Thread 0x7ffff4000700 (LWP 25255)]
qemu-system-x86_64: -drive 
if=ide,id=active-disk0,node-name=active-disk111,throttling.bps-total=70000000,driver=replication,node-name=secondary-replication-driver,mode=secondary,top-id=active-disk0,file.driver=qcow2,file.node-name=active-qcow2-driver,file.file.filename=/mnt/ramfs/active_disk.img,file.file.node-name=active_disk,file.backing.driver=qcow2,file.backing.file.filename=/mnt/ramfs/hidden_disk.img,file.backing.node-name=hidden-qcow2-driver,file.backing.file.node-name=hidden_disk,file.backing.backing=colo-disk: 
Block node is read-only
[Thread 0x7ffff4000700 (LWP 25255) exited]
[Thread 0x7ffff4801700 (LWP 25252) exited]
[Inferior 1 (process 25248) exited with code 01]
Missing separate debuginfos, use: debuginfo-install 
glib2-2.46.2-4.el7.x86_64 glibc-2.17-157.el7_3.4.x86_64 
libacl-2.2.51-12.el7.x86_64 libattr-2.4.46-12.el7.x86_64 
libgcc-4.8.5-11.el7.x86_64 libgcrypt-1.5.3-13.el7_3.1.x86_64 
libgpg-error-1.12-3.el7.x86_64 libstdc++-4.8.5-11.el7.x86_64 
libuuid-2.23.2-33.el7_3.2.x86_64 openssl-libs-1.0.1e-60.el7_3.1.x86_64 
pixman-0.34.0-1.el7.x86_64 zlib-1.2.7-17.el7.x86_64
(gdb)

-- 
Thanks
     -Xie

Re: [Qemu-devel] [PATCH 5/6] block: Fix write/resize permissions for inactive images

[Fam Zheng]

On Fri, 08/18 18:06, Xie Changlong wrote:
> The root casue is when we run replication in secondary, vmstate changes to
> RUN_STATE_INMIGRATE, then blockdev_init() sets bdrv_flags |=
> BDRV_O_INACTIVE. So the whole chain become readonly. I've tried on my side,
> but it seems not easy to fix it. I wonder if there is any way to bypass
> this? Any suggestion would be appreciated.

The non-shared storage migration uses "nbd_server_add -w" at destinition side
where BDRV_O_INACTIVE is set for images like your case, the way it handles it is
by calling bdrv_invalidate_cache(). See nbd_export_new().

See also commit 3dff24f2dffc5f3aa46dc014122012848bd7959d.

I'm not sure if this is enough for block replication?

Fam

[Qemu-devel] [PATCH 6/6] file-posix: Remove .bdrv_inactivate/invalidate_cache

[Kevin Wolf]

Now that the block layer takes care to request a lot less permissions
for inactive nodes, the special-casing in file-posix isn't necessary any
more.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 block/file-posix.c | 33 ---------------------------------
 1 file changed, 33 deletions(-)

diff --git a/block/file-posix.c b/block/file-posix.c
index f1f924b..90a5f53 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -2190,35 +2190,6 @@ static void raw_abort_perm_update(BlockDriverState *bs)
     raw_handle_perm_lock(bs, RAW_PL_ABORT, 0, 0, NULL);
 }
 
-static int raw_inactivate(BlockDriverState *bs)
-{
-    int ret;
-    uint64_t perm = 0;
-    uint64_t shared = BLK_PERM_ALL;
-
-    ret = raw_handle_perm_lock(bs, RAW_PL_PREPARE, perm, shared, NULL);
-    if (ret) {
-        return ret;
-    }
-    raw_handle_perm_lock(bs, RAW_PL_COMMIT, perm, shared, NULL);
-    return 0;
-}
-
-
-static void raw_invalidate_cache(BlockDriverState *bs, Error **errp)
-{
-    BDRVRawState *s = bs->opaque;
-    int ret;
-
-    assert(!(bdrv_get_flags(bs) & BDRV_O_INACTIVE));
-    ret = raw_handle_perm_lock(bs, RAW_PL_PREPARE, s->perm, s->shared_perm,
-                               errp);
-    if (ret) {
-        return;
-    }
-    raw_handle_perm_lock(bs, RAW_PL_COMMIT, s->perm, s->shared_perm, NULL);
-}
-
 BlockDriver bdrv_file = {
     .format_name = "file",
     .protocol_name = "file",
@@ -2249,8 +2220,6 @@ BlockDriver bdrv_file = {
     .bdrv_get_info = raw_get_info,
     .bdrv_get_allocated_file_size
                         = raw_get_allocated_file_size,
-    .bdrv_inactivate = raw_inactivate,
-    .bdrv_invalidate_cache = raw_invalidate_cache,
     .bdrv_check_perm = raw_check_perm,
     .bdrv_set_perm   = raw_set_perm,
     .bdrv_abort_perm_update = raw_abort_perm_update,
@@ -2712,8 +2681,6 @@ static BlockDriver bdrv_host_device = {
     .bdrv_get_info = raw_get_info,
     .bdrv_get_allocated_file_size
                         = raw_get_allocated_file_size,
-    .bdrv_inactivate = raw_inactivate,
-    .bdrv_invalidate_cache = raw_invalidate_cache,
     .bdrv_check_perm = raw_check_perm,
     .bdrv_set_perm   = raw_set_perm,
     .bdrv_abort_perm_update = raw_abort_perm_update,
-- 
1.8.3.1

Re: [Qemu-devel] [PATCH 6/6] file-posix: Remove .bdrv_inactivate/invalidate_cache

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 541 bytes --]

On 05/04/2017 11:52 AM, Kevin Wolf wrote:
> Now that the block layer takes care to request a lot less permissions
> for inactive nodes, the special-casing in file-posix isn't necessary any
> more.
> 
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
>  block/file-posix.c | 33 ---------------------------------
>  1 file changed, 33 deletions(-)
> 

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 0/6] block: Fix op blockers for inactive images

[Kevin Wolf]

Am 04.05.2017 um 18:52 hat Kevin Wolf geschrieben:
> Fam's image locking series introduced some special-casing in the file-posix
> driver that avoids taking locks when the image is inactive. While this works,
> it really isn't the job of the file-posix driver, but the core block layer
> should consider that inactive nodes require a lot less permissions.
> 
> This series integrates op blockers with bdrv_inactivate/invalidate_cache() to
> solve this problem gennerically, and removes the workaround in file-posix.

Applied to the block branch.

Kevin