* [PULL 2/3] docs/interop/bitmaps: use blockdev-backup
2021-11-09 17:52 [PULL 0/3] jobs: deprecate drive-backup qmp command Vladimir Sementsov-Ogievskiy
2021-11-09 17:52 ` [PULL 1/3] docs/block-replication: use blockdev-backup Vladimir Sementsov-Ogievskiy
@ 2021-11-09 17:52 ` Vladimir Sementsov-Ogievskiy
2021-11-09 17:52 ` [PULL 3/3] qapi: deprecate drive-backup Vladimir Sementsov-Ogievskiy
2021-11-10 8:50 ` [PULL 0/3] jobs: deprecate drive-backup qmp command Richard Henderson
3 siblings, 0 replies; 5+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-11-09 17:52 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, jsnow, eblake, vsementsov, peter.maydell,
Kashyap Chamarthy
We are going to deprecate drive-backup, so use modern interface here.
In examples where target image creation is shown, show blockdev-add as
well. If target creation omitted, omit blockdev-add as well.
Reviewed-by: Kashyap Chamarthy <kchamart@redhat.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
docs/interop/bitmaps.rst | 285 +++++++++++++++++++++++++++++----------
1 file changed, 215 insertions(+), 70 deletions(-)
diff --git a/docs/interop/bitmaps.rst b/docs/interop/bitmaps.rst
index 059ad67929..1de46febdc 100644
--- a/docs/interop/bitmaps.rst
+++ b/docs/interop/bitmaps.rst
@@ -539,12 +539,11 @@ other partial disk images on top of a base image to reconstruct a full backup
from the point in time at which the incremental backup was issued.
The "Push Model" here references the fact that QEMU is "pushing" the modified
-blocks out to a destination. We will be using the `drive-backup
-<qemu-qmp-ref.html#index-drive_002dbackup>`_ and `blockdev-backup
-<qemu-qmp-ref.html#index-blockdev_002dbackup>`_ QMP commands to create both
+blocks out to a destination. We will be using the `blockdev-backup
+<qemu-qmp-ref.html#index-blockdev_002dbackup>`_ QMP command to create both
full and incremental backups.
-Both of these commands are jobs, which have their own QMP API for querying and
+The command is a background job, which has its own QMP API for querying and
management documented in `Background jobs
<qemu-qmp-ref.html#Background-jobs>`_.
@@ -557,6 +556,10 @@ create a new incremental backup chain attached to a drive.
This example creates a new, full backup of "drive0" and accompanies it with a
new, empty bitmap that records writes from this point in time forward.
+The target can be created with the help of `blockdev-add
+<qemu-qmp-ref.html#index-blockdev_002dadd>`_ or `blockdev-create
+<qemu-qmp-ref.html#index-blockdev_002dcreate>`_ command.
+
.. note:: Any new writes that happen after this command is issued, even while
the backup job runs, will be written locally and not to the backup
destination. These writes will be recorded in the bitmap
@@ -576,12 +579,11 @@ new, empty bitmap that records writes from this point in time forward.
}
},
{
- "type": "drive-backup",
+ "type": "blockdev-backup",
"data": {
"device": "drive0",
- "target": "/path/to/drive0.full.qcow2",
- "sync": "full",
- "format": "qcow2"
+ "target": "target0",
+ "sync": "full"
}
}
]
@@ -664,12 +666,11 @@ use a transaction to reset the bitmap while making a new full backup:
}
},
{
- "type": "drive-backup",
+ "type": "blockdev-backup",
"data": {
"device": "drive0",
- "target": "/path/to/drive0.new_full.qcow2",
- "sync": "full",
- "format": "qcow2"
+ "target": "target0",
+ "sync": "full"
}
}
]
@@ -728,19 +729,35 @@ Example: First Incremental Backup
$ qemu-img create -f qcow2 drive0.inc0.qcow2 \
-b drive0.full.qcow2 -F qcow2
+#. Add target block node:
+
+ .. code-block:: QMP
+
+ -> {
+ "execute": "blockdev-add",
+ "arguments": {
+ "node-name": "target0",
+ "driver": "qcow2",
+ "file": {
+ "driver": "file",
+ "filename": "drive0.inc0.qcow2"
+ }
+ }
+ }
+
+ <- { "return": {} }
+
#. Issue an incremental backup command:
.. code-block:: QMP
-> {
- "execute": "drive-backup",
+ "execute": "blockdev-backup",
"arguments": {
"device": "drive0",
"bitmap": "bitmap0",
- "target": "drive0.inc0.qcow2",
- "format": "qcow2",
- "sync": "incremental",
- "mode": "existing"
+ "target": "target0",
+ "sync": "incremental"
}
}
@@ -785,20 +802,36 @@ Example: Second Incremental Backup
$ qemu-img create -f qcow2 drive0.inc1.qcow2 \
-b drive0.inc0.qcow2 -F qcow2
+#. Add target block node:
+
+ .. code-block:: QMP
+
+ -> {
+ "execute": "blockdev-add",
+ "arguments": {
+ "node-name": "target0",
+ "driver": "qcow2",
+ "file": {
+ "driver": "file",
+ "filename": "drive0.inc1.qcow2"
+ }
+ }
+ }
+
+ <- { "return": {} }
+
#. Issue a new incremental backup command. The only difference here is that we
have changed the target image below.
.. code-block:: QMP
-> {
- "execute": "drive-backup",
+ "execute": "blockdev-backup",
"arguments": {
"device": "drive0",
"bitmap": "bitmap0",
- "target": "drive0.inc1.qcow2",
- "format": "qcow2",
- "sync": "incremental",
- "mode": "existing"
+ "target": "target0",
+ "sync": "incremental"
}
}
@@ -866,20 +899,36 @@ image:
file for you, but you lose control over format options like
compatibility and preallocation presets.
+#. Add target block node:
+
+ .. code-block:: QMP
+
+ -> {
+ "execute": "blockdev-add",
+ "arguments": {
+ "node-name": "target0",
+ "driver": "qcow2",
+ "file": {
+ "driver": "file",
+ "filename": "drive0.inc2.qcow2"
+ }
+ }
+ }
+
+ <- { "return": {} }
+
#. Issue a new incremental backup command. Apart from the new destination
image, there is no difference from the last two examples.
.. code-block:: QMP
-> {
- "execute": "drive-backup",
+ "execute": "blockdev-backup",
"arguments": {
"device": "drive0",
"bitmap": "bitmap0",
- "target": "drive0.inc2.qcow2",
- "format": "qcow2",
- "sync": "incremental",
- "mode": "existing"
+ "target": "target0",
+ "sync": "incremental"
}
}
@@ -930,6 +979,38 @@ point in time.
$ qemu-img create -f qcow2 drive0.full.qcow2 64G
$ qemu-img create -f qcow2 drive1.full.qcow2 64G
+#. Add target block nodes:
+
+ .. code-block:: QMP
+
+ -> {
+ "execute": "blockdev-add",
+ "arguments": {
+ "node-name": "target0",
+ "driver": "qcow2",
+ "file": {
+ "driver": "file",
+ "filename": "drive0.full.qcow2"
+ }
+ }
+ }
+
+ <- { "return": {} }
+
+ -> {
+ "execute": "blockdev-add",
+ "arguments": {
+ "node-name": "target1",
+ "driver": "qcow2",
+ "file": {
+ "driver": "file",
+ "filename": "drive1.full.qcow2"
+ }
+ }
+ }
+
+ <- { "return": {} }
+
#. Create a full (anchor) backup for each drive, with accompanying bitmaps:
.. code-block:: QMP
@@ -953,21 +1034,19 @@ point in time.
}
},
{
- "type": "drive-backup",
+ "type": "blockdev-backup",
"data": {
"device": "drive0",
- "target": "/path/to/drive0.full.qcow2",
- "sync": "full",
- "format": "qcow2"
+ "target": "target0",
+ "sync": "full"
}
},
{
- "type": "drive-backup",
+ "type": "blockdev-backup",
"data": {
"device": "drive1",
- "target": "/path/to/drive1.full.qcow2",
- "sync": "full",
- "format": "qcow2"
+ "target": "target1",
+ "sync": "full"
}
}
]
@@ -1016,6 +1095,38 @@ point in time.
$ qemu-img create -f qcow2 drive1.inc0.qcow2 \
-b drive1.full.qcow2 -F qcow2
+#. Add target block nodes:
+
+ .. code-block:: QMP
+
+ -> {
+ "execute": "blockdev-add",
+ "arguments": {
+ "node-name": "target0",
+ "driver": "qcow2",
+ "file": {
+ "driver": "file",
+ "filename": "drive0.inc0.qcow2"
+ }
+ }
+ }
+
+ <- { "return": {} }
+
+ -> {
+ "execute": "blockdev-add",
+ "arguments": {
+ "node-name": "target1",
+ "driver": "qcow2",
+ "file": {
+ "driver": "file",
+ "filename": "drive1.inc0.qcow2"
+ }
+ }
+ }
+
+ <- { "return": {} }
+
#. Issue a multi-drive incremental push backup transaction:
.. code-block:: QMP
@@ -1025,25 +1136,21 @@ point in time.
"arguments": {
"actions": [
{
- "type": "drive-backup",
+ "type": "blockev-backup",
"data": {
"device": "drive0",
"bitmap": "bitmap0",
- "format": "qcow2",
- "mode": "existing",
"sync": "incremental",
- "target": "drive0.inc0.qcow2"
+ "target": "target0"
}
},
{
- "type": "drive-backup",
+ "type": "blockdev-backup",
"data": {
"device": "drive1",
"bitmap": "bitmap0",
- "format": "qcow2",
- "mode": "existing",
"sync": "incremental",
- "target": "drive1.inc0.qcow2"
+ "target": "target1"
}
},
]
@@ -1119,19 +1226,35 @@ described above. This example demonstrates the single-job failure case:
$ qemu-img create -f qcow2 drive0.inc0.qcow2 \
-b drive0.full.qcow2 -F qcow2
+#. Add target block node:
+
+ .. code-block:: QMP
+
+ -> {
+ "execute": "blockdev-add",
+ "arguments": {
+ "node-name": "target0",
+ "driver": "qcow2",
+ "file": {
+ "driver": "file",
+ "filename": "drive0.inc0.qcow2"
+ }
+ }
+ }
+
+ <- { "return": {} }
+
#. Attempt to create an incremental backup via QMP:
.. code-block:: QMP
-> {
- "execute": "drive-backup",
+ "execute": "blockdev-backup",
"arguments": {
"device": "drive0",
"bitmap": "bitmap0",
- "target": "drive0.inc0.qcow2",
- "format": "qcow2",
- "sync": "incremental",
- "mode": "existing"
+ "target": "target0",
+ "sync": "incremental"
}
}
@@ -1164,6 +1287,19 @@ described above. This example demonstrates the single-job failure case:
"event": "BLOCK_JOB_COMPLETED"
}
+#. Remove target node:
+
+ .. code-block:: QMP
+
+ -> {
+ "execute": "blockdev-del",
+ "arguments": {
+ "node-name": "target0",
+ }
+ }
+
+ <- { "return": {} }
+
#. Delete the failed image, and re-create it.
.. code:: bash
@@ -1172,20 +1308,36 @@ described above. This example demonstrates the single-job failure case:
$ qemu-img create -f qcow2 drive0.inc0.qcow2 \
-b drive0.full.qcow2 -F qcow2
+#. Add target block node:
+
+ .. code-block:: QMP
+
+ -> {
+ "execute": "blockdev-add",
+ "arguments": {
+ "node-name": "target0",
+ "driver": "qcow2",
+ "file": {
+ "driver": "file",
+ "filename": "drive0.inc0.qcow2"
+ }
+ }
+ }
+
+ <- { "return": {} }
+
#. Retry the command after fixing the underlying problem, such as
freeing up space on the backup volume:
.. code-block:: QMP
-> {
- "execute": "drive-backup",
+ "execute": "blockdev-backup",
"arguments": {
"device": "drive0",
"bitmap": "bitmap0",
- "target": "drive0.inc0.qcow2",
- "format": "qcow2",
- "sync": "incremental",
- "mode": "existing"
+ "target": "target0",
+ "sync": "incremental"
}
}
@@ -1210,7 +1362,8 @@ described above. This example demonstrates the single-job failure case:
Example: Partial Transactional Failures
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-QMP commands like `drive-backup <qemu-qmp-ref.html#index-drive_002dbackup>`_
+QMP commands like `blockdev-backup
+<qemu-qmp-ref.html#index-blockdev_002dbackup>`_
conceptually only start a job, and so transactions containing these commands
may succeed even if the job it created later fails. This might have surprising
interactions with notions of how a "transaction" ought to behave.
@@ -1240,25 +1393,21 @@ and one succeeds:
"arguments": {
"actions": [
{
- "type": "drive-backup",
+ "type": "blockdev-backup",
"data": {
"device": "drive0",
"bitmap": "bitmap0",
- "format": "qcow2",
- "mode": "existing",
"sync": "incremental",
- "target": "drive0.inc0.qcow2"
+ "target": "target0"
}
},
{
- "type": "drive-backup",
+ "type": "blockdev-backup",
"data": {
"device": "drive1",
"bitmap": "bitmap0",
- "format": "qcow2",
- "mode": "existing",
"sync": "incremental",
- "target": "drive1.inc0.qcow2"
+ "target": "target1"
}
}]
}
@@ -1375,25 +1524,21 @@ applied:
},
"actions": [
{
- "type": "drive-backup",
+ "type": "blockdev-backup",
"data": {
"device": "drive0",
"bitmap": "bitmap0",
- "format": "qcow2",
- "mode": "existing",
"sync": "incremental",
- "target": "drive0.inc0.qcow2"
+ "target": "target0"
}
},
{
- "type": "drive-backup",
+ "type": "blockdev-backup",
"data": {
"device": "drive1",
"bitmap": "bitmap0",
- "format": "qcow2",
- "mode": "existing",
"sync": "incremental",
- "target": "drive1.inc0.qcow2"
+ "target": "target1"
}
}]
}
--
2.31.1
^ permalink raw reply related [flat|nested] 5+ messages in thread* [PULL 3/3] qapi: deprecate drive-backup
2021-11-09 17:52 [PULL 0/3] jobs: deprecate drive-backup qmp command Vladimir Sementsov-Ogievskiy
2021-11-09 17:52 ` [PULL 1/3] docs/block-replication: use blockdev-backup Vladimir Sementsov-Ogievskiy
2021-11-09 17:52 ` [PULL 2/3] docs/interop/bitmaps: " Vladimir Sementsov-Ogievskiy
@ 2021-11-09 17:52 ` Vladimir Sementsov-Ogievskiy
2021-11-10 8:50 ` [PULL 0/3] jobs: deprecate drive-backup qmp command Richard Henderson
3 siblings, 0 replies; 5+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-11-09 17:52 UTC (permalink / raw)
To: qemu-block; +Cc: qemu-devel, armbru, jsnow, eblake, vsementsov, peter.maydell
Modern way is using blockdev-add + blockdev-backup, which provides a
lot more control on how target is opened.
As example of drive-backup problems consider the following:
User of drive-backup expects that target will be opened in the same
cache and aio mode as source. Corresponding logic is in
drive_backup_prepare(), where we take bs->open_flags of source.
It works rather bad if source was added by blockdev-add. Assume source
is qcow2 image. On blockdev-add we should specify aio and cache options
for file child of qcow2 node. What happens next:
drive_backup_prepare() looks at bs->open_flags of qcow2 source node.
But there no BDRV_O_NOCAHE neither BDRV_O_NATIVE_AIO: BDRV_O_NOCAHE is
places in bs->file->bs->open_flags, and BDRV_O_NATIVE_AIO is nowhere,
as file-posix parse options and simply set s->use_linux_aio.
The documentation is updated in a minimal way, so that drive-backup is
noted only as a deprecated command, and blockdev-backup used in most of
places.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
docs/about/deprecated.rst | 11 ++++++
docs/interop/live-block-operations.rst | 47 +++++++++++++++++---------
qapi/block-core.json | 5 ++-
qapi/transaction.json | 6 +++-
4 files changed, 51 insertions(+), 18 deletions(-)
diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst
index 5e514fb443..600031210d 100644
--- a/docs/about/deprecated.rst
+++ b/docs/about/deprecated.rst
@@ -239,6 +239,17 @@ single ``bitmap``, the new ``block-export-add`` uses a list of ``bitmaps``.
Member ``values`` in return value elements with meta-type ``enum`` is
deprecated. Use ``members`` instead.
+``drive-backup`` (since 6.2)
+''''''''''''''''''''''''''''
+
+Use ``blockdev-backup`` in combination with ``blockdev-add`` instead.
+This change primarily separates the creation/opening process of the backup
+target with explicit, separate steps. ``blockdev-backup`` uses mostly the
+same arguments as ``drive-backup``, except the ``format`` and ``mode``
+options are removed in favor of using explicit ``blockdev-create`` and
+``blockdev-add`` calls. See :doc:`/interop/live-block-operations` for
+details.
+
System accelerators
-------------------
diff --git a/docs/interop/live-block-operations.rst b/docs/interop/live-block-operations.rst
index 814c29bbe1..39e62c9915 100644
--- a/docs/interop/live-block-operations.rst
+++ b/docs/interop/live-block-operations.rst
@@ -116,8 +116,8 @@ QEMU block layer supports.
(3) ``drive-mirror`` (and ``blockdev-mirror``): Synchronize a running
disk to another image.
-(4) ``drive-backup`` (and ``blockdev-backup``): Point-in-time (live) copy
- of a block device to a destination.
+(4) ``blockdev-backup`` (and the deprecated ``drive-backup``):
+ Point-in-time (live) copy of a block device to a destination.
.. _`Interacting with a QEMU instance`:
@@ -555,13 +555,14 @@ Currently, there are four different kinds:
(3) ``none`` -- Synchronize only the new writes from this point on.
- .. note:: In the case of ``drive-backup`` (or ``blockdev-backup``),
- the behavior of ``none`` synchronization mode is different.
- Normally, a ``backup`` job consists of two parts: Anything
- that is overwritten by the guest is first copied out to
- the backup, and in the background the whole image is
- copied from start to end. With ``sync=none``, it's only
- the first part.
+ .. note:: In the case of ``blockdev-backup`` (or deprecated
+ ``drive-backup``), the behavior of ``none``
+ synchronization mode is different. Normally, a
+ ``backup`` job consists of two parts: Anything that is
+ overwritten by the guest is first copied out to the
+ backup, and in the background the whole image is copied
+ from start to end. With ``sync=none``, it's only the
+ first part.
(4) ``incremental`` -- Synchronize content that is described by the
dirty bitmap
@@ -928,19 +929,22 @@ Shutdown the guest, by issuing the ``quit`` QMP command::
}
-Live disk backup --- ``drive-backup`` and ``blockdev-backup``
--------------------------------------------------------------
+Live disk backup --- ``blockdev-backup`` and the deprecated``drive-backup``
+---------------------------------------------------------------------------
-The ``drive-backup`` (and its newer equivalent ``blockdev-backup``) allows
+The ``blockdev-backup`` (and the deprecated ``drive-backup``) allows
you to create a point-in-time snapshot.
-In this case, the point-in-time is when you *start* the ``drive-backup``
-(or its newer equivalent ``blockdev-backup``) command.
+In this case, the point-in-time is when you *start* the
+``blockdev-backup`` (or deprecated ``drive-backup``) command.
QMP invocation for ``drive-backup``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Note that ``drive-backup`` command is deprecated since QEMU 6.2 and
+will be removed in future.
+
Yet again, starting afresh with our example disk image chain::
[A] <-- [B] <-- [C] <-- [D]
@@ -965,11 +969,22 @@ will be issued, indicating the live block device job operation has
completed, and no further action is required.
+Moving from the deprecated ``drive-backup`` to newer ``blockdev-backup``
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+``blockdev-backup`` differs from ``drive-backup`` in how you specify
+the backup target. With ``blockdev-backup`` you can't specify filename
+as a target. Instead you use ``node-name`` of existing block node,
+which you may add by ``blockdev-add`` or ``blockdev-create`` commands.
+Correspondingly, ``blockdev-backup`` doesn't have ``mode`` and
+``format`` arguments which don't apply to an existing block node. See
+following sections for details and examples.
+
+
Notes on ``blockdev-backup``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-The ``blockdev-backup`` command is equivalent in functionality to
-``drive-backup``, except that it operates at node-level in a Block Driver
+The ``blockdev-backup`` command operates at node-level in a Block Driver
State (BDS) graph.
E.g. the sequence of actions to create a point-in-time backup
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 33e8507d10..1d3dd9cb48 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1709,6 +1709,9 @@
# The operation can be stopped before it has completed using the
# block-job-cancel command.
#
+# Features:
+# @deprecated: This command is deprecated. Use @blockdev-backup instead.
+#
# Returns: - nothing on success
# - If @device is not a valid block device, GenericError
#
@@ -1724,7 +1727,7 @@
#
##
{ 'command': 'drive-backup', 'boxed': true,
- 'data': 'DriveBackup' }
+ 'data': 'DriveBackup', 'features': ['deprecated'] }
##
# @blockdev-backup:
diff --git a/qapi/transaction.json b/qapi/transaction.json
index d175b5f863..381a2df782 100644
--- a/qapi/transaction.json
+++ b/qapi/transaction.json
@@ -54,6 +54,10 @@
# @blockdev-snapshot-sync: since 1.1
# @drive-backup: Since 1.6
#
+# Features:
+# @deprecated: Member @drive-backup is deprecated. Use member
+# @blockdev-backup instead.
+#
# Since: 1.1
##
{ 'enum': 'TransactionActionKind',
@@ -62,7 +66,7 @@
'block-dirty-bitmap-disable', 'block-dirty-bitmap-merge',
'blockdev-backup', 'blockdev-snapshot',
'blockdev-snapshot-internal-sync', 'blockdev-snapshot-sync',
- 'drive-backup' ] }
+ { 'name': 'drive-backup', 'features': [ 'deprecated' ] } ] }
##
# @AbortWrapper:
--
2.31.1
^ permalink raw reply related [flat|nested] 5+ messages in thread