qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: Vladimir Sementsov-Ogievskiy <v.sementsov-og@mail.ru>
To: qemu-block@nongnu.org
Cc: qemu-devel@nongnu.org, kwolf@redhat.com, hreitz@redhat.com
Subject: Re: [PATCH v4 00/45] Transactional block-graph modifying API
Date: Wed, 30 Mar 2022 22:03:56 +0300	[thread overview]
Message-ID: <4d56f439-b1b4-324c-c042-848b5f98c820@mail.ru> (raw)
In-Reply-To: <20220329204107.411011-1-v.sementsov-og@mail.ru>

Outdated :(. I should rebase it on master, there are a lot of conflicts after Emanuele's series.

29.03.2022 23:40, Vladimir Sementsov-Ogievskiy wrote:
> Hi all!
> 
> That's a big series, which unites some of my previous ones, and
> completes them with necessary additions to finally implement block-graph
> modifying API. The series is called "v4" as it inherits
> "[PATCH v3 00/11] blockdev-replace" (among other things).
> 
> After this series, we have blockdev-add, blockdev-del and
> x-blockdev-replace transaction actions, which allows to insert and
> remove filters.
> 
> Additional challenge is to avoid intermediate permission update. That's
> and existing paradigm of block graph modifications: first do all the
> modifications and then refresh the permissions. Now we should bring this
> paradigm to block-graph modifying transactions: if several graph
> modifying commands are sequential in one transaction, permission are
> updated after the last of these commands. The application of this is
> possibility to correct copy-before-write filter permission requirements
> (see last patch).
> 
> I now unite all these things into one series because:
>   - they depend on each other and I have to rebase them together when
>   something needs fix or refactoring
>   - just to resend with my new email address
> If needed, parts may go in separate, and I can split them again if
> necessary.
> 
> So, what is here:
> 
> 1. "[PATCH 00/14] block: cleanup backing and file handling" series,
> unchanged:
> 
>    block: BlockDriver: add .filtered_child_is_backing field
>    block: introduce bdrv_open_file_child() helper
>    block/blklogwrites: don't care to remove bs->file child on failure
>    test-bdrv-graph-mod: update test_parallel_perm_update test case
>    tests-bdrv-drain: bdrv_replace_test driver: declare supports_backing
>    test-bdrv-graph-mod: fix filters to be filters
>    block: document connection between child roles and
>      bs->backing/bs->file
>    block/snapshot: stress that we fallback to primary child
>    Revert "block: Let replace_child_noperm free children"
>    Revert "block: Let replace_child_tran keep indirect pointer"
>    Revert "block: Restructure remove_file_or_backing_child()"
>    Revert "block: Pass BdrvChild ** to replace_child_noperm"
>    block: Manipulate bs->file / bs->backing pointers in .attach/.detach
>    block/snapshot: drop indirection around bdrv_snapshot_fallback_ptr
> 
> 2. implement bdrv_unref_tran() - the key thing to implement blockdev-del
> transaction action later.
> This part inherits from "[PATCH 00/14] block: blockdev-del force=false".
> Still force=false is not realized and qcow2 is untouched, as the target
> now is transactional removement.
> 
>    block: refactor bdrv_remove_file_or_backing_child to bdrv_remove_child
>    block: drop bdrv_detach_child()
>    block: drop bdrv_remove_filter_or_cow_child
>    block: bdrv_refresh_perms(): allow external tran
>    block: refactor bdrv_list_refresh_perms to allow any list of nodes
>    block: make permission update functions public
>    block: add bdrv_try_set_aio_context_tran transaction action
>    block: implemet bdrv_unref_tran()
> 
> 3. Move blockdev.c transactions to util/transactions.c API.
> 
>    blockdev: refactor transaction to use Transaction API
>    blockdev: transactions: rename some things
>    blockdev: qmp_transaction: refactor loop to classic for
>    blockdev: transaction: refactor handling transaction properties
>    blockdev: qmp_transaction: drop extra generic layer
> 
> 4. add blockdev-del transaction action
> 
>    qapi: block: add blockdev-del transaction action
> 
> 5. add blockdev-add transaction action
> (inherits from "[PATCH 0/2] blockdev-add transaction")
> 
>    block: introduce BDRV_O_NOPERM flag
>    block: bdrv_insert_node(): use BDRV_O_NOPERM
>    qapi: block: add blockdev-add transaction action
>    iotests: add blockdev-add-transaction
> 
> 6. add x-blockdev-replace command and transaction action
> (inherits from "[PATCH v3 00/11] blockdev-replace")
> 
>    block-backend: blk_root(): drop const specifier on return type
>    block/export: add blk_by_export_id()
>    block: make bdrv_find_child() function public
>    block: bdrv_replace_child_bs(): move to external transaction
>    qapi: add x-blockdev-replace command
>    qapi: add x-blockdev-replace transaction action
>    block: bdrv_get_xdbg_block_graph(): report export ids
>    iotests.py: qemu_img_create: use imgfmt by default
>    iotests.py: introduce VM.assert_edges_list() method
>    iotests.py: add VM.qmp_check() helper
>    iotests: add filter-insertion
> 
> 7. Correct permission scheme of copy-before-write filter, with help of
> new design of graph-modifying API.
> 
>    block: bdrv_open_inherit: create BlockBackend only when necessary
>    block/copy-before-write: correct permission scheme
> 
>   block.c                                       | 869 ++++++++++--------
>   block/blkdebug.c                              |   9 +-
>   block/blklogwrites.c                          |  11 +-
>   block/blkreplay.c                             |   7 +-
>   block/blkverify.c                             |   9 +-
>   block/block-backend.c                         |  10 +-
>   block/bochs.c                                 |   7 +-
>   block/cloop.c                                 |   7 +-
>   block/commit.c                                |   1 +
>   block/copy-before-write.c                     |  24 +-
>   block/copy-on-read.c                          |   9 +-
>   block/crypto.c                                |  11 +-
>   block/dmg.c                                   |   7 +-
>   block/export/export.c                         |  31 +
>   block/filter-compress.c                       |   6 +-
>   block/mirror.c                                |   1 +
>   block/monitor/block-hmp-cmds.c                |   2 +-
>   block/parallels.c                             |   7 +-
>   block/preallocate.c                           |   9 +-
>   block/qcow.c                                  |   6 +-
>   block/qcow2.c                                 |   8 +-
>   block/qed.c                                   |   8 +-
>   block/raw-format.c                            |   4 +-
>   block/replication.c                           |   8 +-
>   block/snapshot.c                              |  60 +-
>   block/throttle.c                              |   8 +-
>   block/vdi.c                                   |   7 +-
>   block/vhdx.c                                  |   7 +-
>   block/vmdk.c                                  |   7 +-
>   block/vpc.c                                   |   7 +-
>   blockdev.c                                    | 818 +++++++++--------
>   include/block/block.h                         |  72 +-
>   include/block/block_int.h                     |  41 +-
>   include/block/export.h                        |   1 +
>   include/sysemu/block-backend.h                |   3 +-
>   qapi/block-core.json                          |  73 +-
>   qapi/transaction.json                         |  35 +-
>   stubs/blk-by-qdev-id.c                        |   9 +
>   stubs/blk-exp-find-by-blk.c                   |   9 +
>   stubs/meson.build                             |   2 +
>   tests/qemu-iotests/iotests.py                 |  23 +
>   .../tests/blockdev-add-transaction            |  52 ++
>   .../tests/blockdev-add-transaction.out        |   6 +
>   tests/qemu-iotests/tests/filter-insertion     | 253 +++++
>   tests/qemu-iotests/tests/filter-insertion.out |   5 +
>   tests/qemu-iotests/tests/image-fleecing       |  29 +-
>   tests/qemu-iotests/tests/image-fleecing.out   |   2 -
>   tests/unit/test-bdrv-drain.c                  |  11 +-
>   tests/unit/test-bdrv-graph-mod.c              |  94 +-
>   49 files changed, 1715 insertions(+), 990 deletions(-)
>   create mode 100644 stubs/blk-by-qdev-id.c
>   create mode 100644 stubs/blk-exp-find-by-blk.c
>   create mode 100755 tests/qemu-iotests/tests/blockdev-add-transaction
>   create mode 100644 tests/qemu-iotests/tests/blockdev-add-transaction.out
>   create mode 100755 tests/qemu-iotests/tests/filter-insertion
>   create mode 100644 tests/qemu-iotests/tests/filter-insertion.out
> 


-- 
Best regards,
Vladimir


      parent reply	other threads:[~2022-03-30 19:06 UTC|newest]

Thread overview: 47+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-03-29 20:40 [PATCH v4 00/45] Transactional block-graph modifying API Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 01/45] block: BlockDriver: add .filtered_child_is_backing field Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 02/45] block: introduce bdrv_open_file_child() helper Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 03/45] block/blklogwrites: don't care to remove bs->file child on failure Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 04/45] test-bdrv-graph-mod: update test_parallel_perm_update test case Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 05/45] tests-bdrv-drain: bdrv_replace_test driver: declare supports_backing Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 06/45] test-bdrv-graph-mod: fix filters to be filters Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 07/45] block: document connection between child roles and bs->backing/bs->file Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 08/45] block/snapshot: stress that we fallback to primary child Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 09/45] Revert "block: Let replace_child_noperm free children" Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 10/45] Revert "block: Let replace_child_tran keep indirect pointer" Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 11/45] Revert "block: Restructure remove_file_or_backing_child()" Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 12/45] Revert "block: Pass BdrvChild ** to replace_child_noperm" Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 13/45] block: Manipulate bs->file / bs->backing pointers in .attach/.detach Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 14/45] block/snapshot: drop indirection around bdrv_snapshot_fallback_ptr Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 15/45] block: refactor bdrv_remove_file_or_backing_child to bdrv_remove_child Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 16/45] block: drop bdrv_detach_child() Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 17/45] block: drop bdrv_remove_filter_or_cow_child Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 18/45] block: bdrv_refresh_perms(): allow external tran Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 19/45] block: refactor bdrv_list_refresh_perms to allow any list of nodes Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 20/45] block: make permission update functions public Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 21/45] block: add bdrv_try_set_aio_context_tran transaction action Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 22/45] block: implemet bdrv_unref_tran() Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 23/45] blockdev: refactor transaction to use Transaction API Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 24/45] blockdev: transactions: rename some things Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 25/45] blockdev: qmp_transaction: refactor loop to classic for Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 26/45] blockdev: transaction: refactor handling transaction properties Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 27/45] blockdev: qmp_transaction: drop extra generic layer Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 28/45] qapi: block: add blockdev-del transaction action Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 29/45] block: introduce BDRV_O_NOPERM flag Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 30/45] block: bdrv_insert_node(): use BDRV_O_NOPERM Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 31/45] qapi: block: add blockdev-add transaction action Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 32/45] iotests: add blockdev-add-transaction Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 33/45] block-backend: blk_root(): drop const specifier on return type Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 34/45] block/export: add blk_by_export_id() Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 35/45] block: make bdrv_find_child() function public Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 36/45] block: bdrv_replace_child_bs(): move to external transaction Vladimir Sementsov-Ogievskiy
2022-03-29 20:40 ` [PATCH v4 37/45] qapi: add x-blockdev-replace command Vladimir Sementsov-Ogievskiy
2022-03-29 20:41 ` [PATCH v4 38/45] qapi: add x-blockdev-replace transaction action Vladimir Sementsov-Ogievskiy
2022-03-30 10:12 ` [PATCH v4 39/45] block: bdrv_get_xdbg_block_graph(): report export ids Vladimir Sementsov-Ogievskiy
2022-03-30 10:12 ` [PATCH v4 40/45] iotests.py: qemu_img_create: use imgfmt by default Vladimir Sementsov-Ogievskiy
2022-03-30 10:12 ` [PATCH v4 41/45] iotests.py: introduce VM.assert_edges_list() method Vladimir Sementsov-Ogievskiy
2022-03-30 10:12 ` [PATCH v4 42/45] iotests.py: add VM.qmp_check() helper Vladimir Sementsov-Ogievskiy
2022-03-30 10:12 ` [PATCH v4 43/45] iotests: add filter-insertion Vladimir Sementsov-Ogievskiy
2022-03-30 10:12 ` [PATCH v4 44/45] block: bdrv_open_inherit: create BlockBackend only when necessary Vladimir Sementsov-Ogievskiy
2022-03-30 10:12 ` [PATCH v4 45/45] block/copy-before-write: correct permission scheme Vladimir Sementsov-Ogievskiy
2022-03-30 19:03 ` Vladimir Sementsov-Ogievskiy [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=4d56f439-b1b4-324c-c042-848b5f98c820@mail.ru \
    --to=v.sementsov-og@mail.ru \
    --cc=hreitz@redhat.com \
    --cc=kwolf@redhat.com \
    --cc=qemu-block@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).