From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34832) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aoTD5-0004Mn-II for qemu-devel@nongnu.org; Fri, 08 Apr 2016 06:00:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aoTD1-00019K-HB for qemu-devel@nongnu.org; Fri, 08 Apr 2016 06:00:15 -0400 Date: Fri, 8 Apr 2016 11:00:07 +0100 From: Stefan Hajnoczi Message-ID: <20160408100007.GE28423@stefanha-x1.localdomain> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="nHwqXXcoX0o6fKCv" Content-Disposition: inline In-Reply-To: Subject: Re: [Qemu-devel] [PATCH for-2.7 v9 00/11] Support streaming to an intermediate layer List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Alberto Garcia Cc: qemu-devel@nongnu.org, Kevin Wolf , qemu-block@nongnu.org, Max Reitz , Stefan Hajnoczi , Jeff Cody --nHwqXXcoX0o6fKCv Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Apr 04, 2016 at 04:43:50PM +0300, Alberto Garcia wrote: > Hi! >=20 > Half a year later :) here's the new version of the intermediate block > streaming patches. CCing Jeff Cody, block jobs maintainer. > Quick summary: the feature was already working since June but there > were two problems that made it impossible to merge the patches. Here's > the full description: >=20 > https://lists.gnu.org/archive/html/qemu-block/2015-12/msg00180.html >=20 > And here's the TL;DR version: >=20 > 1) Opening a BDS in read-write mode with bdrv_reopen() has the side > effect of reopening its backing chain in read-only mode. >=20 > 2) bdrv_reopen() can trigger the completion of an existing block > job. If that job modifies the backing chain in which that reopen > call is operating, it can crash QEMU. >=20 > Now, these problems are only relevant if there are two block jobs > running at the same time in the same backing chain. This is a nice > feature, but it's not essential for the intermediate block streaming > operation. Therefore I decided to disable it in this version of the > series. >=20 > The way it works is that if you run block-stream on an intermediate > node, it will block the active layer as well, preventing other block > jobs in the same chain. >=20 > Other than that there are no significant changes compared to v8, but a > couple of things are different: >=20 > - The code has been rebased > - The patch that introduced block_job_next() has been split into > several: one that adds the function and the rest that use it in > different parts of the QEMU code. > - Some patches have already been merged into QEMU so they have been > removed from the series. > - The test that ran several block-stream operations in parallel now > verifies that they are forbidden. >=20 > I think that's all. Comments and questions are welcome! >=20 > Berto >=20 > v9: > - Rebase the code > - Block the active layer in order to forbid other block jobs in the > same chain. > - Split the patch that adds block_job_next() into 4 (new patches 1-4). > - Replace the test that performs several block-stream operations in > parallel with one that check that they're forbidden. > - Remove patches that have already been merged. >=20 > v8: https://lists.gnu.org/archive/html/qemu-devel/2015-06/msg05754.html > - Rebased on top of Stefan's block branch (0a35bce416) > - The loop that pauses the block jobs in bdrv_drain_all() is now split > in two: one that iterates the list of block jobs to stop them, and > one that iterates the root bds in order to get the aio contexts. >=20 > v7: https://lists.gnu.org/archive/html/qemu-devel/2015-05/msg02580.html > - Rebased against the current master > - Updated bdrv_drain_all() to use the new block_job_next() API. >=20 > v6: https://lists.gnu.org/archive/html/qemu-devel/2015-04/msg03046.html > - fix the no-op test following Max's suggestions >=20 > v5: https://lists.gnu.org/archive/html/qemu-devel/2015-04/msg03006.html > - Fix a few typos > - Minor documentation updates > - Update test_stream_partial() to test no-ops > - New test case: test_stream_parallel() > - New test case: test_stream_overlapping() >=20 > v4: https://lists.gnu.org/archive/html/qemu-devel/2015-04/msg01878.html > - Refactor find_block_job to use the error from bdrv_lookup_bs() > - Don't use QERR_DEVICE_IN_USE in block_job_create() since we can be > dealing with nodes now. > - Fix @device comment in the BlockJobInfo documentation > - stream_start(): simplify the bdrv_reopen() call and use > bdrv_get_device_or_node_name() for error messages. > - Use a different variable name for BlockDriverState *i > - Documentation fixes in docs/live-block-ops.txt > - Update iotest 30 since now test_device_not_found() returns > GenericError > - Fix test case test_stream_partial() > - Add new test case test_stream_intermediate() > - Fix typos >=20 > v3: https://lists.gnu.org/archive/html/qemu-devel/2015-04/msg00806.html > - Keep a list of block jobs and make qmp_query_block_jobs() iterate > over it. >=20 > v2: https://lists.gnu.org/archive/html/qemu-devel/2015-03/msg04798.html > - The 'block-stream' command does not have a 'node-name' parameter > anymore and reuses 'device' for that purpose. > - Block jobs can now be owned by any intermediate node, and not just > by the ones at the root. query-block-jobs is updated to reflect that > change. > - The 'device' parameter of all 'block-job-*' commands can now take a > node name. > - The BlockJobInfo type and all BLOCK_JOB_* events report the node > name in the 'device' field if the node does not have a device name. > - All intermediate nodes are blocked (and checked for blockers) during > the streaming operation. >=20 > v1: https://lists.gnu.org/archive/html/qemu-devel/2015-02/msg04116.html >=20 > Alberto Garcia (11): > block: keep a list of block jobs > block: use the block job list in bdrv_drain_all() > block: use the block job list in qmp_query_block_jobs() > block: use the block job list in bdrv_close() > block: allow block jobs in any arbitrary node > block: Support streaming to an intermediate layer > block: Add QMP support for streaming to an intermediate layer > docs: Document how to stream to an intermediate layer > qemu-iotests: test streaming to an intermediate layer > qemu-iotests: test overlapping block-stream operations > qemu-iotests: test non-overlapping block-stream operations >=20 > block.c | 29 ++++--------- > block/io.c | 21 ++++++---- > block/stream.c | 39 ++++++++++++++++- > blockdev.c | 68 ++++++++++++++++++------------ > blockjob.c | 18 +++++++- > docs/live-block-ops.txt | 31 +++++++++----- > docs/qmp-events.txt | 8 ++-- > include/block/block_int.h | 5 ++- > include/block/blockjob.h | 14 +++++++ > qapi/block-core.json | 30 ++++++++------ > tests/qemu-iotests/030 | 101 +++++++++++++++++++++++++++++++++++++++= +++++- > tests/qemu-iotests/030.out | 4 +- > 12 files changed, 276 insertions(+), 92 deletions(-) >=20 > --=20 > 2.8.0.rc3 >=20 >=20 --nHwqXXcoX0o6fKCv Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJXB4EnAAoJEJykq7OBq3PIPlEH/3z0arc77sropTpZS63z7UWg HXM1XaQdFFTh2jeU8SOOOEs08Timvou9alkucFvur1COdqkkcBWVTPhbz8ihOswT igx1nKqZObH1XV3Y+8FZHiVcZpg0X5mwHqcOlc4LQtkd8hRDox6Aq/Iec9oNdjhR FM2K0hwHOl/FZqkRvvwVDP1rx06L7f2YBl3gvdkRUxJ5rHlrzeq+pkTbHBGgB+5+ peUl5vC0+A7DpELh47LGHmf1ufrjt1kJ0jogtSOwVgaC57yyuGsnS45jpHemdb2N zvvEOXRHgMFO6Ez1tBuT4i8V5K3G4TMUsH4b1ZJej/Liyl+rVWjrxzUwfIfWklw= =suXy -----END PGP SIGNATURE----- --nHwqXXcoX0o6fKCv--