From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33644) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bWlsy-0005WZ-4I for qemu-devel@nongnu.org; Mon, 08 Aug 2016 10:50:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bWlss-0003uL-7D for qemu-devel@nongnu.org; Mon, 08 Aug 2016 10:50:35 -0400 Date: Mon, 8 Aug 2016 15:50:27 +0100 From: Stefan Hajnoczi Message-ID: <20160808145027.GA25553@stefanha-x1.localdomain> References: <1469602913-20979-1-git-send-email-xiecl.fnst@cn.fujitsu.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="NzB8fVQJ5HfG6fxh" Content-Disposition: inline In-Reply-To: <1469602913-20979-1-git-send-email-xiecl.fnst@cn.fujitsu.com> Subject: Re: [Qemu-devel] [Qemu-block] [PATCH v24 00/12] Block replication for continuous checkpoints List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Changlong Xie Cc: qemu devel , qemu block , Stefan Hajnoczi , Fam Zheng , Max Reitz , Kevin Wolf , Jeff Cody , Wang Weiwei , zhanghailiang , Wen Congyang , Jiang Yunhong , Dong Eddie , Markus Armbruster , "Dr. David Alan Gilbert" , Gonglei , Paolo Bonzini --NzB8fVQJ5HfG6fxh Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jul 27, 2016 at 03:01:41PM +0800, Changlong Xie wrote: > Block replication is a very important feature which is used for > continuous checkpoints(for example: COLO). >=20 > You can get the detailed information about block replication from here: > http://wiki.qemu.org/Features/BlockReplication >=20 > Usage: > Please refer to docs/block-replication.txt >=20 > You can get the patch here: > https://github.com//Pating/qemu/tree/block-replication-v24 >=20 > You can get the patch with framework here: > https://github.com//Pating/qemu/tree/colo_framework_v23 >=20 > TODO: > 1. Continuous block replication. It will be started after basic functions > are accepted. >=20 > Change Log: >=20 > V24: > 1. Address comments from Max > p9: pass NULL to bdrv_lookup_bs(), and introduce bdrv_is_root_node() to c= heck top_bs > p11: perfect @top-id description, and make it #optional=20 > p12: "replication" =3D> "Replication", add docs/block-replication.txt > Note: we need bdrv_is_root_node() in p9, so this patchset is based on kev= in/qmp-node-name,=20 > V23: > 1. Address comments from Stefan and Max, this series introduce p7/p12 > p2. add Copyright for block_backup.h=20 > p7. support configure --disable-replication > p8. update 2.7 to 2.8 > p11. update 2.7 to 2.8, add missing "top-id" > p12. update MAINTAINERS > V22: > 1. Rebase to the lastest code > 2. modify code adapt to the modification of backup_start & commit_active_= start > 3. rewrite io_read & io_write for interface changes=20 > V21: > 1. Rebase to the lastest code > 2. use bdrv_pwrite_zeroes() and BDRV_SECTOR_BITS for p9 > V20 Resend: > 1. Resend to avoid bothering qemu-trivial maintainers > 2. Address comments from Eric, fix header file issue and add a brief comm= it message for p7 > V20: > 1. Rebase to the lastest code > 2. Address comments from stefan > p8:=20 > 1. error_setg() with an error message when check_top_bs() fails.=20 > 2. remove bdrv_ref(s->hidden_disk->bs) since commit 5c438bc6 > 3. use bloc_job_cancel_sync() before active commit > p9:=20 > 1. fix uninitialized 'pattern_buf' > 2. introduce mkstemp(3) to fix unique filenames > 3. use qemu_vfree() for qemu_blockalign() memory > 4. add missing replication_start_all() > 5. remove useless pattern for io_write() > V19: > 1. Rebase to v2.6.0 > 2. Address comments from stefan > p3: a new patch that export interfaces for extra serialization > p8:=20 > 1. call replication_stop() before freeing s->top_id > 2. check top_bs > 3. reopen file readonly in error return paths > 4. enable extra serialization between read and COW > p9: try to hanlde SIGABRT > V18: > p6: add local_err in all replication callbacks to prevent "errp =3D=3D NU= LL" > p7: add missing qemu_iovec_destroy(xxx) > V17: > 1. Rebase to the lastest codes=20 > p2: refactor backup_do_checkpoint addressed comments from Jeff Cody > p4: fix bugs in "drive_add buddy xxx" hmp commands > p6: add "since: 2.7" > p7: fix bug in replication_close(), add missing "qapi/error.h", add test-= replication=20 > p8: add "since: 2.7" > V16: > 1. Rebase to the newest codes > 2. Address comments from Stefan & hailiang > p3: we don't need this patch now > p4: add "top-id" parameters for secondary > p6: fix NULL pointer in replication callbacks, remove unnecessary typedef= s,=20 > add doc comments that explain the semantics of Replication > p7: Refactor AioContext for thread-safe, remove unnecessary get_top_bs() > *Note*: I'm working on replication testcase now, will send out in V17 > V15: > 1. Rebase to the newest codes > 2. Fix typos and coding style addresed Eric's comments > 3. Address Stefan's comments > 1) Make backup_do_checkpoint public, drop the changes on BlockJobDriver > 2) Update the message and description for [PATCH 4/9] > 3) Make replication_(start/stop/do_checkpoint)_all as global interfaces > 4) Introduce AioContext lock to protect start/stop/do_checkpoint callb= acks > 5) Use BdrvChild instead of holding on to BlockDriverState * pointers > 4. Clear BDRV_O_INACTIVE for hidden disk's open_flags since commit 09e0c7= 71 =20 > 5. Introduce replication_get_error_all to check replication status > 6. Remove useless discard interface > V14: > 1. Implement auto complete active commit > 2. Implement active commit block job for replication.c > 3. Address the comments from Stefan, add replication-specific API and data > structure, also remove old block layer APIs > V13: > 1. Rebase to the newest codes > 2. Remove redundant marcos and semicolon in replication.c=20 > 3. Fix typos in block-replication.txt > V12: > 1. Rebase to the newest codes > 2. Use backing reference to replcace 'allow-write-backing-file' > V11: > 1. Reopen the backing file when starting blcok replication if it is not > opened in R/W mode > 2. Unblock BLOCK_OP_TYPE_BACKUP_SOURCE and BLOCK_OP_TYPE_BACKUP_TARGET > when opening backing file > 3. Block the top BDS so there is only one block job for the top BDS and > its backing chain. > V10: > 1. Use blockdev-remove-medium and blockdev-insert-medium to replace backi= ng > reference. > 2. Address the comments from Eric Blake > V9: > 1. Update the error messages > 2. Rebase to the newest qemu > 3. Split child add/delete support. These patches are sent in another patc= hset. > V8: > 1. Address Alberto Garcia's comments > V7: > 1. Implement adding/removing quorum child. Remove the option non-connect. > 2. Simplify the backing refrence option according to Stefan Hajnoczi's su= ggestion > V6: > 1. Rebase to the newest qemu. > V5: > 1. Address the comments from Gong Lei > 2. Speed the failover up. The secondary vm can take over very quickly even > if there are too many I/O requests. > V4: > 1. Introduce a new driver replication to avoid touch nbd and qcow2. > V3: > 1: use error_setg() instead of error_set() > 2. Add a new block job API > 3. Active disk, hidden disk and nbd target uses the same AioContext > 4. Add a testcase to test new hbitmap API > V2: > 1. Redesign the secondary qemu(use image-fleecing) > 2. Use Error objects to return error message > 3. Address the comments from Max Reitz and Eric Blake >=20 > Changlong Xie (5): > Backup: export interfaces for extra serialization > configure: support replication > Introduce new APIs to do replication operation > tests: add unit test case for replication > MAINTAINERS: add maintainer for replication >=20 > Wen Congyang (7): > unblock backup operations in backing file > Backup: clear all bitmap when doing block checkpoint > Link backup into block core > docs: block replication's description > auto complete active commit > Implement new driver for block replication > support replication driver in blockdev-add >=20 > MAINTAINERS | 9 + > Makefile.objs | 1 + > block.c | 17 ++ > block/Makefile.objs | 3 +- > block/backup.c | 59 +++- > block/mirror.c | 13 +- > block/replication.c | 659 +++++++++++++++++++++++++++++++++++++= ++++++ > blockdev.c | 2 +- > configure | 11 + > docs/block-replication.txt | 239 ++++++++++++++++ > include/block/block_backup.h | 39 +++ > include/block/block_int.h | 3 +- > qapi/block-core.json | 36 ++- > qemu-img.c | 2 +- > replication.c | 107 +++++++ > replication.h | 174 ++++++++++++ > tests/.gitignore | 1 + > tests/Makefile.include | 4 + > tests/test-replication.c | 575 +++++++++++++++++++++++++++++++++++++ > 19 files changed, 1937 insertions(+), 17 deletions(-) > create mode 100644 block/replication.c > create mode 100644 docs/block-replication.txt > create mode 100644 include/block/block_backup.h > create mode 100644 replication.c > create mode 100644 replication.h > create mode 100644 tests/test-replication.c >=20 > --=20 > 1.9.3 >=20 >=20 >=20 >=20 Thanks, applied to my block-next tree: https://github.com/stefanha/qemu/commits/block-next Stefan --NzB8fVQJ5HfG6fxh Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJXqJwyAAoJEJykq7OBq3PIMNsH/jpAzgllImvAiHeyNeZLDxve dJkydX0y6HrKkaOST1+MYtFnEYTgpVLb5W1BQl0HL4ZEHVzZImq5rxiebdN/6L/i WLt9ymnV5KTPAsMf81HCrMIvEA/YvMejksuawAL5mLomUUi/J5d8Re/VvWo38hC4 f7AsY+1rYsLyqyBJgWthhoDCVnd+NlOhsouexwux4zNulP09BI75eEWLrTu7o3BN 35nzJ91KsKTyXJPRyV9m7yQH9W/jbkksk4Frwk6/ByXUi0Bgfq2MF+Uyjd6QrE4r fbjuxnhYGKGlV+ReXYdZ8dZn1SqEGi/M89gQEplJEqOURelr4krYlBuxL9y3lME= =ZLHR -----END PGP SIGNATURE----- --NzB8fVQJ5HfG6fxh--