From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60609) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b8qgP-0006P8-5x for qemu-devel@nongnu.org; Fri, 03 Jun 2016 11:06:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b8qgM-0004Cu-JY for qemu-devel@nongnu.org; Fri, 03 Jun 2016 11:06:43 -0400 References: <1464962711-617992-1-git-send-email-vsementsov@virtuozzo.com> From: Eric Blake Message-ID: <57519CFA.4000400@redhat.com> Date: Fri, 3 Jun 2016 09:06:34 -0600 MIME-Version: 1.0 In-Reply-To: <1464962711-617992-1-git-send-email-vsementsov@virtuozzo.com> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="PTn8WSvxfM1CS1XCU2eTUfjbPpJJ5oVKH" Subject: Re: [Qemu-devel] [PATCH] mirror: add target-zeroed flag List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Vladimir Sementsov-Ogievskiy , qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, lcapitulino@redhat.com, armbru@redhat.com, mreitz@redhat.com, kwolf@redhat.com, jcody@redhat.com, den@virtuozzo.com This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --PTn8WSvxfM1CS1XCU2eTUfjbPpJJ5oVKH Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 06/03/2016 08:05 AM, Vladimir Sementsov-Ogievskiy wrote: > Add target-zeroed flag to allow user specify that target is already > zeroed. With this flag set zeroes which was in source before mirror > start will not be copyed. With this flag set, any runs of zeroes in the source before the mirror starts will not be copied. >=20 > Without this libvirt migration of empty disk takes too long time. >=20 > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- >=20 > I've tested it with > time virsh migrate --live test qemu+ssh://other_node/system --copy-stor= age-all Presumably with a libvirt patch to turn on the optional flag. I'm not sure I like this patch. Libvirt uses NBD to implement --copy-storage-all, I think we're better off improving NBD to automatically handle sparse writes, than we are to add a one-off hack that requires libvirt to change. That is, once NBD is smarter, the copy will be faster without needing a tweak. And we ARE working on making NBD smarter (one of my goals for the 2.7 release is to get all the sparse file additions to NBD implemented) That said, I'll still review it. >=20 > Without 'target-zeroed' libvirt migration of vm with empty qcow2 disk o= f > 400Mb to another node takes for me more than 5 minutes. Migration of 5G= b > disk was not finished in 28 minutes. >=20 > With new flag on, migration of 16Tb empty disk takes about a minute. >=20 > block/mirror.c | 16 +++++++++++----- > blockdev.c | 9 ++++++++- > hmp.c | 2 +- > include/block/block_int.h | 2 +- > qapi/block-core.json | 5 ++++- > qmp-commands.hx | 4 +++- > 6 files changed, 28 insertions(+), 10 deletions(-) >=20 > +++ b/hmp.c > @@ -1097,7 +1097,7 @@ void hmp_drive_mirror(Monitor *mon, const QDict *= qdict) > false, NULL, false, NULL, > full ? MIRROR_SYNC_MODE_FULL : MIRROR_SYNC_MODE_T= OP, > true, mode, false, 0, false, 0, false, 0, > - false, 0, false, 0, false, true, &err); > + false, 0, false, 0, false, true, false, false, &e= rr); I'd like for my qapi 'box' patches to land, so that this can be simpler. https://lists.gnu.org/archive/html/qemu-devel/2016-05/msg03569.html > +++ b/qapi/block-core.json > @@ -1154,6 +1154,9 @@ > # written. Both will result in identical contents. > # Default is true. (Since 2.4) > # > +# @target-zeroed: #optional Whether target is already zeroed, so most = of zeroes > +# should not be transferred. (Since 2.7) Grammar suggestion: #optional True if target already reads as all zeroes, so that holes in the source need not be transferred. (Since 2.7) > +++ b/qmp-commands.hx > @@ -1632,7 +1632,7 @@ EQMP > .args_type =3D "sync:s,device:B,target:s,speed:i?,mode:s?,for= mat:s?," > "node-name:s?,replaces:s?," > "on-source-error:s?,on-target-error:s?," > - "unmap:b?," > + "unmap:b?,target-zeroed:b?" > "granularity:i?,buf-size:i?", I'm trying to get rid of .args_type (or rather, Marc-Andre's qapi patches, that are stalled on my qapi patches, do the job); but for now this part is correct. > .mhandler.cmd_new =3D qmp_marshal_drive_mirror, > }, > @@ -1674,6 +1674,8 @@ Arguments: > (BlockdevOnError, default 'report') > - "unmap": whether the target sectors should be discarded where source= has only > zeroes. (json-bool, optional, default true) > +- "target-zeroed": whether target is already zeroed, so most of zeroes= should > + not be transferred. (json-bool, optional, default false) Similar wording as above. --=20 Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org --PTn8WSvxfM1CS1XCU2eTUfjbPpJJ5oVKH Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 Comment: Public key at http://people.redhat.com/eblake/eblake.gpg Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBCAAGBQJXUZz6AAoJEKeha0olJ0Nq5hoIAJ4cAPvR9JBZ5tOkKYSg14pz 2YijaYpVIqzHr25TgSKmAxbIm9azAIH4OnItPAllCVfONZSORFkibYPs8HhtdDbG Els3cLXjkgdNWAB0QSUKXr+BQdJPuxGcrXA8xLV5G/P00ZvOJPWHxNrDJL7dDM4C evvEg0XnvOkVVrumDaj0yOqmSIBkZSgTBrfbdBzZYQGJnnZEm3xMFM7cGTChGU1c c4jbjZ5ECNNiS/0lC/ZIjNbKn54vONZUyFK5QNqIW2zACBmj7EyVu5jxN0bKvnPN nbhfkHvpUNROYz44/dezBCVVGAGIxT4GRSiJrLWDwlG9I3Pt9kDSY/7naOw3lZU= =gB00 -----END PGP SIGNATURE----- --PTn8WSvxfM1CS1XCU2eTUfjbPpJJ5oVKH--