From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45467) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XsxlG-0000iW-L6 for qemu-devel@nongnu.org; Mon, 24 Nov 2014 12:49:23 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XsxlB-000177-Kl for qemu-devel@nongnu.org; Mon, 24 Nov 2014 12:49:18 -0500 Received: from mx1.redhat.com ([209.132.183.28]:58378) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XsxlB-000172-Dk for qemu-devel@nongnu.org; Mon, 24 Nov 2014 12:49:13 -0500 Message-ID: <54736F96.8030503@redhat.com> Date: Mon, 24 Nov 2014 10:49:10 -0700 From: Eric Blake MIME-Version: 1.0 References: <1414336849-21179-1-git-send-email-junmuzi@gmail.com> <1414336849-21179-2-git-send-email-junmuzi@gmail.com> <546F1A4D.50209@redhat.com> In-Reply-To: <546F1A4D.50209@redhat.com> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="olib6OogV6RnGxbeQShQxoUcii0werPCK" Subject: Re: [Qemu-devel] [PATCH v5 1/3] qcow2: Add qcow2_shrink_l1_and_l2_table for qcow2 shrinking List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Max Reitz , Jun Li , qemu-devel@nongnu.org Cc: kwolf@redhat.com, juli@redhat.com, famz@redhat.com, stefanha@redhat.com This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --olib6OogV6RnGxbeQShQxoUcii0werPCK Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 11/21/2014 03:56 AM, Max Reitz wrote: >=20 > Second, why are you truncating the file to offset if offset is smaller > than actual_size? That is always wrong. You are blindly assuming: > (1) that the image consists of only data and no metadata, because you'r= e > using the guest disk size (offset) to shrink the host file > (2) that all that data is tightly packed at the beginning of the file >=20 > (1) is obviously wrong. (2) is wrong as well, because clusters may be > spread all over the host file. We would need defragmentation to solve > that issue. >=20 > Therefore, to shorten the host file, you'd need to walk through the > image and find the highest *host* cluster actually in use. Then you can= > truncate to after that cluster. However, I'd strongly advise against > that because it doesn't bring any actual benefit. Well, there is a minor benefit - the 'query-blockstats' QMP command reports wr_highest_offset, which IS the offset of the highest host cluster already in use, but right now, we only populate that field on the first allocating write, whereas I would like to be able to get at that information even for a read-only file. See also Max's thread for optimizing overlap checks, where I mention this same thing as a side-effect we would get for free when improving overlap checks. > So, as for what I think we do need to do when shrinking (and keep in > mind: The offset given to qcow2_truncate() is the guest size! NOT the > host image size!): >=20 > (1) Determine the first L2 table and the first entry in the table which= > will lie beyond the new guest disk size. > (2) Discard all clusters beginning from there. > (3) Discard all L2 tables which are then completely empty. You may want to also consider discarding refblocks when completely empty, and truncating the size of the reftable if enough trailing refblocks are dropped. On the other hand, just as Max argued that shrinking the L1 table is going to be in the noise, you are also going to be in the noise for trying to shrink the reftable. > (4) Update the header size. >=20 > And that's it. We can either speed up step (2) by implementing it > manually, or we just use bdrv_discard() on the qcow2 BDS (in the > simplest case: bdrv_discard(bs, DIV_ROUND_UP(offset, BDRV_SECTOR_SIZE),= > bs->total_sectors - DIV_ROUND_UP(offset, BDRV_SECTOR_SIZE));. >=20 > We can incorporate step (3) by extending qcow2_discard_clusters() to > free L2 tables when they are empty after discard_single_l2(). But we > don't even have to that now. It's an optimization we can go about later= =2E Same for trimming unused refblocks and/or shrinking the reftable. Also, I think that a future addition of a defragmentation pass would be a more ideal place for tightly packing an image, including trimming reftables to a bare minimum. >=20 > There is one advantage: > - It's extremely simple. It's literally below ten lines of code. >=20 > I think the advantage far outweighs the disadvantage. But I may be > wrong. What do you think? I also like the much simpler approach of leaving holes. --=20 Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org --olib6OogV6RnGxbeQShQxoUcii0werPCK Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: Public key at http://people.redhat.com/eblake/eblake.gpg iQEcBAEBCAAGBQJUc2+WAAoJEKeha0olJ0NqgXwIAI4J2bMPEXC0YzduFU7bY2Fo GyDDsQBloxGKbNuOEcNrretAv5cat9Ed4L/yfgLDc5SR96r+9DSFtWNSzCcsgdJG 0mQM1OeeoxYH3AcvbAPAXptoCA3X5cV82xDSRwZbvf5xyjlSTTSbOy0/RpDn7Zga zbnKsxhIv586EexqTNWcPCKX06i3TQot6M99tVjMCVzCp9Ov2QQpe8bY+xbpbLU7 vLzTeEIhqcvhBuVCT11fRBbvoC9pvO6GeZix3ZLiwHO2/X0ljrWJv8Z8x6M5QgIB 77MwOYMqverLid6kzHIu3ek0AoO8U4+9AcyASdiP2eI6wdFVhjUDmcPLEEZTN0Q= =1R7P -----END PGP SIGNATURE----- --olib6OogV6RnGxbeQShQxoUcii0werPCK--