From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:44112) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZKXDK-0003lc-29 for qemu-devel@nongnu.org; Wed, 29 Jul 2015 15:40:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZKXDE-0003yk-Uv for qemu-devel@nongnu.org; Wed, 29 Jul 2015 15:40:29 -0400 Received: from mx1.redhat.com ([209.132.183.28]:51075) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZKXDE-0003xU-KS for qemu-devel@nongnu.org; Wed, 29 Jul 2015 15:40:24 -0400 Message-ID: <55B92C26.1030507@redhat.com> Date: Wed, 29 Jul 2015 15:40:22 -0400 From: John Snow MIME-Version: 1.0 References: <55AD260D.1060005@redhat.com> <014601d0c358$68094240$381bc6c0$@samsung.com> In-Reply-To: <014601d0c358$68094240$381bc6c0$@samsung.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [PATCH] No change in userland tools after resizing qcow2 image List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: =?UTF-8?B?6rmA7YOc7ZWY?= , qemu-devel@nongnu.org Cc: 'Kevin Wolf' , 'Taeha Kim' On 07/20/2015 09:56 PM, =EA=B9=80=ED=83=9C=ED=95=98 wrote: >> On 07/19/2015 05:24 AM, Taeha Kim wrote: >>> Hello, >>> >>> >>> There is no change in userland tools after resizing qcow2 image excep= t >>> file utility. >>> >>> For example when resize qcow2 image, the "file" utility is detectable >>> increased size. >>> However, the "ls", =E2=80=9Cstat=E2=80=9D, and =E2=80=9Cdu=E2=80=9D u= tility still don't know how many >>> change size of image is changed. >>> The following patch enables to let userland tools - ls, du, stat - >>> know and apply changed size after resizing qcow2 image created by the >>> qemu-img tool. >>> Currently, =E2=80=9Cfile=E2=80=9D utility can only know without this = patch. >>> >>> In addtion, "file" utility sometimes don't know qcow2 image format's >>> version as follows. >>> >>> $ file 100G.qcow2 >>> 100G.qcow2: QEMU QCOW Image (Unknown) >>> >>> >>> So, user can't have any information about qcow2 image size. >>> >>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>> Signed-off-by: Taeha Kim diff --git a/block.c >>> b/block.c index d088ee0..93427f8 100644 >>> --- a/block.c >>> +++ b/block.c >>> @@ -2529,6 +2529,9 @@ int bdrv_truncate(BlockDriverState *bs, int64_t >> offset) >>> if (bs->blk) { >>> blk_dev_resize_cb(bs->blk); >>> } >>> + if (ret =3D=3D 0) { >>> + ret =3D truncate(bs->filename, offset); >>> + } >>> } >>> return ret; >>> } >>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>> >>> >>> $ ./qemu/qemu-img --version >>> qemu-img version 2.3.91, Copyright (c) 2004-2008 Fabrice Bellard >>> >>> The how-to-reproduce is as follows with three reproduction. >>> >>> First let=E2=80=99s say that we create a qcow2 image using qemu-img t= ool as >>> follows. There is still no problem. >>> >>> $ ./qemu/qemu-img create -f qcow2 -o preallocation=3Dmetadata 100G.qc= ow2 >>> 100G Formatting '100G.qcow2', fmt=3Dqcow2 size=3D107374182400 >>> encryption=3Doff >>> cluster_size=3D65536 preallocation=3D'metadata' lazy_refcounts=3Doff >>> refcount_bits=3D16 >>> >>> $ ./qemu/qemu-img check 100G.qcow2 >>> No errors were found on the image. >>> 1638400/1638400 =3D 100.00% allocated, 0.00% fragmented, 0.00% >>> compressed clusters Image end offset: 107390828544 >>> >>> $ ls -l 100G.qcow2 >>> -rw-r--r--. 1 devjames devjames 107390828544 Jul 15 09:55 100G.qcow= 2 >>> >>> $ stat 100G.qcow2 >>> File: =E2=80=98100G.qcow2=E2=80=99 >>> Size: 107390828544 Blocks: 32408 IO Block: 4096 regul= ar file >>> Device: fd00h/64768d Inode: 5778245 Links: 1 >>> Access: (0644/-rw-r--r--) Uid: ( 1000/devjames) Gid: ( 1000/devjam= es) >>> Context: unconfined_u:object_r:user_home_t:s0 >>> Access: 2015-07-15 09:55:17.269620129 +0900 >>> Modify: 2015-07-15 09:55:17.269620129 +0900 >>> Change: 2015-07-15 09:55:17.269620129 +0900 >>> Birth: - >>> >>> $ du -bb 100G.qcow2 >>> 107390828544 100G.qcow2 >>> >>> $ file 100G.qcow2 >>> 100G.qcow2: QEMU QCOW Image (v3), 107374182400 bytes >>> >>> But, from now on there is a problem. >>> >>> Second let=E2=80=99s say we resize the qcow2 image size from 100G to = 200G >>> using current qemu-img without the patch. >>> >>> $ ./qemu/qemu-img resize 100G.qcow2 200G Image resized. >>> >>> $ ./qemu/qemu-img check 100G.qcow2 >>> No errors were found on the image. >>> 1638400/3276800 =3D 50.00% allocated, 0.00% fragmented, 0.00% compres= sed >>> clusters Image end offset: 107390894080 >>> >>> $ ls -l 100G.qcow2 >>> -rw-r--r--. 1 devjames devjames 107390832128 Jul 15 10:02 100G.qcow2 >>> >>> $ stat 100G.qcow2 >>> File: =E2=80=98100G.qcow2=E2=80=99 >>> Size: 107390832128 Blocks: 32416 IO Block: 4096 regul= ar file >>> Device: fd00h/64768d Inode: 5778245 Links: 1 >>> Access: (0644/-rw-r--r--) Uid: ( 1000/devjames) Gid: ( 1000/devjam= es) >>> Context: unconfined_u:object_r:user_home_t:s0 >>> Access: 2015-07-15 10:02:04.842425479 +0900 >>> Modify: 2015-07-15 10:02:04.854425682 +0900 >>> Change: 2015-07-15 10:02:04.854425682 +0900 >>> Birth: - >>> >>> $ du -bb 100G.qcow2 >>> 107390832128 100G.qcow2 >>> >>> $ file 100G.qcow2 >>> 100G.qcow2: QEMU QCOW Image (v3), 214748364800 bytes >>> >>> We can see that =E2=80=9Cls=E2=80=9D, =E2=80=9Cstat=E2=80=9D, and =E2= =80=9Cdu=E2=80=9D utilities don=E2=80=99t know change >>> size of qcow2 image except =E2=80=9Cfile=E2=80=9D one. >>> >>> Third let=E2=80=99s say we resize the qcow2 image size from 100G to 2= 00G using >>> qemu-img with the patch. >>> >>> $ ./qemu/qemu-img resize 100G.qcow2 200G Image resized. >>> >>> $ ./qemu/qemu-img check 100G.qcow2 >>> No errors were found on the image. >>> 1638400/3276800 =3D 50.00% allocated, 0.00% fragmented, 0.00% compres= sed >>> clusters Image end offset: 107390894080 >>> >>> $ ls -l 100G.qcow2 >>> -rw-r--r--. 1 devjames devjames 214748364800 Jul 15 10:08 100G.qcow2 >>> >>> $ stat 100G.qcow2 >>> File: =E2=80=98100G.qcow2=E2=80=99 >>> Size: 214748364800 Blocks: 32416 IO Block: 4096 regul= ar file >>> Device: fd00h/64768d Inode: 5778245 Links: 1 >>> Access: (0644/-rw-r--r--) Uid: ( 1000/devjames) Gid: ( 1000/devjam= es) >>> Context: unconfined_u:object_r:user_home_t:s0 >>> Access: 2015-07-15 10:04:37.595018501 +0900 >>> Modify: 2015-07-15 10:08:01.622484709 +0900 >>> Change: 2015-07-15 10:08:01.622484709 +0900 >>> Birth: - >>> >>> $ du -bb 100G.qcow2 >>> 14748364800 100G.qcow2 >>> >>> $ file 100G.qcow2 >>> 100G.qcow2: QEMU QCOW Image (v3), 214748364800 bytes >>> >>> Now we can see above all four utilities know change size of qcow2 ima= ge. >>> So I made the patch above for the "qemu-img" tool so that "ls", >>> =E2=80=9Cstat=E2=80=9D, and =E2=80=9Cdu=E2=80=9D utility can be appli= ed increased size of the qcow2 >>> image file. >>> >> >=20 > Hello, >=20 > Sorry for late response. At last, I found how to add ">" when the email= is replied in outlook. :) >=20 >> It seems to me that the real issue here is that the resize command you= are >> issuing doesn't respect your original pre-allocation desires. >> >=20 > You're right. At first I thought that the change of resizing with pre-a= llocation needs to be applied in user space. > And the "preallocation" option doesn't have full, for example, "preallo= cation=3Dfull". > I expected that this will enable the qcow2 image to fill increased size= fully when preallocated image is resized. >=20 >> When changing the size of the qcow2 file, it generally just updates th= e >> headers and doesn't pre-allocate those storage blocks. The actual size= of >> the .qcow2 image doesn't take up disk space for empty blocks. That's p= art >> of its design. The reason 'file' can tell the "virtual" size of the di= sk >> is because it is reading the header to do so. >> >=20 > If it updates only headers, I think the "preallocation=3Dmetadata" is r= ight. But user still cannot know the exact size of the qcow2 image withou= t the qemu-img tool. > If people who has only qcow2 images will guess or need many time to kno= w the image size. Because currently qemu-img doesn't support shrinking, p= erhaps the user will have slack space unintentionally. >=20 >> ls and friends are telling you the size of the actual file, as you've >> noticed. >> >> Changing the resize mechanisms to automatically truncate to achieve pr= e- >> allocation is wrong, I'm afraid. >=20 > I agree with you. Could you tell me your opinion about adding the resiz= e mechanisms iff use "preallocation=3Dfull" or "preallocation=3Ddata"? > Of course, new patch will have other approach with internal functions r= elated to qcow2 structure, not just simple userspace API. >=20 >> >> I think if anything, we'd want to allow the qemu-img tool to take opti= ons >> for the resize, concerning pre-allocation strategy. >=20 > I didn't understand what you were saying. Could you please explain to m= e one more time? > What is exactly pre-allocation strategy? You mean that current resize f= unction cannot change any more, or have to keep "as-is"? >=20 What I mean to say is that the qemu-img tool doesn't currently appear to accept arguments for its resize function. That is, there's no way to say e.g. qemu-img resize -o preallocation=3Dmetadata 100G.qcow2 200G because there is no valid "-o" flag for "resize" currently. If it is your goal to resize a preallocated image to a new preallocated image using the qemu-img tool, I think changes to qemu-img are what we wa= nt. >> >> --js >> >=20 > Thank you very much. >=20 > Sincerely, > Taeha >=20 >=20 >>> I have created a VM using qcow2 image after patching using this patch= . >>> The 100G.qcow2 has resized 200G is detected and available. >>> $ qemu-kvm -smp 4 -m 4096 100G.qcow2 -cdrom >>> ../iso/Fedora-Live-Workstation-x86_64-21-5.iso >>> >>> Thanks. >>> >>> >>> >>> Sincerely yours, >>> Taeha Kim >>> Senior Engineer, Network Functions Virtualization Part, Samsung >>> Electronics >>> >=20