From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1LM23S-0000cW-7r for qemu-devel@nongnu.org; Sun, 11 Jan 2009 10:16:46 -0500 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1LM23P-0000bF-Gu for qemu-devel@nongnu.org; Sun, 11 Jan 2009 10:16:44 -0500 Received: from [199.232.76.173] (port=47858 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1LM23P-0000bB-B1 for qemu-devel@nongnu.org; Sun, 11 Jan 2009 10:16:43 -0500 Received: from mx2.redhat.com ([66.187.237.31]:37346) by monty-python.gnu.org with esmtp (Exim 4.60) (envelope-from ) id 1LM23O-00067o-R6 for qemu-devel@nongnu.org; Sun, 11 Jan 2009 10:16:43 -0500 Message-ID: <496A0D5E.7080501@redhat.com> Date: Sun, 11 Jan 2009 17:16:46 +0200 From: Shahar Frank MIME-Version: 1.0 Subject: Re: [Qemu-devel] [PATCH 0/3] info blockstats (block-qcow2): show highest allocated offset (bytes) References: <49664ACA.9050807@redhat.com> <4966560B.30504@codemonkey.ws> <4969BB61.9070803@redhat.com> In-Reply-To: <4969BB61.9070803@redhat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Reply-To: qemu-devel@nongnu.org List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Uri Lublin wrote: > Anthony Liguori wrote: >> Uri Lublin wrote: >>> From: Uri Lublin >>> >>> This patchset let the user know the highest allocated byte of qcow2 >>> images. >>> Actually it's the first unallocated byte after the highest byte written, >>> cluster-size aligned. >>> >>> The highest allocated byte gives a maximal limit (easy to calculate) >>> to the number of bytes allocated for that image, and may hint how >>> many more allocations can be done before we reach end-of-file (end of >>> host block device). >>> Although there may be many free blocks below that number (allocated >>> and freed) >>> the file system can not deallocate those blocks, and they have to be >>> reused >>> by qemu. Also note that due to fragmentation those free blocks may not >>> be used on next allocations. >>> >>> It can be useful for truncation of backing file images (ftruncate). >>> Also it may be useful for defragmentation later (although we'll need >>> the number of free blocks as well). >> >> I'm having trouble seeing the utility of this as it seems to be not >> really reliable. Surely, after a lot of work, you'll have one block >> far at the end of the file, no? I don't see how knowing this location >> helps practically speaking. Can you explain a little more about what >> you want to use this functionality for? >> > > Currently, qcow2 images can only grow, never shrink. > The main usage would be to trigger an appropriate operation when a > threshold is reached. The threshold and operation are defined by a > management application. > Basically we can do one of the following: > 1. Defragment the qcow2 image (simplest way is to qemu-img convert it, > the best is to do it online if possible). > 2. Allocate more space (especially when using LVM) > > I plan on adding another "blockstat" that shows the number of free > bytes/blocks/clusters for a qcow2 image. This would make it easier to > choose the appropriate operation above. > As Uri wrote this patch is part of a patch set that will include also free clusters statistics. Together these new statistics will enable an image repository system to quickly estimate how "fragmented" is a given image, so it can decide if to perform expensive cleaning/shrinking/defragmenting processes on that image. This highest allocated byte is also critical to let you run qcow2 images over raw devices- it give you a good estimate how much space you still have for the image to expand. This is a bit pessimistic statistic, but due the much too simplistic allocation mechanism within qcow2 and the recent multiple clusters allocations optimizations, this may be also a pretty realistic figure. Note that if you run qcow2 over expandable volumes such as LVM volumes, you can use this figure by monitoring process that will expand the volume once a threshold is passed. Shahar > Thanks, > Uri. > >