qcow2 eating up space when formattng Centos

qcow2 eating up space when formattng Centos

[day knight]

Hi All,

I am not sure if this is the right behaviour but qcow2 image seems to
grow when Centos is only formatting the image. I mean it goes upto 30
Gig once evrything is formatted and installed and it is minimal Centos
install with no gui or apps just baseline.

OS = Centos5
Virtualization: KVM
Total Qcow2 Image Created = 1TB

Once Centos is installed the qcow2 image shows as around 30 Gig. I
have done several install and they were all less than 4 gig or even
lesser but this seems to not make sense

Can someone please explain what is going on?

thansks

Re: qcow2 eating up space when formattng Centos

[Philipp Hahn]

[-- Attachment #1: Type: text/plain, Size: 1586 bytes --]

Hello Anonnymous,

On Monday 24 October 2011 03:22:15 day knight wrote:
> I am not sure if this is the right behaviour but qcow2 image seems to
> grow when Centos is only formatting the image. I mean it goes upto 30
> Gig once evrything is formatted and installed and it is minimal Centos
> install with no gui or apps just baseline.
>
> OS = Centos5
> Virtualization: KVM
> Total Qcow2 Image Created = 1TB
>
> Once Centos is installed the qcow2 image shows as around 30 Gig. I
> have done several install and they were all less than 4 gig or even
> lesser but this seems to not make sense
>
> Can someone please explain what is going on?

You didn't tell which file system you're using. ext3 needs to initialize its 
meta data (super blocks, inote tables), which are scattered all over the 
image. Depending on your cluster size fpr the qcow2 file, each (small) write 
takes the space of a full cluster. Add to that the meta-data needed by qcow2 
itself (a two-level tree), 30 GiB seem to be okay.

You might want to try ext4 with "delayed allocation" (IMHO enabled by 
default), which doesn't write all over the range, since initialization of the 
superblock and inode tables is delayed until they are acually needed.

Sincerely
Philipp
-- 
Philipp Hahn           Open Source Software Engineer      hahn@univention.de
Univention GmbH        Linux for Your Business        fon: +49 421 22 232- 0
Mary-Somerville-Str.1  D-28359 Bremen                 fax: +49 421 22 232-99
                                                   http://www.univention.de/

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: qcow2 eating up space when formattng Centos

[Philipp Hahn]

[-- Attachment #1: Type: text/plain, Size: 1315 bytes --]

Hello Anonnymous, cc:kvm

On Friday 28 October 2011 04:48:57 day knight wrote:
> I am using ext3 and your explaination makes sense. May I also ask if
> the qcow image automatically adjusts to the size or is it ever
> expanding image?

That depends: Originally Qcow2 only expands, since Qemu/kvm doesn't know 
what's inside your image file, that is if and what file systems it contains 
and how they're structured. Because of that it can't know which blocks are 
free and which are still used.
Since qemu/kvm-0.15 there is TRIM support (which originally was developed for 
flash-storage media), which — I think — is also supported for Qcow2. If your 
Guest-OS supports TRIM, it can tell the underlaying storage media — Qcow2 — 
that blocks are now unused, which Qemu/kvm can then free.

> I mean lets say I have 100 Gig and write 20 Gig on 
> qcow ext3 image and then delete 10 Gig data. Would it show up as
> 120Gig or 110 once the deletion is done

Sincerely
Philipp
-- 
Philipp Hahn           Open Source Software Engineer      hahn@univention.de
Univention GmbH        Linux for Your Business        fon: +49 421 22 232- 0
Mary-Somerville-Str.1  D-28359 Bremen                 fax: +49 421 22 232-99
                                                   http://www.univention.de/

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: qcow2 eating up space when formattng Centos

[day knight]

Thank you Philipp,

I am using ext3 and your explanation makes sense.

May I also ask if the qcow image automatically adjusts to the size or is it ever
expanding image? I mean lets say I have 100 Gig and write 20 Gig on
qcow ext3 image and then delete 10 Gig data. Would it show up as
120Gig or 110 once the deletion is done

Thank you once again :)
Jonah

On Mon, Oct 24, 2011 at 3:17 AM, Philipp Hahn <hahn@univention.de> wrote:
> Hello Anonnymous,
>
> On Monday 24 October 2011 03:22:15 day knight wrote:
>> I am not sure if this is the right behaviour but qcow2 image seems to
>> grow when Centos is only formatting the image. I mean it goes upto 30
>> Gig once evrything is formatted and installed and it is minimal Centos
>> install with no gui or apps just baseline.
>>
>> OS = Centos5
>> Virtualization: KVM
>> Total Qcow2 Image Created = 1TB
>>
>> Once Centos is installed the qcow2 image shows as around 30 Gig. I
>> have done several install and they were all less than 4 gig or even
>> lesser but this seems to not make sense
>>
>> Can someone please explain what is going on?
>
> You didn't tell which file system you're using. ext3 needs to initialize its
> meta data (super blocks, inote tables), which are scattered all over the
> image. Depending on your cluster size fpr the qcow2 file, each (small) write
> takes the space of a full cluster. Add to that the meta-data needed by qcow2
> itself (a two-level tree), 30 GiB seem to be okay.
>
> You might want to try ext4 with "delayed allocation" (IMHO enabled by
> default), which doesn't write all over the range, since initialization of the
> superblock and inode tables is delayed until they are acually needed.
>
> Sincerely
> Philipp
> --
> Philipp Hahn           Open Source Software Engineer      hahn@univention.de
> Univention GmbH        Linux for Your Business        fon: +49 421 22 232- 0
> Mary-Somerville-Str.1  D-28359 Bremen                 fax: +49 421 22 232-99
>                                                   http://www.univention.de/
>