snapshotting - what data gets shared?

snapshotting - what data gets shared?

[Mathijs Kwik]

Hi all,

I'm used to snapshots with LVM and I would like to compare them to btrfs.

The case I want to compare is the following:
At the moment a snapshot is created, no extra space is needed (maybe
some metadata overhead) and all data is shared between the original
and the snapshot.
In LVM, snapshots work at the block-level, so any changes done to the
original volume trigger a COW to the snapshot.
If LVM is configured to use 4Mb blocks (default), this means that
overwriting a 100k file, will lead to 4Mb "snapshot data" to be backed
up.
A 800Mb file will take around 800Mb.
So, for small files (that are not on the same extent/block) this can
waste quite some space, while for bigger files, or lots of files
"close" to each other, it doesn't matter much.

How is this for btrfs snapshots?
Do they work at the file-level? or also at blocks/extents?

I mean, does changing a 100k file lead to 100k being snapshotted?

What would happen if I have a 20G file (for example a disk image for kvm)?
Would minor changes in that file lead to the entire 20G to be COWed/"backed up"?

Is there a distinction between data and metadata?
Or does touching (ctime/mtime) or visiting (atime) a file cause it to be COWed?

Thanks for any info on this.
Mathijs

Re: snapshotting - what data gets shared?

[Sean Bartell]

On Wed, Jul 14, 2010 at 11:27:39PM +0200, Mathijs Kwik wrote:
> Hi all,
> 
> I'm used to snapshots with LVM and I would like to compare them to btrfs.
> 
> The case I want to compare is the following:
> At the moment a snapshot is created, no extra space is needed (maybe
> some metadata overhead) and all data is shared between the original
> and the snapshot.
> In LVM, snapshots work at the block-level, so any changes done to the
> original volume trigger a COW to the snapshot.
> If LVM is configured to use 4Mb blocks (default), this means that
> overwriting a 100k file, will lead to 4Mb "snapshot data" to be backed
> up.
> A 800Mb file will take around 800Mb.
> So, for small files (that are not on the same extent/block) this can
> waste quite some space, while for bigger files, or lots of files
> "close" to each other, it doesn't matter much.
> 
> How is this for btrfs snapshots?
> Do they work at the file-level? or also at blocks/extents?
> 
> I mean, does changing a 100k file lead to 100k being snapshotted?

Btrfs CoWs file extents, and files can use only the parts of an extent
they need, so a 1-byte change would only require one additional 4K data
block. Of course, metadata also needs to be updated, and will require
a number of additional blocks.

> What would happen if I have a 20G file (for example a disk image for kvm)?
> Would minor changes in that file lead to the entire 20G to be COWed/"backed up"?

No, only the relevant portion.

> Is there a distinction between data and metadata?
> Or does touching (ctime/mtime) or visiting (atime) a file cause it to be COWed?

Metadata is CoWed separately, so there will still only be one copy of
the data.

> Thanks for any info on this.
> Mathijs

Re: snapshotting - what data gets shared?

[Mathijs Kwik]

thanks, sounds very promising

On Thu, Jul 15, 2010 at 12:13 AM, Sean Bartell
<wingedtachikoma@gmail.com> wrote:
> On Wed, Jul 14, 2010 at 11:27:39PM +0200, Mathijs Kwik wrote:
>> Hi all,
>>
>> I'm used to snapshots with LVM and I would like to compare them to btrfs.
>>
>> The case I want to compare is the following:
>> At the moment a snapshot is created, no extra space is needed (maybe
>> some metadata overhead) and all data is shared between the original
>> and the snapshot.
>> In LVM, snapshots work at the block-level, so any changes done to the
>> original volume trigger a COW to the snapshot.
>> If LVM is configured to use 4Mb blocks (default), this means that
>> overwriting a 100k file, will lead to 4Mb "snapshot data" to be backed
>> up.
>> A 800Mb file will take around 800Mb.
>> So, for small files (that are not on the same extent/block) this can
>> waste quite some space, while for bigger files, or lots of files
>> "close" to each other, it doesn't matter much.
>>
>> How is this for btrfs snapshots?
>> Do they work at the file-level? or also at blocks/extents?
>>
>> I mean, does changing a 100k file lead to 100k being snapshotted?
>
> Btrfs CoWs file extents, and files can use only the parts of an extent
> they need, so a 1-byte change would only require one additional 4K data
> block. Of course, metadata also needs to be updated, and will require
> a number of additional blocks.
>
>> What would happen if I have a 20G file (for example a disk image for kvm)?
>> Would minor changes in that file lead to the entire 20G to be COWed/"backed up"?
>
> No, only the relevant portion.
>
>> Is there a distinction between data and metadata?
>> Or does touching (ctime/mtime) or visiting (atime) a file cause it to be COWed?
>
> Metadata is CoWed separately, so there will still only be one copy of
> the data.
>
>> Thanks for any info on this.
>> Mathijs
>

RE: snapshotting - what data gets shared?

[remi.urbillac]

Hi,=20

Just to be sure :=20
LVM snapshots are not COW. They are Copy On First Write.=20
The first time you update one block, the original data needs to be stor=
ed in the snapshot area.=20

The COW, aka Copy On Write, is not a very "good" name, because it can l=
eads to confusion. In fact, the copy means that if you partialy update =
=20

-----Message d'origine-----
De : linux-btrfs-owner@vger.kernel.org [mailto:linux-btrfs-owner@vger.k=
ernel.org] De la part de Mathijs Kwik
Envoy=E9 : jeudi 15 juillet 2010 10:01
=C0 : Mathijs Kwik; linux-btrfs@vger.kernel.org
Objet : Re: snapshotting - what data gets shared?

thanks, sounds very promising

On Thu, Jul 15, 2010 at 12:13 AM, Sean Bartell <wingedtachikoma@gmail.c=
om> wrote:
> On Wed, Jul 14, 2010 at 11:27:39PM +0200, Mathijs Kwik wrote:
>> Hi all,
>>
>> I'm used to snapshots with LVM and I would like to compare them to b=
trfs.
>>
>> The case I want to compare is the following:
>> At the moment a snapshot is created, no extra space is needed (maybe=
=20
>> some metadata overhead) and all data is shared between the original=20
>> and the snapshot.
>> In LVM, snapshots work at the block-level, so any changes done to th=
e=20
>> original volume trigger a COW to the snapshot.
>> If LVM is configured to use 4Mb blocks (default), this means that=20
>> overwriting a 100k file, will lead to 4Mb "snapshot data" to be=20
>> backed up.
>> A 800Mb file will take around 800Mb.
>> So, for small files (that are not on the same extent/block) this can=
=20
>> waste quite some space, while for bigger files, or lots of files=20
>> "close" to each other, it doesn't matter much.
>>
>> How is this for btrfs snapshots?
>> Do they work at the file-level? or also at blocks/extents?
>>
>> I mean, does changing a 100k file lead to 100k being snapshotted?
>
> Btrfs CoWs file extents, and files can use only the parts of an exten=
t=20
> they need, so a 1-byte change would only require one additional 4K=20
> data block. Of course, metadata also needs to be updated, and will=20
> require a number of additional blocks.
>
>> What would happen if I have a 20G file (for example a disk image for=
 kvm)?
>> Would minor changes in that file lead to the entire 20G to be COWed/=
"backed up"?
>
> No, only the relevant portion.
>
>> Is there a distinction between data and metadata?
>> Or does touching (ctime/mtime) or visiting (atime) a file cause it t=
o be COWed?
>
> Metadata is CoWed separately, so there will still only be one copy of=
=20
> the data.
>
>> Thanks for any info on this.
>> Mathijs
>
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in the body of a message to majordomo@vger.kernel.org More majordomo in=
fo at  http://vger.kernel.org/majordomo-info.html

*********************************
This message and any attachments (the "message") are confidential and i=
ntended solely for the addressees.=20
Any unauthorised use or dissemination is prohibited.
Messages are susceptible to alteration.=20
=46rance Telecom Group shall not be liable for the message if altered, =
changed or falsified.
If you are not the intended addressee of this message, please cancel it=
 immediately and inform the sender.
********************************

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: snapshotting - what data gets shared?

[remi.urbillac]

=20
Hi,=20

Just to be sure :=20
LVM snapshots are not COW. They are Copy On First Write (COFW).=20
The first time you update one block, the original data needs to be stor=
ed in the dedicated snapshot area.=20

The COW, aka Copy On Write, is not a very "good" name, because it can l=
eads to confusion. In fact, the copy means that if you partialy update =
one block in the filesystem, you will not update the existing one. You =
will copy it to a new location, and apply the update before writing dat=
a.


When cow, when cofw ?=20
- if you can store the snapshot on an other storage space, it is cofw, =
like snap devices on LVM.
- if you have a lower write performance with a snapshot, then again, it=
 is COFW.
- on COW, snapshots and live filesystem are always on the same disks/ra=
id/pool...=20



Remi


PS : Sorry for the partial send, buggy CTRL key on outlook ^^ ....



-----Message d'origine-----
De : linux-btrfs-owner@vger.kernel.org [mailto:linux-btrfs-owner@vger.k=
ernel.org] De la part de Mathijs Kwik Envoy=E9 : jeudi 15 juillet 2010 =
10:01 =C0 : Mathijs Kwik; linux-btrfs@vger.kernel.org Objet : Re: snaps=
hotting - what data gets shared?

thanks, sounds very promising

On Thu, Jul 15, 2010 at 12:13 AM, Sean Bartell <wingedtachikoma@gmail.c=
om> wrote:
> On Wed, Jul 14, 2010 at 11:27:39PM +0200, Mathijs Kwik wrote:
>> Hi all,
>>
>> I'm used to snapshots with LVM and I would like to compare them to b=
trfs.
>>
>> The case I want to compare is the following:
>> At the moment a snapshot is created, no extra space is needed (maybe=
=20
>> some metadata overhead) and all data is shared between the original=20
>> and the snapshot.
>> In LVM, snapshots work at the block-level, so any changes done to th=
e=20
>> original volume trigger a COW to the snapshot.
>> If LVM is configured to use 4Mb blocks (default), this means that=20
>> overwriting a 100k file, will lead to 4Mb "snapshot data" to be=20
>> backed up.
>> A 800Mb file will take around 800Mb.
>> So, for small files (that are not on the same extent/block) this can=
=20
>> waste quite some space, while for bigger files, or lots of files=20
>> "close" to each other, it doesn't matter much.
>>
>> How is this for btrfs snapshots?
>> Do they work at the file-level? or also at blocks/extents?
>>
>> I mean, does changing a 100k file lead to 100k being snapshotted?
>
> Btrfs CoWs file extents, and files can use only the parts of an exten=
t=20
> they need, so a 1-byte change would only require one additional 4K=20
> data block. Of course, metadata also needs to be updated, and will=20
> require a number of additional blocks.
>
>> What would happen if I have a 20G file (for example a disk image for=
 kvm)?
>> Would minor changes in that file lead to the entire 20G to be COWed/=
"backed up"?
>
> No, only the relevant portion.
>
>> Is there a distinction between data and metadata?
>> Or does touching (ctime/mtime) or visiting (atime) a file cause it t=
o be COWed?
>
> Metadata is CoWed separately, so there will still only be one copy of=
=20
> the data.
>
>> Thanks for any info on this.
>> Mathijs
>
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in the body of a message to majordomo@vger.kernel.org More majordomo in=
fo at  http://vger.kernel.org/majordomo-info.html

*********************************
This message and any attachments (the "message") are confidential and i=
ntended solely for the addressees.=20
Any unauthorised use or dissemination is prohibited.
Messages are susceptible to alteration.=20
=46rance Telecom Group shall not be liable for the message if altered, =
changed or falsified.
If you are not the intended addressee of this message, please cancel it=
 immediately and inform the sender.
********************************

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html