* [linux-lvm] determining snapshot overhead
@ 2015-01-14 7:24 Bram Klein Gunnewiek
2015-01-14 8:34 ` Zdenek Kabelac
0 siblings, 1 reply; 6+ messages in thread
From: Bram Klein Gunnewiek @ 2015-01-14 7:24 UTC (permalink / raw)
To: linux-lvm
Hello,
I am trying to create snapshots of thick volumes that can't break
because they overflow (thin volumes are not an option right now). I
can't find information on how to calculate (in advance) the metadata
overhead of a snapshot when all blocks in the origin volume are changed
(e.g. the snapshot is 100% full). Is there a way to do this?
-Bram
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [linux-lvm] determining snapshot overhead
2015-01-14 7:24 [linux-lvm] determining snapshot overhead Bram Klein Gunnewiek
@ 2015-01-14 8:34 ` Zdenek Kabelac
2015-01-14 10:18 ` Bram Klein Gunnewiek
0 siblings, 1 reply; 6+ messages in thread
From: Zdenek Kabelac @ 2015-01-14 8:34 UTC (permalink / raw)
To: linux-lvm
Dne 14.1.2015 v 08:24 Bram Klein Gunnewiek napsal(a):
> Hello,
>
> I am trying to create snapshots of thick volumes that can't break because they
> overflow (thin volumes are not an option right now). I can't find information
> on how to calculate (in advance) the metadata overhead of a snapshot when all
> blocks in the origin volume are changed (e.g. the snapshot is 100% full). Is
> there a way to do this?
Recent versions of lvm2 (let's say >= 111) are able to determine this size
automatically - just use: 'lvcreate -s -l100%ORIGIN vg/origin'
Regards
Zdenek
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [linux-lvm] determining snapshot overhead
2015-01-14 8:34 ` Zdenek Kabelac
@ 2015-01-14 10:18 ` Bram Klein Gunnewiek
2015-01-14 11:42 ` Zdenek Kabelac
0 siblings, 1 reply; 6+ messages in thread
From: Bram Klein Gunnewiek @ 2015-01-14 10:18 UTC (permalink / raw)
To: LVM general discussion and development
Thanks for your reply. I tested it out and it does not seem to work
(lvm2 version used is 2.02.98-6ubuntu2):
lvcreate -n testvolume3 -L 1G vg
lvcreate -n testvolume3_snapshot -s /dev/vg/testvolume3 -l100%ORIGIN
dd if=/dev/zero of=/dev/vg/testvolume3 bs=1G count=1
After this the snapshot is invalid:
testvolume3_snapshot vg swi-I-s-- 1.00g testvolume3 100.00
I checked out the lvm2 source code and found the methods cow_max_extents
and _cow_max_size in snapshot_manip.c. Do these methods calculate the
exact max size of the snapshot meta data?
On 01/14/2015 09:34 AM, Zdenek Kabelac wrote:
> Dne 14.1.2015 v 08:24 Bram Klein Gunnewiek napsal(a):
>> Hello,
>>
>> I am trying to create snapshots of thick volumes that can't break
>> because they
>> overflow (thin volumes are not an option right now). I can't find
>> information
>> on how to calculate (in advance) the metadata overhead of a snapshot
>> when all
>> blocks in the origin volume are changed (e.g. the snapshot is 100%
>> full). Is
>> there a way to do this?
>
>
> Recent versions of lvm2 (let's say >= 111) are able to determine this
> size automatically - just use: 'lvcreate -s -l100%ORIGIN vg/origin'
>
> Regards
>
> Zdenek
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [linux-lvm] determining snapshot overhead
2015-01-14 10:18 ` Bram Klein Gunnewiek
@ 2015-01-14 11:42 ` Zdenek Kabelac
2015-01-14 12:38 ` Bram Klein Gunnewiek
0 siblings, 1 reply; 6+ messages in thread
From: Zdenek Kabelac @ 2015-01-14 11:42 UTC (permalink / raw)
To: LVM general discussion and development
Dne 14.1.2015 v 11:18 Bram Klein Gunnewiek napsal(a):
> Thanks for your reply. I tested it out and it does not seem to work (lvm2
> version used is 2.02.98-6ubuntu2):
>
yep - too old for this feature being supported.
> lvcreate -n testvolume3 -L 1G vg
> lvcreate -n testvolume3_snapshot -s /dev/vg/testvolume3 -l100%ORIGIN
> dd if=/dev/zero of=/dev/vg/testvolume3 bs=1G count=1
>
> After this the snapshot is invalid:
>
> testvolume3_snapshot vg swi-I-s-- 1.00g testvolume3 100.00
>
> I checked out the lvm2 source code and found the methods cow_max_extents and
> _cow_max_size in snapshot_manip.c. Do these methods calculate the exact max
> size of the snapshot meta data?
yes.
It's probably worth to note there has been some related kernel fixes for old
snaps - otherwise there has been theoretical path to create much bigger
snapshots then the origin size was - though it's nearly impossible to reach
such state.
Zdenek
>
> On 01/14/2015 09:34 AM, Zdenek Kabelac wrote:
>> Dne 14.1.2015 v 08:24 Bram Klein Gunnewiek napsal(a):
>>> Hello,
>>>
>>> I am trying to create snapshots of thick volumes that can't break because they
>>> overflow (thin volumes are not an option right now). I can't find information
>>> on how to calculate (in advance) the metadata overhead of a snapshot when all
>>> blocks in the origin volume are changed (e.g. the snapshot is 100% full). Is
>>> there a way to do this?
>>
>>
>> Recent versions of lvm2 (let's say >= 111) are able to determine this size
>> automatically - just use: 'lvcreate -s -l100%ORIGIN vg/origin'
>>
>> Regards
>>
>> Zdenek
>>
>> _______________________________________________
>> linux-lvm mailing list
>> linux-lvm@redhat.com
>> https://www.redhat.com/mailman/listinfo/linux-lvm
>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [linux-lvm] determining snapshot overhead
2015-01-14 11:42 ` Zdenek Kabelac
@ 2015-01-14 12:38 ` Bram Klein Gunnewiek
2015-01-14 14:26 ` Zdenek Kabelac
0 siblings, 1 reply; 6+ messages in thread
From: Bram Klein Gunnewiek @ 2015-01-14 12:38 UTC (permalink / raw)
To: LVM general discussion and development
On 01/14/2015 12:42 PM, Zdenek Kabelac wrote:
> Dne 14.1.2015 v 11:18 Bram Klein Gunnewiek napsal(a):
>> Thanks for your reply. I tested it out and it does not seem to work
>> (lvm2
>> version used is 2.02.98-6ubuntu2):
>>
>
> yep - too old for this feature being supported.
Ok, then we have to do it ourself since we don't want to go with
backports / custom versions on our systems
>
>> lvcreate -n testvolume3 -L 1G vg
>> lvcreate -n testvolume3_snapshot -s /dev/vg/testvolume3 -l100%ORIGIN
>> dd if=/dev/zero of=/dev/vg/testvolume3 bs=1G count=1
>>
>> After this the snapshot is invalid:
>>
>> testvolume3_snapshot vg swi-I-s-- 1.00g testvolume3 100.00
>>
>> I checked out the lvm2 source code and found the methods
>> cow_max_extents and
>> _cow_max_size in snapshot_manip.c. Do these methods calculate the
>> exact max
>> size of the snapshot meta data?
>
> yes.
>
> It's probably worth to note there has been some related kernel fixes
> for old snaps - otherwise there has been theoretical path to create
> much bigger snapshots then the origin size was - though it's nearly
> impossible to reach such state.
Thanks again. I did some tests and calculations and it seems that the
overhead is ~0.39% of the origin size when chunk sizes of 4KiB are used.
Bigger chunk size means less overhead. If we use 0.5% of the origin
space as meta data overhead we should be on the safe side I think?
> Zdenek
>
>
>>
>> On 01/14/2015 09:34 AM, Zdenek Kabelac wrote:
>>> Dne 14.1.2015 v 08:24 Bram Klein Gunnewiek napsal(a):
>>>> Hello,
>>>>
>>>> I am trying to create snapshots of thick volumes that can't break
>>>> because they
>>>> overflow (thin volumes are not an option right now). I can't find
>>>> information
>>>> on how to calculate (in advance) the metadata overhead of a
>>>> snapshot when all
>>>> blocks in the origin volume are changed (e.g. the snapshot is 100%
>>>> full). Is
>>>> there a way to do this?
>>>
>>>
>>> Recent versions of lvm2 (let's say >= 111) are able to determine
>>> this size
>>> automatically - just use: 'lvcreate -s -l100%ORIGIN vg/origin'
>>>
>>> Regards
>>>
>>> Zdenek
>>>
>>> _______________________________________________
>>> linux-lvm mailing list
>>> linux-lvm@redhat.com
>>> https://www.redhat.com/mailman/listinfo/linux-lvm
>>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>>
>> _______________________________________________
>> linux-lvm mailing list
>> linux-lvm@redhat.com
>> https://www.redhat.com/mailman/listinfo/linux-lvm
>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>>
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [linux-lvm] determining snapshot overhead
2015-01-14 12:38 ` Bram Klein Gunnewiek
@ 2015-01-14 14:26 ` Zdenek Kabelac
0 siblings, 0 replies; 6+ messages in thread
From: Zdenek Kabelac @ 2015-01-14 14:26 UTC (permalink / raw)
To: LVM general discussion and development
Dne 14.1.2015 v 13:38 Bram Klein Gunnewiek napsal(a):
> On 01/14/2015 12:42 PM, Zdenek Kabelac wrote:
>> Dne 14.1.2015 v 11:18 Bram Klein Gunnewiek napsal(a):
>>> Thanks for your reply. I tested it out and it does not seem to work (lvm2
>>> version used is 2.02.98-6ubuntu2):
>>>
>>
>> yep - too old for this feature being supported.
> Ok, then we have to do it ourself since we don't want to go with backports /
> custom versions on our systems
>>
>>> lvcreate -n testvolume3 -L 1G vg
>>> lvcreate -n testvolume3_snapshot -s /dev/vg/testvolume3 -l100%ORIGIN
>>> dd if=/dev/zero of=/dev/vg/testvolume3 bs=1G count=1
>>>
>>> After this the snapshot is invalid:
>>>
>>> testvolume3_snapshot vg swi-I-s-- 1.00g testvolume3 100.00
>>>
>>> I checked out the lvm2 source code and found the methods cow_max_extents and
>>> _cow_max_size in snapshot_manip.c. Do these methods calculate the exact max
>>> size of the snapshot meta data?
>>
>> yes.
>>
>> It's probably worth to note there has been some related kernel fixes for old
>> snaps - otherwise there has been theoretical path to create much bigger
>> snapshots then the origin size was - though it's nearly impossible to reach
>> such state.
>
> Thanks again. I did some tests and calculations and it seems that the overhead
> is ~0.39% of the origin size when chunk sizes of 4KiB are used. Bigger chunk
> size means less overhead. If we use 0.5% of the origin space as meta data
> overhead we should be on the safe side I think?
It cannot be defined as some 'percentage' value - as there are even
combinations of LV sizes and extent size where the percentage might be over
50% (thought those are not really usable in real life).
lvm2 code is the only valid calculation how to estimate full max size of
snapshot for various combinations of those parameters.
Zdenek
>
>
>> Zdenek
>>
>>
>>>
>>> On 01/14/2015 09:34 AM, Zdenek Kabelac wrote:
>>>> Dne 14.1.2015 v 08:24 Bram Klein Gunnewiek napsal(a):
>>>>> Hello,
>>>>>
>>>>> I am trying to create snapshots of thick volumes that can't break because
>>>>> they
>>>>> overflow (thin volumes are not an option right now). I can't find
>>>>> information
>>>>> on how to calculate (in advance) the metadata overhead of a snapshot when
>>>>> all
>>>>> blocks in the origin volume are changed (e.g. the snapshot is 100% full). Is
>>>>> there a way to do this?
>>>>
>>>>
>>>> Recent versions of lvm2 (let's say >= 111) are able to determine this size
>>>> automatically - just use: 'lvcreate -s -l100%ORIGIN vg/origin'
>>>>
>>>> Regards
>>>>
>>>> Zdenek
>>>>
>>>> _______________________________________________
>>>> linux-lvm mailing list
>>>> linux-lvm@redhat.com
>>>> https://www.redhat.com/mailman/listinfo/linux-lvm
>>>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>>>
>>> _______________________________________________
>>> linux-lvm mailing list
>>> linux-lvm@redhat.com
>>> https://www.redhat.com/mailman/listinfo/linux-lvm
>>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>>>
>>
>> _______________________________________________
>> linux-lvm mailing list
>> linux-lvm@redhat.com
>> https://www.redhat.com/mailman/listinfo/linux-lvm
>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2015-01-14 14:26 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-01-14 7:24 [linux-lvm] determining snapshot overhead Bram Klein Gunnewiek
2015-01-14 8:34 ` Zdenek Kabelac
2015-01-14 10:18 ` Bram Klein Gunnewiek
2015-01-14 11:42 ` Zdenek Kabelac
2015-01-14 12:38 ` Bram Klein Gunnewiek
2015-01-14 14:26 ` Zdenek Kabelac
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).