From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx3.redhat.com (mx3.redhat.com [172.16.48.32]) by int-mx2.corp.redhat.com (8.13.1/8.13.1) with ESMTP id m7TKAtYh002744 for ; Fri, 29 Aug 2008 16:10:57 -0400 Received: from rv-out-0506.google.com (rv-out-0506.google.com [209.85.198.225]) by mx3.redhat.com (8.13.8/8.13.8) with ESMTP id m7TKAYog018697 for ; Fri, 29 Aug 2008 16:10:40 -0400 Received: by rv-out-0506.google.com with SMTP id f6so989593rvb.51 for ; Fri, 29 Aug 2008 13:10:33 -0700 (PDT) Message-ID: <1c748a490808291310g274df331ifc76fce539b56954@mail.gmail.com> Date: Fri, 29 Aug 2008 13:10:33 -0700 From: "Larry Dickson" MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_16827_1206899.1220040633630" Subject: [linux-lvm] 15 snapshot limit? Reply-To: LVM general discussion and development List-Id: LVM general discussion and development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , List-Id: To: LVM general discussion and development ------=_Part_16827_1206899.1220040633630 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Hi all, With a 768 GB logical volume in a 2235 GB volume group, and 4 GB of memory, I try to set up lots of snapshots (of 45 or 50 GB each) and can't get past 15. Sometimes I'll succeed in generating more than 15 COWs in /dev/mapper, but lvdisplay never shows more than 15 snapshots in the "LV snapshot status source of" listing under the parent volume. I searched and found 15 snapshots mentioned in a couple places like wiki.samba.org/index.php/Shadow_Copies_with_Snapshots - but nothing really authoritative. Is there really such a limit? Or is it a memory requirement - I seem to remember so many MB memory per GB of storage, but can't find that either. Versions are (variants of) lvm2-2.02.38, device-mapper-1.02.26, kernel-smp-2.6.20 (64 bit). TIA, Larry Dickson Cutting Edge Networked Storage ------=_Part_16827_1206899.1220040633630 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline
Hi all,
 
With a 768 GB logical volume in a 2235 GB volume group, and 4 GB of memory, I try to set up lots of snapshots (of 45 or 50 GB each) and can't get past 15. Sometimes I'll succeed in generating more than 15 COWs in /dev/mapper, but lvdisplay never shows more than 15 snapshots in the "LV snapshot status source of" listing under the parent volume. I searched and found 15 snapshots mentioned in a couple places like wiki.samba.org/index.php/Shadow_Copies_with_Snapshots - but nothing really authoritative.
 
Is there really such a limit? Or is it a memory requirement - I seem to remember so many MB memory per GB of storage, but can't find that either. Versions are (variants of) lvm2-2.02.38, device-mapper-1.02.26, kernel-smp-2.6.20 (64 bit).
 
TIA,
 
Larry Dickson
Cutting Edge Networked Storage
------=_Part_16827_1206899.1220040633630-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from ns3.rdu.redhat.com (ns3.rdu.redhat.com [10.11.255.199]) by int-mx1.corp.redhat.com (8.13.1/8.13.1) with ESMTP id m7TKVqDK006479 for ; Fri, 29 Aug 2008 16:31:52 -0400 Received: from [10.32.4.54] (vpn-4-54.str.redhat.com [10.32.4.54]) by ns3.rdu.redhat.com (8.13.8/8.13.8) with ESMTP id m7TKVpJv002451 for ; Fri, 29 Aug 2008 16:31:51 -0400 Message-ID: <48B85CB6.1080403@redhat.com> Date: Fri, 29 Aug 2008 22:31:50 +0200 From: Milan Broz MIME-Version: 1.0 Subject: Re: [linux-lvm] 15 snapshot limit? References: <1c748a490808291310g274df331ifc76fce539b56954@mail.gmail.com> In-Reply-To: <1c748a490808291310g274df331ifc76fce539b56954@mail.gmail.com> Content-Transfer-Encoding: 7bit Reply-To: LVM general discussion and development List-Id: LVM general discussion and development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , List-Id: Content-Type: text/plain; charset="us-ascii" To: LVM general discussion and development Larry Dickson wrote: > With a 768 GB logical volume in a 2235 GB volume group, and 4 GB of > memory, I try to set up lots of snapshots (of 45 or 50 GB each) and > can't get past 15. Sometimes I'll succeed in generating more than 15 > COWs in /dev/mapper, but lvdisplay never shows more than 15 snapshots in > the "LV snapshot status source of" listing under the parent volume. I > searched and found 15 snapshots mentioned in a couple places like > wiki.samba.org/index.php/Shadow_Copies_with_Snapshots > - but > nothing really authoritative. > > Is there really such a limit? Or is it a memory requirement - I seem to > remember so many MB memory per GB of storage, but can't find that > either. Versions are (variants of) lvm2-2.02.38, device-mapper-1.02.26, > kernel-smp-2.6.20 (64 bit). Hi, no, there is no hard limit for snapshots. But IIRC patches which solves some problems related with this (using private dm-io mempool) were commited in 2.6.22 kernel. But anyway, many snapshot from one origin causes serious performance degradation. (Current snapsot implementation is not ideal in this situation - every write to origin is replicated N-times to COW snapshot area - iow: every snapshot is independent and need copy changed area separately.) (But technically I tested >50 snapshot from one LV - should be limited only by available memory in recent kernels). Milan -- mbroz@redhat.com From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx3.redhat.com (mx3.redhat.com [172.16.48.32]) by int-mx2.corp.redhat.com (8.13.1/8.13.1) with ESMTP id m7TLClgA024141 for ; Fri, 29 Aug 2008 17:12:48 -0400 Received: from rv-out-0506.google.com (rv-out-0506.google.com [209.85.198.238]) by mx3.redhat.com (8.13.8/8.13.8) with ESMTP id m7TLCaWB022901 for ; Fri, 29 Aug 2008 17:12:36 -0400 Received: by rv-out-0506.google.com with SMTP id f6so1011063rvb.51 for ; Fri, 29 Aug 2008 14:12:35 -0700 (PDT) Message-ID: <1c748a490808291412w17dd93f9qcd1454979e77c181@mail.gmail.com> Date: Fri, 29 Aug 2008 14:12:35 -0700 From: "Larry Dickson" Subject: Re: [linux-lvm] 15 snapshot limit? In-Reply-To: <48B85CB6.1080403@redhat.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_17239_17036488.1220044355719" References: <1c748a490808291310g274df331ifc76fce539b56954@mail.gmail.com> <48B85CB6.1080403@redhat.com> Reply-To: LVM general discussion and development List-Id: LVM general discussion and development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , List-Id: To: LVM general discussion and development ------=_Part_17239_17036488.1220044355719 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline You said: > (But technically I tested >50 snapshot from one LV - should be limited > only by available memory in recent kernels) What is that limit? I suspect that is what I have run up against. Larry On 8/29/08, Milan Broz wrote: > > Larry Dickson wrote: > > With a 768 GB logical volume in a 2235 GB volume group, and 4 GB of > > memory, I try to set up lots of snapshots (of 45 or 50 GB each) and > > can't get past 15. Sometimes I'll succeed in generating more than 15 > > COWs in /dev/mapper, but lvdisplay never shows more than 15 snapshots in > > the "LV snapshot status source of" listing under the parent volume. I > > searched and found 15 snapshots mentioned in a couple places like > > wiki.samba.org/index.php/Shadow_Copies_with_Snapshots > > - but > > nothing really authoritative. > > > > Is there really such a limit? Or is it a memory requirement - I seem to > > remember so many MB memory per GB of storage, but can't find that > > either. Versions are (variants of) lvm2-2.02.38, device-mapper-1.02.26, > > kernel-smp-2.6.20 (64 bit). > > Hi, > no, there is no hard limit for snapshots. > > But IIRC patches which solves some problems related with this (using > private dm-io > mempool) were commited in 2.6.22 kernel. > But anyway, many snapshot from one origin causes serious performance > degradation. > (Current snapsot implementation is not ideal in this situation - every > write to origin > is replicated N-times to COW snapshot area - iow: every snapshot is > independent and > need copy changed area separately.) > > (But technically I tested >50 snapshot from one LV - should be limited only > by available > memory in recent kernels). > > Milan > -- > mbroz@redhat.com > > > _______________________________________________ > linux-lvm mailing list > linux-lvm@redhat.com > https://www.redhat.com/mailman/listinfo/linux-lvm > read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ > ------=_Part_17239_17036488.1220044355719 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline
You said:
 
> (But technically I tested >50 snapshot from one LV - should be limited
> only by available memory in recent kernels)
 
What is that limit? I suspect that is what I have run up against.
 
Larry
 
On 8/29/08, Milan Broz <mbroz@redhat.com> wrote:
Larry Dickson wrote:
> With a 768 GB logical volume in a 2235 GB volume group, and 4 GB of
> memory, I try to set up lots of snapshots (of 45 or 50 GB each) and
> can't get past 15. Sometimes I'll succeed in generating more than 15
> COWs in /dev/mapper, but lvdisplay never shows more than 15 snapshots in
> the "LV snapshot status source of" listing under the parent volume. I
> searched and found 15 snapshots mentioned in a couple places like
> wiki.samba.org/index.php/Shadow_Copies_with_Snapshots
> <http://wiki.samba.org/index.php/Shadow_Copies_with_Snapshots> - but
> nothing really authoritative.
>
> Is there really such a limit? Or is it a memory requirement - I seem to
> remember so many MB memory per GB of storage, but can't find that
> either. Versions are (variants of) lvm2-2.02.38, device-mapper-1.02.26,
> kernel-smp-2.6.20 (64 bit).

Hi,
no, there is no hard limit for snapshots.

But IIRC patches which solves some problems related with this (using private dm-io
mempool) were commited in 2.6.22 kernel.
But anyway, many snapshot from one origin causes serious performance degradation.
(Current snapsot implementation is not ideal in this situation - every write to origin
is replicated N-times to COW snapshot area - iow: every snapshot is independent and
need copy changed area separately.)

(But technically I tested >50 snapshot from one LV - should be limited only by available
memory in recent kernels).

Milan
--
mbroz@redhat.com


_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

------=_Part_17239_17036488.1220044355719-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx3.redhat.com (mx3.redhat.com [172.16.48.32]) by int-mx2.corp.redhat.com (8.13.1/8.13.1) with ESMTP id m82EVnCf025254 for ; Tue, 2 Sep 2008 10:31:50 -0400 Received: from rv-out-0506.google.com (rv-out-0506.google.com [209.85.198.238]) by mx3.redhat.com (8.13.8/8.13.8) with ESMTP id m82EVX9B018473 for ; Tue, 2 Sep 2008 10:31:33 -0400 Received: by rv-out-0506.google.com with SMTP id f6so2208455rvb.51 for ; Tue, 02 Sep 2008 07:31:32 -0700 (PDT) Message-ID: <1c748a490809020731i39b214ebhc6e76fd964fc75b4@mail.gmail.com> Date: Tue, 2 Sep 2008 07:31:32 -0700 From: "Larry Dickson" Subject: Re: [linux-lvm] 15 snapshot limit? In-Reply-To: <1c748a490808291412w17dd93f9qcd1454979e77c181@mail.gmail.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_4439_5783173.1220365892098" References: <1c748a490808291310g274df331ifc76fce539b56954@mail.gmail.com> <48B85CB6.1080403@redhat.com> <1c748a490808291412w17dd93f9qcd1454979e77c181@mail.gmail.com> Reply-To: LVM general discussion and development List-Id: LVM general discussion and development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , List-Id: To: LVM general discussion and development ------=_Part_4439_5783173.1220365892098 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Sorry, I did not make that last question clear: What (if any) is the formula by which I can predict the number of snapshots that are possible, given a certain size of the parent volume and/or snapshot and a certain amount of available memory? Milan implied there was such a limit, and I seem to remember it, but I can't seem to find it spelled out anywhere. Larry On 8/29/08, Larry Dickson wrote: > You said: > > > (But technically I tested >50 snapshot from one LV - should be limited > > only by available memory in recent kernels) > > What is that limit? I suspect that is what I have run up against. > > Larry > > On 8/29/08, Milan Broz wrote: >> >> Larry Dickson wrote: >> > With a 768 GB logical volume in a 2235 GB volume group, and 4 GB of >> > memory, I try to set up lots of snapshots (of 45 or 50 GB each) and >> > can't get past 15. Sometimes I'll succeed in generating more than 15 >> > COWs in /dev/mapper, but lvdisplay never shows more than 15 snapshots in >> > the "LV snapshot status source of" listing under the parent volume. I >> > searched and found 15 snapshots mentioned in a couple places like >> > wiki.samba.org/index.php/Shadow_Copies_with_Snapshots >> > - but >> > nothing really authoritative. >> > >> > Is there really such a limit? Or is it a memory requirement - I seem to >> > remember so many MB memory per GB of storage, but can't find that >> > either. Versions are (variants of) lvm2-2.02.38, device-mapper-1.02.26, >> > kernel-smp-2.6.20 (64 bit). >> >> Hi, >> no, there is no hard limit for snapshots. >> >> But IIRC patches which solves some problems related with this (using >> private dm-io >> mempool) were commited in 2.6.22 kernel. >> But anyway, many snapshot from one origin causes serious performance >> degradation. >> (Current snapsot implementation is not ideal in this situation - every >> write to origin >> is replicated N-times to COW snapshot area - iow: every snapshot is >> independent and >> need copy changed area separately.) >> >> (But technically I tested >50 snapshot from one LV - should be limited >> only by available >> memory in recent kernels). >> >> Milan >> -- >> mbroz@redhat.com >> >> >> _______________________________________________ >> linux-lvm mailing list >> linux-lvm@redhat.com >> https://www.redhat.com/mailman/listinfo/linux-lvm >> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ >> > > ------=_Part_4439_5783173.1220365892098 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline
Sorry, I did not make that last question clear:
 
What (if any) is the formula by which I can predict the number of snapshots that are possible, given a certain size of the parent volume and/or snapshot and a certain amount of available memory? Milan implied there was such a limit, and I seem to remember it, but I can't seem to find it spelled out anywhere.
 
Larry

On 8/29/08, Larry Dickson <ldickson@cuttedge.com> wrote:
You said:
 
> (But technically I tested >50 snapshot from one LV - should be limited
> only by available memory in recent kernels)
 
What is that limit? I suspect that is what I have run up against.
 
Larry
 
On 8/29/08, Milan Broz <mbroz@redhat.com> wrote:
Larry Dickson wrote:
> With a 768 GB logical volume in a 2235 GB volume group, and 4 GB of
> memory, I try to set up lots of snapshots (of 45 or 50 GB each) and
> can't get past 15. Sometimes I'll succeed in generating more than 15
> COWs in /dev/mapper, but lvdisplay never shows more than 15 snapshots in
> the "LV snapshot status source of" listing under the parent volume. I
> searched and found 15 snapshots mentioned in a couple places like
> wiki.samba.org/index.php/Shadow_Copies_with_Snapshots
> <http://wiki.samba.org/index.php/Shadow_Copies_with_Snapshots> - but
> nothing really authoritative.
>
> Is there really such a limit? Or is it a memory requirement - I seem to
> remember so many MB memory per GB of storage, but can't find that
> either. Versions are (variants of) lvm2-2.02.38, device-mapper-1.02.26,
> kernel-smp-2.6.20 (64 bit).

Hi,
no, there is no hard limit for snapshots.

But IIRC patches which solves some problems related with this (using private dm-io
mempool) were commited in 2.6.22 kernel.
But anyway, many snapshot from one origin causes serious performance degradation.
(Current snapsot implementation is not ideal in this situation - every write to origin
is replicated N-times to COW snapshot area - iow: every snapshot is independent and
need copy changed area separately.)

(But technically I tested >50 snapshot from one LV - should be limited only by available
memory in recent kernels).

Milan
--
mbroz@redhat.com


_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/


------=_Part_4439_5783173.1220365892098-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx3.redhat.com (mx3.redhat.com [172.16.48.32]) by int-mx2.corp.redhat.com (8.13.1/8.13.1) with ESMTP id m82MKaYR015398 for ; Tue, 2 Sep 2008 18:20:36 -0400 Received: from rv-out-0506.google.com (rv-out-0506.google.com [209.85.198.228]) by mx3.redhat.com (8.13.8/8.13.8) with ESMTP id m82MJsbq025852 for ; Tue, 2 Sep 2008 18:19:55 -0400 Received: by rv-out-0506.google.com with SMTP id f6so2371217rvb.51 for ; Tue, 02 Sep 2008 15:19:54 -0700 (PDT) Message-ID: <1c748a490809021519i294d1d9fp912ce93cbf6f75d2@mail.gmail.com> Date: Tue, 2 Sep 2008 15:19:53 -0700 From: "Larry Dickson" Subject: Re: [linux-lvm] 15 snapshot limit? In-Reply-To: <1c748a490809020731i39b214ebhc6e76fd964fc75b4@mail.gmail.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_10109_6909390.1220393993317" References: <1c748a490808291310g274df331ifc76fce539b56954@mail.gmail.com> <48B85CB6.1080403@redhat.com> <1c748a490808291412w17dd93f9qcd1454979e77c181@mail.gmail.com> <1c748a490809020731i39b214ebhc6e76fd964fc75b4@mail.gmail.com> Reply-To: LVM general discussion and development List-Id: LVM general discussion and development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , List-Id: To: LVM general discussion and development ------=_Part_10109_6909390.1220393993317 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline More details: I found a /var/log/messages note that corresponded exactly to the following bug: ========================== http://www.linux-archive.org/debian-kernel/7067-bug-453254-linux-2-6-18-5-xen-686-unable-create-more-than-32-lvm-snapshots-total.html 11-28-2007, 01:34 AM Peter de Zwart Default Bug#453254: linux-2.6.18-5-xen-686: Unable to create more than 32 LVM snapshots in total Package: linux-2.6.18-5-xen-686 Severity: normal When attempting to create more than 32 LVM snapshots, the following error message is sent from the Kernel: device-mapper: table: 254:142: snapshot: Could not create kcopyd client device-mapper: ioctl: error adding target to table ========================== (End of reference; no follow-up found.) Except that in my case 15 is the max snapshot count, not 32. Increasing memory from 4GB to 6GB does not help. I notice mentions of this "Could not create kcopyd client" bug on the web all the way back to 2004. Does anybody know any workaround? Our customer is not happy about a 15-snapshot limit. Versions are (variants of) lvm2-2.02.38, device-mapper-1.02.26, kernel-smp-2.6.20 (64 bit). Larry On 9/2/08, Larry Dickson wrote: > > Sorry, I did not make that last question clear: > > What (if any) is the formula by which I can predict the number of snapshots > that are possible, given a certain size of the parent volume and/or > snapshot and a certain amount of available memory? Milan implied there was > such a limit, and I seem to remember it, but I can't seem to find it spelled > out anywhere. > > Larry > > On 8/29/08, Larry Dickson wrote: > >> You said: >> >> > (But technically I tested >50 snapshot from one LV - should be limited >> > only by available memory in recent kernels) >> >> What is that limit? I suspect that is what I have run up against. >> >> Larry >> >> On 8/29/08, Milan Broz wrote: >>> >>> Larry Dickson wrote: >>> > With a 768 GB logical volume in a 2235 GB volume group, and 4 GB of >>> > memory, I try to set up lots of snapshots (of 45 or 50 GB each) and >>> > can't get past 15. Sometimes I'll succeed in generating more than 15 >>> > COWs in /dev/mapper, but lvdisplay never shows more than 15 snapshots >>> in >>> > the "LV snapshot status source of" listing under the parent volume. I >>> > searched and found 15 snapshots mentioned in a couple places like >>> > wiki.samba.org/index.php/Shadow_Copies_with_Snapshots >>> > - but >>> > nothing really authoritative. >>> > >>> > Is there really such a limit? Or is it a memory requirement - I seem to >>> > remember so many MB memory per GB of storage, but can't find that >>> > either. Versions are (variants of) lvm2-2.02.38, device-mapper-1.02.26, >>> > kernel-smp-2.6.20 (64 bit). >>> >>> Hi, >>> no, there is no hard limit for snapshots. >>> >>> But IIRC patches which solves some problems related with this (using >>> private dm-io >>> mempool) were commited in 2.6.22 kernel. >>> But anyway, many snapshot from one origin causes serious performance >>> degradation. >>> (Current snapsot implementation is not ideal in this situation - every >>> write to origin >>> is replicated N-times to COW snapshot area - iow: every snapshot is >>> independent and >>> need copy changed area separately.) >>> >>> (But technically I tested >50 snapshot from one LV - should be limited >>> only by available >>> memory in recent kernels). >>> >>> Milan >>> -- >>> mbroz@redhat.com >>> >>> >>> _______________________________________________ >>> linux-lvm mailing list >>> linux-lvm@redhat.com >>> https://www.redhat.com/mailman/listinfo/linux-lvm >>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ >>> >> >> > ------=_Part_10109_6909390.1220393993317 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline
More details: I found a /var/log/messages note that corresponded exactly to the following bug:
==========================

http://www.linux-archive.org/debian-kernel/7067-bug-453254-linux-2-6-18-5-xen-686-unable-create-more-than-32-lvm-snapshots-total.html

11-28-2007, 01:34 AM
Peter de Zwart
    
Default Bug#453254: linux-2.6.18-5-xen-686: Unable to create more than 32 LVM snapshots in total
Package: linux-2.6.18-5-xen-686
Severity: normal


When attempting to create more than 32 LVM snapshots, the following error message is sent from the Kernel:

device-mapper: table: 254:142: snapshot: Could not create kcopyd client
device-mapper: ioctl: error adding target to table

==========================
(End of reference; no follow-up found.) Except that in my case 15 is the max snapshot count, not 32. Increasing memory from 4GB to 6GB does not help. I notice mentions of this "Could not create kcopyd client" bug on the web all the way back to 2004. Does anybody know any workaround? Our customer is not happy about a 15-snapshot limit. Versions are (variants of) lvm2-2.02.38, device-mapper-1.02.26, kernel-smp-2.6.20 (64 bit).

Larry

On 9/2/08, Larry Dickson <ldickson@cuttedge.com> wrote:
Sorry, I did not make that last question clear:
 
What (if any) is the formula by which I can predict the number of snapshots that are possible, given a certain size of the parent volume and/or snapshot and a certain amount of available memory? Milan implied there was such a limit, and I seem to remember it, but I can't seem to find it spelled out anywhere.
 
Larry


On 8/29/08, Larry Dickson <ldickson@cuttedge.com> wrote:
You said:
 
> (But technically I tested >50 snapshot from one LV - should be limited
> only by available memory in recent kernels)
 
What is that limit? I suspect that is what I have run up against.
 
Larry
 
On 8/29/08, Milan Broz <mbroz@redhat.com> wrote:
Larry Dickson wrote:
> With a 768 GB logical volume in a 2235 GB volume group, and 4 GB of
> memory, I try to set up lots of snapshots (of 45 or 50 GB each) and
> can't get past 15. Sometimes I'll succeed in generating more than 15
> COWs in /dev/mapper, but lvdisplay never shows more than 15 snapshots in
> the "LV snapshot status source of" listing under the parent volume. I
> searched and found 15 snapshots mentioned in a couple places like
> wiki.samba.org/index.php/Shadow_Copies_with_Snapshots
> <http://wiki.samba.org/index.php/Shadow_Copies_with_Snapshots> - but
> nothing really authoritative.
>
> Is there really such a limit? Or is it a memory requirement - I seem to
> remember so many MB memory per GB of storage, but can't find that
> either. Versions are (variants of) lvm2-2.02.38, device-mapper-1.02.26,
> kernel-smp-2.6.20 (64 bit).

Hi,
no, there is no hard limit for snapshots.

But IIRC patches which solves some problems related with this (using private dm-io
mempool) were commited in 2.6.22 kernel.
But anyway, many snapshot from one origin causes serious performance degradation.
(Current snapsot implementation is not ideal in this situation - every write to origin
is replicated N-times to COW snapshot area - iow: every snapshot is independent and
need copy changed area separately.)

(But technically I tested >50 snapshot from one LV - should be limited only by available
memory in recent kernels).

Milan
--
mbroz@redhat.com


_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/



------=_Part_10109_6909390.1220393993317-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from ns3.rdu.redhat.com (ns3.rdu.redhat.com [10.11.255.199]) by int-mx1.corp.redhat.com (8.13.1/8.13.1) with ESMTP id m836KTaq032293 for ; Wed, 3 Sep 2008 02:20:29 -0400 Received: from [10.32.4.44] (vpn-4-44.str.redhat.com [10.32.4.44]) by ns3.rdu.redhat.com (8.13.8/8.13.8) with ESMTP id m836KSta020808 for ; Wed, 3 Sep 2008 02:20:29 -0400 Message-ID: <48BE2CAC.3050401@redhat.com> Date: Wed, 03 Sep 2008 08:20:28 +0200 From: Milan Broz MIME-Version: 1.0 Subject: Re: [linux-lvm] 15 snapshot limit? References: <1c748a490808291310g274df331ifc76fce539b56954@mail.gmail.com> <48B85CB6.1080403@redhat.com> <1c748a490808291412w17dd93f9qcd1454979e77c181@mail.gmail.com> <1c748a490809020731i39b214ebhc6e76fd964fc75b4@mail.gmail.com> <1c748a490809021519i294d1d9fp912ce93cbf6f75d2@mail.gmail.com> In-Reply-To: <1c748a490809021519i294d1d9fp912ce93cbf6f75d2@mail.gmail.com> Content-Transfer-Encoding: 7bit Reply-To: LVM general discussion and development List-Id: LVM general discussion and development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , List-Id: Content-Type: text/plain; charset="us-ascii" To: LVM general discussion and development Larry Dickson wrote: > More details: I found a /var/log/messages note that corresponded exactly Please try recent kernel (>2.6.22). Or patched distro kernel, e.g. RHEL/CentOS has patched kernel already since 2007 see https://bugzilla.redhat.com/show_bug.cgi?id=211516 Also recent upstream kernels have a little bit optimized way how to store snapshot exceptions in memory, which should help here too. See https://bugzilla.redhat.com/show_bug.cgi?id=421451 There is no magic formula how many snapshots is possible, previously it was limited by one common mempool (see bug above), and for some reason it fails in cca ~32 snapshot - but it is no longer true (And increasing memory doesn't help with these old unpatched kernels.) With 6GB of memory you should be able create tens of snapshots at least. (But it will be just very slow - every snapshot is independent.) Thanks, Milan From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx3.redhat.com (mx3.redhat.com [172.16.48.32]) by int-mx2.corp.redhat.com (8.13.1/8.13.1) with ESMTP id m8394R9e015158 for ; Wed, 3 Sep 2008 05:04:28 -0400 Received: from sw-gib.mvs.co.il (mvs.co.il [193.16.147.12] (may be forged)) by mx3.redhat.com (8.13.8/8.13.8) with ESMTP id m8394CSK003765 for ; Wed, 3 Sep 2008 05:04:13 -0400 Received: from beta.mvs.co.il (beta [10.253.0.3]) by sw-gib.mvs.co.il (8.13.8/8.13.7) with ESMTP id m83949qb027953 for ; Wed, 3 Sep 2008 12:04:10 +0300 Received: from beta.mvs.co.il (localhost [127.0.0.1]) by beta.mvs.co.il (8.14.1/8.14.1) with ESMTP id m83949Iq010038 for ; Wed, 3 Sep 2008 12:04:09 +0300 Date: Wed, 3 Sep 2008 12:04:09 +0300 Message-Id: <200809030904.m839490W010033@beta.mvs.co.il> From: "Ehud Karni" Subject: Re: [linux-lvm] 15 snapshot limit? In-reply-to: <1c748a490809021519i294d1d9fp912ce93cbf6f75d2@mail.gmail.com> (message from Larry Dickson on Tue, 2 Sep 2008 15:19:53 -0700) References: <1c748a490808291310g274df331ifc76fce539b56954@mail.gmail.com> <48B85CB6.1080403@redhat.com> <1c748a490808291412w17dd93f9qcd1454979e77c181@mail.gmail.com> <1c748a490809020731i39b214ebhc6e76fd964fc75b4@mail.gmail.com> <1c748a490809021519i294d1d9fp912ce93cbf6f75d2@mail.gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Reply-To: ehud@unix.mvs.co.il, LVM general discussion and development List-Id: LVM general discussion and development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , List-Id: Content-Type: text/plain; charset="us-ascii" To: linux-lvm@redhat.com On Tue, 2 Sep 2008 15:19:53 -0700, Larry Dickson wrote: > > in my case 15 is the max > snapshot count, not 32. I followed this thread from the beginning and I have a simple question. What is your Maximum Logical Volume number ? Check it with `vgdisplay' at the "MAX LV" line. If this is the real problem, you can change it with `vgchange'. Ehud. -- Ehud Karni Tel: +972-3-7966-561 /"\ Mivtach - Simon Fax: +972-3-7966-667 \ / ASCII Ribbon Campaign Insurance agencies (USA) voice mail and X Against HTML Mail http://www.mvs.co.il FAX: 1-815-5509341 / \ GnuPG: 98EA398D Better Safe Than Sorry From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx3.redhat.com (mx3.redhat.com [172.16.48.32]) by int-mx2.corp.redhat.com (8.13.1/8.13.1) with ESMTP id m83EmPK4015510 for ; Wed, 3 Sep 2008 10:52:03 -0400 Received: from rv-out-0506.google.com (rv-out-0506.google.com [209.85.198.239]) by mx3.redhat.com (8.13.8/8.13.8) with ESMTP id m83EPeQO019747 for ; Wed, 3 Sep 2008 10:25:41 -0400 Received: by rv-out-0506.google.com with SMTP id f6so2687982rvb.51 for ; Wed, 03 Sep 2008 07:25:40 -0700 (PDT) Message-ID: <1c748a490809030725p2c4da85clf3b195e768212453@mail.gmail.com> Date: Wed, 3 Sep 2008 07:25:39 -0700 From: "Larry Dickson" Subject: Re: [linux-lvm] 15 snapshot limit? In-Reply-To: <200809030904.m839490W010033@beta.mvs.co.il> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_16766_21684092.1220451939306" References: <1c748a490808291310g274df331ifc76fce539b56954@mail.gmail.com> <48B85CB6.1080403@redhat.com> <1c748a490808291412w17dd93f9qcd1454979e77c181@mail.gmail.com> <1c748a490809020731i39b214ebhc6e76fd964fc75b4@mail.gmail.com> <1c748a490809021519i294d1d9fp912ce93cbf6f75d2@mail.gmail.com> <200809030904.m839490W010033@beta.mvs.co.il> Reply-To: LVM general discussion and development List-Id: LVM general discussion and development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , List-Id: To: ehud@unix.mvs.co.il, LVM general discussion and development Cc: Scott Smyth , Michael Ehman , Marcos Busby ------=_Part_16766_21684092.1220451939306 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline It's MAX LV 0 which I suppose means unlimited (or default?) - though I can't seem to find it documented anywhere. I'll check out Milan's upgrades. A "magic formula" sure would be nice, as an unpredictable limit on the number of snapshots is pretty annoying for a customer product. I wonder if something is following a "grab half of what's left" strategy; that would explain a near-independence of memory. There ought to be a way to require ALL resources to be statically assigned. Larry On 9/3/08, Ehud Karni wrote: > > On Tue, 2 Sep 2008 15:19:53 -0700, Larry Dickson wrote: > > > > in my case 15 is the max > > snapshot count, not 32. > > I followed this thread from the beginning and I have a simple question. > What is your Maximum Logical Volume number ? > Check it with `vgdisplay' at the "MAX LV" line. > > If this is the real problem, you can change it with `vgchange'. > > Ehud. > > > -- > Ehud Karni Tel: +972-3-7966-561 /"\ > Mivtach - Simon Fax: +972-3-7966-667 \ / ASCII Ribbon Campaign > Insurance agencies (USA) voice mail and X Against HTML Mail > http://www.mvs.co.il FAX: 1-815-5509341 / \ > GnuPG: 98EA398D Better Safe Than Sorry > > _______________________________________________ > linux-lvm mailing list > linux-lvm@redhat.com > https://www.redhat.com/mailman/listinfo/linux-lvm > read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ > ------=_Part_16766_21684092.1220451939306 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline
It's MAX LV 0 which I suppose means unlimited (or default?) - though I can't seem to find it documented anywhere. I'll check out Milan's upgrades. A "magic formula" sure would be nice, as an unpredictable limit on the number of snapshots is pretty annoying for a customer product. I wonder if something is following a "grab half of what's left" strategy; that would explain a near-independence of memory. There ought to be a way to require ALL resources to be statically assigned.
 
Larry
 
On 9/3/08, Ehud Karni <ehud@unix.mvs.co.il> wrote:
On Tue, 2 Sep 2008 15:19:53 -0700, Larry Dickson wrote:
>
> in my case 15 is the max
> snapshot count, not 32.

I followed this thread from the beginning and I have a simple question.
What is your Maximum Logical Volume number ?
Check it with `vgdisplay' at the "MAX LV" line.

If this is the real problem, you can change it with `vgchange'.

Ehud.


--
Ehud Karni           Tel: +972-3-7966-561  /"\
Mivtach - Simon      Fax: +972-3-7966-667  \ /  ASCII Ribbon Campaign
Insurance agencies   (USA) voice mail and   X   Against   HTML   Mail
http://www.mvs.co.il  FAX:  1-815-5509341  / \
GnuPG: 98EA398D <http://www.keyserver.net/>    Better Safe Than Sorry

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

------=_Part_16766_21684092.1220451939306--