[linux-lvm] Data corruption on large, multi-device filesystem

[linux-lvm] Data corruption on large, multi-device filesystem

[Jens Beyer]

Hi,

I get severe data corruption using an logical volume larger 
then 2 TB. Finally I was able to track down device mappper or 
lvm as last suspects.

My first guess where problems with filesystems but recently 
I tried using md / RAID0 - and didnt have any errors of any 
kind. I would prefer using LVM since we want to use snapshots 
to simplify backup, but I have no clue how to further debug.

On a system with 3 devices each larger then 1 TB and a logical 
volume striped over all devices some data gets corrupted while 
written (or read ?) from disk. This shows up as md5 or crc sums 
changes on sequenced reads of files if filecache is not involved 
(by reading a lot data). 
On ext2fs there are error while writing data (kernel: EXT2-fs error 
(device dm-0): ext2_new_block: Allocating block in system zone -
 block = 722239884), on other filesystems successive fsck/repairs 
shows corrupted metadata.  

The system setup is 
- Three 29160B Adaptec scsi-controller each with one 
  ATA-Disk Raid sized 1240 GB, (dual PIII, HP DL360 G2, 2 GB Ram)
- Volume group over all three devices, logical volume stripped 
  full size (3.7 TB)
- Filesystem either ext2fs/ext3fs (1.34), reiserfs (3.6.13) or 
  xfs (2.6.25)  

- host:~ # lvm version
  LVM version:     2.00.33 (2005-01-07)
  Library version: 1.00.21-ioctl (2005-01-07)
  Driver version:  4.3.0
- 2.6.10 vanilla + 2.6.10-udm1 patches

The problems where initially discovered on 2.6.8, tracked on 2.6.9-udm 
and also occurs if only 2 devices (sum 2.4 TB) are used.

For a limited time I will be able to further debug the system though 
it takes some time to generate more then 2 TB of data 
(max seq read/write rate is ~80 MB/s).

Jens

-- 
Nur tote Fische schwimmen mit dem Strom

Re: [linux-lvm] Data corruption on large, multi-device filesystem

[joe]

I have recently run into this problem also.  I have seen it happen on SuSe 9.2,
Fedora Core 2 and 3, and vanilla kernels 2.6.8.1, 2.6.9, and 2.6.10.
All of my tests were using xfs.

It happens whenever 2 or more devices are striped together with a total volume
size greater than 2TB.  I have played with a single 4TB raid (12x 400GB RAID5)
and did not see any corruption (but I did not fill the disk either).

I initially saw the problem running video files over samba. But have recreated
the problem by simply copying some large (5GB+) files and then checking
md5sums.

I don't see any corruption on the files unless I specify the -i option to
lvcreate.  I usually see data corruption within an hour using my current tests.

Let me know if I can be of any assistance.
Joe


Quoting Jens Beyer <jbe@webde-ag.de>:

>
> Hi,
>
> I get severe data corruption using an logical volume larger
> then 2 TB. Finally I was able to track down device mappper or
> lvm as last suspects.
>
> My first guess where problems with filesystems but recently
> I tried using md / RAID0 - and didnt have any errors of any
> kind. I would prefer using LVM since we want to use snapshots
> to simplify backup, but I have no clue how to further debug.
>
> On a system with 3 devices each larger then 1 TB and a logical
> volume striped over all devices some data gets corrupted while
> written (or read ?) from disk. This shows up as md5 or crc sums
> changes on sequenced reads of files if filecache is not involved
> (by reading a lot data).
> On ext2fs there are error while writing data (kernel: EXT2-fs error
> (device dm-0): ext2_new_block: Allocating block in system zone -
>  block = 722239884), on other filesystems successive fsck/repairs
> shows corrupted metadata.
>
> The system setup is
> - Three 29160B Adaptec scsi-controller each with one
>   ATA-Disk Raid sized 1240 GB, (dual PIII, HP DL360 G2, 2 GB Ram)
> - Volume group over all three devices, logical volume stripped
>   full size (3.7 TB)
> - Filesystem either ext2fs/ext3fs (1.34), reiserfs (3.6.13) or
>   xfs (2.6.25)
>
> - host:~ # lvm version
>   LVM version:     2.00.33 (2005-01-07)
>   Library version: 1.00.21-ioctl (2005-01-07)
>   Driver version:  4.3.0
> - 2.6.10 vanilla + 2.6.10-udm1 patches
>
> The problems where initially discovered on 2.6.8, tracked on 2.6.9-udm
> and also occurs if only 2 devices (sum 2.4 TB) are used.
>
> For a limited time I will be able to further debug the system though
> it takes some time to generate more then 2 TB of data
> (max seq read/write rate is ~80 MB/s).
>
> Jens
>
> --
> Nur tote Fische schwimmen mit dem Strom
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>

Re: [linux-lvm] Data corruption on large, multi-device filesystem

[Randall A. Jones]

joe@eiler.net wrote:

>I have recently run into this problem also.  I have seen it happen on SuSe 9.2,
>Fedora Core 2 and 3, and vanilla kernels 2.6.8.1, 2.6.9, and 2.6.10.
>All of my tests were using xfs.
>
>It happens whenever 2 or more devices are striped together with a total volume
>size greater than 2TB.  I have played with a single 4TB raid (12x 400GB RAID5)
>and did not see any corruption (but I did not fill the disk either).
>
>I initially saw the problem running video files over samba. But have recreated
>the problem by simply copying some large (5GB+) files and then checking
>md5sums.
>
>I don't see any corruption on the files unless I specify the -i option to
>lvcreate.  I usually see data corruption within an hour using my current tests.
>
>  
>
To verify, this corruption you are seeing only happens when you have a 
LV larger than 2TB
and when you use striping specifically with lvcreate -i.
Has anyone experienced data corruption with >2TB LV and no striping?

Randall
-

>Let me know if I can be of any assistance.
>Joe
>
>
>Quoting Jens Beyer <jbe@webde-ag.de>:
>
>  
>
>>Hi,
>>
>>I get severe data corruption using an logical volume larger
>>then 2 TB. Finally I was able to track down device mappper or
>>lvm as last suspects.
>>
>>My first guess where problems with filesystems but recently
>>I tried using md / RAID0 - and didnt have any errors of any
>>kind. I would prefer using LVM since we want to use snapshots
>>to simplify backup, but I have no clue how to further debug.
>>
>>On a system with 3 devices each larger then 1 TB and a logical
>>volume striped over all devices some data gets corrupted while
>>written (or read ?) from disk. This shows up as md5 or crc sums
>>changes on sequenced reads of files if filecache is not involved
>>(by reading a lot data).
>>On ext2fs there are error while writing data (kernel: EXT2-fs error
>>(device dm-0): ext2_new_block: Allocating block in system zone -
>> block = 722239884), on other filesystems successive fsck/repairs
>>shows corrupted metadata.
>>
>>The system setup is
>>- Three 29160B Adaptec scsi-controller each with one
>>  ATA-Disk Raid sized 1240 GB, (dual PIII, HP DL360 G2, 2 GB Ram)
>>- Volume group over all three devices, logical volume stripped
>>  full size (3.7 TB)
>>- Filesystem either ext2fs/ext3fs (1.34), reiserfs (3.6.13) or
>>  xfs (2.6.25)
>>
>>- host:~ # lvm version
>>  LVM version:     2.00.33 (2005-01-07)
>>  Library version: 1.00.21-ioctl (2005-01-07)
>>  Driver version:  4.3.0
>>- 2.6.10 vanilla + 2.6.10-udm1 patches
>>
>>The problems where initially discovered on 2.6.8, tracked on 2.6.9-udm
>>and also occurs if only 2 devices (sum 2.4 TB) are used.
>>
>>For a limited time I will be able to further debug the system though
>>it takes some time to generate more then 2 TB of data
>>(max seq read/write rate is ~80 MB/s).
>>
>>Jens
>>
>>--
>>Nur tote Fische schwimmen mit dem Strom
>>
>>    
>>
-- 
..:.::::
Randall Jones     GST      NASA Goddard Space Flight Center
HPC Visualization Support       http://hpcvis.gsfc.nasa.gov
Scientific Visualization Studio    http://svs.gsfc.nasa.gov
rajones@svs.gsfc.nasa.gov      Code 610.3      301-286-2239

Re: [linux-lvm] Data corruption on large, multi-device filesystem

[Alasdair G Kergon]

On Wed, Jan 19, 2005 at 02:14:43PM -0800, joe@eiler.net wrote:
> I don't see any corruption on the files unless I specify the -i option to
> lvcreate.  
 
That's a useful observation: different code gets run when -i is used, and
I've taken a look and there are two 32-bit limits in that code.  

One appears to be intentional for performance reasons and can be worked 
around by increasing the chunk size (default 64K), though it really should
detect when the limit is exceeded and tell you!

The other unintentional limit (which is the one affecting you I think) 
leads to wrap-around at striped device offsets beyond ~2^32 sectors
i.e. 2^39 bytes into a PV, or ~2^40 bytes (2TB) into an LV in your
case of 2 stripes.  The fix for that is a basic exercise in C.  
[Ref.  stripe_map() in drivers/md/dm-stripe.c]

Alasdair
-- 
agk@redhat.com

Re: [linux-lvm] Data corruption on large, multi-device filesystem

[Alasdair G Kergon]

On Thu, Jan 20, 2005 at 09:06:44AM -0500, Randall A. Jones wrote:
> To verify, this corruption you are seeing only happens when you have a 
> LV larger than 2TB
> and when you use striping specifically with lvcreate -i.
> Has anyone experienced data corruption with >2TB LV and no striping?
 
Does this patch help?


Missing cast causing data corruption on devices with stripes > ~1TB.
--- diff/drivers/md/dm-stripe.c	2005-01-20 17:32:37.000000000 +0000
+++ source/drivers/md/dm-stripe.c	2005-01-20 17:32:26.000000000 +0000
@@ -179,7 +179,7 @@
 
 	bio->bi_bdev = sc->stripe[stripe].dev->bdev;
 	bio->bi_sector = sc->stripe[stripe].physical_start +
-	    (chunk << sc->chunk_shift) + (offset & sc->chunk_mask);
+	    ((sector_t) chunk << sc->chunk_shift) + (offset & sc->chunk_mask);
 	return 1;
 }
 

Alasdair
-- 
agk@redhat.com

Re: [linux-lvm] Data corruption on large, multi-device filesystem

[joe]

Thanks, I'll give it(the patch you sent) a try and let you know what I see.

When you say "chunk size" what lvcreate arg does that map to? the stripe
size(-I) or the chunk size(-c) that goes along with snapshots?

Joe

Quoting Alasdair G Kergon <agk@redhat.com>:

> On Wed, Jan 19, 2005 at 02:14:43PM -0800, joe@eiler.net wrote:
> > I don't see any corruption on the files unless I specify the -i option to
> > lvcreate.
>
> That's a useful observation: different code gets run when -i is used, and
> I've taken a look and there are two 32-bit limits in that code.
>
> One appears to be intentional for performance reasons and can be worked
> around by increasing the chunk size (default 64K), though it really should
> detect when the limit is exceeded and tell you!
>
> The other unintentional limit (which is the one affecting you I think)
> leads to wrap-around at striped device offsets beyond ~2^32 sectors
> i.e. 2^39 bytes into a PV, or ~2^40 bytes (2TB) into an LV in your
> case of 2 stripes.  The fix for that is a basic exercise in C.
> [Ref.  stripe_map() in drivers/md/dm-stripe.c]
>
> Alasdair
> --
> agk@redhat.com
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>

Re: [linux-lvm] Data corruption on large, multi-device filesystem

[Jens Beyer]

Hi,

On Thu, Jan 20, 2005 at 05:59:00PM +0000, Alasdair G Kergon wrote:
> On Thu, Jan 20, 2005 at 09:06:44AM -0500, Randall A. Jones wrote:
> > To verify, this corruption you are seeing only happens when you have a 
> > LV larger than 2TB
> > and when you use striping specifically with lvcreate -i.
> > Has anyone experienced data corruption with >2TB LV and no striping?
>  
> Does this patch help?
> 

No, it doesn't. I just checked multiple times in different setups. 
To address the 'chunksize' Problem mentioned i used lvcreate -i 3 -I 256 . 

Jens


> 
> Missing cast causing data corruption on devices with stripes > ~1TB.
> --- diff/drivers/md/dm-stripe.c	2005-01-20 17:32:37.000000000 +0000
> +++ source/drivers/md/dm-stripe.c	2005-01-20 17:32:26.000000000 +0000
> @@ -179,7 +179,7 @@
>  
>  	bio->bi_bdev = sc->stripe[stripe].dev->bdev;
>  	bio->bi_sector = sc->stripe[stripe].physical_start +
> -	    (chunk << sc->chunk_shift) + (offset & sc->chunk_mask);
> +	    ((sector_t) chunk << sc->chunk_shift) + (offset & sc->chunk_mask);
>  	return 1;
>  }
>  
> 


--
Nur tote Fische schwimmen mit dem Strom

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

Re: [linux-lvm] Data corruption on large, multi-device filesystem

[Jens Beyer]

Hi,

On Thu, Jan 20, 2005 at 05:59:00PM +0000, Alasdair G Kergon wrote:
> On Thu, Jan 20, 2005 at 09:06:44AM -0500, Randall A. Jones wrote:
> > To verify, this corruption you are seeing only happens when you have a 
> > LV larger than 2TB
> > and when you use striping specifically with lvcreate -i.
> > Has anyone experienced data corruption with >2TB LV and no striping?
>  
> Does this patch help?
> 

No, it doesn't. I just checked multiple times in different setups. 
To address the 'chunksize' Problem mentioned i used lvcreate -i 3 -I 256 . 

Jens


> 
> Missing cast causing data corruption on devices with stripes > ~1TB.
> --- diff/drivers/md/dm-stripe.c	2005-01-20 17:32:37.000000000 +0000
> +++ source/drivers/md/dm-stripe.c	2005-01-20 17:32:26.000000000 +0000
> @@ -179,7 +179,7 @@
>  
>  	bio->bi_bdev = sc->stripe[stripe].dev->bdev;
>  	bio->bi_sector = sc->stripe[stripe].physical_start +
> -	    (chunk << sc->chunk_shift) + (offset & sc->chunk_mask);
> +	    ((sector_t) chunk << sc->chunk_shift) + (offset & sc->chunk_mask);
>  	return 1;
>  }
>  
> 


--
Nur tote Fische schwimmen mit dem Strom

Re: Data corruption on large, multi-device filesystem

[joe]

I just thought I would give a quick update on what I have been doing in case
anyone can give me more/better direction.

I incorporated the (sector_t) cast and I also changed stripe_width to sector_t
since my individual devices are bigger than 2TB.

I took lvm out of the equation by using dmsetup and striping together my
devices.  Then I created and mounted and XFS filesystem directly on /dev/dm-0
This still caused corruption.

I have compiled hashdd with largefile support and am now letting it run directly
on /dev/dm-0.  The write finished last night and I am running the read pass now
(man it takes a long time to to write and verify 4+TB)

I have also been putting some debug statements in stripe_map() but this hasn't
shown me anything useful yet.

Joe

Quoting Jens Beyer <jbe@webde-ag.de>:

>
> Hi,
>
> On Thu, Jan 20, 2005 at 05:59:00PM +0000, Alasdair G Kergon wrote:
> > On Thu, Jan 20, 2005 at 09:06:44AM -0500, Randall A. Jones wrote:
> > > To verify, this corruption you are seeing only happens when you have a
> > > LV larger than 2TB
> > > and when you use striping specifically with lvcreate -i.
> > > Has anyone experienced data corruption with >2TB LV and no striping?
> >
> > Does this patch help?
> >
>
> No, it doesn't. I just checked multiple times in different setups.
> To address the 'chunksize' Problem mentioned i used lvcreate -i 3 -I 256 .
>
> Jens
>
>
> >
> > Missing cast causing data corruption on devices with stripes > ~1TB.
> > --- diff/drivers/md/dm-stripe.c	2005-01-20 17:32:37.000000000 +0000
> > +++ source/drivers/md/dm-stripe.c	2005-01-20 17:32:26.000000000 +0000
> > @@ -179,7 +179,7 @@
> >
> >  	bio->bi_bdev = sc->stripe[stripe].dev->bdev;
> >  	bio->bi_sector = sc->stripe[stripe].physical_start +
> > -	    (chunk << sc->chunk_shift) + (offset & sc->chunk_mask);
> > +	    ((sector_t) chunk << sc->chunk_shift) + (offset & sc->chunk_mask);
> >  	return 1;
> >  }
> >
> >
>
>
> --
> Nur tote Fische schwimmen mit dem Strom
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>



_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Data corruption on large, multi-device filesystem

[joe]

I just thought I would give a quick update on what I have been doing in case
anyone can give me more/better direction.

I incorporated the (sector_t) cast and I also changed stripe_width to sector_t
since my individual devices are bigger than 2TB.

I took lvm out of the equation by using dmsetup and striping together my
devices.  Then I created and mounted and XFS filesystem directly on /dev/dm-0
This still caused corruption.

I have compiled hashdd with largefile support and am now letting it run directly
on /dev/dm-0.  The write finished last night and I am running the read pass now
(man it takes a long time to to write and verify 4+TB)

I have also been putting some debug statements in stripe_map() but this hasn't
shown me anything useful yet.

Joe

Quoting Jens Beyer <jbe@webde-ag.de>:

>
> Hi,
>
> On Thu, Jan 20, 2005 at 05:59:00PM +0000, Alasdair G Kergon wrote:
> > On Thu, Jan 20, 2005 at 09:06:44AM -0500, Randall A. Jones wrote:
> > > To verify, this corruption you are seeing only happens when you have a
> > > LV larger than 2TB
> > > and when you use striping specifically with lvcreate -i.
> > > Has anyone experienced data corruption with >2TB LV and no striping?
> >
> > Does this patch help?
> >
>
> No, it doesn't. I just checked multiple times in different setups.
> To address the 'chunksize' Problem mentioned i used lvcreate -i 3 -I 256 .
>
> Jens
>
>
> >
> > Missing cast causing data corruption on devices with stripes > ~1TB.
> > --- diff/drivers/md/dm-stripe.c	2005-01-20 17:32:37.000000000 +0000
> > +++ source/drivers/md/dm-stripe.c	2005-01-20 17:32:26.000000000 +0000
> > @@ -179,7 +179,7 @@
> >
> >  	bio->bi_bdev = sc->stripe[stripe].dev->bdev;
> >  	bio->bi_sector = sc->stripe[stripe].physical_start +
> > -	    (chunk << sc->chunk_shift) + (offset & sc->chunk_mask);
> > +	    ((sector_t) chunk << sc->chunk_shift) + (offset & sc->chunk_mask);
> >  	return 1;
> >  }
> >
> >
>
>
> --
> Nur tote Fische schwimmen mit dem Strom
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>

Re: Data corruption on large, multi-device filesystem

[Jens Beyer]

Hi,

since the two patches (stripe_width and sector_t cast) didnt help, 
I started to put printk in the dm-stripe code. 

At a certain point stripe_map() starts to get called with zero data 
(eg bio->bi_sector = 0 or 2^32, 2^31 ). This lead me to put some debugging 
into dm.c especially into max_io_len() where an inline function dm_round_up()
is used. Interpreting my printk's this funktion doesnt work well with 
offsets larger then 2^32. I did some (very) ugly hack to calculate the
boundary in that case and it seems that the error is gone (though the
full check takes a few days).

Maybe someone with insight on what should happen in dm.c can have 
a look at max_io_len and the inlined function?

Jens

-- 
Nur tote Fische schwimmen mit dem Strom

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Data corruption on large, multi-device filesystem

[Jens Beyer]

Hi,

since the two patches (stripe_width and sector_t cast) didnt help, 
I started to put printk in the dm-stripe code. 

At a certain point stripe_map() starts to get called with zero data 
(eg bio->bi_sector = 0 or 2^32, 2^31 ). This lead me to put some debugging 
into dm.c especially into max_io_len() where an inline function dm_round_up()
is used. Interpreting my printk's this funktion doesnt work well with 
offsets larger then 2^32. I did some (very) ugly hack to calculate the
boundary in that case and it seems that the error is gone (though the
full check takes a few days).

Maybe someone with insight on what should happen in dm.c can have 
a look at max_io_len and the inlined function?

Jens

-- 
Nur tote Fische schwimmen mit dem Strom

Re: Data corruption on large, multi-device filesystem

[Alasdair G Kergon]

On Fri, Jan 28, 2005 at 10:23:17AM +0100, Jens Beyer wrote:
> since the two patches (stripe_width and sector_t cast) didnt help, 

Indeed - you need a version of Kevin's patch too; but I've widened
the scope of this to looking into why these functions are being
called.  (Now down to three 64/32 divisions.)

Alasdair
-- 
agk@redhat.com

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Data corruption on large, multi-device filesystem

[Alasdair G Kergon]

On Fri, Jan 28, 2005 at 10:23:17AM +0100, Jens Beyer wrote:
> since the two patches (stripe_width and sector_t cast) didnt help, 

Indeed - you need a version of Kevin's patch too; but I've widened
the scope of this to looking into why these functions are being
called.  (Now down to three 64/32 divisions.)

Alasdair
-- 
agk@redhat.com

Re: [linux-lvm] Data corruption on large, multi-device filesystem

[Dan Pritts]

Hi folks - I'm new to the list and to the Linux LVM.

I believe I've just run into the same problem described in this thread from 
back in January. 

x86_64 dell, redhat AS 4, kernel 2.6.9 (redhat source, self-compiled to
add xfs) with two ~5TB RAIDs that I'm striping with LVM.  
(it's really the same RAID presenting two LUNs over two parallel fibre
channel busses.)

About 4TB into a "dd if=/dev/zero of=testfile" the filesystem (XFS) started
complaining bitterly of errors.

On the other hand I can write a full 5TB of zero into an XFS created on
one of the raw devices with no errors.

Both XFS & the dm code in the kernel seem to have had some non-trivial updates
between 2.6.9 from redhat and 2.6.11.7, so i'm compiling that now.

Is it believed that this problem should be fixed in the mainline kernel
or are there other LVM patches I should apply?

Also, seems to be a FAQ on this list but my search of the archives didn't
show any definitive answer - is it true that there are no large device
limits with LVM2/device mapper?  

thanks
danno

On Thu, Jan 20, 2005 at 06:38:41PM -0800, joe@eiler.net wrote:
> Thanks, I'll give it(the patch you sent) a try and let you know what I see.
> 
> When you say "chunk size" what lvcreate arg does that map to? the stripe
> size(-I) or the chunk size(-c) that goes along with snapshots?
> 
> Joe
> 
> Quoting Alasdair G Kergon <agk@redhat.com>:
> 
> > On Wed, Jan 19, 2005 at 02:14:43PM -0800, joe@eiler.net wrote:
> > > I don't see any corruption on the files unless I specify the -i option to
> > > lvcreate.
> >
> > That's a useful observation: different code gets run when -i is used, and
> > I've taken a look and there are two 32-bit limits in that code.
> >
> > One appears to be intentional for performance reasons and can be worked
> > around by increasing the chunk size (default 64K), though it really should
> > detect when the limit is exceeded and tell you!
> >
> > The other unintentional limit (which is the one affecting you I think)
> > leads to wrap-around at striped device offsets beyond ~2^32 sectors
> > i.e. 2^39 bytes into a PV, or ~2^40 bytes (2TB) into an LV in your
> > case of 2 stripes.  The fix for that is a basic exercise in C.
> > [Ref.  stripe_map() in drivers/md/dm-stripe.c]
> >
> > Alasdair
> > --
> > agk@redhat.com
> >
> > _______________________________________________
> > linux-lvm mailing list
> > linux-lvm@redhat.com
> > https://www.redhat.com/mailman/listinfo/linux-lvm
> > read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
> >
> 
> 


danno
--
dan pritts - systems administrator - internet2
734/352-4953 office        734/834-7224 mobile