LVM2

LVM2

[Trever L. Adams]

I recently saw Alan Cox say on this list that LVM won't handle more than
2 terabytes. Is this LVM2 or LVM? What is the maximum amount of disk
space LVM2 (or any other RAID/MIRROR capable technology that is in
Linus's kernel) handle? I am talking with various people and we are
looking at Samba on Linux to do several different namespaces (obviously
one tree), most averaging about 3 terabytes, but one would have in
excess of 20 terabytes. We are looking at using 320 to 500 gigabyte
drives in these arrays. (How? IEEE-1394. Which brings a question I will
ask in a second email.)

Is RAID 5 all that bad using this software method? Is RAID 5 available?

Trever Adams
--
"They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety." -- Benjamin Franklin, 1759

Re: LVM2

[Norbert van Nobelen]

A logical volume in LVM will not handle more than 2TB. You can tie together 
the LVs in a volume group, thus going over the 2TB limit. Choose your 
filesystem well though, some have a 2TB limit too.

Disk size: What are you doing with it. 500GB disks are ATA (maybe SATA). ATA 
is good for low end servers or near line storage, SATA can be used equally to 
SCSI (I am going to suffer for this remark).

RAID5 in software works pretty good (survived a failed disk, and recovered 
another failing raid in 1 month). Hardware is better since you don't have a 
boot partition left which is usually just present on one disk (you can mirror 
that yourself ofcourse).

Regards,

Norbert van Nobelen

On Thursday 20 January 2005 20:51, you wrote:
> I recently saw Alan Cox say on this list that LVM won't handle more than
> 2 terabytes. Is this LVM2 or LVM? What is the maximum amount of disk
> space LVM2 (or any other RAID/MIRROR capable technology that is in
> Linus's kernel) handle? I am talking with various people and we are
> looking at Samba on Linux to do several different namespaces (obviously
> one tree), most averaging about 3 terabytes, but one would have in
> excess of 20 terabytes. We are looking at using 320 to 500 gigabyte
> drives in these arrays. (How? IEEE-1394. Which brings a question I will
> ask in a second email.)
>
> Is RAID 5 all that bad using this software method? Is RAID 5 available?
>
> Trever Adams
> --
> "They that can give up essential liberty to obtain a little temporary
> safety deserve neither liberty nor safety." -- Benjamin Franklin, 1759
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: LVM2

[Alasdair G Kergon]

On Thu, Jan 20, 2005 at 10:40:02PM +0100, Norbert van Nobelen wrote:
> A logical volume in LVM will not handle more than 2TB. You can tie together 
> the LVs in a volume group, thus going over the 2TB limit. 

Confused over terminology?
Tie PVs together to form a VG, then divide VG up into LVs.

Size limit depends on metadata format and the kernel: old LVM1 format has 
lower size limits - see the vgcreate man page.

New LVM2 metadata format relaxes those limits and lets you have LVs > 2TB
with a 2.6 kernel.

Alasdair
-- 
agk@redhat.com

Re: LVM2

[Trever L. Adams]

PV = the device
VG = groups of them (the RAID5 array?)
LV = what? the file system?

So, from what you are telling me, and the man page, 2.6.x with LVM2 can
have basically any size of PV, VG, and LV I want.

Am I flawed in my understanding?

Thank you,
Trever

On Thu, 2005-01-20 at 22:02 +0000, Alasdair G Kergon wrote:
> On Thu, Jan 20, 2005 at 10:40:02PM +0100, Norbert van Nobelen wrote:
> > A logical volume in LVM will not handle more than 2TB. You can tie together 
> > the LVs in a volume group, thus going over the 2TB limit. 
> 
> Confused over terminology?
> Tie PVs together to form a VG, then divide VG up into LVs.
> 
> Size limit depends on metadata format and the kernel: old LVM1 format has 
> lower size limits - see the vgcreate man page.
> 
> New LVM2 metadata format relaxes those limits and lets you have LVs > 2TB
> with a 2.6 kernel.
> 
> Alasdair
--
"Assassination is the extreme form of censorship." -- George Bernard
Shaw (1856-1950)

Re: LVM2

[Alasdair G Kergon]

On Thu, Jan 20, 2005 at 03:22:14PM -0700, Trever L. Adams wrote:
> PV = the device
> VG = groups of them (the RAID5 array?)
> LV = what? the file system?
 
http://www.tldp.org/HOWTO/LVM-HOWTO/anatomy.html
http://www.novell.com/products/linuxenterpriseserver8/whitepapers/LVM.pdf
  [Out-of-date now, but descriptions of concepts still useful.]

LVM mailing list for LVM questions:
  https://www.redhat.com/mailman/listinfo/linux-lvm

Alasdair
-- 
agk@redhat.com

Re: LVM2

[Norbert van Nobelen]

Even as LVM user, guess what I used before answering (-:
On Thursday 20 January 2005 23:34, Alasdair G Kergon wrote:
> On Thu, Jan 20, 2005 at 03:22:14PM -0700, Trever L. Adams wrote:
> > PV = the device
> > VG = groups of them (the RAID5 array?)
> > LV = what? the file system?
>
> http://www.tldp.org/HOWTO/LVM-HOWTO/anatomy.html
> http://www.novell.com/products/linuxenterpriseserver8/whitepapers/LVM.pdf
>   [Out-of-date now, but descriptions of concepts still useful.]
>
> LVM mailing list for LVM questions:
>   https://www.redhat.com/mailman/listinfo/linux-lvm
>
> Alasdair

Re: LVM2

[Trever L. Adams]

It is for a group. For the most part it is data access/retention. Writes
and such would be more similar to a desktop. I would use SATA if they
were (nearly) equally priced and there were awesome 1394 to SATA bridge
chips that worked well with Linux. So, right now, I am looking at ATA to
1394.

So, to get 2TB of RAID5 you have 6 500 GB disks right? So, will this
work within on LV? Or is it 2TB of diskspace total? So, are volume
groups pretty fault tolerant if you have a bunch of RAID5 LVs below
them? This is my one worry about this.

Second, you mentioned file systems. We were talking about ext3. I have
never used any others in Linux (barring ext2, minixfs, and fat). I had
heard XFS from IBM was pretty good. I would rather not use reiserfs.

Any recommendations.

Trever

P.S. Why won't an LV support over 2TB?

S.P.S. I am not really worried about the boot and programs drive. They
will be spun down most of the time I am sure.

On Thu, 2005-01-20 at 22:40 +0100, Norbert van Nobelen wrote:
> A logical volume in LVM will not handle more than 2TB. You can tie together 
> the LVs in a volume group, thus going over the 2TB limit. Choose your 
> filesystem well though, some have a 2TB limit too.
> 
> Disk size: What are you doing with it. 500GB disks are ATA (maybe SATA). ATA 
> is good for low end servers or near line storage, SATA can be used equally to 
> SCSI (I am going to suffer for this remark).
> 
> RAID5 in software works pretty good (survived a failed disk, and recovered 
> another failing raid in 1 month). Hardware is better since you don't have a 
> boot partition left which is usually just present on one disk (you can mirror 
> that yourself ofcourse).
> 
> Regards,
> 
> Norbert van Nobelen
> 
> On Thursday 20 January 2005 20:51, you wrote:
> > I recently saw Alan Cox say on this list that LVM won't handle more than
> > 2 terabytes. Is this LVM2 or LVM? What is the maximum amount of disk
> > space LVM2 (or any other RAID/MIRROR capable technology that is in
> > Linus's kernel) handle? I am talking with various people and we are
> > looking at Samba on Linux to do several different namespaces (obviously
> > one tree), most averaging about 3 terabytes, but one would have in
> > excess of 20 terabytes. We are looking at using 320 to 500 gigabyte
> > drives in these arrays. (How? IEEE-1394. Which brings a question I will
> > ask in a second email.)
> >
> > Is RAID 5 all that bad using this software method? Is RAID 5 available?
> >
> > Trever Adams
> > --
> > "They that can give up essential liberty to obtain a little temporary
> > safety deserve neither liberty nor safety." -- Benjamin Franklin, 1759
> >
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > Please read the FAQ at  http://www.tux.org/lkml/
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 
--
"Assassination is the extreme form of censorship." -- George Bernard
Shaw (1856-1950)

Re: LVM2

[William Lee Irwin III]

On Thu, Jan 20, 2005 at 03:17:37PM -0700, Trever L. Adams wrote:
> Second, you mentioned file systems. We were talking about ext3. I have
> never used any others in Linux (barring ext2, minixfs, and fat). I had
> heard XFS from IBM was pretty good. I would rather not use reiserfs.

XFS is from SGI. JFS is from IBM.


-- wli

Re: LVM2

[Jeffrey E. Hundstad]

XFS is an SGI project.
http://oss.sgi.com/

I've been using it for quite a while and am quite happy with it; it is 
very fast and very fault tolerant.  The only warning I'd like to give 
about it is it seems that some Linux developers seem to have a bad taste 
in their mouth when it comes to XFS; go figure.

-- 
jeffrey hundstad

Trever L. Adams wrote:

>It is for a group. For the most part it is data access/retention. Writes
>and such would be more similar to a desktop. I would use SATA if they
>were (nearly) equally priced and there were awesome 1394 to SATA bridge
>chips that worked well with Linux. So, right now, I am looking at ATA to
>1394.
>
>So, to get 2TB of RAID5 you have 6 500 GB disks right? So, will this
>work within on LV? Or is it 2TB of diskspace total? So, are volume
>groups pretty fault tolerant if you have a bunch of RAID5 LVs below
>them? This is my one worry about this.
>
>Second, you mentioned file systems. We were talking about ext3. I have
>never used any others in Linux (barring ext2, minixfs, and fat). I had
>heard XFS from IBM was pretty good. I would rather not use reiserfs.
>
>Any recommendations.
>
>Trever
>
>P.S. Why won't an LV support over 2TB?
>
>S.P.S. I am not really worried about the boot and programs drive. They
>will be spun down most of the time I am sure.
>
>On Thu, 2005-01-20 at 22:40 +0100, Norbert van Nobelen wrote:
>  
>
>>A logical volume in LVM will not handle more than 2TB. You can tie together 
>>the LVs in a volume group, thus going over the 2TB limit. Choose your 
>>filesystem well though, some have a 2TB limit too.
>>
>>Disk size: What are you doing with it. 500GB disks are ATA (maybe SATA). ATA 
>>is good for low end servers or near line storage, SATA can be used equally to 
>>SCSI (I am going to suffer for this remark).
>>
>>RAID5 in software works pretty good (survived a failed disk, and recovered 
>>another failing raid in 1 month). Hardware is better since you don't have a 
>>boot partition left which is usually just present on one disk (you can mirror 
>>that yourself ofcourse).
>>
>>Regards,
>>
>>Norbert van Nobelen
>>
>>On Thursday 20 January 2005 20:51, you wrote:
>>    
>>
>>>I recently saw Alan Cox say on this list that LVM won't handle more than
>>>2 terabytes. Is this LVM2 or LVM? What is the maximum amount of disk
>>>space LVM2 (or any other RAID/MIRROR capable technology that is in
>>>Linus's kernel) handle? I am talking with various people and we are
>>>looking at Samba on Linux to do several different namespaces (obviously
>>>one tree), most averaging about 3 terabytes, but one would have in
>>>excess of 20 terabytes. We are looking at using 320 to 500 gigabyte
>>>drives in these arrays. (How? IEEE-1394. Which brings a question I will
>>>ask in a second email.)
>>>
>>>Is RAID 5 all that bad using this software method? Is RAID 5 available?
>>>
>>>Trever Adams
>>>--
>>>"They that can give up essential liberty to obtain a little temporary
>>>safety deserve neither liberty nor safety." -- Benjamin Franklin, 1759
>>>
>>>-
>>>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>>>the body of a message to majordomo@vger.kernel.org
>>>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>Please read the FAQ at  http://www.tux.org/lkml/
>>>      
>>>
>>-
>>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>>the body of a message to majordomo@vger.kernel.org
>>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>Please read the FAQ at  http://www.tux.org/lkml/
>>
>>    
>>
>--
>"Assassination is the extreme form of censorship." -- George Bernard
>Shaw (1856-1950)
>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at  http://www.tux.org/lkml/
>  
>

Re: LVM2

[Steve Lord]

Trever L. Adams wrote:
> It is for a group. For the most part it is data access/retention. Writes
> and such would be more similar to a desktop. I would use SATA if they
> were (nearly) equally priced and there were awesome 1394 to SATA bridge
> chips that worked well with Linux. So, right now, I am looking at ATA to
> 1394.
> 
> So, to get 2TB of RAID5 you have 6 500 GB disks right? So, will this
> work within on LV? Or is it 2TB of diskspace total? So, are volume
> groups pretty fault tolerant if you have a bunch of RAID5 LVs below
> them? This is my one worry about this.
> 
> Second, you mentioned file systems. We were talking about ext3. I have
> never used any others in Linux (barring ext2, minixfs, and fat). I had
> heard XFS from IBM was pretty good. I would rather not use reiserfs.
> 
> Any recommendations.
> 
> Trever
> 

They all forgot to mention one more limitation, the maximum filesystem
size supported by the address_space structure in linux. If you are running
on ia32, then you get stuck with 2^32 filesystem blocks, or 16 Tbytes in
one filesystem because of the way an address space structure is used to
cache the metadata. If you use an Athlon 64 that limitation goes away.

Steve

Re: LVM2

[Norbert van Nobelen]

With using RAID5 you can choose yourself howmany hot standby/failover disks 
you want to use. The number (or ratio) of disks used for failover of your 
raid will determine the chance that you have when one disk fails and complete 
failure of a raid.
It is still pretty safe just to have 7 active disks and 1 hot standby disk, 
thus getting you 3500 Manufacturers GB (3.34TB)

About LVM on a RAID: 
With hardware RAID it will work for sure.
With software RAID:
Make the RAID partitions and build your RAID.
On that RAID, set the partition type to LVM
And it should work
On LVM you can then use ext3 (your preferred fs)

On Thursday 20 January 2005 23:17, you wrote:
> It is for a group. For the most part it is data access/retention. Writes
> and such would be more similar to a desktop. I would use SATA if they
> were (nearly) equally priced and there were awesome 1394 to SATA bridge
> chips that worked well with Linux. So, right now, I am looking at ATA to
> 1394.
>
> So, to get 2TB of RAID5 you have 6 500 GB disks right? So, will this
> work within on LV? Or is it 2TB of diskspace total? So, are volume
> groups pretty fault tolerant if you have a bunch of RAID5 LVs below
> them? This is my one worry about this.
>
> Second, you mentioned file systems. We were talking about ext3. I have
> never used any others in Linux (barring ext2, minixfs, and fat). I had
> heard XFS from IBM was pretty good. I would rather not use reiserfs.
>
> Any recommendations.
>
> Trever
>
> P.S. Why won't an LV support over 2TB?
>
> S.P.S. I am not really worried about the boot and programs drive. They
> will be spun down most of the time I am sure.
>
> On Thu, 2005-01-20 at 22:40 +0100, Norbert van Nobelen wrote:
> > A logical volume in LVM will not handle more than 2TB. You can tie
> > together the LVs in a volume group, thus going over the 2TB limit. Choose
> > your filesystem well though, some have a 2TB limit too.
> >
> > Disk size: What are you doing with it. 500GB disks are ATA (maybe SATA).
> > ATA is good for low end servers or near line storage, SATA can be used
> > equally to SCSI (I am going to suffer for this remark).
> >
> > RAID5 in software works pretty good (survived a failed disk, and
> > recovered another failing raid in 1 month). Hardware is better since you
> > don't have a boot partition left which is usually just present on one
> > disk (you can mirror that yourself ofcourse).
> >
> > Regards,
> >
> > Norbert van Nobelen
> >
> > On Thursday 20 January 2005 20:51, you wrote:
> > > I recently saw Alan Cox say on this list that LVM won't handle more
> > > than 2 terabytes. Is this LVM2 or LVM? What is the maximum amount of
> > > disk space LVM2 (or any other RAID/MIRROR capable technology that is in
> > > Linus's kernel) handle? I am talking with various people and we are
> > > looking at Samba on Linux to do several different namespaces (obviously
> > > one tree), most averaging about 3 terabytes, but one would have in
> > > excess of 20 terabytes. We are looking at using 320 to 500 gigabyte
> > > drives in these arrays. (How? IEEE-1394. Which brings a question I will
> > > ask in a second email.)
> > >
> > > Is RAID 5 all that bad using this software method? Is RAID 5 available?
> > >
> > > Trever Adams
> > > --
> > > "They that can give up essential liberty to obtain a little temporary
> > > safety deserve neither liberty nor safety." -- Benjamin Franklin, 1759
> > >
> > > -
> > > To unsubscribe from this list: send the line "unsubscribe linux-kernel"
> > > in the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > > Please read the FAQ at  http://www.tux.org/lkml/
> >
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-kernel"
> > in the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > Please read the FAQ at  http://www.tux.org/lkml/
>
> --
> "Assassination is the extreme form of censorship." -- George Bernard
> Shaw (1856-1950)

-- 
<a href="http://www.edusupport.nl">EduSupport: Linux Desktop for schools and 
small to medium business in The Netherlands and Belgium</a>

Re: LVM2

[Kyle Moffett]

On Jan 20, 2005, at 16:40, Norbert van Nobelen wrote:
> RAID5 in software works pretty good (survived a failed disk, and 
> recovered
> another failing raid in 1 month). Hardware is better since you don't 
> have a
> boot partition left which is usually just present on one disk (you can 
> mirror
> that yourself ofcourse).

Err, you _can_ boot completely from a software RAID, it just takes a 
bit more
work.  I have an old PowerMac G4 400MHz with a Promise 20268 controller 
and 3
80GB drives booting from a software RAID.  You just set up a 250-500MB 
boot
partition mirrored with RAID 1 across all drives, then set up a RAID 5 
swap
partition and a RAID 5 LVM partition on each drive.  Once LVM is 
configured
with each remaining filesystem, install your distro (The new 
Debian-installer
does very well) and set up Yaboot/GRUB/whatever to install a boot 
sector on
each drive.  Then set up a RAID+LVM initrd (Debian does this mostly
automatically too), and reboot. This computer boots a custom 2.6.8.1 
kernel,
has 896MB RAM, and a 400MHz CPU, but it reads 41.5MiByte/sec from its 
RAID 5
partition with a 1MiByte blocksize, and has 16.8MiByte/sec over LVM over
RAID 5 with the same blocksize.  I've been following the discussions on 
2.6
instability and "New development model" problems, but AFAICT, 2.6 has 
been
rock stable on this box, which acts as an IPv4/IPv6 
router/firewall/server.

Cheers,
Kyle Moffett

-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCM/CS/IT/U d- s++: a18 C++++>$ UB/L/X/*++++(+)>$ P+++(++++)>$
L++++(+++) E W++(+) N+++(++) o? K? w--- O? M++ V? PS+() PE+(-) Y+
PGP+++ t+(+++) 5 X R? tv-(--) b++++(++) DI+ D+ G e->++++$ h!*()>++$ r  
!y?(-)
------END GEEK CODE BLOCK------

md and RAID 5 [was Re: LVM2]

[Trever L. Adams]

Thank you all for having been so kind in your responses and help.

However, there is one more set of questions I have.

Does the md (software raid) have disk size or raid volume limits?

If I am using such things as USB or 1394 disks, is there a way to use
labels in /etc/raidtab and with the tools so that when the disks, if
they do, get renumbered in /dev that all works fine. I am aware that the
kernel will autodetect these devices, but that the raidtab needs to be
consistent. This is what I am trying to figure out how to do.

Thank you,
Trever Adams
--
"A modest woman, dressed out in all her finery, is the most tremendous
object in the whole creation." -- Goldsmith 

Re: md and RAID 5 [was Re: LVM2]

[Wakko Warner]

Trever L. Adams wrote:
> Thank you all for having been so kind in your responses and help.
> 
> However, there is one more set of questions I have.
> 
> Does the md (software raid) have disk size or raid volume limits?
> 
> If I am using such things as USB or 1394 disks, is there a way to use
> labels in /etc/raidtab and with the tools so that when the disks, if
> they do, get renumbered in /dev that all works fine. I am aware that the
> kernel will autodetect these devices, but that the raidtab needs to be
> consistent. This is what I am trying to figure out how to do.

EVMS can handle this for you.  I've used it with a raid set I made with some
of the drives being on usb and some on ide.

If you use evms, you might want to consider the activation after all disks
have been discovered.

-- 
 Lab tests show that use of micro$oft causes cancer in lab animals

Re: md and RAID 5 [was Re: LVM2]

[Neil Brown]

On Friday January 21, tadams-lists@myrealbox.com wrote:
> Thank you all for having been so kind in your responses and help.
> 
> However, there is one more set of questions I have.
> 
> Does the md (software raid) have disk size or raid volume limits?

2^31 sectors for individual disks.  Arrays do not have this limit.

> 
> If I am using such things as USB or 1394 disks, is there a way to use
> labels in /etc/raidtab and with the tools so that when the disks, if
> they do, get renumbered in /dev that all works fine. I am aware that the
> kernel will autodetect these devices, but that the raidtab needs to be
> consistent. This is what I am trying to figure out how to do.

Scrap raidtools and /etc/raidtab.  Explore "mdadm" and /etc/mdadm.conf

NeilBrown