[linux-lvm] multiple volumes..

[linux-lvm] multiple volumes..

[Anders Widman]

I have a problem which I want to solve as efficiently as possible.

The problem is that I have 13 drives os various sizes (30-120GB) which
I want to combine into one (or a few) logical volumes with redundancy
if one drive would fail.

How would the smartest layout be? I was thinking about multiple
partiontions and then combine them into several RAID 5 chains, and
then use LVM to manage the logical volumes.

There is an expamle of my ideas at http://www.tnonline.net/raid-lvm.png
As you can see there are 8 chains and about 50 GB unallocated data in
this example. Can I minimize the loss even better?

Any help or insight in this would be greatly appreciaded

//Anders

Re: [linux-lvm] multiple volumes..

[Steven Lembark]

> I have a problem which I want to solve as efficiently as possible.
>
> The problem is that I have 13 drives os various sizes (30-120GB) which
> I want to combine into one (or a few) logical volumes with redundancy
> if one drive would fail.
>
> How would the smartest layout be? I was thinking about multiple
> partiontions and then combine them into several RAID 5 chains, and
> then use LVM to manage the logical volumes.
>
> There is an expamle of my ideas at http://www.tnonline.net/raid-lvm.png
> As you can see there are 8 chains and about 50 GB unallocated data in
> this example. Can I minimize the loss even better?
>
> Any help or insight in this would be greatly appreciaded

One way is to mirror the hard drives (requires an even number
of the same size/type) then build your LVM system on top of
the mirrored drives. Nice thing about this is that the messy
part (paring and mirroring the drives) is done once only and
is transparent to the LVM system.

If the drives are a motly collection of stuff, then you
may have to combine them into a VG then create LV's and
mirror those. Problem there is that there is a lot more
work involved in the maintinence: every time you want to
expand a volume you have to deal with mirroring issues.

If all of the storage doesn't need to be strictly mirrored
then you might be able to pair up however many drives you
can, leaving the un-paired ones for a separate LVM of non-
mirrored content used for higher-speed scratch space.

--
Steven Lembark                               2930 W. Palmer
Workhorse Computing                       Chicago, IL 60647
                                            +1 800 762 1582

Re: [linux-lvm] multiple volumes..

[Jon Bendtsen]

Steven Lembark wrote:
 
> > There is an expamle of my ideas at http://www.tnonline.net/raid-lvm.png
> > As you can see there are 8 chains and about 50 GB unallocated data in
> > this example. Can I minimize the loss even better?
> >
> > Any help or insight in this would be greatly appreciaded
> 
> One way is to mirror the hard drives (requires an even number
> of the same size/type) then build your LVM system on top of
> the mirrored drives. Nice thing about this is that the messy
> part (paring and mirroring the drives) is done once only and
> is transparent to the LVM system.

Okay, my suggestion would be to mirror alot, and possibly if you want
to,
to raid0 ontop of the mirrors, but that REQUIRES, that the disks are not
hd<a,b,c,d,e,f,g,h>
You should try to avoid having more than one (active) disk at the same
controller at once.
Anyway, mirror the 80GB disks, and the 75GB disks, possibly raid0 that.
raid0 2 60GB disks, and mirror that against the 120GB, or divide the 120
gb disk in 2, and mirror each partition for one 60GB disk.
The rest of the 60GB disks (4??) can do mirror, or mirror/raid
As for the 30, 40 and those 2*5GB from the 80GB disks, being raid0'ed
against
the 75GB disks, you could mirror the 30, 30 from the 40GB, and the last
10 GB,
well, same procedure as the 120GB disk, and the 2*60GB disks

Or you could just do what you suggested and then only create a mirror
from the
30GB disk, the leftovers from the 75GB disks, and the leftovers from the
120GB
disk.
30+2*5 = 40, which is left from the 120GB disk.

Personaly i dont think it is wise to raid5 over too many disks.
Especialy if
they are located on the same controller.



JonB

Re: [linux-lvm] multiple volumes..

[Adrian Head]

On Wed, 10 Apr 2002 08:01, you wrote:
but that REQUIRES, that the disks are not
> hd<a,b,c,d,e,f,g,h>
> You should try to avoid having more than one (active) disk at the same
> controller at once.

I've always wondered about this statement.  

The idea AFAIK is that unlike SCSI, IDE has the master drive control the 
slave drive on the same controller.  
1) This causes problems where the drives are a different 
model/make/manufactuer as the master drive will downgrade the settings of the 
drives (master & slave) to the lowest common denominator; and therefore, the 
speed in this case is affected.  
2) Having the master drive control the slave drive also causes problems if 
the master fails as then both drives fail or if a drive takes down the chain 
both drives fail.
3) maybe a couple of others that I have forgotten at the moment.

When I was setting up my server at home 1.2GAthlon AsusA7V-133 384M RAM 9x 
IBM 40G drives, 1 Promise Ultra100 & 2 Promise Ultra100TX2 I did a couple of 
tests.  
I found by using a flaky drive from a previous life (a western digital that 
when it would die would take down the controller) that it didn't matter if 
the drive was a single master on a controller, a master on a shared 
controller or a slave on a shared controller.  When the drive would fail - it 
would take the whole machine down with it.  The Kernel would not die but it 
would deadlock waiting for interrupts to return and the only way to fix the 
issue was to hard reset.  Having the flaky dirve as a standalone drive or as 
part of a software RAID made no difference.  So I concluded that at least in 
my setup I gained nothing by only having one IDE drive per controller.

I also ran bonnie++ tests on a software RAID0 using 4 master only drives 
(hde, hdg, hdi, hdk) and 4 master-slave drives (hde, hdf, hdg, hdh).  The 
results of the test were that seek times for the master-slave case were half 
that for the master-master case.  Read times did drop but only by about 
.75M/s from 80M/s but the write times improved by about .25-.75M/s from 
50M/s.  So I concluded that at least for my setup it was better to have the 
master-slave case because the loss of read speed was made up by the increase 
in write speed.  

So my final configuration was that hda, hdb were system disks.  The RAID5 
used hde, hdg, hdi, hdk with a hot spare hdo and the RAID0 used hdm & hdn.

Now the tests weren't what I would call conclusive or proper scientific tests 
but I believe they were valid.  I've always wondered what others have found 
as what I found seems to fly in the face of common rules of thumb.

There is a specific mailing list "linux ide" 
<linux-ide-arrays@lists.math.uh.edu> that deal with large IDE arrays so you 
might like to also ask your question there and see what is suggested.

-- 
Adrian Head

(Public Key available on request.)

Re: [linux-lvm] multiple volumes..

[Anders Widman]

Woha. Lot's of replies :)

1) Speed is generally not the biggest importance. 5-6MB/s would be
enough.

All drives are UDMA 100, and there are 4 UDMA 100 host controllers in
this system. If I want to, I can have 16 drives. The speed varyes
between 25-35MB/s for all drives.

The reason I choose a RAID 5 setup was that one drive could
die (to the point where no data recovery is possible) without loosing
any data. If the system gets very slow, or I might even have to bring
it down to replace the drive, that's fine.

I only need enough security if one drive would fail. Agreed, mirroring
would perhaps be safer, and at least faster when a drive failes. But I
need as much diskspace I can get, and mirroring would consume alot of
that.


Thanks,
Anders



> On Wed, 10 Apr 2002 08:01, you wrote:
> but that REQUIRES, that the disks are not
>> hd<a,b,c,d,e,f,g,h>
>> You should try to avoid having more than one (active) disk at the same
>> controller at once.

> I've always wondered about this statement.  

> The idea AFAIK is that unlike SCSI, IDE has the master drive control the 
> slave drive on the same controller.  
> 1) This causes problems where the drives are a different 
> model/make/manufactuer as the master drive will downgrade the settings of the 
> drives (master & slave) to the lowest common denominator; and therefore, the 
> speed in this case is affected.  
> 2) Having the master drive control the slave drive also causes problems if 
> the master fails as then both drives fail or if a drive takes down the chain 
> both drives fail.
> 3) maybe a couple of others that I have forgotten at the moment.

> When I was setting up my server at home 1.2GAthlon AsusA7V-133 384M RAM 9x 
> IBM 40G drives, 1 Promise Ultra100 & 2 Promise Ultra100TX2 I did a couple of 
> tests.  
> I found by using a flaky drive from a previous life (a western digital that 
> when it would die would take down the controller) that it didn't matter if 
> the drive was a single master on a controller, a master on a shared 
> controller or a slave on a shared controller.  When the drive would fail - it 
> would take the whole machine down with it.  The Kernel would not die but it 
> would deadlock waiting for interrupts to return and the only way to fix the 
> issue was to hard reset.  Having the flaky dirve as a standalone drive or as 
> part of a software RAID made no difference.  So I concluded that at least in 
> my setup I gained nothing by only having one IDE drive per controller.

> I also ran bonnie++ tests on a software RAID0 using 4 master only drives 
> (hde, hdg, hdi, hdk) and 4 master-slave drives (hde, hdf, hdg, hdh).  The 
> results of the test were that seek times for the master-slave case were half 
> that for the master-master case.  Read times did drop but only by about 
> .75M/s from 80M/s but the write times improved by about .25-.75M/s from 
> 50M/s.  So I concluded that at least for my setup it was better to have the 
> master-slave case because the loss of read speed was made up by the increase 
> in write speed.  

> So my final configuration was that hda, hdb were system disks.  The RAID5 
> used hde, hdg, hdi, hdk with a hot spare hdo and the RAID0 used hdm & hdn.

> Now the tests weren't what I would call conclusive or proper scientific tests 
> but I believe they were valid.  I've always wondered what others have found 
> as what I found seems to fly in the face of common rules of thumb.

> There is a specific mailing list "linux ide" 
> <linux-ide-arrays@lists.math.uh.edu> that deal with large IDE arrays so you 
> might like to also ask your question there and see what is suggested.

Re: [linux-lvm] multiple volumes..

[Jon Bendtsen]

Anders Widman wrote:
> 
> Woha. Lot's of replies :)

Yes, and quite alot of uneassesary text. Could you and others possibly
cut
the text you dont reply to away ?? You're waisting bandwidth and
diskspace
for all on the list, and for the list server. If someone REALLLY needs
ALL
the text from the former mail, well they are on the mailing list and got
it
before your email, so they can read it there, or they can go to the
archives
and read it there. So please, cut the uneassesary text avoid, not all,
just
those lines you dont directly respond to.


JonB

Re: [linux-lvm] multiple volumes..

[AJ Lewis]

[-- Attachment #1: Type: text/plain, Size: 1756 bytes --]

On Wed, Apr 10, 2002 at 01:04:21PM +0200, Jon Bendtsen wrote:
> Anders Widman wrote:
> > 
> > Woha. Lot's of replies :)
> 
> Yes, and quite alot of uneassesary text. Could you and others possibly
> cut
> the text you dont reply to away ?? You're waisting bandwidth and
> diskspace
> for all on the list, and for the list server. If someone REALLLY needs
> ALL
> the text from the former mail, well they are on the mailing list and got
> it
> before your email, so they can read it there, or they can go to the
> archives
> and read it there. So please, cut the uneassesary text avoid, not all,
> just
> those lines you dont directly respond to.

I'm sorry, but this rant is just plain silly.  We have plenty of disk space
on the list server to handle people including the previous comments, and I'd
rather not discourage helpful posters.  If you're going to start making this
kind of argument, you might as well start telling people to stop including
their sigs.  Or stop GPG signing their mails.  Or start using snail-mail
'cause it saves on bandwidth.

Oh yeah - it might be helpful for you to fix the width of your e-mail's
'cause you're sending an double the number of lines that are really
necessary and that wastes bandwidth... ;)

Regards,
-- 
AJ Lewis
Sistina Software Inc.                  Voice:  612-638-0500
1313 5th St SE, Suite 111              E-Mail: lewis@sistina.com
Minneapolis, MN 55414
http://www.sistina.com

Current GPG fingerprint =  7792 2F3E 9D86 DA4C 9111 7837 982D 737B 9DBC 4EC9
-----Begin Obligatory Humorous Quote----------------------------------------
Error: Keyboard not attached. Press F1 to continue.
-----End Obligatory Humorous Quote------------------------------------------

[-- Attachment #2: Type: application/pgp-signature, Size: 230 bytes --]

Re: [linux-lvm] multiple volumes..

[Jon Bendtsen]

Anders Widman wrote:
> 
> I have a problem which I want to solve as efficiently as possible.
> 
> The problem is that I have 13 drives os various sizes (30-120GB) which
> I want to combine into one (or a few) logical volumes with redundancy
> if one drive would fail.

13 disks... in one computer. How many controllers do you have ??
Are they all filled up ?? aka, using hd<a,b,c,d,e,f,g,h,...>


> How would the smartest layout be? I was thinking about multiple
> partiontions and then combine them into several RAID 5 chains, and
> then use LVM to manage the logical volumes.

And how many disks would you have in your raid5 thingy ?? Way too
many pr. controller i fear. Only run one (active) ide hd pr.
controller. As for scsi, more, but not many more.

Why not use raid1, or possibly, raid0 ontop of raid1?

 
> There is an expamle of my ideas at http://www.tnonline.net/raid-lvm.png
> As you can see there are 8 chains and about 50 GB unallocated data in
> this example. Can I minimize the loss even better?

Yes, you could avoid it.
Run raid1 on much of it, possibly raid0 on top for speed.
The 30GB disk, and the leftovers from the 2*75GB disks, can
with the leftovers from the 120GB disk be turned into raid1
partitions.

 
> Any help or insight in this would be greatly appreciaded

More info about the disks, ide, scsi, mixed, controllers ??




JonB

Re: [linux-lvm] multiple volumes..

[Petro]

On Tue, Apr 09, 2002 at 11:18:00PM +0200, Anders Widman wrote:
> I have a problem which I want to solve as efficiently as possible.
> The problem is that I have 13 drives os various sizes (30-120GB) which
> I want to combine into one (or a few) logical volumes with redundancy
> if one drive would fail.
> How would the smartest layout be? I was thinking about multiple
> partiontions and then combine them into several RAID 5 chains, and
> then use LVM to manage the logical volumes.

    The problem (off the top of my head) is that (1) this will give
    utter crap performance, and (2) If you're not *really* careful with
    your layout you could wind up losing a lot of data in a crash and
    (3) even if you don't loose data in a crash, a single drive fault
    will still drag performance into the toilet.  

> There is an expamle of my ideas at http://www.tnonline.net/raid-lvm.png
> As you can see there are 8 chains and about 50 GB unallocated data in
> this example. Can I minimize the loss even better?

    Using MD to make some meta-devices on the raw drive do:
    
    Stripe:
    2x60 = 120
    2x60 = 120
    2x60 = 120

    Raw: 
    120x1=120 

    Then layer these devices to become your initial RAID5 volume. 
    
    That leaves 2x75, 2x80, 1x30, and 1x40:

    Linear mode the 30 and 40 to get 70, raid5 across these to get:
    80
    80
    75
    75
    70 (30+40). 

    This loses you about 30 gig. (10+10+5+5). If you really wanted you
    could recover that by working off partitions, but IMO it's just not
    worth it. 

-- 
Share and Enjoy.