[linux-lvm] Fwd: LVM on ATA-Raid Working - but Poorly

[linux-lvm] Fwd: LVM on ATA-Raid Working - but Poorly

[Nathan Dornquast]

Thx, Goetz.

Very good suggestions.  They mostly point out what I suspected, that I will not 
have much luck getting the built in Promise RAID chips to work with LVM.

I am actually running without the PROMISE PDC202{46|62|65|67|68|69|70} 
support.  Some of the How-to's I've come accross say to include it, but when I 
do, I can not boot off the ATA-Raid.

I spent some time with the Promise drivers... they are released for RH 7.2/3, I 
haven't been successful at getting them to install on my Debian 3.0 box.  

I have tried other Promise Controllers (Fasttrak 100 & TX200) - but cannot get 
them to post - disabling built in Promise PDC202675 RAID or switching it to 
UDMA doesn't help.

Tried the Highpoint 404 raid card - not impressed.  Plus, they do not have 
RAID1 support in the kernel - 2.4.19 - only 0.

3ware 7500-4 is on order. ;)  Sadly it adds $260 to box.  

Goetz, I am interested to hear your MD RAID 1/5 - particularly how it handles 
powering off a drive while operating.  Does your box spit, sputter and puke, or 
can you hotswap?  How about with the 3ware card?  Which 3ware card are you 
working with?

I know this RAID info is off-topic.  Feel free to email me directly @ 
nathan@aberon.net.  

Many thanks,

-Nate

..........
Nathan Dornquast
Aberon Technologies, Inc.
2205 California St NE - Suite 003
Minneapolis, MN 55418
w.612.789.9446
c.612.386.6519
..........

Re: [linux-lvm] Fwd: LVM on ATA-Raid Working - but Poorly

[Jon Bendtsen]

Nathan Dornquast wrote:
> 
> Thx, Goetz.

[snip]

> 3ware 7500-4 is on order. ;)  Sadly it adds $260 to box.
> 
> Goetz, I am interested to hear your MD RAID 1/5 - particularly how it handles
> powering off a drive while operating.  Does your box spit, sputter and puke, or
> can you hotswap?  How about with the 3ware card?  Which 3ware card are you
> working with?

i've got 3 3ware cards, and i've tried 3 different kinds.
I've got 2 7410, and one 7810, and i've tried a 7450 (better raid5
performance)
You can hot swap, i've removed drives, i put them back in. You do have
to tell
the controller trough the diskmanagement tool, to use the "new" disk.
But it
works. I've benchmarked all of them vs. software for various raid
devices.
(mirror, stripped, raid10, and raid5)


Unfortunately i forgot to test it with lvm ontop :/




JonB

Re: [linux-lvm] Fwd: LVM on ATA-Raid Working - but Poorly

[Jason L Tibbitts III]

>>>>> "JB" == Jon Bendtsen <jon+lvm@silicide.dk> writes:

JB> Unfortunately i forgot to test it with lvm ontop :/

It works fine:

Total:  1 VG  1 PV  18 LVs (18 LVs open 18 times)
 
Global: 1088155 bytes malloced   IOP version: 10   178 days 23:22:40 active
 
VG:  nas  [1 PV, 18 LV/18 open]  PE Size: 16384 KB
  Usage [KB/PE]: 1101643776 /67239 total  1101643776 /67239 used  0 /0 free
  PV:  [AA] sda9                  1101643776 /67239   1101643776 /67239          0 /0

This is a 1TB array on a 3w7850 (now called a 7500-8) card with 8
Maxtor 160GB disks.  I've made volumes up over 500GB on these, have
resized volumes and filesystems, deleted volumes, etc. all with the
machines live and have experienced absolutely no problems.  This
machine is running (approximately) one of the Red Hat 7.3 beta
releases.

I also have LVM on top of software RAID5 on top of an old 3w6800 card
running in JBOD mode with 8 75GB IBM deathstars.  It works fine when
the disks are actually working (which is rarely).

 - J<

Re: [linux-lvm] Fwd: LVM on ATA-Raid Working - but Poorly

[Jon Bendtsen]

Jason L Tibbitts III wrote:
> 
> >>>>> "JB" == Jon Bendtsen <jon+lvm@silicide.dk> writes:
> 
> JB> Unfortunately i forgot to test it with lvm ontop :/
> 
> It works fine:

yeah, i know, i ment i forgot to benchmark it


[snip]

 
> I also have LVM on top of software RAID5 on top of an old 3w6800 card
> running in JBOD mode with 8 75GB IBM deathstars.  It works fine when
> the disks are actually working (which is rarely).

JBOD ?




JonB

Re: [linux-lvm] Fwd: LVM on ATA-Raid Working - but Poorly

[Jason L Tibbitts III]

>>>>> "JB" == Jon Bendtsen <jon+lvm@silicide.dk> writes:

JB> JBOD ?

Just a Bunch of Disks.  I.e. no hardware RAID arrays; the OS sees
eight independent disks.  Software RAID5 is significantly faster on a
6xxx-series 3ware card since the it was only designed to stripe and
mirror.

 - J<

Re: [linux-lvm] Fwd: LVM on ATA-Raid Working - but Poorly

[Jon Bendtsen]

Jason L Tibbitts III wrote:
> 
> >>>>> "JB" == Jon Bendtsen <jon+lvm@silicide.dk> writes:
> 
> JB> JBOD ?
> 
> Just a Bunch of Disks.  I.e. no hardware RAID arrays; the OS sees
> eight independent disks.  Software RAID5 is significantly faster on a
> 6xxx-series 3ware card since the it was only designed to stripe and
> mirror.

True true, but then your boot device is software raid ??
And lilo ??



JonB

Re: [linux-lvm] Fwd: LVM on ATA-Raid Working - but Poorly

[Jason L Tibbitts III]

>>>>> "JB" == Jon Bendtsen <jon+lvm@silicide.dk> writes:

JB> True true, but then your boot device is software raid ??

Yes, but not on LVM.  /boot and / were mirrored (all eight ways, which
I know is overkill).

JB> And lilo ??

It didn't have any problem; it wrote identical boot blocks to all
eight disks, and would boot with the first disk in the system
disconnected.  This was the first such server I built and I did it
some time ago so I haven't tried with anything more modern like Grub.

 - J<

Re: [linux-lvm] Fwd: LVM on ATA-Raid Working - but Poorly

[Jon Bendtsen]

Jason L Tibbitts III wrote:
> 
> >>>>> "JB" == Jon Bendtsen <jon+lvm@silicide.dk> writes:
> 
> JB> True true, but then your boot device is software raid ??
> 
> Yes, but not on LVM.  /boot and / were mirrored (all eight ways, which
> I know is overkill).
> 
> JB> And lilo ??
> 
> It didn't have any problem; it wrote identical boot blocks to all
> eight disks, and would boot with the first disk in the system
> disconnected.  This was the first such server I built and I did it
> some time ago so I haven't tried with anything more modern like Grub.

Well, grub doesnt matter i use lilo myself.
How did you get lilo to write it to all 8 bootblocks ?



JonB

Re: [linux-lvm] Fwd: LVM on ATA-Raid Working - but Poorly

[Jason L Tibbitts III]

>>>>> "JB" == Jon Bendtsen <jon+lvm@silicide.dk> writes:

JB> How did you get lilo to write it to all 8 bootblocks ?

I didn't do anything special; lilo handled the situation
automatically.

 - J<

Re: [linux-lvm] Fwd: LVM on ATA-Raid Working - but Poorly

[Jon Bendtsen]

Jason L Tibbitts III wrote:
> 
> >>>>> "JB" == Jon Bendtsen <jon+lvm@silicide.dk> writes:
> 
> JB> How did you get lilo to write it to all 8 bootblocks ?
> 
> I didn't do anything special; lilo handled the situation
> automatically.

oh, whats in your lilo.conf then ?



JonB

Re: [linux-lvm] Fwd: LVM on ATA-Raid Working - but Poorly

[Jason L Tibbitts III]

>>>>> "JB" == Jon Bendtsen <jon+lvm@silicide.dk> writes:

JB> oh, whats in your lilo.conf then ?

I had to get the machine booting again; since the IBM disks die so
regularly I can't use the machine for anything useful.  In any case:

boot=/dev/md0
map=/boot/map
install=/boot/boot.b
prompt
timeout=50
message=/boot/message
linear
default=l-2.4.12-ac5_lvm_uh_1

image=/boot/vmlinuz-2.4.12-ac5_lvm_uh_1
        label=l-2.4.12-ac5
        password=XXX
        restricted
        initrd=/boot/initrd-2.4.12-ac5_lvm_uh_1.img
        read-only
        root=/dev/md1

md0 is an eight-way mirror, mounted on /boot.  None of the system
volumes are on LVM.

 - J<

Re: [linux-lvm] Fwd: LVM on ATA-Raid Working - but Poorly

[Jon Bendtsen]

Jason L Tibbitts III wrote:
> 
> >>>>> "JB" == Jon Bendtsen <jon+lvm@silicide.dk> writes:
> 
> JB> oh, whats in your lilo.conf then ?
> 
> I had to get the machine booting again; since the IBM disks die so
> regularly I can't use the machine for anything useful.  In any case:
> 
> boot=/dev/md0

[cut]

> md0 is an eight-way mirror, mounted on /boot.  None of the system


okay, thats how i do it too, i just wondered if you did something smart.



JonB

Re: [linux-lvm] Fwd: LVM on ATA-Raid Working - but Poorly

[Goetz Bock]

On Thu, Oct 17 '02 at 23:36, Nathan Dornquast wrote:
> Very good suggestions.  They mostly point out what I suspected, that I will not 
> have much luck getting the built in Promise RAID chips to work with LVM.
I've two boxes with buildin Fasttrack100 (IIRC) working just fine with
the (colsed source) driver provided by promise, never had any problems,
they are not required to run 24x7 and so I don't care about hotswap and
stuff. The 2 boxes are loadbalancing each other, so one can go down, but
should not take too long to come back up again (and as the boards came
with fasttrack buildin, the customer did not want to spend money on a
3ware card :-( )
 
> I am actually running without the PROMISE PDC202{46|62|65|67|68|69|70} 
> support.  Some of the How-to's I've come accross say to include it, but when I 
> do, I can not boot off the ATA-Raid.
very bad idea, than the controller runs in ATA compatible mode, and that
just sucks (performance, features, stability). With promise support, the
card is detected as a fasttrack and you need the promise driver I've
mentioned above.

> I spent some time with the Promise drivers... they are released for RH 7.2/3, I 
> haven't been successful at getting them to install on my Debian 3.0 box.  
Oh, IC. I'm using redhat 7.3 :-)

> 3ware 7500-4 is on order. ;)  Sadly it adds $260 to box.  
For my newer boxes, that should not go down due to one drive failing, I'm using 
7500-2, its chaep, do RAID1, can boot, work fine, and have the 3dm
managing tool (to hotswap drives). 
I'm just using a 7500-4 in raid1 mode on a paranoids box who wanted
RAID1 and a hotspare ... he pays it, he gets it ;-)

> Goetz, I am interested to hear your MD RAID 1/5 - particularly how it handles 
> powering off a drive while operating.  Does your box spit, sputter and puke, or 
> can you hotswap?
i've two of this boxes, one with a 6850 (old 3ware card) and 8 60GB
(what was top of the line when the box was build) now running in JOBD
mode with MD on top of it (it's a lot faster than native RAID5) there I
can hostswap, but's a pain. The box can go down after about 18:00, so I
just power it down, remove the broken driven and power it up again, if a
drive should fail. 
The seccond if my home server running 6x120GB on 3 promise ATA100 cards.
This box can go down when ever I need it, and all I care is that I've A
LOT of storage with some redundancy. I just can not back it up, but that
does not matter (no RAID5 does not replace backups) as everything on
there can be reconstructed from CDs or the internet ;-)
My homeserver as a RAID1 with 2x40GB with important data on it, that I
can/do backup.

> How about with the 3ware card?  
They are a good pice of work, not cheap, but definetely worth the money
(at last the 7xxx)
> Which 3ware card are you working with?
2x 6850, both where bought do to RAID5 and are used as JBOD with MD
         ontop. 
1x 7500-8: mission critical storage server, 8x120GB, RAID5 rock solid,
           performance is as expected from 32bit bus (mainboard does not
           have 64bit pci)
2x 7500-2: mission critical servers, 2x??GB, RAID1
1x 7500-4: RAID1 with hotspare

as most people on the linux-ide-arrays list will tell you: 3ware 7xxx
are nice/fast/stable cards for RAID5 and IDE, but a version with
64bit/66MHz would be nice.
Personaly I'm looking forward to SATA ... no more flat cables

> I know this RAID info is off-topic.  Feel free to email me directly
> ...
I first thought we're on linux-ide-array already, and than I wanted to
reply privately, but others have replied to the list already, and so did
I (what if not RAID arrays do you want to run LVM on?)
-- 
Goetz Bock       (c) 2002 as     blacknet.de - Munich - Germany   /"\
IT Consultant    GNU FDL 1.1    secure mobile Linux everNETting   \ /
                                                                   X
 ASCII Ribbon Campaign against HTML email & microsoft attachments / \

Re: [linux-lvm] Fwd: LVM on ATA-Raid Working - but Poorly

[Jon Bendtsen]

Goetz Bock wrote:
> 
> On Thu, Oct 17 '02 at 23:36, Nathan Dornquast wrote:


[snip]


> > Goetz, I am interested to hear your MD RAID 1/5 - particularly how it handles
> > powering off a drive while operating.  Does your box spit, sputter and puke, or
> > can you hotswap?
> i've two of this boxes, one with a 6850 (old 3ware card) and 8 60GB
> (what was top of the line when the box was build) now running in JOBD
> mode with MD on top of it (it's a lot faster than native RAID5) there I
> can hostswap, but's a pain. The box can go down after about 18:00, so I

Why is it a pain ??
Pull out the drive, plug in a new, and then raidhotadd it to your md
device.


[snip]


> as most people on the linux-ide-arrays list will tell you: 3ware 7xxx
> are nice/fast/stable cards for RAID5 and IDE, but a version with
> 64bit/66MHz would be nice.

my cards are 64 bit ? Are they not 66 MHz ?
I thought all 64bit cards were 66MHz.


> Personaly I'm looking forward to SATA ... no more flat cables

3ware has a controller for that, with 15 disks *drool
*

 
> > I know this RAID info is off-topic.  Feel free to email me directly
> > ...
> I first thought we're on linux-ide-array already, and than I wanted to
> reply privately, but others have replied to the list already, and so did
> I (what if not RAID arrays do you want to run LVM on?)

oops, you are right, it is (getting) offtopic.




JonB