RE: [linux-lvm] HowTo: EIDE/ATA-100 + RAID 1 + LVM

RE: [linux-lvm] HowTo: EIDE/ATA-100 + RAID 1 + LVM

[Robert_Macaulay]

About the Raid 1.

I had a 4G RAM box(HIMEM64 enabled) with Hardware RAID 1 and LVM beta7,
kernel is from RH 7.1, version 2.4.3-6.

I added 4GB of RAM, kept the kernel the same, and switched to Software RAID
1. Now, I'm getting all sorts of instabilities. For example, I had a APIC
error fly by. umount core dumped. pvscan crashed twice and worked on the
third try(but saw a fraction to the disks, the 4th shot saw them all).
crond randomly crashed. I have the oops from the crond dump, but looked
fairly innocent of lvm code.

I know I changed 2 variables(RAM and RAID). I'm going to go back to HW raid
1 for now, and see if it makes a difference.

Just wondering if this has been seen, and if anyone knows what could have
caused it. The oops follows.

Robert



Unable to handle kernel paging request at virtual address 2e2e3e83
c0144078
Oops: 0000
CPU:    0
EIP:    0010:[<c0144078>]
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010202
eax: 2e2e3e5b   ebx: ccf0dd60   ecx: e839d700   edx: e8123f54
esi: 00000000   edi: e81832c0   ebp: e821f00c   esp: e8123f34
ds: 0018   es: 0018   ss: 0018
Process crond (pid: 817, stackpage=e8123000)
Stack: 000041ed 00000009 00000000 ccf0dd60 00000000 00001000 fffffff4
e821f000
       e821f005 00000007 0af98a86 0804c702 00000000 e821f000 e8123f9c
bffff758
       c014462a e821f000 e8123f9c e8122000 bffffbe0 00000003 c0140ff3
0804c702
Call Trace: [<c014462a>] [<c0140ff3>] [<c012035b>] [<c0106f3b>]
Code: 8b 40 28 85 c0 74 71 be 00 e0 ff ff 21 e6 8b 86 40 03 00 00

>>EIP; c0144078 <path_walk+748/8b0>   <=====
Trace; c014462a <__user_walk+3a/60>
Trace; c0140ff3 <sys_stat64+13/70>
Trace; c012035b <sys_nanosleep+11b/190>
Trace; c0106f3b <system_call+33/38>
Code;  c0144078 <path_walk+748/8b0>
00000000 <_EIP>:
Code;  c0144078 <path_walk+748/8b0>   <=====
   0:   8b 40 28                  mov    0x28(%eax),%eax   <=====
Code;  c014407b <path_walk+74b/8b0>
   3:   85 c0                     test   %eax,%eax
Code;  c014407d <path_walk+74d/8b0>
   5:   74 71                     je     78 <_EIP+0x78> c01440f0
<path_walk+7c0/8b0>
Code;  c014407f <path_walk+74f/8b0>
   7:   be 00 e0 ff ff            mov    $0xffffe000,%esi
Code;  c0144084 <path_walk+754/8b0>
   c:   21 e6                     and    %esp,%esi
Code;  c0144086 <path_walk+756/8b0>
   e:   8b 86 40 03 00 00         mov    0x340(%esi),%eax

Re: [linux-lvm] HowTo: EIDE/ATA-100 + RAID 1 + LVM

[Goetz Bock]

[-- Attachment #1: Type: text/plain, Size: 683 bytes --]


On Thu, Jun 28 '01 at 16:46, Robert_Macaulay@Dell.com wrote:
> I had a 4G RAM box(HIMEM64 enabled) with Hardware RAID 1 and LVM beta7,
> kernel is from RH 7.1, version 2.4.3-6.
What kind of hardware raid?
If it wasn't such a fake IDE one (or a 3ware encalade), I would have
sticked with it.  
 
> [ ... ]
> I know I changed 2 variables(RAM and RAID). I'm going to go back to HW raid
> 1 for now, and see if it makes a difference.
I would try the ram first, I'm not too confident on IA32 boxes handling
more than 4gb correctly (damn it, I'd like to have a box with 1gb ...)

-- 
Goetz Bock                                              IT Consultant
Dipl.-Inf. Univ.

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: [linux-lvm] HowTo: EIDE/ATA-100 + RAID 1 + LVM

[José Luis Domingo López]

On Thursday, 28 June 2001, at 16:46:10 -0500,
Robert_Macaulay@Dell.com wrote:

> About the Raid 1.
> [...]
> I know I changed 2 variables(RAM and RAID). I'm going to go back to HW raid
> 1 for now, and see if it makes a difference.
> 
I would choose hardware RAID instead os software RAID: this reduces CPU
overhead on the server, and avoids performance problems due to excesive
layering of "block devices".

2.4.x isn't "productoin quality" yet on some kinds of hardware, specially
those SMP machines with lots of RAM (requiring BIGMEM). That seems to be
your case, and is a known and (hopefully) soon-to-be-solved problem.

--
Jos� Luis Domingo L�pez
Linux Registered User #189436     Debian GNU/Linux Potato (P166 64 MB RAM)
 
jdomingo EN internautas PUNTO org  => � Spam ? Atente a las consecuencias
jdomingo AT internautas DOT   org  => Spam at your own risk

RE: [linux-lvm] HowTo: EIDE/ATA-100 + RAID 1 + LVM

[Robert_Macaulay]

Just an update. Once I went back to HW raid 1, my problems vanished. It
could still very well be the BIGMEM interacting with RAID 1. The large RAM
ammount doesn't hurt LVM at all(as far I I have seen so far).

> -----Original Message-----
> From: José Luis Domingo López [mailto:jdomingo@internautas.org]
> Sent: Thursday, June 28, 2001 7:57 PM
> To: linux-lvm@sistina.com
> Subject: Re: [linux-lvm] HowTo: EIDE/ATA-100 + RAID 1 + LVM
> 
> 
> On Thursday, 28 June 2001, at 16:46:10 -0500,
> Robert_Macaulay@Dell.com wrote:
> 
> > About the Raid 1.
> > [...]
> > I know I changed 2 variables(RAM and RAID). I'm going to go 
> back to HW raid
> > 1 for now, and see if it makes a difference.
> > 
> I would choose hardware RAID instead os software RAID: this 
> reduces CPU
> overhead on the server, and avoids performance problems due 
> to excesive
> layering of "block devices".
> 
> 2.4.x isn't "productoin quality" yet on some kinds of 
> hardware, specially
> those SMP machines with lots of RAM (requiring BIGMEM). That 
> seems to be
> your case, and is a known and (hopefully) soon-to-be-solved problem.
> 
> --
> José Luis Domingo López
> Linux Registered User #189436     Debian GNU/Linux Potato 
> (P166 64 MB RAM)
>  
> jdomingo EN internautas PUNTO org  => ¿ Spam ? Atente a las 
> consecuencias
> jdomingo AT internautas DOT   org  => Spam at your own risk
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html
> 

[linux-lvm] HowTo: EIDE/ATA-100 + RAID 1 + LVM

[Emmanuel Blot]

Hi,

I don't know much about RAID and LVM.

I have to set up a server, and I'd like to know if it's possible -and reliable- to setup a server
with RAID 1 (mirrored disk) with a pure software solution (not RAID controller) and LVM

* disks: two 40 GB IBM 60GXP (ATA-100).
* motherboard: Asus with AMD760 + DDR (Athlon Thunderbird)
* distribution: SuSE 7.2, with Linux kernel 2.4.4

- Are there some known issues/bugs with this configuration ?
- Where can I start (documentation URL I should know, ...)
- What is the installation order to follow (between software installation, RAID setup, LVM setup, ...)
- Any other suggestion

As the server has to be mostly remote-administrated (from other countries...), reliability is a key.
Is LVM stable enough with RAID-1 and Linux 2.4.x ?

Thanks in advance,
Emmanuel.

Re: [linux-lvm] HowTo: EIDE/ATA-100 + RAID 1 + LVM

[José Luis Domingo López]

On Thursday, 28 June 2001, at 18:06:25 +0100,
Emmanuel Blot wrote:

> Hi,
> 
> I don't know much about RAID and LVM.
> 
> I have to set up a server, and I'd like to know if it's possible -and reliable- to setup a server
> with RAID 1 (mirrored disk) with a pure software solution (not RAID controller) and LVM
> 
> * disks: two 40 GB IBM 60GXP (ATA-100).
> * motherboard: Asus with AMD760 + DDR (Athlon Thunderbird)
> * distribution: SuSE 7.2, with Linux kernel 2.4.4
> 
> - Are there some known issues/bugs with this configuration ?
> - Where can I start (documentation URL I should know, ...)
> - What is the installation order to follow (between software installation, RAID setup, LVM setup, ...)
> - Any other suggestion
> 
> As the server has to be mostly remote-administrated (from other countries...), reliability is a key.
> Is LVM stable enough with RAID-1 and Linux 2.4.x ?
> 
Lately I've been experimenting with layering Linux's softare RAID
implementation with LVM, both LVM on top of RAID and RAID on top of LVM
(the former seems to be more reasonable).

My test hardware is plain _shit_ (P75 with 16 MB RAM and two 6-year old
IDE disks, 400 MB and 800 MB), but was enough to test the aforementioned
configurations. I had no problems at all, and everything seemed to run OK.

However, layering LVM/RAID (or RAID/LVM) can mean a much greater overhead
on IO. My tests _seem_ to show noticeable slowdonws on raw throughput (both
dd and bonnie) and array reconstruction speeds. But a decent hardware
could show a much better behaviour than what I saw.

On the other hand, you must be very careful with designing the setup,
specially with RAID on top of LVM. If you are building a 2-"disk" RAID-1, 
and extents for both "disks" are taken from the same physical hard drive,
your mirror is quite unuseful in case a disk dies.

Another parameter to take into consideration is the filesystem you will be
using, the parameters used to create it, the usage pattern for this
filesystem, AND interaction with the unbderlying "block device". For
example, an ext2 filesystem with 4 KB blocks on top of a stripped LV built
from a 2-disk RAID-0 underlaying device with 32 KB stripe maybe is not
optimal in terms of troughput and/or IO latency.

To get all this working, download a "stable" kernel (2.4.4 gave me no
ptoblems), download LVM patches (version 0.9.1-beta7) and recompile. You
must enable LVM and RAID when using your favourite "make *config".
Although both can be compiled as modules, is not advisable to do so on a
production machine (I've experienced problems with RAID devices not
autostarting upon reboot, manual intervention required). Don't forget to
build LVM tools from the tarball you should have downloaded previously.
Download raidtools2 (version 0.9x) for your distribution.

RAID building and usage is quite well documented on several HOWTO, from
www.linuxdoc.org (mainly Software-RAID-HOWTO).

With respect to LVM, there is the LVM-HOWTO. Not very thorough, but
enough to make it work quickly.

Just one tip to ease your task: LVM on top of RAID is just a matter of
create the RAID device and then making a VG using /dev/md0 as the Physical
Device. Once you have a VG available, create LV (similar to partitions)
and create filesystems on them as usual.

PS: I'm currently trying to finish a HOWTO-like document about Linux's
software RAID usage, with some information about LVM and their mutual
relationships and combined usage and applications. The master document is
being written in Spanish, but as soon as I finish it, I've the intention
to translate it to English. The Spanish version should be ready during
July, and the English one maybe during August or September. Stay tuned :)

-- 
Jos� Luis Domingo L�pez
Linux Registered User #189436     Debian GNU/Linux Potato (P166 64 MB RAM)
 
jdomingo EN internautas PUNTO org  => � Spam ? Atente a las consecuencias
jdomingo AT internautas DOT   org  => Spam at your own risk

Re: [linux-lvm] HowTo: EIDE/ATA-100 + RAID 1 + LVM

[Goetz Bock]

[-- Attachment #1: Type: text/plain, Size: 2620 bytes --]


On Thu, Jun 28 '01 at 18:06, Emmanuel Blot wrote:
> I have to set up a server, and I'd like to know if it's possible 
yes
> - and reliable- to setup a server with RAID 1 (mirrored disk) with a 
perhapes
> pure software solution (not RAID controller) and LVM
> 
> [ ... ]
> * motherboard: Asus with AMD760 + DDR (Athlon Thunderbird)
I don't know about the reliability of the AMD760 ATA100 port, the VIA
one is pritty screwed. If you can you should perhapes get a prommise
ATA100 controller.
> [ ... ]

> - Any other suggestion
Just a few hints and MHO:
- use just one drive per channel (e.g. /dev/hda and /dev/hdc not hdb)
- the 2.4.x series of linux kernels is called stable but probably not
  realy ready for primetime yet. (mainly VM problems)
- with 2.2.x the softraid drivers are a lot older (e.g. not so reliable)
  and you'd have to create your own patches. and the stock 2.2.x kernel
  does not come with the new IDE drivers. But 2.2.x is more stable.
- always get the latest LVM patches, if you can wait for 1.0
- never try RAID on top of LVM. This is nice for testing, but I can't
  see why you might want to do this. 
- If you can partition your disk like this:
  128MB   /dev/hd[ac]1  /boot   for savety put boot /onto its own partition
  (2x)RAM /dev/hd[ac]2  swap    putting swap onto raid does not make
                                sence to me. The 2.4.x kernel NEEDS
                                twice the ammound of RAM as swap. The
                                kernel can do raid0 with swap partitions
                                (on its own)
  rest    /dev/hd[ac]3  md0     put the rest into a RAID1 device

  make /dev/md0 the only PV, with your VG in it.
- put some recovery tools into /boot (this will not realy help you if
  your server stands to far away, but perhapes you can attach a serial
  console connected to a different box ...)
- while the newest LILO beta can boot from a raid1 device, I would not
  stress it and just put one individual version /boot on each harddrive
  (with it's own bootloader) and handsync them (e.g. rsync). You should
  not have to much changes to the /boot partition, so I think this is
  ok.
- after the installation, make a full backup and than realy beat the
  system (focus on the ATA controller, cabeling, disks) to check it.
  Silent data corruption realy sucks, if it results from bad cabeling
  not even RAID will help you.
(- don't use SUSE if you can. Ok, well ignore this, this is just MVHO)

-- 
Goetz Bock                                              IT Consultant
Dipl.-Inf. Univ.

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: [linux-lvm] HowTo: EIDE/ATA-100 + RAID 1 + LVM

[Wolfgang Weisselberg]

Hi, Goetz!

Goetz Bock (bock@blacknet.de) wrote 80 lines:

>   (2x)RAM /dev/hd[ac]2  swap    putting swap onto raid does not make
>                                 sence to me. The 2.4.x kernel NEEDS

Swap on RAID will not give you speed (it will slow you rather),
but a HD failure will not crash the server, because suddenly
the swap has gone.  And without stability even over the loss of
a HD -- what is RAID worth?  Speed you can gain by striping
partitions -- at the cost of having to hope that all HDs
survive, instead of a quorum as with RAID.

-Wolfgang