New RAID-5 800GB array, wich fs ? wich stripe block size ?

New RAID-5 800GB array, wich fs ? wich stripe block size ?

[Aurélien Gouny]

Hi all,

I have a Promise SX6000 card and i juste have bought 6x160 GB HDs that 
make a 800 GB RAID-5 array.
I was wondering wich filesytem I should use (ext3/reiserfs/xfs/jfs/...), 
if I should make one or more partitions and wich stripe block size to 
use knowing that:

- it will be mainly a file-server with mp3s, divx, pictures (1 to 4MB 
each) and some files (openoffice, ...) all shared with Samba.
- it will have a small postgresql database

If someone need more information in order to determine the stripe block 
size or else, just ask ;)

Thanks a lot, (sorry for bad english)
Aurélien
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: New RAID-5 800GB array, wich fs ? wich stripe block size ?

[Bernhard Dobbels]

800 GB in one filesystem is possible, but I think it'll make life easier 
if you have multiple partitions which you can mount wherever you want.

Technically speaking, I prefer the LVM2 solution on which you can make 
and resize partitions on the fly.

I must admit I lost my raid5 with LVM2. The cause will probably have 
been my fault, although I don't know what I did wrong, so I suggest if 
you have critical data, a backup on another medium is still advised.

Good luck.

Bernhard
Aurélien Gouny wrote:
> Hi all,
> 
> I have a Promise SX6000 card and i juste have bought 6x160 GB HDs that 
> make a 800 GB RAID-5 array.
> I was wondering wich filesytem I should use (ext3/reiserfs/xfs/jfs/...), 
> if I should make one or more partitions and wich stripe block size to 
> use knowing that:
> 
> - it will be mainly a file-server with mp3s, divx, pictures (1 to 4MB 
> each) and some files (openoffice, ...) all shared with Samba.
> - it will have a small postgresql database
> 
> If someone need more information in order to determine the stripe block 
> size or else, just ask ;)
> 
> Thanks a lot, (sorry for bad english)
> Aurélien
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: New RAID-5 800GB array, wich fs ? wich stripe block size ?

[Mike Hardy]

That was a tepid review at best! No offense :-)

FWIW - I've got a 1.7TB RAID5 array that handles exactly the files you 
describe (we're probably doing the same thing - networked media server, 
right? With some other stuff on the side). In addition, it also handles 
rsync backups for all the other machines using the --link-dest argument 
and a rotations so it has an unbelievable amount of symlinks in the 
filesystem.

I also have a 1.0TB RAID5 array in a separate machine that I use as a 
backup for the main server, where all the media stuff gets backed up, 
but I don't backup the backups since that would be silly, or overkill at 
least.

I just chose plain vanilla ext3 ('mke2fs -j -m1 /dev/md2') on top of 
plain ol' raid 5. The only option that's different there is the "m1" 
since with an enormous filesystem, reserving 5% of it for root use is a 
bit silly.

With regard to performance, the first thing you'll notice is that unless 
you have gigabit to everywhere, you're limited by network I/O. I am 
anyway. I could saturate a 100Mbit network connection with the read 
speed, and after that, who cares?

Different filesystems will clearly be better for different situations, 
but if you're just looking to serve files over the network you're really 
not going to need to work hard to get it set up "good enough".

I will say though, I've had hardware failures and machine failures take 
the array out before - remember that MTBF is divided by the number of 
parts and arrays usually have lots of parts. Don't forget to backup 
early and often...

-Mike

Bernhard Dobbels wrote:
> 800 GB in one filesystem is possible, but I think it'll make life easier 
> if you have multiple partitions which you can mount wherever you want.
> 
> Technically speaking, I prefer the LVM2 solution on which you can make 
> and resize partitions on the fly.
> 
> I must admit I lost my raid5 with LVM2. The cause will probably have 
> been my fault, although I don't know what I did wrong, so I suggest if 
> you have critical data, a backup on another medium is still advised.
> 
> Good luck.
> 
> Bernhard
> Aurélien Gouny wrote:
> 
>> Hi all,
>>
>> I have a Promise SX6000 card and i juste have bought 6x160 GB HDs that 
>> make a 800 GB RAID-5 array.
>> I was wondering wich filesytem I should use 
>> (ext3/reiserfs/xfs/jfs/...), if I should make one or more partitions 
>> and wich stripe block size to use knowing that:
>>
>> - it will be mainly a file-server with mp3s, divx, pictures (1 to 4MB 
>> each) and some files (openoffice, ...) all shared with Samba.
>> - it will have a small postgresql database
>>
>> If someone need more information in order to determine the stripe 
>> block size or else, just ask ;)
>>
>> Thanks a lot, (sorry for bad english)
>> Aurélien
>> -
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: New RAID-5 800GB array, wich fs ? wich stripe block size ?

[TJ Harrell]

> I just chose plain vanilla ext3 ('mke2fs -j -m1 /dev/md2') on top of
> plain ol' raid 5. The only option that's different there is the "m1"
> since with an enormous filesystem, reserving 5% of it for root use is a
> bit silly.

I have a similar set up. A 600GB RAID-5 serving mp3's, and divx. I use ext3.
The journalling is a lifesaver if you ever have a crash for some reason.
Fsck
on a file-system that large is amazingly slow. I also mount it in
data=writeback
mode to improve performance. The -m option is important, otherwise you
waste a great deal of space. Additionally, the stripe and chunck sizes
should
be matched to the array when you issue the mke2fs. That's something that
was difficult for me to understand, so someone might want to explain that
one.


> With regard to performance, the first thing you'll notice is that unless
> you have gigabit to everywhere, you're limited by network I/O. I am
> anyway. I could saturate a 100Mbit network connection with the read
> speed, and after that, who cares?

I do have gigabit from my fileserver :) The disk bandwidth is actually not
that
great, though. I have yet to figure out why.

> I will say though, I've had hardware failures and machine failures take
> the array out before - remember that MTBF is divided by the number of
> parts and arrays usually have lots of parts. Don't forget to backup
> early and often...

Yes.. RAID is no replacement for backing up!!! My RAID array has proven
extremely durable and reliable through several hardware failures, however,
two drives dying at once, a simple power surge from a faulty PSU or
lightning,
or many other things could kill it. Even mistyping a command during a
recovery
could destroy the array.

> Bernhard Dobbels wrote:
> > Technically speaking, I prefer the LVM2 solution on which you can make
> > and resize partitions on the fly.
> >
> > I must admit I lost my raid5 with LVM2. The cause will probably have
> > been my fault, although I don't know what I did wrong, so I suggest if
> > you have critical data, a backup on another medium is still advised.

I mount my array on /home, and have all data files in the home directory of
my
user account, myself. I can create additional home directories for different
shares.
This is convient to me, although other people may feel otherwise. I do not
use LVM2
for two reasons. It is extra for me to learn to use, and it is an extra
possible point
of failure. The RAID code alone has been found to have bugs and weak code in
the
past. Using LVM2 on top of RAID, which also may have weak code or bugs, is
just
an additional possible fail point. In my opinion, it should not be used
unless it is
significantly useful. In my case, it isn't. I get by without needing it.


I wish you success,
TJ Harrell

Re: New RAID-5 800GB array, wich fs ? wich stripe block size ?

[Gordon Henderson]

On Mon, 19 Jul 2004, Mike Hardy wrote:

> I just chose plain vanilla ext3 ('mke2fs -j -m1 /dev/md2') on top of
> plain ol' raid 5. The only option that's different there is the "m1"
> since with an enormous filesystem, reserving 5% of it for root use is a
> bit silly.

I'm using ext3 on some biggish systems too. Never thought about the -m
option, but I always thought the 5% was for efficiency to stop
fragmentation?

> With regard to performance, the first thing you'll notice is that unless
> you have gigabit to everywhere, you're limited by network I/O. I am
> anyway. I could saturate a 100Mbit network connection with the read
> speed, and after that, who cares?

One thing you might want to look into with ext2/3 is the stride size.
Theres an option in mkfs.ext2/3 to let you set this, and words in the S/W
RAID HowTo describing its use with a RAID5 setup.

However, as you say, it probably doesn't matter if you only have 100Mb
networking! DAThe only time I've seen this make a difference though is
when backing up to a local DLT tape drive.

Basically, you set the stride size to the chunk-size / 4 (if you are using
the default 4K block size) So with a chunk-size of 64, the stride size
will be 16; mkfs -t ext3 -R stride=16 /dev/mdX

See

  http://www.tldp.org/HOWTO/Software-RAID-HOWTO-5.html#ss5.10

and read down to the bottom of the page.

However, I feel that this sort of tuning will really only matter if your
data is of a nature to take full advantage of it - eg. all big files,
access sequentially rather than lots of little files access randomly, or
whatever! I suspect the only way to see what will work for you is to suck
it and see!

> Different filesystems will clearly be better for different situations,
> but if you're just looking to serve files over the network you're really
> not going to need to work hard to get it set up "good enough".

Indeed. One other thing that you might want to invest in is a managed
Ethernet switch. That way to can see if it really is the filesystem thats
a bottleneck (assuming it's a server and not just a filestore for local
applications). I recently checked a network I look after, after the users
were wondering if the network was running flat-out (it's only 100Mb) and
was able to show that actually, 10Mb switches would be almost good enough
for the most-part. MRTG is your friend here! I was able to get more or
less full bandwidth out of it via NFS (running Bonnie) and over 5 times
that locally (again with Bonnie) so I was happy with the servers
performance and happy to blame the applications for being slow to write
their data ;-)

> I will say though, I've had hardware failures and machine failures take
> the array out before - remember that MTBF is divided by the number of
> parts and arrays usually have lots of parts. Don't forget to backup
> early and often...

Absolutely.

Gordon