Encouraged or discouraged? Object database directly on reiserfs

Encouraged or discouraged? Object database directly on reiserfs

[Szabolcs Szasz]

Hi,

I'm creating an OODB for use by ISPs. Now for the prototype 
I use plain filesystem storage, object properties (names, phone 
numbers, URLs etc.) are all stored in multitudes of very small 
files, collected into large directories and there are symlinks all 
over the place...

For any "classical" filesystems this is clearly a horrible abuse...

It may be for the current reiserfs level, too, but much less.

Considering the future (and remembering the Vision doc. at
Namesys), is it encouraged or discouraged to continue the
OODB implementation atop the native filesystem storage?
(Speed is important, but not the primary concern. Unconstrained,
solid and versatile data accessibilty beats it --  that's why the 
fs storage rather than adding a DBMS engine.)

Thanks very much,
Sab

Re: Encouraged or discouraged? Object database directly on reiserfs

[Hans Reiser]

Szabolcs Szasz wrote:

>Hi,
>
>I'm creating an OODB for use by ISPs. Now for the prototype 
>I use plain filesystem storage, object properties (names, phone 
>numbers, URLs etc.) are all stored in multitudes of very small 
>files, collected into large directories and there are symlinks all 
>over the place...
>
>For any "classical" filesystems this is clearly a horrible abuse...
>
>It may be for the current reiserfs level, too, but much less.
>
Take a look at reiser4 next week, it will get a bit easier.

>
>Considering the future (and remembering the Vision doc. at
>Namesys), is it encouraged or discouraged to continue the
>OODB implementation atop the native filesystem storage?
>(Speed is important, but not the primary concern. Unconstrained,
>solid and versatile data accessibilty beats it --  that's why the 
>fs storage rather than adding a DBMS engine.)
>
>Thanks very much,
>Sab
>
>
>
>  
>
Is it a GPL OODB?

If it is a GPL OODB, we encourage you to do it, we'll answere your 
questions and be helpful, but we will try to obsolete you someday.

If it is not a GPL OODB, you might consider forming a commercial 
relationship with us.

Hans

Raid5, LVM and Reiserfs

[darren]

Hi all,

I got 5 pieces of 72GB SCSI harddisk that I would like to implement
raid5 with reiserfs with.

This is the first time I am configuring my linux box on a non-pc (this
is a dell poweredge server), and I figure I should get some advise
first.

Due to the bad performance (or so I heard on google) of the hardware
raid controller, I would like to use software raid5. 

I need on RedHat 7.3 with 2.4.19:

1)	Software RAID5 (any recommended settings?)

2)	Reiserfs (how do I make reiser over raid?)

3) 	I was told I need LVM too??

Any advise abt how I should go about this?

Thanx in advance
Darren

Re: Raid5, LVM and Reiserfs

[Lars Marowsky-Bree]

On 2002-10-20T07:47:50,
   darren <teodarren@myrealbox.com> said:

> I need on RedHat 7.3 with 2.4.19:
> 
> 1)	Software RAID5 (any recommended settings?)

Just as recommended per man 5 raidtab -> parity-algorithm left-symmetric,
chunk-size might need some experimentation, but I recall 32k is usually OK.

> 2)	Reiserfs (how do I make reiser over raid?)

The software raid device will show up as /dev/md0 (or similar); use that
blockdevice for creating the filesystem on.

> 3) 	I was told I need LVM too??

This is certainly a good idea. LVM will allow you to create many logical
volumes on a single software raid device; otherwise every block device you
want (ie, a separate filesystem) will require its own software raid device,
which will be a pain. Especially if you ever need to change sizes of the
devices or add/delete some, LVM will be a god-send.

> Any advise abt how I should go about this?

Carefully, as always ;-)

I suggest you read the software RAID howto. I am surprised that RHAT doesn't
offer selecting software raid right in the installer?


Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>

-- 
Principal Squirrel 
SuSE Labs - Research & Development, SuSE Linux AG
  
"If anything can go wrong, it will." "Chance favors the prepared (mind)."
  -- Capt. Edward A. Murphy            -- Louis Pasteur

Re: Raid5, LVM and Reiserfs

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 570 bytes --]

On Sun, 20 Oct 2002 07:47:50 +0800, darren <teodarren@myrealbox.com>  said:

> Due to the bad performance (or so I heard on google) of the hardware
> raid controller, I would like to use software raid5. 

You may wish to verify further.  Has the hardware raid controller been
identified as being slower than *software* RAID-5?  (Remember - if you do
it in software, you have to do a disk write per disk).  Or is the RAID
controller merely "not as fast as other hardware RAID controllers)?
-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: Raid5, LVM and Reiserfs

[Hans Reiser]

Valdis.Kletnieks@vt.edu wrote:

>On Sun, 20 Oct 2002 07:47:50 +0800, darren <teodarren@myrealbox.com>  said:
>
>  
>
>>Due to the bad performance (or so I heard on google) of the hardware
>>raid controller, I would like to use software raid5. 
>>    
>>
>
>You may wish to verify further.  Has the hardware raid controller been
>identified as being slower than *software* RAID-5?  (Remember - if you do
>it in software, you have to do a disk write per disk).  Or is the RAID
>controller merely "not as fast as other hardware RAID controllers)?
>  
>
It would be surprising if any hardware RAID controller was as bad as 
software RAID.  I agree that measuring is advisable.   I would be 
curious to see numbers on something like that.

Hans

Re: Raid5, LVM and Reiserfs

[Russell Coker]

On Mon, 21 Oct 2002 07:39, Hans Reiser wrote:
> >You may wish to verify further.  Has the hardware raid controller been
> >identified as being slower than *software* RAID-5?  (Remember - if you do
> >it in software, you have to do a disk write per disk).  Or is the RAID
> >controller merely "not as fast as other hardware RAID controllers)?
>
> It would be surprising if any hardware RAID controller was as bad as
> software RAID.  I agree that measuring is advisable.   I would be
> curious to see numbers on something like that.

It's fairly common for low-end RAID hardware to have limitations of memory or 
CPU bandwidth that limit the bulk IO rate.  Hardware RAID that can only 
sustain 10M/s was quite common until very recently.

However such hardware RAID did well on random IO tests, so the result you get 
depends on what exactly you do (as usual).

-- 
http://www.coker.com.au/bonnie++/     Bonnie++ hard drive benchmark
http://www.coker.com.au/postal/       Postal SMTP/POP benchmark
http://www.coker.com.au/projects.html Projects I am working on
http://www.coker.com.au/~russell/     My home page

Re: Raid5, LVM and Reiserfs

[Hans Reiser]

Russell Coker wrote:

>On Mon, 21 Oct 2002 07:39, Hans Reiser wrote:
>  
>
>>>You may wish to verify further.  Has the hardware raid controller been
>>>identified as being slower than *software* RAID-5?  (Remember - if you do
>>>it in software, you have to do a disk write per disk).  Or is the RAID
>>>controller merely "not as fast as other hardware RAID controllers)?
>>>      
>>>
>>It would be surprising if any hardware RAID controller was as bad as
>>software RAID.  I agree that measuring is advisable.   I would be
>>curious to see numbers on something like that.
>>    
>>
>
>It's fairly common for low-end RAID hardware to have limitations of memory or 
>CPU bandwidth that limit the bulk IO rate.  Hardware RAID that can only 
>sustain 10M/s was quite common until very recently.
>
Wow.   I guess marketing folks figured out that most customers would not 
measure it.  10M/s would be amazingly useless as a controller.

>
>However such hardware RAID did well on random IO tests, so the result you get 
>depends on what exactly you do (as usual).
>
>  
>

Re: Raid5, LVM and Reiserfs

[Russell Coker]

On Mon, 21 Oct 2002 08:04, Hans Reiser wrote:
> >It's fairly common for low-end RAID hardware to have limitations of memory
> > or CPU bandwidth that limit the bulk IO rate.  Hardware RAID that can
> > only sustain 10M/s was quite common until very recently.
>
> Wow.   I guess marketing folks figured out that most customers would not
> measure it.  10M/s would be amazingly useless as a controller.

Depends what you are doing.  When running dpkg to install new dpkg such a 
controller performed very well.  Running Bonnie++ gave results that sucked 
badly.

I didn't get a chance to do any real-world performance tests because the 
client went bankrupt too soon.

-- 
http://www.coker.com.au/bonnie++/     Bonnie++ hard drive benchmark
http://www.coker.com.au/postal/       Postal SMTP/POP benchmark
http://www.coker.com.au/projects.html Projects I am working on
http://www.coker.com.au/~russell/     My home page

Re: Raid5, LVM and Reiserfs

[bscott]

On Mon, 21 Oct 2002, at 9:39am, reiser@namesys.com wrote:
> It would be surprising if any hardware RAID controller was as bad as 
> software RAID.

  Funny you should mention that.  I was just discussing this last week on
the <poweredge-linux@dell.com> mailing list.  Apparently, not all RAID
controllers are created equal.  In particular, the design of the integrated
Adaptec RAID controller used in many Dell servers apparently sucks mud when
it comes to performance.  The AMI/LSI MegaRAID controllers that Dell uses as
add-in cards do much better.

-- 
Ben Scott <bscott@ntisys.com>
| The opinions expressed in this message are those of the author and do not |
| necessarily represent the views or policy of any other person, entity or  |
| organization.  All information is provided without warranty of any kind.  |

Re: Raid5, LVM and Reiserfs

[bscott]

On Mon, 21 Oct 2002, at 6:23pm, fcombernous@eprocess.fr wrote:
>> Funny you should mention that.  I was just discussing this last week on
>> the <poweredge-linux@dell.com> mailing list.  Apparently, not all RAID
>> controllers are created equal.  In particular, the design of the
>> integrated Adaptec RAID controller used in many Dell servers apparently
>> sucks mud when it comes to performance.  The AMI/LSI MegaRAID controllers
>> that Dell uses as add-in cards do much better.
>
> Can you give us an URL to look at history of your discussion. I'm
> interested by RAID controler provided by Dell servers.

  You can search the archives here:

http://lists.us.dell.com/htdig/

  A search on, e.g., "RAID performance", will yield many questions but few 
answers, I am afraid.

  The two messages I have posted on this are here:

http://lists.us.dell.com/pipermail/linux-poweredge/2002-October/004340.html
http://lists.us.dell.com/pipermail/linux-poweredge/2002-October/004427.html

but they do not say much more than I already have here.  I did receive a
couple of off-list replies to my first message, which is what I base my
information on.

  Dell kinda sorta acknowledges the problem here

http://lists.us.dell.com/fom-serve/cache/28.html

but they are very tight-lipped about it.  Typical vendor grumble grumble.

-- 
Ben Scott <bscott@ntisys.com>
| The opinions expressed in this message are those of the author and do not |
| necessarily represent the views or policy of any other person, entity or  |
| organization.  All information is provided without warranty of any kind.  |