reiserfs for bkbits.net?

reiserfs for bkbits.net?

[Larry McVoy]

We're moving openlogging back to our offices and I'm experimenting with 
filesystems to see what gives the best performance for BK usage.  Reiserfs
looks pretty good and I'm wondering if anyone knows any reasons that we
shouldn't use it for bkbits.net.  Also, would it help if the journal was
on a different disk?  Most of the bkbits traffic is read so I doubt it.

Please cc me, I'm not on the list.

Re: reiserfs for bkbits.net?

[Sander]

Larry McVoy wrote (ao):
> We're moving openlogging back to our offices and I'm experimenting
> with filesystems to see what gives the best performance for BK usage.
> Reiserfs looks pretty good and I'm wondering if anyone knows any
> reasons that we shouldn't use it for bkbits.net. Also, would it help
> if the journal was on a different disk? Most of the bkbits traffic is
> read so I doubt it.
> 
> Please cc me, I'm not on the list.

I've cc'ed the Reiserfs mailinglist.

IME Reiserfs is a fast and stable fs. If you have the time to benchmark
ext3, reiserfs, jfs and xfs (and ..) with bk then you would know first
hand which fs is best for you. It might be worth the time.

With kind regards, Sander

-- 
Humilis IT Services and Solutions
http://www.humilis.net

Re: reiserfs for bkbits.net?

[Nikita Danilov]

Sander writes:
 > Larry McVoy wrote (ao):
 > > We're moving openlogging back to our offices and I'm experimenting
 > > with filesystems to see what gives the best performance for BK usage.
 > > Reiserfs looks pretty good and I'm wondering if anyone knows any
 > > reasons that we shouldn't use it for bkbits.net. Also, would it help
 > > if the journal was on a different disk? Most of the bkbits traffic is
 > > read so I doubt it.
 > > 
 > > Please cc me, I'm not on the list.
 > 
 > I've cc'ed the Reiserfs mailinglist.
 > 
 > IME Reiserfs is a fast and stable fs. If you have the time to benchmark
 > ext3, reiserfs, jfs and xfs (and ..) with bk then you would know first
 > hand which fs is best for you. It might be worth the time.

I can add that concurrent bk clone of kernel repositories is very good
file system stress tool that we are using while debugging reiser4.

 > 
 > With kind regards, Sander
 > 

Nikita.

Re: reiserfs for bkbits.net?

[Måns Rullgård]

Sander <sander@humilis.net> writes:

> Larry McVoy wrote (ao):
>> We're moving openlogging back to our offices and I'm experimenting
>> with filesystems to see what gives the best performance for BK usage.
>> Reiserfs looks pretty good and I'm wondering if anyone knows any
>> reasons that we shouldn't use it for bkbits.net. Also, would it help
>> if the journal was on a different disk? Most of the bkbits traffic is
>> read so I doubt it.
>> 
>> Please cc me, I'm not on the list.
>
> I've cc'ed the Reiserfs mailinglist.
>
> IME Reiserfs is a fast and stable fs. If you have the time to benchmark
> ext3, reiserfs, jfs and xfs (and ..) with bk then you would know first
> hand which fs is best for you. It might be worth the time.

If someone does any tests, I'd be interested to hear about the
results.

-- 
Måns Rullgård
mru@kth.se

Re: reiserfs for bkbits.net?

[Bryan Whitehead]

Måns Rullgård wrote:
> Sander <sander@humilis.net> writes:
> 
> 
>>Larry McVoy wrote (ao):
>>
>>>We're moving openlogging back to our offices and I'm experimenting
>>>with filesystems to see what gives the best performance for BK usage.
>>>Reiserfs looks pretty good and I'm wondering if anyone knows any
>>>reasons that we shouldn't use it for bkbits.net. Also, would it help
>>>if the journal was on a different disk? Most of the bkbits traffic is
>>>read so I doubt it.
>>>
>>>Please cc me, I'm not on the list.
>>
>>I've cc'ed the Reiserfs mailinglist.
>>
>>IME Reiserfs is a fast and stable fs. If you have the time to benchmark
>>ext3, reiserfs, jfs and xfs (and ..) with bk then you would know first
>>hand which fs is best for you. It might be worth the time.
> 
> 
> If someone does any tests, I'd be interested to hear about the
> results.
> 

http://pcbunn.cacr.caltech.edu/gae/3ware_raid_tests.htm

They needed 200MByte/sec disk transfer speed. this is how they got it.

-- 
Bryan Whitehead
SysAdmin - JPL - Interferometry and Large Optical Systems
Phone: 818 354 2903
driver@jpl.nasa.gov

Re: reiserfs for bkbits.net?

[Måns Rullgård]

Bryan Whitehead <driver@jpl.nasa.gov> writes:

> Måns Rullgård wrote:
>> Sander <sander@humilis.net> writes:
>>
>>>Larry McVoy wrote (ao):
>>>
>>>>We're moving openlogging back to our offices and I'm experimenting
>>>>with filesystems to see what gives the best performance for BK usage.
>>>>Reiserfs looks pretty good and I'm wondering if anyone knows any
>>>>reasons that we shouldn't use it for bkbits.net. Also, would it help
>>>>if the journal was on a different disk? Most of the bkbits traffic is
>>>>read so I doubt it.
>>>>
>>>>Please cc me, I'm not on the list.
>>>
>>>I've cc'ed the Reiserfs mailinglist.
>>>
>>>IME Reiserfs is a fast and stable fs. If you have the time to benchmark
>>>ext3, reiserfs, jfs and xfs (and ..) with bk then you would know first
>>>hand which fs is best for you. It might be worth the time.
>> If someone does any tests, I'd be interested to hear about the
>> results.
>>
>
> http://pcbunn.cacr.caltech.edu/gae/3ware_raid_tests.htm
>
> They needed 200MByte/sec disk transfer speed. this is how they got it.

I was thinking of typical BK workloads on less extreme hardware, in my
case software RAID 0+1 on normal IDE disks.

-- 
Måns Rullgård
mru@kth.se

Re: reiserfs for bkbits.net?

[Larry McVoy]

On Wed, Feb 11, 2004 at 10:29:16AM -0800, Bryan Whitehead wrote:
> http://pcbunn.cacr.caltech.edu/gae/3ware_raid_tests.htm
> 
> They needed 200MByte/sec disk transfer speed. this is how they got it.

Our workload is MUCH less friendly than bonnie.  We typically have lots
of traffic spread over lots of small files.  With 1-3 outstanding 
requests (i.e., just at the point where disk sort does you little good).
-- 
---
Larry McVoy                lm at bitmover.com           http://www.bitkeeper.com

Re: reiserfs for bkbits.net?

[Hans Reiser]

Will you have any chance of extremely large directories?  Do you or your 
users determine the maximum size of directories?

Re: reiserfs for bkbits.net?

[Larry McVoy]

On Wed, Feb 11, 2004 at 05:06:47PM -0800, Hans Reiser wrote:
> Will you have any chance of extremely large directories?  Do you or your 
> users determine the maximum size of directories?

We don't have that problem.  Imagine 500 clones of the Linux kernel all of
which are at least somewhat active on one disk arm.
-- 
---
Larry McVoy                lm at bitmover.com           http://www.bitkeeper.com

Re: reiserfs for bkbits.net?

[Mihai RUSU]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, 11 Feb 2004, Larry McVoy wrote:

> On Wed, Feb 11, 2004 at 10:29:16AM -0800, Bryan Whitehead wrote:
> > http://pcbunn.cacr.caltech.edu/gae/3ware_raid_tests.htm
> > 
> > They needed 200MByte/sec disk transfer speed. this is how they got it.
> 
> Our workload is MUCH less friendly than bonnie.  We typically have lots
> of traffic spread over lots of small files.  With 1-3 outstanding 
> requests (i.e., just at the point where disk sort does you little good).

Hmm, you could try parallel bonnie++ instances then:

- - synchronized
$ bonnie++ -d /path/to/testdir -s0 -n 4096:16000:64000:64 -p 10
then 10 times of 
$ bonnie++ -d /path/to/testdir -s0 -n 4096:16000:64000:64 -y

- - or just run them in background unsynchronized

- -- 
Mihai RUSU                                    Email: dizzy@roedu.net
GPG : http://dizzy.roedu.net/dizzy-gpg.txt    WWW: http://dizzy.roedu.net
                       "Linux is obsolete" -- AST
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)

iD8DBQFAKz2vPZzOzrZY/1QRAjs5AJ9E9wjayUmvKLrhsZ16KMnrm0OqXgCgqWy9
jtyLokxSBNvjqz1b8VfaDiM=
=jHZe
-----END PGP SIGNATURE-----

Re: reiserfs for bkbits.net?

[Edward Shishkin]

Sander wrote:
> 
> Larry McVoy wrote (ao):
> > We're moving openlogging back to our offices and I'm experimenting
> > with filesystems to see what gives the best performance for BK usage.
> > Reiserfs looks pretty good and I'm wondering if anyone knows any
> > reasons that we shouldn't use it for bkbits.net. Also, would it help
> > if the journal was on a different disk?

Not much if you mean performance: here is results for standard journal vs
journal relocated to NVRAM, everything in data logging journal mode:
http://thebsh.namesys.com/benchmarks/journal_relocation_to_NVRAM.html
Edward.

 Most of the bkbits traffic is
> > read so I doubt it.
> >
> > Please cc me, I'm not on the list.
> 
> I've cc'ed the Reiserfs mailinglist.
> 
> IME Reiserfs is a fast and stable fs. If you have the time to benchmark
> ext3, reiserfs, jfs and xfs (and ..) with bk then you would know first
> hand which fs is best for you. It might be worth the time.
> 
> With kind regards, Sander
> 
> --
> Humilis IT Services and Solutions
> http://www.humilis.net

Re: reiserfs for bkbits.net?

[Mihai RUSU]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Larry

On Wed, 11 Feb 2004, Larry McVoy wrote:

> We're moving openlogging back to our offices and I'm experimenting with 
> filesystems to see what gives the best performance for BK usage.  Reiserfs
> looks pretty good and I'm wondering if anyone knows any reasons that we
> shouldn't use it for bkbits.net.  Also, would it help if the journal was
> on a different disk?  Most of the bkbits traffic is read so I doubt it.

I dont have much reiserfs experience on production systems so I cannot say
anything about its stability. However, about the external journal
question, my bonnie++ tests (which I would recommend you do them yourself)
showed not much increase in speed than with internal journal, than say,
xfs with internal or external (but the reiserfs values where much better
in general, at least in the many small files test).

As you said because its mostly read() that shouldnt matter. Dont forget 
noatime mount option :).

- -- 
Mihai RUSU                                    Email: dizzy@roedu.net
GPG : http://dizzy.roedu.net/dizzy-gpg.txt    WWW: http://dizzy.roedu.net
                       "Linux is obsolete" -- AST
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)

iD8DBQFAKl4dPZzOzrZY/1QRAiAEAJ9xvPJ6o0F/m+Lxba581dyqGkZh7ACgj65e
G/ND//hn9QLij39pGdgvkR0=
=49A6
-----END PGP SIGNATURE-----

Re: reiserfs for bkbits.net?

[Larry McVoy]

On Wed, Feb 11, 2004 at 06:53:48PM +0200, Mihai RUSU wrote:
> As you said because its mostly read() that shouldnt matter. Dont forget 
> noatime mount option :).

Yeah, that goes for everyone else too.  BK looks at a lot of files, you
can get quite a boost by turning off atime.
-- 
---
Larry McVoy                lm at bitmover.com           http://www.bitkeeper.com

Re: reiserfs for bkbits.net?

[Erik Hensema]

Larry McVoy (lm@bitmover.com) wrote:
> We're moving openlogging back to our offices and I'm experimenting with 
> filesystems to see what gives the best performance for BK usage.  Reiserfs
> looks pretty good and I'm wondering if anyone knows any reasons that we
> shouldn't use it for bkbits.net.  Also, would it help if the journal was
> on a different disk?  Most of the bkbits traffic is read so I doubt it.

If bitkeeper uses lots of small files and/or many files in a
directory, then reiserfs is the FS for you.

The FS has been stable for a while now and I currently don't see
any reason not to use it.

-- 
Erik Hensema <erik@hensema.net>

Re: reiserfs for bkbits.net?

[Tomas Szepe]

On Feb-11 2004, Wed, 18:41 +0000
Erik Hensema <erik@hensema.net> wrote:

> The FS has been stable for a while now and I currently don't see
> any reason not to use it.

During the last two years, we have deployed some 400+ linux firewall
machines, all of which use reiserfs 3.6 for all of their filesystems.
While some of these boxes live in very wild environments (attics,
cellars, under the bed, in public block-of-flats corridors, ...) and
we've seen hardware die, there have been zero filesystem problems.
I think I can say we're happy with how reiser3 has fared so far.

Sounds a bit like from your favorite marketing department,
but still I thought you might want to know.

-- 
Tomas Szepe <szepe@pinerecords.com>

P.S.  It's also very nice to never have to fsck (that is unless
a broken driver/hardware writes random crap directly to the block
device).

Re: reiserfs for bkbits.net?

[Pavel Machek]

Hi!

> P.S.  It's also very nice to never have to fsck (that is unless
> a broken driver/hardware writes random crap directly to the block
> device).

You should force fsck once in a while. I did and there
were a nasty surprise for me...
				Pavel
-- 
64 bytes from 195.113.31.123: icmp_seq=28 ttl=51 time=448769.1 ms