reiserfs for cluster

reiserfs for cluster

[Juergen Waiblinger]

Hello,

we are running different beowulf clusters and the storage amount increases 
all the time. Until now we have been using Veritas Clusterfilesystem, but 
for different reasons we want to go for an other solution. As we are using 
reiserfs quite happily on the desktops we thought about using this for the 
storagesubsystem attached to the masterserver of the cluster.
there will only be sun and linux machines accessing the files but there 
will be concurrent file access.
Has anybody tried this or could give some other solutions
Any information will be appreciated

TIA will summarize to the list

Re: reiserfs for cluster

[Ookhoi]

Juergen Waiblinger wrote (ao):
> we are running different beowulf clusters and the storage amount
> increases all the time. Until now we have been using Veritas
> Clusterfilesystem, but for different reasons we want to go for an
> other solution. As we are using reiserfs quite happily on the desktops
> we thought about using this for the storagesubsystem attached to the
> masterserver of the cluster.
> there will only be sun and linux machines accessing the files but
> there will be concurrent file access.
> Has anybody tried this or could give some other solutions Any
> information will be appreciated

With the current 300GB sata disks, 3ware controllers and a full tower
which can handle 20 disks you can create a 5.3TB raid. mkreiserfs it,
stress-test it for a while and see if it works for you. Also try xfs and
compare it with reiserfs.

Re: reiserfs for cluster

[Russell Coker]

On Wed, 7 May 2003 00:57, Ookhoi wrote:
> Juergen Waiblinger wrote (ao):
> > we are running different beowulf clusters and the storage amount
> > increases all the time. Until now we have been using Veritas
> > Clusterfilesystem, but for different reasons we want to go for an
> > other solution. As we are using reiserfs quite happily on the desktops
> > we thought about using this for the storagesubsystem attached to the
> > masterserver of the cluster.
> > there will only be sun and linux machines accessing the files but
> > there will be concurrent file access.
> > Has anybody tried this or could give some other solutions Any
> > information will be appreciated
>
> With the current 300GB sata disks, 3ware controllers and a full tower
> which can handle 20 disks you can create a 5.3TB raid. mkreiserfs it,
> stress-test it for a while and see if it works for you. Also try xfs and
> compare it with reiserfs.

From Juergen's description I get the impression that he has a clustered file 
system.  A clustered file system such as GFS is more what he's asking for:
http://www.sistina.com/solutions_fileserve.html

However ReiserFS in a configuration such as you describe may be a viable 
option and will certainly be a lot cheaper.  I believe that ReiserFS does NFS 
well now and it should be a good option.

-- 
http://www.coker.com.au/selinux/   My NSA Security Enhanced Linux packages
http://www.coker.com.au/bonnie++/  Bonnie++ hard drive benchmark
http://www.coker.com.au/postal/    Postal SMTP/POP benchmark
http://www.coker.com.au/~russell/  My home page

Re: reiserfs for cluster

[Gerrit Hannaert]

I've tried something like this on a SAN a while ago, using ext3, but
perhaps the results would not be so different using reiserfs. The
situation we were looking at involved flat-file database updates which
would occur only once every day or even less than that.

Obviously the major issue is who has write-access; but things seem to
work if:
- one machine does the writes, after which the other machines will need
to re-mount the filesystem (probably works best if writes affect new
files, and are not appends/modifications...). Otherwise new files will
not show up because this data is cached (I expect someone else can give
some better low-level explanation and possible workarounds)...
- filesystem on read-only machines is mounted with journaling disabled,
and the noatime,ro options (updating access times are writes, too).
Otherwise the filesystem will not be in a consistent state, and will be
checked every time you mount it.

There may be other alternatives, but this is a simple solution if your
'write demands' are not too high. I haven't done much performance or
scalability tests so I don't know if this method is preferrable over NFS
and such (but it's so much cooler!).

Cheers,

- Gerrit


Juergen Waiblinger wrote:

> Hello,
>
> we are running different beowulf clusters and the storage amount
> increases all the time. Until now we have been using Veritas
> Clusterfilesystem, but for different reasons we want to go for an
> other solution. As we are using reiserfs quite happily on the desktops
> we thought about using this for the storagesubsystem attached to the
> masterserver of the cluster.
> there will only be sun and linux machines accessing the files but
> there will be concurrent file access.
> Has anybody tried this or could give some other solutions
> Any information will be appreciated
>
> TIA will summarize to the list
>