Re: Best filesystems ?

From: Stan Hoeppner <stan@hardwarefreak.com>
To: xfs@oss.sgi.com
Subject: Re: Best filesystems ?
Date: Sat, 23 Oct 2010 16:28:56 -0500	[thread overview]
Message-ID: <4CC35398.6000308@hardwarefreak.com> (raw)
In-Reply-To: <19651.9652.631329.903552@tree.ty.sabi.co.uk>

Peter Grandi put forth on 10/23/2010 1:13 PM:

> * XFS is like JFS, and with somewhat higher scalability both as to
>   sizes and as to higher internal parallelism in the of multiple
>   processes accessing the same file, and has a couple of nice
>   features (mostly barrier support, but also small blocks and large
>   inodes). Its major limitation are internal complexity and should
>   only be used on 64b systems. It can support single filesystems
>   larger than 10-15TB, but that's stretching things.

I'm surprised you mentioned Lustre and not CXFS, especially given the
mailing list you posted this diatribe to.  Beasts of a different breed,
but both cluster oriented.  And, apparently you're not familiar with
NASA Advanced Supercomputing (NAS) Division at Ames Research Center.
They've got some multi hundred terabyte filesystems to show you:

http://www.nas.nasa.gov/Resources/Systems/archive_storage.html

> * Lustre works well as a network parallel large file streaming
>   filesystem. It is however somewhat unstable and great care has to
>   be taken in integration testing because of that.

Integration testing?  I guess you mean the linkers and compilers on the
build machines?  The Lustre client library gets linked directly into
every application that runs on the compute nodes--it's entirely user
space code.  Or at least this is the way it used to be IIRC.  To do any
general work with the files (move/copy/etc) you had to log onto an
object storage server node.  Maybe this has changed with newer revs.

> I currently think that JFS, XFS and Lustre cover more or less all
> common workloads. 

I'll agree with the first two, but Lustre isn't used for "common"
workloads.  See above.

> I occasionally use 'ext2' or NTFS for data
> exchange between Linux and MS-Windows.

Most people use Samba, C/DVD-Rs, or thumb drives for this purpose.  If
you have _that_ much data to move between systems, Samba over GigE
yields similar throughput to mounting a USB or eSATA disk, without the
hassle.  If Samba throws up a wall and you can't tune settings around
it, use rsync or something similar.  Dragging HDs between systems is
so... 80s. ;)

-- 
Stan

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs