File server FS?

File server FS?

[Brian]

We are about to build a fairly large (720GB) file server using Linux.  No 
sane person would actually want to watch this thing fsck, but I've seen 
mixed reports about the functionality of the journaled FSes.

Specifically, I need support for
 * KNFSD - it is a file server, afterall
 * LVM - For snapshots and to add space later
 * Resizing - See last point
 * Quotas - Eventually, but we don't need it just yet

Which, if any, of the journaled FSes support these?
Which one should I go with for a wide range of file and directory sizes?

I have no desire to wipe and restore 720GB of data, so we're pretty much 
stuck with what we launch with.

Thanks
	-- Brian

Re: File server FS?

[Mike Fedyk]

On Tue, Nov 13, 2001 at 05:03:34PM -0500, Brian wrote:
> We are about to build a fairly large (720GB) file server using Linux.  No 
> sane person would actually want to watch this thing fsck, but I've seen 
> mixed reports about the functionality of the journaled FSes.
>

Ext3!

> Specifically, I need support for
>  * KNFSD - it is a file server, afterall

Yep

>  * LVM - For snapshots and to add space later

Yep

>  * Resizing - See last point

There are two utilities to resize ext2, which ext3 is except for an
additional file (journal) after umount.

>  * Quotas - Eventually, but we don't need it just yet
>

A little tricky in Linus' kernel.  Should be sorted out soon.

> Which, if any, of the journaled FSes support these?
> Which one should I go with for a wide range of file and directory sizes?
>

How many files in a single dir?  Reiser is great for that, but not so good
for fragmentation after time on a 70% full or more FS...

There is indexing in development for ext2/3, but that'll be 2.5 work.

Mike

Re: File server FS?

[Sean Elble]

I'd have to recommend XFS for you . . . it supports the kernel mode NFS
server very well, it supports LVM, an XFS file system can be enlarged (not
reduced), and XFS has great quota support, just be sure you use a 3.0 or
greater quota tools package. Why use XFS over Ext3 you ask? XFS is faster,
and scales better, IMHO. Again just my opinion, but I hope that helps.

-----------------------------------------------
Sean P. Elble
Editor, Writer, Co-Webmaster
ReactiveLinux.com (Formerly MaximumLinux.org)
http://www.reactivelinux.com/
elbles@reactivelinux.com
-----------------------------------------------

----- Original Message -----
From: "Mike Fedyk" <mfedyk@matchmail.com>
To: "Brian" <hiryuu@envisiongames.net>
Cc: "linux-kernel" <linux-kernel@vger.kernel.org>
Sent: Tuesday, November 13, 2001 8:53 PM
Subject: Re: File server FS?


> On Tue, Nov 13, 2001 at 05:03:34PM -0500, Brian wrote:
> > We are about to build a fairly large (720GB) file server using Linux.
No
> > sane person would actually want to watch this thing fsck, but I've seen
> > mixed reports about the functionality of the journaled FSes.
> >
>
> Ext3!
>
> > Specifically, I need support for
> >  * KNFSD - it is a file server, afterall
>
> Yep
>
> >  * LVM - For snapshots and to add space later
>
> Yep
>
> >  * Resizing - See last point
>
> There are two utilities to resize ext2, which ext3 is except for an
> additional file (journal) after umount.
>
> >  * Quotas - Eventually, but we don't need it just yet
> >
>
> A little tricky in Linus' kernel.  Should be sorted out soon.
>
> > Which, if any, of the journaled FSes support these?
> > Which one should I go with for a wide range of file and directory sizes?
> >
>
> How many files in a single dir?  Reiser is great for that, but not so good
> for fragmentation after time on a 70% full or more FS...
>
> There is indexing in development for ext2/3, but that'll be 2.5 work.
>
> Mike
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: File server FS?

[Robert Szentmihalyi]

Am Mittwoch, 14. November 2001 03:05 schrieb Sean Elble:
> I'd have to recommend XFS for you . . . it supports the kernel
> mode NFS server very well, it supports LVM, an XFS file system
> can be enlarged (not reduced), and XFS has great quota support,
> just be sure you use a 3.0 or greater quota tools package. Why
> use XFS over Ext3 you ask? XFS is faster, and scales better,
> IMHO. Again just my opinion, but I hope that helps.
>

ACK.
We have built an 800 GB file server for a customer about three 
month ago using XFS on a 3ware RAID.
The server performs great, even under heay load.
The only drawback is that group quotas were not yet supported then.
I don't know if this has changed yet, but it should be fairly easy 
to find out..... :-)

cheers,
 Robert

> -----------------------------------------------
> Sean P. Elble
> Editor, Writer, Co-Webmaster
> ReactiveLinux.com (Formerly MaximumLinux.org)
> http://www.reactivelinux.com/
> elbles@reactivelinux.com
> -----------------------------------------------
>
> ----- Original Message -----
> From: "Mike Fedyk" <mfedyk@matchmail.com>
> To: "Brian" <hiryuu@envisiongames.net>
> Cc: "linux-kernel" <linux-kernel@vger.kernel.org>
> Sent: Tuesday, November 13, 2001 8:53 PM
> Subject: Re: File server FS?
>
> > On Tue, Nov 13, 2001 at 05:03:34PM -0500, Brian wrote:
> > > We are about to build a fairly large (720GB) file server
> > > using Linux.
>
> No
>
> > > sane person would actually want to watch this thing fsck, but
> > > I've seen mixed reports about the functionality of the
> > > journaled FSes.
> >
> > Ext3!
> >
> > > Specifically, I need support for
> > >  * KNFSD - it is a file server, afterall
> >
> > Yep
> >
> > >  * LVM - For snapshots and to add space later
> >
> > Yep
> >
> > >  * Resizing - See last point
> >
> > There are two utilities to resize ext2, which ext3 is except
> > for an additional file (journal) after umount.
> >
> > >  * Quotas - Eventually, but we don't need it just yet
> >
> > A little tricky in Linus' kernel.  Should be sorted out soon.
> >
> > > Which, if any, of the journaled FSes support these?
> > > Which one should I go with for a wide range of file and
> > > directory sizes?
> >
> > How many files in a single dir?  Reiser is great for that, but
> > not so good for fragmentation after time on a 70% full or more
> > FS...
> >
> > There is indexing in development for ext2/3, but that'll be 2.5
> > work.
> >
> > Mike
> > -
> > To unsubscribe from this list: send the line "unsubscribe
> > linux-kernel" in the body of a message to
> > majordomo@vger.kernel.org
> > More majordomo info at 
> > http://vger.kernel.org/majordomo-info.html Please read the FAQ
> > at  http://www.tux.org/lkml/
>
> -
> To unsubscribe from this list: send the line "unsubscribe
> linux-kernel" in the body of a message to
> majordomo@vger.kernel.org
> More majordomo info at 
> http://vger.kernel.org/majordomo-info.html Please read the FAQ at
>  http://www.tux.org/lkml/

-- 
Where do you want to be tomorrow?

Entracom. Building Linux systems.
http://www.entracom.de

Re: File server FS?

[Steve Lord]

On Wed, 2001-11-14 at 04:41, Robert Szentmihalyi wrote:
> Am Mittwoch, 14. November 2001 03:05 schrieb Sean Elble:
> > I'd have to recommend XFS for you . . . it supports the kernel
> > mode NFS server very well, it supports LVM, an XFS file system
> > can be enlarged (not reduced), and XFS has great quota support,
> > just be sure you use a 3.0 or greater quota tools package. Why
> > use XFS over Ext3 you ask? XFS is faster, and scales better,
> > IMHO. Again just my opinion, but I hope that helps.
> >
> 
> ACK.
> We have built an 800 GB file server for a customer about three 
> month ago using XFS on a 3ware RAID.
> The server performs great, even under heay load.
> The only drawback is that group quotas were not yet supported then.
> I don't know if this has changed yet, but it should be fairly easy 
> to find out..... :-)
> 
> cheers,
>  Robert
> 

XFS on linux has had group quota support for quite a while - certainly
longer than 3 months. All the other features are available too.

Steve

-- 

Steve Lord                                      voice: +1-651-683-3511
Principal Engineer, Filesystem Software         email: lord@sgi.com

Re: File server FS?

[Robert Szentmihalyi]

> On Wed, 2001-11-14 at 04:41, Robert Szentmihalyi wrote:
> > Am Mittwoch, 14. November 2001 03:05 schrieb Sean Elble:
> > > I'd have to recommend XFS for you . . . it supports the
> > > kernel mode NFS server very well, it supports LVM, an XFS
> > > file system can be enlarged (not reduced), and XFS has great
> > > quota support, just be sure you use a 3.0 or greater quota
> > > tools package. Why use XFS over Ext3 you ask? XFS is faster,
> > > and scales better, IMHO. Again just my opinion, but I hope
> > > that helps.
> >
> > ACK.
> > We have built an 800 GB file server for a customer about three
> > month ago using XFS on a 3ware RAID.
> > The server performs great, even under heay load.
> > The only drawback is that group quotas were not yet supported
> > then. I don't know if this has changed yet, but it should be
> > fairly easy to find out..... :-)
> >
> > cheers,
> >  Robert
>
> XFS on linux has had group quota support for quite a while -
> certainly longer than 3 months. All the other features are
> available too.

Nice to know :-)
I have not tried it since the FAQ at 
http://oss.sgi.com/projects/xfs/faq.html#quotaswork
said it didn't. (It still does, by the way. Perhaps you could 
update the FAQ :-))

>
> Steve

Robert

-- 
Where do you want to be tomorrow?

Entracom. Building Linux systems.
http://www.entracom.de

Re: File server FS?

[Pavel Machek]

Hi!

> We are about to build a fairly large (720GB) file server using Linux.  No 
> sane person would actually want to watch this thing fsck, but I've seen 
> mixed reports about the functionality of the journaled FSes.

fsck should take hour and a half with 4K blocksize. And I *would* like
to see it fsck ;-))))
								Pavel
-- 
Philips Velo 1: 1"x4"x8", 300gram, 60, 12MB, 40bogomips, linux, mutt,
details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.

Re: File server FS?

[Jamie Lokier]

Mike Fedyk wrote:
> >  * Resizing - See last point
> 
> There are two utilities to resize ext2, which ext3 is except for an
> additional file (journal) after umount.

Two questions:

  1. Does the size of the "appropriately sized journal (given the size
     of the filesystem)" vary with filesystem size?

  2. If so, does resize2fs change the journal size properly?

When I have resized ext3 filesystems, I have removed then recreated the
journal manually because it wasn't clear from the documentation whether
resize2fs does the appropriate thing.

Thanks,
-- Jamie

Re: File server FS?

[Mike Fedyk]

On Sat, Nov 17, 2001 at 06:12:53PM +0000, Jamie Lokier wrote:
> Mike Fedyk wrote:
> > >  * Resizing - See last point
> > 
> > There are two utilities to resize ext2, which ext3 is except for an
> > additional file (journal) after umount.
> 
> Two questions:
> 
>   1. Does the size of the "appropriately sized journal (given the size
>      of the filesystem)" vary with filesystem size?

Journal size has more to do with activity when you are in data journaling
mode.  Otherwise you will be hard pressed to fill a 32MB journal with
meta-data.

> 
>   2. If so, does resize2fs change the journal size properly?
>

As long as resize2fs doesn't change the inode of the journal file you should
be fine.

> When I have resized ext3 filesystems, I have removed then recreated the
> journal manually because it wasn't clear from the documentation whether
> resize2fs does the appropriate thing.
> 

I haven't actually resized any ext2/3 partitions.  Didn't need to.  I'll do
some tests though.

Andrew, Andreas, any official comments?

Mike

Re: File server FS?

[Andrew Morton]

Mike Fedyk wrote:
> 
> I haven't actually resized any ext2/3 partitions.  Didn't need to.  I'll do
> some tests though.

I tested it a while back - it worked OK.  If you could retest that'd be
neat.

The journal shouldn't be affected - it's just a regular file.

mke2fs and tune2fs choose an initial journal size based
on the size of the fs, so if you were increasing the
fs size by a large ratio then there may be a case for
increasing the journal size.  But as you've pointed out,
an 8, 16 or 32 megabyte journal covers an awful lot of metadata.

Re: File server FS?

[Mike Fedyk]

On Sat, Nov 17, 2001 at 02:10:01PM -0800, Andrew Morton wrote:
> Mike Fedyk wrote:
> > 
> > I haven't actually resized any ext2/3 partitions.  Didn't need to.  I'll do
> > some tests though.
>
> I tested it a while back - it worked OK.  If you could retest that'd be
> neat.
> 

I'll try to do that over the next couple weeks.

> The journal shouldn't be affected - it's just a regular file.
>

That's what I meant by "as long as the inode number is the same".  Since it
is a normal file, the only thing ext2resize might overlook would be the
inode number for the jounal that's kept in the super block.  If, in fact
ext2resize does decide to change inode numbers for some reason.  I don't
know if it does. 

> mke2fs and tune2fs choose an initial journal size based
> on the size of the fs, so if you were increasing the
> fs size by a large ratio then there may be a case for
> increasing the journal size.  But as you've pointed out,
> an 8, 16 or 32 megabyte journal covers an awful lot of metadata.

Yep.  It would be more important for data=journal mode.

Can ext2resize change the block size too?  If the journal is larger than
100MB then it would need to be made smaller for 1k blocks 200MB for 2k
blocks, and left at 400MB for 4k blocks.

Mike

Re: File server FS?

[Andreas Dilger]

On Nov 17, 2001  13:55 -0800, Mike Fedyk wrote:
> On Sat, Nov 17, 2001 at 06:12:53PM +0000, Jamie Lokier wrote:
> > Mike Fedyk wrote:
> > > There are two utilities to resize ext2, which ext3 is except for an
> > > additional file (journal) after umount.
> > 
> > Two questions:
> > 
> >   1. Does the size of the "appropriately sized journal (given the size
> >      of the filesystem)" vary with filesystem size?

Only vaguely.  The current size of the journal is mostly guesswork, because
we don't have any tools to measure if the journal is full or not anyways.

> Journal size has more to do with activity when you are in data journaling
> mode.  Otherwise you will be hard pressed to fill a 32MB journal with
> meta-data.

Correct.

> >   2. If so, does resize2fs change the journal size properly?

No, neither does ext2resize.

> As long as resize2fs doesn't change the inode of the journal file you should
> be fine.

Correct.

> > When I have resized ext3 filesystems, I have removed then recreated the
> > journal manually because it wasn't clear from the documentation whether
> > resize2fs does the appropriate thing.

Like Mike says, there should be very minimal impact to the filesystem
operation, unless you are going from, say, a 16MB filesystem to a 500GB
filesystem.  You also have to watch out if you start with a filesystem
smaller than 500MB - you will get 1kB blocks, and you don't want to have
a large filesystem (10's of GB) with a 1kB blocksize.  There is nothing
that resize2fs or ext2resize can do about that, unfortunately.

> I haven't actually resized any ext2/3 partitions.  Didn't need to.  I'll do
> some tests though.

It works just fine with ext2resize, and I'm pretty sure resize2fs also
works on ext3 filesystems.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

Re: File server FS?

[Andrew Morton]

Mike Fedyk wrote:
> 
> Can ext2resize change the block size too?

Nope.  Block size is rather fundamental.

Re: File server FS?

[Andreas Dilger]

On Nov 17, 2001  14:23 -0800, Mike Fedyk wrote:
> That's what I meant by "as long as the inode number is the same".  Since it
> is a normal file, the only thing ext2resize might overlook would be the
> inode number for the jounal that's kept in the super block.  If, in fact
> ext2resize does decide to change inode numbers for some reason.  I don't
> know if it does. 

ext2resize will only move inodes at the end of the fs, and only if you are
shrinking the fs.  I suppose in some cases (creating a journal on an old
fs) there might be a journal not in the first group, but it is unlikely,
since ext2 will always allocate files in the same group as the parent
(the root inode), so it would always be in the first group, unless you
were out of inodes there (unlikely).  With newer e2fsck's, it also moves
the journal to the reserved inode (#8) so it would remove that problem.

With online resizing (not that it works with ext3 yet, but) since you are
only ever growing the fs, you would also not renumber the inodes.

> Can ext2resize change the block size too?  If the journal is larger than
> 100MB then it would need to be made smaller for 1k blocks 200MB for 2k
> blocks, and left at 400MB for 4k blocks.

No, that is a very difficult problem (especially growing the blocksize,
which is what most people would want to do), because none of the 1kB
blocks would be aligned properly.  You would need to move basically all
of the data in the filesystem, at which point you are far better off to
create a new fs and copy over the data - faster and much less likely to
have any problems.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

Re: File server FS?

[Jamie Lokier]

Andreas Dilger wrote:
> > Can ext2resize change the block size too?  If the journal is larger than
> > 100MB then it would need to be made smaller for 1k blocks 200MB for 2k
> > blocks, and left at 400MB for 4k blocks.
> 
> No, that is a very difficult problem (especially growing the blocksize,
> which is what most people would want to do), because none of the 1kB
> blocks would be aligned properly.  You would need to move basically all
> of the data in the filesystem, at which point you are far better off to
> create a new fs and copy over the data - faster and much less likely to
> have any problems.

Well, it's not really faster if you don't have that much spare disk
space.  You end up doing lots of resizes of the old fs, to gradually
make space for the new fs, and between each resize copy over some of the
files.  Oh, and lots of scary "dd" commands to slide the new fs down the
disk as it grows.

I know this because it's what I had to do, on two computers.

cheers,
-- Jamie