tuning ext2 or ReiserFS to avoid fragmentation with large files?

tuning ext2 or ReiserFS to avoid fragmentation with large files?

[Roy Sigurd Karlsbakk]

hi all

I've got this server with lots of ~3GB files and every now and then we
need to add some more or delete some old ones. All files are potentially
read concurrently, so to reduce disk seeks, I've increased the readahead
settings in kernel (/proc/sys/vm/(min|max)-readahead).

Then... A friend of mine told me I could tune the fs (or vfs) to allocate
n kB each time a file is created, and by setting this to whatever I've set
(min|max)-readahead to (currently 1048576), I will reduce the negative
effect of fragmentation to a minimum, since the data blocks will be large,
and read more-or-less sequencially.

Can anyone tell me how to tell the fs or the kernel to allocate n pages/kB
this way? Is it possible? Can I possibly set different sizes per file
system?

Thanks

roy

--
Roy Sigurd Karlsbakk, MCSE, MCNE, CLS, LCA

Computers are like air conditioners.
They stop working when you open Windows.

Re: tuning ext2 or ReiserFS to avoid fragmentation with large files?

[Erik Tews]

On Mon, Dec 03, 2001 at 02:13:22PM +0100, Roy Sigurd Karlsbakk wrote:
> hi all
> 
> I've got this server with lots of ~3GB files and every now and then we
> need to add some more or delete some old ones. All files are potentially
> read concurrently, so to reduce disk seeks, I've increased the readahead
> settings in kernel (/proc/sys/vm/(min|max)-readahead).
> 
> Then... A friend of mine told me I could tune the fs (or vfs) to allocate
> n kB each time a file is created, and by setting this to whatever I've set
> (min|max)-readahead to (currently 1048576), I will reduce the negative
> effect of fragmentation to a minimum, since the data blocks will be large,
> and read more-or-less sequencially.
> 
> Can anyone tell me how to tell the fs or the kernel to allocate n pages/kB
> this way? Is it possible? Can I possibly set different sizes per file
> system?

If I remember right xfs has got a online-defragmentation utility. So
have a look at xfs.

I think xfs works different from reiserfs and ext2 when writing files to
disk which helps avoiding fragmentation. This feature is called
allocation groups.

Re: tuning ext2 or ReiserFS to avoid fragmentation with large files?

[Rasmus Bøg Hansen]

On Tue, 4 Dec 2001, Erik Tews wrote:

> If I remember right xfs has got a online-defragmentation utility. So
> have a look at xfs.
> 
> I think xfs works different from reiserfs and ext2 when writing files to
> disk which helps avoiding fragmentation. This feature is called
> allocation groups.

I *might* be wrong, but isn't the allocation-group thing exactly what 
ext2/ext3 does?

I don't know about reiserfs and fragmentation, however.

Rasmus

-- 
-- [ Rasmus 'Møffe' Bøg Hansen ] ---------------------------------------
Beware of bugs in the above code;
I have only proved it correct, not tried it.
                              - Donald Knuth
--------------------------------- [ moffe at amagerkollegiet dot dk ] --

Re: tuning ext2 or ReiserFS to avoid fragmentation with large files?

[Mike Fedyk]

On Wed, Dec 05, 2001 at 03:17:17AM +0100, Rasmus B?g Hansen wrote:
> On Tue, 4 Dec 2001, Erik Tews wrote:
> 
> > If I remember right xfs has got a online-defragmentation utility. So
> > have a look at xfs.
> > 
> > I think xfs works different from reiserfs and ext2 when writing files to
> > disk which helps avoiding fragmentation. This feature is called
> > allocation groups.
> 
> I *might* be wrong, but isn't the allocation-group thing exactly what 
> ext2/ext3 does?
>

Basically, yes.  They both have the name "group" in some of their feature
lists.  What really matters is *what* they encompass, and *how* they're used.

Can someone in the know comment about the similarity of the ext[23] and xfs
groups?

Re: tuning ext2 or ReiserFS to avoid fragmentation with large files?

[Erik Tews]

On Wed, Dec 05, 2001 at 03:17:17AM +0100, Rasmus Bøg Hansen wrote:
> On Tue, 4 Dec 2001, Erik Tews wrote:
> 
> > If I remember right xfs has got a online-defragmentation utility. So
> > have a look at xfs.
> > 
> > I think xfs works different from reiserfs and ext2 when writing files to
> > disk which helps avoiding fragmentation. This feature is called
> > allocation groups.
> 
> I *might* be wrong, but isn't the allocation-group thing exactly what 
> ext2/ext3 does?
> 
> I don't know about reiserfs and fragmentation, however.

I am sure that xfs is doing that and reiserfs is not doing that.

But I am not sure about ext2 and ext3.

Reiserfs4 is going to be different, and it will have a
online-defragmentation utility.