ext2/3 files per directory limits

ext2/3 files per directory limits

[Ron Peterson]

I would like to better understand ext2/3's performance characteristics.

I'm specifically interested in how ext2/3 will handle a /var/spool/mail
directory w/ ~6000 mbox format inboxes, handling approx 1GB delivered as
75,000 messages daily.  Virtually all access is via imap, w/ approx
~1000 imapd processes running during peak load.  Local delivery is via
procmail, which by default uses both kernel-supported locking calls and
.lock files.

I understand that various tuning parameters will have an impact,
e.g. putting the journal on a separate device, setting the noatime mount
option, etc.  I also understand that there are other mailbox formats and
other strategies for locating mail spools (e.g. in user's home
directories).

I'm interested in people's thoughts on these issues, but I'm mostly
interested in whether or not the scenario I described falls within
ext2/3's designed capabilities.

Best.

-- 
Ron Peterson
Network & Systems Manager
Mount Holyoke College
http://www.mtholyoke.edu/~rpeterso

Re: ext2/3 files per directory limits

[Joel Jaeggli]

On Wed, 23 Feb 2005, Ron Peterson wrote:

> I would like to better understand ext2/3's performance characteristics.
>
> I'm specifically interested in how ext2/3 will handle a /var/spool/mail
> directory w/ ~6000 mbox format inboxes, handling approx 1GB delivered as
> 75,000 messages daily.  Virtually all access is via imap, w/ approx
> ~1000 imapd processes running during peak load.  Local delivery is via
> procmail, which by default uses both kernel-supported locking calls and
> .lock files.

At some point it makes sense to subdivide you mail load because 
serialization of i/o on that one filesystem becomes a bigger issue than 
the performance of your filesystem... We deliver into mbox formatted 
mailboxes inside users homedirs, some folks do a similar thing with 
maildir. In the end you can on make one filesystem so fast. beyond that 
you need more filesystems to acheive any kind of reasonable scaling...

> I understand that various tuning parameters will have an impact,
> e.g. putting the journal on a separate device, setting the noatime mount
> option, etc.  I also understand that there are other mailbox formats and
> other strategies for locating mail spools (e.g. in user's home
> directories).
>
> I'm interested in people's thoughts on these issues, but I'm mostly
> interested in whether or not the scenario I described falls within
> ext2/3's designed capabilities.
>
> Best.
>
>

-- 
-------------------------------------------------------------------------- 
Joel Jaeggli  	       Unix Consulting 	       joelja@darkwing.uoregon.edu 
GPG Key Fingerprint:     5C6E 0104 BAF0 40B0 5BD3 C38B F000 35AB B67F 56B2

Re: ext2/3 files per directory limits

[Lee Revell]

On Wed, 2005-02-23 at 22:11 -0500, Ron Peterson wrote:
> I would like to better understand ext2/3's performance characteristics.
> 
> I'm specifically interested in how ext2/3 will handle a /var/spool/mail
> directory w/ ~6000 mbox format inboxes, handling approx 1GB delivered as
> 75,000 messages daily.  Virtually all access is via imap, w/ approx
> ~1000 imapd processes running during peak load.  Local delivery is via
> procmail, which by default uses both kernel-supported locking calls and
> .lock files.
> 
> I understand that various tuning parameters will have an impact,
> e.g. putting the journal on a separate device, setting the noatime mount
> option, etc.  I also understand that there are other mailbox formats and
> other strategies for locating mail spools (e.g. in user's home
> directories).
> 
> I'm interested in people's thoughts on these issues, but I'm mostly
> interested in whether or not the scenario I described falls within
> ext2/3's designed capabilities.

Yes, ext2 and ext3 can handle that load easily.  You should not have to
do any special tuning.

The real question is why in the world you would want to use mbox format
for this.  It simply does not scale.  Use maildir.

Lee

Re: ext2/3 files per directory limits

[Andrew Morton]

Ron Peterson <rpeterso@mtholyoke.edu> wrote:
>
> I would like to better understand ext2/3's performance characteristics.
> 
> I'm specifically interested in how ext2/3 will handle a /var/spool/mail
> directory w/ ~6000 mbox format inboxes, handling approx 1GB delivered as
> 75,000 messages daily.  Virtually all access is via imap, w/ approx
> ~1000 imapd processes running during peak load.  Local delivery is via
> procmail, which by default uses both kernel-supported locking calls and
> .lock files.
> 
> I understand that various tuning parameters will have an impact,
> e.g. putting the journal on a separate device, setting the noatime mount
> option, etc.  I also understand that there are other mailbox formats and
> other strategies for locating mail spools (e.g. in user's home
> directories).
> 
> I'm interested in people's thoughts on these issues, but I'm mostly
> interested in whether or not the scenario I described falls within
> ext2/3's designed capabilities.
> 

noatime will help.

increasing the journal size _may_ help.

With 6k files per directory you'll benefit from indexed directories
(htree).  Use `tune2fs -O dir_index'.  dir_index isn't available for ext2.