From: Phil Karn <karn@philkarn.net>
To: Paul Anderson <pha@umich.edu>
Cc: Linux fs XFS <xfs@oss.sgi.com>
Subject: Re: I/O hang, possibly XFS, possibly general
Date: Thu, 02 Jun 2011 16:59:25 -0700 [thread overview]
Message-ID: <4DE823DD.7060600@philkarn.net> (raw)
In-Reply-To: <BANLkTim978GhfamN=TEFULP5GdfMu02-7w@mail.gmail.com>
On 6/2/11 2:24 PM, Paul Anderson wrote:
> The data itself has very odd lifecycle behavior, as well - since it is
> research, the different stages are still being sorted out, but some
> stages are essentially write once, read once, maybe keep, maybe
> discard, depending on the research scenario.
...
> The bulk of the work is not small-file - almost all is large files.
Out of curiosity, do your writers use the fallocate() call? If not, how
fragmented do your filesystems get?
Even if most of your data isn't read very often, it seems like a good
idea to minimize its fragmentation because that also reduces
fragmentation of the free list, which makes it easier to keep contiguous
other files that *are* heavily read. Also, fewer extents per file means
less metadata per file, ergo less metadata and log I/O, etc.
When a writer knows in advance how big a file will be, I can't see any
downside to having it call fallocate() to let the file system know. Soon
after I switched to XFS six months ago I've been running locally patched
versions of rsync/tar/cp and so on, and they really do minimize
fragmentation with very little effort.
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
next prev parent reply other threads:[~2011-06-02 23:59 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-06-02 14:42 I/O hang, possibly XFS, possibly general Paul Anderson
2011-06-02 16:17 ` Stan Hoeppner
2011-06-02 18:56 ` Peter Grandi
2011-06-02 21:24 ` Paul Anderson
2011-06-02 23:59 ` Phil Karn [this message]
2011-06-03 0:39 ` Dave Chinner
2011-06-03 2:11 ` Phil Karn
2011-06-03 2:54 ` Dave Chinner
2011-06-03 22:28 ` Phil Karn
2011-06-04 3:12 ` Dave Chinner
2011-06-03 22:19 ` Peter Grandi
2011-06-06 7:29 ` Michael Monnerie
2011-06-07 14:09 ` Peter Grandi
2011-06-08 5:18 ` Dave Chinner
2011-06-08 8:32 ` Michael Monnerie
2011-06-03 0:06 ` Phil Karn
2011-06-03 0:42 ` Christoph Hellwig
2011-06-03 1:39 ` Dave Chinner
2011-06-03 15:59 ` Paul Anderson
2011-06-04 3:15 ` Dave Chinner
2011-06-04 8:14 ` Stan Hoeppner
2011-06-04 10:32 ` Dave Chinner
2011-06-04 12:11 ` Stan Hoeppner
2011-06-04 23:10 ` Dave Chinner
2011-06-05 1:31 ` Stan Hoeppner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4DE823DD.7060600@philkarn.net \
--to=karn@philkarn.net \
--cc=pha@umich.edu \
--cc=xfs@oss.sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox