From: Dave Chinner <david@fromorbit.com>
To: Linda Walsh <xfs@tlinx.org>
Cc: xfs-oss <xfs@oss.sgi.com>
Subject: Re: better perf and memory uage for xfs_fsr? Trivial patch against xfstools-3.16 included...
Date: Fri, 9 Nov 2012 08:29:52 +1100 [thread overview]
Message-ID: <20121108212952.GR6434@dastard> (raw)
In-Reply-To: <509BAABF.3030608@tlinx.org>
On Thu, Nov 08, 2012 at 04:51:11AM -0800, Linda Walsh wrote:
> I was looking at output of xosview and watched a daily run of xfs_fsr take
> off as normal, and do it's normal thing of allocating remaining buffer
> memory during it's run -- and, as seems normal, it would go up to the
> machine's limit, then the kernel's memory reclaiming would kick in for a few
> hundred ms, and take memory usage down from 48G to maybe 30+G.
>
> During this time, I'd see xfs_fsr stop or pause for a bit, then resume and
> I'd see a used-buffer memory look like slow ramps followed by sharp drops
> with xfs_fsr showing i/o gaps as the ramps peaked.
It's not xfs_fsr that is causing this problem. It's other
applications, I think, generating memory pressure and xfs_fsr is the
unfortunate victim...
> I wondered why it lumped all this memory reclaiming and thought to try using
> the posix_fadvise calls in xfs_fsr to tell the kernel what data was unneeded
> and such...
posix_fadvise won't affect fsr at all, because:
int openopts = O_CREAT|O_EXCL|O_RDWR|O_DIRECT;
it uses direct IO, and hence bypasses the page cache. posix_fadvise
only affects buffered IO behaviour (i.e. via the page cache), and
hence will have no effect on IO issued by xfs_fsr.
> It doesn't increase or decrease the memory usage, just makes alloc'ing and
> freeing it in the kernel a bit smoother...
I'd say that's a subjective observation rather than an empirical one
- you're expecting it to improve, so you think it did. ;)
> Maybe other utils might benefit as well, dunno, ...but fsr was the most
> obvious to see the changes with.
Most of the XFS tools that do significant IO use direct IO, so it's
unlikely to make much measurable difference to tool behaviour...
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
prev parent reply other threads:[~2012-11-08 21:27 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-11-08 12:51 better perf and memory uage for xfs_fsr? Trivial patch against xfstools-3.16 included Linda Walsh
2012-11-08 20:30 ` Linda Walsh
2012-11-08 21:39 ` Dave Chinner
2012-11-09 7:10 ` Linda Walsh
2012-11-09 8:16 ` Dave Chinner
2012-11-08 21:29 ` Dave Chinner [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20121108212952.GR6434@dastard \
--to=david@fromorbit.com \
--cc=xfs@oss.sgi.com \
--cc=xfs@tlinx.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox