Re: Questions about XFS

[Roger Oberholtzer <roger@opq.se>]

On Wed, 2013-06-12 at 07:12 -0500, Stan Hoeppner wrote:
> On 6/12/2013 3:26 AM, Roger Oberholtzer wrote:
> ...
> > I have an application that is streaming data to an XFS disk at a
> > sustained 25 MB/sec. This is well below what the hardware supports. The
> > application does fopen/fwrite/fclose (no active flushing or syncing).
> 
> Buffered IO.
> 
> > I see that as my application writes data (the only process writing the
> > only open file on the disk), the system cache grows and grows. Here is
> > the unusual part: periodically, writes take some number of seconds to
> > complete, rather than the typical <50 msecs). The increased time seems
> > to correspond to the increasing size of the page cache.
> 
> Standard Linux buffered IO behavior.  Note this is not XFS specific.

That is correct. But users of XFS, like any others, may experience this
and have a solution. And it seemed to be related the the question in the
original post.

> > If I do:
> > 
> > echo 1 > /proc/sys/vm/drop_caches
> 
> Dumps the page cache forcing your buffered writes to disk.

The interesting thing is that when this is done, and the 3 or 4 GB of
cache goes away, it seems rather quick. Like the pages are not
containing data that must be written. But if that is the case, why the
increasingly long periodic write delays as the cache gets bigger?

> > while the application is runnung, then the writes do not occasionally
> > take longer. Until the cache grows again, and I do the echo again.
> 
> Which seems a bit laborious.
> 
> > I am sure I must be misinterpreting what I see.
> 
> Nope.  The Linux virtual memory system has behaved this way for quite
> some time.  You can teak how long IOs stay in cache.  See dirty_* at
> https://www.kernel.org/doc/Documentation/sysctl/vm.txt

> 
> Given the streaming nature you describe, have you looked at possibly
> using O_DIRECT?

I would really like to avoid this if possible. The data is not in
uniform chunks, so it would need to be buffered in the app to make it
so. The system can obviously keep up with the data rate - as long as it
does not get greedy with all that RAM just sitting there...

I have been thinking that I may need to do an occasional
fflush/fdatasync to be sure the write cache stays reasonably small.


Yours sincerely,

Roger Oberholtzer

Ramböll RST / Systems

Office: Int +46 10-615 60 20
Mobile: Int +46 70-815 1696
roger.oberholtzer@ramboll.se
________________________________________

Ramböll Sverige AB
Krukmakargatan 21
P.O. Box 17009
SE-104 62 Stockholm, Sweden
www.rambollrst.se


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs