Re: definitions for /proc/fs/xfs/stat

[Dave Chinner <david@fromorbit.com>]

On Sun, Jun 16, 2013 at 08:58:58AM -0400, Mark Seger wrote:
> ok, I have a simple reproducer.  try out the following, noting you'll
> obviously have to change the directory pointed to by dname:
> 
> libc=ctypes.CDLL(ctypes.util.find_library('c'), use_errno=True)
> falloc=getattr(libc, 'fallocate')
> 
> fb#!/usr/bin/python -u
> 
> import os
> import sys
> import time
> import ctypes
> import ctypes.util
> from tempfile import mkstemp
> 
> dname = '/srv/node/disk0/mjs'
> fname = 'foo'
> fsize = 1024
> nfiles = 1000
> 
> body = ' ' * fsize
> 
> time0 = time.time()
> for i in range(nfiles):
>     file_name = '%s/%s-%s' % (dname, fname, i)
>     fd, tmppath = mkstemp(dir=dname)
>     falloc(fd, 1, 0, fsize)
>     os.rename(tmppath, file_name)
> 
> elapsed = time.time() - time0
> tbytes = fsize * nfiles
> rate = tbytes/elapsed/1024/1024
> 
> print "DName: %s" % dname
> print "Bytes: %d" % (tbytes/1024/1024)
> print "Time:  %.2f secs" % elapsed
> print "Rate:  %.2f/sec" % rate
> 
> when I run it I see this:
> 
> segerm@az1-sw-object-0006:~$ sudo ./falloc.py
> DName: /srv/node/disk0/mjs
> Bytes: 9
> Time:  5.84 secs
> Rate:  1.67/sec

Running on my 3.11 branch, python 2.7.3, 100TB XFS filesystem, there
is no IO to speak of:

$ tests/falloc.py 
DName: /mnt/scratch/
Bytes: 0
Time:  0.21 secs
Rate:  4.60/sec

> #<----CPU[HYPER]-----><----------Disks-----------><----------Network---------->
> #cpu sys inter  ctxsw KBRead  Reads KBWrit Writes   KBIn  PktIn  KBOut
>  PktOut
>    0   0   110    113      0      0      0      0      0      5      0
>   3
>    1   0  1576   2874      0      0 170240    665      0      3      0
>   2
>    4   3  2248   6623      0      0 406585   1596      0      1      0
>   1
>    4   3  2145   7680      0      0 473600   1850      0      1      0
>   1
>    2   1  2200   7406      0      0 456633   1875      0      2      0
>   1
>    4   3  3696   7401      0      0 454606   1823      0      1      0
>   1
>    3   2  3877   7354      0      0 453537   1806      0      1      0
>   1
>    1   0  1610   2764      0      0 163793    684      0      3      0
>   3

There is no way that fallocate() of 1000x1k files should be causing
450MB/s of IO for 5 seconds.

However, I still have no idea what you are running this test on - as
I asked in another email, can you provide some information about
the system your are seeing this problem on so we can try to work out
what might be causing this?

http://xfs.org/index.php/XFS_FAQ#Q:_What_information_should_I_include_when_reporting_a_problem.3F

> This is the same behavior I'm seeing on swift.  10K 1KB files X 4kb minimal
> block size still comes out to a lot less than the multiple GB of writes
> being reported.  Actually since these whole thing only takes a few seconds
> and I know a single disk can't write that fast maybe it's just a bug in the
> way the kernel is reported allocated preallocated blocks and nothing to do
> with XFS?  Or iis xfs responsible for the stats?

I don't know what stats you are getting your information from. XFS
is only responsible for what is in /proc/fs/xfs/,,,

> If I remove the fallocate call I see the expected amount of disk traffic.

Have you run strace to confirm that your python script is actually
calling fallocate(), and it's not being emulated by python or libc?

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs