CEPH filesystem development
 help / color / mirror / Atom feed
From: Jeff Layton <jlayton-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
To: coreutils-mXXj517/zsQ@public.gmane.org
Cc: adilger-m1MBpc4rdrD3fQ9qLvQP4Q@public.gmane.org,
	dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org,
	ceph-devel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	linux-fsdevel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Subject: [coreutils PATCH v2 0/2] ls: convert to using statx when available
Date: Fri, 13 Sep 2019 06:58:03 -0400	[thread overview]
Message-ID: <20190913105805.24669-1-jlayton@kernel.org> (raw)

v2:
- add wrappers for stat_for_ino and fstat_for_ino, don't factor out loop
  detection
- style cleanups

Sending to a wider distribution list this time, as this may encourage
other filesystem maintainers to flesh out their statx implementations
to take advantage of this.

Original patch description follows:

This patchset converts the ls command to use statx instead of stat when
available. This allows ls to indicate interest in only certain inode
metadata.

This is potentially a win on networked/clustered/distributed
filesystems. In cases where we'd have to do a full, heavyweight stat()
call we can now do a much lighter statx() call.

As a real-world example, consider a filesystem like CephFS where one
client is actively writing to a file and another client does an
ls --color in the same directory. --color means that we need to fetch
the mode of the file.

Doing that with a stat() call means that we have to fetch the size and
mtime in addition to the mode. The MDS in that situation will have to
revoke caps in order to ensure that it has up-to-date values to report,
which disrupts the writer.

This has a measurable affect on performance. I ran a fio sequential
write test on one cephfs client and had a second client do "ls --color"
in a tight loop on the directory that held the file:

Baseline -- no activity on the second client:

  WRITE: bw=76.7MiB/s (80.4MB/s), 76.7MiB/s-76.7MiB/s (80.4MB/s-80.4MB/s), io=4600MiB (4824MB), run=60016-60016msec

Without this patch series, we see a noticable performance hit:

  WRITE: bw=70.4MiB/s (73.9MB/s), 70.4MiB/s-70.4MiB/s (73.9MB/s-73.9MB/s), io=4228MiB (4433MB), run=60012-60012msec

With this patch series, we gain most of that ground back:

  WRITE: bw=75.9MiB/s (79.6MB/s), 75.9MiB/s-75.9MiB/s (79.6MB/s-79.6MB/s), io=4555MiB (4776MB), run=60019-60019msec

Jeff Layton (2):
  stat: move struct statx to struct stat conversion routines to new
    header
  ls: use statx instead of stat when available

 src/ls.c    | 106 ++++++++++++++++++++++++++++++++++++++++++++++++----
 src/stat.c  |  32 +---------------
 src/statx.h |  54 ++++++++++++++++++++++++++
 3 files changed, 154 insertions(+), 38 deletions(-)
 create mode 100644 src/statx.h

-- 
2.21.0

             reply	other threads:[~2019-09-13 10:58 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-09-13 10:58 Jeff Layton [this message]
     [not found] ` <20190913105805.24669-1-jlayton-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
2019-09-13 10:58   ` [coreutils PATCH v2 1/2] stat: move struct statx to struct stat conversion routines to new header Jeff Layton
2019-09-13 10:58   ` [coreutils PATCH v2 2/2] ls: use statx instead of stat when available Jeff Layton

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190913105805.24669-1-jlayton@kernel.org \
    --to=jlayton-dgejt+ai2ygdnm+yrofe0a@public.gmane.org \
    --cc=adilger-m1MBpc4rdrD3fQ9qLvQP4Q@public.gmane.org \
    --cc=ceph-devel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=coreutils-mXXj517/zsQ@public.gmane.org \
    --cc=dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org \
    --cc=linux-fsdevel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox