public inbox for linux-xfs@vger.kernel.org
 help / color / mirror / Atom feed
From: Dave Chinner <david@fromorbit.com>
To: Ben Myers <bpm@sgi.com>
Cc: xfs@oss.sgi.com
Subject: Re: [PATCH 00/27] xfs: current patch queue for 3.11
Date: Thu, 13 Jun 2013 11:58:36 +1000	[thread overview]
Message-ID: <20130613015836.GF29338@dastard> (raw)
In-Reply-To: <20130612141743.GV20932@sgi.com>

On Wed, Jun 12, 2013 at 09:17:43AM -0500, Ben Myers wrote:
> Hey Dave,
> 
> On Wed, Jun 12, 2013 at 08:22:20PM +1000, Dave Chinner wrote:
> > Thoughts, comments, flames?
> 
> Do you have any performance numbers recorded for the block queue plugging for
> bulkstat and the new inode create transaction?

The new inode create transaction doesn't change performance on my
test rigs. It significantly reduces log traffic under heavey create
workloads (up to 50% lower). However, given the reduction is only
from 60MB/s down to 30MB/s at 110,000 inodes/s being created,  log
performance is not a performance limiting factor on any of my test
rigs.

That said, the reason for the change is not so much for immediate
improvements in inode create performance.  Want to allocate a stripe
width of inodes at a time?  The new transaction can do that
atomically....

Ordered buffers allow all sorts of interesting things to be done -
do you want to add an ext3/4 style data=ordered mode? We can do that
with ordered buffers.  Synchronous writes of remote attribute data?
Ordered buffers can be used to make that async and driven by AIL
flushing.

Want to use intent-based logging for operations rather than physical
object logging? Ordered log items and metadata stamped with the last
modification LSN are necessary, and with this icreate transaction we
end up with all the pieces we need to do this....

And for bulkstat, performance differences were documented in
this email where I found the problem:

http://oss.sgi.com/pipermail/xfs/2013-June/026922.html

It took a mulithreaded bulkstat from being IO bound at 450,000
inodes/s @ 220MB/s and 27,000 IOPS to being CPU bound at 750,000
inodes/s @ 350MB/s and 14000 IOPS.

And given that it increased IO sizes from 8k to 16k for 256 byte
inodes and to 32k for 512 byte inodes, that is going to increase
performance on any busy filesystem simply through the fact that
bulkstat IOPS overhead goes down by a factor of 2/4/8/16 depending
on inode size.....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

      reply	other threads:[~2013-06-13  1:58 UTC|newest]

Thread overview: 56+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-06-12 10:22 [PATCH 00/27] xfs: current patch queue for 3.11 Dave Chinner
2013-06-12 10:22 ` [PATCH 01/27] xfs: update mount options documentation Dave Chinner
2013-06-13 13:34   ` Eric Sandeen
2013-06-14  0:40     ` Dave Chinner
2013-06-14  0:53       ` Eric Sandeen
2013-06-12 10:22 ` [PATCH 02/27] xfs: add pluging for bulkstat readahead Dave Chinner
2013-06-12 10:22 ` [PATCH 03/27] xfs: plug directory buffer readahead Dave Chinner
2013-06-12 10:22 ` [PATCH 04/27] xfs: don't use speculative prealloc for small files Dave Chinner
2013-06-12 16:10   ` Brian Foster
2013-06-13  0:50     ` Dave Chinner
2013-06-12 10:22 ` [PATCH 05/27] xfs: don't do IO when creating an new inode Dave Chinner
2013-06-12 10:22 ` [PATCH 06/27] xfs: xfs_ifree doesn't need to modify the inode buffer Dave Chinner
2013-06-12 10:22 ` [PATCH 07/27] xfs: Introduce ordered log vector support Dave Chinner
2013-06-12 10:22 ` [PATCH 08/27] xfs: Introduce an ordered buffer item Dave Chinner
2013-06-12 10:22 ` [PATCH 09/27] xfs: Inode create log items Dave Chinner
2013-06-12 10:22 ` [PATCH 10/27] xfs: Inode create transaction reservations Dave Chinner
2013-06-12 10:22 ` [PATCH 11/27] xfs: Inode create item recovery Dave Chinner
2013-06-12 10:22 ` [PATCH 12/27] xfs: Use inode create transaction Dave Chinner
2013-06-12 10:22 ` [PATCH 13/27] xfs: remove local fork format handling from xfs_bmapi_write() Dave Chinner
2013-06-12 10:22 ` [PATCH 14/27] xfs: move getdents code into it's own file Dave Chinner
2013-06-12 10:22 ` [PATCH 15/27] xfs: reshuffle dir2 definitions around for userspace Dave Chinner
2013-06-17 16:05   ` Christoph Hellwig
2013-06-18 21:12     ` Dave Chinner
2013-06-18 21:35       ` Dave Chinner
2013-06-12 10:22 ` [PATCH 16/27] xfs: split out attribute listing code into separate file Dave Chinner
2013-06-12 10:22 ` [PATCH 17/27] xfs: split out attribute fork truncation " Dave Chinner
2013-06-12 10:22 ` [PATCH 18/27] xfs: split out xfs inode operations " Dave Chinner
2013-06-12 14:05   ` Christoph Hellwig
2013-06-13  1:14     ` Dave Chinner
2013-06-13  8:00       ` Dave Chinner
2013-06-17 15:56         ` Christoph Hellwig
2013-06-17 18:14           ` Ben Myers
2013-06-18 20:40             ` Dave Chinner
2013-06-18 21:37               ` Ben Myers
2013-06-18 22:02                 ` Dave Chinner
2013-06-12 10:22 ` [PATCH 19/27] xfs: consolidate xfs_vnodeops.c into xfs_inode_ops.c Dave Chinner
2013-06-12 13:59   ` Christoph Hellwig
2013-06-13  1:39     ` Dave Chinner
2013-06-17 16:02       ` Christoph Hellwig
2013-06-18 20:55         ` Dave Chinner
2013-06-12 10:22 ` [PATCH 20/27] xfs: move xfs_getbmap to xfs_extent_ops.c Dave Chinner
2013-06-12 10:22 ` [PATCH 21/27] xfs: introduce xfs_sb.c for sharing with libxfs Dave Chinner
2013-06-12 10:22 ` [PATCH 22/27] xfs: move xfs_trans_reservations to xfs_trans.h Dave Chinner
2013-06-12 10:22 ` [PATCH 23/27] xfs: sync minor header differences needed by userspace Dave Chinner
2013-06-12 10:22 ` [PATCH 24/27] xfs: move xfs_bmap_punch_delalloc() to xfs_aops.c Dave Chinner
2013-06-12 14:06   ` Christoph Hellwig
2013-06-13  1:39     ` Dave Chinner
2013-06-12 10:22 ` [PATCH 25/27] xfs: split out transaction reservation code Dave Chinner
2013-06-12 10:22 ` [PATCH 26/27] xfs: minor cleanups Dave Chinner
2013-06-12 10:22 ` [PATCH 27/27] xfs: fix issues that cause userspace warnings Dave Chinner
2013-06-17 19:32   ` Brian Foster
2013-06-18 21:42     ` Dave Chinner
2013-06-12 13:06 ` [PATCH 00/27] xfs: current patch queue for 3.11 Brian Foster
2013-06-13  1:40   ` Dave Chinner
2013-06-12 14:17 ` Ben Myers
2013-06-13  1:58   ` Dave Chinner [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20130613015836.GF29338@dastard \
    --to=david@fromorbit.com \
    --cc=bpm@sgi.com \
    --cc=xfs@oss.sgi.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox