public inbox for linux-xfs@vger.kernel.org
 help / color / mirror / Atom feed
From: Dave Chinner <david@fromorbit.com>
To: xfs@oss.sgi.com
Subject: [PATCH 35/36] repair: Increase default repair parallelism on large filesystems
Date: Wed, 13 Nov 2013 17:40:59 +1100	[thread overview]
Message-ID: <1384324860-25677-36-git-send-email-david@fromorbit.com> (raw)
In-Reply-To: <1384324860-25677-1-git-send-email-david@fromorbit.com>

From: Dave Chinner <dchinner@redhat.com>

Large filesystems or high AG count filesystems generally have more
inherent parallelism in the backing storage. We shoul dmake use of
this by default to speed up repair times. Make xfs_repair use an
"auto-stride" configuration on filesystems with enough AGs to be
considered "multidisk" configurations.

This difference in elaspsed time to repair a 100TB filesystem with
50 million inodes in it with all metadata in flash is:

		Time	IOPS	BW	CPU	RAM
vanilla:	2719s	 2900	 55MB/s	 25%	0.95GB
patched:	 908s	varied	varied	varied	2.33GB

With the patched kernel, there were IO peaks of over 1.3GB/s during
AG scanning. Some phases now run at noticably different speeds
	- phase 3 ran at ~180% CPU, 18,000 IOPS and 130MB/s,
	- phase 4 ran at ~280% CPU, 12,000 IOPS and 100MB/s
	- the other phases were similar to the vanilla repair.

Memory usage is increased because of the increased buffer cache
size as a result of concurrent AG scanning using it.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 repair/xfs_repair.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/repair/xfs_repair.c b/repair/xfs_repair.c
index 78f8363..a863337 100644
--- a/repair/xfs_repair.c
+++ b/repair/xfs_repair.c
@@ -614,6 +614,23 @@ main(int argc, char **argv)
 	inodes_per_cluster = MAX(mp->m_sb.sb_inopblock,
 			XFS_INODE_CLUSTER_SIZE(mp) >> mp->m_sb.sb_inodelog);
 
+	/*
+	 * Automatic striding for high agcount filesystems.
+	 *
+	 * More AGs indicates that the filesystem is either large or can handle
+	 * more IO parallelism. Either way, we should try to process multiple
+	 * AGs at a time in such a configuration to try to saturate the
+	 * underlying storage and speed the repair process. Only do this if
+	 * prefetching is enabled.
+	 *
+	 * Given mkfs defaults for 16AGs for "multidisk" configurations, we want
+	 * to target these for an increase in thread count. Hence a stride value
+	 * of 15 is chosen to ensure we get at least 2 AGs being scanned at once
+	 * on such filesystems.
+	 */
+	if (!ag_stride && glob_agcount >= 16 && do_prefetch)
+		ag_stride = 15;
+
 	if (ag_stride) {
 		thread_count = (glob_agcount + ag_stride - 1) / ag_stride;
 		thread_init();
-- 
1.8.4.rc3

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

  parent reply	other threads:[~2013-11-13  6:41 UTC|newest]

Thread overview: 45+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-11-13  6:40 [PATCH 00/36 V5] xfsprogs: CRC write support for xfs_db + Dave Chinner
2013-11-13  6:40 ` [PATCH 01/36] xfsprogs: fix automatic dependency generation Dave Chinner
2013-11-13  6:40 ` [PATCH 02/36] xfs: fix some minor sparse warnings Dave Chinner
2013-11-13  6:40 ` [PATCH 03/36] xfs: create a shared header file for format-related information Dave Chinner
2013-11-13  6:40 ` [PATCH 04/36] xfs: split dquot buffer operations out Dave Chinner
2013-11-13  6:40 ` [PATCH 05/36] xfs: decouple inode and bmap btree header files Dave Chinner
2013-11-13  6:40 ` [PATCH 06/36] libxfs: unify xfs_btree.c with kernel code Dave Chinner
2013-11-13  6:40 ` [PATCH 07/36] libxfs: bmap btree owner swap support Dave Chinner
2013-11-13  6:40 ` [PATCH 08/36] libxfs: xfs_rtalloc.c becomes xfs_rtbitmap.c Dave Chinner
2013-11-13  6:40 ` [PATCH 09/36] libxfs: bring across inode buffer readahead verifier changes Dave Chinner
2013-11-13  6:40 ` [PATCH 10/36] libxfs: Minor cleanup and bug fix sync Dave Chinner
2013-11-13  6:40 ` [PATCH 11/36] xfs: remove newlines from strings passed to __xfs_printk Dave Chinner
2013-11-13  6:40 ` [PATCH 12/36] xfs: fix the wrong new_size/rnew_size at xfs_iext_realloc_direct() Dave Chinner
2013-11-13  6:40 ` [PATCH 13/36] xfs: fix node forward in xfs_node_toosmall Dave Chinner
2013-11-13  6:40 ` [PATCH 14/36] xfs: don't emit corruption noise on fs probes Dave Chinner
2013-11-13  6:40 ` [PATCH 15/36] libxfs: fix root inode handling inconsistencies Dave Chinner
2013-11-13  6:40 ` [PATCH 16/36] libxfs: stop caching inode structures Dave Chinner
2013-11-13  6:40 ` [PATCH 17/36] db: separate out straight buffer IO from map based IO Dave Chinner
2013-11-13  6:40 ` [PATCH 18/36] db: rewrite bbmap to use xfs_buf_map Dave Chinner
2013-11-13  6:40 ` [PATCH 19/36] libxfs: refactor libxfs_buf_read_map for xfs_db Dave Chinner
2013-11-13  6:40 ` [PATCH 20/36] db: rewrite IO engine to use libxfs Dave Chinner
2013-11-13 16:05   ` Christoph Hellwig
2013-11-13  6:40 ` [PATCH 21/36] db: introduce verifier support into set_cur Dave Chinner
2013-11-13  6:40 ` [PATCH 22/36] db: indicate if the CRC on a buffer is correct or not Dave Chinner
2013-11-13  6:40 ` [PATCH 23/36] db: verify and calculate inode CRCs Dave Chinner
2013-11-13  6:40 ` [PATCH 24/36] db: verify and calculate dquot CRCs Dave Chinner
2013-11-13 16:05   ` Christoph Hellwig
2013-11-13  6:40 ` [PATCH 25/36] db: add a special directory buffer verifier Dave Chinner
2013-11-13  6:40 ` [PATCH 26/36] db: add a special attribute " Dave Chinner
2013-11-13  6:40 ` [PATCH 27/36] db: re-enable write support for v5 filesystems Dave Chinner
2013-11-13  6:40 ` [PATCH 28/36] xfs_db: use inode cluster buffers for inode IO Dave Chinner
2013-11-13  6:40 ` [PATCH 29/36] xfs_db: avoid libxfs buffer lookup warnings Dave Chinner
2013-11-13  6:40 ` [PATCH 30/36] libxfs: work around do_div() not handling 32 bit numerators Dave Chinner
2013-11-13  6:40 ` [PATCH 31/36] db: enable metadump on CRC filesystems Dave Chinner
2013-11-13 16:09   ` Christoph Hellwig
2013-11-13 21:00     ` Dave Chinner
2013-11-14 13:34       ` Christoph Hellwig
2013-11-13  6:40 ` [PATCH 32/36] xfs: support larger inode clusters on v5 filesystems Dave Chinner
2013-11-13  6:40 ` [PATCH 33/36] xfsprogs: kill experimental warnings for " Dave Chinner
2013-11-13  6:40 ` [PATCH 34/36] repair: prefetching is turned off unnecessarily Dave Chinner
2013-11-13  6:40 ` Dave Chinner [this message]
2013-11-13 16:10   ` [PATCH 35/36] repair: Increase default repair parallelism on large filesystems Christoph Hellwig
2013-11-13 21:01     ` Dave Chinner
2013-11-13  6:41 ` [PATCH 36/36] repair: fix leaf node directory data check Dave Chinner
2013-11-14 16:18 ` [PATCH 00/36 V5] xfsprogs: CRC write support for xfs_db + Rich Johnston

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1384324860-25677-36-git-send-email-david@fromorbit.com \
    --to=david@fromorbit.com \
    --cc=xfs@oss.sgi.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox