public inbox for linux-xfs@vger.kernel.org
 help / color / mirror / Atom feed
From: Dave Chinner <david@fromorbit.com>
To: xfs@oss.sgi.com
Subject: [PATCH 00/10, v2] repair: scalability and prefetch fixes
Date: Mon, 24 Feb 2014 17:29:19 +1100	[thread overview]
Message-ID: <1393223369-4696-1-git-send-email-david@fromorbit.com> (raw)

Hi folks,

This is a followup to the patchset posted here:

http://oss.sgi.com/archives/xfs/2013-12/msg00495.html

I've made various changes to address the review comments, and
droped the parallelisation of phase 6 after I realised that is was
causing occasional problem in pahse 6 (i.e. the simple patch didn't
make it entirely threadsafe).

There are various other prefetching fixes added to the series as
well that help with scalability - mainly to do with reworking the
way prefetching runs ahead of the processing threads. This allowed
unbound prefetching when the number of inode chunks was less than
the queue depth (typically 16384 inode chunks or 65536 inode cluster
buffers on 512 byte inode filesystems). This was causing the
prefetching to blow out the caches and result in all metadata being
read twice - once for the readhead, then again when it was actually
necessary.

The prefetching was changed to only readahead the AG following the
current one being processed, hence preventing thrashing and an awful
lot of unnecessary IO and buffer cache churn.

I'm not seeing any problems with this series, and performance on CRC
filesystems is now on a par with non-CRC filesystems so there are no
more known xfs_repair performance issues to be addressed with CRC
enabled filesytems after this patch set is applied.

Comments, review, flames and testing welcome!

Cheers,

Dave.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

             reply	other threads:[~2014-02-24  6:29 UTC|newest]

Thread overview: 29+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-02-24  6:29 Dave Chinner [this message]
2014-02-24  6:29 ` [PATCH 01/10] repair: translation lookups limit scalability Dave Chinner
2014-02-24 20:42   ` Brian Foster
2014-02-25 20:01   ` Christoph Hellwig
2014-02-24  6:29 ` [PATCH 02/10] repair: per AG locks contend for cachelines Dave Chinner
2014-02-24  6:29 ` [PATCH 03/10] libxfs: buffer cache hashing is suboptimal Dave Chinner
2014-02-24  6:29 ` [PATCH 04/10] repair: limit auto-striding concurrency apprpriately Dave Chinner
2014-02-24  6:29 ` [PATCH 05/10] repair: factor out threading setup code Dave Chinner
2014-02-24 20:43   ` Brian Foster
2014-02-24 23:16     ` Dave Chinner
2014-02-24 23:30       ` Brian Foster
2014-02-24  6:29 ` [PATCH 06/10] repair: use a listhead for the dotdot list Dave Chinner
2014-02-25 20:03   ` Christoph Hellwig
2014-02-27  2:06     ` Dave Chinner
2014-02-24  6:29 ` [PATCH 07/10] repair: prefetch runs too far ahead Dave Chinner
2014-02-26  1:52   ` Christoph Hellwig
2014-02-26  5:51     ` Dave Chinner
2014-02-24  6:29 ` [PATCH 08/10] libxfs: remove a couple of locks Dave Chinner
2014-02-25 20:05   ` Christoph Hellwig
2014-02-25 23:43     ` Dave Chinner
2014-02-26  1:54       ` Christoph Hellwig
2014-02-26  5:53         ` Dave Chinner
2014-02-24  6:29 ` [PATCH 09/10] repair: fix prefetch queue limiting Dave Chinner
2014-02-25 20:08   ` Christoph Hellwig
2014-02-24  6:29 ` [PATCH 10/10] repair: BMBT prefetch needs to be CRC aware Dave Chinner
2014-02-25 17:25   ` Christoph Hellwig
2014-02-25 23:51     ` Dave Chinner
2014-02-26  1:40       ` Christoph Hellwig
2014-02-26  1:44   ` Christoph Hellwig

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1393223369-4696-1-git-send-email-david@fromorbit.com \
    --to=david@fromorbit.com \
    --cc=xfs@oss.sgi.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox