All of lore.kernel.org
 help / color / mirror / Atom feed
From: Alex Elder <aelder@sgi.com>
To: wkendall@sgi.com
Cc: xfs@oss.sgi.com
Subject: Re: [PATCH v2 0/9] xfsrestore dirent limitations and scaling issues
Date: Fri, 12 Nov 2010 17:25:50 -0600	[thread overview]
Message-ID: <1289604350.2315.912.camel@doink> (raw)
In-Reply-To: <20101105163500.747192954@sgi.com>

On Fri, 2010-11-05 at 11:35 -0500, wkendall@sgi.com wrote:
> The first two patches in this series remove dirent limitations that
> exist in the current xfsrestore, allowing restore to now handle 4
> billion directory entries. Restore would map 200 GB of memory to do
> so, so don't go thinking this is a good idea. :)  (These two patches
> were previously submitted to the list but I've made some changes to
> them as suggested by Alex Elder.)
> 
> The remaining patches mostly deal with improving restore
> performance, most noticeably on dumps containing upwards of 10
> million inodes/dirents.  This resulted in a 50% improvement in the
> time required to build restore's node table (a mmap'd representation
> of the dump's directory structure), so for interactive restores and
> restoring sub-directories this is very helpful. For full restores
> with millions of files the overall restore time is dominated by
> creating inodes and laying down the data, so the improvements here
> would be less noticeable.
> 
> For dumps with lots of hard links, these changes fix a bug that was
> causing xfsrestore to constantly have to map and unmap segments of
> the node table, leading to horrible performance.
> 
> Several of these patches modify the on-disk state information that
> xfsrestore leaves around for resuming restores. The final patch adds
> versioning information to the on-disk state to detect cases where
> the user tries to resume a restore with an incompatible version of
> xfsrstore (or an incompatible system).

I've reviewed most of these but I'll have to return
to the rest next week.

					-Alex

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

      parent reply	other threads:[~2010-11-12 23:24 UTC|newest]

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-11-05 16:35 [PATCH v2 0/9] xfsrestore dirent limitations and scaling issues wkendall
2010-11-05 16:35 ` [PATCH v2 1/9] xfsrestore: turn off NODECHK wkendall
2010-11-12 23:23   ` Alex Elder
2010-11-05 16:35 ` [PATCH v2 2/9] xfsrestore: change nrh_t from 32 to 64 bits wkendall
2010-11-12 23:24   ` Alex Elder
2010-11-05 16:35 ` [PATCH v2 3/9] xfsrestore: cache path lookups wkendall
2010-11-12 23:25   ` Alex Elder
2010-11-05 16:35 ` [PATCH v2 4/9] xfsrestore: mmap dirent names for faster lookups wkendall
2010-11-12 23:25   ` Alex Elder
2010-11-15 21:51     ` Bill Kendall
2010-11-05 16:35 ` [PATCH v2 5/9] xfsrestore: cleanup node allocation wkendall
2010-11-15 20:38   ` Alex Elder
2010-11-15 21:36     ` Bill Kendall
2010-11-05 16:35 ` [PATCH v2 6/9] xfsrestore: fix node table setup wkendall
2010-11-15 20:38   ` Alex Elder
2010-11-15 21:30     ` Bill Kendall
2010-11-05 16:35 ` [PATCH v2 7/9] xfsrestore: make node lookup more efficient wkendall
2010-11-15 20:38   ` Alex Elder
2010-11-15 22:06     ` Bill Kendall
2010-11-05 16:35 ` [PATCH v2 8/9] xfsrestore: remove nix_t wkendall
2010-11-12 23:25   ` Alex Elder
2010-11-05 16:35 ` [PATCH v2 9/9] xfsrestore: check for compatible xfsrestore wkendall
2010-11-12 23:25   ` Alex Elder
2010-11-12 23:25 ` Alex Elder [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1289604350.2315.912.camel@doink \
    --to=aelder@sgi.com \
    --cc=wkendall@sgi.com \
    --cc=xfs@oss.sgi.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.