public inbox for linux-xfs@vger.kernel.org
 help / color / mirror / Atom feed
From: Dave Chinner <david@fromorbit.com>
To: xfs@oss.sgi.com
Subject: [PATCH 4/5] repair: don't cache large blkmap allocations
Date: Mon, 10 Oct 2011 10:11:49 +1100	[thread overview]
Message-ID: <1318201910-11144-5-git-send-email-david@fromorbit.com> (raw)
In-Reply-To: <1318201910-11144-1-git-send-email-david@fromorbit.com>

From: Dave Chinner <dchinner@redhat.com>

We currently use thread local storage for storing blkmap allocations
from one inode to another as a way of reducing the number of short
term allocations we do. However, the stored allocations can only
ever grow, so once we've done a large allocation we never free than
memory even if we never need that much memory again. This can occur
if we have corrupted extent counts in inodes, and can greatly
increase the memory footprint of the repair process.

Hence if the cached blkmap array id greater than a reasonable number
of extents (say 100,000), then don't store the blkmap in TLS and
instead free it.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 repair/bmap.c   |   20 +++++++++++++++++++-
 repair/dinode.c |    6 ++----
 2 files changed, 21 insertions(+), 5 deletions(-)

diff --git a/repair/bmap.c b/repair/bmap.c
index 3ee5eff..3e53457 100644
--- a/repair/bmap.c
+++ b/repair/bmap.c
@@ -66,12 +66,30 @@ blkmap_alloc(
 
 /*
  * Free a block map.
+ *
+ * If the map is a large, uncommon size (say for hundreds of thousands of
+ * extents) then free it to release the memory. This prevents us from pinning
+ * large tracts of memory due to corrupted fork values or one-off fragmented
+ * files. Otherwise we have nothing to do but keep the memory around for the
+ * next inode
  */
 void
 blkmap_free(
 	blkmap_t	*blkmap)
 {
-	/* nothing to do! - keep the memory around for the next inode */
+	if (!blkmap)
+		return;
+
+	/* consider more than 100k extents rare */
+	if (blkmap->naexts < 100 * 1024)
+		return;
+
+	if (blkmap == pthread_getspecific(dblkmap_key))
+		pthread_setspecific(dblkmap_key, NULL);
+	else
+		pthread_setspecific(ablkmap_key, NULL);
+
+	free(blkmap);
 }
 
 /*
diff --git a/repair/dinode.c b/repair/dinode.c
index 0cedc28..8ad4e94 100644
--- a/repair/dinode.c
+++ b/repair/dinode.c
@@ -2748,8 +2748,7 @@ process_dinode_int(xfs_mount_t *mp,
 		break;
 	}
 
-	if (dblkmap)
-		blkmap_free(dblkmap);
+	blkmap_free(dblkmap);
 
 	/*
 	 * check nlinks feature, if it's a version 1 inode,
@@ -2768,8 +2767,7 @@ clear_bad_out:
 bad_out:
 	*used = is_free;
 	*isa_dir = 0;
-	if (dblkmap)
-		blkmap_free(dblkmap);
+	blkmap_free(dblkmap);
 	return 1;
 }
 
-- 
1.7.5.4

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

  parent reply	other threads:[~2011-10-09 23:12 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-10-09 23:11 [PATCH 0/5, v2] repair: sector size and blkmap fixes Dave Chinner
2011-10-09 23:11 ` [PATCH 1/5] repair: handle repair of image files on large sector size filesystems Dave Chinner
2011-10-09 23:52   ` Christoph Hellwig
2011-10-10  0:17     ` Dave Chinner
2011-10-10  0:19       ` Christoph Hellwig
2011-10-09 23:11 ` [PATCH 2/5] repair: fix some valgrind reported errors on i686 Dave Chinner
2011-10-09 23:45   ` Christoph Hellwig
2011-10-10  0:20     ` Dave Chinner
2011-10-10 13:14       ` Christoph Hellwig
2011-10-09 23:11 ` [PATCH 3/5] repair: handle memory allocation failure from blkmap_grow Dave Chinner
2011-10-09 23:51   ` Christoph Hellwig
2011-10-10  0:10     ` Dave Chinner
2011-10-09 23:11 ` Dave Chinner [this message]
2011-10-09 23:48   ` [PATCH 4/5] repair: don't cache large blkmap allocations Christoph Hellwig
2011-10-10  0:14     ` Dave Chinner
2011-10-10 13:22       ` Christoph Hellwig
2011-10-09 23:11 ` [PATCH 5/5] repair: prevent blkmap extent count overflows Dave Chinner
  -- strict thread matches above, loose matches on Subject: below --
2011-10-10  1:08 [PATCH 0/5, v3] repair: sector size and blkmap fixes Dave Chinner
2011-10-10  1:08 ` [PATCH 4/5] repair: don't cache large blkmap allocations Dave Chinner
2011-10-10 14:15   ` Christoph Hellwig
2011-10-13  9:57   ` Alex Elder

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1318201910-11144-5-git-send-email-david@fromorbit.com \
    --to=david@fromorbit.com \
    --cc=xfs@oss.sgi.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox