From: "Darrick J. Wong" <darrick.wong@oracle.com>
To: tytso@mit.edu, darrick.wong@oracle.com
Cc: linux-ext4@vger.kernel.org
Subject: [PATCH 34/74] e2fsck: in rehash, mark newly allocated extent blocks as found
Date: Tue, 10 Dec 2013 17:22:07 -0800 [thread overview]
Message-ID: <20131211012207.30655.92282.stgit@birch.djwong.org> (raw)
In-Reply-To: <20131211011813.30655.39624.stgit@birch.djwong.org>
When we're rehashing directories, it's possible that an extent block
(or a map block) could be (silently) allocated by the underlying
libext2fs when expanding the directory. This silent allocation is not
captured in block_found_map, which is disastrous if later the rehash
process expands another directory and uses that same block from
before without realizing that it's now in use.
Therefore, if we notice that the free block count has dropped by more
than what e2fsck allocated itself during the expansion, we iterate the
directory's blocks a second time to ensure that these silent
allocations are marked in the found blocks bitmap.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
e2fsck/pass3.c | 39 ++++++++++++++++++++++++++++++++++++++-
1 file changed, 38 insertions(+), 1 deletion(-)
diff --git a/e2fsck/pass3.c b/e2fsck/pass3.c
index 6989f17..dc9d7c1 100644
--- a/e2fsck/pass3.c
+++ b/e2fsck/pass3.c
@@ -758,6 +758,27 @@ static int expand_dir_proc(ext2_filsys fs,
return BLOCK_CHANGED;
}
+/*
+ * Ensure that all blocks are marked in the block_found_map, since it's
+ * possible that the library allocated an extent node block or a block map
+ * block during the directory rebuilding; these new allocations are not
+ * captured in block_found_map. This is bad since we could later use
+ * block_found_map to allocate more blocks.
+ */
+static int find_new_blocks_proc(ext2_filsys fs,
+ blk64_t *blocknr,
+ e2_blkcnt_t blockcnt,
+ blk64_t ref_block EXT2FS_ATTR((unused)),
+ int ref_offset EXT2FS_ATTR((unused)),
+ void *priv_data)
+{
+ struct expand_dir_struct *es = (struct expand_dir_struct *) priv_data;
+ e2fsck_t ctx = es->ctx;
+
+ ext2fs_mark_block_bitmap2(ctx->block_found_map, *blocknr);
+ return 0;
+}
+
errcode_t e2fsck_expand_directory(e2fsck_t ctx, ext2_ino_t dir,
int num, int guaranteed_size)
{
@@ -765,7 +786,7 @@ errcode_t e2fsck_expand_directory(e2fsck_t ctx, ext2_ino_t dir,
errcode_t retval;
struct expand_dir_struct es;
struct ext2_inode inode;
- blk64_t sz;
+ blk64_t sz, before, after;
if (!(fs->flags & EXT2_FLAG_RW))
return EXT2_ET_RO_FILSYS;
@@ -788,11 +809,27 @@ errcode_t e2fsck_expand_directory(e2fsck_t ctx, ext2_ino_t dir,
es.ctx = ctx;
es.dir = dir;
+ before = ext2fs_free_blocks_count(fs->super);
retval = ext2fs_block_iterate3(fs, dir, BLOCK_FLAG_APPEND,
0, expand_dir_proc, &es);
if (es.err)
return es.err;
+ after = ext2fs_free_blocks_count(fs->super);
+
+ /*
+ * If the free block count has dropped by more than the blocks we
+ * allocated ourselves, then we must've allocated some extent/map
+ * blocks. Therefore, we must iterate this dir's blocks again to
+ * ensure that all newly allocated blocks are captured in
+ * block_found_map.
+ */
+ if ((before - after) > es.newblocks) {
+ retval = ext2fs_block_iterate3(fs, dir, BLOCK_FLAG_READ_ONLY,
+ 0, find_new_blocks_proc, &es);
+ if (es.err)
+ return es.err;
+ }
/*
* Update the size and block count fields in the inode.
next prev parent reply other threads:[~2013-12-11 1:22 UTC|newest]
Thread overview: 150+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-12-11 1:18 [PATCH v3 00/74] e2fsprogs patchbomb 12/2013 Darrick J. Wong
2013-12-11 1:18 ` [PATCH 01/74] libext2fs: don't overflow when punching indirect blocks with large blocks Darrick J. Wong
2013-12-12 17:06 ` Theodore Ts'o
2013-12-11 1:18 ` [PATCH 02/74] libext2fs: fix tests that set LARGE_FILE Darrick J. Wong
2013-12-12 17:09 ` Theodore Ts'o
2013-12-11 1:18 ` [PATCH 03/74] mke2fs: load configfile blocksize setting before 64bit checks Darrick J. Wong
2013-12-12 17:27 ` Theodore Ts'o
2013-12-12 22:28 ` Andreas Dilger
2013-12-12 23:13 ` Darrick J. Wong
2013-12-12 23:14 ` [PATCH] mke2fs: clean up kernel version tests Darrick J. Wong
2013-12-11 1:18 ` [PATCH 04/74] libext2fs: use ext2fs_punch() to truncate quota file Darrick J. Wong
2013-12-12 17:28 ` Theodore Ts'o
2013-12-12 17:36 ` Theodore Ts'o
2013-12-12 20:07 ` Darrick J. Wong
2013-12-12 20:56 ` Theodore Ts'o
2013-12-12 21:10 ` Darrick J. Wong
2013-12-11 1:18 ` [PATCH 05/74] debugfs: fix init_filesys help text Darrick J. Wong
2013-12-12 17:37 ` Theodore Ts'o
2013-12-11 1:18 ` [PATCH 06/74] tune2fs: forbid changing uuid on an uninit_bg filesystem Darrick J. Wong
2013-12-15 2:02 ` Theodore Ts'o
2013-12-11 1:19 ` [PATCH 07/74] libext2fs: tweak inline data error wording Darrick J. Wong
2013-12-13 4:33 ` Theodore Ts'o
2013-12-11 1:19 ` [PATCH 08/74] libext2fs: don't allow ridiculously large logical block numbers Darrick J. Wong
2013-12-12 17:41 ` Theodore Ts'o
2013-12-11 1:19 ` [PATCH 09/74] libext2fs: fix another minor grammatical error in the error catalog Darrick J. Wong
2013-12-12 17:42 ` Theodore Ts'o
2013-12-11 1:19 ` [PATCH 10/74] debugfs: fix various minor bogosity Darrick J. Wong
2013-12-12 17:44 ` Theodore Ts'o
2013-12-11 1:19 ` [PATCH 11/74] misc: use the checksum predicate function, not raw flag tests Darrick J. Wong
2013-12-13 4:34 ` Theodore Ts'o
2013-12-11 1:19 ` [PATCH 12/74] libext2fs: make symlinks safe for 64bit blocks and extents Darrick J. Wong
2013-12-12 17:48 ` Theodore Ts'o
2013-12-11 1:19 ` [PATCH 13/74] debugfs: handle 64bit block numbers Darrick J. Wong
2013-12-12 17:49 ` Theodore Ts'o
2013-12-17 17:01 ` Eric Sandeen
2013-12-11 1:19 ` [PATCH 14/74] libext2fs: fileio should use 64bit io routines Darrick J. Wong
2013-12-12 17:50 ` Theodore Ts'o
2013-12-11 1:20 ` [PATCH 15/74] resize2fs: rewrite extent/dir/ea block checksums when migrating Darrick J. Wong
2013-12-13 4:35 ` Theodore Ts'o
2013-12-11 1:20 ` [PATCH 16/74] debugfs: don't leak fd when calling dump_file Darrick J. Wong
2013-12-12 17:51 ` Theodore Ts'o
2013-12-11 1:20 ` [PATCH 17/74] debugfs: don't leak mmp_s memory Darrick J. Wong
2013-12-12 17:52 ` Theodore Ts'o
2013-12-12 22:33 ` Andreas Dilger
2013-12-12 22:44 ` Darrick J. Wong
2013-12-11 1:20 ` [PATCH 18/74] e2fsck: fix memory leaks Darrick J. Wong
2013-12-12 17:58 ` Theodore Ts'o
2013-12-17 16:12 ` Eric Sandeen
2013-12-11 1:20 ` [PATCH 19/74] misc: don't leak file descriptors Darrick J. Wong
2013-12-12 18:06 ` Theodore Ts'o
2013-12-11 1:20 ` [PATCH 20/74] mke2fs: don't leak memory Darrick J. Wong
2013-12-12 18:07 ` Theodore Ts'o
2013-12-11 1:20 ` [PATCH 21/74] e4defrag: don't crash if umounts the filesystem races with us Darrick J. Wong
2013-12-12 18:08 ` Theodore Ts'o
2013-12-11 1:20 ` [PATCH 22/74] e4defrag: defensively check results of sysconf(_SC_PAGESIZE) Darrick J. Wong
2013-12-12 18:09 ` Theodore Ts'o
2013-12-11 1:20 ` [PATCH 23/74] e2image: check return value from check_if_mounted Darrick J. Wong
2013-12-12 18:09 ` Theodore Ts'o
2013-12-11 1:20 ` [PATCH 24/74] dumpe2fs: check return values Darrick J. Wong
2013-12-12 18:10 ` Theodore Ts'o
2013-12-11 1:21 ` [PATCH 25/74] libss: fix fd error handling Darrick J. Wong
2013-12-12 18:11 ` Theodore Ts'o
2013-12-11 1:21 ` [PATCH 26/74] libss: fix memory handling errors Darrick J. Wong
2013-12-12 18:13 ` Theodore Ts'o
2013-12-17 17:04 ` Eric Sandeen
2013-12-18 22:23 ` Darrick J. Wong
2013-12-11 1:21 ` [PATCH 27/74] libquota: fix memory leak Darrick J. Wong
2013-12-12 18:14 ` Theodore Ts'o
2013-12-11 1:21 ` [PATCH 28/74] libext2fs: check return values Darrick J. Wong
2013-12-12 18:15 ` Theodore Ts'o
2013-12-17 16:57 ` Eric Sandeen
2013-12-17 16:59 ` Eric Sandeen
2013-12-11 1:21 ` [PATCH 29/74] libext2fs: fix memory leaks Darrick J. Wong
2013-12-12 18:17 ` Theodore Ts'o
2013-12-11 1:21 ` [PATCH 30/74] libext2fs: fix a broken close() test Darrick J. Wong
2013-12-12 18:18 ` Theodore Ts'o
2013-12-11 1:21 ` [PATCH 31/74] libext2fs: fail fileio write if we can't allocate a block Darrick J. Wong
2013-12-12 18:23 ` Theodore Ts'o
2013-12-11 1:21 ` [PATCH 32/74] libext2fs: fix punching extents when there are no left extents Darrick J. Wong
2013-12-12 18:25 ` Theodore Ts'o
2013-12-11 1:22 ` [PATCH 33/74] libext2fs: don't error out when punching a totally sparse file Darrick J. Wong
2013-12-12 18:26 ` Theodore Ts'o
2013-12-11 1:22 ` Darrick J. Wong [this message]
2013-12-12 18:27 ` [PATCH 34/74] e2fsck: in rehash, mark newly allocated extent blocks as found Theodore Ts'o
2013-12-11 1:22 ` [PATCH 35/74] libext2fs: zero block contents past EOF when setting size Darrick J. Wong
2013-12-12 18:40 ` Theodore Ts'o
2013-12-11 1:22 ` [PATCH 36/74] libext2fs: detect Darrick J. Wong
2013-12-13 4:39 ` Theodore Ts'o
2014-01-15 21:00 ` Darrick J. Wong
2013-12-11 1:22 ` [PATCH 37/74] libext2fs: don't always read backup group descriptors on a 1k-block meta_bg fs Darrick J. Wong
2014-01-11 18:59 ` Theodore Ts'o
2013-12-11 1:22 ` [PATCH 38/74] libext2fs: mark group data blocks when loading block bitmap Darrick J. Wong
2014-01-11 19:08 ` Theodore Ts'o
2013-12-11 1:22 ` [PATCH 39/74] e2fsck: remove uninit block bitmap calculation Darrick J. Wong
2014-01-11 19:08 ` Theodore Ts'o
2013-12-11 1:22 ` [PATCH 40/74] libext2fs: no need to clear BLOCK_UNINIT during ext2fs_reserve_super_and_bgd Darrick J. Wong
2014-01-10 8:17 ` Akira Fujita
2014-01-11 19:18 ` Theodore Ts'o
2013-12-11 1:22 ` [PATCH 41/74] tests: adjust test output to reflect block_uninit calculated block bitmaps Darrick J. Wong
2014-01-11 19:19 ` Theodore Ts'o
2013-12-11 1:22 ` [PATCH 42/74] libext2fs: only punch complete clusters Darrick J. Wong
2013-12-16 4:52 ` Theodore Ts'o
2013-12-11 1:23 ` [PATCH 43/74] libext2fs: don't update the summary counts when doing implied cluster allocation Darrick J. Wong
2013-12-16 4:53 ` Theodore Ts'o
2013-12-11 1:23 ` [PATCH 44/74] e2fsck: only release clusters when shortening a directory during a rehash Darrick J. Wong
2013-12-16 4:55 ` Theodore Ts'o
2013-12-11 1:23 ` [PATCH 45/74] e2fsck: print cluster ranges when encountering bitmap errors Darrick J. Wong
2013-12-16 4:55 ` Theodore Ts'o
2013-12-11 1:23 ` [PATCH 46/74] e2fsck: try implied cluster allocation when expanding a dir Darrick J. Wong
2013-12-16 4:56 ` Theodore Ts'o
2013-12-11 1:23 ` [PATCH 47/74] resize2fs: during shrink, don't free in-use bg data clusters Darrick J. Wong
2013-12-16 5:01 ` Theodore Ts'o
2013-12-16 20:10 ` Darrick J. Wong
2014-02-24 1:39 ` Theodore Ts'o
2013-12-11 1:23 ` [PATCH 48/74] resize2fs: don't free in-use clusters when moving blocks Darrick J. Wong
2014-02-24 1:56 ` Theodore Ts'o
2013-12-11 1:23 ` [PATCH 49/74] mke2fs: set block_validity as a default mount option Darrick J. Wong
2013-12-11 1:23 ` [PATCH 50/74] libext2fs: support allocating uninit blocks in bmap2() Darrick J. Wong
2014-01-11 22:57 ` Theodore Ts'o
2014-01-15 21:11 ` Darrick J. Wong
2014-01-15 22:19 ` Theodore Ts'o
2014-01-15 22:23 ` Theodore Ts'o
2013-12-11 1:23 ` [PATCH 51/74] libext2fs: file IO routines should handle uninit blocks Darrick J. Wong
2013-12-11 1:24 ` [PATCH 52/74] resize2fs: convert fs to and from 64bit mode Darrick J. Wong
2013-12-11 1:24 ` [PATCH 53/74] resize2fs: when toggling 64bit, don't free in-use bg data clusters Darrick J. Wong
2013-12-11 1:24 ` [PATCH 54/74] resize2fs: adjust reserved_gdt_blocks when changing group descriptor size Darrick J. Wong
2013-12-11 1:24 ` [PATCH 55/74] libext2fs: support modifying arbitrary extended attributes Darrick J. Wong
2014-02-24 4:09 ` Theodore Ts'o
2013-12-11 1:24 ` [PATCH 56/74] libext2fs: various tweaks to the xattr editor APIs Darrick J. Wong
2014-02-24 4:10 ` Theodore Ts'o
2013-12-11 1:24 ` [PATCH 57/74] libext2fs: extend xattr api to query number of attrs Darrick J. Wong
2014-02-24 4:10 ` Theodore Ts'o
2013-12-11 1:24 ` [PATCH 58/74] libext2fs: free key/value pairs before reading Darrick J. Wong
2014-02-24 4:10 ` Theodore Ts'o
2013-12-11 1:24 ` [PATCH 59/74] debugfs: dump all extended attributes Darrick J. Wong
2014-02-24 4:10 ` Theodore Ts'o
2013-12-11 1:24 ` [PATCH 60/74] libext2fs: ensure that inline data is always written to ibody Darrick J. Wong
2013-12-11 1:25 ` [PATCH 61/74] libext2fs: fix ext2fs_open2() truncation of the superblock parameter Darrick J. Wong
2013-12-11 1:25 ` [PATCH 62/74] misc: add fuse2fs, a FUSE server for e2fsprogs Darrick J. Wong
2013-12-11 1:25 ` [PATCH 63/74] fuse2fs: translate ACL structures Darrick J. Wong
2013-12-11 1:25 ` [PATCH 64/74] Subject: [PATCH] fuse2fs: support allocating uninit blocks in fallocate Darrick J. Wong
2013-12-11 1:25 ` [PATCH 65/74] fuse2fs: handle 64-bit dates correctly Darrick J. Wong
2013-12-11 1:25 ` [PATCH 67/74] tests: check correct handling of reading and writing uninit extents Darrick J. Wong
2013-12-11 1:25 ` [PATCH 68/74] tests: Add block_validity speed test Darrick J. Wong
2013-12-11 1:26 ` [PATCH 69/74] Subject: [PATCH] tests: test what happens if we run out of space Darrick J. Wong
2013-12-11 1:26 ` [PATCH 70/74] tests: add stale data after truncate test Darrick J. Wong
2013-12-11 1:26 ` [PATCH 71/74] tests: check mapping of really high logical block offsets Darrick J. Wong
2013-12-11 1:26 ` [PATCH 72/74] Subject: [PATCH] tests: enable using fuse2fs with metadata checksum test Darrick J. Wong
2013-12-11 1:26 ` [PATCH 73/74] tests: add large symlink test Darrick J. Wong
2013-12-11 1:26 ` [PATCH 74/74] tests: test date handling Darrick J. Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20131211012207.30655.92282.stgit@birch.djwong.org \
--to=darrick.wong@oracle.com \
--cc=linux-ext4@vger.kernel.org \
--cc=tytso@mit.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).