From: "Darrick J. Wong" <darrick.wong@oracle.com>
To: tytso@mit.edu
Cc: linux-ext4@vger.kernel.org
Subject: Re: [PATCH 18/39] e2fsck: opportunistically recalculate unused inode count and inode_uninit after pass 5
Date: Mon, 27 Oct 2014 13:18:38 -0700 [thread overview]
Message-ID: <20141027201838.GB10057@birch.djwong.org> (raw)
In-Reply-To: <20141025205819.532.37434.stgit@birch.djwong.org>
On Sat, Oct 25, 2014 at 01:58:19PM -0700, Darrick J. Wong wrote:
> At the end of pass5, update the unused inode count to cover any newly
> unused inodes and set inode_uninit if none of the inodes in the group
> are used. This will save us time on future fsck runs since we don't
> need to scan those inodes any more.
>
> Since it's not an error for the unused inode count to be smaller than
> necessary, don't bother if the fs is mounted and we're not fixing
> anything else.
>
> On an aged filesystem that has experienced at least one massive inode
> die-off, this has been shown to reduce future e2fsck times by around
> 10%. The effect is diluted on less heavily used filesystems.
I didn't notice that ext2fs_set_gdt_csum() quietly takes care of this if the
filesystem's open for writes. Therefore, I withdraw this patch and apologize
for the noise.
However, the BLOCK_UNINIT piece is still needed, so I'll issue a new patch #19.
--D
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
> e2fsck/pass5.c | 38 +++++++++++++++++++++++++++++++
> e2fsck/problem.c | 5 ++++
> e2fsck/problem.h | 3 ++
> tests/f_bad_bbitmap/expect.1 | 3 ++
> tests/f_bad_bmap_csum/expect.1 | 3 ++
> tests/f_bad_gdt_csum/expect.1 | 7 +++++-
> tests/f_bad_ibitmap/expect.1 | 3 ++
> tests/f_bad_inode_csum/expect.1 | 3 ++
> tests/f_idata_and_extents/expect.1 | 3 ++
> tests/f_illitable_flexbg/expect.1 | 12 ++++++++++
> tests/f_inlinedata_repair/expect.1 | 3 ++
> tests/f_invalid_extent_symlink/expect.1 | 3 ++
> tests/f_jnl_64bit/expect.1 | 6 +++++
> tests/f_super_bad_csum/expect.1 | 8 +++++--
> tests/f_unused_itable/expect.1 | 6 +++++
> 15 files changed, 103 insertions(+), 3 deletions(-)
>
>
> diff --git a/e2fsck/pass5.c b/e2fsck/pass5.c
> index 64fb7fe..1a225fb 100644
> --- a/e2fsck/pass5.c
> +++ b/e2fsck/pass5.c
> @@ -821,6 +821,44 @@ do_counts:
> } else
> ext2fs_unmark_valid(fs);
> }
> +
> + /*
> + * Opportunistically update the unused inodes count
> + * and set inode_uninit so we can skip scanning unused
> + * inodes during future fsck runs. However, it's not
> + * an error if the unused inode count is smaller
> + * than necessary, so don't bother the user if FS is
> + * mounted and we haven't fixed anything else, to
> + * minimize unnecessary reboots.
> + */
> + if (((ctx->options & E2F_OPT_NO) ||
> + !(ctx->mount_flags & EXT2_MF_MOUNTED) ||
> + ext2fs_test_changed(fs)) &&
> + ext2fs_has_group_desc_csum(fs) &&
> + !ext2fs_bg_flags_test(fs, i, EXT2_BG_INODE_UNINIT)) {
> + ext2_ino_t start, end, j;
> +
> + start = (fs->super->s_inodes_per_group * i) + 1;
> + end = (fs->super->s_inodes_per_group * (i + 1));
> + pctx.group = i;
> + pctx.ino = ext2fs_bg_itable_unused(fs, i);
> + end -= pctx.ino;
> + for (j = end; j >= start; j--)
> + if (ext2fs_test_inode_bitmap2(ctx->inode_used_map, j))
> + break;
> +
> + pctx.ino2 = fs->super->s_inodes_per_group -
> + (j - start + 1);
> + if (pctx.ino != pctx.ino2 &&
> + fix_problem(ctx, PR_5_UNUSED_INODES_COUNT_GROUP,
> + &pctx)) {
> + if (pctx.ino2 == fs->super->s_inodes_per_group)
> + ext2fs_bg_flags_set(fs, i,
> + EXT2_BG_INODE_UNINIT);
> + ext2fs_bg_itable_unused_set(fs, i, pctx.ino2);
> + ext2fs_mark_super_dirty(fs);
> + }
> + }
> }
> if (free_inodes != fs->super->s_free_inodes_count) {
> pctx.group = -1;
> diff --git a/e2fsck/problem.c b/e2fsck/problem.c
> index a4da64b..9c2b2e6 100644
> --- a/e2fsck/problem.c
> +++ b/e2fsck/problem.c
> @@ -1899,6 +1899,11 @@ static struct e2fsck_problem problem_table[] = {
> N_("@g %g @b @B does not match checksum.\n"),
> PROMPT_FIX, PR_LATCH_BBITMAP | PR_PREEN_OK },
>
> + /* Unused inode count for group wrong */
> + { PR_5_UNUSED_INODES_COUNT_GROUP,
> + N_("Unused @i count wrong for @g #%g (%i, counted=%j).\n"),
> + PROMPT_FIX, PR_PREEN_OK | PR_NO_OK },
> +
> /* Post-Pass 5 errors */
>
> /* Recreate journal if E2F_FLAG_JOURNAL_INODE flag is set */
> diff --git a/e2fsck/problem.h b/e2fsck/problem.h
> index 3c28166..6a71fc8 100644
> --- a/e2fsck/problem.h
> +++ b/e2fsck/problem.h
> @@ -1139,6 +1139,9 @@ struct problem_context {
> /* Block bitmap checksum does not match */
> #define PR_5_BLOCK_BITMAP_CSUM_INVALID 0x05001B
>
> +/* Unused inode count for group wrong */
> +#define PR_5_UNUSED_INODES_COUNT_GROUP 0x05001C
> +
> /*
> * Post-Pass 5 errors
> */
> diff --git a/tests/f_bad_bbitmap/expect.1 b/tests/f_bad_bbitmap/expect.1
> index 71ad1bb..af3f632 100644
> --- a/tests/f_bad_bbitmap/expect.1
> +++ b/tests/f_bad_bbitmap/expect.1
> @@ -9,6 +9,9 @@ Pass 5: Checking group summary information
> Block bitmap differences: -(8--10) -(12--17) -(19--31)
> Fix? yes
>
> +Unused inode count wrong for group #0 (0, counted=117).
> +Fix? yes
> +
>
> test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
> test_filesys: 11/128 files (0.0% non-contiguous), 18/512 blocks
> diff --git a/tests/f_bad_bmap_csum/expect.1 b/tests/f_bad_bmap_csum/expect.1
> index ca8f77f..7616751 100644
> --- a/tests/f_bad_bmap_csum/expect.1
> +++ b/tests/f_bad_bmap_csum/expect.1
> @@ -6,6 +6,9 @@ Pass 2: Checking directory structure
> Pass 3: Checking directory connectivity
> Pass 4: Checking reference counts
> Pass 5: Checking group summary information
> +Unused inode count wrong for group #0 (0, counted=117).
> +Fix? yes
> +
> Inode bitmap differences: Group 0 inode bitmap does not match checksum.
> FIXED.
> Block bitmap differences: Group 0 block bitmap does not match checksum.
> diff --git a/tests/f_bad_gdt_csum/expect.1 b/tests/f_bad_gdt_csum/expect.1
> index e14c897..0f496e0 100644
> --- a/tests/f_bad_gdt_csum/expect.1
> +++ b/tests/f_bad_gdt_csum/expect.1
> @@ -6,5 +6,10 @@ Pass 2: Checking directory structure
> Pass 3: Checking directory connectivity
> Pass 4: Checking reference counts
> Pass 5: Checking group summary information
> +Unused inode count wrong for group #0 (0, counted=117).
> +Fix? yes
> +
> +
> +test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
> test_filesys: 11/128 files (0.0% non-contiguous), 18/512 blocks
> -Exit status is 0
> +Exit status is 1
> diff --git a/tests/f_bad_ibitmap/expect.1 b/tests/f_bad_ibitmap/expect.1
> index ea17523..81689e0 100644
> --- a/tests/f_bad_ibitmap/expect.1
> +++ b/tests/f_bad_ibitmap/expect.1
> @@ -9,6 +9,9 @@ Pass 5: Checking group summary information
> Inode bitmap differences: -(12--32)
> Fix? yes
>
> +Unused inode count wrong for group #0 (0, counted=117).
> +Fix? yes
> +
>
> test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
> test_filesys: 11/128 files (0.0% non-contiguous), 18/512 blocks
> diff --git a/tests/f_bad_inode_csum/expect.1 b/tests/f_bad_inode_csum/expect.1
> index b3c628d..b28a349 100644
> --- a/tests/f_bad_inode_csum/expect.1
> +++ b/tests/f_bad_inode_csum/expect.1
> @@ -117,6 +117,9 @@ Fix? yes
> Free inodes count wrong for group #0 (0, counted=32).
> Fix? yes
>
> +Unused inode count wrong for group #0 (0, counted=32).
> +Fix? yes
> +
> Free inodes count wrong (0, counted=32).
> Fix? yes
>
> diff --git a/tests/f_idata_and_extents/expect.1 b/tests/f_idata_and_extents/expect.1
> index 7f7fbf3..aba063a 100644
> --- a/tests/f_idata_and_extents/expect.1
> +++ b/tests/f_idata_and_extents/expect.1
> @@ -26,6 +26,9 @@ Fix? yes
> Free inodes count wrong for group #0 (105, counted=106).
> Fix? yes
>
> +Unused inode count wrong for group #0 (103, counted=105).
> +Fix? yes
> +
> Free inodes count wrong (105, counted=106).
> Fix? yes
>
> diff --git a/tests/f_illitable_flexbg/expect.1 b/tests/f_illitable_flexbg/expect.1
> index fa42a0f..04ce65b 100644
> --- a/tests/f_illitable_flexbg/expect.1
> +++ b/tests/f_illitable_flexbg/expect.1
> @@ -18,6 +18,18 @@ Pass 5: Checking group summary information
> Inode bitmap differences: -(65--128)
> Fix? yes
>
> +Unused inode count wrong for group #0 (0, counted=52).
> +Fix? yes
> +
> +Unused inode count wrong for group #1 (0, counted=64).
> +Fix? yes
> +
> +Unused inode count wrong for group #2 (0, counted=64).
> +Fix? yes
> +
> +Unused inode count wrong for group #3 (0, counted=64).
> +Fix? yes
> +
>
> test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
> test_filesys: 12/256 files (0.0% non-contiguous), 31163/32768 blocks
> diff --git a/tests/f_inlinedata_repair/expect.1 b/tests/f_inlinedata_repair/expect.1
> index faba192..f7eebd3 100644
> --- a/tests/f_inlinedata_repair/expect.1
> +++ b/tests/f_inlinedata_repair/expect.1
> @@ -69,6 +69,9 @@ Pass 5: Checking group summary information
> Directories count wrong for group #0 (7, counted=8).
> Fix? yes
>
> +Unused inode count wrong for group #0 (90, counted=91).
> +Fix? yes
> +
>
> test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
> test_filesys: 28/128 files (0.0% non-contiguous), 18/512 blocks
> diff --git a/tests/f_invalid_extent_symlink/expect.1 b/tests/f_invalid_extent_symlink/expect.1
> index 7bda0b7..cbb2ac5 100644
> --- a/tests/f_invalid_extent_symlink/expect.1
> +++ b/tests/f_invalid_extent_symlink/expect.1
> @@ -6,6 +6,9 @@ Clear? yes
> Pass 3: Checking directory connectivity
> Pass 4: Checking reference counts
> Pass 5: Checking group summary information
> +Unused inode count wrong for group #0 (4, counted=5).
> +Fix? yes
> +
>
> test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
> test_filesys: 11/16 files (9.1% non-contiguous), 21/100 blocks
> diff --git a/tests/f_jnl_64bit/expect.1 b/tests/f_jnl_64bit/expect.1
> index e360e2f..915076b 100644
> --- a/tests/f_jnl_64bit/expect.1
> +++ b/tests/f_jnl_64bit/expect.1
> @@ -7,6 +7,12 @@ Pass 5: Checking group summary information
> Free blocks count wrong (14059, counted=12712).
> Fix? yes
>
> +Unused inode count wrong for group #0 (0, counted=2876).
> +Fix? yes
> +
> +Unused inode count wrong for group #1 (1395, counted=3958).
> +Fix? yes
> +
> Free inodes count wrong (8181, counted=6834).
> Fix? yes
>
> diff --git a/tests/f_super_bad_csum/expect.1 b/tests/f_super_bad_csum/expect.1
> index 25ced5c..a606fab 100644
> --- a/tests/f_super_bad_csum/expect.1
> +++ b/tests/f_super_bad_csum/expect.1
> @@ -5,8 +5,12 @@ Pass 2: Checking directory structure
> Pass 3: Checking directory connectivity
> Pass 4: Checking reference counts
> Pass 5: Checking group summary information
> -Inode bitmap differences: Group 1 inode bitmap does not match checksum.
> -FIXED.
> +Unused inode count wrong for group #0 (0, counted=501).
> +Fix? yes
> +
> +Unused inode count wrong for group #1 (0, counted=512).
> +Fix? yes
> +
>
> test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
> test_filesys: 11/1024 files (0.0% non-contiguous), 1557/16384 blocks
> diff --git a/tests/f_unused_itable/expect.1 b/tests/f_unused_itable/expect.1
> index a4da987..fbc6538 100644
> --- a/tests/f_unused_itable/expect.1
> +++ b/tests/f_unused_itable/expect.1
> @@ -19,9 +19,15 @@ Pass 5: Checking group summary information
> Free inodes count wrong for group #0 (53, counted=51).
> Fix? yes
>
> +Unused inode count wrong for group #0 (0, counted=51).
> +Fix? yes
> +
> Free inodes count wrong for group #1 (64, counted=58).
> Fix? yes
>
> +Unused inode count wrong for group #1 (0, counted=58).
> +Fix? yes
> +
>
> test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
> test_filesys: 19/128 files (0.0% non-contiguous), 165/1000 blocks
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2014-10-27 20:18 UTC|newest]
Thread overview: 55+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-10-25 20:56 [PATCH 00/39] e2fsprogs October 2014 patchbomb, part 6.1 Darrick J. Wong
2014-10-25 20:56 ` [PATCH 01/39] misc: fix compiler warnings Darrick J. Wong
2014-11-04 16:36 ` Theodore Ts'o
2014-10-25 20:56 ` [PATCH 02/39] e2fuzz: exercise fuzzed blocks more aggressively Darrick J. Wong
2014-11-04 16:36 ` Theodore Ts'o
2014-10-25 20:56 ` [PATCH 03/39] libext2fs: directory iteration mustn't walk off the buffer end Darrick J. Wong
2014-11-04 16:37 ` Theodore Ts'o
2014-10-25 20:56 ` [PATCH 04/39] libext2fs: zero the EA block buffer before filling it Darrick J. Wong
2014-11-04 16:47 ` Theodore Ts'o
2014-10-25 20:56 ` [PATCH 05/39] libext2fs: don't memcpy identical pointers when writing a cache block Darrick J. Wong
2014-11-04 16:48 ` Theodore Ts'o
2014-10-25 20:57 ` [PATCH 06/39] misc: fix broken libmagic interaction with plausibility check Darrick J. Wong
2014-11-04 16:50 ` Theodore Ts'o
2014-10-25 20:57 ` [PATCH 07/39] tune2fs: speed up rewriting extent tree when enabling metadata_csum Darrick J. Wong
2014-11-04 16:52 ` Theodore Ts'o
2014-10-25 20:57 ` [PATCH 08/39] tune2fs: don't change metadata_csum on a mounted fs Darrick J. Wong
2014-11-04 16:52 ` Theodore Ts'o
2014-10-25 20:57 ` [PATCH 09/39] resize2fs: don't exit if shrinking sparse_super2 fs to one bg Darrick J. Wong
2014-10-25 20:57 ` [PATCH 10/39] resize2fs: quickly rewrite extent blocks when moving an inode w/ metadata_csum Darrick J. Wong
2014-10-25 20:57 ` [PATCH 11/39] resize2fs: use old_fs to detect per-bg metadata blocks to free Darrick J. Wong
2014-10-25 20:57 ` [PATCH 12/39] resize2fs: don't interpret bitmap shift while crossing flexbg as raid stride Darrick J. Wong
2014-10-25 20:57 ` [PATCH 13/39] resize2fs: set block_uninit in former last bg when expanding fs Darrick J. Wong
2014-10-27 23:29 ` [PATCH v2 13/39] resize2fs: set bg flags and unused inode count when resizing Darrick J. Wong
2014-10-25 20:57 ` [PATCH 14/39] mke2fs: don't zero inode table blocks that are already zeroed Darrick J. Wong
2014-10-25 20:57 ` [PATCH 15/39] dumpe2fs: 80 column outputs, please Darrick J. Wong
2014-10-25 20:58 ` [PATCH 16/39] dumpe2fs: output cleanup Darrick J. Wong
2014-10-25 20:58 ` [PATCH 17/39] e2fsck: fix dangling pointer when dir_info array is resized Darrick J. Wong
2014-11-05 16:12 ` Theodore Ts'o
2014-10-25 20:58 ` [PATCH 18/39] e2fsck: opportunistically recalculate unused inode count and inode_uninit after pass 5 Darrick J. Wong
2014-10-27 20:18 ` Darrick J. Wong [this message]
2014-10-25 20:58 ` [PATCH 19/39] e2fsck: opportunistically set block_uninit " Darrick J. Wong
2014-10-27 23:27 ` [PATCH 19/39] libext2fs: set BLOCK_UNINIT for non-last blockgroups if all blocks are free Darrick J. Wong
2014-10-25 20:58 ` [PATCH 20/39] libext2fs/e2fsck: provide routines to read-ahead metadata Darrick J. Wong
2014-10-25 20:58 ` [PATCH 21/39] e2fsck: read-ahead metadata during passes 1, 2, and 4 Darrick J. Wong
2014-10-25 20:58 ` [PATCH 22/39] libext2fs: ext2fs_new_block2() should call alloc_block hook Darrick J. Wong
2014-10-25 20:58 ` [PATCH 23/39] libext2fs: support BLKZEROOUT/FALLOC_FL_ZERO_RANGE in ext2fs_zero_blocks Darrick J. Wong
2014-10-25 20:58 ` [PATCH 24/39] libext2fs/e2fsck: refactor everyone who writes zero blocks to disk Darrick J. Wong
2014-10-25 20:59 ` [PATCH 25/39] libext2fs: support allocating uninit blocks in bmap2() Darrick J. Wong
2014-10-25 20:59 ` [PATCH 26/39] libext2fs: file IO routines should handle uninit blocks Darrick J. Wong
2014-10-25 20:59 ` [PATCH 27/39] resize2fs: convert fs to and from 64bit mode Darrick J. Wong
2014-10-25 20:59 ` [PATCH 28/39] tests: test resize2fs 32->64 and 64->32bit conversion code Darrick J. Wong
2014-10-25 20:59 ` [PATCH 29/39] libext2fs: find inode goal when allocating blocks Darrick J. Wong
2014-10-25 20:59 ` [PATCH 30/39] libext2fs: find/alloc a range of empty blocks Darrick J. Wong
2014-10-25 20:59 ` [PATCH 31/39] libext2fs: add new hooks to support large allocations Darrick J. Wong
2014-10-25 20:59 ` [PATCH 32/39] libext2fs: implement fallocate Darrick J. Wong
2014-10-25 20:59 ` [PATCH 33/39] libext2fs: use fallocate for creating journals and hugefiles Darrick J. Wong
2014-10-25 21:00 ` [PATCH 34/39] debugfs: implement fallocate Darrick J. Wong
2014-10-25 21:00 ` [PATCH 35/39] tests: test debugfs punch command Darrick J. Wong
2014-10-25 21:00 ` [PATCH 37/39] fuse2fs: translate ACL structures Darrick J. Wong
2014-10-25 21:00 ` [PATCH 38/39] fuse2fs: handle 64-bit dates correctly Darrick J. Wong
2014-10-25 21:00 ` [PATCH 39/39] fuse2fs: implement fallocate Darrick J. Wong
2014-10-27 23:31 ` [PATCH 40/39] e2fsck: fix reporting of unknown htree block inode number Darrick J. Wong
2014-11-05 16:11 ` Theodore Ts'o
2014-10-27 23:32 ` [PATCH 41/39] mke2fs: warn if enabling metadata_csum on a pre-3.18 kernel Darrick J. Wong
2014-11-05 16:20 ` Theodore Ts'o
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20141027201838.GB10057@birch.djwong.org \
--to=darrick.wong@oracle.com \
--cc=linux-ext4@vger.kernel.org \
--cc=tytso@mit.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).