linux-ext4.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Darrick J. Wong" <darrick.wong@oracle.com>
To: tytso@mit.edu
Cc: linux-ext4@vger.kernel.org
Subject: [PATCH 57/54] e2fsck: convert block-mapped files to extents on bigalloc fs
Date: Thu, 19 Mar 2015 14:47:21 -0700	[thread overview]
Message-ID: <20150319214721.GL11031@birch.djwong.org> (raw)
In-Reply-To: <20150127073533.13308.44994.stgit@birch.djwong.org>

As of v4.0, the Linux kernel won't add blocks to a block-mapped file
on a bigalloc filesystem.  Therefore, convert any such files or
directories we find, to prevent fs errors later on.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 e2fsck/pass1.c            |   17 +++++++++++++++++
 e2fsck/problem.c          |    5 +++++
 e2fsck/problem.h          |    3 +++
 tests/f_badcluster/expect |   26 +++++++++++++-------------
 4 files changed, 38 insertions(+), 13 deletions(-)

diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 19e1f33..308eb79 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -3201,6 +3201,23 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx,
 		pctx->num = 0;
 	}
 
+	/*
+	 * The kernel gets mad if we ask it to allocate bigalloc clusters to
+	 * a block mapped file, so rebuild it as an extent file.  We can skip
+	 * symlinks because they're never rewritten.
+	 */
+	if (EXT2_HAS_RO_COMPAT_FEATURE(fs->super,
+			EXT4_FEATURE_RO_COMPAT_BIGALLOC) &&
+	    (LINUX_S_ISREG(inode->i_mode) || LINUX_S_ISDIR(inode->i_mode)) &&
+	    ext2fs_inode_data_blocks2(fs, inode) > 0 &&
+	    (ino == EXT2_ROOT_INO || ino >= EXT2_FIRST_INO(fs->super)) &&
+	    !(inode->i_flags & (EXT4_EXTENTS_FL | EXT4_INLINE_DATA_FL)) &&
+	    fix_problem(ctx, PR_1_NO_BIGALLOC_BLOCKMAP_FILES, pctx)) {
+		pctx->errcode = e2fsck_rebuild_extents_later(ctx, ino);
+		if (pctx->errcode)
+			goto out;
+	}
+
 	if (ctx->dirs_to_hash && pb.is_dir &&
 	    !(ctx->lost_and_found && ctx->lost_and_found == ino) &&
 	    !(inode->i_flags & EXT2_INDEX_FL) &&
diff --git a/e2fsck/problem.c b/e2fsck/problem.c
index 7af0b76..df919da 100644
--- a/e2fsck/problem.c
+++ b/e2fsck/problem.c
@@ -1111,6 +1111,11 @@ static struct e2fsck_problem problem_table[] = {
 	  N_("@i %i @x tree could be more shallow (%b; could be <= %c)\n"),
 	  PROMPT_FIX, PR_NO_OK | PR_PREEN_NO | PR_PREEN_OK },
 
+	/* Inode extent tree could be more shallow */
+	{ PR_1_NO_BIGALLOC_BLOCKMAP_FILES,
+	  N_("@i %i on bigalloc @f cannot be @b mapped.  "),
+	  PROMPT_FIX, 0 },
+
 	/* Pass 1b errors */
 
 	/* Pass 1B: Rescan for duplicate/bad blocks */
diff --git a/e2fsck/problem.h b/e2fsck/problem.h
index ace8d2f..dc0c18a 100644
--- a/e2fsck/problem.h
+++ b/e2fsck/problem.h
@@ -648,6 +648,9 @@ struct problem_context {
 /* extent tree max depth too big */
 #define PR_1_EXTENT_BAD_MAX_DEPTH		0x01007F
 
+/* bigalloc fs cannot have blockmap files */
+#define PR_1_NO_BIGALLOC_BLOCKMAP_FILES		0x010080
+
 /*
  * Pass 1b errors
  */
diff --git a/tests/f_badcluster/expect b/tests/f_badcluster/expect
index b8ce19d..65a1641 100644
--- a/tests/f_badcluster/expect
+++ b/tests/f_badcluster/expect
@@ -19,6 +19,8 @@ Inode 18 logical block 3 (physical block 1201) violates cluster allocation rules
 Will fix in pass 1B.
 Inode 18, i_blocks is 32, should be 64.  Fix? yes
 
+Inode 15 on bigalloc filesystem cannot be block mapped.  Fix? yes
+
 
 Running additional passes to resolve blocks claimed by more than one inode...
 Pass 1B: Rescanning for multiply-claimed blocks
@@ -65,19 +67,20 @@ File /g (inode #18, mod time Tue Jun 17 08:00:50 2014)
   has 1 multiply-claimed block(s), shared with 0 file(s):
 Clone multiply-claimed blocks? yes
 
+Pass 1E: Optimizing extent trees
 Pass 2: Checking directory structure
 Pass 3: Checking directory connectivity
 Pass 4: Checking reference counts
 Pass 5: Checking group summary information
-Free blocks count wrong for group #0 (50, counted=47).
+Free blocks count wrong for group #0 (51, counted=48).
 Fix? yes
 
-Free blocks count wrong (800, counted=752).
+Free blocks count wrong (816, counted=768).
 Fix? yes
 
 
 test_fs: ***** FILE SYSTEM WAS MODIFIED *****
-test_fs: 18/128 files (22.2% non-contiguous), 1296/2048 blocks
+test_fs: 18/128 files (22.2% non-contiguous), 1280/2048 blocks
 Pass 1: Checking inodes, blocks, and sizes
 Inode 12, i_blocks is 64, should be 32.  Fix? yes
 
@@ -94,21 +97,21 @@ Pass 5: Checking group summary information
 Block bitmap differences:  -(1168--1200)
 Fix? yes
 
-Free blocks count wrong for group #0 (47, counted=50).
+Free blocks count wrong for group #0 (48, counted=51).
 Fix? yes
 
-Free blocks count wrong (752, counted=800).
+Free blocks count wrong (768, counted=816).
 Fix? yes
 
 
 test_fs: ***** FILE SYSTEM WAS MODIFIED *****
-test_fs: 18/128 files (5.6% non-contiguous), 1248/2048 blocks
+test_fs: 18/128 files (5.6% non-contiguous), 1232/2048 blocks
 Pass 1: Checking inodes, blocks, and sizes
 Pass 2: Checking directory structure
 Pass 3: Checking directory connectivity
 Pass 4: Checking reference counts
 Pass 5: Checking group summary information
-test_fs: 18/128 files (5.6% non-contiguous), 1248/2048 blocks
+test_fs: 18/128 files (5.6% non-contiguous), 1232/2048 blocks
 debugfs: stat /a
 Inode: 12   Type: regular    Mode:  0644   Flags: 0x80000
 Generation: 1117152157    Version: 0x00000001
@@ -146,19 +149,16 @@ mtime: 0x539ff5b2 -- Tue Jun 17 08:00:50 2014
 EXTENTS:
 (0-1):1216-1217, (2):1218
 debugfs: stat /d
-Inode: 15   Type: regular    Mode:  0644   Flags: 0x0
+Inode: 15   Type: regular    Mode:  0644   Flags: 0x80000
 Generation: 1117152160    Version: 0x00000001
 User:     0   Group:     0   Size: 3072
 File ACL: 0    Directory ACL: 0
-Links: 1   Blockcount: 32
+Links: 1   Blockcount: 0
 Fragment:  Address: 0    Number: 0    Size: 0
 ctime: 0x539ff5b2 -- Tue Jun 17 08:00:50 2014
 atime: 0x539ff5b2 -- Tue Jun 17 08:00:50 2014
 mtime: 0x539ff5b2 -- Tue Jun 17 08:00:50 2014
-BLOCKS:
-(TIND):1650
-TOTAL: 1

  parent reply	other threads:[~2015-03-19 21:47 UTC|newest]

Thread overview: 88+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-01-27  7:35 [PATCH 00/54] e2fsprogs January 2015 patchbomb Darrick J. Wong
2015-01-27  7:35 ` [PATCH 01/54] misc: fix minor testcase problems Darrick J. Wong
2015-01-27 15:55   ` Theodore Ts'o
2015-01-27  7:35 ` [PATCH 02/54] debugfs: document new commands Darrick J. Wong
2015-01-27 15:56   ` Theodore Ts'o
2015-01-27  7:35 ` [PATCH 03/54] debugfs: fix crash in ea_set argument handling Darrick J. Wong
2015-01-27 15:58   ` Theodore Ts'o
2015-01-27  7:35 ` [PATCH 04/54] libext2fs: initialize i_extra_isize when writing EAs Darrick J. Wong
2015-01-27 16:02   ` Theodore Ts'o
2015-01-27  7:36 ` [PATCH 05/54] libext2fs: avoid pointless EA block allocation Darrick J. Wong
2015-01-27 16:07   ` Theodore Ts'o
2015-01-27 19:26     ` Darrick J. Wong
2015-01-27  7:36 ` [PATCH 06/54] libext2fs: strengthen i_extra_isize checks when reading/writing xattrs Darrick J. Wong
2015-01-27 16:08   ` Theodore Ts'o
2015-01-27  7:36 ` [PATCH 07/54] libext2fs: fix tdb.c mmap leak Darrick J. Wong
2015-01-27 16:09   ` Theodore Ts'o
2015-01-27  7:36 ` [PATCH 08/54] resize2fs: fix regression test to not depend on ext4.ko being loaded Darrick J. Wong
2015-01-27 16:10   ` Theodore Ts'o
2015-01-27  7:36 ` [PATCH 09/54] tune2fs: disable csum verification before resizing inode Darrick J. Wong
2015-01-27 16:11   ` Theodore Ts'o
2015-01-27  7:36 ` [PATCH 10/54] tune2fs: abort when trying to enable/disable metadata_csum on mounted fs Darrick J. Wong
2015-01-27 16:26   ` Theodore Ts'o
2015-01-27  7:36 ` [PATCH 11/54] tune2fs: call out to resize2fs for 64bit conversion Darrick J. Wong
2015-01-27 16:31   ` Theodore Ts'o
2015-01-27  7:36 ` [PATCH 12/54] e2fsck: clear i_block[] when there are too many bad mappings on a special inode Darrick J. Wong
2015-01-27 16:32   ` Theodore Ts'o
2015-01-27  7:36 ` [PATCH 13/54] e2fsck: on read error, don't rewrite blocks past the end of the fs Darrick J. Wong
2015-01-27 17:35   ` Theodore Ts'o
2015-01-28 23:35     ` Darrick J. Wong
2015-01-27  7:37 ` [PATCH 14/54] e2fsck: fix the journal recreation message Darrick J. Wong
2015-01-27 18:02   ` Theodore Ts'o
2015-01-27 19:37     ` Darrick J. Wong
2015-01-27  7:37 ` [PATCH 15/54] e2fsck: handle multiple *ind block collisions with critical metadata Darrick J. Wong
2015-01-28 13:52   ` Theodore Ts'o
2015-01-27  7:37 ` [PATCH 16/54] e2fsck: decrement bad count _after_ remapping a duplicate block Darrick J. Wong
2015-01-28 13:58   ` Theodore Ts'o
2015-01-27  7:37 ` [PATCH 17/54] e2fsck: inspect inline dir data as two directory blocks Darrick J. Wong
2015-01-28 15:16   ` Theodore Ts'o
2015-01-27  7:37 ` [PATCH 18/54] e2fsck: improve the inline directory detector Darrick J. Wong
2015-01-28 16:38   ` Theodore Ts'o
2015-01-27  7:37 ` [PATCH 19/54] e2fsck: salvage under-sized dirents by removing them Darrick J. Wong
2015-02-16 15:40   ` Theodore Ts'o
2015-01-27  7:37 ` [PATCH 20/54] e2fsck: add a 'yes to all' response in interactive mode Darrick J. Wong
2015-03-29  2:54   ` Theodore Ts'o
2015-01-27  7:37 ` [PATCH 21/54] libext2fs: zero blocks via FALLOC_FL_ZERO_RANGE in ext2fs_zero_blocks Darrick J. Wong
2015-03-29  3:46   ` Theodore Ts'o
2015-01-27  7:37 ` [PATCH 22/54] libext2fs: ext2fs_new_block2() should call alloc_block hook Darrick J. Wong
2015-03-29  3:08   ` Theodore Ts'o
2015-01-27  7:38 ` [PATCH 23/54] libext2fs: Support readonly filesystem images Darrick J. Wong
2015-03-19 21:32   ` [PATCH v2 " Darrick J. Wong
2015-03-29  3:42     ` Theodore Ts'o
2015-01-27  7:38 ` [PATCH 24/54] libext2fs/e2fsck: provide routines to read-ahead metadata Darrick J. Wong
2015-01-27  7:38 ` [PATCH 25/54] e2fsck: read-ahead metadata during passes 1, 2, and 4 Darrick J. Wong
2015-01-27  7:38 ` [PATCH 26/54] e2fsck: track directories to be rehashed with a bitmap Darrick J. Wong
2015-01-27  7:38 ` [PATCH 27/54] e2fsck: rebuild sparse extent trees/convert non-extent ext3 files Darrick J. Wong
2015-03-19 21:42   ` [PATCH v4 " Darrick J. Wong
2015-01-27  7:38 ` [PATCH 28/54] tests: verify proper rebuilding of sparse extent trees and block map file conversion Darrick J. Wong
2015-01-27  7:38 ` [PATCH 29/54] undo-io: add new calls to and speed up the undo io manager Darrick J. Wong
2015-01-27  7:38 ` [PATCH 30/54] undo-io: be more flexible about setting block size Darrick J. Wong
2015-01-27  7:38 ` [PATCH 31/54] undo-io: use a bitmap to track what we've already written Darrick J. Wong
2015-01-27  7:39 ` [PATCH 32/54] e2undo: fix memory leaks and tweak the error messages somewhat Darrick J. Wong
2015-01-27  7:39 ` [PATCH 33/54] e2undo: ditch tdb file, write everything to a flat file Darrick J. Wong
2015-01-27  7:39 ` [PATCH 34/54] libext2fs: support atexit cleanups Darrick J. Wong
2015-01-27  7:39 ` [PATCH 35/54] e2fsck: optionally create an undo file Darrick J. Wong
2015-01-27  7:39 ` [PATCH 36/54] resize2fs: optionally create " Darrick J. Wong
2015-01-27  7:39 ` [PATCH 37/54] tune2fs: " Darrick J. Wong
2015-01-27  7:39 ` [PATCH 38/54] mke2fs: " Darrick J. Wong
2015-01-27  7:39 ` [PATCH 39/54] debugfs: " Darrick J. Wong
2015-01-27  7:39 ` [PATCH 40/54] tests: test undo file creation in e2fsck/resize2fs/tune2fs/mke2fs Darrick J. Wong
2015-01-27  7:40 ` [PATCH 41/54] tests: test various features of the new e2undo format Darrick J. Wong
2015-01-27  7:40 ` [PATCH 42/54] copy-in: create hardlinks with the correct directory filetype Darrick J. Wong
2015-01-27  7:40 ` [PATCH 43/54] copy-in: for files, only iterate file blocks that are mapped Darrick J. Wong
2015-01-27  7:40 ` [PATCH 44/54] copyin: fix error handling Darrick J. Wong
2015-01-27  7:40 ` [PATCH 45/54] mke2fs: add simple tests and re-alphabetize mke2fs manpage options Darrick J. Wong
2015-01-27  7:40 ` [PATCH 46/54] contrib: script to create minified ext4 image from a directory Darrick J. Wong
2015-01-27  7:40 ` [PATCH 47/54] libext2fs: support allocating uninit blocks in bmap2() Darrick J. Wong
2015-01-27  7:40 ` [PATCH 48/54] libext2fs: find/alloc a range of empty blocks Darrick J. Wong
2015-01-27  7:40 ` [PATCH 49/54] libext2fs: add new hooks to support large allocations Darrick J. Wong
2015-01-27  7:41 ` [PATCH 50/54] libext2fs: implement fallocate Darrick J. Wong
2015-01-27  7:41 ` [PATCH 51/54] libext2fs: use fallocate for creating journals and hugefiles Darrick J. Wong
2015-01-27  7:41 ` [PATCH 52/54] debugfs: implement fallocate Darrick J. Wong
2015-01-27  7:41 ` [PATCH 53/54] tests: test debugfs punch command Darrick J. Wong
2015-03-19 21:44 ` [PATCH 55/54] e2fsck: actually fix inline_data flags problems when user says to do so Darrick J. Wong
2015-03-29  4:05   ` Theodore Ts'o
2015-03-19 21:45 ` [PATCH 56/54] libext2fs: zero hash in ibody extended attributes Darrick J. Wong
2015-03-29  4:13   ` Theodore Ts'o
2015-03-19 21:47 ` Darrick J. Wong [this message]
2015-03-19 23:54 ` [PATCH 58/54] e2fsck: turn inline data symlink into a fast symlink when possible Darrick J. Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20150319214721.GL11031@birch.djwong.org \
    --to=darrick.wong@oracle.com \
    --cc=linux-ext4@vger.kernel.org \
    --cc=tytso@mit.edu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).