linux-ext4.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [RFC PATCH v1 0/6] e2fsprogs 8/2013 patchbomb
@ 2013-08-29  0:43 Darrick J. Wong
  2013-08-29  0:43 ` [PATCH 1/6] libext2fs: ext2fs_dup_handle should not alias MMP buffers Darrick J. Wong
                   ` (5 more replies)
  0 siblings, 6 replies; 13+ messages in thread
From: Darrick J. Wong @ 2013-08-29  0:43 UTC (permalink / raw)
  To: tytso, darrick.wong; +Cc: linux-ext4

I have a pile of random patches that I've been working on.  They're not really
related to each other, though there are some bugfixes at the start.  The first
two patches were sent to linux-ext4 on July 22nd and August 12th and haven't
changed substantially.

The first patch fixes a double-free crash when resize2fs'ing a MMP filesystem.

The next two fix blk_t overflow problems by widening the fields to blk64_t.
One of the patches fixes something that Eric Sandeen was complaining about.

The fourth patch makes it so you can sparse-check e2fsprogs with make C=[12].
Offhand it looks like there are more than a few spots where we should have
converted things to blk64_t but haven't yet.  There are also nastier spots
where 64-bit values get crammed into 32-bit fields (or so sparse thinks) that I
will triage later.

The fifth patch experimentally enables zeroing of inode tables when disabling
checksumming in tune2fs.

The sixth patch modifies resize2fs to toggle 64bit support, with all the
necessary block migrations that might arise from that.  It will probably eat
your data.

I've lightly tested these e2fsprogs changes against the head branch as of a few
days ago.  After a longish while I think I've managed to reconstruct my old
metadata checksumming test program from the old days, so I've posted that
version here:
https://gist.github.com/djwong/6372995

Comments and questions are, as always, welcome.

--D

^ permalink raw reply	[flat|nested] 13+ messages in thread
* [PATCH 1/6] libext2fs: ext2fs_dup_handle should not alias MMP buffers
  2013-08-29  0:43 [RFC PATCH v1 0/6] e2fsprogs 8/2013 patchbomb Darrick J. Wong
@ 2013-08-29  0:43 ` Darrick J. Wong
  2013-09-09 14:39   ` Theodore Ts'o
  2013-08-29  0:43 ` [PATCH 2/6] resize2fs: Use blk64_t and location getters for free_gdp_blocks() Darrick J. Wong
                   ` (4 subsequent siblings)
  5 siblings, 1 reply; 13+ messages in thread
From: Darrick J. Wong @ 2013-08-29  0:43 UTC (permalink / raw)
  To: tytso, darrick.wong; +Cc: linux-ext4

It turns out that resize2fs uses ext2fs_dup_handle to duplicate fs handles.  If
MMP is enabled, this causes both handles to share MMP buffers, which is bad
news when it comes time to free both handles.  Change the code to (we hope) fix
this.  This prevents resize2fs from failing with a double-free error when
handed a MMP filesystem.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 lib/ext2fs/dupfs.c |   25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)


diff --git a/lib/ext2fs/dupfs.c b/lib/ext2fs/dupfs.c
index 64d3124..02721e1 100644
--- a/lib/ext2fs/dupfs.c
+++ b/lib/ext2fs/dupfs.c
@@ -40,6 +40,9 @@ errcode_t ext2fs_dup_handle(ext2_filsys src, ext2_filsys *dest)
 	fs->block_map = 0;
 	fs->badblocks = 0;
 	fs->dblist = 0;
+	fs->mmp_buf = 0;
+	fs->mmp_cmp = 0;
+	fs->mmp_fd = -1;
 
 	io_channel_bumpcount(fs->io);
 	if (fs->icache)
@@ -87,6 +90,28 @@ errcode_t ext2fs_dup_handle(ext2_filsys src, ext2_filsys *dest)
 		if (retval)
 			goto errout;
 	}
+	if (src->mmp_buf) {
+		retval = ext2fs_get_mem(src->blocksize, &fs->mmp_buf);
+		if (retval)
+			goto errout;
+		memcpy(fs->mmp_buf, src->mmp_buf, src->blocksize);
+	}
+	if (src->mmp_fd >= 0) {
+		fs->mmp_fd = dup(src->mmp_fd);
+		if (fs->mmp_fd < 0) {
+			retval = EXT2_ET_MMP_OPEN_DIRECT;
+			goto errout;
+		}
+	}
+	if (src->mmp_cmp) {
+		int align = ext2fs_get_dio_alignment(src->mmp_fd);
+
+		retval = ext2fs_get_memalign(src->blocksize, align,
+					     &fs->mmp_cmp);
+		if (retval)
+			goto errout;
+		memcpy(fs->mmp_cmp, src->mmp_cmp, src->blocksize);
+	}
 	*dest = fs;
 	return 0;
 errout:


^ permalink raw reply related	[flat|nested] 13+ messages in thread
* [PATCH 2/6] resize2fs: Use blk64_t and location getters for free_gdp_blocks()
  2013-08-29  0:43 [RFC PATCH v1 0/6] e2fsprogs 8/2013 patchbomb Darrick J. Wong
  2013-08-29  0:43 ` [PATCH 1/6] libext2fs: ext2fs_dup_handle should not alias MMP buffers Darrick J. Wong
@ 2013-08-29  0:43 ` Darrick J. Wong
  2013-09-09 14:41   ` Theodore Ts'o
  2013-08-29  0:44 ` [PATCH 3/6] e2fsck: Fix incorrect bbitmap checksum failure caused by integer overflow Darrick J. Wong
                   ` (3 subsequent siblings)
  5 siblings, 1 reply; 13+ messages in thread
From: Darrick J. Wong @ 2013-08-29  0:43 UTC (permalink / raw)
  To: tytso, darrick.wong; +Cc: Eric Sandeen, linux-ext4

free_gdp_blocks needs to be taught to use 64-bit fields and the appropriate
getters, otherwise it'll truncate high block numbers (when, say, resizing a
>16T fs) and mark the low numbered group descriptor blocks as free.  Yikes.

Reported-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 resize/resize2fs.c |   41 ++++++++++++++++++++---------------------
 1 file changed, 20 insertions(+), 21 deletions(-)


diff --git a/resize/resize2fs.c b/resize/resize2fs.c
index 3610e7b..76d7aa6 100644
--- a/resize/resize2fs.c
+++ b/resize/resize2fs.c
@@ -272,34 +272,36 @@ static void fix_uninit_block_bitmaps(ext2_filsys fs)
  */
 static void free_gdp_blocks(ext2_filsys fs,
 			    ext2fs_block_bitmap reserve_blocks,
-			    struct ext2_group_desc *gdp)
+			    ext2_filsys old_fs,
+			    dgrp_t group)
 {
-	blk_t	blk;
+	blk64_t	blk;
 	int	j;
 
-	if (gdp->bg_block_bitmap &&
-	    (gdp->bg_block_bitmap < ext2fs_blocks_count(fs->super))) {
-		ext2fs_block_alloc_stats(fs, gdp->bg_block_bitmap, -1);
-		ext2fs_mark_block_bitmap2(reserve_blocks,
-					 gdp->bg_block_bitmap);
+	blk = ext2fs_block_bitmap_loc(old_fs, group);
+	if (blk &&
+	    (blk < ext2fs_blocks_count(fs->super))) {
+		ext2fs_block_alloc_stats2(fs, blk, -1);
+		ext2fs_mark_block_bitmap2(reserve_blocks, blk);
 	}
 
-	if (gdp->bg_inode_bitmap &&
-	    (gdp->bg_inode_bitmap < ext2fs_blocks_count(fs->super))) {
-		ext2fs_block_alloc_stats(fs, gdp->bg_inode_bitmap, -1);
-		ext2fs_mark_block_bitmap2(reserve_blocks,
-					 gdp->bg_inode_bitmap);
+	blk = ext2fs_inode_bitmap_loc(old_fs, group);
+	if (blk &&
+	    (blk < ext2fs_blocks_count(fs->super))) {
+		ext2fs_block_alloc_stats2(fs, blk, -1);
+		ext2fs_mark_block_bitmap2(reserve_blocks, blk);
 	}
 
-	if (gdp->bg_inode_table == 0 ||
-	    (gdp->bg_inode_table >= ext2fs_blocks_count(fs->super)))
+	blk = ext2fs_inode_table_loc(old_fs, group);
+	if (blk == 0 ||
+	    (blk >= ext2fs_blocks_count(fs->super)))
 		return;
 
-	for (blk = gdp->bg_inode_table, j = 0;
+	for (j = 0;
 	     j < fs->inode_blocks_per_group; j++, blk++) {
 		if (blk >= ext2fs_blocks_count(fs->super))
 			break;
-		ext2fs_block_alloc_stats(fs, blk, -1);
+		ext2fs_block_alloc_stats2(fs, blk, -1);
 		ext2fs_mark_block_bitmap2(reserve_blocks, blk);
 	}
 }
@@ -466,11 +468,8 @@ retry:
 		 * and free any blocks associated with their metadata
 		 */
 		for (i = fs->group_desc_count;
-		     i < old_fs->group_desc_count; i++) {
-			free_gdp_blocks(fs, reserve_blocks,
-					ext2fs_group_desc(old_fs,
-						old_fs->group_desc, i));
-		}
+		     i < old_fs->group_desc_count; i++)
+			free_gdp_blocks(fs, reserve_blocks, old_fs, i);
 		retval = 0;
 		goto errout;
 	}


^ permalink raw reply related	[flat|nested] 13+ messages in thread
* [PATCH 3/6] e2fsck: Fix incorrect bbitmap checksum failure caused by integer overflow
  2013-08-29  0:43 [RFC PATCH v1 0/6] e2fsprogs 8/2013 patchbomb Darrick J. Wong
  2013-08-29  0:43 ` [PATCH 1/6] libext2fs: ext2fs_dup_handle should not alias MMP buffers Darrick J. Wong
  2013-08-29  0:43 ` [PATCH 2/6] resize2fs: Use blk64_t and location getters for free_gdp_blocks() Darrick J. Wong
@ 2013-08-29  0:44 ` Darrick J. Wong
  2013-09-16 13:42   ` Theodore Ts'o
  2013-08-29  0:44 ` [PATCH 4/6] e2fsprogs: Add (optional) sparse checking to the build Darrick J. Wong
                   ` (2 subsequent siblings)
  5 siblings, 1 reply; 13+ messages in thread
From: Darrick J. Wong @ 2013-08-29  0:44 UTC (permalink / raw)
  To: tytso, darrick.wong; +Cc: linux-ext4

On a filesystem with more than 2^32 blocks, the block group checksum test will
fail because "i" (the group number) is a 32-bit quantity that is used to
calculate the group's block bitmap block number.  Unfortunately, "i" is not
automatically promoted to 64-bit for this calculation and overflows.  When this
happens, e2fsck will incorrectly report bitmap checksum errors.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 e2fsck/pass5.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)


diff --git a/e2fsck/pass5.c b/e2fsck/pass5.c
index 51e4683..0fe7045 100644
--- a/e2fsck/pass5.c
+++ b/e2fsck/pass5.c
@@ -168,7 +168,7 @@ static void check_block_bitmap_checksum(e2fsck_t ctx)
 
 		blk_itr = EXT2FS_B2C(ctx->fs,
 				     ctx->fs->super->s_first_data_block) +
-			  (i * (nbytes << 3));
+			  ((blk64_t)i * (nbytes << 3));
 		retval = ext2fs_get_block_bitmap_range2(ctx->fs->block_map,
 							blk_itr, nbytes << 3,
 							buf);


^ permalink raw reply related	[flat|nested] 13+ messages in thread
* [PATCH 4/6] e2fsprogs: Add (optional) sparse checking to the build
  2013-08-29  0:43 [RFC PATCH v1 0/6] e2fsprogs 8/2013 patchbomb Darrick J. Wong
                   ` (2 preceding siblings ...)
  2013-08-29  0:44 ` [PATCH 3/6] e2fsck: Fix incorrect bbitmap checksum failure caused by integer overflow Darrick J. Wong
@ 2013-08-29  0:44 ` Darrick J. Wong
  2013-08-29  0:44 ` [PATCH 5/6] tune2fs: Zero inode table when removing checksums Darrick J. Wong
  2013-08-29  0:44 ` [PATCH 6/6] resize2fs: Convert fs to and from 64bit mode Darrick J. Wong
  5 siblings, 0 replies; 13+ messages in thread
From: Darrick J. Wong @ 2013-08-29  0:44 UTC (permalink / raw)
  To: tytso, darrick.wong; +Cc: linux-ext4

Run sparse against source files when building e2fsprogs with 'make C=1'.  If
instead C=2, it configures basic ext2 types for bitwise checking with sparse,
which can help find the (many many) spots where conversion errors are
(possibly) happening.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 Makefile.in             |   13 +++++++++++++
 debugfs/Makefile.in     |    2 ++
 e2fsck/Makefile.in      |    3 +++
 e2fsck/pass1.c          |    2 +-
 ext2ed/Makefile.in      |    1 +
 intl/Makefile.in        |    2 ++
 lib/blkid/Makefile.in   |    2 ++
 lib/e2p/Makefile.in     |    2 ++
 lib/et/Makefile.in      |    2 ++
 lib/ext2fs/Makefile.in  |    2 ++
 lib/ext2fs/ext2fs.h     |   22 ++++++++++++++--------
 lib/quota/Makefile.in   |    2 ++
 lib/ss/Makefile.in      |    2 ++
 lib/uuid/Makefile.in    |    2 ++
 misc/Makefile.in        |    2 ++
 resize/Makefile.in      |    2 ++
 tests/progs/Makefile.in |    2 ++
 util/Makefile.in        |    2 ++
 18 files changed, 58 insertions(+), 9 deletions(-)


diff --git a/Makefile.in b/Makefile.in
index 544ed02..95cbf1b 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -5,6 +5,19 @@ top_builddir = .
 my_dir = .
 INSTALL = @INSTALL@
 
+CHECK=sparse
+CHECK_OPTS=-Wsparse-all -Wno-transparent-union -Wno-return-void -Wno-undef -Wno-non-pointer-null
+ifeq ("$(C)", "2")
+  CHECK_CMD=$(CHECK) $(CHECK_OPTS) -Wbitwise -D__CHECK_ENDIAN__
+else
+  ifeq ("$(C)", "1")
+    CHECK_CMD=$(CHECK) $(CHECK_OPTS)
+   else
+    CHECK_CMD=@true
+  endif
+endif
+export CHECK_CMD
+
 @MCONFIG@
 
 % : %.sh
diff --git a/debugfs/Makefile.in b/debugfs/Makefile.in
index 9a86dc6..afde5ac 100644
--- a/debugfs/Makefile.in
+++ b/debugfs/Makefile.in
@@ -44,6 +44,8 @@ STATIC_DEPLIBS= $(STATIC_LIBEXT2FS) $(DEPSTATIC_LIBSS) \
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(CC) -c $(ALL_CFLAGS) $< -o $@
+	$(E) "	SPARSE $<"
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 
 all:: $(PROGS) $(MANPAGES)
 
diff --git a/e2fsck/Makefile.in b/e2fsck/Makefile.in
index 9c27571..a900b9b 100644
--- a/e2fsck/Makefile.in
+++ b/e2fsck/Makefile.in
@@ -34,10 +34,13 @@ PROFILED_DEPLIBS= $(DEPPROFILED_LIBQUOTA) $(PROFILED_LIBEXT2FS) \
 		  $(DEPPROFILED_LIBUUID) $(DEPPROFILED_LIBE2P)
 
 COMPILE_ET=$(top_builddir)/lib/et/compile_et --build-tree
+CHECK=sparse
 
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(CC) -c $(ALL_CFLAGS) $< -o $@
+	$(E) "	SPARSE $<"
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 @PROFILE_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -g -pg -o profiled/$*.o -c $<
 
 #
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index ba6025b..89a7c20 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -2674,7 +2674,7 @@ static int process_bad_block(ext2_filsys fs,
 	return 0;
 }
 
-static void new_table_block(e2fsck_t ctx, blk_t first_block, int group,
+static void new_table_block(e2fsck_t ctx, blk64_t first_block, int group,
 			    const char *name, int num, blk64_t *new_block)
 {
 	ext2_filsys fs = ctx->fs;
diff --git a/ext2ed/Makefile.in b/ext2ed/Makefile.in
index b3b606a..290f06e 100644
--- a/ext2ed/Makefile.in
+++ b/ext2ed/Makefile.in
@@ -33,6 +33,7 @@ DOCS=   doc/ext2ed-design.pdf doc/user-guide.pdf doc/ext2fs-overview.pdf \
 
 .c.o:
 	$(CC) -c $(ALL_CFLAGS) $< -o $@
+	$(CHECK_CMD) $(ALL_CFLAGS) $<
 
 .SUFFIXES: .sgml .ps .pdf .html
 
diff --git a/intl/Makefile.in b/intl/Makefile.in
index 9dbc84e..6555331 100644
--- a/intl/Makefile.in
+++ b/intl/Makefile.in
@@ -191,6 +191,8 @@ LTV_AGE=4
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(COMPILE) $<
+	$(E) "	SPARSE $<"
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 
 .y.c:
 	$(YACC) $(YFLAGS) --output $@ $<
diff --git a/lib/blkid/Makefile.in b/lib/blkid/Makefile.in
index 0ec8564..ec393fe 100644
--- a/lib/blkid/Makefile.in
+++ b/lib/blkid/Makefile.in
@@ -55,6 +55,8 @@ DEPLIBS_BLKID=	$(DEPSTATIC_LIBBLKID) $(DEPSTATIC_LIBUUID)
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(CC) $(ALL_CFLAGS) -c $< -o $@
+	$(E) "	SPARSE $<"
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 @PROFILE_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -g -pg -o profiled/$*.o -c $<
 @CHECKER_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -checker -g -o checker/$*.o -c $<
 @ELF_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -fPIC -o elfshared/$*.o -c $<
diff --git a/lib/e2p/Makefile.in b/lib/e2p/Makefile.in
index e2d0940..1360155 100644
--- a/lib/e2p/Makefile.in
+++ b/lib/e2p/Makefile.in
@@ -55,6 +55,8 @@ BSDLIB_INSTALL_DIR = $(root_libdir)
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(CC) $(ALL_CFLAGS) -c $< -o $@
+	$(E) "	SPARSE $<"
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 @PROFILE_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -g -pg -o profiled/$*.o -c $<
 @CHECKER_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -checker -g -o checker/$*.o -c $<
 @ELF_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -fPIC -o elfshared/$*.o -c $<
diff --git a/lib/et/Makefile.in b/lib/et/Makefile.in
index 8d1ea26..9a16c80 100644
--- a/lib/et/Makefile.in
+++ b/lib/et/Makefile.in
@@ -43,6 +43,8 @@ BSDLIB_INSTALL_DIR = $(root_libdir)
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(CC) $(ALL_CFLAGS) -c $< -o $@
+	$(E) "	SPARSE $<"
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 @PROFILE_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -g -pg -o profiled/$*.o -c $<
 @CHECKER_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -checker -g -o checker/$*.o -c $<
 @ELF_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -fPIC -o elfshared/$*.o -c $<
diff --git a/lib/ext2fs/Makefile.in b/lib/ext2fs/Makefile.in
index dc7b0d1..d142cf3 100644
--- a/lib/ext2fs/Makefile.in
+++ b/lib/ext2fs/Makefile.in
@@ -202,6 +202,8 @@ all:: ext2fs.pc
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(CC) $(ALL_CFLAGS) -c $< -o $@
+	$(E) "	SPARSE $<"
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 @PROFILE_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -g -pg -o profiled/$*.o -c $<
 @CHECKER_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -checker -g -o checker/$*.o -c $<
 @ELF_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -fPIC -o elfshared/$*.o -c $<
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index 311ceda..b078ec9 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -64,14 +64,20 @@ extern "C" {
 #include <ext2fs/ext3_extents.h>
 #endif /* EXT2_FLAT_INCLUDES */
 
-typedef __u32		ext2_ino_t;
-typedef __u32		blk_t;
-typedef __u64		blk64_t;
-typedef __u32		dgrp_t;
-typedef __u32		ext2_off_t;
-typedef __u64		ext2_off64_t;
-typedef __s64		e2_blkcnt_t;
-typedef __u32		ext2_dirhash_t;
+#ifdef __CHECK_ENDIAN__
+#define __bitwise __attribute__((bitwise))
+#else
+#define __bitwise
+#endif
+
+typedef __u32 __bitwise		ext2_ino_t;
+typedef __u32 __bitwise		blk_t;
+typedef __u64 __bitwise		blk64_t;
+typedef __u32 __bitwise		dgrp_t;
+typedef __u32 __bitwise		ext2_off_t;
+typedef __u64 __bitwise		ext2_off64_t;
+typedef __s64 __bitwise		e2_blkcnt_t;
+typedef __u32 __bitwise		ext2_dirhash_t;
 
 #if EXT2_FLAT_INCLUDES
 #include "com_err.h"
diff --git a/lib/quota/Makefile.in b/lib/quota/Makefile.in
index 720befd..7ea5100 100644
--- a/lib/quota/Makefile.in
+++ b/lib/quota/Makefile.in
@@ -47,6 +47,8 @@ LIBDIR= quota
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(CC) $(ALL_CFLAGS) -c $< -o $@
+	$(E) "	SPARSE $<"
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 @PROFILE_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -g -pg -o profiled/$*.o -c $<
 @CHECKER_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -checker -g -o checker/$*.o -c $<
 #ELF_CMT#	$(Q) $(CC) $(ALL_CFLAGS) -fPIC -o elfshared/$*.o -c $<
diff --git a/lib/ss/Makefile.in b/lib/ss/Makefile.in
index c396f2d..95834fc 100644
--- a/lib/ss/Makefile.in
+++ b/lib/ss/Makefile.in
@@ -34,6 +34,8 @@ MK_CMDS=_SS_DIR_OVERRIDE=. ./mk_cmds
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(CC) $(ALL_CFLAGS) -c $<
+	$(E) "	SPARSE $<"
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 @PROFILE_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -g -pg -o profiled/$*.o -c $<
 @CHECKER_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -checker -g -o checker/$*.o -c $<
 @ELF_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -DSHARED_ELF_LIB -fPIC -o elfshared/$*.o -c $<
diff --git a/lib/uuid/Makefile.in b/lib/uuid/Makefile.in
index 7329467..1d350ee 100644
--- a/lib/uuid/Makefile.in
+++ b/lib/uuid/Makefile.in
@@ -62,6 +62,8 @@ BSDLIB_INSTALL_DIR = $(root_libdir)
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(CC) $(ALL_CFLAGS) -c $< -o $@
+	$(E) "	SPARSE $<"
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 @PROFILE_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -g -pg -o profiled/$*.o -c $<
 @CHECKER_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -checker -g -o checker/$*.o -c $<
 @ELF_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -fPIC -o elfshared/$*.o -c $<
diff --git a/misc/Makefile.in b/misc/Makefile.in
index 8a69855..cae1b9f 100644
--- a/misc/Makefile.in
+++ b/misc/Makefile.in
@@ -100,6 +100,8 @@ COMPILE_ET=$(top_builddir)/lib/et/compile_et --build-tree
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(CC) -c $(ALL_CFLAGS) $< -o $@
+	$(E) "	SPARSE $<"
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 @PROFILE_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -g -pg -o profiled/$*.o -c $<
 
 all:: profiled $(SPROGS) $(UPROGS) $(USPROGS) $(SMANPAGES) $(UMANPAGES) \
diff --git a/resize/Makefile.in b/resize/Makefile.in
index a06b642..e7ea302 100644
--- a/resize/Makefile.in
+++ b/resize/Makefile.in
@@ -38,6 +38,8 @@ DEPSTATIC_LIBS= $(STATIC_LIBE2P) $(STATIC_LIBEXT2FS) $(DEPSTATIC_LIBCOM_ERR)
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(CC) -c $(ALL_CFLAGS) $< -o $@
+	$(E) "	SPARSE $<"
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 
 all:: $(PROGS) $(TEST_PROGS) $(MANPAGES) 
 
diff --git a/tests/progs/Makefile.in b/tests/progs/Makefile.in
index e3c1ef4..6508520 100644
--- a/tests/progs/Makefile.in
+++ b/tests/progs/Makefile.in
@@ -27,6 +27,8 @@ DEPLIBS= $(LIBEXT2FS) $(DEPLIBSS) $(DEPLIBCOM_ERR)
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(CC) -c $(ALL_CFLAGS) $< -o $@
+	$(E) "	SPARSE $<"
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 
 all:: $(PROGS)
 
diff --git a/util/Makefile.in b/util/Makefile.in
index adf0b46..8412442 100644
--- a/util/Makefile.in
+++ b/util/Makefile.in
@@ -16,6 +16,8 @@ SRCS = $(srcdir)/subst.c
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(BUILD_CC) -c $(BUILD_CFLAGS) $< -o $@
+	$(E) "	SPARSE $<"
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 
 PROGS=		subst symlinks
 


^ permalink raw reply related	[flat|nested] 13+ messages in thread
* [PATCH 5/6] tune2fs: Zero inode table when removing checksums
  2013-08-29  0:43 [RFC PATCH v1 0/6] e2fsprogs 8/2013 patchbomb Darrick J. Wong
                   ` (3 preceding siblings ...)
  2013-08-29  0:44 ` [PATCH 4/6] e2fsprogs: Add (optional) sparse checking to the build Darrick J. Wong
@ 2013-08-29  0:44 ` Darrick J. Wong
  2013-09-16 13:53   ` Theodore Ts'o
  2013-08-29  0:44 ` [PATCH 6/6] resize2fs: Convert fs to and from 64bit mode Darrick J. Wong
  5 siblings, 1 reply; 13+ messages in thread
From: Darrick J. Wong @ 2013-08-29  0:44 UTC (permalink / raw)
  To: tytso, darrick.wong; +Cc: linux-ext4

When disabling group checksums, we have to initialize the inode table.  Right
now tune2fs doesn't do this; it merely punts to e2fsck to clean up the mess.
Unfortunately, if the "uninitialized" inode table contains things that look
like inodes (i_link_count > 0, specifically), the e2fsck tries to recover these
inodes.  This leads to it misinterpreting i_blocks as a block map, at which
point it needlessly resurrects phantom inodes and crosslinked file repairs.  As
part of initializing the block bitmaps, we must also mark block group metadata
blocks in use.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 misc/tune2fs.c |  113 ++++++++++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 100 insertions(+), 13 deletions(-)


diff --git a/misc/tune2fs.c b/misc/tune2fs.c
index c38711d..ddf3259 100644
--- a/misc/tune2fs.c
+++ b/misc/tune2fs.c
@@ -614,12 +614,13 @@ static errcode_t rewrite_directory(ext2_filsys fs, ext2_ino_t dir,
 static void rewrite_inodes(ext2_filsys fs)
 {
 	int length = EXT2_INODE_SIZE(fs->super);
-	struct ext2_inode *inode;
+	struct ext2_inode *inode, *zero;
 	char		*ea_buf;
 	ext2_inode_scan	scan;
 	errcode_t	retval;
 	ext2_ino_t	ino;
 	blk64_t		file_acl_block;
+	int		inode_dirty;
 
 	if (fs->super->s_creator_os != EXT2_OS_LINUX)
 		return;
@@ -636,6 +637,12 @@ static void rewrite_inodes(ext2_filsys fs)
 		exit(1);
 	}
 
+	retval = ext2fs_get_memzero(length, &zero);
+	if (retval) {
+		com_err("set_csum", retval, "while allocating memory");
+		exit(1);
+	}
+
 	retval = ext2fs_get_mem(fs->blocksize, &ea_buf);
 	if (retval) {
 		com_err("set_csum", retval, "while allocating memory");
@@ -650,11 +657,25 @@ static void rewrite_inodes(ext2_filsys fs)
 		}
 		if (!ino)
 			break;
+		if (ext2fs_test_inode_bitmap2(fs->inode_map, ino)) {
+			inode_dirty = 1;
+		} else {
+			if (memcmp(inode, zero, length) != 0) {
+				memset(inode, 0, length);
+				inode_dirty = 1;
+			} else {
+				inode_dirty = 0;
+			}
+		}
 
-		retval = ext2fs_write_inode_full(fs, ino, inode, length);
-		if (retval) {
-			com_err("set_csum", retval, "while writing inode");
-			exit(1);
+		if (inode_dirty) {
+			retval = ext2fs_write_inode_full(fs, ino, inode,
+							 length);
+			if (retval) {
+				com_err("set_csum", retval, "while writing "
+					"inode");
+				exit(1);
+			}
 		}
 
 		retval = rewrite_extents(fs, ino, inode);
@@ -691,6 +712,7 @@ static void rewrite_inodes(ext2_filsys fs)
 		}
 	} while (ino);
 
+	ext2fs_free_mem(&zero);
 	ext2fs_free_mem(&inode);
 	ext2fs_free_mem(&ea_buf);
 	ext2fs_close_inode_scan(scan);
@@ -704,8 +726,8 @@ static void rewrite_metadata_checksums(ext2_filsys fs)
 	ext2fs_init_csum_seed(fs);
 	for (i = 0; i < fs->group_desc_count; i++)
 		ext2fs_group_desc_csum_set(fs, i);
-	rewrite_inodes(fs);
 	ext2fs_read_bitmaps(fs);
+	rewrite_inodes(fs);
 	ext2fs_mark_ib_dirty(fs);
 	ext2fs_mark_bb_dirty(fs);
 	ext2fs_mmp_update2(fs, 1);
@@ -733,10 +755,50 @@ static void enable_uninit_bg(ext2_filsys fs)
 	fs->flags &= ~EXT2_FLAG_SUPER_ONLY;
 }
 
+static errcode_t zero_empty_inodes(ext2_filsys fs)
+{
+	int length = EXT2_INODE_SIZE(fs->super);
+	struct ext2_inode *inode;
+	ext2_inode_scan	scan;
+	errcode_t	retval;
+	ext2_ino_t	ino;
+
+	retval = ext2fs_open_inode_scan(fs, 0, &scan);
+	if (retval)
+		goto out;
+
+	retval = ext2fs_get_mem(length, &inode);
+	if (retval)
+		goto out;
+
+	do {
+		retval = ext2fs_get_next_inode_full(scan, &ino, inode, length);
+		if (retval)
+			goto out;
+		if (!ino)
+			break;
+		if (!ext2fs_test_inode_bitmap2(fs->inode_map, ino)) {
+			memset(inode, 0, length);
+			retval = ext2fs_write_inode_full(fs, ino, inode,
+							 length);
+			if (retval)
+				goto out;
+		}
+	} while (1);
+
+out:
+	ext2fs_free_mem(&inode);
+	ext2fs_close_inode_scan(scan);
+	return retval;
+}
+
 static void disable_uninit_bg(ext2_filsys fs, __u32 csum_feature_flag)
 {
 	struct ext2_group_desc *gd;
 	dgrp_t i;
+	errcode_t retval;
+	blk64_t b, c, d;
+	int has_super;
 
 	/* Load bitmaps to ensure that the uninit ones get written out */
 	fs->super->s_feature_ro_compat |= csum_feature_flag;
@@ -745,21 +807,46 @@ static void disable_uninit_bg(ext2_filsys fs, __u32 csum_feature_flag)
 	ext2fs_mark_bb_dirty(fs);
 	fs->super->s_feature_ro_compat &= ~csum_feature_flag;
 
+	/* If we're only turning off uninit_bg, zero the inodes */
+	if (csum_feature_flag == EXT4_FEATURE_RO_COMPAT_GDT_CSUM) {
+		retval = zero_empty_inodes(fs);
+		if (retval) {
+			com_err("disable_uninit_bg", retval,
+				"while zeroing unused inodes");
+			request_fsck_afterwards(fs);
+		}
+	}
+
+	/* The bbitmap is zeroed; we must mark group metadata blocks in use */
 	for (i = 0; i < fs->group_desc_count; i++) {
-		gd = ext2fs_group_desc(fs, fs->group_desc, i);
-		if ((gd->bg_flags & EXT2_BG_INODE_ZEROED) == 0) {
-			/*
-			 * XXX what we really should do is zap
-			 * uninitialized inode tables instead.
-			 */
+		b = ext2fs_block_bitmap_loc(fs, i);
+		ext2fs_mark_block_bitmap2(fs->block_map, EXT2FS_B2C(fs, b));
+		b = ext2fs_inode_bitmap_loc(fs, i);
+		ext2fs_mark_block_bitmap2(fs->block_map, EXT2FS_B2C(fs, b));
+
+		retval = ext2fs_super_and_bgd_loc2(fs, i, &b, &c, &d, NULL);
+		if (retval == 0 && b)
+			ext2fs_mark_block_bitmap2(fs->block_map,
+						  EXT2FS_B2C(fs, b));
+		if (retval == 0 && c)
+			ext2fs_mark_block_bitmap2(fs->block_map,
+						  EXT2FS_B2C(fs, c));
+		if (retval == 0 && d)
+			ext2fs_mark_block_bitmap2(fs->block_map,
+						  EXT2FS_B2C(fs, d));
+		if (retval) {
+			com_err("disable_uninit_bg", retval,
+				"while initializing block bitmaps");
 			request_fsck_afterwards(fs);
-			break;
 		}
+
+		gd = ext2fs_group_desc(fs, fs->group_desc, i);
 		gd->bg_itable_unused = 0;
 		gd->bg_flags = 0;
 		ext2fs_group_desc_csum_set(fs, i);
 	}
 	fs->flags &= ~EXT2_FLAG_SUPER_ONLY;
+	ext2fs_mark_super_dirty(fs);
 }
 
 /*


^ permalink raw reply related	[flat|nested] 13+ messages in thread
* [PATCH 6/6] resize2fs: Convert fs to and from 64bit mode
  2013-08-29  0:43 [RFC PATCH v1 0/6] e2fsprogs 8/2013 patchbomb Darrick J. Wong
                   ` (4 preceding siblings ...)
  2013-08-29  0:44 ` [PATCH 5/6] tune2fs: Zero inode table when removing checksums Darrick J. Wong
@ 2013-08-29  0:44 ` Darrick J. Wong
  2013-09-09 17:29   ` Darrick J. Wong
  5 siblings, 1 reply; 13+ messages in thread
From: Darrick J. Wong @ 2013-08-29  0:44 UTC (permalink / raw)
  To: tytso, darrick.wong; +Cc: linux-ext4

resize2fs does its magic by loading a filesystem, duplicating the in-memory
image of that fs, moving relevant blocks out of the way of whatever new
metadata get created, and finally writing everything back out to disk.
Enabling 64bit mode enlarges the group descriptors, which makes resize2fs a
reasonable vehicle for taking care of the rest of the bookkeeping requirements,
so add to resize2fs the ability to convert a filesystem to 64bit mode and back.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 resize/main.c         |   22 ++++
 resize/resize2fs.8.in |   18 +++
 resize/resize2fs.c    |  264 ++++++++++++++++++++++++++++++++++++++++++++++++-
 resize/resize2fs.h    |    3 +
 4 files changed, 300 insertions(+), 7 deletions(-)


diff --git a/resize/main.c b/resize/main.c
index b648a15..f353351 100644
--- a/resize/main.c
+++ b/resize/main.c
@@ -41,7 +41,7 @@ char *program_name, *device_name, *io_options;
 static void usage (char *prog)
 {
 	fprintf (stderr, _("Usage: %s [-d debug_flags] [-f] [-F] [-M] [-P] "
-			   "[-p] device [new_size]\n\n"), prog);
+			   "[-p] device [-b|-s|new_size]\n\n"), prog);
 
 	exit (1);
 }
@@ -199,7 +199,7 @@ int main (int argc, char ** argv)
 	if (argc && *argv)
 		program_name = *argv;
 
-	while ((c = getopt (argc, argv, "d:fFhMPpS:")) != EOF) {
+	while ((c = getopt(argc, argv, "d:fFhMPpS:bs")) != EOF) {
 		switch (c) {
 		case 'h':
 			usage(program_name);
@@ -225,6 +225,12 @@ int main (int argc, char ** argv)
 		case 'S':
 			use_stride = atoi(optarg);
 			break;
+		case 'b':
+			flags |= RESIZE_ENABLE_64BIT;
+			break;
+		case 's':
+			flags |= RESIZE_DISABLE_64BIT;
+			break;
 		default:
 			usage(program_name);
 		}
@@ -434,7 +440,17 @@ int main (int argc, char ** argv)
 			fs->blocksize / 1024, new_size);
 		exit(1);
 	}
-	if (new_size == ext2fs_blocks_count(fs->super)) {
+	if (flags & RESIZE_DISABLE_64BIT && flags & RESIZE_ENABLE_64BIT) {
+		fprintf(stderr, _("Cannot set and unset 64bit feature.\n"));
+		exit(1);
+	} else if (flags & (RESIZE_DISABLE_64BIT | RESIZE_ENABLE_64BIT)) {
+		new_size = ext2fs_blocks_count(fs->super);
+		if (new_size >= (1ULL << 32)) {
+			fprintf(stderr, _("Cannot change 64bit on a filesystem"
+				" that is larger than 2^32 blocks.\n"));
+			exit(1);
+		}
+	} else if (new_size == ext2fs_blocks_count(fs->super)) {
 		fprintf(stderr, _("The filesystem is already %llu blocks "
 			"long.  Nothing to do!\n\n"), new_size);
 		exit(0);
diff --git a/resize/resize2fs.8.in b/resize/resize2fs.8.in
index 735fc91..d6deee6 100644
--- a/resize/resize2fs.8.in
+++ b/resize/resize2fs.8.in
@@ -8,7 +8,7 @@ resize2fs \- ext2/ext3/ext4 file system resizer
 .SH SYNOPSIS
 .B resize2fs
 [
-.B \-fFpPM
+.B \-fFpPMbs
 ]
 [
 .B \-d
@@ -85,8 +85,21 @@ to shrink the size of filesystem.  Then you may use
 to shrink the size of the partition.  When shrinking the size of
 the partition, make sure you do not make it smaller than the new size
 of the ext2 filesystem!
+.PP
+The
+.B \-b
+and
+.B \-s
+options enable and disable the 64bit feature, respectively.  The resize2fs
+program will, of course, take care of resizing the block group descriptors
+and moving other data blocks out of the way, as needed.  It is not possible
+to resize the filesystem concurrent with changing the 64bit status.
 .SH OPTIONS
 .TP
+.B \-b
+Turns on the 64bit feature, resizes the group descriptors as necessary, and
+moves other metadata out of the way.
+.TP
 .B \-d \fIdebug-flags
 Turns on various resize2fs debugging features, if they have been compiled
 into the binary.
@@ -124,6 +137,9 @@ of what the program is doing.
 .B \-P
 Print the minimum size of the filesystem and exit.
 .TP
+.B \-s
+Turns off the 64bit feature and frees blocks that are no longer in use.
+.TP
 .B \-S \fIRAID-stride
 The
 .B resize2fs
diff --git a/resize/resize2fs.c b/resize/resize2fs.c
index 76d7aa6..fa7f99e 100644
--- a/resize/resize2fs.c
+++ b/resize/resize2fs.c
@@ -53,6 +53,9 @@ static errcode_t ext2fs_calculate_summary_stats(ext2_filsys fs);
 static errcode_t fix_sb_journal_backup(ext2_filsys fs);
 static errcode_t mark_table_blocks(ext2_filsys fs,
 				   ext2fs_block_bitmap bmap);
+static errcode_t resize_group_descriptors(ext2_resize_t rfs, blk64_t new_size);
+static errcode_t move_bg_metadata(ext2_resize_t rfs);
+static errcode_t zero_high_bits_in_inodes(ext2_filsys fs);
 
 /*
  * Some helper CPP macros
@@ -119,13 +122,30 @@ errcode_t resize_fs(ext2_filsys fs, blk64_t *new_size, int flags,
 	if (retval)
 		goto errout;
 
+	init_resource_track(&rtrack, "resize_group_descriptors", fs->io);
+	retval = resize_group_descriptors(rfs, *new_size);
+	if (retval)
+		goto errout;
+	print_resource_track(rfs, &rtrack, fs->io);
+
+	init_resource_track(&rtrack, "move_bg_metadata", fs->io);
+	retval = move_bg_metadata(rfs);
+	if (retval)
+		goto errout;
+	print_resource_track(rfs, &rtrack, fs->io);
+
+	init_resource_track(&rtrack, "zero_high_bits_in_metadata", fs->io);
+	retval = zero_high_bits_in_inodes(rfs->new_fs);
+	if (retval)
+		goto errout;
+	print_resource_track(rfs, &rtrack, fs->io);
+
 	init_resource_track(&rtrack, "adjust_superblock", fs->io);
 	retval = adjust_superblock(rfs, *new_size);
 	if (retval)
 		goto errout;
 	print_resource_track(rfs, &rtrack, fs->io);
 
-
 	init_resource_track(&rtrack, "fix_uninit_block_bitmaps 2", fs->io);
 	fix_uninit_block_bitmaps(rfs->new_fs);
 	print_resource_track(rfs, &rtrack, fs->io);
@@ -221,6 +241,241 @@ errout:
 	return retval;
 }
 
+/* Toggle 64bit mode */
+static errcode_t resize_group_descriptors(ext2_resize_t rfs, blk64_t new_size)
+{
+	void *o, *n, *new_group_desc;
+	dgrp_t i;
+	int copy_size;
+	errcode_t retval;
+
+	if (!(rfs->flags & (RESIZE_DISABLE_64BIT | RESIZE_ENABLE_64BIT)))
+		return 0;
+
+	if (new_size != ext2fs_blocks_count(rfs->new_fs->super) ||
+	    ext2fs_blocks_count(rfs->new_fs->super) >= (1ULL << 32) ||
+	    (rfs->flags & RESIZE_DISABLE_64BIT &&
+	     rfs->flags & RESIZE_ENABLE_64BIT))
+		return EXT2_ET_INVALID_ARGUMENT;
+
+	if (rfs->flags & RESIZE_DISABLE_64BIT) {
+		rfs->new_fs->super->s_feature_incompat &=
+				~EXT4_FEATURE_INCOMPAT_64BIT;
+		rfs->new_fs->super->s_desc_size = EXT2_MIN_DESC_SIZE;
+	} else if (rfs->flags & RESIZE_ENABLE_64BIT) {
+		rfs->new_fs->super->s_feature_incompat |=
+				EXT4_FEATURE_INCOMPAT_64BIT;
+		rfs->new_fs->super->s_desc_size = EXT2_MIN_DESC_SIZE_64BIT;
+	}
+
+	if (EXT2_DESC_SIZE(rfs->old_fs->super) ==
+	    EXT2_DESC_SIZE(rfs->new_fs->super))
+		return 0;
+
+	o = rfs->new_fs->group_desc;
+	rfs->new_fs->desc_blocks = ext2fs_div_ceil(
+			rfs->old_fs->group_desc_count,
+			EXT2_DESC_PER_BLOCK(rfs->new_fs->super));
+	retval = ext2fs_get_arrayzero(rfs->new_fs->desc_blocks,
+				      rfs->old_fs->blocksize, &new_group_desc);
+	if (retval)
+		return retval;
+
+	n = new_group_desc;
+
+	if (EXT2_DESC_SIZE(rfs->old_fs->super) <=
+	    EXT2_DESC_SIZE(rfs->new_fs->super))
+		copy_size = EXT2_DESC_SIZE(rfs->old_fs->super);
+	else
+		copy_size = EXT2_DESC_SIZE(rfs->new_fs->super);
+	for (i = 0; i < rfs->old_fs->group_desc_count; i++) {
+		memcpy(n, o, copy_size);
+		n += EXT2_DESC_SIZE(rfs->new_fs->super);
+		o += EXT2_DESC_SIZE(rfs->old_fs->super);
+	}
+
+	ext2fs_free_mem(&rfs->new_fs->group_desc);
+	rfs->new_fs->group_desc = new_group_desc;
+
+	for (i = 0; i < rfs->old_fs->group_desc_count; i++)
+		ext2fs_group_desc_csum_set(rfs->new_fs, i);
+
+	return 0;
+}
+
+/* Move bitmaps/inode tables out of the way. */
+static errcode_t move_bg_metadata(ext2_resize_t rfs)
+{
+	dgrp_t i;
+	blk64_t b, c, d;
+	ext2fs_block_bitmap old_map, new_map;
+	int old, new;
+	errcode_t retval;
+	int zero = 0, one = 1;
+
+	if (!(rfs->flags & (RESIZE_DISABLE_64BIT | RESIZE_ENABLE_64BIT)))
+		return 0;
+
+	retval = ext2fs_allocate_block_bitmap(rfs->old_fs, "oldfs", &old_map);
+	if (retval)
+		return retval;
+
+	retval = ext2fs_allocate_block_bitmap(rfs->new_fs, "newfs", &new_map);
+	if (retval)
+		goto out;
+
+	/* Construct bitmaps of super/descriptor blocks in old and new fs */
+	for (i = 0; i < rfs->old_fs->group_desc_count; i++) {
+		retval = ext2fs_super_and_bgd_loc2(rfs->old_fs, i, &b, &c, &d,
+						   NULL);
+		if (retval)
+			goto out;
+		ext2fs_mark_block_bitmap2(old_map, EXT2FS_B2C(rfs->old_fs, b));
+		ext2fs_mark_block_bitmap2(old_map, EXT2FS_B2C(rfs->old_fs, c));
+		ext2fs_mark_block_bitmap2(old_map, EXT2FS_B2C(rfs->old_fs, d));
+
+		retval = ext2fs_super_and_bgd_loc2(rfs->new_fs, i, &b, &c, &d,
+						   NULL);
+		if (retval)
+			goto out;
+		ext2fs_mark_block_bitmap2(new_map, EXT2FS_B2C(rfs->new_fs, b));
+		ext2fs_mark_block_bitmap2(new_map, EXT2FS_B2C(rfs->new_fs, c));
+		ext2fs_mark_block_bitmap2(new_map, EXT2FS_B2C(rfs->new_fs, d));
+	}
+
+	/* Find changes in block allocations for bg metadata */
+	for (b = 0; b < ext2fs_blocks_count(rfs->new_fs->super); b++) {
+		c = EXT2FS_B2C(rfs->new_fs, b);
+		old = ext2fs_test_block_bitmap2(old_map, c);
+		new = ext2fs_test_block_bitmap2(new_map, c);
+
+		if (old && !new)
+			ext2fs_unmark_block_bitmap2(rfs->new_fs->block_map, c);
+		else if (!old && new)
+			; /* empty ext2fs_mark_block_bitmap2(new_map, c); */
+		else
+			ext2fs_unmark_block_bitmap2(new_map, c);
+	}
+	/* new_map now shows blocks that have been newly allocated. */
+
+	/* Move any conflicting bitmaps and inode tables */
+	for (i = 0; i < rfs->old_fs->group_desc_count; i++) {
+		b = EXT2FS_B2C(rfs->new_fs,
+			       ext2fs_block_bitmap_loc(rfs->new_fs, i));
+		if (ext2fs_test_block_bitmap2(new_map, b))
+			ext2fs_block_bitmap_loc_set(rfs->new_fs, i, 0);
+
+		b = EXT2FS_B2C(rfs->new_fs,
+			       ext2fs_inode_bitmap_loc(rfs->new_fs, i));
+		if (ext2fs_test_block_bitmap2(new_map, b))
+			ext2fs_inode_bitmap_loc_set(rfs->new_fs, i, 0);
+
+		c = ext2fs_inode_table_loc(rfs->new_fs, i);
+		for (b = 0; b < rfs->new_fs->inode_blocks_per_group; b++) {
+			if (ext2fs_test_block_bitmap2(new_map, b + c)) {
+				ext2fs_inode_table_loc_set(rfs->new_fs, i, 0);
+				break;
+			}
+		}
+	}
+
+out:
+	if (old_map)
+		ext2fs_free_block_bitmap(old_map);
+	if (new_map)
+		ext2fs_free_block_bitmap(new_map);
+	return retval;
+}
+
+/* Zero out the high bits of extent fields */
+static errcode_t zero_high_bits_in_extents(ext2_filsys fs, ext2_ino_t ino,
+				 struct ext2_inode *inode)
+{
+	ext2_extent_handle_t	handle;
+	struct ext2fs_extent	extent;
+	int			op = EXT2_EXTENT_ROOT;
+	errcode_t		errcode;
+
+	if (!(inode->i_flags & EXT4_EXTENTS_FL))
+		return 0;
+
+	errcode = ext2fs_extent_open(fs, ino, &handle);
+	if (errcode)
+		return errcode;
+
+	while (1) {
+		errcode = ext2fs_extent_get(handle, op, &extent);
+		if (errcode)
+			break;
+
+		op = EXT2_EXTENT_NEXT_SIB;
+
+		extent.e_pblk &= 0xFFFFFFFF;
+		errcode = ext2fs_extent_replace(handle, 0, &extent);
+		if (errcode)
+			break;
+	}
+
+	/* Ok if we run off the end */
+	if (errcode == EXT2_ET_EXTENT_NO_NEXT)
+		errcode = 0;
+	return errcode;
+}
+
+/* Zero out the high bits of inodes. */
+static errcode_t zero_high_bits_in_inodes(ext2_filsys fs)
+{
+	int length = EXT2_INODE_SIZE(fs->super);
+	struct ext2_inode *inode = NULL;
+	ext2_inode_scan	scan = NULL;
+	errcode_t	retval;
+	ext2_ino_t	ino;
+	blk64_t		file_acl_block;
+	__u32		sz_mask_high;
+	int		inode_dirty;
+
+	if (fs->super->s_creator_os != EXT2_OS_LINUX)
+		return 0;
+
+	retval = ext2fs_open_inode_scan(fs, 0, &scan);
+	if (retval)
+		return retval;
+
+	retval = ext2fs_get_mem(length, &inode);
+	if (retval)
+		goto out;
+
+	sz_mask_high = (((1ULL << 32) * EXT2_BLOCK_SIZE(fs->super)) - 1) >> 32;
+	do {
+		retval = ext2fs_get_next_inode_full(scan, &ino, inode, length);
+		if (retval)
+			goto out;
+		if (!ino)
+			break;
+		if (!ext2fs_test_inode_bitmap2(fs->inode_map, ino))
+			continue;
+
+		inode->i_size_high &= sz_mask_high;
+		inode->osd2.linux2.l_i_blocks_hi = 0;
+		inode->osd2.linux2.l_i_file_acl_high = 0;
+
+		retval = ext2fs_write_inode_full(fs, ino, inode,
+						 length);
+		if (retval)
+			goto out;
+
+		retval = zero_high_bits_in_extents(fs, ino, inode);
+		if (retval)
+			goto out;
+	} while (ino);
+
+out:
+	if (inode)
+		ext2fs_free_mem(&inode);
+	if (scan)
+		ext2fs_close_inode_scan(scan);
+}
+
 /*
  * Clean up the bitmaps for unitialized bitmaps
  */
@@ -424,7 +679,8 @@ retry:
 	/*
 	 * Reallocate the group descriptors as necessary.
 	 */
-	if (old_fs->desc_blocks != fs->desc_blocks) {
+	if (EXT2_DESC_SIZE(old_fs->super) == EXT2_DESC_SIZE(fs->super) &&
+	    old_fs->desc_blocks != fs->desc_blocks) {
 		retval = ext2fs_resize_mem(old_fs->desc_blocks *
 					   fs->blocksize,
 					   fs->desc_blocks * fs->blocksize,
@@ -939,7 +1195,9 @@ static errcode_t blocks_to_move(ext2_resize_t rfs)
 		new_blocks = fs->desc_blocks + fs->super->s_reserved_gdt_blocks;
 	}
 
-	if (old_blocks == new_blocks) {
+	if (EXT2_DESC_SIZE(rfs->old_fs->super) ==
+	    EXT2_DESC_SIZE(rfs->new_fs->super) &&
+	    old_blocks == new_blocks) {
 		retval = 0;
 		goto errout;
 	}
diff --git a/resize/resize2fs.h b/resize/resize2fs.h
index d425491..e24319d 100644
--- a/resize/resize2fs.h
+++ b/resize/resize2fs.h
@@ -81,6 +81,9 @@ typedef struct ext2_sim_progress *ext2_sim_progmeter;
 #define RESIZE_PERCENT_COMPLETE		0x0100
 #define RESIZE_VERBOSE			0x0200
 
+#define RESIZE_ENABLE_64BIT		0x0400
+#define RESIZE_DISABLE_64BIT		0x0800
+
 /*
  * This structure is used for keeping track of how much resources have
  * been used for a particular resize2fs pass.


^ permalink raw reply related	[flat|nested] 13+ messages in thread
* Re: [PATCH 1/6] libext2fs: ext2fs_dup_handle should not alias MMP buffers
  2013-08-29  0:43 ` [PATCH 1/6] libext2fs: ext2fs_dup_handle should not alias MMP buffers Darrick J. Wong
@ 2013-09-09 14:39   ` Theodore Ts'o
  0 siblings, 0 replies; 13+ messages in thread
From: Theodore Ts'o @ 2013-09-09 14:39 UTC (permalink / raw)
  To: Darrick J. Wong; +Cc: linux-ext4

On Wed, Aug 28, 2013 at 05:43:51PM -0700, Darrick J. Wong wrote:
> It turns out that resize2fs uses ext2fs_dup_handle to duplicate fs handles.  If
> MMP is enabled, this causes both handles to share MMP buffers, which is bad
> news when it comes time to free both handles.  Change the code to (we hope) fix
> this.  This prevents resize2fs from failing with a double-free error when
> handed a MMP filesystem.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Thanks, applied.

					- Ted

^ permalink raw reply	[flat|nested] 13+ messages in thread
* Re: [PATCH 2/6] resize2fs: Use blk64_t and location getters for free_gdp_blocks()
  2013-08-29  0:43 ` [PATCH 2/6] resize2fs: Use blk64_t and location getters for free_gdp_blocks() Darrick J. Wong
@ 2013-09-09 14:41   ` Theodore Ts'o
  0 siblings, 0 replies; 13+ messages in thread
From: Theodore Ts'o @ 2013-09-09 14:41 UTC (permalink / raw)
  To: Darrick J. Wong; +Cc: Eric Sandeen, linux-ext4

On Wed, Aug 28, 2013 at 05:43:57PM -0700, Darrick J. Wong wrote:
> free_gdp_blocks needs to be taught to use 64-bit fields and the appropriate
> getters, otherwise it'll truncate high block numbers (when, say, resizing a
> >16T fs) and mark the low numbered group descriptor blocks as free.  Yikes.
> 
> Reported-by: Eric Sandeen <sandeen@redhat.com>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Applied, thanks.

					- Ted

^ permalink raw reply	[flat|nested] 13+ messages in thread
* Re: [PATCH 6/6] resize2fs: Convert fs to and from 64bit mode
  2013-08-29  0:44 ` [PATCH 6/6] resize2fs: Convert fs to and from 64bit mode Darrick J. Wong
@ 2013-09-09 17:29   ` Darrick J. Wong
  2013-09-09 17:47     ` Theodore Ts'o
  0 siblings, 1 reply; 13+ messages in thread
From: Darrick J. Wong @ 2013-09-09 17:29 UTC (permalink / raw)
  To: tytso; +Cc: linux-ext4

Found some bugs in this; I'll repost the series (without the patches Ted just
merged) in a few days.

--D

On Wed, Aug 28, 2013 at 05:44:23PM -0700, Darrick J. Wong wrote:
> resize2fs does its magic by loading a filesystem, duplicating the in-memory
> image of that fs, moving relevant blocks out of the way of whatever new
> metadata get created, and finally writing everything back out to disk.
> Enabling 64bit mode enlarges the group descriptors, which makes resize2fs a
> reasonable vehicle for taking care of the rest of the bookkeeping requirements,
> so add to resize2fs the ability to convert a filesystem to 64bit mode and back.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
>  resize/main.c         |   22 ++++
>  resize/resize2fs.8.in |   18 +++
>  resize/resize2fs.c    |  264 ++++++++++++++++++++++++++++++++++++++++++++++++-
>  resize/resize2fs.h    |    3 +
>  4 files changed, 300 insertions(+), 7 deletions(-)
> 
> 
> diff --git a/resize/main.c b/resize/main.c
> index b648a15..f353351 100644
> --- a/resize/main.c
> +++ b/resize/main.c
> @@ -41,7 +41,7 @@ char *program_name, *device_name, *io_options;
>  static void usage (char *prog)
>  {
>  	fprintf (stderr, _("Usage: %s [-d debug_flags] [-f] [-F] [-M] [-P] "
> -			   "[-p] device [new_size]\n\n"), prog);
> +			   "[-p] device [-b|-s|new_size]\n\n"), prog);
>  
>  	exit (1);
>  }
> @@ -199,7 +199,7 @@ int main (int argc, char ** argv)
>  	if (argc && *argv)
>  		program_name = *argv;
>  
> -	while ((c = getopt (argc, argv, "d:fFhMPpS:")) != EOF) {
> +	while ((c = getopt(argc, argv, "d:fFhMPpS:bs")) != EOF) {
>  		switch (c) {
>  		case 'h':
>  			usage(program_name);
> @@ -225,6 +225,12 @@ int main (int argc, char ** argv)
>  		case 'S':
>  			use_stride = atoi(optarg);
>  			break;
> +		case 'b':
> +			flags |= RESIZE_ENABLE_64BIT;
> +			break;
> +		case 's':
> +			flags |= RESIZE_DISABLE_64BIT;
> +			break;
>  		default:
>  			usage(program_name);
>  		}
> @@ -434,7 +440,17 @@ int main (int argc, char ** argv)
>  			fs->blocksize / 1024, new_size);
>  		exit(1);
>  	}
> -	if (new_size == ext2fs_blocks_count(fs->super)) {
> +	if (flags & RESIZE_DISABLE_64BIT && flags & RESIZE_ENABLE_64BIT) {
> +		fprintf(stderr, _("Cannot set and unset 64bit feature.\n"));
> +		exit(1);
> +	} else if (flags & (RESIZE_DISABLE_64BIT | RESIZE_ENABLE_64BIT)) {
> +		new_size = ext2fs_blocks_count(fs->super);
> +		if (new_size >= (1ULL << 32)) {
> +			fprintf(stderr, _("Cannot change 64bit on a filesystem"
> +				" that is larger than 2^32 blocks.\n"));
> +			exit(1);
> +		}
> +	} else if (new_size == ext2fs_blocks_count(fs->super)) {
>  		fprintf(stderr, _("The filesystem is already %llu blocks "
>  			"long.  Nothing to do!\n\n"), new_size);
>  		exit(0);
> diff --git a/resize/resize2fs.8.in b/resize/resize2fs.8.in
> index 735fc91..d6deee6 100644
> --- a/resize/resize2fs.8.in
> +++ b/resize/resize2fs.8.in
> @@ -8,7 +8,7 @@ resize2fs \- ext2/ext3/ext4 file system resizer
>  .SH SYNOPSIS
>  .B resize2fs
>  [
> -.B \-fFpPM
> +.B \-fFpPMbs
>  ]
>  [
>  .B \-d
> @@ -85,8 +85,21 @@ to shrink the size of filesystem.  Then you may use
>  to shrink the size of the partition.  When shrinking the size of
>  the partition, make sure you do not make it smaller than the new size
>  of the ext2 filesystem!
> +.PP
> +The
> +.B \-b
> +and
> +.B \-s
> +options enable and disable the 64bit feature, respectively.  The resize2fs
> +program will, of course, take care of resizing the block group descriptors
> +and moving other data blocks out of the way, as needed.  It is not possible
> +to resize the filesystem concurrent with changing the 64bit status.
>  .SH OPTIONS
>  .TP
> +.B \-b
> +Turns on the 64bit feature, resizes the group descriptors as necessary, and
> +moves other metadata out of the way.
> +.TP
>  .B \-d \fIdebug-flags
>  Turns on various resize2fs debugging features, if they have been compiled
>  into the binary.
> @@ -124,6 +137,9 @@ of what the program is doing.
>  .B \-P
>  Print the minimum size of the filesystem and exit.
>  .TP
> +.B \-s
> +Turns off the 64bit feature and frees blocks that are no longer in use.
> +.TP
>  .B \-S \fIRAID-stride
>  The
>  .B resize2fs
> diff --git a/resize/resize2fs.c b/resize/resize2fs.c
> index 76d7aa6..fa7f99e 100644
> --- a/resize/resize2fs.c
> +++ b/resize/resize2fs.c
> @@ -53,6 +53,9 @@ static errcode_t ext2fs_calculate_summary_stats(ext2_filsys fs);
>  static errcode_t fix_sb_journal_backup(ext2_filsys fs);
>  static errcode_t mark_table_blocks(ext2_filsys fs,
>  				   ext2fs_block_bitmap bmap);
> +static errcode_t resize_group_descriptors(ext2_resize_t rfs, blk64_t new_size);
> +static errcode_t move_bg_metadata(ext2_resize_t rfs);
> +static errcode_t zero_high_bits_in_inodes(ext2_filsys fs);
>  
>  /*
>   * Some helper CPP macros
> @@ -119,13 +122,30 @@ errcode_t resize_fs(ext2_filsys fs, blk64_t *new_size, int flags,
>  	if (retval)
>  		goto errout;
>  
> +	init_resource_track(&rtrack, "resize_group_descriptors", fs->io);
> +	retval = resize_group_descriptors(rfs, *new_size);
> +	if (retval)
> +		goto errout;
> +	print_resource_track(rfs, &rtrack, fs->io);
> +
> +	init_resource_track(&rtrack, "move_bg_metadata", fs->io);
> +	retval = move_bg_metadata(rfs);
> +	if (retval)
> +		goto errout;
> +	print_resource_track(rfs, &rtrack, fs->io);
> +
> +	init_resource_track(&rtrack, "zero_high_bits_in_metadata", fs->io);
> +	retval = zero_high_bits_in_inodes(rfs->new_fs);
> +	if (retval)
> +		goto errout;
> +	print_resource_track(rfs, &rtrack, fs->io);
> +
>  	init_resource_track(&rtrack, "adjust_superblock", fs->io);
>  	retval = adjust_superblock(rfs, *new_size);
>  	if (retval)
>  		goto errout;
>  	print_resource_track(rfs, &rtrack, fs->io);
>  
> -
>  	init_resource_track(&rtrack, "fix_uninit_block_bitmaps 2", fs->io);
>  	fix_uninit_block_bitmaps(rfs->new_fs);
>  	print_resource_track(rfs, &rtrack, fs->io);
> @@ -221,6 +241,241 @@ errout:
>  	return retval;
>  }
>  
> +/* Toggle 64bit mode */
> +static errcode_t resize_group_descriptors(ext2_resize_t rfs, blk64_t new_size)
> +{
> +	void *o, *n, *new_group_desc;
> +	dgrp_t i;
> +	int copy_size;
> +	errcode_t retval;
> +
> +	if (!(rfs->flags & (RESIZE_DISABLE_64BIT | RESIZE_ENABLE_64BIT)))
> +		return 0;
> +
> +	if (new_size != ext2fs_blocks_count(rfs->new_fs->super) ||
> +	    ext2fs_blocks_count(rfs->new_fs->super) >= (1ULL << 32) ||
> +	    (rfs->flags & RESIZE_DISABLE_64BIT &&
> +	     rfs->flags & RESIZE_ENABLE_64BIT))
> +		return EXT2_ET_INVALID_ARGUMENT;
> +
> +	if (rfs->flags & RESIZE_DISABLE_64BIT) {
> +		rfs->new_fs->super->s_feature_incompat &=
> +				~EXT4_FEATURE_INCOMPAT_64BIT;
> +		rfs->new_fs->super->s_desc_size = EXT2_MIN_DESC_SIZE;
> +	} else if (rfs->flags & RESIZE_ENABLE_64BIT) {
> +		rfs->new_fs->super->s_feature_incompat |=
> +				EXT4_FEATURE_INCOMPAT_64BIT;
> +		rfs->new_fs->super->s_desc_size = EXT2_MIN_DESC_SIZE_64BIT;
> +	}
> +
> +	if (EXT2_DESC_SIZE(rfs->old_fs->super) ==
> +	    EXT2_DESC_SIZE(rfs->new_fs->super))
> +		return 0;
> +
> +	o = rfs->new_fs->group_desc;
> +	rfs->new_fs->desc_blocks = ext2fs_div_ceil(
> +			rfs->old_fs->group_desc_count,
> +			EXT2_DESC_PER_BLOCK(rfs->new_fs->super));
> +	retval = ext2fs_get_arrayzero(rfs->new_fs->desc_blocks,
> +				      rfs->old_fs->blocksize, &new_group_desc);
> +	if (retval)
> +		return retval;
> +
> +	n = new_group_desc;
> +
> +	if (EXT2_DESC_SIZE(rfs->old_fs->super) <=
> +	    EXT2_DESC_SIZE(rfs->new_fs->super))
> +		copy_size = EXT2_DESC_SIZE(rfs->old_fs->super);
> +	else
> +		copy_size = EXT2_DESC_SIZE(rfs->new_fs->super);
> +	for (i = 0; i < rfs->old_fs->group_desc_count; i++) {
> +		memcpy(n, o, copy_size);
> +		n += EXT2_DESC_SIZE(rfs->new_fs->super);
> +		o += EXT2_DESC_SIZE(rfs->old_fs->super);
> +	}
> +
> +	ext2fs_free_mem(&rfs->new_fs->group_desc);
> +	rfs->new_fs->group_desc = new_group_desc;
> +
> +	for (i = 0; i < rfs->old_fs->group_desc_count; i++)
> +		ext2fs_group_desc_csum_set(rfs->new_fs, i);
> +
> +	return 0;
> +}
> +
> +/* Move bitmaps/inode tables out of the way. */
> +static errcode_t move_bg_metadata(ext2_resize_t rfs)
> +{
> +	dgrp_t i;
> +	blk64_t b, c, d;
> +	ext2fs_block_bitmap old_map, new_map;
> +	int old, new;
> +	errcode_t retval;
> +	int zero = 0, one = 1;
> +
> +	if (!(rfs->flags & (RESIZE_DISABLE_64BIT | RESIZE_ENABLE_64BIT)))
> +		return 0;
> +
> +	retval = ext2fs_allocate_block_bitmap(rfs->old_fs, "oldfs", &old_map);
> +	if (retval)
> +		return retval;
> +
> +	retval = ext2fs_allocate_block_bitmap(rfs->new_fs, "newfs", &new_map);
> +	if (retval)
> +		goto out;
> +
> +	/* Construct bitmaps of super/descriptor blocks in old and new fs */
> +	for (i = 0; i < rfs->old_fs->group_desc_count; i++) {
> +		retval = ext2fs_super_and_bgd_loc2(rfs->old_fs, i, &b, &c, &d,
> +						   NULL);
> +		if (retval)
> +			goto out;
> +		ext2fs_mark_block_bitmap2(old_map, EXT2FS_B2C(rfs->old_fs, b));
> +		ext2fs_mark_block_bitmap2(old_map, EXT2FS_B2C(rfs->old_fs, c));
> +		ext2fs_mark_block_bitmap2(old_map, EXT2FS_B2C(rfs->old_fs, d));
> +
> +		retval = ext2fs_super_and_bgd_loc2(rfs->new_fs, i, &b, &c, &d,
> +						   NULL);
> +		if (retval)
> +			goto out;
> +		ext2fs_mark_block_bitmap2(new_map, EXT2FS_B2C(rfs->new_fs, b));
> +		ext2fs_mark_block_bitmap2(new_map, EXT2FS_B2C(rfs->new_fs, c));
> +		ext2fs_mark_block_bitmap2(new_map, EXT2FS_B2C(rfs->new_fs, d));
> +	}
> +
> +	/* Find changes in block allocations for bg metadata */
> +	for (b = 0; b < ext2fs_blocks_count(rfs->new_fs->super); b++) {
> +		c = EXT2FS_B2C(rfs->new_fs, b);
> +		old = ext2fs_test_block_bitmap2(old_map, c);
> +		new = ext2fs_test_block_bitmap2(new_map, c);
> +
> +		if (old && !new)
> +			ext2fs_unmark_block_bitmap2(rfs->new_fs->block_map, c);
> +		else if (!old && new)
> +			; /* empty ext2fs_mark_block_bitmap2(new_map, c); */
> +		else
> +			ext2fs_unmark_block_bitmap2(new_map, c);
> +	}
> +	/* new_map now shows blocks that have been newly allocated. */
> +
> +	/* Move any conflicting bitmaps and inode tables */
> +	for (i = 0; i < rfs->old_fs->group_desc_count; i++) {
> +		b = EXT2FS_B2C(rfs->new_fs,
> +			       ext2fs_block_bitmap_loc(rfs->new_fs, i));
> +		if (ext2fs_test_block_bitmap2(new_map, b))
> +			ext2fs_block_bitmap_loc_set(rfs->new_fs, i, 0);
> +
> +		b = EXT2FS_B2C(rfs->new_fs,
> +			       ext2fs_inode_bitmap_loc(rfs->new_fs, i));
> +		if (ext2fs_test_block_bitmap2(new_map, b))
> +			ext2fs_inode_bitmap_loc_set(rfs->new_fs, i, 0);
> +
> +		c = ext2fs_inode_table_loc(rfs->new_fs, i);
> +		for (b = 0; b < rfs->new_fs->inode_blocks_per_group; b++) {
> +			if (ext2fs_test_block_bitmap2(new_map, b + c)) {
> +				ext2fs_inode_table_loc_set(rfs->new_fs, i, 0);
> +				break;
> +			}
> +		}
> +	}
> +
> +out:
> +	if (old_map)
> +		ext2fs_free_block_bitmap(old_map);
> +	if (new_map)
> +		ext2fs_free_block_bitmap(new_map);
> +	return retval;
> +}
> +
> +/* Zero out the high bits of extent fields */
> +static errcode_t zero_high_bits_in_extents(ext2_filsys fs, ext2_ino_t ino,
> +				 struct ext2_inode *inode)
> +{
> +	ext2_extent_handle_t	handle;
> +	struct ext2fs_extent	extent;
> +	int			op = EXT2_EXTENT_ROOT;
> +	errcode_t		errcode;
> +
> +	if (!(inode->i_flags & EXT4_EXTENTS_FL))
> +		return 0;
> +
> +	errcode = ext2fs_extent_open(fs, ino, &handle);
> +	if (errcode)
> +		return errcode;
> +
> +	while (1) {
> +		errcode = ext2fs_extent_get(handle, op, &extent);
> +		if (errcode)
> +			break;
> +
> +		op = EXT2_EXTENT_NEXT_SIB;
> +
> +		extent.e_pblk &= 0xFFFFFFFF;
> +		errcode = ext2fs_extent_replace(handle, 0, &extent);
> +		if (errcode)
> +			break;
> +	}
> +
> +	/* Ok if we run off the end */
> +	if (errcode == EXT2_ET_EXTENT_NO_NEXT)
> +		errcode = 0;
> +	return errcode;
> +}
> +
> +/* Zero out the high bits of inodes. */
> +static errcode_t zero_high_bits_in_inodes(ext2_filsys fs)
> +{
> +	int length = EXT2_INODE_SIZE(fs->super);
> +	struct ext2_inode *inode = NULL;
> +	ext2_inode_scan	scan = NULL;
> +	errcode_t	retval;
> +	ext2_ino_t	ino;
> +	blk64_t		file_acl_block;
> +	__u32		sz_mask_high;
> +	int		inode_dirty;
> +
> +	if (fs->super->s_creator_os != EXT2_OS_LINUX)
> +		return 0;
> +
> +	retval = ext2fs_open_inode_scan(fs, 0, &scan);
> +	if (retval)
> +		return retval;
> +
> +	retval = ext2fs_get_mem(length, &inode);
> +	if (retval)
> +		goto out;
> +
> +	sz_mask_high = (((1ULL << 32) * EXT2_BLOCK_SIZE(fs->super)) - 1) >> 32;
> +	do {
> +		retval = ext2fs_get_next_inode_full(scan, &ino, inode, length);
> +		if (retval)
> +			goto out;
> +		if (!ino)
> +			break;
> +		if (!ext2fs_test_inode_bitmap2(fs->inode_map, ino))
> +			continue;
> +
> +		inode->i_size_high &= sz_mask_high;
> +		inode->osd2.linux2.l_i_blocks_hi = 0;
> +		inode->osd2.linux2.l_i_file_acl_high = 0;
> +
> +		retval = ext2fs_write_inode_full(fs, ino, inode,
> +						 length);
> +		if (retval)
> +			goto out;
> +
> +		retval = zero_high_bits_in_extents(fs, ino, inode);
> +		if (retval)
> +			goto out;
> +	} while (ino);
> +
> +out:
> +	if (inode)
> +		ext2fs_free_mem(&inode);
> +	if (scan)
> +		ext2fs_close_inode_scan(scan);
> +}
> +
>  /*
>   * Clean up the bitmaps for unitialized bitmaps
>   */
> @@ -424,7 +679,8 @@ retry:
>  	/*
>  	 * Reallocate the group descriptors as necessary.
>  	 */
> -	if (old_fs->desc_blocks != fs->desc_blocks) {
> +	if (EXT2_DESC_SIZE(old_fs->super) == EXT2_DESC_SIZE(fs->super) &&
> +	    old_fs->desc_blocks != fs->desc_blocks) {
>  		retval = ext2fs_resize_mem(old_fs->desc_blocks *
>  					   fs->blocksize,
>  					   fs->desc_blocks * fs->blocksize,
> @@ -939,7 +1195,9 @@ static errcode_t blocks_to_move(ext2_resize_t rfs)
>  		new_blocks = fs->desc_blocks + fs->super->s_reserved_gdt_blocks;
>  	}
>  
> -	if (old_blocks == new_blocks) {
> +	if (EXT2_DESC_SIZE(rfs->old_fs->super) ==
> +	    EXT2_DESC_SIZE(rfs->new_fs->super) &&
> +	    old_blocks == new_blocks) {
>  		retval = 0;
>  		goto errout;
>  	}
> diff --git a/resize/resize2fs.h b/resize/resize2fs.h
> index d425491..e24319d 100644
> --- a/resize/resize2fs.h
> +++ b/resize/resize2fs.h
> @@ -81,6 +81,9 @@ typedef struct ext2_sim_progress *ext2_sim_progmeter;
>  #define RESIZE_PERCENT_COMPLETE		0x0100
>  #define RESIZE_VERBOSE			0x0200
>  
> +#define RESIZE_ENABLE_64BIT		0x0400
> +#define RESIZE_DISABLE_64BIT		0x0800
> +
>  /*
>   * This structure is used for keeping track of how much resources have
>   * been used for a particular resize2fs pass.
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 13+ messages in thread
* Re: [PATCH 6/6] resize2fs: Convert fs to and from 64bit mode
  2013-09-09 17:29   ` Darrick J. Wong
@ 2013-09-09 17:47     ` Theodore Ts'o
  0 siblings, 0 replies; 13+ messages in thread
From: Theodore Ts'o @ 2013-09-09 17:47 UTC (permalink / raw)
  To: Darrick J. Wong; +Cc: linux-ext4

On Mon, Sep 09, 2013 at 10:29:44AM -0700, Darrick J. Wong wrote:
> Found some bugs in this; I'll repost the series (without the patches Ted just
> merged) in a few days.

Patches 1 and 2 were merged into the maint branch.  I will review and
merge patches 3-5 into the 1.43 branches (master/next), and I'll hold
off on patch #6.

Cheers,

						- Ted

^ permalink raw reply	[flat|nested] 13+ messages in thread
* Re: [PATCH 3/6] e2fsck: Fix incorrect bbitmap checksum failure caused by integer overflow
  2013-08-29  0:44 ` [PATCH 3/6] e2fsck: Fix incorrect bbitmap checksum failure caused by integer overflow Darrick J. Wong
@ 2013-09-16 13:42   ` Theodore Ts'o
  0 siblings, 0 replies; 13+ messages in thread
From: Theodore Ts'o @ 2013-09-16 13:42 UTC (permalink / raw)
  To: Darrick J. Wong; +Cc: linux-ext4

On Wed, Aug 28, 2013 at 05:44:04PM -0700, Darrick J. Wong wrote:
> On a filesystem with more than 2^32 blocks, the block group checksum test will
> fail because "i" (the group number) is a 32-bit quantity that is used to
> calculate the group's block bitmap block number.  Unfortunately, "i" is not
> automatically promoted to 64-bit for this calculation and overflows.  When this
> happens, e2fsck will incorrectly report bitmap checksum errors.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Applied to the next branch, thanks.

						- Ted

^ permalink raw reply	[flat|nested] 13+ messages in thread
* Re: [PATCH 5/6] tune2fs: Zero inode table when removing checksums
  2013-08-29  0:44 ` [PATCH 5/6] tune2fs: Zero inode table when removing checksums Darrick J. Wong
@ 2013-09-16 13:53   ` Theodore Ts'o
  0 siblings, 0 replies; 13+ messages in thread
From: Theodore Ts'o @ 2013-09-16 13:53 UTC (permalink / raw)
  To: Darrick J. Wong; +Cc: linux-ext4

On Wed, Aug 28, 2013 at 05:44:17PM -0700, Darrick J. Wong wrote:
> When disabling group checksums, we have to initialize the inode table.  Right
> now tune2fs doesn't do this; it merely punts to e2fsck to clean up the mess.
> Unfortunately, if the "uninitialized" inode table contains things that look
> like inodes (i_link_count > 0, specifically), the e2fsck tries to recover these
> inodes.  This leads to it misinterpreting i_blocks as a block map, at which
> point it needlessly resurrects phantom inodes and crosslinked file repairs.  As
> part of initializing the block bitmaps, we must also mark block group metadata
> blocks in use.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Applied, thanks.

					- Ted

^ permalink raw reply	[flat|nested] 13+ messages in thread
end of thread, other threads:[~2013-09-16 13:53 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-08-29  0:43 [RFC PATCH v1 0/6] e2fsprogs 8/2013 patchbomb Darrick J. Wong
2013-08-29  0:43 ` [PATCH 1/6] libext2fs: ext2fs_dup_handle should not alias MMP buffers Darrick J. Wong
2013-09-09 14:39   ` Theodore Ts'o
2013-08-29  0:43 ` [PATCH 2/6] resize2fs: Use blk64_t and location getters for free_gdp_blocks() Darrick J. Wong
2013-09-09 14:41   ` Theodore Ts'o
2013-08-29  0:44 ` [PATCH 3/6] e2fsck: Fix incorrect bbitmap checksum failure caused by integer overflow Darrick J. Wong
2013-09-16 13:42   ` Theodore Ts'o
2013-08-29  0:44 ` [PATCH 4/6] e2fsprogs: Add (optional) sparse checking to the build Darrick J. Wong
2013-08-29  0:44 ` [PATCH 5/6] tune2fs: Zero inode table when removing checksums Darrick J. Wong
2013-09-16 13:53   ` Theodore Ts'o
2013-08-29  0:44 ` [PATCH 6/6] resize2fs: Convert fs to and from 64bit mode Darrick J. Wong
2013-09-09 17:29   ` Darrick J. Wong
2013-09-09 17:47     ` Theodore Ts'o

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).