[PATCH 0/5] xfsprogs May 2015 patchbomb

[PATCH 0/5] xfsprogs May 2015 patchbomb

[Darrick J. Wong]

Hi all,

This is a rollup of various fixes for xfsprogs 3.2.3-rc1.

The first patch makes repair stop when it encounters v5 feature bits
which it doesn't understand.  Without it, an old xfs_repair will
trash a filesystem it doesn't know how to fix.

The second patch adds to repair some missing metadata back pointer
checks.

Patch 3 ensures that the .. link in a dir being rebuilt points
to a sane inode number.

Patch 4 implements blockget for v5 filesystems.  This is a second try
at a previous patch which didn't quite catch all the new magic numbers
and had some problems iterating directory index data.

Patch 5 fixes up blocktrash to avoid verifier errors when writing
deliberately corrupted blocks to disk.  It also adds the xfsfuzz.sh
script which will create and populate an XFS image, corrupts it,
then tries to modify/repair/re-modify the image.  I'm not sure this
is appropriate for xfstests, since runs until the kernel crashes or
repair fails to repair or the user ^Cs it.

I've tested these xfsprogs changes against the -next branch as of
5/11.

Comments and questions are, as always, welcome.

--D

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 1/5] xfs_repair: refuse to run if we don't recognize version or feature flags

[Darrick J. Wong]

Apparently xfs_repair running on a v5 filesystem doesn't check the
compat, rocompat, or incompat feature flags for bits that it doesn't
know about, which means that old xfs_repairs can wreak havoc.  So,
strengthen the checks to prevent repair from "repairing" anything it
doesn't understand.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 repair/versions.c |    7 +++++++
 1 file changed, 7 insertions(+)


diff --git a/repair/versions.c b/repair/versions.c
index c1dff72..e60574d 100644
--- a/repair/versions.c
+++ b/repair/versions.c
@@ -141,6 +141,13 @@ parse_sb_version(xfs_sb_t *sb)
 		}
 	}
 
+	/* Look for V5 feature flags we don't know about */
+	if (XFS_SB_VERSION_NUM(sb) >= XFS_SB_VERSION_5 &&
+	    (xfs_sb_has_ro_compat_feature(sb, XFS_SB_FEAT_RO_COMPAT_UNKNOWN) ||
+	     xfs_sb_has_incompat_feature(sb, XFS_SB_FEAT_INCOMPAT_UNKNOWN) ||
+	     xfs_sb_has_compat_feature(sb, XFS_SB_FEAT_COMPAT_UNKNOWN)))
+		issue_warning = 1;
+
 	if (issue_warning)  {
 		do_warn(
 _("This filesystem uses feature(s) not yet supported in this release.\n"

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 2/5] xfs_repair: better checking of v5 metadata fields

[Darrick J. Wong]

Check the UUID, owner, and block number fields during repair, looking for
blocks that fail either the checksum or the data structure verifiers.  For
directories we can simply rebuild corrupt/broken index data, though for
anything else we have to toss out the broken object.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 repair/dir2.c   |    7 ++
 repair/phase6.c |  159 +++++++++++++++++++++++++++++++++++++++++++++++++++++--
 repair/scan.c   |   17 ++++++
 3 files changed, 177 insertions(+), 6 deletions(-)


diff --git a/repair/dir2.c b/repair/dir2.c
index c6d618d..644c214 100644
--- a/repair/dir2.c
+++ b/repair/dir2.c
@@ -198,6 +198,13 @@ _("bad dir magic number 0x%x in inode %" PRIu64 " bno = %u\n"),
 					da_cursor->ino, bno);
 			goto error_out;
 		}
+		/* corrupt node; rebuild the dir. */
+		if (bp->b_error == EFSBADCRC || bp->b_error == EFSCORRUPTED) {
+			do_warn(
+_("corrupt tree block %u for directory inode %" PRIu64 "\n"),
+				bno, da_cursor->ino);
+			goto error_out;
+		}
 		btree = xfs_da3_node_tree_p(node);
 		if (nodehdr.count > mp->m_dir_node_ents)  {
 			libxfs_putbuf(bp);
diff --git a/repair/phase6.c b/repair/phase6.c
index c09b394..efefb2b 100644
--- a/repair/phase6.c
+++ b/repair/phase6.c
@@ -1981,6 +1981,41 @@ longform_dir2_check_leaf(
 		libxfs_putbuf(bp);
 		return 1;
 	}
+
+	/* check v5 metadata */
+	if (leafhdr.magic == XFS_DIR3_LEAF1_MAGIC) {
+		struct xfs_dir3_leaf_hdr *block3 = bp->b_addr;
+
+		/* verify owner */
+		if (be64_to_cpu(block3->info.owner) != ip->i_ino) {
+			do_warn(
+	_("expected owner inode %" PRIu64 ", got %llu, directory block %" PRIu64 "\n"),
+				ip->i_ino, be64_to_cpu(block3->info.owner),
+				bp->b_bn);
+			libxfs_putbuf(bp);
+			return 1;
+		}
+		/* verify block number */
+		if (be64_to_cpu(block3->info.blkno) != bp->b_bn) {
+			do_warn(
+	_("expected block %" PRIu64 ", got %llu, directory inode %" PRIu64 "\n"),
+				bp->b_bn,
+				be64_to_cpu(block3->info.blkno),
+				ip->i_ino);
+			libxfs_putbuf(bp);
+			return 1;
+		}
+		/* verify uuid */
+		if (platform_uuid_compare(&block3->info.uuid,
+					  &mp->m_sb.sb_uuid) != 0) {
+			do_warn(
+	_("wrong FS UUID, directory inode %" PRIu64 " block %" PRIu64 "\n"),
+				ip->i_ino, bp->b_bn);
+			libxfs_putbuf(bp);
+			return 1;
+		}
+	}
+
 	seeval = dir_hash_see_all(hashtab, ents, leafhdr.count, leafhdr.stale);
 	if (dir_hash_check(hashtab, ip, seeval)) {
 		libxfs_putbuf(bp);
@@ -2055,12 +2090,9 @@ longform_dir2_check_node(
 		xfs_dir3_leaf_hdr_from_disk(&leafhdr, leaf);
 		ents = xfs_dir3_leaf_ents_p(leaf);
 		if (!(leafhdr.magic == XFS_DIR2_LEAFN_MAGIC ||
-		      leafhdr.magic == XFS_DIR3_LEAFN_MAGIC)) {
-			if (leafhdr.magic == XFS_DA_NODE_MAGIC ||
-			    leafhdr.magic == XFS_DA3_NODE_MAGIC) {
-				libxfs_putbuf(bp);
-				continue;
-			}
+		      leafhdr.magic == XFS_DIR3_LEAFN_MAGIC ||
+		      leafhdr.magic == XFS_DA_NODE_MAGIC ||
+		      leafhdr.magic == XFS_DA3_NODE_MAGIC)) {
 			do_warn(
 	_("unknown magic number %#x for block %u in directory inode %" PRIu64 "\n"),
 				leafhdr.magic, da_bno, ip->i_ino);
@@ -2068,6 +2100,48 @@ longform_dir2_check_node(
 			return 1;
 		}
 
+		/* check v5 metadata */
+		if (leafhdr.magic == XFS_DIR3_LEAFN_MAGIC ||
+		    leafhdr.magic == XFS_DA3_NODE_MAGIC) {
+			struct xfs_da3_blkinfo	*info = bp->b_addr;
+
+			/* verify owner */
+			if (be64_to_cpu(info->owner) != ip->i_ino) {
+				do_warn(
+	_("expected owner inode %" PRIu64 ", got %llu, directory block %" PRIu64 "\n"),
+					ip->i_ino, be64_to_cpu(info->owner),
+					bp->b_bn);
+				libxfs_putbuf(bp);
+				return 1;
+			}
+			/* verify block number */
+			if (be64_to_cpu(info->blkno) != bp->b_bn) {
+				do_warn(
+	_("expected block %" PRIu64 ", got %llu, directory inode %" PRIu64 "\n"),
+					bp->b_bn,
+					be64_to_cpu(info->blkno),
+					ip->i_ino);
+				libxfs_putbuf(bp);
+				return 1;
+			}
+			/* verify uuid */
+			if (platform_uuid_compare(&info->uuid,
+						  &mp->m_sb.sb_uuid) != 0) {
+				do_warn(
+	_("wrong FS UUID, directory inode %" PRIu64 " block %" PRIu64 "\n"),
+					ip->i_ino, bp->b_bn);
+				libxfs_putbuf(bp);
+				return 1;
+			}
+		}
+
+		/* ignore nodes */
+		if (leafhdr.magic == XFS_DA_NODE_MAGIC ||
+		    leafhdr.magic == XFS_DA3_NODE_MAGIC) {
+			libxfs_putbuf(bp);
+			continue;
+		}
+
 		/*
 		 * If there's a validator error, we need to ensure that we got
 		 * the right ops on the buffer for when we write it back out.
@@ -2121,6 +2195,40 @@ longform_dir2_check_node(
 			libxfs_putbuf(bp);
 			return 1;
 		}
+
+		/* check v5 metadata */
+		if (freehdr.magic == XFS_DIR3_FREE_MAGIC) {
+			struct xfs_dir3_free_hdr *h = bp->b_addr;
+
+			/* verify owner */
+			if (be64_to_cpu(h->hdr.owner) != ip->i_ino) {
+				do_warn(
+	_("expected owner inode %" PRIu64 ", got %llu, directory block %" PRIu64 "\n"),
+					ip->i_ino, be64_to_cpu(h->hdr.owner),
+					bp->b_bn);
+				libxfs_putbuf(bp);
+				return 1;
+			}
+			/* verify block number */
+			if (be64_to_cpu(h->hdr.blkno) != bp->b_bn) {
+				do_warn(
+	_("expected block %" PRIu64 ", got %llu, directory inode %" PRIu64 "\n"),
+					bp->b_bn,
+					be64_to_cpu(h->hdr.blkno),
+					ip->i_ino);
+				libxfs_putbuf(bp);
+				return 1;
+			}
+			/* verify uuid */
+			if (platform_uuid_compare(&h->hdr.uuid,
+						  &mp->m_sb.sb_uuid) != 0) {
+				do_warn(
+	_("wrong FS UUID, directory inode %" PRIu64 " block %" PRIu64 "\n"),
+					ip->i_ino, bp->b_bn);
+				libxfs_putbuf(bp);
+				return 1;
+			}
+		}
 		for (i = used = 0; i < freehdr.nvalid; i++) {
 			if (i + freehdr.firstdb >= freetab->nents ||
 					freetab->ents[i + freehdr.firstdb].v !=
@@ -2212,6 +2320,7 @@ longform_dir2_entry_check(xfs_mount_t	*mp,
 	     da_bno = (xfs_dablk_t)next_da_bno) {
 		const struct xfs_buf_ops *ops;
 		int			 error;
+		struct xfs_dir2_data_hdr *d;
 
 		next_da_bno = da_bno + mp->m_dirblkfsbs - 1;
 		if (bmap_next_offset(NULL, ip, &next_da_bno, XFS_DATA_FORK)) {
@@ -2260,6 +2369,44 @@ longform_dir2_entry_check(xfs_mount_t	*mp,
 			}
 			continue;
 		}
+
+		/* check v5 metadata */
+		d = bplist[db]->b_addr;
+		if (be32_to_cpu(d->magic) == XFS_DIR3_BLOCK_MAGIC ||
+		    be32_to_cpu(d->magic) == XFS_DIR3_DATA_MAGIC) {
+			struct xfs_buf		 *bp = bplist[db];
+			struct xfs_dir3_data_hdr *block3 = bp->b_addr;
+
+			/* verify owner */
+			if (be64_to_cpu(block3->hdr.owner) != ino) {
+				do_warn(
+	_("expected owner inode %" PRIu64 ", got %llu, directory block %" PRIu64 "\n"),
+					ino, be64_to_cpu(block3->hdr.owner),
+					bp->b_bn);
+				fixit++;
+				continue;
+			}
+			/* verify block number */
+			if (be64_to_cpu(block3->hdr.blkno) != bp->b_bn) {
+				do_warn(
+	_("expected block %" PRIu64 ", got %llu, directory inode %" PRIu64 "\n"),
+					bp->b_bn,
+					be64_to_cpu(block3->hdr.blkno),
+					ino);
+				fixit++;
+				continue;
+			}
+			/* verify uuid */
+			if (platform_uuid_compare(&block3->hdr.uuid,
+						  &mp->m_sb.sb_uuid) != 0) {
+				do_warn(
+	_("wrong FS UUID, directory inode %" PRIu64 " block %" PRIu64 "\n"),
+					ino, bp->b_bn);
+				fixit++;
+				continue;
+			}
+		}
+
 		longform_dir2_entry_check_data(mp, ip, num_illegal, need_dot,
 				irec, ino_offset, &bplist[db], hashtab,
 				&freetab, da_bno, isblock);
diff --git a/repair/scan.c b/repair/scan.c
index 12aa782..79fbf83 100644
--- a/repair/scan.c
+++ b/repair/scan.c
@@ -227,6 +227,23 @@ _("expected owner inode %" PRIu64 ", got %llu, bmbt block %" PRIu64 "\n"),
 				ino, be64_to_cpu(block->bb_u.l.bb_owner), bno);
 			return(1);
 		}
+		/* verify block number */
+		if (be64_to_cpu(block->bb_u.l.bb_blkno) !=
+		    XFS_FSB_TO_DADDR(mp, bno)) {
+			do_warn(
+_("expected block %" PRIu64 ", got %llu, bmbt block %" PRIu64 "\n"),
+				XFS_FSB_TO_DADDR(mp, bno),
+				be64_to_cpu(block->bb_u.l.bb_blkno), bno);
+			return(1);
+		}
+		/* verify uuid */
+		if (platform_uuid_compare(&block->bb_u.l.bb_uuid,
+					  &mp->m_sb.sb_uuid) != 0) {
+			do_warn(
+_("wrong FS UUID, bmbt block %" PRIu64 "\n"),
+				bno);
+			return(1);
+		}
 	}
 
 	if (check_dups == 0)  {

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 3/5] xfs_repair: ensure .. is set to a sane ino value when rebuilding dir

[Darrick J. Wong]

When we're rebuilding a directory, ensure that we reinitialize the
directory with a sane parent ('..') inode value.  If we don't, the
error return from xfs_dir_init() is ignored, and the rebuild process
becomes confused and leaves the directory corrupt.  If repair later
discovers that the rebuilt directory is an orphan, it'll try to attach
it to lost+found and abort on the corrupted directory.  Also fix
ignoring the return value of xfs_dir_init().

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 include/xfs_dir2.h     |    2 ++
 libxfs/xfs_dir2_priv.h |    1 -
 repair/phase6.c        |    9 +++++++--
 3 files changed, 9 insertions(+), 3 deletions(-)


diff --git a/include/xfs_dir2.h b/include/xfs_dir2.h
index 3900130..2d41c5f 100644
--- a/include/xfs_dir2.h
+++ b/include/xfs_dir2.h
@@ -100,6 +100,8 @@ extern void xfs_dir2_data_use_free(struct xfs_trans *tp, struct xfs_buf *bp,
 extern struct xfs_dir2_data_free *xfs_dir2_data_freefind(
 		struct xfs_dir2_data_hdr *hdr, struct xfs_dir2_data_unused *dup);
 
+extern int xfs_dir_ino_validate(struct xfs_mount *mp, xfs_ino_t ino);
+
 extern const struct xfs_buf_ops xfs_dir3_block_buf_ops;
 extern const struct xfs_buf_ops xfs_dir3_leafn_buf_ops;
 extern const struct xfs_buf_ops xfs_dir3_leaf1_buf_ops;
diff --git a/libxfs/xfs_dir2_priv.h b/libxfs/xfs_dir2_priv.h
index 1bad84c..926715f 100644
--- a/libxfs/xfs_dir2_priv.h
+++ b/libxfs/xfs_dir2_priv.h
@@ -21,7 +21,6 @@
 struct dir_context;
 
 /* xfs_dir2.c */
-extern int xfs_dir_ino_validate(struct xfs_mount *mp, xfs_ino_t ino);
 extern int xfs_dir2_grow_inode(struct xfs_da_args *args, int space,
 				xfs_dir2_db_t *dbp);
 extern int xfs_dir_cilookup_result(struct xfs_da_args *args,
diff --git a/repair/phase6.c b/repair/phase6.c
index efefb2b..145e497 100644
--- a/repair/phase6.c
+++ b/repair/phase6.c
@@ -1321,7 +1321,8 @@ longform_dir2_rebuild(
 	 * for the libxfs_dir_init() call).
 	 */
 	pip.i_ino = get_inode_parent(irec, ino_offset);
-	if (pip.i_ino == NULLFSINO)
+	if (pip.i_ino == NULLFSINO ||
+	    xfs_dir_ino_validate(mp, pip.i_ino))
 		pip.i_ino = mp->m_sb.sb_rootino;
 
 	xfs_bmap_init(&flist, &firstblock);
@@ -1348,7 +1349,11 @@ longform_dir2_rebuild(
 
 	ASSERT(done);
 
-	libxfs_dir_init(tp, ip, &pip);
+	error = libxfs_dir_init(tp, ip, &pip);
+	if (error) {
+		do_warn(_("xfs_dir_init failed -- error - %d\n"), error);
+		goto out_bmap_cancel;
+	}
 
 	error = libxfs_bmap_finish(&tp, &flist, &committed);
 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 4/5] xfs_db: enable blockget for v5 filesystems

[Darrick J. Wong]

Plumb in the necessary magic number checks and other fixups required
to handle v5 filesystems.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 db/check.c |  225 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++----
 db/type.c  |    3 +
 db/type.h  |    2 -
 3 files changed, 214 insertions(+), 16 deletions(-)


diff --git a/db/check.c b/db/check.c
index c4c972f..8f8096d 100644
--- a/db/check.c
+++ b/db/check.c
@@ -44,7 +44,7 @@ typedef enum {
 	DBM_FREE1,	DBM_FREE2,	DBM_FREELIST,	DBM_INODE,
 	DBM_LOG,	DBM_MISSING,	DBM_QUOTA,	DBM_RTBITMAP,
 	DBM_RTDATA,	DBM_RTFREE,	DBM_RTSUM,	DBM_SB,
-	DBM_SYMLINK,
+	DBM_SYMLINK,	DBM_BTFINO,
 	DBM_NDBM
 } dbm_t;
 
@@ -170,6 +170,7 @@ static const char	*typename[] = {
 	"rtsum",
 	"sb",
 	"symlink",
+	"btfino",
 	NULL
 };
 static int		verbose;
@@ -345,6 +346,9 @@ static void		scanfunc_cnt(struct xfs_btree_block *block, int level,
 static void		scanfunc_ino(struct xfs_btree_block *block, int level,
 				     xfs_agf_t *agf, xfs_agblock_t bno,
 				     int isroot);
+static void		scanfunc_fino(struct xfs_btree_block *block, int level,
+				     xfs_agf_t *agf, xfs_agblock_t bno,
+				     int isroot);
 static void		set_dbmap(xfs_agnumber_t agno, xfs_agblock_t agbno,
 				  xfs_extlen_t len, dbm_t type,
 				  xfs_agnumber_t c_agno, xfs_agblock_t c_agbno);
@@ -799,8 +803,6 @@ blockget_f(
 	 * filters out, or we need to actually do the work to make check support
 	 * crc enabled filesystems.
 	 */
-	if (xfs_sb_version_hascrc(&mp->m_sb))
-		return 0;
 
 	if (!init(argc, argv)) {
 		if (serious_error)
@@ -2223,7 +2225,9 @@ process_data_dir_v2(
 	data = iocur_top->data;
 	block = iocur_top->data;
 	if (be32_to_cpu(block->magic) != XFS_DIR2_BLOCK_MAGIC &&
-			be32_to_cpu(data->magic) != XFS_DIR2_DATA_MAGIC) {
+			be32_to_cpu(data->magic) != XFS_DIR2_DATA_MAGIC &&
+			be32_to_cpu(block->magic) != XFS_DIR3_BLOCK_MAGIC &&
+			be32_to_cpu(data->magic) != XFS_DIR3_DATA_MAGIC) {
 		if (!sflag || v)
 			dbprintf(_("bad directory data magic # %#x for dir ino "
 				 "%lld block %d\n"),
@@ -2234,7 +2238,8 @@ process_data_dir_v2(
 	db = xfs_dir2_da_to_db(mp, dabno);
 	bf = xfs_dir3_data_bestfree_p(data);
 	ptr = (char *)xfs_dir3_data_unused_p(data);
-	if (be32_to_cpu(block->magic) == XFS_DIR2_BLOCK_MAGIC) {
+	if (be32_to_cpu(block->magic) == XFS_DIR2_BLOCK_MAGIC ||
+	    be32_to_cpu(block->magic) == XFS_DIR3_BLOCK_MAGIC) {
 		btp = xfs_dir2_block_tail_p(mp, block);
 		lep = xfs_dir2_block_leaf_p(btp);
 		endptr = (char *)lep;
@@ -2380,7 +2385,8 @@ process_data_dir_v2(
 			(*dot)++;
 		}
 	}
-	if (be32_to_cpu(data->magic) == XFS_DIR2_BLOCK_MAGIC) {
+	if (be32_to_cpu(data->magic) == XFS_DIR2_BLOCK_MAGIC ||
+	    be32_to_cpu(data->magic) == XFS_DIR3_BLOCK_MAGIC) {
 		endptr = (char *)data + mp->m_dirblksize;
 		for (i = stale = 0; lep && i < be32_to_cpu(btp->count); i++) {
 			if ((char *)&lep[i] >= endptr) {
@@ -2412,7 +2418,8 @@ process_data_dir_v2(
 				id->ino, dabno);
 		error++;
 	}
-	if (be32_to_cpu(data->magic) == XFS_DIR2_BLOCK_MAGIC &&
+	if ((be32_to_cpu(data->magic) == XFS_DIR2_BLOCK_MAGIC ||
+	     be32_to_cpu(data->magic) == XFS_DIR3_BLOCK_MAGIC) &&
 	    count != be32_to_cpu(btp->count) - be32_to_cpu(btp->stale)) {
 		if (!sflag || v)
 			dbprintf(_("dir %lld block %d bad block tail count %d "
@@ -2421,7 +2428,8 @@ process_data_dir_v2(
 				be32_to_cpu(btp->stale));
 		error++;
 	}
-	if (be32_to_cpu(data->magic) == XFS_DIR2_BLOCK_MAGIC && 
+	if ((be32_to_cpu(data->magic) == XFS_DIR2_BLOCK_MAGIC ||
+	     be32_to_cpu(data->magic) == XFS_DIR2_BLOCK_MAGIC) &&
 					stale != be32_to_cpu(btp->stale)) {
 		if (!sflag || v)
 			dbprintf(_("dir %lld block %d bad stale tail count %d\n"),
@@ -3007,6 +3015,73 @@ process_leaf_node_dir_v2(
 }
 
 static void
+process_leaf_node_dir_v3_free(
+	inodata_t		*id,
+	int			v,
+	xfs_dablk_t		dabno,
+	freetab_t		*freetab)
+{
+	xfs_dir2_data_off_t	ent;
+	struct xfs_dir3_free	*free;
+	int			i;
+	int			maxent;
+	int			used;
+
+	free = iocur_top->data;
+	maxent = xfs_dir3_free_max_bests(mp);
+	if (be32_to_cpu(free->hdr.firstdb) != xfs_dir2_da_to_db(mp,
+					dabno - mp->m_dirfreeblk) * maxent) {
+		if (!sflag || v)
+			dbprintf(_("bad free block firstdb %d for dir ino %lld "
+				 "block %d\n"),
+				be32_to_cpu(free->hdr.firstdb), id->ino, dabno);
+		error++;
+		return;
+	}
+	if (be32_to_cpu(free->hdr.nvalid) > maxent ||
+				be32_to_cpu(free->hdr.nvalid) < 0 ||
+				be32_to_cpu(free->hdr.nused) > maxent ||
+				be32_to_cpu(free->hdr.nused) < 0 ||
+				be32_to_cpu(free->hdr.nused) >
+					be32_to_cpu(free->hdr.nvalid)) {
+		if (!sflag || v)
+			dbprintf(_("bad free block nvalid/nused %d/%d for dir "
+				 "ino %lld block %d\n"),
+				be32_to_cpu(free->hdr.nvalid),
+				be32_to_cpu(free->hdr.nused), id->ino, dabno);
+		error++;
+		return;
+	}
+	for (used = i = 0; i < be32_to_cpu(free->hdr.nvalid); i++) {
+		if (freetab->nents <= be32_to_cpu(free->hdr.firstdb) + i)
+			ent = NULLDATAOFF;
+		else
+			ent = freetab->ents[be32_to_cpu(free->hdr.firstdb) + i];
+		if (ent != be16_to_cpu(free->bests[i])) {
+			if (!sflag || v)
+				dbprintf(_("bad free block ent %d is %d should "
+					 "be %d for dir ino %lld block %d\n"),
+					i, be16_to_cpu(free->bests[i]), ent,
+					id->ino, dabno);
+			error++;
+		}
+		if (be16_to_cpu(free->bests[i]) != NULLDATAOFF)
+			used++;
+		if (ent != NULLDATAOFF)
+			freetab->ents[be32_to_cpu(free->hdr.firstdb) + i] =
+								NULLDATAOFF;
+	}
+	if (used != be32_to_cpu(free->hdr.nused)) {
+		if (!sflag || v)
+			dbprintf(_("bad free block nused %d should be %d for dir "
+				 "ino %lld block %d\n"),
+				be32_to_cpu(free->hdr.nused), used, id->ino,
+				dabno);
+		error++;
+	}
+}
+
+static void
 process_leaf_node_dir_v2_free(
 	inodata_t		*id,
 	int			v,
@@ -3020,7 +3095,8 @@ process_leaf_node_dir_v2_free(
 	int			used;
 
 	free = iocur_top->data;
-	if (be32_to_cpu(free->hdr.magic) != XFS_DIR2_FREE_MAGIC) {
+	if (be32_to_cpu(free->hdr.magic) != XFS_DIR2_FREE_MAGIC &&
+	    be32_to_cpu(free->hdr.magic) != XFS_DIR3_FREE_MAGIC) {
 		if (!sflag || v)
 			dbprintf(_("bad free block magic # %#x for dir ino %lld "
 				 "block %d\n"),
@@ -3028,6 +3104,10 @@ process_leaf_node_dir_v2_free(
 		error++;
 		return;
 	}
+	if (be32_to_cpu(free->hdr.magic) == XFS_DIR3_FREE_MAGIC) {
+		process_leaf_node_dir_v3_free(id, v, dabno, freetab);
+		return;
+	}
 	maxent = xfs_dir3_free_max_bests(mp);
 	if (be32_to_cpu(free->hdr.firstdb) != xfs_dir2_da_to_db(mp, 
 					dabno - mp->m_dirfreeblk) * maxent) {
@@ -3081,6 +3161,20 @@ process_leaf_node_dir_v2_free(
 	}
 }
 
+/*
+ * Get address of the bestcount field in the single-leaf block.
+ */
+static inline int
+xfs_dir3_leaf_ents_count(struct xfs_dir2_leaf *lp)
+{
+	if (lp->hdr.info.magic == cpu_to_be16(XFS_DIR3_LEAF1_MAGIC) ||
+	    lp->hdr.info.magic == cpu_to_be16(XFS_DIR3_LEAFN_MAGIC)) {
+		struct xfs_dir3_leaf *lp3 = (struct xfs_dir3_leaf *)lp;
+		return be16_to_cpu(lp3->hdr.count);
+	}
+	return be16_to_cpu(lp->hdr.count);
+}
+
 static void
 process_leaf_node_dir_v2_int(
 	inodata_t		*id,
@@ -3091,6 +3185,7 @@ process_leaf_node_dir_v2_int(
 	int			i;
 	__be16			*lbp;
 	xfs_dir2_leaf_t		*leaf;
+	struct xfs_dir3_leaf	*leaf3 = NULL;
 	xfs_dir2_leaf_entry_t	*lep;
 	xfs_dir2_leaf_tail_t	*ltp;
 	xfs_da_intnode_t	*node;
@@ -3099,7 +3194,15 @@ process_leaf_node_dir_v2_int(
 
 	leaf = iocur_top->data;
 	switch (be16_to_cpu(leaf->hdr.info.magic)) {
+	case XFS_DIR3_LEAF1_MAGIC:
+	case XFS_DIR3_LEAFN_MAGIC:
+	case XFS_DA3_NODE_MAGIC:
+		leaf3 = iocur_top->data;
+		break;
+	}
+	switch (be16_to_cpu(leaf->hdr.info.magic)) {
 	case XFS_DIR2_LEAF1_MAGIC:
+	case XFS_DIR3_LEAF1_MAGIC:
 		if (be32_to_cpu(leaf->hdr.info.forw) || 
 					be32_to_cpu(leaf->hdr.info.back)) {
 			if (!sflag || v)
@@ -3139,10 +3242,12 @@ process_leaf_node_dir_v2_int(
 		}
 		break;
 	case XFS_DIR2_LEAFN_MAGIC:
+	case XFS_DIR3_LEAFN_MAGIC:
 		/* if it's at the root location then we can check the
 		 * pointers are null XXX */
 		break;
 	case XFS_DA_NODE_MAGIC:
+	case XFS_DA3_NODE_MAGIC:
 		node = iocur_top->data;
 		xfs_da3_node_hdr_from_disk(&nodehdr, node);
 		if (nodehdr.level < 1 || nodehdr.level > XFS_DA_NODE_MAXDEPTH) {
@@ -3164,7 +3269,7 @@ process_leaf_node_dir_v2_int(
 		return;
 	}
 	lep = xfs_dir3_leaf_ents_p(leaf);
-	for (i = stale = 0; i < be16_to_cpu(leaf->hdr.count); i++) {
+	for (i = stale = 0; i < xfs_dir3_leaf_ents_count(leaf); i++) {
 		if (be32_to_cpu(lep[i].address) == XFS_DIR2_NULL_DATAPTR)
 			stale++;
 		else if (dir_hash_see(be32_to_cpu(lep[i].hashval), 
@@ -3177,7 +3282,14 @@ process_leaf_node_dir_v2_int(
 			error++;
 		}
 	}
-	if (stale != be16_to_cpu(leaf->hdr.stale)) {
+	if (leaf3 && stale != be16_to_cpu(leaf3->hdr.stale)) {
+		if (!sflag || v)
+			dbprintf(_("dir3 %lld block %d stale mismatch "
+				 "%d/%d\n"),
+				 id->ino, dabno, stale,
+				 be16_to_cpu(leaf3->hdr.stale));
+		error++;
+	} else if (!leaf && stale != be16_to_cpu(leaf->hdr.stale)) {
 		if (!sflag || v)
 			dbprintf(_("dir %lld block %d stale mismatch "
 				 "%d/%d\n"),
@@ -3764,6 +3876,12 @@ scan_ag(
 		be32_to_cpu(agi->agi_root),
 		be32_to_cpu(agi->agi_level),
 		1, scanfunc_ino, TYP_INOBT);
+	if (agi->agi_free_root) {
+		scan_sbtree(agf,
+			be32_to_cpu(agi->agi_free_root),
+			be32_to_cpu(agi->agi_free_level),
+			1, scanfunc_fino, TYP_FINOBT);
+	}
 	if (be32_to_cpu(agf->agf_freeblks) != agffreeblks) {
 		if (!sflag)
 			dbprintf(_("agf_freeblks %u, counted %u in ag %u\n"),
@@ -3963,7 +4081,8 @@ scanfunc_bmap(
 
 	agno = XFS_FSB_TO_AGNO(mp, bno);
 	agbno = XFS_FSB_TO_AGBNO(mp, bno);
-	if (be32_to_cpu(block->bb_magic) != XFS_BMAP_MAGIC) {
+	if (be32_to_cpu(block->bb_magic) != XFS_BMAP_MAGIC &&
+	    be32_to_cpu(block->bb_magic) != XFS_BMAP_CRC_MAGIC) {
 		if (!sflag || id->ilist || CHECK_BLIST(bno))
 			dbprintf(_("bad magic # %#x in inode %lld bmbt block "
 				 "%u/%u\n"),
@@ -4028,7 +4147,8 @@ scanfunc_bno(
 	xfs_agnumber_t		seqno = be32_to_cpu(agf->agf_seqno);
 	xfs_agblock_t		lastblock;
 
-	if (be32_to_cpu(block->bb_magic) != XFS_ABTB_MAGIC) {
+	if (be32_to_cpu(block->bb_magic) != XFS_ABTB_MAGIC &&
+	    be32_to_cpu(block->bb_magic) != XFS_ABTB_CRC_MAGIC) {
 		dbprintf(_("bad magic # %#x in btbno block %u/%u\n"),
 			be32_to_cpu(block->bb_magic), seqno, bno);
 		serious_error++;
@@ -4101,7 +4221,8 @@ scanfunc_cnt(
 	xfs_alloc_rec_t		*rp;
 	xfs_extlen_t		lastcount;
 
-	if (be32_to_cpu(block->bb_magic) != XFS_ABTC_MAGIC) {
+	if (be32_to_cpu(block->bb_magic) != XFS_ABTC_MAGIC &&
+	    be32_to_cpu(block->bb_magic) != XFS_ABTC_CRC_MAGIC) {
 		dbprintf(_("bad magic # %#x in btcnt block %u/%u\n"),
 			be32_to_cpu(block->bb_magic), seqno, bno);
 		serious_error++;
@@ -4181,7 +4302,8 @@ scanfunc_ino(
 	xfs_inobt_ptr_t		*pp;
 	xfs_inobt_rec_t		*rp;
 
-	if (be32_to_cpu(block->bb_magic) != XFS_IBT_MAGIC) {
+	if (be32_to_cpu(block->bb_magic) != XFS_IBT_MAGIC &&
+	    be32_to_cpu(block->bb_magic) != XFS_IBT_CRC_MAGIC) {
 		dbprintf(_("bad magic # %#x in inobt block %u/%u\n"),
 			be32_to_cpu(block->bb_magic), seqno, bno);
 		serious_error++;
@@ -4277,6 +4399,79 @@ scanfunc_ino(
 }
 
 static void
+scanfunc_fino(
+	struct xfs_btree_block	*block,
+	int			level,
+	xfs_agf_t		*agf,
+	xfs_agblock_t		bno,
+	int			isroot)
+{
+	xfs_agino_t		agino;
+	xfs_agnumber_t		seqno = be32_to_cpu(agf->agf_seqno);
+	int			i;
+	int			off;
+	xfs_inobt_ptr_t		*pp;
+	xfs_inobt_rec_t		*rp;
+
+	if (be32_to_cpu(block->bb_magic) != XFS_FIBT_MAGIC &&
+	    be32_to_cpu(block->bb_magic) != XFS_FIBT_CRC_MAGIC) {
+		dbprintf(_("bad magic # %#x in finobt block %u/%u\n"),
+			be32_to_cpu(block->bb_magic), seqno, bno);
+		serious_error++;
+		return;
+	}
+	if (be16_to_cpu(block->bb_level) != level) {
+		if (!sflag)
+			dbprintf(_("expected level %d got %d in finobt block "
+				 "%u/%u\n"),
+				level, be16_to_cpu(block->bb_level), seqno, bno);
+		error++;
+	}
+	set_dbmap(seqno, bno, 1, DBM_BTFINO, seqno, bno);
+	if (level == 0) {
+		if (be16_to_cpu(block->bb_numrecs) > mp->m_inobt_mxr[0] ||
+		    (isroot == 0 && be16_to_cpu(block->bb_numrecs) < mp->m_inobt_mnr[0])) {
+			dbprintf(_("bad btree nrecs (%u, min=%u, max=%u) in "
+				 "finobt block %u/%u\n"),
+				be16_to_cpu(block->bb_numrecs), mp->m_inobt_mnr[0],
+				mp->m_inobt_mxr[0], seqno, bno);
+			serious_error++;
+			return;
+		}
+		rp = XFS_INOBT_REC_ADDR(mp, block, 1);
+		for (i = 0; i < be16_to_cpu(block->bb_numrecs); i++) {
+			agino = be32_to_cpu(rp[i].ir_startino);
+			off = XFS_INO_TO_OFFSET(mp, agino);
+			if (off == 0) {
+				if ((sbversion & XFS_SB_VERSION_ALIGNBIT) &&
+				    mp->m_sb.sb_inoalignmt &&
+				    (XFS_INO_TO_AGBNO(mp, agino) %
+				     mp->m_sb.sb_inoalignmt))
+					sbversion &= ~XFS_SB_VERSION_ALIGNBIT;
+				check_set_dbmap(seqno, XFS_AGINO_TO_AGBNO(mp, agino),
+					(xfs_extlen_t)MAX(1,
+						XFS_INODES_PER_CHUNK >>
+						mp->m_sb.sb_inopblog),
+					DBM_INODE, DBM_INODE, seqno, bno);
+			}
+		}
+		return;
+	}
+	if (be16_to_cpu(block->bb_numrecs) > mp->m_inobt_mxr[1] ||
+	    (isroot == 0 && be16_to_cpu(block->bb_numrecs) < mp->m_inobt_mnr[1])) {
+		dbprintf(_("bad btree nrecs (%u, min=%u, max=%u) in finobt block "
+			 "%u/%u\n"),
+			be16_to_cpu(block->bb_numrecs), mp->m_inobt_mnr[1],
+			mp->m_inobt_mxr[1], seqno, bno);
+		serious_error++;
+		return;
+	}
+	pp = XFS_INOBT_PTR_ADDR(mp, block, 1, mp->m_inobt_mxr[1]);
+	for (i = 0; i < be16_to_cpu(block->bb_numrecs); i++)
+		scan_sbtree(agf, be32_to_cpu(pp[i]), level, 0, scanfunc_ino, TYP_FINOBT);
+}
+
+static void
 set_dbmap(
 	xfs_agnumber_t	agno,
 	xfs_agblock_t	agbno,
diff --git a/db/type.c b/db/type.c
index b29f2a4..0aa3137 100644
--- a/db/type.c
+++ b/db/type.c
@@ -70,6 +70,7 @@ static const typ_t	__typtab[] = {
 	{ TYP_SB, "sb", handle_struct, sb_hfld, NULL },
 	{ TYP_SYMLINK, "symlink", handle_string, NULL, NULL },
 	{ TYP_TEXT, "text", handle_text, NULL, NULL },
+	{ TYP_FINOBT, "finobt", handle_struct, inobt_hfld, NULL },
 	{ TYP_NONE, NULL }
 };
 
@@ -104,6 +105,8 @@ static const typ_t	__typtab_crc[] = {
 	{ TYP_SYMLINK, "symlink", handle_struct, symlink_crc_hfld,
 		&xfs_symlink_buf_ops },
 	{ TYP_TEXT, "text", handle_text, NULL, NULL },
+	{ TYP_FINOBT, "finobt", handle_struct, inobt_crc_hfld,
+		&xfs_inobt_buf_ops },
 	{ TYP_NONE, NULL }
 };
 
diff --git a/db/type.h b/db/type.h
index 3bb26f1..e8d8df7 100644
--- a/db/type.h
+++ b/db/type.h
@@ -27,7 +27,7 @@ typedef enum typnm
 	TYP_BMAPBTD, TYP_BNOBT, TYP_CNTBT, TYP_DATA,
 	TYP_DIR2, TYP_DQBLK, TYP_INOBT, TYP_INODATA, TYP_INODE,
 	TYP_LOG, TYP_RTBITMAP, TYP_RTSUMMARY, TYP_SB, TYP_SYMLINK,
-	TYP_TEXT, TYP_NONE
+	TYP_TEXT, TYP_FINOBT, TYP_NONE
 } typnm_t;
 
 #define DB_WRITE 1

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 5/5] xfs_db: enable blocktrash for checksummed filesystems

[Darrick J. Wong]

Disable the write verifiers when we're trashing a block.  With this
in place, create a xfs fuzzer script that formats, populates, corrupts,
tries to use, repairs, and tries again to use a crash test xfs image.
Hopefully this will shake out some v5 filesystem bugs.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 db/check.c    |    7 +
 db/xfsfuzz.sh |  305 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 312 insertions(+)
 create mode 100755 db/xfsfuzz.sh


diff --git a/db/check.c b/db/check.c
index 8f8096d..2c2a02e 100644
--- a/db/check.c
+++ b/db/check.c
@@ -953,6 +953,7 @@ blocktrash_b(
 	int		mask;
 	int		newbit;
 	int		offset;
+	const struct xfs_buf_ops *stashed_ops;
 	static char	*modestr[] = {
 		N_("zeroed"), N_("set"), N_("flipped"), N_("randomized")
 	};
@@ -963,6 +964,8 @@ blocktrash_b(
 	push_cur();
 	set_cur(&typtab[DBM_UNKNOWN],
 		XFS_AGB_TO_DADDR(mp, agno, agbno), blkbb, DB_RING_IGN, NULL);
+	stashed_ops = iocur_top->bp->b_ops;
+	iocur_top->bp->b_ops = NULL;
 	if ((buf = iocur_top->data) == NULL) {
 		dbprintf(_("can't read block %u/%u for trashing\n"), agno, agbno);
 		pop_cur();
@@ -993,6 +996,7 @@ blocktrash_b(
 			buf[byte] &= ~mask;
 	}
 	write_cur();
+	iocur_top->bp->b_ops = stashed_ops;
 	pop_cur();
 	printf(_("blocktrash: %u/%u %s block %d bit%s starting %d:%d %s\n"),
 		agno, agbno, typename[type], len, len == 1 ? "" : "s",
@@ -1049,9 +1053,12 @@ blocktrash_f(
 		   (1 << DBM_BTINO) |
 		   (1 << DBM_DIR) |
 		   (1 << DBM_INODE) |
+		   (1 << DBM_LOG) |
 		   (1 << DBM_QUOTA) |
 		   (1 << DBM_RTBITMAP) |
 		   (1 << DBM_RTSUM) |
+		   (1 << DBM_SYMLINK) |
+		   (1 << DBM_BTFINO) |
 		   (1 << DBM_SB);
 	while ((c = getopt(argc, argv, "0123n:s:t:x:y:")) != EOF) {
 		switch (c) {
diff --git a/db/xfsfuzz.sh b/db/xfsfuzz.sh
new file mode 100755
index 0000000..fc40e97
--- /dev/null
+++ b/db/xfsfuzz.sh
@@ -0,0 +1,305 @@
+#!/bin/bash
+
+# Test harness to fuzz a filesystem over and over...
+# Copyright (C) 2014 Oracle.
+
+DIR=/tmp
+PASSES=10000
+SZ=32m
+SCRIPT_DIR="$(dirname "$0")"
+FEATURES="-m crc=1,finobt=1"
+xEATURES="-m crc=0"
+BLK_SZ=4096
+INODE_SZ=512
+RUN_FSCK=1
+OVERRIDE_PATH=1
+MAX_FSCK=10
+SRCDIR=/etc
+FUZZ_ARGS="-3 -n 32"
+XFS_REPAIR_OPTS="-P"
+
+print_help() {
+	echo "Usage: $0 OPTIONS"
+	echo "-b:	FS block size is this. (${BLK_SZ})"
+	echo "-B:	Corrupt this many bytes per run."
+	echo "-d:	Create test files in this directory. (${DIR})"
+	echo "-f:	Do not run xfs_repair after each pass."
+	echo "-I:	Create inodes of this size. (${INODE_SZ})"
+	echo "-n:	Run this many passes. (${PASSES})"
+	echo "-O:	Pass this to mkfs.xfs."
+	echo "-p:	Use system's xfsprogs tools."
+	echo "-s:	Create FS images of this size. (${SZ})"
+	echo "-S:	Copy files from this dir. (${SRCDIR})"
+	echo "-x:	Run xfs_repair at most this many times. (${MAX_FSCK})"
+	exit 0
+}
+
+GETOPT="d:n:s:O:I:b:B:fpx:S:"
+
+while getopts "${GETOPT}" opt; do
+	case "${opt}" in
+	"B")
+		FUZZ_ARGS="-3 -n ${OPTARG}"
+		;;
+	"d")
+		DIR="${OPTARG}"
+		;;
+	"n")
+		PASSES="${OPTARG}"
+		;;
+	"s")
+		SZ="${OPTARG}"
+		;;
+	"O")
+		FEATURES="${OPTARG}"
+		;;
+	"I")
+		INODE_SZ="${OPTARG}"
+		;;
+	"b")
+		BLK_SZ="${OPTARG}"
+		;;
+	"f")
+		RUN_FSCK=0
+		;;
+	"p")
+		OVERRIDE_PATH=0
+		;;
+	"x")
+		MAX_FSCK="${OPTARG}"
+		;;
+	"S")
+		SRCDIR="${OPTARG}"
+		;;
+	*)
+		print_help
+		;;
+	esac
+done
+
+if [ "${OVERRIDE_PATH}" -gt 0 ]; then
+	PATH="${SCRIPT_DIR}:${SCRIPT_DIR}/../repair/:${SCRIPT_DIR}/../db/:${SCRIPT_DIR}/../mkfs/:${PATH}"
+	export PATH
+fi
+
+TESTDIR="${DIR}/tests/"
+TESTMNT="${DIR}/mnt/"
+BASE_IMG="${DIR}/xfsfuzz.img"
+
+# Set up FS image
+echo "+ create fs image"
+umount "${TESTDIR}"
+umount "${TESTMNT}"
+rm -rf "${TESTDIR}"
+rm -rf "${TESTMNT}"
+mkdir -p "${TESTDIR}"
+mkdir -p "${TESTMNT}"
+rm -rf "${BASE_IMG}"
+truncate -s "${SZ}" "${BASE_IMG}"
+mkfs.xfs -f ${FEATURES} -b "size=${BLK_SZ}" -i "size=${INODE_SZ}" "${BASE_IMG}"
+if [ $? -ne 0 ]; then
+	exit $?
+fi
+
+# Populate FS image
+echo "+ populate fs image"
+modprobe loop
+mount "${BASE_IMG}" "${TESTMNT}" -o loop
+if [ $? -ne 0 ]; then
+	exit $?
+fi
+SRC_SZ="$(du -ks "${SRCDIR}" | awk '{print $1}')"
+FS_SZ="$(( $(stat -f "${TESTMNT}" -c '%a * %S') / 1024 ))"
+NR="$(( (FS_SZ * 6 / 10) / SRC_SZ ))"
+if [ "${NR}" -lt 1 ]; then
+	NR=1
+fi
+echo "+ make ${NR} copies"
+seq 1 "${NR}" | while read nr; do
+	cp -pRdu "${SRCDIR}" "${TESTMNT}/test.${nr}" 2> /dev/null
+done
+umount "${TESTMNT}"
+xfs_repair ${XFS_REPAIR_OPTS} -vn "${BASE_IMG}"
+if [ $? -ne 0 ]; then
+	echo "fsck failed??"
+	exit 1
+fi
+
+# Run tests
+echo "+ run test"
+ret=0
+seq 1 "${PASSES}" | while read pass; do
+	echo "+ pass ${pass}"
+	PASS_IMG="${TESTDIR}/xfsfuzz-${pass}.img"
+	FSCK_IMG="${TESTDIR}/xfsfuzz-${pass}.fsck"
+	FUZZ_LOG="${TESTDIR}/xfsfuzz-${pass}.fuzz.log"
+	OPS_LOG="${TESTDIR}/xfsfuzz-${pass}.ops.log"
+
+	echo "++ copy image"
+	cp "${BASE_IMG}" "${PASS_IMG}"
+	if [ $? -ne 0 ]; then
+		exit $?
+	fi
+	xfs_db -x -c "label xfsfuzz-${pass}" "${PASS_IMG}"
+
+	echo "++ corrupt image"
+	xfs_db -x -c blockget -c "blocktrash ${FUZZ_ARGS}" "${PASS_IMG}" > "${FUZZ_LOG}"
+#	res=$?
+#	if [ "${res}" -ne 0 ]; then
+#		echo "blocktrash returns ${res}"
+#		exit "${res}"
+#	fi
+
+	echo "++ mount image"
+	mount "${PASS_IMG}" "${TESTMNT}" -o loop
+	res=$?
+
+	if [ "${res}" -eq 0 ]; then
+		echo "+++ ls -laR"
+		ls -laR "${TESTMNT}/test.1/" > /dev/null 2> "${OPS_LOG}"
+
+		echo "+++ cat files"
+		find "${TESTMNT}/test.1/" -type f -size -1048576k -print0 | xargs -0 cat > /dev/null 2>> "${OPS_LOG}"
+
+		echo "+++ expand"
+		find "${TESTMNT}/" -type f 2> /dev/null | head -n 50000 | while read f; do
+			attr -l "$f" > /dev/null 2>> "${OPS_LOG}"
+			if [ -f "$f" -a -w "$f" ]; then
+				dd if=/dev/zero bs="${BLK_SZ}" count=1 >> "$f" 2>> "${OPS_LOG}"
+			fi
+			mv "$f" "$f.longer" > /dev/null 2>> "${OPS_LOG}"
+		done
+		sync
+
+		echo "+++ create files"
+		cp -pRdu "${SRCDIR}" "${TESTMNT}/test.moo" 2>> "${OPS_LOG}"
+		sync
+
+		echo "+++ remove files"
+		rm -rf "${TESTMNT}/test.moo" 2>> "${OPS_LOG}"
+
+		umount "${TESTMNT}"
+		res=$?
+		if [ "${res}" -ne 0 ]; then
+			ret=1
+			break
+		fi
+		sync
+	fi
+	if [ "${RUN_FSCK}" -gt 0 ]; then
+		cp "${PASS_IMG}" "${FSCK_IMG}"
+		pass_img_sz="$(stat -c '%s' "${PASS_IMG}")"
+
+		seq 1 "${MAX_FSCK}" | while read fsck_pass; do
+			echo "++ fsck pass ${fsck_pass}: $(which xfs_repair) -v ${FSCK_IMG}"
+			FSCK_LOG="${TESTDIR}/xfsfuzz-${pass}-${fsck_pass}.log"
+			echo "repairing" > "${FSCK_LOG}"
+			xfs_repair ${XFS_REPAIR_OPTS} -v "${FSCK_IMG}" >> "${FSCK_LOG}" 2>&1
+			res=$?
+			if [ "${res}" -eq 0 ]; then
+				echo "reverify" >> "${FSCK_LOG}"
+				xfs_repair ${XFS_REPAIR_OPTS} -v -n "${FSCK_IMG}" >> "${FSCK_LOG}" 2>&1
+				res=$?
+			fi
+			echo "++ fsck returns ${res}"
+			if [ "${res}" -eq 0 ]; then
+				exit 0
+			elif [ "${res}" -eq 2 ]; then
+				# replay log?
+				echo "replaying log" >> "${FSCK_LOG}"
+				dmesg > /tmp/a
+				mount "${FSCK_IMG}" "${TESTMNT}" -o loop
+				res=$?
+				if [ "${res}" -gt 0 ]; then
+					echo "+++ zeroing log"
+					echo "zeroing log" >> "${FSCK_LOG}"
+					xfs_repair ${XFS_REPAIR_OPTS} -L -v "${FSCK_IMG}" >> "${FSCK_LOG}" 2>&1
+				else
+					umount "${TESTMNT}"
+				fi
+				dmesg > /tmp/b
+				diff -u /tmp/a /tmp/b >> "${FSCK_LOG}"
+			elif [ "${fsck_pass}" -eq "${MAX_FSCK}" ]; then
+				echo "++ fsck did not fix in ${MAX_FSCK} passes."
+				exit 1
+			fi
+			if [ "${fsck_pass}" -gt 1 ]; then
+				diff -u "${TESTDIR}/xfsfuzz-${pass}-$((fsck_pass - 1)).log" "${FSCK_LOG}"
+				if [ $? -eq 0 ]; then
+					echo "++ fsck makes no progress"
+					exit 2
+				fi
+			fi
+
+			fsck_img_sz="$(stat -c '%s' "${FSCK_IMG}")"
+			if [ "${fsck_img_sz}" -ne "${pass_img_sz}" ]; then
+				echo "++ fsck image size changed"
+				exit 3
+			fi
+		done
+		fsck_loop_ret=$?
+		if [ "${fsck_loop_ret}" -gt 0 ]; then
+			break;
+		fi
+	fi
+
+	echo "+++ check fs for round 2"
+	FSCK_LOG="${TESTDIR}/xfsfuzz-${pass}-round2.log"
+	xfs_repair ${XFS_REPAIR_OPTS} -v -n "${FSCK_IMG}" > "${FSCK_LOG}" 2>&1
+	res=$?
+	if [ "${res}" -ne 0 ]; then
+		echo "++++ fsck failed."
+		exit 1
+	fi
+
+	echo "++ mount image (2)"
+	mount "${FSCK_IMG}" "${TESTMNT}" -o loop
+	res=$?
+
+	if [ "${res}" -eq 0 ]; then
+		echo "+++ ls -laR (2)"
+		ls -laR "${TESTMNT}/test.1/" > /dev/null 2> "${OPS_LOG}"
+
+		echo "+++ cat files (2)"
+		find "${TESTMNT}/test.1/" -type f -size -1048576k -print0 | xargs -0 cat > /dev/null 2>> "${OPS_LOG}"
+
+		echo "+++ expand (2)"
+		find "${TESTMNT}/" -type f 2> /dev/null | head -n 50000 | while read f; do
+			attr -l "$f" > /dev/null 2>> "${OPS_LOG}"
+			if [ -f "$f" -a -w "$f" ]; then
+				dd if=/dev/zero bs="${BLK_SZ}" count=1 >> "$f" 2>> "${OPS_LOG}"
+			fi
+			mv "$f" "$f.longer" > /dev/null 2>> "${OPS_LOG}"
+		done
+		sync
+
+		echo "+++ create files (2)"
+		cp -pRdu "${SRCDIR}" "${TESTMNT}/test.moo" 2>> "${OPS_LOG}"
+		sync
+
+		echo "+++ remove files (2)"
+		rm -rf "${TESTMNT}/test.moo" 2>> "${OPS_LOG}"
+
+		umount "${TESTMNT}"
+		res=$?
+		if [ "${res}" -ne 0 ]; then
+			ret=1
+			break
+		fi
+		sync
+
+		echo "+++ check fs (2)"
+		xfs_repair ${XFS_REPAIR_OPTS} -v -n "${FSCK_IMG}" >> "${FSCK_LOG}" 2>&1
+		res=$?
+		if [ "${res}" -ne 0 ]; then
+			echo "++++ fsck failed."
+			exit 1
+		fi
+	else
+		echo "++ mount(2) failed with ${res}"
+		exit 1
+	fi
+	rm -rf "${FSCK_IMG}" "${PASS_IMG}" "${FUZZ_LOG}" "${TESTDIR}"/xfsfuzz*.log
+done
+
+exit $ret

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 4/5] xfs_db: enable blockget for v5 filesystems

[Eric Sandeen]

On 5/26/15 5:51 PM, Darrick J. Wong wrote:
> Plumb in the necessary magic number checks and other fixups required
> to handle v5 filesystems.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

> @@ -799,8 +803,6 @@ blockget_f(
>  	 * filters out, or we need to actually do the work to make check support
>  	 * crc enabled filesystems.
>  	 */
> -	if (xfs_sb_version_hascrc(&mp->m_sb))
> -		return 0;

Hm, the comment above these lines says:

        /*
         * XXX: check does not support CRC enabled filesystems. Return
         * immediately, silently, with success but  without doing anything here
         * initially so that xfstests can run without modification on metadata
         * enabled filesystems.
         *
         * XXX: ultimately we need to dump an error message here that xfstests
         * filters out, or we need to actually do the work to make check support
         * crc enabled filesystems.
         */

but now we don't.  So I guess the comment should go too?

I guess check is deprecated now, so the comment is stale anyway...
  
-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 4/5] xfs_db: enable blockget for v5 filesystems

[Darrick J. Wong]

On Tue, May 26, 2015 at 06:07:32PM -0500, Eric Sandeen wrote:
> On 5/26/15 5:51 PM, Darrick J. Wong wrote:
> > Plumb in the necessary magic number checks and other fixups required
> > to handle v5 filesystems.
> > 
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> 
> > @@ -799,8 +803,6 @@ blockget_f(
> >  	 * filters out, or we need to actually do the work to make check support
> >  	 * crc enabled filesystems.
> >  	 */
> > -	if (xfs_sb_version_hascrc(&mp->m_sb))
> > -		return 0;
> 
> Hm, the comment above these lines says:
> 
>         /*
>          * XXX: check does not support CRC enabled filesystems. Return
>          * immediately, silently, with success but  without doing anything here
>          * initially so that xfstests can run without modification on metadata
>          * enabled filesystems.
>          *
>          * XXX: ultimately we need to dump an error message here that xfstests
>          * filters out, or we need to actually do the work to make check support
>          * crc enabled filesystems.
>          */
> 
> but now we don't.  So I guess the comment should go too?
> 
> I guess check is deprecated now, so the comment is stale anyway...

Doh.  Forgot to remove that.  Will resend.

--D

>   
> -Eric
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH v2 4/5] xfs_db: enable blockget for v5 filesystems

[Darrick J. Wong]

Plumb in the necessary magic number checks and other fixups required
to handle v5 filesystems.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 db/check.c |  236 +++++++++++++++++++++++++++++++++++++++++++++++++++++-------
 db/type.c  |    3 +
 db/type.h  |    2 -
 3 files changed, 214 insertions(+), 27 deletions(-)

diff --git a/db/check.c b/db/check.c
index c4c972f..48eeaa8 100644
--- a/db/check.c
+++ b/db/check.c
@@ -44,7 +44,7 @@ typedef enum {
 	DBM_FREE1,	DBM_FREE2,	DBM_FREELIST,	DBM_INODE,
 	DBM_LOG,	DBM_MISSING,	DBM_QUOTA,	DBM_RTBITMAP,
 	DBM_RTDATA,	DBM_RTFREE,	DBM_RTSUM,	DBM_SB,
-	DBM_SYMLINK,
+	DBM_SYMLINK,	DBM_BTFINO,
 	DBM_NDBM
 } dbm_t;
 
@@ -170,6 +170,7 @@ static const char	*typename[] = {
 	"rtsum",
 	"sb",
 	"symlink",
+	"btfino",
 	NULL
 };
 static int		verbose;
@@ -345,6 +346,9 @@ static void		scanfunc_cnt(struct xfs_btree_block *block, int level,
 static void		scanfunc_ino(struct xfs_btree_block *block, int level,
 				     xfs_agf_t *agf, xfs_agblock_t bno,
 				     int isroot);
+static void		scanfunc_fino(struct xfs_btree_block *block, int level,
+				     xfs_agf_t *agf, xfs_agblock_t bno,
+				     int isroot);
 static void		set_dbmap(xfs_agnumber_t agno, xfs_agblock_t agbno,
 				  xfs_extlen_t len, dbm_t type,
 				  xfs_agnumber_t c_agno, xfs_agblock_t c_agbno);
@@ -789,19 +793,6 @@ blockget_f(
 		return 0;
 	}
 
-	/*
-	 * XXX: check does not support CRC enabled filesystems. Return
-	 * immediately, silently, with success but  without doing anything here
-	 * initially so that xfstests can run without modification on metadata
-	 * enabled filesystems.
-	 *
-	 * XXX: ultimately we need to dump an error message here that xfstests
-	 * filters out, or we need to actually do the work to make check support
-	 * crc enabled filesystems.
-	 */
-	if (xfs_sb_version_hascrc(&mp->m_sb))
-		return 0;
-
 	if (!init(argc, argv)) {
 		if (serious_error)
 			exitcode = 3;
@@ -2223,7 +2214,9 @@ process_data_dir_v2(
 	data = iocur_top->data;
 	block = iocur_top->data;
 	if (be32_to_cpu(block->magic) != XFS_DIR2_BLOCK_MAGIC &&
-			be32_to_cpu(data->magic) != XFS_DIR2_DATA_MAGIC) {
+			be32_to_cpu(data->magic) != XFS_DIR2_DATA_MAGIC &&
+			be32_to_cpu(block->magic) != XFS_DIR3_BLOCK_MAGIC &&
+			be32_to_cpu(data->magic) != XFS_DIR3_DATA_MAGIC) {
 		if (!sflag || v)
 			dbprintf(_("bad directory data magic # %#x for dir ino "
 				 "%lld block %d\n"),
@@ -2234,7 +2227,8 @@ process_data_dir_v2(
 	db = xfs_dir2_da_to_db(mp, dabno);
 	bf = xfs_dir3_data_bestfree_p(data);
 	ptr = (char *)xfs_dir3_data_unused_p(data);
-	if (be32_to_cpu(block->magic) == XFS_DIR2_BLOCK_MAGIC) {
+	if (be32_to_cpu(block->magic) == XFS_DIR2_BLOCK_MAGIC ||
+	    be32_to_cpu(block->magic) == XFS_DIR3_BLOCK_MAGIC) {
 		btp = xfs_dir2_block_tail_p(mp, block);
 		lep = xfs_dir2_block_leaf_p(btp);
 		endptr = (char *)lep;
@@ -2380,7 +2374,8 @@ process_data_dir_v2(
 			(*dot)++;
 		}
 	}
-	if (be32_to_cpu(data->magic) == XFS_DIR2_BLOCK_MAGIC) {
+	if (be32_to_cpu(data->magic) == XFS_DIR2_BLOCK_MAGIC ||
+	    be32_to_cpu(data->magic) == XFS_DIR3_BLOCK_MAGIC) {
 		endptr = (char *)data + mp->m_dirblksize;
 		for (i = stale = 0; lep && i < be32_to_cpu(btp->count); i++) {
 			if ((char *)&lep[i] >= endptr) {
@@ -2412,7 +2407,8 @@ process_data_dir_v2(
 				id->ino, dabno);
 		error++;
 	}
-	if (be32_to_cpu(data->magic) == XFS_DIR2_BLOCK_MAGIC &&
+	if ((be32_to_cpu(data->magic) == XFS_DIR2_BLOCK_MAGIC ||
+	     be32_to_cpu(data->magic) == XFS_DIR3_BLOCK_MAGIC) &&
 	    count != be32_to_cpu(btp->count) - be32_to_cpu(btp->stale)) {
 		if (!sflag || v)
 			dbprintf(_("dir %lld block %d bad block tail count %d "
@@ -2421,7 +2417,8 @@ process_data_dir_v2(
 				be32_to_cpu(btp->stale));
 		error++;
 	}
-	if (be32_to_cpu(data->magic) == XFS_DIR2_BLOCK_MAGIC && 
+	if ((be32_to_cpu(data->magic) == XFS_DIR2_BLOCK_MAGIC ||
+	     be32_to_cpu(data->magic) == XFS_DIR2_BLOCK_MAGIC) &&
 					stale != be32_to_cpu(btp->stale)) {
 		if (!sflag || v)
 			dbprintf(_("dir %lld block %d bad stale tail count %d\n"),
@@ -3007,6 +3004,73 @@ process_leaf_node_dir_v2(
 }
 
 static void
+process_leaf_node_dir_v3_free(
+	inodata_t		*id,
+	int			v,
+	xfs_dablk_t		dabno,
+	freetab_t		*freetab)
+{
+	xfs_dir2_data_off_t	ent;
+	struct xfs_dir3_free	*free;
+	int			i;
+	int			maxent;
+	int			used;
+
+	free = iocur_top->data;
+	maxent = xfs_dir3_free_max_bests(mp);
+	if (be32_to_cpu(free->hdr.firstdb) != xfs_dir2_da_to_db(mp,
+					dabno - mp->m_dirfreeblk) * maxent) {
+		if (!sflag || v)
+			dbprintf(_("bad free block firstdb %d for dir ino %lld "
+				 "block %d\n"),
+				be32_to_cpu(free->hdr.firstdb), id->ino, dabno);
+		error++;
+		return;
+	}
+	if (be32_to_cpu(free->hdr.nvalid) > maxent ||
+				be32_to_cpu(free->hdr.nvalid) < 0 ||
+				be32_to_cpu(free->hdr.nused) > maxent ||
+				be32_to_cpu(free->hdr.nused) < 0 ||
+				be32_to_cpu(free->hdr.nused) >
+					be32_to_cpu(free->hdr.nvalid)) {
+		if (!sflag || v)
+			dbprintf(_("bad free block nvalid/nused %d/%d for dir "
+				 "ino %lld block %d\n"),
+				be32_to_cpu(free->hdr.nvalid),
+				be32_to_cpu(free->hdr.nused), id->ino, dabno);
+		error++;
+		return;
+	}
+	for (used = i = 0; i < be32_to_cpu(free->hdr.nvalid); i++) {
+		if (freetab->nents <= be32_to_cpu(free->hdr.firstdb) + i)
+			ent = NULLDATAOFF;
+		else
+			ent = freetab->ents[be32_to_cpu(free->hdr.firstdb) + i];
+		if (ent != be16_to_cpu(free->bests[i])) {
+			if (!sflag || v)
+				dbprintf(_("bad free block ent %d is %d should "
+					 "be %d for dir ino %lld block %d\n"),
+					i, be16_to_cpu(free->bests[i]), ent,
+					id->ino, dabno);
+			error++;
+		}
+		if (be16_to_cpu(free->bests[i]) != NULLDATAOFF)
+			used++;
+		if (ent != NULLDATAOFF)
+			freetab->ents[be32_to_cpu(free->hdr.firstdb) + i] =
+								NULLDATAOFF;
+	}
+	if (used != be32_to_cpu(free->hdr.nused)) {
+		if (!sflag || v)
+			dbprintf(_("bad free block nused %d should be %d for dir "
+				 "ino %lld block %d\n"),
+				be32_to_cpu(free->hdr.nused), used, id->ino,
+				dabno);
+		error++;
+	}
+}
+
+static void
 process_leaf_node_dir_v2_free(
 	inodata_t		*id,
 	int			v,
@@ -3020,7 +3084,8 @@ process_leaf_node_dir_v2_free(
 	int			used;
 
 	free = iocur_top->data;
-	if (be32_to_cpu(free->hdr.magic) != XFS_DIR2_FREE_MAGIC) {
+	if (be32_to_cpu(free->hdr.magic) != XFS_DIR2_FREE_MAGIC &&
+	    be32_to_cpu(free->hdr.magic) != XFS_DIR3_FREE_MAGIC) {
 		if (!sflag || v)
 			dbprintf(_("bad free block magic # %#x for dir ino %lld "
 				 "block %d\n"),
@@ -3028,6 +3093,10 @@ process_leaf_node_dir_v2_free(
 		error++;
 		return;
 	}
+	if (be32_to_cpu(free->hdr.magic) == XFS_DIR3_FREE_MAGIC) {
+		process_leaf_node_dir_v3_free(id, v, dabno, freetab);
+		return;
+	}
 	maxent = xfs_dir3_free_max_bests(mp);
 	if (be32_to_cpu(free->hdr.firstdb) != xfs_dir2_da_to_db(mp, 
 					dabno - mp->m_dirfreeblk) * maxent) {
@@ -3081,6 +3150,20 @@ process_leaf_node_dir_v2_free(
 	}
 }
 
+/*
+ * Get address of the bestcount field in the single-leaf block.
+ */
+static inline int
+xfs_dir3_leaf_ents_count(struct xfs_dir2_leaf *lp)
+{
+	if (lp->hdr.info.magic == cpu_to_be16(XFS_DIR3_LEAF1_MAGIC) ||
+	    lp->hdr.info.magic == cpu_to_be16(XFS_DIR3_LEAFN_MAGIC)) {
+		struct xfs_dir3_leaf *lp3 = (struct xfs_dir3_leaf *)lp;
+		return be16_to_cpu(lp3->hdr.count);
+	}
+	return be16_to_cpu(lp->hdr.count);
+}
+
 static void
 process_leaf_node_dir_v2_int(
 	inodata_t		*id,
@@ -3091,6 +3174,7 @@ process_leaf_node_dir_v2_int(
 	int			i;
 	__be16			*lbp;
 	xfs_dir2_leaf_t		*leaf;
+	struct xfs_dir3_leaf	*leaf3 = NULL;
 	xfs_dir2_leaf_entry_t	*lep;
 	xfs_dir2_leaf_tail_t	*ltp;
 	xfs_da_intnode_t	*node;
@@ -3099,7 +3183,15 @@ process_leaf_node_dir_v2_int(
 
 	leaf = iocur_top->data;
 	switch (be16_to_cpu(leaf->hdr.info.magic)) {
+	case XFS_DIR3_LEAF1_MAGIC:
+	case XFS_DIR3_LEAFN_MAGIC:
+	case XFS_DA3_NODE_MAGIC:
+		leaf3 = iocur_top->data;
+		break;
+	}
+	switch (be16_to_cpu(leaf->hdr.info.magic)) {
 	case XFS_DIR2_LEAF1_MAGIC:
+	case XFS_DIR3_LEAF1_MAGIC:
 		if (be32_to_cpu(leaf->hdr.info.forw) || 
 					be32_to_cpu(leaf->hdr.info.back)) {
 			if (!sflag || v)
@@ -3139,10 +3231,12 @@ process_leaf_node_dir_v2_int(
 		}
 		break;
 	case XFS_DIR2_LEAFN_MAGIC:
+	case XFS_DIR3_LEAFN_MAGIC:
 		/* if it's at the root location then we can check the
 		 * pointers are null XXX */
 		break;
 	case XFS_DA_NODE_MAGIC:
+	case XFS_DA3_NODE_MAGIC:
 		node = iocur_top->data;
 		xfs_da3_node_hdr_from_disk(&nodehdr, node);
 		if (nodehdr.level < 1 || nodehdr.level > XFS_DA_NODE_MAXDEPTH) {
@@ -3164,7 +3258,7 @@ process_leaf_node_dir_v2_int(
 		return;
 	}
 	lep = xfs_dir3_leaf_ents_p(leaf);
-	for (i = stale = 0; i < be16_to_cpu(leaf->hdr.count); i++) {
+	for (i = stale = 0; i < xfs_dir3_leaf_ents_count(leaf); i++) {
 		if (be32_to_cpu(lep[i].address) == XFS_DIR2_NULL_DATAPTR)
 			stale++;
 		else if (dir_hash_see(be32_to_cpu(lep[i].hashval), 
@@ -3177,7 +3271,14 @@ process_leaf_node_dir_v2_int(
 			error++;
 		}
 	}
-	if (stale != be16_to_cpu(leaf->hdr.stale)) {
+	if (leaf3 && stale != be16_to_cpu(leaf3->hdr.stale)) {
+		if (!sflag || v)
+			dbprintf(_("dir3 %lld block %d stale mismatch "
+				 "%d/%d\n"),
+				 id->ino, dabno, stale,
+				 be16_to_cpu(leaf3->hdr.stale));
+		error++;
+	} else if (!leaf && stale != be16_to_cpu(leaf->hdr.stale)) {
 		if (!sflag || v)
 			dbprintf(_("dir %lld block %d stale mismatch "
 				 "%d/%d\n"),
@@ -3764,6 +3865,12 @@ scan_ag(
 		be32_to_cpu(agi->agi_root),
 		be32_to_cpu(agi->agi_level),
 		1, scanfunc_ino, TYP_INOBT);
+	if (agi->agi_free_root) {
+		scan_sbtree(agf,
+			be32_to_cpu(agi->agi_free_root),
+			be32_to_cpu(agi->agi_free_level),
+			1, scanfunc_fino, TYP_FINOBT);
+	}
 	if (be32_to_cpu(agf->agf_freeblks) != agffreeblks) {
 		if (!sflag)
 			dbprintf(_("agf_freeblks %u, counted %u in ag %u\n"),
@@ -3963,7 +4070,8 @@ scanfunc_bmap(
 
 	agno = XFS_FSB_TO_AGNO(mp, bno);
 	agbno = XFS_FSB_TO_AGBNO(mp, bno);
-	if (be32_to_cpu(block->bb_magic) != XFS_BMAP_MAGIC) {
+	if (be32_to_cpu(block->bb_magic) != XFS_BMAP_MAGIC &&
+	    be32_to_cpu(block->bb_magic) != XFS_BMAP_CRC_MAGIC) {
 		if (!sflag || id->ilist || CHECK_BLIST(bno))
 			dbprintf(_("bad magic # %#x in inode %lld bmbt block "
 				 "%u/%u\n"),
@@ -4028,7 +4136,8 @@ scanfunc_bno(
 	xfs_agnumber_t		seqno = be32_to_cpu(agf->agf_seqno);
 	xfs_agblock_t		lastblock;
 
-	if (be32_to_cpu(block->bb_magic) != XFS_ABTB_MAGIC) {
+	if (be32_to_cpu(block->bb_magic) != XFS_ABTB_MAGIC &&
+	    be32_to_cpu(block->bb_magic) != XFS_ABTB_CRC_MAGIC) {
 		dbprintf(_("bad magic # %#x in btbno block %u/%u\n"),
 			be32_to_cpu(block->bb_magic), seqno, bno);
 		serious_error++;
@@ -4101,7 +4210,8 @@ scanfunc_cnt(
 	xfs_alloc_rec_t		*rp;
 	xfs_extlen_t		lastcount;
 
-	if (be32_to_cpu(block->bb_magic) != XFS_ABTC_MAGIC) {
+	if (be32_to_cpu(block->bb_magic) != XFS_ABTC_MAGIC &&
+	    be32_to_cpu(block->bb_magic) != XFS_ABTC_CRC_MAGIC) {
 		dbprintf(_("bad magic # %#x in btcnt block %u/%u\n"),
 			be32_to_cpu(block->bb_magic), seqno, bno);
 		serious_error++;
@@ -4181,7 +4291,8 @@ scanfunc_ino(
 	xfs_inobt_ptr_t		*pp;
 	xfs_inobt_rec_t		*rp;
 
-	if (be32_to_cpu(block->bb_magic) != XFS_IBT_MAGIC) {
+	if (be32_to_cpu(block->bb_magic) != XFS_IBT_MAGIC &&
+	    be32_to_cpu(block->bb_magic) != XFS_IBT_CRC_MAGIC) {
 		dbprintf(_("bad magic # %#x in inobt block %u/%u\n"),
 			be32_to_cpu(block->bb_magic), seqno, bno);
 		serious_error++;
@@ -4277,6 +4388,79 @@ scanfunc_ino(
 }
 
 static void
+scanfunc_fino(
+	struct xfs_btree_block	*block,
+	int			level,
+	xfs_agf_t		*agf,
+	xfs_agblock_t		bno,
+	int			isroot)
+{
+	xfs_agino_t		agino;
+	xfs_agnumber_t		seqno = be32_to_cpu(agf->agf_seqno);
+	int			i;
+	int			off;
+	xfs_inobt_ptr_t		*pp;
+	xfs_inobt_rec_t		*rp;
+
+	if (be32_to_cpu(block->bb_magic) != XFS_FIBT_MAGIC &&
+	    be32_to_cpu(block->bb_magic) != XFS_FIBT_CRC_MAGIC) {
+		dbprintf(_("bad magic # %#x in finobt block %u/%u\n"),
+			be32_to_cpu(block->bb_magic), seqno, bno);
+		serious_error++;
+		return;
+	}
+	if (be16_to_cpu(block->bb_level) != level) {
+		if (!sflag)
+			dbprintf(_("expected level %d got %d in finobt block "
+				 "%u/%u\n"),
+				level, be16_to_cpu(block->bb_level), seqno, bno);
+		error++;
+	}
+	set_dbmap(seqno, bno, 1, DBM_BTFINO, seqno, bno);
+	if (level == 0) {
+		if (be16_to_cpu(block->bb_numrecs) > mp->m_inobt_mxr[0] ||
+		    (isroot == 0 && be16_to_cpu(block->bb_numrecs) < mp->m_inobt_mnr[0])) {
+			dbprintf(_("bad btree nrecs (%u, min=%u, max=%u) in "
+				 "finobt block %u/%u\n"),
+				be16_to_cpu(block->bb_numrecs), mp->m_inobt_mnr[0],
+				mp->m_inobt_mxr[0], seqno, bno);
+			serious_error++;
+			return;
+		}
+		rp = XFS_INOBT_REC_ADDR(mp, block, 1);
+		for (i = 0; i < be16_to_cpu(block->bb_numrecs); i++) {
+			agino = be32_to_cpu(rp[i].ir_startino);
+			off = XFS_INO_TO_OFFSET(mp, agino);
+			if (off == 0) {
+				if ((sbversion & XFS_SB_VERSION_ALIGNBIT) &&
+				    mp->m_sb.sb_inoalignmt &&
+				    (XFS_INO_TO_AGBNO(mp, agino) %
+				     mp->m_sb.sb_inoalignmt))
+					sbversion &= ~XFS_SB_VERSION_ALIGNBIT;
+				check_set_dbmap(seqno, XFS_AGINO_TO_AGBNO(mp, agino),
+					(xfs_extlen_t)MAX(1,
+						XFS_INODES_PER_CHUNK >>
+						mp->m_sb.sb_inopblog),
+					DBM_INODE, DBM_INODE, seqno, bno);
+			}
+		}
+		return;
+	}
+	if (be16_to_cpu(block->bb_numrecs) > mp->m_inobt_mxr[1] ||
+	    (isroot == 0 && be16_to_cpu(block->bb_numrecs) < mp->m_inobt_mnr[1])) {
+		dbprintf(_("bad btree nrecs (%u, min=%u, max=%u) in finobt block "
+			 "%u/%u\n"),
+			be16_to_cpu(block->bb_numrecs), mp->m_inobt_mnr[1],
+			mp->m_inobt_mxr[1], seqno, bno);
+		serious_error++;
+		return;
+	}
+	pp = XFS_INOBT_PTR_ADDR(mp, block, 1, mp->m_inobt_mxr[1]);
+	for (i = 0; i < be16_to_cpu(block->bb_numrecs); i++)
+		scan_sbtree(agf, be32_to_cpu(pp[i]), level, 0, scanfunc_ino, TYP_FINOBT);
+}
+
+static void
 set_dbmap(
 	xfs_agnumber_t	agno,
 	xfs_agblock_t	agbno,
diff --git a/db/type.c b/db/type.c
index b29f2a4..0aa3137 100644
--- a/db/type.c
+++ b/db/type.c
@@ -70,6 +70,7 @@ static const typ_t	__typtab[] = {
 	{ TYP_SB, "sb", handle_struct, sb_hfld, NULL },
 	{ TYP_SYMLINK, "symlink", handle_string, NULL, NULL },
 	{ TYP_TEXT, "text", handle_text, NULL, NULL },
+	{ TYP_FINOBT, "finobt", handle_struct, inobt_hfld, NULL },
 	{ TYP_NONE, NULL }
 };
 
@@ -104,6 +105,8 @@ static const typ_t	__typtab_crc[] = {
 	{ TYP_SYMLINK, "symlink", handle_struct, symlink_crc_hfld,
 		&xfs_symlink_buf_ops },
 	{ TYP_TEXT, "text", handle_text, NULL, NULL },
+	{ TYP_FINOBT, "finobt", handle_struct, inobt_crc_hfld,
+		&xfs_inobt_buf_ops },
 	{ TYP_NONE, NULL }
 };
 
diff --git a/db/type.h b/db/type.h
index 3bb26f1..e8d8df7 100644
--- a/db/type.h
+++ b/db/type.h
@@ -27,7 +27,7 @@ typedef enum typnm
 	TYP_BMAPBTD, TYP_BNOBT, TYP_CNTBT, TYP_DATA,
 	TYP_DIR2, TYP_DQBLK, TYP_INOBT, TYP_INODATA, TYP_INODE,
 	TYP_LOG, TYP_RTBITMAP, TYP_RTSUMMARY, TYP_SB, TYP_SYMLINK,
-	TYP_TEXT, TYP_NONE
+	TYP_TEXT, TYP_FINOBT, TYP_NONE
 } typnm_t;
 
 #define DB_WRITE 1

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 1/5] xfs_repair: refuse to run if we don't recognize version or feature flags

[Eric Sandeen]

On 5/26/15 5:51 PM, Darrick J. Wong wrote:
> Apparently xfs_repair running on a v5 filesystem doesn't check the
> compat, rocompat, or incompat feature flags for bits that it doesn't
> know about, which means that old xfs_repairs can wreak havoc.  So,
> strengthen the checks to prevent repair from "repairing" anything it
> doesn't understand.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
>  repair/versions.c |    7 +++++++
>  1 file changed, 7 insertions(+)
> 
> 
> diff --git a/repair/versions.c b/repair/versions.c
> index c1dff72..e60574d 100644
> --- a/repair/versions.c
> +++ b/repair/versions.c
> @@ -141,6 +141,13 @@ parse_sb_version(xfs_sb_t *sb)
>  		}
>  	}
>  
> +	/* Look for V5 feature flags we don't know about */
> +	if (XFS_SB_VERSION_NUM(sb) >= XFS_SB_VERSION_5 &&
> +	    (xfs_sb_has_ro_compat_feature(sb, XFS_SB_FEAT_RO_COMPAT_UNKNOWN) ||
> +	     xfs_sb_has_incompat_feature(sb, XFS_SB_FEAT_INCOMPAT_UNKNOWN) ||
> +	     xfs_sb_has_compat_feature(sb, XFS_SB_FEAT_COMPAT_UNKNOWN)))
> +		issue_warning = 1;
> +

should this go after the xfs_sb_good_version() check?

Seems like it's more of a fine-grained check on features for a given super
version, and should come after that check.

i.e. if for some reason XFS_SB_VERSION_NUM == 6, the root problem isn't
the feature set, it's the superblock version number.

And; would it be worth printing out what the features are?  I guess we have
no good existing mechanism for that, and could really only print hex values...
still, might be useful for bug reports...

-Eric


>  	if (issue_warning)  {
>  		do_warn(
>  _("This filesystem uses feature(s) not yet supported in this release.\n"
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 2/5] xfs_repair: better checking of v5 metadata fields

[Dave Chinner]

On Tue, May 26, 2015 at 03:51:39PM -0700, Darrick J. Wong wrote:
> Check the UUID, owner, and block number fields during repair, looking for
> blocks that fail either the checksum or the data structure verifiers.  For
> directories we can simply rebuild corrupt/broken index data, though for
> anything else we have to toss out the broken object.

helper function needed here for owner/uuid/blkno info checks. i.e
the header info being checked is a common structure
(xfs_da3_blkinfo) embedded into the different dir header structures,
so no need to repeat the same checks just with different dir3 header
structures....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH v2 2/5] xfs_repair: better checking of v5 metadata fields

[Darrick J. Wong]

Check the UUID, owner, and block number fields during repair, looking for
blocks that fail either the checksum or the data structure verifiers.  For
directories we can simply rebuild corrupt/broken index data, though for
anything else we have to toss out the broken object.

v2: Refactor the directory block header checks into a single function.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 repair/dir2.c   |    7 +++
 repair/phase6.c |  121 ++++++++++++++++++++++++++++++++++++++++++++++++++++---
 repair/scan.c   |   19 ++++++++-
 3 files changed, 140 insertions(+), 7 deletions(-)

diff --git a/repair/dir2.c b/repair/dir2.c
index c6d618d..644c214 100644
--- a/repair/dir2.c
+++ b/repair/dir2.c
@@ -198,6 +198,13 @@ _("bad dir magic number 0x%x in inode %" PRIu64 " bno = %u\n"),
 					da_cursor->ino, bno);
 			goto error_out;
 		}
+		/* corrupt node; rebuild the dir. */
+		if (bp->b_error == EFSBADCRC || bp->b_error == EFSCORRUPTED) {
+			do_warn(
+_("corrupt tree block %u for directory inode %" PRIu64 "\n"),
+				bno, da_cursor->ino);
+			goto error_out;
+		}
 		btree = xfs_da3_node_tree_p(node);
 		if (nodehdr.count > mp->m_dir_node_ents)  {
 			libxfs_putbuf(bp);
diff --git a/repair/phase6.c b/repair/phase6.c
index c09b394..7dd868c 100644
--- a/repair/phase6.c
+++ b/repair/phase6.c
@@ -1925,6 +1925,69 @@ _("entry \"%s\" in dir inode %" PRIu64 " inconsistent with .. value (%" PRIu64 "
 	freetab->ents[db].s = 0;
 }
 
+/* check v5 metadata */
+static int
+__check_dir3_header(
+	xfs_mount_t			*mp,
+	xfs_buf_t			*bp,
+	xfs_ino_t			ino,
+	__be64				owner,
+	__be64				blkno,
+	uuid_t				*uuid)
+{
+
+	/* verify owner */
+	if (be64_to_cpu(owner) != ino) {
+		do_warn(
+_("expected owner inode %" PRIu64 ", got %llu, directory block %" PRIu64 "\n"),
+			ino, be64_to_cpu(owner), bp->b_bn);
+		return 1;
+	}
+	/* verify block number */
+	if (be64_to_cpu(blkno) != bp->b_bn) {
+		do_warn(
+_("expected block %" PRIu64 ", got %llu, directory inode %" PRIu64 "\n"),
+			bp->b_bn, be64_to_cpu(blkno), ino);
+		return 1;
+	}
+	/* verify uuid */
+	if (platform_uuid_compare(uuid,
+				  &mp->m_sb.sb_uuid) != 0) {
+		do_warn(
+_("wrong FS UUID, directory inode %" PRIu64 " block %" PRIu64 "\n"),
+			ino, bp->b_bn);
+		return 1;
+	}
+
+	return 0;
+}
+
+static int
+check_da3_header(
+	xfs_mount_t			*mp,
+	xfs_buf_t			*bp,
+	xfs_ino_t			ino)
+{
+	struct xfs_da3_blkinfo		*info;
+
+	info = bp->b_addr;
+	return __check_dir3_header(mp, bp, ino, info->owner, info->blkno,
+				   &info->uuid);
+}
+
+static int
+check_dir3_header(
+	xfs_mount_t			*mp,
+	xfs_buf_t			*bp,
+	xfs_ino_t			ino)
+{
+	struct xfs_dir3_blk_hdr		*info;
+
+	info = bp->b_addr;
+	return __check_dir3_header(mp, bp, ino, info->owner, info->blkno,
+				   &info->uuid);
+}
+
 /*
  * Check contents of leaf-form block.
  */
@@ -1981,6 +2044,15 @@ longform_dir2_check_leaf(
 		libxfs_putbuf(bp);
 		return 1;
 	}
+
+	if (leafhdr.magic == XFS_DIR3_LEAF1_MAGIC) {
+		error = check_da3_header(mp, bp, ip->i_ino);
+		if (error) {
+			libxfs_putbuf(bp);
+			return error;
+		}
+	}
+
 	seeval = dir_hash_see_all(hashtab, ents, leafhdr.count, leafhdr.stale);
 	if (dir_hash_check(hashtab, ip, seeval)) {
 		libxfs_putbuf(bp);
@@ -2055,12 +2127,9 @@ longform_dir2_check_node(
 		xfs_dir3_leaf_hdr_from_disk(&leafhdr, leaf);
 		ents = xfs_dir3_leaf_ents_p(leaf);
 		if (!(leafhdr.magic == XFS_DIR2_LEAFN_MAGIC ||
-		      leafhdr.magic == XFS_DIR3_LEAFN_MAGIC)) {
-			if (leafhdr.magic == XFS_DA_NODE_MAGIC ||
-			    leafhdr.magic == XFS_DA3_NODE_MAGIC) {
-				libxfs_putbuf(bp);
-				continue;
-			}
+		      leafhdr.magic == XFS_DIR3_LEAFN_MAGIC ||
+		      leafhdr.magic == XFS_DA_NODE_MAGIC ||
+		      leafhdr.magic == XFS_DA3_NODE_MAGIC)) {
 			do_warn(
 	_("unknown magic number %#x for block %u in directory inode %" PRIu64 "\n"),
 				leafhdr.magic, da_bno, ip->i_ino);
@@ -2068,6 +2137,23 @@ longform_dir2_check_node(
 			return 1;
 		}
 
+		/* check v5 metadata */
+		if (leafhdr.magic == XFS_DIR3_LEAFN_MAGIC ||
+		    leafhdr.magic == XFS_DA3_NODE_MAGIC) {
+			error = check_da3_header(mp, bp, ip->i_ino);
+			if (error) {
+				libxfs_putbuf(bp);
+				return error;
+			}
+		}
+
+		/* ignore nodes */
+		if (leafhdr.magic == XFS_DA_NODE_MAGIC ||
+		    leafhdr.magic == XFS_DA3_NODE_MAGIC) {
+			libxfs_putbuf(bp);
+			continue;
+		}
+
 		/*
 		 * If there's a validator error, we need to ensure that we got
 		 * the right ops on the buffer for when we write it back out.
@@ -2121,6 +2207,14 @@ longform_dir2_check_node(
 			libxfs_putbuf(bp);
 			return 1;
 		}
+
+		if (freehdr.magic == XFS_DIR3_FREE_MAGIC) {
+			error = check_dir3_header(mp, bp, ip->i_ino);
+			if (error) {
+				libxfs_putbuf(bp);
+				return error;
+			}
+		}
 		for (i = used = 0; i < freehdr.nvalid; i++) {
 			if (i + freehdr.firstdb >= freetab->nents ||
 					freetab->ents[i + freehdr.firstdb].v !=
@@ -2212,6 +2306,7 @@ longform_dir2_entry_check(xfs_mount_t	*mp,
 	     da_bno = (xfs_dablk_t)next_da_bno) {
 		const struct xfs_buf_ops *ops;
 		int			 error;
+		struct xfs_dir2_data_hdr *d;
 
 		next_da_bno = da_bno + mp->m_dirblkfsbs - 1;
 		if (bmap_next_offset(NULL, ip, &next_da_bno, XFS_DATA_FORK)) {
@@ -2260,6 +2355,20 @@ longform_dir2_entry_check(xfs_mount_t	*mp,
 			}
 			continue;
 		}
+
+		/* check v5 metadata */
+		d = bplist[db]->b_addr;
+		if (be32_to_cpu(d->magic) == XFS_DIR3_BLOCK_MAGIC ||
+		    be32_to_cpu(d->magic) == XFS_DIR3_DATA_MAGIC) {
+			struct xfs_buf		 *bp = bplist[db];
+
+			error = check_dir3_header(mp, bp, ino);
+			if (error) {
+				fixit++;
+				continue;
+			}
+		}
+
 		longform_dir2_entry_check_data(mp, ip, num_illegal, need_dot,
 				irec, ino_offset, &bplist[db], hashtab,
 				&freetab, da_bno, isblock);
diff --git a/repair/scan.c b/repair/scan.c
index 12aa782..6508a47 100644
--- a/repair/scan.c
+++ b/repair/scan.c
@@ -225,7 +225,24 @@ _("expected level %d got %d in inode %" PRIu64 ", (%s fork) bmbt block %" PRIu64
 			do_warn(
 _("expected owner inode %" PRIu64 ", got %llu, bmbt block %" PRIu64 "\n"),
 				ino, be64_to_cpu(block->bb_u.l.bb_owner), bno);
-			return(1);
+			return 1;
+		}
+		/* verify block number */
+		if (be64_to_cpu(block->bb_u.l.bb_blkno) !=
+		    XFS_FSB_TO_DADDR(mp, bno)) {
+			do_warn(
+_("expected block %" PRIu64 ", got %llu, bmbt block %" PRIu64 "\n"),
+				XFS_FSB_TO_DADDR(mp, bno),
+				be64_to_cpu(block->bb_u.l.bb_blkno), bno);
+			return 1;
+		}
+		/* verify uuid */
+		if (platform_uuid_compare(&block->bb_u.l.bb_uuid,
+					  &mp->m_sb.sb_uuid) != 0) {
+			do_warn(
+_("wrong FS UUID, bmbt block %" PRIu64 "\n"),
+				bno);
+			return 1;
 		}
 	}
 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH v2 1/5] xfs_repair: refuse to run if we don't recognize version or feature flags

[Darrick J. Wong]

Apparently xfs_repair running on a v5 filesystem doesn't check the
compat, rocompat, or incompat feature flags for bits that it doesn't
know about, which means that old xfs_repairs can wreak havoc.  So,
strengthen the checks to prevent repair from "repairing" anything it
doesn't understand.

v2: Move the complaint code after the version number check, and print
the actual feature bits that we don't recognize.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 repair/versions.c |   33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/repair/versions.c b/repair/versions.c
index c1dff72..dbe41a4 100644
--- a/repair/versions.c
+++ b/repair/versions.c
@@ -175,6 +175,39 @@ _("WARNING:  you have disallowed superblock-feature-bits-allowed\n"
 		}
 	}
 
+	/* Look for V5 feature flags we don't know about */
+	if (XFS_SB_VERSION_NUM(sb) >= XFS_SB_VERSION_5) {
+		if (xfs_sb_has_ro_compat_feature(sb,
+				XFS_SB_FEAT_RO_COMPAT_UNKNOWN)) {
+			do_warn(
+_("Superblock has unknown read-only compatible features (0x%x) enabled.\n"
+"Using a more recent xfs_repair is recommended.\n"),
+				sb->sb_features_ro_compat &
+						XFS_SB_FEAT_RO_COMPAT_UNKNOWN);
+			issue_warning = 1;
+		}
+		if (xfs_sb_has_incompat_feature(sb,
+				XFS_SB_FEAT_INCOMPAT_UNKNOWN)) {
+			do_warn(
+_("Superblock has unknown read-only compatible features (0x%x) enabled.\n"
+"Using a more recent xfs_repair is recommended.\n"),
+				sb->sb_features_ro_compat &
+						XFS_SB_FEAT_RO_COMPAT_UNKNOWN);
+			issue_warning = 1;
+		}
+		if (xfs_sb_has_compat_feature(sb,
+				XFS_SB_FEAT_COMPAT_UNKNOWN)) {
+			do_warn(
+_("Superblock has unknown compatible features (0x%x) enabled.\n"
+"Using a more recent xfs_repair is recommended.\n"),
+				sb->sb_features_ro_compat &
+						XFS_SB_FEAT_RO_COMPAT_UNKNOWN);
+			issue_warning = 1;
+		}
+		if (issue_warning)
+			return 1;
+	}
+
 	if (xfs_sb_version_hasattr(sb))  {
 		if (!fs_attributes_allowed)  {
 			if (!no_modify)  {

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH v2 1/5] xfs_repair: refuse to run if we don't recognize version or feature flags

[Eric Sandeen]

On 5/27/15 12:45 AM, Darrick J. Wong wrote:
> Apparently xfs_repair running on a v5 filesystem doesn't check the
> compat, rocompat, or incompat feature flags for bits that it doesn't
> know about, which means that old xfs_repairs can wreak havoc.  So,
> strengthen the checks to prevent repair from "repairing" anything it
> doesn't understand.
> 
> v2: Move the complaint code after the version number check, and print
> the actual feature bits that we don't recognize.

Thanks.  That's a bit more wordy, but probably better, in the end.

Reviewed-by: Eric Sandeen <sandeen@redhat.com>

> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
>  repair/versions.c |   33 +++++++++++++++++++++++++++++++++
>  1 file changed, 33 insertions(+)
> 
> diff --git a/repair/versions.c b/repair/versions.c
> index c1dff72..dbe41a4 100644
> --- a/repair/versions.c
> +++ b/repair/versions.c
> @@ -175,6 +175,39 @@ _("WARNING:  you have disallowed superblock-feature-bits-allowed\n"
>  		}
>  	}
>  
> +	/* Look for V5 feature flags we don't know about */
> +	if (XFS_SB_VERSION_NUM(sb) >= XFS_SB_VERSION_5) {
> +		if (xfs_sb_has_ro_compat_feature(sb,
> +				XFS_SB_FEAT_RO_COMPAT_UNKNOWN)) {
> +			do_warn(
> +_("Superblock has unknown read-only compatible features (0x%x) enabled.\n"
> +"Using a more recent xfs_repair is recommended.\n"),
> +				sb->sb_features_ro_compat &
> +						XFS_SB_FEAT_RO_COMPAT_UNKNOWN);
> +			issue_warning = 1;
> +		}
> +		if (xfs_sb_has_incompat_feature(sb,
> +				XFS_SB_FEAT_INCOMPAT_UNKNOWN)) {
> +			do_warn(
> +_("Superblock has unknown read-only compatible features (0x%x) enabled.\n"
> +"Using a more recent xfs_repair is recommended.\n"),
> +				sb->sb_features_ro_compat &
> +						XFS_SB_FEAT_RO_COMPAT_UNKNOWN);
> +			issue_warning = 1;
> +		}
> +		if (xfs_sb_has_compat_feature(sb,
> +				XFS_SB_FEAT_COMPAT_UNKNOWN)) {
> +			do_warn(
> +_("Superblock has unknown compatible features (0x%x) enabled.\n"
> +"Using a more recent xfs_repair is recommended.\n"),
> +				sb->sb_features_ro_compat &
> +						XFS_SB_FEAT_RO_COMPAT_UNKNOWN);
> +			issue_warning = 1;
> +		}
> +		if (issue_warning)
> +			return 1;
> +	}
> +
>  	if (xfs_sb_version_hasattr(sb))  {
>  		if (!fs_attributes_allowed)  {
>  			if (!no_modify)  {
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH v2 1/5] xfs_repair: refuse to run if we don't recognize version or feature flags

[Eric Sandeen]

On 5/27/15 9:54 AM, Eric Sandeen wrote:
> On 5/27/15 12:45 AM, Darrick J. Wong wrote:
>> Apparently xfs_repair running on a v5 filesystem doesn't check the
>> compat, rocompat, or incompat feature flags for bits that it doesn't
>> know about, which means that old xfs_repairs can wreak havoc.  So,
>> strengthen the checks to prevent repair from "repairing" anything it
>> doesn't understand.
>>
>> v2: Move the complaint code after the version number check, and print
>> the actual feature bits that we don't recognize.
> 
> Thanks.  That's a bit more wordy, but probably better, in the end.
> 
> Reviewed-by: Eric Sandeen <sandeen@redhat.com>

Whoops, retract that, thanks for spotting those C&P errors, Fanael!

-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH v2 1/5] xfs_repair: refuse to run if we don't recognize version or feature flags

[Fanael Linithien]

 > Apparently xfs_repair running on a v5 filesystem doesn't check the
 > compat, rocompat, or incompat feature flags for bits that it doesn't
 > know about, which means that old xfs_repairs can wreak havoc. So,
 > strengthen the checks to prevent repair from "repairing" anything it
 > doesn't understand.
 >
 > v2: Move the complaint code after the version number check, and print
 > the actual feature bits that we don't recognize.
 >
 > Signed-off-by: Darrick J. Wong <darrick.wong at oracle.com>
 > ---
 >  repair/versions.c |   33 +++++++++++++++++++++++++++++++++
 >  1 file changed, 33 insertions(+)
 >
 > diff --git a/repair/versions.c b/repair/versions.c
 > index c1dff72..dbe41a4 100644
 > --- a/repair/versions.c
 > +++ b/repair/versions.c
 > @@ -175,6 +175,39 @@ _("WARNING:  you have disallowed 
superblock-feature-bits-allowed\n"
 >          }
 >      }
 >
 > +    /* Look for V5 feature flags we don't know about */
 > +    if (XFS_SB_VERSION_NUM(sb) >= XFS_SB_VERSION_5) {
 > +        if (xfs_sb_has_ro_compat_feature(sb,
 > +                XFS_SB_FEAT_RO_COMPAT_UNKNOWN)) {
 > +            do_warn(
 > +_("Superblock has unknown read-only compatible features (0x%x) 
enabled.\n"
 > +"Using a more recent xfs_repair is recommended.\n"),
 > +                sb->sb_features_ro_compat &
 > +                        XFS_SB_FEAT_RO_COMPAT_UNKNOWN);
 > +            issue_warning = 1;
 > +        }
 > +        if (xfs_sb_has_incompat_feature(sb,
 > +                XFS_SB_FEAT_INCOMPAT_UNKNOWN)) {
 > +            do_warn(
 > +_("Superblock has unknown read-only compatible features (0x%x) 
enabled.\n"
 > +"Using a more recent xfs_repair is recommended.\n"),
 > +                sb->sb_features_ro_compat &
 > +                        XFS_SB_FEAT_RO_COMPAT_UNKNOWN);

Copy/paste error, do you mean incompat?

 > +            issue_warning = 1;
 > +        }
 > +        if (xfs_sb_has_compat_feature(sb,
 > +                XFS_SB_FEAT_COMPAT_UNKNOWN)) {
 > +            do_warn(
 > +_("Superblock has unknown compatible features (0x%x) enabled.\n"
 > +"Using a more recent xfs_repair is recommended.\n"),
 > +                sb->sb_features_ro_compat &
 > +                        XFS_SB_FEAT_RO_COMPAT_UNKNOWN);

compat here?

 > +            issue_warning = 1;
 > +        }
 > +        if (issue_warning)
 > +            return 1;
 > +    }
 > +
 >      if (xfs_sb_version_hasattr(sb))  {
 >          if (!fs_attributes_allowed)  {
 >              if (!no_modify)  {

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH v2 1/5] xfs_repair: refuse to run if we don't recognize version or feature flags

[Eric Sandeen]

On 5/27/15 10:16 AM, Eric Sandeen wrote:
> On 5/27/15 9:54 AM, Eric Sandeen wrote:
>> On 5/27/15 12:45 AM, Darrick J. Wong wrote:
>>> Apparently xfs_repair running on a v5 filesystem doesn't check the
>>> compat, rocompat, or incompat feature flags for bits that it doesn't
>>> know about, which means that old xfs_repairs can wreak havoc.  So,
>>> strengthen the checks to prevent repair from "repairing" anything it
>>> doesn't understand.
>>>
>>> v2: Move the complaint code after the version number check, and print
>>> the actual feature bits that we don't recognize.
>>
>> Thanks.  That's a bit more wordy, but probably better, in the end.
>>
>> Reviewed-by: Eric Sandeen <sandeen@redhat.com>
> 
> Whoops, retract that, thanks for spotting those C&P errors, Fanael!

I wonder if something like:

	/* Look for V5 feature flags we don't know about */
	if (XFS_SB_VERSION_NUM(sb) >= XFS_SB_VERSION_5 &&
	    (xfs_sb_has_ro_compat_feature(sb, XFS_SB_FEAT_RO_COMPAT_UNKNOWN) ||
	     xfs_sb_has_incompat_feature(sb, XFS_SB_FEAT_INCOMPAT_UNKNOWN) ||
	     xfs_sb_has_compat_feature(sb, XFS_SB_FEAT_COMPAT_UNKNOWN))) {
		printf("unknown feature flags 0x%x/0x%x/0x%x\n", 
			sb->sb_features_ro_compat & XFS_SB_FEAT_RO_COMPAT_UNKNOWN,
			sb->sb_features_incompat & XFS_SB_FEAT_INCOMPAT_UNKNOWN,
			sb->sb_features_compat & XFS_SB_FEAT_COMPAT_UNKNOWN);
		...

would suffice, given that the user will need to read code to unerstand the hex
values, anyway.
	
Hm, and as Fanael also pointed out, "compat" features ... should be ok, right,
and can be removed from the exclusions?

-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH v2 1/5] xfs_repair: refuse to run if we don't recognize version or feature flags

[Fanael Linithien]

On 2015-05-27 17:27, Eric Sandeen wrote:
>
> I wonder if something like:
>
> 	/* Look for V5 feature flags we don't know about */
> 	if (XFS_SB_VERSION_NUM(sb) >= XFS_SB_VERSION_5 &&
> 	    (xfs_sb_has_ro_compat_feature(sb, XFS_SB_FEAT_RO_COMPAT_UNKNOWN) ||
> 	     xfs_sb_has_incompat_feature(sb, XFS_SB_FEAT_INCOMPAT_UNKNOWN) ||
> 	     xfs_sb_has_compat_feature(sb, XFS_SB_FEAT_COMPAT_UNKNOWN))) {
> 		printf("unknown feature flags 0x%x/0x%x/0x%x\n",
> 			sb->sb_features_ro_compat & XFS_SB_FEAT_RO_COMPAT_UNKNOWN,
> 			sb->sb_features_incompat & XFS_SB_FEAT_INCOMPAT_UNKNOWN,
> 			sb->sb_features_compat & XFS_SB_FEAT_COMPAT_UNKNOWN);
> 		...
>
> would suffice, given that the user will need to read code to unerstand the hex
> values, anyway.
> 	
> Hm, and as Fanael also pointed out, "compat" features ... should be ok, right,
> and can be removed from the exclusions?

I'm not entirely sure silently ignoring unknown compat features in 
xfs_repair is a good idea. Consider this ext2 example: xattr support is 
a compat flag. It's okay to rw mount a FS with xattrs on some ancient 
(or non-Linux) kernel without xattr support — everything will be fine, 
even though there's no way to access them — but if the fsck tool doesn't 
understand them, it wouldn't be able to diagnose xattr corruption.

I'd either warn the user about unknown compat features, telling them 
they're on their own if something in the FS is still broken; or barf 
outright.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH v2 1/5] xfs_repair: refuse to run if we don't recognize version or feature flags

[Eric Sandeen]

On 5/27/15 11:04 AM, Fanael Linithien wrote:
> I'm not entirely sure silently ignoring unknown compat features in
> xfs_repair is a good idea. Consider this ext2 example: xattr support
> is a compat flag. It's okay to rw mount a FS with xattrs on some
> ancient (or non-Linux) kernel without xattr support — everything will
> be fine, even though there's no way to access them — but if the fsck
> tool doesn't understand them, it wouldn't be able to diagnose xattr
> corruption.

Hm, good point.  We have no "compat" features atm, so it's not really clear;
if nothing else, though, those features will have to go un-checked, if there
is any checking to do.  So yeah, maybe best to keep it as an exclusion.

-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH v2 1/5] xfs_repair: refuse to run if we don't recognize version or feature flags

[Darrick J. Wong]

On Wed, May 27, 2015 at 05:19:00PM +0200, Fanael Linithien wrote:
> > Apparently xfs_repair running on a v5 filesystem doesn't check the
> > compat, rocompat, or incompat feature flags for bits that it doesn't
> > know about, which means that old xfs_repairs can wreak havoc. So,
> > strengthen the checks to prevent repair from "repairing" anything it
> > doesn't understand.
> >
> > v2: Move the complaint code after the version number check, and print
> > the actual feature bits that we don't recognize.
> >
> > Signed-off-by: Darrick J. Wong <darrick.wong at oracle.com>
> > ---
> >  repair/versions.c |   33 +++++++++++++++++++++++++++++++++
> >  1 file changed, 33 insertions(+)
> >
> > diff --git a/repair/versions.c b/repair/versions.c
> > index c1dff72..dbe41a4 100644
> > --- a/repair/versions.c
> > +++ b/repair/versions.c
> > @@ -175,6 +175,39 @@ _("WARNING:  you have disallowed
> superblock-feature-bits-allowed\n"
> >          }
> >      }
> >
> > +    /* Look for V5 feature flags we don't know about */
> > +    if (XFS_SB_VERSION_NUM(sb) >= XFS_SB_VERSION_5) {
> > +        if (xfs_sb_has_ro_compat_feature(sb,
> > +                XFS_SB_FEAT_RO_COMPAT_UNKNOWN)) {
> > +            do_warn(
> > +_("Superblock has unknown read-only compatible features (0x%x)
> enabled.\n"
> > +"Using a more recent xfs_repair is recommended.\n"),
> > +                sb->sb_features_ro_compat &
> > +                        XFS_SB_FEAT_RO_COMPAT_UNKNOWN);
> > +            issue_warning = 1;
> > +        }
> > +        if (xfs_sb_has_incompat_feature(sb,
> > +                XFS_SB_FEAT_INCOMPAT_UNKNOWN)) {
> > +            do_warn(
> > +_("Superblock has unknown read-only compatible features (0x%x)
> enabled.\n"
> > +"Using a more recent xfs_repair is recommended.\n"),
> > +                sb->sb_features_ro_compat &
> > +                        XFS_SB_FEAT_RO_COMPAT_UNKNOWN);
> 
> Copy/paste error, do you mean incompat?
> 
> > +            issue_warning = 1;
> > +        }
> > +        if (xfs_sb_has_compat_feature(sb,
> > +                XFS_SB_FEAT_COMPAT_UNKNOWN)) {
> > +            do_warn(
> > +_("Superblock has unknown compatible features (0x%x) enabled.\n"
> > +"Using a more recent xfs_repair is recommended.\n"),
> > +                sb->sb_features_ro_compat &
> > +                        XFS_SB_FEAT_RO_COMPAT_UNKNOWN);
> 
> compat here?

Err, yes.  Thank you for catching all these.  No more patching past 10pm for
me.... :)

--D

> 
> > +            issue_warning = 1;
> > +        }
> > +        if (issue_warning)
> > +            return 1;
> > +    }
> > +
> >      if (xfs_sb_version_hasattr(sb))  {
> >          if (!fs_attributes_allowed)  {
> >              if (!no_modify)  {
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH v2 1/5] xfs_repair: refuse to run if we don't recognize version or feature flags

[Darrick J. Wong]

On Wed, May 27, 2015 at 06:04:57PM +0200, Fanael Linithien wrote:
> On 2015-05-27 17:27, Eric Sandeen wrote:
> >
> >I wonder if something like:
> >
> >	/* Look for V5 feature flags we don't know about */
> >	if (XFS_SB_VERSION_NUM(sb) >= XFS_SB_VERSION_5 &&
> >	    (xfs_sb_has_ro_compat_feature(sb, XFS_SB_FEAT_RO_COMPAT_UNKNOWN) ||
> >	     xfs_sb_has_incompat_feature(sb, XFS_SB_FEAT_INCOMPAT_UNKNOWN) ||
> >	     xfs_sb_has_compat_feature(sb, XFS_SB_FEAT_COMPAT_UNKNOWN))) {
> >		printf("unknown feature flags 0x%x/0x%x/0x%x\n",
> >			sb->sb_features_ro_compat & XFS_SB_FEAT_RO_COMPAT_UNKNOWN,
> >			sb->sb_features_incompat & XFS_SB_FEAT_INCOMPAT_UNKNOWN,
> >			sb->sb_features_compat & XFS_SB_FEAT_COMPAT_UNKNOWN);
> >		...
> >
> >would suffice, given that the user will need to read code to unerstand the hex
> >values, anyway.

<shrug> I guess repair could do that.  I'd broken things out separately
(key word: broken ;)) to avoid complaining about whichever fields aren't
broken, but I suppose it doesn't really matter.

> >	
> >Hm, and as Fanael also pointed out, "compat" features ... should be ok, right,
> >and can be removed from the exclusions?
> 
> I'm not entirely sure silently ignoring unknown compat features in
> xfs_repair is a good idea. Consider this ext2 example: xattr support
> is a compat flag. It's okay to rw mount a FS with xattrs on some
> ancient (or non-Linux) kernel without xattr support — everything
> will be fine, even though there's no way to access them — but if the
> fsck tool doesn't understand them, it wouldn't be able to diagnose
> xattr corruption.
> 
> I'd either warn the user about unknown compat features, telling them
> they're on their own if something in the FS is still broken; or barf
> outright.

I prefer any repair tool be forthcoming about not knowing what something is.

--D

> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH v3 1/5] xfs_repair: refuse to run if we don't recognize version or feature flags

[Darrick J. Wong]

Apparently xfs_repair running on a v5 filesystem doesn't check the
compat, rocompat, or incompat feature flags for bits that it doesn't
know about, which means that old xfs_repairs can wreak havoc.  So,
strengthen the checks to prevent repair from "repairing" anything it
doesn't understand.

v2: Move the complaint code after the version number check, and print
the actual feature bits that we don't recognize.

v3: Fix some copy-pasta errors, and simpify the warning messages.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 repair/versions.c |   14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/repair/versions.c b/repair/versions.c
index c1dff72..10bcd29 100644
--- a/repair/versions.c
+++ b/repair/versions.c
@@ -175,6 +175,20 @@ _("WARNING:  you have disallowed superblock-feature-bits-allowed\n"
 		}
 	}
 
+	/* Look for V5 feature flags we don't know about */
+	if (XFS_SB_VERSION_NUM(sb) >= XFS_SB_VERSION_5 &&
+	    (xfs_sb_has_compat_feature(sb, XFS_SB_FEAT_COMPAT_UNKNOWN) ||
+	     xfs_sb_has_ro_compat_feature(sb, XFS_SB_FEAT_RO_COMPAT_UNKNOWN) ||
+	     xfs_sb_has_incompat_feature(sb, XFS_SB_FEAT_INCOMPAT_UNKNOWN))) {
+		do_warn(
+_("Superblock has unknown compat/rocompat/incompat features (0x%x/0x%x/0x%x).\n"
+  "Using a more recent xfs_repair is recommended.\n"),
+			sb->sb_features_compat & XFS_SB_FEAT_COMPAT_UNKNOWN,
+			sb->sb_features_ro_compat & XFS_SB_FEAT_RO_COMPAT_UNKNOWN,
+			sb->sb_features_incompat & XFS_SB_FEAT_INCOMPAT_UNKNOWN);
+		return 1;
+	}
+
 	if (xfs_sb_version_hasattr(sb))  {
 		if (!fs_attributes_allowed)  {
 			if (!no_modify)  {

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH v3 1/5] xfs_repair: refuse to run if we don't recognize version or feature flags

[Fanael Linithien]

2015-05-27 20:15 GMT+02:00 Darrick J. Wong <darrick.wong@oracle.com>:
> Apparently xfs_repair running on a v5 filesystem doesn't check the
> compat, rocompat, or incompat feature flags for bits that it doesn't
> know about, which means that old xfs_repairs can wreak havoc.  So,
> strengthen the checks to prevent repair from "repairing" anything it
> doesn't understand.
>
> v2: Move the complaint code after the version number check, and print
> the actual feature bits that we don't recognize.
>
> v3: Fix some copy-pasta errors, and simpify the warning messages.
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---

Reviewed-by: Fanael Linithien <fanael4@gmail.com>

>  repair/versions.c |   14 ++++++++++++++
>  1 file changed, 14 insertions(+)
>
> diff --git a/repair/versions.c b/repair/versions.c
> index c1dff72..10bcd29 100644
> --- a/repair/versions.c
> +++ b/repair/versions.c
> @@ -175,6 +175,20 @@ _("WARNING:  you have disallowed superblock-feature-bits-allowed\n"
>                 }
>         }
>
> +       /* Look for V5 feature flags we don't know about */
> +       if (XFS_SB_VERSION_NUM(sb) >= XFS_SB_VERSION_5 &&
> +           (xfs_sb_has_compat_feature(sb, XFS_SB_FEAT_COMPAT_UNKNOWN) ||
> +            xfs_sb_has_ro_compat_feature(sb, XFS_SB_FEAT_RO_COMPAT_UNKNOWN) ||
> +            xfs_sb_has_incompat_feature(sb, XFS_SB_FEAT_INCOMPAT_UNKNOWN))) {
> +               do_warn(
> +_("Superblock has unknown compat/rocompat/incompat features (0x%x/0x%x/0x%x).\n"
> +  "Using a more recent xfs_repair is recommended.\n"),
> +                       sb->sb_features_compat & XFS_SB_FEAT_COMPAT_UNKNOWN,
> +                       sb->sb_features_ro_compat & XFS_SB_FEAT_RO_COMPAT_UNKNOWN,
> +                       sb->sb_features_incompat & XFS_SB_FEAT_INCOMPAT_UNKNOWN);
> +               return 1;
> +       }
> +
>         if (xfs_sb_version_hasattr(sb))  {
>                 if (!fs_attributes_allowed)  {
>                         if (!no_modify)  {

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 5/5] xfs_db: enable blocktrash for checksummed filesystems

[Darrick J. Wong]

On Tue, May 26, 2015 at 03:51:59PM -0700, Darrick J. Wong wrote:
> Disable the write verifiers when we're trashing a block.  With this
> in place, create a xfs fuzzer script that formats, populates, corrupts,
> tries to use, repairs, and tries again to use a crash test xfs image.
> Hopefully this will shake out some v5 filesystem bugs.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
>  db/check.c    |    7 +
>  db/xfsfuzz.sh |  305 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 312 insertions(+)
>  create mode 100755 db/xfsfuzz.sh
> 
> 
> diff --git a/db/check.c b/db/check.c
> index 8f8096d..2c2a02e 100644
> --- a/db/check.c
> +++ b/db/check.c
> @@ -953,6 +953,7 @@ blocktrash_b(
>  	int		mask;
>  	int		newbit;
>  	int		offset;
> +	const struct xfs_buf_ops *stashed_ops;
>  	static char	*modestr[] = {
>  		N_("zeroed"), N_("set"), N_("flipped"), N_("randomized")
>  	};
> @@ -963,6 +964,8 @@ blocktrash_b(
>  	push_cur();
>  	set_cur(&typtab[DBM_UNKNOWN],

So... seeing as the the TYP_* and DBM_* values don't correspond and every other
user of typtab uses the TYP_ values for array index, what's the point of
unconditionally using the AGF verifier on this block?  Passing NULL as the
first argument doesn't seem to hurt anything.

(DBM_UNKNOWN == TYP_AGF)

Seeing as we're going to trash the block anyway, why bother?

>  		XFS_AGB_TO_DADDR(mp, agno, agbno), blkbb, DB_RING_IGN, NULL);
> +	stashed_ops = iocur_top->bp->b_ops;
> +	iocur_top->bp->b_ops = NULL;
>  	if ((buf = iocur_top->data) == NULL) {
>  		dbprintf(_("can't read block %u/%u for trashing\n"), agno, agbno);
>  		pop_cur();
> @@ -993,6 +996,7 @@ blocktrash_b(
>  			buf[byte] &= ~mask;
>  	}
>  	write_cur();
> +	iocur_top->bp->b_ops = stashed_ops;
>  	pop_cur();
>  	printf(_("blocktrash: %u/%u %s block %d bit%s starting %d:%d %s\n"),
>  		agno, agbno, typename[type], len, len == 1 ? "" : "s",
> @@ -1049,9 +1053,12 @@ blocktrash_f(
>  		   (1 << DBM_BTINO) |
>  		   (1 << DBM_DIR) |
>  		   (1 << DBM_INODE) |
> +		   (1 << DBM_LOG) |

It's useful to be able to fuzz the log, but that'll bias the block selection
towards the log.  Maybe it should be masked off by default in tmask = goodmask
below?

>  		   (1 << DBM_QUOTA) |
>  		   (1 << DBM_RTBITMAP) |
>  		   (1 << DBM_RTSUM) |
> +		   (1 << DBM_SYMLINK) |
> +		   (1 << DBM_BTFINO) |
>  		   (1 << DBM_SB);
>  	while ((c = getopt(argc, argv, "0123n:s:t:x:y:")) != EOF) {
>  		switch (c) {
> diff --git a/db/xfsfuzz.sh b/db/xfsfuzz.sh
> new file mode 100755
> index 0000000..fc40e97
> --- /dev/null
> +++ b/db/xfsfuzz.sh

[working on stuffing this into xfstests]
[maybe I'll see about doing likewise for the ext4 fuzzer]

--D

> @@ -0,0 +1,305 @@
> +#!/bin/bash
> +
> +# Test harness to fuzz a filesystem over and over...
> +# Copyright (C) 2014 Oracle.
> +
> +DIR=/tmp
> +PASSES=10000
> +SZ=32m
> +SCRIPT_DIR="$(dirname "$0")"
> +FEATURES="-m crc=1,finobt=1"
> +xEATURES="-m crc=0"
> +BLK_SZ=4096
> +INODE_SZ=512
> +RUN_FSCK=1
> +OVERRIDE_PATH=1
> +MAX_FSCK=10
> +SRCDIR=/etc
> +FUZZ_ARGS="-3 -n 32"
> +XFS_REPAIR_OPTS="-P"
> +
> +print_help() {
> +	echo "Usage: $0 OPTIONS"
> +	echo "-b:	FS block size is this. (${BLK_SZ})"
> +	echo "-B:	Corrupt this many bytes per run."
> +	echo "-d:	Create test files in this directory. (${DIR})"
> +	echo "-f:	Do not run xfs_repair after each pass."
> +	echo "-I:	Create inodes of this size. (${INODE_SZ})"
> +	echo "-n:	Run this many passes. (${PASSES})"
> +	echo "-O:	Pass this to mkfs.xfs."
> +	echo "-p:	Use system's xfsprogs tools."
> +	echo "-s:	Create FS images of this size. (${SZ})"
> +	echo "-S:	Copy files from this dir. (${SRCDIR})"
> +	echo "-x:	Run xfs_repair at most this many times. (${MAX_FSCK})"
> +	exit 0
> +}
> +
> +GETOPT="d:n:s:O:I:b:B:fpx:S:"
> +
> +while getopts "${GETOPT}" opt; do
> +	case "${opt}" in
> +	"B")
> +		FUZZ_ARGS="-3 -n ${OPTARG}"
> +		;;
> +	"d")
> +		DIR="${OPTARG}"
> +		;;
> +	"n")
> +		PASSES="${OPTARG}"
> +		;;
> +	"s")
> +		SZ="${OPTARG}"
> +		;;
> +	"O")
> +		FEATURES="${OPTARG}"
> +		;;
> +	"I")
> +		INODE_SZ="${OPTARG}"
> +		;;
> +	"b")
> +		BLK_SZ="${OPTARG}"
> +		;;
> +	"f")
> +		RUN_FSCK=0
> +		;;
> +	"p")
> +		OVERRIDE_PATH=0
> +		;;
> +	"x")
> +		MAX_FSCK="${OPTARG}"
> +		;;
> +	"S")
> +		SRCDIR="${OPTARG}"
> +		;;
> +	*)
> +		print_help
> +		;;
> +	esac
> +done
> +
> +if [ "${OVERRIDE_PATH}" -gt 0 ]; then
> +	PATH="${SCRIPT_DIR}:${SCRIPT_DIR}/../repair/:${SCRIPT_DIR}/../db/:${SCRIPT_DIR}/../mkfs/:${PATH}"
> +	export PATH
> +fi
> +
> +TESTDIR="${DIR}/tests/"
> +TESTMNT="${DIR}/mnt/"
> +BASE_IMG="${DIR}/xfsfuzz.img"
> +
> +# Set up FS image
> +echo "+ create fs image"
> +umount "${TESTDIR}"
> +umount "${TESTMNT}"
> +rm -rf "${TESTDIR}"
> +rm -rf "${TESTMNT}"
> +mkdir -p "${TESTDIR}"
> +mkdir -p "${TESTMNT}"
> +rm -rf "${BASE_IMG}"
> +truncate -s "${SZ}" "${BASE_IMG}"
> +mkfs.xfs -f ${FEATURES} -b "size=${BLK_SZ}" -i "size=${INODE_SZ}" "${BASE_IMG}"
> +if [ $? -ne 0 ]; then
> +	exit $?
> +fi
> +
> +# Populate FS image
> +echo "+ populate fs image"
> +modprobe loop
> +mount "${BASE_IMG}" "${TESTMNT}" -o loop
> +if [ $? -ne 0 ]; then
> +	exit $?
> +fi
> +SRC_SZ="$(du -ks "${SRCDIR}" | awk '{print $1}')"
> +FS_SZ="$(( $(stat -f "${TESTMNT}" -c '%a * %S') / 1024 ))"
> +NR="$(( (FS_SZ * 6 / 10) / SRC_SZ ))"
> +if [ "${NR}" -lt 1 ]; then
> +	NR=1
> +fi
> +echo "+ make ${NR} copies"
> +seq 1 "${NR}" | while read nr; do
> +	cp -pRdu "${SRCDIR}" "${TESTMNT}/test.${nr}" 2> /dev/null
> +done
> +umount "${TESTMNT}"
> +xfs_repair ${XFS_REPAIR_OPTS} -vn "${BASE_IMG}"
> +if [ $? -ne 0 ]; then
> +	echo "fsck failed??"
> +	exit 1
> +fi
> +
> +# Run tests
> +echo "+ run test"
> +ret=0
> +seq 1 "${PASSES}" | while read pass; do
> +	echo "+ pass ${pass}"
> +	PASS_IMG="${TESTDIR}/xfsfuzz-${pass}.img"
> +	FSCK_IMG="${TESTDIR}/xfsfuzz-${pass}.fsck"
> +	FUZZ_LOG="${TESTDIR}/xfsfuzz-${pass}.fuzz.log"
> +	OPS_LOG="${TESTDIR}/xfsfuzz-${pass}.ops.log"
> +
> +	echo "++ copy image"
> +	cp "${BASE_IMG}" "${PASS_IMG}"
> +	if [ $? -ne 0 ]; then
> +		exit $?
> +	fi
> +	xfs_db -x -c "label xfsfuzz-${pass}" "${PASS_IMG}"
> +
> +	echo "++ corrupt image"
> +	xfs_db -x -c blockget -c "blocktrash ${FUZZ_ARGS}" "${PASS_IMG}" > "${FUZZ_LOG}"
> +#	res=$?
> +#	if [ "${res}" -ne 0 ]; then
> +#		echo "blocktrash returns ${res}"
> +#		exit "${res}"
> +#	fi
> +
> +	echo "++ mount image"
> +	mount "${PASS_IMG}" "${TESTMNT}" -o loop
> +	res=$?
> +
> +	if [ "${res}" -eq 0 ]; then
> +		echo "+++ ls -laR"
> +		ls -laR "${TESTMNT}/test.1/" > /dev/null 2> "${OPS_LOG}"
> +
> +		echo "+++ cat files"
> +		find "${TESTMNT}/test.1/" -type f -size -1048576k -print0 | xargs -0 cat > /dev/null 2>> "${OPS_LOG}"
> +
> +		echo "+++ expand"
> +		find "${TESTMNT}/" -type f 2> /dev/null | head -n 50000 | while read f; do
> +			attr -l "$f" > /dev/null 2>> "${OPS_LOG}"
> +			if [ -f "$f" -a -w "$f" ]; then
> +				dd if=/dev/zero bs="${BLK_SZ}" count=1 >> "$f" 2>> "${OPS_LOG}"
> +			fi
> +			mv "$f" "$f.longer" > /dev/null 2>> "${OPS_LOG}"
> +		done
> +		sync
> +
> +		echo "+++ create files"
> +		cp -pRdu "${SRCDIR}" "${TESTMNT}/test.moo" 2>> "${OPS_LOG}"
> +		sync
> +
> +		echo "+++ remove files"
> +		rm -rf "${TESTMNT}/test.moo" 2>> "${OPS_LOG}"
> +
> +		umount "${TESTMNT}"
> +		res=$?
> +		if [ "${res}" -ne 0 ]; then
> +			ret=1
> +			break
> +		fi
> +		sync
> +	fi
> +	if [ "${RUN_FSCK}" -gt 0 ]; then
> +		cp "${PASS_IMG}" "${FSCK_IMG}"
> +		pass_img_sz="$(stat -c '%s' "${PASS_IMG}")"
> +
> +		seq 1 "${MAX_FSCK}" | while read fsck_pass; do
> +			echo "++ fsck pass ${fsck_pass}: $(which xfs_repair) -v ${FSCK_IMG}"
> +			FSCK_LOG="${TESTDIR}/xfsfuzz-${pass}-${fsck_pass}.log"
> +			echo "repairing" > "${FSCK_LOG}"
> +			xfs_repair ${XFS_REPAIR_OPTS} -v "${FSCK_IMG}" >> "${FSCK_LOG}" 2>&1
> +			res=$?
> +			if [ "${res}" -eq 0 ]; then
> +				echo "reverify" >> "${FSCK_LOG}"
> +				xfs_repair ${XFS_REPAIR_OPTS} -v -n "${FSCK_IMG}" >> "${FSCK_LOG}" 2>&1
> +				res=$?
> +			fi
> +			echo "++ fsck returns ${res}"
> +			if [ "${res}" -eq 0 ]; then
> +				exit 0
> +			elif [ "${res}" -eq 2 ]; then
> +				# replay log?
> +				echo "replaying log" >> "${FSCK_LOG}"
> +				dmesg > /tmp/a
> +				mount "${FSCK_IMG}" "${TESTMNT}" -o loop
> +				res=$?
> +				if [ "${res}" -gt 0 ]; then
> +					echo "+++ zeroing log"
> +					echo "zeroing log" >> "${FSCK_LOG}"
> +					xfs_repair ${XFS_REPAIR_OPTS} -L -v "${FSCK_IMG}" >> "${FSCK_LOG}" 2>&1
> +				else
> +					umount "${TESTMNT}"
> +				fi
> +				dmesg > /tmp/b
> +				diff -u /tmp/a /tmp/b >> "${FSCK_LOG}"
> +			elif [ "${fsck_pass}" -eq "${MAX_FSCK}" ]; then
> +				echo "++ fsck did not fix in ${MAX_FSCK} passes."
> +				exit 1
> +			fi
> +			if [ "${fsck_pass}" -gt 1 ]; then
> +				diff -u "${TESTDIR}/xfsfuzz-${pass}-$((fsck_pass - 1)).log" "${FSCK_LOG}"
> +				if [ $? -eq 0 ]; then
> +					echo "++ fsck makes no progress"
> +					exit 2
> +				fi
> +			fi
> +
> +			fsck_img_sz="$(stat -c '%s' "${FSCK_IMG}")"
> +			if [ "${fsck_img_sz}" -ne "${pass_img_sz}" ]; then
> +				echo "++ fsck image size changed"
> +				exit 3
> +			fi
> +		done
> +		fsck_loop_ret=$?
> +		if [ "${fsck_loop_ret}" -gt 0 ]; then
> +			break;
> +		fi
> +	fi
> +
> +	echo "+++ check fs for round 2"
> +	FSCK_LOG="${TESTDIR}/xfsfuzz-${pass}-round2.log"
> +	xfs_repair ${XFS_REPAIR_OPTS} -v -n "${FSCK_IMG}" > "${FSCK_LOG}" 2>&1
> +	res=$?
> +	if [ "${res}" -ne 0 ]; then
> +		echo "++++ fsck failed."
> +		exit 1
> +	fi
> +
> +	echo "++ mount image (2)"
> +	mount "${FSCK_IMG}" "${TESTMNT}" -o loop
> +	res=$?
> +
> +	if [ "${res}" -eq 0 ]; then
> +		echo "+++ ls -laR (2)"
> +		ls -laR "${TESTMNT}/test.1/" > /dev/null 2> "${OPS_LOG}"
> +
> +		echo "+++ cat files (2)"
> +		find "${TESTMNT}/test.1/" -type f -size -1048576k -print0 | xargs -0 cat > /dev/null 2>> "${OPS_LOG}"
> +
> +		echo "+++ expand (2)"
> +		find "${TESTMNT}/" -type f 2> /dev/null | head -n 50000 | while read f; do
> +			attr -l "$f" > /dev/null 2>> "${OPS_LOG}"
> +			if [ -f "$f" -a -w "$f" ]; then
> +				dd if=/dev/zero bs="${BLK_SZ}" count=1 >> "$f" 2>> "${OPS_LOG}"
> +			fi
> +			mv "$f" "$f.longer" > /dev/null 2>> "${OPS_LOG}"
> +		done
> +		sync
> +
> +		echo "+++ create files (2)"
> +		cp -pRdu "${SRCDIR}" "${TESTMNT}/test.moo" 2>> "${OPS_LOG}"
> +		sync
> +
> +		echo "+++ remove files (2)"
> +		rm -rf "${TESTMNT}/test.moo" 2>> "${OPS_LOG}"
> +
> +		umount "${TESTMNT}"
> +		res=$?
> +		if [ "${res}" -ne 0 ]; then
> +			ret=1
> +			break
> +		fi
> +		sync
> +
> +		echo "+++ check fs (2)"
> +		xfs_repair ${XFS_REPAIR_OPTS} -v -n "${FSCK_IMG}" >> "${FSCK_LOG}" 2>&1
> +		res=$?
> +		if [ "${res}" -ne 0 ]; then
> +			echo "++++ fsck failed."
> +			exit 1
> +		fi
> +	else
> +		echo "++ mount(2) failed with ${res}"
> +		exit 1
> +	fi
> +	rm -rf "${FSCK_IMG}" "${PASS_IMG}" "${FUZZ_LOG}" "${TESTDIR}"/xfsfuzz*.log
> +done
> +
> +exit $ret
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH v2 2/5] xfs_repair: better checking of v5 metadata fields

[Dave Chinner]

On Tue, May 26, 2015 at 10:44:50PM -0700, Darrick J. Wong wrote:
> Check the UUID, owner, and block number fields during repair, looking for
> blocks that fail either the checksum or the data structure verifiers.  For
> directories we can simply rebuild corrupt/broken index data, though for
> anything else we have to toss out the broken object.
> 
> v2: Refactor the directory block header checks into a single function.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

FYI, just a couple of quick whitespacy things, which I've fixed on
commit.

> diff --git a/repair/phase6.c b/repair/phase6.c
> index c09b394..7dd868c 100644
> --- a/repair/phase6.c
> +++ b/repair/phase6.c
> @@ -1925,6 +1925,69 @@ _("entry \"%s\" in dir inode %" PRIu64 " inconsistent with .. value (%" PRIu64 "
>  	freetab->ents[db].s = 0;
>  }
>  
> +/* check v5 metadata */
> +static int
> +__check_dir3_header(
> +	xfs_mount_t			*mp,
> +	xfs_buf_t			*bp,

Don't use the typedef versions - struct xfs_mount. No need for quite
so much whitespace, either - this sort of thing is fine if there's
no need for larger alignment steps:

	struct xfs_mount *mp,
	struct xfs_buf	*bp,
	....

> +	/* verify uuid */
> +	if (platform_uuid_compare(uuid,
> +				  &mp->m_sb.sb_uuid) != 0) {

No need to split that line.

> +static int
> +check_da3_header(
> +	xfs_mount_t			*mp,
> +	xfs_buf_t			*bp,
> +	xfs_ino_t			ino)
> +{
> +	struct xfs_da3_blkinfo		*info;
> +
> +	info = bp->b_addr;

	struct xfs_da3_blkinfo	*info = bp->b_addr;

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs