[PATCH 00/20] Enable lowmem repair for fs/subvolume tree

[PATCH 00/20] Enable lowmem repair for fs/subvolume tree

[Su Yue]

It can be feched from my github:
https://github.com/Damenly/btrfs-progs.git lowmem_repair

This patchset can repair errors found in fs tree in lowmem mode.

This patchset request includes:
1) Repair inode nbytes error.
2) Repair dir isize error.
3) Repair orpahan inode item.
4) Repair dir item/index missing/mismatch.
5) Repair inode ref missing/mismatch.
6) Repair inode item missing.
7) Repair inode nlink error.
8) Punch file extent hole.
9) Let test-fsck test cases which can be repaired in lowmem mode.

All cases have been tested except fsck-test/006 since it can't be
repaired in original mode.

Qu Wenruo (1):
  btrfs-progs: fsck-check: Allow fsck check test to repair in lowmem
    mode for certain test cases

Su Yue (19):
  btrfs-progs: cmds-check.c: supports inode nbytes fix in lowmem
  btrfs-progs: cmds-check.c: supports dir isize fix in lowmem
  btrfs-progs: cmds-check.c: inode orphan item repair
  btrfs-progs: cmds-check.c: change find_inode_ref's arg
  btrfs-progs: cmds-check.c: modify check_fs_first_inode
  btrfs-progs: cmds-check.c: change find_dir_index/item
  btrfs-progs: cmds-check.c: introduce print_inode_ref
  btrfs-progs: cmds-check.c: print_dir_item_err
  btrfs-progs: cmds-check.c: introduce count_dir_isize
  btrfs-progs: dir-item.c: modify btrfs_insert_dir_item
  btrfs-progs: inode.c: alter btrfs_add_link
  btrfs-progs: cmds-check.c: introduce __create_inode_item
  btrfs-progs: cmds-check.c: repair_inode_item_missing
  btrfs-progs: cmds-check.c: repair_fs_first_inode
  btrfs-progs: cmds-check.c: introduce repair_ternary_lowmem
  btrfs-progs: cmds-check.c: Introduce repair_dir_item
  btrfs-progs: cmds-check.c: repair inode ref
  btrfs-progs: cmds-check.c: repair nlinks lowmem
  btrfs-progs: cmds-check.c: add punch_extent_hole

 cmds-check.c                                       | 1677 +++++++++++++++-----
 convert/main.c                                     |    2 +-
 ctree.h                                            |    2 +-
 dir-item.c                                         |   13 +-
 inode.c                                            |   46 +-
 tests/common.local                                 |   14 +-
 .../fsck-tests/004-no-dir-index/.lowmem_repairable |    0
 .../009-no-dir-item-or-index/.lowmem_repairable    |    0
 .../010-no-rootdir-inode-item/.lowmem_repairable   |    0
 .../011-no-inode-item/.lowmem_repairable           |    0
 .../016-wrong-inode-nbytes/.lowmem_repairable      |    0
 .../017-missing-all-file-extent/.lowmem_repairable |    0
 12 files changed, 1363 insertions(+), 391 deletions(-)
 create mode 100644 tests/fsck-tests/004-no-dir-index/.lowmem_repairable
 create mode 100644 tests/fsck-tests/009-no-dir-item-or-index/.lowmem_repairable
 create mode 100644 tests/fsck-tests/010-no-rootdir-inode-item/.lowmem_repairable
 create mode 100644 tests/fsck-tests/011-no-inode-item/.lowmem_repairable
 create mode 100644 tests/fsck-tests/016-wrong-inode-nbytes/.lowmem_repairable
 create mode 100644 tests/fsck-tests/017-missing-all-file-extent/.lowmem_repairable

-- 
2.11.1

[PATCH 01/20] btrfs-progs: cmds-check.c: supports inode nbytes fix in lowmem

[Su Yue]

Add a function 'repair_inode_nbytes_lowmem' to correct inode item nbytes
error in lowmem mode.

Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
---
 cmds-check.c | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 76 insertions(+), 6 deletions(-)

diff --git a/cmds-check.c b/cmds-check.c
index 9cc1932c..40f9d21e 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -1907,6 +1907,9 @@ static int process_one_leaf_v2(struct btrfs_root *root, struct btrfs_path *path,
 again:
 	err |= check_inode_item(root, path, ext_ref);
 
+	/* remodified cur since check_inode_item may change path */
+	cur = path->nodes[0];
+
 	if (err & LAST_ITEM)
 		goto out;
 
@@ -2256,6 +2259,7 @@ static int walk_down_tree_v2(struct btrfs_root *root, struct btrfs_path *path,
 			}
 			ret = process_one_leaf_v2(root, path, nrefs,
 						  level, ext_ref);
+			cur = path->nodes[*level];
 			break;
 		} else {
 			ret = btrfs_check_node(root, NULL, cur);
@@ -4819,10 +4823,69 @@ static int check_file_extent(struct btrfs_root *root, struct btrfs_key *fkey,
 }
 
 /*
+ * Set inode item nbytes to @nbytes
+ *
+ * Returns <0  means on error
+ * Returns  0  means successful repair
+ */
+static int repair_inode_nbytes_lowmem(struct btrfs_root *root,
+				      struct btrfs_path *path,
+				      u64 ino, u64 nbytes)
+{
+	struct btrfs_trans_handle *trans;
+	struct btrfs_inode_item *ii;
+	struct btrfs_key key;
+	struct btrfs_key research_key;
+	int ret;
+	int ret2;
+
+	key.objectid = ino;
+	key.type = BTRFS_INODE_ITEM_KEY;
+	key.offset = 0;
+	btrfs_item_key_to_cpu(path->nodes[0], &research_key, path->slots[0]);
+	btrfs_release_path(path);
+
+	trans = btrfs_start_transaction(root, 1);
+	if (IS_ERR(trans)) {
+		ret = PTR_ERR(trans);
+		goto out;
+	}
+
+	ret = btrfs_search_slot(trans, root, &key, path, 0, 1);
+	if (ret < 0)
+		goto out;
+	if (ret > 0) {
+		ret = -ENOENT;
+		goto out;
+	}
+
+	ii = btrfs_item_ptr(path->nodes[0], path->slots[0],
+			    struct btrfs_inode_item);
+	btrfs_set_inode_nbytes(path->nodes[0], ii, nbytes);
+	btrfs_mark_buffer_dirty(path->nodes[0]);
+
+	printf("reset nbytes for inode %llu root %llu\n", ino,
+	       root->root_key.objectid);
+
+	btrfs_commit_transaction(trans, root);
+out:
+	if (ret < 0)
+		error("failed to reset nbytes for inode %llu root %llu due to %s",
+		      ino, root->root_key.objectid, strerror(-ret));
+
+	/* research path */
+	btrfs_release_path(path);
+	ret2 = btrfs_search_slot(NULL, root, &research_key, path, 0, 0);
+	return ret2 < 0 ? ret2 : ret;
+}
+
+/*
  * Check INODE_ITEM and related ITEMs (the same inode number)
  * 1. check link count
  * 2. check inode ref/extref
  * 3. check dir item/index
+ * Be Careful, if repair is enable, @path may be changed.
+ * Remember to reassign any context about @path in repair mode.
  *
  * @ext_ref:	the EXTENDED_IREF feature
  *
@@ -4972,9 +5035,17 @@ out:
 		}
 
 		if (nbytes != extent_size) {
-			err |= NBYTES_ERROR;
-			error("root %llu INODE[%llu] nbytes(%llu) not equal to extent_size(%llu)",
-			      root->objectid, inode_id, nbytes, extent_size);
+			if (repair) {
+				ret = repair_inode_nbytes_lowmem(root, path,
+								 inode_id,
+								 extent_size);
+			}
+			if (!repair || ret) {
+				err |= NBYTES_ERROR;
+				error("root %llu INODE[%llu] nbytes(%llu) not equal to extent_size(%llu)",
+				      root->objectid, inode_id, nbytes,
+				      extent_size);
+			}
 		}
 	}
 
@@ -12798,11 +12869,10 @@ int cmd_check(int argc, char **argv)
 	}
 
 	/*
-	 * Not supported yet
+	 * Support partially
 	 */
 	if (repair && check_mode == CHECK_MODE_LOWMEM) {
-		error("low memory mode doesn't support repair yet");
-		exit(1);
+		warning("low memory mode support repair partially");
 	}
 
 	radix_tree_init();
-- 
2.11.1

[PATCH 02/20] btrfs-progs: cmds-check.c: supports dir isize fix in lowmem

[Su Yue]

Add a function 'repair_dir_isize_lowmem' to support dir isize
repair in lowmem mode.

Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
---
 cmds-check.c | 69 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 66 insertions(+), 3 deletions(-)

diff --git a/cmds-check.c b/cmds-check.c
index 40f9d21e..f13ce317 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -4880,6 +4880,62 @@ out:
 }
 
 /*
+ * Set dir isize to @isize
+ *
+ * Returns <0  means on error
+ * Returns  0  means successful repair
+ */
+static int repair_dir_isize_lowmem(struct btrfs_root *root,
+				   struct btrfs_path *path,
+				   u64 ino, u64 isize)
+{
+	struct btrfs_trans_handle *trans;
+	struct btrfs_inode_item *ii;
+	struct btrfs_key key;
+	struct btrfs_key research_key;
+	int ret;
+	int ret2;
+
+	key.objectid = ino;
+	key.type = BTRFS_INODE_ITEM_KEY;
+	key.offset = 0;
+
+	btrfs_item_key_to_cpu(path->nodes[0], &research_key, path->slots[0]);
+	btrfs_release_path(path);
+
+	trans = btrfs_start_transaction(root, 1);
+	if (IS_ERR(trans)) {
+		ret = PTR_ERR(trans);
+		goto out;
+	}
+
+	ret = btrfs_search_slot(trans, root, &key, path, 0, 1);
+	if (ret < 0)
+		goto out;
+	if (ret > 0) {
+		ret = -ENOENT;
+		goto out;
+	}
+
+	ii = btrfs_item_ptr(path->nodes[0], path->slots[0],
+			    struct btrfs_inode_item);
+	btrfs_set_inode_size(path->nodes[0], ii, isize);
+	btrfs_mark_buffer_dirty(path->nodes[0]);
+
+	printf("reset isize for inode %llu root %llu\n", ino,
+	       root->root_key.objectid);
+
+	btrfs_commit_transaction(trans, root);
+out:
+	if (ret < 0)
+		error("failed to reset isize for inode %llu root %llu due to %s",
+		      ino, root->root_key.objectid, strerror(-ret));
+	btrfs_release_path(path);
+	ret2 = btrfs_search_slot(NULL, root, &research_key, path, 0, 0);
+	return ret2 < 0 ? ret2 : ret;
+}
+
+/*
  * Check INODE_ITEM and related ITEMs (the same inode number)
  * 1. check link count
  * 2. check inode ref/extref
@@ -5015,9 +5071,16 @@ out:
 		}
 
 		if (isize != size) {
-			err |= ISIZE_ERROR;
-			error("root %llu DIR INODE [%llu] size(%llu) not equal to %llu",
-			      root->objectid, inode_id, isize, size);
+			if (repair)
+				ret = repair_dir_isize_lowmem(root, path,
+							      inode_id,
+							      size);
+
+			if (!repair || ret) {
+				err |= ISIZE_ERROR;
+				error("root %llu DIR INODE [%llu] size(%llu) not equal to %llu",
+				      root->objectid, inode_id, isize, size);
+			}
 		}
 	} else {
 		if (nlink != refs) {
-- 
2.11.1

[PATCH 03/20] btrfs-progs: cmds-check.c: inode orphan item repair

[Su Yue]

Add a function named 'repair_inode_orphan_item_lowmem'.

Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
---
 cmds-check.c | 50 +++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 49 insertions(+), 1 deletion(-)

diff --git a/cmds-check.c b/cmds-check.c
index f13ce317..fb239968 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -4936,6 +4936,46 @@ out:
 }
 
 /*
+ * repair ORPHAN_ITEM error
+ *
+ * Returns <0  means on error
+ * Returns  0  means successful repair
+ */
+static int repair_inode_orphan_item_lowmem(struct btrfs_root *root,
+					   struct btrfs_path *path, u64 ino)
+{
+	struct btrfs_trans_handle *trans;
+	struct btrfs_key research_key;
+	int ret;
+	int ret2;
+
+	btrfs_item_key_to_cpu(path->nodes[0], &research_key, path->slots[0]);
+	btrfs_release_path(path);
+	trans = btrfs_start_transaction(root, 1);
+	if (IS_ERR(trans)) {
+		ret = PTR_ERR(trans);
+		goto out;
+	}
+
+	ret = btrfs_add_orphan_item(trans, root, path, ino);
+	if (ret)
+		goto out;
+
+	printf("added inode %llu orphan item root %llu", ino,
+	       root->root_key.objectid);
+
+	btrfs_commit_transaction(trans, root);
+out:
+	if (ret < 0)
+		error("failed to add inode %llu orphan item root %llu due to %s",
+		       ino, root->root_key.objectid, strerror(-ret));
+	btrfs_release_path(path);
+	ret2 = btrfs_search_slot(NULL, root, &research_key, path, 0, 0);
+
+	return ret2 < 0 ? ret2 : ret;
+}
+
+/*
  * Check INODE_ITEM and related ITEMs (the same inode number)
  * 1. check link count
  * 2. check inode ref/extref
@@ -5088,7 +5128,15 @@ out:
 			error("root %llu INODE[%llu] nlink(%llu) not equal to inode_refs(%llu)",
 			      root->objectid, inode_id, nlink, refs);
 		} else if (!nlink) {
-			err |= ORPHAN_ITEM;
+			if (repair)
+				ret = repair_inode_orphan_item_lowmem(root,
+								      path,
+								      inode_id);
+			if (!repair || ret) {
+				err |= ORPHAN_ITEM;
+				error("root %llu INODE[%llu] is orphan item",
+				      root->objectid, inode_id);
+			}
 		}
 
 		if (!nbytes && !no_holes && extent_end < isize) {
-- 
2.11.1

[PATCH 04/20] btrfs-progs: cmds-check.c: change find_inode_ref's arg

[Su Yue]

For further lowmem repairs, change the index type u64 to u64 *.
So we could get the index of ref.

Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
---
 cmds-check.c | 24 +++++++++++++++---------
 1 file changed, 15 insertions(+), 9 deletions(-)

diff --git a/cmds-check.c b/cmds-check.c
index fb239968..246f4735 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -4422,22 +4422,23 @@ next:
 }
 
 /*
- * Find INODE_REF/INODE_EXTREF for the given key and check it with the specified
- * DIR_ITEM/DIR_INDEX match.
+ * Find INODE_REF/INODE_EXTREF for the given key and check it with the
+ * specified DIR_ITEM/DIR_INDEX match.Returned with right @index.
  *
  * @root:	the root of the fs/file tree
  * @key:	the key of the INODE_REF/INODE_EXTREF
  * @name:	the name in the INODE_REF/INODE_EXTREF
  * @namelen:	the length of name in the INODE_REF/INODE_EXTREF
- * @index:	the index in the INODE_REF/INODE_EXTREF, for DIR_ITEM set index
- * to (u64)-1
+ * @index_ret:	the index in the INODE_REF/INODE_EXTREF,
+ *              value (64)-1 means do not check index and return
+ *              with matched index.
  * @ext_ref:	the EXTENDED_IREF feature
  *
  * Return 0 if no error occurred.
  * Return >0 for error bitmap
  */
 static int find_inode_ref(struct btrfs_root *root, struct btrfs_key *key,
-			  char *name, int namelen, u64 index,
+			  const char *name, int namelen, u64 *index_ret,
 			  unsigned int ext_ref)
 {
 	struct btrfs_path path;
@@ -4474,7 +4475,8 @@ static int find_inode_ref(struct btrfs_root *root, struct btrfs_key *key,
 
 		ref_namelen = btrfs_inode_ref_name_len(node, ref);
 		ref_index = btrfs_inode_ref_index(node, ref);
-		if (index != (u64)-1 && index != ref_index)
+		if (index_ret && *index_ret != (u64)-1 &&
+		    *index_ret != ref_index)
 			goto next_ref;
 
 		if (ref_namelen <= BTRFS_NAME_LEN) {
@@ -4492,7 +4494,8 @@ static int find_inode_ref(struct btrfs_root *root, struct btrfs_key *key,
 
 		if (len != namelen || strncmp(ref_namebuf, name, len))
 			goto next_ref;
-
+		if (index_ret)
+			*index_ret = ref_index;
 		ret = 0;
 		goto out;
 next_ref:
@@ -4533,7 +4536,8 @@ extref:
 		ref_namelen = btrfs_inode_extref_name_len(node, extref);
 		ref_index = btrfs_inode_extref_index(node, extref);
 		parent = btrfs_inode_extref_parent(node, extref);
-		if (index != (u64)-1 && index != ref_index)
+		if (index_ret && *index_ret != (u64)-1 &&
+		    *index_ret != ref_index)
 			goto next_extref;
 
 		if (parent != dir_id)
@@ -4555,6 +4559,8 @@ extref:
 		if (len != namelen || strncmp(ref_namebuf, name, len))
 			goto next_extref;
 
+		if (index_ret)
+			*index_ret = ref_index;
 		ret = 0;
 		goto out;
 
@@ -4668,7 +4674,7 @@ static int check_dir_item(struct btrfs_root *root, struct btrfs_key *key,
 		location.type = BTRFS_INODE_REF_KEY;
 		location.offset = key->objectid;
 		ret = find_inode_ref(root, &location, namebuf, len,
-				       index, ext_ref);
+				       &index, ext_ref);
 		err |= ret;
 		if (ret & INODE_REF_MISSING)
 			error("root %llu %s[%llu %llu] relative INODE_REF missing namelen %u filename %s filetype %d",
-- 
2.11.1

[PATCH 05/20] btrfs-progs: cmds-check.c: modify check_fs_first_inode

[Su Yue]

Modify check_fs_first_inode to check firt_inode inode item and inode ref.

Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
---
 cmds-check.c | 57 ++++++++++++++++++++++++++++++++++++++++++---------------
 1 file changed, 42 insertions(+), 15 deletions(-)

diff --git a/cmds-check.c b/cmds-check.c
index 246f4735..892a22ba 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -5169,42 +5169,68 @@ out:
 	return err;
 }
 
+/*
+ * check first root dir's inode_item, inde_ref
+ *
+ * returns 0 means no error
+ * returns >0 means error
+ * returns <0 means fatal error
+ */
 static int check_fs_first_inode(struct btrfs_root *root, unsigned int ext_ref)
 {
 	struct btrfs_path path;
 	struct btrfs_key key;
+	struct btrfs_inode_item *ii;
+	u64 index = 0;
+	u32 mode;
 	int err = 0;
 	int ret;
 
-	key.objectid = BTRFS_FIRST_FREE_OBJECTID;
-	key.type = BTRFS_INODE_ITEM_KEY;
-	key.offset = 0;
-
 	/* For root being dropped, we don't need to check first inode */
 	if (btrfs_root_refs(&root->root_item) == 0 &&
 	    btrfs_disk_key_objectid(&root->root_item.drop_progress) >=
-	    key.objectid)
+	    BTRFS_FIRST_FREE_OBJECTID)
 		return 0;
 
+	/*search first inode item */
+	key.objectid = BTRFS_FIRST_FREE_OBJECTID;
+	key.type = BTRFS_INODE_ITEM_KEY;
+	key.offset = 0;
+
 	btrfs_init_path(&path);
 
 	ret = btrfs_search_slot(NULL, root, &key, &path, 0, 0);
-	if (ret < 0)
+	if (ret < 0) {
 		goto out;
-	if (ret > 0) {
-		ret = 0;
+	} else if (ret > 0) {
 		err |= INODE_ITEM_MISSING;
-		error("first inode item of root %llu is missing",
-		      root->objectid);
+	} else {
+		ii = btrfs_item_ptr(path.nodes[0], path.slots[0],
+				    struct btrfs_inode_item);
+		mode = btrfs_inode_mode(path.nodes[0], ii);
+		if (imode_to_type(mode) != BTRFS_FT_DIR)
+			err |= INODE_ITEM_MISMATCH;
 	}
+	btrfs_release_path(&path);
+
+	/* lookup first inode ref */
+	key.offset = BTRFS_FIRST_FREE_OBJECTID;
+	key.type = BTRFS_INODE_REF_KEY;
+
+	ret = find_inode_ref(root, &key, "..", strlen(".."), &index, ext_ref);
+	if (ret < 0)
+		goto out;
+	err |= ret;
 
-	err |= check_inode_item(root, &path, ext_ref);
-	err &= ~LAST_ITEM;
-	if (err && !ret)
-		ret = -EIO;
 out:
 	btrfs_release_path(&path);
-	return ret;
+	if (err & (INODE_ITEM_MISSING | INODE_ITEM_MISMATCH))
+		error("root dir INODE_ITEM is %s",
+		      err & INODE_ITEM_MISMATCH ? "mismatch" : "missing");
+	if (err & INODE_REF_MISSING)
+		error("root dir INODE_REF is missing");
+
+	return ret < 0 ? ret : err;
 }
 
 /*
@@ -5232,6 +5258,7 @@ static int check_fs_root_v2(struct btrfs_root *root, unsigned int ext_ref)
 	 * we will just skip it forever.
 	 */
 	ret = check_fs_first_inode(root, ext_ref);
+	err |= !!ret;
 	if (ret < 0)
 		return ret;
 
-- 
2.11.1

[PATCH 06/20] btrfs-progs: cmds-check.c: change find_dir_index/item

[Su Yue]

For further lowmem_repair, introduce 'find_dir_index' to
get the index by other inode item information.
Remove 'check_dir_item' error msg print.

Adjust 'find_dir_item' args and remove err msg print.

Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
---
 cmds-check.c | 231 +++++++++++++++++++++++++++++++++++++++--------------------
 1 file changed, 155 insertions(+), 76 deletions(-)

diff --git a/cmds-check.c b/cmds-check.c
index 892a22ba..c45dfae4 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -130,6 +130,8 @@ struct data_backref {
 #define LAST_ITEM		(1<<15)	/* Complete this tree traversal */
 #define ROOT_REF_MISSING	(1<<16)	/* ROOT_REF not found */
 #define ROOT_REF_MISMATCH	(1<<17)	/* ROOT_REF found but not match */
+#define DIR_INDEX_MISSING       (1<<18) /* INODE_INDEX not found */
+#define DIR_INDEX_MISMATCH      (1<<19) /* INODE_INDEX found but not match */
 
 static inline struct data_backref* to_data_backref(struct extent_backref *back)
 {
@@ -4133,29 +4135,30 @@ out:
 	return err;
 }
 
+static int find_dir_index(struct btrfs_root *root, u64 dirid, u64 location_id,
+			  u64 *index_ret, char *namebuf, u32 name_len,
+			  u8 file_type);
 /*
  * Find DIR_ITEM/DIR_INDEX for the given key and check it with the specified
  * INODE_REF/INODE_EXTREF match.
  *
  * @root:	the root of the fs/file tree
- * @ref_key:	the key of the INODE_REF/INODE_EXTREF
- * @key:	the key of the DIR_ITEM/DIR_INDEX
- * @index:	the index in the INODE_REF/INODE_EXTREF, be used to
- *		distinguish root_dir between normal dir/file
- * @name:	the name in the INODE_REF/INODE_EXTREF
- * @namelen:	the length of name in the INODE_REF/INODE_EXTREF
- * @mode:	the st_mode of INODE_ITEM
+ * @key:	the key of the DIR_ITEM/DIR_INDEX, key->offset will be right
+ *              value while find index
+ * @location_key: location key of the struct btrfs_dir_item to match
+ * @name:	the name to match
+ * @namelen:	the length of name
+ * @file_type:	the type of file to math
  *
  * Return 0 if no error occurred.
- * Return ROOT_DIR_ERROR if found DIR_ITEM/DIR_INDEX for root_dir.
- * Return DIR_ITEM_MISSING if couldn't find DIR_ITEM/DIR_INDEX for normal
- * dir/file.
- * Return DIR_ITEM_MISMATCH if INODE_REF/INODE_EXTREF and DIR_ITEM/DIR_INDEX
- * not match for normal dir/file.
+ * Return DIR_ITEM_MISSING/DIR_INDEX_MISSING if couldn't find
+ * DIR_ITEM/DIR_INDEX
+ * Return DIR_ITEM_MISMATCH/DIR_INDEX_MISMATCH if INODE_REF/INODE_EXTREF
+ * and DIR_ITEM/DIR_INDEX mismatch
  */
-static int find_dir_item(struct btrfs_root *root, struct btrfs_key *ref_key,
-			 struct btrfs_key *key, u64 index, char *name,
-			 u32 namelen, u32 mode)
+static int find_dir_item(struct btrfs_root *root, struct btrfs_key *key,
+			 struct btrfs_key *location_key, char *name,
+			 u32 namelen, u8 file_type)
 {
 	struct btrfs_path path;
 	struct extent_buffer *node;
@@ -4165,104 +4168,166 @@ static int find_dir_item(struct btrfs_root *root, struct btrfs_key *ref_key,
 	u32 total;
 	u32 cur = 0;
 	u32 len;
-	u32 name_len;
 	u32 data_len;
 	u8 filetype;
 	int slot;
 	int ret;
 
+	/* get the index by traversing all index */
+	if (key->type == BTRFS_DIR_INDEX_KEY && key->offset == (u64)-1) {
+		ret = find_dir_index(root, key->objectid,
+				     location_key->objectid, &key->offset,
+				     name, namelen, file_type);
+		if (ret)
+			ret = DIR_INDEX_MISSING;
+		return ret;
+	}
+
 	btrfs_init_path(&path);
 	ret = btrfs_search_slot(NULL, root, key, &path, 0, 0);
-	if (ret < 0) {
-		ret = DIR_ITEM_MISSING;
+	if (ret) {
+		ret = key->type == BTRFS_DIR_ITEM_KEY ? DIR_ITEM_MISSING :
+			DIR_INDEX_MISSING;
 		goto out;
 	}
 
-	/* Process root dir and goto out*/
-	if (index == 0) {
-		if (ret == 0) {
-			ret = ROOT_DIR_ERROR;
-			error(
-			"root %llu INODE %s[%llu %llu] ROOT_DIR shouldn't have %s",
-				root->objectid,
-				ref_key->type == BTRFS_INODE_REF_KEY ?
-					"REF" : "EXTREF",
-				ref_key->objectid, ref_key->offset,
-				key->type == BTRFS_DIR_ITEM_KEY ?
-					"DIR_ITEM" : "DIR_INDEX");
-		} else {
-			ret = 0;
-		}
+	/* Check whether inode_id/filetype/name match */
+	node = path.nodes[0];
+	slot = path.slots[0];
+	di = btrfs_item_ptr(node, slot, struct btrfs_dir_item);
+	total = btrfs_item_size_nr(node, slot);
+	while (cur < total) {
+		ret = key->type == BTRFS_DIR_ITEM_KEY ?
+			DIR_ITEM_MISMATCH : DIR_INDEX_MISMATCH;
 
-		goto out;
-	}
+		len = btrfs_dir_name_len(node, di);
+		data_len = btrfs_dir_data_len(node, di);
 
-	/* Process normal file/dir */
-	if (ret > 0) {
-		ret = DIR_ITEM_MISSING;
-		error(
-		"root %llu INODE %s[%llu %llu] doesn't have related %s[%llu %llu] namelen %u filename %s filetype %d",
+		btrfs_dir_item_key_to_cpu(node, di, &location);
+		if (location.objectid != location_key->objectid ||
+		    location.type != location_key->type ||
+		    location.offset != location_key->offset)
+			goto next;
+
+		filetype = btrfs_dir_type(node, di);
+		if (file_type != filetype)
+			goto next;
+
+		if (len > BTRFS_NAME_LEN) {
+			len = BTRFS_NAME_LEN;
+			warning("root %llu %s[%llu %llu] name too long %u, trimmed",
 			root->objectid,
-			ref_key->type == BTRFS_INODE_REF_KEY ? "REF" : "EXTREF",
-			ref_key->objectid, ref_key->offset,
 			key->type == BTRFS_DIR_ITEM_KEY ?
-				"DIR_ITEM" : "DIR_INDEX",
-			key->objectid, key->offset, namelen, name,
-			imode_to_type(mode));
+			"DIR_ITEM" : "DIR_INDEX",
+			key->objectid, key->offset, len);
+		}
+		read_extent_buffer(node, namebuf, (unsigned long)(di + 1),
+				   len);
+		if (len != namelen || strncmp(namebuf, name, len))
+			goto next;
+
+		ret = 0;
 		goto out;
+next:
+		len += sizeof(*di) + data_len;
+		di = (struct btrfs_dir_item *)((char *)di + len);
+		cur += len;
 	}
 
+out:
+	btrfs_release_path(&path);
+	return ret;
+}
+
+/*
+ * Find the @index according @ino.
+ * Notice:the time efficiency is O(n)
+ *
+ * @root:	the root of the fs/file tree
+ * @index_ret:	the index as return value
+ * @namebuf:	the name to match
+ * @name_len:	the length of name to match
+ * @file_type:	the file_type of INODE_ITEM to match
+ *
+ * Returns 0 if found and *index will be modified with right value
+ * Returns< 0 not found and *index will be (u64)-1
+ */
+static int find_dir_index(struct btrfs_root *root, u64 dirid, u64 location_id,
+			  u64 *index_ret, char *namebuf, u32 name_len,
+			  u8 file_type)
+{
+	struct btrfs_path path;
+	struct extent_buffer *node;
+	struct btrfs_dir_item *di;
+	struct btrfs_key key;
+	struct btrfs_key location;
+	char name[BTRFS_NAME_LEN] = {0};
+
+	u32 total;
+	u32 cur = 0;
+	u32 len;
+	u32 data_len;
+	u8 filetype;
+	int slot;
+	int ret;
+
+	ASSERT(index_ret);
+
+	/* search from the last index */
+	key.objectid = dirid;
+	key.offset = (u64)-1;
+	key.type = BTRFS_DIR_INDEX_KEY;
+	btrfs_init_path(&path);
+
+	ret = btrfs_search_slot(NULL, root, &key, &path, 0, 0);
+	if (ret < 0 || !ret)
+		return -EIO;
+
+loop:
+	ret = btrfs_previous_item(root, &path, dirid, BTRFS_DIR_INDEX_KEY);
+	if (ret) {
+		ret = -ENOENT;
+		*index_ret = (64)-1;
+		goto out;
+	}
 	/* Check whether inode_id/filetype/name match */
 	node = path.nodes[0];
 	slot = path.slots[0];
 	di = btrfs_item_ptr(node, slot, struct btrfs_dir_item);
 	total = btrfs_item_size_nr(node, slot);
 	while (cur < total) {
-		ret = DIR_ITEM_MISMATCH;
-		name_len = btrfs_dir_name_len(node, di);
+		ret = -ENOENT;
+		len = btrfs_dir_name_len(node, di);
 		data_len = btrfs_dir_data_len(node, di);
 
 		btrfs_dir_item_key_to_cpu(node, di, &location);
-		if (location.objectid != ref_key->objectid ||
-		    location.type !=  BTRFS_INODE_ITEM_KEY ||
+		if (location.objectid != location_id ||
+		    location.type != BTRFS_INODE_ITEM_KEY ||
 		    location.offset != 0)
 			goto next;
 
 		filetype = btrfs_dir_type(node, di);
-		if (imode_to_type(mode) != filetype)
+		if (file_type != filetype)
 			goto next;
 
-		if (name_len <= BTRFS_NAME_LEN) {
-			len = name_len;
-		} else {
+		if (len > BTRFS_NAME_LEN)
 			len = BTRFS_NAME_LEN;
-			warning("root %llu %s[%llu %llu] name too long %u, trimmed",
-			root->objectid,
-			key->type == BTRFS_DIR_ITEM_KEY ?
-			"DIR_ITEM" : "DIR_INDEX",
-			key->objectid, key->offset, name_len);
-		}
-		read_extent_buffer(node, namebuf, (unsigned long)(di + 1), len);
-		if (len != namelen || strncmp(namebuf, name, len))
+
+		read_extent_buffer(node, name, (unsigned long)(di + 1), len);
+		if (len != name_len || strncmp(namebuf, name, len))
 			goto next;
 
+		btrfs_item_key_to_cpu(path.nodes[0], &key, path.slots[0]);
+		*index_ret = key.offset;
 		ret = 0;
 		goto out;
 next:
-		len = sizeof(*di) + name_len + data_len;
+		len += sizeof(*di) + data_len;
 		di = (struct btrfs_dir_item *)((char *)di + len);
 		cur += len;
 	}
-	if (ret == DIR_ITEM_MISMATCH)
-		error(
-		"root %llu INODE %s[%llu %llu] and %s[%llu %llu] mismatch namelen %u filename %s filetype %d",
-			root->objectid,
-			ref_key->type == BTRFS_INODE_REF_KEY ? "REF" : "EXTREF",
-			ref_key->objectid, ref_key->offset,
-			key->type == BTRFS_DIR_ITEM_KEY ?
-				"DIR_ITEM" : "DIR_INDEX",
-			key->objectid, key->offset, namelen, name,
-			imode_to_type(mode));
+	goto loop;
+
 out:
 	btrfs_release_path(&path);
 	return ret;
@@ -4284,6 +4349,7 @@ static int check_inode_ref(struct btrfs_root *root, struct btrfs_key *ref_key,
 			   int mode)
 {
 	struct btrfs_key key;
+	struct btrfs_key location;
 	struct btrfs_inode_ref *ref;
 	char namebuf[BTRFS_NAME_LEN] = {0};
 	u32 total;
@@ -4293,6 +4359,10 @@ static int check_inode_ref(struct btrfs_root *root, struct btrfs_key *ref_key,
 	u64 index;
 	int ret, err = 0;
 
+	location.objectid = ref_key->objectid;
+	location.type = BTRFS_INODE_ITEM_KEY;
+	location.offset = 0;
+
 	ref = btrfs_item_ptr(node, slot, struct btrfs_inode_ref);
 	total = btrfs_item_size_nr(node, slot);
 
@@ -4324,14 +4394,16 @@ next:
 	key.objectid = ref_key->offset;
 	key.type = BTRFS_DIR_INDEX_KEY;
 	key.offset = index;
-	ret = find_dir_item(root, ref_key, &key, index, namebuf, len, mode);
+	ret = find_dir_item(root, &key, &location, namebuf, len,
+			    imode_to_type(mode));
 	err |= ret;
 
 	/* Find related dir_item */
 	key.objectid = ref_key->offset;
 	key.type = BTRFS_DIR_ITEM_KEY;
 	key.offset = btrfs_name_hash(namebuf, len);
-	ret = find_dir_item(root, ref_key, &key, index, namebuf, len, mode);
+	ret = find_dir_item(root, &key, &location, namebuf, len,
+			    imode_to_type(mode));
 	err |= ret;
 
 	len = sizeof(*ref) + name_len;
@@ -4360,6 +4432,7 @@ static int check_inode_extref(struct btrfs_root *root,
 			      int mode)
 {
 	struct btrfs_key key;
+	struct btrfs_key location;
 	struct btrfs_inode_extref *extref;
 	char namebuf[BTRFS_NAME_LEN] = {0};
 	u32 total;
@@ -4371,6 +4444,10 @@ static int check_inode_extref(struct btrfs_root *root,
 	int ret;
 	int err = 0;
 
+	location.objectid = ref_key->objectid;
+	location.type = BTRFS_INODE_ITEM_KEY;
+	location.offset = 0;
+
 	extref = btrfs_item_ptr(node, slot, struct btrfs_inode_extref);
 	total = btrfs_item_size_nr(node, slot);
 
@@ -4401,14 +4478,16 @@ next:
 	key.objectid = parent;
 	key.type = BTRFS_DIR_INDEX_KEY;
 	key.offset = index;
-	ret = find_dir_item(root, ref_key, &key, index, namebuf, len, mode);
+	ret = find_dir_item(root, &key, &location, namebuf, len,
+			    imode_to_type(mode));
 	err |= ret;
 
 	/* find related dir_item */
 	key.objectid = parent;
 	key.type = BTRFS_DIR_ITEM_KEY;
 	key.offset = btrfs_name_hash(namebuf, len);
-	ret = find_dir_item(root, ref_key, &key, index, namebuf, len, mode);
+	ret = find_dir_item(root, &key, &location, namebuf, len,
+			    imode_to_type(mode));
 	err |= ret;
 
 	len = sizeof(*extref) + name_len;
-- 
2.11.1

[PATCH 07/20] btrfs-progs: cmds-check.c: introduce print_inode_ref

[Su Yue]

Introduce 'print_inode_ref' to print error msg while checking inode ref.

Add args 'name_ret' and 'namelen_ret' to 'check_inode_ref' because
they are essential while doing nlinks repair.

Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
---
 cmds-check.c | 93 +++++++++++++++++++++++++++++++++++++++++++++++-------------
 1 file changed, 74 insertions(+), 19 deletions(-)

diff --git a/cmds-check.c b/cmds-check.c
index c45dfae4..24a39e54 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -4334,34 +4334,76 @@ out:
 }
 
 /*
+ * Print inode ref error message
+ */
+static void print_inode_ref_err(struct btrfs_root *root, struct btrfs_key *key,
+				u64 index, const char *namebuf, int name_len,
+				u8 filetype, int err)
+{
+	if (!err)
+		return;
+
+	/*root dir error */
+	if (key->objectid == BTRFS_FIRST_FREE_OBJECTID) {
+		error("root %llu root dir shouldn't have INODE REF[%llu %llu] name %s",
+		      root->objectid, key->objectid, key->offset, namebuf);
+		return;
+	}
+
+	/* normal error */
+	if (err & (DIR_ITEM_MISMATCH | DIR_ITEM_MISSING))
+		error("root %llu DIR ITEM[%llu %llu] %s name %s filetype %u",
+		       root->objectid, key->offset,
+			   btrfs_name_hash(namebuf, name_len),
+		       err & DIR_ITEM_MISMATCH ? "mismath" : "missing",
+		       namebuf, filetype);
+	if (err & (DIR_INDEX_MISMATCH | DIR_INDEX_MISSING))
+		error("root %llu DIR INDEX[%llu %llu] %s name %s filetype %u",
+		       root->objectid, key->offset,
+		       index,
+		       err & DIR_ITEM_MISMATCH ? "mismath" : "missing",
+		       namebuf, filetype);
+}
+
+/*
  * Traverse the given INODE_REF and call find_dir_item() to find related
- * DIR_ITEM/DIR_INDEX.
+ * DIR_ITEM/DIR_INDEX.If repair is enable, research @ref_key and
+ * @path may change.
  *
  * @root:	the root of the fs/file tree
  * @ref_key:	the key of the INODE_REF
+ * @path        the path provides node and slot
  * @refs:	the count of INODE_REF
  * @mode:	the st_mode of INODE_ITEM
+ * @name_ret:   returns with the first ref's name
+ * @name_len_ret:    len of the name_ret
  *
+ * Return <0 on error.
  * Return 0 if no error occurred.
  */
 static int check_inode_ref(struct btrfs_root *root, struct btrfs_key *ref_key,
-			   struct extent_buffer *node, int slot, u64 *refs,
-			   int mode)
+			   struct btrfs_path *path, char *name_ret,
+			   u32 *namelen_ret, u64 *refs, int mode)
 {
 	struct btrfs_key key;
 	struct btrfs_key location;
 	struct btrfs_inode_ref *ref;
+	struct extent_buffer *node;
 	char namebuf[BTRFS_NAME_LEN] = {0};
+	int name_len;
 	u32 total;
 	u32 cur = 0;
-	u32 len;
-	u32 name_len;
+	long len;
 	u64 index;
-	int ret, err = 0;
+	int err = 0;
+	int tmp_err;
+	int slot;
 
 	location.objectid = ref_key->objectid;
 	location.type = BTRFS_INODE_ITEM_KEY;
 	location.offset = 0;
+	node = path->nodes[0];
+	slot = path->slots[0];
 
 	ref = btrfs_item_ptr(node, slot, struct btrfs_inode_ref);
 	total = btrfs_item_size_nr(node, slot);
@@ -4370,6 +4412,7 @@ next:
 	/* Update inode ref count */
 	(*refs)++;
 
+	tmp_err = 0;
 	index = btrfs_inode_ref_index(node, ref);
 	name_len = btrfs_inode_ref_name_len(node, ref);
 	if (name_len <= BTRFS_NAME_LEN) {
@@ -4382,30 +4425,40 @@ next:
 
 	read_extent_buffer(node, namebuf, (unsigned long)(ref + 1), len);
 
-	/* Check root dir ref name */
-	if (index == 0 && strncmp(namebuf, "..", name_len)) {
-		error("root %llu INODE_REF[%llu %llu] ROOT_DIR name shouldn't be %s",
-		      root->objectid, ref_key->objectid, ref_key->offset,
-		      namebuf);
-		err |= ROOT_DIR_ERROR;
+	/* copy the firt name found to name_ret */
+	if (*refs == 1 && name_ret) {
+		memcpy(name_ret, namebuf, len);
+		*namelen_ret = len;
+	}
+	/* Check root dir ref */
+	if (ref_key->objectid == BTRFS_FIRST_FREE_OBJECTID) {
+		if (index != 0 || len != strlen("..") ||
+		    strncmp("..", namebuf, len) ||
+		    ref_key->offset != BTRFS_FIRST_FREE_OBJECTID) {
+			/* set fake err bit so repair will delete the ref */
+			err |= DIR_INDEX_MISSING;
+			err |= DIR_ITEM_MISSING;
+		}
+		goto end;
 	}
 
 	/* Find related DIR_INDEX */
 	key.objectid = ref_key->offset;
 	key.type = BTRFS_DIR_INDEX_KEY;
 	key.offset = index;
-	ret = find_dir_item(root, &key, &location, namebuf, len,
+	tmp_err |= find_dir_item(root, &key, &location, namebuf, len,
 			    imode_to_type(mode));
-	err |= ret;
 
 	/* Find related dir_item */
 	key.objectid = ref_key->offset;
 	key.type = BTRFS_DIR_ITEM_KEY;
 	key.offset = btrfs_name_hash(namebuf, len);
-	ret = find_dir_item(root, &key, &location, namebuf, len,
+	tmp_err |= find_dir_item(root, &key, &location, namebuf, len,
 			    imode_to_type(mode));
-	err |= ret;
-
+end:
+	print_inode_ref_err(root, ref_key, index, namebuf, name_len,
+			    imode_to_type(mode), tmp_err);
+	err |= tmp_err;
 	len = sizeof(*ref) + name_len;
 	ref = (struct btrfs_inode_ref *)((char *)ref + len);
 	cur += len;
@@ -5093,6 +5146,8 @@ static int check_inode_item(struct btrfs_root *root, struct btrfs_path *path,
 	int slot;
 	int ret;
 	int err = 0;
+	char namebuf[BTRFS_NAME_LEN] = {0};
+	u32 name_len = 0;
 
 	node = path->nodes[0];
 	slot = path->slots[0];
@@ -5133,8 +5188,8 @@ static int check_inode_item(struct btrfs_root *root, struct btrfs_path *path,
 
 		switch (key.type) {
 		case BTRFS_INODE_REF_KEY:
-			ret = check_inode_ref(root, &key, node, slot, &refs,
-					      mode);
+			ret = check_inode_ref(root, &key, path, namebuf,
+					      &name_len, &refs, mode);
 			err |= ret;
 			break;
 		case BTRFS_INODE_EXTREF_KEY:
-- 
2.11.1

[PATCH 08/20] btrfs-progs: cmds-check.c: print_dir_item_err

[Su Yue]

Introduce 'print_dir_item_err' to print error msg while
checking dir_item/dir_index.
'check_dir_item' now checks relative dir item and calls
'print_dir_itm_err" to print error msg.

Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
---
 cmds-check.c | 127 ++++++++++++++++++++++++++++++++++++++++-------------------
 1 file changed, 87 insertions(+), 40 deletions(-)

diff --git a/cmds-check.c b/cmds-check.c
index 24a39e54..44abb282 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -4707,25 +4707,61 @@ out:
 	return ret;
 }
 
+static void print_dir_item_err(struct btrfs_root *root, struct btrfs_key *key,
+			       u64 ino, u64 index, const char *namebuf,
+			       int name_len, u8 filetype, int err)
+{
+	if (err & (DIR_ITEM_MISMATCH | DIR_ITEM_MISSING)) {
+		error("root %llu DIR ITEM[%llu %llu] name %s filetype %d %s",
+		      root->objectid, key->objectid, key->offset, namebuf,
+		      filetype,
+		      err & DIR_ITEM_MISMATCH ? "mismath" : "missing");
+	}
+
+	if (err & (DIR_INDEX_MISMATCH | DIR_INDEX_MISSING)) {
+		error("root %llu DIR INDEX[%llu %llu] name %s filetype %d %s",
+		      root->objectid, key->objectid, index, namebuf,
+		      filetype,
+		      err & DIR_ITEM_MISMATCH ? "mismath" : "missing");
+	}
+
+	if (err & (INODE_ITEM_MISSING | INODE_ITEM_MISMATCH)) {
+		error("root %llu INODE_ITEM[%llu] index %llu name %s filetype %d %s",
+		      root->objectid, ino, index, namebuf,
+		      filetype,
+		      err & INODE_ITEM_MISMATCH ? "mismath" : "missing");
+	}
+
+	if (err & INODE_REF_MISSING)
+		error("root %llu INODE REF[%llu, %llu] name %s filetype %u missing",
+		      root->objectid, ino, key->objectid, namebuf, filetype);
+
+}
+
 /*
  * Traverse the given DIR_ITEM/DIR_INDEX and check related INODE_ITEM and
- * call find_inode_ref() to check related INODE_REF/INODE_EXTREF.
+ * call find_inode_ref() to check related INODE_REF/INODE_EXTREF.If repair
+ * is enable, do repair and research by @path->nodes[0].
  *
  * @root:	the root of the fs/file tree
  * @key:	the key of the INODE_REF/INODE_EXTREF
+ * @path:       the path of key
  * @size:	the st_size of the INODE_ITEM
  * @ext_ref:	the EXTENDED_IREF feature
  *
  * Return 0 if no error occurred.
+ * Return >0 if on error
  */
 static int check_dir_item(struct btrfs_root *root, struct btrfs_key *key,
-			  struct extent_buffer *node, int slot, u64 *size,
+			  struct btrfs_path *path, u64 *size,
 			  unsigned int ext_ref)
 {
 	struct btrfs_dir_item *di;
 	struct btrfs_inode_item *ii;
-	struct btrfs_path path;
+	struct btrfs_key key2;
 	struct btrfs_key location;
+	struct extent_buffer *node;
+	int slot;
 	char namebuf[BTRFS_NAME_LEN] = {0};
 	u32 total;
 	u32 cur = 0;
@@ -4737,6 +4773,7 @@ static int check_dir_item(struct btrfs_root *root, struct btrfs_key *key,
 	u64 index;
 	int ret;
 	int err = 0;
+	int tmp_err;
 
 	/*
 	 * For DIR_ITEM set index to (u64)-1, so that find_inode_ref
@@ -4744,11 +4781,14 @@ static int check_dir_item(struct btrfs_root *root, struct btrfs_key *key,
 	 */
 	index = (key->type == BTRFS_DIR_INDEX_KEY) ? key->offset : (u64)-1;
 
+	node = path->nodes[0];
+	slot = path->slots[0];
 	di = btrfs_item_ptr(node, slot, struct btrfs_dir_item);
 	total = btrfs_item_size_nr(node, slot);
 
 	while (cur < total) {
 		data_len = btrfs_dir_data_len(node, di);
+		tmp_err = 0;
 		if (data_len)
 			error("root %llu %s[%llu %llu] data_len shouldn't be %u",
 			      root->objectid, key->type == BTRFS_DIR_ITEM_KEY ?
@@ -4768,58 +4808,61 @@ static int check_dir_item(struct btrfs_root *root, struct btrfs_key *key,
 		}
 		(*size) += name_len;
 
-		read_extent_buffer(node, namebuf, (unsigned long)(di + 1), len);
+		read_extent_buffer(node, namebuf, (unsigned long)(di + 1),
+				   len);
 		filetype = btrfs_dir_type(node, di);
 
-		btrfs_init_path(&path);
 		btrfs_dir_item_key_to_cpu(node, di, &location);
-
 		/* Ignore related ROOT_ITEM check */
 		if (location.type == BTRFS_ROOT_ITEM_KEY)
 			goto next;
 
+		btrfs_release_path(path);
 		/* Check relative INODE_ITEM(existence/filetype) */
-		ret = btrfs_search_slot(NULL, root, &location, &path, 0, 0);
+		ret = btrfs_search_slot(NULL, root, &location, path, 0, 0);
 		if (ret) {
-			err |= INODE_ITEM_MISSING;
-			error("root %llu %s[%llu %llu] couldn't find relative INODE_ITEM[%llu] namelen %u filename %s filetype %x",
-			      root->objectid, key->type == BTRFS_DIR_ITEM_KEY ?
-			      "DIR_ITEM" : "DIR_INDEX", key->objectid,
-			      key->offset, location.objectid, name_len,
-			      namebuf, filetype);
-			goto next;
-		}
-
-		ii = btrfs_item_ptr(path.nodes[0], path.slots[0],
-				    struct btrfs_inode_item);
-		mode = btrfs_inode_mode(path.nodes[0], ii);
-
-		if (imode_to_type(mode) != filetype) {
-			err |= INODE_ITEM_MISMATCH;
-			error("root %llu %s[%llu %llu] relative INODE_ITEM filetype mismatch namelen %u filename %s filetype %d",
-			      root->objectid, key->type == BTRFS_DIR_ITEM_KEY ?
-			      "DIR_ITEM" : "DIR_INDEX", key->objectid,
-			      key->offset, name_len, namebuf, filetype);
+			tmp_err |= INODE_ITEM_MISSING;
+		} else {
+			ii = btrfs_item_ptr(path->nodes[0], path->slots[0],
+					    struct btrfs_inode_item);
+			mode = btrfs_inode_mode(path->nodes[0], ii);
+			if (imode_to_type(mode) != filetype)
+				tmp_err |= INODE_ITEM_MISMATCH;
 		}
 
 		/* Check relative INODE_REF/INODE_EXTREF */
-		location.type = BTRFS_INODE_REF_KEY;
-		location.offset = key->objectid;
-		ret = find_inode_ref(root, &location, namebuf, len,
-				       &index, ext_ref);
-		err |= ret;
-		if (ret & INODE_REF_MISSING)
-			error("root %llu %s[%llu %llu] relative INODE_REF missing namelen %u filename %s filetype %d",
-			      root->objectid, key->type == BTRFS_DIR_ITEM_KEY ?
-			      "DIR_ITEM" : "DIR_INDEX", key->objectid,
-			      key->offset, name_len, namebuf, filetype);
+		key2.objectid = location.objectid;
+		key2.type = BTRFS_INODE_REF_KEY;
+		key2.offset = key->objectid;
+		tmp_err |= find_inode_ref(root, &key2, namebuf, len,
+					  &index, ext_ref);
+
+		/* check relative INDEX/ITEM */
+		key2.objectid = key->objectid;
+		if (key->type == BTRFS_DIR_ITEM_KEY) {
+			key2.type = BTRFS_DIR_INDEX_KEY;
+			key2.offset = index;
+		} else {
+			key2.type = BTRFS_DIR_ITEM_KEY;
+			key2.offset = btrfs_name_hash(namebuf, name_len);
+		}
 
+		tmp_err |= find_dir_item(root, &key2, &location, namebuf,
+					 name_len, filetype);
+		/* find_dir_item may find index */
+		if (key2.type == BTRFS_DIR_INDEX_KEY)
+			index = key2.offset;
 next:
-		btrfs_release_path(&path);
+		btrfs_release_path(path);
+
+		print_dir_item_err(root, key, location.objectid,
+				   index, namebuf, name_len, filetype,
+				   tmp_err);
+
+		err |= tmp_err;
 		len = sizeof(*di) + name_len + data_len;
 		di = (struct btrfs_dir_item *)((char *)di + len);
 		cur += len;
-
 		if (key->type == BTRFS_DIR_INDEX_KEY && cur < total) {
 			error("root %llu DIR_INDEX[%llu %llu] should contain only one entry",
 			      root->objectid, key->objectid, key->offset);
@@ -4827,9 +4870,13 @@ next:
 		}
 	}
 
+	/* research path */
+	btrfs_release_path(path);
+	ret = btrfs_search_slot(NULL, root, key, path, 0, 0);
+	if (ret)
+		err |= ret > 0 ? ENOENT : ret;
 	return err;
 }
-
 /*
  * Check file extent datasum/hole, update the size of the file extents,
  * check and update the last offset of the file extent.
@@ -5209,7 +5256,7 @@ static int check_inode_item(struct btrfs_root *root, struct btrfs_path *path,
 					imode_to_type(mode), key.objectid,
 					key.offset);
 			}
-			ret = check_dir_item(root, &key, node, slot, &size,
+			ret = check_dir_item(root, &key, path, &size,
 					     ext_ref);
 			err |= ret;
 			break;
-- 
2.11.1

[PATCH 09/20] btrfs-progs: cmds-check.c: introduce count_dir_isize

[Su Yue]

Introduce 'count_dir_isize' to get dir isize.
This function is called only under lowmme repair mode.

Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
---
 cmds-check.c | 95 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 92 insertions(+), 3 deletions(-)

diff --git a/cmds-check.c b/cmds-check.c
index 44abb282..685f4f5d 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -4738,6 +4738,93 @@ static void print_dir_item_err(struct btrfs_root *root, struct btrfs_key *key,
 
 }
 
+static int __count_dir_isize(struct btrfs_root *root, u64 ino,
+			 int type, u64 *size_ret)
+{
+	struct btrfs_key key;
+	struct btrfs_path path;
+	u32 len;
+	struct btrfs_dir_item *di;
+	int ret;
+	int cur = 0;
+	int total = 0;
+
+	ASSERT(size_ret);
+	*size_ret = 0;
+
+	key.objectid = ino;
+	key.type = type;
+	key.offset = (u64)-1;
+
+	btrfs_init_path(&path);
+	ret = btrfs_search_slot(NULL, root, &key, &path, 0, 0);
+	if (ret < 0) {
+		ret = -EIO;
+		goto out;
+	}
+	/* if found, go to spacial case */
+	if (ret == 0)
+		goto special_case;
+
+loop:
+	ret = btrfs_previous_item(root, &path, ino, type);
+
+	if (ret) {
+		ret = 0;
+		goto out;
+	}
+
+special_case:
+
+	di = btrfs_item_ptr(path.nodes[0], path.slots[0],
+			    struct btrfs_dir_item);
+	cur = 0;
+	total = btrfs_item_size_nr(path.nodes[0], path.slots[0]);
+
+	while (cur < total) {
+		len = btrfs_dir_name_len(path.nodes[0], di);
+		if (len > BTRFS_NAME_LEN)
+			len = BTRFS_NAME_LEN;
+		*size_ret += len;
+
+		len += btrfs_dir_data_len(path.nodes[0], di);
+		len += sizeof(*di);
+		di = (struct btrfs_dir_item *)((char *)di + len);
+		cur += len;
+	}
+	goto loop;
+
+out:
+	btrfs_release_path(&path);
+	return ret;
+}
+
+static int count_dir_isize(struct btrfs_root *root, u64 ino, u64 *size)
+{
+	ASSERT(size);
+	u64 item_size;
+	u64 index_size;
+	int ret;
+
+	ret = __count_dir_isize(root, ino, BTRFS_DIR_ITEM_KEY,
+				&item_size);
+	if (ret)
+		goto out;
+
+	ret = __count_dir_isize(root, ino, BTRFS_DIR_INDEX_KEY,
+			       &index_size);
+	if (ret)
+		goto out;
+
+	*size = item_size + index_size;
+
+out:
+	if (ret)
+		error("Failed to count root %llu INODE[%llu] root size",
+		      root->objectid, ino);
+	return ret;
+}
+
 /*
  * Traverse the given DIR_ITEM/DIR_INDEX and check related INODE_ITEM and
  * call find_inode_ref() to check related INODE_REF/INODE_EXTREF.If repair
@@ -4807,7 +4894,6 @@ static int check_dir_item(struct btrfs_root *root, struct btrfs_key *key,
 				key->objectid, key->offset);
 		}
 		(*size) += name_len;
-
 		read_extent_buffer(node, namebuf, (unsigned long)(di + 1),
 				   len);
 		filetype = btrfs_dir_type(node, di);
@@ -5256,8 +5342,7 @@ static int check_inode_item(struct btrfs_root *root, struct btrfs_path *path,
 					imode_to_type(mode), key.objectid,
 					key.offset);
 			}
-			ret = check_dir_item(root, &key, path, &size,
-					     ext_ref);
+			ret = check_dir_item(root, &key, path, &size, ext_ref);
 			err |= ret;
 			break;
 		case BTRFS_EXTENT_DATA_KEY:
@@ -5280,6 +5365,10 @@ static int check_inode_item(struct btrfs_root *root, struct btrfs_path *path,
 	}
 
 out:
+	/* Only get isize again since it costs time much */
+	if (repair)
+		count_dir_isize(root, inode_id, &size);
+
 	/* verify INODE_ITEM nlink/isize/nbytes */
 	if (dir) {
 		if (nlink != 1) {
-- 
2.11.1

[PATCH 10/20] btrfs-progs: dir-item.c: modify btrfs_insert_dir_item

[Su Yue]

In the function 'btrfs_insert_dir_item', let it continue to
insert dir index if the dir item is existed and set ret value
to 0.

This further is for further repair.

Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
---
 dir-item.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/dir-item.c b/dir-item.c
index 846fc292..a112ed72 100644
--- a/dir-item.c
+++ b/dir-item.c
@@ -135,7 +135,14 @@ int btrfs_insert_dir_item(struct btrfs_trans_handle *trans, struct btrfs_root
 					name, name_len);
 	if (IS_ERR(dir_item)) {
 		ret = PTR_ERR(dir_item);
-		goto out;
+
+		/* Continue to insert item if existed */
+		if (ret == -EEXIST) {
+			ret = 0;
+			goto next;
+		} else {
+			goto out;
+		}
 	}
 
 	leaf = path->nodes[0];
@@ -149,6 +156,7 @@ int btrfs_insert_dir_item(struct btrfs_trans_handle *trans, struct btrfs_root
 	write_extent_buffer(leaf, name, name_ptr, name_len);
 	btrfs_mark_buffer_dirty(leaf);
 
+next:
 	/* FIXME, use some real flag for selecting the extra index */
 	if (root == root->fs_info->tree_root) {
 		ret = 0;
@@ -162,8 +170,11 @@ int btrfs_insert_dir_item(struct btrfs_trans_handle *trans, struct btrfs_root
 					name, name_len);
 	if (IS_ERR(dir_item)) {
 		ret2 = PTR_ERR(dir_item);
+		if (ret2 == -EEXIST)
+			ret = 0;
 		goto out;
 	}
+
 	leaf = path->nodes[0];
 	btrfs_cpu_key_to_disk(&disk_key, location);
 	btrfs_set_dir_item_key(leaf, dir_item, &disk_key);
-- 
2.11.1

[PATCH 11/20] btrfs-progs: inode.c: alter btrfs_add_link

[Su Yue]

Add an arg 'ignore_existed' to btrfs_add_link.
If ignore_existed=1, continue to add while relative dir index/item
or inode ref is already existed.

This patch is for further repair.

Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
---
 cmds-check.c   |  6 +++---
 convert/main.c |  2 +-
 ctree.h        |  2 +-
 inode.c        | 46 ++++++++++++++++++++++++++--------------------
 4 files changed, 31 insertions(+), 25 deletions(-)

diff --git a/cmds-check.c b/cmds-check.c
index 685f4f5d..bda0849b 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -2855,7 +2855,7 @@ static int reset_nlink(struct btrfs_trans_handle *trans,
 	list_for_each_entry(backref, &rec->backrefs, list) {
 		ret = btrfs_add_link(trans, root, rec->ino, backref->dir,
 				     backref->name, backref->namelen,
-				     backref->filetype, &backref->index, 1);
+				     backref->filetype, &backref->index, 1, 0);
 		if (ret < 0)
 			goto out;
 	}
@@ -2947,7 +2947,7 @@ static int repair_inode_nlinks(struct btrfs_trans_handle *trans,
 			goto out;
 		}
 		ret = btrfs_add_link(trans, root, rec->ino, lost_found_ino,
-				     namebuf, namelen, type, NULL, 1);
+				     namebuf, namelen, type, NULL, 1, 0);
 		/*
 		 * Add ".INO" suffix several times to handle case where
 		 * "FILENAME.INO" is already taken by another file.
@@ -2966,7 +2966,7 @@ static int repair_inode_nlinks(struct btrfs_trans_handle *trans,
 			namelen += count_digits(rec->ino) + 1;
 			ret = btrfs_add_link(trans, root, rec->ino,
 					     lost_found_ino, namebuf,
-					     namelen, type, NULL, 1);
+					     namelen, type, NULL, 1, 0);
 		}
 		if (ret < 0) {
 			fprintf(stderr,
diff --git a/convert/main.c b/convert/main.c
index 8d9f29fa..7607bec1 100644
--- a/convert/main.c
+++ b/convert/main.c
@@ -998,7 +998,7 @@ static int create_image(struct btrfs_root *root,
 	if (ret < 0)
 		goto out;
 	ret = btrfs_add_link(trans, root, ino, BTRFS_FIRST_FREE_OBJECTID, name,
-			     strlen(name), BTRFS_FT_REG_FILE, NULL, 1);
+			     strlen(name), BTRFS_FT_REG_FILE, NULL, 1, 0);
 	if (ret < 0)
 		goto out;
 
diff --git a/ctree.h b/ctree.h
index 0c34ae20..a28e36de 100644
--- a/ctree.h
+++ b/ctree.h
@@ -2779,7 +2779,7 @@ int btrfs_change_inode_flags(struct btrfs_trans_handle *trans,
 			     struct btrfs_root *root, u64 ino, u64 flags);
 int btrfs_add_link(struct btrfs_trans_handle *trans, struct btrfs_root *root,
 		   u64 ino, u64 parent_ino, char *name, int namelen,
-		   u8 type, u64 *index, int add_backref);
+		   u8 type, u64 *index, int add_backref, int ignore_existed);
 int btrfs_unlink(struct btrfs_trans_handle *trans, struct btrfs_root *root,
 		 u64 ino, u64 parent_ino, u64 index, const char *name,
 		 int namelen, int add_orphan);
diff --git a/inode.c b/inode.c
index 991b8ddb..62e8abec 100644
--- a/inode.c
+++ b/inode.c
@@ -161,7 +161,7 @@ out:
  */
 int btrfs_add_link(struct btrfs_trans_handle *trans, struct btrfs_root *root,
 		   u64 ino, u64 parent_ino, char *name, int namelen,
-		   u8 type, u64 *index, int add_backref)
+		   u8 type, u64 *index, int add_backref, int ignore_existed)
 {
 	struct btrfs_path *path;
 	struct btrfs_key key;
@@ -184,33 +184,38 @@ int btrfs_add_link(struct btrfs_trans_handle *trans, struct btrfs_root *root,
 	}
 
 	ret = check_dir_conflict(root, name, namelen, parent_ino, ret_index);
-	if (ret < 0)
+	if (ret < 0 && (!ignore_existed || ret != -EEXIST))
 		goto out;
 
 	/* Add inode ref */
 	if (add_backref) {
 		ret = btrfs_insert_inode_ref(trans, root, name, namelen,
 					     ino, parent_ino, ret_index);
-		if (ret < 0)
+		if (ret < 0 && (!ignore_existed || ret != -EEXIST))
 			goto out;
 
-		/* Update nlinks for the inode */
-		key.objectid = ino;
-		key.type = BTRFS_INODE_ITEM_KEY;
-		key.offset = 0;
-		ret = btrfs_search_slot(trans, root, &key, path, 1, 1);
-		if (ret) {
-			if (ret > 0)
-				ret = -ENOENT;
-			goto out;
+		/* do not update nlinks if existed */
+		if (!ret) {
+			/* Update nlinks for the inode */
+			key.objectid = ino;
+			key.type = BTRFS_INODE_ITEM_KEY;
+			key.offset = 0;
+			ret = btrfs_search_slot(trans, root, &key, path, 1, 1);
+			if (ret) {
+				if (ret > 0)
+					ret = -ENOENT;
+				goto out;
+			}
+			inode_item = btrfs_item_ptr(path->nodes[0],
+						    path->slots[0],
+						    struct btrfs_inode_item);
+			nlink = btrfs_inode_nlink(path->nodes[0], inode_item);
+			nlink++;
+			btrfs_set_inode_nlink(path->nodes[0], inode_item,
+					      nlink);
+			btrfs_mark_buffer_dirty(path->nodes[0]);
+			btrfs_release_path(path);
 		}
-		inode_item = btrfs_item_ptr(path->nodes[0], path->slots[0],
-				    struct btrfs_inode_item);
-		nlink = btrfs_inode_nlink(path->nodes[0], inode_item);
-		nlink++;
-		btrfs_set_inode_nlink(path->nodes[0], inode_item, nlink);
-		btrfs_mark_buffer_dirty(path->nodes[0]);
-		btrfs_release_path(path);
 	}
 
 	/* Add dir_item and dir_index */
@@ -219,6 +224,7 @@ int btrfs_add_link(struct btrfs_trans_handle *trans, struct btrfs_root *root,
 	key.offset = 0;
 	ret = btrfs_insert_dir_item(trans, root, name, namelen, parent_ino,
 				    &key, type, ret_index);
+
 	if (ret < 0)
 		goto out;
 
@@ -561,7 +567,7 @@ int btrfs_mkdir(struct btrfs_trans_handle *trans, struct btrfs_root *root,
 	if (ret)
 		goto out;
 	ret = btrfs_add_link(trans, root, ret_ino, parent_ino, name, namelen,
-			     BTRFS_FT_DIR, NULL, 1);
+			     BTRFS_FT_DIR, NULL, 1, 0);
 	if (ret)
 		goto out;
 out:
-- 
2.11.1

[PATCH 12/20] btrfs-progs: cmds-check.c: introduce __create_inode_item

[Su Yue]

Introduce '__create_inode' to create and insert inode item.
Modify origin 'create_inode_item' call it.
Create 'create_inode_item_lowmem' call it.
The patch is for further lowmem repair.

Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
---
 cmds-check.c | 83 ++++++++++++++++++++++++++++++++++++++----------------------
 1 file changed, 53 insertions(+), 30 deletions(-)

diff --git a/cmds-check.c b/cmds-check.c
index bda0849b..256bfbc9 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -2586,13 +2586,55 @@ static int delete_dir_index(struct btrfs_root *root,
 	return ret;
 }
 
+static int __create_inode_item(struct btrfs_trans_handle *trans,
+			       struct btrfs_root *root, u64 ino, u64 size,
+			       u64 nbytes, u64 nlink, u32 mode)
+{
+	struct btrfs_inode_item ii;
+	time_t now = time(NULL);
+	int ret;
+
+	btrfs_set_stack_inode_size(&ii, size);
+	btrfs_set_stack_inode_nbytes(&ii, nbytes);
+	btrfs_set_stack_inode_nlink(&ii, nlink);
+	btrfs_set_stack_inode_mode(&ii, mode);
+	btrfs_set_stack_inode_generation(&ii, trans->transid);
+	btrfs_set_stack_timespec_nsec(&ii.atime, 0);
+	btrfs_set_stack_timespec_sec(&ii.ctime, now);
+	btrfs_set_stack_timespec_nsec(&ii.ctime, 0);
+	btrfs_set_stack_timespec_sec(&ii.mtime, now);
+	btrfs_set_stack_timespec_nsec(&ii.mtime, 0);
+	btrfs_set_stack_timespec_sec(&ii.otime, 0);
+	btrfs_set_stack_timespec_nsec(&ii.otime, 0);
+
+	ret = btrfs_insert_inode(trans, root, ino, &ii);
+	ASSERT(!ret);
+
+	warning("root %llu inode %llu recreating inode item, this may "
+		"be incomplete, please check permissions and content after "
+		"the fsck completes.\n", (unsigned long long)root->objectid,
+		(unsigned long long)ino);
+
+	return 0;
+}
+
+static int create_inode_item_lowmem(struct btrfs_trans_handle *trans,
+				    struct btrfs_root *root, u64 ino,
+				    u8 filetype)
+{
+	u32 mode = (filetype == BTRFS_FT_DIR ? S_IFDIR : S_IFREG) | 0755;
+
+	return __create_inode_item(trans, root, ino, 0, 0, 0, mode);
+}
+
 static int create_inode_item(struct btrfs_root *root,
 			     struct inode_record *rec,
 			     struct inode_backref *backref, int root_dir)
 {
 	struct btrfs_trans_handle *trans;
-	struct btrfs_inode_item inode_item;
-	time_t now = time(NULL);
+	u64 nlink = 0;
+	u32 mode = 0;
+	u64 size = 0;
 	int ret;
 
 	trans = btrfs_start_transaction(root, 1);
@@ -2601,18 +2643,7 @@ static int create_inode_item(struct btrfs_root *root,
 		return ret;
 	}
 
-	fprintf(stderr, "root %llu inode %llu recreating inode item, this may "
-		"be incomplete, please check permissions and content after "
-		"the fsck completes.\n", (unsigned long long)root->objectid,
-		(unsigned long long)rec->ino);
-
-	memset(&inode_item, 0, sizeof(inode_item));
-	btrfs_set_stack_inode_generation(&inode_item, trans->transid);
-	if (root_dir)
-		btrfs_set_stack_inode_nlink(&inode_item, 1);
-	else
-		btrfs_set_stack_inode_nlink(&inode_item, rec->found_link);
-	btrfs_set_stack_inode_nbytes(&inode_item, rec->found_size);
+	nlink = root_dir ? 1 : rec->found_link;
 	if (rec->found_dir_item) {
 		if (rec->found_file_extent)
 			fprintf(stderr, "root %llu inode %llu has both a dir "
@@ -2620,23 +2651,15 @@ static int create_inode_item(struct btrfs_root *root,
 				"regular file so setting it as a directory\n",
 				(unsigned long long)root->objectid,
 				(unsigned long long)rec->ino);
-		btrfs_set_stack_inode_mode(&inode_item, S_IFDIR | 0755);
-		btrfs_set_stack_inode_size(&inode_item, rec->found_size);
+		mode = S_IFDIR | 0755;
+		size = rec->found_size;
 	} else if (!rec->found_dir_item) {
-		btrfs_set_stack_inode_size(&inode_item, rec->extent_end);
-		btrfs_set_stack_inode_mode(&inode_item, S_IFREG | 0755);
-	}
-	btrfs_set_stack_timespec_sec(&inode_item.atime, now);
-	btrfs_set_stack_timespec_nsec(&inode_item.atime, 0);
-	btrfs_set_stack_timespec_sec(&inode_item.ctime, now);
-	btrfs_set_stack_timespec_nsec(&inode_item.ctime, 0);
-	btrfs_set_stack_timespec_sec(&inode_item.mtime, now);
-	btrfs_set_stack_timespec_nsec(&inode_item.mtime, 0);
-	btrfs_set_stack_timespec_sec(&inode_item.otime, 0);
-	btrfs_set_stack_timespec_nsec(&inode_item.otime, 0);
-
-	ret = btrfs_insert_inode(trans, root, rec->ino, &inode_item);
-	BUG_ON(ret);
+		size = rec->extent_end;
+		mode =  S_IFREG | 0755;
+	}
+
+	ret = __create_inode_item(trans, root, rec->ino, size, rec->nbytes,
+				  nlink, mode);
 	btrfs_commit_transaction(trans, root);
 	return 0;
 }
-- 
2.11.1

[PATCH 13/20] btrfs-progs: cmds-check.c: repair_inode_item_missing

[Su Yue]

Introduce 'repair_inode_item_missing' to fix INODE_ITEM_MISSING
This patch is for further repair.

Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
---
 cmds-check.c | 43 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/cmds-check.c b/cmds-check.c
index 256bfbc9..6dafbd7d 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -5463,6 +5463,49 @@ out:
 }
 
 /*
+ * Insert the missing inode item.
+ *
+ * Returns 0 means success.
+ * Returns <0 means error.
+ */
+static int repair_inode_item_missing(struct btrfs_root *root, u64 ino,
+				     u8 filetype, int err)
+{
+	struct btrfs_key key;
+	struct btrfs_trans_handle *trans;
+	struct btrfs_path path;
+	int ret;
+
+	if (!err)
+		return 0;
+
+	key.objectid = ino;
+	key.type = BTRFS_INODE_ITEM_KEY;
+	key.offset = 0;
+
+	btrfs_init_path(&path);
+	trans = btrfs_start_transaction(root, 1);
+	if (IS_ERR(trans)) {
+		ret = -EIO;
+		goto out;
+	}
+
+	ret = btrfs_search_slot(trans, root, &key, &path, 1, 1);
+	if (ret < 0 || !ret)
+		goto out;
+	/* insert inode item */
+	create_inode_item_lowmem(trans, root, ino, filetype);
+	ret = 0;
+	btrfs_commit_transaction(trans, root);
+out:
+	if (ret)
+		error("Failed to repair root %llu INODE ITEM[%llu] missing",
+		      root->objectid, ino);
+	btrfs_release_path(&path);
+	return ret;
+}
+
+/*
  * check first root dir's inode_item, inde_ref
  *
  * returns 0 means no error
-- 
2.11.1

[PATCH 14/20] btrfs-progs: cmds-check.c: repair_fs_first_inode

[Su Yue]

Introduce 'repair_fs_first_inode' to repair first inode errors.

Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
---
 cmds-check.c | 69 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 69 insertions(+)

diff --git a/cmds-check.c b/cmds-check.c
index 6dafbd7d..85e87884 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -5506,6 +5506,71 @@ out:
 }
 
 /*
+ * Normal INODE_ITEM_MISSING and INODE_REF_MISSING are handled in backref
+ * dir. Root dir should be handled specially because root dir is the root
+ * of fs.
+ *
+ * returns 0 means success
+ * returns <0 means failure
+ */
+static int repair_fs_first_inode(struct btrfs_root *root, int *err_ret)
+{
+	struct btrfs_trans_handle *trans;
+	struct btrfs_key key;
+	struct btrfs_path path;
+	int filetype = BTRFS_FT_DIR;
+	int ret = 0;
+
+	ASSERT(err_ret);
+	btrfs_init_path(&path);
+
+	if (*err_ret & INODE_REF_MISSING) {
+		key.objectid = BTRFS_FIRST_FREE_OBJECTID;
+		key.type = BTRFS_INODE_REF_KEY;
+		key.offset = BTRFS_FIRST_FREE_OBJECTID;
+
+		trans = btrfs_start_transaction(root, 1);
+		if (IS_ERR(trans))
+			return PTR_ERR(trans);
+
+		btrfs_release_path(&path);
+		ret = btrfs_search_slot(trans, root, &key, &path, 1, 1);
+		if (ret < 0 || !ret)
+			goto out;
+
+		ret = btrfs_insert_inode_ref(trans, root, "..", strlen(".."),
+					     BTRFS_FIRST_FREE_OBJECTID,
+					     BTRFS_FIRST_FREE_OBJECTID, 0);
+		if (ret > 0)
+			ret = -ret;
+		if (ret) {
+			error("Failed to insert first inode ref");
+			goto out;
+		}
+		printf("Add INODE_REF[%llu %llu] name %s",
+		       BTRFS_FIRST_FREE_OBJECTID, BTRFS_FIRST_FREE_OBJECTID,
+		       "..");
+		*err_ret &= ~INODE_REF_MISSING;
+		btrfs_commit_transaction(trans, root);
+	}
+
+	if (*err_ret & INODE_ITEM_MISSING) {
+		ret = repair_inode_item_missing(root,
+						BTRFS_FIRST_FREE_OBJECTID,
+						filetype, *err_ret);
+		if (ret)
+			goto out;
+		*err_ret &= ~INODE_ITEM_MISSING;
+	}
+
+out:
+	if (ret)
+		error("Failed to repair first inode");
+	btrfs_release_path(&path);
+	return ret;
+}
+
+/*
  * check first root dir's inode_item, inde_ref
  *
  * returns 0 means no error
@@ -5560,6 +5625,10 @@ static int check_fs_first_inode(struct btrfs_root *root, unsigned int ext_ref)
 
 out:
 	btrfs_release_path(&path);
+
+	if (err && repair)
+		repair_fs_first_inode(root, &err);
+
 	if (err & (INODE_ITEM_MISSING | INODE_ITEM_MISMATCH))
 		error("root dir INODE_ITEM is %s",
 		      err & INODE_ITEM_MISMATCH ? "mismatch" : "missing");
-- 
2.11.1

[PATCH 15/20] btrfs-progs: cmds-check.c: introduce repair_ternary_lowmem

[Su Yue]

Introduce 'repair_ternary_lowmem' to repair error while checking
dir_item/index, inode_ref by the rule:
1. If two of three is missing or mismatched, delete the existed one.
2. If one of three is missing or mismatched, add the missing one.

Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
---
 cmds-check.c | 57 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 57 insertions(+)

diff --git a/cmds-check.c b/cmds-check.c
index 85e87884..9a76107e 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -4388,6 +4388,63 @@ static void print_inode_ref_err(struct btrfs_root *root, struct btrfs_key *key,
 		       namebuf, filetype);
 }
 
+static int repair_inode_item_missing(struct btrfs_root *root, u64 ino,
+				     u8 filetype, int err);
+/*
+ * The ternary contains dir item, dir index and relative inode ref.
+ * the repair function will handle errs: INODE_MISSING, DIR_INDEX_MISSING
+ * DIR_INDEX_MISMATCH, DIR_ITEM_MISSING, DIR_ITEM_MISMATCH by the follow
+ * strategy:
+ * If two of three is missing or mismatched, delete the existed one.
+ * If one of three is missing or mismatched, add the missing one.
+ *
+ * returns 0 mens success.
+ */
+int repair_ternary_lowmem(struct btrfs_root *root, u64 dir_ino, u64 ino,
+			  u64 index, char *name, int name_len, u8 filetype,
+			  int err)
+{
+	struct btrfs_trans_handle *trans;
+	int stage = 0;
+	int ret;
+
+	if (!err)
+		return 0;
+	/*
+	 * stage shall be one of follow valild values:
+	 *	0: Fine, nothing to do.
+	 *	1: One of three is wrong, so add missing one.
+	 *	2: Two of three is wrong, so delete existed one.
+	 */
+	if (err & (DIR_INDEX_MISMATCH | DIR_INDEX_MISSING))
+		++stage;
+	if (err & (DIR_ITEM_MISMATCH | DIR_ITEM_MISSING))
+		++stage;
+	if (err & (INODE_REF_MISSING))
+		++stage;
+
+	/* stage must be smllarer than 3 */
+	ASSERT(stage < 3);
+
+	trans = btrfs_start_transaction(root, 2);
+	if (stage == 2) {
+		ret = btrfs_unlink(trans, root, ino, dir_ino, index, name,
+				   name_len, 0);
+		if (ret)
+			goto out;
+	}
+	if (stage == 1) {
+		ret = btrfs_add_link(trans, root, ino, dir_ino, name, name_len,
+			       filetype, &index, 1, 1);
+		if (ret)
+			goto out;
+	}
+
+	btrfs_commit_transaction(trans, root);
+out:
+	return ret;
+}
+
 /*
  * Traverse the given INODE_REF and call find_dir_item() to find related
  * DIR_ITEM/DIR_INDEX.If repair is enable, research @ref_key and
-- 
2.11.1

[PATCH 16/20] btrfs-progs: cmds-check.c: Introduce repair_dir_item

[Su Yue]

Introduce 'repair_dir_item' to repair dir item/index missing/mismatch
and relative inode item missing while checking.

Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
---
 cmds-check.c | 78 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 78 insertions(+)

diff --git a/cmds-check.c b/cmds-check.c
index 9a76107e..1b35a5fd 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -4906,6 +4906,60 @@ out:
 }
 
 /*
+ * Call repair_inode_item_missing and repair_ternary_lowmem to repair
+ *
+ * @diff_ret:     same as repair_ternary_lowmem
+ *
+ * Returns 0 means success
+ */
+static int repair_dir_item(struct btrfs_root *root, u64 dirid, u64 ino,
+			   u64 index, u8 filetype, char *namebuf, u32 name_len,
+			   int *err_ret)
+{
+	int ret = 0;
+
+	if (*err_ret & INODE_ITEM_MISSING) {
+		ret = repair_inode_item_missing(root, ino, filetype, *err_ret);
+		if (!ret)
+			*err_ret &= ~(INODE_ITEM_MISMATCH |
+				      INODE_ITEM_MISSING);
+	}
+
+	if (*err_ret & ~(INODE_ITEM_MISMATCH | INODE_ITEM_MISSING)) {
+		ret = repair_ternary_lowmem(root, dirid, ino, index, namebuf,
+					    name_len, filetype, *err_ret);
+		if (!ret) {
+			*err_ret &= ~(DIR_INDEX_MISMATCH | DIR_INDEX_MISSING);
+			*err_ret &= ~(DIR_ITEM_MISMATCH | DIR_ITEM_MISSING);
+			*err_ret &= ~(INODE_REF_MISSING);
+		}
+	}
+
+	return ret;
+}
+
+/*
+ * Research @path by the @key, if it fails then change path to previous item.
+ *
+ * returns 0 means success
+ * returns <0 means failure
+ * return >0 means jumped to previous item
+ */
+static int research_path(struct btrfs_root *root, struct btrfs_path *path,
+			 struct btrfs_key *key)
+{
+	int ret;
+	/* research path */
+	btrfs_release_path(path);
+	ret = btrfs_search_slot(NULL, root, key, path, 0, 0);
+	if (ret > 0) {
+		ret = btrfs_previous_item(root, path, key->objectid,
+					  key->type);
+	}
+
+	return ret;
+}
+/*
  * Traverse the given DIR_ITEM/DIR_INDEX and check related INODE_ITEM and
  * call find_inode_ref() to check related INODE_REF/INODE_EXTREF.If repair
  * is enable, do repair and research by @path->nodes[0].
@@ -4941,6 +4995,7 @@ static int check_dir_item(struct btrfs_root *root, struct btrfs_key *key,
 	int ret;
 	int err = 0;
 	int tmp_err;
+	int need_research = 0;
 
 	/*
 	 * For DIR_ITEM set index to (u64)-1, so that find_inode_ref
@@ -4948,10 +5003,22 @@ static int check_dir_item(struct btrfs_root *root, struct btrfs_key *key,
 	 */
 	index = (key->type == BTRFS_DIR_INDEX_KEY) ? key->offset : (u64)-1;
 
+research:
+	if (need_research) {
+		ret = research_path(root, path, key);
+		need_research = 0;
+		if (ret)
+			return ret > 0 ? 0 : ret;
+	}
+
+	err = 0;
+	cur = 0;
 	node = path->nodes[0];
 	slot = path->slots[0];
+
 	di = btrfs_item_ptr(node, slot, struct btrfs_dir_item);
 	total = btrfs_item_size_nr(node, slot);
+	memset(namebuf, 0, sizeof(namebuf) / sizeof(*namebuf));
 
 	while (cur < total) {
 		data_len = btrfs_dir_data_len(node, di);
@@ -5021,6 +5088,16 @@ static int check_dir_item(struct btrfs_root *root, struct btrfs_key *key,
 next:
 		btrfs_release_path(path);
 
+		if (tmp_err && repair) {
+			ret = repair_dir_item(root, key->objectid,
+					      location.objectid, index,
+					      imode_to_type(mode), namebuf,
+					      name_len, &tmp_err);
+			if (!ret) {
+				need_research = 1;
+				goto research;
+			}
+		}
 		print_dir_item_err(root, key, location.objectid,
 				   index, namebuf, name_len, filetype,
 				   tmp_err);
@@ -5043,6 +5120,7 @@ next:
 		err |= ret > 0 ? ENOENT : ret;
 	return err;
 }
+
 /*
  * Check file extent datasum/hole, update the size of the file extents,
  * check and update the last offset of the file extent.
-- 
2.11.1

[PATCH 17/20] btrfs-progs: cmds-check.c: repair inode ref

[Su Yue]

Call repair_ternary_lowmem while checking inode ref.

Introduce 'repair_dir_item' calls above function to repair dir_item.

Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
---
 cmds-check.c | 29 +++++++++++++++++++++++++++--
 1 file changed, 27 insertions(+), 2 deletions(-)

diff --git a/cmds-check.c b/cmds-check.c
index 1b35a5fd..9ac08dfd 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -4445,6 +4445,8 @@ out:
 	return ret;
 }
 
+static int research_path(struct btrfs_root *root, struct btrfs_path *path,
+			 struct btrfs_key *key);
 /*
  * Traverse the given INODE_REF and call find_dir_item() to find related
  * DIR_ITEM/DIR_INDEX.If repair is enable, research @ref_key and
@@ -4475,26 +4477,39 @@ static int check_inode_ref(struct btrfs_root *root, struct btrfs_key *ref_key,
 	u32 cur = 0;
 	long len;
 	u64 index;
-	int err = 0;
+	int ret, err = 0;
 	int tmp_err;
 	int slot;
+	int need_research = 0;
+
+research:
+	if (need_research) {
+		ret = research_path(root, path, ref_key);
+		need_research = 1;
+		if (ret)
+			return ret > 0 ? 0 : ret;
+	}
 
+	err = 0;
+	cur = 0;
+	*refs = 0;
 	location.objectid = ref_key->objectid;
 	location.type = BTRFS_INODE_ITEM_KEY;
 	location.offset = 0;
 	node = path->nodes[0];
 	slot = path->slots[0];
 
+	memset(namebuf, 0, sizeof(namebuf) / sizeof(*namebuf));
 	ref = btrfs_item_ptr(node, slot, struct btrfs_inode_ref);
 	total = btrfs_item_size_nr(node, slot);
 
 next:
 	/* Update inode ref count */
 	(*refs)++;
-
 	tmp_err = 0;
 	index = btrfs_inode_ref_index(node, ref);
 	name_len = btrfs_inode_ref_name_len(node, ref);
+
 	if (name_len <= BTRFS_NAME_LEN) {
 		len = name_len;
 	} else {
@@ -4536,6 +4551,16 @@ next:
 	tmp_err |= find_dir_item(root, &key, &location, namebuf, len,
 			    imode_to_type(mode));
 end:
+	if (tmp_err && repair) {
+		ret = repair_ternary_lowmem(root, ref_key->offset,
+					    ref_key->objectid, index, namebuf,
+					    name_len, imode_to_type(mode),
+					    tmp_err);
+		if (!ret) {
+			need_research = true;
+			goto research;
+		}
+	}
 	print_inode_ref_err(root, ref_key, index, namebuf, name_len,
 			    imode_to_type(mode), tmp_err);
 	err |= tmp_err;
-- 
2.11.1

[PATCH 18/20] btrfs-progs: cmds-check.c: repair nlinks lowmem

[Su Yue]

Introduce 'repair_inode_nlinks_lowmem'.
If ref is 0, move the inode to "lost + found".
Set inode item's nlink to ref_count.

Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
---
 cmds-check.c | 233 +++++++++++++++++++++++++++++++++++++++++++++--------------
 1 file changed, 179 insertions(+), 54 deletions(-)

diff --git a/cmds-check.c b/cmds-check.c
index 9ac08dfd..ae80d5f0 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -2912,15 +2912,17 @@ static int get_highest_inode(struct btrfs_trans_handle *trans,
 	return ret;
 }
 
+static int link_inode_to_lostfound(struct btrfs_trans_handle *trans,
+				   struct btrfs_root *root,
+				   struct btrfs_path *path, u64 ino,
+				   char *name, u32 name_len, u8 filetype,
+				   u64 *ref_count);
 static int repair_inode_nlinks(struct btrfs_trans_handle *trans,
 			       struct btrfs_root *root,
 			       struct btrfs_path *path,
 			       struct inode_record *rec)
 {
-	char *dir_name = "lost+found";
 	char namebuf[BTRFS_NAME_LEN] = {0};
-	u64 lost_found_ino;
-	u32 mode = 0700;
 	u8 type = 0;
 	int namelen = 0;
 	int name_recovered = 0;
@@ -2957,55 +2959,11 @@ static int repair_inode_nlinks(struct btrfs_trans_handle *trans,
 	}
 
 	if (rec->found_link == 0) {
-		ret = get_highest_inode(trans, root, path, &lost_found_ino);
-		if (ret < 0)
-			goto out;
-		lost_found_ino++;
-		ret = btrfs_mkdir(trans, root, dir_name, strlen(dir_name),
-				  BTRFS_FIRST_FREE_OBJECTID, &lost_found_ino,
-				  mode);
-		if (ret < 0) {
-			fprintf(stderr, "Failed to create '%s' dir: %s\n",
-				dir_name, strerror(-ret));
-			goto out;
-		}
-		ret = btrfs_add_link(trans, root, rec->ino, lost_found_ino,
-				     namebuf, namelen, type, NULL, 1, 0);
-		/*
-		 * Add ".INO" suffix several times to handle case where
-		 * "FILENAME.INO" is already taken by another file.
-		 */
-		while (ret == -EEXIST) {
-			/*
-			 * Conflicting file name, add ".INO" as suffix * +1 for '.'
-			 */
-			if (namelen + count_digits(rec->ino) + 1 >
-			    BTRFS_NAME_LEN) {
-				ret = -EFBIG;
-				goto out;
-			}
-			snprintf(namebuf + namelen, BTRFS_NAME_LEN - namelen,
-				 ".%llu", rec->ino);
-			namelen += count_digits(rec->ino) + 1;
-			ret = btrfs_add_link(trans, root, rec->ino,
-					     lost_found_ino, namebuf,
-					     namelen, type, NULL, 1, 0);
-		}
-		if (ret < 0) {
-			fprintf(stderr,
-				"Failed to link the inode %llu to %s dir: %s\n",
-				rec->ino, dir_name, strerror(-ret));
+		ret = link_inode_to_lostfound(trans, root, path, rec->ino,
+					      namebuf, namelen, type,
+					      (u64 *)&rec->found_link);
+		if (ret)
 			goto out;
-		}
-		/*
-		 * Just increase the found_link, don't actually add the
-		 * backref. This will make things easier and this inode
-		 * record will be freed after the repair is done.
-		 * So fsck will not report problem about this inode.
-		 */
-		rec->found_link++;
-		printf("Moving file '%.*s' to '%s' dir since it has no valid backref\n",
-		       namelen, namebuf, dir_name);
 	}
 	printf("Fixed the nlink of inode %llu\n", rec->ino);
 out:
@@ -5430,6 +5388,160 @@ out:
 }
 
 /*
+ * Link inode to dir 'lost+found'. Increase @ref_count.
+ *
+ * Returns 0 means success.
+ * Returns <0 means failure.
+ */
+static int link_inode_to_lostfound(struct btrfs_trans_handle *trans,
+				   struct btrfs_root *root,
+				   struct btrfs_path *path,
+				   u64 ino, char *namebuf, u32 name_len,
+				   u8 filetype, u64 *ref_count)
+{
+	char *dir_name = "lost+found";
+	u64 lost_found_ino;
+	int ret;
+	u32 mode = 0700;
+
+	btrfs_release_path(path);
+	ret = get_highest_inode(trans, root, path, &lost_found_ino);
+	if (ret < 0)
+		goto out;
+	lost_found_ino++;
+
+	ret = btrfs_mkdir(trans, root, dir_name, strlen(dir_name),
+			  BTRFS_FIRST_FREE_OBJECTID, &lost_found_ino,
+			  mode);
+	if (ret < 0) {
+		error("Failed to create '%s' dir: %s\n", dir_name,
+		      strerror(-ret));
+		goto out;
+	}
+	ret = btrfs_add_link(trans, root, ino, lost_found_ino,
+			     namebuf, name_len, filetype, NULL, 1, 0);
+	/*
+	 * Add ".INO" suffix several times to handle case where
+	 * "FILENAME.INO" is already taken by another file.
+	 */
+	while (ret == -EEXIST) {
+		/*
+		 * Conflicting file name, add ".INO" as suffix * +1
+		 * for '.'
+		 */
+		if (name_len + count_digits(ino) + 1 >
+		    BTRFS_NAME_LEN) {
+			ret = -EFBIG;
+			goto out;
+		}
+		snprintf(namebuf + name_len, BTRFS_NAME_LEN - name_len,
+			 ".%llu", ino);
+		name_len += count_digits(ino) + 1;
+		ret = btrfs_add_link(trans, root, ino,
+				     lost_found_ino, namebuf,
+				     name_len, filetype, NULL, 1, 0);
+	}
+	if (ret < 0) {
+		error("Failed to link the inode %llu to %s dir: %s\n",
+		      ino, dir_name, strerror(-ret));
+		goto out;
+	}
+
+	++*ref_count;
+	printf("Moving file '%.*s' to '%s' dir since it has no valid backref\n",
+	       name_len, namebuf, dir_name);
+out:
+	btrfs_release_path(path);
+	if (ret)
+		error("Failed to move file '%.*s' to '%s' dir",
+		      name_len, namebuf, dir_name);
+	return ret;
+}
+
+/* Reset inode_item nlink to @ref_count.
+ * If @ref_count == 0, move it to "lost+found" and increase @ref_count
+ * first.
+ *
+ * @ref_count: the refs counts found
+ * @nlink    : return with value of nlink after repair
+ * Returns 0 means success
+ * Returns <0 means failure
+ */
+static int repair_inode_nlinks_lowmem(struct btrfs_root *root,
+				      struct btrfs_path *path, u64 ino,
+				      const char *name, u32 namelen,
+				      u64 ref_count, u8 filetype, u64 *nlink)
+{
+	struct btrfs_trans_handle *trans;
+	struct btrfs_inode_item *ii;
+	struct btrfs_key key;
+	struct btrfs_key key_store;
+	char namebuf[BTRFS_NAME_LEN] = {0};
+	int name_len;
+	int ret;
+	int ret2;
+	/* save the key */
+	btrfs_item_key_to_cpu(path->nodes[0], &key_store, path->slots[0]);
+
+	if (name && namelen) {
+		ASSERT(namelen <= BTRFS_NAME_LEN);
+		memcpy(namebuf, name, namelen);
+		name_len = namelen;
+	} else {
+		sprintf(namebuf, "%llu", ino);
+		name_len = count_digits(ino);
+		printf("Can't find file name for inode %llu, use %s instead\n",
+		       ino, namebuf);
+	}
+
+	trans = btrfs_start_transaction(root, 1);
+	btrfs_release_path(path);
+	if (ref_count == 0) {
+		ret = link_inode_to_lostfound(trans, root, path, ino, namebuf,
+					      name_len, filetype, &ref_count);
+		if (ret)
+			goto out;
+	}
+	/* reset inode_item's nlink to ref_count */
+	btrfs_release_path(path);
+	key.objectid = ino;
+	key.type = BTRFS_INODE_ITEM_KEY;
+	key.offset = 0;
+
+	ret = btrfs_search_slot(trans, root, &key, path, 0, 1);
+	if (ret > 0)
+		ret = -ENOENT;
+	if (ret)
+		goto out;
+
+	ii = btrfs_item_ptr(path->nodes[0], path->slots[0],
+			    struct btrfs_inode_item);
+	btrfs_set_inode_nlink(path->nodes[0], ii, ref_count);
+	btrfs_mark_buffer_dirty(path->nodes[0]);
+	btrfs_release_path(path);
+
+	btrfs_commit_transaction(trans, root);
+out:
+	btrfs_release_path(path);
+	if (nlink)
+		*nlink = ref_count;
+
+	if (!ret)
+		printf("Fixed the nlink of inode %llu root %llu name %s filetype %u\n",
+		       root->objectid, ino, namebuf, filetype);
+	else
+		printf("Fixed the nlink of inode %llu root %llu name %s filetype %u\n",
+		       root->objectid, ino, namebuf, filetype);
+
+	/* research */
+	btrfs_release_path(path);
+	ret2 = btrfs_search_slot(NULL, root, &key_store, path, 0, 0);
+	if (ret2 < 0)
+		return ret2;
+	return ret;
+}
+
+/*
  * Check INODE_ITEM and related ITEMs (the same inode number)
  * 1. check link count
  * 2. check inode ref/extref
@@ -5554,6 +5666,13 @@ out:
 
 	/* verify INODE_ITEM nlink/isize/nbytes */
 	if (dir) {
+		if ((nlink != 1 || refs != 1) && repair)
+			ret = repair_inode_nlinks_lowmem(root, path,
+							 inode_id,
+							 namebuf, name_len,
+							 refs,
+							 imode_to_type(mode),
+							 &nlink);
 		if (nlink != 1) {
 			err |= LINK_COUNT_ERROR;
 			error("root %llu DIR INODE[%llu] shouldn't have more than one link(%llu)",
@@ -5583,9 +5702,15 @@ out:
 		}
 	} else {
 		if (nlink != refs) {
-			err |= LINK_COUNT_ERROR;
-			error("root %llu INODE[%llu] nlink(%llu) not equal to inode_refs(%llu)",
-			      root->objectid, inode_id, nlink, refs);
+			if (repair)
+				ret = repair_inode_nlinks_lowmem(root,
+								 path,
+								 inode_id,
+								 namebuf,
+								 name_len,
+								 refs,
+								 imode_to_type(mode),
+								 &nlink);
 		} else if (!nlink) {
 			if (repair)
 				ret = repair_inode_orphan_item_lowmem(root,
-- 
2.11.1

[PATCH 19/20] btrfs-progs: cmds-check.c: add punch_extent_hole

[Su Yue]

Introduce 'punch_extent_hole' to punch holes while
repair file extent.

Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
---
 cmds-check.c | 51 +++++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 45 insertions(+), 6 deletions(-)

diff --git a/cmds-check.c b/cmds-check.c
index ae80d5f0..d228fc62 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -4483,6 +4483,7 @@ next:
 		memcpy(name_ret, namebuf, len);
 		*namelen_ret = len;
 	}
+
 	/* Check root dir ref */
 	if (ref_key->objectid == BTRFS_FIRST_FREE_OBJECTID) {
 		if (index != 0 || len != strlen("..") ||
@@ -5105,6 +5106,33 @@ next:
 }
 
 /*
+ * Wrapper function of btrfs_punch_hole.
+ *
+ * Returns 0 means success.
+ */
+static int punch_extent_hole(struct btrfs_root *root, u64 ino, u64 start,
+			      u64 len)
+{
+	struct btrfs_trans_handle *trans;
+	int ret = 0;
+
+	trans = btrfs_start_transaction(root, 1);
+	if (IS_ERR(trans))
+		return PTR_ERR(trans);
+
+	ret = btrfs_punch_hole(trans, root, ino, start, len);
+	if (ret) {
+		error("Failed to add hole [%llu, %llu] in inode [%llu]",
+		      start, len, ino);
+	} else {
+		btrfs_commit_transaction(trans, root);
+		printf("Added hole [%llu, %llu] in inode [%llu]\n", start, len,
+		      ino);
+	}
+	return ret;
+}
+
+/*
  * Check file extent datasum/hole, update the size of the file extents,
  * check and update the last offset of the file extent.
  *
@@ -5222,9 +5250,14 @@ static int check_file_extent(struct btrfs_root *root, struct btrfs_key *fkey,
 		error("root %llu EXTENT_DATA[%llu %llu] shouldn't be hole",
 		      root->objectid, fkey->objectid, fkey->offset);
 	} else if (!no_holes && *end != fkey->offset) {
-		err |= FILE_EXTENT_ERROR;
-		error("root %llu EXTENT_DATA[%llu %llu] interrupt",
-		      root->objectid, fkey->objectid, fkey->offset);
+		if (repair)
+			ret = punch_extent_hole(root, fkey->objectid,
+						*end, fkey->offset - *end);
+		if (!!repair || ret) {
+			err |= FILE_EXTENT_ERROR;
+			error("root %llu EXTENT_DATA[%llu %llu] interrupt",
+			      root->objectid, fkey->objectid, fkey->offset);
+		}
 	}
 
 	*end += extent_num_bytes;
@@ -5724,9 +5757,15 @@ out:
 		}
 
 		if (!nbytes && !no_holes && extent_end < isize) {
-			err |= NBYTES_ERROR;
-			error("root %llu INODE[%llu] size (%llu) should have a file extent hole",
-			      root->objectid, inode_id, isize);
+			if (repair)
+				ret = punch_extent_hole(root, inode_id,
+							extent_end,
+							isize-extent_end);
+			if (!repair || ret) {
+				err |= NBYTES_ERROR;
+				error("root %llu INODE[%llu] size (%llu) should have a file extent hole",
+				      root->objectid, inode_id, isize);
+			}
 		}
 
 		if (nbytes != extent_size) {
-- 
2.11.1

[PATCH 20/20] btrfs-progs: fsck-check: Allow fsck check test to repair in lowmem mode for certain test cases

[Su Yue]

From: Qu Wenruo <quwenruo@cn.fujitsu.com>

Since lowmem mode can repair certain corruptions (mostly in fs tree),
insert a beacon into each fsck test cases to allow some of them be
tested for lowmem mode.

With this patch, fsck option override will check the beacon file
".lowmem_repairbale" in the same directory of the test image, and if the
beacon exists, then it will also run lowmem mode repair to repair the
image.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 tests/common.local                                         | 14 +++++++++++++-
 tests/fsck-tests/004-no-dir-index/.lowmem_repairable       |  0
 .../fsck-tests/009-no-dir-item-or-index/.lowmem_repairable |  0
 .../010-no-rootdir-inode-item/.lowmem_repairable           |  0
 tests/fsck-tests/011-no-inode-item/.lowmem_repairable      |  0
 tests/fsck-tests/016-wrong-inode-nbytes/.lowmem_repairable |  0
 .../017-missing-all-file-extent/.lowmem_repairable         |  0
 7 files changed, 13 insertions(+), 1 deletion(-)
 create mode 100644 tests/fsck-tests/004-no-dir-index/.lowmem_repairable
 create mode 100644 tests/fsck-tests/009-no-dir-item-or-index/.lowmem_repairable
 create mode 100644 tests/fsck-tests/010-no-rootdir-inode-item/.lowmem_repairable
 create mode 100644 tests/fsck-tests/011-no-inode-item/.lowmem_repairable
 create mode 100644 tests/fsck-tests/016-wrong-inode-nbytes/.lowmem_repairable
 create mode 100644 tests/fsck-tests/017-missing-all-file-extent/.lowmem_repairable

diff --git a/tests/common.local b/tests/common.local
index 4f56bb08..af372f16 100644
--- a/tests/common.local
+++ b/tests/common.local
@@ -15,11 +15,23 @@ TEST_ARGS_CHECK=--mode=lowmem
 # gets arguments of a current command and can decide if the argument insertion
 # should happen, eg. if some option combination does not make sense or would
 # break tests
+#
+# Return 0 if we need to skip option override
+# Return 1 if we don't need to skip option override
 _skip_spec()
 {
+	beacon=.lowmem_repairable
+
+	# For loemem repair, only support fs tree repair yet
+	# So we place lowmem repair beacon in the same dir of the
+	# test case
 	if echo "$TEST_ARGS_CHECK" | grep -q 'mode=lowmem' &&
 	   echo "$@" | grep -q -- '--repair'; then
-		return 0
+		dir="$(dirname ${@: -1})"
+		if [ -f ${dir}/${beacon} ]; then
+			return 1;
+		fi
+		return 0;
 	fi
 	return 1
 }
diff --git a/tests/fsck-tests/004-no-dir-index/.lowmem_repairable b/tests/fsck-tests/004-no-dir-index/.lowmem_repairable
new file mode 100644
index 00000000..e69de29b
diff --git a/tests/fsck-tests/009-no-dir-item-or-index/.lowmem_repairable b/tests/fsck-tests/009-no-dir-item-or-index/.lowmem_repairable
new file mode 100644
index 00000000..e69de29b
diff --git a/tests/fsck-tests/010-no-rootdir-inode-item/.lowmem_repairable b/tests/fsck-tests/010-no-rootdir-inode-item/.lowmem_repairable
new file mode 100644
index 00000000..e69de29b
diff --git a/tests/fsck-tests/011-no-inode-item/.lowmem_repairable b/tests/fsck-tests/011-no-inode-item/.lowmem_repairable
new file mode 100644
index 00000000..e69de29b
diff --git a/tests/fsck-tests/016-wrong-inode-nbytes/.lowmem_repairable b/tests/fsck-tests/016-wrong-inode-nbytes/.lowmem_repairable
new file mode 100644
index 00000000..e69de29b
diff --git a/tests/fsck-tests/017-missing-all-file-extent/.lowmem_repairable b/tests/fsck-tests/017-missing-all-file-extent/.lowmem_repairable
new file mode 100644
index 00000000..e69de29b
-- 
2.11.1

Re: [PATCH 00/20] Enable lowmem repair for fs/subvolume tree

[David Sterba]

On Wed, Mar 01, 2017 at 11:13:43AM +0800, Su Yue wrote:
> It can be feched from my github:
> https://github.com/Damenly/btrfs-progs.git lowmem_repair
> 
> This patchset can repair errors found in fs tree in lowmem mode.
> 
> This patchset request includes:
> 1) Repair inode nbytes error.
> 2) Repair dir isize error.
> 3) Repair orpahan inode item.
> 4) Repair dir item/index missing/mismatch.
> 5) Repair inode ref missing/mismatch.
> 6) Repair inode item missing.
> 7) Repair inode nlink error.
> 8) Punch file extent hole.
> 9) Let test-fsck test cases which can be repaired in lowmem mode.

I went through the patchset. At this point it's not close to being
merged. Check and especially repair is IMO an area that deserves more
than just a few lines in the changelog. It should really explain why the
change is correct, what corruptions is it able to repair etc. And the
changelog does not need to repeat the subject line or say what the name
of the new function is. I want to know that you understand what code are
you changing, also helping me to navigate through it as I (or somebody
else) review it. I don't think that repair is the first project one
should attempt in btrfs-progs, but I guess you get advice from Qu.

I saw many small typos or formatting glitches that it started bothering
me. You seem to format arguments at the column of the opening "(" which
looks when it's near the end of line:

+                               ret = repair_dir_isize_lowmem(root, path,
+                                                             inode_id,
+                                                             size);
+


+                       ret = repair_inode_nlinks_lowmem(root, path,
+                                                        inode_id,
+                                                        namebuf, name_len,
+                                                        refs,
+                                                        imode_to_type(mode),
+                                                        &nlink);


+                               ret = repair_inode_nlinks_lowmem(root,
+                                                                path,
+                                                                inode_id,
+                                                                namebuf,
+                                                                name_len,
+                                                                refs,
+                                                                imode_to_type(mode),
+                                                                &nlink);

There's a "prior art" should serve as a deterrent example for anybody using
this style, the infamous balance_leaf from reiserfs, before it was heavily
cleaned up by Jeff:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/reiserfs/do_balan.c?id=60f76eab19e3903e810bdc3ec846c158efcd2e21#n460

Scroll around the file to get the idea what scares me. Arguments can be packed
on next lines indented by one or two tabs. I don't bother to change that where
it does not hurt too much.

Please update the subjects, drop the file name and use just
"btrfs-progs: check: ..." .

We can schedule the changes for 4.12 release, but please start smaller this
time and pick just say 3 fixes so we can work through the preferred style
and quality level so you don't need to update 20 patches each time.

Re: [PATCH 00/20] Enable lowmem repair for fs/subvolume tree

[Su Yue]

On 03/31/2017 12:44 AM, David Sterba wrote:
> On Wed, Mar 01, 2017 at 11:13:43AM +0800, Su Yue wrote:
>> It can be feched from my github:
>> https://github.com/Damenly/btrfs-progs.git lowmem_repair
>>
>> This patchset can repair errors found in fs tree in lowmem mode.
>>
>> This patchset request includes:
>> 1) Repair inode nbytes error.
>> 2) Repair dir isize error.
>> 3) Repair orpahan inode item.
>> 4) Repair dir item/index missing/mismatch.
>> 5) Repair inode ref missing/mismatch.
>> 6) Repair inode item missing.
>> 7) Repair inode nlink error.
>> 8) Punch file extent hole.
>> 9) Let test-fsck test cases which can be repaired in lowmem mode.
>
> I went through the patchset. At this point it's not close to being
> merged. Check and especially repair is IMO an area that deserves more
> than just a few lines in the changelog. It should really explain why the
> change is correct, what corruptions is it able to repair etc. And the
> changelog does not need to repeat the subject line or say what the name
> of the new function is. I want to know that you understand what code are
> you changing, also helping me to navigate through it as I (or somebody
> else) review it. I don't think that repair is the first project one
> should attempt in btrfs-progs, but I guess you get advice from Qu.
>
> I saw many small typos or formatting glitches that it started bothering
> me. You seem to format arguments at the column of the opening "(" which
> looks when it's near the end of line:
>
> +                               ret = repair_dir_isize_lowmem(root, path,
> +                                                             inode_id,
> +                                                             size);
> +
>
>
> +                       ret = repair_inode_nlinks_lowmem(root, path,
> +                                                        inode_id,
> +                                                        namebuf, name_len,
> +                                                        refs,
> +                                                        imode_to_type(mode),
> +                                                        &nlink);
>
>
> +                               ret = repair_inode_nlinks_lowmem(root,
> +                                                                path,
> +                                                                inode_id,
> +                                                                namebuf,
> +                                                                name_len,
> +                                                                refs,
> +                                                                imode_to_type(mode),
> +                                                                &nlink);
>
> There's a "prior art" should serve as a deterrent example for anybody using
> this style, the infamous balance_leaf from reiserfs, before it was heavily
> cleaned up by Jeff:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/reiserfs/do_balan.c?id=60f76eab19e3903e810bdc3ec846c158efcd2e21#n460
>
> Scroll around the file to get the idea what scares me. Arguments can be packed
> on next lines indented by one or two tabs. I don't bother to change that where
> it does not hurt too much.
>
> Please update the subjects, drop the file name and use just
> "btrfs-progs: check: ..." .
>
> We can schedule the changes for 4.12 release, but please start smaller this
> time and pick just say 3 fixes so we can work through the preferred style
> and quality level so you don't need to update 20 patches each time.
>
>

Thanks for your advice. I will reorganize the patches then split
them up into smaller patchsets.

Thanks,
Su