linux-btrfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Gu Jinxiang <gujx@cn.fujitsu.com>
To: <linux-btrfs@vger.kernel.org>
Cc: Qu Wenruo <quwenruo@cn.fujitsu.com>, Su Yue <suy.fnst@cn.fujitsu.com>
Subject: [PATCH v5 03/15] btrfs-progs: csum: Introduce function to read out data csums
Date: Sat, 15 Jul 2017 17:10:41 +0800	[thread overview]
Message-ID: <20170715091053.19725-3-gujx@cn.fujitsu.com> (raw)
In-Reply-To: <20170715091053.19725-1-gujx@cn.fujitsu.com>

From: Qu Wenruo <quwenruo@cn.fujitsu.com>

Introduce a new function: btrfs_read_data_csums(), to read out csums
for sectors in range.

This is quite useful for read out data csum so we don't need to do it
using open code.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
---
 Makefile     |   2 +-
 csum.c       | 134 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 ctree.h      |   4 ++
 kerncompat.h |   3 ++
 utils.h      |   6 +++
 5 files changed, 148 insertions(+), 1 deletion(-)
 create mode 100644 csum.c

diff --git a/Makefile b/Makefile
index b3e2b63..6f734a6 100644
--- a/Makefile
+++ b/Makefile
@@ -96,7 +96,7 @@ objects = ctree.o disk-io.o kernel-lib/radix-tree.o extent-tree.o print-tree.o \
 	  qgroup.o free-space-cache.o kernel-lib/list_sort.o props.o \
 	  kernel-shared/ulist.o qgroup-verify.o backref.o string-table.o task-utils.o \
 	  inode.o file.o find-root.o free-space-tree.o help.o send-dump.o \
-	  fsfeatures.o kernel-lib/tables.o kernel-lib/raid56.o
+	  fsfeatures.o kernel-lib/tables.o kernel-lib/raid56.o csum.o
 cmds_objects = cmds-subvolume.o cmds-filesystem.o cmds-device.o cmds-scrub.o \
 	       cmds-inspect.o cmds-balance.o cmds-send.o cmds-receive.o \
 	       cmds-quota.o cmds-qgroup.o cmds-replace.o cmds-check.o \
diff --git a/csum.c b/csum.c
new file mode 100644
index 0000000..c922ee9
--- /dev/null
+++ b/csum.c
@@ -0,0 +1,134 @@
+/*
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include "kerncompat.h"
+#include "kernel-lib/bitops.h"
+#include "ctree.h"
+#include "utils.h"
+
+/*
+ * TODO:
+ * 1) Add write support for csum
+ *    So we can write new data extents and add csum into csum tree
+ *
+ * Get csums of range[@start, @start + len).
+ *
+ * @start:    Start offset, shall be aligned to sectorsize.
+ * @len:      Length, shall be aligned to sectorsize.
+ * @csum_ret: The size of csum_ret shall be @len / sectorsize * csum_size.
+ * @bit_map:  Every bit corresponds to the offset have csum or not.
+ *            The size in byte of bit_map should be
+ *            calculate_bitmap_len(csum_ret's size / csum_size).
+ *
+ * Returns 0  means success
+ * Returns >0 means on error
+ * Returns <0 means on fatal error
+ */
+
+int btrfs_read_data_csums(struct btrfs_fs_info *fs_info, u64 start, u64 len,
+			  void *csum_ret, unsigned long *bitmap_ret)
+
+{
+	struct btrfs_path path;
+	struct btrfs_key key;
+	struct btrfs_root *csum_root = fs_info->csum_root;
+	u32 item_offset;
+	u32 item_size;
+	u32 final_offset;
+	u32 final_len;
+	u32 sectorsize = fs_info->sectorsize;
+	u16 csum_size = btrfs_super_csum_size(fs_info->super_copy);
+	u64 cur_start;
+	u64 cur_end;
+	int found = 0;
+	int ret;
+
+	ASSERT(IS_ALIGNED(start, sectorsize));
+	ASSERT(IS_ALIGNED(len, sectorsize));
+	ASSERT(csum_ret);
+	ASSERT(bitmap_ret);
+
+	memset(bitmap_ret, 0, calculate_bitmap_len(len / sectorsize));
+	btrfs_init_path(&path);
+
+	key.objectid = BTRFS_EXTENT_CSUM_OBJECTID;
+	key.type = BTRFS_EXTENT_CSUM_KEY;
+	key.offset = start;
+
+	ret = btrfs_search_slot(NULL, csum_root, &key, &path, 0, 0);
+	if (ret < 0)
+		goto out;
+	if (ret > 0) {
+		ret = btrfs_previous_item(csum_root, &path,
+					  BTRFS_EXTENT_CSUM_OBJECTID,
+					  BTRFS_EXTENT_CSUM_KEY);
+		if (ret < 0)
+			goto out;
+	}
+	/* The csum tree may be empty. */
+	if (!btrfs_header_nritems(path.nodes[0]))
+		goto next;
+
+	while (1) {
+		btrfs_item_key_to_cpu(path.nodes[0], &key, path.slots[0]);
+
+		if (!IS_ALIGNED(key.offset, sectorsize)) {
+			error("csum item bytenr %llu is not aligned to %u",
+			      key.offset, sectorsize);
+			ret = -EIO;
+			break;
+		}
+		/* exceeds end */
+		if (key.offset >= start + len)
+			break;
+
+		item_offset = btrfs_item_ptr_offset(path.nodes[0],
+						    path.slots[0]);
+		item_size = btrfs_item_size_nr(path.nodes[0], path.slots[0]);
+
+		if (key.offset + item_size / csum_size * sectorsize < start)
+			goto next;
+
+		/* get start of the extent */
+		cur_start = max(start, key.offset);
+		/* get end of the extent */
+		cur_end = min(start + len, key.offset + item_size / csum_size *
+			      sectorsize);
+
+		final_offset = (cur_start - key.offset) / sectorsize *
+			csum_size + item_offset;
+		final_len = (cur_end - cur_start) / sectorsize * csum_size;
+		read_extent_buffer(path.nodes[0],
+				   (csum_ret + (cur_start - start) /
+				    sectorsize * csum_size),
+				   final_offset, final_len);
+
+		for (u32 i = 0; i != final_len / csum_size; i++)
+			set_bit(i + (cur_start - start) / sectorsize,
+				bitmap_ret);
+
+		found = 1;
+next:
+		ret = btrfs_next_item(csum_root, &path);
+		if (ret)
+			break;
+	}
+out:
+	if (ret >= 0)
+		ret = !found;
+	btrfs_release_path(&path);
+	return ret;
+}
diff --git a/ctree.h b/ctree.h
index 48ae890..19ba64e 100644
--- a/ctree.h
+++ b/ctree.h
@@ -2758,4 +2758,8 @@ int btrfs_punch_hole(struct btrfs_trans_handle *trans,
 int btrfs_read_file(struct btrfs_root *root, u64 ino, u64 start, int len,
 		    char *dest);
 
+/* csum.c */
+int btrfs_read_data_csums(struct btrfs_fs_info *fs_info, u64 start, u64 len,
+			  void *csum_ret, unsigned long *bitmap_ret);
+
 #endif
diff --git a/kerncompat.h b/kerncompat.h
index fa96715..4eb62f6 100644
--- a/kerncompat.h
+++ b/kerncompat.h
@@ -273,6 +273,9 @@ static inline int IS_ERR_OR_NULL(const void *ptr)
 #define round_up(x, y) ((((x)-1) | __round_mask(x, y))+1)
 #define round_down(x, y) ((x) & ~__round_mask(x, y))
 
+#define __KERNEL_DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d))
+#define DIV_ROUND_UP __KERNEL_DIV_ROUND_UP
+
 /*
  * printk
  */
diff --git a/utils.h b/utils.h
index 091f8fa..9c851a2 100644
--- a/utils.h
+++ b/utils.h
@@ -28,6 +28,7 @@
 #include "btrfs-list.h"
 #include "sizes.h"
 #include "messages.h"
+#include "kerncompat.h"
 
 #define BTRFS_SCAN_MOUNTED	(1ULL << 0)
 #define BTRFS_SCAN_LBLKID	(1ULL << 1)
@@ -68,6 +69,11 @@ void units_set_base(unsigned *units, unsigned base);
 #define	PREP_DEVICE_DISCARD	(1U << 1)
 #define	PREP_DEVICE_VERBOSE	(1U << 2)
 
+static int inline calculate_bitmap_len(int nsectors)
+{
+	return (DIV_ROUND_UP(nsectors, BITS_PER_LONG) * sizeof(long));
+}
+
 int btrfs_make_root_dir(struct btrfs_trans_handle *trans,
 			struct btrfs_root *root, u64 objectid);
 int btrfs_prepare_device(int fd, const char *file, u64 *block_count_ret,
-- 
2.9.4




  parent reply	other threads:[~2017-07-15  9:11 UTC|newest]

Thread overview: 25+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-07-15  9:10 [PATCH v5 01/15] btrfs-progs: Introduce new btrfs_map_block function which returns more unified result Gu Jinxiang
2017-07-15  9:10 ` [PATCH v5 02/15] btrfs-progs: Allow __btrfs_map_block_v2 to remove unrelated stripes Gu Jinxiang
2017-07-15  9:10 ` Gu Jinxiang [this message]
2017-07-15  9:10 ` [PATCH v5 04/15] btrfs-progs: scrub: Introduce structures to support offline scrub for RAID56 Gu Jinxiang
2017-07-15  9:10 ` [PATCH v5 05/15] btrfs-progs: scrub: Introduce functions to scrub mirror based tree block Gu Jinxiang
2017-07-15  9:10 ` [PATCH v5 06/15] btrfs-progs: scrub: Introduce functions to scrub mirror based data blocks Gu Jinxiang
2017-07-15  9:10 ` [PATCH v5 07/15] btrfs-progs: scrub: Introduce function to scrub one mirror-based extent Gu Jinxiang
2017-07-15  9:10 ` [PATCH v5 08/15] btrfs-progs: scrub: Introduce function to scrub one data stripe Gu Jinxiang
2017-07-15  9:10 ` [PATCH v5 09/15] btrfs-progs: scrub: Introduce function to verify parities Gu Jinxiang
2017-07-15  9:10 ` [PATCH v5 10/15] btrfs-progs: extent-tree: Introduce function to check if there is any extent in given range Gu Jinxiang
2017-07-15  9:10 ` [PATCH v5 11/15] btrfs-progs: scrub: Introduce function to recover data parity Gu Jinxiang
2017-07-15  9:10 ` [PATCH v5 12/15] btrfs-progs: scrub: Introduce helper to write a full stripe Gu Jinxiang
2017-07-15  9:10 ` [PATCH v5 13/15] btrfs-progs: scrub: Introduce a function to scrub one " Gu Jinxiang
2017-07-15  9:10 ` [PATCH v5 14/15] btrfs-progs: scrub: Introduce function to check a whole block group Gu Jinxiang
2017-07-15  9:10 ` [PATCH v5 15/15] btrfs-progs: scrub: Introduce offline scrub function Gu Jinxiang
2017-07-15  9:20 ` [PATCH v5 01/15] btrfs-progs: Introduce new btrfs_map_block function which returns more unified result Qu Wenruo
2017-07-18  6:33 ` [PATCH 00/15] Btrfs-progs offline scrub Gu Jinxiang
2017-07-19 16:45   ` Marco Lorenzo Crociani
2017-07-20  3:39     ` Qu Wenruo
2017-07-20  8:55       ` Marco Lorenzo Crociani
2017-07-20  9:10         ` Qu Wenruo
2017-07-20  9:17           ` Qu Wenruo
2017-07-20  9:40             ` Marco Lorenzo Crociani
2017-07-20  9:51               ` Qu Wenruo
2017-08-22  9:45   ` Gu, Jinxiang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170715091053.19725-3-gujx@cn.fujitsu.com \
    --to=gujx@cn.fujitsu.com \
    --cc=linux-btrfs@vger.kernel.org \
    --cc=quwenruo@cn.fujitsu.com \
    --cc=suy.fnst@cn.fujitsu.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).