[PATCH 0/4] New 'btrfs chunk list' command

[PATCH 0/4] New 'btrfs chunk list' command

[Goffredo Baroncelli]

This is a revamp of a my previous patches set[1]. After more than
year of attempts these patches were never merged, so I tried to
simplify them and to change a bit the focus. The previous patches set
had the focus to the disk usage.
The aim of these patches now is to show the chunks distribution
among the disks. So a new command 'btrfs chunk list' is added:

$ sudo ./btrfs chunk list /mnt/btrfs1/
Data,RAID6: Size:3.00GiB, Used:1.02MiB
   /dev/vdb	   1.00GiB
   /dev/vdd	   1.00GiB
   /dev/vde	   1.00GiB
   /dev/vdf	   1.00GiB
   /dev/vdg	   1.00GiB

Metadata,RAID5: Size:1.00GiB, Used:112.00KiB
   /dev/vdb	 256.00MiB
   /dev/vdd	 256.00MiB
   /dev/vde	 256.00MiB
   /dev/vdf	 256.00MiB
   /dev/vdg	 256.00MiB

System,RAID1: Size:32.00MiB, Used:16.00KiB
   /dev/vde	  32.00MiB
   /dev/vdg	  32.00MiB

Unallocated:
   /dev/vdb	  48.75GiB
   /dev/vdd	  48.75GiB
   /dev/vde	  48.72GiB
   /dev/vdf	  48.75GiB
   /dev/vdg	  48.72GiB


This command is smart enough to highlight that a disk is not preset
(this happens when a mount -o degraded is used). In case 
/dev/vdb disappears:

$ sudo ./btrfs chunk list /mnt/btrfs1/
Data,RAID6: Size:3.00GiB, Used:1.02MiB
   /dev/vdb	   1.00GiB
   /dev/vde	   1.00GiB
   /dev/vdf	   1.00GiB
   /dev/vdg	   1.00GiB
   [Missing: /dev/vdd]	   1.00GiB

Metadata,RAID5: Size:1.00GiB, Used:112.00KiB
   /dev/vdb	 256.00MiB
   /dev/vde	 256.00MiB
   /dev/vdf	 256.00MiB
   /dev/vdg	 256.00MiB
   [Missing: /dev/vdd]	 256.00MiB

System,RAID1: Size:32.00MiB, Used:16.00KiB
   /dev/vde	  32.00MiB
   /dev/vdg	  32.00MiB

Unallocated:
   /dev/vdb	  48.75GiB
   /dev/vde	  48.72GiB
   /dev/vdf	  48.75GiB
   /dev/vdg	  48.72GiB
   [Missing: /dev/vdd]	  48.75GiB


It is interesting to see what happens after a 
'btrfs balance':

$ sudo ./btrfs chunk list /mnt/btrfs1/
Data,RAID6: Size:2.00GiB, Used:792.00KiB
   /dev/vdb	   1.00GiB
   /dev/vde	   1.00GiB
   /dev/vdf	   1.00GiB
   /dev/vdg	   1.00GiB

Metadata,RAID5: Size:1.03GiB, Used:112.00KiB
   /dev/vdb	 352.00MiB
   /dev/vde	 352.00MiB
   /dev/vdf	 352.00MiB
   /dev/vdg	 352.00MiB

System,RAID1: Size:32.00MiB, Used:16.00KiB
   /dev/vdb	  32.00MiB
   /dev/vdf	  32.00MiB

Unallocated:
   /dev/vdb	  48.63GiB
   /dev/vde	  48.66GiB
   /dev/vdf	  48.63GiB
   /dev/vdg	  48.66GiB
   [Missing: /dev/vdd]	  50.00GiB

And what happens after a 'btrfs device delete missing':

$ sudo ./btrfs chunk list /mnt/btrfs1/
Data,RAID6: Size:2.00GiB, Used:792.00KiB
   /dev/vdb	   1.00GiB
   /dev/vde	   1.00GiB
   /dev/vdf	   1.00GiB
   /dev/vdg	   1.00GiB

Metadata,RAID5: Size:1.03GiB, Used:112.00KiB
   /dev/vdb	 352.00MiB
   /dev/vde	 352.00MiB
   /dev/vdf	 352.00MiB
   /dev/vdg	 352.00MiB

System,RAID1: Size:32.00MiB, Used:16.00KiB
   /dev/vdb	  32.00MiB
   /dev/vdf	  32.00MiB

Unallocated:
   /dev/vdb	  48.63GiB
   /dev/vde	  48.66GiB
   /dev/vdf	  48.63GiB
   /dev/vdg	  48.66GiB

Comments are welcome.
BR
G.Baroncelli



[1] My last attempt was done in 
http://comments.gmane.org/gmane.comp.file-systems.btrfs/32647,
then David resent the patches in 
http://www.spinics.net/lists/linux-btrfs/msg33630.html


-- 
gpg @keyserver.linux.it: Goffredo Baroncelli <kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5

[PATCH 1/4] Move group_type_str() and group_profile_str().

[Goffredo Baroncelli]

Move group_type_str() and group_profile_str() functions to the
util.c file, because these are now used also by the command
'btrfs chunk list'.

Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
---
 cmds-filesystem.c | 43 -------------------------------------------
 utils.c           | 43 +++++++++++++++++++++++++++++++++++++++++++
 utils.h           |  3 +++
 3 files changed, 46 insertions(+), 43 deletions(-)

diff --git a/cmds-filesystem.c b/cmds-filesystem.c
index cd6b3c6..269e758 100644
--- a/cmds-filesystem.c
+++ b/cmds-filesystem.c
@@ -136,49 +136,6 @@ static const char * const cmd_df_usage[] = {
        NULL
 };
 
-static char *group_type_str(u64 flag)
-{
-	u64 mask = BTRFS_BLOCK_GROUP_TYPE_MASK |
-		BTRFS_SPACE_INFO_GLOBAL_RSV;
-
-	switch (flag & mask) {
-	case BTRFS_BLOCK_GROUP_DATA:
-		return "Data";
-	case BTRFS_BLOCK_GROUP_SYSTEM:
-		return "System";
-	case BTRFS_BLOCK_GROUP_METADATA:
-		return "Metadata";
-	case BTRFS_BLOCK_GROUP_DATA|BTRFS_BLOCK_GROUP_METADATA:
-		return "Data+Metadata";
-	case BTRFS_SPACE_INFO_GLOBAL_RSV:
-		return "GlobalReserve";
-	default:
-		return "unknown";
-	}
-}
-
-static char *group_profile_str(u64 flag)
-{
-	switch (flag & BTRFS_BLOCK_GROUP_PROFILE_MASK) {
-	case 0:
-		return "single";
-	case BTRFS_BLOCK_GROUP_RAID0:
-		return "RAID0";
-	case BTRFS_BLOCK_GROUP_RAID1:
-		return "RAID1";
-	case BTRFS_BLOCK_GROUP_RAID5:
-		return "RAID5";
-	case BTRFS_BLOCK_GROUP_RAID6:
-		return "RAID6";
-	case BTRFS_BLOCK_GROUP_DUP:
-		return "DUP";
-	case BTRFS_BLOCK_GROUP_RAID10:
-		return "RAID10";
-	default:
-		return "unknown";
-	}
-}
-
 static int get_df(int fd, struct btrfs_ioctl_space_args **sargs_ret)
 {
 	u64 count = 0;
diff --git a/utils.c b/utils.c
index 2a92416..c9b9e0e 100644
--- a/utils.c
+++ b/utils.c
@@ -2450,3 +2450,46 @@ int find_next_key(struct btrfs_path *path, struct btrfs_key *key)
 	}
 	return 1;
 }
+
+char *group_type_str(u64 flag)
+{
+	u64 mask = BTRFS_BLOCK_GROUP_TYPE_MASK |
+		BTRFS_SPACE_INFO_GLOBAL_RSV;
+
+	switch (flag & mask) {
+	case BTRFS_BLOCK_GROUP_DATA:
+		return "Data";
+	case BTRFS_BLOCK_GROUP_SYSTEM:
+		return "System";
+	case BTRFS_BLOCK_GROUP_METADATA:
+		return "Metadata";
+	case BTRFS_BLOCK_GROUP_DATA|BTRFS_BLOCK_GROUP_METADATA:
+		return "Data+Metadata";
+	case BTRFS_SPACE_INFO_GLOBAL_RSV:
+		return "GlobalReserve";
+	default:
+		return "unknown";
+	}
+}
+
+char *group_profile_str(u64 flag)
+{
+	switch (flag & BTRFS_BLOCK_GROUP_PROFILE_MASK) {
+	case 0:
+		return "single";
+	case BTRFS_BLOCK_GROUP_RAID0:
+		return "RAID0";
+	case BTRFS_BLOCK_GROUP_RAID1:
+		return "RAID1";
+	case BTRFS_BLOCK_GROUP_RAID5:
+		return "RAID5";
+	case BTRFS_BLOCK_GROUP_RAID6:
+		return "RAID6";
+	case BTRFS_BLOCK_GROUP_DUP:
+		return "DUP";
+	case BTRFS_BLOCK_GROUP_RAID10:
+		return "RAID10";
+	default:
+		return "unknown";
+	}
+}
diff --git a/utils.h b/utils.h
index 289e86b..21e3e83 100644
--- a/utils.h
+++ b/utils.h
@@ -161,4 +161,7 @@ static inline u64 btrfs_min_dev_size(u32 leafsize)
 
 int find_next_key(struct btrfs_path *path, struct btrfs_key *key);
 
+char *group_type_str(u64 flag);
+char *group_profile_str(u64 flag);
+
 #endif
-- 
2.1.3

[PATCH 2/4] Add the code for the btrfs chunk list command

[Goffredo Baroncelli]

Add the code for the btrfs chunk list command. The code iterates
through the chunk, grouping these by chunk type (data, metadata,
system) and chunk profiles (linear, dup, raid1, radi5, raid10,
raid6..); then it displays the devices belong each group.
If a device is missing, it is marked as 'Missing'.

Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
---
 cmds-chunk.c | 699 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 699 insertions(+)
 create mode 100644 cmds-chunk.c

diff --git a/cmds-chunk.c b/cmds-chunk.c
new file mode 100644
index 0000000..3afa2b1
--- /dev/null
+++ b/cmds-chunk.c
@@ -0,0 +1,699 @@
+/*
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include <sys/ioctl.h>
+#include <errno.h>
+#include <stdarg.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <linux/fs.h>
+#include <dirent.h>
+#include <linux/limits.h>
+#include <uuid/uuid.h>
+
+#include "utils.h"
+#include "kerncompat.h"
+#include "ctree.h"
+
+#include "commands.h"
+
+#include "version.h"
+
+/*
+ * To store the size information about the chunks:
+ * the chunks info are grouped by the tuple (type, devid, num_stripes),
+ * i.e. if two chunks are of the same type (RAID1, DUP...), are on the
+ * same disk, have the same stripes then their sizes are grouped
+ */
+struct chunk_info {
+	u64	type;
+	u64	size;
+	u64	devid;
+	u64	num_stripes;
+};
+
+/* to store information about the disks */
+struct disk_info {
+	u64	devid;
+	char	path[BTRFS_DEVICE_PATH_NAME_MAX];
+	u64	size;
+};
+
+/*
+ * Add the chunk info to the chunk_info list
+ */
+static int add_info_to_list(struct chunk_info **info_ptr,
+			int *info_count,
+			struct btrfs_chunk *chunk)
+{
+
+	u64 type = btrfs_stack_chunk_type(chunk);
+	u64 size = btrfs_stack_chunk_length(chunk);
+	int num_stripes = btrfs_stack_chunk_num_stripes(chunk);
+	int j;
+
+	for (j = 0 ; j < num_stripes ; j++) {
+		int i;
+		struct chunk_info *p = 0;
+		struct btrfs_stripe *stripe;
+		u64    devid;
+
+		stripe = btrfs_stripe_nr(chunk, j);
+		devid = btrfs_stack_stripe_devid(stripe);
+
+		for (i = 0 ; i < *info_count ; i++)
+			if ((*info_ptr)[i].type == type &&
+			    (*info_ptr)[i].devid == devid &&
+			    (*info_ptr)[i].num_stripes == num_stripes ) {
+				p = (*info_ptr) + i;
+				break;
+			}
+
+		if (!p) {
+			int size = sizeof(struct btrfs_chunk) * (*info_count+1);
+			struct chunk_info *res = realloc(*info_ptr, size);
+
+			if (!res) {
+				fprintf(stderr, "ERROR: not enough memory\n");
+				return -1;
+			}
+
+			*info_ptr = res;
+			p = res + *info_count;
+			(*info_count)++;
+
+			p->devid = devid;
+			p->type = type;
+			p->size = 0;
+			p->num_stripes = num_stripes;
+		}
+
+		p->size += size;
+
+	}
+
+	return 0;
+
+}
+
+/*
+ *  Helper to sort the chunk type
+ */
+static int cmp_chunk_block_group(u64 f1, u64 f2)
+{
+
+	u64 mask;
+
+	if ((f1 & BTRFS_BLOCK_GROUP_TYPE_MASK) ==
+		(f2 & BTRFS_BLOCK_GROUP_TYPE_MASK))
+			mask = BTRFS_BLOCK_GROUP_PROFILE_MASK;
+	else if (f2 & BTRFS_BLOCK_GROUP_SYSTEM)
+			return -1;
+	else if (f1 & BTRFS_BLOCK_GROUP_SYSTEM)
+			return +1;
+	else
+			mask = BTRFS_BLOCK_GROUP_TYPE_MASK;
+
+	if ((f1 & mask) > (f2 & mask))
+		return +1;
+	else if ((f1 & mask) < (f2 & mask))
+		return -1;
+	else
+		return 0;
+}
+
+/*
+ * Helper to sort the chunk
+ */
+static int cmp_chunk_info(const void *a, const void *b)
+{
+	return cmp_chunk_block_group(
+		((struct chunk_info *)a)->type,
+		((struct chunk_info *)b)->type);
+}
+
+/*
+ * This function load all the chunk info from the 'fd' filesystem
+ */
+static int load_chunk_info(int fd,
+			  struct chunk_info **info_ptr,
+			  int *info_count)
+{
+
+	int ret;
+	struct btrfs_ioctl_search_args args;
+	struct btrfs_ioctl_search_key *sk = &args.key;
+	struct btrfs_ioctl_search_header *sh;
+	unsigned long off = 0;
+	int i, e;
+
+	memset(&args, 0, sizeof(args));
+
+	/*
+	 * there may be more than one ROOT_ITEM key if there are
+	 * snapshots pending deletion, we have to loop through
+	 * them.
+	 */
+
+
+	sk->tree_id = BTRFS_CHUNK_TREE_OBJECTID;
+
+	sk->min_objectid = 0;
+	sk->max_objectid = (u64)-1;
+	sk->max_type = 0;
+	sk->min_type = (u8)-1;
+	sk->min_offset = 0;
+	sk->max_offset = (u64)-1;
+	sk->min_transid = 0;
+	sk->max_transid = (u64)-1;
+	sk->nr_items = 4096;
+
+	while (1) {
+
+		ret = ioctl(fd, BTRFS_IOC_TREE_SEARCH, &args);
+		e = errno;
+		if (ret < 0) {
+			fprintf(stderr,
+				"ERROR: can't perform the search - %s\n",
+				strerror(e));
+			return -99;
+		}
+		/* the ioctl returns the number of item it found in nr_items */
+
+		if (sk->nr_items == 0)
+			break;
+
+		off = 0;
+		for (i = 0; i < sk->nr_items; i++) {
+			struct btrfs_chunk *item;
+			sh = (struct btrfs_ioctl_search_header *)(args.buf +
+								  off);
+
+			off += sizeof(*sh);
+			item = (struct btrfs_chunk *)(args.buf + off);
+
+			if (add_info_to_list(info_ptr, info_count, item)) {
+				*info_ptr = 0;
+				free(*info_ptr);
+				return -100;
+			}
+
+			off += sh->len;
+
+			sk->min_objectid = sh->objectid;
+			sk->min_type = sh->type;
+			sk->min_offset = sh->offset+1;
+
+		}
+		if (!sk->min_offset)	/* overflow */
+			sk->min_type++;
+		else
+			continue;
+
+		if (!sk->min_type)
+			sk->min_objectid++;
+		 else
+			continue;
+
+		if (!sk->min_objectid)
+			break;
+	}
+
+	qsort(*info_ptr, *info_count, sizeof(struct chunk_info),
+		cmp_chunk_info);
+
+	return 0;
+
+}
+
+/*
+ * Helper to sort the struct btrfs_ioctl_space_info
+ */
+static int cmp_btrfs_ioctl_space_info(const void *a, const void *b)
+{
+	return cmp_chunk_block_group(
+		((struct btrfs_ioctl_space_info *)a)->flags,
+		((struct btrfs_ioctl_space_info *)b)->flags);
+}
+
+/*
+ * This function load all the information about the space usage
+ */
+static struct btrfs_ioctl_space_args *load_space_info(int fd, const char *path)
+{
+	struct btrfs_ioctl_space_args *sargs = 0, *sargs_orig = 0;
+	int e, ret, count;
+
+	sargs_orig = sargs = malloc(sizeof(struct btrfs_ioctl_space_args));
+	if (!sargs) {
+		fprintf(stderr, "ERROR: not enough memory\n");
+		return NULL;
+	}
+
+	sargs->space_slots = 0;
+	sargs->total_spaces = 0;
+
+	ret = ioctl(fd, BTRFS_IOC_SPACE_INFO, sargs);
+	e = errno;
+	if (ret) {
+		fprintf(stderr,
+			"ERROR: couldn't get space info on '%s' - %s\n",
+			path, strerror(e));
+		free(sargs);
+		return NULL;
+	}
+	if (!sargs->total_spaces) {
+		free(sargs);
+		printf("No chunks found\n");
+		return NULL;
+	}
+
+	count = sargs->total_spaces;
+
+	sargs = realloc(sargs, sizeof(struct btrfs_ioctl_space_args) +
+			(count * sizeof(struct btrfs_ioctl_space_info)));
+	if (!sargs) {
+		free(sargs_orig);
+		fprintf(stderr, "ERROR: not enough memory\n");
+		return NULL;
+	}
+
+	sargs->space_slots = count;
+	sargs->total_spaces = 0;
+
+	ret = ioctl(fd, BTRFS_IOC_SPACE_INFO, sargs);
+	e = errno;
+
+	if (ret) {
+		fprintf(stderr,
+			"ERROR: couldn't get space info on '%s' - %s\n",
+			path, strerror(e));
+		free(sargs);
+		return NULL;
+	}
+
+	qsort(&(sargs->spaces), count, sizeof(struct btrfs_ioctl_space_info),
+		cmp_btrfs_ioctl_space_info);
+
+	return sargs;
+}
+
+/*
+ *  Helper to sort the disk_info structure
+ */
+static int cmp_disk_info(const void *a, const void *b)
+{
+	return strcmp(((struct disk_info *)a)->path,
+			((struct disk_info *)b)->path);
+}
+
+
+/*
+ *	check if the device 'devpath' is used by the btrfs filesystem
+ *	fsid; check devpath against /fs/btrfs/<uuid>/<dev>/dev by
+ *	major, minor
+ */
+static int is_device_used(char *devpath, u8 *fsid) {
+	char fsid_c[BTRFS_UUID_UNPARSED_SIZE];
+	char dirname[PATH_MAX];
+	struct stat devstat;
+	int dev_major, dev_minor;
+	DIR *dir;
+
+
+
+	if (stat(devpath, &devstat)<0)
+		return 0;
+
+	dev_major = major(devstat.st_rdev);
+	dev_minor = minor(devstat.st_rdev);
+
+	uuid_unparse(fsid, fsid_c);
+	snprintf(dirname, PATH_MAX-1,
+			"/sys/fs/btrfs/%s/devices",
+			fsid_c);
+
+	/*
+	 *	if /sys/btrfs doesn't exist; return 0
+	 */
+	if ((dir=opendir(dirname))==NULL)
+		return 0;
+
+	for(;;) {
+		char path[PATH_MAX];
+		struct dirent *direntry;
+		char buf[100];
+		FILE *fp;
+		int mj, mn;
+		char *r;
+
+		direntry = readdir(dir);
+
+		/* entry not found */
+		if (!direntry) {
+			closedir(dir);
+			return 0;
+		}
+
+		if (!strcmp(direntry->d_name, "."))
+			continue;
+		if (!strcmp(direntry->d_name, ".."))
+			continue;
+
+		snprintf(path, PATH_MAX-1,
+			"%s/%s/dev",
+			dirname, direntry->d_name);
+
+		fp = fopen(path, "r");
+		if (!fp)
+			continue;
+
+		r = fgets(buf, 99, fp);
+		fclose(fp);
+		if (!r)
+			continue;
+
+		if (sscanf(buf, "%d:%d", &mj, &mn) != 2)
+			continue;
+
+		if (mj == dev_major && mn == dev_minor) {
+			closedir(dir);
+			return 1;
+		}
+
+	}
+
+	/* we don't reach here */
+	BUG_ON(1);
+}
+
+/*
+ *  This function load the disk_info structure and put them in an array
+ */
+static int load_disks_info(int fd,
+			   struct disk_info **disks_info_ptr,
+			   int *disks_info_count)
+{
+
+	int ret, i, ndevs;
+	struct btrfs_ioctl_fs_info_args fi_args;
+	struct btrfs_ioctl_dev_info_args dev_info;
+	struct disk_info *info;
+
+	*disks_info_count = 0;
+	*disks_info_ptr = 0;
+
+	ret = ioctl(fd, BTRFS_IOC_FS_INFO, &fi_args);
+	if (ret < 0) {
+		fprintf(stderr, "ERROR: cannot get filesystem info\n");
+		return -1;
+	}
+
+	info = malloc(sizeof(struct disk_info) * fi_args.num_devices);
+	if (!info) {
+		fprintf(stderr, "ERROR: not enough memory\n");
+		return -1;
+	}
+
+	for (i = 0, ndevs = 0 ; i <= fi_args.max_id ; i++) {
+
+		BUG_ON(ndevs >= fi_args.num_devices);
+		ret = get_device_info(fd, i, &dev_info);
+
+		if (ret == -ENODEV)
+			continue;
+		if (ret) {
+			fprintf(stderr,
+			    "ERROR: cannot get info about device devid=%d\n",
+			    i);
+			free(info);
+			return -1;
+		}
+
+		info[ndevs].devid = dev_info.devid;
+		if(is_device_used((char*)dev_info.path, fi_args.fsid)) {
+			strcpy(info[ndevs].path, (char *)dev_info.path);
+		} else {
+			snprintf(info[ndevs].path,
+				BTRFS_DEVICE_PATH_NAME_MAX-1,
+				"[Missing: %s]",
+				(char *)dev_info.path);
+		}
+
+		info[ndevs].size = dev_info.total_bytes;
+		++ndevs;
+	}
+
+	BUG_ON(ndevs != fi_args.num_devices);
+	qsort(info, fi_args.num_devices,
+		sizeof(struct disk_info), cmp_disk_info);
+
+	*disks_info_count = fi_args.num_devices;
+	*disks_info_ptr = info;
+
+	return 0;
+
+}
+
+/*
+ *  This function computes the size of a chunk in a disk
+ */
+static u64 calc_chunk_size(struct chunk_info *ci)
+{
+	if (ci->type & BTRFS_BLOCK_GROUP_RAID0)
+		return ci->size / ci->num_stripes;
+	else if (ci->type & BTRFS_BLOCK_GROUP_RAID1)
+		return ci->size ;
+	else if (ci->type & BTRFS_BLOCK_GROUP_DUP)
+		return ci->size ;
+	else if (ci->type & BTRFS_BLOCK_GROUP_RAID5)
+		return ci->size / (ci->num_stripes -1);
+	else if (ci->type & BTRFS_BLOCK_GROUP_RAID6)
+		return ci->size / (ci->num_stripes -2);
+	else if (ci->type & BTRFS_BLOCK_GROUP_RAID10)
+		return ci->size / ci->num_stripes;
+	return ci->size;
+}
+
+/*
+ *  This function prints the unused space per every disk
+ */
+static void print_unused(struct chunk_info *info_ptr,
+			  int info_count,
+			  struct disk_info *disks_info_ptr,
+			  int disks_info_count,
+			  int mode)
+{
+	int i;
+	for (i = 0 ; i < disks_info_count ; i++) {
+
+		int	j;
+		u64	total = 0;
+
+		for (j = 0 ; j < info_count ; j++)
+			if (info_ptr[j].devid == disks_info_ptr[i].devid)
+				total += calc_chunk_size(info_ptr+j);
+
+		printf("   %s\t%10s\n",
+			disks_info_ptr[i].path,
+			pretty_size_mode(disks_info_ptr[i].size-total, mode));
+
+	}
+
+}
+
+/*
+ *  This function prints the allocated chunk per every disk
+ */
+static void print_chunk_disks(u64 chunk_type,
+				struct chunk_info *chunks_info_ptr,
+				int chunks_info_count,
+				struct disk_info *disks_info_ptr,
+				int disks_info_count,
+				int mode)
+{
+	int i;
+
+	for (i = 0 ; i < disks_info_count ; i++) {
+
+		int	j;
+		u64	total = 0;
+
+		for (j = 0 ; j < chunks_info_count ; j++) {
+
+			if (chunks_info_ptr[j].type != chunk_type)
+				continue;
+			if (chunks_info_ptr[j].devid != disks_info_ptr[i].devid)
+				continue;
+
+			total += calc_chunk_size(&(chunks_info_ptr[j]));
+		}
+
+		if (total > 0) {
+			printf("   %s\t%10s\n",
+				disks_info_ptr[i].path,
+				pretty_size_mode(total, mode));
+		}
+	}
+}
+
+/*
+ *  This function print the results of the command btrfs fi disk-usage
+ *  in linear format
+ */
+static void write_chunks_info(int mode,
+					struct btrfs_ioctl_space_args *sargs,
+					struct chunk_info *info_ptr,
+					int info_count,
+					struct disk_info *disks_info_ptr,
+					int disks_info_count)
+{
+	int i;
+
+	for (i = 0; i < sargs->total_spaces; i++) {
+		int j = 0;
+		for (j=0 ; j < info_count ; j++)
+			if (info_ptr[j].type == sargs->spaces[i].flags)
+				break;
+
+		/* skip spaces without chunk */
+		if(j == info_count)
+			continue;
+
+		printf("%s,%s: Size:%s, Used:%s\n",
+			group_type_str(sargs->spaces[i].flags),
+			group_profile_str(sargs->spaces[i].flags),
+			pretty_size_mode(sargs->spaces[i].total_bytes ,
+			    mode),
+			pretty_size_mode(sargs->spaces[i].used_bytes,
+					mode));
+
+		print_chunk_disks(sargs->spaces[i].flags, info_ptr,
+				info_count, disks_info_ptr,
+				disks_info_count, mode);
+		printf("\n");
+
+	}
+
+	printf("Unallocated:\n");
+	print_unused(info_ptr, info_count,
+			disks_info_ptr, disks_info_count,
+			mode);
+
+
+
+}
+
+static int _dump_chunks(int fd, const char *path, int mode)
+{
+	struct btrfs_ioctl_space_args *sargs = 0;
+	int info_count = 0;
+	struct chunk_info *info_ptr = 0;
+	struct disk_info *disks_info_ptr = 0;
+	int disks_info_count = 0;
+	int ret = 0;
+
+	if (load_chunk_info(fd, &info_ptr, &info_count) ||
+	    load_disks_info(fd, &disks_info_ptr, &disks_info_count)) {
+		ret = -1;
+		goto exit;
+	}
+
+	if ((sargs = load_space_info(fd, path)) == NULL) {
+		ret = -1;
+		goto exit;
+	}
+
+	write_chunks_info(mode, sargs,
+				info_ptr, info_count,
+				disks_info_ptr, disks_info_count);
+
+exit:
+
+	if (sargs)
+		free(sargs);
+	if (disks_info_ptr)
+		free(disks_info_ptr);
+	if (info_ptr)
+		free(info_ptr);
+
+	return ret;
+}
+
+static const char * const chunk_cmd_group_usage[] = {
+	"btrfs chunk list <filesystem>",
+	NULL
+};
+
+static const char * const cmd_chunk_list_usage[] = {
+	"btrfs chunk list <filesystem>",
+	"Shows the filesystem chunks.",
+	NULL
+};
+
+static int dump_chunks(const char * path, int mode) {
+
+	int fd, r;
+	DIR *dirp;
+
+	fd = open_file_or_dir(path, &dirp);
+	if (fd < 0) {
+		fprintf(stderr, "ERROR: can't access to '%s'\n", path);
+		return 12;
+	}
+	r = _dump_chunks(fd, path, UNITS_BINARY);
+	close(fd);
+	if (r < 0) {
+		int e = errno;
+		fprintf(stderr, "ERROR: errno = %d, %s\n",
+			e, strerror(e));
+		return 12;
+	}
+
+	return r;
+}
+
+static int cmd_chunk_list(int argc, char **argv)
+{
+	int ret;
+	char *path = NULL;
+
+	if (check_argc_exact(argc, 2))
+		usage(cmd_chunk_list_usage);
+
+	path = argv[1];
+
+	ret = dump_chunks(path, 0);
+	return ret;
+
+}
+
+const struct cmd_group chunk_cmd_group = {
+	chunk_cmd_group_usage, NULL, {
+		{ "list", cmd_chunk_list, cmd_chunk_list_usage, NULL, 0 },
+		NULL_CMD_STRUCT
+	}
+};
+
+int cmd_chunk(int argc, char **argv)
+{
+	return handle_command_group(&chunk_cmd_group, argc, argv);
+}
-- 
2.1.3

[PATCH 3/4] Add the btrfs chunk list command

[Goffredo Baroncelli]

This patch adds the 'btrfs chunk' groups command.

Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
---
 Makefile   | 2 +-
 btrfs.c    | 1 +
 commands.h | 2 ++
 3 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index 4cae30c..1744f9c 100644
--- a/Makefile
+++ b/Makefile
@@ -15,7 +15,7 @@ cmds_objects = cmds-subvolume.o cmds-filesystem.o cmds-device.o cmds-scrub.o \
 	       cmds-inspect.o cmds-balance.o cmds-send.o cmds-receive.o \
 	       cmds-quota.o cmds-qgroup.o cmds-replace.o cmds-check.o \
 	       cmds-restore.o cmds-rescue.o chunk-recover.o super-recover.o \
-	       cmds-property.o
+	       cmds-property.o cmds-chunk.o
 libbtrfs_objects = send-stream.o send-utils.o rbtree.o btrfs-list.o crc32c.o \
 		   uuid-tree.o utils-lib.o rbtree-utils.o
 libbtrfs_headers = send-stream.h send-utils.h send.h rbtree.h btrfs-list.h \
diff --git a/btrfs.c b/btrfs.c
index e83349c..25fbb71 100644
--- a/btrfs.c
+++ b/btrfs.c
@@ -197,6 +197,7 @@ static const struct cmd_group btrfs_cmd_group = {
 		{ "device", cmd_device, NULL, &device_cmd_group, 0 },
 		{ "scrub", cmd_scrub, NULL, &scrub_cmd_group, 0 },
 		{ "check", cmd_check, cmd_check_usage, NULL, 0 },
+		{ "chunk", cmd_chunk, NULL, &chunk_cmd_group, 0 },
 		{ "rescue", cmd_rescue, NULL, &rescue_cmd_group, 0 },
 		{ "restore", cmd_restore, cmd_restore_usage, NULL, 0 },
 		{ "inspect-internal", cmd_inspect, NULL, &inspect_cmd_group, 0 },
diff --git a/commands.h b/commands.h
index 4d870f6..41d69b0 100644
--- a/commands.h
+++ b/commands.h
@@ -80,6 +80,7 @@ extern const struct cmd_group filesystem_cmd_group;
 extern const struct cmd_group balance_cmd_group;
 extern const struct cmd_group device_cmd_group;
 extern const struct cmd_group scrub_cmd_group;
+extern const struct cmd_group chunk_cmd_group;
 extern const struct cmd_group inspect_cmd_group;
 extern const struct cmd_group property_cmd_group;
 extern const struct cmd_group quota_cmd_group;
@@ -101,6 +102,7 @@ int cmd_balance(int argc, char **argv);
 int cmd_device(int argc, char **argv);
 int cmd_scrub(int argc, char **argv);
 int cmd_check(int argc, char **argv);
+int cmd_chunk(int argc, char **argv);
 int cmd_chunk_recover(int argc, char **argv);
 int cmd_super_recover(int argc, char **argv);
 int cmd_inspect(int argc, char **argv);
-- 
2.1.3

[PATCH 4/4] Add man page for the 'btrfs chunk' family commands.

[Goffredo Baroncelli]

Add btrfs-chunk(8) man page, and update btrfs(8) man page.

Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
---
 Documentation/Makefile        |  1 +
 Documentation/btrfs-chunk.txt | 58 +++++++++++++++++++++++++++++++++++++++++++
 Documentation/btrfs.txt       |  5 ++++
 3 files changed, 64 insertions(+)
 create mode 100644 Documentation/btrfs-chunk.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index ef4f1bd..f47f62b 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -18,6 +18,7 @@ MAN8_TXT += mkfs.btrfs.txt
 MAN8_TXT += btrfs-subvolume.txt
 MAN8_TXT += btrfs-filesystem.txt
 MAN8_TXT += btrfs-balance.txt
+MAN8_TXT += btrfs-chunk.txt
 MAN8_TXT += btrfs-device.txt
 MAN8_TXT += btrfs-scrub.txt
 MAN8_TXT += btrfs-check.txt
diff --git a/Documentation/btrfs-chunk.txt b/Documentation/btrfs-chunk.txt
new file mode 100644
index 0000000..85d1ea6
--- /dev/null
+++ b/Documentation/btrfs-chunk.txt
@@ -0,0 +1,58 @@
+btrfs-chunk(8)
+===============
+
+NAME
+----
+btrfs-chunk - control btrfs chunks
+
+SYNOPSIS
+--------
+*btrfs chunk* <subcommand> <args>
+
+DESCRIPTION
+-----------
+*btrfs chunk* is used to control the btrfs chunks.
+
+CHUNK DESCRIPTION
+-----------------
+Block devices are divided into chunks. Chunks may be
+mirrored or striped across multiple devices. The mirroring/striping
+arrangement is transparent to the rest of the filesystem, which simply
+sees the single, logical address space that chunks are mapped into.
+
+There are three main types of chunks: Data, Metadata and System. Sometime
+(when the device is small) Data and Metadata can be grouped togheter; in
+this case the chunk is called 'Data+Metadata'. Usually the Data chunks
+are used to store the file content; but if the data is small enough, this
+may be stored in the Metadata chunk togheter with the filesystem data
+structures.
+
+The striping/mirroring levels (called profiles) may be different
+among the chunk types. The chunk profiles are assigned during the filesystem
+creation and these can be changed by the 'btrfs balance' commands.
+
+See `mkfs.btrfs`(8) and `btrfs-balance`(8) for more details.
+
+SUBCOMMAND
+----------
+*list*  <path>::
+Shows the chunks grouped by type and profile; after each group
+the command lists the devices which host the chunks.
+If a device is not present (i.e. the filesystem is mounted in 'degraded' mode),
+it is marked as 'Missing:'.
+
+EXIT STATUS
+-----------
+*btrf chunk* returns a zero exit status if it succeeds. Non zero is
+returned in case of failure.
+
+AVAILABILITY
+------------
+*btrfs* is part of btrfs-progs.
+Please refer to the btrfs wiki http://btrfs.wiki.kernel.org for
+further details.
+
+SEE ALSO
+--------
+`mkfs.btrfs`(8),
+`btrfs-balance`(8)
diff --git a/Documentation/btrfs.txt b/Documentation/btrfs.txt
index 3bdc6b4..fc02004 100644
--- a/Documentation/btrfs.txt
+++ b/Documentation/btrfs.txt
@@ -41,6 +41,10 @@ COMMANDS
 	Balance btrfs filesystem chunks across single or several devices. +
 	See `btrfs-balance`(8) for details.
 
+*chunk*::
+	Manage filesystem chunk; currently only listing is suported. +
+	See `btrfs-chunk`(8) for details.
+
 *device*::
 	Manage devices managed by btrfs, including add/delete/scan and so
 	on. +
@@ -102,6 +106,7 @@ SEE ALSO
 `btrfs-subvolume`(8),
 `btrfs-filesystem`(8),
 `btrfs-balance`(8),
+`btrfs-chunk`(8),
 `btrfs-device`(8),
 `btrfs-scrub`(8),
 `btrfs-check`(8),
-- 
2.1.3

Re: [PATCH 0/4] New 'btrfs chunk list' command

[Martin Steigerwald]

Hi Goffredo,

Am Dienstag, 25. November 2014, 16:57:21 schrieb Goffredo Baroncelli:
> This is a revamp of a my previous patches set[1]. After more than
> year of attempts these patches were never merged, so I tried to
> simplify them and to change a bit the focus. The previous patches set
> had the focus to the disk usage.
> The aim of these patches now is to show the chunks distribution
> among the disks. So a new command 'btrfs chunk list' is added:
> 
> $ sudo ./btrfs chunk list /mnt/btrfs1/
> Data,RAID6: Size:3.00GiB, Used:1.02MiB
>    /dev/vdb	   1.00GiB
>    /dev/vdd	   1.00GiB
>    /dev/vde	   1.00GiB
>    /dev/vdf	   1.00GiB
>    /dev/vdg	   1.00GiB

I still like your previous patch set *a lot* and wonder why exactly is has 
never been merged. I know there has been quite some discussion about it, but 
in the end I lost track.

Can the chunk list also display the usage inside the chunks?

Thanks,
-- 
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7

Re: [PATCH 0/4] New 'btrfs chunk list' command

[David Sterba]

On Tue, Nov 25, 2014 at 04:57:21PM +0100, Goffredo Baroncelli wrote:
> This is a revamp of a my previous patches set[1]. After more than
> year of attempts these patches were never merged, so I tried to
> simplify them and to change a bit the focus. The previous patches set
> had the focus to the disk usage.

The patches have been pending for last releases but I did not manage to
fix all accounting problems in time and did not want to delay the
releaes due to other important fixes. The 'new df' patchset is being
refreshed each time and is part of the integration branches which might
signify that they haven't been lost.

Re: [PATCH 0/4] New 'btrfs chunk list' command

[Goffredo Baroncelli]

On 11/25/2014 05:13 PM, David Sterba wrote:
> On Tue, Nov 25, 2014 at 04:57:21PM +0100, Goffredo Baroncelli wrote:
>> This is a revamp of a my previous patches set[1]. After more than
>> year of attempts these patches were never merged, so I tried to
>> simplify them and to change a bit the focus. The previous patches set
>> had the focus to the disk usage.
> 
> The patches have been pending for last releases but I did not manage to
> fix all accounting problems in time and did not want to delay the
> releaes due to other important fixes. The 'new df' patchset is being
> refreshed each time and is part of the integration branches which might
> signify that they haven't been lost.
> 
Thanks David for care of that. Unfortunately I didn't heard anything
so I feared that this patch was lost.

I hoped that simplifying the patches set, it would be more simple to
integrate.

BR
G.Baroncelli

-- 
gpg @keyserver.linux.it: Goffredo Baroncelli <kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5

Re: [PATCH 0/4] New 'btrfs chunk list' command

[Goffredo Baroncelli]

On 11/25/2014 05:08 PM, Martin Steigerwald wrote:
> Hi Goffredo,
> 
> Am Dienstag, 25. November 2014, 16:57:21 schrieb Goffredo Baroncelli:
>> This is a revamp of a my previous patches set[1]. After more than
>> year of attempts these patches were never merged, so I tried to
>> simplify them and to change a bit the focus. The previous patches set
>> had the focus to the disk usage.
>> The aim of these patches now is to show the chunks distribution
>> among the disks. So a new command 'btrfs chunk list' is added:
>>
>> $ sudo ./btrfs chunk list /mnt/btrfs1/
>> Data,RAID6: Size:3.00GiB, Used:1.02MiB
>>    /dev/vdb	   1.00GiB
>>    /dev/vdd	   1.00GiB
>>    /dev/vde	   1.00GiB
>>    /dev/vdf	   1.00GiB
>>    /dev/vdg	   1.00GiB
> 
> I still like your previous patch set *a lot* and wonder why exactly is has 
> never been merged. I know there has been quite some discussion about it, but 
> in the end I lost track.

David told [wrote] few minutes ago that this patch is still on the queue.


> 
> Can the chunk list also display the usage inside the chunks?
Unfortunately not. I don't know how it would be possible to get this info.

> 
> Thanks,
> 


-- 
gpg @keyserver.linux.it: Goffredo Baroncelli <kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5

Re: [PATCH 0/4] New 'btrfs chunk list' command

[David Sterba]

On Tue, Nov 25, 2014 at 06:10:29PM +0100, Goffredo Baroncelli wrote:
> > Can the chunk list also display the usage inside the chunks?
> Unfortunately not. I don't know how it would be possible to get this info.

It takes some more parsing of the fs structures, doable via the
SEARCH_TREE ioctl. I have an unpolished patch to show the logical chunks
(though without actual chunk usage), similar can be done for per-device
chunks. The patch was a debugging helper so it hasn't been submitted yet
until it's more user friendly. I'll keep the per-device request in mind.