[PATCH 0/4] btrfs-progs: tune: add --device and --noscan option

[PATCH 0/4] btrfs-progs: tune: add --device and --noscan option

[Anand Jain]

Separated out from the preparatory patch set. Depends on the preparatory
patch set: ("btrfs-progs: cleanup and preparatory around device scan").
This set (along with its preparatory patch) has passed the btrfs-progs
test suite.

By default, btrfstune scans all the block devices in the system.

To scan regular files without mapping them to a loop device, add the
--device option.

The option arguments follow the same pattern as in the "mkfs.btrfs -O"
option.

To indicate not to scan the system for other devices, add the --noscan
option.

For example:

  The command below will scan both regular files and the devices
  provided in the --device option, along with the system block devices.

	btrfstune -m --device /tdev/td1,/tdev/td2 /tdev/td3
  or
	btrfstune -m --device /tdev/td1 --device /tdev/td2 /tdev/td3

  In some cases, if you need to avoid the default system scan for the
  block device, you can use the --noscan option.

	btrfstune -m --noscan --device /tdev/td1,/tdev/td2 /tdev/td3

Anand Jain (5):
  btrfs-progs: tune: consolidate return goto free-out
  btrfs-progs: tune: introduce --device option
  btrfs-progs: docs: update btrfstune --device option
  btrfs-progs: tune: introduce --noscan option
  btrfs-progs: docs: update btrfstune --noscan option

 Documentation/btrfstune.rst |   7 +++
 tune/main.c                 | 103 ++++++++++++++++++++++++++++++------
 2 files changed, 95 insertions(+), 15 deletions(-)

-- 
2.38.1

[PATCH 1/5] btrfs-progs: tune: consolidate return goto free-out

[Anand Jain]

The upcoming "--device" option requires memory to parse devices, which
should be freed before returning from the main() function. As a
preparation for adding the "--device" option to the "btrfstune" command,
provide a consolidated error return exit from the main function with a
"goto" labeled "free_out". The label "free_out" may not make sense
currently, but it will when the "--device" option is added.

There are several return statements within the main function, and
changing all of them in the main "--device" feature patch would deviate
from the actual for the feature changes. Hence, it made sense to create
a preparatory patch.

The return value for any failure remains the same as in the original code.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
 tune/main.c | 27 +++++++++++++++++----------
 1 file changed, 17 insertions(+), 10 deletions(-)

diff --git a/tune/main.c b/tune/main.c
index 0ca1e01282c9..0c8872dcdee5 100644
--- a/tune/main.c
+++ b/tune/main.c
@@ -145,7 +145,7 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 	bool to_fst = false;
 	int csum_type = -1;
 	char *new_fsid_str = NULL;
-	int ret;
+	int ret = 1;
 	u64 super_flags = 0;
 	int fd = -1;
 
@@ -233,18 +233,18 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 	set_argv0(argv);
 	device = argv[optind];
 	if (check_argc_exact(argc - optind, 1))
-		return 1;
+		goto free_out;
 
 	if (random_fsid && new_fsid_str) {
 		error("random fsid can't be used with specified fsid");
-		return 1;
+		goto free_out;
 	}
 	if (!super_flags && !seeding_flag && !(random_fsid || new_fsid_str) &&
 	    !change_metadata_uuid && csum_type == -1 && !to_bg_tree &&
 	    !to_extent_tree && !to_fst) {
 		error("at least one option should be specified");
 		usage(&tune_cmd, 1);
-		return 1;
+		goto free_out;
 	}
 
 	if (new_fsid_str) {
@@ -253,18 +253,21 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 		ret = uuid_parse(new_fsid_str, tmp);
 		if (ret < 0) {
 			error("could not parse UUID: %s", new_fsid_str);
-			return 1;
+			ret = 1;
+			goto free_out;
 		}
 		if (!test_uuid_unique(new_fsid_str)) {
 			error("fsid %s is not unique", new_fsid_str);
-			return 1;
+			ret = 1;
+			goto free_out;
 		}
 	}
 
 	fd = open(device, O_RDWR);
 	if (fd < 0) {
 		error("mount check: cannot open %s: %m", device);
-		return 1;
+		ret = 1;
+		goto free_out;
 	}
 
 	ret = check_mounted_where(fd, device, NULL, 0, NULL,
@@ -273,18 +276,21 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 		errno = -ret;
 		error("could not check mount status of %s: %m", device);
 		close(fd);
-		return 1;
+		ret = 1;
+		goto free_out;
 	} else if (ret) {
 		error("%s is mounted", device);
 		close(fd);
-		return 1;
+		ret = 1;
+		goto free_out;
 	}
 
 	root = open_ctree_fd(fd, device, 0, ctree_flags);
 
 	if (!root) {
 		error("open ctree failed");
-		return 1;
+		ret = 1;
+		goto free_out;
 	}
 
  	if (to_bg_tree) {
@@ -431,5 +437,6 @@ out:
 	close_ctree(root);
 	btrfs_close_all_devices();
 
+free_out:
 	return ret;
 }
-- 
2.38.1

[PATCH 1/1] btrfs-progs: tune: introduce --device option

[Anand Jain]

As of now, btrfstune only accepts one device from the command line and
then scans the system to find other parter devices if any.

However, this method mandates always accessing the file raw image as a
loop device.

This patch modifies btrfstune to accept other devices or reg-files from
the command line using an option --device and scans/registers them.

For example:

	btrfstune -m --device /tdev/td1,/tdev/td2 /tdev/td3
  or
	btrfstune -m --device /tdev/td1 --device /tdev/td2 /tdev/td3

Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
 tune/main.c | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 64 insertions(+), 1 deletion(-)

diff --git a/tune/main.c b/tune/main.c
index 0c8872dcdee5..c9bbada4f7b1 100644
--- a/tune/main.c
+++ b/tune/main.c
@@ -23,6 +23,7 @@
 #include <getopt.h>
 #include <errno.h>
 #include <stdbool.h>
+#include <ctype.h>
 #include <uuid/uuid.h>
 #include "kernel-shared/ctree.h"
 #include "kernel-shared/disk-io.h"
@@ -63,6 +64,37 @@ static int set_super_incompat_flags(struct btrfs_root *root, u64 flags)
 	return ret;
 }
 
+static bool array_append(char **dest, char *src, int *cnt)
+{
+	char *this_tok = strtok(src, ",");
+	int ret_cnt = *cnt;
+
+	while(this_tok != NULL) {
+		ret_cnt++;
+		dest = realloc(dest, sizeof(char *) * ret_cnt);
+		if (!dest)
+			return false;
+
+		dest[ret_cnt - 1] = strdup(this_tok);
+		*cnt = ret_cnt;
+
+		this_tok = strtok(NULL, ",");
+	}
+
+	return true;
+}
+
+static void free_array(char **prt, int cnt)
+{
+	if (!prt)
+		return;
+
+	for (int i = 0; i < cnt; i++)
+		free(prt[i]);
+
+	free(prt);
+}
+
 static int convert_to_fst(struct btrfs_fs_info *fs_info)
 {
 	int ret;
@@ -117,6 +149,7 @@ static const char * const tune_usage[] = {
 	"",
 	"General:",
 	OPTLINE("-f", "allow dangerous operations, make sure that you are aware of the dangers"),
+	OPTLINE("--device", "devices or regular-files of the filesystem to be scanned"),
 	OPTLINE("--help", "print this help"),
 #if EXPERIMENTAL
 	"",
@@ -144,6 +177,8 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 	bool to_bg_tree = false;
 	bool to_fst = false;
 	int csum_type = -1;
+	int argc_devices = 0;
+	char **argv_devices = NULL;
 	char *new_fsid_str = NULL;
 	int ret = 1;
 	u64 super_flags = 0;
@@ -155,7 +190,8 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 		enum { GETOPT_VAL_CSUM = GETOPT_VAL_FIRST,
 		       GETOPT_VAL_ENABLE_BLOCK_GROUP_TREE,
 		       GETOPT_VAL_DISABLE_BLOCK_GROUP_TREE,
-		       GETOPT_VAL_ENABLE_FREE_SPACE_TREE };
+		       GETOPT_VAL_ENABLE_FREE_SPACE_TREE,
+		       GETOPT_VAL_DEVICE };
 		static const struct option long_options[] = {
 			{ "help", no_argument, NULL, GETOPT_VAL_HELP},
 			{ "convert-to-block-group-tree", no_argument, NULL,
@@ -167,6 +203,7 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 #if EXPERIMENTAL
 			{ "csum", required_argument, NULL, GETOPT_VAL_CSUM },
 #endif
+			{ "device", required_argument, NULL, GETOPT_VAL_DEVICE },
 			{ NULL, 0, NULL, 0 }
 		};
 		int c = getopt_long(argc, argv, "S:rxfuU:nmM:", long_options, NULL);
@@ -210,6 +247,21 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 		case GETOPT_VAL_ENABLE_BLOCK_GROUP_TREE:
 			to_bg_tree = true;
 			break;
+		case GETOPT_VAL_DEVICE:
+			if (!argv_devices) {
+				argv_devices = malloc(sizeof(char *));
+				if (!argv_devices) {
+					error("memory alloc failed");
+					return 1;
+				}
+			}
+
+			if (!array_append(argv_devices, optarg,
+					  &argc_devices)) {
+				error("memory alloc failed");
+				goto free_out;
+			}
+			break;
 		case GETOPT_VAL_DISABLE_BLOCK_GROUP_TREE:
 			to_extent_tree = true;
 			break;
@@ -285,6 +337,16 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 		goto free_out;
 	}
 
+	/*
+	 * check_mounted_where() with noscan == true frees the scanned devices
+	 * scan the command line provided device list now.
+	 */
+	if (argv_devices) {
+		ret = btrfs_scan_argv_devices(0, argc_devices, argv_devices);
+		if (ret)
+			goto free_out;
+	}
+
 	root = open_ctree_fd(fd, device, 0, ctree_flags);
 
 	if (!root) {
@@ -438,5 +500,6 @@ out:
 	btrfs_close_all_devices();
 
 free_out:
+	free_array(argv_devices, argc_devices);
 	return ret;
 }
-- 
2.31.1

Re: [PATCH 1/1] btrfs-progs: tune: introduce --device option

[Anand Jain]

Pls read the patch number as 2/5. Thx.

[PATCH 3/5] btrfs-progs: docs: update btrfstune --device option

[Anand Jain]

Update the Documentation/btrsftune.rst to carry the new --device
option.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
 Documentation/btrfstune.rst | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/Documentation/btrfstune.rst b/Documentation/btrfstune.rst
index 0510ad1f4c26..89f4494bbaf0 100644
--- a/Documentation/btrfstune.rst
+++ b/Documentation/btrfstune.rst
@@ -46,6 +46,9 @@ OPTIONS
         Allow dangerous changes, e.g. clear the seeding flag or change fsid.
         Make sure that you are aware of the dangers.
 
+--device
+        List of block devices or regular files that are part of the filesystem.
+
 -m
         (since kernel: 5.0)
 
-- 
2.38.1

[PATCH 4/5] btrfs-progs: tune: introduce --noscan option

[Anand Jain]

The function check_where_mounted() scans the system for other btrfs
devices, but in certain cases, we may need a way to instruct
btrfstune not to perform the system scan and instead only work on the
devices provided through the command line. And so, add an option
--noscan.

For example:

  $ mkfs.btrfs -fq -draid0 -mraid0 ./td1 ./td2

  $ btrfstune -m ./td1
	  warning, device 2 is missing
	  ERROR: could not setup extent tree
	  ERROR: open ctree failed

  $ losetup --find --show ./td2
	/dev/loop4
  $ btrfstune -m ./td1

	Or just
  $ btrfstune -m --device ./td1 ./td2

  The 'noscan' option is optional because there may be scenarios where we
  have a copy that we don't want to modify the fsid. In the following
  scenario, we keep 'td2' out of the metadata_uuid changes, even though
  its loop device is created.

  $ cp td2 td3
  $ btrsftune --noscan --device ./td3 -m ./td1

Thanks

Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
 tune/main.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/tune/main.c b/tune/main.c
index f595bfee0e6a..af0e58973355 100644
--- a/tune/main.c
+++ b/tune/main.c
@@ -150,6 +150,7 @@ static const char * const tune_usage[] = {
 	"General:",
 	OPTLINE("-f", "allow dangerous operations, make sure that you are aware of the dangers"),
 	OPTLINE("--device", "devices or regular-files of the filesystem to be scanned"),
+	OPTLINE("--noscan", "do not scan the devices from the system, use only the listed ones"),
 	OPTLINE("--help", "print this help"),
 #if EXPERIMENTAL
 	"",
@@ -176,6 +177,7 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 	bool to_extent_tree = false;
 	bool to_bg_tree = false;
 	bool to_fst = false;
+	bool noscan = false;
 	int csum_type = -1;
 	int argc_devices = 0;
 	char **argv_devices = NULL;
@@ -191,7 +193,7 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 		       GETOPT_VAL_ENABLE_BLOCK_GROUP_TREE,
 		       GETOPT_VAL_DISABLE_BLOCK_GROUP_TREE,
 		       GETOPT_VAL_ENABLE_FREE_SPACE_TREE,
-		       GETOPT_VAL_DEVICE };
+		       GETOPT_VAL_DEVICE, GETOPT_VAL_NOSCAN };
 		static const struct option long_options[] = {
 			{ "help", no_argument, NULL, GETOPT_VAL_HELP},
 			{ "convert-to-block-group-tree", no_argument, NULL,
@@ -204,6 +206,7 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 			{ "csum", required_argument, NULL, GETOPT_VAL_CSUM },
 #endif
 			{ "device", required_argument, NULL, GETOPT_VAL_DEVICE },
+			{ "noscan", no_argument, NULL, GETOPT_VAL_NOSCAN },
 			{ NULL, 0, NULL, 0 }
 		};
 		int c = getopt_long(argc, argv, "S:rxfuU:nmM:", long_options, NULL);
@@ -273,6 +276,10 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 			csum_type = parse_csum_type(optarg);
 			break;
 #endif
+		case GETOPT_VAL_NOSCAN:
+			ctree_flags |= OPEN_CTREE_NO_DEVICES;
+			noscan = true;
+			break;
 		case GETOPT_VAL_HELP:
 		default:
 			usage(&tune_cmd, c != GETOPT_VAL_HELP);
@@ -320,7 +327,7 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 	}
 
 	ret = check_mounted_where(fd, device, NULL, 0, NULL,
-				  SBREAD_IGNORE_FSID_MISMATCH, false);
+				  SBREAD_IGNORE_FSID_MISMATCH, noscan);
 	if (ret < 0) {
 		errno = -ret;
 		error("could not check mount status of %s: %m", device);
@@ -345,7 +352,6 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 	}
 
 	root = open_ctree_fd(fd, device, 0, ctree_flags);
-
 	if (!root) {
 		error("open ctree failed");
 		ret = 1;
-- 
2.38.1

[PATCH 5/5] btrfs-progs: docs: update btrfstune --noscan option

[Anand Jain]

Update the Documentation/btrsftune.rst to carry the new --noscan
option.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
 Documentation/btrfstune.rst | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/Documentation/btrfstune.rst b/Documentation/btrfstune.rst
index 89f4494bbaf0..46793b3d4a77 100644
--- a/Documentation/btrfstune.rst
+++ b/Documentation/btrfstune.rst
@@ -49,6 +49,10 @@ OPTIONS
 --device
         List of block devices or regular files that are part of the filesystem.
 
+--noscan
+        Do not automatically scan the system for other devices from the same
+        filesystem, only use the devices provided as the arguments.
+
 -m
         (since kernel: 5.0)
 
-- 
2.38.1

Re: [PATCH 0/4] btrfs-progs: tune: add --device and --noscan option

[Anand Jain]

Please ignore this patch bundle and instead use the new patch bundle.

      [PATCH 00/10] btrfs-progs: check and tune: add device and noscan 
options

Thanks.

On 13/6/23 18:47, Anand Jain wrote:
> Separated out from the preparatory patch set. Depends on the preparatory
> patch set: ("btrfs-progs: cleanup and preparatory around device scan").
> This set (along with its preparatory patch) has passed the btrfs-progs
> test suite.
> 
> By default, btrfstune scans all the block devices in the system.
> 
> To scan regular files without mapping them to a loop device, add the
> --device option.
> 
> The option arguments follow the same pattern as in the "mkfs.btrfs -O"
> option.
> 
> To indicate not to scan the system for other devices, add the --noscan
> option.
> 
> For example:
> 
>    The command below will scan both regular files and the devices
>    provided in the --device option, along with the system block devices.
> 
> 	btrfstune -m --device /tdev/td1,/tdev/td2 /tdev/td3
>    or
> 	btrfstune -m --device /tdev/td1 --device /tdev/td2 /tdev/td3
> 
>    In some cases, if you need to avoid the default system scan for the
>    block device, you can use the --noscan option.
> 
> 	btrfstune -m --noscan --device /tdev/td1,/tdev/td2 /tdev/td3
> 
> Anand Jain (5):
>    btrfs-progs: tune: consolidate return goto free-out
>    btrfs-progs: tune: introduce --device option
>    btrfs-progs: docs: update btrfstune --device option
>    btrfs-progs: tune: introduce --noscan option
>    btrfs-progs: docs: update btrfstune --noscan option
> 
>   Documentation/btrfstune.rst |   7 +++
>   tune/main.c                 | 103 ++++++++++++++++++++++++++++++------
>   2 files changed, 95 insertions(+), 15 deletions(-)
>