[PATCH 0/3] libubi / mkfs.ubifs #2

[PATCH 0/3] libubi / mkfs.ubifs #2

[Corentin Chary]

These patchs will allow mkfs.ubifs to write the image directly
on an UBI volume. This time libubi is used. I couldn't manage to
make the Makefile works correctly, even with Mike trick for dependencies.
Mike, could you help me on that ?
You can still make it compile with "cd ubi-utils; make; cd ../mkfs.ubifs; make"

Thanks

Corentin Chary (3):
  libubi: add ubi_is_mapped() function
  libubi: fix multiple memory corruptions
  mkfs.ubifs: use libubi to format UBI volume

 mkfs.ubifs/Makefile        |    5 +-
 mkfs.ubifs/lpt.c           |   10 +-
 mkfs.ubifs/mkfs.ubifs.c    |  187 ++++++++++++++++++++++++++++++++++++--------
 mkfs.ubifs/mkfs.ubifs.h    |    4 +-
 mkfs.ubifs/ubifs.h         |    3 +
 ubi-utils/include/libubi.h |   17 ++++
 ubi-utils/src/libubi.c     |   13 +++-
 7 files changed, 196 insertions(+), 43 deletions(-)

[PATCH 1/3] libubi: add ubi_is_mapped() function

[Corentin Chary]

Signed-off-by: Corentin Chary <corentincj@iksaif.net>
---
 ubi-utils/include/libubi.h |   17 +++++++++++++++++
 ubi-utils/src/libubi.c     |    5 +++++
 2 files changed, 22 insertions(+), 0 deletions(-)

diff --git a/ubi-utils/include/libubi.h b/ubi-utils/include/libubi.h
index 4ffe1e8..f52904d 100644
--- a/ubi-utils/include/libubi.h
+++ b/ubi-utils/include/libubi.h
@@ -414,6 +414,23 @@ int ubi_set_property(int fd, uint8_t property, uint64_t value);
  */
 int ubi_leb_unmap(int fd, int lnum);
 
+/**
+ * ubi_is_mapped - check if logical eraseblock is mapped.
+ * @fd: volume character device file descriptor
+ * @lnum: logical eraseblock number
+ *
+ * This function checks if logical eraseblock @lnum is mapped to a physical
+ * eraseblock. If a logical eraseblock is un-mapped, this does not necessarily
+ * mean it will still be un-mapped after the UBI device is re-attached. The
+ * logical eraseblock may become mapped to the physical eraseblock it was last
+ * mapped to.
+ *
+ * This function returns %1 if the LEB is mapped, %0 if not, and %-1 in case of
+ * failure. If the volume is damaged because of an interrupted update errno
+ * set with %EBADF error code.
+ */
+int ubi_is_mapped(int fd, int lnum);
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/ubi-utils/src/libubi.c b/ubi-utils/src/libubi.c
index 5c8ce9e..a81173d 100644
--- a/ubi-utils/src/libubi.c
+++ b/ubi-utils/src/libubi.c
@@ -1255,3 +1255,8 @@ int ubi_leb_unmap(int fd, int lnum)
 {
 	return ioctl(fd, UBI_IOCEBUNMAP, &lnum);
 }
+
+int ubi_is_mapped(int fd, int lnum)
+{
+	return ioctl(fd, UBI_IOCEBISMAP, &lnum);
+}
-- 
1.6.3

[PATCH 2/3] libubi: fix multiple memory corruptions

[Corentin Chary]

The memset is obviously wrong, and valgrind tells
use there are some uninitialised bytes used after read()

Signed-off-by: Corentin Chary <corentincj@iksaif.net>
---
 ubi-utils/src/libubi.c |    8 +++++---
 1 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/ubi-utils/src/libubi.c b/ubi-utils/src/libubi.c
index a81173d..d748a65 100644
--- a/ubi-utils/src/libubi.c
+++ b/ubi-utils/src/libubi.c
@@ -82,16 +82,17 @@ static int read_positive_ll(const char *file, long long *value)
 	if (fd == -1)
 		return -1;
 
-	rd = read(fd, buf, 50);
+	rd = read(fd, buf, sizeof(buf));
 	if (rd == -1) {
 		sys_errmsg("cannot read \"%s\"", file);
 		goto out_error;
 	}
-	if (rd == 50) {
+	if (rd == sizeof(buf)) {
 		errmsg("contents of \"%s\" is too long", file);
 		errno = EINVAL;
 		goto out_error;
 	}
+	buf[rd] = '\0';
 
 	if (sscanf(buf, "%lld\n", value) != 1) {
 		/* This must be a UBI bug */
@@ -166,6 +167,7 @@ static int read_data(const char *file, void *buf, int buf_len)
 		sys_errmsg("cannot read \"%s\"", file);
 		goto out_error;
 	}
+	((char *)buf)[rd] = '\0';
 
 	/* Make sure all data is read */
 	tmp1 = read(fd, &tmp, 1);
@@ -1244,7 +1246,7 @@ int ubi_set_property(int fd, uint8_t property, uint64_t value)
 {
 	struct ubi_set_prop_req r;
 
-	memset(&r, sizeof(struct ubi_set_prop_req), '\0');
+	memset(&r, 0, sizeof(struct ubi_set_prop_req));
 	r.property = property;
 	r.value = value;
 
-- 
1.6.3

[PATCH 3/3] mkfs.ubifs: use libubi to format UBI volume

[Corentin Chary]

libubi is now used to format directly UBI volume.
Typing mkfs.ubifs /dev/ubi0_0 is now possible.
dtypes should be ok as they are taken from UBIFS code.

Signed-off-by: Corentin Chary <corentincj@iksaif.net>
---
 mkfs.ubifs/Makefile     |    5 +-
 mkfs.ubifs/lpt.c        |   10 +-
 mkfs.ubifs/mkfs.ubifs.c |  187 ++++++++++++++++++++++++++++++++++++++--------
 mkfs.ubifs/mkfs.ubifs.h |    4 +-
 mkfs.ubifs/ubifs.h      |    3 +
 5 files changed, 169 insertions(+), 40 deletions(-)

diff --git a/mkfs.ubifs/Makefile b/mkfs.ubifs/Makefile
index e5bf9ce..a678b0a 100644
--- a/mkfs.ubifs/Makefile
+++ b/mkfs.ubifs/Makefile
@@ -1,8 +1,11 @@
+
+CPPFLAGS += -I../include -I../ubi-utils/include
+
 ALL_SOURCES=*.[ch] hashtable/*.[ch]
 
 TARGETS = mkfs.ubifs
 
-LDLIBS_mkfs.ubifs = -lz -llzo2 -lm -luuid
+LDLIBS_mkfs.ubifs = -lz -llzo2 -lm -luuid -L../ubi-utils/ -lubi
 
 include ../common.mk
 
diff --git a/mkfs.ubifs/lpt.c b/mkfs.ubifs/lpt.c
index f6d4352..60002ff 100644
--- a/mkfs.ubifs/lpt.c
+++ b/mkfs.ubifs/lpt.c
@@ -410,7 +410,7 @@ int create_lpt(struct ubifs_info *c)
 			alen = ALIGN(len, c->min_io_size);
 			set_ltab(c, lnum, c->leb_size - alen, alen - len);
 			memset(p, 0xff, alen - len);
-			err = write_leb(lnum++, alen, buf);
+			err = write_leb(lnum++, alen, buf, UBI_SHORTTERM);
 			if (err)
 				goto out;
 			p = buf;
@@ -452,7 +452,7 @@ int create_lpt(struct ubifs_info *c)
 				set_ltab(c, lnum, c->leb_size - alen,
 					    alen - len);
 				memset(p, 0xff, alen - len);
-				err = write_leb(lnum++, alen, buf);
+				err = write_leb(lnum++, alen, buf, UBI_SHORTTERM);
 				if (err)
 					goto out;
 				p = buf;
@@ -499,7 +499,7 @@ int create_lpt(struct ubifs_info *c)
 			alen = ALIGN(len, c->min_io_size);
 			set_ltab(c, lnum, c->leb_size - alen, alen - len);
 			memset(p, 0xff, alen - len);
-			err = write_leb(lnum++, alen, buf);
+			err = write_leb(lnum++, alen, buf, UBI_SHORTTERM);
 			if (err)
 				goto out;
 			p = buf;
@@ -522,7 +522,7 @@ int create_lpt(struct ubifs_info *c)
 		alen = ALIGN(len, c->min_io_size);
 		set_ltab(c, lnum, c->leb_size - alen, alen - len);
 		memset(p, 0xff, alen - len);
-		err = write_leb(lnum++, alen, buf);
+		err = write_leb(lnum++, alen, buf, UBI_SHORTTERM);
 		if (err)
 			goto out;
 		p = buf;
@@ -542,7 +542,7 @@ int create_lpt(struct ubifs_info *c)
 
 	/* Write remaining buffer */
 	memset(p, 0xff, alen - len);
-	err = write_leb(lnum, alen, buf);
+	err = write_leb(lnum, alen, buf, UBI_SHORTTERM);
 	if (err)
 		goto out;
 
diff --git a/mkfs.ubifs/mkfs.ubifs.c b/mkfs.ubifs/mkfs.ubifs.c
index bedf8a7..c4d427c 100644
--- a/mkfs.ubifs/mkfs.ubifs.c
+++ b/mkfs.ubifs/mkfs.ubifs.c
@@ -95,6 +95,7 @@ struct inum_mapping {
  */
 struct ubifs_info info_;
 static struct ubifs_info *c = &info_;
+static libubi_t ubi;
 
 /* Debug levels are: 0 (none), 1 (statistics), 2 (files) ,3 (more details) */
 int debug_level;
@@ -105,6 +106,7 @@ static int root_len;
 static struct stat root_st;
 static char *output;
 static int out_fd;
+static int out_ubi;
 static int squash_owner;
 
 /* The 'head' (position) which nodes are written */
@@ -154,7 +156,10 @@ static const struct option longopts[] = {
 };
 
 static const char *helptext =
-"Usage: mkfs.ubifs [OPTIONS]\n"
+"Usage: mkfs.ubifs [OPTIONS] [device]\n"
+"Example: mkfs.ubifs ubi0:test\n"
+"         mkfs.ubifs -r /opt/img ubi0:test\n"
+"         mkfs.ubifs -m 512 -e 128KiB -c 100 -r /opt/img -o ubifs.img\n\n"
 "Make a UBIFS file system image from an existing directory tree\n\n"
 "Options:\n"
 "-r, -d, --root=DIR       build file system from directory DIR\n"
@@ -357,11 +362,9 @@ static int validate_options(void)
 {
 	int tmp;
 
-	if (!root)
-		return err_msg("root directory was not specified");
 	if (!output)
-		return err_msg("no output file specified");
-	if (in_path(root, output))
+		return err_msg("no output file or UBI volume specified");
+	if (root && in_path(root, output))
 		return err_msg("output file cannot be in the UBIFS root "
 			       "directory");
 	if (!is_power_of_2(c->min_io_size))
@@ -468,6 +471,34 @@ static long long get_bytes(const char *str)
 	return bytes;
 }
 
+/**
+ * open_ubi - parse UBI device name string and open the UBI device.
+ * @name: UBI volume name
+ *
+ * There are several ways to specify UBI volumes
+ * o /dev/ubiX_Y - UBI device number X, volume Y;
+ *
+ * ubi:name ubiX_Y, etc.. are not implemented right now
+ *
+ * Returns %0 in case of success and %-1 in case of faillure
+ */
+static int open_ubi(const char *name)
+{
+	struct stat st;
+
+	if (stat(name, &st) || !S_ISCHR(st.st_mode))
+		return -1;
+
+	ubi = libubi_open();
+	if (!ubi)
+		return -1;
+	if (ubi_get_vol_info(ubi, name, &c->vi))
+		return -1;
+	if (ubi_get_dev_info1(ubi, c->vi.dev_num, &c->di))
+		return -1;
+	return 0;
+}
+
 static int get_options(int argc, char**argv)
 {
 	int opt, i;
@@ -616,6 +647,20 @@ static int get_options(int argc, char**argv)
 		}
 	}
 
+	if (optind != argc && !output)
+		output = strdup(argv[optind]);
+	if (output)
+		out_ubi = !open_ubi(output);
+
+	if (out_ubi) {
+		c->min_io_size = c->di.min_io_size;
+		c->leb_size = c->vi.leb_size;
+		c->max_leb_cnt = c->vi.rsvd_lebs;
+	}
+
+	if (!output)
+		return err_msg("not output device or file specified");
+
 	if (c->min_io_size == -1)
 		return err_msg("min. I/O unit was not specified "
 			       "(use -h for help)");
@@ -714,36 +759,43 @@ static void prepare_node(void *node, int len)
 }
 
 /**
- * write_leb - copy the image of a LEB to the output file.
+ * write_leb - copy the image of a LEB to the output target
  * @lnum: LEB number
  * @len: length of data in the buffer
  * @buf: buffer (must be at least c->leb_size bytes)
+ * @dtype: expected data type
  */
-int write_leb(int lnum, int len, void *buf)
+int write_leb(int lnum, int len, void *buf, int dtype)
 {
 	off64_t pos = (off64_t)lnum * c->leb_size;
 
 	dbg_msg(3, "LEB %d len %d", lnum, len);
+	memset(buf + len, 0xff, c->leb_size - len);
+	if (out_ubi)
+		if (ubi_leb_change_start(ubi, out_fd, lnum, c->leb_size, dtype))
+			return sys_err_msg("ubi_leb_change_start failed");
+
 	if (lseek64(out_fd, pos, SEEK_SET) != pos)
 		return sys_err_msg("lseek64 failed seeking %lld",
 				   (long long)pos);
 
-	memset(buf + len, 0xff, c->leb_size - len);
-
 	if (write(out_fd, buf, c->leb_size) != c->leb_size)
 		return sys_err_msg("write failed writing %d bytes at pos %lld",
 				   c->leb_size, (long long)pos);
 
 	return 0;
+
 }
 
+
 /**
  * write_empty_leb - copy the image of an empty LEB to the output file.
  * @lnum: LEB number
+ * @dtype: expected data type
  */
-static int write_empty_leb(int lnum)
+static int write_empty_leb(int lnum, int dtype)
 {
-	return write_leb(lnum, 0, leb_buf);
+	return write_leb(lnum, 0, leb_buf, dtype);
 }
 
 /**
@@ -790,8 +842,9 @@ static int do_pad(void *buf, int len)
  * @node: node
  * @len: node length
  * @lnum: LEB number
+ * @dtype: expected data type
  */
-static int write_node(void *node, int len, int lnum)
+static int write_node(void *node, int len, int lnum, int dtype)
 {
 	prepare_node(node, len);
 
@@ -799,7 +852,7 @@ static int write_node(void *node, int len, int lnum)
 
 	len = do_pad(leb_buf, len);
 
-	return write_leb(lnum, len, leb_buf);
+	return write_leb(lnum, len, leb_buf, dtype);
 }
 
 /**
@@ -909,7 +962,7 @@ static int flush_nodes(void)
 	if (!head_offs)
 		return 0;
 	len = do_pad(leb_buf, head_offs);
-	err = write_leb(head_lnum, len, leb_buf);
+	err = write_leb(head_lnum, len, leb_buf, UBI_UNKNOWN);
 	if (err)
 		return err;
 	set_lprops(head_lnum, head_offs, head_flags);
@@ -1011,8 +1064,6 @@ static int add_inode_with_data(struct stat *st, ino_t inum, void *data,
 	ino->mtime_nsec = 0;
 	ino->uid        = cpu_to_le32(st->st_uid);
 	ino->gid        = cpu_to_le32(st->st_gid);
-	ino->uid        = cpu_to_le32(st->st_uid);
-	ino->gid        = cpu_to_le32(st->st_gid);
 	ino->mode       = cpu_to_le32(st->st_mode);
 	ino->flags      = cpu_to_le32(use_flags);
 	ino->data_len   = cpu_to_le32(data_len);
@@ -1583,14 +1634,20 @@ static int write_data(void)
 {
 	int err;
 
-	err = stat(root, &root_st);
-	if (err)
-		return sys_err_msg("bad root file-system directory '%s'", root);
+	if (root) {
+		err = stat(root, &root_st);
+		if (err)
+			return sys_err_msg("bad root file-system directory '%s'"
+					   , root);
+	} else {
+		root_st.st_mtime = time(NULL);
+		root_st.st_atime = root_st.st_ctime = root_st.st_mtime;
+	}
 	root_st.st_uid = root_st.st_gid = 0;
 	root_st.st_mode = S_IFDIR | S_IRWXU | S_IRWXG | S_IRWXO;
 
 	head_flags = 0;
-	err = add_directory(root, UBIFS_ROOT_INO, &root_st, 0);
+	err = add_directory(root, UBIFS_ROOT_INO, &root_st, !root);
 	if (err)
 		return err;
 	err = add_multi_linked_files();
@@ -1833,7 +1890,7 @@ static int set_gc_lnum(void)
 	int err;
 
 	c->gc_lnum = head_lnum++;
-	err = write_empty_leb(c->gc_lnum);
+	err = write_empty_leb(c->gc_lnum, UBI_LONGTERM);
 	if (err)
 		return err;
 	set_lprops(c->gc_lnum, 0, 0);
@@ -1909,7 +1966,7 @@ static int write_super(void)
 	if (c->big_lpt)
 		sup.flags |= cpu_to_le32(UBIFS_FLG_BIGLPT);
 
-	return write_node(&sup, UBIFS_SB_NODE_SZ, UBIFS_SB_LNUM);
+	return write_node(&sup, UBIFS_SB_NODE_SZ, UBIFS_SB_LNUM, UBI_LONGTERM);
 }
 
 /**
@@ -1952,11 +2009,11 @@ static int write_master(void)
 	mst.total_dark   = cpu_to_le64(c->lst.total_dark);
 	mst.leb_cnt      = cpu_to_le32(c->leb_cnt);
 
-	err = write_node(&mst, UBIFS_MST_NODE_SZ, UBIFS_MST_LNUM);
+	err = write_node(&mst, UBIFS_MST_NODE_SZ, UBIFS_MST_LNUM, UBI_SHORTTERM);
 	if (err)
 		return err;
 
-	err = write_node(&mst, UBIFS_MST_NODE_SZ, UBIFS_MST_LNUM + 1);
+	err = write_node(&mst, UBIFS_MST_NODE_SZ, UBIFS_MST_LNUM + 1, UBI_SHORTTERM);
 	if (err)
 		return err;
 
@@ -1976,14 +2033,14 @@ static int write_log(void)
 	cs.ch.node_type = UBIFS_CS_NODE;
 	cs.cmt_no = cpu_to_le64(0);
 
-	err = write_node(&cs, UBIFS_CS_NODE_SZ, lnum);
+	err = write_node(&cs, UBIFS_CS_NODE_SZ, lnum, UBI_UNKNOWN);
 	if (err)
 		return err;
 
 	lnum += 1;
 
 	for (i = 1; i < c->log_lebs; i++, lnum++) {
-		err = write_empty_leb(lnum);
+		err = write_empty_leb(lnum, UBI_UNKNOWN);
 		if (err)
 			return err;
 	}
@@ -2004,7 +2061,7 @@ static int write_lpt(void)
 
 	lnum = c->nhead_lnum + 1;
 	while (lnum <= c->lpt_last) {
-		err = write_empty_leb(lnum++);
+		err = write_empty_leb(lnum++, UBI_SHORTTERM);
 		if (err)
 			return err;
 	}
@@ -2021,7 +2078,7 @@ static int write_orphan_area(void)
 
 	lnum = UBIFS_LOG_LNUM + c->log_lebs + c->lpt_lebs;
 	for (i = 0; i < c->orph_lebs; i++, lnum++) {
-		err = write_empty_leb(lnum);
+		err = write_empty_leb(lnum, UBI_SHORTTERM);
 		if (err)
 			return err;
 	}
@@ -2029,14 +2086,74 @@ static int write_orphan_area(void)
 }
 
 /**
+ * check_volume_empty - check if the UBI volume is empty.
+ *
+ * This function checks if the UBIFS volume is empty by looking if its LEBs are
+ * mapped or not.
+ * Returns zero in case of success and a negative error code in case of
+ * failure.
+ */
+static int check_volume_empty()
+{
+	int lnum, err;
+
+	for (lnum = 0; lnum < c->vi.rsvd_lebs; lnum++) {
+		err = ubi_is_mapped(out_fd, lnum);
+		if (err < 0)
+			return err;
+		if (err == 1) {
+			return 0;
+		}
+	}
+	return 0;
+}
+
+/**
+ * erase an ubi volume
+ */
+static int erase_volume()
+{
+	int lnum, err;
+
+	// FIXME: Ask if the user really want to erase this volume
+	dbg_msg(1, "volume is not empty, erasing ...");
+	for (lnum = 0; lnum < c->vi.rsvd_lebs; lnum++) {
+		err = ubi_is_mapped(out_fd, lnum);
+		if (err == 1) {
+			dbg_msg(3, "erasing leb %d/%d", lnum, c->vi.rsvd_lebs);
+			err = ubi_leb_unmap(out_fd, lnum);
+		}
+		if (err < 0)
+			return err;
+	}
+	dbg_msg(1, "done.");
+	return 0;
+}
+
+/**
  * open_target - open the output file.
  */
 static int open_target(void)
 {
-	out_fd = open(output, O_CREAT | O_RDWR | O_TRUNC,
-		      S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH);
-	if (out_fd == -1)
-		return sys_err_msg("cannot create output file '%s'", output);
+	if (out_ubi) {
+		out_fd = open(output, O_RDWR | O_EXCL);
+
+		if (out_fd == -1)
+			return sys_err_msg("cannot open the UBI volume '%s'",
+					   output);
+		if (ubi_set_property(out_fd, UBI_PROP_DIRECT_WRITE, 1))
+			return sys_err_msg("ubi_set_property failed");
+
+		if (!check_volume_empty())
+			if (erase_volume() < 0)
+				return sys_err_msg("cannot erase the UBI volume");
+	} else {
+		out_fd = open(output, O_CREAT | O_RDWR | O_TRUNC,
+			      S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH);
+		if (out_fd == -1)
+			return sys_err_msg("cannot create output file '%s'",
+					   output);
+	}
 	return 0;
 }
 
@@ -2045,8 +2162,12 @@ static int open_target(void)
  */
 static int close_target(void)
 {
-	if (close(out_fd) == -1)
+	if (ubi)
+		libubi_close(ubi);
+	if (out_fd >= 0 && close(out_fd) == -1)
 		return sys_err_msg("cannot close output file '%s'", output);
+	if (output)
+		free(output);
 	return 0;
 }
 
diff --git a/mkfs.ubifs/mkfs.ubifs.h b/mkfs.ubifs/mkfs.ubifs.h
index 6460bd5..16b34c7 100644
--- a/mkfs.ubifs/mkfs.ubifs.h
+++ b/mkfs.ubifs/mkfs.ubifs.h
@@ -46,7 +46,9 @@
 #include <libgen.h>
 #include <ctype.h>
 #include <uuid/uuid.h>
+#include <sys/file.h>
 
+#include "libubi.h"
 #include "crc32.h"
 #include "defs.h"
 #include "crc16.h"
@@ -128,7 +130,7 @@ extern struct ubifs_info info_;
 
 struct hashtable_itr;
 
-int write_leb(int lnum, int len, void *buf);
+int write_leb(int lnum, int len, void *buf, int dtype);
 int parse_devtable(const char *tbl_file);
 struct path_htbl_element *devtbl_find_path(const char *path);
 struct name_htbl_element *devtbl_find_name(struct path_htbl_element *ph_elt,
diff --git a/mkfs.ubifs/ubifs.h b/mkfs.ubifs/ubifs.h
index 79c7192..0535b6f 100644
--- a/mkfs.ubifs/ubifs.h
+++ b/mkfs.ubifs/ubifs.h
@@ -366,6 +366,9 @@ struct ubifs_info
 	int dead_wm;
 	int dark_wm;
 
+	struct ubi_dev_info di;
+	struct ubi_vol_info vi;
+
 	int gc_lnum;
 	long long rp_size;
 
-- 
1.6.3

Re: [PATCH 3/3] mkfs.ubifs: use libubi to format UBI volume

[Artem Bityutskiy]

Hi,

On Sat, 2009-05-09 at 11:41 +0200, Corentin Chary wrote:
> libubi is now used to format directly UBI volume.
> Typing mkfs.ubifs /dev/ubi0_0 is now possible.
> dtypes should be ok as they are taken from UBIFS code.

Did you test that this works fine on UBI volume and there
are no regressions when you output to a file?

...


>  static const char *helptext =
> -"Usage: mkfs.ubifs [OPTIONS]\n"
> +"Usage: mkfs.ubifs [OPTIONS] [device]\n"
> +"Example: mkfs.ubifs ubi0:test\n"
> +"         mkfs.ubifs -r /opt/img ubi0:test\n"
> +"         mkfs.ubifs -m 512 -e 128KiB -c 100 -r /opt/img -o ubifs.img\n\n"
>  "Make a UBIFS file system image from an existing directory tree\n\n"
>  "Options:\n"
>  "-r, -d, --root=DIR       build file system from directory DIR\n"

I think some more text in the help output which says you may direct
the result to either a file or UBI volume would be nice to have.

> @@ -357,11 +362,9 @@ static int validate_options(void)
>  {
>  	int tmp;
>  
> -	if (!root)
> -		return err_msg("root directory was not specified");
>  	if (!output)
> -		return err_msg("no output file specified");
> -	if (in_path(root, output))
> +		return err_msg("no output file or UBI volume specified");
> +	if (root && in_path(root, output))
>  		return err_msg("output file cannot be in the UBIFS root "
>  			       "directory");

Why you removed

if (!root)
	return err_msg("root directory was not specified");

?

>  	if (!is_power_of_2(c->min_io_size))
> @@ -468,6 +471,34 @@ static long long get_bytes(const char *str)
>  	return bytes;
>  }
>  
> +/**
> + * open_ubi - parse UBI device name string and open the UBI device.
> + * @name: UBI volume name
> + *
> + * There are several ways to specify UBI volumes
> + * o /dev/ubiX_Y - UBI device number X, volume Y;
> + *
> + * ubi:name ubiX_Y, etc.. are not implemented right now
> + *
> + * Returns %0 in case of success and %-1 in case of faillure
> + */
> +static int open_ubi(const char *name)
> +{
> +	struct stat st;
> +
> +	if (stat(name, &st) || !S_ISCHR(st.st_mode))
> +		return -1;
> +
> +	ubi = libubi_open();
> +	if (!ubi)
> +		return -1;
> +	if (ubi_get_vol_info(ubi, name, &c->vi))
> +		return -1;
> +	if (ubi_get_dev_info1(ubi, c->vi.dev_num, &c->di))
> +		return -1;
> +	return 0;
> +}

The comments are incorrect. 'ubi_get_vol_info()' does not depend on the
name of the file at all. It looks at device's major/minor, then looks up
the information in sysfs.
 
>  /**
> - * write_leb - copy the image of a LEB to the output file.
> + * write_leb - copy the image of a LEB to the output target

You removed the "." from the end :-)

> +
>  /**
>   * write_empty_leb - copy the image of an empty LEB to the output file.

Also s/file/target/ ?

> @@ -1011,8 +1064,6 @@ static int add_inode_with_data(struct stat *st, ino_t inum, void *data,
>  	ino->mtime_nsec = 0;
>  	ino->uid        = cpu_to_le32(st->st_uid);
>  	ino->gid        = cpu_to_le32(st->st_gid);
> -	ino->uid        = cpu_to_le32(st->st_uid);
> -	ino->gid        = cpu_to_le32(st->st_gid);
>  	ino->mode       = cpu_to_le32(st->st_mode);
>  	ino->flags      = cpu_to_le32(use_flags);
>  	ino->data_len   = cpu_to_le32(data_len);

Why the 2 lines are removed?

>  /**
> + * check_volume_empty - check if the UBI volume is empty.
> + *
> + * This function checks if the UBIFS volume is empty by looking if its LEBs are
> + * mapped or not.
> + * Returns zero in case of success and a negative error code in case of
> + * failure.
> + */
> +static int check_volume_empty()
check_volume_empty(void)

Comments are untidy.

> +
> +/**
> + * erase an ubi volume
> + */
> +static int erase_volume()

Ditto.

> +
> +	// FIXME: Ask if the user really want to erase this volume
> +	dbg_msg(1, "volume is not empty, erasing ...");
> +	for (lnum = 0; lnum < c->vi.rsvd_lebs; lnum++) {
> +		err = ubi_is_mapped(out_fd, lnum);
> +		if (err == 1) {
> +			dbg_msg(3, "erasing leb %d/%d", lnum, c->vi.rsvd_lebs);
> +			err = ubi_leb_unmap(out_fd, lnum);
> +		}
> +		if (err < 0)
> +			return err;
> +	}
> +	dbg_msg(1, "done.");
> +	return 0;
> +}

Yeah, I think you should either ask the user, or just refuse
writing to it. Users may just 'ubiupdatevol -t' to erase the
volume before invocating mkfs.ubifs.

Probably it is easier for now to just remove whole
'erase_volume()' ?
 
-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: [PATCH 3/3] mkfs.ubifs: use libubi to format UBI volume

[Corentin Chary]

On Wed, May 13, 2009 at 10:12 AM, Artem Bityutskiy
<dedekind@infradead.org> wrote:
> Hi,
>
> On Sat, 2009-05-09 at 11:41 +0200, Corentin Chary wrote:
>> libubi is now used to format directly UBI volume.
>> Typing mkfs.ubifs /dev/ubi0_0 is now possible.
>> dtypes should be ok as they are taken from UBIFS code.
>
> Did you test that this works fine on UBI volume and there
> are no regressions when you output to a file?

I did some test with and without --root, it worked fine.
But I'll do some more test with the corrected patch.

>>  static const char *helptext =
>> -"Usage: mkfs.ubifs [OPTIONS]\n"
>> +"Usage: mkfs.ubifs [OPTIONS] [device]\n"
>> +"Example: mkfs.ubifs ubi0:test\n"
>> +"         mkfs.ubifs -r /opt/img ubi0:test\n"
>> +"         mkfs.ubifs -m 512 -e 128KiB -c 100 -r /opt/img -o ubifs.img\n\n"
>>  "Make a UBIFS file system image from an existing directory tree\n\n"
>>  "Options:\n"
>>  "-r, -d, --root=DIR       build file system from directory DIR\n"
>
> I think some more text in the help output which says you may direct
> the result to either a file or UBI volume would be nice to have.

Yep, and the first example is not valid anymore as this patch don't
handle "ubi:<name>" format.

>> @@ -357,11 +362,9 @@ static int validate_options(void)
>>  {
>>       int tmp;
>>
>> -     if (!root)
>> -             return err_msg("root directory was not specified");
>>       if (!output)
>> -             return err_msg("no output file specified");
>> -     if (in_path(root, output))
>> +             return err_msg("no output file or UBI volume specified");
>> +     if (root && in_path(root, output))
>>               return err_msg("output file cannot be in the UBIFS root "
>>                              "directory");
>
> Why you removed
>
> if (!root)
>        return err_msg("root directory was not specified");
>
> ?

Because I wanted mkfs.ubifs to work without this parameter, like other mkfs.*.
I know mounting an empty volume will works, but we should also be able
to use mkfs.ubifs to format
empty volumes.


> The comments are incorrect. 'ubi_get_vol_info()' does not depend on the
> name of the file at all. It looks at device's major/minor, then looks up
> the information in sysfs.

Yep, I forget to rewrite the comments.

>>  /**
>> - * write_leb - copy the image of a LEB to the output file.
>> + * write_leb - copy the image of a LEB to the output target
>
> You removed the "." from the end :-)

Oups ....

>> +
>>  /**
>>   * write_empty_leb - copy the image of an empty LEB to the output file.

Write empty leb to the target, wich can be a file or an UBI volume ..
I'll clarify that.

>> @@ -1011,8 +1064,6 @@ static int add_inode_with_data(struct stat *st, ino_t inum, void *data,
>>       ino->mtime_nsec = 0;
>>       ino->uid        = cpu_to_le32(st->st_uid);
>>       ino->gid        = cpu_to_le32(st->st_gid);
>> -     ino->uid        = cpu_to_le32(st->st_uid);
>> -     ino->gid        = cpu_to_le32(st->st_gid);
>>       ino->mode       = cpu_to_le32(st->st_mode);
>>       ino->flags      = cpu_to_le32(use_flags);
>>       ino->data_len   = cpu_to_le32(data_len);
>
> Why the 2 lines are removed?

Theses should have been removed in another patch, but look, they are
here twice =).

>> +static int check_volume_empty()
> check_volume_empty(void)
>
> Comments are untidy.

Oups ..

>
> Yeah, I think you should either ask the user, or just refuse
> writing to it. Users may just 'ubiupdatevol -t' to erase the
> volume before invocating mkfs.ubifs.
>
> Probably it is easier for now to just remove whole
> 'erase_volume()' ?
>
mkfs.ext{2/3/4} and mkfs.vfat don't ask for confirmation
I believe mkfs.reiserfs does.
But I won't hurt to ask.
Maybe first we can remove erase_volume(), and then, in another patch,
add erase volume with confirmation.


New patch is comming at the end of the week ...
-- 
Corentin Chary
http://xf.iksaif.net - http://uffs.org

Re: [PATCH 3/3] mkfs.ubifs: use libubi to format UBI volume

[Artem Bityutskiy]

On Wed, 2009-05-13 at 10:38 +0200, Corentin Chary wrote:
> > On Sat, 2009-05-09 at 11:41 +0200, Corentin Chary wrote:
> >> libubi is now used to format directly UBI volume.
> >> Typing mkfs.ubifs /dev/ubi0_0 is now possible.
> >> dtypes should be ok as they are taken from UBIFS code.
> >
> > Did you test that this works fine on UBI volume and there
> > are no regressions when you output to a file?
> 
> I did some test with and without --root, it worked fine.
> But I'll do some more test with the corrected patch.

Ok, thanks.

> Because I wanted mkfs.ubifs to work without this parameter, like other mkfs.*.
> I know mounting an empty volume will works, but we should also be able
> to use mkfs.ubifs to format
> empty volumes.

Ok.

> >>       ino->uid        = cpu_to_le32(st->st_uid);
> >>       ino->gid        = cpu_to_le32(st->st_gid);
> >> -     ino->uid        = cpu_to_le32(st->st_uid);
> >> -     ino->gid        = cpu_to_le32(st->st_gid);
> >>       ino->mode       = cpu_to_le32(st->st_mode);
> >>       ino->flags      = cpu_to_le32(use_flags);
> >>       ino->data_len   = cpu_to_le32(data_len);
> >
> > Why the 2 lines are removed?
> 
> Theses should have been removed in another patch, but look, they are
> here twice =).

Ah, OK.

> > Yeah, I think you should either ask the user, or just refuse
> > writing to it. Users may just 'ubiupdatevol -t' to erase the
> > volume before invocating mkfs.ubifs.
> >
> > Probably it is easier for now to just remove whole
> > 'erase_volume()' ?
> >
> mkfs.ext{2/3/4} and mkfs.vfat don't ask for confirmation
> I believe mkfs.reiserfs does.
> But I won't hurt to ask.
> Maybe first we can remove erase_volume(), and then, in another patch,
> add erase volume with confirmation.

Well, OK, let's be consistent with other mkfs'es then.

> New patch is comming at the end of the week ...

Thanks!

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: [PATCH 0/3] libubi / mkfs.ubifs #2

[Corentin Chary]

On Sat, May 9, 2009 at 11:41 AM, Corentin Chary <corentincj@iksaif.net> wrote:
> These patchs will allow mkfs.ubifs to write the image directly

Again, there are waiting for moderator approval because they have a
suspicious header :/
Can be caused by
X-Bad-Reply: References and In-Reply-To but no 'Re:' in Subject.
but I'm not sure.
I send them with git, maybe next time I won't send them as a thread ?
thanks

-- 
Corentin Chary
http://xf.iksaif.net

Re: [PATCH 0/3] libubi / mkfs.ubifs #2

[Artem Bityutskiy]

On Mon, 2009-05-11 at 09:02 +0200, Corentin Chary wrote:
> On Sat, May 9, 2009 at 11:41 AM, Corentin Chary <corentincj@iksaif.net> wrote:
> > These patchs will allow mkfs.ubifs to write the image directly
> 
> Again, there are waiting for moderator approval because they have a
> suspicious header :/
> Can be caused by
> X-Bad-Reply: References and In-Reply-To but no 'Re:' in Subject.
> but I'm not sure.
> I send them with git, maybe next time I won't send them as a thread ?
> thanks

This is broken MTD ML filter. Patch series are incorrectly caught.
Few years ago dwmw2 was going to fix this :-)

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: [PATCH 0/3] libubi / mkfs.ubifs #2

[Artem Bityutskiy]

On Sat, 2009-05-09 at 11:41 +0200, Corentin Chary wrote:
> These patchs will allow mkfs.ubifs to write the image directly
> on an UBI volume. This time libubi is used. I couldn't manage to
> make the Makefile works correctly, even with Mike trick for dependencies.
> Mike, could you help me on that ?
> You can still make it compile with "cd ubi-utils; make; cd ../mkfs.ubifs; make"

Due to time constraints, I'll look at this a bit later this
week. But at the first glance looks good, thanks.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: [PATCH 0/3] libubi / mkfs.ubifs #2

[Artem Bityutskiy]

On Sat, 2009-05-09 at 11:41 +0200, Corentin Chary wrote:
> These patchs will allow mkfs.ubifs to write the image directly
> on an UBI volume. This time libubi is used. I couldn't manage to
> make the Makefile works correctly, even with Mike trick for dependencies.
> Mike, could you help me on that ?
> You can still make it compile with "cd ubi-utils; make; cd ../mkfs.ubifs; make"

BTW, I've finally push a big patch which adds a new 'mtdinfo'
utility, and which makes 'limbtd' be sysfs-aware, which will
make users' life easier.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: [PATCH 0/3] libubi / mkfs.ubifs #2

[Mike Frysinger]

On Sat, May 9, 2009 at 05:41, Corentin Chary wrote:
> These patchs will allow mkfs.ubifs to write the image directly
> on an UBI volume. This time libubi is used. I couldn't manage to
> make the Makefile works correctly, even with Mike trick for dependencies.
> Mike, could you help me on that ?
> You can still make it compile with "cd ubi-utils; make; cd ../mkfs.ubifs; make"

is there a git tree with these patches sitting in them ?
-mike

Re: [PATCH 0/3] libubi / mkfs.ubifs #2

[Corentin Chary]

On Mon, May 11, 2009 at 9:37 AM, Mike Frysinger <vapier.adi@gmail.com> wrote:
> On Sat, May 9, 2009 at 05:41, Corentin Chary wrote:
>> These patchs will allow mkfs.ubifs to write the image directly
>> on an UBI volume. This time libubi is used. I couldn't manage to
>> make the Makefile works correctly, even with Mike trick for dependencies.
>> Mike, could you help me on that ?
>> You can still make it compile with "cd ubi-utils; make; cd ../mkfs.ubifs; make"
>
> is there a git tree with these patches sitting in them ?
> -mike

Yep
http://git.iksaif.net/?p=users/iksaif/mtd-utils.git;a=shortlog;h=refs/heads/for-upstream
git://git.iksaif.net/users/iksaif/mtd-utils.git for-upstream


-- 
Corentin Chary
http://xf.iksaif.net - http://uffs.org

Re: [PATCH 0/3] libubi / mkfs.ubifs #2

[Artem Bityutskiy]

On Sat, 2009-05-09 at 11:41 +0200, Corentin Chary wrote:
> These patchs will allow mkfs.ubifs to write the image directly
> on an UBI volume. This time libubi is used. I couldn't manage to
> make the Makefile works correctly, even with Mike trick for dependencies.
> Mike, could you help me on that ?
> You can still make it compile with "cd ubi-utils; make; cd ../mkfs.ubifs; make"

Just pushed the first 2 of your patches. Also, there is
a similar problem in libmtd - I've fixed that too. And
I walked through all memset calls and amended/fixed them
as well.

Thanks. Will look at the third patch later.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)