[PATCH 0/2] RFC: Add support for UBI and UBIFS

[PATCH 0/2] RFC: Add support for UBI and UBIFS

[Corentin Chary]

Hi,
Here is two patch to add support for UBI and UBIFS in libblkid
With that, if you type "blkid" it will show you UBIFS filesystem.

The thing is that I'm not sure this patch is really usefull as UBIFS
is a "nodev" filesystem, and it doesn't deal with block devices.

I first started theses patch to be able to do stuff like:
mount /dev/ubi0_0 /mnt/test but we need to patch UBIFS for that.

So, do you think we should be able to do that ? If it's ok, these
two patch are a first step.

Thanks

Corentin Chary (2):
  blkid: add UBI volume support
  blkid: add ubifs support

 lib/blkid/blkidP.h  |    1 +
 lib/blkid/devname.c |   56 ++++++++++++++++++++++++++++++++++-
 lib/blkid/probe.c   |   14 +++++++++
 lib/blkid/probe.h   |   81 +++++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 151 insertions(+), 1 deletions(-)

[PATCH 1/2] blkid: add UBI volume support

[Corentin Chary]

Probe UBI volume under /dev (or /devfs, /devices).
ubi_ctrl skip is hardcoded, maybe we should find a
cleaner way to do that.

Signed-off-by: Corentin Chary <corentincj@iksaif.net>
---
 lib/blkid/blkidP.h  |    1 +
 lib/blkid/devname.c |   56 ++++++++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 56 insertions(+), 1 deletions(-)

diff --git a/lib/blkid/blkidP.h b/lib/blkid/blkidP.h
index e0f11a0..baf82aa 100644
--- a/lib/blkid/blkidP.h
+++ b/lib/blkid/blkidP.h
@@ -117,6 +117,7 @@ extern char *blkid_strndup(const char *s, const int length);
 /*
  * Priority settings for different types of devices
  */
+#define BLKID_PRI_UBI	50
 #define BLKID_PRI_DM	40
 #define BLKID_PRI_EVMS	30
 #define BLKID_PRI_LVM	20
diff --git a/lib/blkid/devname.c b/lib/blkid/devname.c
index b151354..70e85c5 100644
--- a/lib/blkid/devname.c
+++ b/lib/blkid/devname.c
@@ -229,7 +229,9 @@ static void probe_one(blkid_cache cache, const char *ptname,
 		    dev->bid_devno == devno)
 			goto set_pri;
 
-		if (stat(device, &st) == 0 && S_ISBLK(st.st_mode) &&
+		if (stat(device, &st) == 0 &&
+		    (S_ISBLK(st.st_mode) ||
+		     (S_ISCHR(st.st_mode) && !strncmp(ptname, "ubi", 3))) &&
 		    st.st_rdev == devno) {
 			devname = blkid_strdup(device);
 			goto get_dev;
@@ -388,6 +390,57 @@ evms_probe_all(blkid_cache cache, int only_if_new)
 	return num;
 }
 
+static void
+ubi_probe_all(blkid_cache cache, int only_if_new)
+{
+	const char **dirname;
+
+	for (dirname = dirlist; *dirname; dirname++) {
+		DBG(DEBUG_DEVNAME, printf("probing UBI volumes under %s\n",
+					  *dirname));
+
+		DIR		*dir;
+		struct dirent	*iter;
+
+		dir = opendir(*dirname);
+		if (dir == NULL)
+			continue ;
+
+		while ((iter = readdir(dir)) != NULL) {
+			char		*name, *device;
+			struct stat	st;
+			dev_t		dev;
+
+			name = iter->d_name;
+
+			if (!strcmp(name, ".") || !strcmp(name, "..") ||
+			    !strstr(name, "ubi"))
+				continue;
+			if (!strcmp(name, "ubi_ctrl"))
+				continue;
+			device = malloc(strlen(*dirname) + strlen(name) + 2);
+			if (!device)
+				break ;
+			sprintf(device, "%s/%s", *dirname, name);
+			if (stat(device, &st))
+				break ;
+
+			if (!(st.st_rdev & 0xFF)) { // It's an UBI Device
+				free(device);
+				continue ;
+			}
+			dev = st.st_rdev;
+			DBG(DEBUG_DEVNAME, printf("UBI vol %s: devno 0x%04X\n",
+						  device,
+						  (int) dev));
+			probe_one(cache, name, dev, BLKID_PRI_UBI,
+				  only_if_new);
+			free(device);
+		}
+		closedir(dir);
+	}
+}
+
 /*
  * Read the device data for all available block devices in the system.
  */
@@ -419,6 +472,7 @@ static int probe_all(blkid_cache cache, int only_if_new)
 #ifdef VG_DIR
 	lvm_probe_all(cache, only_if_new);
 #endif
+	ubi_probe_all(cache, only_if_new);
 
 	proc = fopen(PROC_PARTITIONS, "r");
 	if (!proc)
-- 
1.6.3.3

[PATCH 2/2] blkid: add ubifs support

[Corentin Chary]

Signed-off-by: Corentin Chary <corentincj@iksaif.net>
---
 lib/blkid/probe.c |   14 +++++++++
 lib/blkid/probe.h |   81 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 95 insertions(+), 0 deletions(-)

diff --git a/lib/blkid/probe.c b/lib/blkid/probe.c
index 8f6cfa6..86278cb 100644
--- a/lib/blkid/probe.c
+++ b/lib/blkid/probe.c
@@ -1384,6 +1384,19 @@ static int probe_btrfs(struct blkid_probe *probe,
 	set_uuid(probe->dev, bs->fsid, 0);
 	return 0;
 }
+
+static int probe_ubifs(struct blkid_probe *probe,
+		       struct blkid_magic *id,
+		       unsigned char *buf)
+{
+	struct ubifs_sb_node *sb;
+
+	sb = (struct ubifs_sb_node *)buf;
+
+	set_uuid(probe->dev, sb->uuid, 0);
+	return 0;
+}
+
 /*
  * Various filesystem magics that we can check for.  Note that kboff and
  * sboff are in kilobytes and bytes respectively.  All magics are in
@@ -1391,6 +1404,7 @@ static int probe_btrfs(struct blkid_probe *probe,
  */
 static struct blkid_magic type_array[] = {
 /*  type     kboff   sboff len  magic			probe */
+  { "ubifs",	 0,      0,  4, "\x31\x18\x10\x06",	probe_ubifs },
   { "oracleasm", 0,	32,  8, "ORCLDISK",		probe_oracleasm },
   { "ntfs",	 0,	 3,  8, "NTFS    ",		probe_ntfs },
   { "jbd",	 1,   0x38,  2, "\123\357",		probe_jbd },
diff --git a/lib/blkid/probe.h b/lib/blkid/probe.h
index 37fc9c0..f71c224 100644
--- a/lib/blkid/probe.h
+++ b/lib/blkid/probe.h
@@ -725,6 +725,87 @@ struct btrfs_super_block {
 	__u8 sys_chunk_array[BTRFS_SYSTEM_CHUNK_ARRAY_SIZE];
 } __attribute__ ((__packed__));
 
+/**
+ * struct ubifs_ch - common header node.
+ * @magic: UBIFS node magic number (%UBIFS_NODE_MAGIC)
+ * @crc: CRC-32 checksum of the node header
+ * @sqnum: sequence number
+ * @len: full node length
+ * @node_type: node type
+ * @group_type: node group type
+ * @padding: reserved for future, zeroes
+ *
+ * Every UBIFS node starts with this common part. If the node has a key, the
+ * key always goes next.
+ */
+struct ubifs_ch {
+	__u32 magic;
+	__u32 crc;
+	__u64 sqnum;
+	__u32 len;
+	__u8 node_type;
+	__u8 group_type;
+	__u8 padding[2];
+} __attribute__ ((packed));
+
+/**
+ * struct ubifs_sb_node - superblock node.
+ * @ch: common header
+ * @padding: reserved for future, zeroes
+ * @key_hash: type of hash function used in keys
+ * @key_fmt: format of the key
+ * @flags: file-system flags (%UBIFS_FLG_BIGLPT, etc)
+ * @min_io_size: minimal input/output unit size
+ * @leb_size: logical eraseblock size in bytes
+ * @leb_cnt: count of LEBs used by file-system
+ * @max_leb_cnt: maximum count of LEBs used by file-system
+ * @max_bud_bytes: maximum amount of data stored in buds
+ * @log_lebs: log size in logical eraseblocks
+ * @lpt_lebs: number of LEBs used for lprops table
+ * @orph_lebs: number of LEBs used for recording orphans
+ * @jhead_cnt: count of journal heads
+ * @fanout: tree fanout (max. number of links per indexing node)
+ * @lsave_cnt: number of LEB numbers in LPT's save table
+ * @fmt_version: UBIFS on-flash format version
+ * @default_compr: default compression algorithm (%UBIFS_COMPR_LZO, etc)
+ * @padding1: reserved for future, zeroes
+ * @rp_uid: reserve pool UID
+ * @rp_gid: reserve pool GID
+ * @rp_size: size of the reserved pool in bytes
+ * @padding2: reserved for future, zeroes
+ * @time_gran: time granularity in nanoseconds
+ * @uuid: UUID generated when the file system image was created
+ * @ro_compat_version: UBIFS R/O compatibility version
+ */
+struct ubifs_sb_node {
+	struct ubifs_ch ch;
+	__u8 padding[2];
+	__u8 key_hash;
+	__u8 key_fmt;
+	__u32 flags;
+	__u32 min_io_size;
+	__u32 leb_size;
+	__u32 leb_cnt;
+	__u32 max_leb_cnt;
+	__u64 max_bud_bytes;
+	__u32 log_lebs;
+	__u32 lpt_lebs;
+	__u32 orph_lebs;
+	__u32 jhead_cnt;
+	__u32 fanout;
+	__u32 lsave_cnt;
+	__u32 fmt_version;
+	__u16 default_compr;
+	__u8 padding1[2];
+	__u32 rp_uid;
+	__u32 rp_gid;
+	__u64 rp_size;
+	__u32 time_gran;
+	__u8 uuid[16];
+	__u32 ro_compat_version;
+	__u8 padding2[3968];
+} __attribute__ ((packed));
+
 /*
  * Byte swap functions
  */
-- 
1.6.3.3

Re: [PATCH 0/2] RFC: Add support for UBI and UBIFS

[Artem Bityutskiy]

On Wed, 2009-07-15 at 11:38 +0200, Corentin Chary wrote:
> Hi,
> Here is two patch to add support for UBI and UBIFS in libblkid
> With that, if you type "blkid" it will show you UBIFS filesystem.
> 
> The thing is that I'm not sure this patch is really usefull as UBIFS
> is a "nodev" filesystem, and it doesn't deal with block devices.
> 
> I first started theses patch to be able to do stuff like:
> mount /dev/ubi0_0 /mnt/test but we need to patch UBIFS for that.

Should not be difficult to implement in UBIFS.

> So, do you think we should be able to do that ? If it's ok, these
> two patch are a first step.

No comments from my side, I do not use the library so I cannot
really comment on this.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: [PATCH 0/2] RFC: Add support for UBI and UBIFS

[Corentin Chary]

On Thu, Jul 16, 2009 at 1:57 PM, Artem Bityutskiy<dedekind@infradead.org> wrote:
> On Wed, 2009-07-15 at 11:38 +0200, Corentin Chary wrote:
>> Hi,
>> Here is two patch to add support for UBI and UBIFS in libblkid
>> With that, if you type "blkid" it will show you UBIFS filesystem.
>>
>> The thing is that I'm not sure this patch is really usefull as UBIFS
>> is a "nodev" filesystem, and it doesn't deal with block devices.
>>
>> I first started theses patch to be able to do stuff like:
>> mount /dev/ubi0_0 /mnt/test but we need to patch UBIFS for that.
>
> Should not be difficult to implement in UBIFS.
>
>> So, do you think we should be able to do that ? If it's ok, these
>> two patch are a first step.
>
> No comments from my side, I do not use the library so I cannot
> really comment on this.
>

AFAIK mount(8) use this library to find the associated filesystem, so
it's needed
to remove the "-t ubifs".

-- 
Corentin Chary
http://xf.iksaif.net - http://uffs.org

Re: [PATCH 0/2] RFC: Add support for UBI and UBIFS

[Artem Bityutskiy]

On Thu, 2009-07-16 at 14:15 +0200, Corentin Chary wrote:
> On Thu, Jul 16, 2009 at 1:57 PM, Artem Bityutskiy<dedekind@infradead.org> wrote:
> > On Wed, 2009-07-15 at 11:38 +0200, Corentin Chary wrote:
> >> Hi,
> >> Here is two patch to add support for UBI and UBIFS in libblkid
> >> With that, if you type "blkid" it will show you UBIFS filesystem.
> >>
> >> The thing is that I'm not sure this patch is really usefull as UBIFS
> >> is a "nodev" filesystem, and it doesn't deal with block devices.
> >>
> >> I first started theses patch to be able to do stuff like:
> >> mount /dev/ubi0_0 /mnt/test but we need to patch UBIFS for that.
> >
> > Should not be difficult to implement in UBIFS.
> >
> >> So, do you think we should be able to do that ? If it's ok, these
> >> two patch are a first step.
> >
> > No comments from my side, I do not use the library so I cannot
> > really comment on this.
> >
> 
> AFAIK mount(8) use this library to find the associated filesystem, so
> it's needed
> to remove the "-t ubifs".

Ah, this would be really cool to have then!

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: [PATCH 0/2] RFC: Add support for UBI and UBIFS

[Artem Bityutskiy]

On 07/15/2009 12:38 PM, Corentin Chary wrote:
> Hi,
> Here is two patch to add support for UBI and UBIFS in libblkid
> With that, if you type "blkid" it will show you UBIFS filesystem.
>
> The thing is that I'm not sure this patch is really usefull as UBIFS
> is a "nodev" filesystem, and it doesn't deal with block devices.
>
> I first started theses patch to be able to do stuff like:
> mount /dev/ubi0_0 /mnt/test but we need to patch UBIFS for that.
>
> So, do you think we should be able to do that ? If it's ok, these
> two patch are a first step.

Theo,

what do you think about this? The feature sounds very useful for
our small comunity.

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

Re: [PATCH 0/2] RFC: Add support for UBI and UBIFS

[Theodore Tso]

On Wed, Jul 15, 2009 at 11:38:42AM +0200, Corentin Chary wrote:
> Hi,
> Here is two patch to add support for UBI and UBIFS in libblkid
> With that, if you type "blkid" it will show you UBIFS filesystem.
> 
> The thing is that I'm not sure this patch is really usefull as UBIFS
> is a "nodev" filesystem, and it doesn't deal with block devices.
> 
> I first started theses patch to be able to do stuff like:
> mount /dev/ubi0_0 /mnt/test but we need to patch UBIFS for that.
> 
> So, do you think we should be able to do that ? If it's ok, these
> two patch are a first step.

Sorry for the delay in getting back to you; things have been pretty
busy this past week or two.  Can you get me small (preferably under
200k) sample filesystem images for the purposes of regression testing?

Also, the blkid library is in the process of being transferred to
util-linux-ng.  So I'm willing to apply the patches, but they also
need to be ported to the version of the blkid library in util-linux-ng
to make sure this feature doesn't get lost when distributions start
transitioning to util-linux-ng.

Regards,

					- Ted

Re: [PATCH 0/2] RFC: Add support for UBI and UBIFS

[Corentin Chary]

[-- Attachment #1: Type: text/plain, Size: 1485 bytes --]

On Mon, Jul 27, 2009 at 1:07 AM, Theodore Tso<tytso@mit.edu> wrote:
> On Wed, Jul 15, 2009 at 11:38:42AM +0200, Corentin Chary wrote:
>> Hi,
>> Here is two patch to add support for UBI and UBIFS in libblkid
>> With that, if you type "blkid" it will show you UBIFS filesystem.
>>
>> The thing is that I'm not sure this patch is really usefull as UBIFS
>> is a "nodev" filesystem, and it doesn't deal with block devices.
>>
>> I first started theses patch to be able to do stuff like:
>> mount /dev/ubi0_0 /mnt/test but we need to patch UBIFS for that.
>>
>> So, do you think we should be able to do that ? If it's ok, these
>> two patch are a first step.
>
> Sorry for the delay in getting back to you; things have been pretty
> busy this past week or two.  Can you get me small (preferably under
> 200k) sample filesystem images for the purposes of regression testing?

No problem. Here is a sample image with the default filesystem.
Maybe Artem can provide more interesting image for regression testing.

> Also, the blkid library is in the process of being transferred to
> util-linux-ng.  So I'm willing to apply the patches, but they also
> need to be ported to the version of the blkid library in util-linux-ng
> to make sure this feature doesn't get lost when distributions start
> transitioning to util-linux-ng.

Ok, I'll port the patch and send them to util-linux-ng ASAP.

Thanks,

-- 
Corentin Chary
http://xf.iksaif.net - http://uffs.org

[-- Attachment #2: ubifs.img.gz --]
[-- Type: application/x-gzip, Size: 742 bytes --]

Re: [PATCH 0/2] RFC: Add support for UBI and UBIFS

[Artem Bityutskiy]

On 07/27/2009 10:34 AM, Corentin Chary wrote:
> No problem. Here is a sample image with the default filesystem.
> Maybe Artem can provide more interesting image for regression testing.

Just few files and dirs there and it should be fine.

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

Re: [PATCH 0/2] RFC: Add support for UBI and UBIFS

[Corentin Chary]

On Sunday 09 August 2009 06:57:20 Artem Bityutskiy wrote:
> On 07/27/2009 10:34 AM, Corentin Chary wrote:
> > No problem. Here is a sample image with the default filesystem.
> > Maybe Artem can provide more interesting image for regression testing.
>
> Just few files and dirs there and it should be fine.

Hi,
Patchs are done for both e2fs-utils and util-linux-ng but I don't have the 
time to send them right now. Except them in two weeks when I'm back ;).
Thanks,
-- 
Corentin Chary
http://xf.iksaif.net - http://uffs.org