[PATCH] mkfs.ubifs: support --squash-uids

[PATCH] mkfs.ubifs: support --squash-uids

[Hamish Moffatt]

mkfs.jffs2 and genext2fs both have a -U/--squash-uids switch which make
them ignore the uid/gids in the directory and set them to 0:0 instead.

Here's my implementation for mkfs.ubifs.

Signed-off-by: Hamish Moffatt <hamish@cloud.net.au>

--
Those tools also have a -P/--squash-perms switch which I haven't
implemented due to lack of need.

Index: mkfs.ubifs/mkfs.ubifs.c
===================================================================
--- mkfs.ubifs/mkfs.ubifs.c	(revision 4400)
+++ mkfs.ubifs/mkfs.ubifs.c	(working copy)
@@ -98,6 +98,7 @@
 static struct stat root_st;
 static char *output;
 static int out_fd;
+static int squash_owner;
 
 /* The 'head' (position) which nodes are written */
 static int head_lnum;
@@ -120,7 +121,7 @@
 /* Inode creation sequence number */
 static unsigned long long creat_sqnum;
 
-static const char *optstring = "d:r:m:o:D:h?vVe:c:g:f:p:k:x:j:l:j:";
+static const char *optstring = "d:r:m:o:D:h?vVe:c:g:f:p:k:x:j:l:j:U";
 
 static const struct option longopts[] = {
 	{"root"        , 1, NULL, 'r'},
@@ -139,6 +140,7 @@
 	{"keyhash"     , 1, NULL, 'k'},
 	{"log-lebs"    , 1, NULL, 'l'},
 	{"orph-lebs"   , 1, NULL, 'p'},
+	{"squash-uids" , 0, NULL, 'U'},
 	{NULL, 0, NULL, 0}
 };
 
@@ -159,6 +161,7 @@
 "  -l, --log-lebs=COUNT    Use COUNT erase blocks for the log\n"
 "  -p, --orph-lebs=COUNT   Use COUNT erase blocks for orphans (default: 1)\n"
 "  -D, --devtable=FILE     Use device table FILE\n"
+"  -U, --squash-uids       Squash owners making all files owned by root\n"
 "  -v, --verbose           Verbose operation\n"
 "  -V, --version           Display version information\n"
 "  -g, --debug=LEVEL       Display debug information (0 - none, 1 - statistics,\n"
@@ -509,6 +512,9 @@
 			if (c->max_bud_bytes < 0)
 				return -1;
 			break;
+		case 'U':
+			squash_owner = 1;
+			break;
 		}
 	}
 
@@ -885,8 +891,8 @@
 	ino->atime.sec  = cpu_to_le32(st->st_atime);
 	ino->ctime.sec  = cpu_to_le32(st->st_ctime);
 	ino->mtime.sec  = cpu_to_le32(st->st_mtime);
-	ino->uid        = cpu_to_le32(st->st_uid);
-	ino->gid        = cpu_to_le32(st->st_gid);
+	ino->uid        = cpu_to_le32((squash_owner) ? 0 : st->st_uid);
+	ino->gid        = cpu_to_le32((squash_owner) ? 0 : st->st_gid);
 	ino->mode       = cpu_to_le32(st->st_mode);
 	ino->flags      = cpu_to_le32(use_flags);
 	ino->data_len   = cpu_to_le32(data_len);
-- 
Hamish Moffatt VK3SB <hamish@debian.org> <hamish@cloud.net.au>

Re: [PATCH] mkfs.ubifs: support --squash-uids

[Artem Bityutskiy]

On Fri, 2008-05-02 at 15:16 +1000, Hamish Moffatt wrote:
> mkfs.jffs2 and genext2fs both have a -U/--squash-uids switch which make
> them ignore the uid/gids in the directory and set them to 0:0 instead.
> 
> Here's my implementation for mkfs.ubifs.
> 
> Signed-off-by: Hamish Moffatt <hamish@cloud.net.au>
> 

Thanks for the patch. Before applying it, we'd like to know what do you
expect to happen if -U is used simultaneously with the device table
(-D). Should we use what device table says for directories which are
created because they are specified in the device table, or we should use
0 anyway? Or we should just prohibit simultaneous usage of -U and -D ?

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: [PATCH] mkfs.ubifs: support --squash-uids

[Hamish Moffatt]

On Mon, May 05, 2008 at 10:31:12AM +0300, Artem Bityutskiy wrote:
> 
> On Fri, 2008-05-02 at 15:16 +1000, Hamish Moffatt wrote:
> > mkfs.jffs2 and genext2fs both have a -U/--squash-uids switch which make
> > them ignore the uid/gids in the directory and set them to 0:0 instead.
> > 
> > Here's my implementation for mkfs.ubifs.
> > 
> > Signed-off-by: Hamish Moffatt <hamish@cloud.net.au>
> > 
> 
> Thanks for the patch. Before applying it, we'd like to know what do you
> expect to happen if -U is used simultaneously with the device table
> (-D). Should we use what device table says for directories which are
> created because they are specified in the device table, or we should use
> 0 anyway? Or we should just prohibit simultaneous usage of -U and -D ?

Good point. mkfs.jffs2 (from my reading of the source code) squashes 
ownership of files read from the specified root directory, but the
ownerships specified in the device table are still used. I think that
makes sense, because you can always specify 0:0 in the device table if
that's what you need.

However it's not what is implemented by my patch :-) Would you like me
to fix it up and resubmit, or will you? I applied the squashing at the
lowest level without realizing the effect on the device table.

thanks,
Hamish
-- 
Hamish Moffatt VK3SB <hamish@debian.org> <hamish@cloud.net.au>

Re: [PATCH] mkfs.ubifs: support --squash-uids

[Artem Bityutskiy]

On Mon, 2008-05-05 at 18:46 +1000, Hamish Moffatt wrote:
> Good point. mkfs.jffs2 (from my reading of the source code) squashes 
> ownership of files read from the specified root directory, but the
> ownerships specified in the device table are still used. I think that
> makes sense, because you can always specify 0:0 in the device table if
> that's what you need.
> 
> However it's not what is implemented by my patch :-) Would you like me
> to fix it up and resubmit, or will you? I applied the squashing at the
> lowest level without realizing the effect on the device table.

I think I can do this, but a bit later. So if it is not urgent for you,
you may want to just wait.

Thanks.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: [PATCH] mkfs.ubifs: support --squash-uids

[Artem Bityutskiy]

On Mon, 2008-05-05 at 18:46 +1000, Hamish Moffatt wrote:
> > Thanks for the patch. Before applying it, we'd like to know what do you
> > expect to happen if -U is used simultaneously with the device table
> > (-D). Should we use what device table says for directories which are
> > created because they are specified in the device table, or we should use
> > 0 anyway? Or we should just prohibit simultaneous usage of -U and -D ?
> 
> Good point. mkfs.jffs2 (from my reading of the source code) squashes 
> ownership of files read from the specified root directory, but the
> ownerships specified in the device table are still used. I think that
> makes sense, because you can always specify 0:0 in the device table if
> that's what you need.
> 
> However it's not what is implemented by my patch :-) Would you like me
> to fix it up and resubmit, or will you? I applied the squashing at the
> lowest level without realizing the effect on the device table.

I've committed your patch and will do the "prefer dev. table" change
later. Just a note - the patch was against some old mkfs.ubifs, so I had
to do conflict resolution. Also note, we changed -p (--orph-lebs) option
to -P at some point. You are unlikely using it - this is just for
information.

Thank you!

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: [PATCH] mkfs.ubifs: support --squash-uids

[Hamish Moffatt]

On Tue, May 06, 2008 at 09:38:36AM +0300, Artem Bityutskiy wrote:
> I've committed your patch and will do the "prefer dev. table" change

Thanks.

> later. Just a note - the patch was against some old mkfs.ubifs, so I had
> to do conflict resolution. Also note, we changed -p (--orph-lebs) option
> to -P at some point. You are unlikely using it - this is just for
> information.

Yes it was against the v0.4 tag. 

mkfs.jffs2 and genext2fs both use -P as the short version of
--squash-perms, so mkfs.ubifs will have to use something different. No
big deal however (although they are pretty similar otherwise).

Also, what is the purpose of --max-leb-cnt? It seems to be just a safety
check that the volume will fit? It's a bit unusual.

thanks
Hamish
-- 
Hamish Moffatt VK3SB <hamish@debian.org> <hamish@cloud.net.au>

Re: [PATCH] mkfs.ubifs: support --squash-uids

[Artem Bityutskiy]

On Tue, 2008-05-06 at 19:46 +1000, Hamish Moffatt wrote:
> Yes it was against the v0.4 tag. 
> 
> mkfs.jffs2 and genext2fs both use -P as the short version of
> --squash-perms, so mkfs.ubifs will have to use something different. No
> big deal however (although they are pretty similar otherwise).

Hmm, ok, then I will make it -p back. I just wanted to reserve -p for
other purposes, but this is not important anymore. I'v pushed
corresponding patch.

> Also, what is the purpose of --max-leb-cnt? It seems to be just a safety
> check that the volume will fit? It's a bit unusual.

It is a form of specifying FS size. But instead of specifying exact FS
size, you specify maximum FS size. This means that it will be possible
to put the image on smaller volumes as well, which is quite handy
because due to random amount of initial NAND bad blocks different
instances of the same device may have a bit different volume sizes.
E.g., rootfs partition may be 500MiB or 501MiB on a 512MiB NAND,
depending on how many initial bad block NAND contains.

Fundamentally, mkfs.ubifs has to know FS size because UBIFS maintains
and stores per-LEB information (like amount of dirty and free space in
each LEB) in so-called LPT area on the media. So obviously, the size of
this area depends on the total amount of LEBs, i.e. on the volume size.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: [PATCH] mkfs.ubifs: support --squash-uids

[Artem Bityutskiy]

On Mon, 2008-05-05 at 11:52 +0300, Artem Bityutskiy wrote:
> > However it's not what is implemented by my patch :-) Would you like me
> > to fix it up and resubmit, or will you? I applied the squashing at the
> > lowest level without realizing the effect on the device table.
> 
> I think I can do this, but a bit later. So if it is not urgent for you,
> you may want to just wait.

Done.
-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: [PATCH] mkfs.ubifs: support --squash-uids

[Artem Bityutskiy]

On Tue, 2008-05-06 at 19:46 +1000, Hamish Moffatt wrote:
> Also, what is the purpose of --max-leb-cnt? It seems to be just a safety
> check that the volume will fit? It's a bit unusual.

Just added this FAQ entry:
http://www.linux-mtd.infradead.org/faq/ubifs.html#L_max_leb_cnt

Let me know if I did not put it clear enough there.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: [PATCH] mkfs.ubifs: support --squash-uids

[Hamish Moffatt]

On Tue, May 06, 2008 at 01:23:12PM +0300, Artem Bityutskiy wrote:
> Fundamentally, mkfs.ubifs has to know FS size because UBIFS maintains
> and stores per-LEB information (like amount of dirty and free space in
> each LEB) in so-called LPT area on the media. So obviously, the size of
> this area depends on the total amount of LEBs, i.e. on the volume size.

OK (and thanks for adding this information to the web site also.)

In my case I am preparing images with mkfs.ubifs, and mounting them
read-only on my embedded target. In that case, could the max LEB count
be calculated automatically to fit the files and the LPT area?

I added support for ubifs images into buildroot. The user must configure
their LEB size, max I/O size, and max LEB count as these are all
required for mkfs.ubifs. Is there a formula to calculate an approximate 
max LEB count from the volume size and LEB size? I will need to improve
the documentation for these parameters shown to the buildroot users
(although probably I'm the only one using the ubifs target so far!).

thanks
Hamish
-- 
Hamish Moffatt VK3SB <hamish@debian.org> <hamish@cloud.net.au>

Re: [PATCH] mkfs.ubifs: support --squash-uids

[Artem Bityutskiy]

On Thu, 2008-05-08 at 10:16 +1000, Hamish Moffatt wrote:
> On Tue, May 06, 2008 at 01:23:12PM +0300, Artem Bityutskiy wrote:
> > Fundamentally, mkfs.ubifs has to know FS size because UBIFS maintains
> > and stores per-LEB information (like amount of dirty and free space in
> > each LEB) in so-called LPT area on the media. So obviously, the size of
> > this area depends on the total amount of LEBs, i.e. on the volume size.
> 
> OK (and thanks for adding this information to the web site also.)
> 
> In my case I am preparing images with mkfs.ubifs, and mounting them
> read-only on my embedded target. In that case, could the max LEB count
> be calculated automatically to fit the files and the LPT area?
> 
> I added support for ubifs images into buildroot. The user must configure
> their LEB size, max I/O size, and max LEB count as these are all
> required for mkfs.ubifs. Is there a formula to calculate an approximate 
> max LEB count from the volume size and LEB size?

Err, max LEB count = count of LEBs in the volume = volume size in
bytes / LEB size. If you know volume size is the same on each device,
just stick with this formula. If you are not 100% sure about volume
size, or you want the FS to automatically re-size when you put it on
larger volumes, then max LEB count = maximum possible volume size in
LEBs.

E.g., you you know your flash is 512MiB, which is 4096 128KiB PEBs, then
you may safely use max LEB count = 4096, because it is absolutely
maximum number. Well, actually UBI will reserve some amount of PEBs for
its own use in any case, so it is better to use 4095.

4095 is better because power of 2 numbers are "boundaries". The larger
max. LEB count is, the more UBIFS reserves for LPT area, the more levels
the LPT tree have, the more bits UBIFS uses for keys and pointers in the
LPT tree. So specifying max LEB count 3000 or 4000 makes few difference
in terms of amount of resources you spend, but 4095 and 4096 do make
slight difference.

But it does not hurt much if you use 4096, that's fine too and you'll
unlikely notice any difference.

>  I will need to improve
> the documentation for these parameters shown to the buildroot users
> (although probably I'm the only one using the ubifs target so far!).

Cool. Feel free to ask for help.

Thanks.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)