[uml-devel] Thoughts on HUMFS and metadata

[uml-devel] Thoughts on HUMFS and metadata

[Paul Wagland]

[-- Attachment #1: Type: text/plain, Size: 1753 bytes --]

Hi all,

Just a bit of brain dumping here... i hope that it is all coherent, if  
not, please let me know and I will try to clarify :-)

I have read through  
<http://marc.theaimsgroup.com/?l=user-mode-linux- 
devel&m=108139168611257&w=4> jdikes "A quick humfs HOWTO", but I still  
have a few questions and/or suggestions.

1. Do files really need to exist in data if they are block, character  
or sockets? Surely it is enough to just look at the metadata for these  
files and create the appropriate dir entry in the UML. Unless it is  
being used for for the host systems inode number? If so, is this only  
for NFS?

2. When COW humfs "partitions" are made, will it be possible to  
separately copy metadata and data? For instance, if I change only the  
uid, gid and/or permissions, do I need to also create a new copy of the  
data, or can I only update my local metadata and use the data from the  
parent?

3. When COW humfs "partitions" are made, how will files be deleted in  
the second humfs? I would assume (but could easily be wrong ;-) ) that  
the easiest way to do COW is for every file access, first check your  
humfs, if it does not exist, then check your parent humfs. If this is  
done, then how do you delete a file? I can see that one easy option is  
to introduce a "deleted" metadata node, that only contains the word  
deleted, and would signify that this file does not exist in this COWed  
copy.

4. Since metadata is basically a copy of "permission data" does it also  
make sense for humfs metadata to include things like capabilities and  
ACL's? (Obviously I think it does, or I would not of raise the question  
;-) )

So, what do you think, insightful thoughts or meandering ravings of a  
crazed mind?

Cheers,
Paul

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 186 bytes --]

Re: [uml-devel] Thoughts on HUMFS and metadata

[Henrik Nordstrom]

On Mon, 19 Apr 2004, Paul Wagland wrote:

> 2. When COW humfs "partitions" are made, will it be possible to  
> separately copy metadata and data?

This makes me remember the Overlay Filesystem 
<url:http://ovlfs.sourceforge.net>. 

Regards
Henrik



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Thoughts on HUMFS and metadata

[Henrik Nordstrom]

On Mon, 19 Apr 2004, Henrik Nordstrom wrote:

> On Mon, 19 Apr 2004, Paul Wagland wrote:
> 
> > 2. When COW humfs "partitions" are made, will it be possible to  
> > separately copy metadata and data?
> 
> This makes me remember the Overlay Filesystem 
> <url:http://ovlfs.sourceforge.net>. 

Looked around a little and there is several of these..

ovlfs Overlay Filesystem
<url:http://ovlfs.sourceforge.net>

translucency lkm
<url:http://translucency.sourceforge.net/>

mini_fo Mini Fan-Out filesystems
<url:http://www.denx.de>
There is also a good technical paper on the subject.


Of these the mini_fo looks very promising.


A nice collection on information regarding overlay or template filesystems
can also be found at the TBVFS Virtual Gathering
<url:http://vserver.13thfloor.at/TBVFS/>

Regards
Henrik



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Thoughts on HUMFS and metadata

[Paul Wagland]

[-- Attachment #1: Type: text/plain, Size: 2211 bytes --]


On Apr 19, 2004, at 20:58, Jeff Dike wrote:

> paul@kungfoocoder.org said:
>> 1. Do files really need to exist in data if they are block, character
>>  or sockets? Surely it is enough to just look at the metadata for
>> these   files and create the appropriate dir entry in the UML.
>
> Permissions are stored with the real file.  This makes "metadata" a 
> bit of a
> misnomer - I just moved the information that absolutely needed to move.

Umm, is that actually valid? For example, a normal user cannot read a 
file with "000" permissions, however root can. However, if the 
permissions are stored on the file, then a "000" file cannot even be 
read be root.

>> 2. When COW humfs "partitions" are made, will it be possible to
>> separately copy metadata and data? For instance, if I change only the
>>  uid, gid and/or permissions, do I need to also create a new copy of
>> the   data, or can I only update my local metadata and use the data
>> from the   parent?
>
> A suitable implementation could optimize the COWing of permissions and 
> other
> non-data.  I'm not sure how worth-while that would be.

Agreed that normally it would not be a large win, but I expect that it 
would be a small price in code for a potential win... though whether it 
is worth having code sitting around that is almost never used...

>> 3. When COW humfs "partitions" are made, how will files be deleted in
>>  the second humfs? [...] I can see that one easy
>> option is   to introduce a "deleted" metadata node, that only contains
>> the word   deleted, and would signify that this file does not exist in
>> this COWed   copy.
>
> That's the plan, more or less.

Cool.

>> 4. Since metadata is basically a copy of "permission data" does it
>> also   make sense for humfs metadata to include things like
>> capabilities and   ACL's?
>
> If they are stored in the filesystem, then it might make sense.

Surely it also makes sense if it is stored in a database? I am not sure 
how else this information can be stored otherwise? I am assuming that 
the various metadata backends will all store the same metadata, just 
with different complexities against different perceived needs.

Just a few more thoughts :-)

Cheers,
Paul

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 186 bytes --]

Re: [uml-devel] Thoughts on HUMFS and metadata

[Jeff Dike]

paul@kungfoocoder.org said:
> 1. Do files really need to exist in data if they are block, character
>  or sockets? Surely it is enough to just look at the metadata for
> these   files and create the appropriate dir entry in the UML. 

Permissions are stored with the real file.  This makes "metadata" a bit of a 
misnomer - I just moved the information that absolutely needed to move.

> 2. When COW humfs "partitions" are made, will it be possible to
> separately copy metadata and data? For instance, if I change only the
>  uid, gid and/or permissions, do I need to also create a new copy of
> the   data, or can I only update my local metadata and use the data
> from the   parent?

A suitable implementation could optimize the COWing of permissions and other
non-data.  I'm not sure how worth-while that would be.

> 3. When COW humfs "partitions" are made, how will files be deleted in
>  the second humfs? I would assume (but could easily be wrong ;-) )
> that   the easiest way to do COW is for every file access, first check
> your   humfs, if it does not exist, then check your parent humfs. If
> this is   done, then how do you delete a file? I can see that one easy
> option is   to introduce a "deleted" metadata node, that only contains
> the word   deleted, and would signify that this file does not exist in
> this COWed   copy.

That's the plan, more or less.

> 4. Since metadata is basically a copy of "permission data" does it
> also   make sense for humfs metadata to include things like
> capabilities and   ACL's? 

If they are stored in the filesystem, then it might make sense.

				Jeff



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Thoughts on HUMFS and metadata

[Paul Wagland]

[-- Attachment #1: Type: text/plain, Size: 1812 bytes --]


On Apr 19, 2004, at 22:15, Jeff Dike wrote:

> paul@kungfoocoder.org said:
>> Umm, is that actually valid? For example, a normal user cannot read a
>> file with "000" permissions, however root can. However, if the
>> permissions are stored on the file, then a "000" file cannot even be
>> read be root.
>
> Ummm, good point.  That needs fixing, as well as the fact that hard 
> links aren't
> supported correctly.

Hard links are a hard problem... pun intended ;-) So long as we can 
assume that only UML modifies the humfs

>> Surely it also makes sense if it is stored in a database? I am not
>> sure  how else this information can be stored otherwise?
>
> Are you talking about extending the normal unix permissions through 
> humfs,
> or supporting this stuff on filesystems that have it?

Unfortunately, as for permissions above, I don't think that we can 
store this stuff on the filesystem, even if the underlying filesystem 
does support it. Since all file accesses are done as the user running 
UML, you certainly cannot store any capability that the user does not 
have, which means, for example, that we cannot give the modprobe 
utility the "load module" capability. On some hardened systems this 
program is the only thing that can load a module, and then root is the 
only user that is allowed to run that program. Personally I think that 
then ACL's become analogous to the above situation for permissions, and 
they might as well be treated in the same way for simplicity sakes.

Note, that since not all FS's are required to support ACL/capabilities, 
they do not need to be in the first cut, so long as the metadata format 
allows for their existence then it can be added later on, indeed it 
sounds like a nice small chunk of work that I would even do for you :-)

Hope this helps some,
Paul

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 186 bytes --]

Re: [uml-devel] Thoughts on HUMFS and metadata

[Jeff Dike]

paul@kungfoocoder.org said:
> Umm, is that actually valid? For example, a normal user cannot read a
> file with "000" permissions, however root can. However, if the
> permissions are stored on the file, then a "000" file cannot even be
> read be root.

Ummm, good point.  That needs fixing, as well as the fact that hard links aren't
supported correctly.

> Surely it also makes sense if it is stored in a database? I am not
> sure  how else this information can be stored otherwise?

Are you talking about extending the normal unix permissions through humfs,
or supporting this stuff on filesystems that have it?

				Jeff



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Thoughts on HUMFS and metadata

[Paul Wagland]

[-- Attachment #1: Type: text/plain, Size: 817 bytes --]


On Apr 19, 2004, at 21:56, Paul Wagland wrote:
> On Apr 19, 2004, at 22:15, Jeff Dike wrote:
>> paul@kungfoocoder.org said:
>>> Umm, is that actually valid? For example, a normal user cannot read a
>>> file with "000" permissions, however root can. However, if the
>>> permissions are stored on the file, then a "000" file cannot even be
>>> read be root.
>>
>> Ummm, good point.  That needs fixing, as well as the fact that hard 
>> links aren't
>> supported correctly.
>
> Hard links are a hard problem... pun intended ;-) So long as we can 
> assume that only UML modifies the humfs

Heh? One day I will learn to talk in complete sentences :-)

I meant to delete the bit above, and ask why aren't hard links 
supported correctly? I am not sure what the file permissions have to do 
with hard links?

Cheers,
Paul

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 186 bytes --]

Re: [uml-devel] Thoughts on HUMFS and metadata

[Jeff Dike]

paul@wagland.net said:
> I am not sure what the file permissions have to do  with hard links?

Nothing, except it's one more thing that needs fixing.

>  ask why aren't hard links  supported correctly? 

See my latest diary entry for the gory details...
	http://user-mode-linux.sourceforge.net/diary.html

				Jeff



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel