[uml-devel] A quick humfs HOWTO

[uml-devel] A quick humfs HOWTO

[Jeff Dike]

As the readers of uml-devel are aware by now, there is a new virtual UML 
filesystem called humfs.  This is a hostfs-like filesystem which does ownerships
right, by separating them from the actual file, ala umsdos.

The reasons for humfs are described in my latest diary entry at 
	http://user-mode-linux.sourceforge.net/diary.html

This post describes how it works, and how to set it up.  This will be expanded
upon and added to the UML site.

Unlike a hostfs-mounted directory, a humfs filesystem has a definite root, and
must be prepared ahead of time.  There is a tool in the latest uml_utilities
called humfsify which does this for you, so don't use the following description
to set one up by hand.

The root contains two directories :
	data - which contains the files
	metadata - which shadows "data" and whose files contain ownership 
		information
and one file
	superblock - which contains information about the filesystem as a 
		whole

For plain files, the corresponding metadata file has the format
	"<uid> <gid>\n"
i.e. "0 0\n" for a uid root, gid root file.

For directories, the format is the same, except that ownership is in the 
"metadata" file in its metadata directory.  In the event that there is a real
"metadata" file, its ownerships will be on the second line of that file.

For example, the directory /foo/bar/ has both data/foo/bar/ and metadata/foo/bar
directories.  Its ownerships are in the file metadata/foo/bar/metadata.  If
there is a /foo/bar/metadata, then its ownerships are also in that file, which
would then look like
	"<directory uid> <directory gid>\n<file uid> <file gid>\n"

This second line thing isn't implemented yet, so don't go creating files called
metadata just now.

For character and block devices the data file exists, but is empty, and the
metadata file has the format
	"<uid> <gid> <type> <major> <minor>\n"

uid and gid are as above.  major and minor are self-explanatory.  type is
either of the characters 'b' and 'c', depending on whether it's a block or
character device.

Unix sockets are similar, except they have no major and minor numbers, and
have a type of 's' :
	"<uid> <gid> s\n"

The superblock file has the format
	"version <number>\nused <number>\ntotal <number>\n"

The version is currently 1.  used is set to the disk space consumed by 
everything in the data directory.  total is set arbitrarily when you set
up the filesystem.  It is the size of the filesystem as seen within UML.

humfsify is in the uml_utilities tarball in ./humfsify/humfsify.  Before running
it, you need to create an empty directory, then copy the filesystem you want
to mount to the "data" subdirectory of the new directory.  cd to the new
directory, and run humfsify as root:
	humfsify <user> <group> <size>

user and group are the uid and gid that UML will be running as, respectively.
size is the size of the filesystem as seen inside UML, specified in bytes.
The suffixes "K", "M", and "G" are accepted, and have the obvious meanings.

humfsify will walk the "data" hierarchy and create the "metadata" hierarchy
and the superblock file.

Once this is done, it is ready for use a UML which is version 2.4.24-2 or later.
To boot UML on this, enable CONFIG_HUMFS, and replace the ubd0=... argument
with 
	root=/dev/root rootflags=<path to humfs root> rootfstype=humfs

UML will boot as it usually does, and you will see a humfs rootfilesystem
when you log in.  All file operations should work as usual, and you should
be able to create files owned by multiple different users and have those
ownerships preserved.

				Jeff



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] A quick humfs HOWTO

[Nuno Silva]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jeff Dike wrote:

[..]

|
| The root contains two directories :
| 	data - which contains the files
| 	metadata - which shadows "data" and whose files contain ownership
| 		information
| and one file
| 	superblock - which contains information about the filesystem as a
| 		whole

Great news, compiling right now! :-)

In the meanwhile I'm thinking about this system and it seems that this
will eat lots of inodes because every file will be present twice in the
[host] filesystem with the metadata version using only a few bytes. Also
every "ls -la" (if it's not cached) in UML will generate a read in the
file's metafile file (say this 3 times quickly! eheheh). With 1000 files
in a directory thats lot's of fopen()s.

Wouldn't it be better to add support for a simple DB, like sleepycat or
CMU cyrus' project skiplist or even something lighter? This way we can
have only 2 files in the humfs "root", the superblock and the metadataDB.

This can save inodes and improve performance because you have only one
file that's indexed! :-)

The downsides:
- - another linker requirement at build time;
- - can't inspect and/or easily modify permissions with emacs. :-)

Any thoughts?

Regards,
Nuno Silva
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFAdPBSOPig54MP17wRAp8zAJ0UWKA5QvAdh4DJSzUfpV+Y5JCywwCgmhz/
S6aXJ9hLzCyJBBNlYSkOVSQ=
=EO+m
-----END PGP SIGNATURE-----


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] A quick humfs HOWTO

[Henrik Nordstrom]

On Thu, 8 Apr 2004, Nuno Silva wrote:

> In the meanwhile I'm thinking about this system and it seems that this
> will eat lots of inodes because every file will be present twice in the
> [host] filesystem with the metadata version using only a few bytes.

Indeed, so the host better use a filesystem which can handle this 
efficiently such as reiserfs if you plan on very large filesystems.

> Also every "ls -la" (if it's not cached) in UML will generate a read in
> the file's metafile file (say this 3 times quickly! eheheh). With 1000
> files in a directory thats lot's of fopen()s.

Indeed.

> Wouldn't it be better to add support for a simple DB, like sleepycat or
> CMU cyrus' project skiplist or even something lighter? This way we can
> have only 2 files in the humfs "root", the superblock and the metadataDB.

Sounds like a cool idea.

> The downsides:
> - - another linker requirement at build time;
> - - can't inspect and/or easily modify permissions with emacs. :-)

Doesn't there exists a emacs .el file for editing DB files? Exists for 
pretty much anything else so why not DB files.. should also be quite 
trivial to add a vim filter for the same. Just db4 tools required as long 
as keys & data are all ascii compatible.

Regards
Henrik



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] A quick humfs HOWTO

[BlaisorBlade]

Alle 10:43, giovedì 8 aprile 2004, Henrik Nordstrom ha scritto:
> On Thu, 8 Apr 2004, Nuno Silva wrote:
> > In the meanwhile I'm thinking about this system and it seems that this
> > will eat lots of inodes because every file will be present twice in the
> > [host] filesystem with the metadata version using only a few bytes.
>
> Indeed, so the host better use a filesystem which can handle this
> efficiently such as reiserfs if you plan on very large filesystems.

Actually, Reiser4 will even be more efficent: first because of the FS itself, 
second because we could then use sys_reiser4. It's a syscall designed for 
reading little amounts of data from little files, without opening a fd, 
calling read() and closing it.

"Casually", it was designed to read little metadata files which store 
attributes. However I think that syscall could also be supported by other 
FS's - at least it is possible. However a compile-time option is worth of 
note.

-- 
Paolo Giarrusso, aka Blaisorblade
Linux registered user n. 292729



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id\x1470&alloc_id638&opÌk
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] A quick humfs HOWTO

[Michael Richardson]

-----BEGIN PGP SIGNED MESSAGE-----


>>>>> "Henrik" == Henrik Nordstrom <uml@henrik.marasystems.com> writes:
    >> In the meanwhile I'm thinking about this system and it seems that
    >> this will eat lots of inodes because every file will be present
    >> twice in the [host] filesystem with the metadata version using
    >> only a few bytes.

    Henrik> Indeed, so the host better use a filesystem which can handle
    Henrik> this efficiently such as reiserfs if you plan on very large
    Henrik> filesystems.

  That's a good idea. So, let's make sure that we can put the file
system data in one place, and the metadata in a different place.

- --
]       ON HUMILITY: to err is human. To moo, bovine.           |  firewalls  [
]   Michael Richardson,    Xelerance Corporation, Ottawa, ON    |net architect[
] mcr@xelerance.com      http://www.sandelman.ottawa.on.ca/mcr/ |device driver[
] panic("Just another Debian GNU/Linux using, kernel hacking, security guy"); [
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)
Comment: Finger me for keys

iQCVAwUBQHqwYoqHRg3pndX9AQE97wP/YDhKSNJQiN5dgm1UckIB69RlvNbf3SFC
notLIbqcPFVvGjyXwZgvoMMAfYnQ7ND/tnbH4IOEGjiUSMXCvK3vHMRB8uYZQltA
jAO8e0+Lt668ysS3BRpX0KdZ8EMCV45NBd2PY8h4IvE9WIfa6tmOSwGgnYtEX5N9
ZPSkuMzGm+g=
=3Grd
-----END PGP SIGNATURE-----


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] A quick humfs HOWTO

[Henrik Nordstrom]

On Mon, 12 Apr 2004, Michael Richardson wrote:

> That's a good idea. So, let's make sure that we can put the file
> system data in one place, and the metadata in a different place.

You already can. See earlier discussion wrt symlinking data etc..

Regards
Henrik



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] A quick humfs HOWTO

[Michael Richardson]

-----BEGIN PGP SIGNED MESSAGE-----


>>>>> "Nuno" == Nuno Silva <nuno.silva@vgertech.com> writes:
    Nuno> In the meanwhile I'm thinking about this system and it seems
    Nuno> that this will eat lots of inodes because every file will be
    Nuno> present twice in the [host] filesystem with the metadata
    Nuno> version using only a few bytes. Also every "ls -la" (if it's
    Nuno> not cached) in UML will generate a read in the file's metafile
    Nuno> file (say this 3 times quickly! eheheh). With 1000 files in a
    Nuno> directory thats lot's of fopen()s.

  Yes, this concerned me too.

    Nuno> Wouldn't it be better to add support for a simple DB, like
    Nuno> sleepycat or CMU cyrus' project skiplist or even something
    Nuno> lighter? This way we can have only 2 files in the humfs
    Nuno> "root", the superblock and the metadataDB.

  Yes, we should the option of doing that, also of having a .db file
per directory. There should *ALSO* be a default. 
  If we can store the information on the plain host, then don't put any
metadata there. A directory should be able to say, "everything below is 
mode X" - this might also be useful for adding more detailed
restrictions.

    Nuno> This can save inodes and improve performance because you have
    Nuno> only one file that's indexed! :-)

    Nuno> The downsides: - - another linker requirement at build time; -
    Nuno> - can't inspect and/or easily modify permissions with
    Nuno> emacs. :-)

  Yes/no.
  If we do it right, the file can be created with "makemap" or some
such. Last I looked .db files have poor locking semantics.

  I'm trying to think about the situation where hostfs stuff is shared
by multiple UMLs. (For instance, I share all of /usr/share...)

- --
]       ON HUMILITY: to err is human. To moo, bovine.           |  firewalls  [
]   Michael Richardson,    Xelerance Corporation, Ottawa, ON    |net architect[
] mcr@xelerance.com      http://www.sandelman.ottawa.on.ca/mcr/ |device driver[
] panic("Just another Debian GNU/Linux using, kernel hacking, security guy"); [

  

  
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)
Comment: Finger me for keys

iQCVAwUBQHqwGoqHRg3pndX9AQFA1AQA0l52UOBFkCC3MvrnNLXBEXMs4gFsXmiq
B6WLaUWOayX7Z7zmscARbPmRyBdKXpjcSHjuTMPHwnoYmVVC6RGiBSNOHeUGEhfZ
4wHSwJwRFbhlM/KYTsTmmVRirGwROpVSAOrHJY+3SyQ8Z9vMLkxnucEe60nF05GS
STH551mEpAs=
=AYWh
-----END PGP SIGNATURE-----


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

[uml-devel] Re: [uml-user] A quick humfs HOWTO

[Rus Foster]

On Wed, 7 Apr 2004, Jeff Dike wrote:

> As the readers of uml-devel are aware by now, there is a new virtual UML
> filesystem called humfs.  This is a hostfs-like filesystem which does ownerships
> right, by separating them from the actual file, ala umsdos.
>
> The reasons for humfs are described in my latest diary entry at
> 	http://user-mode-linux.sourceforge.net/diary.html

Anyone else getting /usr/include/linux/aio_abi.h not found or is it just
me?

Rus
-- 
e: support@vpscolo.com
t: 1-888-327-6330
www.jvds.com - Root on your own box
www.vpscolo.com - Your next hosting company


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

[uml-devel] Re: [uml-user] A quick humfs HOWTO

[Jeff Dike]

rghf@fsck.me.uk said:
> Anyone else getting /usr/include/linux/aio_abi.h not found or is it
> just me?

It's not just you.  Move the include inside the #ifdef HAVE_AIO_ABI.

If you have a 2.6 host and you want to use the new aio stuff, copy aio_abi.h
from a 2.6 kernel tree (include/linux) to /usr/include/linux.

				Jeff



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] A quick humfs HOWTO

[Henrik Nordstrom]

On Wed, 7 Apr 2004, Jeff Dike wrote:

> The root contains two directories :
> 	data - which contains the files
> 	metadata - which shadows "data" and whose files contain ownership 
> 		information
> and one file
> 	superblock - which contains information about the filesystem as a 
> 		whole


Can data be a symlink on the host?

I would suggest adding an option to optionally allow specifying the path
to data, this to allow humfs permission shadows to easily be created of
existing trees. If data can be a symlink this option can be managed in
humfsify.


Also, what happens if there is a mismatches between metadata and data
content?

> The superblock file has the format
> 	"version <number>\nused <number>\ntotal <number>\n"
> 
> The version is currently 1.  used is set to the disk space consumed by 
> everything in the data directory.  total is set arbitrarily when you set
> up the filesystem.  It is the size of the filesystem as seen within UML.

Great!

I wouldsuggest adding a read-only flag, preventing UML from mounting the 
directory read-write.

> user and group are the uid and gid that UML will be running as, respectively.

How is this information used by humfsify? Can these be made optional (then 
use current uid/gid, or don't care)

Regards
Henrik



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] A quick humfs HOWTO

[Jeff Dike]

uml@henrik.marasystems.com said:
> Can data be a symlink on the host?

Yup.

> I would suggest adding an option to optionally allow specifying the
> path to data, this to allow humfs permission shadows to easily be
> created of existing trees. If data can be a symlink this option can be
> managed in humfsify.

With humfsify currently, you set up the data tree, and it takes it from there.
So you can make data a symlink, and it will just work.

At some point, I'll have humsify create the data tree as well, and then it
will be possible to tell it to make it a symlink.

> Also, what happens if there is a mismatches between metadata and data
> content? 

Bad things happen.  The filesystem still needs to be hardened against 
mismatches.

> I wouldsuggest adding a read-only flag, preventing UML from mounting
> the  directory read-write. 

Good idea.

> How is this information used by humfsify? Can these be made optional
> (then  use current uid/gid, or don't care) 

Don't care isn't a possibility.  humfsify chowns everything in the data tree
to this uid and gid.  UML needs write access to everything if root is to
behave as root.

Possible it could run as a normal user, but my expectation is that people
will loop-mount ubd volumes as their data sources, and those are filled with
root-owned files.  The only place that makes sense is when you already have
a tree which is completely owned by the UML user.

				Jeff



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] A quick humfs HOWTO

[Henrik Nordstrom]

On Thu, 8 Apr 2004, Jeff Dike wrote:

> Don't care isn't a possibility.  humfsify chowns everything in the data tree
> to this uid and gid.  UML needs write access to everything if root is to
> behave as root.

Ugh.. this should be removed and just documented as a requirement in my 
opinion, especially considering following of symlinks..

> Possible it could run as a normal user, but my expectation is that people
> will loop-mount ubd volumes as their data sources, and those are filled with
> root-owned files.  The only place that makes sense is when you already have
> a tree which is completely owned by the UML user.

Such as is the case if one today is using hostfs, or setting up a 
reasonable read-only access to some tree but wanting detailed permissions. 
Anyway, these are all extensions to the humfsify tool, not the filesystem 
as such so it should be pretty easy for anyone needing this to get it 
done.

Regards
Henrik



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] A quick humfs HOWTO

[Nick Craig-Wood]

On Wed, Apr 07, 2004 at 10:07:52PM -0400, Jeff Dike wrote:
> As the readers of uml-devel are aware by now, there is a new virtual UML 
> filesystem called humfs.

Sounds like a good solution to lots of problems to me ;-)

Do you think it is easy / possible to make UML boot off a loop mounted
filesystem which is stored in humfs?  This would be an easy migration
path...

> The superblock file has the format
> 	"version <number>\nused <number>\ntotal <number>\n"
> 
> The version is currently 1.  used is set to the disk space consumed by 
> everything in the data directory.  total is set arbitrarily when you set
> up the filesystem.  It is the size of the filesystem as seen within
> UML.

Does uml actually obey this total in the superblock?  Ie does this
setting act as a quota?

And an idea...

Why not make each of the files in the metadata directory a symlink and
store the metadata in the symlink itself?  I believe that there is room in
the inode for short symlinks so you'll save inodes there.  Lots of
dangling symlinks might look a bit untidy, but its a technique thats
been used before (eg emacs session files).

See http://www.charmed.com/txt/ext2.txt and in particular

  Symbolic links are also filesystem objects with inodes.  They
  deserve special mention because the data for them is stored within
  the inode itself if the symlink is less than 60 bytes long.  It uses
  the fields which would normally be used to store the pointers to
  data blocks.  This is a worthwhile optimisation as it we avoid
  allocating a full block for the symlink, and most symlinks are less
  than 60 characters long.

-- 
Nick Craig-Wood
ncw1@axis.demon.co.uk


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] A quick humfs HOWTO

[Henrik Nordstrom]

On Thu, 8 Apr 2004, Nick Craig-Wood wrote:

> Do you think it is easy / possible to make UML boot off a loop mounted
> filesystem which is stored in humfs?  This would be an easy migration
> path...

Eh? Why a loop mounted filesystem for humfs? The whole point of humfs is
to be able to emulate full UNIX semantics including ownership,
permissions, filesystem size etc ontop of a plain file tree without
requiring root on the host..

If you want performance then ubd devices is the path to go.

Please elaborate on what you envision being able to do. Probably I just
misunderstood something.

> Why not make each of the files in the metadata directory a symlink and
> store the metadata in the symlink itself?  I believe that there is room in
> the inode for short symlinks so you'll save inodes there.  Lots of
> dangling symlinks might look a bit untidy, but its a technique thats
> been used before (eg emacs session files).

Good idea for optimization on ext2/ext3 host filesystems.

For host filesystems such as reiserfs there is no difference as the same
kind of optimisation is done on small files as well there (provided the
notail option is not used)

Regards
Henrik



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] A quick humfs HOWTO

[Jeff Dike]

uml@henrik.marasystems.com said:
> Eh? Why a loop mounted filesystem for humfs? The whole point of humfs
> is to be able to emulate full UNIX semantics including ownership,
> permissions, filesystem size etc ontop of a plain file tree without
> requiring root on the host..

humfs was done now because it's the only way to get the memory savings offered
by doing mmap IO.

It turns out that some people need block devices because they repartition
their UMLs, run RAID, etc.  Loop-mounting a disk image located on a humfs
filesystem gives you both.  So, it's more than a transitional thing.

> If you want performance then ubd devices is the path to go. 

That's currently the case (a humfs boot is noticably slower than a ubd boot),
but it's not clear why that should be the case.  There is less code running
inside UML (no block layer or block driver), and it's not obvious to me
that putting stuff in files in a host directory should be slower than 
having it in a disk image.

				Jeff



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] A quick humfs HOWTO

[stian]

> And an idea...
>
> Why not make each of the files in the metadata directory a symlink and
> store the metadata in the symlink itself?  I believe that there is room in
> the inode for short symlinks so you'll save inodes there.  Lots of
> dangling symlinks might look a bit untidy, but its a technique thats
> been used before (eg emacs session files).
>
> See http://www.charmed.com/txt/ext2.txt and in particular
>
>   Symbolic links are also filesystem objects with inodes.  They
>   deserve special mention because the data for them is stored within
>   the inode itself if the symlink is less than 60 bytes long.  It uses
>   the fields which would normally be used to store the pointers to
>   data blocks.  This is a worthwhile optimisation as it we avoid
>   allocating a full block for the symlink, and most symlinks are less
>   than 60 characters long.

What if the place you store all you data does not support symlinks (yes,
most filesystems supports symlinks, but there are a few that don't).

Stan


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] A quick humfs HOWTO

[Jeff Dike]

ncw1@axis.demon.co.uk said:
> Why not make each of the files in the metadata directory a symlink and
> store the metadata in the symlink itself?  I believe that there is
> room in the inode for short symlinks so you'll save inodes there.

Yeah, I'll keep that in mind for a future version.

				Jeff



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] A quick humfs HOWTO

[Jeff Dike]

ncw1@axis.demon.co.uk said:
> Does uml actually obey this total in the superblock?  Ie does this
> setting act as a quota? 

It will, I'm not sure if it does right now.  I think that needs to be put
in the superblock at mount time.

Also, it should be possible to have UML notice changes there and have them
take effect immediately.  With it watching the file with dnotify, and reading
it whenever it changes, you should be able to modify filesystem sizes on the
fly.

				Jeff



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

[uml-devel] Re: A quick humfs HOWTO

[Gerd Knorr]

Jeff Dike <jdike@addtoit.com> writes:

> As the readers of uml-devel are aware by now, there is a new virtual UML 
> filesystem called humfs.  This is a hostfs-like filesystem which does ownerships
> right, by separating them from the actual file, ala umsdos.

Hmm.  Why separate?  These days almost all filesystems in 2.6 support
extended attributes, that would be the ideal place for such metadata
IMHO ...

  Gerd

-- 
http://bigendian.bytesex.org


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

[uml-devel] Re: A quick humfs HOWTO

[Jeff Dike]

kraxel@bytesex.org said:
> Hmm.  Why separate?  These days almost all filesystems in 2.6 support
> extended attributes, that would be the ideal place for such metadata
> IMHO ... 

On 2.4, too?  People are going to be running 2.4 for a while.

For 2.6, that's a good suggestion.

				Jeff



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

[uml-devel] Re: A quick humfs HOWTO

[Gerd Knorr]

On Thu, Apr 08, 2004 at 08:49:54AM -0400, Jeff Dike wrote:
> kraxel@bytesex.org said:
> > Hmm.  Why separate?  These days almost all filesystems in 2.6 support
> > extended attributes, that would be the ideal place for such metadata
> > IMHO ... 
> 
> On 2.4, too?  People are going to be running 2.4 for a while.

Vanilla kernels not.  But patches are available and distro kernels often
have that patched in because samba already uses them for quite some time.
ACLs are also stored in EAs.

  Gerd

-- 
http://bigendian.bytesex.org


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Re: A quick humfs HOWTO

[Robin Green]

[-- Attachment #1: Type: text/plain, Size: 377 bytes --]

On Thu, Apr 08, 2004 at 02:03:49PM +0200, Gerd Knorr wrote:
> Jeff Dike <jdike@addtoit.com> writes:
> Hmm.  Why separate?  These days almost all filesystems in 2.6 support
> extended attributes, that would be the ideal place for such metadata
> IMHO ...

reiserfs 3 doesn't! And reiserfs 4 isn't ready yet ... please don't leave
us reiserfs 3 users behind!
-- 
Robin

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [uml-devel] Re: A quick humfs HOWTO

[Michael Richardson]

-----BEGIN PGP SIGNED MESSAGE-----


>>>>> "Gerd" == Gerd Knorr <kraxel@bytesex.org> writes:
    >> As the readers of uml-devel are aware by now, there is a new
    >> virtual UML filesystem called humfs.  This is a hostfs-like
    >> filesystem which does ownerships right, by separating them from
    >> the actual file, ala umsdos.

    Gerd> Hmm.  Why separate?  These days almost all filesystems in 2.6
    Gerd> support extended attributes, that would be the ideal place for
    Gerd> such metadata IMHO ...

  And, I can serious see running it on top of VFAT or NTFS, or even NFS
or CIFS.

  Why? because I would boot Knoppix in a classroom, then boot a UML on
top of that. The reason for this is to keep the classroom requirements
as simple as possible.

- --
]       ON HUMILITY: to err is human. To moo, bovine.           |  firewalls  [
]   Michael Richardson,    Xelerance Corporation, Ottawa, ON    |net architect[
] mcr@xelerance.com      http://www.sandelman.ottawa.on.ca/mcr/ |device driver[
] panic("Just another Debian GNU/Linux using, kernel hacking, security guy"); [
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)
Comment: Finger me for keys

iQCVAwUBQHqxn4qHRg3pndX9AQFQNgQArdaZExzV7LhZB+52tZsOurPAi97vmxng
m27EIFQvBZBUoS7JC3BEpseSm4s/cf078wfKt/XfBmgMQMDRXhEoD+9mrx7hBDWB
6da0KQffaRI6PZswMfMc17McyKrZx5biqg39+l/0GsF/fmGVnzI/nYIbo5wzOURv
x+CooX9TNZ4=
=qKba
-----END PGP SIGNATURE-----


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel