Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Hans Reiser]

The Idea

You should be able to access metadata about a file the same way you
access the file's data, but with a name based on the filename followed
by a name to select the metadata of interest.

Examples:

cat song_of_silence/metas/owner
cat song_of_silence/metas/permissions
cat 10 > song_of_silence/metas/mixer_defaults/volume
cat song_of_silence/metas/license


But wait!  This idea has flaws.  Let us list them and address them:

Problem: Often we want to access more than one metafile at a time, and
a separate system call for each of them is less efficient than stat or
xattrs.

Solution: Create a system call that can access more than one file at a
time, whether it is a metafile or a data file should be irrelevant.

Problem: Many kinds of metadata need to be updated atomically.

Solution: Atomic filesystem, which allows all files, including data
files, to be updated atomically.

Problem: Many kinds of metadata need to be constrained in their
allowed values.

Solution: Allow general purpose constraints to constrain all files,
including data files also.

Problem: Many kinds of metadata should not be visible to the user
unless he wants to see them.

Solution: Allow all files to be hidden from the user unless he wants
to see them.

Example: metas subdirectory is invisible to readdir

Problem: this approach can pollute the namespace

Answer: choose obscure names

Problem (all credit to Mr. Demidov for identifying this problem, I
argued the other viewpoint, and I can only claim the wisdom to know
that I lost the argument): names like "..metas" are ugly to new users,
who don't really care for languages that use punctuation in their
keywords.

Answer

don't make them too obscure, experienced namespace developers know
that the problem of polluting the namespace is not really as big a
deal as beginners think it is, and Clearcase and the WAFL filesystem
manage to get by just fine, whereas the problem of putting punctuation
marks into names and syntax is a big deal for newbies to the system.
Name it "metas" not "..metas", and users will never experience it as a
real problem, and newbies will never be annoyed by a-rhythmic
punctuation.  Note: if Linus disagrees, it is not the most important
thing in this design, "..metas" isn't the end of the world.

Problem: Many kinds of metadata, such as acls, need to be inherited.
Solution: Allow all files, metafiles and data files both, to be
inherited from other files.

The Idea 2:

When implementing a file plugin, very often there are methods supplied
by that plugin that are unique to that plugin.  These methods are
often well interfaced with as files that can be written to or read
from.  Allow file plugins to define pseudo files that perform these
methods.

Examples:

cat complex_document/pseudos/glued gives an archive of the directory
"complex_document" and everything below it, suitable for emailing.

cat filename/pseudos/backup gives a set of commands that a backup
command can use to create the file "filename" with all of its state
preserved, without the backup program having to understand the
plugin's internals.  (If there is going to be a proliferation of file
plugins, then backup plugin methods are a necessity.

cat password_directory/pseudos/cat gives a concatenation of all the
files in the directory "password_directory".


Why Is This Design Important?

It is all about namespace fragmentation.  If you can access everything
in the OS from the same namespace, the OS is much more powerful.

The expressive power of an OS is not proportional to how many objects
are in the OS, but rather to how many different ways you can combine
them.  Separate namespaces for different types of objects is like
having different railroad track gauges for the different peoples of the
world.

If there are different namespaces for 2 sets of objects, then the
number of possible combinations of the objects is (N/2)^2, which is
less than the number of combinations of the objects if they are in one
namespace, which is N^2.


Why Openat Sucks:

because you can't go cat filenameA/metas/permissions > filenameB/permissions

If cat doesn't work, then we are suffering exactly the problem with
namespace fragmentation that this whole scheme was invented to avoid.

The Origins Of This Idea

I disliked stat() way back in 1984.  This is just one small piece of
my efforts to build tools to allow OS architects to unify namespaces
throughout the OS.  DARPA funded this under the CHATS program, and it
was one of the main components of my proposal.

Schedule:

One piece at a time, I would like to let this mature as a reiser4 only
prototype throughout the 2.7 kernel series, and then have it reviewed
for use by other filesystems after that.  Let us not rush it, please.
I have been studying namespace design since 1984, let someone who is
deeply familiar with namespace design principles have a try at a
coherent design with all the pieces in place first before committee
redesign is attempted.  Please consider that there are advantages to
letting the guy who had the idea and did the work to make it happen be
the maintainer.

My approach is to first produce a working prototype, and only then to
try to make a standard.  It is not an unusual approach in our
community.  Let me take that path, it works better for persons like
me.  I have a half working prototype.  A little more time please.
After 30 years of Unix file system semantics stagnation, there is not
a great rush for this little piece of mild innovation I think.

Conclusion:

Namespace fragmentation is the most important single determinant of OS
expressive power, just as trade fragmentation is the most important
single determinant of national wealth (see Adam Smith's The Wealth of
Nations).  Making objects combine easily is what it is all about.

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Alexander G. M. Smith]

Hans Reiser wrote on Sun, 29 Aug 2004 13:21:44 -0700:
> The Idea [...] Conclusion

Good summary!

> The Idea 2: [...]
> cat filename/pseudos/backup gives a set of commands that a backup
> command can use to create the file "filename" [...]

Are you sneaking in file types there?  Just how does a file know which
plugins it supports?

- Alex

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Hans Reiser]

Alexander G. M. Smith wrote:

>Hans Reiser wrote on Sun, 29 Aug 2004 13:21:44 -0700:
>  
>
>>The Idea [...] Conclusion
>>    
>>
>
>Good summary!
>
>  
>
>>The Idea 2: [...]
>>cat filename/pseudos/backup gives a set of commands that a backup
>>command can use to create the file "filename" [...]
>>    
>>
>
>Are you sneaking in file types there?  Just how does a file know which
>plugins it supports?
>
>- Alex
>
>
>  
>
we have plugins with pluginids, is that what you mean by file type? I 
think they are a bit different from file types.

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Alexander G. M. Smith]

Hans Reiser wrote on Mon, 30 Aug 2004 23:43:13 -0700:
> Alexander G. M. Smith wrote:
> >Are you sneaking in file types there?  Just how does a file know which
> >plugins it supports?
>
> we have plugins with pluginids, is that what you mean by file type? I 
> think they are a bit different from file types.

From your white-paper: "Every file possesses a plugin id, and every
directory possesses a plugin id.  This plugin id will identify a set
of methods."

Functionally this is very close to a file type.  It classifies the
files into related groups, maybe not as finely as a MIME file type
which can distinguish between multiple varieties of text files.

A file type would tell us that a file is a text file and can be opened
by certain applications (text/e-mail can be opened by e-mail reader,
text editor, etc) and have other properties (lists of standard
attributes, default icon).  A plugin ID says that the file can use text
related plugins (like a word count, or XML structure as a subdirectory).

In both cases there is a global repository (I assume) that associates
the file type or plugin ID with a list of things about it.

You could combine the two concepts, just have a file type ID that in the
global repository specifies what plugins it can use as well as the
userland properties (MIME string, etc) of that kind of file.  Or at least
make the type ID available to userland so it can be used there.

Or is this binding irrelevant?  How often does the file type the user
sees not match the plugins desired for the file?  Or can a new subtype
be defined for just that file?  Which may mean that we need something
better than MIME strings for types (something which has inheritance).

- Alex

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[David Masover]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Alexander G. M. Smith wrote:
[...]
| A file type would tell us that a file is a text file and can be opened
| by certain applications (text/e-mail can be opened by e-mail reader,
| text editor, etc) and have other properties (lists of standard
| attributes, default icon).  A plugin ID says that the file can use text
| related plugins (like a word count, or XML structure as a subdirectory).
|
| In both cases there is a global repository (I assume) that associates
| the file type or plugin ID with a list of things about it.

I'm not sure how automatic this is.  Standard things like directories,
block devices, files, symlinks, and so on are all separate plugins.

Other than that, it doesn't seem so automatic.  I have to explicitly
turn on encryption, but I can do it to any kind of file or directory --
assuming the cryptocompress plugin worked yet.

[...]
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQTVIGHgHNmZLgCUhAQKzGg//U3bd6yn4LtArriZY5ldBtEah37MidQMR
SZC+jljIdNdwVap2H5m9dC8P4UbN6FLV/3ZVx4K/sel+E35oSM5QWhGyL0DMsTXw
M2irNqwDzLQr7iY7tOm0sqmAc1vMHjyk+w50tIQ0Kjbe6n4Rg3e9ZPf9iPvhJxx2
tlYcGlqcoJ8C1hfI0yHHWTZIx63TE4eOTDgZ7n2EkmI+23STOmRTKrPT7QXQPgPV
0eaC+xSUVcF+78KBTfg1j4nq4OVdgNUS0dbyfHqKZBwUn76zcfHz6aVwFKLOaMNH
lZ+jzpWYykIFhol/HJOmzO8W7GQfHnpKLNC2rDAlh7eKbCdo7mk2WvyDU/ObSenr
fYyhrDwdUG/WKzAa6WySOWuMnABB16mHiUggc2qMcqkyrkCfwq7X13Ryjis0cmCJ
mkIKZWglZaHZUbOOBKrXnT9zyd8T1DfmxwJ7eg66xnk0m5gQfMYk0Yg6z0VQ6KYv
AmBAkRV70YnC87rVTINN4byqMfTWyySu9EqiKaH0STXFNR/cBA87comIOYp0JPZ8
blCfIR5dIw8e5iPEUhPE623qKT4ljKmiw+xJZcbOczi0dymlXQOagJXE0/NH4IdK
vwBgHUhtoxFWxsDeGW4xEF+mE8YLjXJC5AIxvFIdlOv3N27rvm3w1dPwensj5uFC
iFCQ80xftW8=
=2vbH
-----END PGP SIGNATURE-----

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Hans Reiser]

Alexander G. M. Smith wrote:

>Hans Reiser wrote on Mon, 30 Aug 2004 23:43:13 -0700:
>  
>
>>Alexander G. M. Smith wrote:
>>    
>>
>>>Are you sneaking in file types there?  Just how does a file know which
>>>plugins it supports?
>>>      
>>>
>>we have plugins with pluginids, is that what you mean by file type? I 
>>think they are a bit different from file types.
>>    
>>
>
>>From your white-paper: "Every file possesses a plugin id, and every
>directory possesses a plugin id.  This plugin id will identify a set
>of methods."
>
>Functionally this is very close to a file type.
>
A file type is based on semantics of the contents, and a plugin id is 
based on methods of the object.

>  It classifies the
>files into related groups, maybe not as finely as a MIME file type
>which can distinguish between multiple varieties of text files.
>
>A file type would tell us that a file is a text file and can be opened
>by certain applications (text/e-mail can be opened by e-mail reader,
>text editor, etc) and have other properties (lists of standard
>attributes, default icon).  A plugin ID says that the file can use text
>related plugins (like a word count, or XML structure as a subdirectory).
>
>In both cases there is a global repository (I assume) that associates
>the file type or plugin ID with a list of things about it.
>
>You could combine the two concepts, just have a file type ID that in the
>global repository specifies what plugins it can use as well as the
>userland properties (MIME string, etc) of that kind of file.  Or at least
>make the type ID available to userland so it can be used there.
>
>Or is this binding irrelevant?  How often does the file type the user
>sees not match the plugins desired for the file?  Or can a new subtype
>be defined for just that file?  Which may mean that we need something
>better than MIME strings for types (something which has inheritance).
>
>- Alex
>
>
>  
>
I think that there may be some good use for a filename/metas/type metafile.

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Dr. Giovanni A. Orlando]

Hans Reiser wrote:

> Alexander G. M. Smith wrote:
>
>> Hans Reiser wrote on Mon, 30 Aug 2004 23:43:13 -0700:
>>  
>>
>>> Alexander G. M. Smith wrote:
>>>   
>>>
>>>> Are you sneaking in file types there?  Just how does a file know which
>>>> plugins it supports?
>>>>     
>>>
>>> we have plugins with pluginids, is that what you mean by file type? 
>>> I think they are a bit different from file types.
>>>   
>>
>>
>>> From your white-paper: "Every file possesses a plugin id, and every
>>
>> directory possesses a plugin id.  This plugin id will identify a set
>> of methods."
>>
>> Functionally this is very close to a file type.
>>
> A file type is based on semantics of the contents, and a plugin id is 
> based on methods of the object.


Today I read this info on an article published by an excellent magazine, 
that will appears on October (we are Future Technologies :-) , you know)

Hans, like I comment to you time ago, we plan to implement the un-delete 
for ReiserFS 4, like an external plug-in in 2005.

Thanks,
Giovanni

>

>
>>  It classifies the
>> files into related groups, maybe not as finely as a MIME file type
>> which can distinguish between multiple varieties of text files.
>>
>> A file type would tell us that a file is a text file and can be opened
>> by certain applications (text/e-mail can be opened by e-mail reader,
>> text editor, etc) and have other properties (lists of standard
>> attributes, default icon).  A plugin ID says that the file can use text
>> related plugins (like a word count, or XML structure as a subdirectory).
>>
>> In both cases there is a global repository (I assume) that associates
>> the file type or plugin ID with a list of things about it.
>>
>> You could combine the two concepts, just have a file type ID that in the
>> global repository specifies what plugins it can use as well as the
>> userland properties (MIME string, etc) of that kind of file.  Or at 
>> least
>> make the type ID available to userland so it can be used there.
>>
>> Or is this binding irrelevant?  How often does the file type the user
>> sees not match the plugins desired for the file?  Or can a new subtype
>> be defined for just that file?  Which may mean that we need something
>> better than MIME strings for types (something which has inheritance).
>>
>> - Alex
>>
>>
>>  
>>
> I think that there may be some good use for a filename/metas/type 
> metafile.
>


-- 

-- 

--
Check FT Websites ... 
http://www.futuretg.com  - ftp://ftp.futuretg.com
http://www.FTLinuxCourse.com
	http://www.FTLinuxCourse.com/Certification
http://www.rpmparadaise.org
http://GNULinuxUtilities.com
http://www.YourPersonalOperatingSystem.com

--

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Pavel Machek]

Hi!

> Answer: choose obscure names
> 
> Problem (all credit to Mr. Demidov for identifying this problem, I
> argued the other viewpoint, and I can only claim the wisdom to know
> that I lost the argument): names like "..metas" are ugly to new users,
> who don't really care for languages that use punctuation in their
> keywords.
> 
> Answer
> 
> don't make them too obscure, experienced namespace developers know
> that the problem of polluting the namespace is not really as big a
> deal as beginners think it is, and Clearcase and the WAFL filesystem
> manage to get by just fine, whereas the problem of putting punctuation
> marks into names and syntax is a big deal for newbies to the system.
> Name it "metas" not "..metas", and users will never experience it as a
> real problem, and newbies will never be annoyed by a-rhythmic
> punctuation.  Note: if Linus disagrees, it is not the most important
> thing in this design, "..metas" isn't the end of the world.

What about choosing just "..." instead of "metas"? "metas" is string
that needs translation etc, while "..." is nicely neutral.

cat /sound_of_silence.mp3/.../author

does not look bad, either...
								Pavel
-- 
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Christian Mayrhuber]

On Tuesday 31 August 2004 15:12, Pavel Machek wrote:

> What about choosing just "..." instead of "metas"? "metas" is string
> that needs translation etc, while "..." is nicely neutral.
> 
> cat /sound_of_silence.mp3/.../author

Feel free to read these rather lengthy threads:
http://marc.theaimsgroup.com/?l=reiserfs&m=108039830904541&w=2
http://marc.theaimsgroup.com/?l=reiserfs&m=108112390327773&w=2

-- 
lg, Chris

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Mikaël Cluseau]

Why not a kernel option (accessible through /proc if needed for scripts 
or the like) ?

Christian Mayrhuber wrote:
> On Tuesday 31 August 2004 15:12, Pavel Machek wrote:
> 
> 
>>What about choosing just "..." instead of "metas"? "metas" is string
>>that needs translation etc, while "..." is nicely neutral.
>>
>>cat /sound_of_silence.mp3/.../author
> 
> 
> Feel free to read these rather lengthy threads:
> http://marc.theaimsgroup.com/?l=reiserfs&m=108039830904541&w=2
> http://marc.theaimsgroup.com/?l=reiserfs&m=108112390327773&w=2
> 

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[mjt]

On Thu, Sep 02, 2004 at 01:59:19PM +0200, Mikaël Cluseau wrote:
>Why not a kernel option (accessible through /proc if needed for scripts 
>or the like) ?

What if there has to be a patch for find so it accesses metadata as well,
it seems sort of ugly that it has to check /proc or /sys or somesuch
to access the name.

Same thing with the kernel exporting a read-only environment variable.

Hans is pretty stubborn about this and I guess we'll just have to live
with some custom-patched or default name until we see progs that are
dependant on a single name...

-- 
mjt

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Hans Reiser]

Pavel Machek wrote:

>Hi!
>
>  
>
>>Answer: choose obscure names
>>
>>Problem (all credit to Mr. Demidov for identifying this problem, I
>>argued the other viewpoint, and I can only claim the wisdom to know
>>that I lost the argument): names like "..metas" are ugly to new users,
>>who don't really care for languages that use punctuation in their
>>keywords.
>>
>>Answer
>>
>>don't make them too obscure, experienced namespace developers know
>>that the problem of polluting the namespace is not really as big a
>>deal as beginners think it is, and Clearcase and the WAFL filesystem
>>manage to get by just fine, whereas the problem of putting punctuation
>>marks into names and syntax is a big deal for newbies to the system.
>>Name it "metas" not "..metas", and users will never experience it as a
>>real problem, and newbies will never be annoyed by a-rhythmic
>>punctuation.  Note: if Linus disagrees, it is not the most important
>>thing in this design, "..metas" isn't the end of the world.
>>    
>>
>
>What about choosing just "..." instead of "metas"? "metas" is string
>that needs translation etc, while "..." is nicely neutral.
>
>cat /sound_of_silence.mp3/.../author
>
>does not look bad, either...
>								Pavel
>  
>
"..." is pretty good, but I think it has been used by others, but I 
really forget who.  I could live with "...", but I think "metas" and 
"..metas" will collide less often.  Apparently Meta is a finnish name or 
something, so Linus does not like it.  The exact string is really not 
very important to me.  I agree that "..." is elegant.

Hans

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Pavel Machek]

Hi!

> >What about choosing just "..." instead of "metas"? "metas" is string
> >that needs translation etc, while "..." is nicely neutral.
> >
> >cat /sound_of_silence.mp3/.../author
> >
> >does not look bad, either...

> "..." is pretty good, but I think it has been used by others, but I 
> really forget who.  I could live with "...", but I think "metas" and 
> "..metas" will collide less often.  Apparently Meta is a finnish name or 
> something, so Linus does not like it.  The exact string is really not 
> very important to me.  I agree that "..." is elegant.

Well, "..." is mostly used by script kiddies -- they usually have
their rootkit collection there :-).

It would be nice to decide on one escape into "meta" namespace,
uservfs and similar projects probably should be converted to use same
escape.

[Uservfs currently uses things like cat /tmp/foo.tgz#utar/bar.gz#ugz
... essentially using # as another separator. It should probably be
converted to use same meta escape].

								Pavel

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Romano Giannetti]

On Tue, Sep 07, 2004 at 10:59:11PM +0200, Pavel Machek wrote:

> Well, "..." is mostly used by script kiddies -- they usually have
> their rootkit collection there :-).

Wasn't that ".. "? :-) 

       Romano / Sorry. couldn't resist

-- 
Romano Giannetti             -  Univ. Pontificia Comillas (Madrid, Spain)
Electronic Engineer - phone +34 915 422 800 ext 2416  fax +34 915 596 569

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[William Stearns]

Good day, all,

On Tue, 7 Sep 2004, Hans Reiser wrote:

> >What about choosing just "..." instead of "metas"? "metas" is string
> >that needs translation etc, while "..." is nicely neutral.
> >
> >cat /sound_of_silence.mp3/.../author
> >
> >does not look bad, either...
> >
> "..." is pretty good, but I think it has been used by others, but I 
> really forget who.  I could live with "...", but I think "metas" and 

	Some trojans and rootkits have used it to store files; 
/usr/lib/... , /usr/doc/... , /usr/sbin/... , /usr/local/bin/... , 
/dev/... , and /usr/include/... are used by bobkit.
	This might trigger false positives for rootkit detection tools 
like chkrootkit and rootkit-hunter.
	Cheers,
	- Bill

---------------------------------------------------------------------------
        "What is the most effective Windows NT remote management tool? 
A car."
        -- Network Intrusion Detection, An Analyst's Handbook
           2nd Edition, 2000
           Stephen Northcutt et al, page 147
(Courtesy of Rodrigo Goya <lucent@securenet.com.mx>)
--------------------------------------------------------------------------
William Stearns (wstearns@pobox.com).  Mason, Buildkernel, freedups, p0f,
rsync-backup, ssh-keyinstall, dns-check, more at:   http://www.stearns.org
--------------------------------------------------------------------------

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Robin Rosenberg]

On Tuesday 07 September 2004 23.05, William Stearns wrote:
> somone wrote
> > "..." is pretty good, but I think it has been used by others, but I
> > really forget who.  I could live with "...", but I think "metas" and

In MS-DOS  you can do "cd ..." as a shortcut for ..\.. but only in the cd 
command. VMS used it with fileneme pattern matching to indicate a 
recursive search.

> 	Some trojans and rootkits have used it to store files;
> /usr/lib/... , /usr/doc/... , /usr/sbin/... , /usr/local/bin/... ,
> /dev/... , and /usr/include/... are used by bobkit.
> 	This might trigger false positives for rootkit detection tools
> like chkrootkit and rootkit-hunter.
> 	Cheers,
> 	- Bill

Rootkit detection tools need to be updated and improved regularly anyway. If 
you use old rootkit detection tools, it might be a bonus if you are reminded 
of that fact. That's a feature, not a bug.

Maybe file/./attribute then. /. on a file is currently meaningless. That does 
not avoid the unpleasant fact that has been brought up by others (only to be 
ignored), that the directory syntax does not allow metadata on directories.

I'm not convinced that totally transparent access to meta-data actually 
benefits anyone. If metadata is that useful (which I believe) it may well be
worth fixing those apps that need, and can use them. The rest should just
ignore it, even loose it. 

There are too many apps that will not work properly with the metadata as file 
semantics anyway. Implementing something that only works sometimes is not a 
good idea. A new API would work where you expect it to work, i.e. in new and 
fixed apps, just like ACL's and EA's. (Ok, the ACL's are effective 
everywhere, but they aren't copied to new files or even preserved when 
editing, so emacs and others needs some fixing  anyway).

-- robin

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[Theodore Ts'o]

On Wed, Sep 08, 2004 at 12:09:52AM +0200, Robin Rosenberg wrote:
> Maybe file/./attribute then. /. on a file is currently meaningless. That does 
> not avoid the unpleasant fact that has been brought up by others (only to be 
> ignored), that the directory syntax does not allow metadata on directories.

*Not* that I am endorsing the idea of being able to access metadata
via a standard pathname --- I continue to believe that named streams
are a bad idea that will be an attractive nuisance to application
developers, and if we must do them, then Solaris's openat(2) API is
the best way to proceed --- HOWEVER, if people are insistent on being
able to do this via standard pathnames, and not introducing a new
system call, I would suggest /|/ as the separator as the third least
worst option.  Why?

Any such scheme will violate POSIX and SUS, since we are stealing from
the filename namespace, and thus could cause a previously working
program to stop working --- however, assuming that we don't care about
this, the virtical bar is the least likely to collide with existing
file usages, because of its status as a shell meta-character (i.e.,
pipe).  This means that in order to use it on the shell command line,
programs will have to quote it:

	cat /home/tytso/word.doc/\|/meta/silly-stupid-metadata-or-named-stream

This may seem to be inconvenient, but one very good thing about this
is that PHP and existing Perl scripts already already treat pathnames
that contain pipes with a certain amount of suspicion --- and this is
a good thing!  Otherwise, programs that take input from untrusted
sources (say, URL's or http form posts), may convert such input into a
metadata access, and that may be a very, very, very bad thing.  (For
example, it may mean that you will have accidentally allowed a web
user to read or possibly modify an ACL with whatever privileges of the
CGI-perl or php script.)  By using a pipe character, it avoids this
problem, since secure CGI scripts must be already checking for the
pipe character anyway.

> I'm not convinced that totally transparent access to meta-data actually 
> benefits anyone. If metadata is that useful (which I believe) it may well be
> worth fixing those apps that need, and can use them. The rest should just
> ignore it, even loose it. 

Totally agreed.  As I said above, I would prefer openat(2) to trying
to do this within a standard pathname, and I would prefer not doing it
all since aside from Samba, which is simply trying to maintain
backwards compatibility with a Really Bad Idea, the number of
protocols and data formats (ftp, tar, zip, gzip, cpio, etc., etc.,
etc.) that would need to be revamped is huge. 

						- Ted

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[William Lee Irwin III]

On Wed, Sep 08, 2004 at 12:09:52AM +0200, Robin Rosenberg wrote:
>> Maybe file/./attribute then. /. on a file is currently meaningless.
>> That does not avoid the unpleasant fact that has been brought up by
>> others (only to be ignored), that the directory syntax does not
>> allow metadata on directories.

On Thu, Sep 09, 2004 at 05:03:42AM -0400, Theodore Ts'o wrote:
> *Not* that I am endorsing the idea of being able to access metadata
> via a standard pathname --- I continue to believe that named streams
> are a bad idea that will be an attractive nuisance to application
> developers, and if we must do them, then Solaris's openat(2) API is
> the best way to proceed --- HOWEVER, if people are insistent on being
> able to do this via standard pathnames, and not introducing a new
> system call, I would suggest /|/ as the separator as the third least
> worst option.  Why?

I believe this debate is counterproductive while there are far more
basic and serious issues with reiser4, such as architecture-neutrality
of the interpretation of the on-disk format, still pending.


-- wli

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[Gunnar Ritter]

Theodore Ts'o <tytso@mit.edu> wrote:

> Any such scheme will violate POSIX and SUS, since we are stealing from
> the filename namespace,

Not forcibly. POSIX does not give guarantees that everybody can access
existing files with arbitrary names. If there is a metafile in a directory
which can be looked up using the regular pathname hierarchy but requires
certain conditions to be accessed, there is principally nothing wrong
with this. It would likely be wrong if accessing the name for a
non-existent metafile using one of the POSIX interfaces (e.g. creat())
would result in other than one of the defined actions.

> and thus could cause a previously working program to stop working

POSIX holds a barrier against this. It just does not work using the
pathname resolution specification alone. It works by defining certain
actions for certain file type/system interface combinations. For
example, it is demanded that chdir() fails with ENOTDIR if the target
name is not a directory. Thus if the target is a regular file, chdir()
is required to fail.

If, however, the type of the file in question is neither S_IFREG
nor S_IFDIR nor another of the standardized file types, there are
no prescriptions about what system interfaces must do on them.
POSIX (IEEE Std 1003.1, 2004 Edition) explicitly allows to support
non-standard file types in its Base Definitions, 3. 'Definitions',
3.163 'File'.

Also (Base Definitions, 2.1 'Implementation Conformance', 2.1.1
'Requirements'):

# 4. The system may provide additional utilities, functions, or facilities
#    not required by IEEE Std 1003.1-2001. Non-standard extensions of the
#    utilities, functions, or facilities specified in IEEE Std 1003.1-2001
#    should be identified as such in the system documentation. Non-standard
#    extensions, when used, may change the behavior of utilities, functions,
#    or facilities defined by IEEE Std 1003.1-2001. The conformance document
#    shall define an environment in which an application can be run with the
#    behavior specified by IEEE Std 1003.1-2001. In no case shall such an
#    environment require modification of a Strictly Conforming POSIX
#    Application (see Strictly Conforming POSIX Application ).

For example, Unix domain sockets were not part of POSIX until 2001,
but none of the existing systems violated POSIX by refusing to open()
them.

I'm not sure about the results of pathname lookup using the names of
such implementation-defined file types with slashes attached, but it
would probably be worth to file an interpretation request for this
once there is sufficient demand to support it in Linux.

> --- however, assuming that we don't care about
> this, the virtical bar is the least likely to collide with existing
> file usages, because of its status as a shell meta-character (i.e.,
> pipe).  This means that in order to use it on the shell command line,
> programs will have to quote it:
>
> 	cat /home/tytso/word.doc/\|/meta/silly-stupid-metadata-or-named-stream

POSIX certainly requires this to fail with ENOTDIR if 'word.doc' is
S_IFREG, but if it is something like S_IFSTR, there might be a realistic
chance to have it as an implementation extension without violating POSIX.

	Gunnar

-- 
http://omnibus.ruf.uni-freiburg.de/~gritter

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[Hans Reiser]

Putting \ into filenames makes windows compatibility less trivial. 
Putting | into filenames seems like asking for trouble with shells. 
Asking users to keep track of multiple levels of escapes imposed by 
shells and such is hard on them.

If you think \| is user friendly, oh god, people like you are the reason 
why Unix is hated by many.

Having to explain filename/metas/owner or filename/.../owner or 
filename/..metas/owner (I don't deeply care what string is used in place 
of "metas") is hard enough.

All of that said, if "|" was what people preferred, I could live with it.

Stealing from the namespace has a long history behind it (see WAFL, 
Clearcase, many others), and has never been a real world problem. It is 
not so bad. If you manage to find a historical case where someone made a 
mistake in the past, go ahead and name it, but I think moderate caution 
in such thievery is enough, paranoia is not required. Frankly I think 
the people who get paranoid about stealing a little bit from the 
namespace just aren't experienced enough in such matters.

Making an omelette requires breaking eggs. Making a new semantic layer 
(or adding features to languages generally) requires stealing from the 
namespace. POSIX is a least common denominator of operating systems, not 
something for innovators to follow.

Ted, I encourage you to not innovate and stick with POSIX.;-)

(Oh, and yes, I understand that minimizing the cost of change by being 
artful is desirable.)

Streams are a bad idea. The additional features required to emulate 
streams using files and directories are interesting though. Putting 
metafiles in the fs namespace is an increase in closure for the OS, and 
thus a good thing, because more closure means more connectivity between 
OS components.

Rather few people understand closure though, so I don't expect to do 
well in the politics of this. It is a bit like being for free trade, 
most people will never understand why it is so important because their 
mental gifts are in other matters, and the notion that people need to be 
well connected and free to interact is just way too abstract. That it is 
the single most important determinant of a nation's wealth, oh well.

Namespace connectivity is the single most important determinant of an 
OS's expressive power.

Hans

Theodore Ts'o wrote:

>On Wed, Sep 08, 2004 at 12:09:52AM +0200, Robin Rosenberg wrote:
>  
>
>>Maybe file/./attribute then. /. on a file is currently meaningless. That does 
>>not avoid the unpleasant fact that has been brought up by others (only to be 
>>ignored), that the directory syntax does not allow metadata on directories.
>>    
>>
>
>*Not* that I am endorsing the idea of being able to access metadata
>via a standard pathname --- I continue to believe that named streams
>are a bad idea that will be an attractive nuisance to application
>developers, and if we must do them, then Solaris's openat(2) API is
>the best way to proceed --- HOWEVER, if people are insistent on being
>able to do this via standard pathnames, and not introducing a new
>system call, I would suggest /|/ as the separator as the third least
>worst option.  Why?
>
>Any such scheme will violate POSIX and SUS, since we are stealing from
>the filename namespace, and thus could cause a previously working
>program to stop working --- however, assuming that we don't care about
>this, the virtical bar is the least likely to collide with existing
>file usages, because of its status as a shell meta-character (i.e.,
>pipe).  This means that in order to use it on the shell command line,
>programs will have to quote it:
>
>	cat /home/tytso/word.doc/\|/meta/silly-stupid-metadata-or-named-stream
>
>This may seem to be inconvenient, but one very good thing about this
>is that PHP and existing Perl scripts already already treat pathnames
>that contain pipes with a certain amount of suspicion --- and this is
>a good thing!  Otherwise, programs that take input from untrusted
>sources (say, URL's or http form posts), may convert such input into a
>metadata access, and that may be a very, very, very bad thing.  (For
>example, it may mean that you will have accidentally allowed a web
>user to read or possibly modify an ACL with whatever privileges of the
>CGI-perl or php script.)  By using a pipe character, it avoids this
>problem, since secure CGI scripts must be already checking for the
>pipe character anyway.
>
>  
>
>>I'm not convinced that totally transparent access to meta-data actually 
>>benefits anyone. If metadata is that useful (which I believe) it may well be
>>worth fixing those apps that need, and can use them. The rest should just
>>ignore it, even loose it. 
>>    
>>
>
>Totally agreed.  As I said above, I would prefer openat(2) to trying
>to do this within a standard pathname, and I would prefer not doing it
>all since aside from Samba, which is simply trying to maintain
>backwards compatibility with a Really Bad Idea, the number of
>protocols and data formats (ftp, tar, zip, gzip, cpio, etc., etc.,
>etc.) that would need to be revamped is huge. 
>
>						- Ted
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at  http://www.tux.org/lkml/
>
>
>  
>

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[Paul Jakma]

On Thu, 9 Sep 2004, Hans Reiser wrote:

> Putting \ into filenames makes windows compatibility less trivial.

Err, I think Ted used \ as an example of how to escape |. It is not 
part of the filename.

> Putting | into filenames seems like asking for trouble with shells.

I think that was Ted's precise reason for arguing that | be used. Did 
you even read his rationale? :)

> If you think \| is user friendly, oh god, people like you are the 
> reason why Unix is hated by many.

I think he was arguing | (not \|) is the least worst seperator to 
use.

> Rather few people understand closure though, so I don't expect to 
> do well in the politics of this. It is a bit like being for free 
> trade, most people will never understand why it is so important 
> because their mental gifts are in other matters,

Lots of people understand why free-trade is important. It's taught in 
introductory economics/business classes in secondary school.

If you are similarly underestimating the understanding of those who 
are debating merits of in-name-space streams with you, and 
overestimating your own understanding, you're not going make progress 
in convincing people of their merit (if at all).

regards,
-- 
Paul Jakma	paul@clubi.ie	paul@jakma.org	Key ID: 64A2FF6A
Fortune:
The only way to keep your health is to eat what you don't want, drink what
you don't like, and do what you'd rather not.
 		-- Mark Twain

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[Hans Reiser]

Paul Jakma wrote:

> On Thu, 9 Sep 2004, Hans Reiser wrote:
>
>> Putting \ into filenames makes windows compatibility less trivial.
>
>
> Err, I think Ted used \ as an example of how to escape |. It is not 
> part of the filename.

It is not part of it at one level, but in the shell it is part of it.

>
>> Putting | into filenames seems like asking for trouble with shells.
>
>
> I think that was Ted's precise reason for arguing that | be used. Did 
> you even read his rationale? :)

That trouble is desirable? Yes, I can understand why he might not want 
things to work well.;-)

>
>> If you think \| is user friendly, oh god, people like you are the 
>> reason why Unix is hated by many.
>
>
> I think he was arguing | (not \|) is the least worst seperator to use.
>
>> Rather few people understand closure though, so I don't expect to do 
>> well in the politics of this. It is a bit like being for free trade, 
>> most people will never understand why it is so important because 
>> their mental gifts are in other matters,
>
>
> Lots of people understand why free-trade is important. It's taught in 
> introductory economics/business classes in secondary school.

Have you looked at the political process at all? Or by lots of people, 
do you mean a sizable minority?

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[Paul Jakma]

On Thu, 9 Sep 2004, Hans Reiser wrote:

> It is not part of it at one level, but in the shell it is part of it.

Just one of many applications. Watch Joe-user save their word 
processing file sometime, they'll use spaces, quotes, etc.

> Have you looked at the political process at all? Or by lots of 
> people, do you mean a sizable minority?

Kernel development does require deep understanding by the majority of 
computer users. Only kernel developers need deep understanding. ;)

The real question though is: Have you given Al Viro technical answers 
to his technical questions?

regards,
-- 
Paul Jakma	paul@clubi.ie	paul@jakma.org	Key ID: 64A2FF6A
Fortune:
Malek's Law:
 	Any simple idea will be worded in the most complicated way.

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[Hans Reiser]

Paul Jakma wrote:

> On Thu, 9 Sep 2004, Hans Reiser wrote:
>
>> It is not part of it at one level, but in the shell it is part of it.
>
>
> Just one of many applications. Watch Joe-user save their word 
> processing file sometime, they'll use spaces, quotes, etc.

With great unhappiness they will.

>
>> Have you looked at the political process at all? Or by lots of 
>> people, do you mean a sizable minority?
>
>
> Kernel development does 

did you mean to have a "not" here?

> require deep understanding by the majority of computer users. Only 
> kernel developers need deep understanding. ;)

What makes you think kernel developers have a deep understanding of the 
value of connectivity in the OS? They don't. The average kernel 
developer is not particularly bright. Just ask Ted why htrees are slower 
than reiser4, or ext2 tail combining is slower, and, well, he has no 
clue. He is happy to explain how architects don't do real work and 
should not attend the Linux Kernel Summit, and then when reiser4 blows 
htrees away he undoubtedly still thinks I just take the credit away from 
the programmers who do the real work. They did real work, and they are 
the best in the field, but architecture also matters --- quite a lot 
actually.

Maybe instead of free trade, I should have used anti-trust laws as my 
example. The percentage of persons who analytically understand both that 
free trade is vital and that anti-trust laws are a good thing is very 
small (and especially small at Harvard Law School). Average people can 
understand freedom. Understanding that one is not really free to choose 
to not purchase from a cartel is hard for many. Understanding that free 
markets are only a first approximation and that is why we need 
anti-trust laws is beyond perhaps even most economics PhDs.

This is not due to a lack of education. I once had a boss explain to me 
how many people have trouble understanding orders of magnitude and 
ratios. He particularly meant his boss, who was having trouble 
understanding my report.

We all have mental defects, we just have them in different areas from 
each other. (Forgive me for not enumerating mine....;-) ) Some technical 
matters are understood by much less than 50% of the population. Closure 
is one of them. For most people the value of closure can only be 
understood by using and liking a system that has it, and they are not 
capable of wanting it in advance during the design stages. Codd 
understood the importance of closure. You could sense his frustration at 
being unable to convey it to others in his writings. The search engine 
industry completely misses the importance of closure.

This is why I just want to be left alone to tinker with reiser4. It is 
faster than other filesystems. People should assume I know what I am 
doing, and leave me to tinker in my little fs. 5 years later others will 
follow, or not, I don't care.

>
> The real question though is: Have you given Al Viro technical answers 
> to his technical questions?

Yes, I did. Got no response. Would you like me to post something nice 
and technical to this thread?;-) I can send a summary of my design, and 
the answers I sent to Viro and Linus.

>
> regards,

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[viro]

On Thu, Sep 09, 2004 at 10:04:38PM -0700, Hans Reiser wrote:
> >The real question though is: Have you given Al Viro technical answers 
> >to his technical questions?
> 
> Yes, I did. Got no response.

Liar.

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[Hans Reiser]

viro@parcelfarce.linux.theplanet.co.uk wrote:

>On Thu, Sep 09, 2004 at 10:04:38PM -0700, Hans Reiser wrote:
>  
>
>>>The real question though is: Have you given Al Viro technical answers 
>>>to his technical questions?
>>>      
>>>
>>Yes, I did. Got no response.
>>    
>>
>
>Liar.
>
>
>  
>
What was your technical response then?

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[viro]

On Thu, Sep 09, 2004 at 11:52:46PM -0700, Hans Reiser wrote:
> viro@parcelfarce.linux.theplanet.co.uk wrote:
> 
> >On Thu, Sep 09, 2004 at 10:04:38PM -0700, Hans Reiser wrote:
> > 
> >
> >>>The real question though is: Have you given Al Viro technical answers 
> >>>to his technical questions?
> >>>     
> >>>
> >>Yes, I did. Got no response.
> >>   
> >>
> >
> >Liar.
> >
> >
> > 
> >
> What was your technical response then?

20040908093624.GW23987@parcelfarce.linux.theplanet.co.uk, written in assumption
that the only reply I've got regaring the Message-ID of your "answers" had
been correct.

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[Hans Reiser]

viro@parcelfarce.linux.theplanet.co.uk wrote:

>On Thu, Sep 09, 2004 at 11:52:46PM -0700, Hans Reiser wrote:
>  
>
>>viro@parcelfarce.linux.theplanet.co.uk wrote:
>>
>>    
>>
>>>On Thu, Sep 09, 2004 at 10:04:38PM -0700, Hans Reiser wrote:
>>>
>>>
>>>      
>>>
>>>>>The real question though is: Have you given Al Viro technical answers 
>>>>>to his technical questions?
>>>>>    
>>>>>
>>>>>          
>>>>>
>>>>Yes, I did. Got no response.
>>>>  
>>>>
>>>>        
>>>>
>>>Liar.
>>>
>>>
>>>
>>>
>>>      
>>>
>>What was your technical response then?
>>    
>>
>
>20040908093624.GW23987@parcelfarce.linux.theplanet.co.uk, written in assumption
>that the only reply I've got regaring the Message-ID of your "answers" had
>been correct.
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at  http://www.tux.org/lkml/
>
>
>  
>
Not found in my folder, perhaps you could just forward it.....

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[Lee Revell]

On Fri, 2004-09-10 at 03:30, Hans Reiser wrote:
> viro@parcelfarce.linux.theplanet.co.uk wrote:
> >On Thu, Sep 09, 2004 at 11:52:46PM -0700, Hans Reiser wrote:
> >
> >>viro@parcelfarce.linux.theplanet.co.uk wrote:
> >>
> >>>On Thu, Sep 09, 2004 at 10:04:38PM -0700, Hans Reiser wrote:
> >>>
> >>>>>The real question though is: Have you given Al Viro technical answers 
> >>>>>to his technical questions?          
> >>>>>
> >>>>Yes, I did. Got no response.
> >>>>  
> >>>Liar.   
> >>>
> >>What was your technical response then?
> >
> >20040908093624.GW23987@parcelfarce.linux.theplanet.co.uk, written in assumption
> >that the only reply I've got regaring the Message-ID of your "answers" had
> >been correct.
> >  
> >
> Not found in my folder, perhaps you could just forward it.....

There was a list outage lasting several hours at the height of the
reiser4 thread.  So, before you start with the name calling, please
check the archives to see if your post made it.

Lee

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[viro]

On Fri, Sep 10, 2004 at 12:49:50PM -0400, Lee Revell wrote:
> There was a list outage lasting several hours at the height of the
> reiser4 thread.  So, before you start with the name calling, please
> check the archives to see if your post made it.

http://marc.theaimsgroup.com/?l=linux-kernel&m=109463636524917&w=2

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[Hans Reiser]

viro@parcelfarce.linux.theplanet.co.uk wrote:

>On Thu, Sep 09, 2004 at 10:04:38PM -0700, Hans Reiser wrote:
>  
>
>>>The real question though is: Have you given Al Viro technical answers 
>>>to his technical questions?
>>>      
>>>
>>Yes, I did. Got no response.
>>    
>>
>
>Liar.
>
>
>  
>
I don't think that "Liar." is an appropriate response. If you sent a 
response, just quote it.

I must say that your attitude towards persons contributing to Linux (of 
which this email is the least of it) has over the years lost Linux 
persons much more talented than yourself. We lost the opportunity to 
have one of DARPAs hot young security researchers contribute to us 
because his experience was that Linux maintainers are a collection of 
assholes hostile to new contributors, and he had too much self respect 
to deal with the likes of you. He was a very nice and talented young 
man. Frankly, I look at Linux, and I see all the reasons why I decided 
not to develop for BSD coming to life in Linux now that Linux is more 
successful than BSD. Inner circle, hostility to newcomers, patch 
acceptance based on whose nose is in whose ass, etc., etc.

Hans

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[viro]

On Fri, Sep 10, 2004 at 12:21:50AM -0700, Hans Reiser wrote:
> I don't think that "Liar." is an appropriate response.

To a bold-faced lie?  Yes, it is.

> If you sent a 
> response, just quote it.

I've already posted Message-Id, but if you prefer a quote, fine, here it is:

============================================================================
On Wed, Sep 08, 2004 at 01:21:45PM +0530, Sriram Karra wrote:
> Perhaps this is one? Message-ID: <413578C9.8020305@namesys.com>

OK...

One note before replying: current code deadlocks even if you make ->link()
*ALWAYS* return an error.  It doesn't get to calling the method.  No amount
of "disallow hard links to <something>" is going to help here, obviously.

<quote>
Cycle detection:

We should either 1) make hard links only link to the file aspect of the
file-directory duality, and persons who want to link to the directory
aspect must use symlinks (best short term answer), or 2) ask Alexander
Smith to help us with applying his cycle detection algorithm and gain
the benefit of being able to hard link to directories (if it works well,
best long term answer).
</quote>

... which doesn't address the problem at all.  The question is what to do
with seeing directory "aspect..." in more than one place when we have many
links to file in question.  So much for (1).  And (2) is not feasible with
on-disk fs both due to memory, CPU and IO costs _and_ due to exclusion from
hell you'll need to make it safe.

Re: ambiguity - lots and lots of handwaving on both sides.  FWIW, I agree
with Hans in one (and only one) respect here - openat() as a primary API
(and not a convenient libc function) is an atrocity.  Simply because it
doesn't address operations beyond open (unlinkat(2), anyone?).

However, I still haven't seen any strong arguments for need of this "metas"
stuff _or_ the need to export mode/ownership as files, both for regular
files and for directories.  Aside of "we can do that" [if we solve the
locking issues] and "xattrs are atrocious" [yes, they are; it doesn't make
alternative mechanism any better] there was nothing that even pretended to
be a technical reason.

Note that we also have fun issues with device nodes (Linus' "show partitions"
vs. "show metadata from hosting filesystem"), which makes it even more dubious.
We also have symlinks to deal with (do they have attributes?  where should
that be exported?).

Reserved names have one more problem: to be useful, they'd have to be
hardcoded into applications.  And that will create hell with use of
such applications on existing filesystems.  Again, no feasible scheme
to deal with that in userland code had been proposed so far, AFAICS.

Locking: see above - links to regular files would create directories seen
in many places.  With all related issues...

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[Hans Reiser]

Well I didn't get this response, so whether or not you sent it, it was 
not a lie. Drink less coffee.



viro@parcelfarce.linux.theplanet.co.uk wrote:

>On Fri, Sep 10, 2004 at 12:21:50AM -0700, Hans Reiser wrote:
>  
>
>>I don't think that "Liar." is an appropriate response.
>>    
>>
>
>To a bold-faced lie?  Yes, it is.
>
>  
>
>>If you sent a 
>>response, just quote it.
>>    
>>
>
>I've already posted Message-Id, but if you prefer a quote, fine, here it is:
>
>============================================================================
>On Wed, Sep 08, 2004 at 01:21:45PM +0530, Sriram Karra wrote:
>  
>
>>Perhaps this is one? Message-ID: <413578C9.8020305@namesys.com>
>>    
>>
>
>OK...
>
>One note before replying: current code deadlocks even if you make ->link()
>*ALWAYS* return an error.  It doesn't get to calling the method.  No amount
>of "disallow hard links to <something>" is going to help here, obviously.
>
><quote>
>Cycle detection:
>
>We should either 1) make hard links only link to the file aspect of the
>file-directory duality, and persons who want to link to the directory
>aspect must use symlinks (best short term answer), or 2) ask Alexander
>Smith to help us with applying his cycle detection algorithm and gain
>the benefit of being able to hard link to directories (if it works well,
>best long term answer).
></quote>
>
>... which doesn't address the problem at all.  The question is what to do
>with seeing directory "aspect..." in more than one place when we have many
>links to file in question. 
>
You don't allow people to see the directory aspect in more than one 
place.....

> So much for (1).  And (2) is not feasible with
>on-disk fs both due to memory, CPU and IO costs _and_ due to exclusion from
>hell you'll need to make it safe.
>  
>
Your statement regarding 2) is unsupported by technical argument and I 
think wrong as well.

>Re: ambiguity - lots and lots of handwaving on both sides.  FWIW, I agree
>with Hans in one (and only one) respect here - openat() as a primary API
>(and not a convenient libc function) is an atrocity.  Simply because it
>doesn't address operations beyond open (unlinkat(2), anyone?).
>
>However, I still haven't seen any strong arguments for need of this "metas"
>stuff _or_ the need to export mode/ownership as files, both for regular
>files and for directories.  Aside of "we can do that" [if we solve the
>locking issues] and "xattrs are atrocious" [yes, they are; it doesn't make
>alternative mechanism any better] there was nothing that even pretended to
>be a technical reason.
>  
>
Closure is very technical as a reason. It seems to be above your head 
though. I can say more about it, but not before bed....

>Note that we also have fun issues with device nodes (Linus' "show partitions"
>vs. "show metadata from hosting filesystem"), which makes it even more dubious.
>We also have symlinks to deal with (do they have attributes?  where should
>that be exported?).
>
>Reserved names have one more problem: to be useful, they'd have to be
>hardcoded into applications.  And that will create hell with use of
>such applications on existing filesystems.  Again, no feasible scheme
>to deal with that in userland code had been proposed so far, AFAICS.
>  
>
How is this different from adding any new feature to any program 
(library, kernel, fs, daemon) with competitors, that other programs 
interact with? If you can't cope with change, don't user reiser4..... 
reiser4 still supports stat(),....

>Locking: see above - links to regular files would create directories seen
>in many places. 
>
No, it would only be seen from one parent, unless Smith's solution is used.

> With all related issues...
>
>
>  
>

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[viro]

On Fri, Sep 10, 2004 at 12:46:23AM -0700, Hans Reiser wrote:
> >file-directory duality, and persons who want to link to the directory
> >aspect must use symlinks (best short term answer), or 2) ask Alexander
> >Smith to help us with applying his cycle detection algorithm and gain
> >the benefit of being able to hard link to directories (if it works well,
> >best long term answer).
> ></quote>
> >
> >... which doesn't address the problem at all.  The question is what to do
> >with seeing directory "aspect..." in more than one place when we have many
> >links to file in question. 
> >
> You don't allow people to see the directory aspect in more than one 
> place.....

And which place would that be?  Concrete example: we have 4 links to the
same inode in /bin - /bin/gzip, /bin/gunzip, /bin/zcat and /bin/uncompress.
What should happen upon attempts to look at the metadata of /bin/gzip and
/bin/gunzip simultaneously from completely unrelated processes?  Where
can these guys be found and in case if you say "whoever looks first wins,
another guy gets -EBUSY or some other error" keep in mind that user-triggerable
DoS tend to make sysadmins quite unhappy.
 
> >So much for (1).  And (2) is not feasible with
> >on-disk fs both due to memory, CPU and IO costs _and_ due to exclusion from
> >hell you'll need to make it safe.
> > 
> >
> Your statement regarding 2) is unsupported by technical argument and I 
> think wrong as well.

Uhh...  Hans, think for a second - any algorithm would have to be able to
tell if adding an edge to graph would create a loop.  Consider the following
graph: take two full binary trees of depth N, order their leaves, revert the
edges in one of them and for each leaf of the first tree add an edge leading
to corresponding leaf of the second one.  (IOW, for N=2 you'll get 14 nodes
with the following edges: A->A0, A->A1, A0->A00, A0->A01, A1->A10, A1->A11,
A00->B00, A01->B01, A10->B10, A11->B11, B00->B0, B01->B0, B10->B1, B11->B1,
B0->B, B1->B).  Now, give that tree to your algorithm and start asking if
adding an edge between given two nodes would add a loop.  You'll get
exponential time complexity.  Moreover, the number of nodes you would have
to examine is also exponential.

Note that guy specifically mentioned that his filesystem had been in-core
one.  With disk-based fs you'll either have to keep the entire graph
in-core *or* you will have to walk the damn thing pulling it from disk.

And you need an exclusion against graph modifications while you are looking
through it.  Have fun...

> >Re: ambiguity - lots and lots of handwaving on both sides.  FWIW, I agree
> >with Hans in one (and only one) respect here - openat() as a primary API
> >(and not a convenient libc function) is an atrocity.  Simply because it
> >doesn't address operations beyond open (unlinkat(2), anyone?).
> >
> >However, I still haven't seen any strong arguments for need of this "metas"
> >stuff _or_ the need to export mode/ownership as files, both for regular
> >files and for directories.  Aside of "we can do that" [if we solve the
> >locking issues] and "xattrs are atrocious" [yes, they are; it doesn't make
> >alternative mechanism any better] there was nothing that even pretended to
> >be a technical reason.
> > 
> >
> Closure is very technical as a reason. It seems to be above your head 
> though. I can say more about it, but not before bed....

The word "closure" has more than enough meanings, so I am afraid that
unless you care to specify what exactly you are talking about it will
remain above my head - I am not a telepath.

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[Alan Cox]

On Gwe, 2004-09-10 at 06:04, Hans Reiser wrote:
> > Just one of many applications. Watch Joe-user save their word 
> > processing file sometime, they'll use spaces, quotes, etc.
> With great unhappiness they will.

Its only problematic for the command line users. The GUI doesn't have
some mysterious notion of meta-characters, it provides out of band
information on boundaries.

> This is why I just want to be left alone to tinker with reiser4. It is 
> faster than other filesystems. People should assume I know what I am 
> doing, and leave me to tinker in my little fs. 5 years later others will 
> follow, or not, I don't care.

See I don't care if you tinker with reiser4. I don't care if it turns
out to be a crap fs or a great fs. If its a great fs and scales and
unlike reiser3 can recover well from disk errors then one year I might
even use it.

I do care if you ask me to suffer core API changes for your research,
that in your economics world is an externality. Its a large negative
externality on the part of the userbase so the userbase objects. It
doesn't take a PhD in economics to understand this.

Alan

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[Hans Reiser]

Alan Cox wrote:

>On Gwe, 2004-09-10 at 06:04, Hans Reiser wrote:
>  
>
>>>Just one of many applications. Watch Joe-user save their word 
>>>processing file sometime, they'll use spaces, quotes, etc.
>>>      
>>>
>>With great unhappiness they will.
>>    
>>
>
>Its only problematic for the command line users. The GUI doesn't have
>some mysterious notion of meta-characters, it provides out of band
>information on boundaries.
>  
>
Forgive me, what is out of band information on boundaries?

Most people I know don't use the GUI for executing commands, perhaps 
this is because the existing guis are not good enough yet.

>  
>
>>This is why I just want to be left alone to tinker with reiser4. It is 
>>faster than other filesystems. People should assume I know what I am 
>>doing, and leave me to tinker in my little fs. 5 years later others will 
>>follow, or not, I don't care.
>>    
>>
>
>See I don't care if you tinker with reiser4. I don't care if it turns
>out to be a crap fs or a great fs. If its a great fs and scales and
>unlike reiser3 can recover well from disk errors then one year I might
>even use it.
>  
>
Is there a technical basis for your claim that we have trouble with disk 
errors?

Do you mean badblocks support or what?

>I do care if you ask me to suffer core API changes for your research,
>that in your economics world is an externality. Its a large negative
>externality on the part of the userbase so the userbase objects. It
>doesn't take a PhD in economics to understand this.
>
>Alan
>
>
>
>  
>
I think it would be reasonable for people to say that our approach 
currently has bugs, we should turn metafiles off until we make the bugs 
go away.

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[Alan Cox]

On Gwe, 2004-09-10 at 18:48, Hans Reiser wrote:
> Is there a technical basis for your claim that we have trouble with disk 
> errors?
> 
> Do you mean badblocks support or what?

I mean probability of gettng your data back after a disk loses data. And
the technical basis for my claims is twofold - painful experience is
one, and shooting random blocks of zeros onto a disk and run fsck tools
is another. 

> I think it would be reasonable for people to say that our approach 
> currently has bugs, we should turn metafiles off until we make the bugs 
> go away.

Well reiserfs4 is a lot more than metafiles and new vfs layer concepts.
Clearly those parts of the the fs that don't require core fs changes
belong in the kernel as soon as everyone is happy they are clean enough
and look correct.

Metafiles and openat() are an argument we can all have later.

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[Horst von Brand]

Hans Reiser <reiser@namesys.com> said:
> Paul Jakma wrote:
> > On Thu, 9 Sep 2004, Hans Reiser wrote:
> >> It is not part of it at one level, but in the shell it is part of it.

> > Just one of many applications. Watch Joe-user save their word 
> > processing file sometime, they'll use spaces, quotes, etc.

> With great unhappiness they will.

Given the right GUI, they won't even notice.

> >> Have you looked at the political process at all? Or by lots of 
> >> people, do you mean a sizable minority?

> > Kernel development does 

> did you mean to have a "not" here?

> > require deep understanding by the majority of computer users. Only 
> > kernel developers need deep understanding. ;)

> What makes you think kernel developers have a deep understanding of the 
> value of connectivity in the OS?

Enlighten us. Never heard of "conectivity in the OS" before.

>                                  They don't. The average kernel 
> developer is not particularly bright.

Right.

>                                       Just ask Ted why htrees are slower 
> than reiser4, or ext2 tail combining is slower, and, well, he has no 
> clue.

Great PR move, again: Go on publicly insulting the people who you are
trying to convince.

>       He is happy to explain how architects don't do real work and 
> should not attend the Linux Kernel Summit, and then when reiser4 blows 
> htrees away he undoubtedly still thinks I just take the credit away from 
> the programmers who do the real work. They did real work, and they are 
> the best in the field, but architecture also matters --- quite a lot 
> actually.

Right you are about architecture.

[...]

> This is why I just want to be left alone to tinker with reiser4. It is 
> faster than other filesystems. People should assume I know what I am 
> doing, and leave me to tinker in my little fs. 5 years later others will 
> follow, or not, I don't care.

Great idea.

> > The real question though is: Have you given Al Viro technical answers 
> > to his technical questions?

> Yes, I did.

Haven't seen them.

>             Got no response. Would you like me to post something nice 
> and technical to this thread?;-)

That is what this list is supposed to be about: Technical discussion of the
Linux kernel development.

>                                  I can send a summary of my design, and 
> the answers I sent to Viro and Linus.

Please do. Together with the questions, so we can see the whole picture.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[Horst von Brand]

Hans Reiser <reiser@namesys.com> said:
> Paul Jakma wrote:
> > On Thu, 9 Sep 2004, Hans Reiser wrote:

[...]

> >> Putting \ into filenames makes windows compatibility less trivial.

> > Err, I think Ted used \ as an example of how to escape |. It is not 
> > part of the filename.

> It is not part of it at one level, but in the shell it is part of it.

What are you talking about?

> >> Putting | into filenames seems like asking for trouble with shells.

> > I think that was Ted's precise reason for arguing that | be used. Did 
> > you even read his rationale? :)

> That trouble is desirable?

He didn't say that at all. He said that '|' in filenames is currently
troublesome, so nobody will do it. Besides, '|' is dangerous in some
contexts, so it will probably already be filtered out where it might do
damage without any further work. This is important, you can't expect
everybody rewrite all their applications/web sites just because somebody
might be fooling around with Reiser4 to check it out.

>                            Yes, I can understand why he might not want 
> things to work well.;-)

It is very clear that you don't even try to understand criticism.

> >> If you think \| is user friendly, oh god, people like you are the 
> >> reason why Unix is hated by many.

It isn't. Ted's point is that the alternatives are much worse. Perhaps
using "meta" or something like that is friendlier at first sight, but if
your box is throroughly 0wn3d as a result, I'm sure your impression of
friendliness will radically change.

> > I think he was arguing | (not \|) is the least worst seperator to use.

Exactly.

> >> Rather few people understand closure though, so I don't expect to do 
> >> well in the politics of this.

Sorry for you, but either you convince the head kernel hackers (including
Ted, BTW) that you are right. And they are very hard to convince, only hope
is to present clear, clean solutions to the problems they raise. Nobody has
done so in this flamewar, AFAICS. Politics has very little to do with this.

> >>                               It is a bit like being for free trade, 
> >> most people will never understand why it is so important because 
> >> their mental gifts are in other matters,

Here you aren't talking to the counterpart of Joe Sixpack, you are trying
to convince the Economics Department of an ivy-league university that your
ideas on economics are right and they don't know what they are talking
about. Possible, but I'd call it somewhat unlikely.

> > Lots of people understand why free-trade is important. It's taught in 
> > introductory economics/business classes in secondary school.

> Have you looked at the political process at all? Or by lots of people, 
> do you mean a sizable minority?

Again, no political process in sight.

Please stop the handwawing, and address the core issues: Locking, POSIX
compatibility (or convince people that POSIX is wrong), costs of
implementing this (no, not just kernel implementation; also new
applications, application changes required, user/sysadmin retraining, etc).
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[Davide Inglima]

On Thu, 09 Sep 2004 12:15:02 -0700, Hans Reiser <reiser@namesys.com> wrote:
> Putting \ into filenames makes windows compatibility less trivial.
> Putting | into filenames seems like asking for trouble with shells.
> Asking users to keep track of multiple levels of escapes imposed by
> shells and such is hard on them.
 
> If you think \| is user friendly, oh god, people like you are the reason
> why Unix is hated by many.
 
> Having to explain filename/metas/owner or filename/.../owner or
> filename/..metas/owner (I don't deeply care what string is used in place
> of "metas") is hard enough.

[Sorry if the tone of this e-mail is particularly heated. I used to
use reiserfs and I am really looking towards the insertion of the
final version of reiser4 in the linux kernel, but I wish to publish my
2 cents as user (not as kernel developer). TIA for your attention and
patience in advance...]

The idea of using "metas" (instead of ".metas" and "..metas") however
is ATROCIOUS, as it would mean to have a fine "metas" entry popping up
in every dir, be undeletable, and having some random user pissed at
his random ${OS} because "it puts stuff that does not belong in the
system"...

The main idea about metadata is that you should be able to use
metadata only if you care. And if you use something, THEN you need to
be bugged by its presence but in a way that does not bug everyone's
else... 99% of people out there only care to use metadatas only via
those kind of applications that _really_ support it, like Mp3 players,
Torrentlike apps and CVS-like programs (if they do and really care to
have their metadatas consistent with the reality, that is [1]).

It has no sense for a human being to go and manage thousands of
metadata directories by hand, hence having "metas" right there in the
filesystem is not calling for user-friendlyness... it is only
desperately calling for some n00b# to dd inside it "just for fun" or
"just to make it go away". Putting in the spotlight a potentially
unreliable OR fragile source of informations is only asking for
trouble and is also harrassing for the end-user that couldn't care
less about that "metas" stuff. (what is this metas stuff that is
between my mariah999.jpg and morena001.jpg in my pr0n/ folder? [2])

In the end, using a UNIX-flavoured system matters because things are
shaped to help sysadmin's jobs, and sysadmins are asked for their task
to Read The Finely-written (?) Manuals before inserting the root
password at the login prompts.

The end-users are to be protected from themselves until they are
really ready to read the documentation.

In conclusion: I don't know WAFL, but I won't make Clearcase example a
too forward example... Clearcase naming is useful only inside a
Clearcase-enabled environment and using Clearcase-enabled tools... I
don't think that people at IBM are going to flame me or handle me a
DMCA takedown notice if I put on my filesystem filenames that could
possibly conflict with their system. Instead Reiserfs4 is going to ask
me to sacrifice a viable word for my filenames (no, I don't call my
filenames .something), and to create a workaround for my scripts [3]
while an alternative like .metas or a ..metas would make even more
sense and be more streamlined with Linux/UNIX customs.

Just my 2 cents as random n00b.

Ciao :)
-- 
Davide Inglima

[1] http://www.well.com/~doctorow/metacrap.htm
[2] whoops...
[3] ls -1 | wc -l

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[Helge Hafting]

Theodore Ts'o wrote:

>On Wed, Sep 08, 2004 at 12:09:52AM +0200, Robin Rosenberg wrote:
>  
>
>>Maybe file/./attribute then. /. on a file is currently meaningless. That does 
>>not avoid the unpleasant fact that has been brought up by others (only to be 
>>ignored), that the directory syntax does not allow metadata on directories.
>>    
>>
>
>*Not* that I am endorsing the idea of being able to access metadata
>via a standard pathname --- I continue to believe that named streams
>are a bad idea that will be an attractive nuisance to application
>developers, and if we must do them, then Solaris's openat(2) API is
>the best way to proceed --- HOWEVER, if people are insistent on being
>able to do this via standard pathnames, and not introducing a new
>system call, I would suggest /|/ as the separator as the third least
>worst option.  Why?
>  
>
What's wrong with using / as the separator?  It is already
used to separate components of pathnames.  Named streams
are very much like files in a subdirectory.

This scheme makes for very little change to existing tools,
users may then do a "gimp somefile/icon.jpg"  for example.
Or "ls somefile/*" to see all the named streams/forks.

Helge Hafting

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[Horst von Brand]

Helge Hafting <helge.hafting@hist.no> said:
> Theodore Ts'o wrote:
> >On Wed, Sep 08, 2004 at 12:09:52AM +0200, Robin Rosenberg wrote:

> >>Maybe file/./attribute then. /. on a file is currently
> >>meaningless. That does not avoid the unpleasant fact that has been
> >>brought up by others (only to be ignored), that the directory syntax
> >>does not allow metadata on directories. 

> >*Not* that I am endorsing the idea of being able to access metadata
> >via a standard pathname --- I continue to believe that named streams
> >are a bad idea that will be an attractive nuisance to application
> >developers, and if we must do them, then Solaris's openat(2) API is
> >the best way to proceed --- HOWEVER, if people are insistent on being
> >able to do this via standard pathnames, and not introducing a new
> >system call, I would suggest /|/ as the separator as the third least
> >worst option.  Why?

> What's wrong with using / as the separator?  It is already
> used to separate components of pathnames.  Named streams
> are very much like files in a subdirectory.

/ is separator for directories, POSIX mandates its exact use. No, POSIX
isn't broken here, and even if it was, you have to remain compatible.

> This scheme makes for very little change to existing tools,

... while breaking fundamental assumptions by all programs in a major way,
and no sane solution for legacy applications in sight, with unknown
(probably huge) correctness and security implications...

> users may then do a "gimp somefile/icon.jpg"  for example.
> Or "ls somefile/*" to see all the named streams/forks.

Please don't rehash this one. It is fundamentally broken.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: silent semantic changes in reiser4 (brief attempt to document the idea ofwhat reiser4 wants to do with metafiles and why

[Helge Hafting]

Elladan wrote:

>
>>What's wrong with using / as the separator?  It is already
>>used to separate components of pathnames.  Named streams
>>are very much like files in a subdirectory.
>>
>>This scheme makes for very little change to existing tools,
>>users may then do a "gimp somefile/icon.jpg"  for example.
>>Or "ls somefile/*" to see all the named streams/forks.
>>    
>>
>
>Directories may have metadata as well.
>  
>
They can. That doesn't stand in the way of using "/" to separate
the named stream's name from the file (or directory) that
have the attribute.  "Directories may have metadata" pops
up now and then, and the solution is so blindlingly obvious
that nobody sees it.

A file-as-dir can be implemented as a rather normal directory
attached to the file's name.  The stuff inside may
be interpreted as "attributes" or as something more file-like,
such as the often mentioned thumbnails.

What about a directory then?  It _is_ a directory, so it
support named streams already.  They are usually called "files". :-)
So, if you really want a thumb for your directory, just store a
thumb.jpg in it.

Helge Hafting

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Mike Waychison]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hans Reiser wrote:

>
> Why Openat Sucks:
>
> because you can't go cat filenameA/metas/permissions >
> filenameB/permissions
>
> If cat doesn't work, then we are suffering exactly the problem with
> namespace fragmentation that this whole scheme was invented to avoid.
>

Sure you can.  All you need to do is fix your shell.

- --
Mike Waychison
Sun Microsystems, Inc.
1 (650) 352-5299 voice
1 (416) 202-8336 voice
http://www.sun.com

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
NOTICE:  The opinions expressed in this email are held by me,
and may not represent the views of Sun Microsystems, Inc.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFBNIavdQs4kOxk3/MRAmvAAJ9sZfEgIOfINZsT23pa+Y3nQ4lfMACgmN3q
sdgPzBUg9i/c+roWDrXq5BI=
=poSJ
-----END PGP SIGNATURE-----

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Hubert Chan]

>>>>> "Mike" == Mike Waychison <Michael.Waychison@Sun.COM> writes:

Mike> Hans Reiser wrote:
>>
> Why Openat Sucks:
>> 
>> because you can't go cat filenameA/metas/permissions >
>> filenameB/permissions
>> 
>> If cat doesn't work, then we are suffering exactly the problem with
>> namespace fragmentation that this whole scheme was invented to avoid.

Mike> Sure you can.  All you need to do is fix your shell.

And you'll have to fix every other program that you want to use to edit
metadata/attributes/streams.

The whole point of avoiding namespace fragmentation is to avoid having
to fix programs.

Like I said in another post, the world is bigger than just the command
line.  The command line just gives really easy examples of why Hans (and
others) thinks about things the way he does.

-- 
Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/
PGP/GnuPG key: 1024D/124B61FA
Fingerprint: 96C5 012F 5F74 A5F7 1FF7  5291 AF29 C719 124B 61FA
Key available at wwwkeys.pgp.net.   Encrypted e-mail preferred.

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[V13]

On Sunday 29 August 2004 23:21, Hans Reiser wrote:
> The Idea
>
> You should be able to access metadata about a file the same way you
> access the file's data, but with a name based on the filename followed
> by a name to select the metadata of interest.
>
> Examples:
>
> cat song_of_silence/metas/owner
> cat song_of_silence/metas/permissions
> cat 10 > song_of_silence/metas/mixer_defaults/volume
> cat song_of_silence/metas/license

Maybe I'm crazy but:

 You're talking about a major change in the way filesystems work if this is 
going to be used by other FSs too. If  I understand this correctly it is a 
completely new thing and trying to do it by patching existing well-known 
'primitives' may be wrong. 

  AFAIK and AFAICS the metadata are not files or directories. You can look at 
them as files/dirs but they are not, just like a tar is not a directory. I 
believe that the correct thing to do (tm) is to add a new 'concept' named 
'metadata' (which already exists). This way you'll have files, directories 
and metadata (or whatever you call them). So, each directory can have 
metadatas and files and each file can have metadatas. Then you have to 
provide some new methods of accessing them and not to use chdir() etc. (lets 
say chdir_meta() to enter the meta dir which will work for files too). After 
entering the 'metadir' you'll be able to use existing methods etc to access 
its 'files'.

  This approach doesn't mess with existing things and can be extended for 
other filesystems too.

(Just a thought)

<<V13>>

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Spam]

  

> On Sunday 29 August 2004 23:21, Hans Reiser wrote:
>> The Idea
>>
>> You should be able to access metadata about a file the same way you
>> access the file's data, but with a name based on the filename followed
>> by a name to select the metadata of interest.
>>
>> Examples:
>>
>> cat song_of_silence/metas/owner
>> cat song_of_silence/metas/permissions
>> cat 10 > song_of_silence/metas/mixer_defaults/volume
>> cat song_of_silence/metas/license

> Maybe I'm crazy but:

>  You're talking about a major change in the way filesystems work if this is
> going to be used by other FSs too. If  I understand this correctly it is a
> completely new thing and trying to do it by patching existing well-known
> 'primitives' may be wrong. 

>   AFAIK and AFAICS the metadata are not files or directories. You can look at
> them as files/dirs but they are not, just like a tar is not a directory. I
> believe that the correct thing to do (tm) is to add a new 'concept' named
> 'metadata' (which already exists). This way you'll have files, directories
> and metadata (or whatever you call them). So, each directory can have
> metadatas and files and each file can have metadatas. Then you have to
> provide some new methods of accessing them and not to use chdir() etc. (lets
> say chdir_meta() to enter the meta dir which will work for files too). After
> entering the 'metadir' you'll be able to use existing methods etc to access
> its 'files'.

>   This approach doesn't mess with existing things and can be extended for
> other filesystems too.

> (Just a thought)

  It  is a good thought. However I think they are trying to figure out
  a  way  to have the metadata and streams to be accesible with legacy
  applications.

  The  file-as-directory  concept is one way, which still seem to have
  issues.

  How  are  things  done on Windows platforms when there are files and
  directories  with the same name? In Unix that is imposible. How does
  it  work  for  environments  like  Cygwin  etc? What happen to tools
  that run in them?

> <<V13>>

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Spam]

  

> On Sunday 29 August 2004 23:21, Hans Reiser wrote:
>> The Idea
>>
>> You should be able to access metadata about a file the same way you
>> access the file's data, but with a name based on the filename followed
>> by a name to select the metadata of interest.
>>
>> Examples:
>>
>> cat song_of_silence/metas/owner
>> cat song_of_silence/metas/permissions
>> cat 10 > song_of_silence/metas/mixer_defaults/volume
>> cat song_of_silence/metas/license

> Maybe I'm crazy but:

>  You're talking about a major change in the way filesystems work if this is
> going to be used by other FSs too. If  I understand this correctly it is a
> completely new thing and trying to do it by patching existing well-known
> 'primitives' may be wrong. 

>   AFAIK and AFAICS the metadata are not files or directories. You can look at
> them as files/dirs but they are not, just like a tar is not a directory. I
> believe that the correct thing to do (tm) is to add a new 'concept' named
> 'metadata' (which already exists). This way you'll have files, directories
> and metadata (or whatever you call them). So, each directory can have
> metadatas and files and each file can have metadatas. Then you have to
> provide some new methods of accessing them and not to use chdir() etc. (lets
> say chdir_meta() to enter the meta dir which will work for files too). After
> entering the 'metadir' you'll be able to use existing methods etc to access
> its 'files'.

>   This approach doesn't mess with existing things and can be extended for
> other filesystems too.

> (Just a thought)

  It  is a good thought. However I think they are trying to figure out
  a  way  to have the metadata and streams to be accesible with legacy
  applications.

  The  file-as-directory  concept is one way, which still seem to have
  issues.

  How  are  things  done on Windows platforms when there are files and
  directories  with the same name? In Unix that is imposible. How does
  it  work  for  environments  like  Cygwin  etc? What happen to tools
  that run in them?

> <<V13>>

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Tonnerre]

[-- Attachment #1: Type: text/plain, Size: 760 bytes --]

Salut,

On Tue, Aug 31, 2004 at 08:17:36PM +0200, Spam wrote:
>   How  are  things  done on Windows platforms when there are files and
>   directories  with the same name? In Unix that is imposible. How does
>   it  work  for  environments  like  Cygwin  etc? What happen to tools
>   that run in them?

In  NTFS it's  illegal  IIRC.  At least  the  fs correction  utilities
complain about a block being assigned to two files.

Same with HFS+.

Sometimes  there seem to  be several  things with  the same  name. But
that's because of hidden extensions (.lnk for example).

I'm talking out of the book here, maybe the real-world implementations
of Windows  are different. I can't  tell, I only used  Windows once to
ssh into a screwed-up router.

				Tonnerre


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Spam]

  

> Salut,

> On Tue, Aug 31, 2004 at 08:17:36PM +0200, Spam wrote:
>>   How  are  things  done on Windows platforms when there are files and
>>   directories  with the same name? In Unix that is imposible. How does
>>   it  work  for  environments  like  Cygwin  etc? What happen to tools
>>   that run in them?

> In  NTFS it's  illegal  IIRC.  At least  the  fs correction  utilities
> complain about a block being assigned to two files.

  I  meant  a  file  and a directory with the same name, not two files
  with the same name :) subtle but important difference.

  ie,  you can have a file named "foo" and a directory named "foo" and
  they won't collide.
  

> Same with HFS+.

> Sometimes  there seem to  be several  things with  the same  name. But
> that's because of hidden extensions (.lnk for example).

> I'm talking out of the book here, maybe the real-world implementations
> of Windows  are different. I can't  tell, I only used  Windows once to
> ssh into a screwed-up router.

> 				Tonnerre

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Spam]

  

> Salut,

> On Tue, Aug 31, 2004 at 08:17:36PM +0200, Spam wrote:
>>   How  are  things  done on Windows platforms when there are files and
>>   directories  with the same name? In Unix that is imposible. How does
>>   it  work  for  environments  like  Cygwin  etc? What happen to tools
>>   that run in them?

> In  NTFS it's  illegal  IIRC.  At least  the  fs correction  utilities
> complain about a block being assigned to two files.

  I  meant  a  file  and a directory with the same name, not two files
  with the same name :) subtle but important difference.

  ie,  you can have a file named "foo" and a directory named "foo" and
  they won't collide.
  

> Same with HFS+.

> Sometimes  there seem to  be several  things with  the same  name. But
> that's because of hidden extensions (.lnk for example).

> I'm talking out of the book here, maybe the real-world implementations
> of Windows  are different. I can't  tell, I only used  Windows once to
> ssh into a screwed-up router.

> 				Tonnerre

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Robin Rosenberg]

On Tuesday 31 August 2004 21.38, Spam wrote:
> > Salut,
> >
> > On Tue, Aug 31, 2004 at 08:17:36PM +0200, Spam wrote:
> >>   How  are  things  done on Windows platforms when there are files and
> >>   directories  with the same name? In Unix that is imposible. How does
> >>   it  work  for  environments  like  Cygwin  etc? What happen to tools
> >>   that run in them?
> >
> > In  NTFS it's  illegal  IIRC.  At least  the  fs correction  utilities
> > complain about a block being assigned to two files.
>
>   I  meant  a  file  and a directory with the same name, not two files
>   with the same name :) subtle but important difference.
>
>   ie,  you can have a file named "foo" and a directory named "foo" and
>   they won't collide.

You can't have a file and a directory with the same name in W*.

Alternative data streams don't appear in a normal directory. You
need special API:s and tools to see them. If you know the name of and ADS
you can however access them with standard tools, including cygwin.

echo foo >a.txt
echo bar >a.txt:b.txt

You can also have ADS's on directories. Not sure how Reiserfs4 does that.

mkdir foo
echo bar >foo:a.txt

-- robin

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Robin Rosenberg]

On Tuesday 31 August 2004 21.38, Spam wrote:
> > Salut,
> >
> > On Tue, Aug 31, 2004 at 08:17:36PM +0200, Spam wrote:
> >>   How  are  things  done on Windows platforms when there are files and
> >>   directories  with the same name? In Unix that is imposible. How does
> >>   it  work  for  environments  like  Cygwin  etc? What happen to tools
> >>   that run in them?
> >
> > In  NTFS it's  illegal  IIRC.  At least  the  fs correction  utilities
> > complain about a block being assigned to two files.
>
>   I  meant  a  file  and a directory with the same name, not two files
>   with the same name :) subtle but important difference.
>
>   ie,  you can have a file named "foo" and a directory named "foo" and
>   they won't collide.

You can't have a file and a directory with the same name in W*.

Alternative data streams don't appear in a normal directory. You
need special API:s and tools to see them. If you know the name of and ADS
you can however access them with standard tools, including cygwin.

echo foo >a.txt
echo bar >a.txt:b.txt

You can also have ADS's on directories. Not sure how Reiserfs4 does that.

mkdir foo
echo bar >foo:a.txt

-- robin

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Giuseppe Bilotta]

Robin Rosenberg wrote:
> On Tuesday 31 August 2004 21.38, Spam wrote:
> > > Salut,
> > >
> > > On Tue, Aug 31, 2004 at 08:17:36PM +0200, Spam wrote:
> > >>   How  are  things  done on Windows platforms when there are files and
> > >>   directories  with the same name? In Unix that is imposible. How does
> > >>   it  work  for  environments  like  Cygwin  etc? What happen to tools
> > >>   that run in them?
> > >
> > > In  NTFS it's  illegal  IIRC.  At least  the  fs correction  utilities
> > > complain about a block being assigned to two files.
> >
> >   I  meant  a  file  and a directory with the same name, not two files
> >   with the same name :) subtle but important difference.
> >
> >   ie,  you can have a file named "foo" and a directory named "foo" and
> >   they won't collide.
> 
> You can't have a file and a directory with the same name in W*.
> 
> Alternative data streams don't appear in a normal directory. You
> need special API:s and tools to see them. If you know the name of and ADS
> you can however access them with standard tools, including cygwin.
> 
> echo foo >a.txt
> echo bar >a.txt:b.txt
> 
> You can also have ADS's on directories. Not sure how Reiserfs4 does that.
> 
> mkdir foo
> echo bar >foo:a.txt

The problem with this approach is that some tools will not 
recognize the file:ADS syntax, and complain. This is what 
happens under Windows, at least. Would the same happen under 
Linux, I wonder?

Say that we choose a reserved character (say, #; is # a 
reserved character?) to mark the frontier between the file and 
the ADS/attributes/whatever. Would old programs still be able 
to parse the (commandline or anywhere else) "URL" specification 
and get to the ADS/attribute/whatever?

Say, would cat somefile#an_ads work, or would cat (as it is 
now) complain in some way?

-- 
Giuseppe "Oblomov" Bilotta

Can't you see
It all makes perfect sense
Expressed in dollar and cents
Pounds shillings and pence
                  (Roger Waters)

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[V13]

On Tuesday 31 August 2004 21:17, Spam wrote:
> > On Sunday 29 August 2004 23:21, Hans Reiser wrote:
> >> The Idea
> >>
> >> You should be able to access metadata about a file the same way you
> >> access the file's data, but with a name based on the filename followed
> >> by a name to select the metadata of interest.
> >>
> >> Examples:
> >>
> >> cat song_of_silence/metas/owner
> >> cat song_of_silence/metas/permissions
> >> cat 10 > song_of_silence/metas/mixer_defaults/volume
> >> cat song_of_silence/metas/license
> >
> > Maybe I'm crazy but:
> >
> >  You're talking about a major change in the way filesystems work if this
> > is going to be used by other FSs too. If  I understand this correctly it
> > is a completely new thing and trying to do it by patching existing
> > well-known 'primitives' may be wrong.
> >
> >   AFAIK and AFAICS the metadata are not files or directories. You can
> > look at them as files/dirs but they are not, just like a tar is not a
> > directory. I believe that the correct thing to do (tm) is to add a new
> > 'concept' named 'metadata' (which already exists). This way you'll have
> > files, directories and metadata (or whatever you call them). So, each
> > directory can have metadatas and files and each file can have metadatas.
> > Then you have to provide some new methods of accessing them and not to
> > use chdir() etc. (lets say chdir_meta() to enter the meta dir which will
> > work for files too). After entering the 'metadir' you'll be able to use
> > existing methods etc to access its 'files'.
> >
> >   This approach doesn't mess with existing things and can be extended for
> > other filesystems too.
> >
> > (Just a thought)
>
>   It  is a good thought. However I think they are trying to figure out
>   a  way  to have the metadata and streams to be accesible with legacy
>   applications.

They will be since after chdir_meta() the user will be able to look at the 
metadata just like Hans described it. The only thing that changes (from the 
userland POV) is the way someone can enter the 'metadata directory'. This way 
you don't have to have a special name, just a special function and no 
existing application (like tar) can possibly break because it will not know 
how to enter this 'metadata directory'.

<<V13>>

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Hubert Chan]

>>>>> "V13" == V13  <v13@priest.com> writes:

V13> On Tuesday 31 August 2004 21:17, Spam wrote:

>> It is a good thought. However I think they are trying to figure out a
>> way to have the metadata and streams to be accesible with legacy
>> applications.

V13> They will be since after chdir_meta() the user will be able to look
V13> at the metadata just like Hans described it.

Uh, no.  Legacy means no changes made.  It shouldn't need to be modified
to use chdir_meta.

From what I can see, your proposal isn't much different from the openat
proposal.

V13> The only thing that changes (from the userland POV) is the way
V13> someone can enter the 'metadata directory'. This way you don't have
V13> to have a special name, just a special function and no existing
V13> application (like tar) can possibly break because it will not know
V13> how to enter this 'metadata directory'.

tar won't be able to backup the metadata.  That's the major breakage of
tar that we're worried about.

-- 
Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/
PGP/GnuPG key: 1024D/124B61FA
Fingerprint: 96C5 012F 5F74 A5F7 1FF7  5291 AF29 C719 124B 61FA
Key available at wwwkeys.pgp.net.   Encrypted e-mail preferred.

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Spam]

V13>> The only thing that changes (from the userland POV) is the way
V13>> someone can enter the 'metadata directory'. This way you don't have
V13>> to have a special name, just a special function and no existing
V13>> application (like tar) can possibly break because it will not know
V13>> how to enter this 'metadata directory'.

> tar won't be able to backup the metadata.  That's the major breakage of
> tar that we're worried about.

  However,  if  we do a "cp fileA fileB" then the metadata and streams
  ought  to be copied too, even if "cp" does not support them. This is
  the  real  challenge. Backup tools like tar can be patched just like
  it has so many times before.

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Hubert Chan]

>>>>> "Spam" == Spam  <spam@tnonline.net> writes:

V13> The only thing that changes (from the userland POV) is the way
V13> someone can enter the 'metadata directory'. This way you don't have
V13> to have a special name, just a special function and no existing
V13> application (like tar) can possibly break because it will not know
V13> how to enter this 'metadata directory'.

>> tar won't be able to backup the metadata.  That's the major breakage
>> of tar that we're worried about.

Spam>   However, if we do a "cp fileA fileB" then the metadata and
Spam> streams ought to be copied too, even if "cp" does not support
Spam> them.

Huh?  How do you plan on pulling that off?  Unless cp uses the
not-yet-existing copy syscall, if cp can't see the metadata or streams,
how is it going to copy it?

Spam> This is the real challenge. Backup tools like tar can be patched
Spam> just like it has so many times before.

Yes.  And if we can get file-as-dir, then we only need to patch tar
once, since everything can be exported through that interface.

-- 
Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/
PGP/GnuPG key: 1024D/124B61FA
Fingerprint: 96C5 012F 5F74 A5F7 1FF7  5291 AF29 C719 124B 61FA
Key available at wwwkeys.pgp.net.   Encrypted e-mail preferred.

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Spam]

  

>>>>>> "Spam" == Spam  <spam@tnonline.net> writes:

V13>> The only thing that changes (from the userland POV) is the way
V13>> someone can enter the 'metadata directory'. This way you don't have
V13>> to have a special name, just a special function and no existing
V13>> application (like tar) can possibly break because it will not know
V13>> how to enter this 'metadata directory'.

>>> tar won't be able to backup the metadata.  That's the major breakage
>>> of tar that we're worried about.

Spam>>   However, if we do a "cp fileA fileB" then the metadata and
Spam>> streams ought to be copied too, even if "cp" does not support
Spam>> them.

> Huh?  How do you plan on pulling that off?  Unless cp uses the
> not-yet-existing copy syscall, if cp can't see the metadata or streams,
> how is it going to copy it?

  My first thought would change the API that cp uses to copy the file.
  But  in  an earlier response on this message thread I have found out
  that  there is no such API (well there is, but it is linked into the
  program)  and  cp  in fact itself is doing the copying. this is also
  what  I  objected  against  before. It is a bad design and should be
  attended  much  higher  interest to change than just adding adding a
  new type of file-as-dir.

Spam>> This is the real challenge. Backup tools like tar can be patched
Spam>> just like it has so many times before.

> Yes.  And if we can get file-as-dir, then we only need to patch tar
> once, since everything can be exported through that interface.

  Yes. This seem to be an acceptable way to do things. But next time
  someone comes and want to do changes like this we need to start
  patching things again. If there was an API that was separate from
  the programs then new features could be included much more easily as
  things could be done behind the scenes. ie the "cp fileA fileB"
  would succeed and all the extended attributes, metas, streams etc
  would be copied too. Nothing would ever be lost unless copying to a
  filesystem that doesn't support the special stuff. (as with NTFS
  file streams).

  ~S

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Hubert Chan]

>>>>> "Spam" == Spam  <spam@tnonline.net> writes:

>> Yes.  And if we can get file-as-dir, then we only need to patch tar
>> once, since everything can be exported through that interface.

Spam>   Yes. This seem to be an acceptable way to do things. But next
Spam> time someone comes and want to do changes like this we need to
Spam> start patching things again. ...

But that's my point.  Anything that I can imagine anyone ever wanting to
extend the filesystem to do can be exported through this file-as-dir
system.  ACLs (or insert your favourite access-control scheme here),
xattrs, streams.  If you want to add something new, just make sure that
it can be seen through the file-as-dir interface, and you don't need to
patch tar again.  Want to use mandatory access control instead of
acl's?  Just export through foo.txt/metas/mac_level, and tar will catch
it with no problems.  Want to let your streams have substreams too?
export through foo.txt/streams/bar/streams/baz, and tar will still
catch it.

And you get the added bonus that you can use any program to edit these
extra things, no modifications needed.

-- 
Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/
PGP/GnuPG key: 1024D/124B61FA
Fingerprint: 96C5 012F 5F74 A5F7 1FF7  5291 AF29 C719 124B 61FA
Key available at wwwkeys.pgp.net.   Encrypted e-mail preferred.

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Hubert Chan]

>>>>> "V13" == V13  <v13@priest.com> writes:

V13> Maybe I'm crazy but:

V13>  You're talking about a major change in the way filesystems work if
V13> this is going to be used by other FSs too. If I understand this
V13> correctly it is a completely new thing and trying to do it by
V13> patching existing well-known 'primitives' may be wrong.

V13>   AFAIK and AFAICS the metadata are not files or directories. You
V13> can look at them as files/dirs but they are not, just like a tar is
V13> not a directory. I believe that the correct thing to do (tm) is to
V13> add a new 'concept' named 'metadata' (which already exists). ...

And you'll need a new 'concept' named 'streams'.  And a new concept for
every extension you want/need to add to the filesystem.  By exposing
everything through the normal filesystem interface, you don't need to
make any modifications to your programs to use the new concepts.
Everything just works as expected.  (With the possible exception of
backup-like software, but it is easier to fix just the backup software
than every other program that's out there.)

The hypothetical object /usr/bin/xterm/icon.png looks like a file,
smells like a file, and quacks like a file.  Why shouldn't it be a file?

V13> This way you'll have files, directories and metadata (or whatever
V13> you call them). So, each directory can have metadatas and files and
V13> each file can have metadatas. Then you have to provide some new
V13> methods of accessing them and not to use chdir() etc. (lets say
V13> chdir_meta() to enter the meta dir which will work for files
V13> too). After entering the 'metadir' you'll be able to use existing
V13> methods etc to access its 'files'.

V13>   This approach doesn't mess with existing things and can be
V13> extended for other filesystems too.

Either way, filesystems that want to implement that interface will need
to be changed.  The question is just what kind of interface we will
expose to the users/programs.  And since that's what the question is,
it seems to me that we want to expose it through an interface that the
users are already used to.

-- 
Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/
PGP/GnuPG key: 1024D/124B61FA
Fingerprint: 96C5 012F 5F74 A5F7 1FF7  5291 AF29 C719 124B 61FA
Key available at wwwkeys.pgp.net.   Encrypted e-mail preferred.

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Chris Dawes]

On Tue, 2004-08-31 at 18:55, V13 wrote:
...
>   AFAIK and AFAICS the metadata are not files or directories. You can look at 
> them as files/dirs but they are not, just like a tar is not a directory. I 
> believe that the correct thing to do (tm) is to add a new 'concept' named 
> 'metadata' (which already exists). This way you'll have files, directories 
> and metadata (or whatever you call them). So, each directory can have 
> metadatas and files and each file can have metadatas. Then you have to 
> provide some new methods of accessing them and not to use chdir() etc. (lets 
> say chdir_meta() to enter the meta dir which will work for files too). After 
> entering the 'metadir' you'll be able to use existing methods etc to access 
> its 'files'.

I think this is what Hans is trying to avoid -- it results in creating a
separate namespace for the metadata.

I would agree with the notion that metadata nodes are light-weight files
in that they themselves have no metadata associated with them, but I
don't see why they need separate system-calls (if we can help it).

Chris Dawes.


> 
>   This approach doesn't mess with existing things and can be extended for 
> other filesystems too.
> 
> (Just a thought)
> 
> <<V13>>

Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why

[Hans Reiser]

V13 wrote:

>
>
>  AFAIK and AFAICS the metadata are not files or directories.
>
Yes they are. In reiser4. They might be stored different, but their 
interface is what counts.