Symlink indirection

Symlink indirection

[Andrew Walrond]

Quick question;

Is the number of allowed levels of symlink indirection (if that is the 
right phrase; I mean symlink -> symlink -> ... -> file) dependant on the 
kernel, or libc ? Where is it defined, and can it be changed?

TIA
Andrew

Re: Symlink indirection

[Marc-Christian Petersen]

On Friday 13 December 2002 16:06, Andrew Walrond wrote:

Hi Andrew,

> Is the number of allowed levels of symlink indirection (if that is the
> right phrase; I mean symlink -> symlink -> ... -> file) dependant on the
> kernel, or libc ? Where is it defined, and can it be changed?

fs/namei.c

 if (current->link_count >= 5)

change to a higher value.

So, the answer is: Kernel :)

ciao, Marc

Re: Symlink indirection

[Andrew Walrond]

Thanks Marc

5 is very low isn't it? Certainly marginal for an application I have in 
mind.

What's the reasoning behind it being a) 5 and b) so low?

Marc-Christian Petersen wrote:
> On Friday 13 December 2002 16:06, Andrew Walrond wrote:
> 
> Hi Andrew,
> 
> 
>>Is the number of allowed levels of symlink indirection (if that is the
>>right phrase; I mean symlink -> symlink -> ... -> file) dependant on the
>>kernel, or libc ? Where is it defined, and can it be changed?
> 
> 
> fs/namei.c
> 
>  if (current->link_count >= 5)
> 
> change to a higher value.
> 
> So, the answer is: Kernel :)
> 
> ciao, Marc
> 

Re: Symlink indirection

[Alan Cox]

On Fri, 2002-12-13 at 15:20, Andrew Walrond wrote:
> Thanks Marc
> 
> 5 is very low isn't it? Certainly marginal for an application I have in 
> mind.

8 is more normal, but their are recursion and stack size considerations

Re: Symlink indirection

[Richard B. Johnson]

On Fri, 13 Dec 2002, Marc-Christian Petersen wrote:

> On Friday 13 December 2002 16:06, Andrew Walrond wrote:
> 
> Hi Andrew,
> 
> > Is the number of allowed levels of symlink indirection (if that is the
> > right phrase; I mean symlink -> symlink -> ... -> file) dependant on the
> > kernel, or libc ? Where is it defined, and can it be changed?
> 
> fs/namei.c
> 
>  if (current->link_count >= 5)
> 
> change to a higher value.
> 
> So, the answer is: Kernel :)
> 
> ciao, Marc

No, that thing (whetever it is) is different.

Script started on Fri Dec 13 10:26:30 2002
# file *
foo:        symbolic link to ../foo
typescript: empty
# pwd
/root/foo
# cd *
# cd *
# cd *
# cd *
# cd *
# cd *
# pwd
/root/foo/foo/foo/foo/foo/foo/foo
# cd *
# cd *
# cd *
# pwd
/root/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo
# cd *
# cd *
# cd *
# pwd
/root/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo
# exit
exit

Script done on Fri Dec 13 10:27:21 2002


You can do this until you run out of string-space. Your "link-count"
has something to do with something else.


Cheers,
Dick Johnson
Penguin : Linux version 2.4.18 on an i686 machine (797.90 BogoMips).
Why is the government concerned about the lunatic fringe? Think about it.

Re: Symlink indirection

[James Antill]

"Richard B. Johnson" <root@chaos.analogic.com> writes:

> On Fri, 13 Dec 2002, Marc-Christian Petersen wrote:
> 
> > On Friday 13 December 2002 16:06, Andrew Walrond wrote:
> > 
> > Hi Andrew,
> > 
> > > Is the number of allowed levels of symlink indirection (if that is the
> > > right phrase; I mean symlink -> symlink -> ... -> file) dependant on the
> > > kernel, or libc ? Where is it defined, and can it be changed?
> > 
> > fs/namei.c
> > 
> >  if (current->link_count >= 5)
> > 
> > change to a higher value.
> > 
> > So, the answer is: Kernel :)
> > 
> > ciao, Marc
> 
> No, that thing (whetever it is) is different.
> 
> Script started on Fri Dec 13 10:26:30 2002
> # file *
> foo:        symbolic link to ../foo

 *sigh*, you are following one symlink at a time ... what is this
proving ?

> You can do this until you run out of string-space. Your "link-count"
> has something to do with something else.

 The link count is for recursively following symlinks, as the original
question wanted to know ... and has been discussed on lkml numerous
times.

 Andrew, one extra piece of information you might not know is that the
above value doesn't come into play when the new symlink is the last
element in the new path, then you get a higher value.
 The full code...

        if (current->link_count >= max_recursive_link)
                goto loop;
        if (current->total_link_count >= 40)
                goto loop;
[...]
        current->link_count++;
        current->total_link_count++;
        UPDATE_ATIME(dentry->d_inode);
        err = dentry->d_inode->i_op->follow_link(dentry, nd);
        current->link_count--;

...Ie. a link from /a -> /b/c where "b" is a symlink takes the
"max_recursive_link" value (5 on vanilla kernels) but if "/b/c" was a
symlink then you get to use the 40 value.

-- 
# James Antill -- james@and.org
:0:
* ^From: .*james@and\.org
/dev/null

Re: Symlink indirection

[Andrew Walrond]

Hi James.

Thanks for the info; my application is now entirely plausible :)
And apologies for asking an FAQ. (Google didn't throw up anything useful)

BTW is documented anywhere except the code?

Andrew

> 
>  The link count is for recursively following symlinks, as the original
> question wanted to know ... and has been discussed on lkml numerous
> times.
> 
>  Andrew, one extra piece of information you might not know is that the
> above value doesn't come into play when the new symlink is the last
> element in the new path, then you get a higher value.
>  The full code...
> 
>         if (current->link_count >= max_recursive_link)
>                 goto loop;
>         if (current->total_link_count >= 40)
>                 goto loop;
> [...]
>         current->link_count++;
>         current->total_link_count++;
>         UPDATE_ATIME(dentry->d_inode);
>         err = dentry->d_inode->i_op->follow_link(dentry, nd);
>         current->link_count--;
> 
> ...Ie. a link from /a -> /b/c where "b" is a symlink takes the
> "max_recursive_link" value (5 on vanilla kernels) but if "/b/c" was a
> symlink then you get to use the 40 value.
> 

Re: Symlink indirection

[James Antill]

Andrew Walrond <andrew@walrond.org> writes:

> Hi James.
> 
> Thanks for the info; my application is now entirely plausible :)
> And apologies for asking an FAQ. (Google didn't throw up anything useful)

 Sorry, this was an error on my part, I shouldn't reply to Richard's
posts until I've calmed down.

> BTW is documented anywhere except the code?

 Apart from this mailing list, not AFAIK.

> >  The link count is for recursively following symlinks, as the
> > original
> 
> > question wanted to know ... and has been discussed on lkml numerous
> > times.
> >  Andrew, one extra piece of information you might not know is that
> > the
> 
> > above value doesn't come into play when the new symlink is the last
> > element in the new path, then you get a higher value.
> >  The full code...
> >         if (current->link_count >= max_recursive_link)
> 
> >                 goto loop;
> >         if (current->total_link_count >= 40)
> >                 goto loop;
> > [...]
> >         current->link_count++;
> >         current->total_link_count++;
> >         UPDATE_ATIME(dentry->d_inode);
> >         err = dentry->d_inode->i_op->follow_link(dentry, nd);
> >         current->link_count--;
> > ...Ie. a link from /a -> /b/c where "b" is a symlink takes the
> 
> > "max_recursive_link" value (5 on vanilla kernels) but if "/b/c" was a
> > symlink then you get to use the 40 value.

 The last part of the path is still part of the follow_link, and thus
subject to the non-total link_count value. The only times you get the
bigger value is...

/a     -> /b/c
/b     -> /b.1     -> /b.2     -> /b.3     -> /b.4
/b.4/c -> /b.4/c.1 -> /b.4/c.2 -> /b.4/c.3 -> /b.4/c.4

...here you can open /a even if you'll follow more than ->link_count
links.

-- 
# James Antill -- james@and.org
:0:
* ^From: .*james@and\.org
/dev/null

Re: Symlink indirection

[Andrew Walrond]

No, I think marc was right...

daedalus@bob sym $ ls -l
total 4
lrwxrwxrwx    1 daedalus users           1 Dec 13 16:19 a -> b
lrwxrwxrwx    1 daedalus users           1 Dec 13 16:19 b -> c
lrwxrwxrwx    1 daedalus users           1 Dec 13 16:20 c -> d
lrwxrwxrwx    1 daedalus users           1 Dec 13 16:20 d -> e
lrwxrwxrwx    1 daedalus users           1 Dec 13 16:20 e -> f
lrwxrwxrwx    1 daedalus users           1 Dec 13 16:20 f -> g
lrwxrwxrwx    1 daedalus users           4 Dec 13 16:21 g -> test
-rw-r--r--    1 daedalus users           6 Dec 13 16:18 test
daedalus@bob sym $ cat a
cat: a: Too many levels of symbolic links
daedalus@bob sym $ cat b
cat: b: Too many levels of symbolic links
daedalus@bob sym $ cat c
Hello

Re: Symlink indirection

[Jesse Pollard]

On Friday 13 December 2002 09:30 am, Richard B. Johnson wrote:
> On Fri, 13 Dec 2002, Marc-Christian Petersen wrote:
> > On Friday 13 December 2002 16:06, Andrew Walrond wrote:
> >
> > Hi Andrew,
> >
> > > Is the number of allowed levels of symlink indirection (if that is the
> > > right phrase; I mean symlink -> symlink -> ... -> file) dependant on
> > > the kernel, or libc ? Where is it defined, and can it be changed?
> >
> > fs/namei.c
> >
> >  if (current->link_count >= 5)
> >
> > change to a higher value.
> >
> > So, the answer is: Kernel :)
> >
> > ciao, Marc
>
> No, that thing (whetever it is) is different.
>
> Script started on Fri Dec 13 10:26:30 2002
> # file *
> foo:        symbolic link to ../foo
> typescript: empty
> # pwd
> /root/foo
> # cd *
> # cd *
> # cd *
> # cd *
> # cd *
> # cd *
> # pwd
> /root/foo/foo/foo/foo/foo/foo/foo
> # cd *
> # cd *
> # cd *
> # pwd
> /root/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo
> # cd *
> # cd *
> # cd *
> # pwd
> /root/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo/foo
> # exit
> exit
>
> Script done on Fri Dec 13 10:27:21 2002
>
>
> You can do this until you run out of string-space. Your "link-count"
> has something to do with something else.

Example isn't the same thing:

[pollard@merlin ~/test]$ echo "test" >target
[pollard@merlin ~/test]$ ln -s target a				- 1 link
[pollard@merlin ~/test]$ ln -s a b					- 2 links
[pollard@merlin ~/test]$ ln -s b c					- 3
[pollard@merlin ~/test]$ ln -s c d					- 4
[pollard@merlin ~/test]$ cat d
test
[pollard@merlin ~/test]$ ln -s d e					-5
[pollard@merlin ~/test]$ cat e
test
[pollard@merlin ~/test]$ ln -s e f					-6
[pollard@merlin ~/test]$ cat f
cat: f: Too many levels of symbolic links

This catches problem situations like:

[pollard@merlin ~/test]$ ln -s loop loop
[pollard@merlin ~/test]$ ls -l loop
lrwxrwxrwx    1 pollard  NA0101          4 Dec 13 10:24 loop -> loop
[pollard@merlin ~/test]$ cat loop
cat: loop: Too many levels of symbolic links

-- 
-------------------------------------------------------------------------
Jesse I Pollard, II
Email: pollard@navo.hpc.mil

Any opinions expressed are solely my own.

Re: Symlink indirection

[Richard B. Johnson]

On Fri, 13 Dec 2002, Andrew Walrond wrote:

> Quick question;
> 
> Is the number of allowed levels of symlink indirection (if that is the 
> right phrase; I mean symlink -> symlink -> ... -> file) dependant on the 
> kernel, or libc ? Where is it defined, and can it be changed?
> 
> TIA
> Andrew
> 

Since a symlink is just a file containing a name, the resulting path
length is simply the maximum path length that user-space tools allow.
This should be defined as "PATH_MAX". Posix defines this as 255 characters
but I think posix requires that this be the minimum and all file-name
handling buffers must be at least PATH_MAX in length.

A hard link is just another directory-entry for the same file. This,
therefore follows the same rules. There must be enough space on the
device to contain the number of directory entries, as well as enough
buffer length in the tools necessary to manipulate these "nested"
directories, which are not really "nested" at all. 


Cheers,
Dick Johnson
Penguin : Linux version 2.4.18 on an i686 machine (797.90 BogoMips).
Why is the government concerned about the lunatic fringe? Think about it.

Re: Symlink indirection

[Alfred M. Szmidt]

   This should be defined as "PATH_MAX".

This should only be defined if such an limit exists.  Systems like
GNU/Hurd do not have this limit, but GNU/Linux does.

Re: Symlink indirection

[Jeff Bailey]

[-- Attachment #1: Type: text/plain, Size: 1053 bytes --]

On Fri, 2002-12-13 at 10:23, Richard B. Johnson wrote:

> Since a symlink is just a file containing a name, the resulting path
> length is simply the maximum path length that user-space tools allow.
> This should be defined as "PATH_MAX". Posix defines this as 255 characters
> but I think posix requires that this be the minimum and all file-name
> handling buffers must be at least PATH_MAX in length.

Small nit here, the Posix draft I have says that the definition shall be
omitted from the <limits.h> header on specific implementation where the
corresponding value is equal or greater than the stated minimum, but
where the value can vary depending on the file to which it is applied.

Please don't ever rely on PATH_MAX existing.  pathconf() is a better
tool.  We spend a lot of time chasing authors to explain to them why it
really is okay for the Hurd (i386-pc-gnu)'s libc to omit this
definition.

Tks,
Jeff Bailey 

-- 
When you get to the heart,
use a knife and fork.
 - From instructions on how to eat an artichoke.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Symlink indirection

[Amos Waterland]

I think that you and Richard are dicussing a slightly different issue
than the original poster asked about.  The original question was:

On Fri, 13 Dec 2002, Andrew Walrond wrote:
> Is the number of allowed levels of symlink indirection (if that is the 
> right phrase; I mean symlink -> symlink -> ... -> file) dependant on the 
> kernel, or libc ? Where is it defined, and can it be changed?

To which Richard replied:

> Since a symlink is just a file containing a name, the resulting path
> length is simply the maximum path length that user-space tools allow.
> This should be defined as "PATH_MAX". Posix defines this as 255
> characters but I think posix requires that this be the minimum and all
> file-name handling buffers must be at least PATH_MAX in length.
>
> A hard link is just another directory-entry for the same file. This,
> therefore follows the same rules. There must be enough space on the
> device to contain the number of directory entries, as well as enough
> buffer length in the tools necessary to manipulate these "nested"
> directories, which are not really "nested" at all. 

But Richard is not actually completely correct.  There is a limit of 5
levels of symlink indirection in vanilla 2.4 series Linux kernels.

  % touch 0
  % for i in `seq 1 10`; do ln -s `ls | sort | tail -1` $i; done
  % ls
  0  1  10  2  3  4  5  6  7  8  9
  % cat 5
  % cat 6
  cat: 6: Too many levels of symbolic links
  % strace cat 6 2>&1 | grep 'open("6",'
  open("6", O_RDONLY|O_LARGEFILE) = -1 ELOOP (Too many levels of symbolic links)

This has been discussed by Al Viro et al. many times on lkml.  I believe
that it is not a user-space or POSIX issue, but rather a kernel issue.
Thanks.

Amos Waterland

Re: Symlink indirection

[Richard B. Johnson]

On Fri, 13 Dec 2002, Amos Waterland wrote:

> I think that you and Richard are dicussing a slightly different issue
> than the original poster asked about.  The original question was:
> 
> On Fri, 13 Dec 2002, Andrew Walrond wrote:
> > Is the number of allowed levels of symlink indirection (if that is the 
> > right phrase; I mean symlink -> symlink -> ... -> file) dependant on the 
> > kernel, or libc ? Where is it defined, and can it be changed?
> 
> To which Richard replied:
> 
> > Since a symlink is just a file containing a name, the resulting path
> > length is simply the maximum path length that user-space tools allow.
> > This should be defined as "PATH_MAX". Posix defines this as 255
> > characters but I think posix requires that this be the minimum and all
> > file-name handling buffers must be at least PATH_MAX in length.
> >
> > A hard link is just another directory-entry for the same file. This,
> > therefore follows the same rules. There must be enough space on the
> > device to contain the number of directory entries, as well as enough
> > buffer length in the tools necessary to manipulate these "nested"
> > directories, which are not really "nested" at all. 
> 
> But Richard is not actually completely correct.  There is a limit of 5
> levels of symlink indirection in vanilla 2.4 series Linux kernels.
> 
>   % touch 0
>   % for i in `seq 1 10`; do ln -s `ls | sort | tail -1` $i; done
>   % ls
>   0  1  10  2  3  4  5  6  7  8  9
>   % cat 5
>   % cat 6
>   cat: 6: Too many levels of symbolic links
>   % strace cat 6 2>&1 | grep 'open("6",'
>   open("6", O_RDONLY|O_LARGEFILE) = -1 ELOOP (Too many levels of symbolic links)
> 
> This has been discussed by Al Viro et al. many times on lkml.  I believe
> that it is not a user-space or POSIX issue, but rather a kernel issue.
> Thanks.
> 
> Amos Waterland
> 

Yep. I thought the original poster was talking about following
the links, i.e., "indirection", rather than creating links which
is "definition".

So, the kernel does set a limit on the number of symlinks of the form,

ln -s a b ; ln -s b c ; ln -s c d ; ln -s d e ; # etc.


Cheers,
Dick Johnson
Penguin : Linux version 2.4.18 on an i686 machine (797.90 BogoMips).
Why is the government concerned about the lunatic fringe? Think about it.

Re: Symlink indirection

[Pete Zaitcev]

>> Is the number of allowed levels of symlink indirection (if that is the
>> right phrase; I mean symlink -> symlink -> ... -> file) dependant on the
>> kernel, or libc ? Where is it defined, and can it be changed?
> 
> fs/namei.c
> 
>  if (current->link_count >= 5)
> 
> change to a higher value.

This is vey, very misleading statement. The counter mentioned above
is there to protect stacks from overflow, but our symlink resolution
is largely non-recursive, and certainly not in case of a tail
recursion within the same directory.

Also, consider this message from Al:

----------------------------------------------------
From: Alexander Viro <aviro@redhat.com>
Date: Mon, 3 Jun 2002 13:01:16 -0400

On Mon, Jun 03, 2002 at 12:21:59PM -0400, Matt Wilson wrote:
> SUSv3:
>
> http://www.opengroup.org/onlinepubs/007904975/basedefs/limits.h.html
>
> {_POSIX_SYMLOOP_MAX} The number of symbolic links that can be
>     traversed in the resolution of a pathname in the absence of a
>     loop.  Value: 8

... and that has nothing to said limit of 5.  What we do have limited
is the number of nested symlinks.  4BSD and derived Unices limit the
total amount of symlinks traveresed in pathname resolution.  Example:
if you have

/a -> b
/b/a -> b
/b/b/a -> b
/b/b/b/a -> b
/b/b/b/b/a -> b
/b/b/b/b/b/a -> b
/b/b/b/b/b/b/a -> b
/b/b/b/b/b/b/b/a -> b
/b/b/b/b/b/b/b/b/a -> b
 
the pathname
 
/a/a/a/a/a/a/a/a/a
 
will resolve to
 
/b/b/b/b/b/b/b/b/b
 
and there will be 9 symlink traversals done during the pathname resolution.
However, the depth (i.e. the thing we do limit) is 1 in that case.  OTOH,
with

/1 -> 2/a
/2 -> 3/a
/3 -> 4/a
/4 -> 5/a
/5 -> 6/a
/6 -> 7/a

/1

will resolve to

/7/a/a/a/a/a/a

with 6 symlink traversals and depth 6.  IOW, SuS limit is not an argument
for changing the Linux one - they limit different things.  They are not
independent, of course, since depth of recursion can't be greater than
total amount of symlinks we had traversed, but that's a separate story.

Frankly, all cases when I had seen the nested symlink farms of that
depth would be better served by use of bindings - these are not subject
to any limits on nesting and avoid a lot of PITA inherent to symlink
farms.  To put it another way, nested symlink farms grow from attempts
to work around the lack of bindings.  It's not that you need to replace
all symlinks with bindings, of course - the crown of the tree is usually
OK, it's the trunk that acts as source of pain.

Re: Symlink indirection

[Andrew Walrond]

Pete,

Sorry for being dense, but what do you mean by 'bindings' ? Hard links?

Andrew

> Frankly, all cases when I had seen the nested symlink farms of that
> depth would be better served by use of bindings - these are not subject
> to any limits on nesting and avoid a lot of PITA inherent to symlink
> farms.  To put it another way, nested symlink farms grow from attempts
> to work around the lack of bindings.  It's not that you need to replace
> all symlinks with bindings, of course - the crown of the tree is usually
> OK, it's the trunk that acts as source of pain.

Re: Symlink indirection

[Pete Zaitcev]

> Date: Fri, 13 Dec 2002 16:48:45 +0000
> From: Andrew Walrond <andrew@walrond.org>

> Sorry for being dense, but what do you mean by 'bindings' ? Hard links?

$ man mount

       Since Linux 2.4.0 it is possible to remount part of the file  hierarchy
       somewhere else. The call is
              mount --bind olddir newdir
       After this call the same contents is accessible in two places.

-- Pete

Re: Symlink indirection

[Andrew Walrond]

Of course; Didn't think of that in this context.

My application of symlinks involves overlaying several directories onto 
another, in an order such that any like named files are overridden in a 
defined way.

I had thought about asking the feasibility of [made up name] 
'transparent bindings' which would have this effect

Suppose I might as well ask now ;) Any takers?

Pete Zaitcev wrote:
>>Date: Fri, 13 Dec 2002 16:48:45 +0000
>>From: Andrew Walrond <andrew@walrond.org>
> 
> 
>>Sorry for being dense, but what do you mean by 'bindings' ? Hard links?
> 
> 
> $ man mount
> 
>        Since Linux 2.4.0 it is possible to remount part of the file  hierarchy
>        somewhere else. The call is
>               mount --bind olddir newdir
>        After this call the same contents is accessible in two places.
> 
> -- Pete
> 

Re: Symlink indirection

[Joseph Fannin]

[-- Attachment #1: Type: text/plain, Size: 1599 bytes --]

On Fri, Dec 13, 2002 at 05:04:12PM +0000, Andrew Walrond wrote:
> Of course; Didn't think of that in this context.
> 
> My application of symlinks involves overlaying several directories onto 
> another, in an order such that any like named files are overridden in a 
> defined way.
> 
> I had thought about asking the feasibility of [made up name] 
> 'transparent bindings' which would have this effect
> 
> Suppose I might as well ask now ;) Any takers?

    I don't understand what you are trying to explain.  Do you mean a
union mount, or a variation thereof?

    I thought Al Viro was going to do union mount support for 2.5, but
I haven't heard about it in a while.  Maybe it went in and no one noticed?

 
> Pete Zaitcev wrote:
> >>Date: Fri, 13 Dec 2002 16:48:45 +0000
> >>From: Andrew Walrond <andrew@walrond.org>
> >
> >
> >>Sorry for being dense, but what do you mean by 'bindings' ? Hard links?
> >
> >
> >$ man mount
> >
> >       Since Linux 2.4.0 it is possible to remount part of the file  
> >       hierarchy
> >       somewhere else. The call is
> >              mount --bind olddir newdir
> >       After this call the same contents is accessible in two places.
> >
> >
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

-- 
Joseph Fannin
jhf@rivenstone.net

"That's all I have to say about that." -- Forrest Gump.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Symlink indirection

[Andrew Walrond]

Joseph Fannin wrote:
> 
>     I don't understand what you are trying to explain.  Do you mean a
> union mount, or a variation thereof?
> 
>     I thought Al Viro was going to do union mount support for 2.5, but
> I haven't heard about it in a while.  Maybe it went in and no one noticed?
> 

Hi Joseph

I'm not familiar with the phrase 'union mount' and although google gives 
wads of hits, I can't find a good description of it

What I mean is (contrived example with made-up mount option --overlay)

mkdir a
echo "a/x" > a/x
echo "a/y" > a/y
echo "a/z" > a/z

mkdir b
echo "b/y" > b/y

mkdir c
echo "c/z" > c/z

mkdir d
mount --bind a d
mount --bind --overlay b d
mount --bind --overlay c d

cat d/x
"a/x"

cat d/y
"b/x"

cat d/z
"c/z"

This would be *really* useful and nice. I currently emulate this 
behavior with a bash script which creates hard or soft links, but the 
mounting system would be much nicer, easier to unwind etc.

I assume this isn't possible now (man mount gives no hint), but how 
feasible is it? Has anybody tried to implement this? If yes and No 
perhaps I could (with some initial guidance) have a look at implementing 
this.

I don't use HD's much anymore, so it would need to work for tmpfs.

Re: Symlink indirection

[John Bradford]

> >     I don't understand what you are trying to explain.  Do you mean a
> > union mount, or a variation thereof?
> > 
> >     I thought Al Viro was going to do union mount support for 2.5, but
> > I haven't heard about it in a while.  Maybe it went in and no one noticed?

> I'm not familiar with the phrase 'union mount' and although google gives 
> wads of hits, I can't find a good description of it
> 
> What I mean is (contrived example with made-up mount option --overlay)

> mkdir a
> echo "a/x" > a/x
> echo "a/y" > a/y
> echo "a/z" > a/z
> 
> mkdir b
> echo "b/y" > b/y
> 
> mkdir c
> echo "c/z" > c/z
> 
> mkdir d
> mount --bind a d
> mount --bind --overlay b d
> mount --bind --overlay c d
> 
> cat d/x
> "a/x"
> 
> cat d/y
> "b/x"

Shouldn't that be "b/y"?

> cat d/z
> "c/z"

John.

Re: Symlink indirection

[Andrew Walrond]

Yes, typo - thanks John.

Trying again...

(contrived example with made-up mount option --overlay)

mkdir a
echo "a/x" > a/x
echo "a/y" > a/y
echo "a/z" > a/z

mkdir b
echo "b/y" > b/y

mkdir c
echo "c/z" > c/z

mkdir d
mount --bind a d
mount --bind --overlay b d
mount --bind --overlay c d

cat d/x
"a/x"

cat d/y
"b/y"

cat d/z
"c/z"

Re: Symlink indirection

[John Bradford]

> (contrived example with made-up mount option --overlay)
> 
> mkdir a
> echo "a/x" > a/x
> echo "a/y" > a/y
> echo "a/z" > a/z
> 
> mkdir b
> echo "b/y" > b/y
> 
> mkdir c
> echo "c/z" > c/z
> 
> mkdir d
> mount --bind a d
> mount --bind --overlay b d
> mount --bind --overlay c d
> 
> cat d/x
> "a/x"
> 
> cat d/y
> "b/y"
> 
> cat d/z
> "c/z"

BSD has a mount_union command which does this, as well as mount_null,
which allows you to effectively mount a filesystem more than once.

I'm not sure how well it works, though, it might only work on some
filesystems.

John.

Re: Symlink indirection

[Stephen Wille Padnos]

Andrew Walrond wrote:

> [snip]

> cat d/x
> "a/x"
>
> cat d/y
> "b/y"
>
> cat d/z
> "c/z" 

What would you expect to happen if you then did:
echo "d/w" > d/w

Which physical directory would you expect a new file to go into?

- Steve

Re: Symlink indirection

[Andrew Walrond]

Hi steve

Stephen Wille Padnos wrote:
> 
> What would you expect to happen if you then did:
> echo "d/w" > d/w
> 
> Which physical directory would you expect a new file to go into?
> 

Using my example:

mkdir a
echo "a/x" > a/x
echo "a/y" > a/y
echo "a/z" > a/z

mkdir b
echo "b/y" > b/y

mkdir c
echo "c/z" > c/z

mkdir d
mount --bind a d
mount --bind --overlay b d
mount --bind --overlay c d

cat d/x
"a/x"

cat d/y
"b/y"

cat d/z
"c/z"

Then...

echo "d/w" > d/w would create a new file in directory a.
echo "d/y" > d/y would replace the file b/y
etc...

Is this sort of thing possible, or are there fundamental reasons that 
would make it difficult?

Andrew

Re: Symlink indirection

[Thomas Zimmerman]

[-- Attachment #1: Type: text/plain, Size: 1384 bytes --]

On Sun, 15 Dec 2002 00:41:06 +0000
Andrew Walrond <andrew@walrond.org> wrote:

> Hi steve
> 
> Stephen Wille Padnos wrote:
> > 
> > What would you expect to happen if you then did:
> > echo "d/w" > d/w
> > 
> > Which physical directory would you expect a new file to go into?
> > 
> 
> Using my example:
> 
> mkdir a
> echo "a/x" > a/x
> echo "a/y" > a/y
> echo "a/z" > a/z
> 
> mkdir b
> echo "b/y" > b/y
> 
> mkdir c
> echo "c/z" > c/z
> 
> mkdir d
> mount --bind a d
> mount --bind --overlay b d
> mount --bind --overlay c d
> 
> cat d/x
> "a/x"
> 
> cat d/y
> "b/y"
> 
> cat d/z
> "c/z"
> 
> Then...
> 
> echo "d/w" > d/w would create a new file in directory a.
> echo "d/y" > d/y would replace the file b/y
> etc...
 
I would have expected any changes to /d/* to happen in c; as that was
the *last* change to the mount point. It would allow much niceness, like
NFS root with local changes having persistence, if you "mount -bind
-overlay <smaller local drive> /"

> Is this sort of thing possible, or are there fundamental reasons that 
> would make it difficult?

I'll leave that to greater minds then mine. :)

> Andrew
> 
> -
> To unsubscribe from this list: send the line "unsubscribe
> linux-kernel" in the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Symlink indirection

[James Antill]

Pete Zaitcev <zaitcev@redhat.com> writes:

> >> Is the number of allowed levels of symlink indirection (if that is the
> >> right phrase; I mean symlink -> symlink -> ... -> file) dependant on the
> >> kernel, or libc ? Where is it defined, and can it be changed?
> > 
> > fs/namei.c
> > 
> >  if (current->link_count >= 5)
> > 
> > change to a higher value.
> 
> This is vey, very misleading statement. The counter mentioned above
> is there to protect stacks from overflow, but our symlink resolution
> is largely non-recursive, and certainly not in case of a tail
> recursion within the same directory.

 tail recursion is a bad name, as that implies the last element of the
path can go beyond the above value. A better way is to say that each
element of the path can have at most link_count and the total path can
have at most total_link_count symlinks (or that nested symlinks are
limited to a small number, in Al's words).

-- 
# James Antill -- james@and.org
:0:
* ^From: .*james@and\.org
/dev/null

Re: Symlink indirection

[junkio]

"AW" == Andrew Walrond <andrew@walrond.org> gives an example of
a/{x,y,z}, b/{y,z}, c/z mounted on d/. in that order, later
mounts covering the earlier ones.

AW> echo "d/w" > d/w would create a new file in directory a.

Personally I'd rather expect this to happen in c/.  Imagine a/
being on read-only medium like CD-ROM containing bunch of source
files, b/ to hold patched source, and c/ to hold binaries
resulting from compilation.  That is,

    rm -fr a b c d
    mkdir a b c d
    mount /cdrom a
    mount --bind a d
    mount --bind --overlay b d
    (cd b && bzip2 -d <../patch-2.9.91.bz2 | patch -p1)
    mount --bind --overlay c d
    (cd c && make mrproper && cat ../.config >.config &&
    make oldconfig && make dep && make bzImage)

Back to your example; what do you wish to happen when we do
this?

    $ mv d/z d/zz && test -f d/z && cat d/z

Here we rename d/z (which is really c/z) to zz.  Does this
reveal z that used to be hidden by that, namely b/z, and "cat
d/z" now shows "b/z"?

Or just like the case of creating a new file, does the union
"remember" the fact that the directory "d" should not contain
"z" anymore, and "test -f d/z" fails?

Re: Symlink indirection

[Andrew Walrond]

junkio@cox.net wrote:
> "AW" == Andrew Walrond <andrew@walrond.org> gives an example of
> a/{x,y,z}, b/{y,z}, c/z mounted on d/. in that order, later
> mounts covering the earlier ones.
> 
> AW> echo "d/w" > d/w would create a new file in directory a.
> 
> Personally I'd rather expect this to happen in c/.  Imagine a/
> being on read-only medium like CD-ROM containing bunch of source
> files, b/ to hold patched source, and c/ to hold binaries
> resulting from compilation.  That is,
> 
>     rm -fr a b c d
>     mkdir a b c d
>     mount /cdrom a
>     mount --bind a d
>     mount --bind --overlay b d
>     (cd b && bzip2 -d <../patch-2.9.91.bz2 | patch -p1)
>     mount --bind --overlay c d
>     (cd c && make mrproper && cat ../.config >.config &&
>     make oldconfig && make dep && make bzImage)

A nice example.

Lets ditch --overlay and replace it with:

--transparent
	just like --overlay
--read-transparent
	a bit like a pane of glass. You can see stuff behind it (read) but 
throw a tomato at it and it sticks to the glass (write)

Then, simplifying your example a bit, we can mount the source cd then 
mount a directory --read-transparent over the top to hold both our 
patched source and compiled binaries.

Or if you want to keep patched-source and binaries seperate,

mount /cdrom stuff
mount --bind --read-transparent patched-src stuff

cd stuff
patch the src

mount --bind --read-transparent binaries stuff

compile your code

> 
> Back to your example; what do you wish to happen when we do
> this?
> 
>     $ mv d/z d/zz && test -f d/z && cat d/z
> 
> Here we rename d/z (which is really c/z) to zz.  Does this
> reveal z that used to be hidden by that, namely b/z, and "cat
> d/z" now shows "b/z"?

Yes - exactly

> 
> Or just like the case of creating a new file, does the union
> "remember" the fact that the directory "d" should not contain
> "z" anymore, and "test -f d/z" fails?
> 

No. Thats not necessary.

Re: Symlink indirection

[John Bradford]

> > "AW" == Andrew Walrond <andrew@walrond.org> gives an example of
> > a/{x,y,z}, b/{y,z}, c/z mounted on d/. in that order, later
> > mounts covering the earlier ones.
> > 
> > AW> echo "d/w" > d/w would create a new file in directory a.

I disagree.  It should create it in directory d, even though that is
the mount point.

A union mount should include files from another directory, but writes
should go to the actual named directory.

> > Back to your example; what do you wish to happen when we do
> > this?
> > 
> >     $ mv d/z d/zz && test -f d/z && cat d/z
> > 
> > Here we rename d/z (which is really c/z) to zz.  Does this
> > reveal z that used to be hidden by that, namely b/z, and "cat
> > d/z" now shows "b/z"?
> 
> Yes - exactly

Union mounts should be read only.

If read-write union mounts are needed, I don't think that we should
implement them significantly differently to the way they work in BSD.

John.