Detecting bind-mounts

Detecting bind-mounts

[Michael Tokarev]

Hello.

There are quite some talks on the 'net - questions, not
answers - about detecting bind mounts - be it a directory
or a file.

There are 2 (mostly) different kinds of applications.  One
is cp/tar/find with --same-filesystem option (or equivalent),
that should not cross mountpoints.  And one more, apps like
mountpoint(1) from sysvinit - a utility to determine if a
given path is a mountpoint.

Neither of the two work when two directores on the same
filesystem are bind-mounted.

The usual idiom is to compare st_dev of current directory and
the parent - if they're different that's a mount point.  But
in this case, two st_devs will be the same, so such a mount
point will not be detected.

It is even worse for bind-mounted files (as opposed to dirs):
there's no path/file/.. entry to stat(2), and cutting the
last component from the pathname does not work reliable due
to symlinks (it may be a symlink from different filesystem).

So far I know only one way to detect a bind mount like this,
and it is unreliable anyway.  It is to parse /proc/mounts
and try to find the object(s) in question.  Unreliable because
of, again, symlinks, and possible complex mounts and bind-
mounts.  And this is also very slow - imagine using this way
for find/tar/cp --one-file-system.

Is there some simpler and more reliable way? Maybe use mount
syscall, like we use kill($pid, 0) to check existance of a
process?

And as far as I understand, the same applies to multiple
mounts of the same filesystem.

Thanks!

/mjt

Re: Detecting bind-mounts

[Pádraig Brady]

On 04/11/10 20:45, Michael Tokarev wrote:
> Hello.
> 
> There are quite some talks on the 'net - questions, not
> answers - about detecting bind mounts - be it a directory
> or a file.
> 
> There are 2 (mostly) different kinds of applications.  One
> is cp/tar/find with --same-filesystem option (or equivalent),
> that should not cross mountpoints.  And one more, apps like
> mountpoint(1) from sysvinit - a utility to determine if a
> given path is a mountpoint.
> 
> Neither of the two work when two directores on the same
> filesystem are bind-mounted.
> 
> The usual idiom is to compare st_dev of current directory and
> the parent - if they're different that's a mount point.  But
> in this case, two st_devs will be the same, so such a mount
> point will not be detected.
> 
> It is even worse for bind-mounted files (as opposed to dirs):
> there's no path/file/.. entry to stat(2), and cutting the
> last component from the pathname does not work reliable due
> to symlinks (it may be a symlink from different filesystem).
> 
> So far I know only one way to detect a bind mount like this,
> and it is unreliable anyway.  It is to parse /proc/mounts
> and try to find the object(s) in question.  Unreliable because
> of, again, symlinks, and possible complex mounts and bind-
> mounts.  And this is also very slow - imagine using this way
> for find/tar/cp --one-file-system.
> 
> Is there some simpler and more reliable way? Maybe use mount
> syscall, like we use kill($pid, 0) to check existance of a
> process?
> 
> And as far as I understand, the same applies to multiple
> mounts of the same filesystem.

The `stat` command recently got support for
printing the mount point for a file:
http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=commit;h=ddf6fb86

`stat` will output the alias for a bind mounted file
while `df` will output the initial mount point of its backing device
So you could do something like:

file=.
df_mnt=$(df -P "$file" | sed -n '2s/.* \([^ ]*$\)/\1/p')
stat_mnt=$(stat -c%m "$file")
test "$df_mnt" = "$stat_mnt" || echo "bind mount"

cheers,
Pádraig.

Re: Detecting bind-mounts

[Michael Tokarev]

05.11.2010 13:24, Pádraig Brady wrote:
> On 04/11/10 20:45, Michael Tokarev wrote:
[]
>> There are 2 (mostly) different kinds of applications.  One
>> is cp/tar/find with --same-filesystem option (or equivalent),
>> that should not cross mountpoints.  And one more, apps like
>> mountpoint(1) from sysvinit - a utility to determine if a
>> given path is a mountpoint.
>>
>> Neither of the two work when two directores on the same
>> filesystem are bind-mounted.
[]
> The `stat` command recently got support for
> printing the mount point for a file:
> http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=commit;h=ddf6fb86
>
> `stat` will output the alias for a bind mounted file
> while `df` will output the initial mount point of its backing device
> So you could do something like:
> 
> file=.
> df_mnt=$(df -P "$file" | sed -n '2s/.* \([^ ]*$\)/\1/p')
> stat_mnt=$(stat -c%m "$file")
> test "$df_mnt" = "$stat_mnt" || echo "bind mount"

This is incorrect in two ways.

First of all, stat(1), even after that commit you quote,
still compares st_dev fields, which are the same for this
and parent directory in case of bind mount.  So this version
of stat(1) does _not_ detect a bind mount, unfortunately.

Second, I asked for a low-level way to detect such a mount.
I know how to do it not as efficient as stat(2) and not as
reliable but much simpler than you propose above, in shell
or in C, and I already provided that way in my original
email: we just parse /proc/mounts file, this is faster and
more reliable than the above shell fragment which calls a
few external commands.

In the above example, both stat(1) (even the one with the
commit you refers to) and df(1) reports the same for the
case I'm referring to, the both fails to detect a bind-
mount.

Thanks!

/mjt

Re: Detecting bind-mounts

[Michael Tokarev]

05.11.2010 21:30, Michael Tokarev wrote:
> 05.11.2010 13:24, Pádraig Brady wrote:
>> On 04/11/10 20:45, Michael Tokarev wrote:
> []
>>> There are 2 (mostly) different kinds of applications.  One
>>> is cp/tar/find with --same-filesystem option (or equivalent),
>>> that should not cross mountpoints.  And one more, apps like
>>> mountpoint(1) from sysvinit - a utility to determine if a
>>> given path is a mountpoint.
>>>
>>> Neither of the two work when two directores on the same
>>> filesystem are bind-mounted.
> []
>> The `stat` command recently got support for
>> printing the mount point for a file:
>> http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=commit;h=ddf6fb86
>>
>> `stat` will output the alias for a bind mounted file
>> while `df` will output the initial mount point of its backing device
>> So you could do something like:
>>
>> file=.
>> df_mnt=$(df -P "$file" | sed -n '2s/.* \([^ ]*$\)/\1/p')
>> stat_mnt=$(stat -c%m "$file")
>> test "$df_mnt" = "$stat_mnt" || echo "bind mount"
> 
> This is incorrect in two ways.
> 
> First of all, stat(1), even after that commit you quote,
> still compares st_dev fields, which are the same for this
> and parent directory in case of bind mount.  So this version
> of stat(1) does _not_ detect a bind mount, unfortunately.

And this statement, in turn, is untrue.  I apologize for the
misinformation, it wasn't intentional.  The mentioned commit
adds the ability to detect bind mounts indeed.  but...

> Second, I asked for a low-level way to detect such a mount.
> I know how to do it not as efficient as stat(2) and not as
> reliable but much simpler than you propose above, in shell
> or in C, and I already provided that way in my original
> email: we just parse /proc/mounts file, this is faster and
> more reliable than the above shell fragment which calls a
> few external commands.

.. the way used by stat(1) is to enumerate /proc/mounts --
which is what I were able to come with initially.  It is slow
and unreliable.  Hence I asked if a faster way exist.

/mjt

Re: Detecting bind-mounts

[Pádraig Brady]

On 05/11/10 20:30, Michael Tokarev wrote:
> 05.11.2010 21:30, Michael Tokarev wrote:
>> 05.11.2010 13:24, Pádraig Brady wrote:
>>> On 04/11/10 20:45, Michael Tokarev wrote:
>> []
>>>> There are 2 (mostly) different kinds of applications.  One
>>>> is cp/tar/find with --same-filesystem option (or equivalent),
>>>> that should not cross mountpoints.  And one more, apps like
>>>> mountpoint(1) from sysvinit - a utility to determine if a
>>>> given path is a mountpoint.
>>>>
>>>> Neither of the two work when two directores on the same
>>>> filesystem are bind-mounted.
>> []
>>> The `stat` command recently got support for
>>> printing the mount point for a file:
>>> http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=commit;h=ddf6fb86
>>>
>>> `stat` will output the alias for a bind mounted file
>>> while `df` will output the initial mount point of its backing device
>>> So you could do something like:
>>>
>>> file=.
>>> df_mnt=$(df -P "$file" | sed -n '2s/.* \([^ ]*$\)/\1/p')
>>> stat_mnt=$(stat -c%m "$file")
>>> test "$df_mnt" = "$stat_mnt" || echo "bind mount"
>>
>> This is incorrect in two ways.
>>
>> First of all, stat(1), even after that commit you quote,
>> still compares st_dev fields, which are the same for this
>> and parent directory in case of bind mount.  So this version
>> of stat(1) does _not_ detect a bind mount, unfortunately.
> 
> And this statement, in turn, is untrue.  I apologize for the
> misinformation, it wasn't intentional.  The mentioned commit
> adds the ability to detect bind mounts indeed.  but...
> 
>> Second, I asked for a low-level way to detect such a mount.
>> I know how to do it not as efficient as stat(2) and not as
>> reliable but much simpler than you propose above, in shell
>> or in C, and I already provided that way in my original
>> email: we just parse /proc/mounts file, this is faster and
>> more reliable than the above shell fragment which calls a
>> few external commands.
> 
> .. the way used by stat(1) is to enumerate /proc/mounts --
> which is what I were able to come with initially.  It is slow
> and unreliable.  Hence I asked if a faster way exist.

Sorry I was unclear.

I was just providing reference info about the changes in
the new `stat -c %m` that both showed tools do now exist
to determine if a file/dir is a mount point, and to
validate your "slow way" of detecting (bind) mounts.

As for a fast way, I don't think one exists.
BTW I'm not sure your examples are actually valid.
If a file/dir is bind mounted to the same file system, then
`find -xdev` should be listing it (as it has the same dev).

You want a separate option --same-mount or something,
though I don't know what it would be useful for.

cheers,
Pádraig.

Re: Detecting bind-mounts

[Michael Tokarev]

06.11.2010 03:32, Pádraig Brady wrote:
[]
> As for a fast way, I don't think one exists.
> BTW I'm not sure your examples are actually valid.
> If a file/dir is bind mounted to the same file system, then
> `find -xdev` should be listing it (as it has the same dev).

Think what, say, cp or tar with --one-file-system option are
used for.  It is usually to copy a system to another place.
There, we only want single copy of everything, and with current
situation it will be two, with additional mess with hardlinks
for files wich were hardlinks already (due to optimizations
done by the utils based on link counts).

find -xdev is a bit different since it explicitly mentions
"dev", and in my examples the device is actually the same.
But usage case can be the same as for cp/tar above too, or
may be different.

> You want a separate option --same-mount or something,
> though I don't know what it would be useful for.

Stopping at the bind-mount dir definitely is useful, see
above for the "main" (and very important) usage case (this
can be solved differently on linux too - by cloning a new
namespace and removing the bind mounts before doing that
copy. but this is, again, ugly at best).

Note that this "main" usage case requires fast way to
determine mount points...

/mjt

Re: Detecting bind-mounts

[Pádraig Brady]

On 06/11/10 00:47, Michael Tokarev wrote:
> 06.11.2010 03:32, Pádraig Brady wrote:
> []
>> As for a fast way, I don't think one exists.
>> BTW I'm not sure your examples are actually valid.
>> If a file/dir is bind mounted to the same file system, then
>> `find -xdev` should be listing it (as it has the same dev).
> 
> Think what, say, cp or tar with --one-file-system option are
> used for.  It is usually to copy a system to another place.
> There, we only want single copy of everything, and with current
> situation it will be two, with additional mess with hardlinks
> for files wich were hardlinks already (due to optimizations
> done by the utils based on link counts).

Fair enough, but that starts to overlap with how hardlinks
are handled.  If you're following symlinks then you can't
use the hardlink count. Note the du utility has recently
handled this by using a "di-set" to efficiently exclude
duplicate dev,inode pairs.

> 
> find -xdev is a bit different since it explicitly mentions
> "dev", and in my examples the device is actually the same.
> But usage case can be the same as for cp/tar above too, or
> may be different.
> 
>> You want a separate option --same-mount or something,
>> though I don't know what it would be useful for.
> 
> Stopping at the bind-mount dir definitely is useful, see
> above for the "main" (and very important) usage case (this
> can be solved differently on linux too - by cloning a new
> namespace and removing the bind mounts before doing that
> copy. but this is, again, ugly at best).
> 
> Note that this "main" usage case requires fast way to
> determine mount points...

I can see how it would be useful, yes.

cheers,
Pádraig.