bug with multiple mounts of filesystems in 2.6

bug with multiple mounts of filesystems in 2.6

[John S J Anderson]

  Hi --

  We're working on migrating to the 2.6 kernel series, and one big
  problem has popped up: we have a number of NFS mounts that are
  mounted read-only in one location and read-write in a distinct
  location (on the same machine). With 2.4 series kernels, this worked
  without issue, but with 2.6, it doesn't: it's not possible to mount
  the same filesystem twice with different options for each mount; the
  two mount points have to share the same mount options.

  Furthermore, if you do mount one filesystem at two different places,
  changing the mount options on one mount point ('mount -o remount,rw
  $MOUNT1', for example) also results in the mount options on the
  other mount point being changed. Finally, the information returned
  by 'mount' about the mount points is wrong -- 'mount' will show (for
  example) one mount point being 'rw' and the other being 'ro', when
  in fact attempting to use the 'rw' mount point will show it to be
  read-only. /proc/mounts correctly indicates that both mounts are
  read-only. 

  Further experimentation has shown: 

  * the NFS layer is not involved; it is possible to reproduce this
    problem using just locally-attached filesystems.
  * this affects at least 2.6.3, 2.6.5, and 2.6.7.
  * it happens with mount version 2.11r and 2.12.

  Because the options of the first mount point "wins" (i.e., mounting a
  filesystem once read-write, and then a second time as read-only,
  leaves both mounts in a read-write state), it seems like there's
  some sort of caching optimization going on -- but I haven't looked
  into the code to find out if that guess is correct.
  
  Any pointers towards restoring the 2.4 behavior welcomed. 

thanks,
john.
-- 
Perhaps you have forgotten that this is an engineering discipline, not
some sort of black magic.
  Mark Jason Dominus's Good Advice #11946

Re: bug with multiple mounts of filesystems in 2.6

[Trond Myklebust]

På må , 26/07/2004 klokka 12:29, skreiv John S J Anderson:
>   Hi --
> 
>   We're working on migrating to the 2.6 kernel series, and one big
>   problem has popped up: we have a number of NFS mounts that are
>   mounted read-only in one location and read-write in a distinct
>   location (on the same machine). With 2.4 series kernels, this worked
>   without issue, but with 2.6, it doesn't: it's not possible to mount
>   the same filesystem twice with different options for each mount; the
>   two mount points have to share the same mount options.

That behaviour is no longer supported as it meant that you would have
different superblocks (and hence different out-of-sync caches) between
the 2 mountpoint. It is in any case not a behaviour that is supported on
any other Linux filesystems.

If you want readonly to be an exception, then you will have to move the
MS_RDONLY flag from being a superblock option to being a vfsmount
option, then propagate that vfsmount information down to all the tests
of IS_RDONLY(inode). Not a trivial task, and not one that looms high on
my list of priorities...

Cheers,
  Trond

Re: bug with multiple mounts of filesystems in 2.6

[Mike Waychison]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Trond Myklebust wrote:
> På må , 26/07/2004 klokka 12:29, skreiv John S J Anderson:
>
>>  Hi --
>>
>>  We're working on migrating to the 2.6 kernel series, and one big
>>  problem has popped up: we have a number of NFS mounts that are
>>  mounted read-only in one location and read-write in a distinct
>>  location (on the same machine). With 2.4 series kernels, this worked
>>  without issue, but with 2.6, it doesn't: it's not possible to mount
>>  the same filesystem twice with different options for each mount; the
>>  two mount points have to share the same mount options.
>
>
> That behaviour is no longer supported as it meant that you would have
> different superblocks (and hence different out-of-sync caches) between
> the 2 mountpoint. It is in any case not a behaviour that is supported on
> any other Linux filesystems.
>

How is this any different than having two seperate nfs clients accessing
the same nfs export?

> If you want readonly to be an exception, then you will have to move the
> MS_RDONLY flag from being a superblock option to being a vfsmount
> option, then propagate that vfsmount information down to all the tests
> of IS_RDONLY(inode). Not a trivial task, and not one that looms high on
> my list of priorities...
>

What ever happened to the bind ro patches that were floating around a
couple months ago?
(http://marc.theaimsgroup.com/?t=107932320200005&r=1&w=2)

What is left in getting this done?  Just the touch_file bit Viro
commented on?

- --
Mike Waychison
Sun Microsystems, Inc.
1 (650) 352-5299 voice
1 (416) 202-8336 voice
http://www.sun.com

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
NOTICE:  The opinions expressed in this email are held by me,
and may not represent the views of Sun Microsystems, Inc.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD4DBQFBBXiTdQs4kOxk3/MRAp2CAJ9hLA43GX7breEAuFJp++noSX7hAQCYn7yw
FXXelAMC/NCetjqwC8Q67g==
=rQRx
-----END PGP SIGNATURE-----

Re: bug with multiple mounts of filesystems in 2.6

[Herbert Poetzl]

On Mon, Jul 26, 2004 at 05:33:07PM -0400, Mike Waychison wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Trond Myklebust wrote:
> > På må , 26/07/2004 klokka 12:29, skreiv John S J Anderson:
> >
> >>  Hi --
> >>
> >>  We're working on migrating to the 2.6 kernel series, and one big
> >>  problem has popped up: we have a number of NFS mounts that are
> >>  mounted read-only in one location and read-write in a distinct
> >>  location (on the same machine). With 2.4 series kernels, this worked
> >>  without issue, but with 2.6, it doesn't: it's not possible to mount
> >>  the same filesystem twice with different options for each mount; the
> >>  two mount points have to share the same mount options.
> >
> > That behaviour is no longer supported as it meant that you would have
> > different superblocks (and hence different out-of-sync caches) between
> > the 2 mountpoint. It is in any case not a behaviour that is supported on
> > any other Linux filesystems.
> 
> How is this any different than having two seperate nfs clients accessing
> the same nfs export?
> 
> > If you want readonly to be an exception, then you will have to move the
> > MS_RDONLY flag from being a superblock option to being a vfsmount
> > option, then propagate that vfsmount information down to all the tests
> > of IS_RDONLY(inode). Not a trivial task, and not one that looms high on
> > my list of priorities...
> 
> What ever happened to the bind ro patches that were floating around a
> couple months ago?
> (http://marc.theaimsgroup.com/?t=107932320200005&r=1&w=2)
> 
> What is left in getting this done?  Just the touch_file bit Viro
> commented on?

started with the noatime/nodiratime stuff for inclusion
but that patch was neither commented nor included, so
I put it on hold ... currently I'm planning to update 
this for 2.6.8 ... we'll see ...

best,
Herbert


> - --
> Mike Waychison
> Sun Microsystems, Inc.
> 1 (650) 352-5299 voice
> 1 (416) 202-8336 voice
> http://www.sun.com
> 
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> NOTICE:  The opinions expressed in this email are held by me,
> and may not represent the views of Sun Microsystems, Inc.
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.2.4 (GNU/Linux)
> 
> iD4DBQFBBXiTdQs4kOxk3/MRAp2CAJ9hLA43GX7breEAuFJp++noSX7hAQCYn7yw
> FXXelAMC/NCetjqwC8Q67g==
> =rQRx
> -----END PGP SIGNATURE-----

Re: bug with multiple mounts of filesystems in 2.6

[Trond Myklebust]

På må , 26/07/2004 klokka 17:33, skreiv Mike Waychison:

> How is this any different than having two seperate nfs clients accessing
> the same nfs export?

It isn't, but why do you think that should be a reason for allowing it?

By all means feel free to add "mount --bind -oro" capabilities, but it
is neither useful nor is it necessary to break the NFS caching model in
order to do so.

Cheers,
  Trond

Re: bug with multiple mounts of filesystems in 2.6

[Mike Waychison]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Trond Myklebust wrote:
> På må , 26/07/2004 klokka 17:33, skreiv Mike Waychison:
>
>
>>How is this any different than having two seperate nfs clients accessing
>>the same nfs export?
>
>
> It isn't, but why do you think that should be a reason for allowing it?
>
> By all means feel free to add "mount --bind -oro" capabilities, but it
> is neither useful nor is it necessary to break the NFS caching model in
> order to do so.
>

Agreed.  The two problems are orthogonal. [1]

As an example where sharing the super_block is wrong (albeit probably
just an oversight) is that the protocols (udp vs tcp) are not compared
in nfs_compare_super.  You could argue that the client fhandles should
be different though, I'm not sure..

Another 'bind mount extension' that would be nice to change at the
vfsmount level may be w/rsize, but that is probably a very intrusive
change for nfs and probably not possible.  Thoughts?

[1] - I haven't tested mounting nfs ro, and then mounting nfs rw using
the bind extensions.  Does nfs make any assumptions about the mount
being ro?

- --
Mike Waychison
Sun Microsystems, Inc.
1 (650) 352-5299 voice
1 (416) 202-8336 voice
http://www.sun.com

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
NOTICE:  The opinions expressed in this email are held by me,
and may not represent the views of Sun Microsystems, Inc.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFBBahXdQs4kOxk3/MRAhTCAKCJGOaemEdeDrmtp/tG5Y6fHe+BTgCgkh8v
312wdekZsxms1ShJciogYRQ=
=7Hm7
-----END PGP SIGNATURE-----

Re: bug with multiple mounts of filesystems in 2.6

[Trond Myklebust]

På må , 26/07/2004 klokka 20:56, skreiv Mike Waychison:

> As an example where sharing the super_block is wrong (albeit probably
> just an oversight) is that the protocols (udp vs tcp) are not compared
> in nfs_compare_super.  You could argue that the client fhandles should
> be different though, I'm not sure..

Not an oversight. It's just that we don't really have a model for
trunking a single cache over several different RPC connections.
Basically, it all boils down to the problem that inodes and dentries
have no idea of which namespace you used to access them, and hence even
if you did have space in the vfsmount in which to store an rpc_client
struct and perhaps rsize/wsize ... you still have a reconstruction job
to do.

> Another 'bind mount extension' that would be nice to change at the
> vfsmount level may be w/rsize, but that is probably a very intrusive
> change for nfs and probably not possible.  Thoughts?

The "struct nfs_open_context" I introduce in the latest NFS4_ALL patches
could probably be modified to include vfsmount information.

My question is why do people need to do this? What problems does it
solve?
Normally, rsize/wsize are per-server parameters. Their optimal value
depends above all upon the quality of the network link between client
and server. Ditto for the choice of UDP vs TCP: why would you want to
choose to use both options against a given server?

> [1] - I haven't tested mounting nfs ro, and then mounting nfs rw using
> the bind extensions.  Does nfs make any assumptions about the mount
> being ro?

Nope, however the VFS support for this is missing. That was part of what
Herbert was doing in his patches.

Cheers,
  Trond