Nfsv4 referrals over bind mounted subtree, mount order problem

Nfsv4 referrals over bind mounted subtree, mount order problem

[Prunk Dump]

Hello NFS team !

I have sometimes trouble using NFSv4 referrals and I don't really
understand where the problem come from. Here the configuration steps
that expose the situation :

---------------------------------------------
1) standard nfs4 export (works)
---------------------------------------------
My server export a file tree like this :

/myexport/dirA
/myexport/dirB

To do this I have bind mounted as usual the tree root in the NFS root :

mount --bind /myexport /srv/nfs4/myexport
/srv/nfs4/myexport
172.16.0.0/16(sec=krb5,rw,async,no_subtree_check)  (in /etc/exports)

Everything is OK.

-----------------------------------------------------------------
2) dirA redirection, bind mounted to itself (works)
-----------------------------------------------------------------

I now want to redirect dirA to another server "anotherserver". So I
bind mount the dirA folder to itself.

mount --bind /srv/nfs4/myexport/dirA /srv/nfs4/myexport/dirA
/srv/nfs4/myexport/dirA
172.16.0.0/16(refer=/myexport/dirA@anotherserver,sec=krb5,rw,async,no_subtree_check)
 (in /etc/exports)

Now the redirection works as expected.

----------------------------------------------------------------------------------
3) mount remote dirA on the original server (first problem)
-----------------------------------------------------------------------------------

But the original server need to keep access to the "dirA" files. So I
need to mount the redirected "dirA@anotherserver" tree on "myserver".

mount -t nfs4 -o sec=krb5 anotherserver:/myexport/dirA  /myexport/dirA

But as /myexport is bind mounted, this command create another mount
point on /srv/nfs4/myexport/dirA.

# mount
...
anotherserver:/myexport/dirA on /myexport/dirA ...
anotherserver:/myexport/dirA on /srv/nfs4/myexport/dirA ...

So, as this command create a mount point at the right position, where
the redirection start, I initially think that the initial self binded
mount /srv/nfs4/myexport/dirA is useless now. So I keep the only the
nfs4 mount.

But strangely the redirection does not works anymore. And I don't
understand where is the problem.

######
first solution tried : mount remote to /myexport/dirA, adding an
additional bind mount (don't works)
######

I tried to add another bind mount over the nfs mount. I now have four
mount points !

anotherserver:/myexport/dirA on /myexport/dirA ...
anotherserver:/myexport/dirA on /srv/nfs4/myexport/dirA ...
anotherserver:/myexport/dirA on /srv/nfs4/myexport/dirA ...
anotherserver:/myexport/dirA on /myexport/dirA ...

But this does not solve the problem. No redirection.

######
second solution tried : mount remote nfs directly to
/srv/nfs4/myexport/dirA (works !)
######

If a mount the remote share directly to /srv/nfs4/myexport/dirA :

mount -t nfs4 -o sec=krb5 anotherserver:/myexport/dirA /srv/nfs4/myexport/dirA

as /myexport is bind mounted, this create another mount point on
/myexport/dirA :

# mount
anotherserver:/myexport/dirA on /srv/nfs4/myexport/dirA ...
anotherserver:/myexport/dirA on /myexport/dirA ...

But the mount points are in reverse order now.  And in this case it
works ! I can access the remote share in /myexport/dirA. And the other
clients are redirected to anotherserver.

But I can't understand why...

Why the mount order is so important ?

Thanks !

Baptiste.

Re: Nfsv4 referrals over bind mounted subtree, mount order problem

[Prunk Dump]

2015-10-19 14:09 GMT+02:00 Prunk Dump <prunkdump@gmail.com>:
> Hello NFS team !
>
> I have sometimes trouble using NFSv4 referrals and I don't really
> understand where the problem come from. Here the configuration steps
> that expose the situation :
>
> ---------------------------------------------
> 1) standard nfs4 export (works)
> ---------------------------------------------
> My server export a file tree like this :
>
> /myexport/dirA
> /myexport/dirB
>
> To do this I have bind mounted as usual the tree root in the NFS root :
>
> mount --bind /myexport /srv/nfs4/myexport
> /srv/nfs4/myexport
> 172.16.0.0/16(sec=krb5,rw,async,no_subtree_check)  (in /etc/exports)
>
> Everything is OK.
>
> -----------------------------------------------------------------
> 2) dirA redirection, bind mounted to itself (works)
> -----------------------------------------------------------------
>
> I now want to redirect dirA to another server "anotherserver". So I
> bind mount the dirA folder to itself.
>
> mount --bind /srv/nfs4/myexport/dirA /srv/nfs4/myexport/dirA
> /srv/nfs4/myexport/dirA
> 172.16.0.0/16(refer=/myexport/dirA@anotherserver,sec=krb5,rw,async,no_subtree_check)
>  (in /etc/exports)
>
> Now the redirection works as expected.
>
> ----------------------------------------------------------------------------------
> 3) mount remote dirA on the original server (first problem)
> -----------------------------------------------------------------------------------
>
> But the original server need to keep access to the "dirA" files. So I
> need to mount the redirected "dirA@anotherserver" tree on "myserver".
>
> mount -t nfs4 -o sec=krb5 anotherserver:/myexport/dirA  /myexport/dirA
>
> But as /myexport is bind mounted, this command create another mount
> point on /srv/nfs4/myexport/dirA.
>
> # mount
> ...
> anotherserver:/myexport/dirA on /myexport/dirA ...
> anotherserver:/myexport/dirA on /srv/nfs4/myexport/dirA ...
>
> So, as this command create a mount point at the right position, where
> the redirection start, I initially think that the initial self binded
> mount /srv/nfs4/myexport/dirA is useless now. So I keep the only the
> nfs4 mount.
>
> But strangely the redirection does not works anymore. And I don't
> understand where is the problem.
>
> ######
> first solution tried : mount remote to /myexport/dirA, adding an
> additional bind mount (don't works)
> ######
>
> I tried to add another bind mount over the nfs mount. I now have four
> mount points !
>
> anotherserver:/myexport/dirA on /myexport/dirA ...
> anotherserver:/myexport/dirA on /srv/nfs4/myexport/dirA ...
> anotherserver:/myexport/dirA on /srv/nfs4/myexport/dirA ...
> anotherserver:/myexport/dirA on /myexport/dirA ...
>
> But this does not solve the problem. No redirection.
>
> ######
> second solution tried : mount remote nfs directly to
> /srv/nfs4/myexport/dirA (works !)
> ######
>
> If a mount the remote share directly to /srv/nfs4/myexport/dirA :
>
> mount -t nfs4 -o sec=krb5 anotherserver:/myexport/dirA /srv/nfs4/myexport/dirA
>
> as /myexport is bind mounted, this create another mount point on
> /myexport/dirA :
>
> # mount
> anotherserver:/myexport/dirA on /srv/nfs4/myexport/dirA ...
> anotherserver:/myexport/dirA on /myexport/dirA ...
>
> But the mount points are in reverse order now.  And in this case it
> works ! I can access the remote share in /myexport/dirA. And the other
> clients are redirected to anotherserver.
>
> But I can't understand why...
>
> Why the mount order is so important ?
>
> Thanks !
>
> Baptiste.

Excuse me !

In reality the solution proposed above does not work ! The NFS server
say :  "/myexport/dirA does not support NFS export" as it is a NFS
mounted filesystem.

Can anyone help me !?

Is this possible to avoid duplicating mount point when mounting over
bind mount ?

/myexport and /srv/nfs4/myexport are the same mount. But I want to do
a nfs4 mount on /myexpot/dirA and a bind mount on
/srv/nfs4/myexport/dirA.

Thanks.

Baptiste.

Re: Nfsv4 referrals over bind mounted subtree, mount order problem

[Prunk Dump]

2015-10-20 10:40 GMT+02:00 Prunk Dump <prunkdump@gmail.com>:
> 2015-10-19 14:09 GMT+02:00 Prunk Dump <prunkdump@gmail.com>:
>> Hello NFS team !
>>
>> I have sometimes trouble using NFSv4 referrals and I don't really
>> understand where the problem come from. Here the configuration steps
>> that expose the situation :
>>
>> ---------------------------------------------
>> 1) standard nfs4 export (works)
>> ---------------------------------------------
>> My server export a file tree like this :
>>
>> /myexport/dirA
>> /myexport/dirB
>>
>> To do this I have bind mounted as usual the tree root in the NFS root :
>>
>> mount --bind /myexport /srv/nfs4/myexport
>> /srv/nfs4/myexport
>> 172.16.0.0/16(sec=krb5,rw,async,no_subtree_check)  (in /etc/exports)
>>
>> Everything is OK.
>>
>> -----------------------------------------------------------------
>> 2) dirA redirection, bind mounted to itself (works)
>> -----------------------------------------------------------------
>>
>> I now want to redirect dirA to another server "anotherserver". So I
>> bind mount the dirA folder to itself.
>>
>> mount --bind /srv/nfs4/myexport/dirA /srv/nfs4/myexport/dirA
>> /srv/nfs4/myexport/dirA
>> 172.16.0.0/16(refer=/myexport/dirA@anotherserver,sec=krb5,rw,async,no_subtree_check)
>>  (in /etc/exports)
>>
>> Now the redirection works as expected.
>>
>> ----------------------------------------------------------------------------------
>> 3) mount remote dirA on the original server (first problem)
>> -----------------------------------------------------------------------------------
>>
>> But the original server need to keep access to the "dirA" files. So I
>> need to mount the redirected "dirA@anotherserver" tree on "myserver".
>>
>> mount -t nfs4 -o sec=krb5 anotherserver:/myexport/dirA  /myexport/dirA
>>
>> But as /myexport is bind mounted, this command create another mount
>> point on /srv/nfs4/myexport/dirA.
>>
>> # mount
>> ...
>> anotherserver:/myexport/dirA on /myexport/dirA ...
>> anotherserver:/myexport/dirA on /srv/nfs4/myexport/dirA ...
>>
>> So, as this command create a mount point at the right position, where
>> the redirection start, I initially think that the initial self binded
>> mount /srv/nfs4/myexport/dirA is useless now. So I keep the only the
>> nfs4 mount.
>>
>> But strangely the redirection does not works anymore. And I don't
>> understand where is the problem.
>>
>> ######
>> first solution tried : mount remote to /myexport/dirA, adding an
>> additional bind mount (don't works)
>> ######
>>
>> I tried to add another bind mount over the nfs mount. I now have four
>> mount points !
>>
>> anotherserver:/myexport/dirA on /myexport/dirA ...
>> anotherserver:/myexport/dirA on /srv/nfs4/myexport/dirA ...
>> anotherserver:/myexport/dirA on /srv/nfs4/myexport/dirA ...
>> anotherserver:/myexport/dirA on /myexport/dirA ...
>>
>> But this does not solve the problem. No redirection.
>>
>> ######
>> second solution tried : mount remote nfs directly to
>> /srv/nfs4/myexport/dirA (works !)
>> ######
>>
>> If a mount the remote share directly to /srv/nfs4/myexport/dirA :
>>
>> mount -t nfs4 -o sec=krb5 anotherserver:/myexport/dirA /srv/nfs4/myexport/dirA
>>
>> as /myexport is bind mounted, this create another mount point on
>> /myexport/dirA :
>>
>> # mount
>> anotherserver:/myexport/dirA on /srv/nfs4/myexport/dirA ...
>> anotherserver:/myexport/dirA on /myexport/dirA ...
>>
>> But the mount points are in reverse order now.  And in this case it
>> works ! I can access the remote share in /myexport/dirA. And the other
>> clients are redirected to anotherserver.
>>
>> But I can't understand why...
>>
>> Why the mount order is so important ?
>>
>> Thanks !
>>
>> Baptiste.
>
> Excuse me !
>
> In reality the solution proposed above does not work ! The NFS server
> say :  "/myexport/dirA does not support NFS export" as it is a NFS
> mounted filesystem.
>
> Can anyone help me !?
>
> Is this possible to avoid duplicating mount point when mounting over
> bind mount ?
>
> /myexport and /srv/nfs4/myexport are the same mount. But I want to do
> a nfs4 mount on /myexpot/dirA and a bind mount on
> /srv/nfs4/myexport/dirA.
>
> Thanks.
>
> Baptiste.

I finally found a working solution.

I have added the "private" option on the initial /srv/nfs4/myexport
bind mount. So now the subtree mounts are not duplicated.

I can bind mount /srv/nfs4/myexport/dirA to itself
I can mount the remote nfs4 share on /myexport/dirA

But I would be nice if the redirection can works with an nfs4 mount
point even if an nfs4 filesystem can't be exported by an nfs4 server.

Baptiste.

Re: Nfsv4 referrals over bind mounted subtree, mount order problem

[J. Bruce Fields]

On Mon, Oct 19, 2015 at 02:09:21PM +0200, Prunk Dump wrote:
> Hello NFS team !
> 
> I have sometimes trouble using NFSv4 referrals and I don't really
> understand where the problem come from. Here the configuration steps
> that expose the situation :
> 
> ---------------------------------------------
> 1) standard nfs4 export (works)
> ---------------------------------------------
> My server export a file tree like this :
> 
> /myexport/dirA
> /myexport/dirB
> 
> To do this I have bind mounted as usual the tree root in the NFS root :
> 
> mount --bind /myexport /srv/nfs4/myexport
> /srv/nfs4/myexport
> 172.16.0.0/16(sec=krb5,rw,async,no_subtree_check)  (in /etc/exports)

I'm not sure why you're doing this.

(It used to be necessary to do something like this (with the addition of
an fsid=0 option on the root export) for NFSv4 exports, but that hasn't
been necessary for a while.  You can skip this step and just mount
server:/myexport/dirA from the client.

> Everything is OK.
> 
> -----------------------------------------------------------------
> 2) dirA redirection, bind mounted to itself (works)
> -----------------------------------------------------------------
> 
> I now want to redirect dirA to another server "anotherserver". So I
> bind mount the dirA folder to itself.
> 
> mount --bind /srv/nfs4/myexport/dirA /srv/nfs4/myexport/dirA
> /srv/nfs4/myexport/dirA
> 172.16.0.0/16(refer=/myexport/dirA@anotherserver,sec=krb5,rw,async,no_subtree_check)
>  (in /etc/exports)
> 
> Now the redirection works as expected.
> 
> ----------------------------------------------------------------------------------
> 3) mount remote dirA on the original server (first problem)
> -----------------------------------------------------------------------------------
> 
> But the original server need to keep access to the "dirA" files. So I
> need to mount the redirected "dirA@anotherserver" tree on "myserver".
> 
> mount -t nfs4 -o sec=krb5 anotherserver:/myexport/dirA  /myexport/dirA

I'd recommend mounting it someplace that isn't under an export,
otherwise this will get a bit complicated.

--b.

> 
> But as /myexport is bind mounted, this command create another mount
> point on /srv/nfs4/myexport/dirA.
> 
> # mount
> ...
> anotherserver:/myexport/dirA on /myexport/dirA ...
> anotherserver:/myexport/dirA on /srv/nfs4/myexport/dirA ...
> 
> So, as this command create a mount point at the right position, where
> the redirection start, I initially think that the initial self binded
> mount /srv/nfs4/myexport/dirA is useless now. So I keep the only the
> nfs4 mount.
> 
> But strangely the redirection does not works anymore. And I don't
> understand where is the problem.
> 
> ######
> first solution tried : mount remote to /myexport/dirA, adding an
> additional bind mount (don't works)
> ######
> 
> I tried to add another bind mount over the nfs mount. I now have four
> mount points !
> 
> anotherserver:/myexport/dirA on /myexport/dirA ...
> anotherserver:/myexport/dirA on /srv/nfs4/myexport/dirA ...
> anotherserver:/myexport/dirA on /srv/nfs4/myexport/dirA ...
> anotherserver:/myexport/dirA on /myexport/dirA ...
> 
> But this does not solve the problem. No redirection.
> 
> ######
> second solution tried : mount remote nfs directly to
> /srv/nfs4/myexport/dirA (works !)
> ######
> 
> If a mount the remote share directly to /srv/nfs4/myexport/dirA :
> 
> mount -t nfs4 -o sec=krb5 anotherserver:/myexport/dirA /srv/nfs4/myexport/dirA
> 
> as /myexport is bind mounted, this create another mount point on
> /myexport/dirA :
> 
> # mount
> anotherserver:/myexport/dirA on /srv/nfs4/myexport/dirA ...
> anotherserver:/myexport/dirA on /myexport/dirA ...
> 
> But the mount points are in reverse order now.  And in this case it
> works ! I can access the remote share in /myexport/dirA. And the other
> clients are redirected to anotherserver.
> 
> But I can't understand why...
> 
> Why the mount order is so important ?
> 
> Thanks !
> 
> Baptiste.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html