nfs-utils 1.1.2 + knfsd 2.6.27 (.5) breaks fsid= option

nfs-utils 1.1.2 + knfsd 2.6.27 (.5) breaks fsid= option

[Frank van Maarseveen]

Tested on Debian lenny with nfs-kernel-server 1:1.1.2-6lenny1 and a
2.6.27.5 kernel. /etc/exports says:

	/mp @general(rw,sync,no_root_squash,no_subtree_check,mp,fsid=2886795869)

After actually using it /proc/fs/nfsd/exports says:

	/mp @general(rw,no_root_squash,sync,wdelay,no_subtree_check,fsid=-1408171427,uuid=db4387a6:bed949d0:8f5ef6a2:6a0c

(yes, 2886795869 == (ulong)-1408171427)
However, file handles over the wire now seem to have fsid_type=6
(FSID_UUID16) instead of 1 (FSID_NUM) due to this.

-- 
Frank

Re: nfs-utils 1.1.2 + knfsd 2.6.27 (.5) breaks fsid= option

[Steve Dickson]

Frank van Maarseveen wrote:
> Tested on Debian lenny with nfs-kernel-server 1:1.1.2-6lenny1 and a
> 2.6.27.5 kernel. /etc/exports says:
> 
> 	/mp @general(rw,sync,no_root_squash,no_subtree_check,mp,fsid=2886795869)
> 
> After actually using it /proc/fs/nfsd/exports says:
> 
> 	/mp @general(rw,no_root_squash,sync,wdelay,no_subtree_check,fsid=-1408171427,uuid=db4387a6:bed949d0:8f5ef6a2:6a0c
> 
> (yes, 2886795869 == (ulong)-1408171427)
> However, file handles over the wire now seem to have fsid_type=6
> (FSID_UUID16) instead of 1 (FSID_NUM) due to this.

This is a known problem... The kernel checks UUIDs before FSIDS which cause
FSIDS to be ignored. There are two outstanding proposals to fix this problem.
   1) Move two lines in the kernel so FSIDs are checked before UUIDS
   2) Change mountd to only send down  the FSID or the UUID but
      not both as it does today.
 
Neither proposal has been accepted... yet...

steved.

Re: nfs-utils 1.1.2 + knfsd 2.6.27 (.5) breaks fsid= option

[J. Bruce Fields]

On Wed, Nov 12, 2008 at 08:48:47AM -0500, Steve Dickson wrote:
> Frank van Maarseveen wrote:
> > Tested on Debian lenny with nfs-kernel-server 1:1.1.2-6lenny1 and a
> > 2.6.27.5 kernel. /etc/exports says:
> > 
> > 	/mp @general(rw,sync,no_root_squash,no_subtree_check,mp,fsid=2886795869)
> > 
> > After actually using it /proc/fs/nfsd/exports says:
> > 
> > 	/mp @general(rw,no_root_squash,sync,wdelay,no_subtree_check,fsid=-1408171427,uuid=db4387a6:bed949d0:8f5ef6a2:6a0c
> > 
> > (yes, 2886795869 == (ulong)-1408171427)
> > However, file handles over the wire now seem to have fsid_type=6
> > (FSID_UUID16) instead of 1 (FSID_NUM) due to this.
> 
> This is a known problem... The kernel checks UUIDs before FSIDS which cause
> FSIDS to be ignored. There are two outstanding proposals to fix this problem.
>    1) Move two lines in the kernel so FSIDs are checked before UUIDS
>    2) Change mountd to only send down  the FSID or the UUID but
>       not both as it does today.
>  
> Neither proposal has been accepted... yet...

I'd have a mild preference for the 2nd, but I'd first like to understand
why the new behavior is a problem for Frank.

--b.

Re: nfs-utils 1.1.2 + knfsd 2.6.27 (.5) breaks fsid= option

[Frank van Maarseveen]

On Sun, Nov 16, 2008 at 03:02:11PM -0500, J. Bruce Fields wrote:
> On Wed, Nov 12, 2008 at 08:48:47AM -0500, Steve Dickson wrote:
> > Frank van Maarseveen wrote:
> > > Tested on Debian lenny with nfs-kernel-server 1:1.1.2-6lenny1 and a
> > > 2.6.27.5 kernel. /etc/exports says:
> > > 
> > > 	/mp @general(rw,sync,no_root_squash,no_subtree_check,mp,fsid=2886795869)
> > > 
> > > After actually using it /proc/fs/nfsd/exports says:
> > > 
> > > 	/mp @general(rw,no_root_squash,sync,wdelay,no_subtree_check,fsid=-1408171427,uuid=db4387a6:bed949d0:8f5ef6a2:6a0c
> > > 
> > > (yes, 2886795869 == (ulong)-1408171427)
> > > However, file handles over the wire now seem to have fsid_type=6
> > > (FSID_UUID16) instead of 1 (FSID_NUM) due to this.
> > 
> > This is a known problem... The kernel checks UUIDs before FSIDS which cause
> > FSIDS to be ignored. There are two outstanding proposals to fix this problem.
> >    1) Move two lines in the kernel so FSIDs are checked before UUIDS
> >    2) Change mountd to only send down  the FSID or the UUID but
> >       not both as it does today.
> >  
> > Neither proposal has been accepted... yet...
> 
> I'd have a mild preference for the 2nd, but I'd first like to understand
> why the new behavior is a problem for Frank.

I'm using a patch originally developed by Wendy Cheng for a per-fsid
grace period. It could be triggered by:

	echo fsid >/proc/fs/nfsd/nlm_grace_fsid

where the mentioned fsid is the same as in /etc/exports. I'm not
particularly fond of using fsids for this and AFAIK a similar patch for
per-mountpoint grace period is underway. Correct?

-- 
Frank

Re: nfs-utils 1.1.2 + knfsd 2.6.27 (.5) breaks fsid= option

[J. Bruce Fields]

On Sun, Nov 16, 2008 at 03:02:11PM -0500, bfields wrote:
> On Wed, Nov 12, 2008 at 08:48:47AM -0500, Steve Dickson wrote:
> > Frank van Maarseveen wrote:
> > > Tested on Debian lenny with nfs-kernel-server 1:1.1.2-6lenny1 and a
> > > 2.6.27.5 kernel. /etc/exports says:
> > > 
> > > 	/mp @general(rw,sync,no_root_squash,no_subtree_check,mp,fsid=2886795869)
> > > 
> > > After actually using it /proc/fs/nfsd/exports says:
> > > 
> > > 	/mp @general(rw,no_root_squash,sync,wdelay,no_subtree_check,fsid=-1408171427,uuid=db4387a6:bed949d0:8f5ef6a2:6a0c
> > > 
> > > (yes, 2886795869 == (ulong)-1408171427)
> > > However, file handles over the wire now seem to have fsid_type=6
> > > (FSID_UUID16) instead of 1 (FSID_NUM) due to this.
> > 
> > This is a known problem... The kernel checks UUIDs before FSIDS which cause
> > FSIDS to be ignored. There are two outstanding proposals to fix this problem.
> >    1) Move two lines in the kernel so FSIDs are checked before UUIDS
> >    2) Change mountd to only send down  the FSID or the UUID but
> >       not both as it does today.
> >  
> > Neither proposal has been accepted... yet...
> 
> I'd have a mild preference for the 2nd,

Maybe I should take that back--Neil, don't we need to pass down the uuid
to match the client-provided filehandle type in the case where they're
giving us uuid-style filehandles?  In that case we need to pass down
both so the kernel can choose the desired one.  So if we need to fix
this then that would make SteveD's patch the way to go.

--b.

Re: nfs-utils 1.1.2 + knfsd 2.6.27 (.5) breaks fsid= option

[J. Bruce Fields]

On Sun, Nov 16, 2008 at 09:29:35PM +0100, Frank van Maarseveen wrote:
> On Sun, Nov 16, 2008 at 03:02:11PM -0500, J. Bruce Fields wrote:
> > On Wed, Nov 12, 2008 at 08:48:47AM -0500, Steve Dickson wrote:
> > > Frank van Maarseveen wrote:
> > > > Tested on Debian lenny with nfs-kernel-server 1:1.1.2-6lenny1 and a
> > > > 2.6.27.5 kernel. /etc/exports says:
> > > > 
> > > > 	/mp @general(rw,sync,no_root_squash,no_subtree_check,mp,fsid=2886795869)
> > > > 
> > > > After actually using it /proc/fs/nfsd/exports says:
> > > > 
> > > > 	/mp @general(rw,no_root_squash,sync,wdelay,no_subtree_check,fsid=-1408171427,uuid=db4387a6:bed949d0:8f5ef6a2:6a0c
> > > > 
> > > > (yes, 2886795869 == (ulong)-1408171427)
> > > > However, file handles over the wire now seem to have fsid_type=6
> > > > (FSID_UUID16) instead of 1 (FSID_NUM) due to this.
> > > 
> > > This is a known problem... The kernel checks UUIDs before FSIDS which cause
> > > FSIDS to be ignored. There are two outstanding proposals to fix this problem.
> > >    1) Move two lines in the kernel so FSIDs are checked before UUIDS
> > >    2) Change mountd to only send down  the FSID or the UUID but
> > >       not both as it does today.
> > >  
> > > Neither proposal has been accepted... yet...
> > 
> > I'd have a mild preference for the 2nd, but I'd first like to understand
> > why the new behavior is a problem for Frank.
> 
> I'm using a patch originally developed by Wendy Cheng for a per-fsid
> grace period. It could be triggered by:
> 
> 	echo fsid >/proc/fs/nfsd/nlm_grace_fsid
> 
> where the mentioned fsid is the same as in /etc/exports. I'm not
> particularly fond of using fsids for this and AFAIK a similar patch for
> per-mountpoint grace period is underway. Correct?

Yes.  In the meantime it shouldn't be hard to modify Wendy's patch to
use a filename instead of an fsid--see
fs/nfsd/nfsctl.c:failover_unlock_fs() for an example of an existing nfsd
file that takes a pathname.

But that shouldn't even be necessary--the fsid is still stored with the
filesystem internally, so Wendy's code should still be able to look up
exports using it.

--b.

Re: nfs-utils 1.1.2 + knfsd 2.6.27 (.5) breaks fsid= option

[NeilBrown]

On Mon, November 17, 2008 8:57 am, J. Bruce Fields wrote:
> On Sun, Nov 16, 2008 at 03:02:11PM -0500, bfields wrote:
>> On Wed, Nov 12, 2008 at 08:48:47AM -0500, Steve Dickson wrote:
>> > Frank van Maarseveen wrote:
>> > > Tested on Debian lenny with nfs-kernel-server 1:1.1.2-6lenny1 and a
>> > > 2.6.27.5 kernel. /etc/exports says:
>> > >
>> > > 	/mp
>> @general(rw,sync,no_root_squash,no_subtree_check,mp,fsid=2886795869)
>> > >
>> > > After actually using it /proc/fs/nfsd/exports says:
>> > >
>> > > 	/mp
>> @general(rw,no_root_squash,sync,wdelay,no_subtree_check,fsid=-1408171427,uuid=db4387a6:bed949d0:8f5ef6a2:6a0c
>> > >
>> > > (yes, 2886795869 == (ulong)-1408171427)
>> > > However, file handles over the wire now seem to have fsid_type=6
>> > > (FSID_UUID16) instead of 1 (FSID_NUM) due to this.
>> >
>> > This is a known problem... The kernel checks UUIDs before FSIDS which
>> cause
>> > FSIDS to be ignored. There are two outstanding proposals to fix this
>> problem.
>> >    1) Move two lines in the kernel so FSIDs are checked before UUIDS
>> >    2) Change mountd to only send down  the FSID or the UUID but
>> >       not both as it does today.
>> >
>> > Neither proposal has been accepted... yet...
>>
>> I'd have a mild preference for the 2nd,
>
> Maybe I should take that back--Neil, don't we need to pass down the uuid
> to match the client-provided filehandle type in the case where they're
> giving us uuid-style filehandles?  In that case we need to pass down
> both so the kernel can choose the desired one.  So if we need to fix
> this then that would make SteveD's patch the way to go.

Yes, I seem to be leaning towards option 1 now as well.
We probably never should have given UUIDs to the kernel if an fsid was
given (unless a uuid was given too).  In that case we wouldn't have
an issue here.
But as we do, the law of least surprise suggests that when an fsid is
explicitly given, it should be preferred.

So yeah, let's go with Steve's proposal.

NeilBrown