[PATCH] mountd: Ignore transient and non-fatal filesystem errors in nfsd_export

[PATCH] mountd: Ignore transient and non-fatal filesystem errors in nfsd_export

[trondmy]

From: Trond Myklebust <trond.myklebust@hammerspace.com>

If the mount point check in nfsd_export fails due to a transient error,
then ignore it to avoid spurious NFSERR_STALE errors being returned by
knfsd.

Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 utils/mountd/cache.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/utils/mountd/cache.c b/utils/mountd/cache.c
index 6cba2883026f..93e868341d15 100644
--- a/utils/mountd/cache.c
+++ b/utils/mountd/cache.c
@@ -1411,7 +1411,10 @@ static void nfsd_export(int f)
 
 		if (mp && !*mp)
 			mp = found->m_export.e_path;
-		if (mp && !is_mountpoint(mp))
+		errno = 0;
+		if (mp && !is_mountpoint(mp)) {
+			if (errno != 0 && !path_lookup_error(errno))
+				goto out;
 			/* Exportpoint is not mounted, so tell kernel it is
 			 * not available.
 			 * This will cause it not to appear in the V4 Pseudo-root
@@ -1420,9 +1423,12 @@ static void nfsd_export(int f)
 			 * And filehandle for this mountpoint from an earlier
 			 * mount will block in nfsd.fh lookup.
 			 */
+			xlog(L_WARNING,
+			     "Cannot export path '%s': not a mountpoint",
+			     path);
 			dump_to_cache(f, buf, sizeof(buf), dom, path,
 				      NULL, 60);
-		else if (dump_to_cache(f, buf, sizeof(buf), dom, path,
+		} else if (dump_to_cache(f, buf, sizeof(buf), dom, path,
 					 &found->m_export, 0) < 0) {
 			xlog(L_WARNING,
 			     "Cannot export %s, possibly unsupported filesystem"
-- 
2.26.2

Re: [PATCH] mountd: Ignore transient and non-fatal filesystem errors in nfsd_export

[J. Bruce Fields]

On Tue, Sep 08, 2020 at 05:19:58PM -0400, trondmy@kernel.org wrote:
> From: Trond Myklebust <trond.myklebust@hammerspace.com>
> 
> If the mount point check in nfsd_export fails due to a transient error,
> then ignore it to avoid spurious NFSERR_STALE errors being returned by
> knfsd.

What sort of transient errors?

I guess this makes the upcall (and the original rpc) eventually time
out?

--b.

> 
> Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
> ---
>  utils/mountd/cache.c | 10 ++++++++--
>  1 file changed, 8 insertions(+), 2 deletions(-)
> 
> diff --git a/utils/mountd/cache.c b/utils/mountd/cache.c
> index 6cba2883026f..93e868341d15 100644
> --- a/utils/mountd/cache.c
> +++ b/utils/mountd/cache.c
> @@ -1411,7 +1411,10 @@ static void nfsd_export(int f)
>  
>  		if (mp && !*mp)
>  			mp = found->m_export.e_path;
> -		if (mp && !is_mountpoint(mp))
> +		errno = 0;
> +		if (mp && !is_mountpoint(mp)) {
> +			if (errno != 0 && !path_lookup_error(errno))
> +				goto out;
>  			/* Exportpoint is not mounted, so tell kernel it is
>  			 * not available.
>  			 * This will cause it not to appear in the V4 Pseudo-root
> @@ -1420,9 +1423,12 @@ static void nfsd_export(int f)
>  			 * And filehandle for this mountpoint from an earlier
>  			 * mount will block in nfsd.fh lookup.
>  			 */
> +			xlog(L_WARNING,
> +			     "Cannot export path '%s': not a mountpoint",
> +			     path);
>  			dump_to_cache(f, buf, sizeof(buf), dom, path,
>  				      NULL, 60);
> -		else if (dump_to_cache(f, buf, sizeof(buf), dom, path,
> +		} else if (dump_to_cache(f, buf, sizeof(buf), dom, path,
>  					 &found->m_export, 0) < 0) {
>  			xlog(L_WARNING,
>  			     "Cannot export %s, possibly unsupported filesystem"
> -- 
> 2.26.2

Re: [PATCH] mountd: Ignore transient and non-fatal filesystem errors in nfsd_export

[Trond Myklebust]

On Thu, 2020-09-10 at 13:02 -0400, J. Bruce Fields wrote:
> On Tue, Sep 08, 2020 at 05:19:58PM -0400, trondmy@kernel.org wrote:
> > From: Trond Myklebust <trond.myklebust@hammerspace.com>
> > 
> > If the mount point check in nfsd_export fails due to a transient
> > error,
> > then ignore it to avoid spurious NFSERR_STALE errors being returned
> > by
> > knfsd.
> 
> What sort of transient errors?
> 
> I guess this makes the upcall (and the original rpc) eventually time
> out?

The point here is that is_mountpoint() is making the assumption that
_any_ error from stat() can be used to infer that there is nothing
mounted. In reality, only a subset of errors allow it to make that
assumption (specifically the errors ELOOP, ENAMETOOLONG, ENOENT, and
ENOTDIR).

In our case, we're seeing a problem when the underlying filesystem is a
soft mounted NFSv4.2 client (i.e. we're running our Hammerspace use
case of proxying NFSv4.2 to legacy NFSv3 clients) and that NFS4.2 mount
times out due to a reboot of the underlying server, for instance.

So, yes, in that case we want the upcall to time out instead of poking
knfsd into declaring that the directory is stale.


> 
> --b.
> 
> > Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
> > ---
> >  utils/mountd/cache.c | 10 ++++++++--
> >  1 file changed, 8 insertions(+), 2 deletions(-)
> > 
> > diff --git a/utils/mountd/cache.c b/utils/mountd/cache.c
> > index 6cba2883026f..93e868341d15 100644
> > --- a/utils/mountd/cache.c
> > +++ b/utils/mountd/cache.c
> > @@ -1411,7 +1411,10 @@ static void nfsd_export(int f)
> >  
> >  		if (mp && !*mp)
> >  			mp = found->m_export.e_path;
> > -		if (mp && !is_mountpoint(mp))
> > +		errno = 0;
> > +		if (mp && !is_mountpoint(mp)) {
> > +			if (errno != 0 && !path_lookup_error(errno))
> > +				goto out;
> >  			/* Exportpoint is not mounted, so tell kernel
> > it is
> >  			 * not available.
> >  			 * This will cause it not to appear in the V4
> > Pseudo-root
> > @@ -1420,9 +1423,12 @@ static void nfsd_export(int f)
> >  			 * And filehandle for this mountpoint from an
> > earlier
> >  			 * mount will block in nfsd.fh lookup.
> >  			 */
> > +			xlog(L_WARNING,
> > +			     "Cannot export path '%s': not a
> > mountpoint",
> > +			     path);
> >  			dump_to_cache(f, buf, sizeof(buf), dom, path,
> >  				      NULL, 60);
> > -		else if (dump_to_cache(f, buf, sizeof(buf), dom, path,
> > +		} else if (dump_to_cache(f, buf, sizeof(buf), dom,
> > path,
> >  					 &found->m_export, 0) < 0) {
> >  			xlog(L_WARNING,
> >  			     "Cannot export %s, possibly unsupported
> > filesystem"
> > -- 
> > 2.26.2
-- 
Trond Myklebust
CTO, Hammerspace Inc
4984 El Camino Real, Suite 208
Los Altos, CA 94022
www.hammer.space

Re: [PATCH] mountd: Ignore transient and non-fatal filesystem errors in nfsd_export

[bfields]

On Thu, Sep 10, 2020 at 05:25:19PM +0000, Trond Myklebust wrote:
> On Thu, 2020-09-10 at 13:02 -0400, J. Bruce Fields wrote:
> > On Tue, Sep 08, 2020 at 05:19:58PM -0400, trondmy@kernel.org wrote:
> > > From: Trond Myklebust <trond.myklebust@hammerspace.com>
> > > 
> > > If the mount point check in nfsd_export fails due to a transient
> > > error,
> > > then ignore it to avoid spurious NFSERR_STALE errors being returned
> > > by
> > > knfsd.
> > 
> > What sort of transient errors?
> > 
> > I guess this makes the upcall (and the original rpc) eventually time
> > out?
> 
> The point here is that is_mountpoint() is making the assumption that
> _any_ error from stat() can be used to infer that there is nothing
> mounted. In reality, only a subset of errors allow it to make that
> assumption (specifically the errors ELOOP, ENAMETOOLONG, ENOENT, and
> ENOTDIR).
> 
> In our case, we're seeing a problem when the underlying filesystem is a
> soft mounted NFSv4.2 client (i.e. we're running our Hammerspace use
> case of proxying NFSv4.2 to legacy NFSv3 clients) and that NFS4.2 mount
> times out due to a reboot of the underlying server, for instance.
> 
> So, yes, in that case we want the upcall to time out instead of poking
> knfsd into declaring that the directory is stale.

Thanks for the explanation, I guess I agree that's the best you can do
in that situation.

--b.

> > > Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
> > > ---
> > >  utils/mountd/cache.c | 10 ++++++++--
> > >  1 file changed, 8 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/utils/mountd/cache.c b/utils/mountd/cache.c
> > > index 6cba2883026f..93e868341d15 100644
> > > --- a/utils/mountd/cache.c
> > > +++ b/utils/mountd/cache.c
> > > @@ -1411,7 +1411,10 @@ static void nfsd_export(int f)
> > >  
> > >  		if (mp && !*mp)
> > >  			mp = found->m_export.e_path;
> > > -		if (mp && !is_mountpoint(mp))
> > > +		errno = 0;
> > > +		if (mp && !is_mountpoint(mp)) {
> > > +			if (errno != 0 && !path_lookup_error(errno))
> > > +				goto out;
> > >  			/* Exportpoint is not mounted, so tell kernel
> > > it is
> > >  			 * not available.
> > >  			 * This will cause it not to appear in the V4
> > > Pseudo-root
> > > @@ -1420,9 +1423,12 @@ static void nfsd_export(int f)
> > >  			 * And filehandle for this mountpoint from an
> > > earlier
> > >  			 * mount will block in nfsd.fh lookup.
> > >  			 */
> > > +			xlog(L_WARNING,
> > > +			     "Cannot export path '%s': not a
> > > mountpoint",
> > > +			     path);
> > >  			dump_to_cache(f, buf, sizeof(buf), dom, path,
> > >  				      NULL, 60);
> > > -		else if (dump_to_cache(f, buf, sizeof(buf), dom, path,
> > > +		} else if (dump_to_cache(f, buf, sizeof(buf), dom,
> > > path,
> > >  					 &found->m_export, 0) < 0) {
> > >  			xlog(L_WARNING,
> > >  			     "Cannot export %s, possibly unsupported
> > > filesystem"
> > > -- 
> > > 2.26.2
> -- 
> Trond Myklebust
> CTO, Hammerspace Inc
> 4984 El Camino Real, Suite 208
> Los Altos, CA 94022
> www.hammer.space
> 

Re: [PATCH] mountd: Ignore transient and non-fatal filesystem errors in nfsd_export

[Steve Dickson]

On 9/8/20 5:19 PM, trondmy@kernel.org wrote:
> From: Trond Myklebust <trond.myklebust@hammerspace.com>
> 
> If the mount point check in nfsd_export fails due to a transient error,
> then ignore it to avoid spurious NFSERR_STALE errors being returned by
> knfsd.
> 
> Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
Committed... (tag: nfs-utils-2-5-2-rc5)

steved.
> ---
>  utils/mountd/cache.c | 10 ++++++++--
>  1 file changed, 8 insertions(+), 2 deletions(-)
> 
> diff --git a/utils/mountd/cache.c b/utils/mountd/cache.c
> index 6cba2883026f..93e868341d15 100644
> --- a/utils/mountd/cache.c
> +++ b/utils/mountd/cache.c
> @@ -1411,7 +1411,10 @@ static void nfsd_export(int f)
>  
>  		if (mp && !*mp)
>  			mp = found->m_export.e_path;
> -		if (mp && !is_mountpoint(mp))
> +		errno = 0;
> +		if (mp && !is_mountpoint(mp)) {
> +			if (errno != 0 && !path_lookup_error(errno))
> +				goto out;
>  			/* Exportpoint is not mounted, so tell kernel it is
>  			 * not available.
>  			 * This will cause it not to appear in the V4 Pseudo-root
> @@ -1420,9 +1423,12 @@ static void nfsd_export(int f)
>  			 * And filehandle for this mountpoint from an earlier
>  			 * mount will block in nfsd.fh lookup.
>  			 */
> +			xlog(L_WARNING,
> +			     "Cannot export path '%s': not a mountpoint",
> +			     path);
>  			dump_to_cache(f, buf, sizeof(buf), dom, path,
>  				      NULL, 60);
> -		else if (dump_to_cache(f, buf, sizeof(buf), dom, path,
> +		} else if (dump_to_cache(f, buf, sizeof(buf), dom, path,
>  					 &found->m_export, 0) < 0) {
>  			xlog(L_WARNING,
>  			     "Cannot export %s, possibly unsupported filesystem"
>