Stale NFS mounts

Stale NFS mounts

[Edward Hibbert]

[-- Attachment #1: Type: text/plain, Size: 1191 bytes --]

I'm having a problem with stale mount points.

What we see is that under load all the clients get stale mount points
pretty much simultaneously.  We can rule out various things:
*	There's no indication that the NFS server has rebooted (uptime
indicates it hasn't).
*	There's nothing in /var/log/messages to indicate a problem.  
*	The nfsd processes have a start time that matches uptime, so
they haven't restarted.  
*	We know that the directories we're mounting are not getting
renamed/deleted.

We can cure this by a client reboot, but obviously that's massively
disruptive.

We're wondering whether this is related to nested mounts - the clients
mount a top-level directory, and then subdirectories within it.  There's
circumstantial but not conclusive evidence that we've only seen stale
handles on nested mounts.

So:
*	Are nested mounts safe to use?
*	Are there any common causes of stale mounts which I might not be
aware of?
*	Any suggestions for how to investigate this further?

The NFS server is a Fedora system with kernel 2.6.9-1.667smp.  The NFS
clients are RedHat AS 4.0 2.6.9-11.ELsmp.  Do ask for more details if
needed.

Regards,

Edward.

[-- Attachment #2: Type: text/html, Size: 2426 bytes --]

Re: Stale NFS mounts

[Neil Brown]

On Tuesday November 1, Edward.Hibbert@dataconnection.com wrote:
> 
> So:
> *	Are nested mounts safe to use?

Should be.

> *	Are there any common causes of stale mounts which I might not be
> aware of?

Not sure...

> *	Any suggestions for how to investigate this further?

See below.

> 
> The NFS server is a Fedora system with kernel 2.6.9-1.667smp.  The NFS
> clients are RedHat AS 4.0 2.6.9-11.ELsmp.  Do ask for more details if
> needed.

On the server:
  cat /etc/exports
  cat /proc/fs/nfsd/exports
  grep . /proc/net/rpc/*/content

Do this while it is working, and then again when it isn't working.

NeilBrown


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.
Get Certified Today * Register for a JBoss Training Course
Free Certification Exam for All Training Attendees Through End of 2005
Visit http://www.jboss.com/services/certification for more information
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

RE: Stale NFS mounts

[Edward Hibbert]

[-- Attachment #1: Type: text/plain, Size: 1899 bytes --]

>
> The NFS server is a Fedora system with kernel 2.6.9-1.667smp.  The NFS
> clients are RedHat AS 4.0 2.6.9-11.ELsmp.  Do ask for more details if
> needed.

On the server:
  cat /etc/exports
  cat /proc/fs/nfsd/exports
  grep . /proc/net/rpc/*/content

Ok, I've done this now while it's working - but what should I be looking
for here?  Maybe the last couple of lines changing?

root[vitorbelfort]:/var/log> cat /etc/exports 
/opt/dcl/data/disk1 *(rw,sync,no_root_squash)
/opt/dcl/data/disk2 *(rw,sync,no_root_squash)
root[vitorbelfort]:/var/log> cd /proc/fs/nfsd/exports 
-bash: cd: /proc/fs/nfsd/exports: Not a directory
root[vitorbelfort]:/var/log> cat /proc/fs/nfsd/exports  
# Version 1.1
# Path Client(Flags) # IPs
/opt/dcl/data/disk1     *(rw,no_root_squash,sync,wdelay)
/opt/dcl/data/disk2     *(rw,no_root_squash,sync,wdelay)
root[vitorbelfort]:/var/log>   grep . /proc/net/rpc/*/content
/proc/net/rpc/auth.unix.ip/content:#class IP domain
/proc/net/rpc/auth.unix.ip/content:nfsd 172.19.15.102 *
/proc/net/rpc/auth.unix.ip/content:nfsd 172.19.15.101 *
/proc/net/rpc/auth.unix.ip/content:nfsd 172.19.15.97 *
/proc/net/rpc/auth.unix.ip/content:nfsd 172.19.15.91 *
/proc/net/rpc/auth.unix.ip/content:nfsd 172.19.15.92 *
/proc/net/rpc/auth.unix.ip/content:nfsd 172.19.15.89 *
/proc/net/rpc/nfs4.idtoname/content:#domain type id [name]
/proc/net/rpc/nfs4.nametoid/content:#domain type name [id]
/proc/net/rpc/nfsd.export/content:#path domain(flags)
/proc/net/rpc/nfsd.export/content:/opt/dcl/data/disk1
*(rw,no_root_squash,sync,wdelay)
/proc/net/rpc/nfsd.export/content:/opt/dcl/data/disk2
*(rw,no_root_squash,sync,wdelay)
/proc/net/rpc/nfsd.fh/content:#domain fsidtype fsid [path]
/proc/net/rpc/nfsd.fh/content:* 0 0x0300080000000080 /opt/dcl/data/disk1
/proc/net/rpc/nfsd.fh/content:* 0 0x1100080000000080 /opt/dcl/data/disk2

Regards,

Edward.


[-- Attachment #2: Type: text/html, Size: 2621 bytes --]

RE: Stale NFS mounts

[Edward Hibbert]

[-- Attachment #1: Type: text/plain, Size: 2327 bytes --]

Bit more on this.  
 
If we export a directory X, then mount subdirectories within it (so from
a client, mount X/Y) then we see the problem.
If we export X/Y and mount X/Y then we don't.
 
Can anyone suggest any reason for this?
 
Regards,
 
Edward.

________________________________

From: Edward Hibbert 
Sent: 01 November 2005 10:58
To: 'nfs@lists.sourceforge.net'
Subject: RE: [NFS] Stale NFS mounts



>
> The NFS server is a Fedora system with kernel 2.6.9-1.667smp.  The NFS
> clients are RedHat AS 4.0 2.6.9-11.ELsmp.  Do ask for more details if
> needed.

On the server:
  cat /etc/exports
  cat /proc/fs/nfsd/exports
  grep . /proc/net/rpc/*/content

Ok, I've done this now while it's working - but what should I be looking
for here?  Maybe the last couple of lines changing?

root[vitorbelfort]:/var/log> cat /etc/exports 
/opt/dcl/data/disk1 *(rw,sync,no_root_squash)
/opt/dcl/data/disk2 *(rw,sync,no_root_squash)
root[vitorbelfort]:/var/log> cd /proc/fs/nfsd/exports 
-bash: cd: /proc/fs/nfsd/exports: Not a directory
root[vitorbelfort]:/var/log> cat /proc/fs/nfsd/exports  
# Version 1.1
# Path Client(Flags) # IPs
/opt/dcl/data/disk1     *(rw,no_root_squash,sync,wdelay)
/opt/dcl/data/disk2     *(rw,no_root_squash,sync,wdelay)
root[vitorbelfort]:/var/log>   grep . /proc/net/rpc/*/content
/proc/net/rpc/auth.unix.ip/content:#class IP domain
/proc/net/rpc/auth.unix.ip/content:nfsd 172.19.15.102 *
/proc/net/rpc/auth.unix.ip/content:nfsd 172.19.15.101 *
/proc/net/rpc/auth.unix.ip/content:nfsd 172.19.15.97 *
/proc/net/rpc/auth.unix.ip/content:nfsd 172.19.15.91 *
/proc/net/rpc/auth.unix.ip/content:nfsd 172.19.15.92 *
/proc/net/rpc/auth.unix.ip/content:nfsd 172.19.15.89 *
/proc/net/rpc/nfs4.idtoname/content:#domain type id [name]
/proc/net/rpc/nfs4.nametoid/content:#domain type name [id]
/proc/net/rpc/nfsd.export/content:#path domain(flags)
/proc/net/rpc/nfsd.export/content:/opt/dcl/data/disk1
*(rw,no_root_squash,sync,wdelay)
/proc/net/rpc/nfsd.export/content:/opt/dcl/data/disk2
*(rw,no_root_squash,sync,wdelay)
/proc/net/rpc/nfsd.fh/content:#domain fsidtype fsid [path]
/proc/net/rpc/nfsd.fh/content:* 0 0x0300080000000080 /opt/dcl/data/disk1
/proc/net/rpc/nfsd.fh/content:* 0 0x1100080000000080 /opt/dcl/data/disk2

Regards,

Edward.


[-- Attachment #2: Type: text/html, Size: 4370 bytes --]

Re: Stale NFS mounts

[Alexander Jolk]

Edward Hibbert wrote:
> I'm having a problem with stale mount points.

Might this be related to the problem described in 
http://linux.derkeiler.com/Mailing-Lists/Kernel/2003-09/7161.html?  You 
could try the export option `no_subtree_check' to see whether that helps.

Alex


-- 
Alexander Jolk  * BUF Compagnie * alexj@buf.com
Tel +33-1 42 68 18 28  *  Fax +33-1 42 68 18 29


-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server. Download
it for free - -and be entered to win a 42" plasma tv or your very own
Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs