NFS randomly denying permission to files

NFS randomly denying permission to files

[Don Allingham]

I have recently upgraded my NFS (and NIS) server to a newer system.
Unfortunately, this has caused some very strange errors. Certain
directories (such as .mozilla/firefox) will give Permission denied
errors about 50% of the time.

$ ls
8g3rtccm.default  pluginreg.dat  profiles.ini
$ ls
8g3rtccm.default  pluginreg.dat  profiles.ini
$ ls
ls: .: Permission denied
$ ls
ls: .: Permission denied

The client is using the same mount command as before, only with a
different server name. Both the old and new servers are running Ubuntu
6.10 (one is a older P4, the new one is a dual processor Opteron)

The fstab for the client has the line:

vlsi-fs0:/export /home nfs rw,rsize=8192,wsize=8192,intr,hard 0 0

The host export file is pretty simple:

/export *(rw,no_root_squash,sync)

UIDs and GIDs for all files are the same (even though the NIS server
changed as well).

Any ideas on how to debug this problem?

Thanks
Don
 


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier.
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: NFS randomly denying permission to files

[Neil Brown]

On Thursday February 1, don.allingham@dothill.com wrote:
> 
> Any ideas on how to debug this problem?
> 

Kernel version?

tcpdump -s 0  trace?

no_subtree_check export option?

NeilBrown

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier.
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: NFS randomly denying permission to files

[Don Allingham]

[-- Attachment #1.1: Type: text/plain, Size: 853 bytes --]


Kernel version for the server and clients are (uname -a):

Linux vlsi-fs0 2.6.17-10-generic #2 SMP Tue Dec 5 21:16:35 UTC 2006 x86_64 GNU/Linux

The no_subtree_check option seems to have made a big difference. 

I'm not familar with tcpdump. However, the new system is on a 1G ethernet link, and the old was on a 10/100 link. However, I will investigate this if I start seeing the problem again.

Thanks for the help.

Don


-----Original Message-----
From: Neil Brown [mailto:neilb@suse.de]
Sent: Thu 2/1/2007 3:50 PM
To: Don Allingham
Cc: nfs@lists.sourceforge.net
Subject: Re: [NFS] NFS randomly denying permission to files
 
On Thursday February 1, don.allingham@dothill.com wrote:
> 
> Any ideas on how to debug this problem?
> 

Kernel version?

tcpdump -s 0  trace?

no_subtree_check export option?

NeilBrown


[-- Attachment #1.2: Type: text/html, Size: 1427 bytes --]

[-- Attachment #2: Type: text/plain, Size: 374 bytes --]

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier.
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

[-- Attachment #3: Type: text/plain, Size: 140 bytes --]

_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: NFS randomly denying permission to files

[Neil Brown]

On Thursday February 1, Don.Allingham@dothill.com wrote:
> 
> Kernel version for the server and clients are (uname -a):
> 
> Linux vlsi-fs0 2.6.17-10-generic #2 SMP Tue Dec 5 21:16:35 UTC 2006 x86_64 GNU/Linux
> 
> The no_subtree_check option seems to have made a big difference. 

Hmmm... The bug I was thinking of (affected ext3 with dir_index if you
didn't use no_subtree_check) was fixed will before 2.6.17 came out.

What filesystem are you using?  If ext3, what does
  tune2fs -l /dev/XXXX
say (for the device holding the filesystem).

> 
> I'm not familar with tcpdump. However, the new system is on a 1G
> ethernet link, and the old was on a 10/100 link. However, I will
> investigate this if I start seeing the problem again. 

I wouldn't both.  If no_subtree_check fixes it, then tcpdump won't
show us anything interesting.

NeilBrown

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier.
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: NFS randomly denying permission to files

[Don Allingham]

[-- Attachment #1.1: Type: text/plain, Size: 2949 bytes --]


I've attached the results of tune2fs. The filesystem is a ext3 RAID0 configuration (/dev/md0) of two fibre channel arrays (a 1TB external RAID box configured as a two RAID5 arrays).

root@vlsi-fs0:~# tune2fs -l /dev/md0
tune2fs 1.39 (29-May-2006)
Filesystem volume name:   <none>
Last mounted on:          <not available>
Filesystem UUID:          912e625d-24fc-46ba-ab23-1965d7bc6623
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal resize_inode dir_index filetype needs_recovery sparse_super large_ file
Default mount options:    (none)
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              143622144
Block count:              287242144
Reserved block count:     14362107
Free blocks:              282684137
Free inodes:              143622133
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      955
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         16384
Inode blocks per group:   512
Filesystem created:       Wed Jan 31 21:47:49 2007
Last mount time:          Wed Jan 31 22:06:41 2007
Last write time:          Wed Jan 31 22:06:41 2007
Mount count:              3
Maximum mount count:      38
Last checked:             Wed Jan 31 21:47:49 2007
Check interval:           15552000 (6 months)
Next check after:         Mon Jul 30 22:47:49 2007
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:               128
Journal inode:            8
First orphan inode:       139690025
Default directory hash:   tea
Directory Hash Seed:      628a4a04-a659-46fd-8182-def39afe85fa
Journal backup:           inode blocks


-----Original Message-----
From: Neil Brown [mailto:neilb@suse.de]
Sent: Thu 2/1/2007 6:23 PM
To: Don Allingham
Cc: nfs@lists.sourceforge.net
Subject: Re: [NFS] NFS randomly denying permission to files
 
On Thursday February 1, Don.Allingham@dothill.com wrote:
> 
> Kernel version for the server and clients are (uname -a):
> 
> Linux vlsi-fs0 2.6.17-10-generic #2 SMP Tue Dec 5 21:16:35 UTC 2006 x86_64 GNU/Linux
> 
> The no_subtree_check option seems to have made a big difference. 

Hmmm... The bug I was thinking of (affected ext3 with dir_index if you
didn't use no_subtree_check) was fixed will before 2.6.17 came out.

What filesystem are you using?  If ext3, what does
  tune2fs -l /dev/XXXX
say (for the device holding the filesystem).

> 
> I'm not familar with tcpdump. However, the new system is on a 1G
> ethernet link, and the old was on a 10/100 link. However, I will
> investigate this if I start seeing the problem again. 

I wouldn't both.  If no_subtree_check fixes it, then tcpdump won't
show us anything interesting.

NeilBrown


[-- Attachment #1.2: Type: text/html, Size: 5343 bytes --]

[-- Attachment #2: Type: text/plain, Size: 374 bytes --]

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier.
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

[-- Attachment #3: Type: text/plain, Size: 140 bytes --]

_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs