The missing link -

The missing link -

[David Cureton]

Hi,
    I am using a SuSE 9.2 system as an NFS server and are seeing some
strange results. The server holds our home directories and exports a
number of disks to all the other workstations in the group.  The setup
is somewhat strange in that the first NFS mount, a directory called
"server", containing all the mountpoint for the subsequent mounting of
the other exported partitions


eg.  /server exported from the fileserver contains

drwxr-xr-x   4 root root 288 2005-04-01 15:32 .
lrwxrwxrwx   1 root root  27 2005-03-24 16:25 ./Shared ->
/server/volumes/vol0/Shared
lrwxrwxrwx   1 root root  26 2005-03-24 10:55 ./Staff ->
/server/volumes/vol0/Staff
lrwxrwxrwx   1 root root  27 2005-03-24 10:52 ./users ->
/server/volumes/vol0/users/
lrwxrwxrwx   1 root root  26 2005-03-24 10:51 ./usr1 ->
/server/volumes/vol0/usr1/
lrwxrwxrwx   1 root root  26 2005-03-24 10:51 ./usr2 ->
/server/volumes/vol0/usr2/
lrwxrwxrwx   1 root root  25 2005-03-24 10:53 ./usr3 ->
/server/volumes/vol0/usr3
lrwxrwxrwx   1 root root  25 2005-03-24 10:54 ./usr4 ->
/server/volumes/vol0/usr4
lrwxrwxrwx   1 root root  25 2005-03-24 10:54 ./usr5 ->
/server/volumes/vol0/usr5
drwxr-xr-x   5 root root 120 2005-03-29 13:52 ./volumes
drwxr-xr-x  12 root root 272 2005-04-01 16:00 ./volumes/vol0
drwxr-xr-x   4 root root  80 2005-03-23 17:53 ./volumes/vol1
drwxr-xr-x   2 root root  48 2005-03-29 13:52 ./volumes/vol2


The fstab of a client therefore looks like:

melbourne-fs3:/server      /server    nfs     defaults  0 2
melbourne-fs3:/server/volumes/vol2      /server/volumes/vol2     nfs
defaults  0 2
melbourne-fs3:/server/volumes/vol1      /server/volumes/vol1     nfs
defaults  0 2
melbourne-fs3:/server/volumes/vol0      /server/volumes/vol0     nfs
defaults  0 2


The clients  then have  a link from /Staff  --> /server/Staff which in
it self is a link to /server/volumes/vol0/Staff
i.e. /Staff (link on client's root)  --> /server/Staff (link on nfs
mount) --> /server/volumes/vol0/Staff (Real NFS exported directory)

This works fine most of the time and allows us to  move data between
disks without having to change links of the clients. It also allows us
to add new disks an easily bring them into use.

However on occasion we have noticed that the /server/Staff link just
disappears for some SuSE9.2 clients and the user no longer has access to
there home directory. All other clients still have the link available
and they are fine.

It appears that the client looses the /server/Staff  symbolic link out
of its NFS cache and forgets about it until the cache expires and does a
readdirplus on the /server directory.   All other accesses to the
/server director on the NFS client prior to the cache update  only check
the attibutes of the /server directory and it contents (excluding the
/server/Staff link) and as nothing has changed, uses the cached data.

I am at a loss as to how I can debug this NFS problem  further.  I have
done copious amounts of searching on google and have not found a similar
issue.

Server and client particulars are:

Both SuSE 9.2
Server 'uname -a': Linux melbourne-fs3 2.6.8-24.14-default #1 Tue Mar 29
09:27:43 UTC 2005 i686 athlon i386 GNU/Linux
Kernel based NFS server

Client 'uname -a': Linux cockatoo 2.6.8-24-default #1 Wed Oct 6 09:16:23
UTC 2004 i686 athlon i386 GNU/Linux



Any assistance as to how I should go about debuging such a beast would
be greatly appreciated, especially as it only happens 1 a fortnight
further componding being able to resolve it

David










-------------------------------------------------------
This SF.Net email is sponsored by: NEC IT Guy Games.
Get your fingers limbered up and give it your best shot. 4 great events, 4
opportunities to win big! Highest score wins.NEC IT Guy Games. Play to
win an NEC 61 plasma display. Visit http://www.necitguy.com/?r=20
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: The missing link -

[Trond Myklebust]

m=C3=A5 den 02.05.2005 Klokka 17:49 (+1000) skreiv David Cureton:

> However on occasion we have noticed that the /server/Staff link just
> disappears for some SuSE9.2 clients and the user no longer has access to
> there home directory. All other clients still have the link available
> and they are fine.

What kind of filesystem are you exporting?

When this behaviour happens, what does the client report when you do an
'ls -l' on /server/Staff?

Cheers,
  Trond
--=20
Trond Myklebust <trond.myklebust@fys.uio.no>



-------------------------------------------------------
This SF.Net email is sponsored by: NEC IT Guy Games.
Get your fingers limbered up and give it your best shot. 4 great events, 4
opportunities to win big! Highest score wins.NEC IT Guy Games. Play to
win an NEC 61 plasma display. Visit http://www.necitguy.com/?r=20
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: The missing link -

[David Cureton]

Hi Trond and list,
    Firstly sorry for the triple posting - My email client indicated it=20
could not send, however it just could not put the mail in the sent=20
folder - seems it actually sent the mail just fine :-|

We are  exporting a reiser file system to the clients.=20
 =20
When the behavior is happening there is no '/server/Staff'  link in the=20
ls of '/server' however I have not explicitly tried to do
an  'ls -l /server/Staff' on when the link is missing. Good idea! Next=20
time I see the issue I will do this and report back.

Many thanks,
David


Trond Myklebust wrote:

>m=C3=A5 den 02.05.2005 Klokka 17:49 (+1000) skreiv David Cureton:
>
> =20
>
>>However on occasion we have noticed that the /server/Staff link just
>>disappears for some SuSE9.2 clients and the user no longer has access t=
o
>>there home directory. All other clients still have the link available
>>and they are fine.
>>   =20
>>
>
>What kind of filesystem are you exporting?
>
>When this behaviour happens, what does the client report when you do an
>'ls -l' on /server/Staff?
>
>Cheers,
>  Trond
> =20
>



-------------------------------------------------------
This SF.Net email is sponsored by: NEC IT Guy Games.
Get your fingers limbered up and give it your best shot. 4 great events, =
4
opportunities to win big! Highest score wins.NEC IT Guy Games. Play to
win an NEC 61 plasma display. Visit http://www.necitguy.com/?r=3D20
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: ADV: Re: The missing link -

[David Cureton]

Hi Trond and List,
  =20
      It happened again today. The /server/Staff link went missing.

    "ls -l /server/Staff" explicitly specifying the missing link=20
returned an  "No such file or directory" error. =20
  =20
    "ls -l" in /server  on the effected client shows all but the=20
/server/Staff link.

        Previously I have monitored the NFS traffic where the client=20
does a getattr on the directory /server/Staff.
    Then if proceeds to getattr on the contents of the directory.=20
(Assuming it does not issue a getattr for /server/Staff. It is difficult=20
to map
the raw file handles onto the filenames. How can I do this?)

    Since the client is only doing getattr on the files and directories=20
I am assuming the client thinks that it's cache is OK  and proceeds with=20
it's local
    version of the truth. One that does not include the /server/Staff lin=
k.

    Therefore I tried the following:
       On the NFS server:  'touch test' in the /server diectory=20
therefore causing the client cache of /server to be invalid.
  =20
    On the client ls -la /server...... low and behold  the /server/Staff=20
link is back.

    Therefore it appears to be a client side issue.  It appears that the=20
client forgets about /server/Staff in it's cache?  Anyone else had this=20
issue?

Thanks,
    David





Trond Myklebust wrote:

>m=E5 den 02.05.2005 Klokka 17:49 (+1000) skreiv David Cureton:
>
> =20
>
>>However on occasion we have noticed that the /server/Staff link just
>>disappears for some SuSE9.2 clients and the user no longer has access t=
o
>>there home directory. All other clients still have the link available
>>and they are fine.
>>   =20
>>
>
>What kind of filesystem are you exporting?
>
>When this behaviour happens, what does the client report when you do an
>'ls -l' on /server/Staff?
>
>Cheers,
>  Trond
> =20
>



-------------------------------------------------------
This SF.Net email is sponsored by: NEC IT Guy Games.
Get your fingers limbered up and give it your best shot. 4 great events, =
4
opportunities to win big! Highest score wins.NEC IT Guy Games. Play to
win an NEC 61 plasma display. Visit http://www.necitguy.com/?r=3D20
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs