NFS errors in 2.6

NFS errors in 2.6

[Miles Bader]

My home directory is NFS-mounted from a Solaris server like:

   mccfs10:/mccfs10-4/soft1 /home/soft1	nfs	nfsvers=3,rsize=1024,wsize=1024,noatime,nodiratime	0	0

everything works peachy with linux 2.4.23, and _mostly_ works in linux
2.6.0-test11, but the latter cannot read certain directories; for instance,
if I do `ls -l .gconf' in my homedir, I get an `Invalid Argument' error.

Here's what the directory looks like (in 2.4.23):

   $ ls -ld .gconf
   drwxr-xr-x    6 miles         512 Dec 10 14:15 .gconf
   $ ls -l .gconf
   total 3
   -rw-------    1 miles          31 Oct 29  2002 %gconf.xml
   drwx------   19 miles        1024 Aug 26 10:30 apps
   drwx------    3 miles         512 Oct 29  2002 desktop
   drwx------    4 miles         512 Oct 29  2002 schemas
   drwx------    4 miles         512 Jan 24  2003 system

In 2.6.0-test11, both of those commands fail like:

   $ ls -l .gconf
   ls: .gconf: Invalid Argument

[not sure of the precise syntax of the error message]


I have a packet capture of a failing `ls -l' command like the above;
here's the final packet (previous packets don't seem to have any errors):


Frame 22 (158 bytes on wire, 158 bytes captured)
Ethernet II, Src: 00:01:30:e9:cb:00, Dst: 00:03:47:97:9b:18
Internet Protocol, Src Addr: mccfs10.ucom.lsi.nec.co.jp (10.30.120.156), Dst Addr: mcspd15.ucom.lsi.nec.co.jp (10.30.114.174)
User Datagram Protocol, Src Port: 2049 (2049), Dst Port: 800 (800)
Remote Procedure Call, Type:Reply XID:0x0e72455c
    XID: 0xe72455c (242369884)
    Message Type: Reply (1)
    Program: NFS (100003)
    Program Version: 3
    Procedure: READDIRPLUS (17)
    Reply State: accepted (0)
    This is a reply to a request in frame 21
    Time from request: 0.001394000 seconds
    Verifier
        Flavor: AUTH_NULL (0)
        Length: 0
    Accept State: RPC executed successfully (0)
Network File System, READDIRPLUS Reply  Error:ERR_INVAL
    Program Version: 3
    V3 Procedure: READDIRPLUS (17)
    Status: ERR_INVAL (22)
    dir_attributes
        attributes_follow: value follows (1)
        attributes
            Type: Directory (2)
            mode: 040755
                0... .... .... = not SUID
                .0.. .... .... = not SGID
                ..0. .... .... = not save swapped text
                ...1 .... .... = Read permission for owner
                .... 1... .... = Write permission for owner
                .... .1.. .... = Execute permission for owner
                .... ..1. .... = Read permission for group
                .... ...0 .... = no Write permission for group
                .... .... 1... = Execute permission for group
                .... .... .1.. = Read permission for others
                .... .... ..0. = no Write permission for others
                .... .... ...1 = Execute permission for others
            nlink: 6
            uid: 31295
            gid: 10125
            size: 512
            used: 512
            rdev: 0,0
                specdata1: 0
                specdata2: 0
            fsid: 289359024837623808
            fileid: 4091573
            atime: Dec  9, 2003 13:34:07.040759000
                seconds: 1070944447
                nano seconds: 40759000
            mtime: Dec  9, 2003 12:47:36.827032000
                seconds: 1070941656
                nano seconds: 827032000
            ctime: Dec  9, 2003 12:47:36.827032000
                seconds: 1070941656
                nano seconds: 827032000


Any suggestions?

Thanks,

-Miles
-- 
[|nurgle|]  ddt- demonic? so quake will have an evil kinda setting? one that
            will  make every christian in the world foamm at the mouth?
[iddt]      nurg, that's the goal

Re: NFS errors in 2.6

[Trond Myklebust]

>>>>> " " == Miles Bader <miles@lsi.nec.co.jp> writes:

     > My home directory is NFS-mounted from a Solaris server like:
     >    mccfs10:/mccfs10-4/soft1 /home/soft1 nfs
     >    nfsvers=3,rsize=1024,wsize=1024,noatime,nodiratime 0 0

(BTW: noatime and nodirtime don't make sense in an NFS
environment. There is no way to notify the server not to update
attributes)

     > Frame 22 (158 bytes on wire, 158 bytes captured) Ethernet II,
     > Src: 00:01:30:e9:cb:00, Dst: 00:03:47:97:9b:18 Internet
     > Protocol, Src Addr: mccfs10.ucom.lsi.nec.co.jp (10.30.120.156),
     > Dst Addr: mcspd15.ucom.lsi.nec.co.jp (10.30.114.174) User
     > Datagram Protocol, Src Port: 2049 (2049), Dst Port: 800 (800)
     > Remote Procedure Call, Type:Reply XID:0x0e72455c
     >     XID: 0xe72455c (242369884) Message Type: Reply (1) Program:
     >     NFS (100003) Program Version: 3 Procedure: READDIRPLUS (17)
     >     Reply State: accepted (0) This is a reply to a request in
     >     frame 21 Time from request: 0.001394000 seconds Verifier
     >         Flavor: AUTH_NULL (0) Length: 0
     >     Accept State: RPC executed successfully (0)
     > Network File System, READDIRPLUS Reply Error:ERR_INVAL
     >     Program Version: 3 V3 Procedure: READDIRPLUS (17) Status:
     >     ERR_INVAL (22) dir_attributes

Interesting. That actually looks like an error on the part of your
Solaris server. NFS3ERR_INVAL is not a valid return code for either
READDIR or for READDIRPLUS according to RFC1813.

Is the server being kept up to scratch on the patch side?

Cheers,
  Trond

Re: NFS errors in 2.6

[Miles Bader]

Trond Myklebust <trond.myklebust@fys.uio.no> writes:
>      > Network File System, READDIRPLUS Reply Error:ERR_INVAL
>      >     Program Version: 3 V3 Procedure: READDIRPLUS (17) Status:
>      >     ERR_INVAL (22) dir_attributes
> 
> Interesting. That actually looks like an error on the part of your
> Solaris server. NFS3ERR_INVAL is not a valid return code for either
> READDIR or for READDIRPLUS according to RFC1813.

Hmmm; I did a packet-trace on the 2.4.23 traffic for the same user
command, and it doesn't use READDIRPLUS at all (does 2.4.23 not support
NFS V3, or does it just use a different algorithm for reading
directories?), so maybe the server's just broken.  Gah...

> Is the server being kept up to scratch on the patch side?

No clue, but I suspect the answer is `sort of' (we don't have a real
sysadmin staff, but the people who take care of the servers are not
entirely clueless).  I'll check on it.

Thanks,

-Miles
-- 
We have met the enemy, and he is us.  -- Pogo

Re: NFS errors in 2.6

[Trond Myklebust]

På on , 10/12/2003 klokka 21:19, skreiv Miles Bader:

> Hmmm; I did a packet-trace on the 2.4.23 traffic for the same user
> command, and it doesn't use READDIRPLUS at all (does 2.4.23 not support
> NFS V3, or does it just use a different algorithm for reading
> directories?), so maybe the server's just broken.  Gah...
> 

The 2.4.23 NFSv3 client does not support READDIRPLUS without extra
patches, so that would explain why you haven't seen the problem before.

Note, though, that those patches have been around for almost 3 years,
and have been tested pretty extensively against Solaris servers. I've
never seen any reports of this particular problem before.

If you'd like to try the 2.4.23 readdirplus patches, then you can find
them as part of

http://www.fys.uio.no/~trondmy/src/Linux-2.4.x/2.4.23-rc1/linux-2.4.23-NFS_ALL.dif

however as I said; given your ethereal dump, I suspect something is
screwed on the server itself...

Cheers,
  Trond