state manager failed on NFSv4 server

state manager failed on NFSv4 server

[Jim Rees]

A kernel built from Benny's pnfs-all-latest can no longer mount the EMC
server because nfs4_state_manager() fails.  I suspect it doesn't like
something about the EXCHANGE_ID reply.  This worked before Benny's last
update (late Dec?), and I can bisect if necessary, but wondered if someone
here can take a look and see if there is something obviously wrong.

I put a dump of the call and reply here:
http://www.citi.umich.edu/projects/nfsv4/pnfs/block/emc-ng-state-mgr

I can of course provide more info if that would help.

Re: state manager failed on NFSv4 server

[Trond Myklebust]

On Wed, 2011-01-12 at 13:58 -0500, Jim Rees wrote: 
> A kernel built from Benny's pnfs-all-latest can no longer mount the EMC
> server because nfs4_state_manager() fails.  I suspect it doesn't like
> something about the EXCHANGE_ID reply.  This worked before Benny's last
> update (late Dec?), and I can bisect if necessary, but wondered if someone
> here can take a look and see if there is something obviously wrong.
> 
> I put a dump of the call and reply here:
> http://www.citi.umich.edu/projects/nfsv4/pnfs/block/emc-ng-state-mgr
> 
> I can of course provide more info if that would help.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

That is correct. According to your wireshark dump, your server is
returning an illegal combination of the EXCHGID4_FLAG_USE_* flags.

The combination

        (EXCHGID4_FLAG_USE_PNFS_MDS | EXCHGID4_FLAG_USE_PNFS_DS |
        EXCHGID4_FLAG_USE_NON_PNFS)

is not listed in the table in section 13.1 of RFC5661 as an acceptable
return value.

Cheers
  Trond
-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
Trond.Myklebust@netapp.com
www.netapp.com

Re: state manager failed on NFSv4 server

[Jim Rees]

Trond Myklebust wrote:

  The combination
  
          (EXCHGID4_FLAG_USE_PNFS_MDS | EXCHGID4_FLAG_USE_PNFS_DS |
          EXCHGID4_FLAG_USE_NON_PNFS)
  
  is not listed in the table in section 13.1 of RFC5661 as an acceptable
  return value.

But section 13 doesn't apply to the block layout server, does it?

Re: state manager failed on NFSv4 server

[Trond Myklebust]

On Wed, 2011-01-12 at 19:07 -0500, Jim Rees wrote: 
> Trond Myklebust wrote:
> 
>   The combination
>   
>           (EXCHGID4_FLAG_USE_PNFS_MDS | EXCHGID4_FLAG_USE_PNFS_DS |
>           EXCHGID4_FLAG_USE_NON_PNFS)
>   
>   is not listed in the table in section 13.1 of RFC5661 as an acceptable
>   return value.
> 
> But section 13 doesn't apply to the block layout server, does it?

If you are a block layout server, the only valid return value is

EXCHGID4_FLAG_USE_PNFS_MDS
  
'cos you are clearly not a non-metadata server, and you clearly can't be
a data server.

Cheers
  Trond
-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
Trond.Myklebust@netapp.com
www.netapp.com

Re: state manager failed on NFSv4 server

[Jim Rees]

Trond Myklebust wrote:

  > But section 13 doesn't apply to the block layout server, does it?
  
  If you are a block layout server, the only valid return value is
  
  EXCHGID4_FLAG_USE_PNFS_MDS
    
  'cos you are clearly not a non-metadata server, and you clearly can't be
  a data server.

Sorry to be dense, this is the first time I've thought about this.

At the time of the EXCHANGE_ID call, how is the server supposed to know what
kind of layout is going to be negotiated?  It doesn't yet know whether the
client is even going to ask for a layout, does it?

Re: state manager failed on NFSv4 server

[Trond Myklebust]

On Wed, 2011-01-12 at 19:30 -0500, Jim Rees wrote: 
> Trond Myklebust wrote:
> 
>   > But section 13 doesn't apply to the block layout server, does it?
>   
>   If you are a block layout server, the only valid return value is
>   
>   EXCHGID4_FLAG_USE_PNFS_MDS
>     
>   'cos you are clearly not a non-metadata server, and you clearly can't be
>   a data server.
> 
> Sorry to be dense, this is the first time I've thought about this.
> 
> At the time of the EXCHANGE_ID call, how is the server supposed to know what
> kind of layout is going to be negotiated?  It doesn't yet know whether the
> client is even going to ask for a layout, does it?

It doesn't matter. EXCHGID4_FLAG_USE_PNFS_MDS is the server's way of
advertising to the client that it supports LAYOUTGET and other pNFS
related operations. The client is free to ignore that message if it so
desires, and just use read/write through MDS. The point here is to tell
the client whether or not it should try pNFS if it can.

Cheers
  Trond

-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
Trond.Myklebust@netapp.com
www.netapp.com

Re: state manager failed on NFSv4 server

[Jim Rees]

Trond Myklebust wrote:

  > At the time of the EXCHANGE_ID call, how is the server supposed to know what
  > kind of layout is going to be negotiated?  It doesn't yet know whether the
  > client is even going to ask for a layout, does it?
  
  It doesn't matter. EXCHGID4_FLAG_USE_PNFS_MDS is the server's way of
  advertising to the client that it supports LAYOUTGET and other pNFS
  related operations. The client is free to ignore that message if it so
  desires, and just use read/write through MDS. The point here is to tell
  the client whether or not it should try pNFS if it can.

I guess that makes sense.  But I think section 13 is a bit misleading since
it contains requirements for the block layout too.

13.  NFSv4.1 as a Storage Protocol in pNFS: the File Layout Type

   This section describes the semantics and format of NFSv4.1 file-based
   layouts for pNFS.  NFSv4.1 file-based layouts use the
   LAYOUT4_NFSV4_1_FILES layout type.  The LAYOUT4_NFSV4_1_FILES type
   defines striping data across multiple NFSv4.1 data servers.

Re: state manager failed on NFSv4 server

[Benny Halevy]

On 2011-01-13 02:54, Jim Rees wrote:
> Trond Myklebust wrote:
> 
>   > At the time of the EXCHANGE_ID call, how is the server supposed to know what
>   > kind of layout is going to be negotiated?  It doesn't yet know whether the
>   > client is even going to ask for a layout, does it?
>   
>   It doesn't matter. EXCHGID4_FLAG_USE_PNFS_MDS is the server's way of
>   advertising to the client that it supports LAYOUTGET and other pNFS
>   related operations. The client is free to ignore that message if it so
>   desires, and just use read/write through MDS. The point here is to tell
>   the client whether or not it should try pNFS if it can.
> 
> I guess that makes sense.  But I think section 13 is a bit misleading since
> it contains requirements for the block layout too.

True, but section 18.35. Operation 42: EXCHANGE_ID - Instantiate Client ID
specifically says:

   The EXCHGID4_FLAG_USE_NON_PNFS, EXCHGID4_FLAG_USE_PNFS_MDS, and
   EXCHGID4_FLAG_USE_PNFS_DS bits are described in Section 13.1 and
   convey roles the client ID is to be used for in a pNFS environment.
   The server MUST set one of the acceptable combinations of these bits
   (roles) in eir_flags, as specified in Section 13.1.

Benny

> 
> 13.  NFSv4.1 as a Storage Protocol in pNFS: the File Layout Type
> 
>    This section describes the semantics and format of NFSv4.1 file-based
>    layouts for pNFS.  NFSv4.1 file-based layouts use the
>    LAYOUT4_NFSV4_1_FILES layout type.  The LAYOUT4_NFSV4_1_FILES type
>    defines striping data across multiple NFSv4.1 data servers.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: state manager failed on NFSv4 server

[Andy Adamson]

Hi Jim

The EMC server is returning exchange_id flags = 0x00070000 which is illegal - it's claming the NON_PNFS, PNFS and DS roles. 

#define EXCHGID4_FLAG_USE_NON_PNFS              0x00010000
#define EXCHGID4_FLAG_USE_PNFS_MDS              0x00020000
#define EXCHGID4_FLAG_USE_PNFS_DS               0x00040000
#define EXCHGID4_FLAG_MASK_PNFS                 0x00070000

See Section 13.1 of RFC 5661

-->Andy 

On Jan 12, 2011, at 1:58 PM, Jim Rees wrote:

> A kernel built from Benny's pnfs-all-latest can no longer mount the EMC
> server because nfs4_state_manager() fails.  I suspect it doesn't like
> something about the EXCHANGE_ID reply.  This worked before Benny's last
> update (late Dec?), and I can bisect if necessary, but wondered if someone
> here can take a look and see if there is something obviously wrong.
> 
> I put a dump of the call and reply here:
> http://www.citi.umich.edu/projects/nfsv4/pnfs/block/emc-ng-state-mgr
> 
> I can of course provide more info if that would help.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html