Re: What is NFSv4 READDIR doesn't return a filehandle....

Re: What is NFSv4 READDIR doesn't return a filehandle....

[Christopher T Vogan]

All,

Sorry for this late posting, a coworker was made aware of this thread 
recently and
has these replies to make.  Our server implementation is the one being
complained about in this thread.


Quoted text is from various entries in this forum.


Neil Brown:
===========
> Just a thought: while coping with broken servers would not be a good 
path to
> follow, detecting protocol violations and reporting an error might be...
> should the NFS client treat a missing filehandle and a malformed reply?

The server is allowed to remove an attribute bit from an entry in the 
readdir
reply, this is not "broken" nor is the reply "malformed".  The lack of a 
filehandle in the reply is deterministic.



Trond Myklebust:
================
> > A customer has come across a server which does not return the 
filehandle
> > information (is that allowed?).
> 
> The filehandle attribute is a mandatory attribute according to RFC3530, 
so I believe that the answer is "no".

Mandatory is described in RFS 3530 as that the server must return the 
attribute
on a GETATTR.  (Section 5, page 36).  There is nothing saying that it is
mandatory to return on a READDIR.  Our server will return the filehandle
on a LOOKUP/GETATTR every time.


Trond Myklebust:
================
> My concern is that the client can't objectively judge what constitutes a
> valid filehandle and what does not until it tries to use it in an RPC
> call.

Sure it can.  In the context of this discussion, if the readdir entry
has the filehandle attribute bit off in the reply, there is no filehandle.

I would note in the scenario we sent a trace for, the Linux client had 
a filehandle for the node in question, and "misplaced" it after a 
readdir to the directory containing that node did not return the 
filehandle
for that entry.  So calling the server "broken" is objectively inaccurate.

I would also note that this problem occurred after we upgraded to RHEL 
6.3;
the prior version did not have this issue, and our server did not change 
its
behavior.  My conclusion is that the means to obtain the filehandle was 
either broken by the client adding logic to assume a filehandle and 
obliterate 
the prior value after a readdir reply chose not to return it, or 
previously 
had a code path to lookup the node and obtain the filehandle, and removed 
that code path for some reason.  So again, pointing to server "breakage" 
is objectively inaccurate.

There are many references to "brokenness" or "server bugs" in this 
thread, and I take exception to it from a design and protocol conformance 
point of view.

This condition is easily recoverable, as ANY missing attribute on a 
readdir
is with a lookup/readdir.  Our decision to not return the FH on a readdir 
reply under certain narrow conditions is not one of convenience but of 
limitations of some underlying filesystem types, and if "convenience" is
to be used as an accusative, it can easily be returned to the client's 
refusal to deal with the legal withholding of an attribute on a readdir
reply.


Signed,
Steve Huntington
IBM z/OS NFS


Christopher Vogan
Dept. W98 NFS Development & Test

Re: What is NFSv4 READDIR doesn't return a filehandle....

[Myklebust, Trond]

Adding back in the Ccs...

On Wed, 2013-03-20 at 17:40 -0500, Christopher T Vogan wrote:

> Trond Myklebust:
> ================
> > > A customer has come across a server which does not return the 
> filehandle
> > > information (is that allowed?).
> > 
> > The filehandle attribute is a mandatory attribute according to RFC3530, 
> so I believe that the answer is "no".
> 
> Mandatory is described in RFS 3530 as that the server must return the 
> attribute
> on a GETATTR.  (Section 5, page 36).  There is nothing saying that it is
> mandatory to return on a READDIR.  Our server will return the filehandle
> on a LOOKUP/GETATTR every time.

Section 5.5 lists the filehandle as being REQUIRED, and "primarily for
readdir requests". What's the point of listing an attribute as REQUIRED,
but not for the primary (read "only!") operation where it is useful?

There is an exception allowed: if the object is a referral, then you can
return NFS4ERR_MOVED in the rdattr_error (which is also a REQUIRED
attribute that only applies to READDIR) if the client requests it.
Otherwise, you return NFS4ERR_MOVED in the READDIR result...

IOW: I strongly disagree with your argument, and repeat that your server
is broken as far as the protocol goes...

-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
Trond.Myklebust@netapp.com
www.netapp.com

Re: What is NFSv4 READDIR doesn't return a filehandle....

[Haynes, Tom]

Please note that I have cc'ed the NFSv4 list over at the IETF.

On Mar 20, 2013, at 3:40 PM, Christopher T Vogan <cvogan@us.ibm.com> wrote:

> All,
> 
> Sorry for this late posting, a coworker was made aware of this thread 
> recently and
> has these replies to make.  Our server implementation is the one being
> complained about in this thread.
> 
> 
> Quoted text is from various entries in this forum.
> 
> 
> Neil Brown:
> ===========
>> Just a thought: while coping with broken servers would not be a good 
> path to
>> follow, detecting protocol violations and reporting an error might be...
>> should the NFS client treat a missing filehandle and a malformed reply?
> 
> The server is allowed to remove an attribute bit from an entry in the 
> readdir
> reply, this is not "broken" nor is the reply "malformed".  The lack of a 
> filehandle in the reply is deterministic.
> 
> 
> 
> Trond Myklebust:
> ================
>>> A customer has come across a server which does not return the 
> filehandle
>>> information (is that allowed?).
>> 
>> The filehandle attribute is a mandatory attribute according to RFC3530, 
> so I believe that the answer is "no".
> 
> Mandatory is described in RFS 3530 as that the server must return the 
> attribute
> on a GETATTR.  (Section 5, page 36).  There is nothing saying that it is
> mandatory to return on a READDIR.  Our server will return the filehandle
> on a LOOKUP/GETATTR every time.



GETATTR and READDIR both return attributes. To be precise, they return
a fattr4.

Looking at Section 15.26.4 (roughly pages 277-279 of the most recent copy)
of 3530bis (3530 is on the way to being replaced), READDIR states:

   The READDIR operation retrieves a variable number of entries from a
   filesystem directory and returns client requested attributes for each
   entry along with information to allow the client to request
   additional directory entries in a subsequent READDIR.
...
   On successful return, the server's response will provide a list of
   directory entries.  Each of these entries contains the name of the
   directory entry, a cookie value for that entry, and the associated
   attributes as requested.  The "eof" flag has a value of TRUE if there
   are no more entries in the directory.

Any client implementation has no way to request that any server implementation
return the filehandle because the filehandle is not an attribute which
can be requested. I.e., it is mandatory.

If the intent was to allow the server to not return a filehandle for READDIR but to
allow it to return one for GETATTR, then it would have been made
OPTIONAL.

Whether or not the client used to work with such a server implementation
or not is immaterial as far as standard compliance is concerned.

If you like, we can clean up the corresponding language in 3530bis to
explicitly state that REQUIRED attributes are indeed required whether
in response to GETATTR or READDIR.






> 
> 
> Trond Myklebust:
> ================
>> My concern is that the client can't objectively judge what constitutes a
>> valid filehandle and what does not until it tries to use it in an RPC
>> call.
> 
> Sure it can.  In the context of this discussion, if the readdir entry
> has the filehandle attribute bit off in the reply, there is no filehandle.
> 
> I would note in the scenario we sent a trace for, the Linux client had 
> a filehandle for the node in question, and "misplaced" it after a 
> readdir to the directory containing that node did not return the 
> filehandle
> for that entry.  So calling the server "broken" is objectively inaccurate.
> 
> I would also note that this problem occurred after we upgraded to RHEL 
> 6.3;
> the prior version did not have this issue, and our server did not change 
> its
> behavior.  My conclusion is that the means to obtain the filehandle was 
> either broken by the client adding logic to assume a filehandle and 
> obliterate 
> the prior value after a readdir reply chose not to return it, or 
> previously 
> had a code path to lookup the node and obtain the filehandle, and removed 
> that code path for some reason.  So again, pointing to server "breakage" 
> is objectively inaccurate.
> 
> There are many references to "brokenness" or "server bugs" in this 
> thread, and I take exception to it from a design and protocol conformance 
> point of view.
> 
> This condition is easily recoverable, as ANY missing attribute on a 
> readdir
> is with a lookup/readdir.  Our decision to not return the FH on a readdir 
> reply under certain narrow conditions is not one of convenience but of 
> limitations of some underlying filesystem types, and if "convenience" is
> to be used as an accusative, it can easily be returned to the client's 
> refusal to deal with the legal withholding of an attribute on a readdir
> reply.
> 
> 
> Signed,
> Steve Huntington
> IBM z/OS NFS
> 
> 
> Christopher Vogan
> Dept. W98 NFS Development & Test
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: What is NFSv4 READDIR doesn't return a filehandle....

[Myklebust, Trond]

On Wed, 2013-03-20 at 23:53 +0000, Haynes, Tom wrote:
> Please note that I have cc'ed the NFSv4 list over at the IETF.
> 
> On Mar 20, 2013, at 3:40 PM, Christopher T Vogan <cvogan@us.ibm.com> wrote:
> 
> > All,
> > 
> > Sorry for this late posting, a coworker was made aware of this thread 
> > recently and
> > has these replies to make.  Our server implementation is the one being
> > complained about in this thread.
> > 
> > 
> > Quoted text is from various entries in this forum.
> > 
> > 
> > Neil Brown:
> > ===========
> >> Just a thought: while coping with broken servers would not be a good 
> > path to
> >> follow, detecting protocol violations and reporting an error might be...
> >> should the NFS client treat a missing filehandle and a malformed reply?
> > 
> > The server is allowed to remove an attribute bit from an entry in the 
> > readdir
> > reply, this is not "broken" nor is the reply "malformed".  The lack of a 
> > filehandle in the reply is deterministic.
> > 
> > 
> > 
> > Trond Myklebust:
> > ================
> >>> A customer has come across a server which does not return the 
> > filehandle
> >>> information (is that allowed?).
> >> 
> >> The filehandle attribute is a mandatory attribute according to RFC3530, 
> > so I believe that the answer is "no".
> > 
> > Mandatory is described in RFS 3530 as that the server must return the 
> > attribute
> > on a GETATTR.  (Section 5, page 36).  There is nothing saying that it is
> > mandatory to return on a READDIR.  Our server will return the filehandle
> > on a LOOKUP/GETATTR every time.
> 
> 
> 
> GETATTR and READDIR both return attributes. To be precise, they return
> a fattr4.
> 
> Looking at Section 15.26.4 (roughly pages 277-279 of the most recent copy)
> of 3530bis (3530 is on the way to being replaced), READDIR states:
> 
>    The READDIR operation retrieves a variable number of entries from a
>    filesystem directory and returns client requested attributes for each
>    entry along with information to allow the client to request
>    additional directory entries in a subsequent READDIR.
> ...
>    On successful return, the server's response will provide a list of
>    directory entries.  Each of these entries contains the name of the
>    directory entry, a cookie value for that entry, and the associated
>    attributes as requested.  The "eof" flag has a value of TRUE if there
>    are no more entries in the directory.
> 
> Any client implementation has no way to request that any server implementation
> return the filehandle because the filehandle is not an attribute which
> can be requested. I.e., it is mandatory.
> 
> If the intent was to allow the server to not return a filehandle for READDIR but to
> allow it to return one for GETATTR, then it would have been made
> OPTIONAL.
> 
> Whether or not the client used to work with such a server implementation
> or not is immaterial as far as standard compliance is concerned.
> 
> If you like, we can clean up the corresponding language in 3530bis to
> explicitly state that REQUIRED attributes are indeed required whether
> in response to GETATTR or READDIR.

It might rather be useful to add language to point out that the
"filehandle" attribute SHOULD NOT be used for the GETATTR case. We have
GETFH, and AFAICR, all the language around referrals etc is written with
the assumption that clients _won't_ use GETATTR to retrieve the
filehandle.

Ditto for the case of "rdattr_error", which makes absolutely no sense
whatsoever in a GETATTR request.

-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
Trond.Myklebust@netapp.com
www.netapp.com

Re: [nfsv4] What is NFSv4 READDIR doesn't return a filehandle....

[Rick Macklem]

Trond Myklebust wrote:
> On Wed, 2013-03-20 at 23:53 +0000, Haynes, Tom wrote:
> > Please note that I have cc'ed the NFSv4 list over at the IETF.
> >
> > On Mar 20, 2013, at 3:40 PM, Christopher T Vogan <cvogan@us.ibm.com>
> > wrote:
> >
> > > All,
> > >
> > > Sorry for this late posting, a coworker was made aware of this
> > > thread
> > > recently and
> > > has these replies to make. Our server implementation is the one
> > > being
> > > complained about in this thread.
> > >
> > >
> > > Quoted text is from various entries in this forum.
> > >
> > >
> > > Neil Brown:
> > > ===========
> > >> Just a thought: while coping with broken servers would not be a
> > >> good
> > > path to
> > >> follow, detecting protocol violations and reporting an error
> > >> might be...
> > >> should the NFS client treat a missing filehandle and a malformed
> > >> reply?
> > >
> > > The server is allowed to remove an attribute bit from an entry in
> > > the
> > > readdir
> > > reply, this is not "broken" nor is the reply "malformed". The lack
> > > of a
> > > filehandle in the reply is deterministic.
> > >
> > >
> > >
> > > Trond Myklebust:
> > > ================
> > >>> A customer has come across a server which does not return the
> > > filehandle
> > >>> information (is that allowed?).
> > >>
> > >> The filehandle attribute is a mandatory attribute according to
> > >> RFC3530,
> > > so I believe that the answer is "no".
> > >
> > > Mandatory is described in RFS 3530 as that the server must return
> > > the
> > > attribute
> > > on a GETATTR. (Section 5, page 36). There is nothing saying that
> > > it is
> > > mandatory to return on a READDIR. Our server will return the
> > > filehandle
> > > on a LOOKUP/GETATTR every time.
> >
> >
> >
> > GETATTR and READDIR both return attributes. To be precise, they
> > return
> > a fattr4.
> >
> > Looking at Section 15.26.4 (roughly pages 277-279 of the most recent
> > copy)
> > of 3530bis (3530 is on the way to being replaced), READDIR states:
> >
> >    The READDIR operation retrieves a variable number of entries from
> >    a
> >    filesystem directory and returns client requested attributes for
> >    each
> >    entry along with information to allow the client to request
> >    additional directory entries in a subsequent READDIR.
> > ...
> >    On successful return, the server's response will provide a list
> >    of
> >    directory entries. Each of these entries contains the name of the
> >    directory entry, a cookie value for that entry, and the
> >    associated
> >    attributes as requested. The "eof" flag has a value of TRUE if
> >    there
> >    are no more entries in the directory.
> >
> > Any client implementation has no way to request that any server
> > implementation
> > return the filehandle because the filehandle is not an attribute
> > which
> > can be requested. I.e., it is mandatory.
> >
> > If the intent was to allow the server to not return a filehandle for
> > READDIR but to
> > allow it to return one for GETATTR, then it would have been made
> > OPTIONAL.
> >
Well, I don't see any difference between a mandatory attribute and a
recommended attribute that the server claims to support via the
supported_attributes attribute.

I do believe that the server can choose not to return all of the
mandatory and supported recommended attributes in the readdir reply.
(If not, why have a bitmap of returned attributes?)

One example here is the mounted_on_fileid, which some servers choose
to only "support" for server mount points. (The FreeBSD server returns
this attribute for all file handles, setting it to the same value as
fileid for non-mount-points, but I am pretty sure some other servers
do not return mounted_on_fileid for non-mount-points.)

> > Whether or not the client used to work with such a server
> > implementation
> > or not is immaterial as far as standard compliance is concerned.
> >
> > If you like, we can clean up the corresponding language in 3530bis
> > to
> > explicitly state that REQUIRED attributes are indeed required
> > whether
> > in response to GETATTR or READDIR.
I assume that by REQUIRED you are referring to "mandatory attributes".

> 
> It might rather be useful to add language to point out that the
> "filehandle" attribute SHOULD NOT be used for the GETATTR case. We
> have
> GETFH, and AFAICR, all the language around referrals etc is written
> with
> the assumption that clients _won't_ use GETATTR to retrieve the
> filehandle.
> 
> Ditto for the case of "rdattr_error", which makes absolutely no sense
> whatsoever in a GETATTR request.
> 
And as I mentioned above, can a server choose to return mounted_on_fileid
for only some FHs when it claims to support the attribute? (I think that
is allowed and since I don't see a difference between mandatory and supported
recommended attributes I think the same applies to FH.)

Btw, when a server chooses to not return an FH in the readdir reply although
it was requested, the FreeBSD client "readdirplus" essentially falls back to
a basic "readdir" for the file name (ie. it doesn't prime the name and attribute
caches for it).

rick

> --
> Trond Myklebust
> Linux NFS client maintainer
> 
> NetApp
> Trond.Myklebust@netapp.com
> www.netapp.com
> _______________________________________________
> nfsv4 mailing list
> nfsv4@ietf.org
> https://www.ietf.org/mailman/listinfo/nfsv4

Re: [nfsv4] What is NFSv4 READDIR doesn't return a filehandle....

[Myklebust, Trond]

On Wed, 2013-03-20 at 21:41 -0400, Rick Macklem wrote:
> Trond Myklebust wrote:
> > On Wed, 2013-03-20 at 23:53 +0000, Haynes, Tom wrote:
> > > Please note that I have cc'ed the NFSv4 list over at the IETF.
> > >
> > > On Mar 20, 2013, at 3:40 PM, Christopher T Vogan <cvogan@us.ibm.com>
> > > wrote:
> > >
> > > > All,
> > > >
> > > > Sorry for this late posting, a coworker was made aware of this
> > > > thread
> > > > recently and
> > > > has these replies to make. Our server implementation is the one
> > > > being
> > > > complained about in this thread.
> > > >
> > > >
> > > > Quoted text is from various entries in this forum.
> > > >
> > > >
> > > > Neil Brown:
> > > > ===========
> > > >> Just a thought: while coping with broken servers would not be a
> > > >> good
> > > > path to
> > > >> follow, detecting protocol violations and reporting an error
> > > >> might be...
> > > >> should the NFS client treat a missing filehandle and a malformed
> > > >> reply?
> > > >
> > > > The server is allowed to remove an attribute bit from an entry in
> > > > the
> > > > readdir
> > > > reply, this is not "broken" nor is the reply "malformed". The lack
> > > > of a
> > > > filehandle in the reply is deterministic.
> > > >
> > > >
> > > >
> > > > Trond Myklebust:
> > > > ================
> > > >>> A customer has come across a server which does not return the
> > > > filehandle
> > > >>> information (is that allowed?).
> > > >>
> > > >> The filehandle attribute is a mandatory attribute according to
> > > >> RFC3530,
> > > > so I believe that the answer is "no".
> > > >
> > > > Mandatory is described in RFS 3530 as that the server must return
> > > > the
> > > > attribute
> > > > on a GETATTR. (Section 5, page 36). There is nothing saying that
> > > > it is
> > > > mandatory to return on a READDIR. Our server will return the
> > > > filehandle
> > > > on a LOOKUP/GETATTR every time.
> > >
> > >
> > >
> > > GETATTR and READDIR both return attributes. To be precise, they
> > > return
> > > a fattr4.
> > >
> > > Looking at Section 15.26.4 (roughly pages 277-279 of the most recent
> > > copy)
> > > of 3530bis (3530 is on the way to being replaced), READDIR states:
> > >
> > >    The READDIR operation retrieves a variable number of entries from
> > >    a
> > >    filesystem directory and returns client requested attributes for
> > >    each
> > >    entry along with information to allow the client to request
> > >    additional directory entries in a subsequent READDIR.
> > > ...
> > >    On successful return, the server's response will provide a list
> > >    of
> > >    directory entries. Each of these entries contains the name of the
> > >    directory entry, a cookie value for that entry, and the
> > >    associated
> > >    attributes as requested. The "eof" flag has a value of TRUE if
> > >    there
> > >    are no more entries in the directory.
> > >
> > > Any client implementation has no way to request that any server
> > > implementation
> > > return the filehandle because the filehandle is not an attribute
> > > which
> > > can be requested. I.e., it is mandatory.
> > >
> > > If the intent was to allow the server to not return a filehandle for
> > > READDIR but to
> > > allow it to return one for GETATTR, then it would have been made
> > > OPTIONAL.
> > >
> Well, I don't see any difference between a mandatory attribute and a
> recommended attribute that the server claims to support via the
> supported_attributes attribute.
> 
> I do believe that the server can choose not to return all of the
> mandatory and supported recommended attributes in the readdir reply.
> (If not, why have a bitmap of returned attributes?)

That's for dealing with OPTIONAL attributes. Section 5.2 of RFC3530
states:

"A client may ask for any of these attributes to be returned by setting
a bit in the GETATTR request but must handle the case where the server
does not return them."

Section 5.1 says:

"These MUST be supported by every NFS version 4 client and server in
order to ensure a minimum level of interoperability." There are no
exceptions made for READDIR.

> One example here is the mounted_on_fileid, which some servers choose
> to only "support" for server mount points. (The FreeBSD server returns
> this attribute for all file handles, setting it to the same value as
> fileid for non-mount-points, but I am pretty sure some other servers
> do not return mounted_on_fileid for non-mount-points.)
> 
> > > Whether or not the client used to work with such a server
> > > implementation
> > > or not is immaterial as far as standard compliance is concerned.
> > >
> > > If you like, we can clean up the corresponding language in 3530bis
> > > to
> > > explicitly state that REQUIRED attributes are indeed required
> > > whether
> > > in response to GETATTR or READDIR.
> I assume that by REQUIRED you are referring to "mandatory attributes".

I'm using the language from RFC5661 and RFC3530-bis, since that is the
most up-to-date.

> > It might rather be useful to add language to point out that the
> > "filehandle" attribute SHOULD NOT be used for the GETATTR case. We
> > have
> > GETFH, and AFAICR, all the language around referrals etc is written
> > with
> > the assumption that clients _won't_ use GETATTR to retrieve the
> > filehandle.
> > 
> > Ditto for the case of "rdattr_error", which makes absolutely no sense
> > whatsoever in a GETATTR request.
> > 
> And as I mentioned above, can a server choose to return mounted_on_fileid
> for only some FHs when it claims to support the attribute? (I think that
> is allowed and since I don't see a difference between mandatory and supported
> recommended attributes I think the same applies to FH.)

The fileid and mounted_on_fileid are both OPTIONAL attributes, and as
section 5.2 says, the client is required to be able to deal with the
case where the server does not return them. As stated earlier, there is
no such exception in 5.1.

> Btw, when a server chooses to not return an FH in the readdir reply although
> it was requested, the FreeBSD client "readdirplus" essentially falls back to
> a basic "readdir" for the file name (ie. it doesn't prime the name and attribute
> caches for it).

Implementations are only relevant here in as far as they limit the
protocol changes that RFC3530bis can make. Given that there are clients
out there that assume a strict interpretation of RFC3530, then it is too
late to have RFC3530bis relax that interpretation.

-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
Trond.Myklebust@netapp.com
www.netapp.com

Re: [nfsv4] What is NFSv4 READDIR doesn't return a filehandle....

[Myklebust, Trond]

On Thu, 2013-03-21 at 02:04 +0000, Myklebust, Trond wrote:
> Implementations are only relevant here in as far as they limit the
> protocol changes that RFC3530bis can make. Given that there are clients
> out there that assume a strict interpretation of RFC3530, then it is too
> late to have RFC3530bis relax that interpretation.

To put it differently: if you disagree, then please point to the text in
RFC3530, or RFC5661 (or RFC3530bis at this point) that supports your
position that Mandatory/REQUIRED attributes such as "filehandle" may be
ignored by a server in READDIR requests.
-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
Trond.Myklebust@netapp.com
www.netapp.com

Re: [nfsv4] What is NFSv4 READDIR doesn't return a filehandle....

[Rick Macklem]

Trond Myklebust wrote:
> On Wed, 2013-03-20 at 21:41 -0400, Rick Macklem wrote:
> > Trond Myklebust wrote:
> > > On Wed, 2013-03-20 at 23:53 +0000, Haynes, Tom wrote:
> > > > Please note that I have cc'ed the NFSv4 list over at the IETF.
> > > >
> > > > On Mar 20, 2013, at 3:40 PM, Christopher T Vogan
> > > > <cvogan@us.ibm.com>
> > > > wrote:
> > > >
> > > > > All,
> > > > >
> > > > > Sorry for this late posting, a coworker was made aware of this
> > > > > thread
> > > > > recently and
> > > > > has these replies to make. Our server implementation is the
> > > > > one
> > > > > being
> > > > > complained about in this thread.
> > > > >
> > > > >
> > > > > Quoted text is from various entries in this forum.
> > > > >
> > > > >
> > > > > Neil Brown:
> > > > > ===========
> > > > >> Just a thought: while coping with broken servers would not be
> > > > >> a
> > > > >> good
> > > > > path to
> > > > >> follow, detecting protocol violations and reporting an error
> > > > >> might be...
> > > > >> should the NFS client treat a missing filehandle and a
> > > > >> malformed
> > > > >> reply?
> > > > >
> > > > > The server is allowed to remove an attribute bit from an entry
> > > > > in
> > > > > the
> > > > > readdir
> > > > > reply, this is not "broken" nor is the reply "malformed". The
> > > > > lack
> > > > > of a
> > > > > filehandle in the reply is deterministic.
> > > > >
> > > > >
> > > > >
> > > > > Trond Myklebust:
> > > > > ================
> > > > >>> A customer has come across a server which does not return
> > > > >>> the
> > > > > filehandle
> > > > >>> information (is that allowed?).
> > > > >>
> > > > >> The filehandle attribute is a mandatory attribute according
> > > > >> to
> > > > >> RFC3530,
> > > > > so I believe that the answer is "no".
> > > > >
> > > > > Mandatory is described in RFS 3530 as that the server must
> > > > > return
> > > > > the
> > > > > attribute
> > > > > on a GETATTR. (Section 5, page 36). There is nothing saying
> > > > > that
> > > > > it is
> > > > > mandatory to return on a READDIR. Our server will return the
> > > > > filehandle
> > > > > on a LOOKUP/GETATTR every time.
> > > >
> > > >
> > > >
> > > > GETATTR and READDIR both return attributes. To be precise, they
> > > > return
> > > > a fattr4.
> > > >
> > > > Looking at Section 15.26.4 (roughly pages 277-279 of the most
> > > > recent
> > > > copy)
> > > > of 3530bis (3530 is on the way to being replaced), READDIR
> > > > states:
> > > >
> > > >    The READDIR operation retrieves a variable number of entries
> > > >    from
> > > >    a
> > > >    filesystem directory and returns client requested attributes
> > > >    for
> > > >    each
> > > >    entry along with information to allow the client to request
> > > >    additional directory entries in a subsequent READDIR.
> > > > ...
> > > >    On successful return, the server's response will provide a
> > > >    list
> > > >    of
> > > >    directory entries. Each of these entries contains the name of
> > > >    the
> > > >    directory entry, a cookie value for that entry, and the
> > > >    associated
> > > >    attributes as requested. The "eof" flag has a value of TRUE
> > > >    if
> > > >    there
> > > >    are no more entries in the directory.
> > > >
> > > > Any client implementation has no way to request that any server
> > > > implementation
> > > > return the filehandle because the filehandle is not an attribute
> > > > which
> > > > can be requested. I.e., it is mandatory.
> > > >
> > > > If the intent was to allow the server to not return a filehandle
> > > > for
> > > > READDIR but to
> > > > allow it to return one for GETATTR, then it would have been made
> > > > OPTIONAL.
> > > >
> > Well, I don't see any difference between a mandatory attribute and a
> > recommended attribute that the server claims to support via the
> > supported_attributes attribute.
> >
> > I do believe that the server can choose not to return all of the
> > mandatory and supported recommended attributes in the readdir reply.
> > (If not, why have a bitmap of returned attributes?)
> 
> That's for dealing with OPTIONAL attributes. Section 5.2 of RFC3530
> states:
> 
> "A client may ask for any of these attributes to be returned by
> setting
> a bit in the GETATTR request but must handle the case where the server
> does not return them."
> 
> Section 5.1 says:
> 
> "These MUST be supported by every NFS version 4 client and server in
> order to ensure a minimum level of interoperability." There are no
> exceptions made for READDIR.
> 
Yes, I just read them. I had forgotten (never realized) that for GETATTR
there is the rule that mandatory/required attributes must be returned.

You are also correct that there is no exception for READDIR. In fact,
I don't see any mention of READDIR in Sec. 5.1, 5.2. Both Sec. 5.1 and 5.2
refer specifically to GETATTR when stating whether they must be returned.

Then, when I read the description for READDIR, it seems to say that
the server is to set the rderror attribute if it cannot return all the
requested attributes (no exception for recommended/optional ones) or
fail the entire READDIR if the client hasn't asked for rderror.

Since this isn't what actually happens for mounted_on_fileid, I still
think the same should apply to other attributes requested by the
READDIR request and I don't see that the 5.1, 5.2 rule for GETATTR
applies to READDIR (ie. FH must be returned, but mounted_on_fileid
doesn't need to be).

> > One example here is the mounted_on_fileid, which some servers choose
> > to only "support" for server mount points. (The FreeBSD server
> > returns
> > this attribute for all file handles, setting it to the same value as
> > fileid for non-mount-points, but I am pretty sure some other servers
> > do not return mounted_on_fileid for non-mount-points.)
> >
> > > > Whether or not the client used to work with such a server
> > > > implementation
> > > > or not is immaterial as far as standard compliance is concerned.
> > > >
> > > > If you like, we can clean up the corresponding language in
> > > > 3530bis
> > > > to
> > > > explicitly state that REQUIRED attributes are indeed required
> > > > whether
> > > > in response to GETATTR or READDIR.
> > I assume that by REQUIRED you are referring to "mandatory
> > attributes".
> 
> I'm using the language from RFC5661 and RFC3530-bis, since that is the
> most up-to-date.
> 
I could nitpick and note that rfc3530bis is simply a draft at this point
in time and that RFC3530 is at this time the specification for NFSv4.0.
(The truth is I just haven't bothered reading rfc3530bis. Once it becomes
 an RFC and I get bit by changes, like the "uid # in the owner string"
 change, then I guess I'll have to.;-)

Btw, Sec. 5.1 also states that a client should be able to handle a
server that only supports the mandatory/required attributes.
Have you tested against a server that doesn't support the "fileid"
attribute yet? (I can guarantee that the FreeBSD client will
never work against such a server, so I don't have to worry about
trying to be fully conformant.;-)

rick

> > > It might rather be useful to add language to point out that the
> > > "filehandle" attribute SHOULD NOT be used for the GETATTR case. We
> > > have
> > > GETFH, and AFAICR, all the language around referrals etc is
> > > written
> > > with
> > > the assumption that clients _won't_ use GETATTR to retrieve the
> > > filehandle.
> > >
> > > Ditto for the case of "rdattr_error", which makes absolutely no
> > > sense
> > > whatsoever in a GETATTR request.
> > >
> > And as I mentioned above, can a server choose to return
> > mounted_on_fileid
> > for only some FHs when it claims to support the attribute? (I think
> > that
> > is allowed and since I don't see a difference between mandatory and
> > supported
> > recommended attributes I think the same applies to FH.)
> 
> The fileid and mounted_on_fileid are both OPTIONAL attributes, and as
> section 5.2 says, the client is required to be able to deal with the
> case where the server does not return them. As stated earlier, there
> is
> no such exception in 5.1.
> 
> > Btw, when a server chooses to not return an FH in the readdir reply
> > although
> > it was requested, the FreeBSD client "readdirplus" essentially falls
> > back to
> > a basic "readdir" for the file name (ie. it doesn't prime the name
> > and attribute
> > caches for it).
> 
> Implementations are only relevant here in as far as they limit the
> protocol changes that RFC3530bis can make. Given that there are
> clients
> out there that assume a strict interpretation of RFC3530, then it is
> too
> late to have RFC3530bis relax that interpretation.
> 
> --
> Trond Myklebust
> Linux NFS client maintainer
> 
> NetApp
> Trond.Myklebust@netapp.com
> www.netapp.com
> _______________________________________________
> nfsv4 mailing list
> nfsv4@ietf.org
> https://www.ietf.org/mailman/listinfo/nfsv4

Re: [nfsv4] What is NFSv4 READDIR doesn't return a filehandle....

[Rick Macklem]

Tom Haynes wrote:
> Please note that I have cc'ed the NFSv4 list over at the IETF.
> 
> On Mar 20, 2013, at 3:40 PM, Christopher T Vogan <cvogan@us.ibm.com>
> wrote:
> 
> > All,
> >
> > Sorry for this late posting, a coworker was made aware of this
> > thread
> > recently and
> > has these replies to make. Our server implementation is the one
> > being
> > complained about in this thread.
> >
> >
> > Quoted text is from various entries in this forum.
> >
> >
> > Neil Brown:
> > ===========
> >> Just a thought: while coping with broken servers would not be a
> >> good
> > path to
> >> follow, detecting protocol violations and reporting an error might
> >> be...
> >> should the NFS client treat a missing filehandle and a malformed
> >> reply?
> >
> > The server is allowed to remove an attribute bit from an entry in
> > the
> > readdir
> > reply, this is not "broken" nor is the reply "malformed". The lack
> > of a
> > filehandle in the reply is deterministic.
> >
> >
> >
> > Trond Myklebust:
> > ================
> >>> A customer has come across a server which does not return the
> > filehandle
> >>> information (is that allowed?).
> >>
> >> The filehandle attribute is a mandatory attribute according to
> >> RFC3530,
> > so I believe that the answer is "no".
> >
> > Mandatory is described in RFS 3530 as that the server must return
> > the
> > attribute
> > on a GETATTR. (Section 5, page 36). There is nothing saying that it
> > is
> > mandatory to return on a READDIR. Our server will return the
> > filehandle
> > on a LOOKUP/GETATTR every time.
> 
> 
> 
> GETATTR and READDIR both return attributes. To be precise, they return
> a fattr4.
> 
> Looking at Section 15.26.4 (roughly pages 277-279 of the most recent
> copy)
> of 3530bis (3530 is on the way to being replaced), READDIR states:
> 
> The READDIR operation retrieves a variable number of entries from a
> filesystem directory and returns client requested attributes for each
> entry along with information to allow the client to request
> additional directory entries in a subsequent READDIR.
> ...
> On successful return, the server's response will provide a list of
> directory entries. Each of these entries contains the name of the
> directory entry, a cookie value for that entry, and the associated
> attributes as requested. The "eof" flag has a value of TRUE if there
> are no more entries in the directory.
> 
> Any client implementation has no way to request that any server
> implementation
> return the filehandle because the filehandle is not an attribute which
> can be requested. I.e., it is mandatory.
> 
When I read this, all I think "requested" refers to is whether or not
the client sets the bit "requesting" that attribute.

> If the intent was to allow the server to not return a filehandle for
> READDIR but to
> allow it to return one for GETATTR, then it would have been made
> OPTIONAL.
> 
I don't know what the author's intent was, but since 5.1 only refers
to GETATTR, nothing in RFC3530 indicates that the same rule applies
to READDIR (ie. MANDATORY-->must always return it for READDIR).

> Whether or not the client used to work with such a server
> implementation
> or not is immaterial as far as standard compliance is concerned.
> 
> If you like, we can clean up the corresponding language in 3530bis to
> explicitly state that REQUIRED attributes are indeed required whether
> in response to GETATTR or READDIR.
> 
Personally, I think it should be "clarified" that for READDIR, the server
can choose to not reply any of the attributes. That allows the server implementor
more flexibility and since the old spec. didn't clearly require the REQUIRED
ones, I don't see why the new spec. should, unless it in impractical for clients
to allow this flexibility (and that is where the implementation evidence is
relevent, imho).

rick

> 
> 
> 
> 
> 
> >
> >
> > Trond Myklebust:
> > ================
> >> My concern is that the client can't objectively judge what
> >> constitutes a
> >> valid filehandle and what does not until it tries to use it in an
> >> RPC
> >> call.
> >
> > Sure it can. In the context of this discussion, if the readdir entry
> > has the filehandle attribute bit off in the reply, there is no
> > filehandle.
> >
> > I would note in the scenario we sent a trace for, the Linux client
> > had
> > a filehandle for the node in question, and "misplaced" it after a
> > readdir to the directory containing that node did not return the
> > filehandle
> > for that entry. So calling the server "broken" is objectively
> > inaccurate.
> >
> > I would also note that this problem occurred after we upgraded to
> > RHEL
> > 6.3;
> > the prior version did not have this issue, and our server did not
> > change
> > its
> > behavior. My conclusion is that the means to obtain the filehandle
> > was
> > either broken by the client adding logic to assume a filehandle and
> > obliterate
> > the prior value after a readdir reply chose not to return it, or
> > previously
> > had a code path to lookup the node and obtain the filehandle, and
> > removed
> > that code path for some reason. So again, pointing to server
> > "breakage"
> > is objectively inaccurate.
> >
> > There are many references to "brokenness" or "server bugs" in this
> > thread, and I take exception to it from a design and protocol
> > conformance
> > point of view.
> >
> > This condition is easily recoverable, as ANY missing attribute on a
> > readdir
> > is with a lookup/readdir. Our decision to not return the FH on a
> > readdir
> > reply under certain narrow conditions is not one of convenience but
> > of
> > limitations of some underlying filesystem types, and if
> > "convenience" is
> > to be used as an accusative, it can easily be returned to the
> > client's
> > refusal to deal with the legal withholding of an attribute on a
> > readdir
> > reply.
> >
> >
> > Signed,
> > Steve Huntington
> > IBM z/OS NFS
> >
> >
> > Christopher Vogan
> > Dept. W98 NFS Development & Test
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-nfs"
> > in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
> 
> _______________________________________________
> nfsv4 mailing list
> nfsv4@ietf.org
> https://www.ietf.org/mailman/listinfo/nfsv4

Re: [nfsv4] What is NFSv4 READDIR doesn't return a filehandle....

[Myklebust, Trond]

On Wed, 2013-03-20 at 23:38 -0400, Rick Macklem wrote:
> Trond Myklebust wrote:
> > On Wed, 2013-03-20 at 21:41 -0400, Rick Macklem wrote:
> > > Trond Myklebust wrote:
> > > > On Wed, 2013-03-20 at 23:53 +0000, Haynes, Tom wrote:
> > > > > Please note that I have cc'ed the NFSv4 list over at the IETF.
> > > > >
> > > > > On Mar 20, 2013, at 3:40 PM, Christopher T Vogan
> > > > > <cvogan@us.ibm.com>
> > > > > wrote:
> > > > >
> > > > > > All,
> > > > > >
> > > > > > Sorry for this late posting, a coworker was made aware of this
> > > > > > thread
> > > > > > recently and
> > > > > > has these replies to make. Our server implementation is the
> > > > > > one
> > > > > > being
> > > > > > complained about in this thread.
> > > > > >
> > > > > >
> > > > > > Quoted text is from various entries in this forum.
> > > > > >
> > > > > >
> > > > > > Neil Brown:
> > > > > > ===========
> > > > > >> Just a thought: while coping with broken servers would not be
> > > > > >> a
> > > > > >> good
> > > > > > path to
> > > > > >> follow, detecting protocol violations and reporting an error
> > > > > >> might be...
> > > > > >> should the NFS client treat a missing filehandle and a
> > > > > >> malformed
> > > > > >> reply?
> > > > > >
> > > > > > The server is allowed to remove an attribute bit from an entry
> > > > > > in
> > > > > > the
> > > > > > readdir
> > > > > > reply, this is not "broken" nor is the reply "malformed". The
> > > > > > lack
> > > > > > of a
> > > > > > filehandle in the reply is deterministic.
> > > > > >
> > > > > >
> > > > > >
> > > > > > Trond Myklebust:
> > > > > > ================
> > > > > >>> A customer has come across a server which does not return
> > > > > >>> the
> > > > > > filehandle
> > > > > >>> information (is that allowed?).
> > > > > >>
> > > > > >> The filehandle attribute is a mandatory attribute according
> > > > > >> to
> > > > > >> RFC3530,
> > > > > > so I believe that the answer is "no".
> > > > > >
> > > > > > Mandatory is described in RFS 3530 as that the server must
> > > > > > return
> > > > > > the
> > > > > > attribute
> > > > > > on a GETATTR. (Section 5, page 36). There is nothing saying
> > > > > > that
> > > > > > it is
> > > > > > mandatory to return on a READDIR. Our server will return the
> > > > > > filehandle
> > > > > > on a LOOKUP/GETATTR every time.
> > > > >
> > > > >
> > > > >
> > > > > GETATTR and READDIR both return attributes. To be precise, they
> > > > > return
> > > > > a fattr4.
> > > > >
> > > > > Looking at Section 15.26.4 (roughly pages 277-279 of the most
> > > > > recent
> > > > > copy)
> > > > > of 3530bis (3530 is on the way to being replaced), READDIR
> > > > > states:
> > > > >
> > > > >    The READDIR operation retrieves a variable number of entries
> > > > >    from
> > > > >    a
> > > > >    filesystem directory and returns client requested attributes
> > > > >    for
> > > > >    each
> > > > >    entry along with information to allow the client to request
> > > > >    additional directory entries in a subsequent READDIR.
> > > > > ...
> > > > >    On successful return, the server's response will provide a
> > > > >    list
> > > > >    of
> > > > >    directory entries. Each of these entries contains the name of
> > > > >    the
> > > > >    directory entry, a cookie value for that entry, and the
> > > > >    associated
> > > > >    attributes as requested. The "eof" flag has a value of TRUE
> > > > >    if
> > > > >    there
> > > > >    are no more entries in the directory.
> > > > >
> > > > > Any client implementation has no way to request that any server
> > > > > implementation
> > > > > return the filehandle because the filehandle is not an attribute
> > > > > which
> > > > > can be requested. I.e., it is mandatory.
> > > > >
> > > > > If the intent was to allow the server to not return a filehandle
> > > > > for
> > > > > READDIR but to
> > > > > allow it to return one for GETATTR, then it would have been made
> > > > > OPTIONAL.
> > > > >
> > > Well, I don't see any difference between a mandatory attribute and a
> > > recommended attribute that the server claims to support via the
> > > supported_attributes attribute.
> > >
> > > I do believe that the server can choose not to return all of the
> > > mandatory and supported recommended attributes in the readdir reply.
> > > (If not, why have a bitmap of returned attributes?)
> > 
> > That's for dealing with OPTIONAL attributes. Section 5.2 of RFC3530
> > states:
> > 
> > "A client may ask for any of these attributes to be returned by
> > setting
> > a bit in the GETATTR request but must handle the case where the server
> > does not return them."
> > 
> > Section 5.1 says:
> > 
> > "These MUST be supported by every NFS version 4 client and server in
> > order to ensure a minimum level of interoperability." There are no
> > exceptions made for READDIR.
> > 
> Yes, I just read them. I had forgotten (never realized) that for GETATTR
> there is the rule that mandatory/required attributes must be returned.
> 
> You are also correct that there is no exception for READDIR. In fact,
> I don't see any mention of READDIR in Sec. 5.1, 5.2. Both Sec. 5.1 and 5.2
> refer specifically to GETATTR when stating whether they must be returned.

No, but section 5.5 states very specifically that the "filehandle"
attribute is "primarily for readdir requests". What is the meaning of a
"Mandatory/Required" attribute if it doesn't apply to the primary (read
"only useful") case?

Ditto question for rdattr_error, which (as indicated earlier) _only_
applies to READDIR.

> Then, when I read the description for READDIR, it seems to say that
> the server is to set the rderror attribute if it cannot return all the
> requested attributes (no exception for recommended/optional ones) or
> fail the entire READDIR if the client hasn't asked for rderror.

I accept that.

> Since this isn't what actually happens for mounted_on_fileid, I still
> think the same should apply to other attributes requested by the
> READDIR request and I don't see that the 5.1, 5.2 rule for GETATTR
> applies to READDIR (ie. FH must be returned, but mounted_on_fileid
> doesn't need to be).

Where do you see support for this in the spec?

> > > One example here is the mounted_on_fileid, which some servers choose
> > > to only "support" for server mount points. (The FreeBSD server
> > > returns
> > > this attribute for all file handles, setting it to the same value as
> > > fileid for non-mount-points, but I am pretty sure some other servers
> > > do not return mounted_on_fileid for non-mount-points.)
> > >
> > > > > Whether or not the client used to work with such a server
> > > > > implementation
> > > > > or not is immaterial as far as standard compliance is concerned.
> > > > >
> > > > > If you like, we can clean up the corresponding language in
> > > > > 3530bis
> > > > > to
> > > > > explicitly state that REQUIRED attributes are indeed required
> > > > > whether
> > > > > in response to GETATTR or READDIR.
> > > I assume that by REQUIRED you are referring to "mandatory
> > > attributes".
> > 
> > I'm using the language from RFC5661 and RFC3530-bis, since that is the
> > most up-to-date.
> > 
> I could nitpick and note that rfc3530bis is simply a draft at this point
> in time and that RFC3530 is at this time the specification for NFSv4.0.
> (The truth is I just haven't bothered reading rfc3530bis. Once it becomes
>  an RFC and I get bit by changes, like the "uid # in the owner string"
>  change, then I guess I'll have to.;-)
> 
> Btw, Sec. 5.1 also states that a client should be able to handle a
> server that only supports the mandatory/required attributes.
> Have you tested against a server that doesn't support the "fileid"
> attribute yet? (I can guarantee that the FreeBSD client will
> never work against such a server, so I don't have to worry about
> trying to be fully conformant.;-)

I'm not certain that we've tested against a server that only supports
the mandatory attributes, but I do see some protocol problems there. Not
least, there is a known problem with determining the current value for
the change attribute; hence my proposal of the "change_attr_type"
OPTIONAL attribute for NFSv4.2. Unless someone has a valid objection,
I'm planning on proposing that attribute for REQUIRED status in
NFSv4.3...

-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
Trond.Myklebust@netapp.com
www.netapp.com

Re: [nfsv4] What is NFSv4 READDIR doesn't return a filehandle....

[J. Bruce Fields]

On Wed, Mar 20, 2013 at 09:41:40PM -0400, Rick Macklem wrote:
> Well, I don't see any difference between a mandatory attribute and a
> recommended attribute that the server claims to support via the
> supported_attributes attribute.
> 
> I do believe that the server can choose not to return all of the
> mandatory and supported recommended attributes in the readdir reply.
> (If not, why have a bitmap of returned attributes?)
> 
> One example here is the mounted_on_fileid, which some servers choose
> to only "support" for server mount points. (The FreeBSD server returns
> this attribute for all file handles, setting it to the same value as
> fileid for non-mount-points, but I am pretty sure some other servers
> do not return mounted_on_fileid for non-mount-points.)

That doesn't sound like traditional unix readdir behavior, and isn't the
behavior described by the spec; looking at 5661 5.8.2.23 (haven't
compared other rfcs):

	"If the server detects that there is no mounted point at the
	target file object, then the value for mounted_on_fileid that it
	returns is the same as that of the fileid attribute."

Also, the supported_attrs attribute is a filesystem-wide attribute.  I
don't think we generally allow attributes to vary between "supported"
and "unsupported" on a single filesystem.

--b.

Re: What is NFSv4 READDIR doesn't return a filehandle....

[J. Bruce Fields]

On Wed, Mar 20, 2013 at 05:40:11PM -0500, Christopher T Vogan wrote:
> This condition is easily recoverable, as ANY missing attribute on a 
> readdir
> is with a lookup/readdir.

By the same token, the server can just as well recover by doing that
extra lookup itself, can't it?  (That's more-or-less how the Linux
server gets attributes on readdir.)

Understood that it can be a pain (the Linux server's readdir
implementation has needed some minor surgery over the years to get this
right), but I do think it's the server's responsibility....

--b.

> Our decision to not return the FH on a readdir 
> reply under certain narrow conditions is not one of convenience but of 
> limitations of some underlying filesystem types, and if "convenience" is
> to be used as an accusative, it can easily be returned to the client's 
> refusal to deal with the legal withholding of an attribute on a readdir
> reply.

Re: [nfsv4] What is NFSv4 READDIR doesn't return a filehandle....

[Rick Macklem]

J. Bruce Fields wrote:
> On Wed, Mar 20, 2013 at 09:41:40PM -0400, Rick Macklem wrote:
> > Well, I don't see any difference between a mandatory attribute and a
> > recommended attribute that the server claims to support via the
> > supported_attributes attribute.
> >
> > I do believe that the server can choose not to return all of the
> > mandatory and supported recommended attributes in the readdir reply.
> > (If not, why have a bitmap of returned attributes?)
> >
> > One example here is the mounted_on_fileid, which some servers choose
> > to only "support" for server mount points. (The FreeBSD server
> > returns
> > this attribute for all file handles, setting it to the same value as
> > fileid for non-mount-points, but I am pretty sure some other servers
> > do not return mounted_on_fileid for non-mount-points.)
> 
> That doesn't sound like traditional unix readdir behavior, and isn't
> the
> behavior described by the spec; looking at 5661 5.8.2.23 (haven't
> compared other rfcs):
> 
> "If the server detects that there is no mounted point at the
> target file object, then the value for mounted_on_fileid that it
> returns is the same as that of the fileid attribute."
> 
> Also, the supported_attrs attribute is a filesystem-wide attribute. I
> don't think we generally allow attributes to vary between "supported"
> and "unsupported" on a single filesystem.
> 
> --b.
Well, I'm pretty sure some server did (maybe it has been changed more
recently) would only return mounted_on_fileid in the Readdir reply
if it was at a mount point.

I remember because it broke the FreeBSD client and I had to tweak
the code.

I'll try and remember to stick something in the FreeBSD client to
spot this for the next Bakeathon.

rick