NFSv4 referrals - custom (non-2049) port numbers in fs_locations?

NFSv4 referrals - custom (non-2049) port numbers in fs_locations?

[Martin Wege]

Good morning!

We have questions about NFSv4 referrals:
1. Is there a way to test them in Debian Linux?

2. How does a fs_locations attribute look like when a nonstandard port
like 6666 is used?
RFC5661 says this:

* http://tools.ietf.org/html/rfc5661#section-11.9
* 11.9. The Attribute fs_locations
* An entry in the server array is a UTF-8 string and represents one of a
* traditional DNS host name, IPv4 address, IPv6 address, or a zero-length
* string.  An IPv4 or IPv6 address is represented as a universal address
* (see Section 3.3.9 and [15]), minus the netid, and either with or without
* the trailing ".p1.p2" suffix that represents the port number.  If the
* suffix is omitted, then the default port, 2049, SHOULD be assumed.  A
* zero-length string SHOULD be used to indicate the current address being
* used for the RPC call.

Does anyone have an example of how the content of fs_locations should
look like with a custom port number?

Thanks,
Martin

Re: NFSv4 referrals - custom (non-2049) port numbers in fs_locations?

[Benjamin Coddington]

On 1 Nov 2023, at 5:06, Martin Wege wrote:

> Good morning!
>
> We have questions about NFSv4 referrals:
> 1. Is there a way to test them in Debian Linux?
>
> 2. How does a fs_locations attribute look like when a nonstandard port
> like 6666 is used?
> RFC5661 says this:
>
> * http://tools.ietf.org/html/rfc5661#section-11.9
> * 11.9. The Attribute fs_locations
> * An entry in the server array is a UTF-8 string and represents one of a
> * traditional DNS host name, IPv4 address, IPv6 address, or a zero-length
> * string.  An IPv4 or IPv6 address is represented as a universal address
> * (see Section 3.3.9 and [15]), minus the netid, and either with or without
> * the trailing ".p1.p2" suffix that represents the port number.  If the
> * suffix is omitted, then the default port, 2049, SHOULD be assumed.  A
> * zero-length string SHOULD be used to indicate the current address being
> * used for the RPC call.
>
> Does anyone have an example of how the content of fs_locations should
> look like with a custom port number?

If you keep following the references, you end up with the example in
rfc5665, which gives an example for IPv4:

https://datatracker.ietf.org/doc/html/rfc5665#section-5.2.3.3

Ben

Re: NFSv4 referrals - custom (non-2049) port numbers in fs_locations?

[Martin Wege]

On Wed, Nov 1, 2023 at 3:42 PM Benjamin Coddington <bcodding@redhat.com> wrote:
>
> On 1 Nov 2023, at 5:06, Martin Wege wrote:
>
> > Good morning!
> >
> > We have questions about NFSv4 referrals:
> > 1. Is there a way to test them in Debian Linux?
> >
> > 2. How does a fs_locations attribute look like when a nonstandard port
> > like 6666 is used?
> > RFC5661 says this:
> >
> > * http://tools.ietf.org/html/rfc5661#section-11.9
> > * 11.9. The Attribute fs_locations
> > * An entry in the server array is a UTF-8 string and represents one of a
> > * traditional DNS host name, IPv4 address, IPv6 address, or a zero-length
> > * string.  An IPv4 or IPv6 address is represented as a universal address
> > * (see Section 3.3.9 and [15]), minus the netid, and either with or without
> > * the trailing ".p1.p2" suffix that represents the port number.  If the
> > * suffix is omitted, then the default port, 2049, SHOULD be assumed.  A
> > * zero-length string SHOULD be used to indicate the current address being
> > * used for the RPC call.
> >
> > Does anyone have an example of how the content of fs_locations should
> > look like with a custom port number?
>
> If you keep following the references, you end up with the example in
> rfc5665, which gives an example for IPv4:
>
> https://datatracker.ietf.org/doc/html/rfc5665#section-5.2.3.3

So just <address>.<upper-byte-of-port-number>.<lower-byte-of-port-number>?

How can I test that with the refer= option in /etc/exports? nfsref
does not seem to have a ports option...

Thanks,
Martin

Re: NFSv4 referrals - custom (non-2049) port numbers in fs_locations?

[Benjamin Coddington]

On 10 Nov 2023, at 2:54, Martin Wege wrote:

> On Wed, Nov 1, 2023 at 3:42 PM Benjamin Coddington <bcodding@redhat.com> wrote:
>>
>> On 1 Nov 2023, at 5:06, Martin Wege wrote:
>>
>>> Good morning!
>>>
>>> We have questions about NFSv4 referrals:
>>> 1. Is there a way to test them in Debian Linux?
>>>
>>> 2. How does a fs_locations attribute look like when a nonstandard port
>>> like 6666 is used?
>>> RFC5661 says this:
>>>
>>> * http://tools.ietf.org/html/rfc5661#section-11.9
>>> * 11.9. The Attribute fs_locations
>>> * An entry in the server array is a UTF-8 string and represents one of a
>>> * traditional DNS host name, IPv4 address, IPv6 address, or a zero-length
>>> * string.  An IPv4 or IPv6 address is represented as a universal address
>>> * (see Section 3.3.9 and [15]), minus the netid, and either with or without
>>> * the trailing ".p1.p2" suffix that represents the port number.  If the
>>> * suffix is omitted, then the default port, 2049, SHOULD be assumed.  A
>>> * zero-length string SHOULD be used to indicate the current address being
>>> * used for the RPC call.
>>>
>>> Does anyone have an example of how the content of fs_locations should
>>> look like with a custom port number?
>>
>> If you keep following the references, you end up with the example in
>> rfc5665, which gives an example for IPv4:
>>
>> https://datatracker.ietf.org/doc/html/rfc5665#section-5.2.3.3
>
> So just <address>.<upper-byte-of-port-number>.<lower-byte-of-port-number>?
>
> How can I test that with the refer= option in /etc/exports? nfsref
> does not seem to have a ports option...

Just test it!

I thought the nfsref program actually populates the "trusted.junction.nfs"
xattr, which is part of the "fedfs" project's metadata to link filesystems
together.  I don't think that's what you want here.

Chuck - am I right to say that the nfsref program does not populate
nfsd4_fs_locations on knfsd?

Ben

Re: NFSv4 referrals - custom (non-2049) port numbers in fs_locations?

[Chuck Lever III]

> On Nov 10, 2023, at 8:49 AM, Benjamin Coddington <bcodding@redhat.com> wrote:
> 
> On 10 Nov 2023, at 2:54, Martin Wege wrote:
> 
>> On Wed, Nov 1, 2023 at 3:42 PM Benjamin Coddington <bcodding@redhat.com> wrote:
>>> 
>>> On 1 Nov 2023, at 5:06, Martin Wege wrote:
>>> 
>>>> Good morning!
>>>> 
>>>> We have questions about NFSv4 referrals:
>>>> 1. Is there a way to test them in Debian Linux?
>>>> 
>>>> 2. How does a fs_locations attribute look like when a nonstandard port
>>>> like 6666 is used?
>>>> RFC5661 says this:
>>>> 
>>>> * http://tools.ietf.org/html/rfc5661#section-11.9
>>>> * 11.9. The Attribute fs_locations
>>>> * An entry in the server array is a UTF-8 string and represents one of a
>>>> * traditional DNS host name, IPv4 address, IPv6 address, or a zero-length
>>>> * string.  An IPv4 or IPv6 address is represented as a universal address
>>>> * (see Section 3.3.9 and [15]), minus the netid, and either with or without
>>>> * the trailing ".p1.p2" suffix that represents the port number.  If the
>>>> * suffix is omitted, then the default port, 2049, SHOULD be assumed.  A
>>>> * zero-length string SHOULD be used to indicate the current address being
>>>> * used for the RPC call.
>>>> 
>>>> Does anyone have an example of how the content of fs_locations should
>>>> look like with a custom port number?
>>> 
>>> If you keep following the references, you end up with the example in
>>> rfc5665, which gives an example for IPv4:
>>> 
>>> https://datatracker.ietf.org/doc/html/rfc5665#section-5.2.3.3
>> 
>> So just <address>.<upper-byte-of-port-number>.<lower-byte-of-port-number>?
>> 
>> How can I test that with the refer= option in /etc/exports? nfsref
>> does not seem to have a ports option...
> 
> Just test it!
> 
> I thought the nfsref program actually populates the "trusted.junction.nfs"
> xattr, which is part of the "fedfs" project's metadata to link filesystems
> together.  I don't think that's what you want here.

No, nfsref is what Martin wants.


> Chuck - am I right to say that the nfsref program does not populate
> nfsd4_fs_locations on knfsd?

nfsref is the proper tool to use.

nfsref turns a directory into a junction by doing two things:

1. It adds a trusted.junction.nfs xattr containing the information
   that the server returns when a client does a GETATTR(fs_locations)
   on that directory

2. It updates the directory's mode bits to mark it as a junction

It is mountd that takes either the refer=/replica= export option
or the junction xattr and feeds that to the kernel so it can
construct a GETATTR(fs_locations) response.


--
Chuck Lever

Re: NFSv4 referrals - custom (non-2049) port numbers in fs_locations?

[Cedric Blancher]

On Fri, 10 Nov 2023 at 19:14, Chuck Lever III <chuck.lever@oracle.com> wrote:
>
>
> > On Nov 10, 2023, at 8:49 AM, Benjamin Coddington <bcodding@redhat.com> wrote:
> >
> > On 10 Nov 2023, at 2:54, Martin Wege wrote:
> >
> >> On Wed, Nov 1, 2023 at 3:42 PM Benjamin Coddington <bcodding@redhat.com> wrote:
> >>>
> >>> On 1 Nov 2023, at 5:06, Martin Wege wrote:
> >>>
> >>>> Good morning!
> >>>>
> >>>> We have questions about NFSv4 referrals:
> >>>> 1. Is there a way to test them in Debian Linux?
> >>>>
> >>>> 2. How does a fs_locations attribute look like when a nonstandard port
> >>>> like 6666 is used?
> >>>> RFC5661 says this:
> >>>>
> >>>> * http://tools.ietf.org/html/rfc5661#section-11.9
> >>>> * 11.9. The Attribute fs_locations
> >>>> * An entry in the server array is a UTF-8 string and represents one of a
> >>>> * traditional DNS host name, IPv4 address, IPv6 address, or a zero-length
> >>>> * string.  An IPv4 or IPv6 address is represented as a universal address
> >>>> * (see Section 3.3.9 and [15]), minus the netid, and either with or without
> >>>> * the trailing ".p1.p2" suffix that represents the port number.  If the
> >>>> * suffix is omitted, then the default port, 2049, SHOULD be assumed.  A
> >>>> * zero-length string SHOULD be used to indicate the current address being
> >>>> * used for the RPC call.
> >>>>
> >>>> Does anyone have an example of how the content of fs_locations should
> >>>> look like with a custom port number?
> >>>
> >>> If you keep following the references, you end up with the example in
> >>> rfc5665, which gives an example for IPv4:
> >>>
> >>> https://datatracker.ietf.org/doc/html/rfc5665#section-5.2.3.3
> >>
> >> So just <address>.<upper-byte-of-port-number>.<lower-byte-of-port-number>?
> >>
> >> How can I test that with the refer= option in /etc/exports? nfsref
> >> does not seem to have a ports option...
> >
> > Just test it!
> >
> > I thought the nfsref program actually populates the "trusted.junction.nfs"
> > xattr, which is part of the "fedfs" project's metadata to link filesystems
> > together.  I don't think that's what you want here.
>
> No, nfsref is what Martin wants.
>
>
> > Chuck - am I right to say that the nfsref program does not populate
> > nfsd4_fs_locations on knfsd?
>
> nfsref is the proper tool to use.

nfsref is not being packaged. And likely will not be available for a
couple of years, even if nfs-utils builds it by default.

So what do I have to set in /usr/setfattr to define a junction?

>
> nfsref turns a directory into a junction by doing two things:
>
> 1. It adds a trusted.junction.nfs xattr containing the information
>    that the server returns when a client does a GETATTR(fs_locations)
>    on that directory
>
> 2. It updates the directory's mode bits to mark it as a junction

Which directory mode bits are that?

And maybe that is the reason that having a junction redirect to /home
when /home is controlled by the automounter makes it fail?

Ced
-- 
Cedric Blancher <cedric.blancher@gmail.com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur

Re: NFSv4 referrals - custom (non-2049) port numbers in fs_locations?

[Chuck Lever III]

> On Nov 10, 2023, at 2:54 AM, Martin Wege <martin.l.wege@gmail.com> wrote:
> 
> On Wed, Nov 1, 2023 at 3:42 PM Benjamin Coddington <bcodding@redhat.com> wrote:
>> 
>> On 1 Nov 2023, at 5:06, Martin Wege wrote:
>> 
>>> Good morning!
>>> 
>>> We have questions about NFSv4 referrals:
>>> 1. Is there a way to test them in Debian Linux?
>>> 
>>> 2. How does a fs_locations attribute look like when a nonstandard port
>>> like 6666 is used?
>>> RFC5661 says this:
>>> 
>>> * http://tools.ietf.org/html/rfc5661#section-11.9
>>> * 11.9. The Attribute fs_locations
>>> * An entry in the server array is a UTF-8 string and represents one of a
>>> * traditional DNS host name, IPv4 address, IPv6 address, or a zero-length
>>> * string.  An IPv4 or IPv6 address is represented as a universal address
>>> * (see Section 3.3.9 and [15]), minus the netid, and either with or without
>>> * the trailing ".p1.p2" suffix that represents the port number.  If the
>>> * suffix is omitted, then the default port, 2049, SHOULD be assumed.  A
>>> * zero-length string SHOULD be used to indicate the current address being
>>> * used for the RPC call.
>>> 
>>> Does anyone have an example of how the content of fs_locations should
>>> look like with a custom port number?
>> 
>> If you keep following the references, you end up with the example in
>> rfc5665, which gives an example for IPv4:
>> 
>> https://datatracker.ietf.org/doc/html/rfc5665#section-5.2.3.3
> 
> So just <address>.<upper-byte-of-port-number>.<lower-byte-of-port-number>?

> How can I test that with the refer= option in /etc/exports? nfsref
> does not seem to have a ports option...

Neither refer= nor nfsref support alternate ports for exactly the
same reason: The mountd upcall/downcall, which is how the kernel
learns of referral target locations, needs to be fixed first. Then
support for alternate ports can be implemented in both refer= and
nfsref.


--
Chuck Lever

Re: NFSv4 referrals - custom (non-2049) port numbers in fs_locations?

[Cedric Blancher]

On Mon, 13 Nov 2023 at 17:19, Chuck Lever III <chuck.lever@oracle.com> wrote:
>
>
>
> > On Nov 10, 2023, at 2:54 AM, Martin Wege <martin.l.wege@gmail.com> wrote:
> >
> > On Wed, Nov 1, 2023 at 3:42 PM Benjamin Coddington <bcodding@redhat.com> wrote:
> >>
> >> On 1 Nov 2023, at 5:06, Martin Wege wrote:
> >>
> >>> Good morning!
> >>>
> >>> We have questions about NFSv4 referrals:
> >>> 1. Is there a way to test them in Debian Linux?
> >>>
> >>> 2. How does a fs_locations attribute look like when a nonstandard port
> >>> like 6666 is used?
> >>> RFC5661 says this:
> >>>
> >>> * http://tools.ietf.org/html/rfc5661#section-11.9
> >>> * 11.9. The Attribute fs_locations
> >>> * An entry in the server array is a UTF-8 string and represents one of a
> >>> * traditional DNS host name, IPv4 address, IPv6 address, or a zero-length
> >>> * string.  An IPv4 or IPv6 address is represented as a universal address
> >>> * (see Section 3.3.9 and [15]), minus the netid, and either with or without
> >>> * the trailing ".p1.p2" suffix that represents the port number.  If the
> >>> * suffix is omitted, then the default port, 2049, SHOULD be assumed.  A
> >>> * zero-length string SHOULD be used to indicate the current address being
> >>> * used for the RPC call.
> >>>
> >>> Does anyone have an example of how the content of fs_locations should
> >>> look like with a custom port number?
> >>
> >> If you keep following the references, you end up with the example in
> >> rfc5665, which gives an example for IPv4:
> >>
> >> https://datatracker.ietf.org/doc/html/rfc5665#section-5.2.3.3
> >
> > So just <address>.<upper-byte-of-port-number>.<lower-byte-of-port-number>?
>
> > How can I test that with the refer= option in /etc/exports? nfsref
> > does not seem to have a ports option...
>
> Neither refer= nor nfsref support alternate ports for exactly the
> same reason: The mountd upcall/downcall, which is how the kernel
> learns of referral target locations, needs to be fixed first. Then
> support for alternate ports can be implemented in both refer= and
> nfsref.

Just turn "hostname" into "hostport", i.e. the "hostname" string in
the mountd protocol gets the port number encoded into it. Problem
done. This is seriously a non-brainer, and can be repeated for autofs,
which does not do port number either,

Ced
-- 
Cedric Blancher <cedric.blancher@gmail.com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur

Re: NFSv4 referrals - custom (non-2049) port numbers in fs_locations?

[Chuck Lever III]

> On Nov 13, 2023, at 5:57 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> 
> On Mon, 13 Nov 2023 at 17:19, Chuck Lever III <chuck.lever@oracle.com> wrote:
>> 
>> 
>>> On Nov 10, 2023, at 2:54 AM, Martin Wege <martin.l.wege@gmail.com> wrote:
>>> 
>>> On Wed, Nov 1, 2023 at 3:42 PM Benjamin Coddington <bcodding@redhat.com> wrote:
>>>> 
>>>> On 1 Nov 2023, at 5:06, Martin Wege wrote:
>>>> 
>>>>> Good morning!
>>>>> 
>>>>> We have questions about NFSv4 referrals:
>>>>> 1. Is there a way to test them in Debian Linux?
>>>>> 
>>>>> 2. How does a fs_locations attribute look like when a nonstandard port
>>>>> like 6666 is used?
>>>>> RFC5661 says this:
>>>>> 
>>>>> * http://tools.ietf.org/html/rfc5661#section-11.9
>>>>> * 11.9. The Attribute fs_locations
>>>>> * An entry in the server array is a UTF-8 string and represents one of a
>>>>> * traditional DNS host name, IPv4 address, IPv6 address, or a zero-length
>>>>> * string.  An IPv4 or IPv6 address is represented as a universal address
>>>>> * (see Section 3.3.9 and [15]), minus the netid, and either with or without
>>>>> * the trailing ".p1.p2" suffix that represents the port number.  If the
>>>>> * suffix is omitted, then the default port, 2049, SHOULD be assumed.  A
>>>>> * zero-length string SHOULD be used to indicate the current address being
>>>>> * used for the RPC call.
>>>>> 
>>>>> Does anyone have an example of how the content of fs_locations should
>>>>> look like with a custom port number?
>>>> 
>>>> If you keep following the references, you end up with the example in
>>>> rfc5665, which gives an example for IPv4:
>>>> 
>>>> https://datatracker.ietf.org/doc/html/rfc5665#section-5.2.3.3
>>> 
>>> So just <address>.<upper-byte-of-port-number>.<lower-byte-of-port-number>?
>> 
>>> How can I test that with the refer= option in /etc/exports? nfsref
>>> does not seem to have a ports option...
>> 
>> Neither refer= nor nfsref support alternate ports for exactly the
>> same reason: The mountd upcall/downcall, which is how the kernel
>> learns of referral target locations, needs to be fixed first. Then
>> support for alternate ports can be implemented in both refer= and
>> nfsref.
> 
> Just turn "hostname" into "hostport", i.e. the "hostname" string in
> the mountd protocol gets the port number encoded into it. Problem
> done. This is seriously a non-brainer,

It's not as simple as you think.

As far as I can tell, the mountd upcall/downcall already uses the "@"
character in the refer= value for another purpose. It has the same
problem as using ":" -- it would overload the meaning of the character
and make the refer= value ambiguous and unparsable.

NFSD supports IPv4 addresses, IPv6 addresses, and DNS labels as the
hostname part of each fs_locations entry. DNS label support is one
reason we might have some difficulty using a universal address in
this interface -- the dot notation for the port number bytes looks
no different than the dot notation for subdomains, and we want to
enable alternate ports for both raw IP addresses and DNS labels.

Adding more here means more careful parsing and more to get wrong. In
the kernel we don't have a lot of the beefy parsing library support
that exists in user space. Thus typically user space needs to digest
such arguments and pass the pre-separated tokens to the kernel.


> and can be repeated for autofs,
> which does not do port number either,

autofs is a different subsystem with a different maintainer and a
different set of design goals. Not to mention that it also tries
to stay roughly comparable to the same administrative interfaces
on Solaris and other Unix-like operating systems, since automounter
maps can be shared with non-Linux systems.

Let's focus on the NFS-server specific components for now, please,
and ignore autofs and the mount.nfs command.


--
Chuck Lever

Re: NFSv4 referrals - custom (non-2049) port numbers in fs_locations?

[Martin Wege]

On Tue, Nov 14, 2023 at 3:07 AM Chuck Lever III <chuck.lever@oracle.com> wrote:
>
>
> > On Nov 13, 2023, at 5:57 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> >
> > On Mon, 13 Nov 2023 at 17:19, Chuck Lever III <chuck.lever@oracle.com> wrote:
> >>
> >>
> >>> On Nov 10, 2023, at 2:54 AM, Martin Wege <martin.l.wege@gmail.com> wrote:
> >>>
> >>> On Wed, Nov 1, 2023 at 3:42 PM Benjamin Coddington <bcodding@redhat.com> wrote:
> >>>>
> >>>> On 1 Nov 2023, at 5:06, Martin Wege wrote:
> >>>>
> >>>>> Good morning!
> >>>>>
> >>>>> We have questions about NFSv4 referrals:
> >>>>> 1. Is there a way to test them in Debian Linux?
> >>>>>
> >>>>> 2. How does a fs_locations attribute look like when a nonstandard port
> >>>>> like 6666 is used?
> >>>>> RFC5661 says this:
> >>>>>
> >>>>> * http://tools.ietf.org/html/rfc5661#section-11.9
> >>>>> * 11.9. The Attribute fs_locations
> >>>>> * An entry in the server array is a UTF-8 string and represents one of a
> >>>>> * traditional DNS host name, IPv4 address, IPv6 address, or a zero-length
> >>>>> * string.  An IPv4 or IPv6 address is represented as a universal address
> >>>>> * (see Section 3.3.9 and [15]), minus the netid, and either with or without
> >>>>> * the trailing ".p1.p2" suffix that represents the port number.  If the
> >>>>> * suffix is omitted, then the default port, 2049, SHOULD be assumed.  A
> >>>>> * zero-length string SHOULD be used to indicate the current address being
> >>>>> * used for the RPC call.
> >>>>>
> >>>>> Does anyone have an example of how the content of fs_locations should
> >>>>> look like with a custom port number?
> >>>>
> >>>> If you keep following the references, you end up with the example in
> >>>> rfc5665, which gives an example for IPv4:
> >>>>
> >>>> https://datatracker.ietf.org/doc/html/rfc5665#section-5.2.3.3
> >>>
> >>> So just <address>.<upper-byte-of-port-number>.<lower-byte-of-port-number>?
> >>
> >>> How can I test that with the refer= option in /etc/exports? nfsref
> >>> does not seem to have a ports option...
> >>
> >> Neither refer= nor nfsref support alternate ports for exactly the
> >> same reason: The mountd upcall/downcall, which is how the kernel
> >> learns of referral target locations, needs to be fixed first. Then
> >> support for alternate ports can be implemented in both refer= and
> >> nfsref.
> >
> > Just turn "hostname" into "hostport", i.e. the "hostname" string in
> > the mountd protocol gets the port number encoded into it. Problem
> > done. This is seriously a non-brainer,
>
> It's not as simple as you think.
>
> As far as I can tell, the mountd upcall/downcall already uses the "@"
> character in the refer= value for another purpose. It has the same
> problem as using ":" -- it would overload the meaning of the character
> and make the refer= value ambiguous and unparsable.
>
> NFSD supports IPv4 addresses, IPv6 addresses, and DNS labels as the
> hostname part of each fs_locations entry. DNS label support is one
> reason we might have some difficulty using a universal address in
> this interface -- the dot notation for the port number bytes looks
> no different than the dot notation for subdomains, and we want to
> enable alternate ports for both raw IP addresses and DNS labels.

Which syntax does a DNS label have?

Thanks,
Martin

Re: NFSv4 referrals - custom (non-2049) port numbers in fs_locations?

[Chuck Lever III]

> On Jan 29, 2024, at 6:46 PM, Martin Wege <martin.l.wege@gmail.com> wrote:
> 
> On Tue, Nov 14, 2023 at 3:07 AM Chuck Lever III <chuck.lever@oracle.com> wrote:
>> 
>> 
>>> On Nov 13, 2023, at 5:57 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
>>> 
>>> On Mon, 13 Nov 2023 at 17:19, Chuck Lever III <chuck.lever@oracle.com> wrote:
>>>> 
>>>> 
>>>>> On Nov 10, 2023, at 2:54 AM, Martin Wege <martin.l.wege@gmail.com> wrote:
>>>>> 
>>>>> On Wed, Nov 1, 2023 at 3:42 PM Benjamin Coddington <bcodding@redhat.com> wrote:
>>>>>> 
>>>>>> On 1 Nov 2023, at 5:06, Martin Wege wrote:
>>>>>> 
>>>>>>> Good morning!
>>>>>>> 
>>>>>>> We have questions about NFSv4 referrals:
>>>>>>> 1. Is there a way to test them in Debian Linux?
>>>>>>> 
>>>>>>> 2. How does a fs_locations attribute look like when a nonstandard port
>>>>>>> like 6666 is used?
>>>>>>> RFC5661 says this:
>>>>>>> 
>>>>>>> * http://tools.ietf.org/html/rfc5661#section-11.9
>>>>>>> * 11.9. The Attribute fs_locations
>>>>>>> * An entry in the server array is a UTF-8 string and represents one of a
>>>>>>> * traditional DNS host name, IPv4 address, IPv6 address, or a zero-length
>>>>>>> * string.  An IPv4 or IPv6 address is represented as a universal address
>>>>>>> * (see Section 3.3.9 and [15]), minus the netid, and either with or without
>>>>>>> * the trailing ".p1.p2" suffix that represents the port number.  If the
>>>>>>> * suffix is omitted, then the default port, 2049, SHOULD be assumed.  A
>>>>>>> * zero-length string SHOULD be used to indicate the current address being
>>>>>>> * used for the RPC call.
>>>>>>> 
>>>>>>> Does anyone have an example of how the content of fs_locations should
>>>>>>> look like with a custom port number?
>>>>>> 
>>>>>> If you keep following the references, you end up with the example in
>>>>>> rfc5665, which gives an example for IPv4:
>>>>>> 
>>>>>> https://datatracker.ietf.org/doc/html/rfc5665#section-5.2.3.3
>>>>> 
>>>>> So just <address>.<upper-byte-of-port-number>.<lower-byte-of-port-number>?
>>>> 
>>>>> How can I test that with the refer= option in /etc/exports? nfsref
>>>>> does not seem to have a ports option...
>>>> 
>>>> Neither refer= nor nfsref support alternate ports for exactly the
>>>> same reason: The mountd upcall/downcall, which is how the kernel
>>>> learns of referral target locations, needs to be fixed first. Then
>>>> support for alternate ports can be implemented in both refer= and
>>>> nfsref.
>>> 
>>> Just turn "hostname" into "hostport", i.e. the "hostname" string in
>>> the mountd protocol gets the port number encoded into it. Problem
>>> done. This is seriously a non-brainer,
>> 
>> It's not as simple as you think.
>> 
>> As far as I can tell, the mountd upcall/downcall already uses the "@"
>> character in the refer= value for another purpose. It has the same
>> problem as using ":" -- it would overload the meaning of the character
>> and make the refer= value ambiguous and unparsable.
>> 
>> NFSD supports IPv4 addresses, IPv6 addresses, and DNS labels as the
>> hostname part of each fs_locations entry. DNS label support is one
>> reason we might have some difficulty using a universal address in
>> this interface -- the dot notation for the port number bytes looks
>> no different than the dot notation for subdomains, and we want to
>> enable alternate ports for both raw IP addresses and DNS labels.
> 
> Which syntax does a DNS label have?

I'm not sure what you're asking.

A DNS label is just a hostname (fully-qualified or not). It
never includes a port number.

According to RFC 8881, fs_location4's server field can contain:

 - A DNS label (no port number; 2049 is assumed)

 - An IP presentation address (no port number; 2049 is assumed)

 - a universal address

A universal address is an IP address plus a port number. Therefore
a universal address is the only way an alternate port can be
communicated in an NFSv4 referral.

The RFC is only about wire protocol. It says nothing about the
server's administrative interfaces; those are always up to the
whim of server developers.

In NFSD's case, refer= can specify a DNS label (no port), an IPv4
or IPv6 address (no port), or an IPv4 universal address. This
is because of the punctuation involved with separating the list
of export options, and the punctuation used internally in DNS
labels and IP addresses.

To add support for other combinations would require code changes.


--
Chuck Lever

Re: NFSv4 referrals - custom (non-2049) port numbers in fs_locations?

[Trond Myklebust]

On Mon, 2024-02-05 at 15:13 +0000, Chuck Lever III wrote:
> 
> 
> A DNS label is just a hostname (fully-qualified or not). It
> never includes a port number.
> 
> According to RFC 8881, fs_location4's server field can contain:
> 
>  - A DNS label (no port number; 2049 is assumed)
> 
>  - An IP presentation address (no port number; 2049 is assumed)
> 
>  - a universal address
> 
> A universal address is an IP address plus a port number. Therefore
> a universal address is the only way an alternate port can be
> communicated in an NFSv4 referral.

That's not strictly true. RFC8881 has little to say about how you are
to go about using the DNS hostname provided by fs_locations4. There is
just some non-normative and vague language about using DNS to look up
the addresses.

The use of DNS service records do allow you to look up the full IP
address and port number (i.e. the equivalent of a universal address)
given a fully qualified hostname and a service. While we do not use the
hostname that way in the Linux NFS client today, I see nothing in the
spec that would appear to disallow it at some future time.

> The RFC is only about wire protocol. It says nothing about the
> server's administrative interfaces; those are always up to the
> whim of server developers.
> 
> In NFSD's case, refer= can specify a DNS label (no port), an IPv4
> or IPv6 address (no port), or an IPv4 universal address. This
> is because of the punctuation involved with separating the list
> of export options, and the punctuation used internally in DNS
> labels and IP addresses.
> 
> To add support for other combinations would require code changes.

Agreed, and there would also be the major issue of trying to ensure
backward compatibility if you were to rely on client behaviour that
differs from today.

-- 
Trond Myklebust
Linux NFS client maintainer, Hammerspace
trond.myklebust@hammerspace.com

Re: NFSv4 referrals - custom (non-2049) port numbers in fs_locations?

[Chuck Lever III]

> On Feb 5, 2024, at 11:17 AM, Trond Myklebust <trondmy@hammerspace.com> wrote:
> 
> On Mon, 2024-02-05 at 15:13 +0000, Chuck Lever III wrote:
>> 
>> 
>> A DNS label is just a hostname (fully-qualified or not). It
>> never includes a port number.
>> 
>> According to RFC 8881, fs_location4's server field can contain:
>> 
>>  - A DNS label (no port number; 2049 is assumed)
>> 
>>  - An IP presentation address (no port number; 2049 is assumed)
>> 
>>  - a universal address
>> 
>> A universal address is an IP address plus a port number. Therefore
>> a universal address is the only way an alternate port can be
>> communicated in an NFSv4 referral.
> 
> That's not strictly true. RFC8881 has little to say about how you are
> to go about using the DNS hostname provided by fs_locations4. There is
> just some non-normative and vague language about using DNS to look up
> the addresses.
> 
> The use of DNS service records do allow you to look up the full IP
> address and port number (i.e. the equivalent of a universal address)
> given a fully qualified hostname and a service. While we do not use the
> hostname that way in the Linux NFS client today, I see nothing in the
> spec that would appear to disallow it at some future time.

We absolutely could do that. But first a service name would need to be
reserved, yes?

https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml?search=dns


--
Chuck Lever

Re: NFSv4 referrals - custom (non-2049) port numbers in fs_locations?

[Trond Myklebust]

On Mon, 2024-02-05 at 19:53 +0000, Chuck Lever III wrote:
> 
> 
> > On Feb 5, 2024, at 11:17 AM, Trond Myklebust
> > <trondmy@hammerspace.com> wrote:
> > 
> > On Mon, 2024-02-05 at 15:13 +0000, Chuck Lever III wrote:
> > > 
> > > 
> > > A DNS label is just a hostname (fully-qualified or not). It
> > > never includes a port number.
> > > 
> > > According to RFC 8881, fs_location4's server field can contain:
> > > 
> > >  - A DNS label (no port number; 2049 is assumed)
> > > 
> > >  - An IP presentation address (no port number; 2049 is assumed)
> > > 
> > >  - a universal address
> > > 
> > > A universal address is an IP address plus a port number.
> > > Therefore
> > > a universal address is the only way an alternate port can be
> > > communicated in an NFSv4 referral.
> > 
> > That's not strictly true. RFC8881 has little to say about how you
> > are
> > to go about using the DNS hostname provided by fs_locations4. There
> > is
> > just some non-normative and vague language about using DNS to look
> > up
> > the addresses.
> > 
> > The use of DNS service records do allow you to look up the full IP
> > address and port number (i.e. the equivalent of a universal
> > address)
> > given a fully qualified hostname and a service. While we do not use
> > the
> > hostname that way in the Linux NFS client today, I see nothing in
> > the
> > spec that would appear to disallow it at some future time.
> 
> We absolutely could do that. But first a service name would need to
> be
> reserved, yes?
> 
> https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml?search=dns
> 

What's wrong with the one that is already assigned? I'm talking about:

nfs                2049        tcp    Network File System - Sun    [Brent_Callaghan]                                     [Brent_Callaghan]                                                                                                                                        Defined TXT keys: path=<path to mount point>
                                      Microsystems
nfs                2049        udp    Network File System - Sun    [Brent_Callaghan]                                     [Brent_Callaghan]                                                                                                                                        Defined TXT keys: path=<path to mount point>
                                      Microsystems
nfs                2049       sctp    Network File System                                                                                                                                                    [RFC5665]                                                            Defined TXT keys: path=<path to mount point>

-- 
Trond Myklebust
Linux NFS client maintainer, Hammerspace
trond.myklebust@hammerspace.com

Re: NFSv4 referrals - custom (non-2049) port numbers in fs_locations?

[Chuck Lever]

On Mon, Feb 05, 2024 at 08:34:46PM +0000, Trond Myklebust wrote:
> On Mon, 2024-02-05 at 19:53 +0000, Chuck Lever III wrote:
> > 
> > 
> > > On Feb 5, 2024, at 11:17 AM, Trond Myklebust
> > > <trondmy@hammerspace.com> wrote:
> > > 
> > > On Mon, 2024-02-05 at 15:13 +0000, Chuck Lever III wrote:
> > > > 
> > > > 
> > > > A DNS label is just a hostname (fully-qualified or not). It
> > > > never includes a port number.
> > > > 
> > > > According to RFC 8881, fs_location4's server field can contain:
> > > > 
> > > >  - A DNS label (no port number; 2049 is assumed)
> > > > 
> > > >  - An IP presentation address (no port number; 2049 is assumed)
> > > > 
> > > >  - a universal address
> > > > 
> > > > A universal address is an IP address plus a port number.
> > > > Therefore
> > > > a universal address is the only way an alternate port can be
> > > > communicated in an NFSv4 referral.
> > > 
> > > That's not strictly true. RFC8881 has little to say about how you
> > > are
> > > to go about using the DNS hostname provided by fs_locations4. There
> > > is
> > > just some non-normative and vague language about using DNS to look
> > > up
> > > the addresses.
> > > 
> > > The use of DNS service records do allow you to look up the full IP
> > > address and port number (i.e. the equivalent of a universal
> > > address)
> > > given a fully qualified hostname and a service. While we do not use
> > > the
> > > hostname that way in the Linux NFS client today, I see nothing in
> > > the
> > > spec that would appear to disallow it at some future time.
> > 
> > We absolutely could do that. But first a service name would need to
> > be
> > reserved, yes?
> > 
> > https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml?search=dns
> > 
> 
> What's wrong with the one that is already assigned? I'm talking about:
> 
> nfs                2049        tcp    Network File System - Sun    [Brent_Callaghan]                                     [Brent_Callaghan]                                                                                                                                        Defined TXT keys: path=<path to mount point>
>                                       Microsystems
> nfs                2049        udp    Network File System - Sun    [Brent_Callaghan]                                     [Brent_Callaghan]                                                                                                                                        Defined TXT keys: path=<path to mount point>
>                                       Microsystems
> nfs                2049       sctp    Network File System                                                                                                                                                    [RFC5665]                                                            Defined TXT keys: path=<path to mount point>


For reference:

https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml?search=2049

My search terms excluded these, so I missed them.

I don't see the IANA action request text in RFC 5665. These must
have been specified and requested in some other document, or that
action request was removed before RFC 5665 was published.

Putative example: NFS on port 2050 of example.com:

_nfs._tcp.example.com. 86400 IN SRV 0 5 2050 server1.example.com.

I suppose "mount.nfs" could recognize this mapping as well.

Since the transport protocol is included in the SRV record, I'm not
sure how one would encode "rdma" for NFS on iWARP, which uses a
distinct port (20049).


-- 
Chuck Lever

Re: NFSv4 referrals - custom (non-2049) port numbers in fs_locations?

[Martin Wege]

Good morning!

What is the status here?

Thanks,
Martin

On Mon, Feb 5, 2024 at 8:53 PM Chuck Lever III <chuck.lever@oracle.com> wrote:
>
>
>
> > On Feb 5, 2024, at 11:17 AM, Trond Myklebust <trondmy@hammerspace.com> wrote:
> >
> > On Mon, 2024-02-05 at 15:13 +0000, Chuck Lever III wrote:
> >>
> >>
> >> A DNS label is just a hostname (fully-qualified or not). It
> >> never includes a port number.
> >>
> >> According to RFC 8881, fs_location4's server field can contain:
> >>
> >>  - A DNS label (no port number; 2049 is assumed)
> >>
> >>  - An IP presentation address (no port number; 2049 is assumed)
> >>
> >>  - a universal address
> >>
> >> A universal address is an IP address plus a port number. Therefore
> >> a universal address is the only way an alternate port can be
> >> communicated in an NFSv4 referral.
> >
> > That's not strictly true. RFC8881 has little to say about how you are
> > to go about using the DNS hostname provided by fs_locations4. There is
> > just some non-normative and vague language about using DNS to look up
> > the addresses.
> >
> > The use of DNS service records do allow you to look up the full IP
> > address and port number (i.e. the equivalent of a universal address)
> > given a fully qualified hostname and a service. While we do not use the
> > hostname that way in the Linux NFS client today, I see nothing in the
> > spec that would appear to disallow it at some future time.
>
> We absolutely could do that. But first a service name would need to be
> reserved, yes?
>
> https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml?search=dns
>
>
> --
> Chuck Lever
>
>