Examples for refer= in /etc/exports?

Examples for refer= in /etc/exports?

[Cedric Blancher]

Good morning!

Does anyone have examples of how to use the refer= option in /etc/exports?

Ced
-- 
Cedric Blancher <cedric.blancher@gmail.com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur

BUG in exports(5), no example for refer= Re: Examples for refer= in /etc/exports?

[Cedric Blancher]

On Thu, 2 Nov 2023 at 10:51, Cedric Blancher <cedric.blancher@gmail.com> wrote:
>
> Good morning!
>
> Does anyone have examples of how to use the refer= option in /etc/exports?

Short answer:
To redirect an NFS mount from local machine /ref/baguette to
/export/home/baguette on host 134.49.22.111 add this to Linux
/etc/exports:

/ref *(no_root_squash,refer=/export/home@134.49.22.111)

This is basically an exports(5) manpage bug, which does not provide
ANY examples.

Plus, /ref must not be a dir controlled by the automounter, or a Linux
6.6 kernel will panic

Ced
-- 
Cedric Blancher <cedric.blancher@gmail.com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur

Re: BUG in exports(5), no example for refer= Re: Examples for refer= in /etc/exports?

[Chuck Lever III]

> On Nov 9, 2023, at 7:05 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> 
> On Thu, 2 Nov 2023 at 10:51, Cedric Blancher <cedric.blancher@gmail.com> wrote:
>> 
>> Good morning!
>> 
>> Does anyone have examples of how to use the refer= option in /etc/exports?
> 
> Short answer:
> To redirect an NFS mount from local machine /ref/baguette to
> /export/home/baguette on host 134.49.22.111 add this to Linux
> /etc/exports:
> 
> /ref *(no_root_squash,refer=/export/home@134.49.22.111)
> 
> This is basically an exports(5) manpage bug, which does not provide
> ANY examples.

That's because setting up a referral this way is deprecated. The
preferred way to do it is to use nfsref(8).


> Plus, /ref must not be a dir controlled by the automounter, or a Linux
> 6.6 kernel will panic

Can you open a bug report at bugzilla.linux-nfs.org <http://bugzilla.linux-nfs.org/> (product "kernel"
component "server") and provide the details of the panic?


--
Chuck Lever

Re: BUG in exports(5), no example for refer= Re: Examples for refer= in /etc/exports?

[Cedric Blancher]

On Fri, 10 Nov 2023 at 01:37, Chuck Lever III <chuck.lever@oracle.com> wrote:
>
>
>
> > On Nov 9, 2023, at 7:05 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> >
> > On Thu, 2 Nov 2023 at 10:51, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> >>
> >> Good morning!
> >>
> >> Does anyone have examples of how to use the refer= option in /etc/exports?
> >
> > Short answer:
> > To redirect an NFS mount from local machine /ref/baguette to
> > /export/home/baguette on host 134.49.22.111 add this to Linux
> > /etc/exports:
> >
> > /ref *(no_root_squash,refer=/export/home@134.49.22.111)
> >
> > This is basically an exports(5) manpage bug, which does not provide
> > ANY examples.
>
> That's because setting up a referral this way is deprecated.

Why did you do that?

> The
> preferred way to do it is to use nfsref(8).

nfsref(8) is not shipped by ANY Linux distribution. The configure
switch in nfs-utils to build it is OFF by default, and the
distribution maintainers refuse to enable it because it can be
"dangerous", or may be "experimental". I got many excuses why they
dont want to enable that damn configure option.

Also, stable and oldstable Debian do not have it enabled either.

Seriously, why was refer= in exports(5) depreciated? There is no
realistic replacement, unless you fix every damn Linux distro first.

PS: Sorry for being moody, but I tried to get nfsref(5) working for a
month on Debian bullseye, and it just didn't work.

>
> > Plus, /ref must not be a dir controlled by the automounter, or a Linux
> > 6.6 kernel will panic
>
> Can you open a bug report at bugzilla.linux-nfs.org <http://bugzilla.linux-nfs.org/> (product "kernel"
> component "server") and provide the details of the panic?

Yes, I can

Ced
-- 
Cedric Blancher <cedric.blancher@gmail.com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur

Re: BUG in exports(5), no example for refer= Re: Examples for refer= in /etc/exports?

[Chuck Lever III]

> On Nov 9, 2023, at 7:47 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> 
> On Fri, 10 Nov 2023 at 01:37, Chuck Lever III <chuck.lever@oracle.com> wrote:
>> 
>>> On Nov 9, 2023, at 7:05 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
>>> 
>>> On Thu, 2 Nov 2023 at 10:51, Cedric Blancher <cedric.blancher@gmail.com> wrote:
>>>> 
>>>> Good morning!
>>>> 
>>>> Does anyone have examples of how to use the refer= option in /etc/exports?
>>> 
>>> Short answer:
>>> To redirect an NFS mount from local machine /ref/baguette to
>>> /export/home/baguette on host 134.49.22.111 add this to Linux
>>> /etc/exports:
>>> 
>>> /ref *(no_root_squash,refer=/export/home@134.49.22.111)
>>> 
>>> This is basically an exports(5) manpage bug, which does not provide
>>> ANY examples.
>> 
>> That's because setting up a referral this way is deprecated.
> 
> Why did you do that?

The nfsref command on Linux matches the same command on Solaris.

nfsref was added years ago to manage junctions, as part of FedFS.
The "refer=" export option can't do that... and FedFS has gone
the way of the dodo.


>> The
>> preferred way to do it is to use nfsref(8).
> 
> nfsref(8) is not shipped by ANY Linux distribution.

It is installed on all of my Fedora systems, and it's on my
only RHEL 8 system. That suggests, though I can't immediately
confirm it, that nfsref is packaged also for CentOS, Oracle
Linux, and any other distro that is based on RedHat Enterprise.


> The configure switch in nfs-utils to build it is OFF by default,

You're talking about

  --enable-junction       enable support for NFS junctions [default=no]

Perhaps that default should change -- it's been part of
nfs-utils for five years now. However, that drags in
dependencies for the xml libraries... maybe someone thinks
that's a hazard?


> and the
> distribution maintainers refuse to enable it because it can be
> "dangerous", or may be "experimental". I got many excuses why they
> dont want to enable that damn configure option.
> 
> Also, stable and oldstable Debian do not have it enabled either.

This is an upstream mailing list. We can't answer for what
Linux distributors decide to enable or not.

I've never heard that it was a dangerous feature. If a
distro maintainer has a concern, they should bring it to
upstream.


> Seriously, why was refer= in exports(5) depreciated? There is no
> realistic replacement, unless you fix every damn Linux distro first.

Again, all the RHEL based distros package nfsref, as far
as I am aware. And as you found, refer= still works. It's
simply not documented.

If your distro has decided not to support referrals, there's
not much we can do about that except gently suggest that you
switch to a distro that properly supports them.


> PS: Sorry for being moody, but I tried to get nfsref(5) working for a
> month on Debian bullseye, and it just didn't work.

Did you report problems or ask for help?

If getting it working is that difficult, perhaps your distro's
nfs-utils maintainer couldn't get it working either.


--
Chuck Lever

Re: BUG in exports(5), no example for refer= Re: Examples for refer= in /etc/exports?

[Cedric Blancher]

On Fri, 10 Nov 2023 at 03:19, Chuck Lever III <chuck.lever@oracle.com> wrote:
>
>
>
> > On Nov 9, 2023, at 7:47 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> >
> > On Fri, 10 Nov 2023 at 01:37, Chuck Lever III <chuck.lever@oracle.com> wrote:
> >>
> >>> On Nov 9, 2023, at 7:05 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> >>>
> >>> On Thu, 2 Nov 2023 at 10:51, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> >>>>
> >>>> Good morning!
> >>>>
> >>>> Does anyone have examples of how to use the refer= option in /etc/exports?
> >>>
> >>> Short answer:
> >>> To redirect an NFS mount from local machine /ref/baguette to
> >>> /export/home/baguette on host 134.49.22.111 add this to Linux
> >>> /etc/exports:
> >>>
> >>> /ref *(no_root_squash,refer=/export/home@134.49.22.111)
> >>>
> >>> This is basically an exports(5) manpage bug, which does not provide
> >>> ANY examples.
> >>
> >> That's because setting up a referral this way is deprecated.
> >
> > Why did you do that?
>
> The nfsref command on Linux matches the same command on Solaris.

That does not answer my question. Why did you declare the refer option
in exports(8) depreciated, where it works, and is a much simpler way
than nfsref. nfsref information has to be maintained in yet another
configuration file, and nfsd on Linux already has TOO MANY of them.
Leave alone the fact that there is no Solaris-style INTRO manpage,
which systematically links all config files for nfsd, and all daemons
like rpc.gssd.

The all-in-one /etc/exports and /etc/exports.d/ sounds like a better solution.

>
> nfsref was added years ago to manage junctions, as part of FedFS.
> The "refer=" export option can't do that... and FedFS has gone
> the way of the dodo.

Can NFSv4 clients create junctions? Or just tools on the server side?

> >> The
> >> preferred way to do it is to use nfsref(8).
> >
> > nfsref(8) is not shipped by ANY Linux distribution.
>
> It is installed on all of my Fedora systems, and it's on my
> only RHEL 8 system. That suggests, though I can't immediately
> confirm it, that nfsref is packaged also for CentOS, Oracle
> Linux, and any other distro that is based on RedHat Enterprise.
>
>
> > The configure switch in nfs-utils to build it is OFF by default,
>
> You're talking about
>
>   --enable-junction       enable support for NFS junctions [default=no]

YES, that nightmare.

Nightmare, as it suddenly makes nfs-common depend on libidn11-dev
libcap-dev libldap-dev libsqlite3-dev libxml2-dev liburiparser-dev
libconfig-dev libdevmapper-dev and other packages. Package maintainer
hell.

> Perhaps that default should change -- it's been part of
> nfs-utils for five years now. However, that drags in
> dependencies for the xml libraries... maybe someone thinks
> that's a hazard?

Adding eight new package dependencies (libidn11-dev  libcap-dev
libldap-dev libsqlite3-dev libxml2-dev liburiparser-dev libconfig-dev
libdevmapper-dev...) is a problem, especially the LDAP one.

> > and the
> > distribution maintainers refuse to enable it because it can be
> > "dangerous", or may be "experimental". I got many excuses why they
> > dont want to enable that damn configure option.
> >
> > Also, stable and oldstable Debian do not have it enabled either.
>
> This is an upstream mailing list. We can't answer for what
> Linux distributors decide to enable or not.
>
> I've never heard that it was a dangerous feature. If a
> distro maintainer has a concern, they should bring it to
> upstream.
>
>
> > Seriously, why was refer= in exports(5) depreciated? There is no
> > realistic replacement, unless you fix every damn Linux distro first.
>
> Again, all the RHEL based distros package nfsref, as far
> as I am aware. And as you found, refer= still works. It's
> simply not documented.
>
> If your distro has decided not to support referrals, there's
> not much we can do about that except gently suggest that you
> switch to a distro that properly supports them.
>
>
> > PS: Sorry for being moody, but I tried to get nfsref(5) working for a
> > month on Debian bullseye, and it just didn't work.
>
> Did you report problems or ask for help?

yes

>
> If getting it working is that difficult, perhaps your distro's
> nfs-utils maintainer couldn't get it working either.

Maybe.

Ced
-- 
Cedric Blancher <cedric.blancher@gmail.com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur

Re: BUG in exports(5), no example for refer= Re: Examples for refer= in /etc/exports?

[Chuck Lever III]

> On Nov 9, 2023, at 9:42 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> 
> On Fri, 10 Nov 2023 at 03:19, Chuck Lever III <chuck.lever@oracle.com> wrote:
>> 
>> 
>> 
>>> On Nov 9, 2023, at 7:47 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
>>> 
>>> On Fri, 10 Nov 2023 at 01:37, Chuck Lever III <chuck.lever@oracle.com> wrote:
>>>> 
>>>>> On Nov 9, 2023, at 7:05 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
>>>>> 
>>>>> On Thu, 2 Nov 2023 at 10:51, Cedric Blancher <cedric.blancher@gmail.com> wrote:
>>>>>> 
>>>>>> Good morning!
>>>>>> 
>>>>>> Does anyone have examples of how to use the refer= option in /etc/exports?
>>>>> 
>>>>> Short answer:
>>>>> To redirect an NFS mount from local machine /ref/baguette to
>>>>> /export/home/baguette on host 134.49.22.111 add this to Linux
>>>>> /etc/exports:
>>>>> 
>>>>> /ref *(no_root_squash,refer=/export/home@134.49.22.111)
>>>>> 
>>>>> This is basically an exports(5) manpage bug, which does not provide
>>>>> ANY examples.
>>>> 
>>>> That's because setting up a referral this way is deprecated.
>>> 
>>> Why did you do that?
>> 
>> The nfsref command on Linux matches the same command on Solaris.
> 
> That does not answer my question. Why did you declare the refer option
> in exports(8) depreciated, where it works, and is a much simpler way
> than nfsref. nfsref information has to be maintained in yet another
> configuration file, and nfsd on Linux already has TOO MANY of them.
> Leave alone the fact that there is no Solaris-style INTRO manpage,
> which systematically links all config files for nfsd, and all daemons
> like rpc.gssd.
> 
> The all-in-one /etc/exports and /etc/exports.d/ sounds like a better solution.

There's no plan to remove refer= and replica=. We just prefer
that folks use nfsref because junctions are a richer, more
capable, mechanism than export options.

You are welcome to contribute examples to exports(5).


>> nfsref was added years ago to manage junctions, as part of FedFS.
>> The "refer=" export option can't do that... and FedFS has gone
>> the way of the dodo.
> 
> Can NFSv4 clients create junctions? Or just tools on the server side?

There is no capability in the NFSv4 protocol to create
or modify junctions. The NFSv4 protocol exposes a
redirection to clients by means of GETATTR(fs_locations) --
clients don't see the junction object itself.

A separate protocol, described in RFC 7533, is used for
managing junctions remotely. There are some tools to do
this on Linux, but there has been no demand for this so
they are not packaged.

In addition, a junction object in a filesystem can be
converted atomically from a directory with entries to
a junction and back. That's difficult to do with lines
in /etc/exports.

This is why we prefer nfsref.


>>> The configure switch in nfs-utils to build it is OFF by default,
>> 
>> You're talking about
>> 
>>  --enable-junction       enable support for NFS junctions [default=no]
> 
> YES, that nightmare.
> 
> Nightmare, as it suddenly makes nfs-common depend on libidn11-dev
> libcap-dev libldap-dev libsqlite3-dev libxml2-dev liburiparser-dev
> libconfig-dev libdevmapper-dev and other packages. Package maintainer
> hell.

Several of those have nothing to do with nfsref. That's
just what it takes to build nfs-utils. It's always been
like that.

Can you say exactly what is pulling in the LDAP
dependency? I recall removing all of the LDAP stuff
when I added nfsref to nfs-utils because we wanted
to avoid adding an LDAP requirement.


--
Chuck Lever

Re: BUG in exports(5), no example for refer= Re: Examples for refer= in /etc/exports?

[Martin Wege]

On Fri, Nov 10, 2023 at 1:37 AM Chuck Lever III <chuck.lever@oracle.com> wrote:
>
>
>
> > On Nov 9, 2023, at 7:05 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> >
> > On Thu, 2 Nov 2023 at 10:51, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> >>
> >> Good morning!
> >>
> >> Does anyone have examples of how to use the refer= option in /etc/exports?
> >
> > Short answer:
> > To redirect an NFS mount from local machine /ref/baguette to
> > /export/home/baguette on host 134.49.22.111 add this to Linux
> > /etc/exports:
> >
> > /ref *(no_root_squash,refer=/export/home@134.49.22.111)
> >
> > This is basically an exports(5) manpage bug, which does not provide
> > ANY examples.
>
> That's because setting up a referral this way is deprecated.

Then its time to un-depreciate it. We're actively using it, for four
sites with 2000+ active accounts each.

> The
> preferred way to do it is to use nfsref(8).

which -a nfsref
File not found.

This is on Debian SID. Same on Suse. Same on Mandriva.

Thanks,
Martin

Re: BUG in exports(5), no example for refer= Re: Examples for refer= in /etc/exports?

[Martin Wege]

On Fri, Nov 10, 2023 at 3:20 AM Chuck Lever III <chuck.lever@oracle.com> wrote:
>
>
>
> > On Nov 9, 2023, at 7:47 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> >
> > On Fri, 10 Nov 2023 at 01:37, Chuck Lever III <chuck.lever@oracle.com> wrote:
> >>
> >>> On Nov 9, 2023, at 7:05 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> >>>
> >>> On Thu, 2 Nov 2023 at 10:51, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> >>>>
> >>>> Good morning!
> >>>>
> >>>> Does anyone have examples of how to use the refer= option in /etc/exports?
> >>>
> >>> Short answer:
> >>> To redirect an NFS mount from local machine /ref/baguette to
> >>> /export/home/baguette on host 134.49.22.111 add this to Linux
> >>> /etc/exports:
> >>>
> >>> /ref *(no_root_squash,refer=/export/home@134.49.22.111)
> >>>
> >>> This is basically an exports(5) manpage bug, which does not provide
> >>> ANY examples.
> >>
> >> That's because setting up a referral this way is deprecated.
> >
> > Why did you do that?
>
> The nfsref command on Linux matches the same command on Solaris.
>
> nfsref was added years ago to manage junctions, as part of FedFS.
> The "refer=" export option can't do that...

Where in the kernel is the code for the refer= option? I'd like to get
some of my students to contribute support for custom NFS ports.

I would seriously suggest keeping it for "simple" use cases. nfsref(8)
has ZERO deployment outside RHEL&RHEL clones

> and FedFS has gone
> the way of the dodo.

Why did that happen? ;(

>
> >> The
> >> preferred way to do it is to use nfsref(8).
> >
> > nfsref(8) is not shipped by ANY Linux distribution.
>
> It is installed on all of my Fedora systems, and it's on my
> only RHEL 8 system. That suggests, though I can't immediately
> confirm it, that nfsref is packaged also for CentOS, Oracle
> Linux, and any other distro that is based on RedHat Enterprise.

That are all RHEL clones...

>
> > The configure switch in nfs-utils to build it is OFF by default,
>
> You're talking about
>
>   --enable-junction       enable support for NFS junctions [default=no]
>
> Perhaps that default should change -- it's been part of
> nfs-utils for five years now. However, that drags in
> dependencies for the xml libraries... maybe someone thinks
> that's a hazard?

I would consider it a hazard when the kernel support code drags in XML.
>
>
> > and the
> > distribution maintainers refuse to enable it because it can be
> > "dangerous", or may be "experimental". I got many excuses why they
> > dont want to enable that damn configure option.
> >
> > Also, stable and oldstable Debian do not have it enabled either.
>
> This is an upstream mailing list. We can't answer for what
> Linux distributors decide to enable or not.
>
> I've never heard that it was a dangerous feature. If a
> distro maintainer has a concern, they should bring it to
> upstream.
>
>
> > Seriously, why was refer= in exports(5) depreciated? There is no
> > realistic replacement, unless you fix every damn Linux distro first.
>
> Again, all the RHEL based distros package nfsref, as far
> as I am aware. And as you found, refer= still works. It's
> simply not documented.

Then please document it. You have real world users for that one, who
would be grateful if you don't pull away the floor under their feet.

>
> If your distro has decided not to support referrals, there's
> not much we can do about that except gently suggest that you
> switch to a distro that properly supports them.

You could turn on --enable-junctions by default. And maybe get rid of
the XML dependencies.

Thanks,
Martin

Re: BUG in exports(5), no example for refer= Re: Examples for refer= in /etc/exports?

[Chuck Lever III]

> On Nov 10, 2023, at 3:30 AM, Martin Wege <martin.l.wege@gmail.com> wrote:
> 
> On Fri, Nov 10, 2023 at 3:20 AM Chuck Lever III <chuck.lever@oracle.com> wrote:
>> 
>>> On Nov 9, 2023, at 7:47 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
>>> 
>>> On Fri, 10 Nov 2023 at 01:37, Chuck Lever III <chuck.lever@oracle.com> wrote:
>>>> 
>>>>> On Nov 9, 2023, at 7:05 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
>>>>> 
>>>>> On Thu, 2 Nov 2023 at 10:51, Cedric Blancher <cedric.blancher@gmail.com> wrote:
>>>>>> 
>>>>>> Good morning!
>>>>>> 
>>>>>> Does anyone have examples of how to use the refer= option in /etc/exports?
>>>>> 
>>>>> Short answer:
>>>>> To redirect an NFS mount from local machine /ref/baguette to
>>>>> /export/home/baguette on host 134.49.22.111 add this to Linux
>>>>> /etc/exports:
>>>>> 
>>>>> /ref *(no_root_squash,refer=/export/home@134.49.22.111)
>>>>> 
>>>>> This is basically an exports(5) manpage bug, which does not provide
>>>>> ANY examples.
>>>> 
>>>> That's because setting up a referral this way is deprecated.
>>> 
>>> Why did you do that?
>> 
>> The nfsref command on Linux matches the same command on Solaris.
>> 
>> nfsref was added years ago to manage junctions, as part of FedFS.
>> The "refer=" export option can't do that...
> 
> Where in the kernel is the code for the refer= option? I'd like to get
> some of my students to contribute support for custom NFS ports.

IIRC supporting ports is one thing we can't do right now because
the mountd downcall interface is text-based, and its parser can
barely handle "refer=", let alone specifying a port.

Our plan is to replace the mountd downcall with a netlink protocol
that Lorenzo is working on, and then it would be very straightforward
to add a port to each target location. But getting to a mature
netlink implementation will take a while.


> I would seriously suggest keeping it for "simple" use cases.

We don't have any specific plan to remove support for the
refer=/replica= export options. Deprecated (not "depreciated")
simply means we are in the process of moving to something better
for everyone. So don't build on the old stuff.

However neither you nor Cedric have given a clear reason why
you can't use nfsref other than "Debian won't build it for
us".


> nfsref(8) has ZERO deployment outside RHEL&RHEL clones

Not anything upstream can control. We can help Debian with
packaging concerns, but they haven't brought any to us.


>> and FedFS has gone
>> the way of the dodo.
> 
> Why did that happen? ;(

Complicated to set up (because true FedFS requires LDAP) and no
interest from the user community. pNFS covers the use cases for
referrals as far as people have needed it to. There has been no
demand for it.

Please do not confuse FedFS with support for referrals and
replicas in the NFSv4 protocol. The latter are not going
anywhere.


>>> The configure switch in nfs-utils to build it is OFF by default,
>> 
>> You're talking about
>> 
>>  --enable-junction       enable support for NFS junctions [default=no]
>> 
>> Perhaps that default should change -- it's been part of
>> nfs-utils for five years now. However, that drags in
>> dependencies for the xml libraries... maybe someone thinks
>> that's a hazard?
> 
> I would consider it a hazard when the kernel support code drags in XML.

Can you back up that opinion with some technical facts as to
why an nfs-utils XML dependency is a problem?


>>> and the
>>> distribution maintainers refuse to enable it because it can be
>>> "dangerous", or may be "experimental". I got many excuses why they
>>> dont want to enable that damn configure option.
>>> 
>>> Also, stable and oldstable Debian do not have it enabled either.
>> 
>> This is an upstream mailing list. We can't answer for what
>> Linux distributors decide to enable or not.
>> 
>> I've never heard that it was a dangerous feature. If a
>> distro maintainer has a concern, they should bring it to
>> upstream.
>> 
>> 
>>> Seriously, why was refer= in exports(5) depreciated? There is no
>>> realistic replacement, unless you fix every damn Linux distro first.
>> 
>> Again, all the RHEL based distros package nfsref, as far
>> as I am aware. And as you found, refer= still works. It's
>> simply not documented.
> 
> Then please document it. You have real world users for that one, who
> would be grateful if you don't pull away the floor under their feet.

Let me underscore this: we're not removing the referral capability
from the kernel. No one's feet are going anywhere.

The administrative interface for this mechanism is going to change
at some point. It's simply taken much longer than anticipated.

And again, you are welcome to contribute patches for the man
page. That's kind of the point of open source. You don't go
around demanding support for yada or updates for such-and-such
man page. You contribute them yourself if they are missing.


>> If your distro has decided not to support referrals, there's
>> not much we can do about that except gently suggest that you
>> switch to a distro that properly supports them.
> 
> You could turn on --enable-junctions by default. And maybe get rid of
> the XML dependencies.

It's not easy to get rid of the XML library dependencies
because the junction information stored in the
trusted.junction.nfs xattr is stored in an XML byte stream.

To replace XML itself, we'd need to either invent a private
serialization format for the junction information, or go with
something else that adds a library dependency, like JSON or
yaml. I don't see how that is a win.


--
Chuck Lever

Re: BUG in exports(5), no example for refer= Re: Examples for refer= in /etc/exports?

[Chuck Lever III]

> On Nov 10, 2023, at 2:11 AM, Martin Wege <martin.l.wege@gmail.com> wrote:
> 
> On Fri, Nov 10, 2023 at 1:37 AM Chuck Lever III <chuck.lever@oracle.com> wrote:
>> 
>>> On Nov 9, 2023, at 7:05 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
>>> 
>>> On Thu, 2 Nov 2023 at 10:51, Cedric Blancher <cedric.blancher@gmail.com> wrote:
>>>> 
>>>> Good morning!
>>>> 
>>>> Does anyone have examples of how to use the refer= option in /etc/exports?
>>> 
>>> Short answer:
>>> To redirect an NFS mount from local machine /ref/baguette to
>>> /export/home/baguette on host 134.49.22.111 add this to Linux
>>> /etc/exports:
>>> 
>>> /ref *(no_root_squash,refer=/export/home@134.49.22.111)
>>> 
>>> This is basically an exports(5) manpage bug, which does not provide
>>> ANY examples.
>> 
>> That's because setting up a referral this way is deprecated.
> 
> Then its time to un-depreciate it.

Not "depreciate". The term is deprecate.


> We're actively using it, for four
> sites with 2000+ active accounts each.

Your admins are using the referral capability in the kernel. The
"refer=" export option is its current administrative interface.
We are replacing that interface, not the referral capability
itself.


>> The preferred way to do it is to use nfsref(8).
> 
> which -a nfsref
> File not found.
> 
> This is on Debian SID. Same on Suse. Same on Mandriva.

Then the right way forward is to get them to package it.

If they have concerns about it, please forward them to us.


--
Chuck Lever

Change "hostname" to "hostport" in text-based mountd downcall Re: BUG in exports(5), no example for refer= Re: Examples for refer= in /etc/exports?

[Cedric Blancher]

On Fri, 10 Nov 2023 at 20:17, Chuck Lever III <chuck.lever@oracle.com> wrote:
>
>
>
> > On Nov 10, 2023, at 3:30 AM, Martin Wege <martin.l.wege@gmail.com> wrote:
> >
> > On Fri, Nov 10, 2023 at 3:20 AM Chuck Lever III <chuck.lever@oracle.com> wrote:
> >>
> >>> On Nov 9, 2023, at 7:47 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> >>>
> >>> On Fri, 10 Nov 2023 at 01:37, Chuck Lever III <chuck.lever@oracle.com> wrote:
> >>>>
> >>>>> On Nov 9, 2023, at 7:05 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> >>>>>
> >>>>> On Thu, 2 Nov 2023 at 10:51, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> >>>>>>
> >>>>>> Good morning!
> >>>>>>
> >>>>>> Does anyone have examples of how to use the refer= option in /etc/exports?
> >>>>>
> >>>>> Short answer:
> >>>>> To redirect an NFS mount from local machine /ref/baguette to
> >>>>> /export/home/baguette on host 134.49.22.111 add this to Linux
> >>>>> /etc/exports:
> >>>>>
> >>>>> /ref *(no_root_squash,refer=/export/home@134.49.22.111)
> >>>>>
> >>>>> This is basically an exports(5) manpage bug, which does not provide
> >>>>> ANY examples.
> >>>>
> >>>> That's because setting up a referral this way is deprecated.
> >>>
> >>> Why did you do that?
> >>
> >> The nfsref command on Linux matches the same command on Solaris.
> >>
> >> nfsref was added years ago to manage junctions, as part of FedFS.
> >> The "refer=" export option can't do that...
> >
> > Where in the kernel is the code for the refer= option? I'd like to get
> > some of my students to contribute support for custom NFS ports.
>
> IIRC supporting ports is one thing we can't do right now because
> the mountd downcall interface is text-based, and its parser can
> barely handle "refer=", let alone specifying a port.

I had a chat this weekend with Roland Mainz (who works on the NFSv4
driver for Windows these days):
...
That is the old mistake to think that "hostname" and "port" are
separate entities. We had this kind of debate at SUN/MPK17 several
times. Host and TCP port are the "location of the server", and they
are inseparable.
...
The RFCs even help out with that one, for example RFC1738 (URL RFC)
defines a "hostport" in Page 18.
And that's what I actually did for ms-nfs41-client: NIH&Institute
Pasteur needed custom TCP server ports, and I implemented this by
changing the communication of nfs41_driver.sys (kernel) to userland
NFSv4 client daemon (nfsd_debug.exe) from "hostname" to "hostport",
and added the port number in the UNC, so Windows always uses (and
remembers!) the port number together with the hostname.
Or easier: I changed "hostname" to "hostport" to embed the port number
in the up-/downcalls for mount requests - and that is what the Linux
people can use too.
...

> Our plan is to replace the mountd downcall with a netlink protocol
> that Lorenzo is working on, and then it would be very straightforward
> to add a port to each target location. But getting to a mature
> netlink implementation will take a while.

Yeah, instead of waiting for NetLink you could implement Roland's
suggestion, and change "hostname" to "hostport" in your test-based
mount protocol, and technically everywhere else, like /proc/mounts and
the /sbin/mount output.
So instead of:
mount -t nfs -o port=4444 10.10.0.10:/backups /var/backups
you could use
mount -t nfs 10.10.0.10@4444:/backups /var/backups

The same applies to refer= - just change from "hostname" to
"hostport", and the text-based mountd downcall can stay the same (e.g.
so "foobarhost" changes to "foobarhost@444" in the mountd download.)

Ced
-- 
Cedric Blancher <cedric.blancher@gmail.com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur

No bugzilla account confirmation email from bugzilla.linux-nfs.org Re: BUG in exports(5), no example for refer= Re: Examples for refer= in /etc/exports?

[Cedric Blancher]

On Fri, 10 Nov 2023 at 01:37, Chuck Lever III <chuck.lever@oracle.com> wrote:
>
>
>
> > On Nov 9, 2023, at 7:05 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> >
> > On Thu, 2 Nov 2023 at 10:51, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> >>
> >> Good morning!
> >>
> >> Does anyone have examples of how to use the refer= option in /etc/exports?
> >
> > Short answer:
> > To redirect an NFS mount from local machine /ref/baguette to
> > /export/home/baguette on host 134.49.22.111 add this to Linux
> > /etc/exports:
> >
> > /ref *(no_root_squash,refer=/export/home@134.49.22.111)
> >
> > This is basically an exports(5) manpage bug, which does not provide
> > ANY examples.
>
> That's because setting up a referral this way is deprecated. The
> preferred way to do it is to use nfsref(8).
>
>
> > Plus, /ref must not be a dir controlled by the automounter, or a Linux
> > 6.6 kernel will panic
>
> Can you open a bug report at bugzilla.linux-nfs.org <http://bugzilla.linux-nfs.org/> (product "kernel"
> component "server") and provide the details of the panic?

I tried, but I never got the bugzilla account confirmation email from
bugzilla.linux-nfs.org

Also, the https certificate of bugzilla.linux-nfs.org is somehow
wrong, Chrome does not like it

Ced
-- 
Cedric Blancher <cedric.blancher@gmail.com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur

Re: Change "hostname" to "hostport" in text-based mountd downcall Re: BUG in exports(5), no example for refer= Re: Examples for refer= in /etc/exports?

[Steve Dickson]

Hello Ced,

On 11/12/23 8:01 PM, Cedric Blancher wrote:
> On Fri, 10 Nov 2023 at 20:17, Chuck Lever III <chuck.lever@oracle.com> wrote:
>>
>>
>>
>>> On Nov 10, 2023, at 3:30 AM, Martin Wege <martin.l.wege@gmail.com> wrote:
>>>
>>> On Fri, Nov 10, 2023 at 3:20 AM Chuck Lever III <chuck.lever@oracle.com> wrote:
>>>>
>>>>> On Nov 9, 2023, at 7:47 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
>>>>>
>>>>> On Fri, 10 Nov 2023 at 01:37, Chuck Lever III <chuck.lever@oracle.com> wrote:
>>>>>>
>>>>>>> On Nov 9, 2023, at 7:05 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
>>>>>>>
>>>>>>> On Thu, 2 Nov 2023 at 10:51, Cedric Blancher <cedric.blancher@gmail.com> wrote:
>>>>>>>>
>>>>>>>> Good morning!
>>>>>>>>
>>>>>>>> Does anyone have examples of how to use the refer= option in /etc/exports?
>>>>>>>
>>>>>>> Short answer:
>>>>>>> To redirect an NFS mount from local machine /ref/baguette to
>>>>>>> /export/home/baguette on host 134.49.22.111 add this to Linux
>>>>>>> /etc/exports:
>>>>>>>
>>>>>>> /ref *(no_root_squash,refer=/export/home@134.49.22.111)
>>>>>>>
>>>>>>> This is basically an exports(5) manpage bug, which does not provide
>>>>>>> ANY examples.
>>>>>>
>>>>>> That's because setting up a referral this way is deprecated.
>>>>>
>>>>> Why did you do that?
>>>>
>>>> The nfsref command on Linux matches the same command on Solaris.
>>>>
>>>> nfsref was added years ago to manage junctions, as part of FedFS.
>>>> The "refer=" export option can't do that...
>>>
>>> Where in the kernel is the code for the refer= option? I'd like to get
>>> some of my students to contribute support for custom NFS ports.
>>
>> IIRC supporting ports is one thing we can't do right now because
>> the mountd downcall interface is text-based, and its parser can
>> barely handle "refer=", let alone specifying a port.
> 
> I had a chat this weekend with Roland Mainz (who works on the NFSv4
> driver for Windows these days):
> ...
> That is the old mistake to think that "hostname" and "port" are
> separate entities. We had this kind of debate at SUN/MPK17 several
> times. Host and TCP port are the "location of the server", and they
> are inseparable.
> ...
> The RFCs even help out with that one, for example RFC1738 (URL RFC)
> defines a "hostport" in Page 18.
> And that's what I actually did for ms-nfs41-client: NIH&Institute
> Pasteur needed custom TCP server ports, and I implemented this by
> changing the communication of nfs41_driver.sys (kernel) to userland
> NFSv4 client daemon (nfsd_debug.exe) from "hostname" to "hostport",
> and added the port number in the UNC, so Windows always uses (and
> remembers!) the port number together with the hostname.
> Or easier: I changed "hostname" to "hostport" to embed the port number
> in the up-/downcalls for mount requests - and that is what the Linux
> people can use too.
> ...
> 
>> Our plan is to replace the mountd downcall with a netlink protocol
>> that Lorenzo is working on, and then it would be very straightforward
>> to add a port to each target location. But getting to a mature
>> netlink implementation will take a while.
> 
> Yeah, instead of waiting for NetLink you could implement Roland's
> suggestion, and change "hostname" to "hostport" in your test-based
> mount protocol, and technically everywhere else, like /proc/mounts and
> the /sbin/mount output.
> So instead of:
> mount -t nfs -o port=4444 10.10.0.10:/backups /var/backups
> you could use
> mount -t nfs 10.10.0.10@4444:/backups /var/backups
> 
> The same applies to refer= - just change from "hostname" to
> "hostport", and the text-based mountd downcall can stay the same (e.g.
> so "foobarhost" changes to "foobarhost@444" in the mountd download.)
My suggestion is you and Roland attend the next Bakeathon, this
spring, which will have remote access for testing and
talk about this on one of @2pm talks.

steved.

Re: Change "hostname" to "hostport" in text-based mountd downcall Re: BUG in exports(5), no example for refer= Re: Examples for refer= in /etc/exports?

[Cedric Blancher]

On Mon, 13 Nov 2023 at 16:48, Steve Dickson <steved@redhat.com> wrote:
>
> Hello Ced,
>
> On 11/12/23 8:01 PM, Cedric Blancher wrote:
> > On Fri, 10 Nov 2023 at 20:17, Chuck Lever III <chuck.lever@oracle.com> wrote:
> >>
> >>
> >>
> >>> On Nov 10, 2023, at 3:30 AM, Martin Wege <martin.l.wege@gmail.com> wrote:
> >>>
> >>> On Fri, Nov 10, 2023 at 3:20 AM Chuck Lever III <chuck.lever@oracle.com> wrote:
> >>>>
> >>>>> On Nov 9, 2023, at 7:47 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> >>>>>
> >>>>> On Fri, 10 Nov 2023 at 01:37, Chuck Lever III <chuck.lever@oracle.com> wrote:
> >>>>>>
> >>>>>>> On Nov 9, 2023, at 7:05 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> >>>>>>>
> >>>>>>> On Thu, 2 Nov 2023 at 10:51, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> >>>>>>>>
> >>>>>>>> Good morning!
> >>>>>>>>
> >>>>>>>> Does anyone have examples of how to use the refer= option in /etc/exports?
> >>>>>>>
> >>>>>>> Short answer:
> >>>>>>> To redirect an NFS mount from local machine /ref/baguette to
> >>>>>>> /export/home/baguette on host 134.49.22.111 add this to Linux
> >>>>>>> /etc/exports:
> >>>>>>>
> >>>>>>> /ref *(no_root_squash,refer=/export/home@134.49.22.111)
> >>>>>>>
> >>>>>>> This is basically an exports(5) manpage bug, which does not provide
> >>>>>>> ANY examples.
> >>>>>>
> >>>>>> That's because setting up a referral this way is deprecated.
> >>>>>
> >>>>> Why did you do that?
> >>>>
> >>>> The nfsref command on Linux matches the same command on Solaris.
> >>>>
> >>>> nfsref was added years ago to manage junctions, as part of FedFS.
> >>>> The "refer=" export option can't do that...
> >>>
> >>> Where in the kernel is the code for the refer= option? I'd like to get
> >>> some of my students to contribute support for custom NFS ports.
> >>
> >> IIRC supporting ports is one thing we can't do right now because
> >> the mountd downcall interface is text-based, and its parser can
> >> barely handle "refer=", let alone specifying a port.
> >
> > I had a chat this weekend with Roland Mainz (who works on the NFSv4
> > driver for Windows these days):
> > ...
> > That is the old mistake to think that "hostname" and "port" are
> > separate entities. We had this kind of debate at SUN/MPK17 several
> > times. Host and TCP port are the "location of the server", and they
> > are inseparable.
> > ...
> > The RFCs even help out with that one, for example RFC1738 (URL RFC)
> > defines a "hostport" in Page 18.
> > And that's what I actually did for ms-nfs41-client: NIH&Institute
> > Pasteur needed custom TCP server ports, and I implemented this by
> > changing the communication of nfs41_driver.sys (kernel) to userland
> > NFSv4 client daemon (nfsd_debug.exe) from "hostname" to "hostport",
> > and added the port number in the UNC, so Windows always uses (and
> > remembers!) the port number together with the hostname.
> > Or easier: I changed "hostname" to "hostport" to embed the port number
> > in the up-/downcalls for mount requests - and that is what the Linux
> > people can use too.
> > ...
> >
> >> Our plan is to replace the mountd downcall with a netlink protocol
> >> that Lorenzo is working on, and then it would be very straightforward
> >> to add a port to each target location. But getting to a mature
> >> netlink implementation will take a while.
> >
> > Yeah, instead of waiting for NetLink you could implement Roland's
> > suggestion, and change "hostname" to "hostport" in your test-based
> > mount protocol, and technically everywhere else, like /proc/mounts and
> > the /sbin/mount output.
> > So instead of:
> > mount -t nfs -o port=4444 10.10.0.10:/backups /var/backups
> > you could use
> > mount -t nfs 10.10.0.10@4444:/backups /var/backups
> >
> > The same applies to refer= - just change from "hostname" to
> > "hostport", and the text-based mountd downcall can stay the same (e.g.
> > so "foobarhost" changes to "foobarhost@444" in the mountd download.)
> My suggestion is you and Roland attend the next Bakeathon, this
> spring, which will have remote access for testing and
> talk about this on one of @2pm talks.

Where is the bakeathon hosted? San Francisco?

Ced
-- 
Cedric Blancher <cedric.blancher@gmail.com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur

Re: Change "hostname" to "hostport" in text-based mountd downcall Re: BUG in exports(5), no example for refer= Re: Examples for refer= in /etc/exports?

[Cedric Blancher]

?

On Sun, 19 Nov 2023 at 18:17, Cedric Blancher <cedric.blancher@gmail.com> wrote:
>
> On Mon, 13 Nov 2023 at 16:48, Steve Dickson <steved@redhat.com> wrote:
> >
> > Hello Ced,
> >
> > On 11/12/23 8:01 PM, Cedric Blancher wrote:
> > > On Fri, 10 Nov 2023 at 20:17, Chuck Lever III <chuck.lever@oracle.com> wrote:
> > >>
> > >>
> > >>
> > >>> On Nov 10, 2023, at 3:30 AM, Martin Wege <martin.l.wege@gmail.com> wrote:
> > >>>
> > >>> On Fri, Nov 10, 2023 at 3:20 AM Chuck Lever III <chuck.lever@oracle.com> wrote:
> > >>>>
> > >>>>> On Nov 9, 2023, at 7:47 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> > >>>>>
> > >>>>> On Fri, 10 Nov 2023 at 01:37, Chuck Lever III <chuck.lever@oracle.com> wrote:
> > >>>>>>
> > >>>>>>> On Nov 9, 2023, at 7:05 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> > >>>>>>>
> > >>>>>>> On Thu, 2 Nov 2023 at 10:51, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> > >>>>>>>>
> > >>>>>>>> Good morning!
> > >>>>>>>>
> > >>>>>>>> Does anyone have examples of how to use the refer= option in /etc/exports?
> > >>>>>>>
> > >>>>>>> Short answer:
> > >>>>>>> To redirect an NFS mount from local machine /ref/baguette to
> > >>>>>>> /export/home/baguette on host 134.49.22.111 add this to Linux
> > >>>>>>> /etc/exports:
> > >>>>>>>
> > >>>>>>> /ref *(no_root_squash,refer=/export/home@134.49.22.111)
> > >>>>>>>
> > >>>>>>> This is basically an exports(5) manpage bug, which does not provide
> > >>>>>>> ANY examples.
> > >>>>>>
> > >>>>>> That's because setting up a referral this way is deprecated.
> > >>>>>
> > >>>>> Why did you do that?
> > >>>>
> > >>>> The nfsref command on Linux matches the same command on Solaris.
> > >>>>
> > >>>> nfsref was added years ago to manage junctions, as part of FedFS.
> > >>>> The "refer=" export option can't do that...
> > >>>
> > >>> Where in the kernel is the code for the refer= option? I'd like to get
> > >>> some of my students to contribute support for custom NFS ports.
> > >>
> > >> IIRC supporting ports is one thing we can't do right now because
> > >> the mountd downcall interface is text-based, and its parser can
> > >> barely handle "refer=", let alone specifying a port.
> > >
> > > I had a chat this weekend with Roland Mainz (who works on the NFSv4
> > > driver for Windows these days):
> > > ...
> > > That is the old mistake to think that "hostname" and "port" are
> > > separate entities. We had this kind of debate at SUN/MPK17 several
> > > times. Host and TCP port are the "location of the server", and they
> > > are inseparable.
> > > ...
> > > The RFCs even help out with that one, for example RFC1738 (URL RFC)
> > > defines a "hostport" in Page 18.
> > > And that's what I actually did for ms-nfs41-client: NIH&Institute
> > > Pasteur needed custom TCP server ports, and I implemented this by
> > > changing the communication of nfs41_driver.sys (kernel) to userland
> > > NFSv4 client daemon (nfsd_debug.exe) from "hostname" to "hostport",
> > > and added the port number in the UNC, so Windows always uses (and
> > > remembers!) the port number together with the hostname.
> > > Or easier: I changed "hostname" to "hostport" to embed the port number
> > > in the up-/downcalls for mount requests - and that is what the Linux
> > > people can use too.
> > > ...
> > >
> > >> Our plan is to replace the mountd downcall with a netlink protocol
> > >> that Lorenzo is working on, and then it would be very straightforward
> > >> to add a port to each target location. But getting to a mature
> > >> netlink implementation will take a while.
> > >
> > > Yeah, instead of waiting for NetLink you could implement Roland's
> > > suggestion, and change "hostname" to "hostport" in your test-based
> > > mount protocol, and technically everywhere else, like /proc/mounts and
> > > the /sbin/mount output.
> > > So instead of:
> > > mount -t nfs -o port=4444 10.10.0.10:/backups /var/backups
> > > you could use
> > > mount -t nfs 10.10.0.10@4444:/backups /var/backups
> > >
> > > The same applies to refer= - just change from "hostname" to
> > > "hostport", and the text-based mountd downcall can stay the same (e.g.
> > > so "foobarhost" changes to "foobarhost@444" in the mountd download.)
> > My suggestion is you and Roland attend the next Bakeathon, this
> > spring, which will have remote access for testing and
> > talk about this on one of @2pm talks.
>
> Where is the bakeathon hosted? San Francisco?
>
> Ced
> --
> Cedric Blancher <cedric.blancher@gmail.com>
> [https://plus.google.com/u/0/+CedricBlancher/]
> Institute Pasteur



-- 
Cedric Blancher <cedric.blancher@gmail.com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur

Re: Change "hostname" to "hostport" in text-based mountd downcall Re: BUG in exports(5), no example for refer= Re: Examples for refer= in /etc/exports?

[Cedric Blancher]

On Mon, 13 Nov 2023 at 02:01, Cedric Blancher <cedric.blancher@gmail.com> wrote:
>
> On Fri, 10 Nov 2023 at 20:17, Chuck Lever III <chuck.lever@oracle.com> wrote:
> >
> >
> >
> > > On Nov 10, 2023, at 3:30 AM, Martin Wege <martin.l.wege@gmail.com> wrote:
> > >
> > > On Fri, Nov 10, 2023 at 3:20 AM Chuck Lever III <chuck.lever@oracle.com> wrote:
> > >>
> > >>> On Nov 9, 2023, at 7:47 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> > >>>
> > >>> On Fri, 10 Nov 2023 at 01:37, Chuck Lever III <chuck.lever@oracle.com> wrote:
> > >>>>
> > >>>>> On Nov 9, 2023, at 7:05 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> > >>>>>
> > >>>>> On Thu, 2 Nov 2023 at 10:51, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> > >>>>>>
> > >>>>>> Good morning!
> > >>>>>>
> > >>>>>> Does anyone have examples of how to use the refer= option in /etc/exports?
> > >>>>>
> > >>>>> Short answer:
> > >>>>> To redirect an NFS mount from local machine /ref/baguette to
> > >>>>> /export/home/baguette on host 134.49.22.111 add this to Linux
> > >>>>> /etc/exports:
> > >>>>>
> > >>>>> /ref *(no_root_squash,refer=/export/home@134.49.22.111)
> > >>>>>
> > >>>>> This is basically an exports(5) manpage bug, which does not provide
> > >>>>> ANY examples.
> > >>>>
> > >>>> That's because setting up a referral this way is deprecated.
> > >>>
> > >>> Why did you do that?
> > >>
> > >> The nfsref command on Linux matches the same command on Solaris.
> > >>
> > >> nfsref was added years ago to manage junctions, as part of FedFS.
> > >> The "refer=" export option can't do that...
> > >
> > > Where in the kernel is the code for the refer= option? I'd like to get
> > > some of my students to contribute support for custom NFS ports.
> >
> > IIRC supporting ports is one thing we can't do right now because
> > the mountd downcall interface is text-based, and its parser can
> > barely handle "refer=", let alone specifying a port.
>
> I had a chat this weekend with Roland Mainz (who works on the NFSv4
> driver for Windows these days):
> ...
> That is the old mistake to think that "hostname" and "port" are
> separate entities. We had this kind of debate at SUN/MPK17 several
> times. Host and TCP port are the "location of the server", and they
> are inseparable.
> ...
> The RFCs even help out with that one, for example RFC1738 (URL RFC)
> defines a "hostport" in Page 18.
> And that's what I actually did for ms-nfs41-client: NIH&Institute
> Pasteur needed custom TCP server ports, and I implemented this by
> changing the communication of nfs41_driver.sys (kernel) to userland
> NFSv4 client daemon (nfsd_debug.exe) from "hostname" to "hostport",
> and added the port number in the UNC, so Windows always uses (and
> remembers!) the port number together with the hostname.
> Or easier: I changed "hostname" to "hostport" to embed the port number
> in the up-/downcalls for mount requests - and that is what the Linux
> people can use too.
> ...
>
> > Our plan is to replace the mountd downcall with a netlink protocol
> > that Lorenzo is working on, and then it would be very straightforward
> > to add a port to each target location. But getting to a mature
> > netlink implementation will take a while.
>
> Yeah, instead of waiting for NetLink you could implement Roland's
> suggestion, and change "hostname" to "hostport" in your test-based
> mount protocol, and technically everywhere else, like /proc/mounts and
> the /sbin/mount output.
> So instead of:
> mount -t nfs -o port=4444 10.10.0.10:/backups /var/backups
> you could use
> mount -t nfs 10.10.0.10@4444:/backups /var/backups
>
> The same applies to refer= - just change from "hostname" to
> "hostport", and the text-based mountd downcall can stay the same (e.g.
> so "foobarhost" changes to "foobarhost@444" in the mountd download.)

This issue is still open, and BURNING!

Ced
-- 
Cedric Blancher <cedric.blancher@gmail.com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur

refer= syntax in /etc/exports for custom non-2049 TCP ports ? / was: Re: Change "hostname" to "hostport" in text-based mountd downcall Re: BUG in exports(5), no example for refer= Re: Examples for refer= in /etc/exports?

[Roland Mainz]

On Mon, Nov 13, 2023 at 2:01 AM Cedric Blancher
<cedric.blancher@gmail.com> wrote:
> On Fri, 10 Nov 2023 at 20:17, Chuck Lever III <chuck.lever@oracle.com> wrote:
> > > On Nov 10, 2023, at 3:30 AM, Martin Wege <martin.l.wege@gmail.com> wrote:
> > > On Fri, Nov 10, 2023 at 3:20 AM Chuck Lever III <chuck.lever@oracle.com> wrote:
> > >>> On Nov 9, 2023, at 7:47 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
[snip]
> Yeah, instead of waiting for NetLink you could implement Roland's
> suggestion, and change "hostname" to "hostport" in your test-based
> mount protocol, and technically everywhere else, like /proc/mounts and
> the /sbin/mount output.
> So instead of:
> mount -t nfs -o port=4444 10.10.0.10:/backups /var/backups
> you could use
> mount -t nfs 10.10.0.10@4444:/backups /var/backups
>
> The same applies to refer= - just change from "hostname" to
> "hostport", and the text-based mountd downcall can stay the same (e.g.
> so "foobarhost" changes to "foobarhost@444" in the mountd download.)
[snip]

What would be the correct syntax to specify a custom (non-2049) TCP
port for refer= in /etc/exports ?

Would this work:
---- snip ----
`/ref *(no_root_squash,refer=/export/home@134.49.22.111:32049)
---- snip ----

----

Bye,
Roland
-- 
  __ .  . __
 (o.\ \/ /.o) roland.mainz@nrubsig.org
  \__\/\/__/  MPEG specialist, C&&JAVA&&Sun&&Unix programmer
  /O /==\ O\  TEL +49 641 3992797
 (;O/ \/ \O;)

Re: refer= syntax in /etc/exports for custom non-2049 TCP ports ? / was: Re: Change "hostname" to "hostport" in text-based mountd downcall Re: BUG in exports(5), no example for refer= Re: Examples for refer= in /etc/exports?

[Chuck Lever III]

> On Jan 29, 2024, at 6:44 AM, Roland Mainz <roland.mainz@nrubsig.org> wrote:
> 
> On Mon, Nov 13, 2023 at 2:01 AM Cedric Blancher
> <cedric.blancher@gmail.com> wrote:
>> On Fri, 10 Nov 2023 at 20:17, Chuck Lever III <chuck.lever@oracle.com> wrote:
>>>> On Nov 10, 2023, at 3:30 AM, Martin Wege <martin.l.wege@gmail.com> wrote:
>>>> On Fri, Nov 10, 2023 at 3:20 AM Chuck Lever III <chuck.lever@oracle.com> wrote:
>>>>>> On Nov 9, 2023, at 7:47 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> [snip]
>> Yeah, instead of waiting for NetLink you could implement Roland's
>> suggestion, and change "hostname" to "hostport" in your test-based
>> mount protocol, and technically everywhere else, like /proc/mounts and
>> the /sbin/mount output.
>> So instead of:
>> mount -t nfs -o port=4444 10.10.0.10:/backups /var/backups
>> you could use
>> mount -t nfs 10.10.0.10@4444:/backups /var/backups
>> 
>> The same applies to refer= - just change from "hostname" to
>> "hostport", and the text-based mountd downcall can stay the same (e.g.
>> so "foobarhost" changes to "foobarhost@444" in the mountd download.)
> [snip]
> 
> What would be the correct syntax to specify a custom (non-2049) TCP
> port for refer= in /etc/exports ?
> 
> Would this work:
> ---- snip ----
> `/ref *(no_root_squash,refer=/export/home@134.49.22.111:32049)
> ---- snip ----

Hello Roland -

Although generic NFSv4 referral support has been in NFSD for
many years, NFSD currently does not implement alternate ports
in referrals. It is on our enhancement request list.


--
Chuck Lever

Re: refer= syntax in /etc/exports for custom non-2049 TCP ports ? / was: Re: Change "hostname" to "hostport" in text-based mountd downcall Re: BUG in exports(5), no example for refer= Re: Examples for refer= in /etc/exports?

[Roland Mainz]

On Mon, Jan 29, 2024 at 3:14 PM Chuck Lever III <chuck.lever@oracle.com> wrote:
>
>
>
> > On Jan 29, 2024, at 6:44 AM, Roland Mainz <roland.mainz@nrubsig.org> wrote:
> >
> > On Mon, Nov 13, 2023 at 2:01 AM Cedric Blancher
> > <cedric.blancher@gmail.com> wrote:
> >> On Fri, 10 Nov 2023 at 20:17, Chuck Lever III <chuck.lever@oracle.com> wrote:
> >>>> On Nov 10, 2023, at 3:30 AM, Martin Wege <martin.l.wege@gmail.com> wrote:
> >>>> On Fri, Nov 10, 2023 at 3:20 AM Chuck Lever III <chuck.lever@oracle.com> wrote:
> >>>>>> On Nov 9, 2023, at 7:47 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> > [snip]
> >> Yeah, instead of waiting for NetLink you could implement Roland's
> >> suggestion, and change "hostname" to "hostport" in your test-based
> >> mount protocol, and technically everywhere else, like /proc/mounts and
> >> the /sbin/mount output.
> >> So instead of:
> >> mount -t nfs -o port=4444 10.10.0.10:/backups /var/backups
> >> you could use
> >> mount -t nfs 10.10.0.10@4444:/backups /var/backups
> >>
> >> The same applies to refer= - just change from "hostname" to
> >> "hostport", and the text-based mountd downcall can stay the same (e.g.
> >> so "foobarhost" changes to "foobarhost@444" in the mountd download.)
> > [snip]
> >
> > What would be the correct syntax to specify a custom (non-2049) TCP
> > port for refer= in /etc/exports ?
> >
> > Would this work:
> > ---- snip ----
> > `/ref *(no_root_squash,refer=/export/home@134.49.22.111:32049)
> > ---- snip ----
>
> Hello Roland -
>
> Although generic NFSv4 referral support has been in NFSD for
> many years, NFSD currently does not implement alternate ports
> in referrals.

I know, but the question is about the syntax in /etc/exports. The idea
is to use the same syntax for other NFSv4 server implementations (like
nfs4j) ...

For context: I have a ticket open for the ms-nfs41-client to get the
referral support with custom (non-2049) TCP ports fixed...

> It is on our enhancement request list.

Thanks... :-)

----

Bye,
Roland
-- 
  __ .  . __
 (o.\ \/ /.o) roland.mainz@nrubsig.org
  \__\/\/__/  MPEG specialist, C&&JAVA&&Sun&&Unix programmer
  /O /==\ O\  TEL +49 641 3992797
 (;O/ \/ \O;)

Re: refer= syntax in /etc/exports for custom non-2049 TCP ports ? / was: Re: Change "hostname" to "hostport" in text-based mountd downcall Re: BUG in exports(5), no example for refer= Re: Examples for refer= in /etc/exports?

[Chuck Lever III]

> On Jan 29, 2024, at 10:07 AM, Roland Mainz <roland.mainz@nrubsig.org> wrote:
> 
> On Mon, Jan 29, 2024 at 3:14 PM Chuck Lever III <chuck.lever@oracle.com> wrote:
>> 
>>> On Jan 29, 2024, at 6:44 AM, Roland Mainz <roland.mainz@nrubsig.org> wrote:
>>> 
>>> On Mon, Nov 13, 2023 at 2:01 AM Cedric Blancher
>>> <cedric.blancher@gmail.com> wrote:
>>>> On Fri, 10 Nov 2023 at 20:17, Chuck Lever III <chuck.lever@oracle.com> wrote:
>>>>>> On Nov 10, 2023, at 3:30 AM, Martin Wege <martin.l.wege@gmail.com> wrote:
>>>>>> On Fri, Nov 10, 2023 at 3:20 AM Chuck Lever III <chuck.lever@oracle.com> wrote:
>>>>>>>> On Nov 9, 2023, at 7:47 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
>>> [snip]
>>>> Yeah, instead of waiting for NetLink you could implement Roland's
>>>> suggestion, and change "hostname" to "hostport" in your test-based
>>>> mount protocol, and technically everywhere else, like /proc/mounts and
>>>> the /sbin/mount output.
>>>> So instead of:
>>>> mount -t nfs -o port=4444 10.10.0.10:/backups /var/backups
>>>> you could use
>>>> mount -t nfs 10.10.0.10@4444:/backups /var/backups
>>>> 
>>>> The same applies to refer= - just change from "hostname" to
>>>> "hostport", and the text-based mountd downcall can stay the same (e.g.
>>>> so "foobarhost" changes to "foobarhost@444" in the mountd download.)
>>> [snip]
>>> 
>>> What would be the correct syntax to specify a custom (non-2049) TCP
>>> port for refer= in /etc/exports ?
>>> 
>>> Would this work:
>>> ---- snip ----
>>> `/ref *(no_root_squash,refer=/export/home@134.49.22.111:32049)
>>> ---- snip ----
>> 
>> Hello Roland -
>> 
>> Although generic NFSv4 referral support has been in NFSD for
>> many years, NFSD currently does not implement alternate ports
>> in referrals.
> 
> I know, but the question is about the syntax in /etc/exports. The idea
> is to use the same syntax for other NFSv4 server implementations (like
> nfs4j) ...

We're planning not to support alternate ports via the refer= export
option. Instead, we plan to add the ability to specify an alternate
port via the "nfsref" command.

The refer= export option (that is, using this UI for setting up NFSv4
referrals) has been experimental since it was introduced, and has a
number of limitations that we hope to avoid by using "nfsref" instead.

As an alternative, Solaris, for instance, does not use the /etc/export
interface mechanism at all, preferring instead the "share" and "nfsref"
commands. (though as far as I am aware, Solaris does not implement
support for alternate ports in referrals either).

Solaris has support for reparse points (as does FreeBSD). "nfsref"
is supposed to be a mechanism for editing those, and theoretically
reparse points were supposed to handle more than just NFSv4 referrals.

Unfortunately I was never able to generate a lot of appetite in the
Linux kernel community for implementing reparse points in our file
systems. Our "nfsref" command is therefore somewhat limited.

HTH

--
Chuck Lever

Re: refer= syntax in /etc/exports for custom non-2049 TCP ports ? / was: Re: Change "hostname" to "hostport" in text-based mountd downcall Re: BUG in exports(5), no example for refer= Re: Examples for refer= in /etc/exports?

[Martin Wege]

On Mon, Jan 29, 2024 at 4:51 PM Chuck Lever III <chuck.lever@oracle.com> wrote:
>
>
> > On Jan 29, 2024, at 10:07 AM, Roland Mainz <roland.mainz@nrubsig.org> wrote:
> >
> > On Mon, Jan 29, 2024 at 3:14 PM Chuck Lever III <chuck.lever@oracle.com> wrote:
> >>
> >>> On Jan 29, 2024, at 6:44 AM, Roland Mainz <roland.mainz@nrubsig.org> wrote:
> >>>
> >>> On Mon, Nov 13, 2023 at 2:01 AM Cedric Blancher
> >>> <cedric.blancher@gmail.com> wrote:
> >>>> On Fri, 10 Nov 2023 at 20:17, Chuck Lever III <chuck.lever@oracle.com> wrote:
> >>>>>> On Nov 10, 2023, at 3:30 AM, Martin Wege <martin.l.wege@gmail.com> wrote:
> >>>>>> On Fri, Nov 10, 2023 at 3:20 AM Chuck Lever III <chuck.lever@oracle.com> wrote:
> >>>>>>>> On Nov 9, 2023, at 7:47 PM, Cedric Blancher <cedric.blancher@gmail.com> wrote:
> >>> [snip]
> >>>> Yeah, instead of waiting for NetLink you could implement Roland's
> >>>> suggestion, and change "hostname" to "hostport" in your test-based
> >>>> mount protocol, and technically everywhere else, like /proc/mounts and
> >>>> the /sbin/mount output.
> >>>> So instead of:
> >>>> mount -t nfs -o port=4444 10.10.0.10:/backups /var/backups
> >>>> you could use
> >>>> mount -t nfs 10.10.0.10@4444:/backups /var/backups
> >>>>
> >>>> The same applies to refer= - just change from "hostname" to
> >>>> "hostport", and the text-based mountd downcall can stay the same (e.g.
> >>>> so "foobarhost" changes to "foobarhost@444" in the mountd download.)
> >>> [snip]
> >>>
> >>> What would be the correct syntax to specify a custom (non-2049) TCP
> >>> port for refer= in /etc/exports ?
> >>>
> >>> Would this work:
> >>> ---- snip ----
> >>> `/ref *(no_root_squash,refer=/export/home@134.49.22.111:32049)
> >>> ---- snip ----
> >>
> >> Hello Roland -
> >>
> >> Although generic NFSv4 referral support has been in NFSD for
> >> many years, NFSD currently does not implement alternate ports
> >> in referrals.
> >
> > I know, but the question is about the syntax in /etc/exports. The idea
> > is to use the same syntax for other NFSv4 server implementations (like
> > nfs4j) ...
>
> We're planning not to support alternate ports via the refer= export
> option. Instead, we plan to add the ability to specify an alternate
> port via the "nfsref" command.

But that was NOT Rolands question. The question was which syntax would
work ('Would this work:'), as this is for other NFSv4 server software
such as nfs4j, which tries to be compatible with Linux.

>
> The refer= export option (that is, using this UI for setting up NFSv4
> referrals) has been experimental since it was introduced, and has a
> number of limitations that we hope to avoid by using "nfsref" instead.

The problem I see is that - if Linux nfsref gets fixed to include
custom ports - then it will take at least 3-5 years until this version
is readily available in ALL stable versions of all distributions.
Any improvements or fixes to refer= is available with the next Linux
kernel patch, and I'd be more than happy to pay $$$$ to customer
support to have such a patch backported to a stable branch.

>
> As an alternative, Solaris, for instance, does not use the /etc/export
> interface mechanism at all, preferring instead the "share" and "nfsref"
> commands. (though as far as I am aware, Solaris does not implement
> support for alternate ports in referrals either).
>
> Solaris has support for reparse points (as does FreeBSD). "nfsref"
> is supposed to be a mechanism for editing those, and theoretically
> reparse points were supposed to handle more than just NFSv4 referrals.

How would a reparse point with a custom NFSv4 port look like?

>
> Unfortunately I was never able to generate a lot of appetite in the
> Linux kernel community for implementing reparse points in our file
> systems. Our "nfsref" command is therefore somewhat limited.

Did you document this somewhere?

Thanks,
Martin

Re: refer= syntax in /etc/exports for custom non-2049 TCP ports ? / was: Re: Change "hostname" to "hostport" in text-based mountd downcall Re: BUG in exports(5), no example for refer= Re: Examples for refer= in /etc/exports?

[Chuck Lever III]

> On Jan 29, 2024, at 10:07 AM, Roland Mainz <roland.mainz@nrubsig.org> wrote:
> 
> On Mon, Jan 29, 2024 at 3:14 PM Chuck Lever III <chuck.lever@oracle.com> wrote:
>> 
>>> On Jan 29, 2024, at 6:44 AM, Roland Mainz <roland.mainz@nrubsig.org> wrote:
>>> 
>>> What would be the correct syntax to specify a custom (non-2049) TCP
>>> port for refer= in /etc/exports ?
>>> 
>>> Would this work:
>>> ---- snip ----
>>> `/ref *(no_root_squash,refer=/export/home@134.49.22.111:32049)
>>> ---- snip ----
>> 
>> Hello Roland -
>> 
>> Although generic NFSv4 referral support has been in NFSD for
>> many years, NFSD currently does not implement alternate ports
>> in referrals.
> 
> I know, but the question is about the syntax in /etc/exports. The idea
> is to use the same syntax for other NFSv4 server implementations (like
> nfs4j) ...
> 
> For context: I have a ticket open for the ms-nfs41-client to get the
> referral support with custom (non-2049) TCP ports fixed...

Here's a workaround that might be used with current
versions of NFSD. I'm going to walk through this because
there is some information that could be useful for
ms-nfs41-client.

Referrals are conveyed to NFSv4 clients using the
fs_locations attribute of GETATTR. This attribute conveys
a list of alternate locations a client can use to access
the share it's looking for.

Each item in the list looks like this:

struct fs_location4 {
	utf8str_cis	server<>;
	pathname4	rootpath;
};

The content of the server<> field is either a hostname or
an IP address. RFC 8881 Section 11.16 says:

> An IPv4 or IPv6 address is represented as a universal address (see Section 3.3.9 and [12]), minus the netid, and either with or without the trailing ".p1.p2" suffix that represents the port number. If the suffix is omitted, then the default port, 2049, SHOULD be assumed.


Therefore there is only one way to convey an alternate
port. You can't do it when the server<> field contains a
DNS label; the server has to spell it out with a universal
address. This is part of the NFSv4 protocol, it's not an
NFSD limitation.

(When NFSD supports this properly, it will need to resolve
each hostname to an IP address, when an alternate port is
present, before it builds the fs_location4 list item).

It turns out that our mountd passes the "host" part of the
refer= option to the kernel without much additional
checking, so you can specify a full IPv4 universal address
like so:

/export/referral   *(refer=/export/yada@192.168.1.77 <mailto:refer=/export/yada@192.168.1.77>.8.1)

That is, it's an IPv4 address with a .p1.p2 suffix.

p1 = port DIV 256
p2 = port MOD 256

Here, .8.1 is port 2049.

I've tried some simple testing, and it seems to work with
both refer= and nfsref.

This refer= port syntax probably won't work for IPv6
addresses, and definitely will not for DNS labels. Thus
I consider this a hack and not a long-term solution for
NFSD.

Regarding Roland's question about copying this syntax for
other NFSv4 servers: I recommend that you don't do that.
You'll end up copying its mistakes and limitations. To
amplify my previous explanation: this is unfinished work,
and there are other, better ways to handle referrals.

Moreover, you can't copy what doesn't exist. Since NFSD
currently does not support alternate ports in any kind
of robust way, there's nothing to copy. Each server
implementation will have to invent their own.


--
Chuck Lever