Re: [nfs-utils PATCH] exports: Fix referrals when --enable-junction=no

[Steve Dickson <steved@redhat.com>]

On 12/3/24 7:43 AM, Scott Mayhew wrote:
> On Mon, 02 Dec 2024, Steve Dickson wrote:
> 
>> Hey,
>>
>> On 12/2/24 3:30 PM, Scott Mayhew wrote:
>>> Commit 15dc0bea ("exportd: Moved cache upcalls routines into
>>> libexport.a") caused write_fsloc() to be elided when junction support is
>>> disabled.  Get rid of the bogus #ifdef HAVE_JUNCTION_SUPPORT blocks so
>>> that referrals work again (the only #ifdef HAVE_JUNCTION_SUPPORT should
>>> be around actual junction code).
>> Why not just take the enable_junction config variable
>> out of configure.ac as well?
>>
>> If we want junctions/referrals (which are the same)
>> IMHO... on all the time... Lets not be able to
>> turn them off at all?
>>
>> Point being... if we are going remove 3 of the 4
>> HAVE_JUNCTION_SUPPORT ifdefs... let get ride of
>> all of them.
> 
> Junctions and referrals are _not_ the same.  A junction is one mechanism
> that can be used to generate a referral.  The other way to generate a
> referral is with an export entry, and that is the method that stopped
> working after 15dc0bea.
> 
> When you set up a referral via an export entry you use the refer=
> export option, and the directory must be a mountpoint so that nfsd will
> consult the export cache when the client tries to access the directory.
> 
> For example, I set up the following in /etc/exports:
> 
> /export *(rw,insecure,no_root_squash)
> /export/ref     *(rw,insecure,no_root_squash,refer=/export@192.168.124.66)
> 
> After the client tries to access /export/ref, this is what I see when I
> dump the export cache without my fix:
> 
> [root@rawhide ~]# cat /proc/net/rpc/nfsd.export/content
> #path domain(flags)
> /       *(ro,insecure,no_root_squash,sync,no_wdelay,no_subtree_check,v4root,fsid=0,sec=1)
> /export *(rw,insecure,no_root_squash,sync,wdelay,no_subtree_check,uuid=c4eeda84:ea1a4dcd:a043fdc1:372d7878,sec=1)
> /export/ref     *(rw,insecure,no_root_squash,sync,wdelay,no_subtree_check,uuid=c4eeda84:ea1a4dcd:a043fdc1:372d7878,sec=1)
> 
> Notice there's no refer= option.  So when the client does a LOOKUP of
> /export/ref, the server treats it as a normal directory... it doesn't
> return NFS4ERR_MOVED and so the client doesn't know to query
> fs_locations.
> 
> Here is what the export cache looks like with my fix:
> 
> [root@rawhide ~]# cat /proc/net/rpc/nfsd.export/content
> #path domain(flags)
> /export *(rw,insecure,no_root_squash,sync,wdelay,no_subtree_check,uuid=c4eeda84:ea1a4dcd:a043fdc1:372d7878,sec=1)
> /       *(ro,insecure,no_root_squash,sync,no_wdelay,no_subtree_check,v4root,fsid=0,sec=1)
> /export/ref     *(rw,insecure,no_root_squash,sync,wdelay,no_subtree_check,refer=/export@192.168.124.66,uuid=c4eeda84:ea1a4dcd:a043fdc1:372d7878,sec=1)
> 
> Note the refer= option is present, and the referral works normally.
> 
> A junction is basically a fancy directory that has the user/group/other
> mode bits set to 0 and the sticky bit turned on.  The original mode bits
> are stored in the trusted.junction.mode extended attribute and the
> referral information is stored in the trusted.junction.nfs extended
> attribute.
> 
> Continuing with my previous example, I have this in my /etc/exports
> 
> [root@rawhide ~]# cat /etc/exports
> /export *(rw,insecure,no_root_squash)
> /export/ref     *(rw,insecure,no_root_squash,refer=/export@192.168.124.66)
> 
> Let's add a referral using a junction.
> 
> [root@rawhide ~]# nfsref add /export/junc 192.168.124.66 /export
> Created junction /export/junc
> 
> In this case, /export/junc didn't previously exist, so the nfsref tool
> created it.  If /export/junc did already exist, then the original mode
> would be stored in the trusted.junction.mode and the original contents
> of the directory would be hidden from the client (as well as
> non-privileged users on the server).
> 
> You can look up the referral info using 'nfsref lookup':
> 
> [root@rawhide ~]# nfsref lookup /export/junc
> 192.168.124.66:/export
> 
>          NFS port:       2049
>          Valid for:      0
>          Currency:       -1
>          Flags:          varsub(false)
>          GenFlags:       writable(false), going(false), split(true)
>          TransFlags:     rdma(true)
>          Class:          simul(0), handle(0), fileid(0)
>          Class:          writever(0), change(0), readdir(0)
>          Read:           rank(0), order(0)
>          Write:          rank(0), order(0)
> 
> Or you can just use getfattr if you want to see the raw xml:
> 
> [root@rawhide ~]# getfattr --only-values -d -m trusted.junction.nfs /export/junc
> getfattr: Removing leading '/' from absolute path names
> <?xml version="1.0" encoding="UTF-8"?>
> <junction>
>    <savedmode bits="755"/>
>    <fileset>
>      <location>
>        <host name="192.168.124.66"/>
>        <path>
>          <component>export</component>
>        </path>
>        <currency>-1</currency>
>        <genflags writable="false" going="false" split="true"/>
>        <transflags rdma="true"/>
>        <class simul="0" handle="0" fileid="0" writever="0" change="0" readdir="0"/>
>        <read rank="0" order="0"/>
>        <write rank="0" order="0"/>
>        <flags varsub="false"/>
>        <validfor>0</validfor>
>      </location>
>    </fileset>
> </junction>
> 
> Note that since the /export/junc referral is stored in a junction, it
> doesn't appear in the export info:
> 
> [root@rawhide ~]# exportfs -v
> /export         <world>(sync,wdelay,hide,no_subtree_check,sec=sys,rw,insecure,no_root_squash,no_all_squash)
> /export/ref     <world>(sync,wdelay,hide,no_subtree_check,refer=/export@192.168.124.66,sec=sys,rw,insecure,no_root_squash,no_all_squash)
> 
>  From the client's standpoint, both style referrals work the same:
> 
> root@aion:~# mount 192.168.124.26:/export /mnt/t
> root@aion:~# ls /mnt/t
> junc  ref
> root@aion:~# ls /mnt/t/ref
> file
> root@aion:~# cat /mnt/t/ref/file
> I am on the referral server.
> root@aion:~# grep nfs4 /proc/mounts
> 192.168.124.26:/export /mnt/t nfs4 rw,relatime,vers=4.2,rsize=524288,wsize=524288,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.124.1,local_lock=none,addr=192.168.124.26 0 0
> 192.168.124.66:/export /mnt/t/ref nfs4 rw,relatime,vers=4.2,rsize=262144,wsize=262144,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.124.1,local_lock=none,addr=192.168.124.66 0 0
> root@aion:~# ls /mnt/t/junc
> file
> root@aion:~# cat /mnt/t/junc/file
> I am on the referral server.
> root@aion:~# grep nfs4 /proc/mounts
> 192.168.124.26:/export /mnt/t nfs4 rw,relatime,vers=4.2,rsize=524288,wsize=524288,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.124.1,local_lock=none,addr=192.168.124.26 0 0
> 192.168.124.66:/export /mnt/t/ref nfs4 rw,relatime,vers=4.2,rsize=262144,wsize=262144,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.124.1,local_lock=none,addr=192.168.124.66 0 0
> 192.168.124.66:/export /mnt/t/junc nfs4 rw,relatime,vers=4.2,rsize=262144,wsize=262144,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.124.1,local_lock=none,addr=192.168.124.66 0 0
> 
> 
> So if you want to get rid of that last #ifdef HAVE_JUNCTION_SUPPORT
> then you have 2 options:
> 
> a) get rid of junctions entirely, leaving users with only the old
> (relatively speaking) method for configuring referrals
> b) force all packagers of nfs-utils to pull in the extra dependencies
> needed to support junctions, which is the exact opposite of what the
> Debian folks are requesting.
> 
> Or you can take the patch and we can continue to have both style
> referrals.
Very good explanation... we should put this on linux-nfs.org on
how to uses both junctions and referrals...

I'll let the distros decide how they want to deal with this.

thanks!

steved.

> 
> -Scott
>>
>> steved.
>>>
>>> Fixes: 15dc0bea ("exportd: Moved cache upcalls routines into libexport.a")
>>> Signed-off-by: Scott Mayhew <smayhew@redhat.com>
>>> ---
>>>    support/export/cache.c | 7 -------
>>>    1 file changed, 7 deletions(-)
>>>
>>> diff --git a/support/export/cache.c b/support/export/cache.c
>>> index 6c0a44a3..3a8e57cf 100644
>>> --- a/support/export/cache.c
>>> +++ b/support/export/cache.c
>>> @@ -34,10 +34,7 @@
>>>    #include "pseudoflavors.h"
>>>    #include "xcommon.h"
>>>    #include "reexport.h"
>>> -
>>> -#ifdef HAVE_JUNCTION_SUPPORT
>>>    #include "fsloc.h"
>>> -#endif
>>>    #ifdef USE_BLKID
>>>    #include "blkid/blkid.h"
>>> @@ -999,7 +996,6 @@ static void nfsd_retry_fh(struct delayed *d)
>>>    	*dp = d;
>>>    }
>>> -#ifdef HAVE_JUNCTION_SUPPORT
>>>    static void write_fsloc(char **bp, int *blen, struct exportent *ep)
>>>    {
>>>    	struct servers *servers;
>>> @@ -1022,7 +1018,6 @@ static void write_fsloc(char **bp, int *blen, struct exportent *ep)
>>>    	qword_addint(bp, blen, servers->h_referral);
>>>    	release_replicas(servers);
>>>    }
>>> -#endif
>>>    static void write_secinfo(char **bp, int *blen, struct exportent *ep, int flag_mask, int extra_flag)
>>>    {
>>> @@ -1120,9 +1115,7 @@ static int dump_to_cache(int f, char *buf, int blen, char *domain,
>>>    		qword_addint(&bp, &blen, exp->e_anongid);
>>>    		qword_addint(&bp, &blen, fsidnum);
>>> -#ifdef HAVE_JUNCTION_SUPPORT
>>>    		write_fsloc(&bp, &blen, exp);
>>> -#endif
>>>    		write_secinfo(&bp, &blen, exp, flag_mask, do_fsidnum ? NFSEXP_FSID : 0);
>>>    		if (exp->e_uuid == NULL || different_fs) {
>>>    			char u[16];
>>
>