Re: nfs4.1+: workaround for defunct clientaddr?

[Manfred Schwarb <manfred99@gmx.ch>]

Am 03.10.22 um 16:18 schrieb Olga Kornievskaia:
>  Hi Manfred,
>
> What's the purpose of segregating your connections? You don't want
> your backup traffic "interfering" with your regular operation.
> However, the assumption is that between 2 NICs the backup traffic and
> regular traffic could co-exists, correct? In that case why not use
> session trunking?  What you are correctly experiencing is that with
> 4.1+ the 2nd mount discovers that in the 2nd mount it's the same
> server the client is talking to (even if it's thru a different IPs)
> and the client will drop the new connection.
>

I did not know that I'm supposed to want this ;-)  Sounds interesting.
I searched for "replica" and "discovertrunking" but did not find anything.
I'm on Linux 5.14 / nfs-kernel-server-2.1.1. Did you mean "nconnect"
or is my Linux installation simply too old? Probably. 2.1.1 is 5 years old.
I'm on openSUSE Leap 15.4, on idea why they ship such old userspace components.


> For session trunking, you can configure your linux server (I'm
> assuming it is, if not that might be a problem) to support session
> trunking (by using replica=<> option). Then you can also add
> "discovertrunking" option to your mount command and then the client
> will discover the 2 available paths to the server. You wouldn't need 2
> mounts and you'd have both NICs available to serve your combined
> regular and backup traffic. This would be the solution to utilize both
> of the NICs (network paths) you have available between the client and
> the server.
>
>
> On Mon, Oct 3, 2022 at 9:27 AM Manfred Schwarb <manfred99@gmx.ch> wrote:
>>
>> Am 03.10.22 um 14:26 schrieb Jeff Layton:
>>> On Mon, 2022-10-03 at 13:55 +0200, Manfred Schwarb wrote:
>>>> Am 03.10.22 um 13:39 schrieb Jeff Layton:
>>>>> On Sun, 2022-10-02 at 14:35 +0200, Manfred Schwarb wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I have 2 boxes connected with 2 network cards each, one
>>>>>> crossover connection and one connection via LAN.
>>>>>> I want to use the crossover connection for backup,
>>>>>> so I want to be able to select exactly this wire when
>>>>>> doing my NFS backup transfers. Everything interconnected via NFS4.1
>>>>>> and automount.
>>>>>>
>>>>>> Now the thing is, if there is an already existing connection
>>>>>> via LAN, I am not able to select the crossover connection,
>>>>>> there is some session reuse against my will.
>>>>>>
>>>>>> automount config:
>>>>>> /net/192.168.99.1  -fstype=nfs4,nfsvers=4,minorversion=1,clientaddr=192.168.99.100   /  192.168.99.1:/
>>>>>> /net2/192.168.98.1 -fstype=nfs4,nfsvers=4,minorversion=1,clientaddr=192.168.98.100   /  192.168.98.1:/
>>>>>>
>>>>>> mount -l:
>>>>>> 192.168.99.1:/data on /net/192.168.99.1/data type nfs4 (...,clientaddr=192.168.99.100,addr=192.168.99.1)
>>>>>> 192.168.99.1:/data on /net2/192.168.98.1/data type nfs4 (...,clientaddr=192.168.99.100,addr=192.168.99.1)
>>>>>>
>>>>>> As you see, both connections are on "192.168.99.1:/data", and the backup runs
>>>>>> over the same wire as all user communication, which is not desired.
>>>>>> This even happens if I explicitly set some clientaddr= option.
>>>>>>
>>>>>> Now I found two workarounds:
>>>>>> - downgrade to NFS 4.0, clientaddr seems to work with it
>>>>>> - choose different NFS versions, i.e. one connection with
>>>>>>   minorversion=1 and the other with minorversion=2
>>>>>>
>>>>>> Both possibilities seem a bit lame to me.
>>>>>> Are there some other (recommended) variants which do what I want?
>>>>>>
>>>>>> It seems different minor versions result in different "nfs4_unique_id" values,
>>>>>> and therefore no session sharing occurs. But why do different network
>>>>>> interfaces (via explicitly set clientaddr= by user) not result in different
>>>>>> "nfs4_unique_id" values?
>>>>>>
>>>>>> Thanks for any comments and advice,
>>>>>> Manfred
>>>>>
>>>>> That sounds like a bug. We probably need to compare the clientaddr
>>>>> values in nfs_compare_super or nfs_compare_mount_options so that it
>>>>> doesn't match if the clientaddrs are different.
>>>>>
>>>
>>>
>>> Actually, I take it back, clientaddr is specifically advertised as being
>>> for NFSv4.0 only. The workaround for you is "nosharecache", which will
>>> force the mount under /net2 to get a new superblock altogether.
>>
>> But clientaddr is silently accepted on NFS4.1+, and seemingly silently does nothing.
>>
>> The point is, RFC5661 explicitely tells
>> "NFS minor version 1 is deemed superior to NFS minor version 0 with no loss of functionality".
>>
>> So this behavior comes as a surprise.
>>
>>>
>>>>> As a workaround, you can probably mount the second mount with
>>>>> -o nosharecache and get what you want.
>>>>
>>>> Indeed, nosharecache works. But the man page has some scary words for it:
>>>>   "This is considered a data risk".
>>>>
>>>
>>> Yeah, it does sound scary but it's not a huge issue unless you're doing
>>> I/O to the same files at the same time via both mounts. With
>>> "sharecache" (the default) you get better cache coherency in that
>>> situation since the inode and its pagecache are the same.
>>>
>>
>> So I guess this is equivalent to the minorversion=1/minorversion=2 trick
>> cache coherency wise then?
>>
>>
>>> With "nosharecache" you need to be more careful to flush caches, etc. if
>>> you are doing reads and writes to the same files via different paths. If
>>> you need careful coordination there, then you probably want to use file
>>> locking.
>>
>> Thanks for these explanations, it is appreciated!
>> Manfred
>>
>>> --
>>> Jeff Layton <jlayton@kernel.org>
>>