[PATCH pynfs] RPLY14: reconnect before retransmitting the call

[PATCH pynfs] RPLY14: reconnect before retransmitting the call

[Jeff Layton]

This test recently started failing with a change to the Linux kernel,
but we believe that the test is wrong. This test replays the same call
with the same XID over the same socket without reconnecting in between.

There is language in RFC1813 that implies that a reconnect is necessary
on a retransmit on a reliable transport like TCP. Change the test to
reconnect the socket before retransmitting.

Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
 nfs4.0/lib/rpc/rpc.py           | 25 +++++++++++++++++++++++++
 nfs4.0/servertests/st_replay.py | 10 ++++++++--
 2 files changed, 33 insertions(+), 2 deletions(-)

diff --git a/nfs4.0/lib/rpc/rpc.py b/nfs4.0/lib/rpc/rpc.py
index 7a80241aa127..c530847a46d9 100644
--- a/nfs4.0/lib/rpc/rpc.py
+++ b/nfs4.0/lib/rpc/rpc.py
@@ -315,6 +315,31 @@ class RPCClient(object):
         out.settimeout(self.timeout)
         self.lock.release()
         return out
+
+    def reconnect_same_port(self):
+        """Drop the current connection and reconnect from the same local
+        port, so the server sees a new transport with an unchanged
+        (address, port) cache key. Used to test duplicate reply cache
+        behavior across connections.
+
+        The old socket is reset (RST) rather than closed gracefully: a
+        graceful close leaves the 4-tuple unusable until the server
+        also closes, while an RST frees the source port immediately.
+        """
+        t = threading.currentThread()
+        self.lock.acquire()
+        old = self._socket[t]
+        saddr = old.getsockname()
+        old.setsockopt(socket.SOL_SOCKET, socket.SO_LINGER,
+                       struct.pack('ii', 1, 0))
+        old.close()
+        out = self._socket[t] = socket.socket(self.af, socket.SOCK_STREAM)
+        out.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+        out.bind(saddr)
+        out.connect((self.remotehost, self.remoteport))
+        out.settimeout(self.timeout)
+        self.lock.release()
+        return out
         
     def send(self, procedure, data=b'', program=None, version=None):
         """Send an RPC call to the server
diff --git a/nfs4.0/servertests/st_replay.py b/nfs4.0/servertests/st_replay.py
index 48e5363432ec..fc8282151909 100644
--- a/nfs4.0/servertests/st_replay.py
+++ b/nfs4.0/servertests/st_replay.py
@@ -4,7 +4,7 @@ from xdrdef.nfs4_type import *
 import nfs_ops
 op = nfs_ops.NFS4ops()
 
-def _replay(env, c, ops, error=NFS4_OK):
+def _replay(env, c, ops, error=NFS4_OK, newconn=False):
     # Can send in an error list, but replays must return same error as orig
     if type(error) is list:
         check_funct = check
@@ -18,6 +18,12 @@ def _replay(env, c, ops, error=NFS4_OK):
     try:
         c.get_new_xid = lambda : xid
 
+        if newconn:
+            # Reconnect from the same source port before replaying:
+            # servers may drop a retransmit that arrives on the same
+            # connection as the original call.
+            c.reconnect_same_port()
+
         # note: this is really cheesy: we happen to know the current
         # Linux server implementation will drop a replay if it comes
         # "too quickly" (<.02 seconds).
@@ -260,4 +266,4 @@ def testMkdirReplay(t, env):
     c = env.c1
     c.init_connection()
     ops = c.go_home() + [op.create(createtype4(NF4DIR), t.word(), {})]
-    _replay(env, c, ops)
+    _replay(env, c, ops, newconn=True)

---
base-commit: cd4701827a8261fedbfb4c6e39029fb9671321a6
change-id: 20260923-rply14-d24b334cd6fa

Best regards,
-- 
Jeff Layton <jlayton@kernel.org>

Re: [PATCH pynfs] RPLY14: reconnect before retransmitting the call

[Chuck Lever]

On Wed, Sep 23, 2026, at 1:43 PM, Jeff Layton wrote:
> This test recently started failing with a change to the Linux kernel,
> but we believe that the test is wrong. This test replays the same call
> with the same XID over the same socket without reconnecting in between.
>
> There is language in RFC1813 that implies that a reconnect is necessary
> on a retransmit on a reliable transport like TCP.

I'd like to see a specific Section number and maybe even a block
quote.

But RFC 1813 is not normative for NFSv4.0. Does RFC 7530 have
anything to say about retransmission of NFS requests on
connection-oriented RPC transports?

One more below.


> Change the test to
> reconnect the socket before retransmitting.
>
> Signed-off-by: Jeff Layton <jlayton@kernel.org>
> ---
>  nfs4.0/lib/rpc/rpc.py           | 25 +++++++++++++++++++++++++
>  nfs4.0/servertests/st_replay.py | 10 ++++++++--
>  2 files changed, 33 insertions(+), 2 deletions(-)
>
> diff --git a/nfs4.0/lib/rpc/rpc.py b/nfs4.0/lib/rpc/rpc.py
> index 7a80241aa127..c530847a46d9 100644
> --- a/nfs4.0/lib/rpc/rpc.py
> +++ b/nfs4.0/lib/rpc/rpc.py
> @@ -315,6 +315,31 @@ class RPCClient(object):
>          out.settimeout(self.timeout)
>          self.lock.release()
>          return out
> +
> +    def reconnect_same_port(self):
> +        """Drop the current connection and reconnect from the same local
> +        port, so the server sees a new transport with an unchanged
> +        (address, port) cache key. Used to test duplicate reply cache
> +        behavior across connections.
> +
> +        The old socket is reset (RST) rather than closed gracefully: a
> +        graceful close leaves the 4-tuple unusable until the server
> +        also closes, while an RST frees the source port immediately.
> +        """
> +        t = threading.currentThread()
> +        self.lock.acquire()
> +        old = self._socket[t]
> +        saddr = old.getsockname()
> +        old.setsockopt(socket.SOL_SOCKET, socket.SO_LINGER,
> +                       struct.pack('ii', 1, 0))
> +        old.close()
> +        out = self._socket[t] = socket.socket(self.af, socket.SOCK_STREAM)
> +        out.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
> +        out.bind(saddr)
> +        out.connect((self.remotehost, self.remoteport))
> +        out.settimeout(self.timeout)
> +        self.lock.release()
> +        return out
>         
>      def send(self, procedure, data=b'', program=None, version=None):
>          """Send an RPC call to the server
> diff --git a/nfs4.0/servertests/st_replay.py b/nfs4.0/servertests/st_replay.py
> index 48e5363432ec..fc8282151909 100644
> --- a/nfs4.0/servertests/st_replay.py
> +++ b/nfs4.0/servertests/st_replay.py
> @@ -4,7 +4,7 @@ from xdrdef.nfs4_type import *
>  import nfs_ops
>  op = nfs_ops.NFS4ops()
> 
> -def _replay(env, c, ops, error=NFS4_OK):
> +def _replay(env, c, ops, error=NFS4_OK, newconn=False):
>      # Can send in an error list, but replays must return same error as orig
>      if type(error) is list:
>          check_funct = check
> @@ -18,6 +18,12 @@ def _replay(env, c, ops, error=NFS4_OK):
>      try:
>          c.get_new_xid = lambda : xid
> 
> +        if newconn:
> +            # Reconnect from the same source port before replaying:
> +            # servers may drop a retransmit that arrives on the same
> +            # connection as the original call.

This comment echoes the documenting comment in front of reconnect_same_port
and can be dropped.


> +            c.reconnect_same_port()
> +
>          # note: this is really cheesy: we happen to know the current
>          # Linux server implementation will drop a replay if it comes
>          # "too quickly" (<.02 seconds).
> @@ -260,4 +266,4 @@ def testMkdirReplay(t, env):
>      c = env.c1
>      c.init_connection()
>      ops = c.go_home() + [op.create(createtype4(NF4DIR), t.word(), {})]
> -    _replay(env, c, ops)
> +    _replay(env, c, ops, newconn=True)
>
> ---
> base-commit: cd4701827a8261fedbfb4c6e39029fb9671321a6
> change-id: 20260923-rply14-d24b334cd6fa
>
> Best regards,
> -- 
> Jeff Layton <jlayton@kernel.org>

-- 
Chuck Lever (Come to NFS bake-a-thon! https://nfsv4bat.org)

Re: [PATCH pynfs] RPLY14: reconnect before retransmitting the call

[NeilBrown]

On Thu, 24 Sep 2026, Jeff Layton wrote:
> This test recently started failing with a change to the Linux kernel,
> but we believe that the test is wrong. This test replays the same call
> with the same XID over the same socket without reconnecting in between.
> 
> There is language in RFC1813 that implies that a reconnect is necessary
> on a retransmit on a reliable transport like TCP. Change the test to
> reconnect the socket before retransmitting.

I don't think this is a sufficient fix.

In the test, the client receives the first reply.  This means the server
will have seen a TCP ACK and will have marked the reply a acked, so it
can be cleaned up at any time.

Currently the cleaning of acked replies is minimal.  We need a per-xprt
LRU for that to be effective, and I believe Chuck has plans for that.

Currently the reply is still in cache even though it has been acked.  So
when a new request comes on the same connection, the request is
deliberately dropped because it cannot possibly be useful.  When
a new request comes on a different connection, the cached reply is found
and returned.
But once we have a per-xprt lru, the cached reply will almost certainly
have been cleaned so it won't be found and the request will be repeated.

To properly test the DRC over TCP with the new tcp-ack detection, we
would need to ensure the client doesn't see the first reply, or at least
doesn't send a tcp-ack.  I can only imagine doing this with an iptables
rule which blocks the ack.  This would allow the client to see the
reply, then abort the connection and retry and confirm the same response.

NeilBrown

> 
> Signed-off-by: Jeff Layton <jlayton@kernel.org>
> ---
>  nfs4.0/lib/rpc/rpc.py           | 25 +++++++++++++++++++++++++
>  nfs4.0/servertests/st_replay.py | 10 ++++++++--
>  2 files changed, 33 insertions(+), 2 deletions(-)
> 
> diff --git a/nfs4.0/lib/rpc/rpc.py b/nfs4.0/lib/rpc/rpc.py
> index 7a80241aa127..c530847a46d9 100644
> --- a/nfs4.0/lib/rpc/rpc.py
> +++ b/nfs4.0/lib/rpc/rpc.py
> @@ -315,6 +315,31 @@ class RPCClient(object):
>          out.settimeout(self.timeout)
>          self.lock.release()
>          return out
> +
> +    def reconnect_same_port(self):
> +        """Drop the current connection and reconnect from the same local
> +        port, so the server sees a new transport with an unchanged
> +        (address, port) cache key. Used to test duplicate reply cache
> +        behavior across connections.
> +
> +        The old socket is reset (RST) rather than closed gracefully: a
> +        graceful close leaves the 4-tuple unusable until the server
> +        also closes, while an RST frees the source port immediately.
> +        """
> +        t = threading.currentThread()
> +        self.lock.acquire()
> +        old = self._socket[t]
> +        saddr = old.getsockname()
> +        old.setsockopt(socket.SOL_SOCKET, socket.SO_LINGER,
> +                       struct.pack('ii', 1, 0))
> +        old.close()
> +        out = self._socket[t] = socket.socket(self.af, socket.SOCK_STREAM)
> +        out.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
> +        out.bind(saddr)
> +        out.connect((self.remotehost, self.remoteport))
> +        out.settimeout(self.timeout)
> +        self.lock.release()
> +        return out
>          
>      def send(self, procedure, data=b'', program=None, version=None):
>          """Send an RPC call to the server
> diff --git a/nfs4.0/servertests/st_replay.py b/nfs4.0/servertests/st_replay.py
> index 48e5363432ec..fc8282151909 100644
> --- a/nfs4.0/servertests/st_replay.py
> +++ b/nfs4.0/servertests/st_replay.py
> @@ -4,7 +4,7 @@ from xdrdef.nfs4_type import *
>  import nfs_ops
>  op = nfs_ops.NFS4ops()
>  
> -def _replay(env, c, ops, error=NFS4_OK):
> +def _replay(env, c, ops, error=NFS4_OK, newconn=False):
>      # Can send in an error list, but replays must return same error as orig
>      if type(error) is list:
>          check_funct = check
> @@ -18,6 +18,12 @@ def _replay(env, c, ops, error=NFS4_OK):
>      try:
>          c.get_new_xid = lambda : xid
>  
> +        if newconn:
> +            # Reconnect from the same source port before replaying:
> +            # servers may drop a retransmit that arrives on the same
> +            # connection as the original call.
> +            c.reconnect_same_port()
> +
>          # note: this is really cheesy: we happen to know the current
>          # Linux server implementation will drop a replay if it comes
>          # "too quickly" (<.02 seconds).
> @@ -260,4 +266,4 @@ def testMkdirReplay(t, env):
>      c = env.c1
>      c.init_connection()
>      ops = c.go_home() + [op.create(createtype4(NF4DIR), t.word(), {})]
> -    _replay(env, c, ops)
> +    _replay(env, c, ops, newconn=True)
> 
> ---
> base-commit: cd4701827a8261fedbfb4c6e39029fb9671321a6
> change-id: 20260923-rply14-d24b334cd6fa
> 
> Best regards,
> -- 
> Jeff Layton <jlayton@kernel.org>
> 
> 

Re: [PATCH pynfs] RPLY14: reconnect before retransmitting the call

[Chuck Lever]

On Thu, Sep 24, 2026, at 5:46 PM, NeilBrown wrote:
> On Thu, 24 Sep 2026, Jeff Layton wrote:
>> This test recently started failing with a change to the Linux kernel,
>> but we believe that the test is wrong. This test replays the same call
>> with the same XID over the same socket without reconnecting in between.
>> 
>> There is language in RFC1813 that implies that a reconnect is necessary
>> on a retransmit on a reliable transport like TCP. Change the test to
>> reconnect the socket before retransmitting.
>
> I don't think this is a sufficient fix.
>
> In the test, the client receives the first reply.  This means the server
> will have seen a TCP ACK and will have marked the reply a acked, so it
> can be cleaned up at any time.
>
> Currently the cleaning of acked replies is minimal.  We need a per-xprt
> LRU for that to be effective, and I believe Chuck has plans for that.
>
> Currently the reply is still in cache even though it has been acked.  So
> when a new request comes on the same connection, the request is
> deliberately dropped because it cannot possibly be useful.  When
> a new request comes on a different connection, the cached reply is found
> and returned.
> But once we have a per-xprt lru, the cached reply will almost certainly
> have been cleaned so it won't be found and the request will be repeated.
>
> To properly test the DRC over TCP with the new tcp-ack detection, we
> would need to ensure the client doesn't see the first reply, or at least
> doesn't send a tcp-ack.  I can only imagine doing this with an iptables
> rule which blocks the ack.  This would allow the client to see the
> reply, then abort the connection and retry and confirm the same response.

Following up on Neil's remarks: pynfs is supposed to check XDR and
spec compliance. DRC behavior is not specified in any RFC, compared
with NFSv4.1 sessions, which is fully specified.

Perhaps the better course of action is to retire RPLY14.


-- 
Chuck Lever (Come to NFS bake-a-thon! https://nfsv4bat.org)