[PATCH v2] smb: client: let smbd_post_send_iter() respect the peers max_send_size and transmit all data

[PATCH v2] smb: client: let smbd_post_send_iter() respect the peers max_send_size and transmit all data

[Stefan Metzmacher]

We should not send smbdirect_data_transfer messages larger than
the negotiated max_send_size, typically 1364 bytes, which means
24 bytes of the smbdirect_data_transfer header + 1340 payload bytes.

This happened when doing an SMB2 write with more than 1340 bytes
(which is done inline as it's below rdma_readwrite_threshold).

It means the peer resets the connection.

When testing between cifs.ko and ksmbd.ko something like this
is logged:

client:

    CIFS: VFS: RDMA transport re-established
    siw: got TERMINATE. layer 1, type 2, code 2
    siw: got TERMINATE. layer 1, type 2, code 2
    siw: got TERMINATE. layer 1, type 2, code 2
    siw: got TERMINATE. layer 1, type 2, code 2
    siw: got TERMINATE. layer 1, type 2, code 2
    siw: got TERMINATE. layer 1, type 2, code 2
    siw: got TERMINATE. layer 1, type 2, code 2
    siw: got TERMINATE. layer 1, type 2, code 2
    siw: got TERMINATE. layer 1, type 2, code 2
    CIFS: VFS: \\carina Send error in SessSetup = -11
    smb2_reconnect: 12 callbacks suppressed
    CIFS: VFS: reconnect tcon failed rc = -11
    CIFS: VFS: reconnect tcon failed rc = -11
    CIFS: VFS: reconnect tcon failed rc = -11
    CIFS: VFS: SMB: Zero rsize calculated, using minimum value 65536

and:

    CIFS: VFS: RDMA transport re-established
    siw: got TERMINATE. layer 1, type 2, code 2
    CIFS: VFS: smbd_recv:1894 disconnected
    siw: got TERMINATE. layer 1, type 2, code 2

The ksmbd dmesg is showing things like:

    smb_direct: Recv error. status='local length error (1)' opcode=128
    smb_direct: disconnected
    smb_direct: Recv error. status='local length error (1)' opcode=128
    ksmbd: smb_direct: disconnected
    ksmbd: sock_read failed: -107

As smbd_post_send_iter() limits the transmitted number of bytes
we need loop over it in order to transmit the whole iter.

Cc: Steve French <sfrench@samba.org>
Cc: David Howells <dhowells@redhat.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: linux-cifs@vger.kernel.org
Cc: <stable+noautosel@kernel.org> # sp->max_send_size should be info->max_send_size in backports
Fixes: 3d78fe73fa12 ("cifs: Build the RDMA SGE list directly from an iterator")
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
 fs/smb/client/smbdirect.c | 31 +++++++++++++++++++++++++++----
 1 file changed, 27 insertions(+), 4 deletions(-)

diff --git a/fs/smb/client/smbdirect.c b/fs/smb/client/smbdirect.c
index d00adce37f09..7162c0e3acb1 100644
--- a/fs/smb/client/smbdirect.c
+++ b/fs/smb/client/smbdirect.c
@@ -907,8 +907,10 @@ static int smbd_post_send_iter(struct smbd_connection *info,
 			.local_dma_lkey	= sc->ib.pd->local_dma_lkey,
 			.direction	= DMA_TO_DEVICE,
 		};
+		size_t payload_len = umin(*_remaining_data_length,
+					  sp->max_send_size - sizeof(*packet));
 
-		rc = smb_extract_iter_to_rdma(iter, *_remaining_data_length,
+		rc = smb_extract_iter_to_rdma(iter, payload_len,
 					      &extract);
 		if (rc < 0)
 			goto err_dma;
@@ -1013,6 +1015,27 @@ static int smbd_post_send_empty(struct smbd_connection *info)
 	return smbd_post_send_iter(info, NULL, &remaining_data_length);
 }
 
+static int smbd_post_send_full_iter(struct smbd_connection *info,
+				    struct iov_iter *iter,
+				    int *_remaining_data_length)
+{
+	int rc = 0;
+
+	/*
+	 * smbd_post_send_iter() respects the
+	 * negotiated max_send_size, so we need to
+	 * loop until the full iter is posted
+	 */
+
+	while (iov_iter_count(iter) > 0) {
+		rc = smbd_post_send_iter(info, iter, _remaining_data_length);
+		if (rc < 0)
+			break;
+	}
+
+	return rc;
+}
+
 /*
  * Post a receive request to the transport
  * The remote peer can only send data when a receive request is posted
@@ -1956,14 +1979,14 @@ int smbd_send(struct TCP_Server_Info *server,
 			klen += rqst->rq_iov[i].iov_len;
 		iov_iter_kvec(&iter, ITER_SOURCE, rqst->rq_iov, rqst->rq_nvec, klen);
 
-		rc = smbd_post_send_iter(info, &iter, &remaining_data_length);
+		rc = smbd_post_send_full_iter(info, &iter, &remaining_data_length);
 		if (rc < 0)
 			break;
 
 		if (iov_iter_count(&rqst->rq_iter) > 0) {
 			/* And then the data pages if there are any */
-			rc = smbd_post_send_iter(info, &rqst->rq_iter,
-						 &remaining_data_length);
+			rc = smbd_post_send_full_iter(info, &rqst->rq_iter,
+						      &remaining_data_length);
 			if (rc < 0)
 				break;
 		}
-- 
2.34.1

Re: [PATCH v2] smb: client: let smbd_post_send_iter() respect the peers max_send_size and transmit all data

[David Howells]

Stefan Metzmacher <metze@samba.org> wrote:

> We should not send smbdirect_data_transfer messages larger than
> the negotiated max_send_size, typically 1364 bytes, which means
> 24 bytes of the smbdirect_data_transfer header + 1340 payload bytes.
> 
> This happened when doing an SMB2 write with more than 1340 bytes
> (which is done inline as it's below rdma_readwrite_threshold).
> 
> It means the peer resets the connection.
> 
> When testing between cifs.ko and ksmbd.ko something like this
> is logged:
> 
> client:
> 
>     CIFS: VFS: RDMA transport re-established
>     siw: got TERMINATE. layer 1, type 2, code 2
>     siw: got TERMINATE. layer 1, type 2, code 2
>     siw: got TERMINATE. layer 1, type 2, code 2
>     siw: got TERMINATE. layer 1, type 2, code 2
>     siw: got TERMINATE. layer 1, type 2, code 2
>     siw: got TERMINATE. layer 1, type 2, code 2
>     siw: got TERMINATE. layer 1, type 2, code 2
>     siw: got TERMINATE. layer 1, type 2, code 2
>     siw: got TERMINATE. layer 1, type 2, code 2
>     CIFS: VFS: \\carina Send error in SessSetup = -11
>     smb2_reconnect: 12 callbacks suppressed
>     CIFS: VFS: reconnect tcon failed rc = -11
>     CIFS: VFS: reconnect tcon failed rc = -11
>     CIFS: VFS: reconnect tcon failed rc = -11
>     CIFS: VFS: SMB: Zero rsize calculated, using minimum value 65536
> 
> and:
> 
>     CIFS: VFS: RDMA transport re-established
>     siw: got TERMINATE. layer 1, type 2, code 2
>     CIFS: VFS: smbd_recv:1894 disconnected
>     siw: got TERMINATE. layer 1, type 2, code 2
> 
> The ksmbd dmesg is showing things like:
> 
>     smb_direct: Recv error. status='local length error (1)' opcode=128
>     smb_direct: disconnected
>     smb_direct: Recv error. status='local length error (1)' opcode=128
>     ksmbd: smb_direct: disconnected
>     ksmbd: sock_read failed: -107
> 
> As smbd_post_send_iter() limits the transmitted number of bytes
> we need loop over it in order to transmit the whole iter.
> 
> Cc: Steve French <sfrench@samba.org>
> Cc: David Howells <dhowells@redhat.com>
> Cc: Tom Talpey <tom@talpey.com>
> Cc: linux-cifs@vger.kernel.org
> Cc: <stable+noautosel@kernel.org> # sp->max_send_size should be info->max_send_size in backports
> Fixes: 3d78fe73fa12 ("cifs: Build the RDMA SGE list directly from an iterator")
> Signed-off-by: Stefan Metzmacher <metze@samba.org>

Reviewed-by: David Howells <dhowells@redhat.com>
Tested-by: David Howells <dhowells@redhat.com>

Re: [PATCH v2] smb: client: let smbd_post_send_iter() respect the peers max_send_size and transmit all data

[Steve French]

Added to cifs-2.6.git for-next and updated with rb and tested-by from
David and tested-by from Meetakshi

On Wed, Jun 25, 2025 at 4:03 AM David Howells <dhowells@redhat.com> wrote:
>
> Stefan Metzmacher <metze@samba.org> wrote:
>
> > We should not send smbdirect_data_transfer messages larger than
> > the negotiated max_send_size, typically 1364 bytes, which means
> > 24 bytes of the smbdirect_data_transfer header + 1340 payload bytes.
> >
> > This happened when doing an SMB2 write with more than 1340 bytes
> > (which is done inline as it's below rdma_readwrite_threshold).
> >
> > It means the peer resets the connection.
> >
> > When testing between cifs.ko and ksmbd.ko something like this
> > is logged:
> >
> > client:
> >
> >     CIFS: VFS: RDMA transport re-established
> >     siw: got TERMINATE. layer 1, type 2, code 2
> >     siw: got TERMINATE. layer 1, type 2, code 2
> >     siw: got TERMINATE. layer 1, type 2, code 2
> >     siw: got TERMINATE. layer 1, type 2, code 2
> >     siw: got TERMINATE. layer 1, type 2, code 2
> >     siw: got TERMINATE. layer 1, type 2, code 2
> >     siw: got TERMINATE. layer 1, type 2, code 2
> >     siw: got TERMINATE. layer 1, type 2, code 2
> >     siw: got TERMINATE. layer 1, type 2, code 2
> >     CIFS: VFS: \\carina Send error in SessSetup = -11
> >     smb2_reconnect: 12 callbacks suppressed
> >     CIFS: VFS: reconnect tcon failed rc = -11
> >     CIFS: VFS: reconnect tcon failed rc = -11
> >     CIFS: VFS: reconnect tcon failed rc = -11
> >     CIFS: VFS: SMB: Zero rsize calculated, using minimum value 65536
> >
> > and:
> >
> >     CIFS: VFS: RDMA transport re-established
> >     siw: got TERMINATE. layer 1, type 2, code 2
> >     CIFS: VFS: smbd_recv:1894 disconnected
> >     siw: got TERMINATE. layer 1, type 2, code 2
> >
> > The ksmbd dmesg is showing things like:
> >
> >     smb_direct: Recv error. status='local length error (1)' opcode=128
> >     smb_direct: disconnected
> >     smb_direct: Recv error. status='local length error (1)' opcode=128
> >     ksmbd: smb_direct: disconnected
> >     ksmbd: sock_read failed: -107
> >
> > As smbd_post_send_iter() limits the transmitted number of bytes
> > we need loop over it in order to transmit the whole iter.
> >
> > Cc: Steve French <sfrench@samba.org>
> > Cc: David Howells <dhowells@redhat.com>
> > Cc: Tom Talpey <tom@talpey.com>
> > Cc: linux-cifs@vger.kernel.org
> > Cc: <stable+noautosel@kernel.org> # sp->max_send_size should be info->max_send_size in backports
> > Fixes: 3d78fe73fa12 ("cifs: Build the RDMA SGE list directly from an iterator")
> > Signed-off-by: Stefan Metzmacher <metze@samba.org>
>
> Reviewed-by: David Howells <dhowells@redhat.com>
> Tested-by: David Howells <dhowells@redhat.com>
>
>


-- 
Thanks,

Steve

Re: [PATCH v2] smb: client: let smbd_post_send_iter() respect the peers max_send_size and transmit all data

[Tom Talpey]

LGTM

Reviewed-by: Tom Talpey <tom@talpey.com>


It appears that the "type 2 code 2" terminate packet the client is
receiving is actually incorrect, it should be a "type 2 code 5" to
properly indicate the length error. The siw maintainer suggests I
send a patch, which I'll do.

Metze - the samba.org server is rejecting me as a spammer based on
some sort of IP address blacklist. I don't have any way to fix that,
so I'm sorry if you only see my replies on the list.

Tom.

On 6/25/2025 12:17 PM, Steve French wrote:
> Added to cifs-2.6.git for-next and updated with rb and tested-by from
> David and tested-by from Meetakshi
> 
> On Wed, Jun 25, 2025 at 4:03 AM David Howells <dhowells@redhat.com> wrote:
>>
>> Stefan Metzmacher <metze@samba.org> wrote:
>>
>>> We should not send smbdirect_data_transfer messages larger than
>>> the negotiated max_send_size, typically 1364 bytes, which means
>>> 24 bytes of the smbdirect_data_transfer header + 1340 payload bytes.
>>>
>>> This happened when doing an SMB2 write with more than 1340 bytes
>>> (which is done inline as it's below rdma_readwrite_threshold).
>>>
>>> It means the peer resets the connection.
>>>
>>> When testing between cifs.ko and ksmbd.ko something like this
>>> is logged:
>>>
>>> client:
>>>
>>>      CIFS: VFS: RDMA transport re-established
>>>      siw: got TERMINATE. layer 1, type 2, code 2
>>>      siw: got TERMINATE. layer 1, type 2, code 2
>>>      siw: got TERMINATE. layer 1, type 2, code 2
>>>      siw: got TERMINATE. layer 1, type 2, code 2
>>>      siw: got TERMINATE. layer 1, type 2, code 2
>>>      siw: got TERMINATE. layer 1, type 2, code 2
>>>      siw: got TERMINATE. layer 1, type 2, code 2
>>>      siw: got TERMINATE. layer 1, type 2, code 2
>>>      siw: got TERMINATE. layer 1, type 2, code 2
>>>      CIFS: VFS: \\carina Send error in SessSetup = -11
>>>      smb2_reconnect: 12 callbacks suppressed
>>>      CIFS: VFS: reconnect tcon failed rc = -11
>>>      CIFS: VFS: reconnect tcon failed rc = -11
>>>      CIFS: VFS: reconnect tcon failed rc = -11
>>>      CIFS: VFS: SMB: Zero rsize calculated, using minimum value 65536
>>>
>>> and:
>>>
>>>      CIFS: VFS: RDMA transport re-established
>>>      siw: got TERMINATE. layer 1, type 2, code 2
>>>      CIFS: VFS: smbd_recv:1894 disconnected
>>>      siw: got TERMINATE. layer 1, type 2, code 2
>>>
>>> The ksmbd dmesg is showing things like:
>>>
>>>      smb_direct: Recv error. status='local length error (1)' opcode=128
>>>      smb_direct: disconnected
>>>      smb_direct: Recv error. status='local length error (1)' opcode=128
>>>      ksmbd: smb_direct: disconnected
>>>      ksmbd: sock_read failed: -107
>>>
>>> As smbd_post_send_iter() limits the transmitted number of bytes
>>> we need loop over it in order to transmit the whole iter.
>>>
>>> Cc: Steve French <sfrench@samba.org>
>>> Cc: David Howells <dhowells@redhat.com>
>>> Cc: Tom Talpey <tom@talpey.com>
>>> Cc: linux-cifs@vger.kernel.org
>>> Cc: <stable+noautosel@kernel.org> # sp->max_send_size should be info->max_send_size in backports
>>> Fixes: 3d78fe73fa12 ("cifs: Build the RDMA SGE list directly from an iterator")
>>> Signed-off-by: Stefan Metzmacher <metze@samba.org>
>>
>> Reviewed-by: David Howells <dhowells@redhat.com>
>> Tested-by: David Howells <dhowells@redhat.com>
>>
>>
> 
>