Re: [PATCH 11/12] cifs: Fix problem with encrypted RDMA data read

[Stefan Metzmacher <metze@samba.org>]

Am 31.01.23 um 19:28 schrieb David Howells:
> When the cifs client is talking to the ksmbd server by RDMA and the ksmbd
> server has "smb3 encryption = yes" in its config file, the normal PDU
> stream is encrypted, but the directly-delivered data isn't in the stream
> (and isn't encrypted), but is rather delivered by DDP/RDMA packets (at
> least with IWarp).
> 
> Currently, the direct delivery fails with:
> 
>     buf can not contain only a part of read data
>     WARNING: CPU: 0 PID: 4619 at fs/cifs/smb2ops.c:4731 handle_read_data+0x393/0x405
>     ...
>     RIP: 0010:handle_read_data+0x393/0x405
>     ...
>      smb3_handle_read_data+0x30/0x37
>      receive_encrypted_standard+0x141/0x224
>      cifs_demultiplex_thread+0x21a/0x63b
>      kthread+0xe7/0xef
>      ret_from_fork+0x22/0x30
> 
> The problem apparently stemming from the fact that it's trying to manage
> the decryption, but the data isn't in the smallbuf, the bigbuf or the page
> array).
> 
> This can be fixed simply by inserting an extra case into handle_read_data()
> that checks to see if use_rdma_mr is true, and if it is, just setting
> rdata->got_bytes to the length of data delivered and allowing normal
> continuation.
> 
> This can be seen in an IWarp packet trace.  With the upstream code, it does
> a DDP/RDMA packet, which produces the warning above and then retries,
> retrieving the data inline, spread across several SMBDirect messages that
> get glued together into a single PDU.  With the patch applied, only the
> DDP/RDMA packet is seen.
> 
> Note that this doesn't happen if the server isn't told to encrypt stuff and
> it does also happen with softRoCE.
> 
> Signed-off-by: David Howells <dhowells@redhat.com>
> cc: Steve French <smfrench@gmail.com>
> cc: Tom Talpey <tom@talpey.com>
> cc: Long Li <longli@microsoft.com>
> cc: Namjae Jeon <linkinjeon@kernel.org>
> cc: Stefan Metzmacher <metze@samba.org>
> cc: linux-cifs@vger.kernel.org
> 
> Link: https://lore.kernel.org/r/166855224228.1998592.2212551359609792175.stgit@warthog.procyon.org.uk/ # v1
> ---
>   fs/cifs/smb2ops.c | 3 +++
>   1 file changed, 3 insertions(+)
> 
> diff --git a/fs/cifs/smb2ops.c b/fs/cifs/smb2ops.c
> index cea578a45ed8..73b66ac86abf 100644
> --- a/fs/cifs/smb2ops.c
> +++ b/fs/cifs/smb2ops.c
> @@ -4733,6 +4733,9 @@ handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,
>   		if (length < 0)
>   			return length;
>   		rdata->got_bytes = data_len;
> +	} else if (use_rdma_mr) {
> +		/* The data was delivered directly by RDMA. */
> +		rdata->got_bytes = data_len;

I actually don't understand why this would only be a problem with encryption.

I guess there's much more needed and data_offset should most likely be
ignored completely in the rdma offload case. So I'd guess its just luck
that we don't trigger the below warning/error more often.

>   	} else {
>   		/* read response payload cannot be in both buf and pages */
>   		WARN_ONCE(1, "buf can not contain only a part of read data");
>