[PATCH net] net/smc: set the lkey of every SGE in smcr_tx_rdma_writes()

[PATCH net] net/smc: set the lkey of every SGE in smcr_tx_rdma_writes()

[Bryam Vargas via B4 Relay]

From: Bryam Vargas <hexlabsecurity@proton.me>

smcr_tx_rdma_writes() sets the lkey of an SGE only for a virtually
contiguous send buffer; for a physically contiguous one it leaves the
lkey the work request slot already holds. With net.smc.smcr_buf_type=2
a send buffer falls back to vzalloc() when alloc_pages() fails, so one
link carries both kinds, and a slot last used by a virtually contiguous
buffer posts the DMA address of a physically contiguous one under the
earlier buffer's MR lkey. That write fails, the link goes down and,
with no link to switch to, the link group is terminated with all of
its connections.

Set the lkey on both branches: the MR's for a virtually contiguous
buffer, local_dma_lkey otherwise.

Fixes: b8d199451c99 ("net/smc: Allow virtually contiguous sndbufs or RMBs for SMC-R")
Cc: stable@vger.kernel.org
Signed-off-by: Bryam Vargas <hexlabsecurity@proton.me>
---
Repro on SMC-Rv2 with soft-RoCE (rxe) over a veth pair between two
namespaces, one link, 64 KiB send buffers; smc.ko built from the
base-commit below, with and without the patch, on a v7.4-rc2 KASAN
kernel. smcr_buf_type=2 is set in init_net, where the rxe devices are.
A test-only knob in smc.ko fails alloc_pages() for the server's first
send buffer, so connection A gets a virtually contiguous send buffer
and connection B a physically contiguous one. The server sends 1 MiB on
A, then 1 MiB on B, in 16 KiB writes.

Unpatched, B's first RDMA write is posted with the lkey of A's MR
(recorded by test code added to smc.ko). None of its 16384 bytes reach
the peer, though send() returned 16384. The write and the CDC message
behind it both complete with IB_WC_WR_FLUSH_ERR. By reading rxe (not
measured), the requester rejects the write, puts the QP in the error
state and flushes the send queue. The link goes down and the link group
is terminated, so A is aborted too: its next send() fails with EPIPE.
Patched, both streams arrive intact and the link stays up. Both builds
also deliver both streams intact with smcr_buf_type=0, with
smcr_buf_type=1, and with smcr_buf_type=2 without the knob. No KASAN
splat in any run.
---
 net/smc/smc_tx.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/net/smc/smc_tx.c b/net/smc/smc_tx.c
index 0f0c6df3bf2c..2c8bcaccad83 100644
--- a/net/smc/smc_tx.c
+++ b/net/smc/smc_tx.c
@@ -393,9 +393,9 @@ static int smcr_tx_rdma_writes(struct smc_connection *conn, size_t len,
 			sge[srcchunk].addr = conn->sndbuf_desc->is_vm ?
 				(virt_addr + src_off) : (base_addr + src_off);
 			sge[srcchunk].length = src_len;
-			if (conn->sndbuf_desc->is_vm)
-				sge[srcchunk].lkey =
-					conn->sndbuf_desc->mr[link->link_idx]->lkey;
+			sge[srcchunk].lkey = conn->sndbuf_desc->is_vm ?
+				conn->sndbuf_desc->mr[link->link_idx]->lkey :
+				link->roce_pd->local_dma_lkey;
 			num_sges++;
 
 			src_off += src_len;

---
base-commit: ead6fa9a0ab194f02a2f36d634b193e9c5f9a1c6
change-id: 20261114-b4-disp-dd99cd41-246f1595153d

Best regards,
--  
Bryam Vargas <hexlabsecurity@proton.me>

Re: [PATCH net] net/smc: set the lkey of every SGE in smcr_tx_rdma_writes()

[sashiko-bot]

> From: Bryam Vargas <hexlabsecurity@proton.me>
> 
> smcr_tx_rdma_writes() sets the lkey of an SGE only for a virtually
> contiguous send buffer; for a physically contiguous one it leaves the
> lkey the work request slot already holds. With net.smc.smcr_buf_type=2
> a send buffer falls back to vzalloc() when alloc_pages() fails, so one
> link carries both kinds, and a slot last used by a virtually contiguous
> buffer posts the DMA address of a physically contiguous one under the
> earlier buffer's MR lkey. That write fails, the link goes down and,
> with no link to switch to, the link group is terminated with all of
> its connections.
> 
> Set the lkey on both branches: the MR's for a virtually contiguous
> buffer, local_dma_lkey otherwise.
> 
> Fixes: b8d199451c99 ("net/smc: Allow virtually contiguous sndbufs or RMBs for SMC-R")
> Cc: stable@vger.kernel.org
> Signed-off-by: Bryam Vargas <hexlabsecurity@proton.me>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261114-b4-disp-dd99cd41-v1-1-009e1c022346@proton.me?part=1