* [PATCH net-next v5 0/2] net/rds: scope-aware sendmsg connection cache @ 2026-10-17 3:23 Allison Henderson 2026-10-17 3:23 ` [PATCH net-next v5 1/2] net/rds: keep the connected peer's scope id apart from the bound one Allison Henderson 2026-10-17 3:23 ` [PATCH net-next v5 2/2] net/rds: include the scope id in the sendmsg connection cache check Allison Henderson 0 siblings, 2 replies; 6+ messages in thread From: Allison Henderson @ 2026-10-17 3:23 UTC (permalink / raw) To: netdev, linux-rdma, pabeni, edumazet, kuba, horms; +Cc: achender Hi all, This is v5 of the sendmsg connection-cache scope fix (v1 at [1], v2 at [2], v3 at [3], v4 at [4]), now against net-next as Jakub suggested: neither issue has been observed in the field, and both were found by inspection while reworking this cache for the connection-lifetime series. Patch 1 gives the connected peer's link-local scope its own field. rds_connect() used to store it in rs_bound_scope_id, the scope of the socket's own bound address, which a later bind() to a non-link-local address resets to 0 and which a later connect() pollutes for a globally bound socket. A send without a destination and getpeername() take the connected peer's scope from that field; recvmsg() reports a link-local sender with the interface its connection came in on. Patch 2 compares c_dev_if with the request's scope in the cache-hit test, so the cache uses the same key as rds_conn_lookup(). Changes since v4 [4]: - Patch 1: recvmsg() takes a link-local sender's scope from the connection the message arrived on (c_dev_if) and reports 0 for any other sender, instead of the socket's connected or bound scope - a connected socket receives from any peer, so a per-socket scope was wrong for every sender but the connected one and could send a reply out the wrong link (review of v4). getpeername() and recvmsg() now agree for the connected peer. - Retargeted at net-next. Changes since v3 [3]: - Patch 1: a destination-less send falls back to the bound scope for a non-link-local peer, so send() and sendto() to the connected peer compute the same scope; getpeername() and recvmsg() report the connected peer's scope instead of the bound scope. [1] https://lore.kernel.org/netdev/20260921215046.174745-1-achender@kernel.org/ [2] https://lore.kernel.org/netdev/20260927062312.169097-1-achender@kernel.org/ [3] https://lore.kernel.org/netdev/20261003163526.250805-1-achender@kernel.org/ [4] https://lore.kernel.org/netdev/20261008031426.1142344-1-achender@kernel.org/ Thank you, Allison Allison Henderson (2): net/rds: keep the connected peer's scope id apart from the bound one net/rds: include the scope id in the sendmsg connection cache check net/rds/af_rds.c | 15 +++++++++------ net/rds/bind.c | 4 ++-- net/rds/rds.h | 2 ++ net/rds/recv.c | 9 ++++++++- net/rds/send.c | 9 +++++++-- 5 files changed, 28 insertions(+), 11 deletions(-) -- 2.50.1 ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH net-next v5 1/2] net/rds: keep the connected peer's scope id apart from the bound one 2026-10-17 3:23 [PATCH net-next v5 0/2] net/rds: scope-aware sendmsg connection cache Allison Henderson @ 2026-10-17 3:23 ` Allison Henderson 2026-10-17 3:29 ` sashiko-bot 2026-10-18 3:27 ` netdev-bot+sashiko 2026-10-17 3:23 ` [PATCH net-next v5 2/2] net/rds: include the scope id in the sendmsg connection cache check Allison Henderson 1 sibling, 2 replies; 6+ messages in thread From: Allison Henderson @ 2026-10-17 3:23 UTC (permalink / raw) To: netdev, linux-rdma, pabeni, edumazet, kuba, horms; +Cc: achender rds_connect() has nowhere to keep the scope of a link-local peer, so it stores it in rs_bound_scope_id, the scope of the socket's own bound address, for rds_bind() to check a later link-local bind against. That field is then also what a send without a destination uses as the request's scope, what getpeername() and recvmsg() report as the peer's scope, and what a later bind() overwrites: rds_add_bound() stores the bound address's scope unconditionally, which for a non-link-local address is 0. So after connect(fe80::x%ifA) followed by bind(global), a send() with no destination asks for fe80::x with scope 0. rds_conn_lookup() keys on the interface, so that finds or creates a connection with c_dev_if 0, which the TCP transport then tries to connect through sin6_scope_id 0 and tcp_v6_connect() rejects for a link-local peer: the connected socket's data is queued on a connection that can never come up. In the other order, bind(global) then connect(fe80::x%ifB), the connect leaves a global-bound socket with rs_bound_scope_id ifB, so sends to other link-local peers are refused as off-link and sends to global peers inherit a meaningless interface. Give the connected peer its own rs_conn_scope_id. rds_connect() records it there and leaves the bound scope alone, rds_bind()'s connected-socket check compares against it, and the destination-less send takes its scope from it - falling back to the bound scope for a non-link-local peer, so that send() and sendto() to the connected peer compute the same scope, and so the same connection. getpeername() reports the connected peer with its own scope. recvmsg() used to report every sender with the socket's bound scope; a connected socket receives from any peer, so neither the bound scope nor the connected peer's scope is right for an arbitrary sender. It now reports a link-local sender with the interface its connection came in on (c_dev_if, which the TCP accept path takes from the incoming socket) and any other sender with scope 0, the inet6 convention, so an application can reply to the address it was given. Found by inspection while reworking the sendmsg connection cache; not observed in the field. Fixes: 1e2b44e78eea ("rds: Enable RDS IPv6 support") Assisted-by: Claude-Code:claude-fable-5 Signed-off-by: Allison Henderson <achender@kernel.org> --- net/rds/af_rds.c | 15 +++++++++------ net/rds/bind.c | 4 ++-- net/rds/rds.h | 2 ++ net/rds/recv.c | 9 ++++++++- net/rds/send.c | 6 +++++- 5 files changed, 26 insertions(+), 10 deletions(-) diff --git a/net/rds/af_rds.c b/net/rds/af_rds.c index d5defe9172e3..e84d00bbe9a1 100644 --- a/net/rds/af_rds.c +++ b/net/rds/af_rds.c @@ -136,8 +136,7 @@ static int rds_getname(struct socket *sock, struct sockaddr *uaddr, sin6->sin6_port = rs->rs_conn_port; sin6->sin6_addr = rs->rs_conn_addr; sin6->sin6_flowinfo = 0; - /* scope_id is the same as in the bound address. */ - sin6->sin6_scope_id = rs->rs_bound_scope_id; + sin6->sin6_scope_id = rs->rs_conn_scope_id; uaddr_len = sizeof(*sin6); } } else { @@ -572,6 +571,7 @@ static int rds_connect(struct socket *sock, struct sockaddr_unsized *uaddr, } ipv6_addr_set_v4mapped(sin->sin_addr.s_addr, &rs->rs_conn_addr); rs->rs_conn_port = sin->sin_port; + rs->rs_conn_scope_id = 0; break; #if IS_ENABLED(CONFIG_IPV6) @@ -616,11 +616,14 @@ static int rds_connect(struct socket *sock, struct sockaddr_unsized *uaddr, ret = -EINVAL; break; } - /* Remember the connected address scope ID. It will - * be checked against the binding local address when - * the socket is bound. + /* Remember the connected address scope ID. It is + * checked against the binding local address when + * the socket is bound, and gives a send without a + * destination its scope. */ - rs->rs_bound_scope_id = sin6->sin6_scope_id; + rs->rs_conn_scope_id = sin6->sin6_scope_id; + } else { + rs->rs_conn_scope_id = 0; } rs->rs_conn_addr = sin6->sin6_addr; rs->rs_conn_port = sin6->sin6_port; diff --git a/net/rds/bind.c b/net/rds/bind.c index f800d920d969..3ac59cd512a2 100644 --- a/net/rds/bind.c +++ b/net/rds/bind.c @@ -233,8 +233,8 @@ int rds_bind(struct socket *sock, struct sockaddr_unsized *uaddr, int addr_len) * non-link local address (scope_id is 0). */ if (!ipv6_addr_any(&rs->rs_conn_addr) && scope_id && - rs->rs_bound_scope_id && - scope_id != rs->rs_bound_scope_id) { + rs->rs_conn_scope_id && + scope_id != rs->rs_conn_scope_id) { ret = -EINVAL; goto out; } diff --git a/net/rds/rds.h b/net/rds/rds.h index 2db49573dacd..9b1ffc49c0a6 100644 --- a/net/rds/rds.h +++ b/net/rds/rds.h @@ -646,6 +646,8 @@ struct rds_sock { struct in6_addr rs_conn_addr; #define rs_conn_addr_v4 rs_conn_addr.s6_addr32[3] __be16 rs_conn_port; + /* scope of rs_conn_addr when it is link-local, 0 otherwise */ + __u32 rs_conn_scope_id; struct rds_transport *rs_transport; /* diff --git a/net/rds/recv.c b/net/rds/recv.c index 6204e577a90a..a036162539e5 100644 --- a/net/rds/recv.c +++ b/net/rds/recv.c @@ -789,7 +789,14 @@ int rds_recvmsg(struct socket *sock, struct msghdr *msg, size_t size, sin6->sin6_port = inc->i_hdr.h_sport; sin6->sin6_addr = inc->i_saddr; sin6->sin6_flowinfo = 0; - sin6->sin6_scope_id = rs->rs_bound_scope_id; + /* A link-local sender is reached through the + * interface its connection came in on; any + * other sender has no scope. + */ + sin6->sin6_scope_id = + (ipv6_addr_type(&inc->i_saddr) & + IPV6_ADDR_LINKLOCAL) ? + inc->i_conn->c_dev_if : 0; msg->msg_namelen = sizeof(*sin6); } } diff --git a/net/rds/send.c b/net/rds/send.c index 1afa981e5c06..7b525a6f7eac 100644 --- a/net/rds/send.c +++ b/net/rds/send.c @@ -1256,7 +1256,11 @@ int rds_sendmsg(struct socket *sock, struct msghdr *msg, size_t payload_len) lock_sock(sk); daddr = rs->rs_conn_addr; dport = rs->rs_conn_port; - scope_id = rs->rs_bound_scope_id; + /* The connected peer's scope for a link-local peer, else + * the bound scope, as an explicit send to that peer would + * compute below. + */ + scope_id = rs->rs_conn_scope_id ?: rs->rs_bound_scope_id; release_sock(sk); } -- 2.50.1 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH net-next v5 1/2] net/rds: keep the connected peer's scope id apart from the bound one 2026-10-17 3:23 ` [PATCH net-next v5 1/2] net/rds: keep the connected peer's scope id apart from the bound one Allison Henderson @ 2026-10-17 3:29 ` sashiko-bot 2026-10-18 3:27 ` netdev-bot+sashiko 1 sibling, 0 replies; 6+ messages in thread From: sashiko-bot @ 2026-10-17 3:29 UTC (permalink / raw) To: Allison Henderson, linux-rdma > rds_connect() has nowhere to keep the scope of a link-local peer, so > it stores it in rs_bound_scope_id, the scope of the socket's own > bound address, for rds_bind() to check a later link-local bind > against. That field is then also what a send without a destination > uses as the request's scope, what getpeername() and recvmsg() report > as the peer's scope, and what a later bind() overwrites: > rds_add_bound() stores the bound address's scope unconditionally, > which for a non-link-local address is 0. > > So after connect(fe80::x%ifA) followed by bind(global), a send() with > no destination asks for fe80::x with scope 0. rds_conn_lookup() keys > on the interface, so that finds or creates a connection with c_dev_if > 0, which the TCP transport then tries to connect through > sin6_scope_id 0 and tcp_v6_connect() rejects for a link-local peer: > the connected socket's data is queued on a connection that can never > [ ... ] > observed in the field. > > Fixes: 1e2b44e78eea ("rds: Enable RDS IPv6 support") > Assisted-by: Claude-Code:claude-fable-5 > Signed-off-by: Allison Henderson <achender@kernel.org> Sashiko has reviewed this patch and found no issues. -- Sashiko AI review · https://sashiko.dev/#/patchset/20261017032358.2563121-1-achender@kernel.org?part=1 ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH net-next v5 1/2] net/rds: keep the connected peer's scope id apart from the bound one 2026-10-17 3:23 ` [PATCH net-next v5 1/2] net/rds: keep the connected peer's scope id apart from the bound one Allison Henderson 2026-10-17 3:29 ` sashiko-bot @ 2026-10-18 3:27 ` netdev-bot+sashiko 1 sibling, 0 replies; 6+ messages in thread From: netdev-bot+sashiko @ 2026-10-18 3:27 UTC (permalink / raw) To: achender; +Cc: netdev, linux-rdma, pabeni, edumazet, kuba, horms Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider. Critical: 0 · High: 0 · Medium: 1 (1) · Low: 1 If you believe none of the issues are legitimate, please reply to each issue of Medium severity or higher and explain why the AI is wrong. If any of them are legitimate and you plan to respin, please let patchwork know by including "pw-bot: cr" as a separate line at the end of your reply (one such reply per series is enough). - [Medium] In rds_sendmsg() (net/rds/send.c), the rs_conn cache-hit test at this commit compares only c_faddr and c_tos. - [Low] rds_recvmsg() (net/rds/recv.c:792-799) now reports a link-local sender's sin6_scope_id as inc->i_conn->c_dev_if, with the comment 'A… Pre-existing issues: - [Medium] rds_sendmsg() (net/rds/send.c:1256-1264) reads the scope for a send with no destination in one lock_sock() section and checks that… -- net/rds: keep the connected peer's scope id apart from the bound one This patch adds rs_conn_scope_id to struct rds_sock. With it, rds_connect() no longer stores a link-local peer's scope in rs_bound_scope_id. A send without a destination, getpeername() and the connected-socket check in rds_bind() now use the connected peer's scope, and recvmsg() takes a link-local sender's scope from the incoming connection's c_dev_if. [ ... ] > diff --git a/net/rds/recv.c b/net/rds/recv.c > index 6204e577a90ae..a036162539e5e 100644 > --- a/net/rds/recv.c > +++ b/net/rds/recv.c > @@ -789,7 +789,14 @@ int rds_recvmsg(struct socket *sock, struct msghdr *msg, size_t size, > sin6->sin6_port = inc->i_hdr.h_sport; > sin6->sin6_addr = inc->i_saddr; > sin6->sin6_flowinfo = 0; > - sin6->sin6_scope_id = rs->rs_bound_scope_id; > + /* A link-local sender is reached through the > + * interface its connection came in on; any > + * other sender has no scope. > + */ > + sin6->sin6_scope_id = > + (ipv6_addr_type(&inc->i_saddr) & > + IPV6_ADDR_LINKLOCAL) ? > + inc->i_conn->c_dev_if : 0; [Severity: Low] Is c_dev_if always the interface the connection came in on? For RDS/TCP it is, because the accept path takes it from the accepted socket's sk_bound_dev_if. For RDS/IB it looks more like a guess. In rds_ib_cm_handle_connect(), when the peer address is link-local and our local address is global, the ifindex is looked up from our own address: net/rds/ib_cm.c:rds_ib_cm_handle_connect() { ... } else if (ipv6_addr_type(saddr6) & IPV6_ADDR_LINKLOCAL) { /* Use our address to find the correct index. */ ifindex = __rds_find_ifindex(&init_net, daddr6); ... } __rds_find_ifindex() returns the first netdev that carries that address. The comment above it says the incoming interface of an RDMA connect request cannot be determined. So on the IB transport, can recvmsg() on a globally bound socket return fe80::peer%ifX where ifX is not the peer's link? Replies sent through RDS still look fine. A sendto() to the reported address uses scope_id ifX, which matches the key of the incoming IB connection, so rds_conn_lookup() returns that same connection. What's left is that the new comment and the commit message describe c_dev_if as the arrival interface. That only holds for TCP. Could the comment say that for IB this is a best-effort guess? > diff --git a/net/rds/send.c b/net/rds/send.c > index 1afa981e5c06d..7b525a6f7eac2 100644 > --- a/net/rds/send.c > +++ b/net/rds/send.c > @@ -1256,7 +1256,11 @@ int rds_sendmsg(struct socket *sock, struct msghdr *msg, size_t payload_len) > lock_sock(sk); > daddr = rs->rs_conn_addr; > dport = rs->rs_conn_port; > - scope_id = rs->rs_bound_scope_id; > + /* The connected peer's scope for a link-local peer, else > + * the bound scope, as an explicit send to that peer would > + * compute below. > + */ > + scope_id = rs->rs_conn_scope_id ?: rs->rs_bound_scope_id; > release_sock(sk); > } [Severity: Medium] At this commit, does the rs_conn cache in rds_sendmsg() respect this scope_id? Further down, the cache-hit test still compares only the peer address and tos: if (rs->rs_conn && ipv6_addr_equal(&rs->rs_conn->c_faddr, &daddr) && rs->rs_tos == rs->rs_conn->c_tos) { conn = rs->rs_conn; rds_conn_lookup(), on the other hand, treats c_dev_if as part of a connection's identity. This patch also makes a new sequence possible on a socket bound to a global address: connect(fe80::1%ifA) send() connect(fe80::1%ifB) send() Before this patch, the second connect() failed with -EINVAL, because the first one had stored ifA in rs_bound_scope_id and rds_connect() compared against it. Now the second connect() succeeds, and the second send() seems to reuse the cached ifA connection instead of one for ifB. The next patch in the series, "net/rds: include the scope id in the sendmsg connection cache check", adds rs->rs_conn->c_dev_if == scope_id to the cache test, so the final tree is fine. Could that patch go before this one, so this intermediate commit doesn't send on the wrong interface? [Severity: Medium] This isn't a bug introduced by this patch, but the scope is read here in one lock_sock() section, and the bound check is done in a second one: lock_sock(sk); if (ipv6_addr_any(&rs->rs_bound_addr) || ipv6_addr_any(&daddr)) { ... } else if (namelen != 0) { Could a concurrent bind() run between the two sections? Take a socket connected to a global peer. If rds_bind()->rds_add_bound() binds fe80::a%ifA after scope_id has already been read as 0, the send continues with scope_id 0 against a link-local bound address. The namelen == 0 path skips the scope reconciliation that sendto() gets. The result would be a connection with c_dev_if 0, which the TCP transport cannot bring up. The baseline had the same split, with scope_id = rs->rs_bound_scope_id read in the first section, so this patch changes neither the window nor the outcome. -- Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261017032358.2563121-1-achender%40kernel.org ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH net-next v5 2/2] net/rds: include the scope id in the sendmsg connection cache check 2026-10-17 3:23 [PATCH net-next v5 0/2] net/rds: scope-aware sendmsg connection cache Allison Henderson 2026-10-17 3:23 ` [PATCH net-next v5 1/2] net/rds: keep the connected peer's scope id apart from the bound one Allison Henderson @ 2026-10-17 3:23 ` Allison Henderson 2026-10-17 3:29 ` sashiko-bot 1 sibling, 1 reply; 6+ messages in thread From: Allison Henderson @ 2026-10-17 3:23 UTC (permalink / raw) To: netdev, linux-rdma, pabeni, edumazet, kuba, horms; +Cc: achender rds_sendmsg() reuses the connection cached in rs->rs_conn when its peer address and ToS match the request. The interface index is part of a connection's identity as well: rds_conn_create_outgoing() passes the request's scope_id down as dev_if, and rds_conn_lookup() compares c_dev_if, so sends to the same link-local address through two interfaces are two different connections. The cache-hit test never looked at it. A socket bound to a non-link-local address has rs_bound_scope_id 0, and the scope check at the top of rds_sendmsg() accepts any non-zero destination scope for such a socket. So after a send to fe80::x%ifA, a send to fe80::x%ifB hits the cached ifA connection and the datagram leaves through ifA, to whichever peer answers to that address there. Compare c_dev_if with the request's scope_id in the cache test, so that such a send takes the lookup path and finds, or creates, the ifB connection - the same key the lookup uses. With the previous patch a send without a destination carries the connected peer's scope, or the bound scope for a non-link-local peer, which is what an explicit send to that peer computes, so the comparison is exact for it too. Found by inspection while reworking this cache for the connection lifetime series on net-next; not observed in the field. Fixes: 1e2b44e78eea ("rds: Enable RDS IPv6 support") Assisted-by: Claude-Code:claude-fable-5 Signed-off-by: Allison Henderson <achender@kernel.org> --- net/rds/send.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/net/rds/send.c b/net/rds/send.c index 7b525a6f7eac..f38967f65af1 100644 --- a/net/rds/send.c +++ b/net/rds/send.c @@ -1346,7 +1346,8 @@ int rds_sendmsg(struct socket *sock, struct msghdr *msg, size_t payload_len) /* rds_conn_create has a spinlock that runs with IRQ off. * Caching the conn in the socket helps a lot. */ if (rs->rs_conn && ipv6_addr_equal(&rs->rs_conn->c_faddr, &daddr) && - rs->rs_tos == rs->rs_conn->c_tos) { + rs->rs_tos == rs->rs_conn->c_tos && + rs->rs_conn->c_dev_if == scope_id) { conn = rs->rs_conn; } else { conn = rds_conn_create_outgoing(sock_net(sock->sk), -- 2.50.1 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH net-next v5 2/2] net/rds: include the scope id in the sendmsg connection cache check 2026-10-17 3:23 ` [PATCH net-next v5 2/2] net/rds: include the scope id in the sendmsg connection cache check Allison Henderson @ 2026-10-17 3:29 ` sashiko-bot 0 siblings, 0 replies; 6+ messages in thread From: sashiko-bot @ 2026-10-17 3:29 UTC (permalink / raw) To: Allison Henderson, linux-rdma > rds_sendmsg() reuses the connection cached in rs->rs_conn when its > peer address and ToS match the request. The interface index is part > of a connection's identity as well: rds_conn_create_outgoing() passes > the request's scope_id down as dev_if, and rds_conn_lookup() compares > c_dev_if, so sends to the same link-local address through two > interfaces are two different connections. The cache-hit test never > looked at it. > > A socket bound to a non-link-local address has rs_bound_scope_id 0, > and the scope check at the top of rds_sendmsg() accepts any non-zero > destination scope for such a socket. So after a send to fe80::x%ifA, > a send to fe80::x%ifB hits the cached ifA connection and the datagram > leaves through ifA, to whichever peer answers to that address there. > > Compare c_dev_if with the request's scope_id in the cache test, so > that such a send takes the lookup path and finds, or creates, the ifB > connection - the same key the lookup uses. With the previous patch a > send without a destination carries the connected peer's scope, or the > bound scope for a non-link-local peer, which is what an explicit send > to that peer computes, so the comparison is exact for it too. > > Found by inspection while reworking this cache for the connection > lifetime series on net-next; not observed in the field. > > Fixes: 1e2b44e78eea ("rds: Enable RDS IPv6 support") > Assisted-by: Claude-Code:claude-fable-5 > Signed-off-by: Allison Henderson <achender@kernel.org> Sashiko has reviewed this patch and found no issues. -- Sashiko AI review · https://sashiko.dev/#/patchset/20261017032358.2563121-1-achender@kernel.org?part=2 ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2026-10-18 3:27 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-10-17 3:23 [PATCH net-next v5 0/2] net/rds: scope-aware sendmsg connection cache Allison Henderson 2026-10-17 3:23 ` [PATCH net-next v5 1/2] net/rds: keep the connected peer's scope id apart from the bound one Allison Henderson 2026-10-17 3:29 ` sashiko-bot 2026-10-18 3:27 ` netdev-bot+sashiko 2026-10-17 3:23 ` [PATCH net-next v5 2/2] net/rds: include the scope id in the sendmsg connection cache check Allison Henderson 2026-10-17 3:29 ` sashiko-bot
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.