[PATCH AUTOSEL 6.19-5.10] net/rds: Clear reconnect pending bit

[PATCH AUTOSEL 6.19-5.10] net/rds: Clear reconnect pending bit

[Sasha Levin]

From: Håkon Bugge <haakon.bugge@oracle.com>

[ Upstream commit b89fc7c2523b2b0750d91840f4e52521270d70ed ]

When canceling the reconnect worker, care must be taken to reset the
reconnect-pending bit. If the reconnect worker has not yet been
scheduled before it is canceled, the reconnect-pending bit will stay
on forever.

Signed-off-by: Håkon Bugge <haakon.bugge@oracle.com>
Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Link: https://patch.msgid.link/20260203055723.1085751-6-achender@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---

LLM Generated explanations, may be completely bogus:

### 3. BUG MECHANISM — CLEAR AND CRITICAL

Now the full picture is clear:

**The flow:**
1. `rds_queue_reconnect()` at line 138 sets `RDS_RECONNECT_PENDING` and
   queues the delayed work.
2. `rds_connect_worker()` (the worker function) at line 173 clears
   `RDS_RECONNECT_PENDING` when it runs.
3. `rds_conn_path_connect_if_down()` at line 911 uses
   `test_and_set_bit(RDS_RECONNECT_PENDING, ...)` — if the bit is
   already set, it returns without queuing work, trusting that a
   reconnect is already pending.

**The bug:**
In `rds_conn_shutdown()`, `cancel_delayed_work_sync()` cancels the
queued worker. If the worker hadn't started yet, it never runs
`clear_bit(RDS_RECONNECT_PENDING, ...)`. The bit stays set. Then when
`rds_conn_path_connect_if_down()` is later called (e.g., when trying to
send data), `test_and_set_bit` finds the bit already set and skips
queuing — **forever**. The connection can never reconnect.

This is a **permanent connection failure** bug — once triggered, the RDS
connection path is effectively dead until the system is rebooted or the
module is reloaded.

### 4. CLASSIFICATION

- **Bug type:** State corruption / logic bug leading to permanent loss
  of network connectivity
- **Severity:** HIGH — RDS is used in production Oracle RAC clusters and
  RDMA-based environments
- **Trigger:** Race between shutdown and reconnect scheduling —
  realistic in production with network flaps

### 5. SCOPE AND RISK

- **Change size:** 1 functional line (plus 1 blank line) — extremely
  small and surgical
- **Files changed:** 1 file (`net/rds/connection.c`)
- **Risk of regression:** Very low — `clear_bit` is idempotent. If the
  worker already ran and cleared the bit, clearing it again is harmless.
  If the worker didn't run, this is the correct fix.
- **The fix is placed correctly:** After `cancel_delayed_work_sync()`
  guarantees the worker won't run, and before `rds_queue_reconnect()`
  which will set the bit again if needed.

### 6. STABLE KERNEL CRITERIA

- **Obviously correct:** Yes — the logic is straightforward and well-
  explained
- **Fixes a real bug:** Yes — permanent loss of RDS connectivity
- **Important issue:** Yes — affects network reliability for RDS users
  (Oracle, RDMA environments)
- **Small and contained:** Yes — 1 line in 1 file
- **No new features:** Correct — purely a bug fix
- **No new APIs:** Correct

### 7. DEPENDENCY CHECK

This fix is self-contained. It only adds a `clear_bit()` call after an
existing `cancel_delayed_work_sync()` call. The `RDS_RECONNECT_PENDING`
flag and the surrounding code have been in the kernel for a long time.
This should apply cleanly to any stable tree that has the RDS subsystem.

### Conclusion

This is a textbook stable backport candidate: a tiny, obviously correct,
one-line fix for a serious bug (permanent connection failure) in
networking code used in production environments. The fix has zero risk
of regression due to the idempotent nature of `clear_bit`, and the bug
mechanism is clearly explained and verified through code analysis.

**YES**

 net/rds/connection.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/net/rds/connection.c b/net/rds/connection.c
index ad8027e6f54ef..dbfea6fa11260 100644
--- a/net/rds/connection.c
+++ b/net/rds/connection.c
@@ -429,6 +429,8 @@ void rds_conn_shutdown(struct rds_conn_path *cp)
 	 * to the conn hash, so we never trigger a reconnect on this
 	 * conn - the reconnect is always triggered by the active peer. */
 	cancel_delayed_work_sync(&cp->cp_conn_w);
+
+	clear_bit(RDS_RECONNECT_PENDING, &cp->cp_flags);
 	rcu_read_lock();
 	if (!hlist_unhashed(&conn->c_hash_node)) {
 		rcu_read_unlock();
-- 
2.51.0

[PATCH AUTOSEL 6.19-5.10] net/rds: No shortcut out of RDS_CONN_ERROR

[Sasha Levin]

From: Gerd Rausch <gerd.rausch@oracle.com>

[ Upstream commit ad22d24be635c6beab6a1fdd3f8b1f3c478d15da ]

RDS connections carry a state "rds_conn_path::cp_state"
and transitions from one state to another and are conditional
upon an expected state: "rds_conn_path_transition."

There is one exception to this conditionality, which is
"RDS_CONN_ERROR" that can be enforced by "rds_conn_path_drop"
regardless of what state the condition is currently in.

But as soon as a connection enters state "RDS_CONN_ERROR",
the connection handling code expects it to go through the
shutdown-path.

The RDS/TCP multipath changes added a shortcut out of
"RDS_CONN_ERROR" straight back to "RDS_CONN_CONNECTING"
via "rds_tcp_accept_one_path" (e.g. after "rds_tcp_state_change").

A subsequent "rds_tcp_reset_callbacks" can then transition
the state to "RDS_CONN_RESETTING" with a shutdown-worker queued.

That'll trip up "rds_conn_init_shutdown", which was
never adjusted to handle "RDS_CONN_RESETTING" and subsequently
drops the connection with the dreaded "DR_INV_CONN_STATE",
which leaves "RDS_SHUTDOWN_WORK_QUEUED" on forever.

So we do two things here:

a) Don't shortcut "RDS_CONN_ERROR", but take the longer
   path through the shutdown code.

b) Add "RDS_CONN_RESETTING" to the expected states in
  "rds_conn_init_shutdown" so that we won't error out
  and get stuck, if we ever hit weird state transitions
  like this again."

Signed-off-by: Gerd Rausch <gerd.rausch@oracle.com>
Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Link: https://patch.msgid.link/20260122055213.83608-2-achender@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---

LLM Generated explanations, may be completely bogus:

## Analysis of net/rds: No shortcut out of RDS_CONN_ERROR

### 1. COMMIT MESSAGE ANALYSIS

The commit message is detailed and clearly describes a **state machine
bug** in the RDS (Reliable Datagram Sockets) TCP multipath code:

- A connection in `RDS_CONN_ERROR` state was being shortcut directly to
  `RDS_CONN_CONNECTING` via `rds_tcp_accept_one_path`, bypassing the
  required shutdown path.
- This leads to a subsequent transition to `RDS_CONN_RESETTING` with a
  shutdown worker queued.
- `rds_conn_init_shutdown` was never adjusted to handle
  `RDS_CONN_RESETTING`, causing it to drop the connection with
  `DR_INV_CONN_STATE`.
- This leaves `RDS_SHUTDOWN_WORK_QUEUED` set forever, effectively
  **permanently breaking the connection**.

The description of "leaves RDS_SHUTDOWN_WORK_QUEUED on forever" is a
serious consequence — it means the RDS connection gets permanently
stuck, which is essentially a hang/denial of service for RDS users.

### 2. CODE CHANGE ANALYSIS

The fix has two parts:

**Part A: `net/rds/tcp_listen.c` — Remove the shortcut from
RDS_CONN_ERROR**

```c
- if (rds_conn_path_transition(cp, RDS_CONN_DOWN,
- RDS_CONN_CONNECTING) ||
- rds_conn_path_transition(cp, RDS_CONN_ERROR,
- RDS_CONN_CONNECTING)) {
+ if (rds_conn_path_transition(cp, RDS_CONN_DOWN,
+                              RDS_CONN_CONNECTING)) {
```

This removes the problematic shortcut that allowed `RDS_CONN_ERROR` →
`RDS_CONN_CONNECTING` directly, which bypassed the shutdown path. The
comment explaining this shortcut behavior is also removed.

**Part B: `net/rds/connection.c` — Handle RDS_CONN_RESETTING in
shutdown**

```c
 if (!rds_conn_path_transition(cp, RDS_CONN_UP,
                               RDS_CONN_DISCONNECTING) &&
     !rds_conn_path_transition(cp, RDS_CONN_ERROR,
+                              RDS_CONN_DISCONNECTING) &&
+    !rds_conn_path_transition(cp, RDS_CONN_RESETTING,
                               RDS_CONN_DISCONNECTING)) {
```

This adds `RDS_CONN_RESETTING` as an acceptable state to transition from
during shutdown, making the shutdown code more robust against unexpected
state transitions.

### 3. BUG CLASSIFICATION

This is a **state machine bug** that leads to:
- **Connection hang**: RDS connections get permanently stuck with
  `RDS_SHUTDOWN_WORK_QUEUED` set forever
- **Denial of service**: Users of RDS (common in Oracle database
  clusters) lose connectivity
- This is effectively a **deadlock/livelock** in the connection state
  machine

### 4. SCOPE AND RISK ASSESSMENT

- **Files changed**: 2 files
- **Lines changed**: Very small — removing 3 lines from one function,
  adding 2 lines to another
- **Subsystem**: net/rds (RDS networking, widely used in Oracle
  environments)
- **Risk**: LOW — The changes are surgical and well-contained:
  - Part A removes a transition that was causing the bug (conservative
    change)
  - Part B adds defensive handling for an additional state in the
    shutdown path (safe addition)
- **Could this break something?**: Removing the shortcut means
  connections in ERROR state will take the longer path through shutdown.
  This is the correct/expected behavior and should not cause
  regressions.

### 5. USER IMPACT

RDS is used extensively in:
- Oracle RAC (Real Application Clusters) database environments
- High-availability enterprise systems
- Oracle Cloud Infrastructure

A permanently stuck connection is a serious production issue for these
users. The bug is in the multipath code path, which is actively used in
modern RDS deployments.

### 6. STABLE KERNEL CRITERIA

- **Obviously correct**: Yes — the commit message explains the problem
  clearly, and the fix is straightforward state machine correction
- **Fixes a real bug**: Yes — connections getting permanently stuck is a
  real, user-impacting bug
- **Important issue**: Yes — connection hangs in enterprise networking
  code
- **Small and contained**: Yes — minimal changes to 2 files in the same
  subsystem
- **No new features**: Correct — this only fixes state machine
  transitions
- **Tested**: The commit comes from Oracle developers (Gerd Rausch,
  Allison Henderson) who maintain RDS and was accepted by the net
  maintainer (Jakub Kicinski)

### 7. DEPENDENCY CHECK

The fix is self-contained. It modifies existing state transition logic
that has been present since the multipath changes were added. The
`RDS_CONN_RESETTING` state and the relevant functions all exist in
stable kernels that have RDS TCP multipath support.

### Summary

This is a clear bug fix for a state machine issue in RDS TCP multipath
that causes connections to get permanently stuck. The fix is small,
surgical, well-understood, and comes from the RDS subsystem maintainers
at Oracle. It fixes a real production issue (permanent connection hang)
with minimal risk of regression. It meets all stable kernel criteria.

**YES**

 net/rds/connection.c | 2 ++
 net/rds/tcp_listen.c | 5 -----
 2 files changed, 2 insertions(+), 5 deletions(-)

diff --git a/net/rds/connection.c b/net/rds/connection.c
index 68bc88cce84ec..ad8027e6f54ef 100644
--- a/net/rds/connection.c
+++ b/net/rds/connection.c
@@ -382,6 +382,8 @@ void rds_conn_shutdown(struct rds_conn_path *cp)
 		if (!rds_conn_path_transition(cp, RDS_CONN_UP,
 					      RDS_CONN_DISCONNECTING) &&
 		    !rds_conn_path_transition(cp, RDS_CONN_ERROR,
+					      RDS_CONN_DISCONNECTING) &&
+		    !rds_conn_path_transition(cp, RDS_CONN_RESETTING,
 					      RDS_CONN_DISCONNECTING)) {
 			rds_conn_path_error(cp,
 					    "shutdown called in state %d\n",
diff --git a/net/rds/tcp_listen.c b/net/rds/tcp_listen.c
index 820d3e20de195..27b6107ddc28d 100644
--- a/net/rds/tcp_listen.c
+++ b/net/rds/tcp_listen.c
@@ -59,9 +59,6 @@ void rds_tcp_keepalive(struct socket *sock)
  * socket and force a reconneect from smaller -> larger ip addr. The reason
  * we special case cp_index 0 is to allow the rds probe ping itself to itself
  * get through efficiently.
- * Since reconnects are only initiated from the node with the numerically
- * smaller ip address, we recycle conns in RDS_CONN_ERROR on the passive side
- * by moving them to CONNECTING in this function.
  */
 static
 struct rds_tcp_connection *rds_tcp_accept_one_path(struct rds_connection *conn)
@@ -86,8 +83,6 @@ struct rds_tcp_connection *rds_tcp_accept_one_path(struct rds_connection *conn)
 		struct rds_conn_path *cp = &conn->c_path[i];
 
 		if (rds_conn_path_transition(cp, RDS_CONN_DOWN,
-					     RDS_CONN_CONNECTING) ||
-		    rds_conn_path_transition(cp, RDS_CONN_ERROR,
 					     RDS_CONN_CONNECTING)) {
 			return cp->cp_transport_data;
 		}
-- 
2.51.0