All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 0/1] scsi: target: iscsi: fix use-after-free in iscsit_stop_session()
@ 2026-11-21 16:38 Vitalii Dodonov
  2026-11-21 16:38 ` [PATCH 1/1] " Vitalii Dodonov
  0 siblings, 1 reply; 3+ messages in thread
From: Vitalii Dodonov @ 2026-11-21 16:38 UTC (permalink / raw)
  To: linux-scsi; +Cc: stable, target-devel

The iSCSI target panics with a general protection fault inside
iscsit_stop_session() when a session that has more than one connection is
torn down while another thread closes one of those connections.

iscsit_stop_session() walks sess->sess_conn_list with
list_for_each_entry_safe() and drops sess->conn_lock inside the loop.  A
connection that is being shut down concurrently runs iscsit_close_connection(),
which removes itself from the list with list_del() under the same lock and
leaves LIST_POISON1 in conn->conn_list.next.  Pinning the next connection with
iscsit_inc_conn_usage_count() keeps the object alive, but not the list links,
so the following iteration computes conn_tmp = list_next_entry() from the
poisoned pointer and passes it to iscsit_inc_conn_usage_count(), which takes a
spinlock at *(conn_tmp + offsetof(conn_usage_lock)) and faults on
0xdead000000000004.

Crash, 7.3.0-11-generic, LIO target with MaxConnections=8, Microsoft iSCSI
initiator holding 8 MC/S connections, graceful reboot of the initiator while
I/O is in flight:

  Oops: general protection fault, probably for non-canonical address 0xdead000000000004: 0000 [#1] SMP PTI
  CPU: 0 UID: 0 PID: 9028 Comm: kworker/0:2 Not tainted 7.3.0-11-generic #12-Ubuntu PREEMPT(lazy)
  Workqueue: events iscsi_target_do_login_rx [iscsi_target_mod]
  RIP: 0010:_raw_spin_lock_bh+0x18/0x50
  RDI: dead000000000004   RBX: deacfffffffffcb0   R15: deacfffffffffcb0
  Call Trace:
   ? iscsit_inc_conn_usage_count+0x1b/0x40 [iscsi_target_mod]
   iscsit_stop_session+0x7d/0x1b0 [iscsi_target_mod]
   iscsi_check_for_session_reinstatement+0x1e1/0x290 [iscsi_target_mod]
   iscsi_target_handle_csg_one+0x135/0x160 [iscsi_target_mod]
   iscsi_target_do_login+0x32/0xe0 [iscsi_target_mod]
   iscsi_target_do_login_rx+0x2d2/0x420 [iscsi_target_mod]
   process_one_work+0x1dc/0x3f0
   ...
  Kernel panic - not syncing: Fatal exception in interrupt

The same fault is reachable from the logout path:
iscsit_stop_session <- iscsit_logout_post_handler <- iscsit_response_queue <-
iscsi_target_tx_thread, with the identical 0xdead000000000004.

Where it was first seen (field report).  A Proxmox VE host (7.0.14-9-pve,
later 7.0.14-28-pve) exported zvols through LIO to Windows 11 clients using
the Microsoft iSCSI initiator with MC/S.  The host had been stable for 95 days
with MaxConnections=1; after raising it to 2 there were four panics in one
night, all with the same general protection fault at 0xdead000000000004 and
the logout stack above, and the journal of one of them contains "Received
logout request on connection that is not in logged in state".  The panic also
reproduced on 7.0.14-28-pve, so it is not tied to a single build.  The module
carrying this fix has been running on that host for 9 days with
MaxConnections=4 and 72 active iSCSI connections, without a single panic.

Reproducer (any MC/S-capable initiator; open-iscsi does not implement MC/S, so
a Microsoft iSCSI initiator was used):

 1. LIO target with MaxConnections=8, no CHAP, generate_node_acls=1.  The
    value lives in configfs and is not exported by targetcli:

      echo 8 > /sys/kernel/config/target/iscsi/<iqn>/tpgt_1/param/MaxConnections

    With MaxConnections=1 the extra connection is refused with
    ISDSC_TOO_MANY_CONNECTIONS and nothing happens, so the target has to be
    configured first.

 2. Initiator: log in, for example on Windows

      Connect-IscsiTarget -NodeAddress <iqn> -TargetPortalAddress <ip>

    and then add the remaining connections to the *same* session, one call
    per connection:

      iscsicli SessionList                 # take the "Session Id"
      iscsicli ListInitiators              # -> ROOT\ISCSIPRT\0000_0
      iscsicli QAddConnection <SessionId> ROOT\ISCSIPRT\0000_0 <portal-ip>

    Repeat until the session holds MaxConnections connections.
    AddIScsiConnectionW() does the same thing programmatically, and the
    "MCS..." button in the iSCSI Initiator GUI (target -> Properties ->
    Sessions) as well.

 3. Bring the LUN online, mount a filesystem on it and start writing to it.

 4. Reboot the initiator gracefully while the I/O is still in flight; repeat.
    The stock module crashed on the 2nd-4th cycle (2 panics in ~20 minutes).

Results: stock module 2 crashes in 8 cycles, patched module 0 in 118 cycles
(the target stayed up for the whole run).

Patch 1/1 takes a snapshot of the connections, with a reference on each, while
conn_lock is held, and then works only on that snapshot; the order in which the
connections are reinstated is unchanged.

Vitalii Dodonov (1):
  scsi: target: iscsi: fix use-after-free in iscsit_stop_session()

 drivers/target/iscsi/iscsi_target.c | 68 ++++++++++++++++++++---------
 1 file changed, 47 insertions(+), 21 deletions(-)

^ permalink raw reply	[flat|nested] 3+ messages in thread

* [PATCH 1/1] scsi: target: iscsi: fix use-after-free in iscsit_stop_session()
  2026-11-21 16:38 [PATCH 0/1] scsi: target: iscsi: fix use-after-free in iscsit_stop_session() Vitalii Dodonov
@ 2026-11-21 16:38 ` Vitalii Dodonov
  2026-11-21 16:50   ` sashiko-bot
  0 siblings, 1 reply; 3+ messages in thread
From: Vitalii Dodonov @ 2026-11-21 16:38 UTC (permalink / raw)
  To: linux-scsi; +Cc: stable, target-devel

iscsit_stop_session() walks sess->sess_conn_list with
list_for_each_entry_safe() while dropping sess->conn_lock inside the loop.
A connection that is being shut down concurrently removes itself from that
list with list_del() under the same lock, which leaves LIST_POISON1 in
conn->conn_list.next.

Taking a reference on the next connection keeps the object alive, but not
the list links: the next iteration computes conn_tmp = list_next_entry()
from the poisoned pointer and passes it to iscsit_inc_conn_usage_count(),
which takes a spinlock inside the bogus struct and faults on
0xdead000000000004.

This is reachable whenever a session has more than one connection: on
logout (iscsit_logout_post_handler -> iscsit_stop_session) and on session
reinstatement, when a new login arrives while the old connections are
being closed (iscsi_check_for_session_reinstatement -> iscsit_stop_session).
With MaxConnections=1 list_is_last() is true, conn_tmp is never
dereferenced and the bug stays hidden, which is why single-connection
setups never hit it.

Fix it by taking a snapshot of the connections (with a reference on each)
while holding conn_lock, and using only the snapshot afterwards.  The
allocation happens before conn_lock is taken, on paths that already block
in iscsit_cause_connection_reinstatement(..., 1) and in
wait_for_completion(&sess->session_wait_comp).  The order in which the
connections are reinstated is unchanged.

Tested with a LIO target (MaxConnections=8) and a Microsoft iSCSI
initiator that adds MC/S connections and then reboots gracefully while
I/O is in flight: the stock module panics in 2 of 8 cycles (general
protection fault, address 0xdead000000000004), the patched module
survives 118 of 118 cycles.

Fixes: e48354ce078c ("iscsi-target: Add iSCSI fabric support for target v4.1")
Cc: stable@vger.kernel.org
Signed-off-by: Vitalii Dodonov <dreamer.two@gmail.com>
---
 drivers/target/iscsi/iscsi_target.c | 68 ++++++++++++++++++++---------
 1 file changed, 47 insertions(+), 21 deletions(-)

diff --git a/drivers/target/iscsi/iscsi_target.c b/drivers/target/iscsi/iscsi_target.c
index f3b7569..e644fe0 100644
--- a/drivers/target/iscsi/iscsi_target.c
+++ b/drivers/target/iscsi/iscsi_target.c
@@ -4672,36 +4672,62 @@ void iscsit_stop_session(
 	int session_sleep,
 	int connection_sleep)
 {
-	u16 conn_count = atomic_read(&sess->nconn);
-	struct iscsit_conn *conn, *conn_tmp = NULL;
-	int is_last;
+	struct iscsit_conn *conn, **conns = NULL;
+	unsigned int i, n, conn_count = 0;
+
+	/*
+	 * sess_conn_list may only be walked while holding sess->conn_lock: a
+	 * connection that is being shut down concurrently runs
+	 * iscsit_close_connection(), which removes itself from that list with
+	 * list_del() under the same lock and leaves LIST_POISON1 in
+	 * conn->conn_list.next.  Pinning the connection itself is not enough,
+	 * because the list links are poisoned as well, so take a snapshot of
+	 * the connections (with a reference on each) and work on that snapshot
+	 * only.
+	 *
+	 * The sleeping variant is called from process context only --
+	 * iscsi_target_tx_thread(), the login workqueue (session
+	 * reinstatement), configfs, iscsit_release_sessions_for_tpg() -- with
+	 * no locks held, and this function already blocks below in
+	 * iscsit_cause_connection_reinstatement(conn, 1) and in
+	 * wait_for_completion(&sess->session_wait_comp), so the GFP_KERNEL
+	 * allocation does not add a new constraint.
+	 */
+	if (connection_sleep) {
+		conn_count = atomic_read(&sess->nconn);
+		if (conn_count) {
+			conns = kcalloc(conn_count, sizeof(*conns), GFP_KERNEL);
+			if (!conns)
+				pr_warn("Unable to allocate %u connection slots for session %u, shutting down without waiting\n",
+					conn_count, sess->sid);
+		}
+	}
 
 	spin_lock_bh(&sess->conn_lock);
 
-	if (connection_sleep) {
-		list_for_each_entry_safe(conn, conn_tmp, &sess->sess_conn_list,
-				conn_list) {
-			if (conn_count == 0)
+	if (connection_sleep && conns) {
+		i = 0;
+		list_for_each_entry(conn, &sess->sess_conn_list, conn_list) {
+			if (i == conn_count)
 				break;
-
-			if (list_is_last(&conn->conn_list, &sess->sess_conn_list)) {
-				is_last = 1;
-			} else {
-				iscsit_inc_conn_usage_count(conn_tmp);
-				is_last = 0;
-			}
 			iscsit_inc_conn_usage_count(conn);
+			conns[i++] = conn;
+		}
+		spin_unlock_bh(&sess->conn_lock);
 
-			spin_unlock_bh(&sess->conn_lock);
+		for (n = 0; n < i; n++) {
+			conn = conns[n];
 			iscsit_cause_connection_reinstatement(conn, 1);
-			spin_lock_bh(&sess->conn_lock);
-
 			iscsit_dec_conn_usage_count(conn);
-			if (is_last == 0)
-				iscsit_dec_conn_usage_count(conn_tmp);
-			conn_count--;
 		}
-	} else {
+		kfree(conns);
+
+		spin_lock_bh(&sess->conn_lock);
+	} else if (!connection_sleep || conn_count) {
+		/*
+		 * Either the caller does not want to wait, or the snapshot could
+		 * not be allocated: tear the connections down without waiting.
+		 */
 		list_for_each_entry(conn, &sess->sess_conn_list, conn_list)
 			iscsit_cause_connection_reinstatement(conn, 0);
 	}

^ permalink raw reply related	[flat|nested] 3+ messages in thread

* Re: [PATCH 1/1] scsi: target: iscsi: fix use-after-free in iscsit_stop_session()
  2026-11-21 16:38 ` [PATCH 1/1] " Vitalii Dodonov
@ 2026-11-21 16:50   ` sashiko-bot
  0 siblings, 0 replies; 3+ messages in thread
From: sashiko-bot @ 2026-11-21 16:50 UTC (permalink / raw)
  To: Vitalii Dodonov, linux-scsi; +Cc: target-devel

Thank you for your contribution! Sashiko AI review found no new issues
in this patch, but noticed 1 related pre-existing issue it does not
introduce:
- [High] scsi: target: iscsi: session leak when closing failed session
  with nconn == 0 [1]
--

commit 29536f098b4fb525d53737f9024a60f1ba499f18
Author: Vitalii Dodonov <dreamer.two@gmail.com>

scsi: target: iscsi: fix use-after-free in iscsit_stop_session()

This commit fixes a use-after-free in iscsit_stop_session() by taking a
reference-counted snapshot of sess->sess_conn_list under sess->conn_lock
before dropping the lock to reinstate connections.

> diff --git a/drivers/target/iscsi/iscsi_target.c b/drivers/target/iscsi/iscsi_target.c
> index 03d618cd76721..bd56dc76fbca8 100644
> --- a/drivers/target/iscsi/iscsi_target.c
> +++ b/drivers/target/iscsi/iscsi_target.c
> @@ -4669,36 +4669,62 @@ void iscsit_stop_session(
>  	int session_sleep,
>  	int connection_sleep)
>  {
> -	u16 conn_count = atomic_read(&sess->nconn);
> -	struct iscsit_conn *conn, *conn_tmp = NULL;
> -	int is_last;
> +	struct iscsit_conn *conn, **conns = NULL;
> +	unsigned int i, n, conn_count = 0;
> +
> +	/*
> +	 * sess_conn_list may only be walked while holding sess->conn_lock: a
> +	 * connection that is being shut down concurrently runs
> +	 * iscsit_close_connection(), which removes itself from that list with
> +	 * list_del() under the same lock and leaves LIST_POISON1 in
> +	 * conn->conn_list.next.  Pinning the connection itself is not enough,
> +	 * because the list links are poisoned as well, so take a snapshot of
> +	 * the connections (with a reference on each) and work on that snapshot
> +	 * only.
> +	 *
> +	 * The sleeping variant is called from process context only --
> +	 * iscsi_target_tx_thread(), the login workqueue (session
> +	 * reinstatement), configfs, iscsit_release_sessions_for_tpg() -- with
> +	 * no locks held, and this function already blocks below in
> +	 * iscsit_cause_connection_reinstatement(conn, 1) and in
> +	 * wait_for_completion(&sess->session_wait_comp), so the GFP_KERNEL
> +	 * allocation does not add a new constraint.
> +	 */
> +	if (connection_sleep) {
> +		conn_count = atomic_read(&sess->nconn);

[Severity: High]
This problem wasn't introduced by this patch, but can closing or
reinstating a session in TARG_SESS_STATE_FAILED when sess->nconn is
already 0 leak the session [1]?

When the last connection closes without falling back to ERL=0 (for
example on ISCSI_LOGOUT_REASON_CLOSE_CONNECTION),
iscsit_close_connection() moves the session to TARG_SESS_STATE_FAILED
with sess->nconn == 0 and starts the Time2Retain timer.  Commit
57c46e9f33da ("scsi: target: fix hang when multiple threads try to
destroy the same iscsi session") removed the direct
iscsit_close_session() calls from iscsi_check_for_session_reinstatement(),
lio_tpg_close_session(), and iscsit_release_sessions_for_tpg(), relying
on iscsit_stop_session() to tear down the remaining connections so the
last connection calls iscsit_close_session().

If one of those paths runs while sess->nconn is already 0, it stops the
Time2Retain timer and either returns early on TARG_SESS_STATE_FAILED:

drivers/target/iscsi/iscsi_target_login.c:iscsi_check_for_session_reinstatement() {
    ...
	spin_lock_bh(&sess->conn_lock);
	if (sess->session_state == TARG_SESS_STATE_FAILED) {
		spin_unlock_bh(&sess->conn_lock);
		iscsit_dec_session_usage_count(sess);
		return 0;
	}
	spin_unlock_bh(&sess->conn_lock);

	iscsit_stop_session(sess, 1, 1);
    ...
}

or calls iscsit_stop_session() with an empty sess->sess_conn_list.  With
no connection left to run iscsit_close_connection(),
iscsit_close_session() never runs, leaking struct iscsit_session and
struct se_session and leaving core_tpg_del_initiator_node_acl() hung in
wait_for_completion(&acl->acl_free_comp).

[1] https://sashiko.dev/#/bug/linux-305f6726-98b0-4cd9-9238-5325539f4459

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/179527913740.32224.2426414360492240680@gmail.com?part=1

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2026-11-21 16:50 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-21 16:38 [PATCH 0/1] scsi: target: iscsi: fix use-after-free in iscsit_stop_session() Vitalii Dodonov
2026-11-21 16:38 ` [PATCH 1/1] " Vitalii Dodonov
2026-11-21 16:50   ` sashiko-bot

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.