* [PATCH] scsi: libiscsi_tcp: hold back_lock across the copy of a Data-In
@ 2026-11-20 14:52 Yehyeong Lee
2026-11-20 15:15 ` sashiko-bot
0 siblings, 1 reply; 2+ messages in thread
From: Yehyeong Lee @ 2026-11-20 14:52 UTC (permalink / raw)
To: Lee Duncan, Chris Leech, Mike Christie, Martin K . Petersen,
James E . J . Bottomley
Cc: linux-scsi, linux-kernel
iscsi_tcp_hdr_dissect() aims the receive segment at a command's
scatterlist and drops back_lock, and the copy itself then runs once per
skb in iscsi_tcp_recv_skb() with nothing held:
rc = iscsi_segment_seek_sg(&tcp_conn->in.segment, sdb->table.sgl,
sdb->table.nents, ...);
spin_unlock(&session->back_lock);
The SCSI error handler finishes a task while such a copy is pending, and
it leaves the receive side running on purpose while it does.
iscsi_eh_abort() says so itself:
/*
* we do not stop the recv side because targets have been
* good and have never sent us a successful tmf response
* then sent more data for the cmd.
*/
iscsi_eh_device_reset() and iscsi_eh_target_reset() do the same, calling
iscsi_suspend_tx() and not iscsi_suspend_rx() before they fail the tasks.
fail_scsi_task() and fail_scsi_tasks() complete the task the receive path
is copying into, iscsi_free_task() hands the command back with
scsi_done(), and its scatterlist returns to the sg mempool while
tcp_conn->in.segment still points into it.
Looking the command up again before each round of copying would not help.
The error handler can fail the task between the lookup and the copy, and
a lookup cannot tell the command apart from its successor either:
task->itt is the index of a preallocated slot and is fixed for the life
of the session, so the next command to take that slot carries the same
ITT and the same session age, and iscsi_itt_to_ctask() returns it as
running while segment->sg still points into the freed scatterlist.
So let the lock that releases a command also guard the copy into it.
iscsi_tcp_cleanup_task() already runs from iscsi_free_task() with
back_lock held, immediately before the command is handed back, so
remember which command a Data-In is being copied into and drop that
pointer there. iscsi_tcp_recv_skb() then holds back_lock for the whole
of the copy and refuses the rest of the PDU with ISCSI_ERR_BAD_ITT if the
pointer is gone, as it does one step earlier. Cleanup and copy now
exclude each other, and a slot cannot be reused before the cleanup that
invalidated the pointer has run. Only a Data-In pays for this: it is the
only receive segment aimed at a scatterlist, every other one is a linear
buffer that belongs to the connection.
The lock is dropped again before segment->done(), which for a Data-In is
iscsi_tcp_process_data_in() and takes back_lock itself. Holding it over
the copy is bounded by one skb and adds no sleeping call; the copy maps
its pages with kmap_atomic(). It is taken with _bh because the receive
path is not only a softirq: iscsi_sw_tcp_recv_data_work() runs it from a
work item with softirqs enabled when iscsi_recv_from_iscsi_q is set, and
every process context that takes back_lock in libiscsi takes it that way.
A target that holds part of a Data-In until the command's LUN has been
reset, and then sends the rest, gets the copy to run through the freed
scatterlist and KASAN to report it:
BUG: KASAN: slab-use-after-free in iscsi_tcp_segment_done+0xe4b/0x1490
Read of size 4 at addr ffff88800d44cc8c by task swapper/0/0
CPU: 0 UID: 0 PID: 0 Comm: swapper/0 Tainted: G B 7.4.0-rc2-00478-ga189f19ef663 #10 PREEMPT(lazy)
Tainted: [B]=BAD_PAGE
Hardware name: QEMU Ubuntu 24.04 PC v2 (i440FX + PIIX, arch_caps fix, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
Call Trace:
<IRQ>
dump_stack_lvl+0x53/0x70
print_report+0xd0/0x630
kasan_report+0xce/0x100
iscsi_tcp_segment_done+0xe4b/0x1490
iscsi_tcp_recv_skb+0x924/0x15f0
iscsi_sw_tcp_recv+0x12f/0x390
__tcp_read_sock+0x1ab/0x810
iscsi_sw_tcp_data_ready+0x18b/0x510
tcp_data_queue+0x1f2d/0x4b70
tcp_rcv_established+0x8a8/0x3990
tcp_v4_do_rcv+0x45b/0x970
tcp_v4_rcv+0x223d/0x3b90
ip_protocol_deliver_rcu+0x61/0x2e0
ip_local_deliver_finish+0x329/0x4b0
ip_local_deliver+0x18f/0x2d0
ip_sublist_rcv_finish+0x138/0x1b0
ip_sublist_rcv+0x3ae/0x670
ip_list_rcv+0x266/0x380
__netif_receive_skb_list_core+0x5c3/0x860
netif_receive_skb_list_internal+0x553/0xbb0
gro_complete.constprop.0+0x3ed/0x5c0
dev_gro_receive+0xa31/0x23c0
gro_receive_skb+0x279/0x960
e1000_clean_rx_irq+0x602/0x1160
e1000_clean+0x7eb/0x25a0
__napi_poll+0x9c/0x520
net_rx_action+0x8f1/0xfb0
handle_softirqs+0x1b3/0x600
__irq_exit_rcu+0xfc/0x160
common_interrupt+0x7f/0x90
</IRQ>
Allocated by task 75:
kasan_save_stack+0x33/0x60
kasan_save_track+0x14/0x30
__kasan_slab_alloc+0x6e/0x70
kmem_cache_alloc_noprof+0x10e/0x340
mempool_alloc_noprof+0xcd/0x180
__sg_alloc_table+0x23e/0x410
sg_alloc_table_chained+0x47/0x180
scsi_alloc_sgtables+0x17f/0xc50
sd_init_command+0x20e/0x2e70
scsi_queue_rq+0x1070/0x2f60
blk_mq_dispatch_rq_list+0x390/0x2100
__blk_mq_sched_dispatch_requests+0xa7a/0x1550
blk_mq_sched_dispatch_requests+0xa7/0x140
blk_mq_run_work_fn+0x105/0x150
process_one_work+0x697/0x1060
worker_thread+0x45b/0xd10
kthread+0x2c8/0x3b0
ret_from_fork+0x36e/0x5a0
ret_from_fork_asm+0x1a/0x30
Freed by task 130:
kasan_save_stack+0x33/0x60
kasan_save_track+0x14/0x30
kasan_save_free_info+0x3b/0x60
__kasan_slab_free+0x43/0x70
kmem_cache_free+0xba/0x3a0
mempool_free+0xbd/0x110
__sg_free_table+0xa8/0x170
scsi_end_request+0x329/0x5d0
scsi_io_completion+0x4b0/0xda0
blk_complete_reqs+0xa4/0xe0
handle_softirqs+0x1b3/0x600
do_softirq+0x3d/0x60
__local_bh_enable_ip+0x6a/0x70
iscsi_eh_device_reset+0x7f1/0xa10
scsi_ioctl_reset+0x3a8/0x5b0
scsi_ioctl+0x7ac/0x1360
blkdev_ioctl+0x385/0x520
__x64_sys_ioctl+0x134/0x1c0
do_syscall_64+0xdf/0x4c0
entry_SYSCALL_64_after_hwframe+0x77/0x7f
The buggy address belongs to the object at ffff88800d44cc80
which belongs to the cache sgpool-16 of size 512
The buggy address is located 12 bytes inside of
freed 512-byte region [ffff88800d44cc80, ffff88800d44ce80)
Fixes: da32dd681f7a ("[SCSI] iscsi_tcp: rewrite recv path")
Cc: stable@vger.kernel.org
Signed-off-by: Yehyeong Lee <yhlee@isslab.korea.ac.kr>
---
Reproduced against a userspace target reached through a proxy that holds
a 64 KiB read's whole Data-In, lets a LUN reset through, and then sends
the header with the first 2048 bytes and the remaining 63488 bytes after
a gap. A test-only delay parks the receive path for 20 ms once it has
reached the copy, and the reset path waits for that park before it fails
the tasks, so that the command is given back inside the window. The
receive softirq is kept on one cpu and the reader and the reset on
others. Every boot is a fresh one, under KASAN.
Measured on top of the series named below. Without this patch all 3
boots report, 51 reports each, of which 4 name the freed scatterlist and
the rest are the copy writing through it and into the freed data pages.
With it none of 5 do: the reset waits on back_lock until the copy has
finished, the 63488 bytes land in the command's own buffer, and the reset
then fails the command as it asked to. A 64 KiB read through the same
proxy with the injection turned off is unaffected in both arms. Ten
32 MiB direct reads, on kernels built without the test delays and without
KASAN, take 1.23 to 1.39 s without the patch and 1.23 to 1.34 s with it,
three boots of three measurements each per arm.
The report above is the first one that names the scatterlist. The boot's
first report is the same copy writing into a freed data page, which is
also what set the taint.
This is a follow-up to "scsi: libiscsi_tcp: do not accept a Data-In for a
finished command", which refused a Data-In whose command had already gone
but argued that while a copy is pending only the receive path can finish
the task. The error handler does too, which is what this patch answers.
https://lore.kernel.org/r/20261117101500.1894608-5-yhlee@isslab.korea.ac.kr
Sashiko raised the same case on that patch:
https://lore.kernel.org/all/sashiko-outbox-218654@kernel.org/
Applies on its own, and on top of that series, to mkp-scsi/for-next
(08fec21faae4), origin/master (65e4fd942f72) and next/master
(adcad7f4e6332c).
drivers/scsi/libiscsi_tcp.c | 37 +++++++++++++++++++++++++++++++++++++
include/scsi/libiscsi_tcp.h | 7 +++++++
2 files changed, 44 insertions(+)
diff --git a/drivers/scsi/libiscsi_tcp.c b/drivers/scsi/libiscsi_tcp.c
index eeb853c6b66d49..79489793b076d7 100644
--- a/drivers/scsi/libiscsi_tcp.c
+++ b/drivers/scsi/libiscsi_tcp.c
@@ -442,9 +442,17 @@ iscsi_tcp_data_recv_prep(struct iscsi_tcp_conn *tcp_conn)
*/
void iscsi_tcp_cleanup_task(struct iscsi_task *task)
{
+ struct iscsi_tcp_conn *tcp_conn = task->conn->dd_data;
struct iscsi_tcp_task *tcp_task = task->dd_data;
struct iscsi_r2t_info *r2t;
+ /*
+ * Runs under back_lock just before the command is given back, and
+ * the receive path copies into its scatterlist under the same lock.
+ */
+ if (tcp_conn->in.datain_task == task)
+ tcp_conn->in.datain_task = NULL;
+
/* nothing to do for mgmt */
if (!task->sc)
return;
@@ -745,6 +753,7 @@ iscsi_tcp_hdr_dissect(struct iscsi_conn *conn, struct iscsi_hdr *hdr)
tcp_task->data_offset,
tcp_conn->in.datalen);
task->last_xfer = jiffies;
+ tcp_conn->in.datain_task = task;
rc = iscsi_segment_seek_sg(&tcp_conn->in.segment,
sdb->table.sgl,
sdb->table.nents,
@@ -908,8 +917,10 @@ int iscsi_tcp_recv_skb(struct iscsi_conn *conn, struct sk_buff *skb,
{
struct iscsi_tcp_conn *tcp_conn = conn->dd_data;
struct iscsi_segment *segment = &tcp_conn->in.segment;
+ struct iscsi_session *session = conn->session;
struct skb_seq_state seq;
unsigned int consumed = 0;
+ bool copy_locked = false;
int rc = 0;
ISCSI_DBG_TCP(conn, "in %d bytes\n", skb->len - offset);
@@ -926,6 +937,26 @@ int iscsi_tcp_recv_skb(struct iscsi_conn *conn, struct sk_buff *skb,
return 0;
}
+ /*
+ * Hold back_lock across a Data-In copy so the command cannot be
+ * given back while it runs. _bh: this also runs from a work item.
+ */
+ if (segment->sg) {
+ struct iscsi_task *task;
+
+ spin_lock_bh(&session->back_lock);
+ task = tcp_conn->in.datain_task;
+ if (!task || task->state != ISCSI_TASK_RUNNING) {
+ spin_unlock_bh(&session->back_lock);
+ iscsi_conn_printk(KERN_ERR, conn,
+ "Data-In for a command that is gone\n");
+ *status = ISCSI_TCP_CONN_ERR;
+ iscsi_conn_failure(conn, ISCSI_ERR_BAD_ITT);
+ return 0;
+ }
+ copy_locked = true;
+ }
+
if (offloaded) {
segment->total_copied = segment->total_size;
goto segment_done;
@@ -958,6 +989,10 @@ int iscsi_tcp_recv_skb(struct iscsi_conn *conn, struct sk_buff *skb,
}
segment_done:
+ if (copy_locked) {
+ spin_unlock_bh(&session->back_lock);
+ copy_locked = false;
+ }
*status = ISCSI_TCP_SEGMENT_DONE;
ISCSI_DBG_TCP(conn, "segment done\n");
rc = segment->done(tcp_conn, segment);
@@ -970,6 +1005,8 @@ int iscsi_tcp_recv_skb(struct iscsi_conn *conn, struct sk_buff *skb,
/* The done() functions sets up the next segment. */
skb_done:
+ if (copy_locked)
+ spin_unlock_bh(&session->back_lock);
conn->rxdata_octets += consumed;
return consumed;
}
diff --git a/include/scsi/libiscsi_tcp.h b/include/scsi/libiscsi_tcp.h
index ef53d4bea28a07..70967bd936e0bd 100644
--- a/include/scsi/libiscsi_tcp.h
+++ b/include/scsi/libiscsi_tcp.h
@@ -45,6 +45,13 @@ struct iscsi_tcp_recv {
struct iscsi_hdr *hdr;
struct iscsi_segment segment;
+ /*
+ * The command a Data-In is being copied into. Set when the segment
+ * is aimed at its scatterlist and cleared when it is given back, both
+ * with the session back_lock held.
+ */
+ struct iscsi_task *datain_task;
+
/* Allocate buffer for BHS + AHS */
uint32_t hdr_buf[64];
--
2.43.0
^ permalink raw reply related [flat|nested] 2+ messages in thread* Re: [PATCH] scsi: libiscsi_tcp: hold back_lock across the copy of a Data-In
2026-11-20 14:52 [PATCH] scsi: libiscsi_tcp: hold back_lock across the copy of a Data-In Yehyeong Lee
@ 2026-11-20 15:15 ` sashiko-bot
0 siblings, 0 replies; 2+ messages in thread
From: sashiko-bot @ 2026-11-20 15:15 UTC (permalink / raw)
To: Yehyeong Lee; +Cc: linux-scsi
Thank you for your contribution! Sashiko AI review found 4 potential
issues to consider:
- [High] scsi: libiscsi_tcp: race on ITT slot reuse in
iscsi_tcp_process_data_in()
- [High] scsi: libiscsi_tcp: kmap_atomic() held across back_lock unlock
at skb_done
- [High] scsi: libiscsi_tcp: missing ISCSI_TASK_RUNNING check in
iscsi_tcp_hdr_dissect()
- [Low] scsi: libiscsi_tcp: Clang context-analysis warning in
iscsi_tcp_recv_skb()
--
commit 137946721880763b95233f1c2f065be24dc7e099
Author: Yehyeong Lee <yhlee@isslab.korea.ac.kr>
scsi: libiscsi_tcp: hold back_lock across the copy of a Data-In
This commit tracks the active Data-In task in tcp_conn->in.datain_task and
holds session->back_lock across the Data-In scatterlist copy in
iscsi_tcp_recv_skb() so that error handlers cannot free the command's
scatterlist while the copy is in progress.
> diff --git a/drivers/scsi/libiscsi_tcp.c b/drivers/scsi/libiscsi_tcp.c
> index 20ec3ba45067f..50c58e938321e 100644
> --- a/drivers/scsi/libiscsi_tcp.c
> +++ b/drivers/scsi/libiscsi_tcp.c
[ ... ]
> @@ -744,6 +752,7 @@ iscsi_tcp_hdr_dissect(struct iscsi_conn *conn, struct iscsi_hdr *hdr)
> tcp_task->data_offset,
> tcp_conn->in.datalen);
> task->last_xfer = jiffies;
> + tcp_conn->in.datain_task = task;
> rc = iscsi_segment_seek_sg(&tcp_conn->in.segment,
> sdb->table.sgl,
> sdb->table.nents,
[Severity: High]
Should iscsi_tcp_hdr_dissect() verify that task->state ==
ISCSI_TASK_RUNNING before accepting a Data-In header and setting
tcp_conn->in.datain_task?
iscsi_itt_to_ctask() only checks task->sc and session->age, so it also
returns newly queued commands in ISCSI_TASK_PENDING on conn->cmdqueue. If
a stray Data-In header arrives for a slot occupied by a pending command and
iscsi_prep_scsi_cmd_pdu() transitions that command to ISCSI_TASK_RUNNING
before the payload skb arrives, iscsi_tcp_recv_skb() sees the matching
datain_task in ISCSI_TASK_RUNNING and copies the stale Data-In payload into
the new command's scatterlist. In addition, a zero-length Data-In with
ISCSI_FLAG_DATA_STATUS calls __iscsi_complete_pdu() right below without
ever reaching iscsi_tcp_recv_skb(), freeing an ISCSI_TASK_PENDING task
while it is still linked on conn->cmdqueue and causing a NULL task->sc
dereference in iscsi_prep_scsi_cmd_pdu().
> @@ -907,8 +916,10 @@ int iscsi_tcp_recv_skb(struct iscsi_conn *conn, struct sk_buff *skb,
> {
> struct iscsi_tcp_conn *tcp_conn = conn->dd_data;
> struct iscsi_segment *segment = &tcp_conn->in.segment;
> + struct iscsi_session *session = conn->session;
> struct skb_seq_state seq;
> unsigned int consumed = 0;
> + bool copy_locked = false;
> int rc = 0;
>
> ISCSI_DBG_TCP(conn, "in %d bytes\n", skb->len - offset);
> @@ -925,6 +936,26 @@ int iscsi_tcp_recv_skb(struct iscsi_conn *conn, struct sk_buff *skb,
> return 0;
> }
>
> + /*
> + * Hold back_lock across a Data-In copy so the command cannot be
> + * given back while it runs. _bh: this also runs from a work item.
> + */
> + if (segment->sg) {
> + struct iscsi_task *task;
> +
> + spin_lock_bh(&session->back_lock);
> + task = tcp_conn->in.datain_task;
> + if (!task || task->state != ISCSI_TASK_RUNNING) {
> + spin_unlock_bh(&session->back_lock);
> + iscsi_conn_printk(KERN_ERR, conn,
> + "Data-In for a command that is gone\n");
> + *status = ISCSI_TCP_CONN_ERR;
> + iscsi_conn_failure(conn, ISCSI_ERR_BAD_ITT);
> + return 0;
> + }
> + copy_locked = true;
> + }
> +
> if (offloaded) {
> segment->total_copied = segment->total_size;
> goto segment_done;
> @@ -957,6 +988,10 @@ int iscsi_tcp_recv_skb(struct iscsi_conn *conn, struct sk_buff *skb,
> }
>
> segment_done:
> + if (copy_locked) {
> + spin_unlock_bh(&session->back_lock);
> + copy_locked = false;
> + }
> *status = ISCSI_TCP_SEGMENT_DONE;
> ISCSI_DBG_TCP(conn, "segment done\n");
> rc = segment->done(tcp_conn, segment);
[Severity: High]
Can dropping session->back_lock before segment->done() let
iscsi_tcp_process_data_in() complete a newly queued command that reuses
the freed ITT slot?
CPU0 (softirq) CPU1 (EH / queuecommand)
iscsi_tcp_recv_skb()
spin_lock_bh(&back_lock)
iscsi_tcp_segment_recv()
spin_unlock_bh(&back_lock)
iscsi_eh_device_reset()
fail_scsi_tasks()
spin_lock_bh(&back_lock)
iscsi_free_task()
datain_task = NULL
spin_unlock_bh(&back_lock)
iscsi_queuecommand()
/* reuses the same ITT slot */
segment->done()
iscsi_tcp_process_data_in()
iscsi_complete_pdu()
spin_lock(&back_lock)
iscsi_itt_to_ctask()
iscsi_complete_task() <-- completes the new command
When ISCSI_FLAG_DATA_STATUS is set on the Data-In header,
iscsi_tcp_process_data_in() calls iscsi_complete_pdu() without checking
tcp_conn->in.datain_task, so __iscsi_complete_pdu() looks up hdr->itt
afresh via iscsi_itt_to_ctask(). If the new command is still in
ISCSI_TASK_PENDING on conn->cmdqueue, completing it frees the task and sets
task->sc = NULL while it stays linked on conn->cmdqueue, causing a NULL
pointer dereference in iscsi_prep_scsi_cmd_pdu(). Also, when software data
digests are enabled, iscsi_tcp_segment_splice_digest() sets
segment->sg = NULL before receiving the 4-byte digest, so a subsequent skb
carrying the digest skips the if (segment->sg) check and reaches
segment->done() after the task was already freed.
> @@ -969,6 +1004,8 @@ int iscsi_tcp_recv_skb(struct iscsi_conn *conn, struct sk_buff *skb,
> /* The done() functions sets up the next segment. */
>
> skb_done:
> + if (copy_locked)
> + spin_unlock_bh(&session->back_lock);
> conn->rxdata_octets += consumed;
> return consumed;
> }
[Severity: High]
Does unlocking session->back_lock at skb_done leave segment->sg_mapped
mapped with kmap_atomic()?
When an skb ends before the Data-In payload is finished,
iscsi_tcp_segment_done() calls iscsi_tcp_segment_map(), which calls
kmap_atomic(sg_page(sg)), and the unmap does not happen until
iscsi_sw_tcp_recv_data() finishes tcp_read_sock():
drivers/scsi/libiscsi_tcp.c:iscsi_tcp_segment_done() {
...
segment->copied += copied;
if (segment->copied < segment->size) {
iscsi_tcp_segment_map(segment, recv);
return 0;
}
...
}
On a multi-skb Data-In, skb_done releases session->back_lock while
kmap_atomic() is still active, allowing an error handler on another CPU to
free the command and its scatterlist pages while segment->sg_mapped remains
mapped, and the next skb in tcp_read_sock() then calls
spin_lock_bh(&session->back_lock) while inside kmap_atomic() /
pagefault_disable() (which violates sleeping spinlock rules on
CONFIG_PREEMPT_RT).
[Severity: Low]
Does the conditional locking across copy_locked trigger a Clang
-Wthread-safety-analysis warning when CONFIG_WARN_CONTEXT_ANALYSIS=y?
drivers/scsi/Makefile enables CONTEXT_ANALYSIS := y, and Clang's thread
safety analyzer does not track boolean lock flags across basic blocks.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261120145243.3124991-1-yhlee@isslab.korea.ac.kr?part=1
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2026-11-20 15:15 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-20 14:52 [PATCH] scsi: libiscsi_tcp: hold back_lock across the copy of a Data-In Yehyeong Lee
2026-11-20 15:15 ` sashiko-bot
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox