linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH RFC 0/3] extend hung task blocker tracking to rwsems
@ 2025-06-12  4:19 Lance Yang
  2025-06-12  4:19 ` [PATCH RFC 1/3] locking/rwsem: make owner helpers globally available Lance Yang
                   ` (3 more replies)
  0 siblings, 4 replies; 15+ messages in thread
From: Lance Yang @ 2025-06-12  4:19 UTC (permalink / raw)
  To: akpm
  Cc: zi.li, anna.schumaker, boqun.feng, joel.granados, jstultz,
	kent.overstreet, leonylgao, linux-kernel, longman, mhiramat,
	mingo, mingzhe.yang, peterz, rostedt, senozhatsky, tfiga, will

Hi all,

Inspired by mutex blocker tracking[1], and having already extended it to
semaphores, let's now add support for reader-writer semaphores (rwsems).

The approach is simple: when a task enters TASK_UNINTERRUPTIBLE while
waiting for an rwsem, we just call hung_task_set_blocker(). The hung task
detector can then query the rwsem's owner to identify the lock holder.

Tracking works reliably for writers, as there can only be a single writer
holding the lock, and its task struct is stored in the owner field.

The main challenge lies with readers. The owner field points to only one
of many concurrent readers, so we might lose track of the blocker if that
specific reader unlocks, even while others remain. This is not a
significant issue, however. In practice, long-lasting lock contention is
almost always caused by a writer. Therefore, reliably tracking the writer
is the primary goal of this patch series ;)

With this change, the hung task detector can now show blocker task's info
like below:

[Thu Jun 12 11:01:33 2025] INFO: task rw_sem_thread2:36526 blocked for more than 122 seconds.
[Thu Jun 12 11:01:33 2025]       Tainted: G S         O        6.16.0-rc1 #1
[Thu Jun 12 11:01:33 2025] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[Thu Jun 12 11:01:33 2025] task:rw_sem_thread2  state:D stack:0     pid:36526 tgid:36526 ppid:2      task_flags:0x208040 flags:0x00004000
[Thu Jun 12 11:01:33 2025] Call Trace:
[Thu Jun 12 11:01:33 2025]  <TASK>
[Thu Jun 12 11:01:33 2025]  __schedule+0x7c7/0x1930
[Thu Jun 12 11:01:33 2025]  ? __pfx___schedule+0x10/0x10
[Thu Jun 12 11:01:33 2025]  ? _raw_spin_lock_irq+0x8a/0xe0
[Thu Jun 12 11:01:33 2025]  ? __pfx__raw_spin_lock_irq+0x10/0x10
[Thu Jun 12 11:01:33 2025]  schedule+0x6a/0x180
[Thu Jun 12 11:01:33 2025]  schedule_preempt_disabled+0x15/0x30
[Thu Jun 12 11:01:33 2025]  rwsem_down_write_slowpath+0x447/0x1090
[Thu Jun 12 11:01:33 2025]  ? __pfx_rwsem_down_write_slowpath+0x10/0x10
[Thu Jun 12 11:01:33 2025]  ? __pfx___schedule+0x10/0x10
[Thu Jun 12 11:01:33 2025]  ? __pfx___might_resched+0x10/0x10
[Thu Jun 12 11:01:33 2025]  ? __pfx_thread2_func+0x10/0x10 [rw_sem_test_2]
[Thu Jun 12 11:01:33 2025]  down_write+0x125/0x140
[Thu Jun 12 11:01:33 2025]  ? __pfx_down_write+0x10/0x10
[Thu Jun 12 11:01:33 2025]  ? msleep+0x91/0xf0
[Thu Jun 12 11:01:33 2025]  ? __raw_spin_lock_irqsave+0x8c/0xf0
[Thu Jun 12 11:01:33 2025]  thread2_func+0x37/0x70 [rw_sem_test_2]
[Thu Jun 12 11:01:33 2025]  kthread+0x39f/0x750
[Thu Jun 12 11:01:33 2025]  ? __pfx_kthread+0x10/0x10
[Thu Jun 12 11:01:33 2025]  ? __pfx__raw_spin_lock_irq+0x10/0x10
[Thu Jun 12 11:01:33 2025]  ? __pfx_kthread+0x10/0x10
[Thu Jun 12 11:01:33 2025]  ret_from_fork+0x25d/0x320
[Thu Jun 12 11:01:33 2025]  ? __pfx_kthread+0x10/0x10
[Thu Jun 12 11:01:33 2025]  ret_from_fork_asm+0x1a/0x30
[Thu Jun 12 11:01:33 2025]  </TASK>
[Thu Jun 12 11:01:33 2025] INFO: task rw_sem_thread2:36526 <writer> blocked on an rw-semaphore likely owned by task rw_sem_thread1:36525 <writer>
[Thu Jun 12 11:01:33 2025] task:rw_sem_thread1  state:S stack:0     pid:36525 tgid:36525 ppid:2      task_flags:0x208040 flags:0x00004000
[Thu Jun 12 11:01:33 2025] Call Trace:
[Thu Jun 12 11:01:33 2025]  <TASK>
[Thu Jun 12 11:01:33 2025]  __schedule+0x7c7/0x1930
[Thu Jun 12 11:01:33 2025]  ? __pfx___schedule+0x10/0x10
[Thu Jun 12 11:01:33 2025]  ? __mod_timer+0x304/0xa80
[Thu Jun 12 11:01:33 2025]  ? irq_work_queue+0x6a/0xa0
[Thu Jun 12 11:01:33 2025]  ? __pfx_vprintk_emit+0x10/0x10
[Thu Jun 12 11:01:33 2025]  schedule+0x6a/0x180
[Thu Jun 12 11:01:33 2025]  schedule_timeout+0xfb/0x230
[Thu Jun 12 11:01:33 2025]  ? __pfx_schedule_timeout+0x10/0x10
[Thu Jun 12 11:01:33 2025]  ? __pfx_process_timeout+0x10/0x10
[Thu Jun 12 11:01:33 2025]  ? down_write+0xc4/0x140
[Thu Jun 12 11:01:33 2025]  msleep_interruptible+0xbe/0x150
[Thu Jun 12 11:01:33 2025]  ? __pfx_thread1_func+0x10/0x10 [rw_sem_test_2]
[Thu Jun 12 11:01:33 2025]  thread1_func+0x37/0x60 [rw_sem_test_2]
[Thu Jun 12 11:01:33 2025]  kthread+0x39f/0x750
[Thu Jun 12 11:01:33 2025]  ? __pfx_kthread+0x10/0x10
[Thu Jun 12 11:01:33 2025]  ? __pfx__raw_spin_lock_irq+0x10/0x10
[Thu Jun 12 11:01:33 2025]  ? __pfx_kthread+0x10/0x10
[Thu Jun 12 11:01:33 2025]  ret_from_fork+0x25d/0x320
[Thu Jun 12 11:01:33 2025]  ? __pfx_kthread+0x10/0x10
[Thu Jun 12 11:01:33 2025]  ret_from_fork_asm+0x1a/0x30
[Thu Jun 12 11:01:33 2025]  </TASK>

[1] https://lore.kernel.org/all/174046694331.2194069.15472952050240807469.stgit@mhiramat.tok.corp.google.com/

Thanks,
Lance

Lance Yang (3):
  locking/rwsem: make owner helpers globally available
  locking/rwsem: clear reader-owner on unlock to reduce false positives
  hung_task: extend hung task blocker tracking to rwsems

 include/linux/hung_task.h | 18 +++++++++---------
 include/linux/rwsem.h     | 12 ++++++++++++
 kernel/hung_task.c        | 29 +++++++++++++++++++++++++----
 kernel/locking/rwsem.c    | 31 +++++++++++++++++++++++--------
 4 files changed, 69 insertions(+), 21 deletions(-)

-- 
2.49.0


^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2025-06-24  7:38 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-06-12  4:19 [PATCH RFC 0/3] extend hung task blocker tracking to rwsems Lance Yang
2025-06-12  4:19 ` [PATCH RFC 1/3] locking/rwsem: make owner helpers globally available Lance Yang
2025-06-24  0:17   ` Masami Hiramatsu
2025-06-24  1:35     ` Lance Yang
2025-06-12  4:19 ` [PATCH RFC 2/3] locking/rwsem: clear reader-owner on unlock to reduce false positives Lance Yang
2025-06-24  0:26   ` Masami Hiramatsu
2025-06-24  1:44     ` Lance Yang
2025-06-24  3:53       ` Masami Hiramatsu
2025-06-24  5:02         ` Lance Yang
2025-06-24  6:13           ` Masami Hiramatsu
2025-06-24  7:38             ` Lance Yang
2025-06-12  4:19 ` [PATCH RFC 3/3] hung_task: extend hung task blocker tracking to rwsems Lance Yang
2025-06-24  3:57   ` Masami Hiramatsu
2025-06-24  4:25     ` Lance Yang
2025-06-23 12:33 ` [PATCH RFC 0/3] " Lance Yang

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).