[ 21.763347] WARNING: possible irq lock inversion dependency detected [ 21.763349] 4.16.0-11958-g16e205cf42da #168 Not tainted [ 21.763350] -------------------------------------------------------- [ 21.763352] kworker/1:1H/129 just changed the state of lock: [ 21.763353] 000000007d3db786 (&rq->gstate_seq){+.+.}, at: null_queue_rq+0x70/0xc8 [null_blk] [ 21.763360] but this lock was taken by another, SOFTIRQ-safe lock in the past: [ 21.763361] (&(&dq->lock)->rlock){..-.} [ 21.763363] and interrupts could create inverse lock ordering between them. [ 21.763366] other info that might help us debug this: [ 21.763368] Possible interrupt unsafe locking scenario: [ 21.763369] CPU0 CPU1 [ 21.763370] ---- ---- [ 21.763371] lock(&rq->gstate_seq); [ 21.763374] local_irq_disable(); [ 21.763375] lock(&(&dq->lock)->rlock); [ 21.763377] lock(&rq->gstate_seq); [ 21.763379] [ 21.763380] lock(&(&dq->lock)->rlock); [ 21.763382] *** DEADLOCK *** [ 21.763385] 3 locks held by kworker/1:1H/129: [ 21.763399] #0: 000000003a6e153c ((wq_completion)"kblockd"){+.+.}, at: process_one_work+0x1a0/0x6f0 [ 21.763406] #1: 00000000b52029e8 ((work_completion)(&(&hctx->run_work)->work)){+.+.}, at: process_one_work+0x1a0/0x6f0 [ 21.763411] #2: 000000001fde8b5d (rcu_read_lock){....}, at: hctx_lock+0x30/0x110 [ 21.763416] the shortest dependencies between 2nd lock and 1st lock: [ 21.763418] -> (&(&dq->lock)->rlock){..-.} ops: 55959128899584 { [ 21.763422] IN-SOFTIRQ-W at: [ 21.763426] __lock_acquire+0x4e6/0x13f0 [ 21.763428] lock_acquire+0xf6/0x290 [ 21.763431] _raw_spin_lock_irq+0x6a/0xb0 [ 21.763434] dasd_block_tasklet+0xf8/0x400 [ 21.763436] tasklet_action_common.isra.5+0x82/0x110 [ 21.763438] __do_softirq+0xd2/0x5d8 [ 21.763441] irq_exit+0x132/0x138 [ 21.763443] do_IRQ+0xaa/0xf0 [ 21.763445] io_int_handler+0x138/0x31c [ 21.763447] enabled_wait+0x6a/0x138 [ 21.763449] enabled_wait+0x52/0x138 [ 21.763450] arch_cpu_idle+0x32/0x50 [ 21.763452] default_idle_call+0x3e/0x68 [ 21.763454] do_idle+0xea/0x1c0 [ 21.763456] cpu_startup_entry+0x3e/0x48 [ 21.763458] smp_start_secondary+0x13e/0x158 [ 21.763460] restart_int_handler+0x5a/0x70 [ 21.763462] (null) [ 21.763463] INITIAL USE at: [ 21.763465] __lock_acquire+0x26c/0x13f0 [ 21.763467] lock_acquire+0xf6/0x290 [ 21.763469] _raw_spin_lock_irq+0x6a/0xb0 [ 21.763472] do_dasd_request+0x50/0x470 [ 21.763474] __blk_mq_try_issue_directly+0x1ca/0x208 [ 21.763476] blk_mq_try_issue_directly+0x5c/0x108 [ 21.763478] blk_mq_make_request+0x4b8/0xa20 [ 21.763480] generic_make_request+0xea/0x2c8 [ 21.763482] submit_bio+0xa4/0x1a0 [ 21.763485] submit_bh_wbc+0x1c4/0x218 [ 21.763487] block_read_full_page+0x366/0x408 [ 21.763491] do_read_cache_page+0x1b2/0x3d0 [ 21.763493] read_cache_page+0x30/0x40 [ 21.763497] read_dev_sector+0x58/0xe8 [ 21.763498] read_lba.isra.0+0x12a/0x1b8 [ 21.763500] efi_partition+0x198/0x660 [ 21.763501] check_partition+0x15c/0x2b8 [ 21.763503] rescan_partitions+0xea/0x3f8 [ 21.763505] blkdev_reread_part+0x44/0x68 [ 21.763507] dasd_scan_partitions+0x90/0x148 [ 21.763509] dasd_change_state+0x8cc/0xb28 [ 21.763511] do_kick_device+0x4c/0x98 [ 21.763513] process_one_work+0x25c/0x6f0 [ 21.763515] worker_thread+0x52/0x400 [ 21.763518] kthread+0x15a/0x178 [ 21.763520] kernel_thread_starter+0x6/0xc [ 21.763522] kernel_thread_starter+0x0/0xc [ 21.763523] } [ 21.763527] ... key at: [<0000000001bd0e08>] __key.43428+0x0/0x8 [ 21.763528] ... acquired at: [ 21.763530] __lock_acquire+0x130a/0x13f0 [ 21.763531] lock_acquire+0xf6/0x290 [ 21.763533] blk_mq_start_request+0xde/0x300 [ 21.763534] do_dasd_request+0x17a/0x470 [ 21.763536] __blk_mq_try_issue_directly+0x1ca/0x208 [ 21.763538] blk_mq_try_issue_directly+0x5c/0x108 [ 21.763539] blk_mq_make_request+0x4b8/0xa20 [ 21.763541] generic_make_request+0xea/0x2c8 [ 21.763543] submit_bio+0xa4/0x1a0 [ 21.763544] submit_bh_wbc+0x1c4/0x218 [ 21.763546] block_read_full_page+0x366/0x408 [ 21.763548] do_read_cache_page+0x1b2/0x3d0 [ 21.763550] read_cache_page+0x30/0x40 [ 21.763552] read_dev_sector+0x58/0xe8 [ 21.763553] read_lba.isra.0+0x12a/0x1b8 [ 21.763555] efi_partition+0x198/0x660 [ 21.763556] check_partition+0x15c/0x2b8 [ 21.763558] rescan_partitions+0xea/0x3f8 [ 21.763560] blkdev_reread_part+0x44/0x68 [ 21.763561] dasd_scan_partitions+0x90/0x148 [ 21.763563] dasd_change_state+0x8cc/0xb28 [ 21.763564] do_kick_device+0x4c/0x98 [ 21.763566] process_one_work+0x25c/0x6f0 [ 21.763568] worker_thread+0x52/0x400 [ 21.763570] kthread+0x15a/0x178 [ 21.763571] kernel_thread_starter+0x6/0xc [ 21.763573] kernel_thread_starter+0x0/0xc [ 21.763575] -> (&rq->gstate_seq){+.+.} ops: 17862768984064 { [ 21.763579] HARDIRQ-ON-W at: [ 21.763581] __lock_acquire+0x3da/0x13f0 [ 21.763583] lock_acquire+0xf6/0x290 [ 21.763585] blk_mq_start_request+0xde/0x300 [ 21.763588] null_queue_rq+0x70/0xc8 [null_blk] [ 21.763590] blk_mq_dispatch_rq_list+0x186/0x678 [ 21.763592] blk_mq_do_dispatch_sched+0x90/0x128 [ 21.763593] blk_mq_sched_dispatch_requests+0x168/0x200 [ 21.763595] __blk_mq_run_hw_queue+0xd0/0x128 [ 21.763597] process_one_work+0x25c/0x6f0 [ 21.763599] worker_thread+0x52/0x400 [ 21.763601] kthread+0x15a/0x178 [ 21.763602] kernel_thread_starter+0x6/0xc [ 21.763604] kernel_thread_starter+0x0/0xc [ 21.763606] SOFTIRQ-ON-W at: [ 21.763608] __lock_acquire+0x3fc/0x13f0 [ 21.763609] lock_acquire+0xf6/0x290 [ 21.763611] blk_mq_start_request+0xde/0x300 [ 21.763614] null_queue_rq+0x70/0xc8 [null_blk] [ 21.763615] blk_mq_dispatch_rq_list+0x186/0x678 [ 21.763617] blk_mq_do_dispatch_sched+0x90/0x128 [ 21.763619] blk_mq_sched_dispatch_requests+0x168/0x200 [ 21.763621] __blk_mq_run_hw_queue+0xd0/0x128 [ 21.763623] process_one_work+0x25c/0x6f0 [ 21.763625] worker_thread+0x52/0x400 [ 21.763627] kthread+0x15a/0x178 [ 21.763628] kernel_thread_starter+0x6/0xc [ 21.763630] kernel_thread_starter+0x0/0xc [ 21.763631] INITIAL USE at: [ 21.763634] __lock_acquire+0x26c/0x13f0 [ 21.763635] lock_acquire+0xf6/0x290 [ 21.763637] blk_mq_start_request+0xde/0x300 [ 21.763639] do_dasd_request+0x17a/0x470 [ 21.763640] __blk_mq_try_issue_directly+0x1ca/0x208 [ 21.763642] blk_mq_try_issue_directly+0x5c/0x108 [ 21.763644] blk_mq_make_request+0x4b8/0xa20 [ 21.763646] generic_make_request+0xea/0x2c8 [ 21.763647] submit_bio+0xa4/0x1a0 [ 21.763649] submit_bh_wbc+0x1c4/0x218 [ 21.763651] block_read_full_page+0x366/0x408 [ 21.763653] do_read_cache_page+0x1b2/0x3d0 [ 21.763655] read_cache_page+0x30/0x40 [ 21.763657] read_dev_sector+0x58/0xe8 [ 21.763659] read_lba.isra.0+0x12a/0x1b8 [ 21.763660] efi_partition+0x198/0x660 [ 21.763662] check_partition+0x15c/0x2b8 [ 21.763663] rescan_partitions+0xea/0x3f8 [ 21.763666] blkdev_reread_part+0x44/0x68 [ 21.763667] dasd_scan_partitions+0x90/0x148 [ 21.763669] dasd_change_state+0x8cc/0xb28 [ 21.763671] do_kick_device+0x4c/0x98 [ 21.763673] process_one_work+0x25c/0x6f0 [ 21.763675] worker_thread+0x52/0x400 [ 21.763676] kthread+0x15a/0x178 [ 21.763678] kernel_thread_starter+0x6/0xc [ 21.763680] kernel_thread_starter+0x0/0xc [ 21.763681] } [ 21.763684] ... key at: [<0000000001bc22e8>] __key.43671+0x0/0x8 [ 21.763685] ... acquired at: [ 21.763687] check_usage_backwards+0xa2/0x140 [ 21.763688] mark_lock+0x3ee/0x6f8 [ 21.763690] __lock_acquire+0x3fc/0x13f0 [ 21.763692] lock_acquire+0xf6/0x290 [ 21.763693] blk_mq_start_request+0xde/0x300 [ 21.763696] null_queue_rq+0x70/0xc8 [null_blk] [ 21.763697] blk_mq_dispatch_rq_list+0x186/0x678 [ 21.763699] blk_mq_do_dispatch_sched+0x90/0x128 [ 21.763701] blk_mq_sched_dispatch_requests+0x168/0x200 [ 21.763702] __blk_mq_run_hw_queue+0xd0/0x128 [ 21.763704] process_one_work+0x25c/0x6f0 [ 21.763706] worker_thread+0x52/0x400 [ 21.763707] kthread+0x15a/0x178 [ 21.763709] kernel_thread_starter+0x6/0xc [ 21.763711] kernel_thread_starter+0x0/0xc [ 21.763713] stack backtrace: [ 21.763716] CPU: 1 PID: 129 Comm: kworker/1:1H Not tainted 4.16.0-11958-g16e205cf42da #168 [ 21.763718] Hardware name: IBM 3906 M04 704 (LPAR) [ 21.763720] Workqueue: kblockd blk_mq_run_work_fn [ 21.763722] Call Trace: [ 21.763725] ([<00000000001151d4>] show_stack+0x9c/0x108) [ 21.763728] [<00000000008c3e92>] dump_stack+0x9a/0xd8 [ 21.763730] [<00000000001b8f02>] print_irq_inversion_bug+0x1ba/0x230 [ 21.763732] [<00000000001b915a>] check_usage_backwards+0xa2/0x140 [ 21.763733] [<00000000001ba7e6>] mark_lock+0x3ee/0x6f8 [ 21.763735] [<00000000001bb524>] __lock_acquire+0x3fc/0x13f0 [ 21.763737] [<00000000001bcd8e>] lock_acquire+0xf6/0x290 [ 21.763739] [<00000000005824b6>] blk_mq_start_request+0xde/0x300 [ 21.763741] [<000003ff803d34f8>] null_queue_rq+0x70/0xc8 [null_blk] [ 21.763743] [<00000000005857c6>] blk_mq_dispatch_rq_list+0x186/0x678 [ 21.763745] [<000000000058b1d0>] blk_mq_do_dispatch_sched+0x90/0x128 [ 21.763747] [<000000000058bba8>] blk_mq_sched_dispatch_requests+0x168/0x200 [ 21.763749] [<0000000000583ad0>] __blk_mq_run_hw_queue+0xd0/0x128 [ 21.763751] [<000000000016ace4>] process_one_work+0x25c/0x6f0 [ 21.763753] [<000000000016b1ca>] worker_thread+0x52/0x400 [ 21.763754] [<00000000001730ba>] kthread+0x15a/0x178 [ 21.763756] [<00000000008e68ba>] kernel_thread_starter+0x6/0xc [ 21.763758] [<00000000008e68b4>] kernel_thread_starter+0x0/0xc [ 21.763759] INFO: lockdep is turned off. [ 21.764713] null: module loaded