From: TomK <tk@mdevsys.com>
To: linux-scsi@vger.kernel.org
Subject: Re: Kernel 4.8.4: INFO: task kworker/u16:8:289 blocked for more than 120 seconds.
Date: Mon, 24 Oct 2016 00:45:22 -0400 [thread overview]
Message-ID: <0851dba9-ad6c-c419-dc92-93c5d7d2f54a@mdevsys.com> (raw)
In-Reply-To: <64ef386e-8de2-0b1c-9322-f02530a3e783@mdevsys.com>
On 10/24/2016 12:32 AM, TomK wrote:
> On 10/23/2016 10:03 PM, TomK wrote:
>> Hey,
>>
>> Has anyone seen this and could have a workaround? Seems like it is more
>> Kernel related with various apps not just target apparently not but
>> wondering if there is an interim solution
>> (https://access.redhat.com/solutions/408833)
>>
>> Getting this message after few minutes of usage from the QLA2xxx driver.
>> This is after some activity on an ESXi server (15 VM's) that I'm
>> connecting to this HBA. I've tried the following tuning parameters but
>> there was no change in behaviour:
>>
>> vm.dirty_background_ratio = 5
>> vm.dirty_ratio = 10
>>
>> Details:
>>
>>
>> Oct 23 21:28:25 mbpc-pc kernel: hpet1: lost 9600 rtc interrupts
>> Oct 23 21:28:29 mbpc-pc kernel: ABORT_TASK: Found referenced qla2xxx
>> task_tag: 1128612
>> Oct 23 21:28:42 mbpc-pc kernel: ABORT_TASK: Sending
>> TMR_FUNCTION_COMPLETE for ref_tag: 1128612
>> Oct 23 21:28:42 mbpc-pc kernel: ABORT_TASK: Found referenced qla2xxx
>> task_tag: 1129116
>> Jan 6 23:52:00 192.168.0.2 syslog: dhcpfwd : dhcp forwarder daemon
>> successfully started
>> Oct 23 21:30:18 mbpc-pc kernel: hpet1: lost 9600 rtc interrupts
>> Jan 6 23:54:01 192.168.0.2 syslog: dhcpfwd : dhcp forwarder daemon
>> successfully started
>> Oct 23 21:32:16 mbpc-pc kernel: hpet1: lost 9600 rtc interrupts
>> Oct 23 21:32:24 mbpc-pc kernel: INFO: task kworker/u16:8:289 blocked for
>> more than 120 seconds.
>> Oct 23 21:32:24 mbpc-pc kernel: Not tainted 4.8.4 #2
>> Oct 23 21:32:24 mbpc-pc kernel: "echo 0 >
>> /proc/sys/kernel/hung_task_timeout_secs" disables this message.
>> Oct 23 21:32:24 mbpc-pc kernel: kworker/u16:8 D ffff88011113ba18 0
>> 289 2 0x00000000
>> Oct 23 21:32:24 mbpc-pc kernel: Workqueue: tmr-fileio target_tmr_work
>> [target_core_mod]
>> Oct 23 21:32:24 mbpc-pc kernel: ffff88011113ba18 0000000000000400
>> ffff880049e926c0 ffff88011113b998
>> Oct 23 21:32:24 mbpc-pc kernel: ffff880111134600 ffffffff81f99ca0
>> ffffffff81f998ef ffff880100000000
>> Oct 23 21:32:24 mbpc-pc kernel: ffffffff812f27d9 0000000000000000
>> ffffe8ffffc9a000 ffff880000000000
>> Oct 23 21:32:24 mbpc-pc kernel: Call Trace:
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff812f27d9>] ? number+0x2e9/0x310
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8162c040>] schedule+0x40/0xb0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff81080169>] ?
>> start_flush_work+0x49/0x180
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8162e7ec>]
>> schedule_timeout+0x9c/0xe0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810802ba>] ?
>> flush_work+0x1a/0x40
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810bd15c>] ?
>> console_unlock+0x35c/0x380
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8162cfa0>]
>> wait_for_completion+0xc0/0xf0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810923e0>] ?
>> try_to_wake_up+0x260/0x260
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffffa08f6f84>]
>> __transport_wait_for_tasks+0xb4/0x1b0 [target_core_mod]
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810bdd1f>] ?
>> vprintk_default+0x1f/0x30
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8115cc5c>] ? printk+0x46/0x48
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffffa08f70c4>]
>> transport_wait_for_tasks+0x44/0x60 [target_core_mod]
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffffa08f46e2>]
>> core_tmr_abort_task+0xf2/0x160 [target_core_mod]
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffffa08f6aa4>]
>> target_tmr_work+0x154/0x160 [target_core_mod]
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff81080639>]
>> process_one_work+0x189/0x4e0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8162c040>] ? schedule+0x40/0xb0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8108150d>]
>> worker_thread+0x16d/0x520
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810923f2>] ?
>> default_wake_function+0x12/0x20
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810a6f06>] ?
>> __wake_up_common+0x56/0x90
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810813a0>] ?
>> maybe_create_worker+0x110/0x110
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8162c040>] ? schedule+0x40/0xb0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810813a0>] ?
>> maybe_create_worker+0x110/0x110
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff81085fec>] kthread+0xcc/0xf0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8109130e>] ?
>> schedule_tail+0x1e/0xc0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8162f60f>]
>> ret_from_fork+0x1f/0x40
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff81085f20>] ?
>> kthread_freezable_should_stop+0x70/0x70
>> Oct 23 21:32:24 mbpc-pc kernel: INFO: task kworker/1:48:6089 blocked for
>> more than 120 seconds.
>> Oct 23 21:32:24 mbpc-pc kernel: Not tainted 4.8.4 #2
>> Oct 23 21:32:24 mbpc-pc kernel: "echo 0 >
>> /proc/sys/kernel/hung_task_timeout_secs" disables this message.
>> Oct 23 21:32:24 mbpc-pc kernel: kworker/1:48 D ffff88004017f968 0
>> 6089 2 0x00000080
>> Oct 23 21:32:24 mbpc-pc kernel: Workqueue: events qlt_free_session_done
>> [qla2xxx]
>> Oct 23 21:32:24 mbpc-pc kernel: ffff88004017f968 ffff88004017f8f8
>> ffff88011a83a300 0000000000000004
>> Oct 23 21:32:24 mbpc-pc kernel: ffff88004017a600 ffff88004017f938
>> ffffffff810a0bb6 ffff880100000000
>> Oct 23 21:32:24 mbpc-pc kernel: ffff880110fd0840 ffff880000000000
>> ffffffff81090728 ffff880100000000
>> Oct 23 21:32:24 mbpc-pc kernel: Call Trace:
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810a0bb6>] ?
>> enqueue_task_fair+0x66/0x410
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff81090728>] ?
>> check_preempt_curr+0x78/0x90
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8109075d>] ?
>> ttwu_do_wakeup+0x1d/0xf0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8162c040>] schedule+0x40/0xb0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff81090de0>] ?
>> ttwu_queue+0x180/0x190
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8162e7ec>]
>> schedule_timeout+0x9c/0xe0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8162cfa0>]
>> wait_for_completion+0xc0/0xf0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810923e0>] ?
>> try_to_wake_up+0x260/0x260
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffffa08f76ad>]
>> target_wait_for_sess_cmds+0x4d/0x1b0 [target_core_mod]
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffffa00e7188>] ?
>> qla2x00_post_work+0x58/0x70 [qla2xxx]
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffffa0286f69>]
>> tcm_qla2xxx_free_session+0x49/0x90 [tcm_qla2xxx]
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffffa01447e9>]
>> qlt_free_session_done+0xf9/0x3d0 [qla2xxx]
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff815092fc>] ?
>> dbs_work_handler+0x5c/0x90
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8107f960>] ?
>> pwq_dec_nr_in_flight+0x50/0xa0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff81080639>]
>> process_one_work+0x189/0x4e0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810d060c>] ?
>> del_timer_sync+0x4c/0x60
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8108131e>] ?
>> maybe_create_worker+0x8e/0x110
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8162c040>] ? schedule+0x40/0xb0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8108150d>]
>> worker_thread+0x16d/0x520
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810923f2>] ?
>> default_wake_function+0x12/0x20
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810a6f06>] ?
>> __wake_up_common+0x56/0x90
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810813a0>] ?
>> maybe_create_worker+0x110/0x110
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8162c040>] ? schedule+0x40/0xb0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810813a0>] ?
>> maybe_create_worker+0x110/0x110
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff81085fec>] kthread+0xcc/0xf0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8109130e>] ?
>> schedule_tail+0x1e/0xc0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8162f60f>]
>> ret_from_fork+0x1f/0x40
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff81085f20>] ?
>> kthread_freezable_should_stop+0x70/0x70
>> Jan 6 23:56:00 192.168.0.2 syslog: dhcpfwd : dhcp forwarder daemon
>> successfully started
>> Oct 23 21:34:22 mbpc-pc kernel: hpet1: lost 9600 rtc interrupts
>> Oct 23 21:34:22 mbpc-pc kernel: hpet1: lost 3 rtc interrupts
>> Oct 23 21:34:27 mbpc-pc kernel: INFO: task kworker/u16:8:289 blocked for
>> more than 120 seconds.
>> Oct 23 21:34:27 mbpc-pc kernel: Not tainted 4.8.4 #2
>> Oct 23 21:34:27 mbpc-pc kernel: "echo 0 >
>> /proc/sys/kernel/hung_task_timeout_secs" disables this message.
>> Oct 23 21:34:27 mbpc-pc kernel: kworker/u16:8 D ffff88011113ba18 0
>> 289 2 0x00000000
>> Oct 23 21:34:27 mbpc-pc kernel: Workqueue: tmr-fileio target_tmr_work
>> [target_core_mod]
>> Oct 23 21:34:27 mbpc-pc kernel: ffff88011113ba18 0000000000000400
>> ffff880049e926c0 ffff88011113b998
>> Oct 23 21:34:27 mbpc-pc kernel: ffff880111134600 ffffffff81f99ca0
>> ffffffff81f998ef ffff880100000000
>> Oct 23 21:34:27 mbpc-pc kernel: ffffffff812f27d9 0000000000000000
>> ffffe8ffffc9a000 ffff880000000000
>> Oct 23 21:34:27 mbpc-pc kernel: Call Trace:
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff812f27d9>] ? number+0x2e9/0x310
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8162c040>] schedule+0x40/0xb0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff81080169>] ?
>> start_flush_work+0x49/0x180
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8162e7ec>]
>> schedule_timeout+0x9c/0xe0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810802ba>] ?
>> flush_work+0x1a/0x40
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810bd15c>] ?
>> console_unlock+0x35c/0x380
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8162cfa0>]
>> wait_for_completion+0xc0/0xf0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810923e0>] ?
>> try_to_wake_up+0x260/0x260
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffffa08f6f84>]
>> __transport_wait_for_tasks+0xb4/0x1b0 [target_core_mod]
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810bdd1f>] ?
>> vprintk_default+0x1f/0x30
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8115cc5c>] ? printk+0x46/0x48
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffffa08f70c4>]
>> transport_wait_for_tasks+0x44/0x60 [target_core_mod]
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffffa08f46e2>]
>> core_tmr_abort_task+0xf2/0x160 [target_core_mod]
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffffa08f6aa4>]
>> target_tmr_work+0x154/0x160 [target_core_mod]
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff81080639>]
>> process_one_work+0x189/0x4e0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8162c040>] ? schedule+0x40/0xb0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8108150d>]
>> worker_thread+0x16d/0x520
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810923f2>] ?
>> default_wake_function+0x12/0x20
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810a6f06>] ?
>> __wake_up_common+0x56/0x90
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810813a0>] ?
>> maybe_create_worker+0x110/0x110
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8162c040>] ? schedule+0x40/0xb0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810813a0>] ?
>> maybe_create_worker+0x110/0x110
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff81085fec>] kthread+0xcc/0xf0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8109130e>] ?
>> schedule_tail+0x1e/0xc0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8162f60f>]
>> ret_from_fork+0x1f/0x40
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff81085f20>] ?
>> kthread_freezable_should_stop+0x70/0x70
>> Oct 23 21:34:27 mbpc-pc kernel: INFO: task kworker/1:48:6089 blocked for
>> more than 120 seconds.
>> Oct 23 21:34:27 mbpc-pc kernel: Not tainted 4.8.4 #2
>> Oct 23 21:34:27 mbpc-pc kernel: "echo 0 >
>> /proc/sys/kernel/hung_task_timeout_secs" disables this message.
>> Oct 23 21:34:27 mbpc-pc kernel: kworker/1:48 D ffff88004017f968 0
>> 6089 2 0x00000080
>> Oct 23 21:34:27 mbpc-pc kernel: Workqueue: events qlt_free_session_done
>> [qla2xxx]
>> Oct 23 21:34:27 mbpc-pc kernel: ffff88004017f968 ffff88004017f8f8
>> ffff88011a83a300 0000000000000004
>> Oct 23 21:34:27 mbpc-pc kernel: ffff88004017a600 ffff88004017f938
>> ffffffff810a0bb6 ffff880100000000
>> Oct 23 21:34:27 mbpc-pc kernel: ffff880110fd0840 ffff880000000000
>> ffffffff81090728 ffff880100000000
>> Oct 23 21:34:27 mbpc-pc kernel: Call Trace:
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810a0bb6>] ?
>> enqueue_task_fair+0x66/0x410
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff81090728>] ?
>> check_preempt_curr+0x78/0x90
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8109075d>] ?
>> ttwu_do_wakeup+0x1d/0xf0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8162c040>] schedule+0x40/0xb0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff81090de0>] ?
>> ttwu_queue+0x180/0x190
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8162e7ec>]
>> schedule_timeout+0x9c/0xe0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8162cfa0>]
>> wait_for_completion+0xc0/0xf0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810923e0>] ?
>> try_to_wake_up+0x260/0x260
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffffa08f76ad>]
>> target_wait_for_sess_cmds+0x4d/0x1b0 [target_core_mod]
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffffa00e7188>] ?
>> qla2x00_post_work+0x58/0x70 [qla2xxx]
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffffa0286f69>]
>> tcm_qla2xxx_free_session+0x49/0x90 [tcm_qla2xxx]
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffffa01447e9>]
>> qlt_free_session_done+0xf9/0x3d0 [qla2xxx]
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff815092fc>] ?
>> dbs_work_handler+0x5c/0x90
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8107f960>] ?
>> pwq_dec_nr_in_flight+0x50/0xa0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff81080639>]
>> process_one_work+0x189/0x4e0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810d060c>] ?
>> del_timer_sync+0x4c/0x60
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8108131e>] ?
>> maybe_create_worker+0x8e/0x110
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8162c040>] ? schedule+0x40/0xb0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8108150d>]
>> worker_thread+0x16d/0x520
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810923f2>] ?
>> default_wake_function+0x12/0x20
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810a6f06>] ?
>> __wake_up_common+0x56/0x90
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810813a0>] ?
>> maybe_create_worker+0x110/0x110
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8162c040>] ? schedule+0x40/0xb0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810813a0>] ?
>> maybe_create_worker+0x110/0x110
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff81085fec>] kthread+0xcc/0xf0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8109130e>] ?
>> schedule_tail+0x1e/0xc0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8162f60f>]
>> ret_from_fork+0x1f/0x40
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff81085f20>] ?
>> kthread_freezable_should_stop+0x70/0x70
>> Oct 23 21:36:04 mbpc-pc kernel: hpet1: lost 9600 rtc interrupts
>> Oct 23 21:36:04 mbpc-pc kernel: hpet1: lost 3 rtc interrupts
>> Jan 6 23:58:00 192.168.0.2 syslog: dhcpfwd : dhcp forwarder daemon
>> successfully started
>> Oct 23 21:36:30 mbpc-pc kernel: INFO: task kworker/u16:8:289 blocked for
>> more than 120 seconds.
>> Oct 23 21:36:30 mbpc-pc kernel: Not tainted 4.8.4 #2
>> Oct 23 21:36:30 mbpc-pc kernel: "echo 0 >
>> /proc/sys/kernel/hung_task_timeout_secs" disables this message.
>> Oct 23 21:36:30 mbpc-pc kernel: kworker/u16:8 D ffff88011113ba18 0
>> 289 2 0x00000000
>> Oct 23 21:36:30 mbpc-pc kernel: Workqueue: tmr-fileio target_tmr_work
>> [target_core_mod]
>> Oct 23 21:36:30 mbpc-pc kernel: ffff88011113ba18 0000000000000400
>> ffff880049e926c0 ffff88011113b998
>> Oct 23 21:36:30 mbpc-pc kernel: ffff880111134600 ffffffff81f99ca0
>> ffffffff81f998ef ffff880100000000
>> Oct 23 21:36:30 mbpc-pc kernel: ffffffff812f27d9 0000000000000000
>> ffffe8ffffc9a000 ffff880000000000
>> Oct 23 21:36:30 mbpc-pc kernel: Call Trace:
>> Oct 23 21:36:30 mbpc-pc kernel: [<ffffffff812f27d9>] ? number+0x2e9/0x310
>> Oct 23 21:36:30 mbpc-pc kernel: [<ffffffff8162c040>] schedule+0x40/0xb0
>> Oct 23 21:36:30 mbpc-pc kernel: [<ffffffff81080169>] ?
>> start_flush_work+0x49/0x180
>> Oct 23 21:36:30 mbpc-pc kernel: [<ffffffff8162e7ec>]
>> schedule_timeout+0x9c/0xe0
>> Oct 23 21:36:30 mbpc-pc kernel: [<ffffffff810802ba>] ?
>> flush_work+0x1a/0x40
>> Oct 23 21:36:30 mbpc-pc kernel: [<ffffffff810bd15c>] ?
>> console_unlock+0x35c/0x380
>> Oct 23 21:36:30 mbpc-pc kernel: [<ffffffff8162cfa0>]
>> wait_for_completion+0xc0/0xf0
>> Oct 23 21:36:30 mbpc-pc kernel: [<ffffffff810923e0>] ?
>> try_to_wake_up+0x260/0x260
>> Oct 23 21:36:30 mbpc-pc kernel: [<ffffffffa08f6f84>]
>> __transport_wait_for_tasks+0xb4/0x1b0 [target_core_mod]
>> Oct 23 21:36:30 mbpc-pc kernel: [<ffffffff810bdd1f>] ?
>> vprintk_default+0x1f/0x30
>>
>>
>
>
> Including the full log:
>
> http://microdevsys.com/linux-lio/messages-mailing-list
>
When tryint to shut down target using /etc/init.d/target stop, the
following is printed repeatedly:
Oct 24 00:39:48 mbpc-pc kernel: qla2xxx [0000:04:00.0]-e837:20:
ABTS_RECV_24XX: instance 0
Oct 24 00:39:48 mbpc-pc kernel: qla2xxx [0000:04:00.0]-f811:20:
qla_target(0): task abort (s_id=1:5:0, tag=1177068, param=0)
Oct 24 00:39:48 mbpc-pc kernel: qla2xxx [0000:04:00.0]-f812:20:
qla_target(0): task abort for non-existant session
Oct 24 00:39:48 mbpc-pc kernel: qla2xxx [0000:04:00.0]-f80e:20:
Scheduling work (type 1, prm ffff880093365680) to find session for param
ffff88010f8c7680 (size 64, tgt ffff880111f06600)
Oct 24 00:39:48 mbpc-pc kernel: qla2xxx [0000:04:00.0]-f800:20: Sess
work (tgt ffff880111f06600)
Oct 24 00:39:48 mbpc-pc kernel: qla2xxx [0000:04:00.0]-e806:20: Sending
task mgmt ABTS response (ha=ffff88010fae0000, atio=ffff880093365694,
status=4
Oct 24 00:39:48 mbpc-pc kernel: qla2xxx [0000:04:00.0]-e838:20:
ABTS_RESP_24XX: compl_status 31
Oct 24 00:39:48 mbpc-pc kernel: qla2xxx [0000:04:00.0]-e807:20: Sending
retry TERM EXCH CTIO7 (ha=ffff88010fae0000)
Oct 24 00:39:48 mbpc-pc kernel: qla2xxx [0000:04:00.0]-e806:20: Sending
task mgmt ABTS response (ha=ffff88010fae0000, atio=ffff88010f8c76c0,
status=0
Oct 24 00:39:48 mbpc-pc kernel: qla2xxx [0000:04:00.0]-e838:20:
ABTS_RESP_24XX: compl_status 0
Oct 24 00:39:48 mbpc-pc kernel: qla2xxx [0000:04:00.0]-e872:20:
qlt_24xx_atio_pkt_all_vps: qla_target(0): type 6 ox_id 029c
Oct 24 00:39:48 mbpc-pc kernel: qla2xxx [0000:04:00.0]-3861:20: New
command while device ffff880111f06600 is shutting down
Oct 24 00:39:48 mbpc-pc kernel: qla2xxx [0000:04:00.0]-e859:20:
qla_target: Unable to send command to target for req, ignoring.
+ when I disable the ports on the brocade switch that we're using then
try to stop target, the following is printed:
Oct 24 00:41:31 mbpc-pc kernel: qla2xxx [0000:04:00.1]-680a:21: Loop
down - seconds remaining 231.
Oct 24 00:41:32 mbpc-pc kernel: qla2xxx [0000:04:00.0]-680a:20: Loop
down - seconds remaining 153.
Oct 24 00:41:32 mbpc-pc kernel: ------------[ cut here ]------------
Oct 24 00:41:32 mbpc-pc kernel: WARNING: CPU: 2 PID: 8615 at
lib/list_debug.c:33 __list_add+0xbe/0xd0
Oct 24 00:41:32 mbpc-pc kernel: list_add corruption. prev->next should
be next (ffff88009e83b330), but was ffff88011fc972a0.
(prev=ffff880118ada4c0).
Oct 24 00:41:32 mbpc-pc kernel: Modules linked in: tcm_qla2xxx tcm_fc
tcm_loop target_core_file target_core_iblock target_core_pscsi
target_core_mod configfs ip6table_filter ip6_tables ebtable_nat ebtables
nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack xt_CHECKSUM
iptable_mangle bridge nfsd lockd grace nfs_acl auth_rpcgss autofs4 it87
hwmon_vid bnx2fc cnic uio fcoe libfcoe libfc 8021q garp stp llc ppdev
parport_pc parport sunrpc cpufreq_ondemand bonding ipv6 crc_ccitt
ipt_REJECT nf_reject_ipv4 xt_multiport iptable_filter ip_tables fuse
vfat fat xfs vhost_net macvtap macvlan vhost tun uinput raid456
async_raid6_recov async_pq async_xor xor async_memcpy async_tx raid6_pq
libcrc32c joydev sg serio_raw e1000 kvm_amd kvm irqbypass r8169 mii
pcspkr k10temp snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel
snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm
snd_timer snd soundcore i2c_piix4 i2c_core wmi acpi_cpufreq shpchp ext4
mbcache jbd2 qla2xxx scsi_transport_fc floppy firewire_ohci f
Oct 24 00:41:32 mbpc-pc kernel: irewire_core crc_itu_t sd_mod pata_acpi
ata_generic pata_jmicron ahci libahci usb_storage dm_mirror
dm_region_hash dm_log dm_mod
Oct 24 00:41:32 mbpc-pc kernel: CPU: 2 PID: 8615 Comm: kworker/2:3 Not
tainted 4.8.4 #2
Oct 24 00:41:32 mbpc-pc kernel: Hardware name: Gigabyte Technology Co.,
Ltd. GA-890XA-UD3/GA-890XA-UD3, BIOS FC 08/02/2010
Oct 24 00:41:32 mbpc-pc kernel: 0000000000000000 ffff880092b83b48
ffffffff812e88e9 ffffffff8130753e
Oct 24 00:41:32 mbpc-pc kernel: ffff880092b83ba8 ffff880092b83ba8
0000000000000000 ffff880092b83b98
Oct 24 00:41:32 mbpc-pc kernel: ffffffff81066a7d ffff88000058f952
0000002100000046 ffffffff8101eae8
Oct 24 00:41:32 mbpc-pc kernel: Call Trace:
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff812e88e9>] dump_stack+0x51/0x78
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8130753e>] ? __list_add+0xbe/0xd0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff81066a7d>] __warn+0xfd/0x120
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8101eae8>] ?
__switch_to+0x398/0x7e0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff81066b59>]
warn_slowpath_fmt+0x49/0x50
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8130753e>] __list_add+0xbe/0xd0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8107d0b2>]
move_linked_works+0x62/0x90
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8108070c>]
process_one_work+0x25c/0x4e0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8162c040>] ? schedule+0x40/0xb0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8108150d>]
worker_thread+0x16d/0x520
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8162bb3d>] ?
__schedule+0x2fd/0x6a0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff810923f2>] ?
default_wake_function+0x12/0x20
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff810a6f06>] ?
__wake_up_common+0x56/0x90
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff810813a0>] ?
maybe_create_worker+0x110/0x110
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8162c040>] ? schedule+0x40/0xb0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff810813a0>] ?
maybe_create_worker+0x110/0x110
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff81085fec>] kthread+0xcc/0xf0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8109130e>] ?
schedule_tail+0x1e/0xc0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8162f60f>] ret_from_fork+0x1f/0x40
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff81085f20>] ?
kthread_freezable_should_stop+0x70/0x70
Oct 24 00:41:32 mbpc-pc kernel: ---[ end trace 713a9071c9f5f464 ]---
Oct 24 00:41:32 mbpc-pc kernel: ------------[ cut here ]------------
Oct 24 00:41:32 mbpc-pc kernel: WARNING: CPU: 2 PID: 8615 at
lib/list_debug.c:36 __list_add+0x9c/0xd0
Oct 24 00:41:32 mbpc-pc kernel: list_add double add:
new=ffff880118ada4c0, prev=ffff880118ada4c0, next=ffff88009e83b330.
Oct 24 00:41:32 mbpc-pc kernel: Modules linked in: tcm_qla2xxx tcm_fc
tcm_loop target_core_file target_core_iblock target_core_pscsi
target_core_mod configfs ip6table_filter ip6_tables ebtable_nat ebtables
nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack xt_CHECKSUM
iptable_mangle bridge nfsd lockd grace nfs_acl auth_rpcgss autofs4 it87
hwmon_vid bnx2fc cnic uio fcoe libfcoe libfc 8021q garp stp llc ppdev
parport_pc parport sunrpc cpufreq_ondemand bonding ipv6 crc_ccitt
ipt_REJECT nf_reject_ipv4 xt_multiport iptable_filter ip_tables fuse
vfat fat xfs vhost_net macvtap macvlan vhost tun uinput raid456
async_raid6_recov async_pq async_xor xor async_memcpy async_tx raid6_pq
libcrc32c joydev sg serio_raw e1000 kvm_amd kvm irqbypass r8169 mii
pcspkr k10temp snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel
snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm
snd_timer snd soundcore i2c_piix4 i2c_core wmi acpi_cpufreq shpchp ext4
mbcache jbd2 qla2xxx scsi_transport_fc floppy firewire_ohci f
Oct 24 00:41:32 mbpc-pc kernel: irewire_core crc_itu_t sd_mod pata_acpi
ata_generic pata_jmicron ahci libahci usb_storage dm_mirror
dm_region_hash dm_log dm_mod
Oct 24 00:41:32 mbpc-pc kernel: CPU: 2 PID: 8615 Comm: kworker/2:3
Tainted: G W 4.8.4 #2
Oct 24 00:41:32 mbpc-pc kernel: Hardware name: Gigabyte Technology Co.,
Ltd. GA-890XA-UD3/GA-890XA-UD3, BIOS FC 08/02/2010
Oct 24 00:41:32 mbpc-pc kernel: 0000000000000000 ffff880092b83b48
ffffffff812e88e9 ffffffff8130751c
Oct 24 00:41:32 mbpc-pc kernel: ffff880092b83ba8 ffff880092b83ba8
0000000000000000 ffff880092b83b98
Oct 24 00:41:32 mbpc-pc kernel: ffffffff81066a7d ffff88000058f952
0000002400000046 ffffffff8101eae8
Oct 24 00:41:32 mbpc-pc kernel: Call Trace:
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff812e88e9>] dump_stack+0x51/0x78
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8130751c>] ? __list_add+0x9c/0xd0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff81066a7d>] __warn+0xfd/0x120
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8101eae8>] ?
__switch_to+0x398/0x7e0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff81066b59>]
warn_slowpath_fmt+0x49/0x50
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8130751c>] __list_add+0x9c/0xd0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8107d0b2>]
move_linked_works+0x62/0x90
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8108070c>]
process_one_work+0x25c/0x4e0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8162c040>] ? schedule+0x40/0xb0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8108150d>]
worker_thread+0x16d/0x520
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8162bb3d>] ?
__schedule+0x2fd/0x6a0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff810923f2>] ?
default_wake_function+0x12/0x20
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff810a6f06>] ?
__wake_up_common+0x56/0x90
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff810813a0>] ?
maybe_create_worker+0x110/0x110
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8162c040>] ? schedule+0x40/0xb0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff810813a0>] ?
maybe_create_worker+0x110/0x110
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff81085fec>] kthread+0xcc/0xf0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8109130e>] ?
schedule_tail+0x1e/0xc0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8162f60f>] ret_from_fork+0x1f/0x40
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff81085f20>] ?
kthread_freezable_should_stop+0x70/0x70
Oct 24 00:41:32 mbpc-pc kernel: ---[ end trace 713a9071c9f5f465 ]---
Oct 24 00:41:32 mbpc-pc kernel: qla2xxx [0000:04:00.1]-680a:21: Loop
down - seconds remaining 230.
Oct 24 00:41:33 mbpc-pc kernel: qla2xxx [0000:04:00.0]-680a:20: Loop
down - seconds remaining 152.
--
Cheers,
Tom K.
-------------------------------------------------------------------------------------
Living on earth is expensive, but it includes a free trip around the sun.
next prev parent reply other threads:[~2016-10-24 4:45 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-10-24 2:03 Kernel 4.8.4: INFO: task kworker/u16:8:289 blocked for more than 120 seconds TomK
2016-10-24 4:32 ` TomK
2016-10-24 4:45 ` TomK [this message]
2016-10-24 6:36 ` Nicholas A. Bellinger
2016-10-25 5:28 ` TomK
2016-10-26 2:05 ` TomK
2016-10-26 7:20 ` Nicholas A. Bellinger
2016-10-26 12:08 ` TomK
2016-10-28 6:01 ` TomK
2016-10-29 7:50 ` Nicholas A. Bellinger
2016-10-29 18:10 ` TomK
2016-10-29 21:44 ` Nicholas A. Bellinger
2016-10-30 18:50 ` TomK
2016-11-01 2:44 ` TomK
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0851dba9-ad6c-c419-dc92-93c5d7d2f54a@mdevsys.com \
--to=tk@mdevsys.com \
--cc=linux-scsi@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox