* mvsas panics and dies when attached to a port extender on newer kernels
@ 2015-04-13 15:11 Adam Talbot
2015-04-23 11:10 ` Ceriel Jacobs
0 siblings, 1 reply; 12+ messages in thread
From: Adam Talbot @ 2015-04-13 15:11 UTC (permalink / raw)
To: linux-scsi
[-- Attachment #1: Type: text/plain, Size: 794 bytes --]
To make a very long debugging story short, I think there is an issues/bug
with the mvsas driver. It works, with older kernels, and breaks on
newer kernels.
My Debian Jessie system was running great on a 3.18 kernel. Changed
cases to a newer supermicro case with a SAS expander backplane (SAS933EL). That
was the only hardware change. Now, when ever I boot, the system kernel panics.
3.2.65-1+deb7u2 works
3.9.0 Gentoo CD works
3.16+ all fail
Attached are 3 kernel panics on 3.16+ kernels.
Motherboard is a Supermicro X8SIE, with a "Marvell Technology Group Ltd.
88SE6440 SAS/SATA PCIe controller"
Is this a known bug?
At this point I have two options:
Stick with the old kernel (yuck)
Buy a new card running a better supported chipset
Any help would be greatly appreciated
Thanks
Adam
[-- Attachment #2: 3.18.0-kernel-panic.txt --]
[-- Type: text/plain, Size: 10168 bytes --]
[ 5.295367] scsi host0: mvsas
[ 5.555699] BUG: unable to handle kernel paging request at 00000030747371bc
[ 5.562813] IP: [<ffffffffa00cd7ed>] mvs_task_prep+0x78d/0xe40 [mvsas]
[ 5.569440] PGD 0
[ 5.571555] Oops: 0000 [#1] SMP
[ 5.574934] Modules linked in: usb_storage hid_generic usbhid hid ata_generic mvsas libsas ata_piix ehci_pci ehci_hcd libata scsi_transport_sas e1000e ptp crc32c_intel usbcore pps_core usb_common scsi_mod
[ 5.594188] CPU: 3 PID: 78 Comm: kworker/u16:1 Not tainted 3.18.0-trunk-amd64 #1 Debian 3.18.5-1~exp1
[ 5.603467] Hardware name: Supermicro X8SIE/X8SIE, BIOS 1.2a 06/27/2012
[ 5.610651] Workqueue: events_unbound async_run_entry_fn
[ 5.616056] task: ffff88032fb6cc40 ti: ffff88032fb78000 task.ti: ffff88032fb78000
[ 5.624779] RIP: 0010:[<ffffffffa00cd7ed>] [<ffffffffa00cd7ed>] mvs_task_prep+0x78d/0xe40 [mvsas]
[ 5.633853] RSP: 0018:ffff88032fb7b888 EFLAGS: 00010097
[ 5.639217] RAX: 000000000000002c RBX: 0000003074736f68 RCX: 0000000000000000
[ 5.646399] RDX: ffff8800372c3618 RSI: 0000000000000000 RDI: 0000000000001000
[ 5.653580] RBP: 0000000000000000 R08: ffff88032f572400 R09: ffff8800372a8000
[ 5.660763] R10: ffffc900018f6000 R11: 0000000000010000 R12: 0000000000000000
[ 5.667947] R13: ffff8800372c3618 R14: ffff8800372c0000 R15: ffff88032f23ce40
[ 5.675129] FS: 0000000000000000(0000) GS:ffff88033fcc0000(0000) knlGS:0000000000000000
[ 5.683284] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 5.689079] CR2: 00000030747371bc CR3: 0000000001813000 CR4: 00000000000007e0
[ 5.696261] Stack:
[ 5.698323] ffff88032fb7b934 ffff8800372c3618 0000000000000000 ffff8800372c2678
[ 5.705983] ffff8800372c1e38 ffff8800372a8000 ffff88032f572400 ffff88032fb7bfd8
[ 5.713642] 0000000000013740 ffff88032fb6cc40 0000000300000001 ffffffff8109904a
[ 5.721310] Call Trace:
[ 5.723810] [<ffffffff8109904a>] ? select_idle_sibling+0x10a/0x120
[ 5.730125] [<ffffffffa00cdef7>] ? mvs_task_exec.isra.13+0x57/0xd0 [mvsas]
[ 5.737135] [<ffffffffa00ce9f9>] ? mvs_queue_command+0x2e9/0x2f0 [mvsas]
[ 5.743971] [<ffffffffa017a7cc>] ? sas_ata_qc_issue+0x22c/0x270 [libsas]
[ 5.750808] [<ffffffffa028e845>] ? ata_qc_issue+0x1f5/0x380 [libata]
[ 5.757296] [<ffffffffa028ecbf>] ? ata_exec_internal_sg+0x2ef/0x5f0 [libata]
[ 5.764480] [<ffffffffa028f018>] ? ata_exec_internal+0x58/0xa0 [libata]
[ 5.771229] [<ffffffffa017a4cd>] ? sas_ata_hard_reset+0x9d/0x140 [libsas]
[ 5.778151] [<ffffffffa0291f35>] ? ata_phys_link_offline+0x15/0x30 [libata]
[ 5.785247] [<ffffffffa028f3b5>] ? ata_dev_read_id+0x205/0x5c0 [libata]
[ 5.791997] [<ffffffffa017a430>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 5.798487] [<ffffffffa02996c1>] ? ata_eh_schedule_probe+0x141/0x180 [libata]
[ 5.805774] [<ffffffffa029c5da>] ? ata_eh_recover+0x7da/0x1370 [libata]
[ 5.812524] [<ffffffffa0291f50>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 5.819619] [<ffffffffa017a430>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 5.826109] [<ffffffffa0291dc0>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 5.833119] [<ffffffffa017a430>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 5.839609] [<ffffffffa0291f50>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 5.846703] [<ffffffffa017a430>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 5.853194] [<ffffffffa029db06>] ? ata_do_eh+0x46/0xb0 [libata]
[ 5.859251] [<ffffffffa0291dc0>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 5.866262] [<ffffffff8101c206>] ? native_sched_clock+0x26/0x90
[ 5.872318] [<ffffffffa029d698>] ? ata_scsi_port_error_handler+0x448/0x7b0 [libata]
[ 5.880122] [<ffffffffa017abb6>] ? async_sas_ata_eh+0x46/0x70 [libsas]
[ 5.886786] [<ffffffff8108a973>] ? async_run_entry_fn+0x43/0x160
[ 5.892930] [<ffffffff81082a8e>] ? process_one_work+0x14e/0x3f0
[ 5.898985] [<ffffffff8108338b>] ? worker_thread+0x6b/0x4a0
[ 5.904693] [<ffffffff81083320>] ? rescuer_thread+0x290/0x290
[ 5.910578] [<ffffffff810880e3>] ? kthread+0xd3/0xf0
[ 5.915680] [<ffffffff81088010>] ? kthread_create_on_node+0x180/0x180
[ 5.922260] [<ffffffff8154543c>] ? ret_from_fork+0x7c/0xb0
[ 5.927880] [<ffffffff81088010>] ? kthread_create_on_node+0x180/0x180
[ 5.934455] Code: 8b 80 58 01 00 00 48 8b 1c c8 0f 84 29 06 00 00 41 8b 46 58 48 8d 4c ad 00 bf 00 10 00 00 48 8d 4c 4d 00 41 89 84 ce d4 55 02 00 <8b> 8b 54 02 00 00 49 8b 76 48 d3 e7 0f b6 8a 84 00 00 00 81 cf
[ 5.956722] RIP [<ffffffffa00cd7ed>] mvs_task_prep+0x78d/0xe40 [mvsas]
[ 5.963438] RSP <ffff88032fb7b888>
[ 5.966973] CR2: 00000030747371bc
[ 5.970337] ---[ end trace d307165136b3807d ]---
[ 5.987492] BUG: unable to handle kernel paging request at ffffffffffffffd8
[ 5.994605] IP: [<ffffffff810886fc>] kthread_data+0xc/0x20
[ 6.000183] PGD 1816067 PUD 1818067 PMD 0
[ 6.004483] Oops: 0000 [#2] SMP
[ 6.007871] Modules linked in: usb_storage hid_generic usbhid hid ata_generic mvsas libsas ata_piix ehci_pci ehci_hcd libata scsi_transport_sas e1000e ptp crc32c_intel usbcore pps_core usb_common scsi_mod
[ 6.027123] CPU: 3 PID: 78 Comm: kworker/u16:1 Tainted: G D 3.18.0-trunk-amd64 #1 Debian 3.18.5-1~exp1
[ 6.037529] Hardware name: Supermicro X8SIE/X8SIE, BIOS 1.2a 06/27/2012
[ 6.044715] task: ffff88032fb6cc40 ti: ffff88032fb78000 task.ti: ffff88032fb78000
[ 6.052258] RIP: 0010:[<ffffffff810886fc>] [<ffffffff810886fc>] kthread_data+0xc/0x20
[ 6.060288] RSP: 0018:ffff88032fb7b540 EFLAGS: 00010002
[ 6.065644] RAX: 0000000000000000 RBX: 0000000000000003 RCX: 0000000000000000
[ 6.072827] RDX: 000000000000000f RSI: 0000000000000003 RDI: ffff88032fb6cc40
[ 6.080009] RBP: ffff88032fb6cc40 R08: 0000000000000000 R09: 000000000000001f
[ 6.087192] R10: ffffffff81afdec0 R11: 000000000000001a R12: ffff88033fcd3740
[ 6.094374] R13: ffff88032fb6d0a8 R14: 0000000000000003 R15: 0000000000030001
[ 6.101560] FS: 0000000000000000(0000) GS:ffff88033fcc0000(0000) knlGS:0000000000000000
[ 6.109712] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 6.115507] CR2: 0000000000000028 CR3: 0000000001813000 CR4: 00000000000007e0
[ 6.122690] Stack:
[ 6.124753] ffffffff8108383d 0000000000000000 ffffffff81540fbf 0000000000000000
[ 6.132422] ffffffff8106c24e 0000000000013740 ffff88032fb7bfd8 0000000000013740
[ 6.140090] ffff88032fb6cc40 0000000000000008 ffff88032fb6d2c8 ffff88032fb6cc30
[ 6.147749] Call Trace:
[ 6.150244] [<ffffffff8108383d>] ? wq_worker_sleeping+0xd/0xa0
[ 6.156215] [<ffffffff81540fbf>] ? __schedule+0x66f/0x7f0
[ 6.161751] [<ffffffff8106c24e>] ? release_task+0x36e/0x470
[ 6.167460] [<ffffffff8106dc67>] ? do_exit+0x857/0xad0
[ 6.172739] [<ffffffff810166b7>] ? oops_end+0x97/0xe0
[ 6.177928] [<ffffffff8105a0f6>] ? no_context+0x106/0x380
[ 6.183463] [<ffffffff8105a8c6>] ? __do_page_fault+0xb6/0x530
[ 6.189349] [<ffffffff812c7c20>] ? cpumask_next_and+0x30/0x50
[ 6.195230] [<ffffffff8109fca2>] ? load_balance+0x1f2/0x880
[ 6.200941] [<ffffffff8101c206>] ? native_sched_clock+0x26/0x90
[ 6.206997] [<ffffffff815474e8>] ? page_fault+0x28/0x30
[ 6.212361] [<ffffffffa00cd7ed>] ? mvs_task_prep+0x78d/0xe40 [mvsas]
[ 6.218850] [<ffffffffa00cde25>] ? mvs_task_prep+0xdc5/0xe40 [mvsas]
[ 6.225340] [<ffffffff8109904a>] ? select_idle_sibling+0x10a/0x120
[ 6.231657] [<ffffffffa00cdef7>] ? mvs_task_exec.isra.13+0x57/0xd0 [mvsas]
[ 6.238667] [<ffffffffa00ce9f9>] ? mvs_queue_command+0x2e9/0x2f0 [mvsas]
[ 6.245502] [<ffffffffa017a7cc>] ? sas_ata_qc_issue+0x22c/0x270 [libsas]
[ 6.252340] [<ffffffffa028e845>] ? ata_qc_issue+0x1f5/0x380 [libata]
[ 6.258830] [<ffffffffa028ecbf>] ? ata_exec_internal_sg+0x2ef/0x5f0 [libata]
[ 6.266013] [<ffffffffa028f018>] ? ata_exec_internal+0x58/0xa0 [libata]
[ 6.272759] [<ffffffffa017a4cd>] ? sas_ata_hard_reset+0x9d/0x140 [libsas]
[ 6.279685] [<ffffffffa0291f35>] ? ata_phys_link_offline+0x15/0x30 [libata]
[ 6.286788] [<ffffffffa028f3b5>] ? ata_dev_read_id+0x205/0x5c0 [libata]
[ 6.293537] [<ffffffffa017a430>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 6.300028] [<ffffffffa02996c1>] ? ata_eh_schedule_probe+0x141/0x180 [libata]
[ 6.307315] [<ffffffffa029c5da>] ? ata_eh_recover+0x7da/0x1370 [libata]
[ 6.314065] [<ffffffffa0291f50>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 6.321161] [<ffffffffa017a430>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 6.327651] [<ffffffffa0291dc0>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 6.334659] [<ffffffffa017a430>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 6.341150] [<ffffffffa0291f50>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 6.348246] [<ffffffffa017a430>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 6.354736] [<ffffffffa029db06>] ? ata_do_eh+0x46/0xb0 [libata]
[ 6.360792] [<ffffffffa0291dc0>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 6.367801] [<ffffffff8101c206>] ? native_sched_clock+0x26/0x90
[ 6.373859] [<ffffffffa029d698>] ? ata_scsi_port_error_handler+0x448/0x7b0 [libata]
[ 6.381665] [<ffffffffa017abb6>] ? async_sas_ata_eh+0x46/0x70 [libsas]
[ 6.388327] [<ffffffff8108a973>] ? async_run_entry_fn+0x43/0x160
[ 6.394468] [<ffffffff81082a8e>] ? process_one_work+0x14e/0x3f0
[ 6.400527] [<ffffffff8108338b>] ? worker_thread+0x6b/0x4a0
[ 6.406236] [<ffffffff81083320>] ? rescuer_thread+0x290/0x290
[ 6.412119] [<ffffffff810880e3>] ? kthread+0xd3/0xf0
[ 6.417221] [<ffffffff81088010>] ? kthread_create_on_node+0x180/0x180
[ 6.423798] [<ffffffff8154543c>] ? ret_from_fork+0x7c/0xb0
[ 6.429421] [<ffffffff81088010>] ? kthread_create_on_node+0x180/0x180
[ 6.435995] Code: 10 04 00 00 48 8b 40 c8 48 c1 e8 02 83 e0 01 c3 66 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 48 8b 87 10 04 00 00 <48> 8b 40 d8 c3 66 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 66
[ 6.458265] RIP [<ffffffff810886fc>] kthread_data+0xc/0x20
[ 6.463931] RSP <ffff88032fb7b540>
[ 6.467466] CR2: ffffffffffffffd8
[ 6.470827] ---[ end trace d307165136b3807e ]---
[ 6.475490] Fixing recursive fault but reboot is needed!
[-- Attachment #3: 3.16.0-kernel-panic.txt --]
[-- Type: text/plain, Size: 9317 bytes --]
[ 5.192410] scsi0 : mvsas
[ 5.456795] general protection fault: 0000 [#1] SMP
[ 5.461913] Modules linked in: usb_storage hid_generic usbhid hid ata_generic mvsas libsas ata_piix ehci_pci libata ehci_hcd scsi_transport_sas e1000e ptp crc32c_intel pps_core usbcore scsi_mod usb_common
[ 5.481174] CPU: 3 PID: 138 Comm: kworker/u16:5 Not tainted 3.16.0-4-amd64 #1 Debian 3.16.7-ckt7-1
[ 5.490194] Hardware name: Supermicro X8SIE/X8SIE, BIOS 1.2a 06/27/2012
[ 5.497378] Workqueue: events_unbound async_run_entry_fn
[ 5.502791] task: ffff880036f68010 ti: ffff880036f74000 task.ti: ffff880036f74000
[ 5.511515] RIP: 0010:[<ffffffffa00bd90f>] [<ffffffffa00bd90f>] mvs_task_prep+0x73f/0xd50 [mvsas]
[ 5.520596] RSP: 0018:ffff880036f77948 EFLAGS: 00010097
[ 5.525960] RAX: 000000000000002c RBX: cccccccccccccccc RCX: 0000000000000000
[ 5.533142] RDX: 0000000000000000 RSI: ffff880036ea55b8 RDI: ffff880036e80000
[ 5.540325] RBP: 0000000000000000 R08: ffff880036e836a0 R09: ffff880036e70000
[ 5.547508] R10: ffffc900018f0000 R11: 0000000000000005 R12: ffff880036e80000
[ 5.554692] R13: ffff88032f4bf600 R14: ffff880036e836a0 R15: ffff88032f123e40
[ 5.561875] FS: 0000000000000000(0000) GS:ffff88033fcc0000(0000) knlGS:0000000000000000
[ 5.570028] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 5.575825] CR2: 00007ffb9332c000 CR3: 0000000001813000 CR4: 00000000000007e0
[ 5.583006] Stack:
[ 5.585070] ffff880036e81e38 ffff880036f779ec ffff880036ea55b8 ffff880300000001
[ 5.592737] ffff880036e82678 ffff880036ea55b0 ffff880036e836a0 ffff880036e70000
[ 5.600404] 0000000000012f00 ffff880036f68010 ffff880332584418 ffff880036f77a68
[ 5.608063] Call Trace:
[ 5.610562] [<ffffffffa00bdf77>] ? mvs_task_exec.isra.13+0x57/0xd0 [mvsas]
[ 5.617571] [<ffffffffa00bea71>] ? mvs_queue_command+0x301/0x310 [mvsas]
[ 5.624407] [<ffffffff8118e2bf>] ? kmem_cache_alloc+0x21f/0x450
[ 5.630464] [<ffffffffa011c3ac>] ? sas_ata_qc_issue+0x22c/0x270 [libsas]
[ 5.637303] [<ffffffffa026272c>] ? ata_qc_issue+0x16c/0x390 [libata]
[ 5.643789] [<ffffffffa0262be2>] ? ata_exec_internal_sg+0x292/0x570 [libata]
[ 5.650974] [<ffffffffa0262f18>] ? ata_exec_internal+0x58/0xa0 [libata]
[ 5.657721] [<ffffffffa011c0ae>] ? sas_ata_hard_reset+0x9e/0x140 [libsas]
[ 5.664647] [<ffffffffa0263325>] ? ata_dev_read_id+0x275/0x560 [libata]
[ 5.671395] [<ffffffffa026d5a4>] ? ata_eh_schedule_probe+0x104/0x150 [libata]
[ 5.678681] [<ffffffffa02707d4>] ? ata_eh_recover+0x814/0x1350 [libata]
[ 5.685432] [<ffffffffa0265d40>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 5.692526] [<ffffffffa011c010>] ? sas_ata_printk+0x80/0x80 [libsas]
[ 5.699018] [<ffffffffa0265bc0>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 5.706025] [<ffffffff8150768a>] ? printk+0x54/0x56
[ 5.711042] [<ffffffffa0265d40>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 5.718140] [<ffffffffa011c010>] ? sas_ata_printk+0x80/0x80 [libsas]
[ 5.724628] [<ffffffffa0271de4>] ? ata_do_eh+0x44/0xb0 [libata]
[ 5.730685] [<ffffffffa0265bc0>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 5.737694] [<ffffffffa0271908>] ? ata_scsi_port_error_handler+0x518/0x8f0 [libata]
[ 5.745500] [<ffffffffa011c786>] ? async_sas_ata_eh+0x46/0x70 [libsas]
[ 5.752162] [<ffffffff8108d872>] ? async_run_entry_fn+0x32/0x120
[ 5.758307] [<ffffffff810814e2>] ? process_one_work+0x172/0x420
[ 5.764362] [<ffffffff81081b73>] ? worker_thread+0x113/0x4f0
[ 5.770159] [<ffffffff8150c291>] ? __schedule+0x2b1/0x710
[ 5.775694] [<ffffffff81081a60>] ? rescuer_thread+0x2d0/0x2d0
[ 5.781578] [<ffffffff81087dfd>] ? kthread+0xbd/0xe0
[ 5.786682] [<ffffffff81087d40>] ? kthread_create_on_node+0x180/0x180
[ 5.793259] [<ffffffff8150febc>] ? ret_from_fork+0x7c/0xb0
[ 5.798883] [<ffffffff81087d40>] ? kthread_create_on_node+0x180/0x180
[ 5.805458] Code: 92 b8 02 00 00 41 80 b8 84 00 00 00 7f 48 8b 80 58 01 00 00 48 8b 1c d0 0f 84 9d 05 00 00 41 8b 44 24 58 48 8b 74 24 10 89 46 1c <8b> 8b 54 02 00 00 be 00 10 00 00 41 8b 54 24 58 49 8b 44 24 48
[ 5.827742] RIP [<ffffffffa00bd90f>] mvs_task_prep+0x73f/0xd50 [mvsas]
[ 5.834457] RSP <ffff880036f77948>
[ 5.837992] ---[ end trace ab01b27d6a351042 ]---
[ 5.842671] BUG: unable to handle kernel paging request at ffffffffffffffd8
[ 5.849783] IP: [<ffffffff8108838c>] kthread_data+0xc/0x20
[ 5.855374] PGD 1816067 PUD 1818067 PMD 0
[ 5.859679] Oops: 0000 [#2] SMP
[ 5.863058] Modules linked in: usb_storage hid_generic usbhid hid ata_generic mvsas libsas ata_piix ehci_pci libata ehci_hcd scsi_transport_sas e1000e ptp crc32c_intel pps_core usbcore scsi_mod usb_common
[ 5.882319] CPU: 3 PID: 138 Comm: kworker/u16:5 Tainted: G D 3.16.0-4-amd64 #1 Debian 3.16.7-ckt7-1
[ 5.892379] Hardware name: Supermicro X8SIE/X8SIE, BIOS 1.2a 06/27/2012
[ 5.899567] task: ffff880036f68010 ti: ffff880036f74000 task.ti: ffff880036f74000
[ 5.907109] RIP: 0010:[<ffffffff8108838c>] [<ffffffff8108838c>] kthread_data+0xc/0x20
[ 5.915140] RSP: 0018:ffff880036f77798 EFLAGS: 00010002
[ 5.920504] RAX: 0000000000000000 RBX: 0000000000000003 RCX: 000000000000000f
[ 5.927686] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff880036f68010
[ 5.934869] RBP: ffff880036f68010 R08: 0000000000000000 R09: 0000000000000000
[ 5.942052] R10: ffffffff81adf150 R11: 0000000000000000 R12: ffff88033fcd2f00
[ 5.949235] R13: 0000000000000003 R14: 0000000000000000 R15: ffff880036f68010
[ 5.956417] FS: 0000000000000000(0000) GS:ffff88033fcc0000(0000) knlGS:0000000000000000
[ 5.964571] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 5.970367] CR2: 0000000000000028 CR3: 0000000001813000 CR4: 00000000000007e0
[ 5.977550] Stack:
[ 5.979612] ffffffff81081fbd ffff880036f68470 ffffffff8150c43d 0000000000012f00
[ 5.987281] ffff880036f77fd8 0000000000012f00 ffff880036f68010 ffff880036f68690
[ 5.994948] ffff880036f68368 ffff880036f68000 ffff8803329752b0 ffff880036f68000
[ 6.002608] Call Trace:
[ 6.005104] [<ffffffff81081fbd>] ? wq_worker_sleeping+0xd/0x80
[ 6.011075] [<ffffffff8150c43d>] ? __schedule+0x45d/0x710
[ 6.016611] [<ffffffff81069f0f>] ? do_exit+0x6ef/0xa60
[ 6.021890] [<ffffffff810163b7>] ? oops_end+0x97/0xe0
[ 6.027078] [<ffffffff81511f48>] ? general_protection+0x28/0x30
[ 6.033134] [<ffffffffa00bd90f>] ? mvs_task_prep+0x73f/0xd50 [mvsas]
[ 6.039625] [<ffffffffa00bdec5>] ? mvs_task_prep+0xcf5/0xd50 [mvsas]
[ 6.046115] [<ffffffffa00bdf77>] ? mvs_task_exec.isra.13+0x57/0xd0 [mvsas]
[ 6.053124] [<ffffffffa00bea71>] ? mvs_queue_command+0x301/0x310 [mvsas]
[ 6.059960] [<ffffffff8118e2bf>] ? kmem_cache_alloc+0x21f/0x450
[ 6.066015] [<ffffffffa011c3ac>] ? sas_ata_qc_issue+0x22c/0x270 [libsas]
[ 6.072852] [<ffffffffa026272c>] ? ata_qc_issue+0x16c/0x390 [libata]
[ 6.079341] [<ffffffffa0262be2>] ? ata_exec_internal_sg+0x292/0x570 [libata]
[ 6.086526] [<ffffffffa0262f18>] ? ata_exec_internal+0x58/0xa0 [libata]
[ 6.093274] [<ffffffffa011c0ae>] ? sas_ata_hard_reset+0x9e/0x140 [libsas]
[ 6.100198] [<ffffffffa0263325>] ? ata_dev_read_id+0x275/0x560 [libata]
[ 6.106947] [<ffffffffa026d5a4>] ? ata_eh_schedule_probe+0x104/0x150 [libata]
[ 6.114233] [<ffffffffa02707d4>] ? ata_eh_recover+0x814/0x1350 [libata]
[ 6.118548] scsi 5:0:0:0: Direct-Access Generic USB EDC 1.00 PQ: 0 ANSI: 2
[ 6.129139] [<ffffffffa0265d40>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 6.136242] [<ffffffffa011c010>] ? sas_ata_printk+0x80/0x80 [libsas]
[ 6.142732] [<ffffffffa0265bc0>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 6.149740] [<ffffffff8150768a>] ? printk+0x54/0x56
[ 6.154759] [<ffffffffa0265d40>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 6.161852] [<ffffffffa011c010>] ? sas_ata_printk+0x80/0x80 [libsas]
[ 6.168345] [<ffffffffa0271de4>] ? ata_do_eh+0x44/0xb0 [libata]
[ 6.174401] [<ffffffffa0265bc0>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 6.181410] [<ffffffffa0271908>] ? ata_scsi_port_error_handler+0x518/0x8f0 [libata]
[ 6.189215] [<ffffffffa011c786>] ? async_sas_ata_eh+0x46/0x70 [libsas]
[ 6.195879] [<ffffffff8108d872>] ? async_run_entry_fn+0x32/0x120
[ 6.202022] [<ffffffff810814e2>] ? process_one_work+0x172/0x420
[ 6.208077] [<ffffffff81081b73>] ? worker_thread+0x113/0x4f0
[ 6.213874] [<ffffffff8150c291>] ? __schedule+0x2b1/0x710
[ 6.219410] [<ffffffff81081a60>] ? rescuer_thread+0x2d0/0x2d0
[ 6.225293] [<ffffffff81087dfd>] ? kthread+0xbd/0xe0
[ 6.230397] [<ffffffff81087d40>] ? kthread_create_on_node+0x180/0x180
[ 6.236973] [<ffffffff8150febc>] ? ret_from_fork+0x7c/0xb0
[ 6.242597] [<ffffffff81087d40>] ? kthread_create_on_node+0x180/0x180
[ 6.249172] Code: 08 04 00 00 48 8b 40 c8 48 c1 e8 02 83 e0 01 c3 66 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 48 8b 87 08 04 00 00 <48> 8b 40 d8 c3 66 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 66
[ 6.271449] RIP [<ffffffff8108838c>] kthread_data+0xc/0x20
[ 6.277115] RSP <ffff880036f77798>
[ 6.280649] CR2: ffffffffffffffd8
[ 6.284012] ---[ end trace ab01b27d6a351043 ]---
[ 6.288681] Fixing recursive fault but reboot is needed!
[-- Attachment #4: 3.19.0-kernel-panic.txt --]
[-- Type: text/plain, Size: 28319 bytes --]
[ 5.022094] scsi host4: mvsas
[ 5.162509] floppy0: no floppy controllers found
[ 5.286606] BUG: unable to handle kernel paging request at 0000009a2484b963
[ 5.293727] IP: [<ffffffffa00f1877>] mvs_task_exec.isra.13+0x827/0xf10 [mvsas]
[ 5.301066] PGD 0
[ 5.303181] Oops: 0000 [#1] SMP
[ 5.306560] Modules linked in: usb_storage hid_generic usbhid hid ata_generic mvsas ehci_pci libsas ata_piix ehci_hcd libata e1000e scsi_transport_sas ptp crc32c_intel pps_core usbcore scsi_mod usb_common
[ 5.325822] CPU: 7 PID: 146 Comm: kworker/u16:6 Not tainted 3.19.0-trunk-amd64 #1 Debian 3.19.3-1~exp1
[ 5.335188] Hardware name: Supermicro X8SIE/X8SIE, BIOS 1.2a 06/27/2012
[ 5.342373] Workqueue: events_unbound async_run_entry_fn
[ 5.347778] task: ffff88033113e450 ti: ffff88032d868000 task.ti: ffff88032d868000
[ 5.356537] RIP: 0010:[<ffffffffa00f1877>] [<ffffffffa00f1877>] mvs_task_exec.isra.13+0x827/0xf10 [mvsas]
[ 5.366310] RSP: 0018:ffff88032d86b948 EFLAGS: 00010097
[ 5.371673] RAX: 000000000000002d RBX: ffff88032d8037b0 RCX: 0000009a2484b70f
[ 5.378855] RDX: 000000000000000b RSI: 0000000000001000 RDI: ffff88032d800000
[ 5.386040] RBP: 0000000000000001 R08: ffff88032d8037b0 R09: ffff88032e173200
[ 5.393221] R10: ffff8800ba230040 R11: 0000000000010000 R12: ffff88032dc55c00
[ 5.400404] R13: ffff88032d800000 R14: 0000000000000096 R15: ffff88032d800008
[ 5.407588] FS: 0000000000000000(0000) GS:ffff88033fdc0000(0000) knlGS:0000000000000000
[ 5.415740] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 5.421537] CR2: 0000009a2484b963 CR3: 0000000001813000 CR4: 00000000000007e0
[ 5.428721] Stack:
[ 5.430782] 0000000003aa7d20 ffff88032d8037b0 ffff88032d801e38 ffff8800ba230040
[ 5.438442] 0000000000000001 ffff88032d802678 ffff88032e173200 0000009a2484b70f
[ 5.446100] ffff88032d8813f0 0000000000000286 ffff88032d8813f0 0000000000000286
[ 5.453760] Call Trace:
[ 5.456260] [<ffffffffa00d8c13>] ? sas_ata_qc_issue+0x1a3/0x250 [libsas]
[ 5.463104] [<ffffffffa0259875>] ? ata_qc_issue+0x1f5/0x380 [libata]
[ 5.469594] [<ffffffffa0259cef>] ? ata_exec_internal_sg+0x2ef/0x5f0 [libata]
[ 5.476784] [<ffffffffa025a048>] ? ata_exec_internal+0x58/0xa0 [libata]
[ 5.483533] [<ffffffffa00d89f2>] ? sas_ata_hard_reset+0xa2/0x120 [libsas]
[ 5.490456] [<ffffffffa025cfa5>] ? ata_phys_link_offline+0x15/0x30 [libata]
[ 5.497551] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 5.504042] [<ffffffffa025a525>] ? ata_dev_read_id+0x345/0x5d0 [libata]
[ 5.510792] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 5.517283] [<ffffffffa02647f1>] ? ata_eh_schedule_probe+0x141/0x180 [libata]
[ 5.524570] [<ffffffffa02676ef>] ? ata_eh_recover+0x7df/0x13b0 [libata]
[ 5.531318] [<ffffffffa025cfc0>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 5.538412] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 5.544904] [<ffffffffa025ce30>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 5.551912] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 5.558404] [<ffffffffa025cfc0>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 5.565497] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 5.571990] [<ffffffffa0268c56>] ? ata_do_eh+0x46/0xb0 [libata]
[ 5.578046] [<ffffffffa025ce30>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 5.585056] [<ffffffff8101d146>] ? native_sched_clock+0x26/0x90
[ 5.591112] [<ffffffffa02687e8>] ? ata_scsi_port_error_handler+0x448/0x7b0 [libata]
[ 5.598917] [<ffffffffa00d9046>] ? async_sas_ata_eh+0x46/0x70 [libsas]
[ 5.605579] [<ffffffff8108ca33>] ? async_run_entry_fn+0x43/0x160
[ 5.611725] [<ffffffff810849cd>] ? process_one_work+0x14d/0x3f0
[ 5.617778] [<ffffffff810853fb>] ? worker_thread+0x6b/0x4a0
[ 5.623488] [<ffffffff81085390>] ? rescuer_thread+0x310/0x310
[ 5.629371] [<ffffffff8108a1a3>] ? kthread+0xd3/0xf0
[ 5.634475] [<ffffffff8108a0d0>] ? kthread_create_on_node+0x180/0x180
[ 5.641054] [<ffffffff815509d8>] ? ret_from_fork+0x58/0x90
[ 5.646674] [<ffffffff8108a0d0>] ? kthread_create_on_node+0x180/0x180
[ 5.653251] Code: 8b 80 08 01 00 00 48 8b 0c d0 0f 84 e6 05 00 00 41 8b 45 58 48 8d 54 ad 00 be 00 10 00 00 48 8d 54 55 00 41 89 84 d5 d4 55 02 00 <8b> 89 54 02 00 00 49 8b 55 48 d3 e6 41 0f b6 88 84 00 00 00 81
[ 5.675544] RIP [<ffffffffa00f1877>] mvs_task_exec.isra.13+0x827/0xf10 [mvsas]
[ 5.682970] RSP <ffff88032d86b948>
[ 5.686505] CR2: 0000009a2484b963
[ 5.689867] ---[ end trace 50dd3aeb0a903cc2 ]---
[ 5.694551] BUG: unable to handle kernel paging request at ffffffffffffffd8
[ 5.701667] IP: [<ffffffff8108a7bc>] kthread_data+0xc/0x20
[ 5.707247] PGD 1816067 PUD 1818067 PMD 0
[ 5.711554] Oops: 0000 [#2] SMP
[ 5.714941] Modules linked in: usb_storage hid_generic usbhid hid ata_generic mvsas ehci_pci libsas ata_piix ehci_hcd libata e1000e scsi_transport_sas ptp crc32c_intel pps_core usbcore scsi_mod usb_common
[ 5.734195] CPU: 7 PID: 146 Comm: kworker/u16:6 Tainted: G D 3.19.0-trunk-amd64 #1 Debian 3.19.3-1~exp1
[ 5.744695] Hardware name: Supermicro X8SIE/X8SIE, BIOS 1.2a 06/27/2012
[ 5.751883] task: ffff88033113e450 ti: ffff88032d868000 task.ti: ffff88032d868000
[ 5.759424] RIP: 0010:[<ffffffff8108a7bc>] [<ffffffff8108a7bc>] kthread_data+0xc/0x20
[ 5.767455] RSP: 0018:ffff88032d86b600 EFLAGS: 00010002
[ 5.772820] RAX: 0000000000000000 RBX: 0000000000000007 RCX: 0000000000000000
[ 5.780003] RDX: 000000000000000f RSI: 0000000000000007 RDI: ffff88033113e450
[ 5.787185] RBP: ffff88033113e450 R08: 0000000000000000 R09: 000000000000001f
[ 5.794367] R10: ffffffff81af9330 R11: 000000000000001a R12: 0000000000000000
[ 5.801551] R13: ffff88033113e8b8 R14: 0000000000000007 R15: ffff880331601230
[ 5.808733] FS: 0000000000000000(0000) GS:ffff88033fdc0000(0000) knlGS:0000000000000000
[ 5.816886] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 5.822684] CR2: 0000000000000028 CR3: 0000000001813000 CR4: 00000000000007e0
[ 5.829865] Stack:
[ 5.831928] ffffffff810858ad ffff88033fdd4140 ffffffff8154c701 0000000000000000
[ 5.839588] ffffffff8106e1dd 0000000000014140 ffff88032d86bfd8 0000000000014140
[ 5.847256] ffff88033113e450 0000000000000008 ffff88033113e450 ffff88033113ead8
[ 5.854924] Call Trace:
[ 5.857420] [<ffffffff810858ad>] ? wq_worker_sleeping+0xd/0xa0
[ 5.863391] [<ffffffff8154c701>] ? __schedule+0x651/0x800
[ 5.868927] [<ffffffff8106e1dd>] ? release_task+0x36d/0x460
[ 5.874635] [<ffffffff8106fab2>] ? do_exit+0x782/0xad0
[ 5.879913] [<ffffffff81017636>] ? oops_end+0x96/0xe0
[ 5.885105] [<ffffffff8105c1b6>] ? no_context+0x106/0x380
[ 5.890639] [<ffffffff8105c90e>] ? __do_page_fault+0xae/0x550
[ 5.896524] [<ffffffff810a3350>] ? pick_next_task_fair+0x1b0/0x850
[ 5.902840] [<ffffffff810125eb>] ? __switch_to+0x14b/0x5d0
[ 5.908464] [<ffffffff81552be8>] ? page_fault+0x28/0x30
[ 5.913826] [<ffffffffa00f1877>] ? mvs_task_exec.isra.13+0x827/0xf10 [mvsas]
[ 5.921010] [<ffffffffa00f1e71>] ? mvs_task_exec.isra.13+0xe21/0xf10 [mvsas]
[ 5.928192] [<ffffffffa00d8c13>] ? sas_ata_qc_issue+0x1a3/0x250 [libsas]
[ 5.935029] [<ffffffffa0259875>] ? ata_qc_issue+0x1f5/0x380 [libata]
[ 5.941518] [<ffffffffa0259cef>] ? ata_exec_internal_sg+0x2ef/0x5f0 [libata]
[ 5.948702] [<ffffffffa025a048>] ? ata_exec_internal+0x58/0xa0 [libata]
[ 5.955449] [<ffffffffa00d89f2>] ? sas_ata_hard_reset+0xa2/0x120 [libsas]
[ 5.962375] [<ffffffffa025cfa5>] ? ata_phys_link_offline+0x15/0x30 [libata]
[ 5.969477] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 5.975967] [<ffffffffa025a525>] ? ata_dev_read_id+0x345/0x5d0 [libata]
[ 5.982716] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 5.989208] [<ffffffffa02647f1>] ? ata_eh_schedule_probe+0x141/0x180 [libata]
[ 5.996496] [<ffffffffa02676ef>] ? ata_eh_recover+0x7df/0x13b0 [libata]
[ 6.003245] [<ffffffffa025cfc0>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 6.010338] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 6.016829] [<ffffffffa025ce30>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 6.023838] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 6.030329] [<ffffffffa025cfc0>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 6.037425] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 6.043915] [<ffffffffa0268c56>] ? ata_do_eh+0x46/0xb0 [libata]
[ 6.049973] [<ffffffffa025ce30>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 6.056980] [<ffffffff8101d146>] ? native_sched_clock+0x26/0x90
[ 6.063038] [<ffffffffa02687e8>] ? ata_scsi_port_error_handler+0x448/0x7b0 [libata]
[ 6.070843] [<ffffffffa00d9046>] ? async_sas_ata_eh+0x46/0x70 [libsas]
[ 6.077506] [<ffffffff8108ca33>] ? async_run_entry_fn+0x43/0x160
[ 6.083648] [<ffffffff810849cd>] ? process_one_work+0x14d/0x3f0
[ 6.089705] [<ffffffff810853fb>] ? worker_thread+0x6b/0x4a0
[ 6.095414] [<ffffffff81085390>] ? rescuer_thread+0x310/0x310
[ 6.101297] [<ffffffff8108a1a3>] ? kthread+0xd3/0xf0
[ 6.106402] [<ffffffff8108a0d0>] ? kthread_create_on_node+0x180/0x180
[ 6.112976] [<ffffffff815509d8>] ? ret_from_fork+0x58/0x90
[ 6.118601] [<ffffffff8108a0d0>] ? kthread_create_on_node+0x180/0x180
[ 6.125175] Code: 10 04 00 00 48 8b 40 c8 48 c1 e8 02 83 e0 01 c3 66 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 48 8b 87 10 04 00 00 <48> 8b 40 d8 c3 66 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 66
[ 6.147461] RIP [<ffffffff8108a7bc>] kthread_data+0xc/0x20
[ 6.153128] RSP <ffff88032d86b600>
[ 6.156662] CR2: ffffffffffffffd8
[ 6.160024] ---[ end trace 50dd3aeb0a903cc3 ]---
[ 6.163260] scsi 5:0:0:0: Direct-Access Generic USB EDC 1.00 PQ: 0 ANSI: 2
[ 6.172847] Fixing recursive fault but reboot is needed!
[ 16.487514] random: nonblocking pool is initialized
[ 28.480381] INFO: rcu_sched detected stalls on CPUs/tasks: { 3 7} (detected by 2, t=5581 jiffies, g=-188, c=-189, q=50)
[ 28.491481] Task dump for CPU 3:
[ 28.494765] swapper/3 R running task 0 0 1 0x00000008
[ 28.501963] 000038ad1b4fe8cc ffffffffffffffff ffffffffffffff4e ffffffff8142366e
[ 28.509633] 0000000000000010 0000000000000217 ffff880331677e88 0000000000000018
[ 28.517301] ffffffff8142365e 0000000000000092 ffffffff818f60c0 ffffe8ffffcc6de0
[ 28.524969] Call Trace:
[ 28.527466] [<ffffffff8142366e>] ? cpuidle_enter_state+0x5e/0x160
[ 28.533693] [<ffffffff8142365e>] ? cpuidle_enter_state+0x4e/0x160
[ 28.539924] [<ffffffff810a936d>] ? cpu_startup_entry+0x34d/0x3f0
[ 28.546068] [<ffffffff810d9cc0>] ? tick_check_new_device+0xe0/0x110
[ 28.552472] [<ffffffff8104642e>] ? start_secondary+0x19e/0x1d0
[ 28.558438] Task dump for CPU 7:
[ 28.561715] kworker/u16:6 D ffff88032d800008 0 146 0 0x00000000
[ 28.568920] 0000000000000096 ffff88032d800000 ffff88032dc55c00 0000000000000001
[ 28.576582] ffff88032d8037b0 0000000000010000 ffff8800ba230040 ffff88032e173200
[ 28.584250] ffff88032d8037b0 000000000000002d 0000009a2484b70f 000000000000000b
[ 28.591919] Call Trace:
[ 28.594417] [<ffffffffa00f1877>] ? mvs_task_exec.isra.13+0x827/0xf10 [mvsas]
[ 28.601599] [<ffffffffa00f1e71>] ? mvs_task_exec.isra.13+0xe21/0xf10 [mvsas]
[ 28.608782] [<ffffffffa00d8c13>] ? sas_ata_qc_issue+0x1a3/0x250 [libsas]
[ 28.615618] [<ffffffffa0259875>] ? ata_qc_issue+0x1f5/0x380 [libata]
[ 28.622107] [<ffffffffa0259cef>] ? ata_exec_internal_sg+0x2ef/0x5f0 [libata]
[ 28.629290] [<ffffffffa025a048>] ? ata_exec_internal+0x58/0xa0 [libata]
[ 28.636040] [<ffffffffa00d89f2>] ? sas_ata_hard_reset+0xa2/0x120 [libsas]
[ 28.642962] [<ffffffffa025cfa5>] ? ata_phys_link_offline+0x15/0x30 [libata]
[ 28.651367] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 28.657858] [<ffffffffa025a525>] ? ata_dev_read_id+0x345/0x5d0 [libata]
[ 28.664607] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 28.671098] [<ffffffffa02647f1>] ? ata_eh_schedule_probe+0x141/0x180 [libata]
[ 28.678384] [<ffffffffa02676ef>] ? ata_eh_recover+0x7df/0x13b0 [libata]
[ 28.685134] [<ffffffffa025cfc0>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 28.692227] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 28.698719] [<ffffffffa025ce30>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 28.705727] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 28.712217] [<ffffffffa025cfc0>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 28.719314] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 28.725803] [<ffffffffa0268c56>] ? ata_do_eh+0x46/0xb0 [libata]
[ 28.731859] [<ffffffffa025ce30>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 28.738868] [<ffffffff8101d146>] ? native_sched_clock+0x26/0x90
[ 28.744928] [<ffffffffa02687e8>] ? ata_scsi_port_error_handler+0x448/0x7b0 [libata]
[ 28.752732] [<ffffffffa00d9046>] ? async_sas_ata_eh+0x46/0x70 [libsas]
[ 28.759394] [<ffffffff8108ca33>] ? async_run_entry_fn+0x43/0x160
[ 28.765537] [<ffffffff810849cd>] ? process_one_work+0x14d/0x3f0
[ 28.771593] [<ffffffff810853fb>] ? worker_thread+0x6b/0x4a0
[ 28.777304] [<ffffffff81085390>] ? rescuer_thread+0x310/0x310
[ 28.783186] [<ffffffff8108a1a3>] ? kthread+0xd3/0xf0
[ 28.788290] [<ffffffff8108a0d0>] ? kthread_create_on_node+0x180/0x180
[ 28.794867] [<ffffffff815509d8>] ? ret_from_fork+0x58/0x90
[ 28.800488] [<ffffffff8108a0d0>] ? kthread_create_on_node+0x180/0x180
[ 35.049176] ------------[ cut here ]------------
[ 35.053847] WARNING: CPU: 7 PID: 146 at /build/linux-gzeV2n/linux-3.19.3/kernel/watchdog.c:290 watchdog_overflow_callback+0x92/0xc0()
[ 35.065906] Watchdog detected hard LOCKUP on cpu 7
[ 35.070560] Modules linked in: usb_storage hid_generic usbhid hid ata_generic mvsas ehci_pci libsas ata_piix ehci_hcd libata e1000e scsi_transport_sas ptp crc32c_intel pps_core usbcore scsi_mod usb_common
[ 35.090107] CPU: 7 PID: 146 Comm: kworker/u16:6 Tainted: G D 3.19.0-trunk-amd64 #1 Debian 3.19.3-1~exp1
[ 35.100607] Hardware name: Supermicro X8SIE/X8SIE, BIOS 1.2a 06/27/2012
[ 35.107796] 0000000000000000 ffffffff81723ea8 ffffffff8154b001 ffff88033fdc5b60
[ 35.115483] ffffffff8106ced1 ffff880331146800[ 35.118656] ------------[ cut here ]------------
[ 35.118658] WARNING: CPU: 3 PID: 0 at /build/linux-gzeV2n/linux-3.19.3/kernel/watchdog.c:290 watchdog_overflow_callback+0x92/0xc0()
[ 35.118659] Watchdog detected hard LOCKUP on cpu 3Modules linked in: usb_storage hid_generic usbhid hid ata_generic mvsas ehci_pci libsas ata_piix ehci_hcd libata e1000e scsi_transport_sas ptp crc32c_intel pps_core usbcore scsi_mod usb_common
[ 35.159023] 0000000000000000 ffff88033fdc5c80
[ 35.163866] ffff88033fdc5ef8 0000000000000000 ffffffff8106cf4a ffffffff81723e80
[ 35.171536] Call Trace:
[ 35.174029] <NMI> [<ffffffff8154b001>] ? dump_stack+0x40/0x50
[ 35.180089] [<ffffffff8106ced1>] ? warn_slowpath_common+0x81/0xb0
[ 35.186316] [<ffffffff8106cf4a>] ? warn_slowpath_fmt+0x4a/0x50
[ 35.192287] [<ffffffff81109cd2>] ? watchdog_overflow_callback+0x92/0xc0
[ 35.199037] [<ffffffff81145e16>] ? __perf_event_overflow+0x86/0x230
[ 35.205442] [<ffffffff8102a689>] ? x86_perf_event_set_period+0xc9/0x170
[ 35.212190] [<ffffffff810319da>] ? intel_pmu_handle_irq+0x1ba/0x3a0
[ 35.218623] [<ffffffff8102962a>] ? perf_event_nmi_handler+0x2a/0x50
[ 35.225021] [<ffffffff8101d146>] ? native_sched_clock+0x26/0x90
[ 35.231078] [<ffffffff81017e16>] ? nmi_handle+0x86/0x120
[ 35.236527] [<ffffffff81018370>] ? default_do_nmi+0x40/0x110
[ 35.242325] [<ffffffff810184c8>] ? do_nmi+0x88/0xd0
[ 35.247341] [<ffffffff81552f61>] ? end_repeat_nmi+0x1e/0x2e
[ 35.253051] [<ffffffff8106fc06>] ? do_exit+0x8d6/0xad0
[ 35.258329] [<ffffffff81550512>] ? _raw_spin_lock_irq+0x32/0x40
[ 35.264383] [<ffffffff81550512>] ? _raw_spin_lock_irq+0x32/0x40
[ 35.270440] [<ffffffff81550512>] ? _raw_spin_lock_irq+0x32/0x40
[ 35.276495] <<EOE>> [<ffffffff8154c153>] ? __schedule+0xa3/0x800
[ 35.282812] [<ffffffff81549e3c>] ? printk+0x4d/0x52
[ 35.287831] [<ffffffff8106fc06>] ? do_exit+0x8d6/0xad0
[ 35.293107] [<ffffffff81017636>] ? oops_end+0x96/0xe0
[ 35.298297] [<ffffffff8105c1b6>] ? no_context+0x106/0x380
[ 35.303835] [<ffffffff8105c90e>] ? __do_page_fault+0xae/0x550
[ 35.309718] [<ffffffff812d7fa5>] ? vsnprintf+0x295/0x5d0
[ 35.315165] [<ffffffff812d80df>] ? vsnprintf+0x3cf/0x5d0
[ 35.320616] [<ffffffff812d92c3>] ? delay_tsc+0x33/0x60
[ 35.325892] [<ffffffff81552be8>] ? page_fault+0x28/0x30
[ 35.331257] [<ffffffff8108a7bc>] ? kthread_data+0xc/0x20
[ 35.336706] [<ffffffff810858ad>] ? wq_worker_sleeping+0xd/0xa0
[ 35.342675] [<ffffffff8154c701>] ? __schedule+0x651/0x800
[ 35.348211] [<ffffffff8106e1dd>] ? release_task+0x36d/0x460
[ 35.353923] [<ffffffff8106fab2>] ? do_exit+0x782/0xad0
[ 35.359199] [<ffffffff81017636>] ? oops_end+0x96/0xe0
[ 35.364390] [<ffffffff8105c1b6>] ? no_context+0x106/0x380
[ 35.369925] [<ffffffff8105c90e>] ? __do_page_fault+0xae/0x550
[ 35.375808] [<ffffffff810a3350>] ? pick_next_task_fair+0x1b0/0x850
[ 35.382126] [<ffffffff810125eb>] ? __switch_to+0x14b/0x5d0
[ 35.387749] [<ffffffff81552be8>] ? page_fault+0x28/0x30
[ 35.393114] [<ffffffffa00f1877>] ? mvs_task_exec.isra.13+0x827/0xf10 [mvsas]
[ 35.400296] [<ffffffffa00f1e71>] ? mvs_task_exec.isra.13+0xe21/0xf10 [mvsas]
[ 35.407478] [<ffffffffa00d8c13>] ? sas_ata_qc_issue+0x1a3/0x250 [libsas]
[ 35.414314] [<ffffffffa0259875>] ? ata_qc_issue+0x1f5/0x380 [libata]
[ 35.420805] [<ffffffffa0259cef>] ? ata_exec_internal_sg+0x2ef/0x5f0 [libata]
[ 35.427987] [<ffffffffa025a048>] ? ata_exec_internal+0x58/0xa0 [libata]
[ 35.434737] [<ffffffffa00d89f2>] ? sas_ata_hard_reset+0xa2/0x120 [libsas]
[ 35.441662] [<ffffffffa025cfa5>] ? ata_phys_link_offline+0x15/0x30 [libata]
[ 35.448764] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 35.455256] [<ffffffffa025a525>] ? ata_dev_read_id+0x345/0x5d0 [libata]
[ 35.462002] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 35.468494] [<ffffffffa02647f1>] ? ata_eh_schedule_probe+0x141/0x180 [libata]
[ 35.475781] [<ffffffffa02676ef>] ? ata_eh_recover+0x7df/0x13b0 [libata]
[ 35.482556] [<ffffffffa025cfc0>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 35.489652] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 35.496143] [<ffffffffa025ce30>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 35.503149] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 35.509641] [<ffffffffa025cfc0>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 35.516736] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 35.523227] [<ffffffffa0268c56>] ? ata_do_eh+0x46/0xb0 [libata]
[ 35.529284] [<ffffffffa025ce30>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 35.536290] [<ffffffff8101d146>] ? native_sched_clock+0x26/0x90
[ 35.542350] [<ffffffffa02687e8>] ? ata_scsi_port_error_handler+0x448/0x7b0 [libata]
[ 35.550155] [<ffffffffa00d9046>] ? async_sas_ata_eh+0x46/0x70 [libsas]
[ 35.556819] [<ffffffff8108ca33>] ? async_run_entry_fn+0x43/0x160
[ 35.562960] [<ffffffff810849cd>] ? process_one_work+0x14d/0x3f0
[ 35.569016] [<ffffffff810853fb>] ? worker_thread+0x6b/0x4a0
[ 35.574726] [<ffffffff81085390>] ? rescuer_thread+0x310/0x310
[ 35.580610] [<ffffffff8108a1a3>] ? kthread+0xd3/0xf0
[ 35.585714] [<ffffffff8108a0d0>] ? kthread_create_on_node+0x180/0x180
[ 35.592289] [<ffffffff815509d8>] ? ret_from_fork+0x58/0x90
[ 35.597912] [<ffffffff8108a0d0>] ? kthread_create_on_node+0x180/0x180
[ 35.604488] ---[ end trace 50dd3aeb0a903cc4 ]---
[ 35.604489] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G D 3.19.0-trunk-amd64 #1 Debian 3.19.3-1~exp1
[ 35.604490] Hardware name: Supermicro X8SIE/X8SIE, BIOS 1.2a 06/27/2012
[ 35.604492] 0000000000000000 ffffffff81723ea8 ffffffff8154b001 ffff88033fcc5b60
[ 35.604493] ffffffff8106ced1 ffff880331003800 0000000000000000 ffff88033fcc5c80
[ 35.604494] ffff88033fcc5ef8 0000000000000000 ffffffff8106cf4a ffffffff81723e80
[ 35.604494] Call Trace:
[ 35.604496] <NMI> [<ffffffff8154b001>] ? dump_stack+0x40/0x50
[ 35.604498] [<ffffffff8106ced1>] ? warn_slowpath_common+0x81/0xb0
[ 35.604499] [<ffffffff8106cf4a>] ? warn_slowpath_fmt+0x4a/0x50
[ 35.604500] [<ffffffff81109cd2>] ? watchdog_overflow_callback+0x92/0xc0
[ 35.604502] [<ffffffff81145e16>] ? __perf_event_overflow+0x86/0x230
[ 35.604503] [<ffffffff8102a689>] ? x86_perf_event_set_period+0xc9/0x170
[ 35.604504] [<ffffffff810319da>] ? intel_pmu_handle_irq+0x1ba/0x3a0
[ 35.604506] [<ffffffff8102962a>] ? perf_event_nmi_handler+0x2a/0x50
[ 35.604508] [<ffffffff8101d146>] ? native_sched_clock+0x26/0x90
[ 35.604509] [<ffffffff81017e16>] ? nmi_handle+0x86/0x120
[ 35.604510] [<ffffffff81018370>] ? default_do_nmi+0x40/0x110
[ 35.604511] [<ffffffff810184c8>] ? do_nmi+0x88/0xd0
[ 35.604513] [<ffffffff81552f61>] ? end_repeat_nmi+0x1e/0x2e
[ 35.604514] [<ffffffff815506d2>] ? _raw_spin_lock+0x32/0x40
[ 35.604516] [<ffffffff815506d2>] ? _raw_spin_lock+0x32/0x40
[ 35.604517] [<ffffffff815506d2>] ? _raw_spin_lock+0x32/0x40
[ 35.604523] <<EOE>> <IRQ> [<ffffffffa00f5310>] ? mvs_64xx_isr+0x30/0x100 [mvsas]
[ 35.604527] [<ffffffffa00ef15f>] ? mvs_interrupt+0x6f/0xa0 [mvsas]
[ 35.604528] [<ffffffff810bcd35>] ? handle_irq_event_percpu+0x75/0x190
[ 35.604530] [<ffffffff810bce88>] ? handle_irq_event+0x38/0x50
[ 35.604531] [<ffffffff810bfa94>] ? handle_fasteoi_irq+0x84/0x140
[ 35.604533] [<ffffffff8101648d>] ? handle_irq+0x1d/0x30
[ 35.604534] [<ffffffff81553986>] ? do_IRQ+0x46/0xe0
[ 35.604536] [<ffffffff8155182d>] ? common_interrupt+0x6d/0x6d
[ 35.604538] <EOI> [<ffffffff8142366e>] ? cpuidle_enter_state+0x5e/0x160
[ 35.604539] [<ffffffff8142365e>] ? cpuidle_enter_state+0x4e/0x160
[ 35.604540] [<ffffffff810a936d>] ? cpu_startup_entry+0x34d/0x3f0
[ 35.604542] [<ffffffff810d9cc0>] ? tick_check_new_device+0xe0/0x110
[ 35.604544] [<ffffffff8104642e>] ? start_secondary+0x19e/0x1d0
[ 35.604545] ---[ end trace 50dd3aeb0a903cc5 ]---
[ 38.645536] ------------[ cut here ]------------
[ 38.650204] WARNING: CPU: 1 PID: 0 at /build/linux-gzeV2n/linux-3.19.3/kernel/watchdog.c:290 watchdog_overflow_callback+0x92/0xc0()
[ 38.662090] Watchdog detected hard LOCKUP on cpu 1
[ 38.666742] Modules linked in: usb_storage hid_generic usbhid hid ata_generic mvsas ehci_pci libsas ata_piix ehci_hcd libata e1000e scsi_transport_sas ptp crc32c_intel pps_core usbcore scsi_mod usb_common
[ 38.686238] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G D W 3.19.0-trunk-amd64 #1 Debian 3.19.3-1~exp1
[ 38.696210] Hardware name: Supermicro X8SIE/X8SIE, BIOS 1.2a 06/27/2012
[ 38.703392] 0000000000000000 ffffffff81723ea8 ffffffff8154b001 ffff88033fc45b60
[ 38.711052] ffffffff8106ced1 ffff880331631800 0000000000000001 ffff88033fc45c80
[ 38.718719] ffff88033fc45ef8 0000000000000000 ffffffff8106cf4a ffffffff81723e80
[ 38.727549] Call Trace:
[ 38.730045] <NMI> [<ffffffff8154b001>] ? dump_stack+0x40/0x50
[ 38.736103] [<ffffffff8106ced1>] ? warn_slowpath_common+0x81/0xb0
[ 38.742331] [<ffffffff8106cf4a>] ? warn_slowpath_fmt+0x4a/0x50
[ 38.748301] [<ffffffff81109cd2>] ? watchdog_overflow_callback+0x92/0xc0
[ 38.755051] [<ffffffff81145e16>] ? __perf_event_overflow+0x86/0x230
[ 38.761454] [<ffffffff8102a689>] ? x86_perf_event_set_period+0xc9/0x170
[ 38.768204] [<ffffffff810319da>] ? intel_pmu_handle_irq+0x1ba/0x3a0
[ 38.774608] [<ffffffff8102962a>] ? perf_event_nmi_handler+0x2a/0x50
[ 38.781009] [<ffffffff8101d146>] ? native_sched_clock+0x26/0x90
[ 38.787067] [<ffffffff81017e16>] ? nmi_handle+0x86/0x120
[ 38.792516] [<ffffffff81018370>] ? default_do_nmi+0x40/0x110
[ 38.798313] [<ffffffff810184c8>] ? do_nmi+0x88/0xd0
[ 38.803330] [<ffffffff81552f61>] ? end_repeat_nmi+0x1e/0x2e
[ 38.809038] [<ffffffff815506d7>] ? _raw_spin_lock+0x37/0x40
[ 38.814749] [<ffffffff815506d7>] ? _raw_spin_lock+0x37/0x40
[ 38.820457] [<ffffffff815506d7>] ? _raw_spin_lock+0x37/0x40
[ 38.826168] <<EOE>> <IRQ> [<ffffffff810a5272>] ? sched_rt_period_timer+0xe2/0x2e0
[ 38.834124] [<ffffffff810a5190>] ? put_prev_task_rt+0x50/0x50
[ 38.840007] [<ffffffff810cd50b>] ? __run_hrtimer+0x6b/0x1c0
[ 38.845714] [<ffffffff8101cc25>] ? read_tsc+0x5/0x10
[ 38.850810] [<ffffffff810cd919>] ? hrtimer_interrupt+0xf9/0x230
[ 38.856868] [<ffffffff81054506>] ? hpet_interrupt_handler+0x16/0x40
[ 38.863270] [<ffffffff810bcd35>] ? handle_irq_event_percpu+0x75/0x190
[ 38.869845] [<ffffffff810bce88>] ? handle_irq_event+0x38/0x50
[ 38.875730] [<ffffffff810bfd81>] ? handle_edge_irq+0x71/0x120
[ 38.881613] [<ffffffff8101648d>] ? handle_irq+0x1d/0x30
[ 38.887011] [<ffffffff81553986>] ? do_IRQ+0x46/0xe0
[ 38.892027] [<ffffffff8155182d>] ? common_interrupt+0x6d/0x6d
[ 38.897910] <EOI> [<ffffffff8142366e>] ? cpuidle_enter_state+0x5e/0x160
[ 38.904832] [<ffffffff8142365e>] ? cpuidle_enter_state+0x4e/0x160
[ 38.911064] [<ffffffff810a936d>] ? cpu_startup_entry+0x34d/0x3f0
[ 38.917207] [<ffffffff8104642e>] ? start_secondary+0x19e/0x1d0
[ 38.923175] ---[ end trace 50dd3aeb0a903cc6 ]---
[ 44.432170] NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [systemd-udevd:101]
[ 44.440098] Modules linked in: usb_storage hid_generic usbhid hid ata_generic mvsas ehci_pci libsas ata_piix ehci_hcd libata e1000e scsi_transport_sas ptp crc32c_intel pps_core usbcore scsi_mod usb_common
[ 44.459360] CPU: 0 PID: 101 Comm: systemd-udevd Tainted: G D W 3.19.0-trunk-amd64 #1 Debian 3.19.3-1~exp1
[ 44.469853] Hardware name: Supermicro X8SIE/X8SIE, BIOS 1.2a 06/27/2012
[ 44.477036] task: ffff8803312e60d0 ti: ffff88032e1f8000 task.ti: ffff88032e1f8000
[ 44.484582] RIP: 0010:[<ffffffff810e0ca2>] [<ffffffff810e0ca2>] smp_call_function_many+0x212/0x270
[ 44.493739] RSP: 0018:ffff88032e1fbaf8 EFLAGS: 00000202
[ 44.499102] RAX: ffff88033fcd7918 RBX: 00000000000000fe RCX: 0000000000000003
[ 44.506287] RDX: ffff88033fcd7918 RSI: 0000000000000200 RDI: 0000000000000000
[ 44.513468] RBP: ffff88033fc14f40 R08: ffff88033fc14f48 R09: 0000000000000000
[ 44.520651] R10: 0000000000000008 R11: 000000000000a44a R12: 0000000000000002
[ 44.527834] R13: 0000000000000000 R14: 0000000000000293 R15: ffff88033fff3d80
[ 44.535018] FS: 00007f89f9312880(0000) GS:ffff88033fc00000(0000) knlGS:0000000000000000
[ 44.543171] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 44.548966] CR2: 00007f89f9319000 CR3: 000000032e061000 CR4: 00000000000007f0
[ 44.556149] Stack:
[ 44.558211] 0000000200000141 0000000100000000 0000004000000040 ffff88032e1fbc40
[ 44.565871] ffffffff81062fa0 ffff88032e1fbb70 ffff88033fdd0428 ffff88032e1fbc48
[ 44.573530] ffff88032e1fbb18 ffffffff810e0e68 ffff88032e1fbc40 ffff88033fdd04c0
[ 44.581190] Call Trace:
[ 44.583686] [<ffffffff81062fa0>] ? rbt_memtype_copy_nth_element+0xa0/0xa0
[ 44.590609] [<ffffffff810e0e68>] ? on_each_cpu+0x28/0x50
[ 44.596060] [<ffffffff81063710>] ? flush_tlb_kernel_range+0x60/0x70
[ 44.602463] [<ffffffff8118a553>] ? __purge_vmap_area_lazy+0x3c3/0x430
[ 44.609040] [<ffffffff810e1f30>] ? mod_find_symname+0x90/0x90
[ 44.614921] [<ffffffff8118a72c>] ? vm_unmap_aliases+0x16c/0x180
[ 44.620981] [<ffffffff8105f5ad>] ? change_page_attr_set_clr+0xdd/0x450
[ 44.627641] [<ffffffff8105fdcd>] ? set_memory_ro+0x2d/0x40
[ 44.633263] [<ffffffff810e31da>] ? set_section_ro_nx+0x7a/0x90
[ 44.639235] [<ffffffff810e6430>] ? load_module+0x1a80/0x2740
[ 44.645032] [<ffffffff810e725d>] ? SyS_finit_module+0x7d/0xa0
[ 44.650914] [<ffffffff81550a8d>] ? system_call_fast_compare_end+0xc/0x11
[ 44.657749] Code: 81 00 89 c1 0f 8d 7c fe ff ff 48 98 48 8b 55 00 48 03 14 c5 40 4a 8f 81 f6 42 18 01 48 89 d0 74 c8 0f 1f 84 00 00 00 00 00 f3 90 <f6> 40 18 01 75 f8 eb b6 0f b6 4c 24 0c 48 83 c4 18 4c 89 ea 5b
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: mvsas panics and dies when attached to a port extender on newer kernels
2015-04-13 15:11 mvsas panics and dies when attached to a port extender on newer kernels Adam Talbot
@ 2015-04-23 11:10 ` Ceriel Jacobs
2015-04-23 16:27 ` Adam Talbot
0 siblings, 1 reply; 12+ messages in thread
From: Ceriel Jacobs @ 2015-04-23 11:10 UTC (permalink / raw)
To: Adam Talbot, linux-scsi
Your mvsas port expander bug might be related to bug# 81861
https://bugzilla.kernel.org/show_bug.cgi?id=81861
Adam Talbot schreef op 13-04-15 om 17:11:
> To make a very long debugging story short, I think there is an issues/bug
> with the mvsas driver. It works, with older kernels, and breaks on
> newer kernels.
>
> My Debian Jessie system was running great on a 3.18 kernel. Changed
> cases to a newer supermicro case with a SAS expander backplane (SAS933EL). That
> was the only hardware change. Now, when ever I boot, the system kernel panics.
>
> 3.2.65-1+deb7u2 works
> 3.9.0 Gentoo CD works
> 3.16+ all fail
> Attached are 3 kernel panics on 3.16+ kernels.
>
> Motherboard is a Supermicro X8SIE, with a "Marvell Technology Group Ltd.
> 88SE6440 SAS/SATA PCIe controller"
>
> Is this a known bug?
>
> At this point I have two options:
> Stick with the old kernel (yuck)
> Buy a new card running a better supported chipset
>
> Any help would be greatly appreciated
> Thanks
> Adam
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: mvsas panics and dies when attached to a port extender on newer kernels
2015-04-23 11:10 ` Ceriel Jacobs
@ 2015-04-23 16:27 ` Adam Talbot
0 siblings, 0 replies; 12+ messages in thread
From: Adam Talbot @ 2015-04-23 16:27 UTC (permalink / raw)
To: Ceriel Jacobs; +Cc: linux-scsi
Oh! Glad to see it was not just me suffering from this bug. Still
running a patched 4.0.0 kernel through the SAS expander. Everything
seems happy.
On Thu, Apr 23, 2015 at 4:10 AM, Ceriel Jacobs <linux-scsi@crashplan.pro> wrote:
> Your mvsas port expander bug might be related to bug# 81861
>
> https://bugzilla.kernel.org/show_bug.cgi?id=81861
>
> Adam Talbot schreef op 13-04-15 om 17:11:
>>
>> To make a very long debugging story short, I think there is an issues/bug
>> with the mvsas driver. It works, with older kernels, and breaks on
>> newer kernels.
>>
>> My Debian Jessie system was running great on a 3.18 kernel. Changed
>> cases to a newer supermicro case with a SAS expander backplane (SAS933EL).
>> That
>> was the only hardware change. Now, when ever I boot, the system kernel
>> panics.
>>
>> 3.2.65-1+deb7u2 works
>> 3.9.0 Gentoo CD works
>> 3.16+ all fail
>> Attached are 3 kernel panics on 3.16+ kernels.
>>
>> Motherboard is a Supermicro X8SIE, with a "Marvell Technology Group Ltd.
>> 88SE6440 SAS/SATA PCIe controller"
>>
>> Is this a known bug?
>>
>> At this point I have two options:
>> Stick with the old kernel (yuck)
>> Buy a new card running a better supported chipset
>>
>> Any help would be greatly appreciated
>> Thanks
>> Adam
>>
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* mvsas panics and dies when attached to a port extender on newer kernels
@ 2015-04-14 21:03 Adam Talbot
2015-04-14 21:16 ` James Bottomley
0 siblings, 1 reply; 12+ messages in thread
From: Adam Talbot @ 2015-04-14 21:03 UTC (permalink / raw)
To: linux-scsi
[-- Attachment #1: Type: text/plain, Size: 789 bytes --]
To make a very long debugging story short, I think there is an issues/bug
with the mvsas driver. It works, with older kernels, and breaks on
newer kernels.
My Debian Jessie system was running great on a 3.18 kernel. Changed
cases to a newer supermicro case with a SAS expander backplane (SAS933EL). That
was the only hardware change. Now, when ever I boot, the system kernel panics.
3.2.65-1+deb7u2 works
3.9.0 Gentoo CD works
3.16+ all fail
Attached are 3 kernel panics on 3.16+ kernels.
Motherboard is a Supermicro X8SIE, with a "Marvell Technology Group Ltd.
88SE6440 SAS/SATA PCIe controller"
Is this a known bug?
At this point I have two options:
Stick with the old kernel (yuck)
Buy a new card running a better supported chipset
Any help would be greatly appreciated
Thanks
[-- Attachment #2: 3.18.0-kernel-panic.txt --]
[-- Type: text/plain, Size: 10168 bytes --]
[ 5.295367] scsi host0: mvsas
[ 5.555699] BUG: unable to handle kernel paging request at 00000030747371bc
[ 5.562813] IP: [<ffffffffa00cd7ed>] mvs_task_prep+0x78d/0xe40 [mvsas]
[ 5.569440] PGD 0
[ 5.571555] Oops: 0000 [#1] SMP
[ 5.574934] Modules linked in: usb_storage hid_generic usbhid hid ata_generic mvsas libsas ata_piix ehci_pci ehci_hcd libata scsi_transport_sas e1000e ptp crc32c_intel usbcore pps_core usb_common scsi_mod
[ 5.594188] CPU: 3 PID: 78 Comm: kworker/u16:1 Not tainted 3.18.0-trunk-amd64 #1 Debian 3.18.5-1~exp1
[ 5.603467] Hardware name: Supermicro X8SIE/X8SIE, BIOS 1.2a 06/27/2012
[ 5.610651] Workqueue: events_unbound async_run_entry_fn
[ 5.616056] task: ffff88032fb6cc40 ti: ffff88032fb78000 task.ti: ffff88032fb78000
[ 5.624779] RIP: 0010:[<ffffffffa00cd7ed>] [<ffffffffa00cd7ed>] mvs_task_prep+0x78d/0xe40 [mvsas]
[ 5.633853] RSP: 0018:ffff88032fb7b888 EFLAGS: 00010097
[ 5.639217] RAX: 000000000000002c RBX: 0000003074736f68 RCX: 0000000000000000
[ 5.646399] RDX: ffff8800372c3618 RSI: 0000000000000000 RDI: 0000000000001000
[ 5.653580] RBP: 0000000000000000 R08: ffff88032f572400 R09: ffff8800372a8000
[ 5.660763] R10: ffffc900018f6000 R11: 0000000000010000 R12: 0000000000000000
[ 5.667947] R13: ffff8800372c3618 R14: ffff8800372c0000 R15: ffff88032f23ce40
[ 5.675129] FS: 0000000000000000(0000) GS:ffff88033fcc0000(0000) knlGS:0000000000000000
[ 5.683284] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 5.689079] CR2: 00000030747371bc CR3: 0000000001813000 CR4: 00000000000007e0
[ 5.696261] Stack:
[ 5.698323] ffff88032fb7b934 ffff8800372c3618 0000000000000000 ffff8800372c2678
[ 5.705983] ffff8800372c1e38 ffff8800372a8000 ffff88032f572400 ffff88032fb7bfd8
[ 5.713642] 0000000000013740 ffff88032fb6cc40 0000000300000001 ffffffff8109904a
[ 5.721310] Call Trace:
[ 5.723810] [<ffffffff8109904a>] ? select_idle_sibling+0x10a/0x120
[ 5.730125] [<ffffffffa00cdef7>] ? mvs_task_exec.isra.13+0x57/0xd0 [mvsas]
[ 5.737135] [<ffffffffa00ce9f9>] ? mvs_queue_command+0x2e9/0x2f0 [mvsas]
[ 5.743971] [<ffffffffa017a7cc>] ? sas_ata_qc_issue+0x22c/0x270 [libsas]
[ 5.750808] [<ffffffffa028e845>] ? ata_qc_issue+0x1f5/0x380 [libata]
[ 5.757296] [<ffffffffa028ecbf>] ? ata_exec_internal_sg+0x2ef/0x5f0 [libata]
[ 5.764480] [<ffffffffa028f018>] ? ata_exec_internal+0x58/0xa0 [libata]
[ 5.771229] [<ffffffffa017a4cd>] ? sas_ata_hard_reset+0x9d/0x140 [libsas]
[ 5.778151] [<ffffffffa0291f35>] ? ata_phys_link_offline+0x15/0x30 [libata]
[ 5.785247] [<ffffffffa028f3b5>] ? ata_dev_read_id+0x205/0x5c0 [libata]
[ 5.791997] [<ffffffffa017a430>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 5.798487] [<ffffffffa02996c1>] ? ata_eh_schedule_probe+0x141/0x180 [libata]
[ 5.805774] [<ffffffffa029c5da>] ? ata_eh_recover+0x7da/0x1370 [libata]
[ 5.812524] [<ffffffffa0291f50>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 5.819619] [<ffffffffa017a430>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 5.826109] [<ffffffffa0291dc0>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 5.833119] [<ffffffffa017a430>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 5.839609] [<ffffffffa0291f50>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 5.846703] [<ffffffffa017a430>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 5.853194] [<ffffffffa029db06>] ? ata_do_eh+0x46/0xb0 [libata]
[ 5.859251] [<ffffffffa0291dc0>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 5.866262] [<ffffffff8101c206>] ? native_sched_clock+0x26/0x90
[ 5.872318] [<ffffffffa029d698>] ? ata_scsi_port_error_handler+0x448/0x7b0 [libata]
[ 5.880122] [<ffffffffa017abb6>] ? async_sas_ata_eh+0x46/0x70 [libsas]
[ 5.886786] [<ffffffff8108a973>] ? async_run_entry_fn+0x43/0x160
[ 5.892930] [<ffffffff81082a8e>] ? process_one_work+0x14e/0x3f0
[ 5.898985] [<ffffffff8108338b>] ? worker_thread+0x6b/0x4a0
[ 5.904693] [<ffffffff81083320>] ? rescuer_thread+0x290/0x290
[ 5.910578] [<ffffffff810880e3>] ? kthread+0xd3/0xf0
[ 5.915680] [<ffffffff81088010>] ? kthread_create_on_node+0x180/0x180
[ 5.922260] [<ffffffff8154543c>] ? ret_from_fork+0x7c/0xb0
[ 5.927880] [<ffffffff81088010>] ? kthread_create_on_node+0x180/0x180
[ 5.934455] Code: 8b 80 58 01 00 00 48 8b 1c c8 0f 84 29 06 00 00 41 8b 46 58 48 8d 4c ad 00 bf 00 10 00 00 48 8d 4c 4d 00 41 89 84 ce d4 55 02 00 <8b> 8b 54 02 00 00 49 8b 76 48 d3 e7 0f b6 8a 84 00 00 00 81 cf
[ 5.956722] RIP [<ffffffffa00cd7ed>] mvs_task_prep+0x78d/0xe40 [mvsas]
[ 5.963438] RSP <ffff88032fb7b888>
[ 5.966973] CR2: 00000030747371bc
[ 5.970337] ---[ end trace d307165136b3807d ]---
[ 5.987492] BUG: unable to handle kernel paging request at ffffffffffffffd8
[ 5.994605] IP: [<ffffffff810886fc>] kthread_data+0xc/0x20
[ 6.000183] PGD 1816067 PUD 1818067 PMD 0
[ 6.004483] Oops: 0000 [#2] SMP
[ 6.007871] Modules linked in: usb_storage hid_generic usbhid hid ata_generic mvsas libsas ata_piix ehci_pci ehci_hcd libata scsi_transport_sas e1000e ptp crc32c_intel usbcore pps_core usb_common scsi_mod
[ 6.027123] CPU: 3 PID: 78 Comm: kworker/u16:1 Tainted: G D 3.18.0-trunk-amd64 #1 Debian 3.18.5-1~exp1
[ 6.037529] Hardware name: Supermicro X8SIE/X8SIE, BIOS 1.2a 06/27/2012
[ 6.044715] task: ffff88032fb6cc40 ti: ffff88032fb78000 task.ti: ffff88032fb78000
[ 6.052258] RIP: 0010:[<ffffffff810886fc>] [<ffffffff810886fc>] kthread_data+0xc/0x20
[ 6.060288] RSP: 0018:ffff88032fb7b540 EFLAGS: 00010002
[ 6.065644] RAX: 0000000000000000 RBX: 0000000000000003 RCX: 0000000000000000
[ 6.072827] RDX: 000000000000000f RSI: 0000000000000003 RDI: ffff88032fb6cc40
[ 6.080009] RBP: ffff88032fb6cc40 R08: 0000000000000000 R09: 000000000000001f
[ 6.087192] R10: ffffffff81afdec0 R11: 000000000000001a R12: ffff88033fcd3740
[ 6.094374] R13: ffff88032fb6d0a8 R14: 0000000000000003 R15: 0000000000030001
[ 6.101560] FS: 0000000000000000(0000) GS:ffff88033fcc0000(0000) knlGS:0000000000000000
[ 6.109712] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 6.115507] CR2: 0000000000000028 CR3: 0000000001813000 CR4: 00000000000007e0
[ 6.122690] Stack:
[ 6.124753] ffffffff8108383d 0000000000000000 ffffffff81540fbf 0000000000000000
[ 6.132422] ffffffff8106c24e 0000000000013740 ffff88032fb7bfd8 0000000000013740
[ 6.140090] ffff88032fb6cc40 0000000000000008 ffff88032fb6d2c8 ffff88032fb6cc30
[ 6.147749] Call Trace:
[ 6.150244] [<ffffffff8108383d>] ? wq_worker_sleeping+0xd/0xa0
[ 6.156215] [<ffffffff81540fbf>] ? __schedule+0x66f/0x7f0
[ 6.161751] [<ffffffff8106c24e>] ? release_task+0x36e/0x470
[ 6.167460] [<ffffffff8106dc67>] ? do_exit+0x857/0xad0
[ 6.172739] [<ffffffff810166b7>] ? oops_end+0x97/0xe0
[ 6.177928] [<ffffffff8105a0f6>] ? no_context+0x106/0x380
[ 6.183463] [<ffffffff8105a8c6>] ? __do_page_fault+0xb6/0x530
[ 6.189349] [<ffffffff812c7c20>] ? cpumask_next_and+0x30/0x50
[ 6.195230] [<ffffffff8109fca2>] ? load_balance+0x1f2/0x880
[ 6.200941] [<ffffffff8101c206>] ? native_sched_clock+0x26/0x90
[ 6.206997] [<ffffffff815474e8>] ? page_fault+0x28/0x30
[ 6.212361] [<ffffffffa00cd7ed>] ? mvs_task_prep+0x78d/0xe40 [mvsas]
[ 6.218850] [<ffffffffa00cde25>] ? mvs_task_prep+0xdc5/0xe40 [mvsas]
[ 6.225340] [<ffffffff8109904a>] ? select_idle_sibling+0x10a/0x120
[ 6.231657] [<ffffffffa00cdef7>] ? mvs_task_exec.isra.13+0x57/0xd0 [mvsas]
[ 6.238667] [<ffffffffa00ce9f9>] ? mvs_queue_command+0x2e9/0x2f0 [mvsas]
[ 6.245502] [<ffffffffa017a7cc>] ? sas_ata_qc_issue+0x22c/0x270 [libsas]
[ 6.252340] [<ffffffffa028e845>] ? ata_qc_issue+0x1f5/0x380 [libata]
[ 6.258830] [<ffffffffa028ecbf>] ? ata_exec_internal_sg+0x2ef/0x5f0 [libata]
[ 6.266013] [<ffffffffa028f018>] ? ata_exec_internal+0x58/0xa0 [libata]
[ 6.272759] [<ffffffffa017a4cd>] ? sas_ata_hard_reset+0x9d/0x140 [libsas]
[ 6.279685] [<ffffffffa0291f35>] ? ata_phys_link_offline+0x15/0x30 [libata]
[ 6.286788] [<ffffffffa028f3b5>] ? ata_dev_read_id+0x205/0x5c0 [libata]
[ 6.293537] [<ffffffffa017a430>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 6.300028] [<ffffffffa02996c1>] ? ata_eh_schedule_probe+0x141/0x180 [libata]
[ 6.307315] [<ffffffffa029c5da>] ? ata_eh_recover+0x7da/0x1370 [libata]
[ 6.314065] [<ffffffffa0291f50>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 6.321161] [<ffffffffa017a430>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 6.327651] [<ffffffffa0291dc0>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 6.334659] [<ffffffffa017a430>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 6.341150] [<ffffffffa0291f50>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 6.348246] [<ffffffffa017a430>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 6.354736] [<ffffffffa029db06>] ? ata_do_eh+0x46/0xb0 [libata]
[ 6.360792] [<ffffffffa0291dc0>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 6.367801] [<ffffffff8101c206>] ? native_sched_clock+0x26/0x90
[ 6.373859] [<ffffffffa029d698>] ? ata_scsi_port_error_handler+0x448/0x7b0 [libata]
[ 6.381665] [<ffffffffa017abb6>] ? async_sas_ata_eh+0x46/0x70 [libsas]
[ 6.388327] [<ffffffff8108a973>] ? async_run_entry_fn+0x43/0x160
[ 6.394468] [<ffffffff81082a8e>] ? process_one_work+0x14e/0x3f0
[ 6.400527] [<ffffffff8108338b>] ? worker_thread+0x6b/0x4a0
[ 6.406236] [<ffffffff81083320>] ? rescuer_thread+0x290/0x290
[ 6.412119] [<ffffffff810880e3>] ? kthread+0xd3/0xf0
[ 6.417221] [<ffffffff81088010>] ? kthread_create_on_node+0x180/0x180
[ 6.423798] [<ffffffff8154543c>] ? ret_from_fork+0x7c/0xb0
[ 6.429421] [<ffffffff81088010>] ? kthread_create_on_node+0x180/0x180
[ 6.435995] Code: 10 04 00 00 48 8b 40 c8 48 c1 e8 02 83 e0 01 c3 66 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 48 8b 87 10 04 00 00 <48> 8b 40 d8 c3 66 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 66
[ 6.458265] RIP [<ffffffff810886fc>] kthread_data+0xc/0x20
[ 6.463931] RSP <ffff88032fb7b540>
[ 6.467466] CR2: ffffffffffffffd8
[ 6.470827] ---[ end trace d307165136b3807e ]---
[ 6.475490] Fixing recursive fault but reboot is needed!
[-- Attachment #3: 3.16.0-kernel-panic.txt --]
[-- Type: text/plain, Size: 9317 bytes --]
[ 5.192410] scsi0 : mvsas
[ 5.456795] general protection fault: 0000 [#1] SMP
[ 5.461913] Modules linked in: usb_storage hid_generic usbhid hid ata_generic mvsas libsas ata_piix ehci_pci libata ehci_hcd scsi_transport_sas e1000e ptp crc32c_intel pps_core usbcore scsi_mod usb_common
[ 5.481174] CPU: 3 PID: 138 Comm: kworker/u16:5 Not tainted 3.16.0-4-amd64 #1 Debian 3.16.7-ckt7-1
[ 5.490194] Hardware name: Supermicro X8SIE/X8SIE, BIOS 1.2a 06/27/2012
[ 5.497378] Workqueue: events_unbound async_run_entry_fn
[ 5.502791] task: ffff880036f68010 ti: ffff880036f74000 task.ti: ffff880036f74000
[ 5.511515] RIP: 0010:[<ffffffffa00bd90f>] [<ffffffffa00bd90f>] mvs_task_prep+0x73f/0xd50 [mvsas]
[ 5.520596] RSP: 0018:ffff880036f77948 EFLAGS: 00010097
[ 5.525960] RAX: 000000000000002c RBX: cccccccccccccccc RCX: 0000000000000000
[ 5.533142] RDX: 0000000000000000 RSI: ffff880036ea55b8 RDI: ffff880036e80000
[ 5.540325] RBP: 0000000000000000 R08: ffff880036e836a0 R09: ffff880036e70000
[ 5.547508] R10: ffffc900018f0000 R11: 0000000000000005 R12: ffff880036e80000
[ 5.554692] R13: ffff88032f4bf600 R14: ffff880036e836a0 R15: ffff88032f123e40
[ 5.561875] FS: 0000000000000000(0000) GS:ffff88033fcc0000(0000) knlGS:0000000000000000
[ 5.570028] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 5.575825] CR2: 00007ffb9332c000 CR3: 0000000001813000 CR4: 00000000000007e0
[ 5.583006] Stack:
[ 5.585070] ffff880036e81e38 ffff880036f779ec ffff880036ea55b8 ffff880300000001
[ 5.592737] ffff880036e82678 ffff880036ea55b0 ffff880036e836a0 ffff880036e70000
[ 5.600404] 0000000000012f00 ffff880036f68010 ffff880332584418 ffff880036f77a68
[ 5.608063] Call Trace:
[ 5.610562] [<ffffffffa00bdf77>] ? mvs_task_exec.isra.13+0x57/0xd0 [mvsas]
[ 5.617571] [<ffffffffa00bea71>] ? mvs_queue_command+0x301/0x310 [mvsas]
[ 5.624407] [<ffffffff8118e2bf>] ? kmem_cache_alloc+0x21f/0x450
[ 5.630464] [<ffffffffa011c3ac>] ? sas_ata_qc_issue+0x22c/0x270 [libsas]
[ 5.637303] [<ffffffffa026272c>] ? ata_qc_issue+0x16c/0x390 [libata]
[ 5.643789] [<ffffffffa0262be2>] ? ata_exec_internal_sg+0x292/0x570 [libata]
[ 5.650974] [<ffffffffa0262f18>] ? ata_exec_internal+0x58/0xa0 [libata]
[ 5.657721] [<ffffffffa011c0ae>] ? sas_ata_hard_reset+0x9e/0x140 [libsas]
[ 5.664647] [<ffffffffa0263325>] ? ata_dev_read_id+0x275/0x560 [libata]
[ 5.671395] [<ffffffffa026d5a4>] ? ata_eh_schedule_probe+0x104/0x150 [libata]
[ 5.678681] [<ffffffffa02707d4>] ? ata_eh_recover+0x814/0x1350 [libata]
[ 5.685432] [<ffffffffa0265d40>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 5.692526] [<ffffffffa011c010>] ? sas_ata_printk+0x80/0x80 [libsas]
[ 5.699018] [<ffffffffa0265bc0>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 5.706025] [<ffffffff8150768a>] ? printk+0x54/0x56
[ 5.711042] [<ffffffffa0265d40>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 5.718140] [<ffffffffa011c010>] ? sas_ata_printk+0x80/0x80 [libsas]
[ 5.724628] [<ffffffffa0271de4>] ? ata_do_eh+0x44/0xb0 [libata]
[ 5.730685] [<ffffffffa0265bc0>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 5.737694] [<ffffffffa0271908>] ? ata_scsi_port_error_handler+0x518/0x8f0 [libata]
[ 5.745500] [<ffffffffa011c786>] ? async_sas_ata_eh+0x46/0x70 [libsas]
[ 5.752162] [<ffffffff8108d872>] ? async_run_entry_fn+0x32/0x120
[ 5.758307] [<ffffffff810814e2>] ? process_one_work+0x172/0x420
[ 5.764362] [<ffffffff81081b73>] ? worker_thread+0x113/0x4f0
[ 5.770159] [<ffffffff8150c291>] ? __schedule+0x2b1/0x710
[ 5.775694] [<ffffffff81081a60>] ? rescuer_thread+0x2d0/0x2d0
[ 5.781578] [<ffffffff81087dfd>] ? kthread+0xbd/0xe0
[ 5.786682] [<ffffffff81087d40>] ? kthread_create_on_node+0x180/0x180
[ 5.793259] [<ffffffff8150febc>] ? ret_from_fork+0x7c/0xb0
[ 5.798883] [<ffffffff81087d40>] ? kthread_create_on_node+0x180/0x180
[ 5.805458] Code: 92 b8 02 00 00 41 80 b8 84 00 00 00 7f 48 8b 80 58 01 00 00 48 8b 1c d0 0f 84 9d 05 00 00 41 8b 44 24 58 48 8b 74 24 10 89 46 1c <8b> 8b 54 02 00 00 be 00 10 00 00 41 8b 54 24 58 49 8b 44 24 48
[ 5.827742] RIP [<ffffffffa00bd90f>] mvs_task_prep+0x73f/0xd50 [mvsas]
[ 5.834457] RSP <ffff880036f77948>
[ 5.837992] ---[ end trace ab01b27d6a351042 ]---
[ 5.842671] BUG: unable to handle kernel paging request at ffffffffffffffd8
[ 5.849783] IP: [<ffffffff8108838c>] kthread_data+0xc/0x20
[ 5.855374] PGD 1816067 PUD 1818067 PMD 0
[ 5.859679] Oops: 0000 [#2] SMP
[ 5.863058] Modules linked in: usb_storage hid_generic usbhid hid ata_generic mvsas libsas ata_piix ehci_pci libata ehci_hcd scsi_transport_sas e1000e ptp crc32c_intel pps_core usbcore scsi_mod usb_common
[ 5.882319] CPU: 3 PID: 138 Comm: kworker/u16:5 Tainted: G D 3.16.0-4-amd64 #1 Debian 3.16.7-ckt7-1
[ 5.892379] Hardware name: Supermicro X8SIE/X8SIE, BIOS 1.2a 06/27/2012
[ 5.899567] task: ffff880036f68010 ti: ffff880036f74000 task.ti: ffff880036f74000
[ 5.907109] RIP: 0010:[<ffffffff8108838c>] [<ffffffff8108838c>] kthread_data+0xc/0x20
[ 5.915140] RSP: 0018:ffff880036f77798 EFLAGS: 00010002
[ 5.920504] RAX: 0000000000000000 RBX: 0000000000000003 RCX: 000000000000000f
[ 5.927686] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff880036f68010
[ 5.934869] RBP: ffff880036f68010 R08: 0000000000000000 R09: 0000000000000000
[ 5.942052] R10: ffffffff81adf150 R11: 0000000000000000 R12: ffff88033fcd2f00
[ 5.949235] R13: 0000000000000003 R14: 0000000000000000 R15: ffff880036f68010
[ 5.956417] FS: 0000000000000000(0000) GS:ffff88033fcc0000(0000) knlGS:0000000000000000
[ 5.964571] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 5.970367] CR2: 0000000000000028 CR3: 0000000001813000 CR4: 00000000000007e0
[ 5.977550] Stack:
[ 5.979612] ffffffff81081fbd ffff880036f68470 ffffffff8150c43d 0000000000012f00
[ 5.987281] ffff880036f77fd8 0000000000012f00 ffff880036f68010 ffff880036f68690
[ 5.994948] ffff880036f68368 ffff880036f68000 ffff8803329752b0 ffff880036f68000
[ 6.002608] Call Trace:
[ 6.005104] [<ffffffff81081fbd>] ? wq_worker_sleeping+0xd/0x80
[ 6.011075] [<ffffffff8150c43d>] ? __schedule+0x45d/0x710
[ 6.016611] [<ffffffff81069f0f>] ? do_exit+0x6ef/0xa60
[ 6.021890] [<ffffffff810163b7>] ? oops_end+0x97/0xe0
[ 6.027078] [<ffffffff81511f48>] ? general_protection+0x28/0x30
[ 6.033134] [<ffffffffa00bd90f>] ? mvs_task_prep+0x73f/0xd50 [mvsas]
[ 6.039625] [<ffffffffa00bdec5>] ? mvs_task_prep+0xcf5/0xd50 [mvsas]
[ 6.046115] [<ffffffffa00bdf77>] ? mvs_task_exec.isra.13+0x57/0xd0 [mvsas]
[ 6.053124] [<ffffffffa00bea71>] ? mvs_queue_command+0x301/0x310 [mvsas]
[ 6.059960] [<ffffffff8118e2bf>] ? kmem_cache_alloc+0x21f/0x450
[ 6.066015] [<ffffffffa011c3ac>] ? sas_ata_qc_issue+0x22c/0x270 [libsas]
[ 6.072852] [<ffffffffa026272c>] ? ata_qc_issue+0x16c/0x390 [libata]
[ 6.079341] [<ffffffffa0262be2>] ? ata_exec_internal_sg+0x292/0x570 [libata]
[ 6.086526] [<ffffffffa0262f18>] ? ata_exec_internal+0x58/0xa0 [libata]
[ 6.093274] [<ffffffffa011c0ae>] ? sas_ata_hard_reset+0x9e/0x140 [libsas]
[ 6.100198] [<ffffffffa0263325>] ? ata_dev_read_id+0x275/0x560 [libata]
[ 6.106947] [<ffffffffa026d5a4>] ? ata_eh_schedule_probe+0x104/0x150 [libata]
[ 6.114233] [<ffffffffa02707d4>] ? ata_eh_recover+0x814/0x1350 [libata]
[ 6.118548] scsi 5:0:0:0: Direct-Access Generic USB EDC 1.00 PQ: 0 ANSI: 2
[ 6.129139] [<ffffffffa0265d40>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 6.136242] [<ffffffffa011c010>] ? sas_ata_printk+0x80/0x80 [libsas]
[ 6.142732] [<ffffffffa0265bc0>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 6.149740] [<ffffffff8150768a>] ? printk+0x54/0x56
[ 6.154759] [<ffffffffa0265d40>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 6.161852] [<ffffffffa011c010>] ? sas_ata_printk+0x80/0x80 [libsas]
[ 6.168345] [<ffffffffa0271de4>] ? ata_do_eh+0x44/0xb0 [libata]
[ 6.174401] [<ffffffffa0265bc0>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 6.181410] [<ffffffffa0271908>] ? ata_scsi_port_error_handler+0x518/0x8f0 [libata]
[ 6.189215] [<ffffffffa011c786>] ? async_sas_ata_eh+0x46/0x70 [libsas]
[ 6.195879] [<ffffffff8108d872>] ? async_run_entry_fn+0x32/0x120
[ 6.202022] [<ffffffff810814e2>] ? process_one_work+0x172/0x420
[ 6.208077] [<ffffffff81081b73>] ? worker_thread+0x113/0x4f0
[ 6.213874] [<ffffffff8150c291>] ? __schedule+0x2b1/0x710
[ 6.219410] [<ffffffff81081a60>] ? rescuer_thread+0x2d0/0x2d0
[ 6.225293] [<ffffffff81087dfd>] ? kthread+0xbd/0xe0
[ 6.230397] [<ffffffff81087d40>] ? kthread_create_on_node+0x180/0x180
[ 6.236973] [<ffffffff8150febc>] ? ret_from_fork+0x7c/0xb0
[ 6.242597] [<ffffffff81087d40>] ? kthread_create_on_node+0x180/0x180
[ 6.249172] Code: 08 04 00 00 48 8b 40 c8 48 c1 e8 02 83 e0 01 c3 66 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 48 8b 87 08 04 00 00 <48> 8b 40 d8 c3 66 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 66
[ 6.271449] RIP [<ffffffff8108838c>] kthread_data+0xc/0x20
[ 6.277115] RSP <ffff880036f77798>
[ 6.280649] CR2: ffffffffffffffd8
[ 6.284012] ---[ end trace ab01b27d6a351043 ]---
[ 6.288681] Fixing recursive fault but reboot is needed!
[-- Attachment #4: 3.19.0-kernel-panic.txt --]
[-- Type: text/plain, Size: 28319 bytes --]
[ 5.022094] scsi host4: mvsas
[ 5.162509] floppy0: no floppy controllers found
[ 5.286606] BUG: unable to handle kernel paging request at 0000009a2484b963
[ 5.293727] IP: [<ffffffffa00f1877>] mvs_task_exec.isra.13+0x827/0xf10 [mvsas]
[ 5.301066] PGD 0
[ 5.303181] Oops: 0000 [#1] SMP
[ 5.306560] Modules linked in: usb_storage hid_generic usbhid hid ata_generic mvsas ehci_pci libsas ata_piix ehci_hcd libata e1000e scsi_transport_sas ptp crc32c_intel pps_core usbcore scsi_mod usb_common
[ 5.325822] CPU: 7 PID: 146 Comm: kworker/u16:6 Not tainted 3.19.0-trunk-amd64 #1 Debian 3.19.3-1~exp1
[ 5.335188] Hardware name: Supermicro X8SIE/X8SIE, BIOS 1.2a 06/27/2012
[ 5.342373] Workqueue: events_unbound async_run_entry_fn
[ 5.347778] task: ffff88033113e450 ti: ffff88032d868000 task.ti: ffff88032d868000
[ 5.356537] RIP: 0010:[<ffffffffa00f1877>] [<ffffffffa00f1877>] mvs_task_exec.isra.13+0x827/0xf10 [mvsas]
[ 5.366310] RSP: 0018:ffff88032d86b948 EFLAGS: 00010097
[ 5.371673] RAX: 000000000000002d RBX: ffff88032d8037b0 RCX: 0000009a2484b70f
[ 5.378855] RDX: 000000000000000b RSI: 0000000000001000 RDI: ffff88032d800000
[ 5.386040] RBP: 0000000000000001 R08: ffff88032d8037b0 R09: ffff88032e173200
[ 5.393221] R10: ffff8800ba230040 R11: 0000000000010000 R12: ffff88032dc55c00
[ 5.400404] R13: ffff88032d800000 R14: 0000000000000096 R15: ffff88032d800008
[ 5.407588] FS: 0000000000000000(0000) GS:ffff88033fdc0000(0000) knlGS:0000000000000000
[ 5.415740] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 5.421537] CR2: 0000009a2484b963 CR3: 0000000001813000 CR4: 00000000000007e0
[ 5.428721] Stack:
[ 5.430782] 0000000003aa7d20 ffff88032d8037b0 ffff88032d801e38 ffff8800ba230040
[ 5.438442] 0000000000000001 ffff88032d802678 ffff88032e173200 0000009a2484b70f
[ 5.446100] ffff88032d8813f0 0000000000000286 ffff88032d8813f0 0000000000000286
[ 5.453760] Call Trace:
[ 5.456260] [<ffffffffa00d8c13>] ? sas_ata_qc_issue+0x1a3/0x250 [libsas]
[ 5.463104] [<ffffffffa0259875>] ? ata_qc_issue+0x1f5/0x380 [libata]
[ 5.469594] [<ffffffffa0259cef>] ? ata_exec_internal_sg+0x2ef/0x5f0 [libata]
[ 5.476784] [<ffffffffa025a048>] ? ata_exec_internal+0x58/0xa0 [libata]
[ 5.483533] [<ffffffffa00d89f2>] ? sas_ata_hard_reset+0xa2/0x120 [libsas]
[ 5.490456] [<ffffffffa025cfa5>] ? ata_phys_link_offline+0x15/0x30 [libata]
[ 5.497551] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 5.504042] [<ffffffffa025a525>] ? ata_dev_read_id+0x345/0x5d0 [libata]
[ 5.510792] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 5.517283] [<ffffffffa02647f1>] ? ata_eh_schedule_probe+0x141/0x180 [libata]
[ 5.524570] [<ffffffffa02676ef>] ? ata_eh_recover+0x7df/0x13b0 [libata]
[ 5.531318] [<ffffffffa025cfc0>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 5.538412] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 5.544904] [<ffffffffa025ce30>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 5.551912] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 5.558404] [<ffffffffa025cfc0>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 5.565497] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 5.571990] [<ffffffffa0268c56>] ? ata_do_eh+0x46/0xb0 [libata]
[ 5.578046] [<ffffffffa025ce30>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 5.585056] [<ffffffff8101d146>] ? native_sched_clock+0x26/0x90
[ 5.591112] [<ffffffffa02687e8>] ? ata_scsi_port_error_handler+0x448/0x7b0 [libata]
[ 5.598917] [<ffffffffa00d9046>] ? async_sas_ata_eh+0x46/0x70 [libsas]
[ 5.605579] [<ffffffff8108ca33>] ? async_run_entry_fn+0x43/0x160
[ 5.611725] [<ffffffff810849cd>] ? process_one_work+0x14d/0x3f0
[ 5.617778] [<ffffffff810853fb>] ? worker_thread+0x6b/0x4a0
[ 5.623488] [<ffffffff81085390>] ? rescuer_thread+0x310/0x310
[ 5.629371] [<ffffffff8108a1a3>] ? kthread+0xd3/0xf0
[ 5.634475] [<ffffffff8108a0d0>] ? kthread_create_on_node+0x180/0x180
[ 5.641054] [<ffffffff815509d8>] ? ret_from_fork+0x58/0x90
[ 5.646674] [<ffffffff8108a0d0>] ? kthread_create_on_node+0x180/0x180
[ 5.653251] Code: 8b 80 08 01 00 00 48 8b 0c d0 0f 84 e6 05 00 00 41 8b 45 58 48 8d 54 ad 00 be 00 10 00 00 48 8d 54 55 00 41 89 84 d5 d4 55 02 00 <8b> 89 54 02 00 00 49 8b 55 48 d3 e6 41 0f b6 88 84 00 00 00 81
[ 5.675544] RIP [<ffffffffa00f1877>] mvs_task_exec.isra.13+0x827/0xf10 [mvsas]
[ 5.682970] RSP <ffff88032d86b948>
[ 5.686505] CR2: 0000009a2484b963
[ 5.689867] ---[ end trace 50dd3aeb0a903cc2 ]---
[ 5.694551] BUG: unable to handle kernel paging request at ffffffffffffffd8
[ 5.701667] IP: [<ffffffff8108a7bc>] kthread_data+0xc/0x20
[ 5.707247] PGD 1816067 PUD 1818067 PMD 0
[ 5.711554] Oops: 0000 [#2] SMP
[ 5.714941] Modules linked in: usb_storage hid_generic usbhid hid ata_generic mvsas ehci_pci libsas ata_piix ehci_hcd libata e1000e scsi_transport_sas ptp crc32c_intel pps_core usbcore scsi_mod usb_common
[ 5.734195] CPU: 7 PID: 146 Comm: kworker/u16:6 Tainted: G D 3.19.0-trunk-amd64 #1 Debian 3.19.3-1~exp1
[ 5.744695] Hardware name: Supermicro X8SIE/X8SIE, BIOS 1.2a 06/27/2012
[ 5.751883] task: ffff88033113e450 ti: ffff88032d868000 task.ti: ffff88032d868000
[ 5.759424] RIP: 0010:[<ffffffff8108a7bc>] [<ffffffff8108a7bc>] kthread_data+0xc/0x20
[ 5.767455] RSP: 0018:ffff88032d86b600 EFLAGS: 00010002
[ 5.772820] RAX: 0000000000000000 RBX: 0000000000000007 RCX: 0000000000000000
[ 5.780003] RDX: 000000000000000f RSI: 0000000000000007 RDI: ffff88033113e450
[ 5.787185] RBP: ffff88033113e450 R08: 0000000000000000 R09: 000000000000001f
[ 5.794367] R10: ffffffff81af9330 R11: 000000000000001a R12: 0000000000000000
[ 5.801551] R13: ffff88033113e8b8 R14: 0000000000000007 R15: ffff880331601230
[ 5.808733] FS: 0000000000000000(0000) GS:ffff88033fdc0000(0000) knlGS:0000000000000000
[ 5.816886] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 5.822684] CR2: 0000000000000028 CR3: 0000000001813000 CR4: 00000000000007e0
[ 5.829865] Stack:
[ 5.831928] ffffffff810858ad ffff88033fdd4140 ffffffff8154c701 0000000000000000
[ 5.839588] ffffffff8106e1dd 0000000000014140 ffff88032d86bfd8 0000000000014140
[ 5.847256] ffff88033113e450 0000000000000008 ffff88033113e450 ffff88033113ead8
[ 5.854924] Call Trace:
[ 5.857420] [<ffffffff810858ad>] ? wq_worker_sleeping+0xd/0xa0
[ 5.863391] [<ffffffff8154c701>] ? __schedule+0x651/0x800
[ 5.868927] [<ffffffff8106e1dd>] ? release_task+0x36d/0x460
[ 5.874635] [<ffffffff8106fab2>] ? do_exit+0x782/0xad0
[ 5.879913] [<ffffffff81017636>] ? oops_end+0x96/0xe0
[ 5.885105] [<ffffffff8105c1b6>] ? no_context+0x106/0x380
[ 5.890639] [<ffffffff8105c90e>] ? __do_page_fault+0xae/0x550
[ 5.896524] [<ffffffff810a3350>] ? pick_next_task_fair+0x1b0/0x850
[ 5.902840] [<ffffffff810125eb>] ? __switch_to+0x14b/0x5d0
[ 5.908464] [<ffffffff81552be8>] ? page_fault+0x28/0x30
[ 5.913826] [<ffffffffa00f1877>] ? mvs_task_exec.isra.13+0x827/0xf10 [mvsas]
[ 5.921010] [<ffffffffa00f1e71>] ? mvs_task_exec.isra.13+0xe21/0xf10 [mvsas]
[ 5.928192] [<ffffffffa00d8c13>] ? sas_ata_qc_issue+0x1a3/0x250 [libsas]
[ 5.935029] [<ffffffffa0259875>] ? ata_qc_issue+0x1f5/0x380 [libata]
[ 5.941518] [<ffffffffa0259cef>] ? ata_exec_internal_sg+0x2ef/0x5f0 [libata]
[ 5.948702] [<ffffffffa025a048>] ? ata_exec_internal+0x58/0xa0 [libata]
[ 5.955449] [<ffffffffa00d89f2>] ? sas_ata_hard_reset+0xa2/0x120 [libsas]
[ 5.962375] [<ffffffffa025cfa5>] ? ata_phys_link_offline+0x15/0x30 [libata]
[ 5.969477] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 5.975967] [<ffffffffa025a525>] ? ata_dev_read_id+0x345/0x5d0 [libata]
[ 5.982716] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 5.989208] [<ffffffffa02647f1>] ? ata_eh_schedule_probe+0x141/0x180 [libata]
[ 5.996496] [<ffffffffa02676ef>] ? ata_eh_recover+0x7df/0x13b0 [libata]
[ 6.003245] [<ffffffffa025cfc0>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 6.010338] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 6.016829] [<ffffffffa025ce30>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 6.023838] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 6.030329] [<ffffffffa025cfc0>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 6.037425] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 6.043915] [<ffffffffa0268c56>] ? ata_do_eh+0x46/0xb0 [libata]
[ 6.049973] [<ffffffffa025ce30>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 6.056980] [<ffffffff8101d146>] ? native_sched_clock+0x26/0x90
[ 6.063038] [<ffffffffa02687e8>] ? ata_scsi_port_error_handler+0x448/0x7b0 [libata]
[ 6.070843] [<ffffffffa00d9046>] ? async_sas_ata_eh+0x46/0x70 [libsas]
[ 6.077506] [<ffffffff8108ca33>] ? async_run_entry_fn+0x43/0x160
[ 6.083648] [<ffffffff810849cd>] ? process_one_work+0x14d/0x3f0
[ 6.089705] [<ffffffff810853fb>] ? worker_thread+0x6b/0x4a0
[ 6.095414] [<ffffffff81085390>] ? rescuer_thread+0x310/0x310
[ 6.101297] [<ffffffff8108a1a3>] ? kthread+0xd3/0xf0
[ 6.106402] [<ffffffff8108a0d0>] ? kthread_create_on_node+0x180/0x180
[ 6.112976] [<ffffffff815509d8>] ? ret_from_fork+0x58/0x90
[ 6.118601] [<ffffffff8108a0d0>] ? kthread_create_on_node+0x180/0x180
[ 6.125175] Code: 10 04 00 00 48 8b 40 c8 48 c1 e8 02 83 e0 01 c3 66 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 48 8b 87 10 04 00 00 <48> 8b 40 d8 c3 66 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 66
[ 6.147461] RIP [<ffffffff8108a7bc>] kthread_data+0xc/0x20
[ 6.153128] RSP <ffff88032d86b600>
[ 6.156662] CR2: ffffffffffffffd8
[ 6.160024] ---[ end trace 50dd3aeb0a903cc3 ]---
[ 6.163260] scsi 5:0:0:0: Direct-Access Generic USB EDC 1.00 PQ: 0 ANSI: 2
[ 6.172847] Fixing recursive fault but reboot is needed!
[ 16.487514] random: nonblocking pool is initialized
[ 28.480381] INFO: rcu_sched detected stalls on CPUs/tasks: { 3 7} (detected by 2, t=5581 jiffies, g=-188, c=-189, q=50)
[ 28.491481] Task dump for CPU 3:
[ 28.494765] swapper/3 R running task 0 0 1 0x00000008
[ 28.501963] 000038ad1b4fe8cc ffffffffffffffff ffffffffffffff4e ffffffff8142366e
[ 28.509633] 0000000000000010 0000000000000217 ffff880331677e88 0000000000000018
[ 28.517301] ffffffff8142365e 0000000000000092 ffffffff818f60c0 ffffe8ffffcc6de0
[ 28.524969] Call Trace:
[ 28.527466] [<ffffffff8142366e>] ? cpuidle_enter_state+0x5e/0x160
[ 28.533693] [<ffffffff8142365e>] ? cpuidle_enter_state+0x4e/0x160
[ 28.539924] [<ffffffff810a936d>] ? cpu_startup_entry+0x34d/0x3f0
[ 28.546068] [<ffffffff810d9cc0>] ? tick_check_new_device+0xe0/0x110
[ 28.552472] [<ffffffff8104642e>] ? start_secondary+0x19e/0x1d0
[ 28.558438] Task dump for CPU 7:
[ 28.561715] kworker/u16:6 D ffff88032d800008 0 146 0 0x00000000
[ 28.568920] 0000000000000096 ffff88032d800000 ffff88032dc55c00 0000000000000001
[ 28.576582] ffff88032d8037b0 0000000000010000 ffff8800ba230040 ffff88032e173200
[ 28.584250] ffff88032d8037b0 000000000000002d 0000009a2484b70f 000000000000000b
[ 28.591919] Call Trace:
[ 28.594417] [<ffffffffa00f1877>] ? mvs_task_exec.isra.13+0x827/0xf10 [mvsas]
[ 28.601599] [<ffffffffa00f1e71>] ? mvs_task_exec.isra.13+0xe21/0xf10 [mvsas]
[ 28.608782] [<ffffffffa00d8c13>] ? sas_ata_qc_issue+0x1a3/0x250 [libsas]
[ 28.615618] [<ffffffffa0259875>] ? ata_qc_issue+0x1f5/0x380 [libata]
[ 28.622107] [<ffffffffa0259cef>] ? ata_exec_internal_sg+0x2ef/0x5f0 [libata]
[ 28.629290] [<ffffffffa025a048>] ? ata_exec_internal+0x58/0xa0 [libata]
[ 28.636040] [<ffffffffa00d89f2>] ? sas_ata_hard_reset+0xa2/0x120 [libsas]
[ 28.642962] [<ffffffffa025cfa5>] ? ata_phys_link_offline+0x15/0x30 [libata]
[ 28.651367] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 28.657858] [<ffffffffa025a525>] ? ata_dev_read_id+0x345/0x5d0 [libata]
[ 28.664607] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 28.671098] [<ffffffffa02647f1>] ? ata_eh_schedule_probe+0x141/0x180 [libata]
[ 28.678384] [<ffffffffa02676ef>] ? ata_eh_recover+0x7df/0x13b0 [libata]
[ 28.685134] [<ffffffffa025cfc0>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 28.692227] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 28.698719] [<ffffffffa025ce30>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 28.705727] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 28.712217] [<ffffffffa025cfc0>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 28.719314] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 28.725803] [<ffffffffa0268c56>] ? ata_do_eh+0x46/0xb0 [libata]
[ 28.731859] [<ffffffffa025ce30>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 28.738868] [<ffffffff8101d146>] ? native_sched_clock+0x26/0x90
[ 28.744928] [<ffffffffa02687e8>] ? ata_scsi_port_error_handler+0x448/0x7b0 [libata]
[ 28.752732] [<ffffffffa00d9046>] ? async_sas_ata_eh+0x46/0x70 [libsas]
[ 28.759394] [<ffffffff8108ca33>] ? async_run_entry_fn+0x43/0x160
[ 28.765537] [<ffffffff810849cd>] ? process_one_work+0x14d/0x3f0
[ 28.771593] [<ffffffff810853fb>] ? worker_thread+0x6b/0x4a0
[ 28.777304] [<ffffffff81085390>] ? rescuer_thread+0x310/0x310
[ 28.783186] [<ffffffff8108a1a3>] ? kthread+0xd3/0xf0
[ 28.788290] [<ffffffff8108a0d0>] ? kthread_create_on_node+0x180/0x180
[ 28.794867] [<ffffffff815509d8>] ? ret_from_fork+0x58/0x90
[ 28.800488] [<ffffffff8108a0d0>] ? kthread_create_on_node+0x180/0x180
[ 35.049176] ------------[ cut here ]------------
[ 35.053847] WARNING: CPU: 7 PID: 146 at /build/linux-gzeV2n/linux-3.19.3/kernel/watchdog.c:290 watchdog_overflow_callback+0x92/0xc0()
[ 35.065906] Watchdog detected hard LOCKUP on cpu 7
[ 35.070560] Modules linked in: usb_storage hid_generic usbhid hid ata_generic mvsas ehci_pci libsas ata_piix ehci_hcd libata e1000e scsi_transport_sas ptp crc32c_intel pps_core usbcore scsi_mod usb_common
[ 35.090107] CPU: 7 PID: 146 Comm: kworker/u16:6 Tainted: G D 3.19.0-trunk-amd64 #1 Debian 3.19.3-1~exp1
[ 35.100607] Hardware name: Supermicro X8SIE/X8SIE, BIOS 1.2a 06/27/2012
[ 35.107796] 0000000000000000 ffffffff81723ea8 ffffffff8154b001 ffff88033fdc5b60
[ 35.115483] ffffffff8106ced1 ffff880331146800[ 35.118656] ------------[ cut here ]------------
[ 35.118658] WARNING: CPU: 3 PID: 0 at /build/linux-gzeV2n/linux-3.19.3/kernel/watchdog.c:290 watchdog_overflow_callback+0x92/0xc0()
[ 35.118659] Watchdog detected hard LOCKUP on cpu 3Modules linked in: usb_storage hid_generic usbhid hid ata_generic mvsas ehci_pci libsas ata_piix ehci_hcd libata e1000e scsi_transport_sas ptp crc32c_intel pps_core usbcore scsi_mod usb_common
[ 35.159023] 0000000000000000 ffff88033fdc5c80
[ 35.163866] ffff88033fdc5ef8 0000000000000000 ffffffff8106cf4a ffffffff81723e80
[ 35.171536] Call Trace:
[ 35.174029] <NMI> [<ffffffff8154b001>] ? dump_stack+0x40/0x50
[ 35.180089] [<ffffffff8106ced1>] ? warn_slowpath_common+0x81/0xb0
[ 35.186316] [<ffffffff8106cf4a>] ? warn_slowpath_fmt+0x4a/0x50
[ 35.192287] [<ffffffff81109cd2>] ? watchdog_overflow_callback+0x92/0xc0
[ 35.199037] [<ffffffff81145e16>] ? __perf_event_overflow+0x86/0x230
[ 35.205442] [<ffffffff8102a689>] ? x86_perf_event_set_period+0xc9/0x170
[ 35.212190] [<ffffffff810319da>] ? intel_pmu_handle_irq+0x1ba/0x3a0
[ 35.218623] [<ffffffff8102962a>] ? perf_event_nmi_handler+0x2a/0x50
[ 35.225021] [<ffffffff8101d146>] ? native_sched_clock+0x26/0x90
[ 35.231078] [<ffffffff81017e16>] ? nmi_handle+0x86/0x120
[ 35.236527] [<ffffffff81018370>] ? default_do_nmi+0x40/0x110
[ 35.242325] [<ffffffff810184c8>] ? do_nmi+0x88/0xd0
[ 35.247341] [<ffffffff81552f61>] ? end_repeat_nmi+0x1e/0x2e
[ 35.253051] [<ffffffff8106fc06>] ? do_exit+0x8d6/0xad0
[ 35.258329] [<ffffffff81550512>] ? _raw_spin_lock_irq+0x32/0x40
[ 35.264383] [<ffffffff81550512>] ? _raw_spin_lock_irq+0x32/0x40
[ 35.270440] [<ffffffff81550512>] ? _raw_spin_lock_irq+0x32/0x40
[ 35.276495] <<EOE>> [<ffffffff8154c153>] ? __schedule+0xa3/0x800
[ 35.282812] [<ffffffff81549e3c>] ? printk+0x4d/0x52
[ 35.287831] [<ffffffff8106fc06>] ? do_exit+0x8d6/0xad0
[ 35.293107] [<ffffffff81017636>] ? oops_end+0x96/0xe0
[ 35.298297] [<ffffffff8105c1b6>] ? no_context+0x106/0x380
[ 35.303835] [<ffffffff8105c90e>] ? __do_page_fault+0xae/0x550
[ 35.309718] [<ffffffff812d7fa5>] ? vsnprintf+0x295/0x5d0
[ 35.315165] [<ffffffff812d80df>] ? vsnprintf+0x3cf/0x5d0
[ 35.320616] [<ffffffff812d92c3>] ? delay_tsc+0x33/0x60
[ 35.325892] [<ffffffff81552be8>] ? page_fault+0x28/0x30
[ 35.331257] [<ffffffff8108a7bc>] ? kthread_data+0xc/0x20
[ 35.336706] [<ffffffff810858ad>] ? wq_worker_sleeping+0xd/0xa0
[ 35.342675] [<ffffffff8154c701>] ? __schedule+0x651/0x800
[ 35.348211] [<ffffffff8106e1dd>] ? release_task+0x36d/0x460
[ 35.353923] [<ffffffff8106fab2>] ? do_exit+0x782/0xad0
[ 35.359199] [<ffffffff81017636>] ? oops_end+0x96/0xe0
[ 35.364390] [<ffffffff8105c1b6>] ? no_context+0x106/0x380
[ 35.369925] [<ffffffff8105c90e>] ? __do_page_fault+0xae/0x550
[ 35.375808] [<ffffffff810a3350>] ? pick_next_task_fair+0x1b0/0x850
[ 35.382126] [<ffffffff810125eb>] ? __switch_to+0x14b/0x5d0
[ 35.387749] [<ffffffff81552be8>] ? page_fault+0x28/0x30
[ 35.393114] [<ffffffffa00f1877>] ? mvs_task_exec.isra.13+0x827/0xf10 [mvsas]
[ 35.400296] [<ffffffffa00f1e71>] ? mvs_task_exec.isra.13+0xe21/0xf10 [mvsas]
[ 35.407478] [<ffffffffa00d8c13>] ? sas_ata_qc_issue+0x1a3/0x250 [libsas]
[ 35.414314] [<ffffffffa0259875>] ? ata_qc_issue+0x1f5/0x380 [libata]
[ 35.420805] [<ffffffffa0259cef>] ? ata_exec_internal_sg+0x2ef/0x5f0 [libata]
[ 35.427987] [<ffffffffa025a048>] ? ata_exec_internal+0x58/0xa0 [libata]
[ 35.434737] [<ffffffffa00d89f2>] ? sas_ata_hard_reset+0xa2/0x120 [libsas]
[ 35.441662] [<ffffffffa025cfa5>] ? ata_phys_link_offline+0x15/0x30 [libata]
[ 35.448764] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 35.455256] [<ffffffffa025a525>] ? ata_dev_read_id+0x345/0x5d0 [libata]
[ 35.462002] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 35.468494] [<ffffffffa02647f1>] ? ata_eh_schedule_probe+0x141/0x180 [libata]
[ 35.475781] [<ffffffffa02676ef>] ? ata_eh_recover+0x7df/0x13b0 [libata]
[ 35.482556] [<ffffffffa025cfc0>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 35.489652] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 35.496143] [<ffffffffa025ce30>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 35.503149] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 35.509641] [<ffffffffa025cfc0>] ? ata_phys_link_offline+0x30/0x30 [libata]
[ 35.516736] [<ffffffffa00d8950>] ? sas_ata_printk+0x90/0x90 [libsas]
[ 35.523227] [<ffffffffa0268c56>] ? ata_do_eh+0x46/0xb0 [libata]
[ 35.529284] [<ffffffffa025ce30>] ? ata_phys_link_online+0x30/0x30 [libata]
[ 35.536290] [<ffffffff8101d146>] ? native_sched_clock+0x26/0x90
[ 35.542350] [<ffffffffa02687e8>] ? ata_scsi_port_error_handler+0x448/0x7b0 [libata]
[ 35.550155] [<ffffffffa00d9046>] ? async_sas_ata_eh+0x46/0x70 [libsas]
[ 35.556819] [<ffffffff8108ca33>] ? async_run_entry_fn+0x43/0x160
[ 35.562960] [<ffffffff810849cd>] ? process_one_work+0x14d/0x3f0
[ 35.569016] [<ffffffff810853fb>] ? worker_thread+0x6b/0x4a0
[ 35.574726] [<ffffffff81085390>] ? rescuer_thread+0x310/0x310
[ 35.580610] [<ffffffff8108a1a3>] ? kthread+0xd3/0xf0
[ 35.585714] [<ffffffff8108a0d0>] ? kthread_create_on_node+0x180/0x180
[ 35.592289] [<ffffffff815509d8>] ? ret_from_fork+0x58/0x90
[ 35.597912] [<ffffffff8108a0d0>] ? kthread_create_on_node+0x180/0x180
[ 35.604488] ---[ end trace 50dd3aeb0a903cc4 ]---
[ 35.604489] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G D 3.19.0-trunk-amd64 #1 Debian 3.19.3-1~exp1
[ 35.604490] Hardware name: Supermicro X8SIE/X8SIE, BIOS 1.2a 06/27/2012
[ 35.604492] 0000000000000000 ffffffff81723ea8 ffffffff8154b001 ffff88033fcc5b60
[ 35.604493] ffffffff8106ced1 ffff880331003800 0000000000000000 ffff88033fcc5c80
[ 35.604494] ffff88033fcc5ef8 0000000000000000 ffffffff8106cf4a ffffffff81723e80
[ 35.604494] Call Trace:
[ 35.604496] <NMI> [<ffffffff8154b001>] ? dump_stack+0x40/0x50
[ 35.604498] [<ffffffff8106ced1>] ? warn_slowpath_common+0x81/0xb0
[ 35.604499] [<ffffffff8106cf4a>] ? warn_slowpath_fmt+0x4a/0x50
[ 35.604500] [<ffffffff81109cd2>] ? watchdog_overflow_callback+0x92/0xc0
[ 35.604502] [<ffffffff81145e16>] ? __perf_event_overflow+0x86/0x230
[ 35.604503] [<ffffffff8102a689>] ? x86_perf_event_set_period+0xc9/0x170
[ 35.604504] [<ffffffff810319da>] ? intel_pmu_handle_irq+0x1ba/0x3a0
[ 35.604506] [<ffffffff8102962a>] ? perf_event_nmi_handler+0x2a/0x50
[ 35.604508] [<ffffffff8101d146>] ? native_sched_clock+0x26/0x90
[ 35.604509] [<ffffffff81017e16>] ? nmi_handle+0x86/0x120
[ 35.604510] [<ffffffff81018370>] ? default_do_nmi+0x40/0x110
[ 35.604511] [<ffffffff810184c8>] ? do_nmi+0x88/0xd0
[ 35.604513] [<ffffffff81552f61>] ? end_repeat_nmi+0x1e/0x2e
[ 35.604514] [<ffffffff815506d2>] ? _raw_spin_lock+0x32/0x40
[ 35.604516] [<ffffffff815506d2>] ? _raw_spin_lock+0x32/0x40
[ 35.604517] [<ffffffff815506d2>] ? _raw_spin_lock+0x32/0x40
[ 35.604523] <<EOE>> <IRQ> [<ffffffffa00f5310>] ? mvs_64xx_isr+0x30/0x100 [mvsas]
[ 35.604527] [<ffffffffa00ef15f>] ? mvs_interrupt+0x6f/0xa0 [mvsas]
[ 35.604528] [<ffffffff810bcd35>] ? handle_irq_event_percpu+0x75/0x190
[ 35.604530] [<ffffffff810bce88>] ? handle_irq_event+0x38/0x50
[ 35.604531] [<ffffffff810bfa94>] ? handle_fasteoi_irq+0x84/0x140
[ 35.604533] [<ffffffff8101648d>] ? handle_irq+0x1d/0x30
[ 35.604534] [<ffffffff81553986>] ? do_IRQ+0x46/0xe0
[ 35.604536] [<ffffffff8155182d>] ? common_interrupt+0x6d/0x6d
[ 35.604538] <EOI> [<ffffffff8142366e>] ? cpuidle_enter_state+0x5e/0x160
[ 35.604539] [<ffffffff8142365e>] ? cpuidle_enter_state+0x4e/0x160
[ 35.604540] [<ffffffff810a936d>] ? cpu_startup_entry+0x34d/0x3f0
[ 35.604542] [<ffffffff810d9cc0>] ? tick_check_new_device+0xe0/0x110
[ 35.604544] [<ffffffff8104642e>] ? start_secondary+0x19e/0x1d0
[ 35.604545] ---[ end trace 50dd3aeb0a903cc5 ]---
[ 38.645536] ------------[ cut here ]------------
[ 38.650204] WARNING: CPU: 1 PID: 0 at /build/linux-gzeV2n/linux-3.19.3/kernel/watchdog.c:290 watchdog_overflow_callback+0x92/0xc0()
[ 38.662090] Watchdog detected hard LOCKUP on cpu 1
[ 38.666742] Modules linked in: usb_storage hid_generic usbhid hid ata_generic mvsas ehci_pci libsas ata_piix ehci_hcd libata e1000e scsi_transport_sas ptp crc32c_intel pps_core usbcore scsi_mod usb_common
[ 38.686238] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G D W 3.19.0-trunk-amd64 #1 Debian 3.19.3-1~exp1
[ 38.696210] Hardware name: Supermicro X8SIE/X8SIE, BIOS 1.2a 06/27/2012
[ 38.703392] 0000000000000000 ffffffff81723ea8 ffffffff8154b001 ffff88033fc45b60
[ 38.711052] ffffffff8106ced1 ffff880331631800 0000000000000001 ffff88033fc45c80
[ 38.718719] ffff88033fc45ef8 0000000000000000 ffffffff8106cf4a ffffffff81723e80
[ 38.727549] Call Trace:
[ 38.730045] <NMI> [<ffffffff8154b001>] ? dump_stack+0x40/0x50
[ 38.736103] [<ffffffff8106ced1>] ? warn_slowpath_common+0x81/0xb0
[ 38.742331] [<ffffffff8106cf4a>] ? warn_slowpath_fmt+0x4a/0x50
[ 38.748301] [<ffffffff81109cd2>] ? watchdog_overflow_callback+0x92/0xc0
[ 38.755051] [<ffffffff81145e16>] ? __perf_event_overflow+0x86/0x230
[ 38.761454] [<ffffffff8102a689>] ? x86_perf_event_set_period+0xc9/0x170
[ 38.768204] [<ffffffff810319da>] ? intel_pmu_handle_irq+0x1ba/0x3a0
[ 38.774608] [<ffffffff8102962a>] ? perf_event_nmi_handler+0x2a/0x50
[ 38.781009] [<ffffffff8101d146>] ? native_sched_clock+0x26/0x90
[ 38.787067] [<ffffffff81017e16>] ? nmi_handle+0x86/0x120
[ 38.792516] [<ffffffff81018370>] ? default_do_nmi+0x40/0x110
[ 38.798313] [<ffffffff810184c8>] ? do_nmi+0x88/0xd0
[ 38.803330] [<ffffffff81552f61>] ? end_repeat_nmi+0x1e/0x2e
[ 38.809038] [<ffffffff815506d7>] ? _raw_spin_lock+0x37/0x40
[ 38.814749] [<ffffffff815506d7>] ? _raw_spin_lock+0x37/0x40
[ 38.820457] [<ffffffff815506d7>] ? _raw_spin_lock+0x37/0x40
[ 38.826168] <<EOE>> <IRQ> [<ffffffff810a5272>] ? sched_rt_period_timer+0xe2/0x2e0
[ 38.834124] [<ffffffff810a5190>] ? put_prev_task_rt+0x50/0x50
[ 38.840007] [<ffffffff810cd50b>] ? __run_hrtimer+0x6b/0x1c0
[ 38.845714] [<ffffffff8101cc25>] ? read_tsc+0x5/0x10
[ 38.850810] [<ffffffff810cd919>] ? hrtimer_interrupt+0xf9/0x230
[ 38.856868] [<ffffffff81054506>] ? hpet_interrupt_handler+0x16/0x40
[ 38.863270] [<ffffffff810bcd35>] ? handle_irq_event_percpu+0x75/0x190
[ 38.869845] [<ffffffff810bce88>] ? handle_irq_event+0x38/0x50
[ 38.875730] [<ffffffff810bfd81>] ? handle_edge_irq+0x71/0x120
[ 38.881613] [<ffffffff8101648d>] ? handle_irq+0x1d/0x30
[ 38.887011] [<ffffffff81553986>] ? do_IRQ+0x46/0xe0
[ 38.892027] [<ffffffff8155182d>] ? common_interrupt+0x6d/0x6d
[ 38.897910] <EOI> [<ffffffff8142366e>] ? cpuidle_enter_state+0x5e/0x160
[ 38.904832] [<ffffffff8142365e>] ? cpuidle_enter_state+0x4e/0x160
[ 38.911064] [<ffffffff810a936d>] ? cpu_startup_entry+0x34d/0x3f0
[ 38.917207] [<ffffffff8104642e>] ? start_secondary+0x19e/0x1d0
[ 38.923175] ---[ end trace 50dd3aeb0a903cc6 ]---
[ 44.432170] NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [systemd-udevd:101]
[ 44.440098] Modules linked in: usb_storage hid_generic usbhid hid ata_generic mvsas ehci_pci libsas ata_piix ehci_hcd libata e1000e scsi_transport_sas ptp crc32c_intel pps_core usbcore scsi_mod usb_common
[ 44.459360] CPU: 0 PID: 101 Comm: systemd-udevd Tainted: G D W 3.19.0-trunk-amd64 #1 Debian 3.19.3-1~exp1
[ 44.469853] Hardware name: Supermicro X8SIE/X8SIE, BIOS 1.2a 06/27/2012
[ 44.477036] task: ffff8803312e60d0 ti: ffff88032e1f8000 task.ti: ffff88032e1f8000
[ 44.484582] RIP: 0010:[<ffffffff810e0ca2>] [<ffffffff810e0ca2>] smp_call_function_many+0x212/0x270
[ 44.493739] RSP: 0018:ffff88032e1fbaf8 EFLAGS: 00000202
[ 44.499102] RAX: ffff88033fcd7918 RBX: 00000000000000fe RCX: 0000000000000003
[ 44.506287] RDX: ffff88033fcd7918 RSI: 0000000000000200 RDI: 0000000000000000
[ 44.513468] RBP: ffff88033fc14f40 R08: ffff88033fc14f48 R09: 0000000000000000
[ 44.520651] R10: 0000000000000008 R11: 000000000000a44a R12: 0000000000000002
[ 44.527834] R13: 0000000000000000 R14: 0000000000000293 R15: ffff88033fff3d80
[ 44.535018] FS: 00007f89f9312880(0000) GS:ffff88033fc00000(0000) knlGS:0000000000000000
[ 44.543171] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 44.548966] CR2: 00007f89f9319000 CR3: 000000032e061000 CR4: 00000000000007f0
[ 44.556149] Stack:
[ 44.558211] 0000000200000141 0000000100000000 0000004000000040 ffff88032e1fbc40
[ 44.565871] ffffffff81062fa0 ffff88032e1fbb70 ffff88033fdd0428 ffff88032e1fbc48
[ 44.573530] ffff88032e1fbb18 ffffffff810e0e68 ffff88032e1fbc40 ffff88033fdd04c0
[ 44.581190] Call Trace:
[ 44.583686] [<ffffffff81062fa0>] ? rbt_memtype_copy_nth_element+0xa0/0xa0
[ 44.590609] [<ffffffff810e0e68>] ? on_each_cpu+0x28/0x50
[ 44.596060] [<ffffffff81063710>] ? flush_tlb_kernel_range+0x60/0x70
[ 44.602463] [<ffffffff8118a553>] ? __purge_vmap_area_lazy+0x3c3/0x430
[ 44.609040] [<ffffffff810e1f30>] ? mod_find_symname+0x90/0x90
[ 44.614921] [<ffffffff8118a72c>] ? vm_unmap_aliases+0x16c/0x180
[ 44.620981] [<ffffffff8105f5ad>] ? change_page_attr_set_clr+0xdd/0x450
[ 44.627641] [<ffffffff8105fdcd>] ? set_memory_ro+0x2d/0x40
[ 44.633263] [<ffffffff810e31da>] ? set_section_ro_nx+0x7a/0x90
[ 44.639235] [<ffffffff810e6430>] ? load_module+0x1a80/0x2740
[ 44.645032] [<ffffffff810e725d>] ? SyS_finit_module+0x7d/0xa0
[ 44.650914] [<ffffffff81550a8d>] ? system_call_fast_compare_end+0xc/0x11
[ 44.657749] Code: 81 00 89 c1 0f 8d 7c fe ff ff 48 98 48 8b 55 00 48 03 14 c5 40 4a 8f 81 f6 42 18 01 48 89 d0 74 c8 0f 1f 84 00 00 00 00 00 f3 90 <f6> 40 18 01 75 f8 eb b6 0f b6 4c 24 0c 48 83 c4 18 4c 89 ea 5b
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: mvsas panics and dies when attached to a port extender on newer kernels
2015-04-14 21:03 Adam Talbot
@ 2015-04-14 21:16 ` James Bottomley
2015-04-14 21:41 ` Adam Talbot
2015-04-14 21:46 ` Laurence Oberman
0 siblings, 2 replies; 12+ messages in thread
From: James Bottomley @ 2015-04-14 21:16 UTC (permalink / raw)
To: Adam Talbot; +Cc: linux-scsi
On Tue, 2015-04-14 at 14:03 -0700, Adam Talbot wrote:
> To make a very long debugging story short, I think there is an issues/bug
> with the mvsas driver. It works, with older kernels, and breaks on
> newer kernels.
>
> My Debian Jessie system was running great on a 3.18 kernel. Changed
> cases to a newer supermicro case with a SAS expander backplane (SAS933EL). That
> was the only hardware change. Now, when ever I boot, the system kernel panics.
>
> 3.2.65-1+deb7u2 works
> 3.9.0 Gentoo CD works
> 3.16+ all fail
> Attached are 3 kernel panics on 3.16+ kernels.
>
> Motherboard is a Supermicro X8SIE, with a "Marvell Technology Group Ltd.
> 88SE6440 SAS/SATA PCIe controller"
>
> Is this a known bug?
Well, you're the only person that's reported it so far.
I think based on the above is that your configuration is a single
expander attached SATA device ... and if you move it to be non expander
attached it works fine?
> At this point I have two options:
> Stick with the old kernel (yuck)
> Buy a new card running a better supported chipset
>
> Any help would be greatly appreciated
> Thanks
You didn't specify: does 3.15 work? At least the highest working kernel
version would help me narrow down potential problems.
James
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: mvsas panics and dies when attached to a port extender on newer kernels
2015-04-14 21:16 ` James Bottomley
@ 2015-04-14 21:41 ` Adam Talbot
2015-04-16 5:16 ` James Bottomley
2015-04-14 21:46 ` Laurence Oberman
1 sibling, 1 reply; 12+ messages in thread
From: Adam Talbot @ 2015-04-14 21:41 UTC (permalink / raw)
To: James Bottomley; +Cc: linux-scsi
Removing the sas expander and attaching the SATA drives directly works
just fine. I had to limp along with the drives direct attached for a
while, while debugging.
Here is the dump from: Linux version 3.15.1-031501-generic. Just tested it.
[ 8.455203] scsi4 : mvsas
[ 8.663885] floppy0: no floppy controllers found
[ 8.720580] BUG: unable to handle kernel NULL pointer dereference
at 0000000000000254
[ 8.728583] IP: [<ffffffffa006f5b0>] mvs_task_prep_ata+0x80/0x3a0 [mvsas]
[ 8.735470] PGD 0
[ 8.737586] Oops: 0000 [#1] SMP
[ 8.740965] Modules linked in: uas hid_generic usbkbd usbmouse
usbhid hid e1000e mvsas ptp libsas pata_acpi pps_core
scsi_transport_sas
[ 8.753918] CPU: 3 PID: 193 Comm: kworker/u16:8 Not tainted
3.15.1-031501-generic #201406161841
[ 8.762677] Hardware name: Supermicro X8SIE/X8SIE, BIOS 1.2a 06/27/2012
[ 8.769863] Workqueue: events_unbound async_run_entry_fn
[ 8.775268] task: ffff88032a031930 ti: ffff88032a038000 task.ti:
ffff88032a038000
[ 8.782814] RIP: 0010:[<ffffffffa006f5b0>] [<ffffffffa006f5b0>]
mvs_task_prep_ata+0x80/0x3a0 [mvsas]
[ 8.792146] RSP: 0018:ffff88032a03b7c8 EFLAGS: 00010097
[ 8.797508] RAX: 0000000000000030 RBX: ffff88032a780000 RCX: 0000000000000000
[ 8.804691] RDX: ffff88032a7838c0 RSI: ffff88032a7a5668 RDI: ffff88032a780000
[ 8.811873] RBP: ffff88032a03b838 R08: 0000000000000002 R09: 0000000000001000
[ 8.819056] R10: 0000000000040000 R11: ffff88032a7a5660 R12: ffff88032a60c280
[ 8.826239] R13: ffff88032a03b898 R14: ffff88032a758080 R15: ffff88032b2c7000
[ 8.833422] FS: 0000000000000000(0000) GS:ffff88033fcc0000(0000)
knlGS:0000000000000000
[ 8.841575] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 8.847373] CR2: 0000000000000254 CR3: 0000000001c12000 CR4: 00000000000007e0
[ 8.854556] Stack:
[ 8.856617] ffff88032a03b818 ffffffff00000002 ffff88032a03b7f8
0000000000000000
[ 8.864277] ffff88032a7838c0 0000000000000046 ffff88032a781e38
ffffffff811a435e
[ 8.871936] 0000000000014500 ffff88032a60c280 ffff88032a7838c0
ffff88032a780000
[ 8.879604] Call Trace:
[ 8.882104] [<ffffffff811a435e>] ? dma_pool_alloc+0xce/0x100
[ 8.887897] [<ffffffffa00704bb>] mvs_task_prep+0x58b/0x620 [mvsas]
[ 8.894214] [<ffffffffa00705b8>] mvs_task_exec.isra.14+0x68/0xf0 [mvsas]
[ 8.901051] [<ffffffffa0071159>] mvs_queue_command+0x39/0x40 [mvsas]
[ 8.907542] [<ffffffffa00568ab>] sas_ata_qc_issue+0x28b/0x2d0 [libsas]
[ 8.914203] [<ffffffffa0051e1f>] ? sas_ex_phy_discover+0xaf/0xe0 [libsas]
[ 8.921127] [<ffffffff8152520f>] ata_qc_issue+0x18f/0x2d0
[ 8.926660] [<ffffffff81525648>] ata_exec_internal_sg+0x2f8/0x5d0
[ 8.932891] [<ffffffff81525992>] ata_exec_internal+0x72/0xb0
[ 8.938686] [<ffffffff8152618a>] ata_do_dev_read_id+0x2a/0x30
[ 8.944572] [<ffffffffa00564b0>] ?
sas_ata_internal_abort+0x120/0x120 [libsas]
[ 8.951945] [<ffffffff815263d5>] ata_dev_read_id+0x245/0x460
[ 8.957741] [<ffffffff81532b1c>] ? ata_eh_reset+0x24c/0xe20
[ 8.963452] [<ffffffff81531a78>] ata_eh_revalidate_and_attach+0x198/0x3a0
[ 8.970373] [<ffffffff81533ee9>] ata_eh_recover+0x599/0x7e0
[ 8.976084] [<ffffffff815283e0>] ? sata_print_link_status+0xc0/0xc0
[ 8.982488] [<ffffffffa00564b0>] ?
sas_ata_internal_abort+0x120/0x120 [libsas]
[ 8.989858] [<ffffffff81528930>] ? sata_std_hardreset+0x50/0x50
[ 8.995916] [<ffffffffa00564b0>] ?
sas_ata_internal_abort+0x120/0x120 [libsas]
[ 9.003289] [<ffffffff81528930>] ? sata_std_hardreset+0x50/0x50
[ 9.009347] [<ffffffffa00564b0>] ?
sas_ata_internal_abort+0x120/0x120 [libsas]
[ 9.016719] [<ffffffff815348c2>] ata_do_eh+0x52/0xc0
[ 9.021823] [<ffffffff815283e0>] ? sata_print_link_status+0xc0/0xc0
[ 9.028225] [<ffffffff81534977>] ata_std_error_handler+0x47/0x80
[ 9.034370] [<ffffffff8152fa78>] ? ata_eh_handle_port_resume+0x38/0x160
[ 9.041119] [<ffffffff8153459b>] ata_scsi_port_error_handler+0x39b/0x5a0
[ 9.047956] [<ffffffffa00562c5>] async_sas_ata_eh+0x55/0x90 [libsas]
[ 9.054444] [<ffffffff8109770b>] async_run_entry_fn+0x3b/0x140
[ 9.060415] [<ffffffff8108935f>] process_one_work+0x17f/0x4c0
[ 9.066298] [<ffffffff8108a57b>] worker_thread+0x11b/0x3d0
[ 9.071921] [<ffffffff8108a460>] ? manage_workers.isra.21+0x190/0x190
[ 9.078496] [<ffffffff810913c9>] kthread+0xc9/0xe0
[ 9.083427] [<ffffffff81091300>] ? flush_kthread_worker+0xb0/0xb0
[ 9.089657] [<ffffffff817845bc>] ret_from_fork+0x7c/0xb0
[ 9.095106] [<ffffffff81091300>] ? flush_kthread_worker+0xb0/0xb0
[ 9.101335] Code: 00 00 48 8b 0c c8 0f 84 a7 02 00 00 44 89 c0 41
b9 00 10 00 00 48 8d 34 80 48 8d 04 70 48 8d b4 c3 b8 55 02 00 8b 43
58 89 46 1c <8b> 89 54 02 00 00 44 89 c0 8b 7b 58 0d 00 00 00 70 4c 8b
53 48
[ 9.123612] RIP [<ffffffffa006f5b0>] mvs_task_prep_ata+0x80/0x3a0 [mvsas]
[ 9.130587] RSP <ffff88032a03b7c8>
[ 9.134121] CR2: 0000000000000254
[ 9.137484] ---[ end trace 626da177ae6396db ]---
On Tue, Apr 14, 2015 at 2:16 PM, James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
> On Tue, 2015-04-14 at 14:03 -0700, Adam Talbot wrote:
>> To make a very long debugging story short, I think there is an issues/bug
>> with the mvsas driver. It works, with older kernels, and breaks on
>> newer kernels.
>>
>> My Debian Jessie system was running great on a 3.18 kernel. Changed
>> cases to a newer supermicro case with a SAS expander backplane (SAS933EL). That
>> was the only hardware change. Now, when ever I boot, the system kernel panics.
>>
>> 3.2.65-1+deb7u2 works
>> 3.9.0 Gentoo CD works
>> 3.16+ all fail
>> Attached are 3 kernel panics on 3.16+ kernels.
>>
>> Motherboard is a Supermicro X8SIE, with a "Marvell Technology Group Ltd.
>> 88SE6440 SAS/SATA PCIe controller"
>>
>> Is this a known bug?
>
> Well, you're the only person that's reported it so far.
>
> I think based on the above is that your configuration is a single
> expander attached SATA device ... and if you move it to be non expander
> attached it works fine?
>
>> At this point I have two options:
>> Stick with the old kernel (yuck)
>> Buy a new card running a better supported chipset
>>
>> Any help would be greatly appreciated
>> Thanks
>
> You didn't specify: does 3.15 work? At least the highest working kernel
> version would help me narrow down potential problems.
>
> James
>
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: mvsas panics and dies when attached to a port extender on newer kernels
2015-04-14 21:41 ` Adam Talbot
@ 2015-04-16 5:16 ` James Bottomley
2015-04-16 17:26 ` Adam Talbot
0 siblings, 1 reply; 12+ messages in thread
From: James Bottomley @ 2015-04-16 5:16 UTC (permalink / raw)
To: Adam Talbot; +Cc: linux-scsi
On Tue, 2015-04-14 at 14:41 -0700, Adam Talbot wrote:
> Removing the sas expander and attaching the SATA drives directly works
> just fine. I had to limp along with the drives direct attached for a
> while, while debugging.
Well, that narrows it down. It looks like there's a longstanding bug in
mvs_task_prep_ata() where the physical PHY field is populated by taking
an index through the HBA phy table. This field is ignored for STP but
the phy table is too small and it uses the expander phy number to index
it (hence the GPF as we fall off the end of the phy table trying to
dereference sas_phy->id).
This should fix the problem.
James
---
diff --git a/drivers/scsi/mvsas/mv_sas.c b/drivers/scsi/mvsas/mv_sas.c
index 2d5ab6d..454536c 100644
--- a/drivers/scsi/mvsas/mv_sas.c
+++ b/drivers/scsi/mvsas/mv_sas.c
@@ -441,14 +441,11 @@ static u32 mvs_get_ncq_tag(struct sas_task *task, u32 *tag)
static int mvs_task_prep_ata(struct mvs_info *mvi,
struct mvs_task_exec_info *tei)
{
- struct sas_ha_struct *sha = mvi->sas;
struct sas_task *task = tei->task;
struct domain_device *dev = task->dev;
struct mvs_device *mvi_dev = dev->lldd_dev;
struct mvs_cmd_hdr *hdr = tei->hdr;
struct asd_sas_port *sas_port = dev->port;
- struct sas_phy *sphy = dev->phy;
- struct asd_sas_phy *sas_phy = sha->sas_phy[sphy->number];
struct mvs_slot_info *slot;
void *buf_prd;
u32 tag = tei->tag, hdr_tag;
@@ -468,7 +465,7 @@ static int mvs_task_prep_ata(struct mvs_info *mvi,
slot->tx = mvi->tx_prod;
del_q = TXQ_MODE_I | tag |
(TXQ_CMD_STP << TXQ_CMD_SHIFT) |
- (MVS_PHY_ID << TXQ_PHY_SHIFT) |
+ ((sas_port->phy_mask & TXQ_PHY_MASK) << TXQ_PHY_SHIFT) |
(mvi_dev->taskfileset << TXQ_SRS_SHIFT);
mvi->tx[mvi->tx_prod] = cpu_to_le32(del_q);
^ permalink raw reply related [flat|nested] 12+ messages in thread* Re: mvsas panics and dies when attached to a port extender on newer kernels
2015-04-16 5:16 ` James Bottomley
@ 2015-04-16 17:26 ` Adam Talbot
2015-04-16 17:28 ` James Bottomley
0 siblings, 1 reply; 12+ messages in thread
From: Adam Talbot @ 2015-04-16 17:26 UTC (permalink / raw)
To: James Bottomley; +Cc: linux-scsi
Wow, I forgot how long it takes to compile a full kernel. Glad I ran
Gentoo for a few years and knew how to compile and apply patches. I
will admit I had to dust off some mental cobwebs.
Pre-patched 4.0.0 kernel tree: Oops, as expected
Patched 4.0.0 kernel tree: IT WORKED!!!!! Basic mount, and checking a
few files all looks good. I will start a RAID check as that should
really push the driver. I will report back tomorrow when it finishes.
Logs below.
[ 5.072154] scsi host4: mvsas
[ 5.180706] floppy0: no floppy controllers found
[ 5.339613] ata12.00: ATA-8: ST32000542AS, CC35, max UDMA/133
[ 5.339616] ata10.00: ATA-7: HDS725050KLA360, K2AOAD1A, max UDMA/133
[ 5.339624] ata10.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[ 5.339892] ata11.00: ATA-7: HDS725050KLA360, K2AOAD1A, max UDMA/133
[ 5.339893] ata11.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[ 5.340146] ata13.00: ATA-8: ST2000DL003-9VT166, CC32, max UDMA/133
[ 5.340147] ata13.00: 3907029168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[ 5.341015] ata10.00: configured for UDMA/133
[ 5.341344] ata11.00: configured for UDMA/133
[ 5.341406] ata13.00: configured for UDMA/133
[ 5.373207] ata5.00: ATA-8: WDC WD20EADS-11R6B1, 80.00A80, max UDMA/133
[ 5.373208] ata5.00: 3907029168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[ 5.374223] ata8.00: ATA-8: WDC WD20EADS-42R6B0, 02.00A02, max UDMA/133
[ 5.374224] ata8.00: 3907029168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[ 5.379766] ata5.00: configured for UDMA/133
[ 5.380106] ata8.00: configured for UDMA/133
[ 5.397396] ata7.00: ATA-8: WDC WD20EARS-00S8B1, 80.00A80, max UDMA/133
[ 5.397396] ata7.00: 3907029168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[ 5.403539] ata7.00: configured for UDMA/133
[ 5.451846] ata12.00: 3907029168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[ 5.460457] ata12.00: configured for UDMA/133
[ 5.738535] ata6.00: ATA-8: WDC WD20EADS-11R6B1, 80.00A80, max UDMA/133
[ 5.745214] ata6.00: 3907029168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[ 5.745267] ata9.00: ATA-8: WDC WD20EADS-42R6B0, 02.00A02, max UDMA/133
[ 5.745268] ata9.00: 3907029168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[ 5.765626] ata9.00: configured for UDMA/133
[ 5.771347] ata6.00: configured for UDMA/133
[ 5.784708] scsi 4:0:0:0: Direct-Access ATA WDC
WD20EADS-11R 0A80 PQ: 0 ANSI: 5
[ 5.793137] scsi 4:0:1:0: Direct-Access ATA WDC
WD20EADS-11R 0A80 PQ: 0 ANSI: 5
[ 5.801560] scsi 4:0:2:0: Direct-Access ATA WDC
WD20EARS-00S 0A80 PQ: 0 ANSI: 5
[ 5.809982] scsi 4:0:3:0: Direct-Access ATA WDC
WD20EADS-42R 0A02 PQ: 0 ANSI: 5
[ 5.818404] scsi 4:0:4:0: Direct-Access ATA WDC
WD20EADS-42R 0A02 PQ: 0 ANSI: 5
[ 5.826816] scsi 4:0:5:0: Direct-Access ATA
HDS725050KLA360 AD1A PQ: 0 ANSI: 5
[ 5.835171] scsi 4:0:6:0: Direct-Access ATA
HDS725050KLA360 AD1A PQ: 0 ANSI: 5
[ 5.843526] scsi 4:0:7:0: Direct-Access ATA ST32000542AS
CC35 PQ: 0 ANSI: 5
[ 5.851928] scsi 4:0:8:0: Direct-Access ATA
ST2000DL003-9VT1 CC32 PQ: 0 ANSI: 5
[ 5.862669] scsi 4:0:9:0: Enclosure LSILOGIC SASX28 A.1
7014 PQ: 0 ANSI: 3
[ 6.140482] scsi 5:0:0:0: Direct-Access Generic USB EDC
1.00 PQ: 0 ANSI: 2
[ 6.148955] sd 5:0:0:0: Attached scsi generic sg2 type 0
[ 6.149610] sd 5:0:0:0: [sdc] 2007040 512-byte logical blocks:
(1.02 GB/980 MiB)
[ 6.150218] sd 5:0:0:0: [sdc] Write Protect is off
[ 6.150840] sd 5:0:0:0: [sdc] No Caching mode page found
[ 6.150841] sd 5:0:0:0: [sdc] Assuming drive cache: write through
[ 6.153552] sdc: sdc1
[ 6.156225] sd 5:0:0:0: [sdc] Attached SCSI disk
[ 6.185605] sd 4:0:0:0: [sdd] 3907029168 512-byte logical blocks:
(2.00 TB/1.81 TiB)
[ 6.185633] sd 4:0:0:0: Attached scsi generic sg3 type 0
[ 6.185799] sd 4:0:1:0: Attached scsi generic sg4 type 0
[ 6.185801] sd 4:0:1:0: [sde] 3907029168 512-byte logical blocks:
(2.00 TB/1.81 TiB)
[ 6.185802] sd 4:0:1:0: [sde] 4096-byte physical blocks
[ 6.185837] sd 4:0:1:0: [sde] Write Protect is off
[ 6.185848] sd 4:0:1:0: [sde] Write cache: enabled, read cache:
enabled, doesn't support DPO or FUA
[ 6.185978] sd 4:0:2:0: [sdf] 3907029168 512-byte logical blocks:
(2.00 TB/1.81 TiB)
[ 6.185986] sd 4:0:2:0: Attached scsi generic sg5 type 0
[ 6.186059] sd 4:0:2:0: [sdf] Write Protect is off
[ 6.186148] sd 4:0:3:0: [sdg] 3907029168 512-byte logical blocks:
(2.00 TB/1.81 TiB)
[ 6.186149] sd 4:0:2:0: [sdf] Write cache: enabled, read cache:
enabled, doesn't support DPO or FUA
[ 6.186163] sd 4:0:3:0: Attached scsi generic sg6 type 0
[ 6.186188] sde: sde1
[ 6.186205] sd 4:0:3:0: [sdg] Write Protect is off
[ 6.186234] sd 4:0:3:0: [sdg] Write cache: enabled, read cache:
enabled, doesn't support DPO or FUA
[ 6.186395] sd 4:0:4:0: [sdh] 3907029168 512-byte logical blocks:
(2.00 TB/1.81 TiB)
[ 6.186417] sd 4:0:1:0: [sde] Attached SCSI disk
[ 6.186439] sd 4:0:4:0: [sdh] Write Protect is off
[ 6.186441] sd 4:0:4:0: Attached scsi generic sg7 type 0
[ 6.186461] sd 4:0:4:0: [sdh] Write cache: enabled, read cache:
enabled, doesn't support DPO or FUA
[ 6.186628] sd 4:0:5:0: [sdi] 976773168 512-byte logical blocks:
(500 GB/465 GiB)
[ 6.186670] sd 4:0:5:0: Attached scsi generic sg8 type 0
[ 6.186685] sdf: sdf1
[ 6.186696] sdg: sdg1
[ 6.186726] sd 4:0:5:0: [sdi] Write Protect is off
[ 6.186877] sdh: sdh1
[ 6.186882] sd 4:0:6:0: [sdj] 976773168 512-byte logical blocks:
(500 GB/465 GiB)
[ 6.186905] sd 4:0:2:0: [sdf] Attached SCSI disk
[ 6.186911] sd 4:0:6:0: Attached scsi generic sg9 type 0
[ 6.186925] sd 4:0:5:0: [sdi] Write cache: enabled, read cache:
enabled, doesn't support DPO or FUA
[ 6.186945] sd 4:0:3:0: [sdg] Attached SCSI disk
[ 6.186948] sd 4:0:6:0: [sdj] Write Protect is off
[ 6.186977] sd 4:0:6:0: [sdj] Write cache: enabled, read cache:
enabled, doesn't support DPO or FUA
[ 6.187142] sd 4:0:4:0: [sdh] Attached SCSI disk
[ 6.187171] sd 4:0:7:0: [sdk] 3907029168 512-byte logical blocks:
(2.00 TB/1.81 TiB)
[ 6.187198] sd 4:0:7:0: Attached scsi generic sg10 type 0
[ 6.187240] sd 4:0:7:0: [sdk] Write Protect is off
[ 6.187266] sd 4:0:7:0: [sdk] Write cache: enabled, read cache:
enabled, doesn't support DPO or FUA
[ 6.187355] sd 4:0:8:0: [sdl] 3907029168 512-byte logical blocks:
(2.00 TB/1.81 TiB)
[ 6.187377] sd 4:0:8:0: [sdl] Write Protect is off
[ 6.187394] sd 4:0:8:0: [sdl] Write cache: enabled, read cache:
enabled, doesn't support DPO or FUA
[ 6.187400] sd 4:0:8:0: Attached scsi generic sg11 type 0
[ 6.187589] scsi 4:0:9:0: Attached scsi generic sg12 type 13
[ 6.200457] sdl: sdl1
[ 6.200601] sd 4:0:8:0: [sdl] Attached SCSI disk
[ 6.202898] sdi: sdi1
[ 6.203316] sd 4:0:5:0: [sdi] Attached SCSI disk
[ 6.203662] random: nonblocking pool is initialized
[ 6.203687] sdj: sdj1
[ 6.204047] sd 4:0:6:0: [sdj] Attached SCSI disk
[ 6.207504] sdk: sdk1
[ 6.207764] sd 4:0:7:0: [sdk] Attached SCSI disk
[ 6.488728] sd 4:0:0:0: [sdd] 4096-byte physical blocks
[ 6.494046] sd 4:0:0:0: [sdd] Write Protect is off
[ 6.498906] sd 4:0:0:0: [sdd] Write cache: enabled, read cache:
enabled, doesn't support DPO or FUA
[ 6.508482] sdd: sdd1
[ 6.511045] sd 4:0:0:0: [sdd] Attached SCSI disk
[ 6.654475] md: bind<sdl1>
[ 6.668281] md: bind<sdh1>
[ 6.672779] md: bind<sdd1>
[ 6.677573] md: bind<sdj1>
[ 6.681852] md: bind<sdi1>
[ 6.685906] md/raid1:md125: active with 2 out of 2 mirrors
[ 6.686221] md: bind<sde1>
[ 6.694288] md125: detected capacity change from 0 to 499972440064
[ 6.697903] md: bind<sdk1>
[ 6.707702] md: bind<sdg1>
[ 6.789676] md: bind<sdf1>
[ 6.864048] raid6: sse2x1 7299 MB/s
[ 6.932065] raid6: sse2x2 8507 MB/s
[ 7.000052] raid6: sse2x4 9334 MB/s
[ 7.003883] raid6: using algorithm sse2x4 (9334 MB/s)
[ 7.009030] raid6: using ssse3x2 recovery algorithm
[ 7.016950] xor: measuring software checksum speed
[ 7.060033] prefetch64-sse: 12476.000 MB/sec
[ 7.104097] generic_sse: 11050.000 MB/sec
[ 7.108414] xor: using function: prefetch64-sse (12476.000 MB/sec)
[ 7.115066] async_tx: api initialized (async)
[ 7.120852] md: raid6 personality registered for level 6
[ 7.126224] md: raid5 personality registered for level 5
[ 7.131583] md: raid4 personality registered for level 4
[ 7.137118] md/raid:md126: device sdf1 operational as raid disk 2
[ 7.143266] md/raid:md126: device sdg1 operational as raid disk 3
[ 7.149407] md/raid:md126: device sdk1 operational as raid disk 6
[ 7.155550] md/raid:md126: device sde1 operational as raid disk 1
[ 7.161693] md/raid:md126: device sdd1 operational as raid disk 0
[ 7.167836] md/raid:md126: device sdh1 operational as raid disk 4
[ 7.173979] md/raid:md126: device sdl1 operational as raid disk 5
[ 7.180538] md/raid:md126: allocated 0kB
[ 7.184619] md/raid:md126: raid level 6 active with 7 out of 7
devices, algorithm 2
On Wed, Apr 15, 2015 at 10:16 PM, James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
> On Tue, 2015-04-14 at 14:41 -0700, Adam Talbot wrote:
>> Removing the sas expander and attaching the SATA drives directly works
>> just fine. I had to limp along with the drives direct attached for a
>> while, while debugging.
>
> Well, that narrows it down. It looks like there's a longstanding bug in
> mvs_task_prep_ata() where the physical PHY field is populated by taking
> an index through the HBA phy table. This field is ignored for STP but
> the phy table is too small and it uses the expander phy number to index
> it (hence the GPF as we fall off the end of the phy table trying to
> dereference sas_phy->id).
>
> This should fix the problem.
>
> James
>
> ---
>
> diff --git a/drivers/scsi/mvsas/mv_sas.c b/drivers/scsi/mvsas/mv_sas.c
> index 2d5ab6d..454536c 100644
> --- a/drivers/scsi/mvsas/mv_sas.c
> +++ b/drivers/scsi/mvsas/mv_sas.c
> @@ -441,14 +441,11 @@ static u32 mvs_get_ncq_tag(struct sas_task *task, u32 *tag)
> static int mvs_task_prep_ata(struct mvs_info *mvi,
> struct mvs_task_exec_info *tei)
> {
> - struct sas_ha_struct *sha = mvi->sas;
> struct sas_task *task = tei->task;
> struct domain_device *dev = task->dev;
> struct mvs_device *mvi_dev = dev->lldd_dev;
> struct mvs_cmd_hdr *hdr = tei->hdr;
> struct asd_sas_port *sas_port = dev->port;
> - struct sas_phy *sphy = dev->phy;
> - struct asd_sas_phy *sas_phy = sha->sas_phy[sphy->number];
> struct mvs_slot_info *slot;
> void *buf_prd;
> u32 tag = tei->tag, hdr_tag;
> @@ -468,7 +465,7 @@ static int mvs_task_prep_ata(struct mvs_info *mvi,
> slot->tx = mvi->tx_prod;
> del_q = TXQ_MODE_I | tag |
> (TXQ_CMD_STP << TXQ_CMD_SHIFT) |
> - (MVS_PHY_ID << TXQ_PHY_SHIFT) |
> + ((sas_port->phy_mask & TXQ_PHY_MASK) << TXQ_PHY_SHIFT) |
> (mvi_dev->taskfileset << TXQ_SRS_SHIFT);
> mvi->tx[mvi->tx_prod] = cpu_to_le32(del_q);
>
>
>
>
>
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: mvsas panics and dies when attached to a port extender on newer kernels
2015-04-16 17:26 ` Adam Talbot
@ 2015-04-16 17:28 ` James Bottomley
2015-04-16 17:31 ` Adam Talbot
0 siblings, 1 reply; 12+ messages in thread
From: James Bottomley @ 2015-04-16 17:28 UTC (permalink / raw)
To: Adam Talbot; +Cc: linux-scsi
On Thu, 2015-04-16 at 10:26 -0700, Adam Talbot wrote:
> Wow, I forgot how long it takes to compile a full kernel. Glad I ran
> Gentoo for a few years and knew how to compile and apply patches. I
> will admit I had to dust off some mental cobwebs.
>
> Pre-patched 4.0.0 kernel tree: Oops, as expected
> Patched 4.0.0 kernel tree: IT WORKED!!!!! Basic mount, and checking a
> few files all looks good. I will start a RAID check as that should
> really push the driver. I will report back tomorrow when it finishes.
Could you also check the direct ATA attachment case to make sure I
didn't screw that up. The fix is based on a theory about how the driver
operates rather than any actual documentation.
Thanks,
James
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: mvsas panics and dies when attached to a port extender on newer kernels
2015-04-16 17:28 ` James Bottomley
@ 2015-04-16 17:31 ` Adam Talbot
2015-04-17 2:26 ` Adam Talbot
0 siblings, 1 reply; 12+ messages in thread
From: Adam Talbot @ 2015-04-16 17:31 UTC (permalink / raw)
To: James Bottomley; +Cc: linux-scsi
Oh! Good idea. ;-)
I will test it in 6~8 hours, once the raid check finishes.
On Thu, Apr 16, 2015 at 10:28 AM, James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
> On Thu, 2015-04-16 at 10:26 -0700, Adam Talbot wrote:
>> Wow, I forgot how long it takes to compile a full kernel. Glad I ran
>> Gentoo for a few years and knew how to compile and apply patches. I
>> will admit I had to dust off some mental cobwebs.
>>
>> Pre-patched 4.0.0 kernel tree: Oops, as expected
>> Patched 4.0.0 kernel tree: IT WORKED!!!!! Basic mount, and checking a
>> few files all looks good. I will start a RAID check as that should
>> really push the driver. I will report back tomorrow when it finishes.
>
> Could you also check the direct ATA attachment case to make sure I
> didn't screw that up. The fix is based on a theory about how the driver
> operates rather than any actual documentation.
>
> Thanks,
>
> James
>
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: mvsas panics and dies when attached to a port extender on newer kernels
2015-04-16 17:31 ` Adam Talbot
@ 2015-04-17 2:26 ` Adam Talbot
0 siblings, 0 replies; 12+ messages in thread
From: Adam Talbot @ 2015-04-17 2:26 UTC (permalink / raw)
To: James Bottomley; +Cc: linux-scsi
Tested against main RAID6, 7 disk array, with sas extender and work
with out error.
Tested against 2X mirror of SSD's, direct attached, and worked with out error.
Check was a simple RAID check. "echo check > /sys/block/md126/md/sync_action"
Patched against:
root@nas:~# uname -a
Linux nas 4.0.0 #1 SMP Thu Apr 16 09:05:59 PDT 2015 x86_64 GNU/Linux
Many thanks to all involved in helping me debug this.
Should this patch be tested by a few other then added to the kernel tree?
On Thu, Apr 16, 2015 at 10:31 AM, Adam Talbot <ajtalbot1@gmail.com> wrote:
> Oh! Good idea. ;-)
> I will test it in 6~8 hours, once the raid check finishes.
>
> On Thu, Apr 16, 2015 at 10:28 AM, James Bottomley
> <James.Bottomley@hansenpartnership.com> wrote:
>> On Thu, 2015-04-16 at 10:26 -0700, Adam Talbot wrote:
>>> Wow, I forgot how long it takes to compile a full kernel. Glad I ran
>>> Gentoo for a few years and knew how to compile and apply patches. I
>>> will admit I had to dust off some mental cobwebs.
>>>
>>> Pre-patched 4.0.0 kernel tree: Oops, as expected
>>> Patched 4.0.0 kernel tree: IT WORKED!!!!! Basic mount, and checking a
>>> few files all looks good. I will start a RAID check as that should
>>> really push the driver. I will report back tomorrow when it finishes.
>>
>> Could you also check the direct ATA attachment case to make sure I
>> didn't screw that up. The fix is based on a theory about how the driver
>> operates rather than any actual documentation.
>>
>> Thanks,
>>
>> James
>>
>>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: mvsas panics and dies when attached to a port extender on newer kernels
2015-04-14 21:16 ` James Bottomley
2015-04-14 21:41 ` Adam Talbot
@ 2015-04-14 21:46 ` Laurence Oberman
1 sibling, 0 replies; 12+ messages in thread
From: Laurence Oberman @ 2015-04-14 21:46 UTC (permalink / raw)
To: James Bottomley; +Cc: Adam Talbot, linux-scsi@vger.kernel.org
Any chance you can capture a vmcore (kernel only pages), I will
provide an upload location.
Thanks
Laurence
On Tue, Apr 14, 2015 at 5:16 PM, James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
> On Tue, 2015-04-14 at 14:03 -0700, Adam Talbot wrote:
>> To make a very long debugging story short, I think there is an issues/bug
>> with the mvsas driver. It works, with older kernels, and breaks on
>> newer kernels.
>>
>> My Debian Jessie system was running great on a 3.18 kernel. Changed
>> cases to a newer supermicro case with a SAS expander backplane (SAS933EL). That
>> was the only hardware change. Now, when ever I boot, the system kernel panics.
>>
>> 3.2.65-1+deb7u2 works
>> 3.9.0 Gentoo CD works
>> 3.16+ all fail
>> Attached are 3 kernel panics on 3.16+ kernels.
>>
>> Motherboard is a Supermicro X8SIE, with a "Marvell Technology Group Ltd.
>> 88SE6440 SAS/SATA PCIe controller"
>>
>> Is this a known bug?
>
> Well, you're the only person that's reported it so far.
>
> I think based on the above is that your configuration is a single
> expander attached SATA device ... and if you move it to be non expander
> attached it works fine?
>
>> At this point I have two options:
>> Stick with the old kernel (yuck)
>> Buy a new card running a better supported chipset
>>
>> Any help would be greatly appreciated
>> Thanks
>
> You didn't specify: does 3.15 work? At least the highest working kernel
> version would help me narrow down potential problems.
>
> James
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2015-04-23 16:27 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-04-13 15:11 mvsas panics and dies when attached to a port extender on newer kernels Adam Talbot
2015-04-23 11:10 ` Ceriel Jacobs
2015-04-23 16:27 ` Adam Talbot
-- strict thread matches above, loose matches on Subject: below --
2015-04-14 21:03 Adam Talbot
2015-04-14 21:16 ` James Bottomley
2015-04-14 21:41 ` Adam Talbot
2015-04-16 5:16 ` James Bottomley
2015-04-16 17:26 ` Adam Talbot
2015-04-16 17:28 ` James Bottomley
2015-04-16 17:31 ` Adam Talbot
2015-04-17 2:26 ` Adam Talbot
2015-04-14 21:46 ` Laurence Oberman
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox