* ipr boot failure caused by MSI (2.6.30-rc1+)
@ 2009-05-21 17:15 James Bottomley
2009-05-21 17:23 ` James Bottomley
2009-05-21 18:47 ` Brian King
0 siblings, 2 replies; 7+ messages in thread
From: James Bottomley @ 2009-05-21 17:15 UTC (permalink / raw)
To: Wayne Boyer, Brian King; +Cc: linux-scsi
Kernels after 2.6.30-rc1 stopped booting on my powerstation. The ipr
just times out and refuses to probe devices. If I let it drop into the
initramfs system, this is what the interrupts shows:
(initramfs) cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3
16: 20 10 13 11 MPIC Level pata_amd
20: 0 0 0 0 MPIC Level ohci_hcd:usb1, ohci_hcd:usb2
21: 0 0 0 0 MPIC-U3MSI Edge ipr
68: 37 37 48 37 MPIC Edge serial
251: 10 71 69 72 MPIC Edge ipi call function
252: 1555 1779 1372 1155 MPIC Edge ipi reschedule
253: 0 0 0 0 MPIC Edge ipi call function single
254: 0 0 0 0 MPIC Edge ipi debugger
BAD: 416
So you see the IPR is the only device not receiving them.
I can fix the boot hang by reverting
commit 5a9ef25b14d39b8413364df12cb8d9bb7a673a32
Author: Wayne Boyer <wayneb@linux.vnet.ibm.com>
Date: Fri Jan 23 09:17:35 2009 -0800
[SCSI] ipr: add MSI support
The system in question is:
SYSTEM INFORMATION
Processor = PowerPC,970MP @ 2500 MHz
I/O Bridge = U4 (4.4)
SMP Size = 4 (#0 #1 #2 #3)
Boot-Date = 2009-04-21 17:13:36
Memory = 2 GB of RAM @ 666 MHz
Board Type = Bimini (7047191/0000000/1)
MFG Date = 1608
Part No. = 10N8748
FRU No. = 10N7182
FRU Serial = YL30W8106038
UUID = 00000000000000000000000000000000
Flashside = 1 (temporary)
Version = HEAD
Build Date = 12-04-2008 16:13
James
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: ipr boot failure caused by MSI (2.6.30-rc1+)
2009-05-21 17:15 ipr boot failure caused by MSI (2.6.30-rc1+) James Bottomley
@ 2009-05-21 17:23 ` James Bottomley
2009-05-21 18:47 ` Brian King
1 sibling, 0 replies; 7+ messages in thread
From: James Bottomley @ 2009-05-21 17:23 UTC (permalink / raw)
To: Wayne Boyer; +Cc: Brian King, linux-scsi
On Thu, 2009-05-21 at 12:15 -0500, James Bottomley wrote:
> I can fix the boot hang by reverting
>
> commit 5a9ef25b14d39b8413364df12cb8d9bb7a673a32
> Author: Wayne Boyer <wayneb@linux.vnet.ibm.com>
> Date: Fri Jan 23 09:17:35 2009 -0800
>
> [SCSI] ipr: add MSI support
>
Just as a follow up, after reverting the above and booting successfully,
this is what /proc/interrupts now shows:
claymoor:~# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3
16: 31 24 35 19 MPIC Level pata_amd, serial
17: 374 421 434 381 MPIC Level ipr
20: 0 0 0 0 MPIC Level ohci_hcd:usb1, ohci_hcd:usb2
21: 135 140 152 146 MPIC-U3MSI Edge eth1
68: 59 66 60 57 MPIC Edge serial
251: 65 124 119 108 MPIC Edge ipi call function
252: 1018 1383 1127 746 MPIC Edge ipi reschedule
253: 0 0 0 0 MPIC Edge ipi call function single
254: 0 0 0 0 MPIC Edge ipi debugger
BAD: 621
So MSI seems to be being used successfully on the ethernet controller,
pointing to an ipr specific problem.
James
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: ipr boot failure caused by MSI (2.6.30-rc1+)
2009-05-21 17:15 ipr boot failure caused by MSI (2.6.30-rc1+) James Bottomley
2009-05-21 17:23 ` James Bottomley
@ 2009-05-21 18:47 ` Brian King
2009-05-21 19:51 ` James Bottomley
1 sibling, 1 reply; 7+ messages in thread
From: Brian King @ 2009-05-21 18:47 UTC (permalink / raw)
To: James Bottomley; +Cc: Wayne Boyer, Brian King, linux-scsi, ppc-dev
cc'ing linuxppc-dev...
-Brian
James Bottomley wrote:
> Kernels after 2.6.30-rc1 stopped booting on my powerstation. The ipr
> just times out and refuses to probe devices. If I let it drop into the
> initramfs system, this is what the interrupts shows:
>
> (initramfs) cat /proc/interrupts
> CPU0 CPU1 CPU2 CPU3
> 16: 20 10 13 11 MPIC Level pata_amd
> 20: 0 0 0 0 MPIC Level ohci_hcd:usb1, ohci_hcd:usb2
> 21: 0 0 0 0 MPIC-U3MSI Edge ipr
> 68: 37 37 48 37 MPIC Edge serial
> 251: 10 71 69 72 MPIC Edge ipi call function
> 252: 1555 1779 1372 1155 MPIC Edge ipi reschedule
> 253: 0 0 0 0 MPIC Edge ipi call function single
> 254: 0 0 0 0 MPIC Edge ipi debugger
> BAD: 416
>
> So you see the IPR is the only device not receiving them.
>
> I can fix the boot hang by reverting
>
> commit 5a9ef25b14d39b8413364df12cb8d9bb7a673a32
> Author: Wayne Boyer <wayneb@linux.vnet.ibm.com>
> Date: Fri Jan 23 09:17:35 2009 -0800
>
> [SCSI] ipr: add MSI support
>
> The system in question is:
>
> SYSTEM INFORMATION
> Processor = PowerPC,970MP @ 2500 MHz
> I/O Bridge = U4 (4.4)
> SMP Size = 4 (#0 #1 #2 #3)
> Boot-Date = 2009-04-21 17:13:36
> Memory = 2 GB of RAM @ 666 MHz
> Board Type = Bimini (7047191/0000000/1)
> MFG Date = 1608
> Part No. = 10N8748
> FRU No. = 10N7182
> FRU Serial = YL30W8106038
> UUID = 00000000000000000000000000000000
> Flashside = 1 (temporary)
> Version = HEAD
> Build Date = 12-04-2008 16:13
>
> James
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Brian King
Linux on Power Virtualization
IBM Linux Technology Center
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: ipr boot failure caused by MSI (2.6.30-rc1+)
2009-05-21 18:47 ` Brian King
@ 2009-05-21 19:51 ` James Bottomley
2009-05-22 16:23 ` James Bottomley
0 siblings, 1 reply; 7+ messages in thread
From: James Bottomley @ 2009-05-21 19:51 UTC (permalink / raw)
To: Brian King; +Cc: Wayne Boyer, linux-scsi, ppc-dev
On Thu, 2009-05-21 at 13:47 -0500, Brian King wrote:
> cc'ing linuxppc-dev...
>
> -Brian
>
>
> James Bottomley wrote:
> > Kernels after 2.6.30-rc1 stopped booting on my powerstation. The ipr
> > just times out and refuses to probe devices. If I let it drop into the
> > initramfs system, this is what the interrupts shows:
> >
> > (initramfs) cat /proc/interrupts
> > CPU0 CPU1 CPU2 CPU3
> > 16: 20 10 13 11 MPIC Level pata_amd
> > 20: 0 0 0 0 MPIC Level ohci_hcd:usb1, ohci_hcd:usb2
> > 21: 0 0 0 0 MPIC-U3MSI Edge ipr
> > 68: 37 37 48 37 MPIC Edge serial
> > 251: 10 71 69 72 MPIC Edge ipi call function
> > 252: 1555 1779 1372 1155 MPIC Edge ipi reschedule
> > 253: 0 0 0 0 MPIC Edge ipi call function single
> > 254: 0 0 0 0 MPIC Edge ipi debugger
> > BAD: 416
> >
> > So you see the IPR is the only device not receiving them.
> >
> > I can fix the boot hang by reverting
> >
> > commit 5a9ef25b14d39b8413364df12cb8d9bb7a673a32
> > Author: Wayne Boyer <wayneb@linux.vnet.ibm.com>
> > Date: Fri Jan 23 09:17:35 2009 -0800
> >
> > [SCSI] ipr: add MSI support
> >
> > The system in question is:
> >
> > SYSTEM INFORMATION
> > Processor = PowerPC,970MP @ 2500 MHz
> > I/O Bridge = U4 (4.4)
> > SMP Size = 4 (#0 #1 #2 #3)
> > Boot-Date = 2009-04-21 17:13:36
> > Memory = 2 GB of RAM @ 666 MHz
> > Board Type = Bimini (7047191/0000000/1)
> > MFG Date = 1608
> > Part No. = 10N8748
> > FRU No. = 10N7182
> > FRU Serial = YL30W8106038
> > UUID = 00000000000000000000000000000000
> > Flashside = 1 (temporary)
> > Version = HEAD
> > Build Date = 12-04-2008 16:13
OK, so as an update, I booted to the initrd and inserted the network
modules, which are also MSI enabled and this is what I get:
(initramfs) cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3
16: 14 11 11 18 MPIC Level pata_amd
20: 0 0 0 0 MPIC Level ohci_hcd:usb1, ohci_hcd:usb2
21: 0 0 0 0 MPIC-U3MSI Edge ipr
22: 1 0 1 0 MPIC-U3MSI Edge eth0
23: 0 2 1 0 MPIC-U3MSI Edge eth1
68: 193 166 113 177 MPIC Edge serial
251: 16 65 71 70 MPIC Edge ipi call function
252: 1574 1804 1346 1289 MPIC Edge ipi reschedule
253: 0 0 0 0 MPIC Edge ipi call function single
254: 0 0 0 0 MPIC Edge ipi debugger
BAD: 1866
So clearly the MSI interrupts to the network cards are working and it
looks like just a local problem with the ipr rather than a platform
problem with MSI.
James
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: ipr boot failure caused by MSI (2.6.30-rc1+)
2009-05-21 19:51 ` James Bottomley
@ 2009-05-22 16:23 ` James Bottomley
2009-05-29 22:01 ` Brian King
0 siblings, 1 reply; 7+ messages in thread
From: James Bottomley @ 2009-05-22 16:23 UTC (permalink / raw)
To: Brian King; +Cc: ppc-dev, Wayne Boyer, linux-scsi
On Thu, 2009-05-21 at 14:51 -0500, James Bottomley wrote:
> On Thu, 2009-05-21 at 13:47 -0500, Brian King wrote:
> > cc'ing linuxppc-dev...
> >
> > -Brian
> >
> >
> > James Bottomley wrote:
> > > Kernels after 2.6.30-rc1 stopped booting on my powerstation. The ipr
> > > just times out and refuses to probe devices. If I let it drop into the
> > > initramfs system, this is what the interrupts shows:
> > >
> > > (initramfs) cat /proc/interrupts
> > > CPU0 CPU1 CPU2 CPU3
> > > 16: 20 10 13 11 MPIC Level pata_amd
> > > 20: 0 0 0 0 MPIC Level ohci_hcd:usb1, ohci_hcd:usb2
> > > 21: 0 0 0 0 MPIC-U3MSI Edge ipr
> > > 68: 37 37 48 37 MPIC Edge serial
> > > 251: 10 71 69 72 MPIC Edge ipi call function
> > > 252: 1555 1779 1372 1155 MPIC Edge ipi reschedule
> > > 253: 0 0 0 0 MPIC Edge ipi call function single
> > > 254: 0 0 0 0 MPIC Edge ipi debugger
> > > BAD: 416
> > >
> > > So you see the IPR is the only device not receiving them.
> > >
> > > I can fix the boot hang by reverting
> > >
> > > commit 5a9ef25b14d39b8413364df12cb8d9bb7a673a32
> > > Author: Wayne Boyer <wayneb@linux.vnet.ibm.com>
> > > Date: Fri Jan 23 09:17:35 2009 -0800
> > >
> > > [SCSI] ipr: add MSI support
> > >
> > > The system in question is:
> > >
> > > SYSTEM INFORMATION
> > > Processor = PowerPC,970MP @ 2500 MHz
> > > I/O Bridge = U4 (4.4)
> > > SMP Size = 4 (#0 #1 #2 #3)
> > > Boot-Date = 2009-04-21 17:13:36
> > > Memory = 2 GB of RAM @ 666 MHz
> > > Board Type = Bimini (7047191/0000000/1)
> > > MFG Date = 1608
> > > Part No. = 10N8748
> > > FRU No. = 10N7182
> > > FRU Serial = YL30W8106038
> > > UUID = 00000000000000000000000000000000
> > > Flashside = 1 (temporary)
> > > Version = HEAD
> > > Build Date = 12-04-2008 16:13
>
> OK, so as an update, I booted to the initrd and inserted the network
> modules, which are also MSI enabled and this is what I get:
>
> (initramfs) cat /proc/interrupts
> CPU0 CPU1 CPU2 CPU3
> 16: 14 11 11 18 MPIC Level pata_amd
> 20: 0 0 0 0 MPIC Level ohci_hcd:usb1, ohci_hcd:usb2
> 21: 0 0 0 0 MPIC-U3MSI Edge ipr
> 22: 1 0 1 0 MPIC-U3MSI Edge eth0
> 23: 0 2 1 0 MPIC-U3MSI Edge eth1
> 68: 193 166 113 177 MPIC Edge serial
> 251: 16 65 71 70 MPIC Edge ipi call function
> 252: 1574 1804 1346 1289 MPIC Edge ipi reschedule
> 253: 0 0 0 0 MPIC Edge ipi call function single
> 254: 0 0 0 0 MPIC Edge ipi debugger
> BAD: 1866
>
> So clearly the MSI interrupts to the network cards are working and it
> looks like just a local problem with the ipr rather than a platform
> problem with MSI.
I saw the quirk fix for this go by:
http://ozlabs.org/pipermail/linuxppc-dev/2009-May/072436.html
Is there an easy way to trigger an interrupt on this device? Preferably
in ipr_probe_ioa() so we can at least print out if the interrupts are
misrouted and fall back from MSI to normal using the PCI infrastructure?
James
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: ipr boot failure caused by MSI (2.6.30-rc1+)
2009-05-22 16:23 ` James Bottomley
@ 2009-05-29 22:01 ` Brian King
2009-06-11 3:23 ` Wayne Boyer
0 siblings, 1 reply; 7+ messages in thread
From: Brian King @ 2009-05-29 22:01 UTC (permalink / raw)
To: James Bottomley; +Cc: ppc-dev, Wayne Boyer, linux-scsi
James Bottomley wrote:
> On Thu, 2009-05-21 at 14:51 -0500, James Bottomley wrote:
>> On Thu, 2009-05-21 at 13:47 -0500, Brian King wrote:
>>> cc'ing linuxppc-dev...
>>>
>>> -Brian
>>>
>>>
>>> James Bottomley wrote:
>>>> Kernels after 2.6.30-rc1 stopped booting on my powerstation. The ipr
>>>> just times out and refuses to probe devices. If I let it drop into the
>>>> initramfs system, this is what the interrupts shows:
>>>>
>>>> (initramfs) cat /proc/interrupts
>>>> CPU0 CPU1 CPU2 CPU3
>>>> 16: 20 10 13 11 MPIC Level pata_amd
>>>> 20: 0 0 0 0 MPIC Level ohci_hcd:usb1, ohci_hcd:usb2
>>>> 21: 0 0 0 0 MPIC-U3MSI Edge ipr
>>>> 68: 37 37 48 37 MPIC Edge serial
>>>> 251: 10 71 69 72 MPIC Edge ipi call function
>>>> 252: 1555 1779 1372 1155 MPIC Edge ipi reschedule
>>>> 253: 0 0 0 0 MPIC Edge ipi call function single
>>>> 254: 0 0 0 0 MPIC Edge ipi debugger
>>>> BAD: 416
>>>>
>>>> So you see the IPR is the only device not receiving them.
>>>>
>>>> I can fix the boot hang by reverting
>>>>
>>>> commit 5a9ef25b14d39b8413364df12cb8d9bb7a673a32
>>>> Author: Wayne Boyer <wayneb@linux.vnet.ibm.com>
>>>> Date: Fri Jan 23 09:17:35 2009 -0800
>>>>
>>>> [SCSI] ipr: add MSI support
>>>>
>>>> The system in question is:
>>>>
>>>> SYSTEM INFORMATION
>>>> Processor = PowerPC,970MP @ 2500 MHz
>>>> I/O Bridge = U4 (4.4)
>>>> SMP Size = 4 (#0 #1 #2 #3)
>>>> Boot-Date = 2009-04-21 17:13:36
>>>> Memory = 2 GB of RAM @ 666 MHz
>>>> Board Type = Bimini (7047191/0000000/1)
>>>> MFG Date = 1608
>>>> Part No. = 10N8748
>>>> FRU No. = 10N7182
>>>> FRU Serial = YL30W8106038
>>>> UUID = 00000000000000000000000000000000
>>>> Flashside = 1 (temporary)
>>>> Version = HEAD
>>>> Build Date = 12-04-2008 16:13
>> OK, so as an update, I booted to the initrd and inserted the network
>> modules, which are also MSI enabled and this is what I get:
>>
>> (initramfs) cat /proc/interrupts
>> CPU0 CPU1 CPU2 CPU3
>> 16: 14 11 11 18 MPIC Level pata_amd
>> 20: 0 0 0 0 MPIC Level ohci_hcd:usb1, ohci_hcd:usb2
>> 21: 0 0 0 0 MPIC-U3MSI Edge ipr
>> 22: 1 0 1 0 MPIC-U3MSI Edge eth0
>> 23: 0 2 1 0 MPIC-U3MSI Edge eth1
>> 68: 193 166 113 177 MPIC Edge serial
>> 251: 16 65 71 70 MPIC Edge ipi call function
>> 252: 1574 1804 1346 1289 MPIC Edge ipi reschedule
>> 253: 0 0 0 0 MPIC Edge ipi call function single
>> 254: 0 0 0 0 MPIC Edge ipi debugger
>> BAD: 1866
>>
>> So clearly the MSI interrupts to the network cards are working and it
>> looks like just a local problem with the ipr rather than a platform
>> problem with MSI.
>
> I saw the quirk fix for this go by:
>
> http://ozlabs.org/pipermail/linuxppc-dev/2009-May/072436.html
>
> Is there an easy way to trigger an interrupt on this device? Preferably
> in ipr_probe_ioa() so we can at least print out if the interrupts are
> misrouted and fall back from MSI to normal using the PCI infrastructure?
I just talked with one of the adapter firmware developers and it sounds like
this might be possible. I'll work with Wayne on coding something up to try.
-Brian
--
Brian King
Linux on Power Virtualization
IBM Linux Technology Center
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: ipr boot failure caused by MSI (2.6.30-rc1+)
2009-05-29 22:01 ` Brian King
@ 2009-06-11 3:23 ` Wayne Boyer
0 siblings, 0 replies; 7+ messages in thread
From: Wayne Boyer @ 2009-06-11 3:23 UTC (permalink / raw)
To: Brian King; +Cc: James Bottomley, ppc-dev, linux-scsi
Brian King wrote:
> James Bottomley wrote:
>> On Thu, 2009-05-21 at 14:51 -0500, James Bottomley wrote:
>> I saw the quirk fix for this go by:
>>
>> http://ozlabs.org/pipermail/linuxppc-dev/2009-May/072436.html
>>
>> Is there an easy way to trigger an interrupt on this device? Preferably
>> in ipr_probe_ioa() so we can at least print out if the interrupts are
>> misrouted and fall back from MSI to normal using the PCI infrastructure?
>
> I just talked with one of the adapter firmware developers and it sounds like
> this might be possible. I'll work with Wayne on coding something up to try.
>
> -Brian
>
I've put together some code to trigger the test MSI and either fall back to
LSI if it isn't detected, or continue on using MSI. In the routine that
does the setup and test I have this call to request_irq:
rc = request_irq(pdev->irq, ipr_test_intr, 0, IPR_NAME, ioa_cfg);
Note that the flags value is '0'. The question came up as to whether or
not this value should really be IRQF_SHARED. Given that this is for an MSI
interrupt and MSI interrupts are not shared by definition, IRQF_SHARED did
not seem correct.
What should the flags value be in this case?
--
Wayne Boyer
IBM - Beaverton, Oregon
LTC S/W Development - eServerIO
(503) 578-5236, T/L 775-5236
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2009-06-11 3:23 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2009-05-21 17:15 ipr boot failure caused by MSI (2.6.30-rc1+) James Bottomley
2009-05-21 17:23 ` James Bottomley
2009-05-21 18:47 ` Brian King
2009-05-21 19:51 ` James Bottomley
2009-05-22 16:23 ` James Bottomley
2009-05-29 22:01 ` Brian King
2009-06-11 3:23 ` Wayne Boyer
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox