FW: MSI-X on 2.6.15

FW: MSI-X on 2.6.15

[Miller, Mike (OS Dev)]

Greg KH,
We have the same results on 2.6.15, the MSI-X table is all zeroes. See
below. Any ideas of what to do do next? The driver works on x86_64. Is
there any thing extra I need to do on ia64?

Andrew, can you try 2.6.16-rc1 and/or the rc1-git4 kernels?

Thanks,
mikem

> -----Original Message-----
> From: Patterson, Andrew D (Linux R&D) 
> Sent: Friday, January 27, 2006 6:13 PM
> To: Miller, Mike (OS Dev)
> Subject: MSI-X on 2.6.15
> 
> Mike,
> 
> Here is what I get on 2.6.15.  I may look familiar.
> 
> Andrew
> 
> HP CISS Driver (v 2.6.8)
> GSI 45 (level, low) -> CPU 0 (0x0000) vector 52
> ACPI: PCI Interrupt 0000:46:01.0[A] -> GSI 45 (level, low) -> IRQ 52
> cciss: using DAC cycles
>       blocks= 143305920 block_size= 512
>       heads= 255, sectors= 32, cylinders= 17562
> 
>       blocks= 143305920 block_size= 512
>       heads= 255, sectors= 32, cylinders= 17562
> 
>       blocks= 71065440 block_size= 512
>       heads= 255, sectors= 32, cylinders= 8709
> 
>       blocks= 143305920 block_size= 512
>       heads= 255, sectors= 32, cylinders= 17562
> 
>  cciss/c0d0: p1 p2 p3
>       blocks= 143305920 block_size= 512
>       heads= 255, sectors= 32, cylinders= 17562
> 
>  cciss/c0d1: p1 p2 p3
>       blocks= 71065440 block_size= 512
>       heads= 255, sectors= 32, cylinders= 8709
> 
>  cciss/c0d2: p1 p2 p3
> GSI 63 (level, low) -> CPU 1 (0x0100) vector 53
> ACPI: PCI Interrupt 0000:4a:00.0[A] -> GSI 63 (level, low) -> IRQ 53
> cciss: offset = 0xfe000 table offset = 0xfe000 BIR = 0x0
> cciss: 0: vector = 0,msg data = 0, msg upper addr = 0,msg addr = 0
> cciss: 1: vector = 0,msg data = 0, msg upper addr = 0,msg addr = 0
> cciss: 2: vector = 0,msg data = 0, msg upper addr = 0,msg addr = 0
> cciss: 3: vector = 0,msg data = 0, msg upper addr = 0,msg addr = 0
> cciss: using DAC cycles
> GSI 71 (level, low) -> CPU 2 (0x0200) vector 59
> ACPI: PCI Interrupt 0000:88:00.0[A] -> GSI 71 (level, low) -> IRQ 59
> cciss: offset = 0xfe000 table offset = 0xfe000 BIR = 0x0
> cciss: 0: vector = 0,msg data = 0, msg upper addr = 0,msg addr = 0
> cciss: 1: vector = 0,msg data = 0, msg upper addr = 0,msg addr = 0
> cciss: 2: vector = 0,msg data = 0, msg upper addr = 0,msg addr = 0
> cciss: 3: vector = 0,msg data = 0, msg upper addr = 0,msg addr = 0
> cciss: using DAC cycles
>       blocks= 143305920 block_size= 512
>       heads= 255, sectors= 32, cylinders= 17562
> 
> 

Re: FW: MSI-X on 2.6.15

[Greg KH]

On Mon, Jan 30, 2006 at 10:33:50AM -0600, Miller, Mike (OS Dev) wrote:
> Greg KH,
> We have the same results on 2.6.15, the MSI-X table is all zeroes. See
> below. Any ideas of what to do do next? The driver works on x86_64. Is
> there any thing extra I need to do on ia64?

ia64 didn't really have msi support before the latest -mm kernel, right
Mark?

> Andrew, can you try 2.6.16-rc1 and/or the rc1-git4 kernels?

How about the -mm kernel?

thanks,

greg k-h

Re: FW: MSI-X on 2.6.15

[Ashok Raj]

On Mon, Jan 30, 2006 at 09:38:52AM -0800, Greg KH wrote:
> 
>    On Mon, Jan 30, 2006 at 10:33:50AM -0600, Miller, Mike (OS Dev) wrote:
>    > Greg KH,
>    >  We  have the same results on 2.6.15, the MSI-X table is all zeroes.
>    See
>    >  below. Any ideas of what to do do next? The driver works on x86_64.
>    Is
>    > there any thing extra I need to do on ia64?
> 
>    ia64  didn't  really  have  msi  support before the latest -mm kernel,
>    right
>    Mark?

It wasnt enabled in any default configs, but i believe its functional
even earlier. Atleast when i changed the method for irq migration via 
/proc, i remember testing on ia64 on ixgb driver. 

I havent followed the register_ops() discussion but believe it just changed
the default vector allocation to arch/platform types. But the core
msi should have worked even earlier.

Cheers,
ashok

Re: FW: MSI-X on 2.6.15

[Mark Maule]

As Ashok Raj has already responded, there was support for APIC-based MSI[-X]
on ia64 before my patches.  I personally do not know if it worked or not, my
suspicion is that it didn't get much airtime due to MSI being off by default
until somewhat recently.

I believe MSI was working on zx1 after my patch, so I suspect it worked there
before my patch as well.  I can't speak to MSI-X.

Mike, is your driver capable of MSI (vs. MSI-X)?  As a datapoint, could you
try that?

Mark

On Mon, Jan 30, 2006 at 09:38:52AM -0800, Greg KH wrote:
> On Mon, Jan 30, 2006 at 10:33:50AM -0600, Miller, Mike (OS Dev) wrote:
> > Greg KH,
> > We have the same results on 2.6.15, the MSI-X table is all zeroes. See
> > below. Any ideas of what to do do next? The driver works on x86_64. Is
> > there any thing extra I need to do on ia64?
> 
> ia64 didn't really have msi support before the latest -mm kernel, right
> Mark?
> 
> > Andrew, can you try 2.6.16-rc1 and/or the rc1-git4 kernels?
> 
> How about the -mm kernel?
> 
> thanks,
> 
> greg k-h

RE: FW: MSI-X on 2.6.15

[Miller, Mike (OS Dev)]

> -----Original Message-----
> From: Mark Maule [mailto:maule@sgi.com] 
> Sent: Monday, January 30, 2006 1:10 PM
> To: Greg KH
> Cc: Miller, Mike (OS Dev); linux-kernel@vger.kernel.org; 
> linux-scsi@vger.kernel.org; Patterson, Andrew D (Linux R&D)
> Subject: Re: FW: MSI-X on 2.6.15
> 
> As Ashok Raj has already responded, there was support for 
> APIC-based MSI[-X] on ia64 before my patches.  I personally 
> do not know if it worked or not, my suspicion is that it 
> didn't get much airtime due to MSI being off by default until 
> somewhat recently.
> 
> I believe MSI was working on zx1 after my patch, so I suspect 
> it worked there before my patch as well.  I can't speak to MSI-X.
> 
> Mike, is your driver capable of MSI (vs. MSI-X)?  As a 
> datapoint, could you try that?

The ASIC on this particular board is broken for MSI. I have a different
board that I can ship out along with a patch for MSI only. Also just
found out that this chipset is the follow on to the zx1.
I've had MSI work on zx1 based systems and PCI-X controllers. This is
the first time trying it with PCI-E and the new chipset.
The trace we captured looks normal though all the init stuff. The host
and controller talk to each other until we try to send the interrupt
message data. It goes to address 0 (so it says) and never returns.

mikem

> 
> Mark
> 
> On Mon, Jan 30, 2006 at 09:38:52AM -0800, Greg KH wrote:
> > On Mon, Jan 30, 2006 at 10:33:50AM -0600, Miller, Mike (OS 
> Dev) wrote:
> > > Greg KH,
> > > We have the same results on 2.6.15, the MSI-X table is 
> all zeroes. 
> > > See below. Any ideas of what to do do next? The driver works on 
> > > x86_64. Is there any thing extra I need to do on ia64?
> > 
> > ia64 didn't really have msi support before the latest -mm kernel, 
> > right Mark?
> > 
> > > Andrew, can you try 2.6.16-rc1 and/or the rc1-git4 kernels?
> > 
> > How about the -mm kernel?
> > 
> > thanks,
> > 
> > greg k-h
> 

Re: FW: MSI-X on 2.6.15

[Roland Dreier]

    Greg> ia64 didn't really have msi support before the latest -mm
    Greg> kernel, right Mark?

No, I know Grant Grundler has been using MSI-X on ia64 with a PCI-X
InfiniBand HCA and the ib_mthca driver for quite a while, at least
since 2.6.11 or so.  ib_mthca uses 3 separate interrupts in MSI-X
mode, so even multiple messages from a single device are fine.

 - R.