* Re: dma restriction on Itanium 2
2004-01-17 1:48 dma restriction on Itanium 2 Kallol Biswas
@ 2004-01-17 2:14 ` Grant Grundler
2004-01-17 3:58 ` Grant Grundler
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Grant Grundler @ 2004-01-17 2:14 UTC (permalink / raw)
To: linux-ia64
On Fri, Jan 16, 2004 at 05:48:15PM -0800, Kallol Biswas wrote:
> We have been seeing a problem on itanium 2 running linux 2.4.21-4.EL
> with our 10GB ethernet card.
> The error is PCI related. On PARISC systems the DMA address could not
> span page boundary due to hardware limitation,
Which HW limitation on parisc?
Are you talking about under HPUX?
Calling pci_map_single() can map up to 64-pages under parisc-linux.
(And that's a semi-arbitrary SW limit I designed into the implementation)
pci_alloc_consistent() has the same limitation since it uses
pci_map_single().
> do we have the same limitation on Itanium 2 systems also?
Under linux ZX1 DMA support has very similar constraints
HW-wise as PARISC *except* for 64-bit capable devices.
64-bit devices are not limited to 64-pages on ZX1.
> If the transmit or receive
> descriptors take more than a page then they should be aligned such a way
> that DMA does not happen over a page right?
transmit and recieve descriptors should be using memory allocated
with pci_alloc_consistent(). I'm not aware of any page boundary
limitation there. What's missing in 2.4.X is the ability
to set 64-bit attribute for PCI consistent memory.
grant
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: dma restriction on Itanium 2
2004-01-17 1:48 dma restriction on Itanium 2 Kallol Biswas
2004-01-17 2:14 ` Grant Grundler
@ 2004-01-17 3:58 ` Grant Grundler
2004-01-17 18:47 ` Christian Hinkelbein
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Grant Grundler @ 2004-01-17 3:58 UTC (permalink / raw)
To: linux-ia64
Kallol,
please continue to include the linux-ia64 list in your replies.
On Fri, Jan 16, 2004 at 06:35:53PM -0800, Kallol Biswas wrote:
> Yes, I was talking about under HP-UX.
>
> Just to verify, inability to do DMA over page boundary on PA-RISC
> running HP-UX was a software limitation or hardware? You are saying
> it was a hp-ux(software) limitation, right?
Yes. HPUX has a similar interface for DMA mapping but a completely
different VM subsystem. Linux guarantees contiguous physical blocks
for multi-page memory allocations. HPUX does not. The linux DMA
subsystem uses that assumption.
I'm hoping someone who is better at decoding MCA data can
take a quick look at the data you posted earlier. If no one
else does by next week, I'll work on it then.
grant
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: dma restriction on Itanium 2
2004-01-17 1:48 dma restriction on Itanium 2 Kallol Biswas
2004-01-17 2:14 ` Grant Grundler
2004-01-17 3:58 ` Grant Grundler
@ 2004-01-17 18:47 ` Christian Hinkelbein
2004-01-19 6:14 ` Grant Grundler
2004-01-19 16:00 ` Alex Williamson
4 siblings, 0 replies; 6+ messages in thread
From: Christian Hinkelbein @ 2004-01-17 18:47 UTC (permalink / raw)
To: linux-ia64
Grant Grundler wrote:
> Yes. HPUX has a similar interface for DMA mapping but a completely
> different VM subsystem. Linux guarantees contiguous physical blocks
> for multi-page memory allocations. HPUX does not. The linux DMA
> subsystem uses that assumption.
you are talking about kernel-mode __get_free_pages and kmalloc,
and not user-mode malloc, arent you?
christian
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: dma restriction on Itanium 2
2004-01-17 1:48 dma restriction on Itanium 2 Kallol Biswas
` (2 preceding siblings ...)
2004-01-17 18:47 ` Christian Hinkelbein
@ 2004-01-19 6:14 ` Grant Grundler
2004-01-19 16:00 ` Alex Williamson
4 siblings, 0 replies; 6+ messages in thread
From: Grant Grundler @ 2004-01-19 6:14 UTC (permalink / raw)
To: linux-ia64
On Sat, Jan 17, 2004 at 07:47:10PM +0100, Christian Hinkelbein wrote:
> Grant Grundler wrote:
> >Yes. HPUX has a similar interface for DMA mapping but a completely
> >different VM subsystem. Linux guarantees contiguous physical blocks
> >for multi-page memory allocations. HPUX does not. The linux DMA
> >subsystem uses that assumption.
> you are talking about kernel-mode __get_free_pages and kmalloc,
> and not user-mode malloc, arent you?
yes - kmalloc.
Sorry - I assumed it was clear this whole discussion
was about kernel internals.
grant
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: dma restriction on Itanium 2
2004-01-17 1:48 dma restriction on Itanium 2 Kallol Biswas
` (3 preceding siblings ...)
2004-01-19 6:14 ` Grant Grundler
@ 2004-01-19 16:00 ` Alex Williamson
4 siblings, 0 replies; 6+ messages in thread
From: Alex Williamson @ 2004-01-19 16:00 UTC (permalink / raw)
To: linux-ia64
On Fri, 2004-01-16 at 20:58, Grant Grundler wrote:
> I'm hoping someone who is better at decoding MCA data can
> take a quick look at the data you posted earlier. If no one
> else does by next week, I'll work on it then.
I took a stab at it, but I don't have my rope to slot decoder ring
handy. It's a combination of parity errors and iommu translation
errors. Here's what I get:
Error Status: 0x4a1800
ERR_ERROR Detecton of PATH_ERROR
Control: detected on control signal
Responder: responder detected
Overflow
Error Type: 0x2
System Error
This was detected on bus 0x40, which according to lspci has nothing
plugged into it. Potential stray address down the rope?
Error Status: 0xa1800 - Same as above w/o overflow
Error Type: 0x2
Bus ID: 0x60
Nothing plugged into this slot either
Error Status: 0x91600
ERR_PARITY Bus Parity Error
Address: detected on address
Requestor detected
I beleive this one is on bus 0x80. The OEM data confirms a parity error
on rope 4.
Error Status: 0x91100 ERR_MAP - Virt address not found in iotlb/iopdir
This one looks to be on rope 3, bus 0x60. The OEM data shows the DMA
target address as 0x8000000042448684. The next one is the same thing on
rope 3 to 0x800000004ee68a00. Following that is another parity error on
rope 3.
The log show the same error record id twice (22843435). RHEL 3 has a
bug in their CPE reporting/salinfo that never clears the CPE logs. You
need to do it manually either by doing 'errdump clear' at the EFI shell
or by using the /proc/sal/cpe files. I believe the latter is done by
'echo "clear 0" > /proc/sal/cpe/data'. Since much of the data doesn't
appear to match your current slot config, it's possible we're looking at
old events here. Thanks,
Alex
^ permalink raw reply [flat|nested] 6+ messages in thread