Re: BCM91x80A/B PCI DMA problems

[Martin Michlmayr <tbm@cyrius.com>]

* Mark E Mason <mark.e.mason@broadcom.com> [2006-03-13 18:31]:
> Again -- which PCI slot are you seeing this with?

The one you recommended (i.e. the one closest to the CPU), but I see
the same with the other slots.

I have done some more research today with the help of Lennert
Buytenhek, an ARM hacker.  While we were not able to pin down the
exact problem, I think we have enough information so you can look into
it and hopefully suggest a fix.

Finding 1: while it fails with > 1 GB RAM, using 512 MB or 1 GB works.

Finding 2: with > 1 GB RAM, we're getting addresses over 4 GB in
sg->dma_address.  We put some printks into arch/mips/mm/dma-coherent.c
and while everything looks okay with 1 GB of RAM, with 2 GB I get e.g.
    map page a8000000063f36c0 to addr 000000017fc68000

That's an address > 4G.  Lennert thinks that it's "most likely going
to stuff it into a 32bit address register somewhere" and: "I can't see
how it can work at all if it passes the >32bit pci address to the
device.  it should detect, hey, this is above 4G, so then allocate a
buffer below 4G, copy it into there, and pass _that_ buffer to the
device.  that's called dma bounce buffering."

Finding 3: the memory layout is weird.
memory: 000000000fe91e00 @ 0000000000000000 (usable)
memory: 000000001ffffe00 @ 0000000080000000 (usable)
memory: 000000000ffffe00 @ 00000000c0000000 (usable)
memory: 000000003ffffe00 @ 0000000140000000 (usable)

Lennert, "if the pci controller doesn't compensate for such a weird
layout, you're bound to see pci issues."

Finding 4: the host bridge has some "unassigned" memory.  Why?

0001:00:04.0 Host bridge: Broadcom Corporation: Unknown device 0014 (rev 01)
         Flags: bus master, fast devsel, latency 0, IRQ 255
         Memory at 60000000 (32-bit, prefetchable) [size=16M]
         Memory at <unassigned> (32-bit, prefetchable)
         Memory at 70000000 (32-bit, prefetchable) [size=4K]
         Memory at <unassigned> (64-bit, prefetchable)

Does this information help?  Also, we were wondering how to find out
whether a driver is okay with 64-bit dma addresses.

Thanks a lot.
-- 
Martin Michlmayr
http://www.cyrius.com/