PCI QSPAN2 resource conflicts

PCI QSPAN2 resource conflicts

[Jeff Studer]

Hello list,

I am working with a linux 2.4.2 kernel in an MPC860 custom board using a
Tundra QSPAN2 PCI controller.  I have set up configuration space and can
successfully probe for PCI cards. When I load PCI drivers, I get into
trouble.

When I insmod the 3c59x.o module, I can access I/O space, but my MAC
address is read as 00:00:00:00:00:00.  I can't find manuals for this
card to verify what is going wrong, but I can see data in the I/O space
when I use our ADS board and the mpc8bug program.

When I load another driver specific to this project, therefore the need
for PCI at all, I get resource conflicts.  The 3c59x.o driver actually
uses the value I set bus->resource->start to,  whereas the other PCI
driver uses 0.

I've tried various setups for the resource.  I have set
bus->resource->start to the address in the QSPAN2's Slave0 address
setting. And I also tried do this and using the arch/ppc/mbxboot/pci.c's
functions: probe_addresses and map_pci_addrs.

I really wish there was a detailed HOWTO for all this, but in the
meantime, does anyone have any idea what are the proper steps to
complete this PCI support and/or what I may be doing wrong?

Thank you,

Jeff Studer
Software Engineer
Aquila Technologies Group

jstuder@aquilagroup.com
ph:  505-828-9100
fax: 505-828-9115


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: PCI QSPAN2 resource conflicts

[Jeff Studer]

I downloaded lspci and ran it on my embedded system,
here is the results:

# lspci -vv
00:03.0 Class 0000: 16b2:5100 (prog-if 02)
        Control: I/O- Mem- BusMaster- SpecCycle-
MemWINV- VGASnoop- ParErr- Step
ping- SERR- FastB2B-
        Status: Cap- 66Mhz- UDF- FastB2B- ParErr-
DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR-
        Interrupt: pin A routed to IRQ 6
        Region 0: I/O ports at <ignored> [disabled]
[size=65]
        Region 1: I/O ports at <ignored> [disabled]
[size=129]

00:04.0 Class 0200: 10b7:9055 (rev 30)
        Subsystem: 10b7:9055
        Control: I/O+ Mem+ BusMaster+ SpecCycle-
MemWINV- VGASnoop- ParErr- Step
ping- SERR- FastB2B-
        Status: Cap+ 66Mhz- UDF- FastB2B- ParErr-
DEVSEL=medium >TAbort- <TAbort
- <MAbort- >SERR- <PERR-
        Latency: 0 (2500ns min, 2500ns max)
        Interrupt: pin A routed to IRQ 8
        Region 0: I/O ports at f4000000 [size=129]
        Region 1: Memory at f5ffff80 (32-bit,
non-prefetchable) [size=129]
        Expansion ROM at <unassigned> [disabled]
[size=128K]
        Capabilities: [dc] Power Management version 1
                Flags: PMEClk- DSI- D1+ D2+
AuxCurrent=0mA PME(D0-,D1+,D2+,D3hot
+,D3cold-)
                Status: D0 PME-Enable- DSel=0 DScale=0
PME-

# lspci -xxx
00:03.0 Class 0000: 16b2:5100
00: b2 16 00 51 00 00 00 00 00 02 00 00 00 40 00 00
10: 41 ff ff f5 81 ff ff f5 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 06 01 0a 0a
40: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
50: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
60: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
70: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
80: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
90: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
a0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
b0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
c0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
d0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
e0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
f0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff

00:04.0 Class 0200: 10b7:9055 (rev 30)
00: b7 10 55 90 07 00 10 02 30 00 00 02 00 00 00 00
10: 01 00 00 f4 80 ff ff f5 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 b7 10 55 90
30: 00 00 00 00 dc 00 00 00 00 00 00 00 08 01 0a 0a
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 01 00 01 76
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

The arch/ppc/kernel/qspan_pci.c file doesn't have a
debug macro, and its basically totally rewritten anyway,
so I can't provide any information that way.  I'll gladly
add any printks for various pieces of data in whatever
functions needed to provide any information that would
help.

The backplane has 2 PCI slots. In the first slot is our
PCI card, in the 2nd slot is the 3c905B-TXNM 3COM
card.  If I can provide any more information that could
help, please let me know.


Peter Desnoyers wrote:

> Two pieces of information that would be useful for anyone to debug this:
>
> 1. lspci -vv and lspci -xx output
> 2. console output of boot with DEBUG turned on in the qspan file.  (not
> the bootloader one, but kernel/qspan_pci.c or whatever)
>
> Send them to the list and a few of us might have some ideas.
>
> If all else fails, do what I did - call the Tundra people for help.
> (although in my case it was a hardware issue. They were pretty helpful,
> though.)
>
> Jeff Studer wrote:
> >
> > I am working with a linux 2.4.2 kernel in an MPC860 custom board using a
> > Tundra QSPAN2 PCI controller.  I have set up configuration space and can
> > successfully probe for PCI cards. When I load PCI drivers, I get into
> > trouble.
> >
> > When I insmod the 3c59x.o module, I can access I/O space, but my MAC
> > address is read as 00:00:00:00:00:00.  I can't find manuals for this
> > card to verify what is going wrong, but I can see data in the I/O space
> > when I use our ADS board and the mpc8bug program.
> >
> > When I load another driver specific to this project, therefore the need
> > for PCI at all, I get resource conflicts.  The 3c59x.o driver actually
> > uses the value I set bus->resource->start to,  whereas the other PCI
> > driver uses 0.
> >
> > I've tried various setups for the resource.  I have set
> > bus->resource->start to the address in the QSPAN2's Slave0 address
> > setting. And I also tried do this and using the arch/ppc/mbxboot/pci.c's
> > functions: probe_addresses and map_pci_addrs.
> >
> > I really wish there was a detailed HOWTO for all this, but in the
> > meantime, does anyone have any idea what are the proper steps to
> > complete this PCI support and/or what I may be doing wrong?

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: PCI QSPAN2 resource conflicts

[Peter Desnoyers]

An aside - when I was bringing up PCI on our system, I used a
tulip-based card (Staples has one from Linksys for $20) because the 3com
vortex driver had some issues with PPC or vice versa.  Or something like
that.  (I don't remember the details, and it could have been pilot
error)

Walking through your lspci output:

> # lspci -xxx
> 00:03.0 Class 0000: 16b2:5100
> 00: b2 16 00 51 00 00 00 00 00 02 00 00 00 40 00 00
> 10: 41 ff ff f5 81 ff ff f5 00 00 00 00 00 00 00 00
> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 30: 00 00 00 00 00 00 00 00 00 00 00 00 06 01 0a 0a

vendor = 0x16b2, device = 0x5100   - I assume it's your device
status = 0x0000  weird - I would expect at least 0x0200 (devsel=medium)
- 0s in the 0x0600 mask is devsel=fast
command = 0x0000 weirder - no bus master enable, no I/O access enable,
no memory access enable.  I.e. it does nothing except suck electricity
and emit heat.

base reg 0: f5ffff41 = i/o, addr=f5ffff40, size=64
base reg 1: f5ffff81 = i/o, addr=f5ffff80, size=128

I assume you've set the QSpan up correctly with (in your case) an I/O
target image of size 0x01000000 at 0xf5000000 and a memory target image
of the same size at 0xf4000000.

I'm not sure why lspci -vv is displaying the region size incorrectly as
65 and 129, unless you're supposed to know that .

Now on to the 3Com device:

> 00:04.0 Class 0200: 10b7:9055 (rev 30)
> 00: b7 10 55 90 07 00 10 02 30 00 00 02 00 00 00 00
> 10: 01 00 00 f4 80 ff ff f5 00 00 00 00 00 00 00 00
> 20: 00 00 00 00 00 00 00 00 00 00 00 00 b7 10 55 90
> 30: 00 00 00 00 dc 00 00 00 00 00 00 00 08 01 0a 0a

status = 0x0210 - devsel = medium
command = 0x0007 - master, i/o, memory all enabled

Base registers don't make sense compared to the previous chip:

br0 = f4000001, i/o at 0xf4000000, size 128 (weird - assigned from
bottom of range here, not top)
br1 = f5ffff80, memory at 0xf5ffff80 size 128

You can't have it both ways - 0xf5ffffxx is either I/O or memory, but
not both.  I'm getting a feeling that the values in the base registers
of your device are completely fictitious, and might not come from the
PCI enumeration code.  Or you're doing the PCI enumeration in a
bootloader and doing it wrong...

Note also that the memory space at f5ffff80 totally overlaps with the
first I/O region of your device, besides being the wrong type.  But the
conflict shouldn't matter, since your device is completely turned off,
and isn't going to do anything.  So you shouldn't be loading a driver
for your device, because it isn't there, and if you don't you won't get
any resource conflicts.

Looks like you've got a hardware issue in your device, compounded by the
fact that the PCI bus enumeration code is actually assigning addresses
to your device even though it's disabled.  (although it's probably not
the fault of that code, as I assume it's writing something to the
command register and it's just not sticking)

You're not loading an FPGA after the OS is up and then poking in these
values by hand, are you?

--
.....................................................................
 Peter Desnoyers            (781) 457-1165   pdesnoyers@chinook.com
 Chinook Communications     (617) 661-1979   pjd@fred.cambridge.ma.us
 100 Hayden Ave, Lexington MA 02421

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: PCI QSPAN2 resource conflicts

[Peter Desnoyers]

I wrote:
>
> br0 = f4000001, i/o at 0xf4000000, size 128 (weird - assigned from
> bottom of range here, not top)
> br1 = f5ffff80, memory at 0xf5ffff80 size 128
>
> You can't have it both ways - 0xf5ffffxx is either I/O or memory, but
> not both.

I'm forgetting - the values in the registers are actual PCI I/O and
memory addresses, so there's no issue if they overlap.  (the physical
addresses mapped to them on the CPU side can't overlap, though) And when
I was talking about target image programming in the QSpan, I should have
been referring to the bus addresses, not the physical addresses.

However, I still get the feeling that there's something badly wrong with
the way you've got your base registers mapped on the two devices, plus
your device is just plain turned off.

--
.....................................................................
 Peter Desnoyers            (781) 457-1165   pdesnoyers@chinook.com
 Chinook Communications     (617) 661-1979   pjd@fred.cambridge.ma.us
 100 Hayden Ave, Lexington MA 02421

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: PCI QSPAN2 resource conflicts

[Jeff Studer]

Basically, my problem is setting up resources, or at least for our PCI
card, thats the case.  I can't get past resource allocation for it.

My current understanding is that we pick a safe base address to use to
access PCI I/O and Memory space.  We also set up the amount of PCI I/O and
Memory space we want to be able to access.  We set this address up in the
Slave0 -- in our case, we only get Slave0 -- and this size, and do the same
with the chipselect for the Slave0 image.  We then use these same values to
set up the PCI system's bus resources.  How much space is recommended?  I
picked 0xf4000000 and 64Mbs of PCI I/O / Memory space.

I also modified the inb/outb/inw/outw/inl/outl to set the QSpan2 Slave0 for
I/O space access, and readb/writeb/readw/writew/readl/writel to set Slave0
for PCI Memory access, as our board can't use Slave1 at all.

As far as setting up the device resources, I'm not entirely clear on what I
should do, if anything, in this regard.  I wasn't sure if the rest of
Linux's PCI system would do that, given the bus resources settings I made.

I did try using the probe_addresses() and map_pci_addrs() functions from
arch/ppc/mbxboot/pci.c, but I don't clearly understand what the address
mapping function is trying to do.

Am I making things overly complicated, or overly simplified?

Peter Desnoyers wrote:

> I'm forgetting - the values in the registers are actual PCI I/O and
> memory addresses, so there's no issue if they overlap.  (the physical
> addresses mapped to them on the CPU side can't overlap, though) And when
> I was talking about target image programming in the QSpan, I should have
> been referring to the bus addresses, not the physical addresses.

Me, too.  Is there a standardized algorithm to do this?

>
> However, I still get the feeling that there's something badly wrong with
> the way you've got your base registers mapped on the two devices, plus
> your device is just plain turned off.

Thanks again for taking the time to help me.

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/