750 and Discovery II coherency with PCI....

750 and Discovery II coherency with PCI....

[Steven J. Hill]

Greetings.

I am writing a device driver for a PMC card that is nothing
fancy. It is not a PCI-X card though. My Linux driver properly
detects the device, 'ioremaps' the PCI IO and MEM regions then
attempts to start writing the devices registers for setup
and configuration. I am not getting valid reads and there
appears to be some coherency issues. I have verified that no
other drivers or resources in the system are utilizing the
addresses that are returned to me from 'ioremap'. I have the
following code snippet:

#define shunt_read_8(offset)		\
	*((volatile u8 *) (shunt_dev->ctrl_base + offset))
[SNIP]
		svalue[0] = shunt_read_8(SHUNT_ADDRESS + 0x220);
		svalue[1] = shunt_read_8(SHUNT_ADDRESS + 0x221);
		svalue[2] = shunt_read_8(SHUNT_ADDRESS + 0x222);
		svalue[3] = shunt_read_8(SHUNT_ADDRESS + 0x223);

		printk("0x%02x\n", svalue[0]);
		printk("0x%02x\n", svalue[1]);
		printk("0x%02x\n", svalue[2]);
		printk("0x%02x\n", svalue[3]);

If I read the byte addresses above, which are in the remapped PCI I/O address
space and I get the following output:

0x00
0x00
0x00
0x00

which is of course, incorrect. I even tried using the 'in_8' macro functions
in 'asm-ppc/io.h'. I get the same results. Now, if I instead do the following
code snippet:

		printk("0x%02x\n", shunt_read_8(SHUNT_ADDRESS + 0x220));
		printk("0x%02x\n", shunt_read_8(SHUNT_ADDRESS + 0x220));
		printk("0x%02x\n", shunt_read_8(SHUNT_ADDRESS + 0x220));
		printk("0x%02x\n", shunt_read_8(SHUNT_ADDRESS + 0x221));
		printk("0x%02x\n", shunt_read_8(SHUNT_ADDRESS + 0x222));
		printk("0x%02x\n", shunt_read_8(SHUNT_ADDRESS + 0x223));

I now get the output of:

0x00
0x81
0x81
0x02
0x0e
0x0e

The first time I read 0x220, it is incorrect, but the following two times and
the other remaining values are now correct. Unfortunately, this device is the
only PMC device that I have to test. I do not have any other PMC cards. This
behavior exists in both the 2.4 kernel and in my 2.6 kernel. The virtual
addresses received from the 'ioremap' call are identical on both 2.4 and 2.6,
just for reference. I tried adding in memory barriers and 'eieio' calls, but
I did not expect those to make a difference...and it did not. Has anyone ever
seen similar behavior? Preferably I would like to hear from people working
with Discovery II parts. I have documentation under NDA, but I am still working
my way through it. Thanks in advance.

-Steve

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: 750 and Discovery II coherency with PCI....

[Brian Waite]

Steve,
I have been using the disco II now for quite a while I have been able
to make 4 different PCI cards work without many problems. Can you look
at your memory windows and make sure the mappings are set up
correctly? Is this a single or a dual processor? I would start by
decoding the PCI memory windows that sounds like where you might be
getting into trouble. Do you have a PCI bus analyzer handy to make
sure you are putting the right addresses on the bus?
Thanks
Brian


On Mon, 02 Aug 2004 22:26:36 -0500, Steven J. Hill
<sjhill@realitydiluted.com> wrote:
>
> Greetings.
>
> I am writing a device driver for a PMC card that is nothing
> fancy. It is not a PCI-X card though. My Linux driver properly
> detects the device, 'ioremaps' the PCI IO and MEM regions then
> attempts to start writing the devices registers for setup
> and configuration. I am not getting valid reads and there
> appears to be some coherency issues. I have verified that no
> other drivers or resources in the system are utilizing the
> addresses that are returned to me from 'ioremap'. I have the
> following code snippet:
>
> #define shunt_read_8(offset)            \
>         *((volatile u8 *) (shunt_dev->ctrl_base + offset))
> [SNIP]
>                 svalue[0] = shunt_read_8(SHUNT_ADDRESS + 0x220);
>                 svalue[1] = shunt_read_8(SHUNT_ADDRESS + 0x221);
>                 svalue[2] = shunt_read_8(SHUNT_ADDRESS + 0x222);
>                 svalue[3] = shunt_read_8(SHUNT_ADDRESS + 0x223);
>
>                 printk("0x%02x\n", svalue[0]);
>                 printk("0x%02x\n", svalue[1]);
>                 printk("0x%02x\n", svalue[2]);
>                 printk("0x%02x\n", svalue[3]);
>
> If I read the byte addresses above, which are in the remapped PCI I/O address
> space and I get the following output:
>
> 0x00
> 0x00
> 0x00
> 0x00
>
> which is of course, incorrect. I even tried using the 'in_8' macro functions
> in 'asm-ppc/io.h'. I get the same results. Now, if I instead do the following
> code snippet:
>
>                 printk("0x%02x\n", shunt_read_8(SHUNT_ADDRESS + 0x220));
>                 printk("0x%02x\n", shunt_read_8(SHUNT_ADDRESS + 0x220));
>                 printk("0x%02x\n", shunt_read_8(SHUNT_ADDRESS + 0x220));
>                 printk("0x%02x\n", shunt_read_8(SHUNT_ADDRESS + 0x221));
>                 printk("0x%02x\n", shunt_read_8(SHUNT_ADDRESS + 0x222));
>                 printk("0x%02x\n", shunt_read_8(SHUNT_ADDRESS + 0x223));
>
> I now get the output of:
>
> 0x00
> 0x81
> 0x81
> 0x02
> 0x0e
> 0x0e
>
> The first time I read 0x220, it is incorrect, but the following two times and
> the other remaining values are now correct. Unfortunately, this device is the
> only PMC device that I have to test. I do not have any other PMC cards. This
> behavior exists in both the 2.4 kernel and in my 2.6 kernel. The virtual
> addresses received from the 'ioremap' call are identical on both 2.4 and 2.6,
> just for reference. I tried adding in memory barriers and 'eieio' calls, but
> I did not expect those to make a difference...and it did not. Has anyone ever
> seen similar behavior? Preferably I would like to hear from people working
> with Discovery II parts. I have documentation under NDA, but I am still working
> my way through it. Thanks in advance.
>
> -Steve
>
>
>

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: 750 and Discovery II coherency with PCI....

[Mark A. Greer]

Steve,

You leave out a lot of pertinent information that would help us help
you.  For example, what board are you using?  Is it a board with a known
good Linux board port?  What kernel base are you using (the *exact* tree
and version of that tree; there are several 2.4 trees)?  Are you sure
that the PCI I/O & MEM space allocations are correct (e.g., did you run
pci_auto)?

Somc comments follow...

Steven J. Hill wrote:

>
> Greetings.
>
> I am writing a device driver for a PMC card that is nothing
> fancy. It is not a PCI-X card though. My Linux driver properly
> detects the device, 'ioremaps' the PCI IO


PCI I/O space is already "ioremap'd" for you, don't do it again.
Assuming your board code in arch/ppc/platforms is correct and you are
using the correct PCI I/O space access routines (e.g., inb/outb and
friends), it will just work.  Make sure those assumptions are valid.

There are many, many linux drivers available for you to look at, please
use them for examples on how to access PCI I/O and MEM space.  Also,
there is lots of good info in .../Documentation/*.  It will be well
worth your time to read through it.

> and MEM regions then
> attempts to start writing the devices registers for setup
> and configuration. I am not getting valid reads and there
> appears to be some coherency issues.


What makes you think there are coherency issues?

> I have verified that no
> other drivers or resources in the system are utilizing the
> addresses that are returned to me from 'ioremap'.


Of course not, ioremap returns the virtual address from the virt->CPU
phys mapping it just made.  That's always going to look fine.  Its the
CPU physical, PCI I/O, and PCI MEM addresses that you need to make sure
don't overlap.

> I have the
> following code snippet:
>
> #define shunt_read_8(offset)        \
>     *((volatile u8 *) (shunt_dev->ctrl_base + offset))


Don't do that.  Use the proper access routines that are already
available to you.

Also, I see nothing remotely related to cache coherency here.  You only
talk about accessing registers in ioremap'd areas.  Caches are not
involved nor are you looking at areas in memory where your device DMA'd
data into or out of.

Mark


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: 750 and Discovery II coherency with PCI....

[Benjamin Herrenschmidt]

On Tue, 2004-08-03 at 13:26, Steven J. Hill wrote:
> Greetings.
>
> I am writing a device driver for a PMC card that is nothing
> fancy. It is not a PCI-X card though. My Linux driver properly
> detects the device, 'ioremaps' the PCI IO and MEM regions then
> attempts to start writing the devices registers for setup
> and configuration. I am not getting valid reads and there
> appears to be some coherency issues. I have verified that no
> other drivers or resources in the system are utilizing the
> addresses that are returned to me from 'ioremap'. I have the
> following code snippet:

There is no coherency issues with non cacheable (ioremap'ed)
access of that kind... Note that if you are dealing with IO space,
then you should use inX/outX accessors and not ioremap.

Ben.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/