G4 + Linux + PCI device + x86 driver = 0

G4 + Linux + PCI device + x86 driver = 0

[David W. Patmore]

Hi,

I'm trying to port a device driver from working x86 code over to a G4 (early
350MHz) running LinuxPPC.  The driver will load up, but the memory access
isn't working.

Linux version: 2.2.6 Sat Oct 30 1999
gcc version: egcs-2.91.66

warning sign:  I get warning message "pcidrv.o was compiled for kernel
version
2.2.6-15apmac, while this kernel is version 2.2.6".  I force the driver to
load
using "insmod -f pcidrv.o".

The original x86 (Redhat 6.0) driver uses virt_to_bus() to get the address
to write to.  In LinuxPPC, that function is not available (not in name
table).  I guess that I'm supposed to use ioremap(), but that doesn't seem
to do it for me either.

Code snip:
	ul_reg_addr = p_dev->base_address[0];

	pul_remapped =ioremap( ul_reg_addr, 32 );

	printk( "pcidrv: base addr %08X \n", ul_reg_addr );
	printk( "pcidrv: pul_remapped %08X \n", pul_remapped );
	printk( "pcidrv: remapped data: %08X \n", *pul_remapped );

	iounmap( pul_remapped );

The outcome of running this code is:  pci base address (0x80890000),
remapped (0xC8271000), data (0xFFFFFFFF).  The data is a status word which I
expect to be not
all "F"s.  Note that iounmap is "TBD" in the source, so each time the code
runs per
reboot, iomap returns a new address.

If anyone has some advice, I'd be very pleased to hear it.  I need to go on
the G4, because that is what I have on hand.

Thanks very much,

David Patmore


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: G4 + Linux + PCI device + x86 driver = 0

[Takashi Oe]

On Sat, 13 Nov 1999, David W. Patmore wrote:

[...]
> The original x86 (Redhat 6.0) driver uses virt_to_bus() to get the address
> to write to.  In LinuxPPC, that function is not available (not in name
> table).  I guess that I'm supposed to use ioremap(), but that doesn't seem
> to do it for me either.
> 
> Code snip:
> 	ul_reg_addr = p_dev->base_address[0];
> 
> 	pul_remapped =ioremap( ul_reg_addr, 32 );
> 
> 	printk( "pcidrv: base addr %08X \n", ul_reg_addr );
> 	printk( "pcidrv: pul_remapped %08X \n", pul_remapped );
> 	printk( "pcidrv: remapped data: %08X \n", *pul_remapped );
> 
> 	iounmap( pul_remapped );
> 
> The outcome of running this code is:  pci base address (0x80890000),
> remapped (0xC8271000), data (0xFFFFFFFF).  The data is a status word which I
> expect to be not
> all "F"s.  Note that iounmap is "TBD" in the source, so each time the code

Most likely, you need to enable I/O or/and memory access by hand.  OF
probably didn't enable it automatically.  For example,

	(void) pci_read_config_word(p_dev, PCI_COMMAND, &command);
	if (!(command & PCI_COMMAND_IO)) {
		command |= PCI_COMMAND_IO;
		(void) pci_write_config_word(p_dev, PCI_COMMAND, command);
		(void) pci_read_config_word(p_dev, PCI_COMMAND, &command);
		if (!(command & PCI_COMMAND_IO)) {
			printk(KERN_DEBUG "IO access enable failed\n");
			return 0;
		}
	}
	if (!(command & PCI_COMMAND_MEMORY)) {
		command |= PCI_COMMAND_MEMORY;
		(void) pci_write_config_word(p_dev, PCI_COMMAND, command);
		(void) pci_read_config_word(p_dev, PCI_COMMAND, &command);
		if (!(command & PCI_COMMAND_MEMORY)) {
			printk(KERN_DEBUG "Memory access enable failed\n");
			return 0;
		}
	}

This would make *pul_remapped return something different, I'd think.  See
other ppc drivers for more details.  You may need additional PCI fixups.


Takashi Oe


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: G4 + Linux + PCI device + x86 driver = 0

[Paul Mackerras]

On Sun, 14 Nov 1999, David W. Patmore wrote:

> The original x86 (Redhat 6.0) driver uses virt_to_bus() to get the address
> to write to.  In LinuxPPC, that function is not available (not in name
> table).  I guess that I'm supposed to use ioremap(), but that doesn't seem
> to do it for me either.

virt_to_bus is an extern inline function in <asm/io.h>, so you should be
able to use it.  What undefined symbols do you get when you try?  (Note
that you have to compile with at least -O2 otherwise gcc doesn't do inline
functions.)

If you want to access the device register within your driver, ioremap is
what you want, though, rather than virt_to_bus.  You would use virt_to_bus
to translate the virtual address of some kernel memory into a bus address
to program into a DMA controller.

> The outcome of running this code is:  pci base address (0x80890000),
> remapped (0xC8271000), data (0xFFFFFFFF).  The data is a status word which I
> expect to be not
> all "F"s.

Most likely the device doesn't have PCI memory space accesses enabled, as
Ryuichi pointed out.

Paul.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: G4 + Linux + PCI device + x86 driver = 0

[Adrian Cox]

"David W. Patmore" wrote:
[snip]
> The original x86 (Redhat 6.0) driver uses virt_to_bus() to get the address
> to write to.  In LinuxPPC, that function is not available (not in name
> table).  I guess that I'm supposed to use ioremap(), but that doesn't seem
> to do it for me either.

You should include <asm/io.h>. But virt_to_bus(), phys_to_virt(), and
their relatives, only apply to RAM.

> Code snip:
>         ul_reg_addr = p_dev->base_address[0];
> 
>         pul_remapped =ioremap( ul_reg_addr, 32 );
> 
>         printk( "pcidrv: base addr %08X \n", ul_reg_addr );
>         printk( "pcidrv: pul_remapped %08X \n", pul_remapped );
>         printk( "pcidrv: remapped data: %08X \n", *pul_remapped );
> 
>         iounmap( pul_remapped );

Note that as a general principal of cross platform Linux programming,
the thing returned by ioremap is not a true pointer, and should only be
used with readl(), writel(), etc.

These are not your problem - the problem is most likely the device not
being enabled by the boot firmware. This may mean that 
(1) The PCI fixup code is broken on the G4s.
(2) The device has unusual startup requirements.

- Adrian Cox, AG Electronics

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Update: RE: G4 + Linux + PCI device + x86 driver = 0

[David W. Patmore]

I posted a question about getting a PCI device working on LinuxPPC (G4!).
Thanks to those who responded; your assistance made a big difference.  I
finally finished my project, and wanted to post a few dribs that I had to
deal with (that were different from the Linux x86 driver).

1.  I had to enable memory writes (PCI_COMMAND_MEMORY), which was somehow
handled automataically on x86, but not here.
    {
    pci_read_config_word( pDev, PCI_COMMAND, &value );
    value |= PCI_COMMAND_MEMORY;
    pci_write_config_word( pDev, PCI_COMMAND, value );
    }
    For good measure, I also set PCI_COMMAND_MASTER, though I also call
pci_set_master(), which covers the issue.

2.  I had to byte-swap all pointers that I supply to my device.  For
portability, I made a set of macros, "SYS_TO_LE() and LE_TO_SYS(), etc. so
that both endian architectures could work out.

3.  I had to byte-swap all data that I supply to my device, except the data
that is handled as a byte stream.  This involved major discussions with my
colleagues, because of the need to consider Big Endian, Little Endian, and
Network Byte Order.

Now I'm going to put my feet up and take an asprin.  Thanks again for your
help.

David Patmore
www.BlueSteelNet.com
> -----Original Message-----
> From: apc@multi59.netcomi.com [mailto:apc@multi59.netcomi.com]On Behalf
> Of Adrian Cox
> Sent: Monday, November 15, 1999 1:34 AM
> To: David W. Patmore
> Cc: linuxppc-dev@lists.linuxppc.org
> Subject: Re: G4 + Linux + PCI device + x86 driver = 0
>
>
> "David W. Patmore" wrote:
> [snip]
> > The original x86 (Redhat 6.0) driver uses virt_to_bus() to get
> the address
> > to write to.  In LinuxPPC, that function is not available (not in name
> > table).  I guess that I'm supposed to use ioremap(), but that
> doesn't seem
> > to do it for me either.
>
> You should include <asm/io.h>. But virt_to_bus(), phys_to_virt(), and
> their relatives, only apply to RAM.
>
> > Code snip:
> >         ul_reg_addr = p_dev->base_address[0];
> >
> >         pul_remapped =ioremap( ul_reg_addr, 32 );
> >
> >         printk( "pcidrv: base addr %08X \n", ul_reg_addr );
> >         printk( "pcidrv: pul_remapped %08X \n", pul_remapped );
> >         printk( "pcidrv: remapped data: %08X \n", *pul_remapped );
> >
> >         iounmap( pul_remapped );
>
> Note that as a general principal of cross platform Linux programming,
> the thing returned by ioremap is not a true pointer, and should only be
> used with readl(), writel(), etc.
>
> These are not your problem - the problem is most likely the device not
> being enabled by the boot firmware. This may mean that
> (1) The PCI fixup code is broken on the G4s.
> (2) The device has unusual startup requirements.
>
> - Adrian Cox, AG Electronics
>


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Update: RE: G4 + Linux + PCI device + x86 driver = 0

[David Edelsohn]

>>>>> "David W Patmore" writes:

avid> 1.  I had to enable memory writes (PCI_COMMAND_MEMORY), which was somehow
David> handled automataically on x86, but not here.
David> {
David> pci_read_config_word( pDev, PCI_COMMAND, &value );
David> value |= PCI_COMMAND_MEMORY;
David> pci_write_config_word( pDev, PCI_COMMAND, value );
David> }
David> For good measure, I also set PCI_COMMAND_MASTER, though I also call
David> pci_set_master(), which covers the issue.

	It is up to the device driver or the card to set all of these bits
if necessary.

David> 2.  I had to byte-swap all pointers that I supply to my device.  For
David> portability, I made a set of macros, "SYS_TO_LE() and LE_TO_SYS(), etc. so
David> that both endian architectures could work out.

	Linux already provides macros like le16_to_cpu(), etc. for this. 

David

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: endian-swapping

[Hollis R Blanchard]

On Fri, 3 Dec 1999, David Edelsohn wrote:
> 
> 	Linux already provides macros like le16_to_cpu(), etc. for this. 

For the sake of someone (like me) who doesn't know how to use these, can you
give an example? Just a snippet like

#include <endian.h>

#ifdef __BIG_ENDIAN
    cpu_to_le16(blah);
#endif

I don't know which headers are supposed to be used for this. endian.h, for
example, does a #define __BIG_ENDIAN 4321, so __BIG_ENDIAN should *always* be
defined... so that doesn't seem like the right header to include... but lots
of drivers do that exact #ifdef test.

Also, the macros are in which header? I'm having trouble finding them too...

-Hollis


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: endian-swapping

[David Edelsohn]

>>>>> Hollis R Blanchard writes:

Hollis> On Fri, 3 Dec 1999, David Edelsohn wrote:
>> 
>> Linux already provides macros like le16_to_cpu(), etc. for this. 

Hollis> For the sake of someone (like me) who doesn't know how to use these, can you
Hollis> give an example?

	See drivers/net/pcnet32.c for one example of its use.  There are
no #ifdef's like NetBSD uses.  le16_to_cpu() is a no-op on LE machines, so
one uses it directly like 

short x = le16_to_cpu(dev_flag);

Hollis> Also, the macros are in which header? I'm having trouble finding them too...

	The macros seem to be included via <asm/bitops.h> which then
include <linux/byteorder/big_endian.h> (for example).

David

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: endian-swapping

[Gabriel Paubert]

On Fri, 3 Dec 1999, Hollis R Blanchard wrote:

> 
> On Fri, 3 Dec 1999, David Edelsohn wrote:
> > 
> > 	Linux already provides macros like le16_to_cpu(), etc. for this. 
> 
> For the sake of someone (like me) who doesn't know how to use these, can you
> give an example? Just a snippet like
> 
> #include <endian.h>
> 
> #ifdef __BIG_ENDIAN
>     cpu_to_le16(blah);
> #endif
> 
> I don't know which headers are supposed to be used for this. endian.h, for
> example, does a #define __BIG_ENDIAN 4321, so __BIG_ENDIAN should *always* be
> defined... so that doesn't seem like the right header to include... but lots
> of drivers do that exact #ifdef test.
> 
> Also, the macros are in which header? I'm having trouble finding them too...

#include <asm/byteorder.h> 

Several other header files include it but you should not rely on it since
for example asm/bitops.h includes it on moost architectures but not on
Intel. 

	Gabriel.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Update: RE: G4 + Linux + PCI device + x86 driver = 0

[Michel Lanners]

Hi David,

On   3 Dec, this message from David W. Patmore echoed through cyberspace:
> I posted a question about getting a PCI device working on LinuxPPC (G4!).
> Thanks to those who responded; your assistance made a big difference.  I
> finally finished my project, and wanted to post a few dribs that I had to
> deal with (that were different from the Linux x86 driver).

;-)

> 1.  I had to enable memory writes (PCI_COMMAND_MEMORY), which was somehow
> handled automataically on x86, but not here.

As far as I understand, i86 BIOS does this by default. However,
OpenFirmware does not. I've heard of numerous people run into this;
I've put together a kernel patch that adds a PCI fixup function on
PowerMacs, which enables memory and IO access if necessary.

>     For good measure, I also set PCI_COMMAND_MASTER, though I also call
> pci_set_master(), which covers the issue.

This, however, should have been enabled by OF. Are you sure setting
master 'sticks'? There is no other way to detect whether a deivce
supports bus mastering other then setting PCI_COMMAND_MASTER, and then
checking it really got set.

> 2.  I had to byte-swap all pointers that I supply to my device.  For
> portability, I made a set of macros, "SYS_TO_LE() and LE_TO_SYS(), etc. so
> that both endian architectures could work out.

Why do you need to pass pointers to your device? For DMA? If so, you
should not only byteswap your pointers, but also convert them between
CPU address space and PCI bus address space. Some PCI host bridges do
address transaltion, so you're not guaranteed both sides see the same
address. I use:

pci_device->dma_target = virt_to_bus(my_buffer);

I'ts defined in asm/io.h

> 3.  I had to byte-swap all data that I supply to my device, except the data
> that is handled as a byte stream.  This involved major discussions with my
> colleagues, because of the need to consider Big Endian, Little Endian, and
> Network Byte Order.

David pointed out le16_to_cpu() et al; personally, I use in_/out_le32
et al for access to PCI memory space like this:

out_le32 (&device->register, val);

These functions are available under both big- and little-endian
systems, and always do 'the right thing'. They are defined in
asm-ppc/io.h, and include I/O barrier instructions (preventing
reordering on the CPU).

If you don't want I/O barriers, there are also ld_/st_le_32 et al,
which are defined in asm-ppc/byteorder.h (included from asm/io.h).

Michel

-------------------------------------------------------------------------
Michel Lanners                 |  " Read Philosophy.  Study Art.
23, Rue Paul Henkes            |    Ask Questions.  Make Mistakes.
L-1710 Luxembourg              |
email   mlan@cpu.lu            |
http://www.cpu.lu/~mlan        |                     Learn Always. "


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Update: RE: G4 + Linux + PCI device + x86 driver = 0

[Geert Uytterhoeven]

On Sat, 4 Dec 1999, Michel Lanners wrote:
> > 3.  I had to byte-swap all data that I supply to my device, except the data
> > that is handled as a byte stream.  This involved major discussions with my
> > colleagues, because of the need to consider Big Endian, Little Endian, and
> > Network Byte Order.
> 
> David pointed out le16_to_cpu() et al; personally, I use in_/out_le32
> et al for access to PCI memory space like this:
> 
> out_le32 (&device->register, val);

Which is wrong! Portable PCI memory space accesses must be done using
{read,write}[bwl]()!

See linux/Documentation/IO-mapping.txt 

Gr{oetje,eeting}s,
--
Geert Uytterhoeven -- Linux/{m68k~Amiga,PPC~CHRP} -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Update: RE: G4 + Linux + PCI device + x86 driver = 0

[Michel Lanners]

Hi Geert,

On   5 Dec, this message from Geert Uytterhoeven echoed through cyberspace:
>> David pointed out le16_to_cpu() et al; personally, I use in_/out_le32
>> et al for access to PCI memory space like this:
>> 
>> out_le32 (&device->register, val);
> 
> Which is wrong! Portable PCI memory space accesses must be done using
> {read,write}[bwl]()!

Ooopppss.... Thanks for letting me know. I guess it doesn't make much
difference, because the code I use it in is purely PowerMac, but
anyway... I'll go read...

> See linux/Documentation/IO-mapping.txt 

....that ;-)

Michel

-------------------------------------------------------------------------
Michel Lanners                 |  " Read Philosophy.  Study Art.
23, Rue Paul Henkes            |    Ask Questions.  Make Mistakes.
L-1710 Luxembourg              |
email   mlan@cpu.lu            |
http://www.cpu.lu/~mlan        |                     Learn Always. "


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/