Linux running on a PCI Option device?

Linux running on a PCI Option device?

[Jon Ringle]

Hello,

I am working on a new board that will have Linux running on an xscale 
processor. This board will be a PCI Option device. I currently have a IXDP465 
eval board which has a PCI Option connector that I will use for prototyping. 
>From what I can tell so far, Linux wants to scan the PCI bus for devices as 
if it is the PCI host. Is there any provision in Linux so that it can take on 
the role of a PCI option rather than a PCI host?

Jon

PS. Please CC me on replies.

Re: Linux running on a PCI Option device?

[Greg Ungerer]

Hi Jon,

Jon Ringle wrote:
> I am working on a new board that will have Linux running on an xscale 
> processor. This board will be a PCI Option device. I currently have a IXDP465 
> eval board which has a PCI Option connector that I will use for prototyping. 
> From what I can tell so far, Linux wants to scan the PCI bus for devices as 
> if it is the PCI host. Is there any provision in Linux so that it can take on 
> the role of a PCI option rather than a PCI host?

Have a look at the code in arch/arm/mach-ixp4xx/common-pci.c, in
the function ixp4xx_pci_preinit().

It does a check on whether the PCI bus is configured as HOST or not.
I don't know if that code support is enough for it all to work right
though (I certainly haven't tried it on either the 425 or 465...)

Regards
Greg



------------------------------------------------------------------------
Greg Ungerer  --  Chief Software Dude       EMAIL:     gerg@snapgear.com
SnapGear -- a CyberGuard Company            PHONE:       +61 7 3435 2888
825 Stanley St,                             FAX:         +61 7 3891 3630
Woolloongabba, QLD, 4102, Australia         WEB: http://www.SnapGear.com

Re: Linux running on a PCI Option device?

[Jon Ringle]

On Thursday 09 February 2006 01:43 am, Greg Ungerer wrote:
> Hi Jon,
>
> Jon Ringle wrote:
> > I am working on a new board that will have Linux running on an xscale
> > processor. This board will be a PCI Option device. I currently have a
> > IXDP465 eval board which has a PCI Option connector that I will use for
> > prototyping. From what I can tell so far, Linux wants to scan the PCI bus
> > for devices as if it is the PCI host. Is there any provision in Linux so
> > that it can take on the role of a PCI option rather than a PCI host?
>
> Have a look at the code in arch/arm/mach-ixp4xx/common-pci.c, in
> the function ixp4xx_pci_preinit().
>
> It does a check on whether the PCI bus is configured as HOST or not.
> I don't know if that code support is enough for it all to work right
> though (I certainly haven't tried it on either the 425 or 465...)

Something that I don't quite understand is how I'm supposed to make vendor Id 
information available to the PCI host. Any ideas there?

TIA,

Jon

Re: Linux running on a PCI Option device?

[Greg Ungerer]

Hi Jon,

Jon Ringle wrote:
> On Thursday 09 February 2006 01:43 am, Greg Ungerer wrote:
>>Jon Ringle wrote:
>>>I am working on a new board that will have Linux running on an xscale
>>>processor. This board will be a PCI Option device. I currently have a
>>>IXDP465 eval board which has a PCI Option connector that I will use for
>>>prototyping. From what I can tell so far, Linux wants to scan the PCI bus
>>>for devices as if it is the PCI host. Is there any provision in Linux so
>>>that it can take on the role of a PCI option rather than a PCI host?
>>
>>Have a look at the code in arch/arm/mach-ixp4xx/common-pci.c, in
>>the function ixp4xx_pci_preinit().
>>
>>It does a check on whether the PCI bus is configured as HOST or not.
>>I don't know if that code support is enough for it all to work right
>>though (I certainly haven't tried it on either the 425 or 465...)
> 
> 
> Something that I don't quite understand is how I'm supposed to make vendor Id 
> information available to the PCI host. Any ideas there?

No, sorry, no idea. I haven't looked at it in that much detail.

Regards
Greg



------------------------------------------------------------------------
Greg Ungerer  --  Chief Software Dude       EMAIL:     gerg@snapgear.com
SnapGear -- a CyberGuard Company            PHONE:       +61 7 3435 2888
825 Stanley St,                             FAX:         +61 7 3891 3630
Woolloongabba, QLD, 4102, Australia         WEB: http://www.SnapGear.com

Re: Linux running on a PCI Option device?

[Jon Ringle]

On Thursday 09 February 2006 01:43 am, Greg Ungerer wrote:
> Hi Jon,
>
> Jon Ringle wrote:
> > I am working on a new board that will have Linux running on an xscale
> > processor. This board will be a PCI Option device. I currently have a
> > IXDP465 eval board which has a PCI Option connector that I will use for
> > prototyping. From what I can tell so far, Linux wants to scan the PCI bus
> > for devices as if it is the PCI host. Is there any provision in Linux so
> > that it can take on the role of a PCI option rather than a PCI host?
>
> Have a look at the code in arch/arm/mach-ixp4xx/common-pci.c, in
> the function ixp4xx_pci_preinit().
>
> It does a check on whether the PCI bus is configured as HOST or not.
> I don't know if that code support is enough for it all to work right
> though (I certainly haven't tried it on either the 425 or 465...)

When I have the IXDP465 in PCI Option mode, Linux still writes to pci 
configuration space which confuses the heck out of the PCI Host (Windows 
2003). What do I need to do in order to have Linux work as a PCI option but 
still not mess with the pci configuration, and leave that task to the PCI 
Host?

Jon

Re: Linux running on a PCI Option device?

[linux-os (Dick Johnson)]

On Tue, 28 Feb 2006, Jon Ringle wrote:

> On Thursday 09 February 2006 01:43 am, Greg Ungerer wrote:
>> Hi Jon,
>>
>> Jon Ringle wrote:
>>> I am working on a new board that will have Linux running on an xscale
>>> processor. This board will be a PCI Option device. I currently have a
>>> IXDP465 eval board which has a PCI Option connector that I will use for
>>> prototyping. From what I can tell so far, Linux wants to scan the PCI bus
>>> for devices as if it is the PCI host. Is there any provision in Linux so
>>> that it can take on the role of a PCI option rather than a PCI host?
>>
>> Have a look at the code in arch/arm/mach-ixp4xx/common-pci.c, in
>> the function ixp4xx_pci_preinit().
>>
>> It does a check on whether the PCI bus is configured as HOST or not.
>> I don't know if that code support is enough for it all to work right
>> though (I certainly haven't tried it on either the 425 or 465...)
>
> When I have the IXDP465 in PCI Option mode, Linux still writes to pci
> configuration space which confuses the heck out of the PCI Host (Windows
> 2003). What do I need to do in order to have Linux work as a PCI option but
> still not mess with the pci configuration, and leave that task to the PCI
> Host?
>
> Jon

But anything on the PCI bus will get written, at least to find out
the length of the address space. Please read about the PCI System
Architecture. There are several writes that are mandatory. If
somebody is attempting to configure the PCI devices, the following
will occur.

(1) The BIOS will find some available address-space and put it
into any base-address register that has memory-space enabled
in the command register.

(2) The BIOS will find some available I/O space and put it into
a base-address register, too.

This all occurs long before any OS is booted. These writes
will occur. Then, the addresses may be re-written by any OS
including Linux. Additional writes will occur when the
device is enabled and when the cache-line size or the
latency timer is written.

If the device is a host, and you don't want the BIOS to
hack with it, then you need to turn off writes from the
the host-bus side. This is something your FPGA designer
needs to provide.

Usually, option boards use a PCI interface chip such as
a PLX PCI-9656BA. It would not normally BE a PCI device
on the Host side. The host CPU would communicate with it
on a local bus.


Cheers,
Dick Johnson
Penguin : Linux version 2.6.15.4 on an i686 machine (5589.54 BogoMips).
Warning : 98.36% of all statistics are fiction, book release in April.
_
\x1a\x04

****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

Re: Linux running on a PCI Option device?

[Krzysztof Halasa]

"linux-os \(Dick Johnson\)" <linux-os@analogic.com> writes:

> Usually, option boards use a PCI interface chip such as
> a PLX PCI-9656BA. It would not normally BE a PCI device
> on the Host side. The host CPU would communicate with it
> on a local bus.

Correct but IPX4xx CPU has a PCI bus interface built in (in addition
to interfaces for ROM, RAM, GPIO, Ethernet and serial).
-- 
Krzysztof Halasa

Re: Linux running on a PCI Option device?

[Jon Ringle]

On Tuesday 28 February 2006 04:13 pm, linux-os (Dick Johnson) wrote:
> On Tue, 28 Feb 2006, Jon Ringle wrote:
> > When I have the IXDP465 in PCI Option mode, Linux still writes to pci
> > configuration space which confuses the heck out of the PCI Host (Windows
> > 2003). What do I need to do in order to have Linux work as a PCI option
> > but still not mess with the pci configuration, and leave that task to the
> > PCI Host?
> >
> > Jon
>
> But anything on the PCI bus will get written, at least to find out
> the length of the address space. Please read about the PCI System
> Architecture. There are several writes that are mandatory. If
> somebody is attempting to configure the PCI devices, the following
> will occur.

Thanks for the book reference. I am reading it.

> (1) The BIOS will find some available address-space and put it
> into any base-address register that has memory-space enabled
> in the command register.
>
> (2) The BIOS will find some available I/O space and put it into
> a base-address register, too.
>
> This all occurs long before any OS is booted. These writes
> will occur.

That's all well and fine. However, I think I need to explain a bit better the 
hardware topology here.

             +--------------------+
             |     POST/BIOS      |
             |    Windows 2003    |
             |       Pentium      |
             |      PCI Host      |
             |      PCI Arbiter   |
             +--------------------+
                       |
<-------- PCI bus ---------------------------------->
    |                       |
+---------+           +---------------------------+    
| Various |           | IXDP465 (PCI option mode) |
|  other  |           |         Redboot           |
|   PCI   |           |      Linux 2.6.16-rc5     |
| Devices |           +---------------------------+
+---------+

When power is applied to the system, several things are happening 
simultaneously, as I understand it:

1) The POST/BIOS code executes on the Pentium side that should do the actions 
you describe above.
2) The PCI devices are configuring themselves.
3) The IXDP465 is executing Redboot bootloader, which hands off control to 
Linux.

As it turns out, Linux completes it's bootup before Windows bootup even 
begins, and it seems that Linux changes the configuration of the various 
other PCI devices that happen to be on the system as well. I need to get 
Linux to leave the configuration of other PCI devices it finds alone. It 
should only mess with it's own configuration. Why should Linux need to change 
the configuration of other PCI devices when it is fulfilling the role of a 
PCI device itself?

Thanks,

Jon

Re: Linux running on a PCI Option device?

[Adrian Cox]

On Thu, 2006-03-02 at 17:07 -0500, Jon Ringle wrote:

> As it turns out, Linux completes it's bootup before Windows bootup even 
> begins, and it seems that Linux changes the configuration of the various 
> other PCI devices that happen to be on the system as well. I need to get 
> Linux to leave the configuration of other PCI devices it finds alone. It 
> should only mess with it's own configuration. Why should Linux need to change 
> the configuration of other PCI devices when it is fulfilling the role of a 
> PCI device itself?

Have you disabled CONFIG_PCI?

CONFIG_PCI is the configuration option for a PCI host, just as
CONFIG_USB is the configuration option for a USB host. Linux contains
code for CONFIG_USB_GADGET, but what you need is the non-existent
CONFIG_PCI_GADGET.

If you're running on a PCI option device (unless using a 21555
non-transparent bridge), you need to disable CONFIG_PCI and write your
own driver for the PCI option device functionality.

-- 
Adrian Cox <adrian@humboldt.co.uk>

Re: Linux running on a PCI Option device?

[Jon Ringle]

On Friday 03 March 2006 04:13 am, Adrian Cox wrote:
> On Thu, 2006-03-02 at 17:07 -0500, Jon Ringle wrote:
> > As it turns out, Linux completes it's bootup before Windows bootup even
> > begins, and it seems that Linux changes the configuration of the various
> > other PCI devices that happen to be on the system as well. I need to get
> > Linux to leave the configuration of other PCI devices it finds alone. It
> > should only mess with it's own configuration. Why should Linux need to
> > change the configuration of other PCI devices when it is fulfilling the
> > role of a PCI device itself?
>
> Have you disabled CONFIG_PCI?
>
> CONFIG_PCI is the configuration option for a PCI host, just as
> CONFIG_USB is the configuration option for a USB host. Linux contains
> code for CONFIG_USB_GADGET, but what you need is the non-existent
> CONFIG_PCI_GADGET.
>
> If you're running on a PCI option device (unless using a 21555
> non-transparent bridge), you need to disable CONFIG_PCI and write your
> own driver for the PCI option device functionality.

Another requirement that I have that makes it difficult for me to disable 
CONFIG_PCI is that the hardware component that is running Windows (and 
therefore the PCI host) is optional hardware. If the Windows part is not 
present, then the IXP will be configured (via hardware means) as a PCI host. 
So, I need to detect at run time whether the IXP is in PCI option or PCI host 
mode. If it is in PCI host mode then the code encapuslated by CONFIG_PCI must 
be available.

I can see now that I also have a need to have a "CONFIG_PCI_GADGET".

Jon

Re: Linux running on a PCI Option device?

[Adrian Cox]

On Fri, 2006-03-03 at 09:09 -0500, Jon Ringle wrote:
> On Friday 03 March 2006 04:13 am, Adrian Cox wrote:
> > If you're running on a PCI option device (unless using a 21555
> > non-transparent bridge), you need to disable CONFIG_PCI and write your
> > own driver for the PCI option device functionality.
> 
> Another requirement that I have that makes it difficult for me to disable 
> CONFIG_PCI is that the hardware component that is running Windows (and 
> therefore the PCI host) is optional hardware. If the Windows part is not 
> present, then the IXP will be configured (via hardware means) as a PCI host. 
> So, I need to detect at run time whether the IXP is in PCI option or PCI host 
> mode. If it is in PCI host mode then the code encapuslated by CONFIG_PCI must 
> be available.

Based on only a quick look at the code: if the Windows host is present,
don't call pci_common_init() in ixdp425_pci_init(). You then need to
write the entire CONFIG_PCI_GADGET code, and ensure that you only enable
it if the Windows host is present.

The greater problem is that these changes may work all the way up into
your application. If the Windows host is present, then you need its
cooperation to find other devices on the PCI bus.

-- 
Adrian Cox <adrian@humboldt.co.uk>

Re: Linux running on a PCI Option device?

[Jon Ringle]

On Friday 03 March 2006 09:40 am, Adrian Cox wrote:
> Based on only a quick look at the code: if the Windows host is present,
> don't call pci_common_init() in ixdp425_pci_init().

Doing this will prevent the code in ixp4xx_pci_preinit() from executing which 
handles some initialization for both PCI host and option modes. Should I go 
ahead and explicitly call ixp4xx_pci_preinit() from ixdp425_pci_init() if in 
PCI option mode?

Jon

Re: Linux running on a PCI Option device?

[Adrian Cox]

On Fri, 2006-03-03 at 13:31 -0500, Jon Ringle wrote:
> On Friday 03 March 2006 09:40 am, Adrian Cox wrote:
> > Based on only a quick look at the code: if the Windows host is present,
> > don't call pci_common_init() in ixdp425_pci_init().
> 
> Doing this will prevent the code in ixp4xx_pci_preinit() from executing which 
> handles some initialization for both PCI host and option modes. Should I go 
> ahead and explicitly call ixp4xx_pci_preinit() from ixdp425_pci_init() if in 
> PCI option mode?

At this point I have to give up - I'm not an ixp4xx expert. While I have
developed a PCI option device under Linux, it was a PowerPC with a PLX
bridge and some custom logic.

-- 
Adrian Cox <adrian@humboldt.co.uk>

Re: Linux running on a PCI Option device?

[Greg Ungerer]

Hi Jon,

Jon Ringle wrote:
> On Friday 03 March 2006 09:40 am, Adrian Cox wrote:
> 
>>Based on only a quick look at the code: if the Windows host is present,
>>don't call pci_common_init() in ixdp425_pci_init().
> 
> 
> Doing this will prevent the code in ixp4xx_pci_preinit() from executing which 
> handles some initialization for both PCI host and option modes. Should I go 
> ahead and explicitly call ixp4xx_pci_preinit() from ixdp425_pci_init() if in 
> PCI option mode?

The older 2.4 kernel code for the IXP425 support pretty much did it
this way (which is what makes it look like it supports the device
in option mode).

Code snippet from ixp425_pci_init() in arch/arm/mach-ixp425/ixp425-pci.c
(from patches for 2.4 kernels to support ixp425):

        ...
        if (ixp425_pci_is_host())
         {
                 local_write_config_word(PCI_COMMAND, PCI_COMMAND_MASTER |
                         PCI_COMMAND_MEMORY);

                 DBG("allocating hose\n");
                 hose = pcibios_alloc_controller();
                 if (!hose)
                 panic("Could not allocate PCI hose");

                 hose->first_busno = 0;
                 hose->last_busno = 0;
                 hose->io_space.start = 0;
                 hose->io_space.end = 0xffffffff;
                 hose->mem_space.start = 0x48000000;
                 hose->mem_space.end = 0x4bffffff;

                 /* autoconfig the bus */ DBG("AUTOCONFIG\n");
                 hose->last_busno = pciauto_bus_scan(hose, 0);

                 /* scan the bus */
                 DBG("SCANNING THE BUS\n");
                 pci_scan_bus(0, &ixp425_ops, sysdata);
          }


Pretty much the rest of the PCI init is the same, it just
doesn't do the bus scan.

Regards
Greg


------------------------------------------------------------------------
Greg Ungerer  --  Chief Software Dude       EMAIL:     gerg@snapgear.com
SnapGear -- a Secure Computing Company      PHONE:       +61 7 3435 2888
825 Stanley St,                             FAX:         +61 7 3891 3630
Woolloongabba, QLD, 4102, Australia         WEB: http://www.SnapGear.com