[PATCH]PCI on SWARM

[PATCH]PCI on SWARM

[Manish Lachwani]

Hello Ralf

This small patch is required to get PCI working on the Broadcom SWARM 
board in 2.6. Without this patch, the PCI bus scan is skipped due to 
resource conflict. Tested using the E100 network card

Thanks
Manish Lachwani

Index: linux/arch/mips/pci/pci-sb1250.c
===================================================================
--- linux.orig/arch/mips/pci/pci-sb1250.c
+++ linux/arch/mips/pci/pci-sb1250.c
@@ -209,7 +209,7 @@
        pci_probe_only = 1;
 
        /* set resource limit to avoid errors */
-       ioport_resource.end = 0x0000ffff;       /* 32MB reserved by 
sb1250 */
+       ioport_resource.end = 0xffffffff;       /* 32MB reserved by 
sb1250 */
        iomem_resource.end = 0xffffffff;        /* no HT support yet */
 
        cfg_space =

Re: [PATCH]PCI on SWARM

[Ralf Baechle]

On Wed, Oct 13, 2004 at 07:23:26PM -0700, Manish Lachwani wrote:

> This small patch is required to get PCI working on the Broadcom SWARM 
> board in 2.6. Without this patch, the PCI bus scan is skipped due to 
> resource conflict. Tested using the E100 network card

> -       ioport_resource.end = 0x0000ffff;       /* 32MB reserved by 
> sb1250 */
> +       ioport_resource.end = 0xffffffff;       /* 32MB reserved by 
> sb1250 */

I'm too lazy to dig up the actual numbers from the BCM1250 manuals but
it definately does not have 4GB of port address space.

  Ralf

Re: [PATCH]PCI on SWARM

[Maciej W. Rozycki]

On Thu, 14 Oct 2004, Ralf Baechle wrote:

> > This small patch is required to get PCI working on the Broadcom SWARM 
> > board in 2.6. Without this patch, the PCI bus scan is skipped due to 
> > resource conflict. Tested using the E100 network card
> 
> > -       ioport_resource.end = 0x0000ffff;       /* 32MB reserved by 
> > sb1250 */
> > +       ioport_resource.end = 0xffffffff;       /* 32MB reserved by 
> > sb1250 */
> 
> I'm too lazy to dig up the actual numbers from the BCM1250 manuals but
> it definately does not have 4GB of port address space.

 The doc states low 25 bits are used for I/O addressing, matching the
comment above (surprise!, surprise!), so I guess the constant sought for
the bit mask above is 0x01ffffff.  If that turns out to work, I can apply
an update (can I, Ralf?).

  Maciej

Re: [PATCH]PCI on SWARM

[Manish Lachwani]

Maciej W. Rozycki wrote:

>On Thu, 14 Oct 2004, Ralf Baechle wrote:
>
>  
>
>>>This small patch is required to get PCI working on the Broadcom SWARM 
>>>board in 2.6. Without this patch, the PCI bus scan is skipped due to 
>>>resource conflict. Tested using the E100 network card
>>>      
>>>
>>>-       ioport_resource.end = 0x0000ffff;       /* 32MB reserved by 
>>>sb1250 */
>>>+       ioport_resource.end = 0xffffffff;       /* 32MB reserved by 
>>>sb1250 */
>>>      
>>>
>>I'm too lazy to dig up the actual numbers from the BCM1250 manuals but
>>it definately does not have 4GB of port address space.
>>    
>>
>
> The doc states low 25 bits are used for I/O addressing, matching the
>comment above (surprise!, surprise!), so I guess the constant sought for
>the bit mask above is 0x01ffffff.  If that turns out to work, I can apply
>an update (can I, Ralf?).
>
>  Maciej
>

Hello !

Honestly, I dont have the manual to determine the port address space 
bits. Hence, I set it to this value to MAX (i.e. 0xffffffff). Probably, 
should have mentioned that when sending the patch. Do you want me to try 
with this value (0x01ffffff) ?

Thanks
Manish Lachwani

Re: [PATCH]PCI on SWARM

[Maciej W. Rozycki]

On Thu, 14 Oct 2004, Manish Lachwani wrote:

> Honestly, I dont have the manual to determine the port address space 

 Well, have you considered getting one?

> bits. Hence, I set it to this value to MAX (i.e. 0xffffffff). Probably, 
> should have mentioned that when sending the patch. Do you want me to try 
> with this value (0x01ffffff) ?

 Absolutely -- unfortunately I cannot test the change myself ATM.

  Maciej

Re: [PATCH]PCI on SWARM

[Ralf Baechle]

On Thu, Oct 14, 2004 at 11:32:25PM +0100, Maciej W. Rozycki wrote:

> > Honestly, I dont have the manual to determine the port address space 
> 
>  Well, have you considered getting one?

Afaik it's now available without NDA from http://swarm.broadcom.com .

> > bits. Hence, I set it to this value to MAX (i.e. 0xffffffff). Probably, 
> > should have mentioned that when sending the patch. Do you want me to try 
> > with this value (0x01ffffff) ?
> 
>  Absolutely -- unfortunately I cannot test the change myself ATM.

Sure, go ahead.  This btw should match with the pci_controller definition
which is looking fishy also.

  Ralf

Re: [PATCH]PCI on SWARM

[Manish Lachwani]

Maciej W. Rozycki wrote:

>On Thu, 14 Oct 2004, Manish Lachwani wrote:
>
>  
>
>>Honestly, I dont have the manual to determine the port address space 
>>    
>>
>
> Well, have you considered getting one?
>
>  
>
>>bits. Hence, I set it to this value to MAX (i.e. 0xffffffff). Probably, 
>>should have mentioned that when sending the patch. Do you want me to try 
>>with this value (0x01ffffff) ?
>>    
>>
>
> Absolutely -- unfortunately I cannot test the change myself ATM.
>
>  Maciej
>  
>

Hello !

This value works well.

Thanks
Manish Lachwani

Re: [PATCH]PCI on SWARM

[Maciej W. Rozycki]

On Fri, 15 Oct 2004, Ralf Baechle wrote:

> Sure, go ahead.  This btw should match with the pci_controller definition
> which is looking fishy also.

 Tough.  Both the PCI memory and the PCI I/O spaces are mapped in several
areas, depending on the byte lane swapping policy needed and whether
64-bit addressing is feasible or not.  We'd need two areas for I/O and
four for memory (plus another one for the 40-bit HT address space).

  Maciej

Re: [PATCH]PCI on SWARM

[Jes Sorensen]

>>>>> "Maciej" == Maciej W Rozycki <macro@linux-mips.org> writes:

Maciej> On Fri, 15 Oct 2004, Ralf Baechle wrote:
>> Sure, go ahead.  This btw should match with the pci_controller
>> definition which is looking fishy also.

Maciej>  Tough.  Both the PCI memory and the PCI I/O spaces are mapped
Maciej> in several areas, depending on the byte lane swapping policy
Maciej> needed and whether 64-bit addressing is feasible or not.  We'd
Maciej> need two areas for I/O and four for memory (plus another one
Maciej> for the 40-bit HT address space).

Dual address cycles, ie. 64 bit addressing is fscked on the 1250 from
what I remember. Correct way to work around this is to stick all
physical memory outside the 32 bit space into ZONE_HIGHMEM - had a
patch for 2.4, but I lost it ages ago ;-(

One can just hope Broadcom will learn how to make chips some day ;-(

Regards,
Jes

Re: [PATCH]PCI on SWARM

[Ralf Baechle]

On Mon, Oct 18, 2004 at 04:44:17AM -0400, Jes Sorensen wrote:

> Dual address cycles, ie. 64 bit addressing is fscked on the 1250 from
> what I remember. Correct way to work around this is to stick all
> physical memory outside the 32 bit space into ZONE_HIGHMEM - had a patch
> for 2.4, but I lost it ages ago ;-(

The Momentum Jaguar suffers from similar problems, so I've implemented
that for Linux 2.6 as CONFIG_LIMITED_DMA.

> One can just hope Broadcom will learn how to make chips some day ;-(

Well, they've just done their homework, also known as the 1450.  Pick
your red pen and give grades ;-)

  Ralf

Re: [PATCH]PCI on SWARM

[Maciej W. Rozycki]

On Mon, 18 Oct 2004, Jes Sorensen wrote:

> Maciej>  Tough.  Both the PCI memory and the PCI I/O spaces are mapped
> Maciej> in several areas, depending on the byte lane swapping policy
> Maciej> needed and whether 64-bit addressing is feasible or not.  We'd
> Maciej> need two areas for I/O and four for memory (plus another one
> Maciej> for the 40-bit HT address space).
> 
> Dual address cycles, ie. 64 bit addressing is fscked on the 1250 from
> what I remember. Correct way to work around this is to stick all
> physical memory outside the 32 bit space into ZONE_HIGHMEM - had a
> patch for 2.4, but I lost it ages ago ;-(

 The BCM1250A's PCI is obviously 32-bit only (the host bridge is
explicitly documented not to support DACs), but you need to use 64-bit
addressing to access the whole 4GB range -- the bus is decoded starting
from 0xf800000000 and 0xf900000000, with a different byte lane swapping
policy for each of these areas.  For limited access for 32-bit software,
you can use alternate mappings at 0x40000000 and 0x60000000, but they
provide only 512MB of space each.

 I don't think something like DACs exists for HT.

  Maciej

Re: [PATCH]PCI on SWARM

[Jes Sorensen]

>>>>> "Ralf" == Ralf Baechle <ralf@linux-mips.org> writes:

Ralf> On Mon, Oct 18, 2004 at 04:44:17AM -0400, Jes Sorensen wrote:
>> Dual address cycles, ie. 64 bit addressing is fscked on the 1250
>> from what I remember. Correct way to work around this is to stick
>> all physical memory outside the 32 bit space into ZONE_HIGHMEM -
>> had a patch for 2.4, but I lost it ages ago ;-(

Ralf> The Momentum Jaguar suffers from similar problems, so I've
Ralf> implemented that for Linux 2.6 as CONFIG_LIMITED_DMA.

Wouldn't it be better to always have the highmem zone configured and
then just fill it up with pages if you have a b0rked platform? The
cost of this is miniscule.

>> One can just hope Broadcom will learn how to make chips some day
>> ;-(

Ralf> Well, they've just done their homework, also known as the 1450.
Ralf> Pick your red pen and give grades ;-)

The moment I receive one in my mailbox ;-)

Cheers,
Jes

Re: [PATCH]PCI on SWARM

[Jes Sorensen]

>>>>> "Maciej" == Maciej W Rozycki <macro@linux-mips.org> writes:

Maciej> On Mon, 18 Oct 2004, Jes Sorensen wrote: 
>>  Dual address cycles, ie. 64 bit addressing is fscked on the 1250
>> from what I remember. Correct way to work around this is to stick
>> all physical memory outside the 32 bit space into ZONE_HIGHMEM -
>> had a patch for 2.4, but I lost it ages ago ;-(

Maciej>  The BCM1250A's PCI is obviously 32-bit only (the host bridge
Maciej> is explicitly documented not to support DACs), but you need to
Maciej> use 64-bit addressing to access the whole 4GB range -- the bus
Maciej> is decoded starting from 0xf800000000 and 0xf900000000, with a
Maciej> different byte lane swapping policy for each of these areas.
Maciej> For limited access for 32-bit software, you can use alternate
Maciej> mappings at 0x40000000 and 0x60000000, but they provide only
Maciej> 512MB of space each.

Yeah it seems to be documented, it's highly unfortunate that anybody
will release a chip which doesn't support DAC but will support memory
outside the 4GB range. I'd be highly embarrassed if I had done the
chip, but thats just me.

I think you are right about the HT part.

Cheers,
JEs

Re: [PATCH]PCI on SWARM

[Ralf Baechle]

On Wed, Oct 20, 2004 at 01:57:36AM -0400, Jes Sorensen wrote:

> Yeah it seems to be documented, it's highly unfortunate that anybody
> will release a chip which doesn't support DAC but will support memory
> outside the 4GB range. I'd be highly embarrassed if I had done the
> chip, but thats just me.

The memory map which apparently has been created with a shotgun
aggrevates the problem ...

  Ralf

Re: [PATCH]PCI on SWARM

[Ralf Baechle]

On Wed, Oct 20, 2004 at 01:56:02AM -0400, Jes Sorensen wrote:

> Ralf> On Mon, Oct 18, 2004 at 04:44:17AM -0400, Jes Sorensen wrote:
> >> Dual address cycles, ie. 64 bit addressing is fscked on the 1250
> >> from what I remember. Correct way to work around this is to stick
> >> all physical memory outside the 32 bit space into ZONE_HIGHMEM -
> >> had a patch for 2.4, but I lost it ages ago ;-(
> 
> Ralf> The Momentum Jaguar suffers from similar problems, so I've
> Ralf> implemented that for Linux 2.6 as CONFIG_LIMITED_DMA.
> 
> Wouldn't it be better to always have the highmem zone configured and
> then just fill it up with pages if you have a b0rked platform?

On these platform I enable CONFIG_HIGHMEM and because they don't support
real highmem in sense of requiring other mappings than KSEG0 for access
by the processor I replace kmap() etc. with the non-highmem variants
from <linux/highmem.h>.  Filling in page->virtual on startup and defining
WANT_PAGE_VIRTUAL makes sure the mapping overhead stays low.  An ugly
solution but probably the only one acceptable upstream and fortunately
there are now nicer variants of the architecture available that don't
need such tricks.

> The cost of this is miniscule.

I wish it was.  The kernel has this preference for ZONE_NORMAL over
ZONE_HIGHMEM but on this particular platform ZONE_HIGHMEM with it's
memory controller integrated into the CPU itself provides better performance
than ZONE_NORMAL which resides in memory controller that's separate from
the processor in an external system controller.

If further optimizations to were of interest I'd probably try to play a
bit with build_zonelists_node() ...

  Ralf