Non-contiguous physical memory

Non-contiguous physical memory

[Aaron Pace]

Hello,

I'm working on a design using a Freescale MPC8572 processor.
We are using 4 gigs of memory, and also need a window of 512 megs for
PCI-E devices.
What I have done is set up the first 2G of memory from 0x0 - 0x7f, the
PCI windows from 0x8 - 0x9f, localbus devices + CCSRBAR from 0xf -
0xffffffff, and the second 2G of ram from 0x1.0000.0000 -
0x1.8000.0000.
I've got this set up in U-boot (although it only uses the low mem),
but Linux will only use the first contiguous physical area (the
message is "Only using first contiguous memory region").
Is it possible to have multiple non-contiguous physical memory chunks
used for memory allocation?
If not, is there a better way to set this up without losing large
chunks of memory?

Thanks,
Aaron

Re: Non-contiguous physical memory

[Kumar Gala]

On Jan 21, 2009, at 9:18 AM, Aaron Pace wrote:

> Hello,
>
> I'm working on a design using a Freescale MPC8572 processor.
> We are using 4 gigs of memory, and also need a window of 512 megs for
> PCI-E devices.
> What I have done is set up the first 2G of memory from 0x0 - 0x7f, the
> PCI windows from 0x8 - 0x9f, localbus devices + CCSRBAR from 0xf -
> 0xffffffff, and the second 2G of ram from 0x1.0000.0000 -
> 0x1.8000.0000.
> I've got this set up in U-boot (although it only uses the low mem),
> but Linux will only use the first contiguous physical area (the
> message is "Only using first contiguous memory region").
> Is it possible to have multiple non-contiguous physical memory chunks
> used for memory allocation?
> If not, is there a better way to set this up without losing large
> chunks of memory?

Its possible in that Linux supports this.  However the PPC32 code does  
not exist and would need to be added to support non-contiguous memory  
ranges.

What exact PCI-E needs do you have?  Is PCI-E performance critical?   
Is (are) your pci device(s) 64-bit address capable?

I ask because depending on the answers doing straight 4G and PCI above  
that range might be sufficient for your needs.

- k

Re: Non-contiguous physical memory

[Aaron Pace]

>
> Its possible in that Linux supports this.  However the PPC32 code does not
> exist and would need to be added to support non-contiguous memory ranges.
>
> What exact PCI-E needs do you have?  Is PCI-E performance critical?  Is
> (are) your pci device(s) 64-bit address capable?
>
> I ask because depending on the answers doing straight 4G and PCI above that
> range might be sufficient for your needs.
>
> - k
>

Hello, thanks for responding.
The first take at this used your idea of mapping the PCI-E ranges
above the 4G boundary.
We have a localbus bridge device on the PCI-E bus that only supports
32-bit MMIO, but I believe we could satisfactorily work around that
problem using the ATMU registers.
However, one question that I had that I wasn't able to verify to my
satisfaction was whether I could ioremap a 36-bit physical address
when building for ppc32, as the PCI-E devices have memory ranges that
need to be accessible from userspace.  If that is possible, then this
may be the way to go, although it would still be nice not to lose the
128 meg for the localbus & CCSR region.

Thanks,
-Aaron

Re: Non-contiguous physical memory

[Kumar Gala]

On Jan 21, 2009, at 12:00 PM, Aaron Pace wrote:

>>
>> Its possible in that Linux supports this.  However the PPC32 code  
>> does not
>> exist and would need to be added to support non-contiguous memory  
>> ranges.
>>
>> What exact PCI-E needs do you have?  Is PCI-E performance  
>> critical?  Is
>> (are) your pci device(s) 64-bit address capable?
>>
>> I ask because depending on the answers doing straight 4G and PCI  
>> above that
>> range might be sufficient for your needs.
>>
>> - k
>>
>
> Hello, thanks for responding.
> The first take at this used your idea of mapping the PCI-E ranges
> above the 4G boundary.
> We have a localbus bridge device on the PCI-E bus that only supports
> 32-bit MMIO, but I believe we could satisfactorily work around that
> problem using the ATMU registers.
> However, one question that I had that I wasn't able to verify to my
> satisfaction was whether I could ioremap a 36-bit physical address
> when building for ppc32, as the PCI-E devices have memory ranges that
> need to be accessible from userspace.  If that is possible, then this
> may be the way to go, although it would still be nice not to lose the
> 128 meg for the localbus & CCSR region.

you can get 36-bit physical addresses but you need to enable "Large  
physical address support".

- k

Re: Non-contiguous physical memory

[Michele Pallaro]

Hello,
	I have a similar problem with my custom board CPU mpc8548 (E500V2) with
4Gbytes of RAM

	  0x0000_0000 --    0x7FFF_FFFF  2Gbytes
	0x1_0000_0000 -- 0x1_7FFF_FFFF   2Gbytes

I Enable the option PHYS_64BIT, and set in dts

memory {
                #address-cells = <2>;
                #size-cells = <1>;   
                device_type = "memory";
                reg = <00000000 00000000 00000000 7FFFFFFF
                       00000001 00000000 00000000 7FFFFFFF>;
        };  

But the kernel can see just 2Gbytes of memory.
How the kernel can see the other 2Gbytes ?

	Michele
	

On Wed, 2009-01-21 at 11:28 -0600, Kumar Gala wrote:
> On Jan 21, 2009, at 9:18 AM, Aaron Pace wrote:
> 
> > Hello,
> >
> > I'm working on a design using a Freescale MPC8572 processor.
> > We are using 4 gigs of memory, and also need a window of 512 megs for
> > PCI-E devices.
> > What I have done is set up the first 2G of memory from 0x0 - 0x7f, the
> > PCI windows from 0x8 - 0x9f, localbus devices + CCSRBAR from 0xf -
> > 0xffffffff, and the second 2G of ram from 0x1.0000.0000 -
> > 0x1.8000.0000.
> > I've got this set up in U-boot (although it only uses the low mem),
> > but Linux will only use the first contiguous physical area (the
> > message is "Only using first contiguous memory region").
> > Is it possible to have multiple non-contiguous physical memory chunks
> > used for memory allocation?
> > If not, is there a better way to set this up without losing large
> > chunks of memory?
> 
> Its possible in that Linux supports this.  However the PPC32 code does  
> not exist and would need to be added to support non-contiguous memory  
> ranges.
> 
> What exact PCI-E needs do you have?  Is PCI-E performance critical?   
> Is (are) your pci device(s) 64-bit address capable?
> 
> I ask because depending on the answers doing straight 4G and PCI above  
> that range might be sufficient for your needs.
> 
> - k
> _______________________________________________
> Linuxppc-dev mailing list
> Linuxppc-dev@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-dev

Re: Non-contiguous physical memory

[Kumar Gala]

On Jan 23, 2009, at 11:52 AM, Michele Pallaro wrote:

> Hello,
> 	I have a similar problem with my custom board CPU mpc8548 (E500V2)  
> with
> 4Gbytes of RAM
>
> 	  0x0000_0000 --    0x7FFF_FFFF  2Gbytes
> 	0x1_0000_0000 -- 0x1_7FFF_FFFF   2Gbytes
>
> I Enable the option PHYS_64BIT, and set in dts
>
> memory {
>                #address-cells = <2>;
>                #size-cells = <1>;
>                device_type = "memory";
>                reg = <00000000 00000000 00000000 7FFFFFFF
>                       00000001 00000000 00000000 7FFFFFFF>;
>        };
>
> But the kernel can see just 2Gbytes of memory.
> How the kernel can see the other 2Gbytes ?
>
> 	Michele

This should be:

reg = < 0x00000000 0x00000000 0x00000000 0x80000000
         0x00000001 0x00000000 0x00000000 0x80000000 >;

Its a start, size pair.

- k