[U-Boot] PCIE_MEM_BUS for Freescale SoC

[Scott Wood <scottwood@freescale.com>]

On Mon, 2015-11-23 at 09:00 -0800, York Sun wrote:
> 
> On 11/23/2015 08:24 AM, Scott Wood wrote:
> > On Sat, 2015-11-21 at 15:29 -0800, York Sun wrote:
> > > 
> > > On 11/21/2015 02:55 PM, York Sun wrote:
> > > > Roy,
> > > > 
> > > > Do you remember the reason why we use different virtual memory address
> > > > from pci
> > > > bus address with 36-bit? For example
> > > > 
> > > > include/configs/P1022DS.h-496-#define CONFIG_SYS_PCIE1_MEM_VIRT
> > > > 0xc0000000
> > > > include/configs/P1022DS.h-497-#ifdef CONFIG_PHYS_64BIT
> > > > include/configs/P1022DS.h:498:#define CONFIG_SYS_PCIE1_MEM_BUS 
> > > >  0xe0000000
> > > > include/configs/P1022DS.h-499-#define CONFIG_SYS_PCIE1_MEM_PHYS
> > > > 0xc40000000ull
> > > > include/configs/P1022DS.h-500-#else
> > > > include/configs/P1022DS.h:501:#define CONFIG_SYS_PCIE1_MEM_BUS 
> > > >  0xc0000000
> > > > include/configs/P1022DS.h-502-#define CONFIG_SYS_PCIE1_MEM_PHYS
> > > > 0xc0000000
> > > > include/configs/P1022DS.h-503-#endif
> > > > 
> > > > As far as I can tell, the following is the mapping
> > > > 
> > > > TLB: MEM_VIRT=>MEM_PHYS
> > > > PCI: MEM_BUS=>MEM_PHYS
> > > > LAW: MEM_PHYS=>pcie interface
> > > > 
> > > > Being different for MEM_VIRT and MEM_BUS cause confusion. When I run
> > > > "pci
> > > > header" command to show the BARs, I expect I can use "md" to access
> > > > the
> > > > BAR
> > > > address. That's not the case if MEM_BUS is different from MEM_VIRT.
> > > > 
> > > > I forget why we did this for 36-bit addressing. The MEM_VIRT is the
> > > > same
> > > > as
> > > > MEM_BUS for 32-bit addressing. And why do we use the same MEM_BUS
> > > > address
> > > > for
> > > > all PCIe hose? I know they are not conflicting, but is it necessary?
> > > 
> > > (I hope Becky and Kumar still follow this mailing list)
> > > 
> > > I dug out an old commit 4c78d4a6c01621721b732418e1c6da684a56bbb1 by
> > > Becky
> > > Bruce.
> > > She believed overlapping the bus address for PCI controllers leaves more
> > > space.
> > > That's true. But we haven't use more than 512MB in u-boot. If we do need
> > > more
> > > space, we can easily move things around if we have PHYS_64BIT. If no
> > > objection,
> > > I'd like to change this back.
> > 
> > I object.  It's not about how much RAM is used in U-Boot; it's about how
> > much
> > memory the OS needs to bounce-buffer for DMA.  The addresses set up by U
> > -Boot
> > should match what's in the device tree.  Yes, on 85xx Linux reprograms the
> > ATMU based on the device tree rather than trusting U-Boot, but that
> > doesn't
> > mean every OS does.
> 
> Isn't this backward? I mean the device tree should match u-boot,

Yes, and I don't want to deal with the churn of these device trees changing.

>  or u-boot should fix up the device tree, right?

In theory yes, but then you really are affecting Linux's usage.

> I checked several device tree files in kernel. We are not using more than
> 512MB
> for each PCI. What's the benefit of using the same address 0xe0000000?

Reducing the amount of RAM whose address matches a PCI bus address and thus
can't be used for PCI DMA.

-Scott