* [U-Boot] PCIE_MEM_BUS for Freescale SoC @ 2015-11-21 22:55 York Sun 2015-11-21 23:29 ` York Sun 0 siblings, 1 reply; 7+ messages in thread From: York Sun @ 2015-11-21 22:55 UTC (permalink / raw) To: u-boot Roy, Do you remember the reason why we use different virtual memory address from pci bus address with 36-bit? For example include/configs/P1022DS.h-496-#define CONFIG_SYS_PCIE1_MEM_VIRT 0xc0000000 include/configs/P1022DS.h-497-#ifdef CONFIG_PHYS_64BIT include/configs/P1022DS.h:498:#define CONFIG_SYS_PCIE1_MEM_BUS 0xe0000000 include/configs/P1022DS.h-499-#define CONFIG_SYS_PCIE1_MEM_PHYS 0xc40000000ull include/configs/P1022DS.h-500-#else include/configs/P1022DS.h:501:#define CONFIG_SYS_PCIE1_MEM_BUS 0xc0000000 include/configs/P1022DS.h-502-#define CONFIG_SYS_PCIE1_MEM_PHYS 0xc0000000 include/configs/P1022DS.h-503-#endif As far as I can tell, the following is the mapping TLB: MEM_VIRT=>MEM_PHYS PCI: MEM_BUS=>MEM_PHYS LAW: MEM_PHYS=>pcie interface Being different for MEM_VIRT and MEM_BUS cause confusion. When I run "pci header" command to show the BARs, I expect I can use "md" to access the BAR address. That's not the case if MEM_BUS is different from MEM_VIRT. I forget why we did this for 36-bit addressing. The MEM_VIRT is the same as MEM_BUS for 32-bit addressing. And why do we use the same MEM_BUS address for all PCIe hose? I know they are not conflicting, but is it necessary? Let me know if you have any concerns. Otherwise I will change all platforms to use consistent addresses. York ^ permalink raw reply [flat|nested] 7+ messages in thread
* [U-Boot] PCIE_MEM_BUS for Freescale SoC 2015-11-21 22:55 [U-Boot] PCIE_MEM_BUS for Freescale SoC York Sun @ 2015-11-21 23:29 ` York Sun 2015-11-23 16:24 ` Scott Wood 0 siblings, 1 reply; 7+ messages in thread From: York Sun @ 2015-11-21 23:29 UTC (permalink / raw) To: u-boot On 11/21/2015 02:55 PM, York Sun wrote: > Roy, > > Do you remember the reason why we use different virtual memory address from pci > bus address with 36-bit? For example > > include/configs/P1022DS.h-496-#define CONFIG_SYS_PCIE1_MEM_VIRT 0xc0000000 > include/configs/P1022DS.h-497-#ifdef CONFIG_PHYS_64BIT > include/configs/P1022DS.h:498:#define CONFIG_SYS_PCIE1_MEM_BUS 0xe0000000 > include/configs/P1022DS.h-499-#define CONFIG_SYS_PCIE1_MEM_PHYS 0xc40000000ull > include/configs/P1022DS.h-500-#else > include/configs/P1022DS.h:501:#define CONFIG_SYS_PCIE1_MEM_BUS 0xc0000000 > include/configs/P1022DS.h-502-#define CONFIG_SYS_PCIE1_MEM_PHYS 0xc0000000 > include/configs/P1022DS.h-503-#endif > > As far as I can tell, the following is the mapping > > TLB: MEM_VIRT=>MEM_PHYS > PCI: MEM_BUS=>MEM_PHYS > LAW: MEM_PHYS=>pcie interface > > Being different for MEM_VIRT and MEM_BUS cause confusion. When I run "pci > header" command to show the BARs, I expect I can use "md" to access the BAR > address. That's not the case if MEM_BUS is different from MEM_VIRT. > > I forget why we did this for 36-bit addressing. The MEM_VIRT is the same as > MEM_BUS for 32-bit addressing. And why do we use the same MEM_BUS address for > all PCIe hose? I know they are not conflicting, but is it necessary? (I hope Becky and Kumar still follow this mailing list) I dug out an old commit 4c78d4a6c01621721b732418e1c6da684a56bbb1 by Becky Bruce. She believed overlapping the bus address for PCI controllers leaves more space. That's true. But we haven't use more than 512MB in u-boot. If we do need more space, we can easily move things around if we have PHYS_64BIT. If no objection, I'd like to change this back. York ^ permalink raw reply [flat|nested] 7+ messages in thread
* [U-Boot] PCIE_MEM_BUS for Freescale SoC 2015-11-21 23:29 ` York Sun @ 2015-11-23 16:24 ` Scott Wood 2015-11-23 17:00 ` York Sun 0 siblings, 1 reply; 7+ messages in thread From: Scott Wood @ 2015-11-23 16:24 UTC (permalink / raw) To: u-boot On Sat, 2015-11-21 at 15:29 -0800, York Sun wrote: > > On 11/21/2015 02:55 PM, York Sun wrote: > > Roy, > > > > Do you remember the reason why we use different virtual memory address > > from pci > > bus address with 36-bit? For example > > > > include/configs/P1022DS.h-496-#define CONFIG_SYS_PCIE1_MEM_VIRT 0xc0000000 > > include/configs/P1022DS.h-497-#ifdef CONFIG_PHYS_64BIT > > include/configs/P1022DS.h:498:#define CONFIG_SYS_PCIE1_MEM_BUS 0xe0000000 > > include/configs/P1022DS.h-499-#define CONFIG_SYS_PCIE1_MEM_PHYS > > 0xc40000000ull > > include/configs/P1022DS.h-500-#else > > include/configs/P1022DS.h:501:#define CONFIG_SYS_PCIE1_MEM_BUS 0xc0000000 > > include/configs/P1022DS.h-502-#define CONFIG_SYS_PCIE1_MEM_PHYS 0xc0000000 > > include/configs/P1022DS.h-503-#endif > > > > As far as I can tell, the following is the mapping > > > > TLB: MEM_VIRT=>MEM_PHYS > > PCI: MEM_BUS=>MEM_PHYS > > LAW: MEM_PHYS=>pcie interface > > > > Being different for MEM_VIRT and MEM_BUS cause confusion. When I run "pci > > header" command to show the BARs, I expect I can use "md" to access the > > BAR > > address. That's not the case if MEM_BUS is different from MEM_VIRT. > > > > I forget why we did this for 36-bit addressing. The MEM_VIRT is the same > > as > > MEM_BUS for 32-bit addressing. And why do we use the same MEM_BUS address > > for > > all PCIe hose? I know they are not conflicting, but is it necessary? > > (I hope Becky and Kumar still follow this mailing list) > > I dug out an old commit 4c78d4a6c01621721b732418e1c6da684a56bbb1 by Becky > Bruce. > She believed overlapping the bus address for PCI controllers leaves more > space. > That's true. But we haven't use more than 512MB in u-boot. If we do need > more > space, we can easily move things around if we have PHYS_64BIT. If no > objection, > I'd like to change this back. I object. It's not about how much RAM is used in U-Boot; it's about how much memory the OS needs to bounce-buffer for DMA. The addresses set up by U-Boot should match what's in the device tree. Yes, on 85xx Linux reprograms the ATMU based on the device tree rather than trusting U-Boot, but that doesn't mean every OS does. -Scott ^ permalink raw reply [flat|nested] 7+ messages in thread
* [U-Boot] PCIE_MEM_BUS for Freescale SoC 2015-11-23 16:24 ` Scott Wood @ 2015-11-23 17:00 ` York Sun 2015-11-23 17:07 ` Scott Wood 0 siblings, 1 reply; 7+ messages in thread From: York Sun @ 2015-11-23 17:00 UTC (permalink / raw) To: u-boot On 11/23/2015 08:24 AM, Scott Wood wrote: > On Sat, 2015-11-21 at 15:29 -0800, York Sun wrote: >> >> On 11/21/2015 02:55 PM, York Sun wrote: >>> Roy, >>> >>> Do you remember the reason why we use different virtual memory address >>> from pci >>> bus address with 36-bit? For example >>> >>> include/configs/P1022DS.h-496-#define CONFIG_SYS_PCIE1_MEM_VIRT 0xc0000000 >>> include/configs/P1022DS.h-497-#ifdef CONFIG_PHYS_64BIT >>> include/configs/P1022DS.h:498:#define CONFIG_SYS_PCIE1_MEM_BUS 0xe0000000 >>> include/configs/P1022DS.h-499-#define CONFIG_SYS_PCIE1_MEM_PHYS >>> 0xc40000000ull >>> include/configs/P1022DS.h-500-#else >>> include/configs/P1022DS.h:501:#define CONFIG_SYS_PCIE1_MEM_BUS 0xc0000000 >>> include/configs/P1022DS.h-502-#define CONFIG_SYS_PCIE1_MEM_PHYS 0xc0000000 >>> include/configs/P1022DS.h-503-#endif >>> >>> As far as I can tell, the following is the mapping >>> >>> TLB: MEM_VIRT=>MEM_PHYS >>> PCI: MEM_BUS=>MEM_PHYS >>> LAW: MEM_PHYS=>pcie interface >>> >>> Being different for MEM_VIRT and MEM_BUS cause confusion. When I run "pci >>> header" command to show the BARs, I expect I can use "md" to access the >>> BAR >>> address. That's not the case if MEM_BUS is different from MEM_VIRT. >>> >>> I forget why we did this for 36-bit addressing. The MEM_VIRT is the same >>> as >>> MEM_BUS for 32-bit addressing. And why do we use the same MEM_BUS address >>> for >>> all PCIe hose? I know they are not conflicting, but is it necessary? >> >> (I hope Becky and Kumar still follow this mailing list) >> >> I dug out an old commit 4c78d4a6c01621721b732418e1c6da684a56bbb1 by Becky >> Bruce. >> She believed overlapping the bus address for PCI controllers leaves more >> space. >> That's true. But we haven't use more than 512MB in u-boot. If we do need >> more >> space, we can easily move things around if we have PHYS_64BIT. If no >> objection, >> I'd like to change this back. > > I object. It's not about how much RAM is used in U-Boot; it's about how much > memory the OS needs to bounce-buffer for DMA. The addresses set up by U-Boot > should match what's in the device tree. Yes, on 85xx Linux reprograms the > ATMU based on the device tree rather than trusting U-Boot, but that doesn't > mean every OS does. Isn't this backward? I mean the device tree should match u-boot, or u-boot should fix up the device tree, right? I checked several device tree files in kernel. We are not using more than 512MB for each PCI. What's the benefit of using the same address 0xe0000000? York ^ permalink raw reply [flat|nested] 7+ messages in thread
* [U-Boot] PCIE_MEM_BUS for Freescale SoC 2015-11-23 17:00 ` York Sun @ 2015-11-23 17:07 ` Scott Wood 2015-11-23 17:11 ` York Sun 2015-11-23 19:56 ` York Sun 0 siblings, 2 replies; 7+ messages in thread From: Scott Wood @ 2015-11-23 17:07 UTC (permalink / raw) To: u-boot On Mon, 2015-11-23 at 09:00 -0800, York Sun wrote: > > On 11/23/2015 08:24 AM, Scott Wood wrote: > > On Sat, 2015-11-21 at 15:29 -0800, York Sun wrote: > > > > > > On 11/21/2015 02:55 PM, York Sun wrote: > > > > Roy, > > > > > > > > Do you remember the reason why we use different virtual memory address > > > > from pci > > > > bus address with 36-bit? For example > > > > > > > > include/configs/P1022DS.h-496-#define CONFIG_SYS_PCIE1_MEM_VIRT > > > > 0xc0000000 > > > > include/configs/P1022DS.h-497-#ifdef CONFIG_PHYS_64BIT > > > > include/configs/P1022DS.h:498:#define CONFIG_SYS_PCIE1_MEM_BUS > > > > 0xe0000000 > > > > include/configs/P1022DS.h-499-#define CONFIG_SYS_PCIE1_MEM_PHYS > > > > 0xc40000000ull > > > > include/configs/P1022DS.h-500-#else > > > > include/configs/P1022DS.h:501:#define CONFIG_SYS_PCIE1_MEM_BUS > > > > 0xc0000000 > > > > include/configs/P1022DS.h-502-#define CONFIG_SYS_PCIE1_MEM_PHYS > > > > 0xc0000000 > > > > include/configs/P1022DS.h-503-#endif > > > > > > > > As far as I can tell, the following is the mapping > > > > > > > > TLB: MEM_VIRT=>MEM_PHYS > > > > PCI: MEM_BUS=>MEM_PHYS > > > > LAW: MEM_PHYS=>pcie interface > > > > > > > > Being different for MEM_VIRT and MEM_BUS cause confusion. When I run > > > > "pci > > > > header" command to show the BARs, I expect I can use "md" to access > > > > the > > > > BAR > > > > address. That's not the case if MEM_BUS is different from MEM_VIRT. > > > > > > > > I forget why we did this for 36-bit addressing. The MEM_VIRT is the > > > > same > > > > as > > > > MEM_BUS for 32-bit addressing. And why do we use the same MEM_BUS > > > > address > > > > for > > > > all PCIe hose? I know they are not conflicting, but is it necessary? > > > > > > (I hope Becky and Kumar still follow this mailing list) > > > > > > I dug out an old commit 4c78d4a6c01621721b732418e1c6da684a56bbb1 by > > > Becky > > > Bruce. > > > She believed overlapping the bus address for PCI controllers leaves more > > > space. > > > That's true. But we haven't use more than 512MB in u-boot. If we do need > > > more > > > space, we can easily move things around if we have PHYS_64BIT. If no > > > objection, > > > I'd like to change this back. > > > > I object. It's not about how much RAM is used in U-Boot; it's about how > > much > > memory the OS needs to bounce-buffer for DMA. The addresses set up by U > > -Boot > > should match what's in the device tree. Yes, on 85xx Linux reprograms the > > ATMU based on the device tree rather than trusting U-Boot, but that > > doesn't > > mean every OS does. > > Isn't this backward? I mean the device tree should match u-boot, Yes, and I don't want to deal with the churn of these device trees changing. > or u-boot should fix up the device tree, right? In theory yes, but then you really are affecting Linux's usage. > I checked several device tree files in kernel. We are not using more than > 512MB > for each PCI. What's the benefit of using the same address 0xe0000000? Reducing the amount of RAM whose address matches a PCI bus address and thus can't be used for PCI DMA. -Scott ^ permalink raw reply [flat|nested] 7+ messages in thread
* [U-Boot] PCIE_MEM_BUS for Freescale SoC 2015-11-23 17:07 ` Scott Wood @ 2015-11-23 17:11 ` York Sun 2015-11-23 19:56 ` York Sun 1 sibling, 0 replies; 7+ messages in thread From: York Sun @ 2015-11-23 17:11 UTC (permalink / raw) To: u-boot On 11/23/2015 09:07 AM, Scott Wood wrote: > On Mon, 2015-11-23 at 09:00 -0800, York Sun wrote: >> >> On 11/23/2015 08:24 AM, Scott Wood wrote: >>> On Sat, 2015-11-21 at 15:29 -0800, York Sun wrote: >>>> >>>> On 11/21/2015 02:55 PM, York Sun wrote: >>>>> Roy, >>>>> >>>>> Do you remember the reason why we use different virtual memory address >>>>> from pci >>>>> bus address with 36-bit? For example >>>>> >>>>> include/configs/P1022DS.h-496-#define CONFIG_SYS_PCIE1_MEM_VIRT >>>>> 0xc0000000 >>>>> include/configs/P1022DS.h-497-#ifdef CONFIG_PHYS_64BIT >>>>> include/configs/P1022DS.h:498:#define CONFIG_SYS_PCIE1_MEM_BUS >>>>> 0xe0000000 >>>>> include/configs/P1022DS.h-499-#define CONFIG_SYS_PCIE1_MEM_PHYS >>>>> 0xc40000000ull >>>>> include/configs/P1022DS.h-500-#else >>>>> include/configs/P1022DS.h:501:#define CONFIG_SYS_PCIE1_MEM_BUS >>>>> 0xc0000000 >>>>> include/configs/P1022DS.h-502-#define CONFIG_SYS_PCIE1_MEM_PHYS >>>>> 0xc0000000 >>>>> include/configs/P1022DS.h-503-#endif >>>>> >>>>> As far as I can tell, the following is the mapping >>>>> >>>>> TLB: MEM_VIRT=>MEM_PHYS >>>>> PCI: MEM_BUS=>MEM_PHYS >>>>> LAW: MEM_PHYS=>pcie interface >>>>> >>>>> Being different for MEM_VIRT and MEM_BUS cause confusion. When I run >>>>> "pci >>>>> header" command to show the BARs, I expect I can use "md" to access >>>>> the >>>>> BAR >>>>> address. That's not the case if MEM_BUS is different from MEM_VIRT. >>>>> >>>>> I forget why we did this for 36-bit addressing. The MEM_VIRT is the >>>>> same >>>>> as >>>>> MEM_BUS for 32-bit addressing. And why do we use the same MEM_BUS >>>>> address >>>>> for >>>>> all PCIe hose? I know they are not conflicting, but is it necessary? >>>> >>>> (I hope Becky and Kumar still follow this mailing list) >>>> >>>> I dug out an old commit 4c78d4a6c01621721b732418e1c6da684a56bbb1 by >>>> Becky >>>> Bruce. >>>> She believed overlapping the bus address for PCI controllers leaves more >>>> space. >>>> That's true. But we haven't use more than 512MB in u-boot. If we do need >>>> more >>>> space, we can easily move things around if we have PHYS_64BIT. If no >>>> objection, >>>> I'd like to change this back. >>> >>> I object. It's not about how much RAM is used in U-Boot; it's about how >>> much >>> memory the OS needs to bounce-buffer for DMA. The addresses set up by U >>> -Boot >>> should match what's in the device tree. Yes, on 85xx Linux reprograms the >>> ATMU based on the device tree rather than trusting U-Boot, but that >>> doesn't >>> mean every OS does. >> >> Isn't this backward? I mean the device tree should match u-boot, > > Yes, and I don't want to deal with the churn of these device trees changing. > >> or u-boot should fix up the device tree, right? > > In theory yes, but then you really are affecting Linux's usage. > >> I checked several device tree files in kernel. We are not using more than >> 512MB >> for each PCI. What's the benefit of using the same address 0xe0000000? > > Reducing the amount of RAM whose address matches a PCI bus address and thus > can't be used for PCI DMA. > That sounds like a good reason. OK, I will defer my patch until we have a stronger need to access the BARs under u-boot, or something else changes. York ^ permalink raw reply [flat|nested] 7+ messages in thread
* [U-Boot] PCIE_MEM_BUS for Freescale SoC 2015-11-23 17:07 ` Scott Wood 2015-11-23 17:11 ` York Sun @ 2015-11-23 19:56 ` York Sun 1 sibling, 0 replies; 7+ messages in thread From: York Sun @ 2015-11-23 19:56 UTC (permalink / raw) To: u-boot On 11/23/2015 09:07 AM, Scott Wood wrote: > On Mon, 2015-11-23 at 09:00 -0800, York Sun wrote: >> >> On 11/23/2015 08:24 AM, Scott Wood wrote: >>> On Sat, 2015-11-21 at 15:29 -0800, York Sun wrote: >>>> >>>> On 11/21/2015 02:55 PM, York Sun wrote: >>>>> Roy, >>>>> >>>>> Do you remember the reason why we use different virtual memory address >>>>> from pci >>>>> bus address with 36-bit? For example >>>>> >>>>> include/configs/P1022DS.h-496-#define CONFIG_SYS_PCIE1_MEM_VIRT >>>>> 0xc0000000 >>>>> include/configs/P1022DS.h-497-#ifdef CONFIG_PHYS_64BIT >>>>> include/configs/P1022DS.h:498:#define CONFIG_SYS_PCIE1_MEM_BUS >>>>> 0xe0000000 >>>>> include/configs/P1022DS.h-499-#define CONFIG_SYS_PCIE1_MEM_PHYS >>>>> 0xc40000000ull >>>>> include/configs/P1022DS.h-500-#else >>>>> include/configs/P1022DS.h:501:#define CONFIG_SYS_PCIE1_MEM_BUS >>>>> 0xc0000000 >>>>> include/configs/P1022DS.h-502-#define CONFIG_SYS_PCIE1_MEM_PHYS >>>>> 0xc0000000 >>>>> include/configs/P1022DS.h-503-#endif >>>>> >>>>> As far as I can tell, the following is the mapping >>>>> >>>>> TLB: MEM_VIRT=>MEM_PHYS >>>>> PCI: MEM_BUS=>MEM_PHYS >>>>> LAW: MEM_PHYS=>pcie interface >>>>> >>>>> Being different for MEM_VIRT and MEM_BUS cause confusion. When I run >>>>> "pci >>>>> header" command to show the BARs, I expect I can use "md" to access >>>>> the >>>>> BAR >>>>> address. That's not the case if MEM_BUS is different from MEM_VIRT. >>>>> >>>>> I forget why we did this for 36-bit addressing. The MEM_VIRT is the >>>>> same >>>>> as >>>>> MEM_BUS for 32-bit addressing. And why do we use the same MEM_BUS >>>>> address >>>>> for >>>>> all PCIe hose? I know they are not conflicting, but is it necessary? >>>> >>>> (I hope Becky and Kumar still follow this mailing list) >>>> >>>> I dug out an old commit 4c78d4a6c01621721b732418e1c6da684a56bbb1 by >>>> Becky >>>> Bruce. >>>> She believed overlapping the bus address for PCI controllers leaves more >>>> space. >>>> That's true. But we haven't use more than 512MB in u-boot. If we do need >>>> more >>>> space, we can easily move things around if we have PHYS_64BIT. If no >>>> objection, >>>> I'd like to change this back. >>> >>> I object. It's not about how much RAM is used in U-Boot; it's about how >>> much >>> memory the OS needs to bounce-buffer for DMA. The addresses set up by U >>> -Boot >>> should match what's in the device tree. Yes, on 85xx Linux reprograms the >>> ATMU based on the device tree rather than trusting U-Boot, but that >>> doesn't >>> mean every OS does. >> >> Isn't this backward? I mean the device tree should match u-boot, > > Yes, and I don't want to deal with the churn of these device trees changing. > >> or u-boot should fix up the device tree, right? > > In theory yes, but then you really are affecting Linux's usage. > >> I checked several device tree files in kernel. We are not using more than >> 512MB >> for each PCI. What's the benefit of using the same address 0xe0000000? > > Reducing the amount of RAM whose address matches a PCI bus address and thus > can't be used for PCI DMA. > Another thought, Linux scans and enumerates PCIe regardless U-boot. The BARs are setup according to device tree. So setting done u-boot is overwritten. And, reusing 0xe0000000 for all PCI(e) controllers only applies to PHYS_64BIT in current code. Isn't address space more crowed for 32-bit? Regarding the ATMU, if OS relies on u-boot to setup correct ATMU, it should have correct device tree as well. Those being said, I can leave the code as is. It is not broken badly, but confusing if one tries to access BAR space manually. York ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2015-11-23 19:56 UTC | newest] Thread overview: 7+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2015-11-21 22:55 [U-Boot] PCIE_MEM_BUS for Freescale SoC York Sun 2015-11-21 23:29 ` York Sun 2015-11-23 16:24 ` Scott Wood 2015-11-23 17:00 ` York Sun 2015-11-23 17:07 ` Scott Wood 2015-11-23 17:11 ` York Sun 2015-11-23 19:56 ` York Sun
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox