[PATCH 1/5] ARM: use phys_addr_t for DMA zone sizes

[PATCH 1/5] ARM: use phys_addr_t for DMA zone sizes

[Rob Herring]

From: Rob Herring <rob.herring@calxeda.com>

In order to specify a DMA zone size of 4GB on LPAE systems, the sizes need
to be 64-bit. So make machine_desc.dma_zone_size and arm_dma_zone_size be
phys_addr_t instead of unsigned long.

Signed-off-by: Rob Herring <rob.herring@calxeda.com>
---
 arch/arm/include/asm/mach/arch.h | 2 +-
 arch/arm/mm/init.c               | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm/include/asm/mach/arch.h b/arch/arm/include/asm/mach/arch.h
index 441efc4..d91b168 100644
--- a/arch/arm/include/asm/mach/arch.h
+++ b/arch/arm/include/asm/mach/arch.h
@@ -35,7 +35,7 @@ struct machine_desc {
 	unsigned int		nr_irqs;	/* number of IRQs */
 
 #ifdef CONFIG_ZONE_DMA
-	unsigned long		dma_zone_size;	/* size of DMA-able area */
+	phys_addr_t		dma_zone_size;	/* size of DMA-able area */
 #endif
 
 	unsigned int		video_start;	/* start of video RAM	*/
diff --git a/arch/arm/mm/init.c b/arch/arm/mm/init.c
index 15225d8..c0bb66e 100644
--- a/arch/arm/mm/init.c
+++ b/arch/arm/mm/init.c
@@ -207,7 +207,7 @@ static void __init arm_bootmem_init(unsigned long start_pfn,
 
 #ifdef CONFIG_ZONE_DMA
 
-unsigned long arm_dma_zone_size __read_mostly;
+phys_addr_t arm_dma_zone_size __read_mostly;
 EXPORT_SYMBOL(arm_dma_zone_size);
 
 /*
-- 
1.8.1.2

[PATCH 2/5] ARM: make dma_addr_t 64-bit on LPAE

[Rob Herring]

From: Rob Herring <rob.herring@calxeda.com>

LPAE systems may have 64-bit capable DMA, so dma_addr_t needs to be
64-bit.

Signed-off-by: Rob Herring <rob.herring@calxeda.com>
---
 arch/arm/mm/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm/mm/Kconfig b/arch/arm/mm/Kconfig
index 6cacdc8..9f6f3d7 100644
--- a/arch/arm/mm/Kconfig
+++ b/arch/arm/mm/Kconfig
@@ -625,7 +625,7 @@ config ARCH_PHYS_ADDR_T_64BIT
 	def_bool ARM_LPAE
 
 config ARCH_DMA_ADDR_T_64BIT
-	bool
+	def_bool ARM_LPAE
 
 config ARM_THUMB
 	bool "Support Thumb user binaries" if !CPU_THUMBONLY
-- 
1.8.1.2

[PATCH 3/5] ARM: highbank: enable DMA zone for LPAE

[Rob Herring]

From: Rob Herring <rob.herring@calxeda.com>

Some devices are restricted to 32-bit DMA. Thus the platform dma_zone_size
needs to be set. Otherwise dma-mapping code is complaining, e.g.

calxedaxgmac fff50000.ethernet: coherent DMA mask 0xffffffff is smaller
than system GFP_DMA mask 0xffffffffffffffff

Signed-off-by: Andreas Herrmann <andreas.herrmann@calxeda.com>
Signed-off-by: Rob Herring <rob.herring@calxeda.com>
---
 arch/arm/mach-highbank/Kconfig    | 1 +
 arch/arm/mach-highbank/highbank.c | 3 +++
 2 files changed, 4 insertions(+)

diff --git a/arch/arm/mach-highbank/Kconfig b/arch/arm/mach-highbank/Kconfig
index cd9fcb1..efe4961 100644
--- a/arch/arm/mach-highbank/Kconfig
+++ b/arch/arm/mach-highbank/Kconfig
@@ -18,3 +18,4 @@ config ARCH_HIGHBANK
 	select PL320_MBOX
 	select SPARSE_IRQ
 	select USE_OF
+	select ZONE_DMA if ARM_LPAE
diff --git a/arch/arm/mach-highbank/highbank.c b/arch/arm/mach-highbank/highbank.c
index 8881579..1894dcf 100644
--- a/arch/arm/mach-highbank/highbank.c
+++ b/arch/arm/mach-highbank/highbank.c
@@ -176,6 +176,9 @@ static const char *highbank_match[] __initconst = {
 };
 
 DT_MACHINE_START(HIGHBANK, "Highbank")
+#ifdef CONFIG_ZONE_DMA
+	.dma_zone_size	= (4ULL * SZ_1G),
+#endif
 	.smp		= smp_ops(highbank_smp_ops),
 	.init_irq	= highbank_init_irq,
 	.init_time	= highbank_timer_init,
-- 
1.8.1.2

[PATCH 4/5] ARM: highbank: select ARCH_HAS_HOLES_MEMORYMODEL

[Rob Herring]

From: Rob Herring <rob.herring@calxeda.com>

On Midway, the first 4G of memory starts at 0 and the rest of memory
(4GB+) starts at 0x2_00000000, so we need to enable pfn_valid checks
by selecting ARCH_HAS_HOLES_MEMORYMODEL.

Signed-off-by: Rob Herring <rob.herring@calxeda.com>
---
 arch/arm/mach-highbank/Kconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm/mach-highbank/Kconfig b/arch/arm/mach-highbank/Kconfig
index efe4961..8b2948f 100644
--- a/arch/arm/mach-highbank/Kconfig
+++ b/arch/arm/mach-highbank/Kconfig
@@ -1,6 +1,7 @@
 config ARCH_HIGHBANK
 	bool "Calxeda ECX-1000/2000 (Highbank/Midway)" if ARCH_MULTI_V7
 	select ARCH_HAS_CPUFREQ
+	select ARCH_HAS_HOLES_MEMORYMODEL
 	select ARCH_HAS_OPP
 	select ARCH_WANT_OPTIONAL_GPIOLIB
 	select ARM_AMBA
-- 
1.8.1.2

[PATCH 5/5] ARM: highbank: select required errata work-arounds

[Rob Herring]

From: Rob Herring <rob.herring@calxeda.com>

Ensure necessary errata work-arounds are always enabled for Highbank
and Midway platforms. Highbank requires 764369 and 764369. Midway requires
798181, but only the first half of the work-around (no IPI). Support for
skipping the IPI is introduced in another commit.

Signed-off-by: Rob Herring <rob.herring@calxeda.com>
---
 arch/arm/mach-highbank/Kconfig | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/arch/arm/mach-highbank/Kconfig b/arch/arm/mach-highbank/Kconfig
index 8b2948f..386835e 100644
--- a/arch/arm/mach-highbank/Kconfig
+++ b/arch/arm/mach-highbank/Kconfig
@@ -5,6 +5,9 @@ config ARCH_HIGHBANK
 	select ARCH_HAS_OPP
 	select ARCH_WANT_OPTIONAL_GPIOLIB
 	select ARM_AMBA
+	select ARM_ERRATA_764369
+	select ARM_ERRATA_775420
+	select ARM_ERRATA_798181
 	select ARM_GIC
 	select ARM_TIMER_SP804
 	select CACHE_L2X0
-- 
1.8.1.2

[PATCH 3/5] ARM: highbank: enable DMA zone for LPAE

[Russell King - ARM Linux]

On Mon, Aug 05, 2013 at 05:50:07PM -0500, Rob Herring wrote:
> From: Rob Herring <rob.herring@calxeda.com>
> 
> Some devices are restricted to 32-bit DMA. Thus the platform dma_zone_size
> needs to be set. Otherwise dma-mapping code is complaining, e.g.
> 
> calxedaxgmac fff50000.ethernet: coherent DMA mask 0xffffffff is smaller
> than system GFP_DMA mask 0xffffffffffffffff

Note that the patches I posted last week will fix that in a much nicer
way, as we will only error if we actually have sufficient memory that
the condition becomes problematical.

The other problem here is that we need to get better separation between
what's a bus address and what's a physical address.

[PATCH 2/5] ARM: make dma_addr_t 64-bit on LPAE

[Russell King - ARM Linux]

On Mon, Aug 05, 2013 at 05:50:06PM -0500, Rob Herring wrote:
> From: Rob Herring <rob.herring@calxeda.com>
> 
> LPAE systems may have 64-bit capable DMA, so dma_addr_t needs to be
> 64-bit.

The question I put to you here is: any can your DMA controllers produce
more than 32-bits of address?

Yes, physical memory may be mapped at addresses greater than 4GB phys.
That doesn't mean that dma_addr_t also becomes 64-bit.

dma_addr_t is the bus address - the address you program into the DMA
controller.  If you're only ever writing it into a 32-bit address
register then you can only address 32-bits of memory, even though that
may be in the physical address range of 4GB-8GB.  In such cases where
that applies to all DMA controllers in the system, dma_addr_t should
still be 32-bit.

This is where it's broken to think that DMA addresses are the same as
physical addresses.  They aren't.

[PATCH 2/5] ARM: make dma_addr_t 64-bit on LPAE

[Rob Herring]

On Mon, Aug 5, 2013 at 7:15 PM, Russell King - ARM Linux
<linux@arm.linux.org.uk> wrote:
> On Mon, Aug 05, 2013 at 05:50:06PM -0500, Rob Herring wrote:
>> From: Rob Herring <rob.herring@calxeda.com>
>>
>> LPAE systems may have 64-bit capable DMA, so dma_addr_t needs to be
>> 64-bit.
>
> The question I put to you here is: any can your DMA controllers produce
> more than 32-bits of address?

Yes. SATA is 64-bit. And the ones that are 32-bit only have an IOMMU
in front of them.

> Yes, physical memory may be mapped at addresses greater than 4GB phys.
> That doesn't mean that dma_addr_t also becomes 64-bit.
>
> dma_addr_t is the bus address - the address you program into the DMA
> controller.  If you're only ever writing it into a 32-bit address
> register then you can only address 32-bits of memory, even though that
> may be in the physical address range of 4GB-8GB.  In such cases where
> that applies to all DMA controllers in the system, dma_addr_t should
> still be 32-bit.
>
> This is where it's broken to think that DMA addresses are the same as
> physical addresses.  They aren't.

Understood, but it's not that clear once IOMMUs are in the mix. While
IOMMU drivers will return 32-bit addresses at the DMA API level, I'd
think internally they will need 64-bit addresses.

I'm fine with just selecting this in ARCH_HIGHBANK, but the machine
kconfig selects are getting rather long. We'll be cleaning those up at
some point if we keep selecting everything at the machine level.

Rob

[PATCH 3/5] ARM: highbank: enable DMA zone for LPAE

[Rob Herring]

On Mon, Aug 5, 2013 at 7:11 PM, Russell King - ARM Linux
<linux@arm.linux.org.uk> wrote:
> On Mon, Aug 05, 2013 at 05:50:07PM -0500, Rob Herring wrote:
>> From: Rob Herring <rob.herring@calxeda.com>
>>
>> Some devices are restricted to 32-bit DMA. Thus the platform dma_zone_size
>> needs to be set. Otherwise dma-mapping code is complaining, e.g.
>>
>> calxedaxgmac fff50000.ethernet: coherent DMA mask 0xffffffff is smaller
>> than system GFP_DMA mask 0xffffffffffffffff
>
> Note that the patches I posted last week will fix that in a much nicer
> way, as we will only error if we actually have sufficient memory that
> the condition becomes problematical.

But in the Midway case it is problematic as we do have more than 4GB
with some masters being 32-bit. DMA zone is still the right way to
limit allocations to less than 4GB dma address, right?

> The other problem here is that we need to get better separation between
> what's a bus address and what's a physical address.

Right, but for Midway they are actually the same. There is no h/w
translation of 32-bit bus addresses to a >4GB physical address other
than with an IOMMU.

Rob

[PATCH 2/5] ARM: make dma_addr_t 64-bit on LPAE

[Russell King - ARM Linux]

On Mon, Aug 05, 2013 at 08:59:18PM -0500, Rob Herring wrote:
> On Mon, Aug 5, 2013 at 7:15 PM, Russell King - ARM Linux
> <linux@arm.linux.org.uk> wrote:
> > On Mon, Aug 05, 2013 at 05:50:06PM -0500, Rob Herring wrote:
> >> From: Rob Herring <rob.herring@calxeda.com>
> >>
> >> LPAE systems may have 64-bit capable DMA, so dma_addr_t needs to be
> >> 64-bit.
> >
> > The question I put to you here is: any can your DMA controllers produce
> > more than 32-bits of address?
> 
> Yes. SATA is 64-bit. And the ones that are 32-bit only have an IOMMU
> in front of them.
> 
> > Yes, physical memory may be mapped at addresses greater than 4GB phys.
> > That doesn't mean that dma_addr_t also becomes 64-bit.
> >
> > dma_addr_t is the bus address - the address you program into the DMA
> > controller.  If you're only ever writing it into a 32-bit address
> > register then you can only address 32-bits of memory, even though that
> > may be in the physical address range of 4GB-8GB.  In such cases where
> > that applies to all DMA controllers in the system, dma_addr_t should
> > still be 32-bit.
> >
> > This is where it's broken to think that DMA addresses are the same as
> > physical addresses.  They aren't.
> 
> Understood, but it's not that clear once IOMMUs are in the mix. While
> IOMMU drivers will return 32-bit addresses at the DMA API level, I'd
> think internally they will need 64-bit addresses.
> 
> I'm fine with just selecting this in ARCH_HIGHBANK, but the machine
> kconfig selects are getting rather long. We'll be cleaning those up at
> some point if we keep selecting everything at the machine level.

Given that this breaks PL08x, I think you need to drop this change.  We
need the root problem fixed, as I have done in my dma-masks patches,
rather than endlessly plastering around this.

If we don't solve the root problem first, this issue is just going to get
really crappy with workaround on top of more workarounds, and it's going
to be impossible to undo all that crap later.