[PATCH v2] arm64: Allow for different DMA and CPU bus offsets

[PATCH v2] arm64: Allow for different DMA and CPU bus offsets

[Alexander Graf]

On arm64, all SoCs we supported so far either have an IOMMU or have bus
addresses equal to CPU addresses.

However, with the Raspberry Pi 3 coming up, this is no longer true. To
allow DMA to work with an AArch64 kernel on those devices, let's allow
devices to have DMA offsets again.

Signed-off-by: Alexander Graf <agraf@suse.de>

---

This patch may conflict with another patch titled "swiotlb: prefix dma_to_phys
and phys_to_dma functions" which is in flight, but hasn't seen an update since
March.

Since this patch is very small and isolated to arm64, I'd prefer to keep them
separate rather than combine them. So if the other patch gets accepted first,
I'm happy to rebase this patch on top of a topic branch that would address the
rename.

v1 -> v2:

  - Use PAGE_SHIFT explicitly rather than __pfn_to_phys and __phys_to_pfn
---
 arch/arm64/include/asm/dma-mapping.h | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/include/asm/dma-mapping.h b/arch/arm64/include/asm/dma-mapping.h
index ba437f0..c14e9cd 100644
--- a/arch/arm64/include/asm/dma-mapping.h
+++ b/arch/arm64/include/asm/dma-mapping.h
@@ -66,12 +66,16 @@ static inline bool is_device_dma_coherent(struct device *dev)
 
 static inline dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr)
 {
-	return (dma_addr_t)paddr;
+	dma_addr_t dev_addr = (dma_addr_t)paddr;
+
+	return dev_addr - ((dma_addr_t)dev->dma_pfn_offset << PAGE_SHIFT);
 }
 
 static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t dev_addr)
 {
-	return (phys_addr_t)dev_addr;
+	phys_addr_t paddr = (phys_addr_t)dev_addr;
+
+	return paddr + ((phys_addr_t)dev->dma_pfn_offset << PAGE_SHIFT);
 }
 
 static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size)
@@ -86,5 +90,14 @@ static inline void dma_mark_clean(void *addr, size_t size)
 {
 }
 
+/* Override for dma_max_pfn() */
+static inline unsigned long dma_max_pfn(struct device *dev)
+{
+	dma_addr_t dma_max = (dma_addr_t)*dev->dma_mask;
+
+	return (ulong)dma_to_phys(dev, dma_max) >> PAGE_SHIFT;
+}
+#define dma_max_pfn(dev) dma_max_pfn(dev)
+
 #endif	/* __KERNEL__ */
 #endif	/* __ASM_DMA_MAPPING_H */
-- 
1.8.5.6

[PATCH v2] arm64: Allow for different DMA and CPU bus offsets

[Arnd Bergmann]

On Wednesday 18 May 2016 14:53:30 Alexander Graf wrote:
> On arm64, all SoCs we supported so far either have an IOMMU or have bus
> addresses equal to CPU addresses.
> 
> However, with the Raspberry Pi 3 coming up, this is no longer true. To
> allow DMA to work with an AArch64 kernel on those devices, let's allow
> devices to have DMA offsets again.
> 
> Signed-off-by: Alexander Graf <agraf@suse.de>
> 
> 

Acked-by: Arnd Bergmann <arnd@arndb.de>

[PATCH v2] arm64: Allow for different DMA and CPU bus offsets

[Catalin Marinas]

On Wed, May 18, 2016 at 02:53:30PM +0200, Alexander Graf wrote:
> On arm64, all SoCs we supported so far either have an IOMMU or have bus
> addresses equal to CPU addresses.
> 
> However, with the Raspberry Pi 3 coming up, this is no longer true. To
> allow DMA to work with an AArch64 kernel on those devices, let's allow
> devices to have DMA offsets again.
> 
> Signed-off-by: Alexander Graf <agraf@suse.de>

We were aware about this issue but we've been waiting for an actual use
case. I'll queue this for 4.8 (given that RPi3 is not supported by the
arm64 kernel, I don't see the rush to merge it earlier).

Thanks.

-- 
Catalin

[PATCH v2] arm64: Allow for different DMA and CPU bus offsets

[Alexander Graf]

On 05/18/2016 03:27 PM, Catalin Marinas wrote:
> On Wed, May 18, 2016 at 02:53:30PM +0200, Alexander Graf wrote:
>> On arm64, all SoCs we supported so far either have an IOMMU or have bus
>> addresses equal to CPU addresses.
>>
>> However, with the Raspberry Pi 3 coming up, this is no longer true. To
>> allow DMA to work with an AArch64 kernel on those devices, let's allow
>> devices to have DMA offsets again.
>>
>> Signed-off-by: Alexander Graf <agraf@suse.de>
> We were aware about this issue but we've been waiting for an actual use
> case. I'll queue this for 4.8 (given that RPi3 is not supported by the
> arm64 kernel, I don't see the rush to merge it earlier).

Sure, works for me. That way Eric can try to push the other few bits 
necessary to run an arm64 kernel on the RPi3 towards 4.8 as well :).


Alex

[PATCH v2] arm64: Allow for different DMA and CPU bus offsets

[Catalin Marinas]

On Wed, May 18, 2016 at 03:49:17PM +0200, Alexander Graf wrote:
> On 05/18/2016 03:27 PM, Catalin Marinas wrote:
> >On Wed, May 18, 2016 at 02:53:30PM +0200, Alexander Graf wrote:
> >>On arm64, all SoCs we supported so far either have an IOMMU or have bus
> >>addresses equal to CPU addresses.
> >>
> >>However, with the Raspberry Pi 3 coming up, this is no longer true. To
> >>allow DMA to work with an AArch64 kernel on those devices, let's allow
> >>devices to have DMA offsets again.
> >>
> >>Signed-off-by: Alexander Graf <agraf@suse.de>
> >We were aware about this issue but we've been waiting for an actual use
> >case. I'll queue this for 4.8 (given that RPi3 is not supported by the
> >arm64 kernel, I don't see the rush to merge it earlier).
> 
> Sure, works for me. That way Eric can try to push the other few bits
> necessary to run an arm64 kernel on the RPi3 towards 4.8 as well :).

As long as the firmware is PSCI capable ;)

-- 
Catalin

[PATCH v2] arm64: Allow for different DMA and CPU bus offsets

[Alexander Graf]

On 05/18/2016 04:26 PM, Catalin Marinas wrote:
> On Wed, May 18, 2016 at 03:49:17PM +0200, Alexander Graf wrote:
>> On 05/18/2016 03:27 PM, Catalin Marinas wrote:
>>> On Wed, May 18, 2016 at 02:53:30PM +0200, Alexander Graf wrote:
>>>> On arm64, all SoCs we supported so far either have an IOMMU or have bus
>>>> addresses equal to CPU addresses.
>>>>
>>>> However, with the Raspberry Pi 3 coming up, this is no longer true. To
>>>> allow DMA to work with an AArch64 kernel on those devices, let's allow
>>>> devices to have DMA offsets again.
>>>>
>>>> Signed-off-by: Alexander Graf <agraf@suse.de>
>>> We were aware about this issue but we've been waiting for an actual use
>>> case. I'll queue this for 4.8 (given that RPi3 is not supported by the
>>> arm64 kernel, I don't see the rush to merge it earlier).
>> Sure, works for me. That way Eric can try to push the other few bits
>> necessary to run an arm64 kernel on the RPi3 towards 4.8 as well :).
> As long as the firmware is PSCI capable ;)

So spin tables won't work for you? IIRC we don't own EL3 on there.


Alex

[PATCH v2] arm64: Allow for different DMA and CPU bus offsets

[Catalin Marinas]

On Wed, May 18, 2016 at 04:34:12PM +0200, Alexander Graf wrote:
> On 05/18/2016 04:26 PM, Catalin Marinas wrote:
> >On Wed, May 18, 2016 at 03:49:17PM +0200, Alexander Graf wrote:
> >>On 05/18/2016 03:27 PM, Catalin Marinas wrote:
> >>>On Wed, May 18, 2016 at 02:53:30PM +0200, Alexander Graf wrote:
> >>>>On arm64, all SoCs we supported so far either have an IOMMU or have bus
> >>>>addresses equal to CPU addresses.
> >>>>
> >>>>However, with the Raspberry Pi 3 coming up, this is no longer true. To
> >>>>allow DMA to work with an AArch64 kernel on those devices, let's allow
> >>>>devices to have DMA offsets again.
> >>>>
> >>>>Signed-off-by: Alexander Graf <agraf@suse.de>
> >>>We were aware about this issue but we've been waiting for an actual use
> >>>case. I'll queue this for 4.8 (given that RPi3 is not supported by the
> >>>arm64 kernel, I don't see the rush to merge it earlier).
> >>Sure, works for me. That way Eric can try to push the other few bits
> >>necessary to run an arm64 kernel on the RPi3 towards 4.8 as well :).
> >As long as the firmware is PSCI capable ;)
> 
> So spin tables won't work for you?

They only work for booting, not cpuidle or hotplug.

> IIRC we don't own EL3 on there.

Then how do you switch to AArch64?

-- 
Catalin

[PATCH v2] arm64: Allow for different DMA and CPU bus offsets

[Alexander Graf]

On 05/18/2016 05:19 PM, Catalin Marinas wrote:
> On Wed, May 18, 2016 at 04:34:12PM +0200, Alexander Graf wrote:
>> On 05/18/2016 04:26 PM, Catalin Marinas wrote:
>>> On Wed, May 18, 2016 at 03:49:17PM +0200, Alexander Graf wrote:
>>>> On 05/18/2016 03:27 PM, Catalin Marinas wrote:
>>>>> On Wed, May 18, 2016 at 02:53:30PM +0200, Alexander Graf wrote:
>>>>>> On arm64, all SoCs we supported so far either have an IOMMU or have bus
>>>>>> addresses equal to CPU addresses.
>>>>>>
>>>>>> However, with the Raspberry Pi 3 coming up, this is no longer true. To
>>>>>> allow DMA to work with an AArch64 kernel on those devices, let's allow
>>>>>> devices to have DMA offsets again.
>>>>>>
>>>>>> Signed-off-by: Alexander Graf <agraf@suse.de>
>>>>> We were aware about this issue but we've been waiting for an actual use
>>>>> case. I'll queue this for 4.8 (given that RPi3 is not supported by the
>>>>> arm64 kernel, I don't see the rush to merge it earlier).
>>>> Sure, works for me. That way Eric can try to push the other few bits
>>>> necessary to run an arm64 kernel on the RPi3 towards 4.8 as well :).
>>> As long as the firmware is PSCI capable ;)
>> So spin tables won't work for you?
> They only work for booting, not cpuidle or hotplug.

Well, booting is a good step forward I'd say :).

>
>> IIRC we don't own EL3 on there.
> Then how do you switch to AArch64?

You write into config.txt that you want to boot into AArch64 EL2 and the 
raspberry pi firmware takes care of the rest.


Alex

[PATCH v2] arm64: Allow for different DMA and CPU bus offsets

[Arnd Bergmann]

On Wednesday 18 May 2016 17:33:41 Alexander Graf wrote:
> >
> >> IIRC we don't own EL3 on there.
> > Then how do you switch to AArch64?
> 
> You write into config.txt that you want to boot into AArch64 EL2 and the 
> raspberry pi firmware takes care of the rest.
> 
> 

Have you tried telling the firmware that you'd like to boot into AArch64 EL3? ;-)

	Arnd

[PATCH v2] arm64: Allow for different DMA and CPU bus offsets

[Alexander Graf]

On 05/18/2016 06:08 PM, Arnd Bergmann wrote:
> On Wednesday 18 May 2016 17:33:41 Alexander Graf wrote:
>>>> IIRC we don't own EL3 on there.
>>> Then how do you switch to AArch64?
>> You write into config.txt that you want to boot into AArch64 EL2 and the
>> raspberry pi firmware takes care of the rest.
>>
>>
> Have you tried telling the firmware that you'd like to boot into AArch64 EL3? ;-)

Yes, but then you don't get any SMP initialization (so we'd have to redo 
that in U-Boot, or port ATF) and you lose the usual workflow on defining 
how the hat device tree overlays work.

Overall, I think it's an interesting goal and you're more than happy to 
be my guest, but I think that spin tables are good enough for the device 
for now :). At least we'd be able to move people away from 32bit code.


Alex

[PATCH v2] arm64: Allow for different DMA and CPU bus offsets

[Eric Anholt]

Catalin Marinas <catalin.marinas@arm.com> writes:

> On Wed, May 18, 2016 at 02:53:30PM +0200, Alexander Graf wrote:
>> On arm64, all SoCs we supported so far either have an IOMMU or have bus
>> addresses equal to CPU addresses.
>> 
>> However, with the Raspberry Pi 3 coming up, this is no longer true. To
>> allow DMA to work with an AArch64 kernel on those devices, let's allow
>> devices to have DMA offsets again.
>> 
>> Signed-off-by: Alexander Graf <agraf@suse.de>
>
> We were aware about this issue but we've been waiting for an actual use
> case. I'll queue this for 4.8 (given that RPi3 is not supported by the
> arm64 kernel, I don't see the rush to merge it earlier).

Sounds good to me.  I'm looking forward to getting the other pieces in
place for 4.8.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20160518/d6a194aa/attachment.sig>

[PATCH v2] arm64: Allow for different DMA and CPU bus offsets

[Catalin Marinas]

On Wed, May 18, 2016 at 02:27:41PM +0100, Catalin Marinas wrote:
> On Wed, May 18, 2016 at 02:53:30PM +0200, Alexander Graf wrote:
> > On arm64, all SoCs we supported so far either have an IOMMU or have bus
> > addresses equal to CPU addresses.
> > 
> > However, with the Raspberry Pi 3 coming up, this is no longer true. To
> > allow DMA to work with an AArch64 kernel on those devices, let's allow
> > devices to have DMA offsets again.
> > 
> > Signed-off-by: Alexander Graf <agraf@suse.de>
> 
> We were aware about this issue but we've been waiting for an actual use
> case. I'll queue this for 4.8 (given that RPi3 is not supported by the
> arm64 kernel, I don't see the rush to merge it earlier).

In case this patch goes in via the arm-soc tree (together with other
RPi3 patches):

Acked-by: Catalin Marinas <catalin.marinas@arm.com>

[PATCH v2] arm64: Allow for different DMA and CPU bus offsets

[Eric Anholt]

Catalin Marinas <catalin.marinas@arm.com> writes:

> On Wed, May 18, 2016 at 02:27:41PM +0100, Catalin Marinas wrote:
>> On Wed, May 18, 2016 at 02:53:30PM +0200, Alexander Graf wrote:
>> > On arm64, all SoCs we supported so far either have an IOMMU or have bus
>> > addresses equal to CPU addresses.
>> > 
>> > However, with the Raspberry Pi 3 coming up, this is no longer true. To
>> > allow DMA to work with an AArch64 kernel on those devices, let's allow
>> > devices to have DMA offsets again.
>> > 
>> > Signed-off-by: Alexander Graf <agraf@suse.de>
>> 
>> We were aware about this issue but we've been waiting for an actual use
>> case. I'll queue this for 4.8 (given that RPi3 is not supported by the
>> arm64 kernel, I don't see the rush to merge it earlier).
>
> In case this patch goes in via the arm-soc tree (together with other
> RPi3 patches):
>
> Acked-by: Catalin Marinas <catalin.marinas@arm.com>

Thanks!  I've pulled this into my for-next branch.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20160607/2c4ab37a/attachment-0001.sig>

[PATCH v2] arm64: Allow for different DMA and CPU bus offsets

[Sinan Kaya]

On 5/18/2016 8:53 AM, Alexander Graf wrote:
> ---
> 
> This patch may conflict with another patch titled "swiotlb: prefix dma_to_phys
> and phys_to_dma functions" which is in flight, but hasn't seen an update since
> March.

Thanks for the heads up. Due to missing API in DMA API, I couldn't make progress
with the series.

I need dma_map_resource API to get merged. After that one of the drivers
that actually uses dma_to_phys need to be changed to use dma_map_resource. 

Then the rest of the series can come.

-- 
Sinan Kaya
Qualcomm Technologies, Inc. on behalf of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project