[PATCH] Documentation: DMA-API: Clarify semantics of dma_set_mask_and_coherent

[PATCH] Documentation: DMA-API: Clarify semantics of dma_set_mask_and_coherent

[Punit Agrawal]

The dma mapping api howto gives the impression that using the
dma_set_mask_and_coherent (and related DMA APIs) will cause the kernel
to check all the components in the path from the device to memory for
addressing restrictions. In systems with address translations between
the device and memory (e.g., when using IOMMU), this implies that a
successful call to set set dma mask has checked the addressing
constraints of the intermediaries as well.

For the IOMMU drivers in the tree, the check is actually performed while
allocating the DMA buffer rather than when the DMA mask is
configured. For MMUs that do not support the full device addressing
capability, the allocations are made from a reduced address space.

Update the documentation to clarify that even though the call to
dma_set_mask_and_coherent succeeds, it may not be possible to use the
full addressing capability of the device.

Signed-off-by: Punit Agrawal <punit.agrawal@arm.com>
Cc: Jonathan Corbet <corbet@lwn.net>
---
 Documentation/DMA-API-HOWTO.txt | 39 +++++++++++++++++++++++----------------
 1 file changed, 23 insertions(+), 16 deletions(-)

diff --git a/Documentation/DMA-API-HOWTO.txt b/Documentation/DMA-API-HOWTO.txt
index 979228b..240d1ee 100644
--- a/Documentation/DMA-API-HOWTO.txt
+++ b/Documentation/DMA-API-HOWTO.txt
@@ -159,39 +159,46 @@ support 64-bit addressing (DAC) for all transactions.  And at least
 one platform (SGI SN2) requires 64-bit consistent allocations to
 operate correctly when the IO bus is in PCI-X mode.
 
-For correct operation, you must interrogate the kernel in your device
-probe routine to see if the DMA controller on the machine can properly
-support the DMA addressing limitation your device has.  It is good
+For correct operation, you must inform the kernel in your device probe
+routine to see if the DMA controller on the machine can properly
+support the DMA addressing capabilities your device has.  It is good
 style to do this even if your device holds the default setting,
 because this shows that you did think about these issues wrt. your
 device.
 
-The query is performed via a call to dma_set_mask_and_coherent():
+The call to inform the kernel is performed via a call to
+dma_set_mask_and_coherent():
 
 	int dma_set_mask_and_coherent(struct device *dev, u64 mask);
 
-which will query the mask for both streaming and coherent APIs together.
-If you have some special requirements, then the following two separate
-queries can be used instead:
+which will set the mask for both streaming and coherent APIs together.
+If there are some special requirements, then the following two
+separate functions can be used instead:
 
-	The query for streaming mappings is performed via a call to
-	dma_set_mask():
+	The configuration for streaming mappings is performed via a
+	call to dma_set_mask():
 
 		int dma_set_mask(struct device *dev, u64 mask);
 
-	The query for consistent allocations is performed via a call
-	to dma_set_coherent_mask():
+	The configuration for consistent allocations is performed via
+	a call to dma_set_coherent_mask():
 
 		int dma_set_coherent_mask(struct device *dev, u64 mask);
 
 Here, dev is a pointer to the device struct of your device, and mask
 is a bit mask describing which bits of an address your device
 supports.  It returns zero if your card can perform DMA properly on
-the machine given the address mask you provided.  In general, the
-device struct of your device is embedded in the bus-specific device
-struct of your device.  For example, &pdev->dev is a pointer to the
-device struct of a PCI device (pdev is a pointer to the PCI device
-struct of your device).
+the machine given the address mask you provided.  Subsequent to
+calling the above apis, DMA allocations will be made from address
+space that conforms to the mask.  The DMA allocation space maybe
+further restricted if devices along the patch to memory have stricter
+addressing requirements than the device performing the DMA, e.g.,
+IOMMU.
+
+In general, the device struct of your device is embedded in the
+bus-specific device struct of your device.  For example, &pdev->dev is
+a pointer to the device struct of a PCI device (pdev is a pointer to
+the PCI device struct of your device).
 
 If it returns non-zero, your device cannot perform DMA properly on
 this platform, and attempting to do so will result in undefined
-- 
2.9.3

[PATCH] Documentation: DMA-API: Clarify semantics of dma_set_mask_and_coherent

[Punit Agrawal]

The dma mapping api howto gives the impression that using the
dma_set_mask_and_coherent (and related DMA APIs) will cause the kernel
to check all the components in the path from the device to memory for
addressing restrictions. In systems with address translations between
the device and memory (e.g., when using IOMMU), this implies that a
successful call to set set dma mask has checked the addressing
constraints of the intermediaries as well.

For the IOMMU drivers in the tree, the check is actually performed while
allocating the DMA buffer rather than when the DMA mask is
configured. For MMUs that do not support the full device addressing
capability, the allocations are made from a reduced address space.

Update the documentation to clarify that even though the call to
dma_set_mask_and_coherent succeeds, it may not be possible to use the
full addressing capability of the device.

Signed-off-by: Punit Agrawal <punit.agrawal@arm.com>
Cc: Jonathan Corbet <corbet@lwn.net>
---
 Documentation/DMA-API-HOWTO.txt | 39 +++++++++++++++++++++++----------------
 1 file changed, 23 insertions(+), 16 deletions(-)

diff --git a/Documentation/DMA-API-HOWTO.txt b/Documentation/DMA-API-HOWTO.txt
index 979228b..240d1ee 100644
--- a/Documentation/DMA-API-HOWTO.txt
+++ b/Documentation/DMA-API-HOWTO.txt
@@ -159,39 +159,46 @@ support 64-bit addressing (DAC) for all transactions.  And at least
 one platform (SGI SN2) requires 64-bit consistent allocations to
 operate correctly when the IO bus is in PCI-X mode.
 
-For correct operation, you must interrogate the kernel in your device
-probe routine to see if the DMA controller on the machine can properly
-support the DMA addressing limitation your device has.  It is good
+For correct operation, you must inform the kernel in your device probe
+routine to see if the DMA controller on the machine can properly
+support the DMA addressing capabilities your device has.  It is good
 style to do this even if your device holds the default setting,
 because this shows that you did think about these issues wrt. your
 device.
 
-The query is performed via a call to dma_set_mask_and_coherent():
+The call to inform the kernel is performed via a call to
+dma_set_mask_and_coherent():
 
 	int dma_set_mask_and_coherent(struct device *dev, u64 mask);
 
-which will query the mask for both streaming and coherent APIs together.
-If you have some special requirements, then the following two separate
-queries can be used instead:
+which will set the mask for both streaming and coherent APIs together.
+If there are some special requirements, then the following two
+separate functions can be used instead:
 
-	The query for streaming mappings is performed via a call to
-	dma_set_mask():
+	The configuration for streaming mappings is performed via a
+	call to dma_set_mask():
 
 		int dma_set_mask(struct device *dev, u64 mask);
 
-	The query for consistent allocations is performed via a call
-	to dma_set_coherent_mask():
+	The configuration for consistent allocations is performed via
+	a call to dma_set_coherent_mask():
 
 		int dma_set_coherent_mask(struct device *dev, u64 mask);
 
 Here, dev is a pointer to the device struct of your device, and mask
 is a bit mask describing which bits of an address your device
 supports.  It returns zero if your card can perform DMA properly on
-the machine given the address mask you provided.  In general, the
-device struct of your device is embedded in the bus-specific device
-struct of your device.  For example, &pdev->dev is a pointer to the
-device struct of a PCI device (pdev is a pointer to the PCI device
-struct of your device).
+the machine given the address mask you provided.  Subsequent to
+calling the above apis, DMA allocations will be made from address
+space that conforms to the mask.  The DMA allocation space maybe
+further restricted if devices along the patch to memory have stricter
+addressing requirements than the device performing the DMA, e.g.,
+IOMMU.
+
+In general, the device struct of your device is embedded in the
+bus-specific device struct of your device.  For example, &pdev->dev is
+a pointer to the device struct of a PCI device (pdev is a pointer to
+the PCI device struct of your device).
 
 If it returns non-zero, your device cannot perform DMA properly on
 this platform, and attempting to do so will result in undefined
-- 
2.9.3

[PATCH] Documentation: DMA-API: Clarify semantics of dma_set_mask_and_coherent

[Jonathan Corbet]

On Mon, 17 Oct 2016 16:26:23 +0100
Punit Agrawal <punit.agrawal@arm.com> wrote:

> The dma mapping api howto gives the impression that using the
> dma_set_mask_and_coherent (and related DMA APIs) will cause the kernel
> to check all the components in the path from the device to memory for
> addressing restrictions. In systems with address translations between
> the device and memory (e.g., when using IOMMU), this implies that a
> successful call to set set dma mask has checked the addressing
> constraints of the intermediaries as well.
> 
> For the IOMMU drivers in the tree, the check is actually performed while
> allocating the DMA buffer rather than when the DMA mask is
> configured. For MMUs that do not support the full device addressing
> capability, the allocations are made from a reduced address space.
> 
> Update the documentation to clarify that even though the call to
> dma_set_mask_and_coherent succeeds, it may not be possible to use the
> full addressing capability of the device.

OK, so I guess I can buy this.  But...

> Signed-off-by: Punit Agrawal <punit.agrawal@arm.com>
> Cc: Jonathan Corbet <corbet@lwn.net>
> ---
>  Documentation/DMA-API-HOWTO.txt | 39 +++++++++++++++++++++++----------------
>  1 file changed, 23 insertions(+), 16 deletions(-)
> 
> diff --git a/Documentation/DMA-API-HOWTO.txt b/Documentation/DMA-API-HOWTO.txt
> index 979228b..240d1ee 100644
> --- a/Documentation/DMA-API-HOWTO.txt
> +++ b/Documentation/DMA-API-HOWTO.txt
> @@ -159,39 +159,46 @@ support 64-bit addressing (DAC) for all transactions.  And at least
>  one platform (SGI SN2) requires 64-bit consistent allocations to
>  operate correctly when the IO bus is in PCI-X mode.
>  
> -For correct operation, you must interrogate the kernel in your device
> -probe routine to see if the DMA controller on the machine can properly
> -support the DMA addressing limitation your device has.  It is good
> +For correct operation, you must inform the kernel in your device probe
> +routine to see if the DMA controller on the machine can properly
> +support the DMA addressing capabilities your device has.  It is good

Here it's still saying "to see if the DMA controller on the machine can
properly support the DMA addressing capabilities your device has".  So
you've not really changed the sense of this sentence here.

If I understand things correctly, the calls in question are storing the
device's limitations; they will only fail if the kernel is entirely
unable to work within the indicated range, right?  I don't think there's
ever been any guarantee that the system as a whole could use the entire
range that is addressable by the device.  I have no objection to making
that more clear, but let's actually make it more clear by saying what the
functions are actually doing.

Make sense, or am I missing something here?

Thanks,

jon

Re: [PATCH] Documentation: DMA-API: Clarify semantics of dma_set_mask_and_coherent

[Jonathan Corbet]

On Mon, 17 Oct 2016 16:26:23 +0100
Punit Agrawal <punit.agrawal@arm.com> wrote:

> The dma mapping api howto gives the impression that using the
> dma_set_mask_and_coherent (and related DMA APIs) will cause the kernel
> to check all the components in the path from the device to memory for
> addressing restrictions. In systems with address translations between
> the device and memory (e.g., when using IOMMU), this implies that a
> successful call to set set dma mask has checked the addressing
> constraints of the intermediaries as well.
> 
> For the IOMMU drivers in the tree, the check is actually performed while
> allocating the DMA buffer rather than when the DMA mask is
> configured. For MMUs that do not support the full device addressing
> capability, the allocations are made from a reduced address space.
> 
> Update the documentation to clarify that even though the call to
> dma_set_mask_and_coherent succeeds, it may not be possible to use the
> full addressing capability of the device.

OK, so I guess I can buy this.  But...

> Signed-off-by: Punit Agrawal <punit.agrawal@arm.com>
> Cc: Jonathan Corbet <corbet@lwn.net>
> ---
>  Documentation/DMA-API-HOWTO.txt | 39 +++++++++++++++++++++++----------------
>  1 file changed, 23 insertions(+), 16 deletions(-)
> 
> diff --git a/Documentation/DMA-API-HOWTO.txt b/Documentation/DMA-API-HOWTO.txt
> index 979228b..240d1ee 100644
> --- a/Documentation/DMA-API-HOWTO.txt
> +++ b/Documentation/DMA-API-HOWTO.txt
> @@ -159,39 +159,46 @@ support 64-bit addressing (DAC) for all transactions.  And at least
>  one platform (SGI SN2) requires 64-bit consistent allocations to
>  operate correctly when the IO bus is in PCI-X mode.
>  
> -For correct operation, you must interrogate the kernel in your device
> -probe routine to see if the DMA controller on the machine can properly
> -support the DMA addressing limitation your device has.  It is good
> +For correct operation, you must inform the kernel in your device probe
> +routine to see if the DMA controller on the machine can properly
> +support the DMA addressing capabilities your device has.  It is good

Here it's still saying "to see if the DMA controller on the machine can
properly support the DMA addressing capabilities your device has".  So
you've not really changed the sense of this sentence here.

If I understand things correctly, the calls in question are storing the
device's limitations; they will only fail if the kernel is entirely
unable to work within the indicated range, right?  I don't think there's
ever been any guarantee that the system as a whole could use the entire
range that is addressable by the device.  I have no objection to making
that more clear, but let's actually make it more clear by saying what the
functions are actually doing.

Make sense, or am I missing something here?

Thanks,

jon

[PATCH] Documentation: DMA-API: Clarify semantics of dma_set_mask_and_coherent

[Joerg Roedel]

On Fri, Oct 21, 2016 at 03:09:16PM -0600, Jonathan Corbet wrote:
> On Mon, 17 Oct 2016 16:26:23 +0100
> Punit Agrawal <punit.agrawal@arm.com> wrote:
> 
> > The dma mapping api howto gives the impression that using the
> > dma_set_mask_and_coherent (and related DMA APIs) will cause the kernel
> > to check all the components in the path from the device to memory for
> > addressing restrictions. In systems with address translations between
> > the device and memory (e.g., when using IOMMU), this implies that a
> > successful call to set set dma mask has checked the addressing
> > constraints of the intermediaries as well.

This is basically true when you have DMA controllers in the path from
device to memory. But it is not true for IOMMUs, because IOMMU drivers
are consumers of the dma-masks, they don't really restrict them. An
IOMMU driver knows the limitations of IOMMU hardware and counts that in
when allocating an address for a dma-buffer.

So long story short: Any IOMMU restrictions in address space size don't
need to be represented in the dma-mask for a device.



	Joerg

Re: [PATCH] Documentation: DMA-API: Clarify semantics of dma_set_mask_and_coherent

[Joerg Roedel]

On Fri, Oct 21, 2016 at 03:09:16PM -0600, Jonathan Corbet wrote:
> On Mon, 17 Oct 2016 16:26:23 +0100
> Punit Agrawal <punit.agrawal@arm.com> wrote:
> 
> > The dma mapping api howto gives the impression that using the
> > dma_set_mask_and_coherent (and related DMA APIs) will cause the kernel
> > to check all the components in the path from the device to memory for
> > addressing restrictions. In systems with address translations between
> > the device and memory (e.g., when using IOMMU), this implies that a
> > successful call to set set dma mask has checked the addressing
> > constraints of the intermediaries as well.

This is basically true when you have DMA controllers in the path from
device to memory. But it is not true for IOMMUs, because IOMMU drivers
are consumers of the dma-masks, they don't really restrict them. An
IOMMU driver knows the limitations of IOMMU hardware and counts that in
when allocating an address for a dma-buffer.

So long story short: Any IOMMU restrictions in address space size don't
need to be represented in the dma-mask for a device.



	Joerg

[PATCH] Documentation: DMA-API: Clarify semantics of dma_set_mask_and_coherent

[Punit Agrawal]

Joerg Roedel <joro@8bytes.org> writes:

> On Fri, Oct 21, 2016 at 03:09:16PM -0600, Jonathan Corbet wrote:
>> On Mon, 17 Oct 2016 16:26:23 +0100
>> Punit Agrawal <punit.agrawal@arm.com> wrote:
>> 
>> > The dma mapping api howto gives the impression that using the
>> > dma_set_mask_and_coherent (and related DMA APIs) will cause the kernel
>> > to check all the components in the path from the device to memory for
>> > addressing restrictions. In systems with address translations between
>> > the device and memory (e.g., when using IOMMU), this implies that a
>> > successful call to set set dma mask has checked the addressing
>> > constraints of the intermediaries as well.
>
> This is basically true when you have DMA controllers in the path from
> device to memory. But it is not true for IOMMUs, because IOMMU drivers
> are consumers of the dma-masks, they don't really restrict them. An
> IOMMU driver knows the limitations of IOMMU hardware and counts that in
> when allocating an address for a dma-buffer.

Yes, that's what I'd found looking at the IOMMU drivers in the tree.

>
> So long story short: Any IOMMU restrictions in address space size don't
> need to be represented in the dma-mask for a device.

That was another rabbit hole I'd spend some time in - whether IOMMU
restrictions need to be factored into the dma_mask for devices.

As size(dma_mask) > size(iommu supported address size) still works, I
came to the conclusion that the documentation can maybe help clarify
this.

This patch is an attempt to update the documentation to inform the user
that even if the dma_set_mask call succeeds, the system may still not
use the entire device address space.

Thanks for clarifying a few of my doubts.

Punit

>
>
>
> 	Joerg

Re: [PATCH] Documentation: DMA-API: Clarify semantics of dma_set_mask_and_coherent

[Punit Agrawal]

Joerg Roedel <joro@8bytes.org> writes:

> On Fri, Oct 21, 2016 at 03:09:16PM -0600, Jonathan Corbet wrote:
>> On Mon, 17 Oct 2016 16:26:23 +0100
>> Punit Agrawal <punit.agrawal@arm.com> wrote:
>> 
>> > The dma mapping api howto gives the impression that using the
>> > dma_set_mask_and_coherent (and related DMA APIs) will cause the kernel
>> > to check all the components in the path from the device to memory for
>> > addressing restrictions. In systems with address translations between
>> > the device and memory (e.g., when using IOMMU), this implies that a
>> > successful call to set set dma mask has checked the addressing
>> > constraints of the intermediaries as well.
>
> This is basically true when you have DMA controllers in the path from
> device to memory. But it is not true for IOMMUs, because IOMMU drivers
> are consumers of the dma-masks, they don't really restrict them. An
> IOMMU driver knows the limitations of IOMMU hardware and counts that in
> when allocating an address for a dma-buffer.

Yes, that's what I'd found looking at the IOMMU drivers in the tree.

>
> So long story short: Any IOMMU restrictions in address space size don't
> need to be represented in the dma-mask for a device.

That was another rabbit hole I'd spend some time in - whether IOMMU
restrictions need to be factored into the dma_mask for devices.

As size(dma_mask) > size(iommu supported address size) still works, I
came to the conclusion that the documentation can maybe help clarify
this.

This patch is an attempt to update the documentation to inform the user
that even if the dma_set_mask call succeeds, the system may still not
use the entire device address space.

Thanks for clarifying a few of my doubts.

Punit

>
>
>
> 	Joerg

[PATCH] Documentation: DMA-API: Clarify semantics of dma_set_mask_and_coherent

[Punit Agrawal]

Jonathan Corbet <corbet@lwn.net> writes:

> On Mon, 17 Oct 2016 16:26:23 +0100
> Punit Agrawal <punit.agrawal@arm.com> wrote:
>
>> The dma mapping api howto gives the impression that using the
>> dma_set_mask_and_coherent (and related DMA APIs) will cause the kernel
>> to check all the components in the path from the device to memory for
>> addressing restrictions. In systems with address translations between
>> the device and memory (e.g., when using IOMMU), this implies that a
>> successful call to set set dma mask has checked the addressing
>> constraints of the intermediaries as well.
>> 
>> For the IOMMU drivers in the tree, the check is actually performed while
>> allocating the DMA buffer rather than when the DMA mask is
>> configured. For MMUs that do not support the full device addressing
>> capability, the allocations are made from a reduced address space.
>> 
>> Update the documentation to clarify that even though the call to
>> dma_set_mask_and_coherent succeeds, it may not be possible to use the
>> full addressing capability of the device.
>
> OK, so I guess I can buy this.  But...
>
>> Signed-off-by: Punit Agrawal <punit.agrawal@arm.com>
>> Cc: Jonathan Corbet <corbet@lwn.net>
>> ---
>>  Documentation/DMA-API-HOWTO.txt | 39 +++++++++++++++++++++++----------------
>>  1 file changed, 23 insertions(+), 16 deletions(-)
>> 
>> diff --git a/Documentation/DMA-API-HOWTO.txt b/Documentation/DMA-API-HOWTO.txt
>> index 979228b..240d1ee 100644
>> --- a/Documentation/DMA-API-HOWTO.txt
>> +++ b/Documentation/DMA-API-HOWTO.txt
>> @@ -159,39 +159,46 @@ support 64-bit addressing (DAC) for all transactions.  And at least
>>  one platform (SGI SN2) requires 64-bit consistent allocations to
>>  operate correctly when the IO bus is in PCI-X mode.
>>  
>> -For correct operation, you must interrogate the kernel in your device
>> -probe routine to see if the DMA controller on the machine can properly
>> -support the DMA addressing limitation your device has.  It is good
>> +For correct operation, you must inform the kernel in your device probe
>> +routine to see if the DMA controller on the machine can properly
>> +support the DMA addressing capabilities your device has.  It is good
>
> Here it's still saying "to see if the DMA controller on the machine can
> properly support the DMA addressing capabilities your device has".  So
> you've not really changed the sense of this sentence here.

You're right - the changes don't go far enough.

How about dropping the bit so that it now reads -

"For correct operation, in your device probe routine, you must inform
the DMA addressing capabilities of your device to the kernel."

>
> If I understand things correctly, the calls in question are storing the
> device's limitations; they will only fail if the kernel is entirely
> unable to work within the indicated range, right?  I don't think there's
> ever been any guarantee that the system as a whole could use the entire
> range that is addressable by the device.

That matches my understanding as well. I was just trying to make it more
explicit with this patch.

> I have no objection to making
> that more clear, but let's actually make it more clear by saying what the
> functions are actually doing.
>
> Make sense, or am I missing something here?

Let me know if the change I suggest is an improvement and I'll send out
a v2.

Thanks,
Punit

>
> Thanks,
>
> jon

Re: [PATCH] Documentation: DMA-API: Clarify semantics of dma_set_mask_and_coherent

[Punit Agrawal]

Jonathan Corbet <corbet@lwn.net> writes:

> On Mon, 17 Oct 2016 16:26:23 +0100
> Punit Agrawal <punit.agrawal@arm.com> wrote:
>
>> The dma mapping api howto gives the impression that using the
>> dma_set_mask_and_coherent (and related DMA APIs) will cause the kernel
>> to check all the components in the path from the device to memory for
>> addressing restrictions. In systems with address translations between
>> the device and memory (e.g., when using IOMMU), this implies that a
>> successful call to set set dma mask has checked the addressing
>> constraints of the intermediaries as well.
>> 
>> For the IOMMU drivers in the tree, the check is actually performed while
>> allocating the DMA buffer rather than when the DMA mask is
>> configured. For MMUs that do not support the full device addressing
>> capability, the allocations are made from a reduced address space.
>> 
>> Update the documentation to clarify that even though the call to
>> dma_set_mask_and_coherent succeeds, it may not be possible to use the
>> full addressing capability of the device.
>
> OK, so I guess I can buy this.  But...
>
>> Signed-off-by: Punit Agrawal <punit.agrawal@arm.com>
>> Cc: Jonathan Corbet <corbet@lwn.net>
>> ---
>>  Documentation/DMA-API-HOWTO.txt | 39 +++++++++++++++++++++++----------------
>>  1 file changed, 23 insertions(+), 16 deletions(-)
>> 
>> diff --git a/Documentation/DMA-API-HOWTO.txt b/Documentation/DMA-API-HOWTO.txt
>> index 979228b..240d1ee 100644
>> --- a/Documentation/DMA-API-HOWTO.txt
>> +++ b/Documentation/DMA-API-HOWTO.txt
>> @@ -159,39 +159,46 @@ support 64-bit addressing (DAC) for all transactions.  And at least
>>  one platform (SGI SN2) requires 64-bit consistent allocations to
>>  operate correctly when the IO bus is in PCI-X mode.
>>  
>> -For correct operation, you must interrogate the kernel in your device
>> -probe routine to see if the DMA controller on the machine can properly
>> -support the DMA addressing limitation your device has.  It is good
>> +For correct operation, you must inform the kernel in your device probe
>> +routine to see if the DMA controller on the machine can properly
>> +support the DMA addressing capabilities your device has.  It is good
>
> Here it's still saying "to see if the DMA controller on the machine can
> properly support the DMA addressing capabilities your device has".  So
> you've not really changed the sense of this sentence here.

You're right - the changes don't go far enough.

How about dropping the bit so that it now reads -

"For correct operation, in your device probe routine, you must inform
the DMA addressing capabilities of your device to the kernel."

>
> If I understand things correctly, the calls in question are storing the
> device's limitations; they will only fail if the kernel is entirely
> unable to work within the indicated range, right?  I don't think there's
> ever been any guarantee that the system as a whole could use the entire
> range that is addressable by the device.

That matches my understanding as well. I was just trying to make it more
explicit with this patch.

> I have no objection to making
> that more clear, but let's actually make it more clear by saying what the
> functions are actually doing.
>
> Make sense, or am I missing something here?

Let me know if the change I suggest is an improvement and I'll send out
a v2.

Thanks,
Punit

>
> Thanks,
>
> jon

[PATCH] Documentation: DMA-API: Clarify semantics of dma_set_mask_and_coherent

[Arnd Bergmann]

On Friday, October 21, 2016 3:09:16 PM CEST Jonathan Corbet wrote:
> On Mon, 17 Oct 2016 16:26:23 +0100
> Punit Agrawal <punit.agrawal@arm.com> wrote:
> 
> > The dma mapping api howto gives the impression that using the
> > dma_set_mask_and_coherent (and related DMA APIs) will cause the kernel
> > to check all the components in the path from the device to memory for
> > addressing restrictions. In systems with address translations between
> > the device and memory (e.g., when using IOMMU), this implies that a
> > successful call to set set dma mask has checked the addressing
> > constraints of the intermediaries as well.
> > 
> > For the IOMMU drivers in the tree, the check is actually performed while
> > allocating the DMA buffer rather than when the DMA mask is
> > configured. For MMUs that do not support the full device addressing
> > capability, the allocations are made from a reduced address space.
> > 
> > Update the documentation to clarify that even though the call to
> > dma_set_mask_and_coherent succeeds, it may not be possible to use the
> > full addressing capability of the device.
> 
> OK, so I guess I can buy this.  But...
> 
> > Signed-off-by: Punit Agrawal <punit.agrawal@arm.com>
> > Cc: Jonathan Corbet <corbet@lwn.net>
> > ---
> >  Documentation/DMA-API-HOWTO.txt | 39 +++++++++++++++++++++++----------------
> >  1 file changed, 23 insertions(+), 16 deletions(-)
> > 
> > diff --git a/Documentation/DMA-API-HOWTO.txt b/Documentation/DMA-API-HOWTO.txt
> > index 979228b..240d1ee 100644
> > --- a/Documentation/DMA-API-HOWTO.txt
> > +++ b/Documentation/DMA-API-HOWTO.txt
> > @@ -159,39 +159,46 @@ support 64-bit addressing (DAC) for all transactions.  And at least
> >  one platform (SGI SN2) requires 64-bit consistent allocations to
> >  operate correctly when the IO bus is in PCI-X mode.
> >  
> > -For correct operation, you must interrogate the kernel in your device
> > -probe routine to see if the DMA controller on the machine can properly
> > -support the DMA addressing limitation your device has.  It is good
> > +For correct operation, you must inform the kernel in your device probe
> > +routine to see if the DMA controller on the machine can properly
> > +support the DMA addressing capabilities your device has.  It is good
> 
> Here it's still saying "to see if the DMA controller on the machine can
> properly support the DMA addressing capabilities your device has".  So
> you've not really changed the sense of this sentence here.
> 
> If I understand things correctly, the calls in question are storing the
> device's limitations; they will only fail if the kernel is entirely
> unable to work within the indicated range, right?  I don't think there's
> ever been any guarantee that the system as a whole could use the entire
> range that is addressable by the device.  I have no objection to making
> that more clear, but let's actually make it more clear by saying what the
> functions are actually doing.
> 
> Make sense, or am I missing something here?

The call is a two-way interface, and the existing text tries to convey
that already: The device tells the kernel whether it is limited (< 32
bit mask) or if it can support extended addresses (> 32 bit mask),
or just handles the default 32bit mask, and the kernel should come
back saying whether that mask allows a correct operation of the device
on the given platform, as well as set it up correctly that way.

What exactly happens in dma_set_mask() and the related interfaces
is highly platform specific, including:

- if the mask is smaller than the smallest memory zone and the
  swiotlb bounce buffers (if any) don't fit inside it, it has to
  fail

- if the device claims to support larger mask, but the bus it
  connects to does not (e.g. a 32-bit PCI host), it may
  also fail (or succeed if there is no RAM outside of the
  intersection of the two masks)

- if the mask is large enough to cover all RAM, we can bypass
  the IOMMU and use a direct mapping

- if swiotlb is enabled or an IOMMU is present, any mask that
  includes the bounce buffer area (or the virtual address space
  of the IOMMU) should succeed.

	Arnd

Re: [PATCH] Documentation: DMA-API: Clarify semantics of dma_set_mask_and_coherent

[Arnd Bergmann]

On Friday, October 21, 2016 3:09:16 PM CEST Jonathan Corbet wrote:
> On Mon, 17 Oct 2016 16:26:23 +0100
> Punit Agrawal <punit.agrawal@arm.com> wrote:
> 
> > The dma mapping api howto gives the impression that using the
> > dma_set_mask_and_coherent (and related DMA APIs) will cause the kernel
> > to check all the components in the path from the device to memory for
> > addressing restrictions. In systems with address translations between
> > the device and memory (e.g., when using IOMMU), this implies that a
> > successful call to set set dma mask has checked the addressing
> > constraints of the intermediaries as well.
> > 
> > For the IOMMU drivers in the tree, the check is actually performed while
> > allocating the DMA buffer rather than when the DMA mask is
> > configured. For MMUs that do not support the full device addressing
> > capability, the allocations are made from a reduced address space.
> > 
> > Update the documentation to clarify that even though the call to
> > dma_set_mask_and_coherent succeeds, it may not be possible to use the
> > full addressing capability of the device.
> 
> OK, so I guess I can buy this.  But...
> 
> > Signed-off-by: Punit Agrawal <punit.agrawal@arm.com>
> > Cc: Jonathan Corbet <corbet@lwn.net>
> > ---
> >  Documentation/DMA-API-HOWTO.txt | 39 +++++++++++++++++++++++----------------
> >  1 file changed, 23 insertions(+), 16 deletions(-)
> > 
> > diff --git a/Documentation/DMA-API-HOWTO.txt b/Documentation/DMA-API-HOWTO.txt
> > index 979228b..240d1ee 100644
> > --- a/Documentation/DMA-API-HOWTO.txt
> > +++ b/Documentation/DMA-API-HOWTO.txt
> > @@ -159,39 +159,46 @@ support 64-bit addressing (DAC) for all transactions.  And at least
> >  one platform (SGI SN2) requires 64-bit consistent allocations to
> >  operate correctly when the IO bus is in PCI-X mode.
> >  
> > -For correct operation, you must interrogate the kernel in your device
> > -probe routine to see if the DMA controller on the machine can properly
> > -support the DMA addressing limitation your device has.  It is good
> > +For correct operation, you must inform the kernel in your device probe
> > +routine to see if the DMA controller on the machine can properly
> > +support the DMA addressing capabilities your device has.  It is good
> 
> Here it's still saying "to see if the DMA controller on the machine can
> properly support the DMA addressing capabilities your device has".  So
> you've not really changed the sense of this sentence here.
> 
> If I understand things correctly, the calls in question are storing the
> device's limitations; they will only fail if the kernel is entirely
> unable to work within the indicated range, right?  I don't think there's
> ever been any guarantee that the system as a whole could use the entire
> range that is addressable by the device.  I have no objection to making
> that more clear, but let's actually make it more clear by saying what the
> functions are actually doing.
> 
> Make sense, or am I missing something here?

The call is a two-way interface, and the existing text tries to convey
that already: The device tells the kernel whether it is limited (< 32
bit mask) or if it can support extended addresses (> 32 bit mask),
or just handles the default 32bit mask, and the kernel should come
back saying whether that mask allows a correct operation of the device
on the given platform, as well as set it up correctly that way.

What exactly happens in dma_set_mask() and the related interfaces
is highly platform specific, including:

- if the mask is smaller than the smallest memory zone and the
  swiotlb bounce buffers (if any) don't fit inside it, it has to
  fail

- if the device claims to support larger mask, but the bus it
  connects to does not (e.g. a 32-bit PCI host), it may
  also fail (or succeed if there is no RAM outside of the
  intersection of the two masks)

- if the mask is large enough to cover all RAM, we can bypass
  the IOMMU and use a direct mapping

- if swiotlb is enabled or an IOMMU is present, any mask that
  includes the bounce buffer area (or the virtual address space
  of the IOMMU) should succeed.

	Arnd