Problems with dma_alloc_coherent()

Problems with dma_alloc_coherent()

[John Whitney]

[-- Attachment #1: Type: text/plain, Size: 1465 bytes --]

In the process of reworking my Generic DMA core for the 2.6 kernel, I
noticed that dma_alloc_coherent() returns NULL in all cases on a
cache-coherent platform.  Instead, I think it would be better if this
routine returned a block of contiguous memory (not cache-inhibited), as
pci_alloc_consistent() does.

To that end, I am submitting the attached patch which modifies
dma_alloc_coherent() and dma_free_coherent() to do just this for
cache-coherent platforms.
I was going to add a check for dev->dma_mask in dma_alloc_coherent()
(to add GFP_DMA to the flags), but the dma_supported() macro always
returns 1, which would indicate that this is superfluous.

John


--- dma-mapping.h.orig	2004-04-01 10:38:33.000000000 -0500
+++ dma-mapping.h	2004-04-01 10:49:32.000000000 -0500
@@ -35,7 +35,18 @@
  		return pci_alloc_consistent(to_pci_dev(dev), size, dma_handle);
  #endif

-	return consistent_alloc(flag, size, dma_handle);
+#ifdef CONFIG_NOT_COHERENT_CACHE
+    return consistent_alloc(flag, size, dma_handle);
+#else
+	void *ret;
+
+	ret = (void *)__get_free_pages(flag, get_order(size));
+	if (ret != NULL) {
+		*dma_handle = virt_to_bus(ret);
+	}
+
+    return ret;
+#endif /* CONFIG_NOT_COHERENT_CACHE */
  }

  static inline void
@@ -49,7 +60,11 @@
  	}
  #endif

-	consistent_free(vaddr);
+#ifdef CONFIG_NOT_COHERENT_CACHE
+	consistent_free(vaddr, size);
+#else
+	free_pages((unsigned long)vaddr, get_order(size));
+#endif
  }

  static inline dma_addr_t



[-- Attachment #2: dma-mapping.h.patch --]
[-- Type: application/octet-stream, Size: 791 bytes --]

--- dma-mapping.h.orig	2004-04-01 10:38:33.000000000 -0500
+++ dma-mapping.h	2004-04-01 10:49:32.000000000 -0500
@@ -35,7 +35,18 @@
 		return pci_alloc_consistent(to_pci_dev(dev), size, dma_handle);
 #endif

-	return consistent_alloc(flag, size, dma_handle);
+#ifdef CONFIG_NOT_COHERENT_CACHE
+    return consistent_alloc(flag, size, dma_handle);
+#else
+	void *ret;
+
+	ret = (void *)__get_free_pages(flag, get_order(size));
+	if (ret != NULL) {
+		*dma_handle = virt_to_bus(ret);
+	}
+
+    return ret;
+#endif /* CONFIG_NOT_COHERENT_CACHE */
 }

 static inline void
@@ -49,7 +60,11 @@
 	}
 #endif

-	consistent_free(vaddr);
+#ifdef CONFIG_NOT_COHERENT_CACHE
+	consistent_free(vaddr, size);
+#else
+	free_pages((unsigned long)vaddr, get_order(size));
+#endif
 }

 static inline dma_addr_t

Re: Problems with dma_alloc_coherent()

[John Whitney]

[-- Attachment #1: Type: text/plain, Size: 827 bytes --]

Whoops.  I accidentally had some previous changes in the patch I
submitted.  Here is the cleaned up version.

--- dma-mapping.h.orig	2004-04-01 10:38:33.000000000 -0500
+++ dma-mapping.h	2004-04-01 11:27:44.000000000 -0500
@@ -35,7 +35,18 @@
  		return pci_alloc_consistent(to_pci_dev(dev), size, dma_handle);
  #endif

+#ifdef CONFIG_NOT_COHERENT_CACHE
  	return consistent_alloc(flag, size, dma_handle);
+#else
+	void *ret;
+
+	ret = (void *)__get_free_pages(flag, get_order(size));
+	if (ret != NULL) {
+		*dma_handle = virt_to_bus(ret);
+	}
+
+    return ret;
+#endif /* CONFIG_NOT_COHERENT_CACHE */
  }

  static inline void
@@ -49,7 +60,11 @@
  	}
  #endif

+#ifdef CONFIG_NOT_COHERENT_CACHE
  	consistent_free(vaddr);
+#else
+	free_pages((unsigned long)vaddr, get_order(size));
+#endif
  }

  static inline dma_addr_t



[-- Attachment #2: dma-mapping.h.patch --]
[-- Type: application/octet-stream, Size: 705 bytes --]

--- dma-mapping.h.orig	2004-04-01 10:38:33.000000000 -0500
+++ dma-mapping.h	2004-04-01 11:27:44.000000000 -0500
@@ -35,7 +35,18 @@
 		return pci_alloc_consistent(to_pci_dev(dev), size, dma_handle);
 #endif

+#ifdef CONFIG_NOT_COHERENT_CACHE
 	return consistent_alloc(flag, size, dma_handle);
+#else
+	void *ret;
+
+	ret = (void *)__get_free_pages(flag, get_order(size));
+	if (ret != NULL) {
+		*dma_handle = virt_to_bus(ret);
+	}
+
+    return ret;
+#endif /* CONFIG_NOT_COHERENT_CACHE */
 }

 static inline void
@@ -49,7 +60,11 @@
 	}
 #endif

+#ifdef CONFIG_NOT_COHERENT_CACHE
 	consistent_free(vaddr);
+#else
+	free_pages((unsigned long)vaddr, get_order(size));
+#endif
 }

 static inline dma_addr_t

Re: Problems with dma_alloc_coherent()

[Dan Malek]

John Whitney wrote:

> To that end, I am submitting the attached patch which modifies
> dma_alloc_coherent() and dma_free_coherent() to do just this for
> cache-coherent platforms.

I think you should change/implement consistent_alloc()/free() to do
the right thing rather that use the #ifdef here.  Also, ensure the
ancillary function (like the dma sync/sg functions) work properly as
well.  They are probably no-ops (well, not the sg functions) but
look for those anyway.

There are also reasons to allocate un-cached DMA regions on
cache coherent platforms (but I will not debate the value in doing so),
so keep that under consideration.

Thanks.


	-- Dan


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[Tom Rini]

On Thu, Apr 01, 2004 at 10:59:56AM -0500, John Whitney wrote:

> In the process of reworking my Generic DMA core for the 2.6 kernel, I
> noticed that dma_alloc_coherent() returns NULL in all cases on a
> cache-coherent platform.  Instead, I think it would be better if this
> routine returned a block of contiguous memory (not cache-inhibited), as
> pci_alloc_consistent() does.
>
> To that end, I am submitting the attached patch which modifies
> dma_alloc_coherent() and dma_free_coherent() to do just this for
> cache-coherent platforms.
> I was going to add a check for dev->dma_mask in dma_alloc_coherent()
> (to add GFP_DMA to the flags), but the dma_supported() macro always
> returns 1, which would indicate that this is superfluous.

I'd like to suggest that what we do is make dma_alloc_coherent and
dma_free_coherent steal the logic of pci_alloc/free_consistent outright,
moving us one step closer to using <asm-generic/pci-dma-compat.h> (PCI
DMA API implemented in terms of the generic DMA API).

--
Tom Rini
http://gate.crashing.org/~trini/

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[Matt Porter]

On Thu, Apr 01, 2004 at 10:59:56AM -0500, John Whitney wrote:
> In the process of reworking my Generic DMA core for the 2.6 kernel, I
> noticed that dma_alloc_coherent() returns NULL in all cases on a
> cache-coherent platform.  Instead, I think it would be better if this
> routine returned a block of contiguous memory (not cache-inhibited), as
> pci_alloc_consistent() does.

Yeah, that's really broken.  The changeover to dma-mapping.h is
incomplete.  Tom and I were just talking about this because I
think it exposes a bigger problem.  What I originally wanted
was the PCI DMA API defined in terms of the generic DMA API
on ppc32.  He just pointed out asm-generic/pci-dma-compat.h that
we can leverage to do that.

> To that end, I am submitting the attached patch which modifies
> dma_alloc_coherent() and dma_free_coherent() to do just this for
> cache-coherent platforms.

Your patch fixes the problem but exposes the bigger issue that we
are starting to duplicate code since on ppc32 PCI DMA API and DMA
API are handled the same on the backend.

In the meantime (I think Tom volunteered to clean this up!), I think
this needs to go in so we are at least functional.

-Matt

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[John Whitney]

>
> Yeah, that's really broken.  The changeover to dma-mapping.h is
> incomplete.  Tom and I were just talking about this because I
> think it exposes a bigger problem.  What I originally wanted
> was the PCI DMA API defined in terms of the generic DMA API
> on ppc32.  He just pointed out asm-generic/pci-dma-compat.h that
> we can leverage to do that.
>

I noticed that all the PCI and DMA coherency routines seem to assume
that DMA will occur between memory and a bus device (all physical
addresses returned are converted to a bus address).  Is this really the
desired action?  I would have prefered to have the PCI routines return
bus-correct physical addresses, and the DMA routines return processor
physical addresses (so they can be used for non-bus-related DMA, to a
direct memory-mapped encryption chip, for example).  Is this the
eventual implementation of those functions, or are all DMA transactions
expected to be to or from a PCI-based device?

John


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[Matt Porter]

On Thu, Apr 01, 2004 at 12:51:54PM -0500, John Whitney wrote:
> >
> > Yeah, that's really broken.  The changeover to dma-mapping.h is
> > incomplete.  Tom and I were just talking about this because I
> > think it exposes a bigger problem.  What I originally wanted
> > was the PCI DMA API defined in terms of the generic DMA API
> > on ppc32.  He just pointed out asm-generic/pci-dma-compat.h that
> > we can leverage to do that.
> >
>
> I noticed that all the PCI and DMA coherency routines seem to assume
> that DMA will occur between memory and a bus device (all physical
> addresses returned are converted to a bus address).  Is this really the
> desired action?  I would have prefered to have the PCI routines return
> bus-correct physical addresses, and the DMA routines return processor
> physical addresses (so they can be used for non-bus-related DMA, to a
> direct memory-mapped encryption chip, for example).  Is this the
> eventual implementation of those functions, or are all DMA transactions
> expected to be to or from a PCI-based device?

Yes, the DMA API and PCI DMA API have been stated (repeatedly) to
be _only_ for [system memory]<->[device].  Trying to use them
otherwise is incorrect.  I believe DaveM's suggestion was, "If you
want a device-to-device DMA API then create a separate one".

It's another thing that embedded people will have to take the
time to introduce...but most embedded Linux people are too busy
with their apps on 2.4, maintaining boards, etc. to fix kernel
APIs. Don't expect the desktop/server folks to care. :)

-Matt

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[Eugene Surovegin]

On Thu, Apr 01, 2004 at 12:51:54PM -0500, John Whitney wrote:
> I noticed that all the PCI and DMA coherency routines seem to assume
> that DMA will occur between memory and a bus device (all physical
> addresses returned are converted to a bus address).  Is this really the
> desired action?  I would have prefered to have the PCI routines return
> bus-correct physical addresses, and the DMA routines return processor
> physical addresses (so they can be used for non-bus-related DMA, to a
> direct memory-mapped encryption chip, for example).  Is this the
> eventual implementation of those functions, or are all DMA transactions
> expected to be to or from a PCI-based device?

Hmm, I don't understand this, bus != PCI bus. All devices sit on some kind of
bus, even your encryption chip.

As far as I understand, DMA API was added exactly because we may have different
(from PCI) buses.

Current implementation just relies on the fact that PCI devices view system
memory the same way as other-bus devices (e.g. OCP devices which sit on OPB).

In more general case, yes, "views" can be different for each bus type or even
for each bus...

Eugene.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[Eugene Surovegin]

On Thu, Apr 01, 2004 at 10:19:26AM -0800, Eugene Surovegin wrote:
> On Thu, Apr 01, 2004 at 12:51:54PM -0500, John Whitney wrote:
> > I noticed that all the PCI and DMA coherency routines seem to assume
> > that DMA will occur between memory and a bus device (all physical
> > addresses returned are converted to a bus address).  Is this really the
> > desired action?  I would have prefered to have the PCI routines return
> > bus-correct physical addresses, and the DMA routines return processor
> > physical addresses (so they can be used for non-bus-related DMA, to a
> > direct memory-mapped encryption chip, for example).  Is this the
> > eventual implementation of those functions, or are all DMA transactions
> > expected to be to or from a PCI-based device?
>
> Hmm, I don't understand this, bus != PCI bus. All devices sit on some kind of
> bus, even your encryption chip.
>
> As far as I understand, DMA API was added exactly because we may have different
> (from PCI) buses.
>
> Current implementation just relies on the fact that PCI devices view system
> memory the same way as other-bus devices (e.g. OCP devices which sit on OPB).
>
> In more general case, yes, "views" can be different for each bus type or even
> for each bus...

Let me clarify my point.

You are right that returning PCI-bus address _may_ not be very good. I think
it's OK for now, because it should work for the reason I pointed earlier.

In future, implementation may change...

What you are proposing isn't better than the current PCI-bus addresses. Phys
address is just an address on processor local bus, so changing PCI-bus as a
default to PLB as a default doesn't seem wise.

Correct implementation should devise DMA-addresses (and actual bus) from the
_device_ passed to DMA API function...

Eugene.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[John Whitney]

> Hmm, I don't understand this, bus != PCI bus. All devices sit on some
> kind of
> bus, even your encryption chip.
>

Yes, but both pci_alloc_consistent() and dma_alloc_coherent() return
physical addresses made with virt_to_bus() (at least they do in the
PowerPC tree).  This routine specifically adds the PCI-bus view of the
address space to the physical address produced.  I don't understand why
two different APIs are needed, if they both do the same thing.

This means that the address produced by pci_alloc_... and dma_alloc_...
are incorrect for a device like a direct memory-mapped encryption chip,
or in my case a northbridge DMA controller.

John


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[Eugene Surovegin]

On Thu, Apr 01, 2004 at 01:33:42PM -0500, John Whitney wrote:
> Yes, but both pci_alloc_consistent() and dma_alloc_coherent() return
> physical addresses made with virt_to_bus() (at least they do in the
> PowerPC tree).  This routine specifically adds the PCI-bus view of the
> address space to the physical address produced.  I don't understand why
> two different APIs are needed, if they both do the same thing.

They do the same thing _now_, because it's a _new_ API. This _may_ change in
future.

DMA API is needed, because generic code uses it, most archs have default generic
implementation which use PCI DMA API and I don't see _any_ problems with that.

Some archs may have _different_ implementations which DON'T use PCI DMA API.

>
> This means that the address produced by pci_alloc_... and dma_alloc_...
> are incorrect for a device like a direct memory-mapped encryption chip,
> or in my case a northbridge DMA controller.

It depends, on all systems I use PCI view == phys address, so it'll work.

Eugene


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[John Whitney]

> They do the same thing _now_, because it's a _new_ API. This _may_
> change in
> future.
>
> DMA API is needed, because generic code uses it, most archs have
> default generic
> implementation which use PCI DMA API and I don't see _any_ problems
> with that.
>
> Some archs may have _different_ implementations which DON'T use PCI
> DMA API.
>
>>
>> This means that the address produced by pci_alloc_... and
>> dma_alloc_...
>> are incorrect for a device like a direct memory-mapped encryption
>> chip,
>> or in my case a northbridge DMA controller.
>
> It depends, on all systems I use PCI view == phys address, so it'll
> work.
>

And in actuality, it will work in my case as well.  However, it won't
work for some single-board computers I've worked on, so I'd like to
have an API that works for everything.

It would be nice if there was a dma_bus_offset field in the device
structure that got set by the device creator.  PCI devices would get
PCI_DRAM_OFFSET, PLB devices would get 0, OCP devices would get
whatever they need.

Then the API could add the appropriate offset for the device passed in,
rather than relying on the static virt_to_bus() mapping, which is PCI
specific.  As a default, with a NULL device given, I think it should
return a PLB physical address.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[Dan Malek]

Eugene Surovegin wrote:

> Hmm, I don't understand this, bus != PCI bus.

Isn't that necessary due to bridge windows?  If != PCI bus I guess we
assume OCP that are connected to a 1:1 mapped internal bus, which
may not be a valid assumption either.

> Current implementation just relies on the fact that PCI devices view system
> memory the same way as other-bus devices (e.g. OCP devices which sit on OPB).

I don't think so.  PCI devices are known to go through bridges, while
others are assumed to not.

> In more general case, yes, "views" can be different for each bus type or even
> for each bus...

So, we should always have some kind of 'bus type' device handle, that provides
this information.  It's clear that all Linux APIs are designed from the
viewpoint of PC-like workstations.  They have always fallen very short when
pressed into the more prevalent embedded computing space :-)


	-- Dan


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[Eugene Surovegin]

On Thu, Apr 01, 2004 at 01:55:10PM -0500, Dan Malek wrote:
> >Current implementation just relies on the fact that PCI devices view system
> >memory the same way as other-bus devices (e.g. OCP devices which sit on
> >OPB).
>
> I don't think so.  PCI devices are known to go through bridges, while
> others are assumed to not.

Well, bridge doesn't necessarily mean there is an address translation :).
Currently 4xx systems (not sure all, at least ones I use) use PCI == phys
address mapping, and yes, bridge is setup so this is the case.

Although, I understand in more general case, bridge will translate address.

> >In more general case, yes, "views" can be different for each bus type or
> >even
> >for each bus...
>
> So, we should always have some kind of 'bus type' device handle, that
> provides
> this information.  It's clear that all Linux APIs are designed from the
> viewpoint of PC-like workstations.  They have always fallen very short when
> pressed into the more prevalent embedded computing space :-)

I think we should use "device" parameter passed to DMA API for such stuff, it
already has a member "struct bus_type* bus".

Eugene

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[John Whitney]

> I think we should use "device" parameter passed to DMA API for such
> stuff, it
> already has a member "struct bus_type* bus".
>

This effectively makes the DMA API a superset of the PCI DMA API (the
DMA API would recognize a PCI device, and handle it accordingly)?  It
sounds like a good solution to me, although I'd like to see the
"device" structure get that DMA offset I mentioned, so the DMA API
wouldn't have to have a special case for each new bus-type added (PLB,
PCI, OCP, CPM, whatever), as it does now for PCI vs. non-PCI.  (That
assumes bus offset is the only difference that needs to be handled...)


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[Eugene Surovegin]

On Thu, Apr 01, 2004 at 02:10:00PM -0500, John Whitney wrote:
> >I think we should use "device" parameter passed to DMA API for such
> >stuff, it
> >already has a member "struct bus_type* bus".
> >
>
> This effectively makes the DMA API a superset of the PCI DMA API (the
> DMA API would recognize a PCI device, and handle it accordingly)?

Yes, I think it's a goal.

> It
> sounds like a good solution to me, although I'd like to see the
> "device" structure get that DMA offset I mentioned, so the DMA API
> wouldn't have to have a special case for each new bus-type added (PLB,
> PCI, OCP, CPM, whatever), as it does now for PCI vs. non-PCI.  (That
> assumes bus offset is the only difference that needs to be handled...)

I think we should add such info to a structure which describes the _bus_, not
_device_.
I'm not "new-device-model" guru, so I cannot comment _how_ this can be
implemented though :)

Eugene.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[John Whitney]

> I think we should add such info to a structure which describes the
> _bus_, not
> _device_.
> I'm not "new-device-model" guru, so I cannot comment _how_ this can be
> implemented though :)

Yes, after thinking about it, it made more sense to modify the "struct
bus_type" structure to have the bus memory map offset.  We could then
have an OCP bus type, PLB bus type, etc.  However, that would be a core
kernel change, and I don't know how easy it is to get those pushed back
into the mainline.

Also, after looking at the kernel code more (I'm fairly new to 2.6,
which I'm sure you've deduced), it looks like most routines expect
instances of type "bus_type" to be enumeratable/hot-swappable, and many
of the busses I'd use in the embedded world wouldn't fit.  I don't know
how easy or difficult it would be to use the bus_type code construct in
this fashion.

John


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[Matt Porter]

On Thu, Apr 01, 2004 at 10:59:56AM -0800, Eugene Surovegin wrote:
> On Thu, Apr 01, 2004 at 01:55:10PM -0500, Dan Malek wrote:
> > >Current implementation just relies on the fact that PCI devices view system
> > >memory the same way as other-bus devices (e.g. OCP devices which sit on
> > >OPB).
> >
> > I don't think so.  PCI devices are known to go through bridges, while
> > others are assumed to not.
>
> Well, bridge doesn't necessarily mean there is an address translation :).
> Currently 4xx systems (not sure all, at least ones I use) use PCI == phys
> address mapping, and yes, bridge is setup so this is the case.

Just FYI, not all 4xx systems, just the ones in the kernel tree today.

-Matt

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[Michael R. Zucca]

Eugene Surovegin wrote:
> On Thu, Apr 01, 2004 at 02:10:00PM -0500, John Whitney wrote:

> I think we should add such info to a structure which describes the _bus_, not
> _device_.
> I'm not "new-device-model" guru, so I cannot comment _how_ this can be
> implemented though :)

A bus description alone is not sufficient. A generic DMA API should take
both a source and destination device. Between any two sources and
destinations there may be several intervening bridges, which potentially
means programming those bridges and/or doing several address translations.

--
----------------------------------------------
  Michael Zucca - mrz5149@acm.org
----------------------------------------------
  "I'm too old to use Emacs." -- Rod MacDonald
----------------------------------------------


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[Eugene Surovegin]

On Thu, Apr 01, 2004 at 03:52:20PM -0500, Michael R. Zucca wrote:
>
> >I think we should add such info to a structure which describes the _bus_,
> >not
> >_device_.
> >I'm not "new-device-model" guru, so I cannot comment _how_ this can be
> >implemented though :)
>
> A bus description alone is not sufficient. A generic DMA API should take
> both a source and destination device.

well, DMA API we are talking about handles only device <-> system memory, so
only _one_ device (which is currently done) is need.

> Between any two sources and
> destinations there may be several intervening bridges, which potentially
> means programming those bridges and/or doing several address translations.

In case of several bridges, I think it's simpler just to assign _different_
buses for each bus connected by a bridge.

Eugene.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[Michael R. Zucca]

Eugene Surovegin wrote:
> On Thu, Apr 01, 2004 at 03:52:20PM -0500, Michael R. Zucca wrote:
?
> In case of several bridges, I think it's simpler just to assign _different_
> buses for each bus connected by a bridge.

Ok, I see. So if you assume the CPU is a target/source, you can infer
any intervening bridges/busses. I assume the bridge/bus hierarchy is
stored somewhere so you can do the appropriate programming/translation
for each bridge.

--
----------------------------------------------
  Michael Zucca - mrz5149@acm.org
----------------------------------------------
  "I'm too old to use Emacs." -- Rod MacDonald
----------------------------------------------


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[Christoph Hellwig]

On Thu, Apr 01, 2004 at 02:10:00PM -0500, John Whitney wrote:
> sounds like a good solution to me, although I'd like to see the
> "device" structure get that DMA offset I mentioned, so the DMA API
> wouldn't have to have a special case for each new bus-type added (PLB,

That doesn't work because there's much more complex bridges that do
complicated remapping of the bus addresses and it's not a simple offset.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[John Whitney]

As an exemplification of the problem I am having with the current
DMA/PCI API:

I am developing a Generic DMA driver.  DMA providers register
themselves with it.  Clients ask the DMA core driver to please have a
registered driver transfer the data from one location to another, and
call them back when it is complete.

Now, let's look at my options:

1. The DMA-core API accepts physical addresses only for source and
destination.

This technically can't be done.  The virt_to_phys() API, I am told, is
deprecated and shouldn't be used by the kernel.  However, the client
code can't call dma_map_single() or dma_alloc_coherent(), because it
doesn't KNOW which providers are available and will be servicing the
request.

2. Okay, we'll make the DMA-core API accept only virtual addresses,
which it can map itself to the final driver.

Okay... this works well for the local memory (__get_free_pages() and
all), but the client has a physical address of a device it wants to
write to... how does it get a virtual address?  ioremap().  Bzzt!  The
address returned by ioremap is fully virtual, which dma_map_single()
can't accept (as it calls virt_to_bus(), which doesn't handle
fully-virtual addresses).

3. Well, darn.  How about the client calls
dma_map_single/dma_alloc_coherent with a NULL device, to get a physical
address for allocated local memory to pass to the DMA core?

Well, that sort of works.  However, do ALL platforms add the PCI bus
address-map-offset to the value?  How does the core (processor
independent) code know what to subract off to get back to the base
physical address?  There is no bus_to_phys() to phys_to_bus() routines,
and these would be deprecated even if they did exist.  How does it know
what to add to get back a bus-offset-phys-address suitable for the DMA
controller?  I guess the DMA core COULD call bus_to_virt() (although it
is deprecated), and then call dma_map_single() for the appropriate
device...

4. I suppose I COULD have a far more complicated dma_mem_to_dev() and
dma_dev_to_mem() and dma_dev_to_dev() and dma_mem_to_mem() kind of
interface, which mixed virtual and physical addresses.  Ugh.

Am I missing anything that should have made this all simpler?  If the
DMA/PCI API routines simply accepted ANY virtual address (fully-virtual
or semi-virtual), I wouldn't have this problem.  I'd simply accept
virtual addresses for the source and destination.

John


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[Michael R. Zucca]

John Whitney wrote:
>
> As an exemplification of the problem I am having with the current
> DMA/PCI API:
>
> I am developing a Generic DMA driver.  DMA providers register
> themselves with it.  Clients ask the DMA core driver to please have a
> registered driver transfer the data from one location to another, and
> call them back when it is complete.

If CPU memory is assumed to be a source/target and the other
source/target is the device, then you should take a DMA direction, a
virtual address, a device or bus, and a bus-local address. If CPU memory
isn't the only target or the DMA engine is between a bunch of bridges,
you might need to take a source bus and bus-local address, and a
destination bus and bus-local address.

If you have the bus and the bus-local address you should be able to
deduce what bridges you have to program/translate-through to do the
transfer with the DMA engine. Once you know that you should be able to
translate a bus-local address to whatever address is appropriate for the
DMA engine and program any intervening bridges appropriately.

Does Linux currently have the smarts to do this? Beats me, but it should
if it's ever going to hope to deal with the crazy array of busses and
bridges coming out of the minds of hardware engineers these days. :-)

--
----------------------------------------------
  Michael Zucca - mrz5149@acm.org
----------------------------------------------
  "I'm too old to use Emacs." -- Rod MacDonald
----------------------------------------------


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[John Whitney]

On Apr 2, 2004, at 1:50 PM, Michael R. Zucca wrote:

> John Whitney wrote:
>> As an exemplification of the problem I am having with the current
>> DMA/PCI API:
>> I am developing a Generic DMA driver.  DMA providers register
>> themselves with it.  Clients ask the DMA core driver to please have a
>> registered driver transfer the data from one location to another, and
>> call them back when it is complete.
>
> If CPU memory is assumed to be a source/target and the other
> source/target is the device, then you should take a DMA direction, a
> virtual address, a device or bus, and a bus-local address. If CPU
> memory isn't the only target or the DMA engine is between a bunch of
> bridges, you might need to take a source bus and bus-local address,
> and a destination bus and bus-local address.
>

The "device" will always be the DMA controller (since it is moving data
from one place to another), and all physical addresses should be from
the controller's perspective.  However, the client does not know what
DMA controller might be used (or even which ones exists), as it has
been abstracted.  Direction is not needed to be specified.  The source
will ALWAYS be "to device" the destination will always be "from
device".  The DMA controller device just needs source and destination
physical addresses (adjusted for its memory space).

I'm not trying to handle wierd bus scenarios... I'm just trying to
figure out how to go from a virtual address to a DMA-controller
specific one, with the constraints I have been told about with the
LinuxPPC kernel, in an abstracted interface.  As I outlined in the
previous message, I can't see a way to do this.


Heck, let's make it simple.  Let's say I have a high-speed serial port,
and I want to DMA data to it with a known DMA controller (not
abstracted) with a standard "device" structure associated with it.  The
serial driver has the physical address of the 1-byte transmit register
of the serial port that it wants to DMA to.  How does he make this into
a physical address the known DMA controller device can use?  There is
no phys_to_bus() macro, and it would be deprecated if it existed.
There is no DMA/PCI API for taking a physical address and converting it
to a dma_addr_t for a specific device (the DMA controller).  I can't
ioremap() the device, and pass that virtual address to the DMA API, as
it will produce a bad physical address and I have been told this is the
desired way for the API to work.

What's generic method is left, that doesn't require platform-specific
code?

John


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[Michael R. Zucca]

John Whitney wrote:

> I'm not trying to handle wierd bus scenarios... I'm just trying to
> figure out how to go from a virtual address to a DMA-controller specific
> one, with the constraints I have been told about with the LinuxPPC
> kernel, in an abstracted interface.  As I outlined in the previous
> message, I can't see a way to do this.

This is an approach that might have been good a number of years ago, but
I don't think it's going to cut it these days. DMA engines can be
anywhere and there can be a lot of intervening bridges. You can't always
assume that programming a DMA engine is just a matter of doing
virt_to_phys() and, voila, you're done. These days, you might have lots
of bridges in the way that might need to have mappings setup, and there
might be limited mapping resources so you can't just statically map the
world with the intention that the source or destination will have a
permanent address on the bus the DMA engine sits on.

A generic DMA API might do something like:

dma_move(main_memory_device, virtual_address,
          serial_port_device, pci_address, length) ;

or

dma_move(bulk_memory_card, pci_address,
          serial_port, pci_address, length) ;

A generic DMA API could then pick the best DMA controller at it's
disposal to push data from one bus to another. Once it has picked the
right DMA engine it would know what bridges lie between the DMA engine
and the source and destination devices. At that point it could translate
each of the given addresses so that they can be programmed into the DMA
engine.

> Heck, let's make it simple.  Let's say I have a high-speed serial port,
> and I want to DMA data to it with a known DMA controller (not
> abstracted) with a standard "device" structure associated with it.  The
> serial driver has the physical address of the 1-byte transmit register
> of the serial port that it wants to DMA to.  How does he make this into
> a physical address the known DMA controller device can use?  There is no
> phys_to_bus() macro, and it would be deprecated if it existed.  There is
> no DMA/PCI API for taking a physical address and converting it to a
> dma_addr_t for a specific device (the DMA controller).  I can't
> ioremap() the device, and pass that virtual address to the DMA API, as
> it will produce a bad physical address and I have been told this is the
> desired way for the API to work.

On 2.4, at least, I believe you are supposed to pci_map_* the item if
it's on PCI, take the address, slam in the DMA engine, and when the
transfer is complete, pci_unamp_* the item. I think, though, that this
might even be some hackery since I think the address you get from
pci_map may or may not be appropriate for the DMA engine depending on
where it lives.

Good bus support is kind of lacking in Linux :( It would be nice if
Linux had something more like NetBSD's bus_space/bus_dma stuff, or
something even better.

If what you need is generic address translation, I don't think that
exists yet in Linux. If you need it, you may have to cook it up yourself.

--
----------------------------------------------
  Michael Zucca - mrz5149@acm.org
----------------------------------------------
  "I'm too old to use Emacs." -- Rod MacDonald
----------------------------------------------


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[John Whitney]

> This is an approach that might have been good a number of years ago,
> but I don't think it's going to cut it these days. DMA engines can be
> anywhere and there can be a lot of intervening bridges. You can't
> always assume that programming a DMA engine is just a matter of doing
> virt_to_phys() and, voila, you're done. These days, you might have
> lots of bridges in the way that might need to have mappings setup, and
> there might be limited mapping resources so you can't just statically
> map the world with the intention that the source or destination will
> have a permanent address on the bus the DMA engine sits on.
>
> A generic DMA API might do something like:
>
> dma_move(main_memory_device, virtual_address,
>          serial_port_device, pci_address, length) ;
>
> or
>
> dma_move(bulk_memory_card, pci_address,
>          serial_port, pci_address, length) ;
> A generic DMA API could then pick the best DMA controller at it's
> disposal to push data from one bus to another. Once it has picked the
> right DMA engine it would know what bridges lie between the DMA engine
> and the source and destination devices. At that point it could
> translate each of the given addresses so that they can be programmed
> into the DMA engine.

In fact, there is only one address translation that is important: the
processor's view of the end-points' physical address to the DMA
controller's view.  The bridges should handle all other necessary
translations transparently (I won't swear to that, but I can't think of
any example that contradicts that, although you might not be able to
"get there" from the DMA controller if it is on a different bus...).

However, I'm not trying to come up with a good interface, really.  I'm
trying to show "you can't get there from here" with the current DMA API
implementation.

> On 2.4, at least, I believe you are supposed to pci_map_* the item if
> it's on PCI, take the address, slam in the DMA engine, and when the
> transfer is complete, pci_unamp_* the item. I think, though, that this
> might even be some hackery since I think the address you get from
> pci_map may or may not be appropriate for the DMA engine depending on
> where it lives.

That's the problem.  pci_map_* and dma_map_* are not ALLOWED to take
fully-virtual addresses, the kind that ioremap() produces (they produce
totally bogus DMA addresses, as they just subtract off PCI_DRAM_OFFSET
currently).  Without that being allowed, how do I convert the
processor-local-bus physical address I have for a direct memory mapped
device, and generically translate it to the DMA controller's address
space?

I've been thinking about changes to the API that would help this, and
have come up with the following (for the lists' consideration, with
possible submission to the kernel mailing list):

1. dma_addr_t should be changed from a flat physical address variable
to a structure, containing both the physical address and the device it
corresponds to.
2. A macro should exist to convert this to a flat physical address:
unsigned long dma_addr_to_phys (dma_addr_t *address);
3. A macro should exist to convert it from one device's address space
to another's: dma_convert_addr (device *to_dev, dma_addr_t *to_address,
dma_addr_t *from_address);  If, due to dma_mask reasons the address is
not valid for the requested device, this could return an error
indicator, indicating the caller would have to allocate new memory and
copy the contents over.
4. The processor should have a "device" associated with it, and this
should be assumed to be the device when it is specified as NULL to the
DMA API routines: dma_alloc_coherent (NULL, ...) would mean "allocate
contiguous/coherent memory for me, with the processor's address space
used for the physical address".
5. There should be a way of taking a local bus physical address, and
converting it to the device's address space (dma_addr_t):
dma_convert_phys (device *to_dev, dma_addr_t *address, unsigned long
phys_addr).  Again, this could return an error indicator if the
dma_mask shows the address won't work.
6. With the above changes, the device or bus objects would have to
include their address-space offset from the PLB address space.  This
would handle non-PCI buses, and allow you to convert from one to
another (device1->PLB->device2).
7. It would also be REALLY nice if the DMA/PCI API could take fully
virtual addresses with the ..._map_single()/_map_sg() calls.  They are
implemented in architecture-specific areas that know how to walk their
own page tables, so why NOT allow them?  I understand the dangers of
having contiguous pages mapped to non-contiguous physical addresses,
but that's a lot easier to explain than "virtual addresses from here
will work, but not from there, and also not from THERE if your platform
is not cache-coherent).


The problem I see is that the current API assumes sysmem<->dev, with
the device itself doing the transfer.  I'm wanting to add a generic
address-space-translation API to it that will allow other kinds of DMA
transfers, and handle busses other than the PCI bus.

The main impact I see this having on drivers written with the current
API is that they need to call dma_addr_to_phys() instead of just using
the dma_addr_t as a physical address.

Comments?

John


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[Paul Mackerras]

John Whitney writes:

> As an exemplification of the problem I am having with the current
> DMA/PCI API:
>
> I am developing a Generic DMA driver.  DMA providers register
> themselves with it.  Clients ask the DMA core driver to please have a
> registered driver transfer the data from one location to another, and
> call them back when it is complete.

It sounds like you have the model of a DMA controller that is separate
from any I/O device that needs to transfer data to/from memory, and
you want to use it to transfer data from one memory location to
another.  Just like in the bad old days of the PC/XT. :)

That is not the model that PCI/DMA API is designed for.  The model it
is designed for is one where you have a bridge, often with an IOMMU
that remaps bus addresses to memory addresses on a page granularity,
and behind that you have a bus-master capable device.  Thus the
PCI/DMA API takes care of setting up an appropriate translation in the
IOMMU (if any) and returning the bus address that the driver should
use.  The way that the driver gives the bus address to the device is
very device-dependent and isn't handled by the API.

If you want to write code to handle a separate DMA controller, go
ahead.  You will need to design your own separate API for it.  I
wouldn't try to make it too general, though, since there are very few
systems these days that have DMA controllers (as distinct from
bus-master capable I/O devices).

Paul.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[Adrian Cox]

On Fri, 2004-04-02 at 23:54, Paul Mackerras wrote:

> If you want to write code to handle a separate DMA controller, go
> ahead.  You will need to design your own separate API for it.  I
> wouldn't try to make it too general, though, since there are very few
> systems these days that have DMA controllers (as distinct from
> bus-master capable I/O devices).

Actually, they are extremely common in the embedded market. On my desk I
have three Linux systems with this class of DMA engine: an MPC107/7410,
an ARM9 from Cirrus, and an ARM9 from TI. Most system-on-a-chip
processors have DMA engines which can move between any two of memory,
PCI, and internal peripherals.

- Adrian Cox


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[Adrian Cox]

On Fri, 2004-04-02 at 22:01, John Whitney wrote:

> I've been thinking about changes to the API that would help this, and
> have come up with the following (for the lists' consideration, with
> possible submission to the kernel mailing list):
>
> 1. dma_addr_t should be changed from a flat physical address variable
> to a structure, containing both the physical address and the device it
> corresponds to.

Why not leave dma_addr_t alone, and create a new type for this class of
DMA engine. The new type would be a structure. The API then would be:

dma_move(struct gen_dma *src, struct gen_dma *dest, unsigned len);

Then struct gen_dma contains a union, to allow it to refer to memory
(using the current DMA api), PCI devices, or on-chip peripherals:
struct gen_dma {
	int bus_type;
	union {
		dma_addr_t memory_address;
		unsigned long bus_address;
        }
}
#define DMA_BUS_TYPE_MEM    0
#define DMA_BUS_TYPE_PCI    1

And each platform can define extra bus types
#define DMA_BUS_TYPE_ODDBUS 2

> 2. A macro should exist to convert this to a flat physical address:
> unsigned long dma_addr_to_phys (dma_addr_t *address);

The macro to convert a struct gen_dma to a flat address would be
specific to the platform. It wouldn't be in any global header. The
conversion would only happen one line of code before writing the flat
address into the DMA engine's register.

> 3. A macro should exist to convert it from one device's address space
> to another's: dma_convert_addr (device *to_dev, dma_addr_t *to_address,
> dma_addr_t *from_address);  If, due to dma_mask reasons the address is
> not valid for the requested device, this could return an error
> indicator, indicating the caller would have to allocate new memory and
> copy the contents over.

Again, easiest to implement inside the platform specific DMA controller
code.

> 4. The processor should have a "device" associated with it, and this
> should be assumed to be the device when it is specified as NULL to the
> DMA API routines: dma_alloc_coherent (NULL, ...) would mean "allocate
> contiguous/coherent memory for me, with the processor's address space
> used for the physical address".

Actually, what might be useful would be a NULL device to indicate the
platform DMA engine.

> 5. There should be a way of taking a local bus physical address, and
> converting it to the device's address space (dma_addr_t):
> dma_convert_phys (device *to_dev, dma_addr_t *address, unsigned long
> phys_addr).  Again, this could return an error indicator if the
> dma_mask shows the address won't work.

I don't really see why we need this.

> The problem I see is that the current API assumes sysmem<->dev, with
> the device itself doing the transfer.  I'm wanting to add a generic
> address-space-translation API to it that will allow other kinds of DMA
> transfers, and handle busses other than the PCI bus.

I think that this generic API can live inside the platform specific
implementation of the DMA controller.

There are three important sorts of address:

local memory - the DMA api gives us a handle to this
PCI devices - addresses found through struct pci_dev
on-chip peripherals, platform buses - addresses defined in platform
header files

> The main impact I see this having on drivers written with the current
> API is that they need to call dma_addr_to_phys() instead of just using
> the dma_addr_t as a physical address.
>
> Comments?

Leave the current API alone. It's fine for the common class of
peripherals. This is a completely new API for embedded devices.

I've not encountered a chip with more than one platform DMA controller.
Rather than having a top level API with low-level providers, consider
picking the implementation at Kconfig time.

- Adrian Cox


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[John Whitney]

>> 1. dma_addr_t should be changed from a flat physical address variable
>> to a structure, containing both the physical address and the device it
>> corresponds to.
>
> Why not leave dma_addr_t alone, and create a new type for this class of
> DMA engine. The new type would be a structure. The API then would be:
>

One reason I wanted them to be in the same system was for the following
scenario:

I created a standard DMA memory area, passed the dma_addr_t to an
ethernet controller, and received a packet.  Now I want to pass that
packet to a decryption engine via an onboard DMA controller.  If I
could convert the DMA address suitable for the ethernet controller to
one suitable for the DMA controller, nothing else would be necessary.
Otherwise, I'll possibly have to walk the page table to convert the
virtual address to the area they got back to a physical address
suitable for the DMA controller.

> dma_move(struct gen_dma *src, struct gen_dma *dest, unsigned len);
>
> Then struct gen_dma contains a union, to allow it to refer to memory
> (using the current DMA api), PCI devices, or on-chip peripherals:
> struct gen_dma {
> 	int bus_type;
> 	union {
> 		dma_addr_t memory_address;
> 		unsigned long bus_address;
>         }
> }

This is effectively my interface, although I take an array of
src/dst/length/flag entries for scatter-gather type DMAs.  Flags allow
specification of special transfers such as "destination-held 1 byte".

> #define DMA_BUS_TYPE_MEM    0
> #define DMA_BUS_TYPE_PCI    1
>
> And each platform can define extra bus types
> #define DMA_BUS_TYPE_ODDBUS 2
>

You used dma_addr_t in your gen_dma struct type.  From what I have
seen, a dma_addr_t is ALWAYS in the PCI-bus address space.  Perhaps
just an "unsigned long" to contain a physical address in the local-bus
memory space, and the DMA driver converts this to its own address space
(more on this below)?

>> 2. A macro should exist to convert this to a flat physical address:
>> unsigned long dma_addr_to_phys (dma_addr_t *address);
>
> The macro to convert a struct gen_dma to a flat address would be
> specific to the platform. It wouldn't be in any global header. The
> conversion would only happen one line of code before writing the flat
> address into the DMA engine's register.

Are there any multi-platform DMA controllers that might make this
difficult, I wonder?

>> 5. There should be a way of taking a local bus physical address, and
>> converting it to the device's address space (dma_addr_t):
>> dma_convert_phys (device *to_dev, dma_addr_t *address, unsigned long
>> phys_addr).  Again, this could return an error indicator if the
>> dma_mask shows the address won't work.
>
> I don't really see why we need this.

To convert a PLB specific address to another device's address-space.
With the current DMA API, you can't do this without the address being
in RAM (otherwise, it wouldn't have a semi-virtual address).  On the
PowerPC side, this would usualy have been a no-op, or an addition of
PCI_DRAM_OFFSET to the given address.  Whether this is needed for the
"new" API we are discussing is another matter.

> Leave the current API alone. It's fine for the common class of
> peripherals. This is a completely new API for embedded devices.
>
> I've not encountered a chip with more than one platform DMA controller.
> Rather than having a top level API with low-level providers, consider
> picking the implementation at Kconfig time.
>

As long as each driver has a common interface, so that I could write a
module that uses that interface, and works with different DMA
controllers on different platforms, I don't have a problem with doing
it this way.

One of the nice things about having the DMA-core, however, was that it
accepted virtual addresses and handled cache consistency.  If the
driver is accepting physical addresses only, it can't do that.  This
means the client would have to allocate space using the DMA/PCI to get
the same coherency, which means all addresses will be
PCI-address-space.  Should the individual drivers accept this and just
subtract off PCI_DRAM_OFFSET (or equivalent) to get back to a physical
address?  This is what would make life hell for a cross-platform DMA
controller.

Good suggestions, thanks!

John


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[Brad Boyer]

On Sat, Apr 03, 2004 at 08:54:18AM +0100, Adrian Cox wrote:
> I've not encountered a chip with more than one platform DMA controller.
> Rather than having a top level API with low-level providers, consider
> picking the implementation at Kconfig time.

It's generally better not to decide this sort of thing at compile time
if you want it to be portable. If you want an example of a kernel that
can do everything, try compiling Linux for m68k sometime. You can have
one kernel that can support Amigas, Ataris, Macs, and a few others all
at once. I'm sure it goes without saying that pretty much everything
other than the base CPU is different on each platform.  In fact, even
just with 68k based Macs, Apple changed stuff so often that any two
models may have almost nothing in common.

	Brad Boyer
	flar@allandria.com


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[Paul Mackerras]

Brad Boyer writes:

> It's generally better not to decide this sort of thing at compile time
> if you want it to be portable. If you want an example of a kernel that
> can do everything, try compiling Linux for m68k sometime. You can have
> one kernel that can support Amigas, Ataris, Macs, and a few others all
> at once. I'm sure it goes without saying that pretty much everything
> other than the base CPU is different on each platform.  In fact, even
> just with 68k based Macs, Apple changed stuff so often that any two
> models may have almost nothing in common.

We tend not to do that with kernels for embedded systems, mainly
because memory is at a premium and people developing embedded systems
don't want to include any code that isn't going to get used.

Paul.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[Adrian Cox]

On Sat, 2004-04-03 at 18:33, Brad Boyer wrote:

> It's generally better not to decide this sort of thing at compile time
> if you want it to be portable. If you want an example of a kernel that
> can do everything, try compiling Linux for m68k sometime. You can have
> one kernel that can support Amigas, Ataris, Macs, and a few others all
> at once. I'm sure it goes without saying that pretty much everything
> other than the base CPU is different on each platform.  In fact, even
> just with 68k based Macs, Apple changed stuff so often that any two
> models may have almost nothing in common.

In the sort of embedded systems which have these DMA controllers, there
is generally no way for Linux to tell which hardware it is running on.
For example, there is no way to tell that this MPC8260 board was made by
North American Veeblefetzer, or that GPIO 15 is the MII clock and GPIO
21 is the MII data, or that those two lines were reassigned on the next
revision of the board. We're already stuck with compiling for a very
specific model, and we're frequently trying to save space in the boot
flash.

- Adrian Cox


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[Benjamin Herrenschmidt]

> Actually, they are extremely common in the embedded market. On my desk I
> have three Linux systems with this class of DMA engine: an MPC107/7410,
> an ARM9 from Cirrus, and an ARM9 from TI. Most system-on-a-chip
> processors have DMA engines which can move between any two of memory,
> PCI, and internal peripherals.

Right, but in this case, those systems also usually don't have an iommu
and usually provide a simple 1:1 mapping between memory and bus addresses ;)

Ben.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Problems with dma_alloc_coherent()

[Adrian Cox]

On Sat, 2004-04-03 at 13:43, John Whitney wrote:

> One of the nice things about having the DMA-core, however, was that it
> accepted virtual addresses and handled cache consistency.  If the
> driver is accepting physical addresses only, it can't do that.  This
> means the client would have to allocate space using the DMA/PCI to get
> the same coherency, which means all addresses will be
> PCI-address-space.  Should the individual drivers accept this and just
> subtract off PCI_DRAM_OFFSET (or equivalent) to get back to a physical
> address?  This is what would make life hell for a cross-platform DMA
> controller.

What I would like is a general API for a block move between memory and a
dumb PCI peripheral. Something like mempcy_toio() but with the
expectation of it sleeping.

Your local bus peripheral might benefit from a standard API, but I'm
still not convinced that there will be many cross-platform DMA
controllers. All the ones I've seen are tightly coupled to the
northbridge or system-on-a-chip internal bus. I'd still prefer a shared
interface rather than implementation, and I'd suggest taking an
interface proposal to the linux-arm-kernel and linuxppc-embedded lists.

- Adrian Cox


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/