dma_alloc_coherent() on PPC32: physical addresses above 2G possible?

dma_alloc_coherent() on PPC32: physical addresses above 2G possible?

[Stefan Richter]

Hi all,

I have to implement a workaround for a PCI device which gets into 
trouble if descriptors are located at 32bit addresses, while 31bit 
addresses are fine.  I would like to avoid this workaround on machines 
on which dma_alloc_coherent() won't ever go at memory above 2 GB.

Is defined(CONFIG_PPC32) a safe test for this?  I'm under the impression 
that defined(CONFIG_X86_32) is safe.

Are there any other means to detect when the workaround can be omitted, 
at compile time or at runtime?

PS:  I don't want to set the DMA mask of this device to DMA_31BIT_MASK 
because that would be detrimental to other functions of the device. It's 
a TI TSB43AB22A FireWire controller.
-- 
Stefan Richter
-=====-==--- -=== =-=--
http://arcgraph.de/sr/

Re: dma_alloc_coherent() on PPC32: physical addresses above 2G possible?

[Arjan van de Ven]

On Sun, 20 Jul 2008 20:36:23 +0200
Stefan Richter <stefanr@s5r6.in-berlin.de> wrote:

> PS:  I don't want to set the DMA mask of this device to
> DMA_31BIT_MASK because that would be detrimental to other functions
> of the device. It's a TI TSB43AB22A FireWire controller.

Hi,

just want to mention that you can set the coherent mask separately from
the generic mask... is that sufficient for your load?
(you can even set it just for this allocation..)

-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: dma_alloc_coherent() on PPC32: physical addresses above 2G possible?

[Stefan Richter]

Arjan van de Ven wrote:
> On Sun, 20 Jul 2008 20:36:23 +0200
> Stefan Richter <stefanr@s5r6.in-berlin.de> wrote:
> 
>> PS:  I don't want to set the DMA mask of this device to
>> DMA_31BIT_MASK because that would be detrimental to other functions
>> of the device. It's a TI TSB43AB22A FireWire controller.
> 
> Hi,
> 
> just want to mention that you can set the coherent mask separately from
> the generic mask... is that sufficient for your load?
> (you can even set it just for this allocation..)

Hmm.  Would that be done this way?
During probe:

	if (chip_is_tsb43ab22a) {
		if (dma_supported(dev, DMA_31BIT_MASK))
			chip->needs_dma_mask_workaround = 1;
		else
			chip->needs_some_other_workaround = 1;
	}

Later on:

	if (dev->needs_dma_mask_workaround)
		pci_set_consistent_dma_mask(pdev, DMA_31BIT_MASK);
	allocate_something_special;
	if (dev->needs_dma_mask_workaround)
		pci_set_consistent_dma_mask(pdev, DMA_32BIT_MASK);

Or is there a variant of dma_alloc_coherent() which directly accepts a
mask argument?
-- 
Stefan Richter
-=====-==--- -=== =-=--
http://arcgraph.de/sr/

Re: dma_alloc_coherent() on PPC32: physical addresses above 2G possible?

[Arjan van de Ven]

On Sun, 20 Jul 2008 21:25:51 +0200
Stefan Richter <stefanr@s5r6.in-berlin.de> wrote:

> 
> Later on:
> 
> 	if (dev->needs_dma_mask_workaround)
> 		pci_set_consistent_dma_mask(pdev, DMA_31BIT_MASK);
> 	allocate_something_special;
> 	if (dev->needs_dma_mask_workaround)
> 		pci_set_consistent_dma_mask(pdev, DMA_32BIT_MASK);
> 
> Or is there a variant of dma_alloc_coherent() which directly accepts a
> mask argument?

something like this.
But realistically, how many consistent/coherent allocations do you have?
some ring buffers and other one time stuff surely... but not after that?

-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: dma_alloc_coherent() on PPC32: physical addresses above 2G possible?

[Stefan Richter]

Arjan van de Ven wrote:
> On Sun, 20 Jul 2008 21:25:51 +0200
> Stefan Richter <stefanr@s5r6.in-berlin.de> wrote:
>> 	if (dev->needs_dma_mask_workaround)
>> 		pci_set_consistent_dma_mask(pdev, DMA_31BIT_MASK);
>> 	allocate_something_special;
>> 	if (dev->needs_dma_mask_workaround)
>> 		pci_set_consistent_dma_mask(pdev, DMA_32BIT_MASK);
...
> something like this.
> But realistically, how many consistent/coherent allocations do you have?
> some ring buffers and other one time stuff surely... but not after that?

It's for DMA programs (i.e. lists of descriptors), not for the data DMA
buffers themselves.  FireWire controllers have several DMA contexts for
isochronous and asynchronous reception and transmission, and some
others.  Only context programs for isochronous reception need the
workaround.

We are dynamically appending new descriptors during runtime.  I.e. the
affected allocations happen during setup of the DMA context and
sometimes while the DMA context is active.  Particularly, in
drivers/firewire/fw-ohci.c:
	ohci_allocate_iso_context
		context_init
			context_add_buffer  <--
and
	ohci_queue_iso
		ohci_queue_iso_receive_dualbuffer
			context_get_descriptors
				context_add_buffer  <--

The other unaffected context types use context_add_buffer too.
-- 
Stefan Richter
-=====-==--- -=== =-=--
http://arcgraph.de/sr/

Re: dma_alloc_coherent() on PPC32: physical addresses above 2G possible?

[Arjan van de Ven]

On Sun, 20 Jul 2008 21:25:51 +0200
Stefan Richter <stefanr@s5r6.in-berlin.de> wrote:

> Arjan van de Ven wrote:
> > On Sun, 20 Jul 2008 20:36:23 +0200
> > Stefan Richter <stefanr@s5r6.in-berlin.de> wrote:
> > 
> >> PS:  I don't want to set the DMA mask of this device to
> >> DMA_31BIT_MASK because that would be detrimental to other functions
> >> of the device. It's a TI TSB43AB22A FireWire controller.
> > 
> > Hi,
> > 
> > just want to mention that you can set the coherent mask separately
> > from the generic mask... is that sufficient for your load?
> > (you can even set it just for this allocation..)
> 
> Hmm.  Would that be done this way?
> During probe:
> 
> 	if (chip_is_tsb43ab22a) {
> 		if (dma_supported(dev, DMA_31BIT_MASK))
> 			chip->needs_dma_mask_workaround = 1;
> 		else
> 			chip->needs_some_other_workaround = 1;
> 	}

btw it might be nicer to make this

chip->something_special_mask = DMA_31BIT_MASK;

then you can just use the mask from this struct rather than another
check


-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: dma_alloc_coherent() on PPC32: physical addresses above 2G possible?

[Benjamin Herrenschmidt]

On Sun, 2008-07-20 at 20:36 +0200, Stefan Richter wrote:
> Hi all,
> 
> I have to implement a workaround for a PCI device which gets into 
> trouble if descriptors are located at 32bit addresses, while 31bit 
> addresses are fine.  I would like to avoid this workaround on machines 
> on which dma_alloc_coherent() won't ever go at memory above 2 GB.
> 
> Is defined(CONFIG_PPC32) a safe test for this?  I'm under the impression 
> that defined(CONFIG_X86_32) is safe.

CONFIG_PPC32 -may- be safe today... the  current implementation
for DMA coherent machines seem to have been copied from x86 or so :-) It
sets GFP_DMA if the coherent_dma_mask < 32 bits, but we don't have
separate ZONE_NORMAL and ZONE_DMA anyway, so that is irrelevant. In any
case, it won't give you memory in highmem, so you should be safe.

In the  non-DMA coherent case, make sure you don't pass __GFP_HIGHMEM.

There's some work in progress to support swiotlb and more advanced DMA
mapping ops for PPC32 (using function pointers like PPC64), so things
will probably change. However, the need for 31 bits DMA seem to be
common enough that we should probably do something specific about it,
for example, ensure that lowmem is never > 2G (shouldn't be a big deal)
and thus make dma masks < 32 bits always clear __GFP_HIGHMEM from the
allocation mask.

Ben.