Re: PCI DMA to small buffers on cache-incoherent arch

Albert D. Cahalan
2002-06-09T06:33:39Z

David S. Miller writes:
> From: "Albert D. Cahalan" <acahalan@cs.uml.edu>

>> On a non-SMP system, would it be OK to map all the memory
>> without memory coherency enabled? You seem to be implying that
>> one only needs to implement some mechanism in pci_map_single()
>> to handle flushing cache lines (write back, then invalidate).
>>
>> This would be useful for Macs.
>
> It's just avoiding flushing by effecting flushing the cache after
> every load/store the cpu does, so of course it would work.
>
> It would be slow as hell, but it would work.

I don't see why it would have to be slow. Mapping the memory
with coherency disabled will reduce bus traffic for all the
regular kernel code doing non-DMA stuff. (coherency requires
that the CPU generate address-only bus operations)

The obvious concern is that some kernel code might touch
a cache line after the invalidate but before the DMA is
done. If that could be considered a bug, then every non-SMP
Mac could gain some speed by turning off coherency.
Maybe the x86 MTRRs allow for something similar.

For device --> memory DMA:

1. write back cache lines that cross unaligned boundries
2. start the DMA
3. invalidate the above cache lines
4. invalidate cache lines that are fully inside the DMA

For memory --> device DMA:

1. write back all cache lines affected by the DMA
2. start the DMA
3. invalidate the above cache lines