Re: drm + 4GB RAM + swiotlb = drm craps out

[David Miller <davem@davemloft.net>]

From: "Dave Airlie" <airlied@gmail.com>
Date: Mon, 2 Apr 2007 09:44:41 +1000

> Okay I've got a bug reported before and now again about > 4GB + radeon
> blows up the DRM... on Intel hw...
> 
> What the drm currently does for the PCI GART table is it allocates a
> chunk of memory (8MB) with vmalloc_32(), then when it decides to use
> it it goes through every page of it calls pci_map_single() (with
> PCI_DMA_TODEVICE, which is probably wrong...) with every page from the
> vmalloc mapping and puts the bus addresses of the pages into the PCI
> GART table on the GPU.
> 
> So when swiotlb happens, as you can guess it all falls apart as the
> drm never calls sync functions at any stage...

You would have hit this on any platform that does caching
in the PCI controller as well.

> The main problem is the ring buffer and scratch write back, these
> values are read/write from both the CPU and GPU quite a lot, so this
> leads me to think I should really just be using dma_alloc_coherent for
> the whole lot, however this is an 8MB mapping and possibly could be
> getting larger in the future and dynamic as we do dynamic PCIEGART
> support for the radeons...
> 
> So I suppose I'm asking for ideas on the "correct" way to do this, and
> perhaps any quick way to patch up the problem I'm seeing now by making
> swiotlb not get involved ....

Coherent memory was created for precisely the case where the cpu
and the device frequently access the memory.

8MB is indeed a lot for the kind of allocation that the coherent
DMA implementation uses.

Does it really have to be all in one big 8MB chunk?  I doubt it.
Perhaps you can therefore create multiple DMA pools instead?  See
include/linux/dmapool.h