Re: Proper way to check for restricted DMA addressing from device driver

[Christoph Hellwig <hch@lst.de>]

What you really needs is something like the dma_alloc_pages interface
here:

   http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/dma_alloc_pages

which has been preempted by a few other things, and the fact that
the AMD SEV encryption bit breaks various assumptions made in this
interface..

On Wed, Feb 26, 2020 at 04:44:14PM +0100, Lucas Stach wrote:
> Hi all,
> 
> I'm currently struggling with how to properly check for restricted DMA
> addressing from a device driver side. The basic issue I'm facing is
> that I have a embedded GPU, which isn't able to address all system
> memory due to interconnect being restricted to 32bit addressing. The
> limits are properly described in the system device-tree and thus
> SWIOTLB is working.
> 
> However graphics buffers are large and graphics drivers really like to
> keep the dma mapping alive for performance reasons, which means I'm
> running out of SWIOTLB space pretty easily, aside from the obvious
> performance implications of SWIOTLB.
> 
> As 3 out of the maximum 4GB system memory are located in the DMA32 zone
> and thus located in the GPU addressable space, I just want to avoid
> allocating graphics buffers outside of the DMA32 zone.
> 
> To add the DMA32 restriction to my drivers allocations, I need a
> reliable way from the device driver side to check if the GPU is in such
> a restricted system. What I'm currently doing in my WIP patch is this:
> 
>  /*
>   * If the GPU is part of a system with only 32bit bus addressing
>   * capabilities, request pages for our SHM backend buffers from the
>   * DMA32 zone to avoid performance killing SWIOTLB bounce buffering.
>   */
>  if (*gpu->dev->dma_mask < BIT_ULL(32) && !device_iommu_mapped(gpu->dev))
>          priv->shm_gfp_mask |= GFP_DMA32;
> 
> However I'm not sure if there are edge cases where this check would
> fool me. Is there any better way to check for DMA addressing
> restrictions from the device driver side?
> 
> Regards,
> Lucas
---end quoted text---