Re: [PATCH 0/5] Implement SMMU passthrough using the default domain

[Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>]

On Thu, Feb 02, 2017 at 09:15:19PM +0530, Sricharan wrote:
> Hi Rob,
> 
> >-----Original Message-----
> >From: linux-arm-kernel [mailto:linux-arm-kernel-bounces-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org] On Behalf Of Rob Clark
> >Sent: Thursday, February 02, 2017 8:33 PM
> >To: Joerg Roedel <joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org>
> >Cc: Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; Sricharan <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>; linux-arm-
> >kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org
> >Subject: Re: [PATCH 0/5] Implement SMMU passthrough using the default domain
> >
> >On Thu, Jan 26, 2017 at 12:18 PM, Joerg Roedel <joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org> wrote:
> >> On Tue, Jan 24, 2017 at 08:42:23PM +0530, Sricharan wrote:
> >>> Thanks for this series. We had a case with the GPU.
> >>> The GPU's iommu was setup by kernel and the GPU
> >>> also does dynamic updates for on-the-fly switching between
> >>> process pagetables.  GPU driver was not using DMA domain and
> >>> the GPU's firmware was always expecting to run out  of contextbank
> >>>  '0' (although not correct) , which was not the case after the DMA domain
> >>> was made default  as '0' was getting allocated for DMA domain and
> >>> there were concerns about reusing the DMA domain as well.
> >>> Now with this series, looks there is an way out of that that can be tried.
> >>>
> >>> So should the default domain not be per device specific selectable ?
> >>
> >> Note that iommu-drivers can request direct-mapping for any given device
> >> on its initializtion. This is used on x86 for devices that need a 1-1
> >> mapping for some reason.
> >>
> >> Also device drivers can use the iommu-api and assign their own domain to
> >> a device, which allows them to manage the dma address space on their
> >> own.
> >
> >Part of the problem is that dev->archdata.dma_ops gets wired up to
> >iommu_dma_ops.  Which isn't so bad on it's own, except that cache ops
> >are not exposed to drivers, forcing us to use dma-mapping API
> >(dma_map_sg, etc) for cache operations.
> >
> >Possibly we should just expose cache op's to drivers bypass this abuse
> >of dma-mapping.
> >
> [1], with this, when the default domain in not DOMAIN_DMA, then
> dev->archdata.dma_ops is not set to iommu_dma_ops , instead remains
> to be swiotlb_ops. Is that not correct for gpu's unmanaged domain case ?
> 
> https://www.spinics.net/lists/arm-kernel/msg556209.html
> 
> >btw, Will, we definitely want this to *not* rely on kcmdline for the
> >gpu with it's own private iommu case..
> 
> Ya, that changes behavior for all devices and some might want 
> DMA_DOMAIN and some UNMANAGED.

My patch changes the *default* domain. When would this ever be UNMANAGED?

If you're using an UNMANAGED domain, I think you need to take care of the
DMA ops yourself. There are likely missing functions to do that, but they
should be added in a separate series.

Will