Re: IOMMU Query

[Stepan Moskovchenko <stepanm@codeaurora.org>]

On 3/23/2011 2:08 AM, Abhilash Kesavan wrote:
> Hi all,
>
Hello

> We are investigating if the generic IOMMU API (include/linux/iommu.h)
> can be used
> for the IOMMU on our arm-based boards. I noticed that msm's IOMMU code
> (arch/arm/mach-msm/iommu.c) uses the genric IOMMU API and have a few queries
> regarding the supported features. Following are some
> features/requirements of our
> system:
>
> 1) Multiple IOMMU's  - consistency between parallel IOMMUs must be maintained
> 2) Supports varying page sizes - 4K, 64K, 1M, 16M i.e. a sg list of
> contiguous physical
> memory blocks be made available. Thus, giving us mappings from the
> conservative 4K
> to large1M reducing TLB misses.
> 3) Multiple VA allocations can be associated with the same PA space.
>
Consistency between multiple IOMMUs can be maintained. Map and unmap 
operations happen on a 'domain' structure (which is intended to abstract 
a page table), and a domain can be attached to one or more IOMMU 
devices, assuming that all your IOMMUs use the same page table format. 
If not, you will need to put more "higher-level" information into your 
domain and then translate it into the HW-specific format for each IOMMU 
that you have. But it sounds pretty unlikely that you would have 
multiple IOMMUs in the same system with different page table formats. 
Thus, I think your consistency requirement should be supported, as long 
as your map/unmap function knows to iterate over all the IOMMUs attached 
to that domain and do any hardware-specific maintenance that is needed 
in addition to just updating the page table (such as invalidating 
IOTLBs, etc). The MSM driver currently does this.

Varying mapping sizes can be supported, but you will likely need an 
additional layer to organize arbitrarily-sized mappings into something 
that the IOMMU layer can easily digest. The IOMMU API takes a page order 
as an argument, and you can mix and match these (as the MSM driver does) 
but the chunk breakdown would have to come from the layer above.

Multiple VA allocations for the same PA should also be possible, and 
should really not be affected by the IOMMU API (nothing really prevents 
you from passing the same PA to multiple calls to iommu_map()). Just 
keep in mind that your multiple VA allocations need to be mapped with 
consistent attributes as per ARM spec, at least on the CPU side (not 
sure if your IOMMU HW also has this requirement).

> Who manages the virtual address space of IO device that has IOMMU? I think the
> generic iommu.h API cannot handle virtual address space and memory allocation.
> Is it just a IOMMU handler? Should we have another virtual memory
> manager for IOMMU?
>
The IOMMU layer itself does not handle virtual space management. Its 
sole responsibility is to set up the mappings. You would need something 
else to do the management for you. There have been some discussions 
about it in the past...

> Can you please tell me if the current msm sysmmu code supports some/all of the
> above features, so that we can use it as reference. Also, any pointers on areas
> which would need work would be appreciated.
>
The MSM driver supports most of the features. The only thing that is 
stubbed out right now is domain_has_cap. I believe the only place this 
is currently used is to determine cache coherency capabilities. While 
the MSM IOMMU does support cache coherent access, this needs to be 
configured through the mapping attributes to be "fully correct". I hope 
to have the has_cap stuff properly supported at a later time.

> Lastly, is there any driver making use of msm's IOMMU code - mainline
> or otherwise ?
> I wanted to check the usage of the same.
>
I don't think there is any usage of the MSM driver in mainline at this 
time. You could look around for usage examples of the API in other 
places (like KVM) but I doubt that will have the usage that you are 
looking for. There are currently efforts to use it in MSM code, but that 
code has not been sent out yet.

I hope this helps.
Steve

> Regards,
> Abhilash
> --
> To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
Sent by an employee of the Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum