[PATCH 0/4] iommu: Support non-paging IOMMUs

[PATCH 0/4] iommu: Support non-paging IOMMUs

[Joerg Roedel]

Hi Varun,

here is a patch-set that implements (what I think are) the necessary
IOMMU-API extensions to support the PAMU. Please have a look at it and
tell me if it fits your needs.


Regards,

	Joerg

[PATCH 1/4] iommu: Make sure DOMAIN_ATTR_MAX is really the maximum

[Joerg Roedel]

Move it to the end of the list.

Signed-off-by: Joerg Roedel <joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org>
---
 include/linux/iommu.h |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index f3b99e1..7e6ce72 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -58,8 +58,8 @@ struct iommu_domain {
 #define IOMMU_CAP_INTR_REMAP		0x2	/* isolates device intrs */
 
 enum iommu_attr {
-	DOMAIN_ATTR_MAX,
 	DOMAIN_ATTR_GEOMETRY,
+	DOMAIN_ATTR_MAX,
 };
 
 #ifdef CONFIG_IOMMU_API
-- 
1.7.9.5

[PATCH 2/4] iommu: Check for valid pgsize_bitmap in iommu_map/unmap

[Joerg Roedel]

In case the page-size bitmap is zero the code path in
iommu_map and iommu_unmap is undefined. Make it defined and
return -ENODEV in this case.

Signed-off-by: Joerg Roedel <joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org>
---
 drivers/iommu/iommu.c |    6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index ddbdaca..622360f 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -734,7 +734,8 @@ int iommu_map(struct iommu_domain *domain, unsigned long iova,
 	size_t orig_size = size;
 	int ret = 0;
 
-	if (unlikely(domain->ops->map == NULL))
+	if (unlikely(domain->ops->unmap == NULL ||
+		     domain->ops->pgsize_bitmap == 0UL))
 		return -ENODEV;
 
 	/* find out the minimum page size supported */
@@ -808,7 +809,8 @@ size_t iommu_unmap(struct iommu_domain *domain, unsigned long iova, size_t size)
 	size_t unmapped_page, unmapped = 0;
 	unsigned int min_pagesz;
 
-	if (unlikely(domain->ops->unmap == NULL))
+	if (unlikely(domain->ops->unmap == NULL ||
+		     domain->ops->pgsize_bitmap == 0UL))
 		return -ENODEV;
 
 	/* find out the minimum page size supported */
-- 
1.7.9.5

[PATCH 3/4] iommu: Implement DOMAIN_ATTR_PAGING attribute

[Joerg Roedel]

This attribute of a domain can be queried to find out if the
domain supports setting up page-tables using the iommu_map()
and iommu_unmap() functions.

Signed-off-by: Joerg Roedel <joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org>
---
 drivers/iommu/iommu.c |    5 +++++
 include/linux/iommu.h |    1 +
 2 files changed, 6 insertions(+)

diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 622360f..ab9dafd 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -869,6 +869,7 @@ int iommu_domain_get_attr(struct iommu_domain *domain,
 			  enum iommu_attr attr, void *data)
 {
 	struct iommu_domain_geometry *geometry;
+	bool *paging;
 	int ret = 0;
 
 	switch (attr) {
@@ -877,6 +878,10 @@ int iommu_domain_get_attr(struct iommu_domain *domain,
 		*geometry = domain->geometry;
 
 		break;
+	case DOMAIN_ATTR_PAGING:
+		paging  = data;
+		*paging = (domain->ops->pgsize_bitmap != 0UL);
+		break;
 	default:
 		if (!domain->ops->domain_get_attr)
 			return -EINVAL;
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index 7e6ce72..26066f5 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -59,6 +59,7 @@ struct iommu_domain {
 
 enum iommu_attr {
 	DOMAIN_ATTR_GEOMETRY,
+	DOMAIN_ATTR_PAGING,
 	DOMAIN_ATTR_MAX,
 };
 
-- 
1.7.9.5

[PATCH 4/4] iommu: Add domain window handling functions

[Joerg Roedel]

Add the iommu_domain_wnd_enable() and iommu_domain_wnd_disable()
functions to the IOMMU-API. These functions will be used to
setup domains that are based on subwindows and not on
paging.

Signed-off-by: Joerg Roedel <joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org>
---
 drivers/iommu/iommu.c |   20 ++++++++++++++++++++
 include/linux/iommu.h |   18 ++++++++++++++++++
 2 files changed, 38 insertions(+)

diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index ab9dafd..55ae3bf 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -852,6 +852,26 @@ size_t iommu_unmap(struct iommu_domain *domain, unsigned long iova, size_t size)
 }
 EXPORT_SYMBOL_GPL(iommu_unmap);
 
+
+int iommu_domain_wnd_enable(struct iommu_domain *domain, u32 window,
+			    unsigned long offset, size_t size)
+{
+	if (unlikely(domain->ops->domain_wnd_enable == NULL))
+		return -ENODEV;
+
+	return domain->ops->domain_wnd_enable(domain, window, offset, size);
+}
+EXPORT_SYMBOL_GPL(iommu_domain_wnd_enable);
+
+void iommu_domain_wnd_disable(struct iommu_domain *domain, u32 window)
+{
+	if (unlikely(domain->ops->domain_wnd_disable == NULL))
+		return;
+
+	return domain->ops->domain_wnd_disable(domain, window);
+}
+EXPORT_SYMBOL_GPL(iommu_domain_wnd_disable);
+
 static int __init iommu_init(void)
 {
 	iommu_group_kset = kset_create_and_add("iommu_groups",
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index 26066f5..f01657e 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -90,6 +90,9 @@ struct iommu_ops {
 		   phys_addr_t paddr, size_t size, int prot);
 	size_t (*unmap)(struct iommu_domain *domain, unsigned long iova,
 		     size_t size);
+	int (*domain_wnd_enable)(struct iommu_domain *domain, u32 window,
+				   unsigned long offset, size_t size);
+	void (*domain_wnd_disable)(struct iommu_domain *domain, u32 window);
 	phys_addr_t (*iova_to_phys)(struct iommu_domain *domain,
 				    unsigned long iova);
 	int (*domain_has_cap)(struct iommu_domain *domain,
@@ -123,6 +126,9 @@ extern int iommu_map(struct iommu_domain *domain, unsigned long iova,
 		     phys_addr_t paddr, size_t size, int prot);
 extern size_t iommu_unmap(struct iommu_domain *domain, unsigned long iova,
 		       size_t size);
+extern int iommu_domain_wnd_enable(struct iommu_domain *domain, u32 window,
+				   unsigned long offset, size_t size);
+extern void iommu_domain_wnd_disable(struct iommu_domain *domain, u32 window);
 extern phys_addr_t iommu_iova_to_phys(struct iommu_domain *domain,
 				      unsigned long iova);
 extern int iommu_domain_has_cap(struct iommu_domain *domain,
@@ -240,6 +246,18 @@ static inline int iommu_unmap(struct iommu_domain *domain, unsigned long iova,
 	return -ENODEV;
 }
 
+static inline int iommu_domain_wnd_enable(struct iommu_domain *domain,
+					  u32 window, unsigned long offset,
+					  size_t size)
+{
+	return -ENODEV;
+}
+
+static inline void iommu_domain_wnd_disable(struct iommu_domain *domain,
+					    u32 window)
+{
+}
+
 static inline phys_addr_t iommu_iova_to_phys(struct iommu_domain *domain,
 					     unsigned long iova)
 {
-- 
1.7.9.5

RE: [PATCH 4/4] iommu: Add domain window handling functions

[Sethi Varun-B16395]

> -----Original Message-----
> From: Joerg Roedel [mailto:joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org]
> Sent: Thursday, January 31, 2013 3:14 AM
> To: Sethi Varun-B16395
> Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Joerg
> Roedel
> Subject: [PATCH 4/4] iommu: Add domain window handling functions
> 
> Add the iommu_domain_wnd_enable() and iommu_domain_wnd_disable()
> functions to the IOMMU-API. These functions will be used to setup domains
> that are based on subwindows and not on paging.
> 
> Signed-off-by: Joerg Roedel <joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org>
> ---
>  drivers/iommu/iommu.c |   20 ++++++++++++++++++++
>  include/linux/iommu.h |   18 ++++++++++++++++++
>  2 files changed, 38 insertions(+)
> 
> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index
> ab9dafd..55ae3bf 100644
> --- a/drivers/iommu/iommu.c
> +++ b/drivers/iommu/iommu.c
> @@ -852,6 +852,26 @@ size_t iommu_unmap(struct iommu_domain *domain,
> unsigned long iova, size_t size)  }  EXPORT_SYMBOL_GPL(iommu_unmap);
> 
> +
> +int iommu_domain_wnd_enable(struct iommu_domain *domain, u32 window,
> +			    unsigned long offset, size_t size) {
> +	if (unlikely(domain->ops->domain_wnd_enable == NULL))
> +		return -ENODEV;
> +
> +	return domain->ops->domain_wnd_enable(domain, window, offset,
> size); }
> +EXPORT_SYMBOL_GPL(iommu_domain_wnd_enable);
> +
> +void iommu_domain_wnd_disable(struct iommu_domain *domain, u32 window)
> +{
> +	if (unlikely(domain->ops->domain_wnd_disable == NULL))
> +		return;
> +
> +	return domain->ops->domain_wnd_disable(domain, window); }
> +EXPORT_SYMBOL_GPL(iommu_domain_wnd_disable);
> +
>  static int __init iommu_init(void)
>  {
>  	iommu_group_kset = kset_create_and_add("iommu_groups",
> diff --git a/include/linux/iommu.h b/include/linux/iommu.h index
> 26066f5..f01657e 100644
> --- a/include/linux/iommu.h
> +++ b/include/linux/iommu.h
> @@ -90,6 +90,9 @@ struct iommu_ops {
>  		   phys_addr_t paddr, size_t size, int prot);
>  	size_t (*unmap)(struct iommu_domain *domain, unsigned long iova,
>  		     size_t size);
> +	int (*domain_wnd_enable)(struct iommu_domain *domain, u32 window,
> +				   unsigned long offset, size_t size);
> +	void (*domain_wnd_disable)(struct iommu_domain *domain, u32
> window);
>  	phys_addr_t (*iova_to_phys)(struct iommu_domain *domain,
>  				    unsigned long iova);
>  	int (*domain_has_cap)(struct iommu_domain *domain, @@ -123,6 +126,9
> @@ extern int iommu_map(struct iommu_domain *domain, unsigned long iova,
>  		     phys_addr_t paddr, size_t size, int prot);  extern
> size_t iommu_unmap(struct iommu_domain *domain, unsigned long iova,
>  		       size_t size);
> +extern int iommu_domain_wnd_enable(struct iommu_domain *domain, u32
> window,
> +				   unsigned long offset, size_t size); extern
> void
> +iommu_domain_wnd_disable(struct iommu_domain *domain, u32 window);
>  extern phys_addr_t iommu_iova_to_phys(struct iommu_domain *domain,
>  				      unsigned long iova);
>  extern int iommu_domain_has_cap(struct iommu_domain *domain, @@ -240,6
> +246,18 @@ static inline int iommu_unmap(struct iommu_domain *domain,
> unsigned long iova,
>  	return -ENODEV;
>  }
> 
> +static inline int iommu_domain_wnd_enable(struct iommu_domain *domain,
> +					  u32 window, unsigned long offset,
> +					  size_t size)
> +{
> +	return -ENODEV;
> +}
> +
> +static inline void iommu_domain_wnd_disable(struct iommu_domain *domain,
> +					    u32 window)
> +{
> +}
> +
>  static inline phys_addr_t iommu_iova_to_phys(struct iommu_domain
> *domain,
>  					     unsigned long iova)
>  {
We would need a corresponding physical address in the iommu_domain_wnd_enable call. The sub windows can point to physically discontiguous locations. Also, although we support partial mappings where the sub window size < geometry_size/max_sub_windows, but the mapping would always start from the sub window base (sub window base address would be aligned to (geometry size)/max_sub_windows). The user of the API would have to ensure that the iova is aligned to the max sub window size. So, offset is not relevant in our case as it would always be zero. The size should be u64 in order to accommodate window sizes supported by PAMU.


int iommu_domain_wnd_enable(struct iommu_domain *domain, u32 window, phys_addr_t paddr, u64 size)

We need a mechanism to determine the maximum number of subwindows supported by PAMU. How about representing it in the iommu_domain structure:
struct  iommu_domain {
	struct iommu_ops *ops;
	void *priv;
	iommu_fault_handler_t handler;
	void *handler_token;
	struct iommu_domain_geometry geometry;
	u32 max_sub_windows; -----> maximum number of sub windows supported by the hardware.
}

Also, we would need to set the number of sub windows for a geometry, for that again we would need a new domain attribute (DOMAIN_ATTR_SUBWINDOWS).

-Varun

Re: [PATCH 4/4] iommu: Add domain window handling functions

[Joerg Roedel]

On Thu, Jan 31, 2013 at 09:32:26AM +0000, Sethi Varun-B16395 wrote:
> We need a mechanism to determine the maximum number of subwindows supported by PAMU. How about representing it in the iommu_domain structure:
> struct  iommu_domain {
> 	struct iommu_ops *ops;
> 	void *priv;
> 	iommu_fault_handler_t handler;
> 	void *handler_token;
> 	struct iommu_domain_geometry geometry;
> 	u32 max_sub_windows; -----> maximum number of sub windows supported by the hardware.
> }

I`ll leave that flag to the private-data of the IOMMU domain. Instead I
added a DOMAIN_ATTR_WINDOWS attribute to get/set the number of
subwindows. I'll post the updated patch-set soon.


	Joerg

Re: [PATCH 4/4] iommu: Add domain window handling functions

[Scott Wood]

On 02/04/2013 07:02:14 AM, Joerg Roedel wrote:
> On Thu, Jan 31, 2013 at 09:32:26AM +0000, Sethi Varun-B16395 wrote:
> > We need a mechanism to determine the maximum number of subwindows  
> supported by PAMU. How about representing it in the iommu_domain  
> structure:
> > struct  iommu_domain {
> > 	struct iommu_ops *ops;
> > 	void *priv;
> > 	iommu_fault_handler_t handler;
> > 	void *handler_token;
> > 	struct iommu_domain_geometry geometry;
> > 	u32 max_sub_windows; -----> maximum number of sub windows  
> supported by the hardware.
> > }
> 
> I`ll leave that flag to the private-data of the IOMMU domain. Instead  
> I
> added a DOMAIN_ATTR_WINDOWS attribute to get/set the number of
> subwindows. I'll post the updated patch-set soon.

If it's private data, how does the caller know what sort of geometry it  
can request?

-Scott

RE: [PATCH 4/4] iommu: Add domain window handling functions

[Yoder Stuart-B08248]

> -----Original Message-----
> From: Wood Scott-B07421
> Sent: Monday, February 04, 2013 5:36 PM
> To: Joerg Roedel
> Cc: Sethi Varun-B16395; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Wood Scott-
> B07421; Yoder Stuart-B08248
> Subject: Re: [PATCH 4/4] iommu: Add domain window handling functions
> 
> On 02/04/2013 07:02:14 AM, Joerg Roedel wrote:
> > On Thu, Jan 31, 2013 at 09:32:26AM +0000, Sethi Varun-B16395 wrote:
> > > We need a mechanism to determine the maximum number of subwindows
> > supported by PAMU. How about representing it in the iommu_domain
> > structure:
> > > struct  iommu_domain {
> > > 	struct iommu_ops *ops;
> > > 	void *priv;
> > > 	iommu_fault_handler_t handler;
> > > 	void *handler_token;
> > > 	struct iommu_domain_geometry geometry;
> > > 	u32 max_sub_windows; -----> maximum number of sub windows
> > supported by the hardware.
> > > }
> >
> > I`ll leave that flag to the private-data of the IOMMU domain. Instead
> > I
> > added a DOMAIN_ATTR_WINDOWS attribute to get/set the number of
> > subwindows. I'll post the updated patch-set soon.
> 
> If it's private data, how does the caller know what sort of geometry it
> can request?

The way I thought this could work based on the patch that Joerg posted today
was that when the domain is created the caller could read the DOMAIN_ATTR_WINDOWS 
domain_get_windows() to determine the max.  The initial/default value is the
max.  The caller could then set a smaller value if they wanted to.

Stuart