[PATCH 1/5] iommu: Allow taking a reference on a group directly

[PATCH 1/5] iommu: Allow taking a reference on a group directly

[Robin Murphy]

iommu_group_get_for_dev() expects that the IOMMU driver's device_group
callback return a group with a reference held for the given device.
Whilst allocating a new group is fine, and pci_device_group() correctly
handles reusing an existing group, there is no general means for IOMMU
drivers doing their own group lookup to take additional references on an
existing group pointer without having to also store device pointers or
resort to elaborate trickery.

Add an IOMMU-driver-specific function to fill the hole.

Signed-off-by: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>
---
 drivers/iommu/iommu.c | 14 ++++++++++++++
 include/linux/iommu.h |  1 +
 2 files changed, 15 insertions(+)

diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 9a2f1960873b..b0b052bc6bb5 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -552,6 +552,20 @@ struct iommu_group *iommu_group_get(struct device *dev)
 EXPORT_SYMBOL_GPL(iommu_group_get);
 
 /**
+ * __iommu_group_get - Increment reference on a group
+ * @group: the group to use, must not be NULL
+ *
+ * This function may be called by internal iommu driver group management
+ * when the context of a struct device pointer is not available.  It is
+ * not for general use.  Returns the given group for convenience.
+ */
+struct iommu_group *__iommu_group_get(struct iommu_group *group)
+{
+	kobject_get(group->devices_kobj);
+	return group;
+}
+
+/**
  * iommu_group_put - Decrement group reference
  * @group: the group to use
  *
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index 436dc21318af..3f5d3f9d2440 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -253,6 +253,7 @@ extern void iommu_group_remove_device(struct device *dev);
 extern int iommu_group_for_each_dev(struct iommu_group *group, void *data,
 				    int (*fn)(struct device *, void *));
 extern struct iommu_group *iommu_group_get(struct device *dev);
+extern struct iommu_group *__iommu_group_get(struct iommu_group *group);
 extern void iommu_group_put(struct iommu_group *group);
 extern int iommu_group_register_notifier(struct iommu_group *group,
 					 struct notifier_block *nb);
-- 
2.10.2.dirty

[PATCH 2/5] iommu/arm-smmu: Fix group refcounting

[Robin Murphy]

When arm_smmu_device_group() finds an existing group due to Stream ID
aliasing, it should be taking an additional reference on that group.

Reported-by: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
Signed-off-by: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>
---
 drivers/iommu/arm-smmu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c
index 8f7281444551..9f760cfba4fa 100644
--- a/drivers/iommu/arm-smmu.c
+++ b/drivers/iommu/arm-smmu.c
@@ -1478,7 +1478,7 @@ static struct iommu_group *arm_smmu_device_group(struct device *dev)
 	}
 
 	if (group)
-		return group;
+		return __iommu_group_get(group);
 
 	if (dev_is_pci(dev))
 		group = pci_device_group(dev);
-- 
2.10.2.dirty

[PATCH 3/5] iommu/amd: Fix group refcounting

[Robin Murphy]

If acpihid_device_group() finds an existing group for the relevant
devid, it should be taking an additional reference on that group.

Signed-off-by: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>
---
 drivers/iommu/amd_iommu.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
index 754595ee11b6..4ef05691ceb1 100644
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -373,6 +373,8 @@ static struct iommu_group *acpihid_device_group(struct device *dev)
 
 	if (!entry->group)
 		entry->group = generic_device_group(dev);
+	else
+		__iommu_group_get(entry->group);
 
 	return entry->group;
 }
-- 
2.10.2.dirty

[PATCH 4/5] iommu/mediatek: Fix M4Uv2 group refcounting

[Robin Murphy]

For each subsequent device assigned to the m4u_group after its initial
allocation, we need to take an additional reference.

Signed-off-by: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>
---
 drivers/iommu/mtk_iommu.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/iommu/mtk_iommu.c b/drivers/iommu/mtk_iommu.c
index b12c12d74c33..f5591f6aef49 100644
--- a/drivers/iommu/mtk_iommu.c
+++ b/drivers/iommu/mtk_iommu.c
@@ -410,6 +410,8 @@ static struct iommu_group *mtk_iommu_device_group(struct device *dev)
 		data->m4u_group = iommu_group_alloc();
 		if (IS_ERR(data->m4u_group))
 			dev_err(dev, "Failed to allocate M4U IOMMU group\n");
+	} else {
+		__iommu_group_get(data->m4u_group);
 	}
 	return data->m4u_group;
 }
-- 
2.10.2.dirty

[PATCH 5/5] iommu/mediatek: Fix M4Uv1 group refcounting

[Robin Murphy]

For each subsequent device assigned to the m4u_group after its initial
allocation, we need to take an additional reference.

Signed-off-by: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>
---
 drivers/iommu/mtk_iommu_v1.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/iommu/mtk_iommu_v1.c b/drivers/iommu/mtk_iommu_v1.c
index b8aeb0768483..1b736990373f 100644
--- a/drivers/iommu/mtk_iommu_v1.c
+++ b/drivers/iommu/mtk_iommu_v1.c
@@ -502,6 +502,8 @@ static struct iommu_group *mtk_iommu_device_group(struct device *dev)
 		data->m4u_group = iommu_group_alloc();
 		if (IS_ERR(data->m4u_group))
 			dev_err(dev, "Failed to allocate M4U IOMMU group\n");
+	} else {
+		__iommu_group_get(data->m4u_group);
 	}
 	return data->m4u_group;
 }
-- 
2.10.2.dirty

RE: [PATCH 1/5] iommu: Allow taking a reference on a group directly

[Sricharan]

Hi,

>-----Original Message-----
>From: Robin Murphy [mailto:robin.murphy-5wv7dgnIgG8@public.gmane.org]
>Sent: Wednesday, November 09, 2016 6:17 PM
>To: joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org
>Cc: will.deacon-5wv7dgnIgG8@public.gmane.org; sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org
>Subject: [PATCH 1/5] iommu: Allow taking a reference on a group directly
>
>iommu_group_get_for_dev() expects that the IOMMU driver's device_group
>callback return a group with a reference held for the given device.
>Whilst allocating a new group is fine, and pci_device_group() correctly
>handles reusing an existing group, there is no general means for IOMMU
>drivers doing their own group lookup to take additional references on an
>existing group pointer without having to also store device pointers or
>resort to elaborate trickery.
>
>Add an IOMMU-driver-specific function to fill the hole.
>
>Signed-off-by: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>
>---
> drivers/iommu/iommu.c | 14 ++++++++++++++
> include/linux/iommu.h |  1 +
> 2 files changed, 15 insertions(+)
>
>diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
>index 9a2f1960873b..b0b052bc6bb5 100644
>--- a/drivers/iommu/iommu.c
>+++ b/drivers/iommu/iommu.c
>@@ -552,6 +552,20 @@ struct iommu_group *iommu_group_get(struct device *dev)
> EXPORT_SYMBOL_GPL(iommu_group_get);
>
> /**
>+ * __iommu_group_get - Increment reference on a group
>+ * @group: the group to use, must not be NULL
>+ *
>+ * This function may be called by internal iommu driver group management
>+ * when the context of a struct device pointer is not available.  It is
>+ * not for general use.  Returns the given group for convenience.
>+ */
>+struct iommu_group *__iommu_group_get(struct iommu_group *group)
>+{
>+	kobject_get(group->devices_kobj);
>+	return group;
>+}
>+
>+/**

Acked-by: sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org

Regards,
 Sricharan

Re: [PATCH 1/5] iommu: Allow taking a reference on a group directly

[Will Deacon]

On Wed, Nov 09, 2016 at 12:47:24PM +0000, Robin Murphy wrote:
> iommu_group_get_for_dev() expects that the IOMMU driver's device_group
> callback return a group with a reference held for the given device.
> Whilst allocating a new group is fine, and pci_device_group() correctly
> handles reusing an existing group, there is no general means for IOMMU
> drivers doing their own group lookup to take additional references on an
> existing group pointer without having to also store device pointers or
> resort to elaborate trickery.
> 
> Add an IOMMU-driver-specific function to fill the hole.
> 
> Signed-off-by: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>
> ---
>  drivers/iommu/iommu.c | 14 ++++++++++++++
>  include/linux/iommu.h |  1 +
>  2 files changed, 15 insertions(+)
> 
> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
> index 9a2f1960873b..b0b052bc6bb5 100644
> --- a/drivers/iommu/iommu.c
> +++ b/drivers/iommu/iommu.c
> @@ -552,6 +552,20 @@ struct iommu_group *iommu_group_get(struct device *dev)
>  EXPORT_SYMBOL_GPL(iommu_group_get);
>  
>  /**
> + * __iommu_group_get - Increment reference on a group
> + * @group: the group to use, must not be NULL
> + *
> + * This function may be called by internal iommu driver group management
> + * when the context of a struct device pointer is not available.  It is
> + * not for general use.  Returns the given group for convenience.
> + */
> +struct iommu_group *__iommu_group_get(struct iommu_group *group)
> +{
> +	kobject_get(group->devices_kobj);
> +	return group;
> +}

This probably either wants sticking in a header or exporting to modules.
That said, why do we need the underscores and the comment about internal
group management? That's pretty much already the case for iommu_group_get.

Of course, removing the underscores gives you a naming conflict, but we
could just call it something like "iommu_group_get_ref".

Will

Re: [PATCH 1/5] iommu: Allow taking a reference on a group directly

[Robin Murphy]

On 09/11/16 17:25, Will Deacon wrote:
> On Wed, Nov 09, 2016 at 12:47:24PM +0000, Robin Murphy wrote:
>> iommu_group_get_for_dev() expects that the IOMMU driver's device_group
>> callback return a group with a reference held for the given device.
>> Whilst allocating a new group is fine, and pci_device_group() correctly
>> handles reusing an existing group, there is no general means for IOMMU
>> drivers doing their own group lookup to take additional references on an
>> existing group pointer without having to also store device pointers or
>> resort to elaborate trickery.
>>
>> Add an IOMMU-driver-specific function to fill the hole.
>>
>> Signed-off-by: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>
>> ---
>>  drivers/iommu/iommu.c | 14 ++++++++++++++
>>  include/linux/iommu.h |  1 +
>>  2 files changed, 15 insertions(+)
>>
>> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
>> index 9a2f1960873b..b0b052bc6bb5 100644
>> --- a/drivers/iommu/iommu.c
>> +++ b/drivers/iommu/iommu.c
>> @@ -552,6 +552,20 @@ struct iommu_group *iommu_group_get(struct device *dev)
>>  EXPORT_SYMBOL_GPL(iommu_group_get);
>>  
>>  /**
>> + * __iommu_group_get - Increment reference on a group
>> + * @group: the group to use, must not be NULL
>> + *
>> + * This function may be called by internal iommu driver group management
>> + * when the context of a struct device pointer is not available.  It is
>> + * not for general use.  Returns the given group for convenience.
>> + */
>> +struct iommu_group *__iommu_group_get(struct iommu_group *group)
>> +{
>> +	kobject_get(group->devices_kobj);
>> +	return group;
>> +}
> 
> This probably either wants sticking in a header or exporting to modules.
> That said, why do we need the underscores and the comment about internal
> group management? That's pretty much already the case for iommu_group_get.

The definition of struct iommu_group is private to iommu.c, so any
touching of the members has to be in here. The comment is to contrast
with iommu_group_get()'s "This function is called by iommu drivers and
users". This one is explicitly not for users of the API (DMA mapping,
VFIO, etc.), as they really have no business messing with refcounts
directly, and should always be operating in the context of a device;
it's only for the benefit of anyone *implementing* the API. And since
IOMMU drivers aren't modular (yet... ;)) there's no cause for an export.

> Of course, removing the underscores gives you a naming conflict, but we
> could just call it something like "iommu_group_get_ref".

Ideally, this would be the iommu_group_get() to precisely match
iommu_group_put(), and the existing function would renamed something
like iommu_dev_group_get() (or perhaps even all external uses converted
over to iommu_group_get_for_dev()), but that would be an awful lot of
churn for little obvious benefit. Similarly, I nearly added the below
hunk, but it didn't seem worth the bother.

Robin.

----->8-----
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 9a2f1960873b..89d509c59019 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -545,7 +545,7 @@ struct iommu_group *iommu_group_get(struct device *dev)
 	struct iommu_group *group = dev->iommu_group;

 	if (group)
-		kobject_get(group->devices_kobj);
+		__iommu_group_get(group);

 	return group;
 }

Re: [PATCH 1/5] iommu: Allow taking a reference on a group directly

[Will Deacon]

On Wed, Nov 09, 2016 at 05:46:16PM +0000, Robin Murphy wrote:
> On 09/11/16 17:25, Will Deacon wrote:
> > On Wed, Nov 09, 2016 at 12:47:24PM +0000, Robin Murphy wrote:
> >> iommu_group_get_for_dev() expects that the IOMMU driver's device_group
> >> callback return a group with a reference held for the given device.
> >> Whilst allocating a new group is fine, and pci_device_group() correctly
> >> handles reusing an existing group, there is no general means for IOMMU
> >> drivers doing their own group lookup to take additional references on an
> >> existing group pointer without having to also store device pointers or
> >> resort to elaborate trickery.
> >>
> >> Add an IOMMU-driver-specific function to fill the hole.
> >>
> >> Signed-off-by: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>
> >> ---
> >>  drivers/iommu/iommu.c | 14 ++++++++++++++
> >>  include/linux/iommu.h |  1 +
> >>  2 files changed, 15 insertions(+)
> >>
> >> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
> >> index 9a2f1960873b..b0b052bc6bb5 100644
> >> --- a/drivers/iommu/iommu.c
> >> +++ b/drivers/iommu/iommu.c
> >> @@ -552,6 +552,20 @@ struct iommu_group *iommu_group_get(struct device *dev)
> >>  EXPORT_SYMBOL_GPL(iommu_group_get);
> >>  
> >>  /**
> >> + * __iommu_group_get - Increment reference on a group
> >> + * @group: the group to use, must not be NULL
> >> + *
> >> + * This function may be called by internal iommu driver group management
> >> + * when the context of a struct device pointer is not available.  It is
> >> + * not for general use.  Returns the given group for convenience.
> >> + */
> >> +struct iommu_group *__iommu_group_get(struct iommu_group *group)
> >> +{
> >> +	kobject_get(group->devices_kobj);
> >> +	return group;
> >> +}
> > 
> > This probably either wants sticking in a header or exporting to modules.
> > That said, why do we need the underscores and the comment about internal
> > group management? That's pretty much already the case for iommu_group_get.
> 
> The definition of struct iommu_group is private to iommu.c, so any
> touching of the members has to be in here. The comment is to contrast
> with iommu_group_get()'s "This function is called by iommu drivers and
> users". This one is explicitly not for users of the API (DMA mapping,
> VFIO, etc.), as they really have no business messing with refcounts
> directly, and should always be operating in the context of a device;

But they can already do it if they want to, using the horrible group id
hack that's been doing the rounds. The IOMMU API is already low-level
enough, so I don't think trying to split it up like this is helpful.
Hell, people can even just dip in and bump the kobject directly, or grab a
handle to a device in the group already and call iommu_group_get.

That said, it doesn't look like iommu_group_get_by_id actually has any
callers in tree, so maybe we could kill it.

> it's only for the benefit of anyone *implementing* the API. And since
> IOMMU drivers aren't modular (yet... ;)) there's no cause for an export.
> 
> > Of course, removing the underscores gives you a naming conflict, but we
> > could just call it something like "iommu_group_get_ref".
> 
> Ideally, this would be the iommu_group_get() to precisely match
> iommu_group_put(), and the existing function would renamed something
> like iommu_dev_group_get() (or perhaps even all external uses converted
> over to iommu_group_get_for_dev()), but that would be an awful lot of
> churn for little obvious benefit. Similarly, I nearly added the below
> hunk, but it didn't seem worth the bother.

I'd still rather the new function was renamed. We already have the group,
so calling a weird underscore version of iommu_group_get is really
counter-intuitive.

Joerg -- do you have a preference?

Will

Re: [PATCH 1/5] iommu: Allow taking a reference on a group directly

[Joerg Roedel]

On Wed, Nov 09, 2016 at 06:00:59PM +0000, Will Deacon wrote:
> I'd still rather the new function was renamed. We already have the group,
> so calling a weird underscore version of iommu_group_get is really
> counter-intuitive.
> 
> Joerg -- do you have a preference?

iommu_group_ref_get sound best to me. Unfortunatly C has no function
overloading ;)


	Joerg