[PATCH v5] xen/iommu: smmu: Advertise when the SMMU support coherent table walk

[PATCH v5] xen/iommu: smmu: Advertise when the SMMU support coherent table walk

[Julien Grall]

When SMMU doesn't support coherent table walk, Xen may need to clean
updated PT (see commit 4c5f4cb "xen/arm: p2m: Clean cache PT when the
IOMMU doesn't support coherent walk").

If one SMMU of the platform doesn't support coherent table walk, the
feature is disabled for the whole platform. This is because device is
assigned to a domain after the page table are populated.

This could impact performance on domain which doesn't use device
passthrough. But, as the spec strongly recommends the support of this
feature for mainstream platform, I expect server will always have SMMUs
supporting coherent table walk. If not, we may need to enable this feature
per-domain.

Signed-off-by: Julien Grall <julien.grall@linaro.org>

---
    I've just noticed that the support on the previous driver (i.e in
    Xen 4.5) may incorrectly expose this feature when all the SMMUs is
    not supporting coherent table walk.

    I'm not sure, if I should send a patch for it.

    Also I didn't squash this patch into "xen/iommu: smmu: Add Xen specific
    code to be able to use the driver" to help for review and to catch
    possible error in this patch.

    Changes in v5:
        - Create a separate function to find the SMMU
        - Typoes

    Changes in v4:
        - Browse the list to retrieve the last SMMU added.
        - Take the spinlock

    Changes in v3:
        - Patch added
---
 xen/drivers/passthrough/arm/smmu.c | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)

diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c
index 7675767..a7a7da9 100644
--- a/xen/drivers/passthrough/arm/smmu.c
+++ b/xen/drivers/passthrough/arm/smmu.c
@@ -2531,6 +2531,13 @@ MODULE_LICENSE("GPL v2");
 /* Xen only supports stage-2 translation, so force the value to 2. */
 static int force_stage = 2;
 
+/*
+ * Platform features. It indicates the list of features supported by all
+ * SMMUs.
+ * Actually we only care about coherent table walk.
+ */
+static u32 platform_features = ARM_SMMU_FEAT_COHERENT_WALK;
+
 static void arm_smmu_iotlb_flush_all(struct domain *d)
 {
 	struct arm_smmu_xen_domain *smmu_domain = domain_hvm_iommu(d)->arch.priv;
@@ -2668,6 +2675,10 @@ static int arm_smmu_iommu_domain_init(struct domain *d)
 
 	domain_hvm_iommu(d)->arch.priv = xen_domain;
 
+	/* Coherent walk can be enabled only when all SMMUs support it. */
+	if (platform_features & ARM_SMMU_FEAT_COHERENT_WALK)
+		iommu_set_feature(d, IOMMU_FEAT_COHERENT_WALK);
+
 	return 0;
 }
 
@@ -2738,10 +2749,28 @@ static const struct iommu_ops arm_smmu_iommu_ops = {
     .unmap_page = arm_smmu_unmap_page,
 };
 
+static __init const struct arm_smmu_device *find_smmu(const struct device *dev)
+{
+	struct arm_smmu_device *smmu;
+	bool found = false;
+
+	spin_lock(&arm_smmu_devices_lock);
+	list_for_each_entry(smmu, &arm_smmu_devices, list) {
+		if (smmu->dev == dev) {
+			found = true;
+			break;
+		}
+	}
+	spin_unlock(&arm_smmu_devices_lock);
+
+	return (found) ? smmu : NULL;
+}
+
 static __init int arm_smmu_dt_init(struct dt_device_node *dev,
 				   const void *data)
 {
 	int rc;
+	const struct arm_smmu_device *smmu;
 
 	/*
 	 * Even if the device can't be initialized, we don't want to
@@ -2755,6 +2784,12 @@ static __init int arm_smmu_dt_init(struct dt_device_node *dev,
 
 	iommu_set_ops(&arm_smmu_iommu_ops);
 
+	/* Find the last SMMU added and retrieve its features. */
+	smmu = find_smmu(dt_to_dev(dev));
+	BUG_ON(smmu == NULL);
+
+	platform_features &= smmu->features;
+
 	return 0;
 }
 
-- 
2.1.4

Re: [PATCH v5] xen/iommu: smmu: Advertise when the SMMU support coherent table walk

[Ian Campbell]

On Mon, 2015-03-02 at 15:42 +0000, Julien Grall wrote:
> When SMMU doesn't support coherent table walk, Xen may need to clean
> updated PT (see commit 4c5f4cb "xen/arm: p2m: Clean cache PT when the
> IOMMU doesn't support coherent walk").
> 
> If one SMMU of the platform doesn't support coherent table walk, the
> feature is disabled for the whole platform. This is because device is
> assigned to a domain after the page table are populated.
> 
> This could impact performance on domain which doesn't use device
> passthrough. But, as the spec strongly recommends the support of this
> feature for mainstream platform, I expect server will always have SMMUs
> supporting coherent table walk. If not, we may need to enable this feature
> per-domain.
> 
> Signed-off-by: Julien Grall <julien.grall@linaro.org>

Acked + applied,thanks.

Re: [PATCH v5] xen/iommu: smmu: Advertise when the SMMU support coherent table walk

[Ian Campbell]

On Mon, 2015-03-02 at 15:42 +0000, Julien Grall wrote:
>     I've just noticed that the support on the previous driver (i.e in
>     Xen 4.5) may incorrectly expose this feature when all the SMMUs is
>     not supporting coherent table walk.
> 
>     I'm not sure, if I should send a patch for it.

Probably not a lot of point given the dearth of hardware where it would
actually matter?

Ian.

Re: [PATCH v5] xen/iommu: smmu: Advertise when the SMMU support coherent table walk

[Julien Grall]

On 02/03/2015 17:04, Ian Campbell wrote:
> On Mon, 2015-03-02 at 15:42 +0000, Julien Grall wrote:
>>      I've just noticed that the support on the previous driver (i.e in
>>      Xen 4.5) may incorrectly expose this feature when all the SMMUs is
>>      not supporting coherent table walk.
>>
>>      I'm not sure, if I should send a patch for it.
>
> Probably not a lot of point given the dearth of hardware where it would
> actually matter?

IGiven there is a lake of PCI support in the SMMU for Xen 4.5, I guess 
we can consider the driver not usable in new hardware.

Regards,

-- 
Julien Grall