[PATCH 2/2] pciehp_driver.c: PCIe hot plug for Intel iommu - call IOMMU API in hot remove

[PATCH 2/2] pciehp_driver.c: PCIe hot plug for Intel iommu - call IOMMU API in hot remove

[Fenghua Yu]

IOMMU API iommu_detach_devcie() is called to hot remove a PCIe device.

A hot added device will be added into an IOMMU domain when it first does IOMMU
op. So there is no need to add more code for hot add. 

Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>

---

 drivers/pci/pci-driver.c |    8 ++++++++
 1 files changed, 8 insertions(+)

diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c
index e5d47be..b8f0592 100644
--- a/drivers/pci/pci-driver.c
+++ b/drivers/pci/pci-driver.c
@@ -17,6 +17,7 @@
 #include <linux/slab.h>
 #include <linux/sched.h>
 #include <linux/cpu.h>
+#include <linux/iommu.h>
 #include "pci.h"
 
 struct pci_dynid {
@@ -390,6 +391,13 @@ static int pci_device_remove(struct device * dev)
 	 */
 
 	pci_dev_put(pci_dev);
+
+	/*
+	 * Remove this pci device from its iommu domain.
+	 */
+	if (iommu_found())
+		iommu_detach_device(NULL, dev);
+
 	return 0;
 }
 

Re: [PATCH 2/2] pciehp_driver.c: PCIe hot plug for Intel iommu - call IOMMU API in hot remove

[Joerg Roedel]

On Fri, Oct 02, 2009 at 02:18:44PM -0700, Fenghua Yu wrote:
> IOMMU API iommu_detach_devcie() is called to hot remove a PCIe device.
> 
> A hot added device will be added into an IOMMU domain when it first does IOMMU
> op. So there is no need to add more code for hot add. 
> 
> Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
> 
> ---
> 
>  drivers/pci/pci-driver.c |    8 ++++++++
>  1 files changed, 8 insertions(+)
> 
> diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c
> index e5d47be..b8f0592 100644
> --- a/drivers/pci/pci-driver.c
> +++ b/drivers/pci/pci-driver.c
> @@ -17,6 +17,7 @@
>  #include <linux/slab.h>
>  #include <linux/sched.h>
>  #include <linux/cpu.h>
> +#include <linux/iommu.h>
>  #include "pci.h"
>  
>  struct pci_dynid {
> @@ -390,6 +391,13 @@ static int pci_device_remove(struct device * dev)
>  	 */
>  
>  	pci_dev_put(pci_dev);
> +
> +	/*
> +	 * Remove this pci device from its iommu domain.
> +	 */
> +	if (iommu_found())
> +		iommu_detach_device(NULL, dev);
> +
>  	return 0;
>  }

NACK!

This breaks with AMD IOMMU driver.
Just implement a VT-d specific notifier callback for device removal on
the pci bus.

	Joerg

Re: [PATCH 2/2] pciehp_driver.c: PCIe hot plug for Intel iommu - call IOMMU API in hot remove

[Fenghua Yu]

> NACK!
> 
> This breaks with AMD IOMMU driver.
> Just implement a VT-d specific notifier callback for device removal on
> the pci bus.

You are right. Will change to AMD's method.

Thanks.

-Fenghua

[PATCH] PCIe hot plug for Intel iommu

[Fenghua Yu]

To support PCIe hot plug in IOMMU, we register a notifier to respond to device
change action.

When the notifier gets BUS_NOTIFY_UNBOUND_DRIVER, it removes the device from its
DMAR domain.

A hot added device will be added into an IOMMU domain when it first does IOMMU
op. So there is no need to add more code for hot add. 

Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>

---

 intel-iommu.c |   29 +++++++++++++++++++++++++++++
 1 files changed, 29 insertions(+)

diff --git a/drivers/pci/intel-iommu.c b/drivers/pci/intel-iommu.c
index 855dd7c..d8b8cfc 100644
--- a/drivers/pci/intel-iommu.c
+++ b/drivers/pci/intel-iommu.c
@@ -3193,6 +3193,33 @@ static int __init init_iommu_sysfs(void)
 }
 #endif	/* CONFIG_PM */
 
+/*
+ * Here we only respond to action of unbound device from driver.
+ *
+ * Added device is not attached to its DMAR domain here yet. That will happen
+ * when mapping the device to iova.
+ */
+static int device_notifier(struct notifier_block *nb,
+				  unsigned long action, void *data)
+{
+	struct device *dev = data;
+	struct pci_dev *pdev = to_pci_dev(dev);
+	struct dmar_domain *domain;
+
+	domain = find_domain(pdev);
+	if (!domain)
+		return 0;
+
+	if (action == BUS_NOTIFY_UNBOUND_DRIVER && !iommu_pass_through)
+		domain_remove_one_dev_info(domain, pdev);
+
+	return 0;
+}
+
+static struct notifier_block device_nb = {
+	.notifier_call = device_notifier,
+};
+
 int __init intel_iommu_init(void)
 {
 	int ret = 0;
@@ -3245,6 +3272,8 @@ int __init intel_iommu_init(void)
 
 	register_iommu(&intel_iommu_ops);
 
+	bus_register_notifier(&pci_bus_type, &device_nb);
+
 	return 0;
 }
 

Re: [PATCH] PCIe hot plug for Intel iommu

[Yinghai Lu]

On Tue, Oct 6, 2009 at 11:23 AM, Fenghua Yu <fenghua.yu@intel.com> wrote:
> To support PCIe hot plug in IOMMU, we register a notifier to respond to device
> change action.
>
> When the notifier gets BUS_NOTIFY_UNBOUND_DRIVER, it removes the device from its
> DMAR domain.
>
> A hot added device will be added into an IOMMU domain when it first does IOMMU
> op. So there is no need to add more code for hot add.
>
> Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
>
> ---
>
>  intel-iommu.c |   29 +++++++++++++++++++++++++++++
>  1 files changed, 29 insertions(+)
>
> diff --git a/drivers/pci/intel-iommu.c b/drivers/pci/intel-iommu.c
> index 855dd7c..d8b8cfc 100644
> --- a/drivers/pci/intel-iommu.c
> +++ b/drivers/pci/intel-iommu.c
> @@ -3193,6 +3193,33 @@ static int __init init_iommu_sysfs(void)
>  }
>  #endif /* CONFIG_PM */
>
> +/*
> + * Here we only respond to action of unbound device from driver.
> + *
> + * Added device is not attached to its DMAR domain here yet. That will happen
> + * when mapping the device to iova.
> + */
> +static int device_notifier(struct notifier_block *nb,
> +                                 unsigned long action, void *data)
> +{
> +       struct device *dev = data;
> +       struct pci_dev *pdev = to_pci_dev(dev);
> +       struct dmar_domain *domain;
> +
> +       domain = find_domain(pdev);
> +       if (!domain)
> +               return 0;
> +
> +       if (action == BUS_NOTIFY_UNBOUND_DRIVER && !iommu_pass_through)
> +               domain_remove_one_dev_info(domain, pdev);
> +
> +       return 0;
> +}
> +
> +static struct notifier_block device_nb = {
> +       .notifier_call = device_notifier,
> +};
> +
>  int __init intel_iommu_init(void)
>  {
>        int ret = 0;
> @@ -3245,6 +3272,8 @@ int __init intel_iommu_init(void)
>
>        register_iommu(&intel_iommu_ops);
>
> +       bus_register_notifier(&pci_bus_type, &device_nb);
> +
>        return 0;
>  }
>

it seems this one should be get into 2.6.32....and .stable

otherwise second hot plug in the same pcie slot will not work with dma.

YH