[Patch V2 0/9] Refine generic/PCI MSI irqodmian interfaces

[Patch V2 0/9] Refine generic/PCI MSI irqodmian interfaces

[Jiang Liu]

This patch set is based on tip/irq/irqdomain and tries to refine
interfaces to support irqdomain for generic MSI and PCI MSI.

Patch 1 is just minor fixes for tip/irq/irqdomain.

Patch 2-4 introduce mechanisms to help reduce irqdomain users' code size.
When converting XEN to use hierarchy irqdomain, I found the code size
increases about 100 lines. So we still need helpers to reduce code size.

Patch 5 introduces some helpers to hide struct msi_desc implementation
details, so later we could move msi_list from struct pci_dev into
struct device to enable generic MSI support.

Patch 6 introduces msi_domain_{alloc|free}_irqs() which generalize
pci_msi_domain_alloc_irqs() to support generic MSI.

Patch 7 introduces default data structures and callback implementations
to support msi_domain_alloc_irqs(), so reduce burden on generic MSI
users.

Patch 8 converts PCI MSI to use generic MSI interfaces, and also
implement default callbacks for PCI MSI.

Patch 9 introduces a mechanism to replace arch_setup_msi_irq()/
arch_setup_msi_irqs()/arch_teardown_msi_irq()/arch_teardown_msi_irqs().

With this patch set applied, the generic MSI and PCI MSI interfaces
are much easier to use. For extreme case, you only need to define
a "struct msi_domain_info" and don't need to implement any callbacks,
just using the default callbacks is OK:)

This patch set is also a preparation for:
1) Kill all weak functions in drivers/pci/msi.c
2) Implement support for non-PCI-compliant MSI device

It has been tested on x86 platforms, comments are welcomed!

V1->V2:
1) Fix bugs reported by Marc
2) Rename MSI_FLAG_USE_DEF_OPS as MSI_FLAG_USE_DEF_DOM_OPS
3) Add support of MSI_FLAG_USE_DEF_CHIP_OPS
4) Include linxu/irqhandler.h to use irq_flow_handler_t instead of "void *"
5) Change iterfaces to pass in "nvect" to msi_domain_ops.msi_prepare()
6) Add new interfaces/flags to help reduce irqdomain users' code size

Jiang Liu (8):
  PCI, MSI: Fix errors caused by commit e5f1a59c4e12
  irqdomain: Implement a method to automatically call parent domain's
    alloc/free
  irqdomain: Introduce helper function irq_domain_add_hierarchy()
  PCI, MSI: Introduce helpers to hide struct msi_desc implementation
    details
  genirq: Introduce msi_domain_{alloc|free}_irqs()
  genirq: Provide default callbacks for msi_domain_ops
  PCI, MSI: Refine irqdomain interfaces to simplify its usage
  PCI, MSI: Provide mechanism to alloc/free MSI/MSIX interrupt from
    irqdomain

Yingjoe Chen (1):
  irqdomain: Use consistent prototype for irq_domain_free_irqs_*

 drivers/pci/msi.c         |  181 ++++++++++++++++++++++++++++++++++++---------
 include/linux/irqdomain.h |   33 ++++-----
 include/linux/msi.h       |   99 ++++++++++++++++++++++---
 kernel/irq/irqdomain.c    |   93 +++++++++++++++++++++--
 kernel/irq/msi.c          |  169 ++++++++++++++++++++++++++++++++++++++++--
 5 files changed, 502 insertions(+), 73 deletions(-)

-- 
1.7.10.4

[Patch V2 1/9] PCI, MSI: Fix errors caused by commit e5f1a59c4e12

[Jiang Liu]

Better to fold into commit e5f1a59c4e12 ("PCI/MSI: Rename write_msi_msg()
to pci_write_msi_msg()").

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
---
 drivers/pci/msi.c   |    2 +-
 include/linux/msi.h |    2 --
 2 files changed, 1 insertion(+), 3 deletions(-)

diff --git a/drivers/pci/msi.c b/drivers/pci/msi.c
index 564850b1316e..9c53b865cb1b 100644
--- a/drivers/pci/msi.c
+++ b/drivers/pci/msi.c
@@ -1084,7 +1084,7 @@ void pci_msi_domain_write_msg(struct irq_data *irq_data, struct msi_msg *msg)
 	 * MSI message denotes a contiguous group of IRQs, written for 0th IRQ.
 	 */
 	if (desc->irq == irq_data->irq)
-		pci_write_msi_msg(desc, msg);
+		__pci_write_msi_msg(desc, msg);
 }
 
 /*
diff --git a/include/linux/msi.h b/include/linux/msi.h
index 8112a17cdca1..190c7abbec84 100644
--- a/include/linux/msi.h
+++ b/include/linux/msi.h
@@ -44,11 +44,9 @@ struct msi_desc {
 	struct msi_msg msg;
 };
 
-#ifdef CONFIG_PCI_MSI
 void __pci_read_msi_msg(struct msi_desc *entry, struct msi_msg *msg);
 void __pci_write_msi_msg(struct msi_desc *entry, struct msi_msg *msg);
 void pci_write_msi_msg(unsigned int irq, struct msi_msg *msg);
-#endif
 
 /*
  * The arch hooks to setup up msi irqs. Those functions are
-- 
1.7.10.4

[Patch V2 2/9] irqdomain: Use consistent prototype for irq_domain_free_irqs_*

[Jiang Liu]

From: Yingjoe Chen <yingjoe.chen@mediatek.com>

When using irq_domain_free_irqs_top() directly in irq_domain_ops, gcc
generate the following warnings:

../drivers/irqchip/irq-gic.c:879:2: warning: initialization from incompatible pointer type [enabled by default]
../drivers/irqchip/irq-gic.c:879:2: warning: (near initialization for 'gic_irq_domain_hierarchy_ops.free') [enabled by default]

Change to use consistent prototype for all irq_domain_free_irqs*

Better to fold this into "irqdomain: Introduce new interfaces to
support hierarchy irqdomains".

Signed-off-by: Yingjoe Chen <yingjoe.chen@mediatek.com>
Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
---
 include/linux/irqdomain.h |    9 +++++----
 kernel/irq/irqdomain.c    |    8 ++++----
 2 files changed, 9 insertions(+), 8 deletions(-)

diff --git a/include/linux/irqdomain.h b/include/linux/irqdomain.h
index 640a1ec54772..fbe542967c20 100644
--- a/include/linux/irqdomain.h
+++ b/include/linux/irqdomain.h
@@ -270,12 +270,13 @@ extern void irq_domain_set_info(struct irq_domain *domain, unsigned int virq,
 				void *handler_data, const char *handler_name);
 extern void irq_domain_reset_irq_data(struct irq_data *irq_data);
 extern void irq_domain_free_irqs_common(struct irq_domain *domain,
-					int virq, int nr_irqs);
+					unsigned int virq,
+					unsigned int nr_irqs);
 extern void irq_domain_free_irqs_top(struct irq_domain *domain,
-				     int virq, int nr_irqs);
+				     unsigned int virq, unsigned int nr_irqs);
 
 static inline int irq_domain_alloc_irqs_parent(struct irq_domain *domain,
-				int irq_base, unsigned int nr_irqs, void *arg)
+			unsigned int irq_base, unsigned int nr_irqs, void *arg)
 {
 	if (domain->parent && domain->parent->ops->alloc)
 		return domain->parent->ops->alloc(domain->parent, irq_base,
@@ -284,7 +285,7 @@ static inline int irq_domain_alloc_irqs_parent(struct irq_domain *domain,
 }
 
 static inline void irq_domain_free_irqs_parent(struct irq_domain *domain,
-					int irq_base, unsigned int nr_irqs)
+				unsigned int irq_base, unsigned int nr_irqs)
 {
 	if (domain->parent && domain->parent->ops->free)
 		domain->parent->ops->free(domain->parent, irq_base, nr_irqs);
diff --git a/kernel/irq/irqdomain.c b/kernel/irq/irqdomain.c
index 8b0eddee0b21..705fb573e509 100644
--- a/kernel/irq/irqdomain.c
+++ b/kernel/irq/irqdomain.c
@@ -899,8 +899,8 @@ void irq_domain_reset_irq_data(struct irq_data *irq_data)
 	irq_data->chip_data = NULL;
 }
 
-void irq_domain_free_irqs_common(struct irq_domain *domain, int virq,
-				 int nr_irqs)
+void irq_domain_free_irqs_common(struct irq_domain *domain, unsigned int virq,
+				 unsigned int nr_irqs)
 {
 	int i;
 	struct irq_data *irq_data;
@@ -913,8 +913,8 @@ void irq_domain_free_irqs_common(struct irq_domain *domain, int virq,
 	irq_domain_free_irqs_parent(domain, virq, nr_irqs);
 }
 
-void irq_domain_free_irqs_top(struct irq_domain *domain, int virq,
-			      int nr_irqs)
+void irq_domain_free_irqs_top(struct irq_domain *domain, unsigned int virq,
+			      unsigned int nr_irqs)
 {
 	int i;
 
-- 
1.7.10.4

[Patch V2 3/9] irqdomain: Implement a method to automatically call parent domain's alloc/free

[Jiang Liu]

Add a flags to irq_domain.flags to control whether the irqdomain core
should automatically call parent irqdomain's alloc/free callbacks. It
help to reduce hierarchy irqdomains users' code size.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
---
 include/linux/irqdomain.h |   24 ++++++-----------
 kernel/irq/irqdomain.c    |   66 +++++++++++++++++++++++++++++++++++++++++++--
 2 files changed, 72 insertions(+), 18 deletions(-)

diff --git a/include/linux/irqdomain.h b/include/linux/irqdomain.h
index fbe542967c20..ad92b0b15945 100644
--- a/include/linux/irqdomain.h
+++ b/include/linux/irqdomain.h
@@ -130,6 +130,8 @@ struct irq_domain {
 };
 
 #define	IRQ_DOMAIN_FLAG_HIERARCHY	0x1
+/* Framework automatically calls parent domain's alloc()/free() */
+#define	IRQ_DOMAIN_FLAG_AUTO_RECURSIVE	0x2
 #define	IRQ_DOMAIN_FLAG_ARCH1		0x10000
 
 #ifdef CONFIG_IRQ_DOMAIN
@@ -274,22 +276,12 @@ extern void irq_domain_free_irqs_common(struct irq_domain *domain,
 					unsigned int nr_irqs);
 extern void irq_domain_free_irqs_top(struct irq_domain *domain,
 				     unsigned int virq, unsigned int nr_irqs);
-
-static inline int irq_domain_alloc_irqs_parent(struct irq_domain *domain,
-			unsigned int irq_base, unsigned int nr_irqs, void *arg)
-{
-	if (domain->parent && domain->parent->ops->alloc)
-		return domain->parent->ops->alloc(domain->parent, irq_base,
-						  nr_irqs, arg);
-	return -ENOSYS;
-}
-
-static inline void irq_domain_free_irqs_parent(struct irq_domain *domain,
-				unsigned int irq_base, unsigned int nr_irqs)
-{
-	if (domain->parent && domain->parent->ops->free)
-		domain->parent->ops->free(domain->parent, irq_base, nr_irqs);
-}
+extern int irq_domain_alloc_irqs_parent(struct irq_domain *domain,
+					unsigned int irq_base,
+					unsigned int nr_irqs, void *arg);
+extern void irq_domain_free_irqs_parent(struct irq_domain *domain,
+					unsigned int irq_base,
+					unsigned int nr_irqs);
 
 static inline bool irq_domain_is_hierarchy(struct irq_domain *domain)
 {
diff --git a/kernel/irq/irqdomain.c b/kernel/irq/irqdomain.c
index 705fb573e509..03fc7d018324 100644
--- a/kernel/irq/irqdomain.c
+++ b/kernel/irq/irqdomain.c
@@ -925,6 +925,43 @@ void irq_domain_free_irqs_top(struct irq_domain *domain, unsigned int virq,
 	irq_domain_free_irqs_common(domain, virq, nr_irqs);
 }
 
+static bool irq_domain_is_auto_recursive(struct irq_domain *domain)
+{
+	return domain->flags & IRQ_DOMAIN_FLAG_AUTO_RECURSIVE;
+}
+
+static void irq_domain_free_irqs_recursive(struct irq_domain *domain,
+					   unsigned int irq_base,
+					   unsigned int nr_irqs)
+{
+	domain->ops->free(domain, irq_base, nr_irqs);
+	if (irq_domain_is_auto_recursive(domain)) {
+		BUG_ON(!domain->parent);
+		irq_domain_free_irqs_recursive(domain->parent, irq_base,
+					       nr_irqs);
+	}
+}
+
+static int irq_domain_alloc_irqs_recursive(struct irq_domain *domain,
+					   unsigned int irq_base,
+					   unsigned int nr_irqs, void *arg)
+{
+	int ret = 0;
+	struct irq_domain *parent = domain->parent;
+	bool recursive = irq_domain_is_auto_recursive(domain);
+
+	BUG_ON(recursive && !parent);
+	if (recursive)
+		ret = irq_domain_alloc_irqs_recursive(parent, irq_base,
+						      nr_irqs, arg);
+	if (ret >= 0)
+		ret = domain->ops->alloc(domain, irq_base, nr_irqs, arg);
+	if (ret < 0 && recursive)
+		irq_domain_free_irqs_recursive(parent, irq_base, nr_irqs);
+
+	return ret;
+}
+
 /**
  * __irq_domain_alloc_irqs - Allocate IRQs from domain
  * @domain: domain to allocate from
@@ -981,7 +1018,7 @@ int __irq_domain_alloc_irqs(struct irq_domain *domain, int irq_base,
 	}
 
 	mutex_lock(&irq_domain_mutex);
-	ret = domain->ops->alloc(domain, virq, nr_irqs, arg);
+	ret = irq_domain_alloc_irqs_recursive(domain, virq, nr_irqs, arg);
 	if (ret < 0) {
 		mutex_unlock(&irq_domain_mutex);
 		goto out_free_irq_data;
@@ -1016,13 +1053,38 @@ void irq_domain_free_irqs(unsigned int virq, unsigned int nr_irqs)
 	mutex_lock(&irq_domain_mutex);
 	for (i = 0; i < nr_irqs; i++)
 		irq_domain_remove_irq(virq + i);
-	data->domain->ops->free(data->domain, virq, nr_irqs);
+	irq_domain_free_irqs_recursive(data->domain, virq, nr_irqs);
 	mutex_unlock(&irq_domain_mutex);
 
 	irq_domain_free_irq_data(virq, nr_irqs);
 	irq_free_descs(virq, nr_irqs);
 }
 
+int irq_domain_alloc_irqs_parent(struct irq_domain *domain,
+				 unsigned int irq_base, unsigned int nr_irqs,
+				 void *arg)
+{
+	/* irq_domain_alloc_irqs_recursive() has called parent's alloc() */
+	if (irq_domain_is_auto_recursive(domain))
+		return 0;
+
+	domain = domain->parent;
+	if (domain)
+		return irq_domain_alloc_irqs_recursive(domain, irq_base,
+						       nr_irqs, arg);
+
+	return -ENOSYS;
+}
+
+void irq_domain_free_irqs_parent(struct irq_domain *domain,
+				 unsigned int irq_base, unsigned int nr_irqs)
+{
+	/* irq_domain_free_irqs_recursive() will call parent's free */
+	if (!irq_domain_is_auto_recursive(domain))
+		irq_domain_free_irqs_recursive(domain->parent, irq_base,
+					       nr_irqs);
+}
+
 /**
  * irq_domain_activate_irq - Call domain_ops->activate recursively to activate
  *			     interrupt
-- 
1.7.10.4

[Patch V2 4/9] irqdomain: Introduce helper function irq_domain_add_hierarchy()

[Jiang Liu]

Introduce helper function irq_domain_add_hierarchy(), which creates
a linear irqdomain if parameter 'size' is not zero, otherwise creates
a tree irqdomain.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
---
 include/linux/irqdomain.h |    4 ++++
 kernel/irq/irqdomain.c    |   19 +++++++++++++++++++
 2 files changed, 23 insertions(+)

diff --git a/include/linux/irqdomain.h b/include/linux/irqdomain.h
index ad92b0b15945..30e8c753fa98 100644
--- a/include/linux/irqdomain.h
+++ b/include/linux/irqdomain.h
@@ -248,6 +248,10 @@ int irq_domain_xlate_onetwocell(struct irq_domain *d, struct device_node *ctrlr,
 extern struct irq_data *irq_domain_get_irq_data(struct irq_domain *domain,
 						unsigned int virq);
 #ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
+extern struct irq_domain *irq_domain_add_hierarchy(struct irq_domain *parent,
+			unsigned int flags, unsigned int size,
+			struct device_node *node,
+			const struct irq_domain_ops *ops, void *host_data);
 extern int __irq_domain_alloc_irqs(struct irq_domain *domain, int irq_base,
 				   unsigned int nr_irqs, int node, void *arg,
 				   bool realloc);
diff --git a/kernel/irq/irqdomain.c b/kernel/irq/irqdomain.c
index 03fc7d018324..73c66d6f9ffc 100644
--- a/kernel/irq/irqdomain.c
+++ b/kernel/irq/irqdomain.c
@@ -742,6 +742,25 @@ static int irq_domain_alloc_descs(int virq, unsigned int cnt,
 }
 
 #ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
+struct irq_domain *irq_domain_add_hierarchy(struct irq_domain *parent,
+			unsigned int flags, unsigned int size,
+			struct device_node *node,
+			const struct irq_domain_ops *ops, void *host_data)
+{
+	struct irq_domain *domain;
+
+	if (size)
+		domain = irq_domain_add_linear(node, size, ops, host_data);
+	else
+		domain = irq_domain_add_tree(node, ops, host_data);
+	if (domain) {
+		domain->parent = parent;
+		domain->flags |= flags;
+	}
+
+	return domain;
+}
+
 static void irq_domain_insert_irq(int virq)
 {
 	struct irq_data *data;
-- 
1.7.10.4

[Patch V2 5/9] PCI, MSI: Introduce helpers to hide struct msi_desc implementation details

[Jiang Liu]

Introduce helpers to hide struct msi_desc implementation details,
so we could easily support non-PCI-compliant MSI devices later by
moving msi_list into struct device.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
---
 include/linux/msi.h |   19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/include/linux/msi.h b/include/linux/msi.h
index 190c7abbec84..714716a3ffdd 100644
--- a/include/linux/msi.h
+++ b/include/linux/msi.h
@@ -44,6 +44,25 @@ struct msi_desc {
 	struct msi_msg msg;
 };
 
+/* Helpers to hide struct msi_desc implementation details */
+#define msi_desc_to_dev(desc)		(&(desc)->dev.dev)
+#define dev_to_msi_list(dev)		(&to_pci_dev((dev))->msi_list)
+#define first_msi_entry(dev)		\
+	list_first_entry(dev_to_msi_list((dev)), struct msi_desc, list)
+#define for_each_msi_entry(desc, dev)	\
+	list_for_each_entry((desc), dev_to_msi_list((dev)), list)
+
+#ifdef CONFIG_PCI_MSI
+#define first_pci_msi_entry(pdev)	first_msi_entry(&(pdev)->dev)
+#define for_each_pci_msi_entry(desc, pdev)	\
+	for_each_msi_entry((desc), &(pdev)->dev)
+
+static inline struct pci_dev *msi_desc_to_pci_dev(struct msi_desc *desc)
+{
+	return desc->dev;
+}
+#endif /* CONFIG_PCI_MSI */
+
 void __pci_read_msi_msg(struct msi_desc *entry, struct msi_msg *msg);
 void __pci_write_msi_msg(struct msi_desc *entry, struct msi_msg *msg);
 void pci_write_msi_msg(unsigned int irq, struct msi_msg *msg);
-- 
1.7.10.4

[Patch V2 6/9] genirq: Introduce msi_domain_{alloc|free}_irqs()

[Jiang Liu]

Introduce msi_domain_{alloc|free}_irqs() to alloc/free interrupts
from generic MSI irqdomain.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
---
 include/linux/msi.h |   39 +++++++++++++++++++++++++++++++++
 kernel/irq/msi.c    |   60 +++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 99 insertions(+)

diff --git a/include/linux/msi.h b/include/linux/msi.h
index 714716a3ffdd..7283e155afef 100644
--- a/include/linux/msi.h
+++ b/include/linux/msi.h
@@ -3,6 +3,7 @@
 
 #include <linux/kobject.h>
 #include <linux/list.h>
+#include <asm/hw_irq.h>		/* for msi_alloc_info_t */
 
 struct msi_msg {
 	u32	address_lo;	/* low 32 bits of msi message address */
@@ -100,7 +101,29 @@ struct irq_chip;
 struct device_node;
 struct msi_domain_info;
 
+#ifndef NUM_MSI_ALLOC_SCRATCHPAD_REGS
+#define NUM_MSI_ALLOC_SCRATCHPAD_REGS	2
+#endif
+
+/*
+ * Default structure for MSI interrupt allocation.
+ * Arch may overwrite it by defining msi_alloc_info_t.
+ */
+struct msi_alloc_info {
+	struct msi_desc			*desc;
+	irq_hw_number_t			hwirq;
+	union {
+		unsigned long		ul;
+		void			*ptr;
+	} scratchpad[NUM_MSI_ALLOC_SCRATCHPAD_REGS];
+};
+
+#ifndef msi_alloc_info_t
+typedef struct msi_alloc_info msi_alloc_info_t;
+#endif
+
 struct msi_domain_ops {
+	/* Callbacks for msi_create_irq_domain() */
 	void		(*calc_hwirq)(struct msi_domain_info *info, void *arg,
 				      struct msi_desc *desc);
 	irq_hw_number_t	(*get_hwirq)(struct msi_domain_info *info, void *arg);
@@ -111,6 +134,19 @@ struct msi_domain_ops {
 	void		(*msi_free)(struct irq_domain *domain,
 				    struct msi_domain_info *info,
 				    unsigned int virq);
+
+	/* Callbacks for msi_irq_domain_alloc_irqs() based on msi_descs */
+	int		(*msi_check)(struct irq_domain *domain,
+				     struct msi_domain_info *info,
+				     struct device *dev);
+	int		(*msi_prepare)(struct irq_domain *domain,
+				       struct device *dev, int nvec,
+				       msi_alloc_info_t *arg);
+	void		(*msi_finish)(msi_alloc_info_t *arg, int retval);
+	void		(*set_desc)(msi_alloc_info_t *arg,
+				    struct msi_desc *desc);
+	int		(*handle_error)(struct irq_domain *domain,
+					struct msi_desc *desc, int error);
 };
 
 struct msi_domain_info {
@@ -125,6 +161,9 @@ int msi_domain_set_affinity(struct irq_data *data, const struct cpumask *mask,
 struct irq_domain *msi_create_irq_domain(struct device_node *of_node,
 					 struct msi_domain_info *info,
 					 struct irq_domain *parent);
+int msi_domain_alloc_irqs(struct irq_domain *domain, struct device *dev,
+			  int nvec);
+void msi_domain_free_irqs(struct irq_domain *domain, struct device *dev);
 struct msi_domain_info *msi_get_domain_info(struct irq_domain *domain);
 
 #endif /* CONFIG_GENERIC_MSI_IRQ_DOMAIN */
diff --git a/kernel/irq/msi.c b/kernel/irq/msi.c
index 94d6d87ee23e..ce054566edf5 100644
--- a/kernel/irq/msi.c
+++ b/kernel/irq/msi.c
@@ -13,6 +13,9 @@
 #include <linux/irqdomain.h>
 #include <linux/msi.h>
 
+/* Temparory solution for building, will be removed later */
+#include <linux/pci.h>
+
 void __get_cached_msi_msg(struct msi_desc *entry, struct msi_msg *msg)
 {
 	*msg = entry->msg;
@@ -124,6 +127,63 @@ struct irq_domain *msi_create_irq_domain(struct device_node *of_node,
 	return domain;
 }
 
+int msi_domain_alloc_irqs(struct irq_domain *domain, struct device *dev,
+			  int nvec)
+{
+	int i, ret, virq = -1;
+	msi_alloc_info_t arg;
+	struct msi_desc *desc;
+	struct msi_domain_info *info = domain->host_data;
+	struct msi_domain_ops *ops = info->ops;
+
+	ret = ops->msi_check(domain, info, dev);
+	if (ret == 0)
+		ret = ops->msi_prepare(domain, dev, nvec, &arg);
+	if (ret)
+		return ret;
+
+	for_each_msi_entry(desc, dev) {
+		ops->set_desc(&arg, desc);
+
+		virq = __irq_domain_alloc_irqs(domain, -1, desc->nvec_used,
+					       dev_to_node(dev), &arg, false);
+		if (virq < 0) {
+			ret = -ENOSPC;
+			if (ops->handle_error)
+				ret = ops->handle_error(domain, desc, ret);
+			if (ops->msi_finish)
+				ops->msi_finish(&arg, ret);
+			return ret;
+		}
+
+		for (i = 0; i < desc->nvec_used; i++)
+			irq_set_msi_desc_off(virq, i, desc);
+	}
+
+	if (ops->msi_finish)
+		ops->msi_finish(&arg, 0);
+
+	for_each_msi_entry(desc, dev) {
+		if (desc->nvec_used == 1)
+			dev_dbg(dev, "irq %d for MSI\n", virq);
+		else
+			dev_dbg(dev, "irq [%d-%d] for MSI\n",
+				virq, virq + desc->nvec_used - 1);
+	}
+
+	return 0;
+}
+
+void msi_domain_free_irqs(struct irq_domain *domain, struct device *dev)
+{
+	struct msi_desc *desc;
+
+	for_each_msi_entry(desc, dev) {
+		irq_domain_free_irqs(desc->irq, desc->nvec_used);
+		desc->irq = 0;
+	}
+}
+
 struct msi_domain_info *msi_get_domain_info(struct irq_domain *domain)
 {
 	return (struct msi_domain_info *)domain->host_data;
-- 
1.7.10.4

[Patch V2 7/9] genirq: Provide default callbacks for msi_domain_ops

[Jiang Liu]

Extend struct msi_domain_info and provide default callbacks for
msi_domain_ops.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
---
 include/linux/msi.h |   29 ++++++++++---
 kernel/irq/msi.c    |  113 +++++++++++++++++++++++++++++++++++++++++++++++----
 2 files changed, 129 insertions(+), 13 deletions(-)

diff --git a/include/linux/msi.h b/include/linux/msi.h
index 7283e155afef..96ee8c3b98cc 100644
--- a/include/linux/msi.h
+++ b/include/linux/msi.h
@@ -3,6 +3,7 @@
 
 #include <linux/kobject.h>
 #include <linux/list.h>
+#include <linux/irqhandler.h>
 #include <asm/hw_irq.h>		/* for msi_alloc_info_t */
 
 struct msi_msg {
@@ -124,13 +125,15 @@ typedef struct msi_alloc_info msi_alloc_info_t;
 
 struct msi_domain_ops {
 	/* Callbacks for msi_create_irq_domain() */
-	void		(*calc_hwirq)(struct msi_domain_info *info, void *arg,
+	void		(*calc_hwirq)(struct msi_domain_info *info,
+				      msi_alloc_info_t *arg,
 				      struct msi_desc *desc);
-	irq_hw_number_t	(*get_hwirq)(struct msi_domain_info *info, void *arg);
+	irq_hw_number_t	(*get_hwirq)(struct msi_domain_info *info,
+				     msi_alloc_info_t *arg);
 	int		(*msi_init)(struct irq_domain *domain,
 				    struct msi_domain_info *info,
 				    unsigned int virq, irq_hw_number_t hwirq,
-				    void *arg);
+				    msi_alloc_info_t *arg);
 	void		(*msi_free)(struct irq_domain *domain,
 				    struct msi_domain_info *info,
 				    unsigned int virq);
@@ -150,15 +153,31 @@ struct msi_domain_ops {
 };
 
 struct msi_domain_info {
+	u32			flags;
 	struct msi_domain_ops	*ops;
 	struct irq_chip		*chip;
-	void			*data;
+	void			*chip_data;	/* optional chip data */
+	irq_flow_handler_t	handler;	/* optional flow handler */
+	void			*handler_data;	/* optional handler data */
+	const char		*handler_name;	/* optional handler name */
+	void			*data;		/* optional private data */
 };
 
+/* Use default MSI domain ops if possible */
+#define MSI_FLAG_USE_DEF_DOM_OPS	0x1
+/* Use default MSI chip ops if possible */
+#define MSI_FLAG_USE_DEF_CHIP_OPS	0x2
+/* Build identity map between hwirq and irq */
+#define MSI_FLAG_IDENTITY_MAP		0x10
+/* Support multiple PCI MSI interrupts */
+#define MSI_FLAG_MULTI_PCI_MSI		0x100
+/* Support PCI MSIX interrupts */
+#define MSI_FLAG_PCI_MSIX		0x200
+
 int msi_domain_set_affinity(struct irq_data *data, const struct cpumask *mask,
 			    bool force);
 
-struct irq_domain *msi_create_irq_domain(struct device_node *of_node,
+struct irq_domain *msi_create_irq_domain(struct device_node *node,
 					 struct msi_domain_info *info,
 					 struct irq_domain *parent);
 int msi_domain_alloc_irqs(struct irq_domain *domain, struct device *dev,
diff --git a/kernel/irq/msi.c b/kernel/irq/msi.c
index ce054566edf5..ff15fd643b3d 100644
--- a/kernel/irq/msi.c
+++ b/kernel/irq/msi.c
@@ -9,6 +9,8 @@
  * This file contains common code to support Message Signalled Interrupt for
  * PCI compatible and non PCI compatible devices.
  */
+#include <linux/types.h>
+#include <linux/device.h>
 #include <linux/irq.h>
 #include <linux/irqdomain.h>
 #include <linux/msi.h>
@@ -114,17 +116,108 @@ static struct irq_domain_ops msi_domain_ops = {
 	.deactivate	= msi_domain_deactivate,
 };
 
-struct irq_domain *msi_create_irq_domain(struct device_node *of_node,
+#ifndef msi_alloc_info_t
+static irq_hw_number_t msi_domain_ops_get_hwirq(struct msi_domain_info *info,
+						struct msi_alloc_info *arg)
+{
+	return arg->hwirq;
+}
+
+static int msi_domain_ops_prepare(struct irq_domain *domain, struct device *dev,
+				  int nvec, struct msi_alloc_info *arg)
+{
+	memset(arg, 0, sizeof(*arg));
+
+	return 0;
+}
+
+static void msi_domain_ops_set_desc(struct msi_alloc_info *arg,
+				    struct msi_desc *desc)
+{
+	arg->desc = desc;
+}
+#else
+#define msi_domain_ops_get_hwirq	NULL
+#define msi_domain_ops_prepare		NULL
+#define msi_domain_ops_set_desc		NULL
+#endif /* msi_alloc_info_t */
+
+static int msi_domain_ops_init(struct irq_domain *domain,
+			       struct msi_domain_info *info,
+			       unsigned int virq, irq_hw_number_t hwirq,
+			       msi_alloc_info_t *arg)
+{
+	irq_domain_set_hwirq_and_chip(domain, virq, hwirq, info->chip,
+				      info->chip_data);
+	if (info->handler && info->handler_name) {
+		__irq_set_handler(virq, info->handler, 0, info->handler_name);
+		if (info->handler_data)
+			irq_set_handler_data(virq, info->handler_data);
+	}
+
+	return 0;
+}
+
+static int msi_domain_ops_check(struct irq_domain *domain,
+				struct msi_domain_info *info,
+				struct device *dev)
+{
+	return 0;
+}
+
+static struct msi_domain_ops msi_domain_ops_default = {
+	.get_hwirq	= msi_domain_ops_get_hwirq,
+	.msi_init	= msi_domain_ops_init,
+	.msi_check	= msi_domain_ops_check,
+	.msi_prepare	= msi_domain_ops_prepare,
+	.set_desc	= msi_domain_ops_set_desc,
+};
+
+static void msi_domain_update_dom_ops(struct msi_domain_info *info)
+{
+	struct msi_domain_ops *ops = info->ops;
+
+	if (ops == NULL) {
+		info->ops = &msi_domain_ops_default;
+		return;
+	}
+
+	if (ops->get_hwirq == NULL)
+		ops->get_hwirq = msi_domain_ops_default.get_hwirq;
+	if (ops->msi_init == NULL)
+		ops->msi_init = msi_domain_ops_default.msi_init;
+	if (ops->msi_check == NULL)
+		ops->msi_check = msi_domain_ops_default.msi_check;
+	if (ops->msi_prepare == NULL)
+		ops->msi_prepare = msi_domain_ops_default.msi_prepare;
+	if (ops->set_desc == NULL)
+		ops->set_desc = msi_domain_ops_default.set_desc;
+}
+
+static void msi_domain_update_chip_ops(struct msi_domain_info *info)
+{
+	struct irq_chip *chip = info->chip;
+
+	BUG_ON(!chip);
+	if (!chip->irq_mask)
+		chip->irq_mask = mask_msi_irq;
+	if (!chip->irq_unmask)
+		chip->irq_unmask = unmask_msi_irq;
+	if (!chip->irq_set_affinity)
+		chip->irq_set_affinity = msi_domain_set_affinity;
+}
+
+struct irq_domain *msi_create_irq_domain(struct device_node *node,
 					 struct msi_domain_info *info,
 					 struct irq_domain *parent)
 {
-	struct irq_domain *domain;
+	if (info->flags & MSI_FLAG_USE_DEF_DOM_OPS)
+		msi_domain_update_dom_ops(info);
+	if (info->flags & MSI_FLAG_USE_DEF_CHIP_OPS)
+		msi_domain_update_chip_ops(info);
 
-	domain = irq_domain_add_tree(of_node, &msi_domain_ops, info);
-	if (domain)
-		domain->parent = parent;
-
-	return domain;
+	return irq_domain_add_hierarchy(parent, 0, 0, node, &msi_domain_ops,
+					info);
 }
 
 int msi_domain_alloc_irqs(struct irq_domain *domain, struct device *dev,
@@ -144,8 +237,12 @@ int msi_domain_alloc_irqs(struct irq_domain *domain, struct device *dev,
 
 	for_each_msi_entry(desc, dev) {
 		ops->set_desc(&arg, desc);
+		if (info->flags & MSI_FLAG_IDENTITY_MAP)
+			virq = (int)ops->get_hwirq(info, &arg);
+		else
+			virq = -1;
 
-		virq = __irq_domain_alloc_irqs(domain, -1, desc->nvec_used,
+		virq = __irq_domain_alloc_irqs(domain, virq, desc->nvec_used,
 					       dev_to_node(dev), &arg, false);
 		if (virq < 0) {
 			ret = -ENOSPC;
-- 
1.7.10.4

[Patch V2 8/9] PCI, MSI: Refine irqdomain interfaces to simplify its usage

[Jiang Liu]

Refine irqdomain interfaces to simplify its usage.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
---
 drivers/pci/msi.c   |  119 ++++++++++++++++++++++++++++++++++++++-------------
 include/linux/msi.h |   13 +++---
 2 files changed, 98 insertions(+), 34 deletions(-)

diff --git a/drivers/pci/msi.c b/drivers/pci/msi.c
index 9c53b865cb1b..1dd5d93bd4b8 100644
--- a/drivers/pci/msi.c
+++ b/drivers/pci/msi.c
@@ -1099,38 +1099,99 @@ irq_hw_number_t pci_msi_domain_calc_hwirq(struct pci_dev *dev,
 		(pci_domain_nr(dev->bus) & 0xFFFFFFFF) << 27;
 }
 
-int pci_msi_domain_alloc_irqs(struct irq_domain *domain, int type,
-			      struct pci_dev *dev, void *arg)
+static inline bool pci_msi_desc_is_multi_msi(struct msi_desc *desc)
 {
-	struct msi_domain_info *info = domain->host_data;
-	int node = dev_to_node(&dev->dev);
-	struct msi_desc *desc;
-	int i, virq;
-
-	list_for_each_entry(desc, &dev->msi_list, list) {
-		if (info->ops->calc_hwirq)
-			info->ops->calc_hwirq(info, arg, desc);
-
-		virq = irq_domain_alloc_irqs(domain, desc->nvec_used,
-					     node, arg);
-		if (virq < 0) {
-			/* Special handling for pci_enable_msi_range(). */
-			if (type == PCI_CAP_ID_MSI && desc->nvec_used > 1)
-				return 1;
-			else
-				return -ENOSPC;
-		}
-		for (i = 0; i < desc->nvec_used; i++)
-			irq_set_msi_desc_off(virq, i, desc);
-	}
+	return !desc->msi_attrib.is_msix && desc->nvec_used > 1;
+}
 
-	list_for_each_entry(desc, &dev->msi_list, list)
-		if (desc->nvec_used == 1)
-			dev_dbg(&dev->dev, "irq %d for MSI/MSI-X\n", virq);
-		else
-			dev_dbg(&dev->dev, "irq [%d-%d] for MSI/MSI-X\n",
-				virq, virq + desc->nvec_used - 1);
+int pci_msi_domain_check_cap(struct irq_domain *domain,
+			     struct msi_domain_info *info, struct device *dev)
+{
+	struct msi_desc *desc = first_pci_msi_entry(to_pci_dev(dev));
+
+	/* Special handling to support pci_enable_msi_range() */
+	if (pci_msi_desc_is_multi_msi(desc) &&
+	    !(info->flags & MSI_FLAG_MULTI_PCI_MSI))
+		return 1;
+	else if (desc->msi_attrib.is_msix && !(info->flags & MSI_FLAG_PCI_MSIX))
+		return -ENOTSUPP;
 
 	return 0;
 }
+
+static int pci_msi_domain_handle_error(struct irq_domain *domain,
+				       struct msi_desc *desc, int error)
+{
+	/* Special handling to support pci_enable_msi_range() */
+	if (pci_msi_desc_is_multi_msi(desc) && error == -ENOSPC)
+		return 1;
+
+	return error;
+}
+
+#ifndef msi_alloc_info_t
+static void pci_msi_domain_set_desc(struct msi_alloc_info *arg,
+				    struct msi_desc *desc)
+{
+	arg->desc = desc;
+	arg->hwirq = pci_msi_domain_calc_hwirq(msi_desc_to_pci_dev(desc),
+					       desc);
+}
+#else
+#define pci_msi_domain_set_desc		NULL
+#endif
+
+static struct msi_domain_ops pci_msi_domain_ops_default = {
+	.set_desc	= pci_msi_domain_set_desc,
+	.msi_check	= pci_msi_domain_check_cap,
+	.handle_error	= pci_msi_domain_handle_error,
+};
+
+static void pci_msi_domain_update_dom_ops(struct msi_domain_info *info)
+{
+	struct msi_domain_ops *ops = info->ops;
+
+	if (ops == NULL) {
+		info->ops = &pci_msi_domain_ops_default;
+	} else {
+		if (ops->set_desc == NULL)
+			ops->set_desc = pci_msi_domain_set_desc;
+		if (ops->msi_check == NULL)
+			ops->msi_check = pci_msi_domain_check_cap;
+		if (ops->handle_error == NULL)
+			ops->handle_error = pci_msi_domain_handle_error;
+	}
+}
+
+static void pci_msi_domain_update_chip_ops(struct msi_domain_info *info)
+{
+	struct irq_chip *chip = info->chip;
+
+	BUG_ON(!chip);
+	if (!chip->irq_write_msi_msg)
+		chip->irq_write_msi_msg = pci_msi_domain_write_msg;
+}
+
+struct irq_domain *pci_msi_create_irq_domain(struct device_node *node,
+					     struct msi_domain_info *info,
+					     struct irq_domain *parent)
+{
+	if (info->flags & MSI_FLAG_USE_DEF_DOM_OPS)
+		pci_msi_domain_update_dom_ops(info);
+	if (info->flags & MSI_FLAG_USE_DEF_CHIP_OPS)
+		pci_msi_domain_update_chip_ops(info);
+
+	return msi_create_irq_domain(node, info, parent);
+}
+
+int pci_msi_domain_alloc_irqs(struct irq_domain *domain, struct pci_dev *dev,
+			      int nvec, int type)
+{
+	return msi_domain_alloc_irqs(domain, &dev->dev, nvec);
+}
+
+void pci_msi_domain_free_irqs(struct irq_domain *domain, struct pci_dev *dev)
+{
+	msi_domain_free_irqs(domain, &dev->dev);
+}
 #endif /* CONFIG_PCI_MSI_IRQ_DOMAIN */
diff --git a/include/linux/msi.h b/include/linux/msi.h
index 96ee8c3b98cc..0463e361fef8 100644
--- a/include/linux/msi.h
+++ b/include/linux/msi.h
@@ -125,9 +125,6 @@ typedef struct msi_alloc_info msi_alloc_info_t;
 
 struct msi_domain_ops {
 	/* Callbacks for msi_create_irq_domain() */
-	void		(*calc_hwirq)(struct msi_domain_info *info,
-				      msi_alloc_info_t *arg,
-				      struct msi_desc *desc);
 	irq_hw_number_t	(*get_hwirq)(struct msi_domain_info *info,
 				     msi_alloc_info_t *arg);
 	int		(*msi_init)(struct irq_domain *domain,
@@ -189,10 +186,16 @@ struct msi_domain_info *msi_get_domain_info(struct irq_domain *domain);
 
 #ifdef CONFIG_PCI_MSI_IRQ_DOMAIN
 void pci_msi_domain_write_msg(struct irq_data *irq_data, struct msi_msg *msg);
-int pci_msi_domain_alloc_irqs(struct irq_domain *domain, int type,
-			      struct pci_dev *dev, void *arg);
+struct irq_domain *pci_msi_create_irq_domain(struct device_node *node,
+					     struct msi_domain_info *info,
+					     struct irq_domain *parent);
+int pci_msi_domain_alloc_irqs(struct irq_domain *domain, struct pci_dev *dev,
+			      int nvec, int type);
+void pci_msi_domain_free_irqs(struct irq_domain *domain, struct pci_dev *dev);
 irq_hw_number_t pci_msi_domain_calc_hwirq(struct pci_dev *dev,
 					  struct msi_desc *desc);
+int pci_msi_domain_check_cap(struct irq_domain *domain,
+			     struct msi_domain_info *info, struct device *dev);
 #endif /* CONFIG_PCI_MSI_IRQ_DOMAIN */
 
 #endif /* LINUX_MSI_H */
-- 
1.7.10.4

[Patch V2 9/9] PCI, MSI: Provide mechanism to alloc/free MSI/MSIX interrupt from irqdomain

[Jiang Liu]

Provide mechanism to directly alloc/free MSI/MSIX interrupt from
irqdomain, which will be used to replace arch_setup_msi_irq()/
arch_setup_msi_irqs()/arch_teardown_msi_irq()/arch_teardown_msi_irqs().

To kill weak functions, this patch introduce a new weak function
arch_get_pci_msi_domain(), which is to retrieve the MSI irqdomain
for a PCI device. This weak function could be killed once we get
a common way to associate MSI domain with PCI device.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
---
 drivers/pci/msi.c   |   60 +++++++++++++++++++++++++++++++++++++++++++++++----
 include/linux/msi.h |    3 +++
 2 files changed, 59 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/msi.c b/drivers/pci/msi.c
index 1dd5d93bd4b8..a66a4eaf8111 100644
--- a/drivers/pci/msi.c
+++ b/drivers/pci/msi.c
@@ -27,8 +27,41 @@ static int pci_msi_enable = 1;
 
 #define msix_table_size(flags)	((flags & PCI_MSIX_FLAGS_QSIZE) + 1)
 
-/* Arch hooks */
+#ifdef CONFIG_PCI_MSI_IRQ_DOMAIN
+static struct irq_domain *pci_msi_default_domain;
+
+struct irq_domain * __weak arch_get_pci_msi_domain(struct pci_dev *dev)
+{
+	return pci_msi_default_domain;
+}
+
+static int pci_msi_setup_msi_irqs(struct pci_dev *dev, int nvec, int type)
+{
+	struct irq_domain *domain;
 
+	domain = arch_get_pci_msi_domain(dev);
+	if (domain)
+		return pci_msi_domain_alloc_irqs(domain, dev, nvec, type);
+
+	return arch_setup_msi_irqs(dev, nvec, type);
+}
+
+static void pci_msi_teardown_msi_irqs(struct pci_dev *dev)
+{
+	struct irq_domain *domain;
+
+	domain = arch_get_pci_msi_domain(dev);
+	if (domain)
+		pci_msi_domain_free_irqs(domain, dev);
+	else
+		arch_teardown_msi_irqs(dev);
+}
+#else
+#define pci_msi_setup_msi_irqs		arch_setup_msi_irqs
+#define pci_msi_teardown_msi_irqs	arch_teardown_msi_irqs
+#endif
+
+/* Arch hooks */
 int __weak arch_setup_msi_irq(struct pci_dev *dev, struct msi_desc *desc)
 {
 	struct msi_chip *chip = dev->bus->msi;
@@ -329,7 +362,7 @@ static void free_msi_irqs(struct pci_dev *dev)
 			for (i = 0; i < entry->nvec_used; i++)
 				BUG_ON(irq_has_action(entry->irq + i));
 
-	arch_teardown_msi_irqs(dev);
+	pci_msi_teardown_msi_irqs(dev);
 
 	list_for_each_entry_safe(entry, tmp, &dev->msi_list, list) {
 		if (entry->msi_attrib.is_msix) {
@@ -581,7 +614,7 @@ static int msi_capability_init(struct pci_dev *dev, int nvec)
 	list_add_tail(&entry->list, &dev->msi_list);
 
 	/* Configure MSI capability structure */
-	ret = arch_setup_msi_irqs(dev, nvec, PCI_CAP_ID_MSI);
+	ret = pci_msi_setup_msi_irqs(dev, nvec, PCI_CAP_ID_MSI);
 	if (ret) {
 		msi_mask_irq(entry, mask, ~mask);
 		free_msi_irqs(dev);
@@ -696,7 +729,7 @@ static int msix_capability_init(struct pci_dev *dev,
 	if (ret)
 		return ret;
 
-	ret = arch_setup_msi_irqs(dev, nvec, PCI_CAP_ID_MSIX);
+	ret = pci_msi_setup_msi_irqs(dev, nvec, PCI_CAP_ID_MSIX);
 	if (ret)
 		goto out_avail;
 
@@ -1194,4 +1227,23 @@ void pci_msi_domain_free_irqs(struct irq_domain *domain, struct pci_dev *dev)
 {
 	msi_domain_free_irqs(domain, &dev->dev);
 }
+
+struct irq_domain *pci_msi_create_default_irq_domain(struct device_node *node,
+		 struct msi_domain_info *info, struct irq_domain *parent)
+{
+	struct irq_domain *domain;
+	static DEFINE_MUTEX(pci_msi_domain_lock);
+
+	mutex_lock(&pci_msi_domain_lock);
+	if (pci_msi_default_domain) {
+		pr_err("PCI: default irq domain for PCI MSI has already been created.\n");
+		domain = NULL;
+	} else {
+		domain = pci_msi_create_irq_domain(node, info, parent);
+		pci_msi_default_domain = domain;
+	}
+	mutex_unlock(&pci_msi_domain_lock);
+
+	return domain;
+}
 #endif /* CONFIG_PCI_MSI_IRQ_DOMAIN */
diff --git a/include/linux/msi.h b/include/linux/msi.h
index 0463e361fef8..ba173a6a007e 100644
--- a/include/linux/msi.h
+++ b/include/linux/msi.h
@@ -192,6 +192,9 @@ struct irq_domain *pci_msi_create_irq_domain(struct device_node *node,
 int pci_msi_domain_alloc_irqs(struct irq_domain *domain, struct pci_dev *dev,
 			      int nvec, int type);
 void pci_msi_domain_free_irqs(struct irq_domain *domain, struct pci_dev *dev);
+struct irq_domain *pci_msi_create_default_irq_domain(struct device_node *node,
+		 struct msi_domain_info *info, struct irq_domain *parent);
+
 irq_hw_number_t pci_msi_domain_calc_hwirq(struct pci_dev *dev,
 					  struct msi_desc *desc);
 int pci_msi_domain_check_cap(struct irq_domain *domain,
-- 
1.7.10.4