[RFC v2 PATCH 00/14] Implement generic IPI support mechanism

[RFC v2 PATCH 00/14] Implement generic IPI support mechanism

[Qais Yousef]

This RFC series attempts to implement a generic IPI layer for reserving and sending IPIs.

It is based on the discussion in these links

	https://lkml.org/lkml/2015/8/26/713
	https://lkml.org/lkml/2015/9/29/875

In summary. We need a generic IPI layer to allow driver code to send IPIs to coprocessor
without caring about implementation details of the underlying controller.

Also it will help in making SMP IPI support more generic.

The goal is to have a mechanism to dynamically reserve an IPI to destination CPUs and
provide a single virq to send an IPI to any of these CPUs using generic irq_send_ipi()
API.

This v2 addresses the comments from v1 and implements a simpler mapping mechanism and moves
the irq_send_ipi() to be part of irqchip instead of irqdomain.

The implementation falls more natural and fits into place now (hopefully). So hopefully next
series would be non RFC. The only thing I haven't addressed is whether we want to make
request_percpu_irq() enable a coprocessor or defer that to the coprocessor itself.

This series is based on Linus tree. I couldn't compile test it because MIPS compilation was
broken due to other reasons. I expect some brokeness because of the introduction of
struct irq_common_data which is not present on the 4.1 tree I was testing my code on before
porting it to Linus tip. I will fix these issues and introduce proper accessors for accessing
struct ipi_mask given that the concept is approved.

I hope my commit messages aren't too terse.

Credit goes to Thomas for spec'ing and outlining the proper way to get this new API in.

Qais Yousef (14):
  irq: add new IRQ_DOMAIN_FLAGS_IPI
  irq: add GENERIC_IRQ_IPI Kconfig symbol
  irq: add new struct ipi_mask
  irq: add a new irq_send_ipi() to irq_chip
  irq: add struct ipi_mask to irq_data
  irq: add struct ipi_mapping and its helper functions
  irq: add a new generic IPI reservation code to irq core
  irq: implement irq_send_ipi
  MIPS: add support for generic SMP IPI support
  MIPS: make smp CMP, CPS and MT use the new generic IPI functions
  MIPS: delete smp-gic.c
  irqchip: mips-gic: add a IPI hierarchy domain
  irqchip: mips-gic: implement the new irq_send_ipi
  irqchip: mips-gic: remove IPI init code

 arch/mips/Kconfig                |   6 --
 arch/mips/include/asm/smp-ops.h  |   5 +-
 arch/mips/kernel/Makefile        |   1 -
 arch/mips/kernel/smp-cmp.c       |   4 +-
 arch/mips/kernel/smp-cps.c       |   4 +-
 arch/mips/kernel/smp-gic.c       |  64 -----------
 arch/mips/kernel/smp-mt.c        |   2 +-
 arch/mips/kernel/smp.c           | 117 ++++++++++++++++++++
 drivers/irqchip/Kconfig          |   2 +
 drivers/irqchip/irq-mips-gic.c   | 225 ++++++++++++++++++++++++---------------
 include/linux/irq.h              |  43 ++++++++
 include/linux/irqchip/mips-gic.h |   3 -
 include/linux/irqdomain.h        |  19 ++++
 kernel/irq/Kconfig               |   4 +
 kernel/irq/irqdomain.c           |  84 +++++++++++++++
 kernel/irq/manage.c              | 103 ++++++++++++++++++
 16 files changed, 517 insertions(+), 169 deletions(-)
 delete mode 100644 arch/mips/kernel/smp-gic.c

-- 
2.1.0

[RFC v2 PATCH 01/14] irq: add new IRQ_DOMAIN_FLAGS_IPI

[Qais Yousef]

This flag will be used to identify an IPI domain.

Signed-off-by: Qais Yousef <qais.yousef@imgtec.com>
---
 include/linux/irqdomain.h | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/include/linux/irqdomain.h b/include/linux/irqdomain.h
index d3ca79236fb0..9b3dc6c2a3cc 100644
--- a/include/linux/irqdomain.h
+++ b/include/linux/irqdomain.h
@@ -153,6 +153,9 @@ enum {
 	/* Core calls alloc/free recursive through the domain hierarchy. */
 	IRQ_DOMAIN_FLAG_AUTO_RECURSIVE	= (1 << 1),
 
+	/* Irq domain is an IPI domain */
+	IRQ_DOMAIN_FLAG_IPI		= (1 << 2),
+
 	/*
 	 * Flags starting from IRQ_DOMAIN_FLAG_NONCORE are reserved
 	 * for implementation specific purposes and ignored by the
@@ -326,6 +329,11 @@ static inline bool irq_domain_is_hierarchy(struct irq_domain *domain)
 {
 	return domain->flags & IRQ_DOMAIN_FLAG_HIERARCHY;
 }
+
+static inline bool irq_domain_is_ipi(struct irq_domain *domain)
+{
+	return domain->flags & IRQ_DOMAIN_FLAG_IPI;
+}
 #else	/* CONFIG_IRQ_DOMAIN_HIERARCHY */
 static inline void irq_domain_activate_irq(struct irq_data *data) { }
 static inline void irq_domain_deactivate_irq(struct irq_data *data) { }
@@ -339,6 +347,11 @@ static inline bool irq_domain_is_hierarchy(struct irq_domain *domain)
 {
 	return false;
 }
+
+static inline bool irq_domain_is_ipi(struct irq_domain *domain)
+{
+	return false;
+}
 #endif	/* CONFIG_IRQ_DOMAIN_HIERARCHY */
 
 #else /* CONFIG_IRQ_DOMAIN */
-- 
2.1.0

[RFC v2 PATCH 02/14] irq: add GENERIC_IRQ_IPI Kconfig symbol

[Qais Yousef]

irqchip should select this config to denote it supports generic IPI.

This will aid generic arch code to know when it can use generic IPI layer.

Signed-off-by: Qais Yousef <qais.yousef@imgtec.com>
---
 kernel/irq/Kconfig | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/kernel/irq/Kconfig b/kernel/irq/Kconfig
index 9a76e3beda54..cb5044c54288 100644
--- a/kernel/irq/Kconfig
+++ b/kernel/irq/Kconfig
@@ -60,6 +60,10 @@ config IRQ_DOMAIN_HIERARCHY
 	bool
 	select IRQ_DOMAIN
 
+# Generic IRQ IPI support
+config GENERIC_IRQ_IPI
+	bool
+
 # Generic MSI interrupt support
 config GENERIC_MSI_IRQ
 	bool
-- 
2.1.0

[RFC v2 PATCH 03/14] irq: add new struct ipi_mask

[Qais Yousef]

cpumask is limited to NR_CPUS. introduce ipi_mask which allows us to address
cpu range that is higher than NR_CPUS which is required for drivers to send
IPIs for coprocessor that are outside Linux CPU range.

Signed-off-by: Qais Yousef <qais.yousef@imgtec.com>
---
 include/linux/irq.h | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/include/linux/irq.h b/include/linux/irq.h
index 11bf09288ddb..4b537e4d393b 100644
--- a/include/linux/irq.h
+++ b/include/linux/irq.h
@@ -125,6 +125,21 @@ enum {
 struct msi_desc;
 struct irq_domain;
 
+ /**
+ * struct ipi_mask - IPI mask information
+ * @cpumask: bitmap of cpumasks
+ * @nbits: number of bits in cpumask
+ * @global: whether the mask is SMP IPI ie: subset of cpu_possible_mask or not
+ *
+ * ipi_mask is similar to cpumask, but it provides nbits that's configurable
+ * rather than fixed to NR_CPUS.
+ */
+struct ipi_mask {
+	unsigned long	*cpumask;
+	unsigned int	nbits;
+	bool		global;
+};
+
 /**
  * struct irq_common_data - per irq data shared by all irqchips
  * @state_use_accessors: status information for irq chip functions.
-- 
2.1.0

Re: [RFC v2 PATCH 03/14] irq: add new struct ipi_mask

[Thomas Gleixner]

On Tue, 13 Oct 2015, Qais Yousef wrote:
> cpumask is limited to NR_CPUS. introduce ipi_mask which allows us to address
> cpu range that is higher than NR_CPUS which is required for drivers to send
> IPIs for coprocessor that are outside Linux CPU range.
> 
> Signed-off-by: Qais Yousef <qais.yousef@imgtec.com>
> ---
>  include/linux/irq.h | 15 +++++++++++++++
>  1 file changed, 15 insertions(+)
> 
> diff --git a/include/linux/irq.h b/include/linux/irq.h
> index 11bf09288ddb..4b537e4d393b 100644
> --- a/include/linux/irq.h
> +++ b/include/linux/irq.h
> @@ -125,6 +125,21 @@ enum {
>  struct msi_desc;
>  struct irq_domain;
>  
> + /**
> + * struct ipi_mask - IPI mask information
> + * @cpumask: bitmap of cpumasks
> + * @nbits: number of bits in cpumask
> + * @global: whether the mask is SMP IPI ie: subset of cpu_possible_mask or not
> + *
> + * ipi_mask is similar to cpumask, but it provides nbits that's configurable
> + * rather than fixed to NR_CPUS.
> + */
> +struct ipi_mask {
> +	unsigned long	*cpumask;
> +	unsigned int	nbits;
> +	bool		global;
> +};

Can you make that:

struct ipi_mask {
	unsigned int	nbits;
	bool		global;
	unsigned long	cpu_bitmap[];
};

That allows you to allocate the data structure in one go. So the
ipi_mask in irq_data_common becomes a pointer which is only filled in
when ipi_mask is actually used.

Note, I renamed cpumask to cpu_bitmap to avoid confusion with
cpumasks.

We also want a helper function

   struct cpumask *irq_data_get_ipi_mask(struct irq_data *data);

so we can use normal cpumask operations for the majority of cases.

Thanks,

	tglx

Re: [RFC v2 PATCH 03/14] irq: add new struct ipi_mask

[Qais Yousef]

On 10/13/2015 02:26 PM, Thomas Gleixner wrote:
> On Tue, 13 Oct 2015, Qais Yousef wrote:
>> cpumask is limited to NR_CPUS. introduce ipi_mask which allows us to address
>> cpu range that is higher than NR_CPUS which is required for drivers to send
>> IPIs for coprocessor that are outside Linux CPU range.
>>
>> Signed-off-by: Qais Yousef <qais.yousef@imgtec.com>
>> ---
>>   include/linux/irq.h | 15 +++++++++++++++
>>   1 file changed, 15 insertions(+)
>>
>> diff --git a/include/linux/irq.h b/include/linux/irq.h
>> index 11bf09288ddb..4b537e4d393b 100644
>> --- a/include/linux/irq.h
>> +++ b/include/linux/irq.h
>> @@ -125,6 +125,21 @@ enum {
>>   struct msi_desc;
>>   struct irq_domain;
>>   
>> + /**
>> + * struct ipi_mask - IPI mask information
>> + * @cpumask: bitmap of cpumasks
>> + * @nbits: number of bits in cpumask
>> + * @global: whether the mask is SMP IPI ie: subset of cpu_possible_mask or not
>> + *
>> + * ipi_mask is similar to cpumask, but it provides nbits that's configurable
>> + * rather than fixed to NR_CPUS.
>> + */
>> +struct ipi_mask {
>> +	unsigned long	*cpumask;
>> +	unsigned int	nbits;
>> +	bool		global;
>> +};
> Can you make that:
>
> struct ipi_mask {
> 	unsigned int	nbits;
> 	bool		global;
> 	unsigned long	cpu_bitmap[];
> };
>
> That allows you to allocate the data structure in one go. So the
> ipi_mask in irq_data_common becomes a pointer which is only filled in
> when ipi_mask is actually used.
>
> Note, I renamed cpumask to cpu_bitmap to avoid confusion with
> cpumasks.
>
> We also want a helper function
>
>     struct cpumask *irq_data_get_ipi_mask(struct irq_data *data);
>
> so we can use normal cpumask operations for the majority of cases.
>
>

Will do.

Thanks,
Qais

[RFC v2 PATCH 04/14] irq: add a new irq_send_ipi() to irq_chip

[Qais Yousef]

Introduce the new function to allow generic IPI send mechanism to be
used from drivers code.

Signed-off-by: Qais Yousef <qais.yousef@imgtec.com>
---
 include/linux/irq.h | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/include/linux/irq.h b/include/linux/irq.h
index 4b537e4d393b..504133671985 100644
--- a/include/linux/irq.h
+++ b/include/linux/irq.h
@@ -369,6 +369,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
  * @irq_get_irqchip_state:	return the internal state of an interrupt
  * @irq_set_irqchip_state:	set the internal state of a interrupt
  * @irq_set_vcpu_affinity:	optional to target a vCPU in a virtual machine
+ * @irq_send_ipi:	send an IPI to destination cpus
  * @flags:		chip specific flags
  */
 struct irq_chip {
@@ -413,6 +414,8 @@ struct irq_chip {
 
 	int		(*irq_set_vcpu_affinity)(struct irq_data *data, void *vcpu_info);
 
+	void		(*irq_send_ipi)(struct irq_data *data, const struct ipi_mask *dest);
+
 	unsigned long	flags;
 };
 
-- 
2.1.0

[RFC v2 PATCH 05/14] irq: add struct ipi_mask to irq_data

[Qais Yousef]

It has a similar role to affinity mask, but tracks the IPI affinity instead.

Signed-off-by: Qais Yousef <qais.yousef@imgtec.com>
---
 include/linux/irq.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/include/linux/irq.h b/include/linux/irq.h
index 504133671985..b000b217ea24 100644
--- a/include/linux/irq.h
+++ b/include/linux/irq.h
@@ -157,6 +157,7 @@ struct irq_common_data {
 	void			*handler_data;
 	struct msi_desc		*msi_desc;
 	cpumask_var_t		affinity;
+	struct ipi_mask		ipi_mask;
 };
 
 /**
-- 
2.1.0

Re: [RFC v2 PATCH 05/14] irq: add struct ipi_mask to irq_data

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 2821 bytes --]

Hi Qais,

[auto build test WARNING on v4.3-rc5 -- if it's inappropriate base, please suggest rules for selecting the more suitable base]

url:    https://github.com/0day-ci/linux/commits/Qais-Yousef/Implement-generic-IPI-support-mechanism/20151013-182314
reproduce: make htmldocs

All warnings (new ones prefixed by >>):

>> include/linux/irq.h:161: warning: No description found for parameter 'ipi_mask'
   kernel/irq/handle.c:1: warning: no structured comments found

vim +/ipi_mask +161 include/linux/irq.h

0d0b4c86 Jiang Liu       2015-06-01  145   * @state_use_accessors: status information for irq chip functions.
0d0b4c86 Jiang Liu       2015-06-01  146   *			Use accessor functions to deal with it
449e9cae Jiang Liu       2015-06-01  147   * @node:		node index useful for balancing
af7080e0 Jiang Liu       2015-06-01  148   * @handler_data:	per-IRQ data for the irq_chip methods
9df872fa Jiang Liu       2015-06-03  149   * @affinity:		IRQ affinity on SMP
b237721c Jiang Liu       2015-06-01  150   * @msi_desc:		MSI descriptor
0d0b4c86 Jiang Liu       2015-06-01  151   */
0d0b4c86 Jiang Liu       2015-06-01  152  struct irq_common_data {
0d0b4c86 Jiang Liu       2015-06-01  153  	unsigned int		state_use_accessors;
449e9cae Jiang Liu       2015-06-01  154  #ifdef CONFIG_NUMA
449e9cae Jiang Liu       2015-06-01  155  	unsigned int		node;
449e9cae Jiang Liu       2015-06-01  156  #endif
af7080e0 Jiang Liu       2015-06-01  157  	void			*handler_data;
b237721c Jiang Liu       2015-06-01  158  	struct msi_desc		*msi_desc;
9df872fa Jiang Liu       2015-06-03  159  	cpumask_var_t		affinity;
9bed7dd6 Qais Yousef     2015-10-13  160  	struct ipi_mask		ipi_mask;
0d0b4c86 Jiang Liu       2015-06-01 @161  };
0d0b4c86 Jiang Liu       2015-06-01  162  
0d0b4c86 Jiang Liu       2015-06-01  163  /**
0d0b4c86 Jiang Liu       2015-06-01  164   * struct irq_data - per irq chip data passed down to chip functions
966dc736 Thomas Gleixner 2013-05-06  165   * @mask:		precomputed bitmask for accessing the chip registers
ff7dcd44 Thomas Gleixner 2010-09-27  166   * @irq:		interrupt number
08a543ad Grant Likely    2011-07-26  167   * @hwirq:		hardware interrupt number, local to the interrupt domain
0d0b4c86 Jiang Liu       2015-06-01  168   * @common:		point to data shared by all irqchips
ff7dcd44 Thomas Gleixner 2010-09-27  169   * @chip:		low level interrupt hardware access

:::::: The code at line 161 was first introduced by commit
:::::: 0d0b4c866bcce647f40d73efe5e90aeeb079050a genirq: Introduce struct irq_common_data to host shared irq data

:::::: TO: Jiang Liu <jiang.liu@linux.intel.com>
:::::: CC: Thomas Gleixner <tglx@linutronix.de>

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 6062 bytes --]

Re: [RFC v2 PATCH 05/14] irq: add struct ipi_mask to irq_data

[Davidlohr Bueso]

On Tue, 13 Oct 2015, Qais Yousef wrote:

>It has a similar role to affinity mask, but tracks the IPI affinity instead.
>
>Signed-off-by: Qais Yousef <qais.yousef@imgtec.com>
>---
> include/linux/irq.h | 1 +
> 1 file changed, 1 insertion(+)
>
>diff --git a/include/linux/irq.h b/include/linux/irq.h
>index 504133671985..b000b217ea24 100644
>--- a/include/linux/irq.h
>+++ b/include/linux/irq.h
>@@ -157,6 +157,7 @@ struct irq_common_data {
> 	void			*handler_data;
> 	struct msi_desc		*msi_desc;
> 	cpumask_var_t		affinity;
>+	struct ipi_mask		ipi_mask;
> };

This should be folded into patch 3, no?

Re: [RFC v2 PATCH 05/14] irq: add struct ipi_mask to irq_data

[Qais Yousef]

On 10/14/2015 03:50 PM, Davidlohr Bueso wrote:
> On Tue, 13 Oct 2015, Qais Yousef wrote:
>
>> It has a similar role to affinity mask, but tracks the IPI affinity 
>> instead.
>>
>> Signed-off-by: Qais Yousef <qais.yousef@imgtec.com>
>> ---
>> include/linux/irq.h | 1 +
>> 1 file changed, 1 insertion(+)
>>
>> diff --git a/include/linux/irq.h b/include/linux/irq.h
>> index 504133671985..b000b217ea24 100644
>> --- a/include/linux/irq.h
>> +++ b/include/linux/irq.h
>> @@ -157,6 +157,7 @@ struct irq_common_data {
>>     void            *handler_data;
>>     struct msi_desc        *msi_desc;
>>     cpumask_var_t        affinity;
>> +    struct ipi_mask        ipi_mask;
>> };
>
> This should be folded into patch 3, no?

For me they are 2 separate changes, hence the 2 commits. They could be 
folded but I prefer the septation unless there's a strong opinion not to.

Thanks,
Qais

[RFC v2 PATCH 06/14] irq: add struct ipi_mapping and its helper functions

[Qais Yousef]

struct ipi_mapping will provide a mechanism for irqdomain/architure
code to fill out the mapping for the generic code later to implement
generic IPI reserve and send functions.

Signed-off-by: Qais Yousef <qais.yousef@imgtec.com>
---
 include/linux/irq.h | 21 +++++++++++++++++++
 kernel/irq/manage.c | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 80 insertions(+)

diff --git a/include/linux/irq.h b/include/linux/irq.h
index b000b217ea24..c3d0f26c3eff 100644
--- a/include/linux/irq.h
+++ b/include/linux/irq.h
@@ -964,4 +964,25 @@ static inline u32 irq_reg_readl(struct irq_chip_generic *gc,
 		return readl(gc->reg_base + reg_offset);
 }
 
+#define INVALID_HWIRQ	-1
+
+/**
+ * struct ipi_mapping - IPI mapping information object
+ * @nr_hwirqs: number of hwirqs mapped
+ * @nr_cpus: number of cpus the controller can talk to
+ * @cpumap: per cpu hwirq mapping table
+ */
+struct ipi_mapping {
+	unsigned int nr_hwirqs;
+	unsigned int nr_cpus;
+	unsigned int *cpumap;
+};
+
+struct ipi_mapping *irq_alloc_ipi_mapping(unsigned int nr_cpus);
+void irq_free_ipi_mapping(struct ipi_mapping *map);
+int irq_map_ipi(struct ipi_mapping *map,
+		unsigned int cpu, irq_hw_number_t hwirq);
+int irq_unmap_ipi(struct ipi_mapping *map,
+		  unsigned int cpu, irq_hw_number_t *hwirq);
+
 #endif /* _LINUX_IRQ_H */
diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
index f9a59f6cabd2..9a9bc0822c8f 100644
--- a/kernel/irq/manage.c
+++ b/kernel/irq/manage.c
@@ -1924,3 +1924,62 @@ int irq_set_irqchip_state(unsigned int irq, enum irqchip_irq_state which,
 	return err;
 }
 EXPORT_SYMBOL_GPL(irq_set_irqchip_state);
+
+struct ipi_mapping *irq_alloc_ipi_mapping(unsigned int nr_cpus)
+{
+	struct ipi_mapping *map;
+	int i;
+
+	map = kzalloc(sizeof(struct ipi_mapping), GFP_KERNEL);
+	if (!map)
+		return NULL;
+
+	map->nr_cpus = nr_cpus;
+
+	map->cpumap = kmalloc(sizeof(irq_hw_number_t) * nr_cpus, GFP_KERNEL);
+	if (!map->cpumap) {
+		kfree(map);
+		return NULL;
+	}
+
+	for (i = 0; i < nr_cpus; i++)
+		map->cpumap[i] = INVALID_HWIRQ;
+
+	return map;
+}
+
+void irq_free_ipi_mapping(struct ipi_mapping *map)
+{
+	kfree(map->cpumap);
+	kfree(map);
+}
+
+int irq_map_ipi(struct ipi_mapping *map,
+		unsigned int cpu, irq_hw_number_t hwirq)
+{
+	if (cpu >= map->nr_cpus)
+		return -EINVAL;
+
+	map->cpumap[cpu] = hwirq;
+	map->nr_hwirqs++;
+
+	return 0;
+}
+
+int irq_unmap_ipi(struct ipi_mapping *map,
+		  unsigned int cpu, irq_hw_number_t *hwirq)
+{
+	if (cpu >= map->nr_cpus)
+		return -EINVAL;
+
+	if (map->cpumap[cpu] == INVALID_HWIRQ)
+		return -EINVAL;
+
+	if (hwirq)
+		*hwirq = map->cpumap[cpu];
+
+	map->cpumap[cpu] = INVALID_HWIRQ;
+	map->nr_hwirqs--;
+
+	return 0;
+}
-- 
2.1.0

Re: [RFC v2 PATCH 06/14] irq: add struct ipi_mapping and its helper functions

[Thomas Gleixner]

On Tue, 13 Oct 2015, Qais Yousef wrote:

> struct ipi_mapping will provide a mechanism for irqdomain/architure
> code to fill out the mapping for the generic code later to implement
> generic IPI reserve and send functions.
> 
> Signed-off-by: Qais Yousef <qais.yousef@imgtec.com>
> ---
>  include/linux/irq.h | 21 +++++++++++++++++++
>  kernel/irq/manage.c | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 80 insertions(+)
> 
> diff --git a/include/linux/irq.h b/include/linux/irq.h
> index b000b217ea24..c3d0f26c3eff 100644
> --- a/include/linux/irq.h
> +++ b/include/linux/irq.h
> @@ -964,4 +964,25 @@ static inline u32 irq_reg_readl(struct irq_chip_generic *gc,
>  		return readl(gc->reg_base + reg_offset);
>  }
>  
> +#define INVALID_HWIRQ	-1
> +
> +/**
> + * struct ipi_mapping - IPI mapping information object
> + * @nr_hwirqs: number of hwirqs mapped
> + * @nr_cpus: number of cpus the controller can talk to
> + * @cpumap: per cpu hwirq mapping table
> + */
> +struct ipi_mapping {
> +	unsigned int nr_hwirqs;
> +	unsigned int nr_cpus;
> +	unsigned int *cpumap;
> +};

Again, you can avoid seperate allocations and pointer indirections by
s/*cpumap/cpumap[]/
      
> +struct ipi_mapping *irq_alloc_ipi_mapping(unsigned int nr_cpus)
> +{
> +	struct ipi_mapping *map;
> +	int i;
> +
> +	map = kzalloc(sizeof(struct ipi_mapping), GFP_KERNEL);
> +	if (!map)
> +		return NULL;
> +
> +	map->nr_cpus = nr_cpus;
> +
> +	map->cpumap = kmalloc(sizeof(irq_hw_number_t) * nr_cpus, GFP_KERNEL);
> +	if (!map->cpumap) {
> +		kfree(map);
> +		return NULL;
> +	}
> +	for (i = 0; i < nr_cpus; i++)
> +		map->cpumap[i] = INVALID_HWIRQ;

memset please

> +
> +	return map;
> +}
> +
> +void irq_free_ipi_mapping(struct ipi_mapping *map)
> +{
> +	kfree(map->cpumap);
> +	kfree(map);
> +}
> +
> +int irq_map_ipi(struct ipi_mapping *map,
> +		unsigned int cpu, irq_hw_number_t hwirq)
> +{
> +	if (cpu >= map->nr_cpus)
> +		return -EINVAL;
> +
> +	map->cpumap[cpu] = hwirq;
> +	map->nr_hwirqs++;
> +
> +	return 0;
> +}
> +
> +int irq_unmap_ipi(struct ipi_mapping *map,
> +		  unsigned int cpu, irq_hw_number_t *hwirq)
> +{
> +	if (cpu >= map->nr_cpus)
> +		return -EINVAL;
> +
> +	if (map->cpumap[cpu] == INVALID_HWIRQ)
> +		return -EINVAL;
> +
> +	if (hwirq)
> +		*hwirq = map->cpumap[cpu];

Why do we store hwirq in unmap?

All these new functions lack kerneldoc comments.

Thanks,

	tglx

Re: [RFC v2 PATCH 06/14] irq: add struct ipi_mapping and its helper functions

[Qais Yousef]

On 10/13/2015 02:31 PM, Thomas Gleixner wrote:
> On Tue, 13 Oct 2015, Qais Yousef wrote:
>
> +
> +int irq_unmap_ipi(struct ipi_mapping *map,
> +		  unsigned int cpu, irq_hw_number_t *hwirq)
> +{
> +	if (cpu >= map->nr_cpus)
> +		return -EINVAL;
> +
> +	if (map->cpumap[cpu] == INVALID_HWIRQ)
> +		return -EINVAL;
> +
> +	if (hwirq)
> +		*hwirq = map->cpumap[cpu];
> Why do we store hwirq in unmap?

So that the irqchip driver can return the hwirq to its available IPI pool.

>
> All these new functions lack kerneldoc comments.

Apologies about the missing docs here and in other places.

Thanks,
Qais

[RFC v2 PATCH 07/14] irq: add a new generic IPI reservation code to irq core

[Qais Yousef]

Add a generic mechanism to dynamically allocate an IPI.

With this change the user can call irq_reserve_ipi() to dynamically allocate an
IPI and use the associate virq to send one to 1 or more cpus.

Signed-off-by: Qais Yousef <qais.yousef@imgtec.com>
---
 include/linux/irqdomain.h |  6 ++++
 kernel/irq/irqdomain.c    | 84 +++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 90 insertions(+)

diff --git a/include/linux/irqdomain.h b/include/linux/irqdomain.h
index 9b3dc6c2a3cc..f5003f5fd530 100644
--- a/include/linux/irqdomain.h
+++ b/include/linux/irqdomain.h
@@ -41,6 +41,7 @@ struct irq_domain;
 struct of_device_id;
 struct irq_chip;
 struct irq_data;
+struct ipi_mask;
 
 /* Number of irqs reserved for a legacy isa controller */
 #define NUM_ISA_INTERRUPTS	16
@@ -280,6 +281,11 @@ int irq_domain_xlate_onetwocell(struct irq_domain *d, struct device_node *ctrlr,
 			const u32 *intspec, unsigned int intsize,
 			irq_hw_number_t *out_hwirq, unsigned int *out_type);
 
+/* IPI functions */
+unsigned int irq_reserve_ipi(struct irq_domain *domain,
+			     const struct ipi_mask *dest);
+void irq_destroy_ipi(unsigned int irq);
+
 /* V2 interfaces to support hierarchy IRQ domains. */
 extern struct irq_data *irq_domain_get_irq_data(struct irq_domain *domain,
 						unsigned int virq);
diff --git a/kernel/irq/irqdomain.c b/kernel/irq/irqdomain.c
index dc9d27c0c158..781407f7d692 100644
--- a/kernel/irq/irqdomain.c
+++ b/kernel/irq/irqdomain.c
@@ -749,6 +749,90 @@ static int irq_domain_alloc_descs(int virq, unsigned int cnt,
 	return virq;
 }
 
+/**
+ * irq_reserve_ipi() - setup an IPI to destination cpumask
+ * @domain: IPI domain
+ * @dest: cpumask of cpus to receive the IPI
+ *
+ * Allocate a virq that can be used to send IPI to any CPU in dest mask.
+ *
+ * On success it'll return linux irq number and 0 on failure
+ */
+unsigned int irq_reserve_ipi(struct irq_domain *domain,
+			     const struct ipi_mask *dest)
+{
+	struct irq_data *data;
+	int virq;
+	unsigned int nr_irqs;
+
+	if (domain == NULL)
+		domain = irq_default_domain; /* need a separate ipi_default_domain? */
+
+	if (domain == NULL) {
+		pr_warn("Must provide a valid IPI domain!\n");
+		return 0;
+	}
+
+	if (!irq_domain_is_ipi(domain)) {
+		pr_warn("Not an IPI domain!\n");
+		return 0;
+	}
+
+	/* always allocate a virq per cpu */
+	nr_irqs = bitmap_weight(dest->cpumask, dest->nbits);;
+
+	virq = irq_domain_alloc_descs(-1, nr_irqs, 0, NUMA_NO_NODE);
+	if (virq <= 0) {
+		pr_warn("Can't reserve IPI, failed to alloc descs\n");
+		return 0;
+	}
+
+	/* we are reusing hierarchy alloc function, should we create another one? */
+	virq = __irq_domain_alloc_irqs(domain, virq, nr_irqs, NUMA_NO_NODE,
+					(void *) dest, true);
+	if (virq <= 0) {
+		pr_warn("Can't reserve IPI, failed to alloc irqs\n");
+		goto free_descs;
+	}
+
+	data = irq_get_irq_data(virq);
+	bitmap_copy(data->ipi_mask.cpumask, dest->cpumask, dest->nbits);
+	data->ipi_mask.nbits = dest->nbits;
+
+	return virq;
+
+free_descs:
+	irq_free_descs(virq, nr_irqs);
+	return 0;
+}
+
+/**
+ * irq_destroy_ipi() - unreserve an IPI that was previously allocated
+ * @irq: linux irq number to be destroyed
+ *
+ * Return an IPI allocated with irq_reserve_ipi() to the system.
+ */
+void irq_destroy_ipi(unsigned int irq)
+{
+	struct irq_data *data = irq_get_irq_data(irq);
+	struct irq_domain *domain;
+
+	if (!irq || !data)
+		return;
+
+	domain = data->domain;
+	if (WARN_ON(domain == NULL))
+		return;
+
+	if (!irq_domain_is_ipi(domain)) {
+		pr_warn("Not an IPI domain!\n");
+		return;
+	}
+
+	irq_domain_free_irqs(irq,
+		bitmap_weight(data->ipi_mask.cpumask, data->ipi_mask.nbits));
+}
+
 #ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
 /**
  * irq_domain_add_hierarchy - Add a irqdomain into the hierarchy
-- 
2.1.0

Re: [RFC v2 PATCH 07/14] irq: add a new generic IPI reservation code to irq core

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 1779 bytes --]

Hi Qais,

[auto build test ERROR on v4.3-rc5 -- if it's inappropriate base, please suggest rules for selecting the more suitable base]

url:    https://github.com/0day-ci/linux/commits/Qais-Yousef/Implement-generic-IPI-support-mechanism/20151013-182314
config: x86_64-allnoconfig (attached as .config)
reproduce:
        # save the attached .config to linux build tree
        make ARCH=x86_64 

All errors (new ones prefixed by >>):

   kernel/irq/irqdomain.c: In function 'irq_reserve_ipi':
>> kernel/irq/irqdomain.c:799:18: error: 'struct irq_data' has no member named 'ipi_mask'
     bitmap_copy(data->ipi_mask.cpumask, dest->cpumask, dest->nbits);
                     ^
   kernel/irq/irqdomain.c:800:6: error: 'struct irq_data' has no member named 'ipi_mask'
     data->ipi_mask.nbits = dest->nbits;
         ^
   kernel/irq/irqdomain.c: In function 'irq_destroy_ipi':
   kernel/irq/irqdomain.c:833:21: error: 'struct irq_data' has no member named 'ipi_mask'
      bitmap_weight(data->ipi_mask.cpumask, data->ipi_mask.nbits));
                        ^
   kernel/irq/irqdomain.c:833:45: error: 'struct irq_data' has no member named 'ipi_mask'
      bitmap_weight(data->ipi_mask.cpumask, data->ipi_mask.nbits));
                                                ^

vim +799 kernel/irq/irqdomain.c

   793		if (virq <= 0) {
   794			pr_warn("Can't reserve IPI, failed to alloc irqs\n");
   795			goto free_descs;
   796		}
   797	
   798		data = irq_get_irq_data(virq);
 > 799		bitmap_copy(data->ipi_mask.cpumask, dest->cpumask, dest->nbits);
   800		data->ipi_mask.nbits = dest->nbits;
   801	
   802		return virq;

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 6030 bytes --]

Re: [RFC v2 PATCH 07/14] irq: add a new generic IPI reservation code to irq core

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 3114 bytes --]

Hi Qais,

[auto build test ERROR on v4.3-rc5 -- if it's inappropriate base, please suggest rules for selecting the more suitable base]

url:    https://github.com/0day-ci/linux/commits/Qais-Yousef/Implement-generic-IPI-support-mechanism/20151013-182314
config: sh-titan_defconfig (attached as .config)
reproduce:
        wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        make.cross ARCH=sh 

All errors (new ones prefixed by >>):

   kernel/irq/irqdomain.c: In function 'irq_reserve_ipi':
>> kernel/irq/irqdomain.c:791:2: error: implicit declaration of function '__irq_domain_alloc_irqs' [-Werror=implicit-function-declaration]
   kernel/irq/irqdomain.c:799:18: error: 'struct irq_data' has no member named 'ipi_mask'
   kernel/irq/irqdomain.c:800:6: error: 'struct irq_data' has no member named 'ipi_mask'
   kernel/irq/irqdomain.c: In function 'irq_destroy_ipi':
>> kernel/irq/irqdomain.c:832:2: error: implicit declaration of function 'irq_domain_free_irqs' [-Werror=implicit-function-declaration]
   kernel/irq/irqdomain.c:833:21: error: 'struct irq_data' has no member named 'ipi_mask'
   kernel/irq/irqdomain.c:833:45: error: 'struct irq_data' has no member named 'ipi_mask'
   cc1: some warnings being treated as errors

vim +/__irq_domain_alloc_irqs +791 kernel/irq/irqdomain.c

   785		if (virq <= 0) {
   786			pr_warn("Can't reserve IPI, failed to alloc descs\n");
   787			return 0;
   788		}
   789	
   790		/* we are reusing hierarchy alloc function, should we create another one? */
 > 791		virq = __irq_domain_alloc_irqs(domain, virq, nr_irqs, NUMA_NO_NODE,
   792						(void *) dest, true);
   793		if (virq <= 0) {
   794			pr_warn("Can't reserve IPI, failed to alloc irqs\n");
   795			goto free_descs;
   796		}
   797	
   798		data = irq_get_irq_data(virq);
   799		bitmap_copy(data->ipi_mask.cpumask, dest->cpumask, dest->nbits);
   800		data->ipi_mask.nbits = dest->nbits;
   801	
   802		return virq;
   803	
   804	free_descs:
   805		irq_free_descs(virq, nr_irqs);
   806		return 0;
   807	}
   808	
   809	/**
   810	 * irq_destroy_ipi() - unreserve an IPI that was previously allocated
   811	 * @irq: linux irq number to be destroyed
   812	 *
   813	 * Return an IPI allocated with irq_reserve_ipi() to the system.
   814	 */
   815	void irq_destroy_ipi(unsigned int irq)
   816	{
   817		struct irq_data *data = irq_get_irq_data(irq);
   818		struct irq_domain *domain;
   819	
   820		if (!irq || !data)
   821			return;
   822	
   823		domain = data->domain;
   824		if (WARN_ON(domain == NULL))
   825			return;
   826	
   827		if (!irq_domain_is_ipi(domain)) {
   828			pr_warn("Not an IPI domain!\n");
   829			return;
   830		}
   831	
 > 832		irq_domain_free_irqs(irq,
   833			bitmap_weight(data->ipi_mask.cpumask, data->ipi_mask.nbits));
   834	}
   835	

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 15297 bytes --]

Re: [RFC v2 PATCH 07/14] irq: add a new generic IPI reservation code to irq core

[Thomas Gleixner]

On Tue, 13 Oct 2015, Qais Yousef wrote:
> +/**
> + * irq_reserve_ipi() - setup an IPI to destination cpumask
> + * @domain: IPI domain
> + * @dest: cpumask of cpus to receive the IPI
> + *
> + * Allocate a virq that can be used to send IPI to any CPU in dest mask.
> + *
> + * On success it'll return linux irq number and 0 on failure
> + */
> +unsigned int irq_reserve_ipi(struct irq_domain *domain,
> +			     const struct ipi_mask *dest)
> +{
> +	struct irq_data *data;
> +	int virq;
> +	unsigned int nr_irqs;

Please order them so:

+	struct irq_data *data;
+	unsigned int nr_irqs;
+	int virq;

Much simpler to read.

> +	if (domain == NULL)
> +		domain = irq_default_domain; /* need a separate ipi_default_domain? */

No tail comments please.

We should neither use irq_default_domain nor have an
ipi_default_domain.

> +
> +	if (domain == NULL) {
> +		pr_warn("Must provide a valid IPI domain!\n");
> +		return 0;
> +	}
> +
> +	if (!irq_domain_is_ipi(domain)) {
> +		pr_warn("Not an IPI domain!\n");
> +		return 0;
> +	}
> +
> +	/* always allocate a virq per cpu */
> +	nr_irqs = bitmap_weight(dest->cpumask, dest->nbits);;

Double semicolon

> +
> +	virq = irq_domain_alloc_descs(-1, nr_irqs, 0, NUMA_NO_NODE);
> +	if (virq <= 0) {
> +		pr_warn("Can't reserve IPI, failed to alloc descs\n");
> +		return 0;
> +	}
> +
> +	/* we are reusing hierarchy alloc function, should we create another one? */
> +	virq = __irq_domain_alloc_irqs(domain, virq, nr_irqs, NUMA_NO_NODE,
> +					(void *) dest, true);
> +	if (virq <= 0) {
> +		pr_warn("Can't reserve IPI, failed to alloc irqs\n");
> +		goto free_descs;
> +	}
> +
> +	data = irq_get_irq_data(virq);
> +	bitmap_copy(data->ipi_mask.cpumask, dest->cpumask, dest->nbits);
> +	data->ipi_mask.nbits = dest->nbits;

This does only initialize the first virq data. What about the others?

> +	return virq;
> +
> +free_descs:
> +	irq_free_descs(virq, nr_irqs);
> +	return 0;
> +}
> +
> +/**
> + * irq_destroy_ipi() - unreserve an IPI that was previously allocated
> + * @irq: linux irq number to be destroyed
> + *
> + * Return an IPI allocated with irq_reserve_ipi() to the system.

That wants to explain that it actually destroys a number of virqs not
just the primary one.

Thanks,

	tglx

Re: [RFC v2 PATCH 07/14] irq: add a new generic IPI reservation code to irq core

[Qais Yousef]

On 10/13/2015 02:37 PM, Thomas Gleixner wrote:
> On Tue, 13 Oct 2015, Qais Yousef wrote:
>
>> +	if (domain == NULL)
>> +		domain = irq_default_domain; /* need a separate ipi_default_domain? */
> No tail comments please.
>
> We should neither use irq_default_domain nor have an
> ipi_default_domain.

OK though I understood that you were OK with using the irq_default_domain.

This means that arch code must parse the DT for an IPI domain. I think 
I've seen arch code using the root FDT to search for a specific node. 
I'll try to do something similar to search for an IPI domain.

>> +
>> +	virq = irq_domain_alloc_descs(-1, nr_irqs, 0, NUMA_NO_NODE);
>> +	if (virq <= 0) {
>> +		pr_warn("Can't reserve IPI, failed to alloc descs\n");
>> +		return 0;
>> +	}
>> +
>> +	/* we are reusing hierarchy alloc function, should we create another one? */
>> +	virq = __irq_domain_alloc_irqs(domain, virq, nr_irqs, NUMA_NO_NODE,
>> +					(void *) dest, true);
>> +	if (virq <= 0) {
>> +		pr_warn("Can't reserve IPI, failed to alloc irqs\n");
>> +		goto free_descs;
>> +	}
>> +
>> +	data = irq_get_irq_data(virq);
>> +	bitmap_copy(data->ipi_mask.cpumask, dest->cpumask, dest->nbits);
>> +	data->ipi_mask.nbits = dest->nbits;
> This does only initialize the first virq data. What about the others?

Right I missed that. I'll fix it.


>
>> +	return virq;
>> +
>> +free_descs:
>> +	irq_free_descs(virq, nr_irqs);
>> +	return 0;
>> +}
>> +
>> +/**
>> + * irq_destroy_ipi() - unreserve an IPI that was previously allocated
>> + * @irq: linux irq number to be destroyed
>> + *
>> + * Return an IPI allocated with irq_reserve_ipi() to the system.
> That wants to explain that it actually destroys a number of virqs not
> just the primary one.
>
>

OK I'll expand on that.

Thanks,
Qais

Re: [RFC v2 PATCH 07/14] irq: add a new generic IPI reservation code to irq core

[Qais Yousef]

Hi Thomas,

On 10/13/2015 03:38 PM, Qais Yousef wrote:
> On 10/13/2015 02:37 PM, Thomas Gleixner wrote:
>> On Tue, 13 Oct 2015, Qais Yousef wrote:
>>
>>> +    if (domain == NULL)
>>> +        domain = irq_default_domain; /* need a separate 
>>> ipi_default_domain? */
>> No tail comments please.
>>
>> We should neither use irq_default_domain nor have an
>> ipi_default_domain.
>
> OK though I understood that you were OK with using the 
> irq_default_domain.
>
> This means that arch code must parse the DT for an IPI domain. I think 
> I've seen arch code using the root FDT to search for a specific node. 
> I'll try to do something similar to search for an IPI domain.

I'm having an issue here. I made the arch code look for an IPI domain 
but we have some platforms on MIPS that don't support device tree. I 
can't find how I can go away with that without using irq_default_domain.

Also irq_default_domain is understandably only visible within irqdomain 
code, so I can't make the arch code fallback to passing 
irq_default_domain if the platform doesn't support DT.

Are you OK with keeping this in irq_reserve_ipi()? Alternatively I could 
modify irq_find_host(), which I'm using to find the IPI domain, return 
irq_default_domain() if the passed node is NULL.

Or maybe there's a better option I couldn't think of?

Thanks,
Qais

Re: [RFC v2 PATCH 07/14] irq: add a new generic IPI reservation code to irq core

[Qais Yousef]

On 10/28/2015 03:46 PM, Qais Yousef wrote:
> Hi Thomas,
>
> On 10/13/2015 03:38 PM, Qais Yousef wrote:
>> On 10/13/2015 02:37 PM, Thomas Gleixner wrote:
>>> On Tue, 13 Oct 2015, Qais Yousef wrote:
>>>
>>>> +    if (domain == NULL)
>>>> +        domain = irq_default_domain; /* need a separate 
>>>> ipi_default_domain? */
>>> No tail comments please.
>>>
>>> We should neither use irq_default_domain nor have an
>>> ipi_default_domain.
>>
>> OK though I understood that you were OK with using the 
>> irq_default_domain.
>>
>> This means that arch code must parse the DT for an IPI domain. I 
>> think I've seen arch code using the root FDT to search for a specific 
>> node. I'll try to do something similar to search for an IPI domain.
>
> I'm having an issue here. I made the arch code look for an IPI domain 
> but we have some platforms on MIPS that don't support device tree. I 
> can't find how I can go away with that without using irq_default_domain.
>
> Also irq_default_domain is understandably only visible within 
> irqdomain code, so I can't make the arch code fallback to passing 
> irq_default_domain if the platform doesn't support DT.
>
> Are you OK with keeping this in irq_reserve_ipi()? Alternatively I 
> could modify irq_find_host(), which I'm using to find the IPI domain, 
> return irq_default_domain() if the passed node is NULL.
>
> Or maybe there's a better option I couldn't think of?

I think I can fix this in the GIC driver by using my own 
irq_domain->ops->match() function to ignore when of_node is NULL to cope 
with platforms without DT.

Sorry for the noise.

Thanks,
Qais

[RFC v2 PATCH 08/14] irq: implement irq_send_ipi

[Qais Yousef]

There are 2 variants. __irq_desc_send_ipi() is meant to be used by arch code to
save the desc lookup when doing SMP IPIs.

irq_send_ipi() is meant for drivers that want to send IPIs to coprocessors they
interact with.

Signed-off-by: Qais Yousef <qais.yousef@imgtec.com>
---
 include/linux/irq.h |  3 +++
 kernel/irq/manage.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 47 insertions(+)

diff --git a/include/linux/irq.h b/include/linux/irq.h
index c3d0f26c3eff..32c740ac95b4 100644
--- a/include/linux/irq.h
+++ b/include/linux/irq.h
@@ -985,4 +985,7 @@ int irq_map_ipi(struct ipi_mapping *map,
 int irq_unmap_ipi(struct ipi_mapping *map,
 		  unsigned int cpu, irq_hw_number_t *hwirq);
 
+int __irq_desc_send_ipi(struct irq_desc *desc, const struct ipi_mask *dest);
+int irq_send_ipi(unsigned int virq, const struct ipi_mask *dest);
+
 #endif /* _LINUX_IRQ_H */
diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
index 9a9bc0822c8f..f2425116a243 100644
--- a/kernel/irq/manage.c
+++ b/kernel/irq/manage.c
@@ -1983,3 +1983,47 @@ int irq_unmap_ipi(struct ipi_mapping *map,
 
 	return 0;
 }
+
+int __irq_desc_send_ipi(struct irq_desc *desc, const struct ipi_mask *dest)
+{
+	struct irq_data *data = irq_desc_get_irq_data(desc);
+	struct irq_chip *chip = irq_data_get_irq_chip(data);
+
+	if (!chip || !chip->irq_send_ipi)
+		return -EINVAL;
+
+	/*
+	 * Do not validate the mask for IPIs marked global. These are
+	 * regular IPIs so we can avoid the operation as their target
+	 * mask is the cpu_possible_mask.
+	 */
+	if (!dest->global) {
+		if (!bitmap_subset(dest->cpumask, data->ipi_mask.cpumask,
+				   dest->nbits))
+			return -EINVAL;
+	}
+
+	chip->irq_send_ipi(data, dest);
+	return 0;
+}
+
+/**
+ * irq_send_ipi() - send an IPI to target CPU(s)
+ * @irq: linux irq number from irq_reserve_ipi()
+ * @dest: dest CPU(s), must be the same or a subset of the mask passed to
+ *	  irq_reserve_ipi()
+ *
+ * Sends an IPI to all cpus in dest mask.
+ *
+ * Returns 0 on success and errno otherwise..
+ */
+int irq_send_ipi(unsigned int virq, const struct ipi_mask *dest)
+{
+	struct irq_desc *desc = irq_to_desc(virq);
+
+	if (!desc)
+		return -EINVAL;
+
+	return __irq_desc_send_ipi(desc, dest);
+}
+EXPORT_SYMBOL(irq_send_ipi);
-- 
2.1.0

Re: [RFC v2 PATCH 08/14] irq: implement irq_send_ipi

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 1250 bytes --]

Hi Qais,

[auto build test ERROR on v4.3-rc5 -- if it's inappropriate base, please suggest rules for selecting the more suitable base]

url:    https://github.com/0day-ci/linux/commits/Qais-Yousef/Implement-generic-IPI-support-mechanism/20151013-182314
config: x86_64-allnoconfig (attached as .config)
reproduce:
        # save the attached .config to linux build tree
        make ARCH=x86_64 

All errors (new ones prefixed by >>):

   kernel/irq/manage.c: In function '__irq_desc_send_ipi':
>> kernel/irq/manage.c:2001:41: error: 'struct irq_data' has no member named 'ipi_mask'
      if (!bitmap_subset(dest->cpumask, data->ipi_mask.cpumask,
                                            ^

vim +2001 kernel/irq/manage.c

  1995		/*
  1996		 * Do not validate the mask for IPIs marked global. These are
  1997		 * regular IPIs so we can avoid the operation as their target
  1998		 * mask is the cpu_possible_mask.
  1999		 */
  2000		if (!dest->global) {
> 2001			if (!bitmap_subset(dest->cpumask, data->ipi_mask.cpumask,
  2002					   dest->nbits))
  2003				return -EINVAL;
  2004		}

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 6030 bytes --]

Re: [RFC v2 PATCH 08/14] irq: implement irq_send_ipi

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 3369 bytes --]

Hi Qais,

[auto build test WARNING on v4.3-rc5 -- if it's inappropriate base, please suggest rules for selecting the more suitable base]

url:    https://github.com/0day-ci/linux/commits/Qais-Yousef/Implement-generic-IPI-support-mechanism/20151013-182314
config: x86_64-randconfig-s0-10131828 (attached as .config)
reproduce:
        # save the attached .config to linux build tree
        make ARCH=x86_64 

All warnings (new ones prefixed by >>):

   In file included from include/uapi/linux/stddef.h:1:0,
                    from include/linux/stddef.h:4,
                    from include/uapi/linux/posix_types.h:4,
                    from include/uapi/linux/types.h:13,
                    from include/linux/types.h:5,
                    from include/linux/smp.h:10,
                    from include/linux/irq.h:12,
                    from kernel/irq/manage.c:12:
   kernel/irq/manage.c: In function '__irq_desc_send_ipi':
   kernel/irq/manage.c:2001:41: error: 'struct irq_data' has no member named 'ipi_mask'
      if (!bitmap_subset(dest->cpumask, data->ipi_mask.cpumask,
                                            ^
   include/linux/compiler.h:147:28: note: in definition of macro '__trace_if'
     if (__builtin_constant_p((cond)) ? !!(cond) :   \
                               ^
>> kernel/irq/manage.c:2001:3: note: in expansion of macro 'if'
      if (!bitmap_subset(dest->cpumask, data->ipi_mask.cpumask,
      ^
   kernel/irq/manage.c:2001:41: error: 'struct irq_data' has no member named 'ipi_mask'
      if (!bitmap_subset(dest->cpumask, data->ipi_mask.cpumask,
                                            ^
   include/linux/compiler.h:147:40: note: in definition of macro '__trace_if'
     if (__builtin_constant_p((cond)) ? !!(cond) :   \
                                           ^
>> kernel/irq/manage.c:2001:3: note: in expansion of macro 'if'
      if (!bitmap_subset(dest->cpumask, data->ipi_mask.cpumask,
      ^
   kernel/irq/manage.c:2001:41: error: 'struct irq_data' has no member named 'ipi_mask'
      if (!bitmap_subset(dest->cpumask, data->ipi_mask.cpumask,
                                            ^
   include/linux/compiler.h:158:16: note: in definition of macro '__trace_if'
      ______r = !!(cond);     \
                   ^
>> kernel/irq/manage.c:2001:3: note: in expansion of macro 'if'
      if (!bitmap_subset(dest->cpumask, data->ipi_mask.cpumask,
      ^

vim +/if +2001 kernel/irq/manage.c

  1985	}
  1986	
  1987	int __irq_desc_send_ipi(struct irq_desc *desc, const struct ipi_mask *dest)
  1988	{
  1989		struct irq_data *data = irq_desc_get_irq_data(desc);
  1990		struct irq_chip *chip = irq_data_get_irq_chip(data);
  1991	
  1992		if (!chip || !chip->irq_send_ipi)
  1993			return -EINVAL;
  1994	
  1995		/*
  1996		 * Do not validate the mask for IPIs marked global. These are
  1997		 * regular IPIs so we can avoid the operation as their target
  1998		 * mask is the cpu_possible_mask.
  1999		 */
  2000		if (!dest->global) {
> 2001			if (!bitmap_subset(dest->cpumask, data->ipi_mask.cpumask,
  2002					   dest->nbits))
  2003				return -EINVAL;
  2004		}
  2005	
  2006		chip->irq_send_ipi(data, dest);
  2007		return 0;
  2008	}
  2009	

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 22902 bytes --]

Re: [RFC v2 PATCH 08/14] irq: implement irq_send_ipi

[Thomas Gleixner]

On Tue, 13 Oct 2015, Qais Yousef wrote:

Lacks kerneldoc

> +int __irq_desc_send_ipi(struct irq_desc *desc, const struct ipi_mask *dest)
> +{
> +	struct irq_data *data = irq_desc_get_irq_data(desc);
> +	struct irq_chip *chip = irq_data_get_irq_chip(data);
> +
> +	if (!chip || !chip->irq_send_ipi)
> +		return -EINVAL;
> +
> +	/*
> +	 * Do not validate the mask for IPIs marked global. These are
> +	 * regular IPIs so we can avoid the operation as their target
> +	 * mask is the cpu_possible_mask.
> +	 */
> +	if (!dest->global) {
> +		if (!bitmap_subset(dest->cpumask, data->ipi_mask.cpumask,
> +				   dest->nbits))
> +			return -EINVAL;
> +	}

This looks half thought out. You rely on the caller getting the global
bit right. There should be a sanity check for this versus
data->ipi_mask and also you need to validate nbits.

> +EXPORT_SYMBOL(irq_send_ipi);

EXPORT_SYMBOL_GPL please

Thanks,

	tglx

Re: [RFC v2 PATCH 08/14] irq: implement irq_send_ipi

[Qais Yousef]

On 10/13/2015 02:40 PM, Thomas Gleixner wrote:
> On Tue, 13 Oct 2015, Qais Yousef wrote:
>
> Lacks kerneldoc
>
>> +int __irq_desc_send_ipi(struct irq_desc *desc, const struct ipi_mask *dest)
>> +{
>> +	struct irq_data *data = irq_desc_get_irq_data(desc);
>> +	struct irq_chip *chip = irq_data_get_irq_chip(data);
>> +
>> +	if (!chip || !chip->irq_send_ipi)
>> +		return -EINVAL;
>> +
>> +	/*
>> +	 * Do not validate the mask for IPIs marked global. These are
>> +	 * regular IPIs so we can avoid the operation as their target
>> +	 * mask is the cpu_possible_mask.
>> +	 */
>> +	if (!dest->global) {
>> +		if (!bitmap_subset(dest->cpumask, data->ipi_mask.cpumask,
>> +				   dest->nbits))
>> +			return -EINVAL;
>> +	}
> This looks half thought out. You rely on the caller getting the global
> bit right. There should be a sanity check for this versus
> data->ipi_mask and also you need to validate nbits.

Yes I might have rushed this part as I did it last. I'll improve it.

>
>> +EXPORT_SYMBOL(irq_send_ipi);
> EXPORT_SYMBOL_GPL please
>
>

OK.

Thanks,
Qais

[RFC v2 PATCH 09/14] MIPS: add support for generic SMP IPI support

[Qais Yousef]

Use the new generic IPI layer to provide generic SMP IPI support if the irqchip
supports it.

Signed-off-by: Qais Yousef <qais.yousef@imgtec.com>
---
 arch/mips/kernel/smp.c | 117 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 117 insertions(+)

diff --git a/arch/mips/kernel/smp.c b/arch/mips/kernel/smp.c
index bd4385a8e6e8..6cbadfd17439 100644
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -39,6 +39,7 @@
 #include <asm/processor.h>
 #include <asm/idle.h>
 #include <asm/r4k-timer.h>
+#include <asm/mips-cpc.h>
 #include <asm/mmu_context.h>
 #include <asm/time.h>
 #include <asm/setup.h>
@@ -79,6 +80,11 @@ static cpumask_t cpu_core_setup_map;
 
 cpumask_t cpu_coherent_mask;
 
+#ifdef CONFIG_GENERIC_IRQ_IPI
+static struct irq_desc *call_desc;
+static struct irq_desc *sched_desc;
+#endif
+
 static inline void set_cpu_sibling_map(int cpu)
 {
 	int i;
@@ -145,6 +151,117 @@ void register_smp_ops(struct plat_smp_ops *ops)
 	mp_ops = ops;
 }
 
+#ifdef CONFIG_GENERIC_IRQ_IPI
+void generic_smp_send_ipi_single(int cpu, unsigned int action)
+{
+	generic_smp_send_ipi_mask(cpumask_of(cpu), action);
+}
+
+void generic_smp_send_ipi_mask(const struct cpumask *mask, unsigned int action)
+{
+	unsigned long flags;
+	unsigned int core;
+	int cpu;
+	struct ipi_mask ipi_mask;
+
+	ipi_mask.cpumask = ((struct cpumask *)mask)->bits;
+	ipi_mask.nbits = NR_CPUS;
+	ipi_mask.global = true;
+
+	local_irq_save(flags);
+
+	switch (action) {
+	case SMP_CALL_FUNCTION:
+		__irq_desc_send_ipi(call_desc, &ipi_mask);
+		break;
+
+	case SMP_RESCHEDULE_YOURSELF:
+		__irq_desc_send_ipi(sched_desc, &ipi_mask);
+		break;
+
+	default:
+		BUG();
+	}
+
+	if (mips_cpc_present() && (core != current_cpu_data.core)) {
+		for_each_cpu(cpu, mask) {
+			core = cpu_data[cpu].core;
+			while (!cpumask_test_cpu(cpu, &cpu_coherent_mask)) {
+				mips_cpc_lock_other(core);
+				write_cpc_co_cmd(CPC_Cx_CMD_PWRUP);
+				mips_cpc_unlock_other();
+			}
+		}
+	}
+
+	local_irq_restore(flags);
+}
+
+
+static irqreturn_t ipi_resched_interrupt(int irq, void *dev_id)
+{
+	scheduler_ipi();
+
+	return IRQ_HANDLED;
+}
+
+static irqreturn_t ipi_call_interrupt(int irq, void *dev_id)
+{
+	smp_call_function_interrupt();
+
+	return IRQ_HANDLED;
+}
+
+static struct irqaction irq_resched = {
+	.handler	= ipi_resched_interrupt,
+	.flags		= IRQF_PERCPU,
+	.name		= "IPI resched"
+};
+
+static struct irqaction irq_call = {
+	.handler	= ipi_call_interrupt,
+	.flags		= IRQF_PERCPU,
+	.name		= "IPI call"
+};
+
+static __init void smp_ipi_init_one(unsigned int virq,
+				    struct irqaction *action)
+{
+	int ret;
+
+	irq_set_handler(virq, handle_percpu_irq);
+	ret = setup_irq(virq, action);
+	BUG_ON(ret);
+}
+
+static int __init generic_smp_ipi_init(void)
+{
+	unsigned int call_virq, sched_virq;
+	struct ipi_mask ipi_mask;
+	int cpu;
+
+	ipi_mask.cpumask = ((struct cpumask *)cpu_possible_mask)->bits;
+	ipi_mask.nbits = NR_CPUS;
+
+	call_virq = irq_reserve_ipi(NULL, &ipi_mask);
+	BUG_ON(!call_virq);
+
+	sched_virq = irq_reserve_ipi(NULL, &ipi_mask);
+	BUG_ON(!sched_virq);
+
+	for_each_cpu(cpu, cpu_possible_mask) {
+		smp_ipi_init_one(call_virq + cpu, &irq_call);
+		smp_ipi_init_one(sched_virq + cpu, &irq_resched);
+	}
+
+	call_desc = irq_to_desc(call_virq);
+	sched_desc = irq_to_desc(sched_virq);
+
+	return 0;
+}
+early_initcall(generic_smp_ipi_init);
+#endif
+
 /*
  * First C code run on the secondary CPUs after being started up by
  * the master.
-- 
2.1.0

Re: [RFC v2 PATCH 09/14] MIPS: add support for generic SMP IPI support

[Thomas Gleixner]

On Tue, 13 Oct 2015, Qais Yousef wrote:
>  
> +#ifdef CONFIG_GENERIC_IRQ_IPI
> +void generic_smp_send_ipi_single(int cpu, unsigned int action)

Please use a mips name space. This suggests that it s completely
generic, which is not true.

> +{
> +	generic_smp_send_ipi_mask(cpumask_of(cpu), action);
> +}
> +
> +void generic_smp_send_ipi_mask(const struct cpumask *mask, unsigned int action)

Ditto.

> +{
> +	unsigned long flags;
> +	unsigned int core;
> +	int cpu;
> +	struct ipi_mask ipi_mask;
> +
> +	ipi_mask.cpumask = ((struct cpumask *)mask)->bits;

We have accessors for that. Hmm, so for this case we must make the
ipi_mask different:

struct ipi_mask {
	unsigned int	nbits;
	bool		global;
	union {
       	     struct cpumask *mask;
	     unsigned long  cpu_bitmap[];
	};
};

Thanks,

	tglx

Re: [RFC v2 PATCH 09/14] MIPS: add support for generic SMP IPI support

[Qais Yousef]

On 10/13/2015 02:48 PM, Thomas Gleixner wrote:
> On Tue, 13 Oct 2015, Qais Yousef wrote:
>>   
>> +#ifdef CONFIG_GENERIC_IRQ_IPI
>> +void generic_smp_send_ipi_single(int cpu, unsigned int action)
> Please use a mips name space. This suggests that it s completely
> generic, which is not true.
>
>> +{
>> +	generic_smp_send_ipi_mask(cpumask_of(cpu), action);
>> +}
>> +
>> +void generic_smp_send_ipi_mask(const struct cpumask *mask, unsigned int action)
> Ditto.

OK.

>> +{
>> +	unsigned long flags;
>> +	unsigned int core;
>> +	int cpu;
>> +	struct ipi_mask ipi_mask;
>> +
>> +	ipi_mask.cpumask = ((struct cpumask *)mask)->bits;
> We have accessors for that. Hmm, so for this case we must make the
> ipi_mask different:
>
> struct ipi_mask {
> 	unsigned int	nbits;
> 	bool		global;
> 	union {
>         	     struct cpumask *mask;
> 	     unsigned long  cpu_bitmap[];
> 	};
> };
>

Right. This looks cleaner for sure. Not sure why I haven't though about 
this.

Thanks,
Qais

[RFC v2 PATCH 10/14] MIPS: make smp CMP, CPS and MT use the new generic IPI functions

[Qais Yousef]

Only the SMP variants that use GIC were converted as it's the only irqchip that
will have the support for generic IPI for now which will be introduced in
the following patches.

Signed-off-by: Qais Yousef <qais.yousef@imgtec.com>
---
 arch/mips/include/asm/smp-ops.h | 5 +++--
 arch/mips/kernel/smp-cmp.c      | 4 ++--
 arch/mips/kernel/smp-cps.c      | 4 ++--
 arch/mips/kernel/smp-mt.c       | 2 +-
 4 files changed, 8 insertions(+), 7 deletions(-)

diff --git a/arch/mips/include/asm/smp-ops.h b/arch/mips/include/asm/smp-ops.h
index 6ba1fb8b11e2..5e83aec0a14c 100644
--- a/arch/mips/include/asm/smp-ops.h
+++ b/arch/mips/include/asm/smp-ops.h
@@ -44,8 +44,9 @@ static inline void plat_smp_setup(void)
 	mp_ops->smp_setup();
 }
 
-extern void gic_send_ipi_single(int cpu, unsigned int action);
-extern void gic_send_ipi_mask(const struct cpumask *mask, unsigned int action);
+extern void generic_smp_send_ipi_single(int cpu, unsigned int action);
+extern void generic_smp_send_ipi_mask(const struct cpumask *mask,
+				      unsigned int action);
 
 #else /* !CONFIG_SMP */
 
diff --git a/arch/mips/kernel/smp-cmp.c b/arch/mips/kernel/smp-cmp.c
index d5e0f949dc48..a70080500e04 100644
--- a/arch/mips/kernel/smp-cmp.c
+++ b/arch/mips/kernel/smp-cmp.c
@@ -149,8 +149,8 @@ void __init cmp_prepare_cpus(unsigned int max_cpus)
 }
 
 struct plat_smp_ops cmp_smp_ops = {
-	.send_ipi_single	= gic_send_ipi_single,
-	.send_ipi_mask		= gic_send_ipi_mask,
+	.send_ipi_single	= generic_smp_send_ipi_single,
+	.send_ipi_mask		= generic_smp_send_ipi_mask,
 	.init_secondary		= cmp_init_secondary,
 	.smp_finish		= cmp_smp_finish,
 	.boot_secondary		= cmp_boot_secondary,
diff --git a/arch/mips/kernel/smp-cps.c b/arch/mips/kernel/smp-cps.c
index c88937745b4e..dd6834564ecb 100644
--- a/arch/mips/kernel/smp-cps.c
+++ b/arch/mips/kernel/smp-cps.c
@@ -444,8 +444,8 @@ static struct plat_smp_ops cps_smp_ops = {
 	.boot_secondary		= cps_boot_secondary,
 	.init_secondary		= cps_init_secondary,
 	.smp_finish		= cps_smp_finish,
-	.send_ipi_single	= gic_send_ipi_single,
-	.send_ipi_mask		= gic_send_ipi_mask,
+	.send_ipi_single	= generic_smp_send_ipi_single,
+	.send_ipi_mask		= generic_smp_send_ipi_mask,
 #ifdef CONFIG_HOTPLUG_CPU
 	.cpu_disable		= cps_cpu_disable,
 	.cpu_die		= cps_cpu_die,
diff --git a/arch/mips/kernel/smp-mt.c b/arch/mips/kernel/smp-mt.c
index 86311a164ef1..33793abe5a20 100644
--- a/arch/mips/kernel/smp-mt.c
+++ b/arch/mips/kernel/smp-mt.c
@@ -121,7 +121,7 @@ static void vsmp_send_ipi_single(int cpu, unsigned int action)
 
 #ifdef CONFIG_MIPS_GIC
 	if (gic_present) {
-		gic_send_ipi_single(cpu, action);
+		generic_smp_send_ipi_single(cpu, action);
 		return;
 	}
 #endif
-- 
2.1.0

Re: [RFC v2 PATCH 10/14] MIPS: make smp CMP, CPS and MT use the new generic IPI functions

[Thomas Gleixner]

On Tue, 13 Oct 2015, Qais Yousef wrote:

> Only the SMP variants that use GIC were converted as it's the only irqchip that
> will have the support for generic IPI for now which will be introduced in
> the following patches.

You break bisectability. Introduce the new gic magic first and then
switch it over.

Thanks,

	tglx

Re: [RFC v2 PATCH 10/14] MIPS: make smp CMP, CPS and MT use the new generic IPI functions

[Qais Yousef]

On 10/13/2015 02:49 PM, Thomas Gleixner wrote:
> On Tue, 13 Oct 2015, Qais Yousef wrote:
>
>> Only the SMP variants that use GIC were converted as it's the only irqchip that
>> will have the support for generic IPI for now which will be introduced in
>> the following patches.
> You break bisectability. Introduce the new gic magic first and then
> switch it over.
>
>

Yes I was trying to make the review easier by grouping things together. 
I might need to squash some of these last patches together to avoid 
breaking bisectability.

Thanks,
Qais

[RFC v2 PATCH 11/14] MIPS: delete smp-gic.c

[Qais Yousef]

We now have a generic IPI layer that will use GIC automatically
if it's compiled in.

Signed-off-by: Qais Yousef <qais.yousef@imgtec.com>
---
 arch/mips/Kconfig          |  6 -----
 arch/mips/kernel/Makefile  |  1 -
 arch/mips/kernel/smp-gic.c | 64 ----------------------------------------------
 3 files changed, 71 deletions(-)
 delete mode 100644 arch/mips/kernel/smp-gic.c

diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig
index e3aa5b0b4ef1..5a73c1217af7 100644
--- a/arch/mips/Kconfig
+++ b/arch/mips/Kconfig
@@ -2123,7 +2123,6 @@ config MIPS_MT_SMP
 	select CPU_MIPSR2_IRQ_VI
 	select CPU_MIPSR2_IRQ_EI
 	select SYNC_R4K
-	select MIPS_GIC_IPI
 	select MIPS_MT
 	select SMP
 	select SMP_UP
@@ -2221,7 +2220,6 @@ config MIPS_VPE_APSP_API_MT
 config MIPS_CMP
 	bool "MIPS CMP framework support (DEPRECATED)"
 	depends on SYS_SUPPORTS_MIPS_CMP && !CPU_MIPSR6
-	select MIPS_GIC_IPI
 	select SMP
 	select SYNC_R4K
 	select SYS_SUPPORTS_SMP
@@ -2241,7 +2239,6 @@ config MIPS_CPS
 	select MIPS_CM
 	select MIPS_CPC
 	select MIPS_CPS_PM if HOTPLUG_CPU
-	select MIPS_GIC_IPI
 	select SMP
 	select SYNC_R4K if (CEVT_R4K || CSRC_R4K)
 	select SYS_SUPPORTS_HOTPLUG_CPU
@@ -2259,9 +2256,6 @@ config MIPS_CPS_PM
 	select MIPS_CPC
 	bool
 
-config MIPS_GIC_IPI
-	bool
-
 config MIPS_CM
 	bool
 
diff --git a/arch/mips/kernel/Makefile b/arch/mips/kernel/Makefile
index d982be1ea1c3..a4bfc41d46b5 100644
--- a/arch/mips/kernel/Makefile
+++ b/arch/mips/kernel/Makefile
@@ -51,7 +51,6 @@ obj-$(CONFIG_MIPS_MT_FPAFF)	+= mips-mt-fpaff.o
 obj-$(CONFIG_MIPS_MT_SMP)	+= smp-mt.o
 obj-$(CONFIG_MIPS_CMP)		+= smp-cmp.o
 obj-$(CONFIG_MIPS_CPS)		+= smp-cps.o cps-vec.o
-obj-$(CONFIG_MIPS_GIC_IPI)	+= smp-gic.o
 obj-$(CONFIG_MIPS_SPRAM)	+= spram.o
 
 obj-$(CONFIG_MIPS_VPE_LOADER)	+= vpe.o
diff --git a/arch/mips/kernel/smp-gic.c b/arch/mips/kernel/smp-gic.c
deleted file mode 100644
index 5f0ab5bcd01e..000000000000
--- a/arch/mips/kernel/smp-gic.c
+++ /dev/null
@@ -1,64 +0,0 @@
-/*
- * Copyright (C) 2013 Imagination Technologies
- * Author: Paul Burton <paul.burton@imgtec.com>
- *
- * Based on smp-cmp.c:
- *  Copyright (C) 2007 MIPS Technologies, Inc.
- *  Author: Chris Dearman (chris@mips.com)
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation;  either version 2 of the  License, or (at your
- * option) any later version.
- */
-
-#include <linux/irqchip/mips-gic.h>
-#include <linux/printk.h>
-
-#include <asm/mips-cpc.h>
-#include <asm/smp-ops.h>
-
-void gic_send_ipi_single(int cpu, unsigned int action)
-{
-	unsigned long flags;
-	unsigned int intr;
-	unsigned int core = cpu_data[cpu].core;
-
-	pr_debug("CPU%d: %s cpu %d action %u status %08x\n",
-		 smp_processor_id(), __func__, cpu, action, read_c0_status());
-
-	local_irq_save(flags);
-
-	switch (action) {
-	case SMP_CALL_FUNCTION:
-		intr = plat_ipi_call_int_xlate(cpu);
-		break;
-
-	case SMP_RESCHEDULE_YOURSELF:
-		intr = plat_ipi_resched_int_xlate(cpu);
-		break;
-
-	default:
-		BUG();
-	}
-
-	gic_send_ipi(intr);
-
-	if (mips_cpc_present() && (core != current_cpu_data.core)) {
-		while (!cpumask_test_cpu(cpu, &cpu_coherent_mask)) {
-			mips_cpc_lock_other(core);
-			write_cpc_co_cmd(CPC_Cx_CMD_PWRUP);
-			mips_cpc_unlock_other();
-		}
-	}
-
-	local_irq_restore(flags);
-}
-
-void gic_send_ipi_mask(const struct cpumask *mask, unsigned int action)
-{
-	unsigned int i;
-
-	for_each_cpu(i, mask)
-		gic_send_ipi_single(i, action);
-}
-- 
2.1.0

[RFC v2 PATCH 12/14] irqchip: mips-gic: add a IPI hierarchy domain

[Qais Yousef]

Add a new ipi domain on top of the normal domain.

We set it as the default domain to allow arch code to use it without looking up
DT.

Signed-off-by: Qais Yousef <qais.yousef@imgtec.com>
---
 drivers/irqchip/Kconfig        |   2 +
 drivers/irqchip/irq-mips-gic.c | 126 ++++++++++++++++++++++++++++++++++++++---
 2 files changed, 121 insertions(+), 7 deletions(-)

diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
index 27b52c8729cd..d9a9c8c8cbe1 100644
--- a/drivers/irqchip/Kconfig
+++ b/drivers/irqchip/Kconfig
@@ -167,6 +167,8 @@ config KEYSTONE_IRQ
 
 config MIPS_GIC
 	bool
+	select GENERIC_IRQ_IPI
+	select IRQ_DOMAIN_HIERARCHY
 	select MIPS_CM
 
 config INGENIC_IRQ
diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c
index aeaa061f0dbf..9c8ece1d90f2 100644
--- a/drivers/irqchip/irq-mips-gic.c
+++ b/drivers/irqchip/irq-mips-gic.c
@@ -33,11 +33,14 @@ static void __iomem *gic_base;
 static struct gic_pcpu_mask pcpu_masks[NR_CPUS];
 static DEFINE_SPINLOCK(gic_lock);
 static struct irq_domain *gic_irq_domain;
+static struct irq_domain *gic_ipi_domain;
 static int gic_shared_intrs;
 static int gic_vpes;
 static unsigned int gic_cpu_pin;
 static unsigned int timer_cpu_pin;
 static struct irq_chip gic_level_irq_controller, gic_edge_irq_controller;
+DECLARE_BITMAP(ipi_intrs, GIC_MAX_INTRS);
+DECLARE_BITMAP(ipi_resrv, GIC_MAX_INTRS);
 
 static void __gic_irq_dispatch(void);
 
@@ -335,8 +338,14 @@ static void gic_handle_shared_int(bool chained)
 
 	intr = find_first_bit(pending, gic_shared_intrs);
 	while (intr != gic_shared_intrs) {
-		virq = irq_linear_revmap(gic_irq_domain,
-					 GIC_SHARED_TO_HWIRQ(intr));
+		if (test_bit(intr, ipi_intrs)) {
+			virq = irq_linear_revmap(gic_ipi_domain,
+					GIC_SHARED_TO_HWIRQ(intr));
+		} else {
+			virq = irq_linear_revmap(gic_irq_domain,
+					GIC_SHARED_TO_HWIRQ(intr));
+		}
+
 		if (chained)
 			generic_handle_irq(virq);
 		else
@@ -741,7 +750,7 @@ static int gic_local_irq_domain_map(struct irq_domain *d, unsigned int virq,
 }
 
 static int gic_shared_irq_domain_map(struct irq_domain *d, unsigned int virq,
-				     irq_hw_number_t hw)
+				     irq_hw_number_t hw, unsigned int vpe)
 {
 	int intr = GIC_HWIRQ_TO_SHARED(hw);
 	unsigned long flags;
@@ -751,9 +760,8 @@ static int gic_shared_irq_domain_map(struct irq_domain *d, unsigned int virq,
 
 	spin_lock_irqsave(&gic_lock, flags);
 	gic_map_to_pin(intr, gic_cpu_pin);
-	/* Map to VPE 0 by default */
-	gic_map_to_vpe(intr, 0);
-	set_bit(intr, pcpu_masks[0].pcpu_mask);
+	gic_map_to_vpe(intr, vpe);
+	set_bit(intr, pcpu_masks[vpe].pcpu_mask);
 	spin_unlock_irqrestore(&gic_lock, flags);
 
 	return 0;
@@ -764,7 +772,7 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int virq,
 {
 	if (GIC_HWIRQ_TO_LOCAL(hw) < GIC_NUM_LOCAL_INTRS)
 		return gic_local_irq_domain_map(d, virq, hw);
-	return gic_shared_irq_domain_map(d, virq, hw);
+	return gic_shared_irq_domain_map(d, virq, hw, 0);
 }
 
 static int gic_irq_domain_xlate(struct irq_domain *d, struct device_node *ctrlr,
@@ -791,6 +799,98 @@ static const struct irq_domain_ops gic_irq_domain_ops = {
 	.xlate = gic_irq_domain_xlate,
 };
 
+static int gic_ipi_domain_xlate(struct irq_domain *d, struct device_node *ctrlr,
+				const u32 *intspec, unsigned int intsize,
+				irq_hw_number_t *out_hwirq,
+				unsigned int *out_type)
+{
+	/*
+	 * There's nothing to translate here. hwirq is dynamically allocated and
+	 * the irq type is always edge triggered.
+	 * */
+	*out_hwirq = 0;
+	*out_type = IRQ_TYPE_EDGE_RISING;
+
+	return 0;
+}
+
+static int gic_ipi_domain_alloc(struct irq_domain *d, unsigned int virq,
+				unsigned int nr_irqs, void *arg)
+{
+	struct ipi_mask *cpumask = arg;
+	int cpu, ret;
+	irq_hw_number_t hwirq;
+	struct ipi_mapping *map;
+
+	map = irq_alloc_ipi_mapping(gic_vpes);
+	if (!map)
+		return -ENOMEM;
+
+	if (cpumask->nbits > map->nr_cpus)
+		return -EINVAL;
+
+	cpu = find_first_bit(cpumask->cpumask, cpumask->nbits);
+	while (cpu != cpumask->nbits) {
+		hwirq = find_first_bit(ipi_resrv, gic_shared_intrs);
+		if (hwirq == gic_shared_intrs) {
+			ret = -ENOMEM;
+			goto out;
+		}
+
+		bitmap_clear(ipi_resrv, hwirq, 1);
+		bitmap_set(ipi_intrs, hwirq, 1);
+
+		ret = irq_map_ipi(map, cpu, hwirq);
+		if (ret)
+			goto out;
+
+		hwirq = GIC_SHARED_TO_HWIRQ(hwirq);
+		ret = irq_domain_set_hwirq_and_chip(d, virq + cpu, hwirq,
+						&gic_edge_irq_controller, map);
+		if (ret)
+			goto out;
+
+		ret = gic_shared_irq_domain_map(d, virq + cpu, hwirq, cpu);
+		if (ret)
+			goto out;
+
+		ret = irq_set_irq_type(virq + cpu, IRQ_TYPE_EDGE_RISING);
+		if (ret)
+			goto out;
+
+		cpu = find_next_bit(cpumask->cpumask, cpumask->nbits, cpu + 1);
+	}
+
+	return 0;
+out:
+	irq_free_ipi_mapping(map);
+	return ret;
+}
+
+void gic_ipi_domain_free(struct irq_domain *d, unsigned int virq,
+			 unsigned int nr_irqs)
+{
+	struct ipi_mapping *map = irq_get_chip_data(virq);
+	irq_hw_number_t hwirq;
+	int ret, cpu;
+
+	for (cpu = 0; cpu < map->nr_cpus; cpu++) {
+		ret = irq_unmap_ipi(map, cpu, &hwirq);
+		if (!ret) {
+			bitmap_set(ipi_resrv, hwirq, 1);
+			bitmap_clear(ipi_intrs, hwirq, 1);
+		}
+	}
+
+	irq_free_ipi_mapping(map);
+}
+
+static struct irq_domain_ops gic_ipi_domain_ops = {
+	.xlate = gic_ipi_domain_xlate,
+	.alloc = gic_ipi_domain_alloc,
+	.free = gic_ipi_domain_free,
+};
+
 static void __init __gic_init(unsigned long gic_base_addr,
 			      unsigned long gic_addrspace_size,
 			      unsigned int cpu_vec, unsigned int irqbase,
@@ -850,6 +950,18 @@ static void __init __gic_init(unsigned long gic_base_addr,
 	if (!gic_irq_domain)
 		panic("Failed to add GIC IRQ domain");
 
+	gic_ipi_domain = irq_domain_add_hierarchy(gic_irq_domain, IRQ_DOMAIN_FLAG_IPI,
+						  GIC_NUM_LOCAL_INTRS + gic_shared_intrs,
+						  NULL, &gic_ipi_domain_ops, NULL);
+	if (!gic_ipi_domain)
+		panic("Failed to add GIC IPI domain");
+
+	/* set IPI domain as default */
+	irq_set_default_host(gic_ipi_domain);
+
+	/* Make the last 2 * NR_CPUS available for IPIs */
+	bitmap_set(ipi_resrv, gic_shared_intrs - 2 * NR_CPUS, 2 * NR_CPUS);
+
 	gic_basic_init();
 
 	gic_ipi_init();
-- 
2.1.0

[RFC v2 PATCH 13/14] irqchip: mips-gic: implement the new irq_send_ipi

[Qais Yousef]

Now irqchip has a irq_send_ipi() function, implement gic_send_ipi()
to make use of it

Signed-off-by: Qais Yousef <qais.yousef@imgtec.com>
---
 drivers/irqchip/irq-mips-gic.c | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)

diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c
index 9c8ece1d90f2..764470375cd0 100644
--- a/drivers/irqchip/irq-mips-gic.c
+++ b/drivers/irqchip/irq-mips-gic.c
@@ -266,9 +266,24 @@ static void gic_bind_eic_interrupt(int irq, int set)
 		  GIC_VPE_EIC_SS(irq), set);
 }
 
-void gic_send_ipi(unsigned int intr)
+static void gic_send_ipi(struct irq_data *d, const struct ipi_mask *cpumask)
 {
-	gic_write(GIC_REG(SHARED, GIC_SH_WEDGE), GIC_SH_WEDGE_SET(intr));
+	struct ipi_mapping *map = irq_data_get_irq_chip_data(d);
+	irq_hw_number_t intr;
+	int vpe;
+
+	if (!map)
+		return;
+
+	vpe = find_first_bit(cpumask->cpumask, cpumask->nbits);
+	while (vpe != cpumask->nbits) {
+		intr = map->cpumap[vpe];
+		if (intr == INVALID_HWIRQ)
+			continue;
+		gic_write(GIC_REG(SHARED, GIC_SH_WEDGE), GIC_SH_WEDGE_SET(intr));
+
+		vpe = find_next_bit(cpumask->cpumask, cpumask->nbits, vpe + 1);
+	}
 }
 
 int gic_get_c0_compare_int(void)
@@ -476,6 +491,7 @@ static struct irq_chip gic_edge_irq_controller = {
 #ifdef CONFIG_SMP
 	.irq_set_affinity	=	gic_set_affinity,
 #endif
+	.irq_send_ipi		=	gic_send_ipi,
 };
 
 static void gic_handle_local_int(bool chained)
-- 
2.1.0

[RFC v2 PATCH 14/14] irqchip: mips-gic: remove IPI init code

[Qais Yousef]

It's now all handled in generic arch code.

Signed-off-by: Qais Yousef <qais.yousef@imgtec.com>
---
 drivers/irqchip/irq-mips-gic.c   | 79 ----------------------------------------
 include/linux/irqchip/mips-gic.h |  3 --
 2 files changed, 82 deletions(-)

diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c
index 764470375cd0..8a56965d5980 100644
--- a/drivers/irqchip/irq-mips-gic.c
+++ b/drivers/irqchip/irq-mips-gic.c
@@ -585,83 +585,6 @@ static void gic_irq_dispatch(struct irq_desc *desc)
 	gic_handle_shared_int(true);
 }
 
-#ifdef CONFIG_MIPS_GIC_IPI
-static int gic_resched_int_base;
-static int gic_call_int_base;
-
-unsigned int plat_ipi_resched_int_xlate(unsigned int cpu)
-{
-	return gic_resched_int_base + cpu;
-}
-
-unsigned int plat_ipi_call_int_xlate(unsigned int cpu)
-{
-	return gic_call_int_base + cpu;
-}
-
-static irqreturn_t ipi_resched_interrupt(int irq, void *dev_id)
-{
-	scheduler_ipi();
-
-	return IRQ_HANDLED;
-}
-
-static irqreturn_t ipi_call_interrupt(int irq, void *dev_id)
-{
-	generic_smp_call_function_interrupt();
-
-	return IRQ_HANDLED;
-}
-
-static struct irqaction irq_resched = {
-	.handler	= ipi_resched_interrupt,
-	.flags		= IRQF_PERCPU,
-	.name		= "IPI resched"
-};
-
-static struct irqaction irq_call = {
-	.handler	= ipi_call_interrupt,
-	.flags		= IRQF_PERCPU,
-	.name		= "IPI call"
-};
-
-static __init void gic_ipi_init_one(unsigned int intr, int cpu,
-				    struct irqaction *action)
-{
-	int virq = irq_create_mapping(gic_irq_domain,
-				      GIC_SHARED_TO_HWIRQ(intr));
-	int i;
-
-	gic_map_to_vpe(intr, mips_cm_vp_id(cpu));
-	for (i = 0; i < NR_CPUS; i++)
-		clear_bit(intr, pcpu_masks[i].pcpu_mask);
-	set_bit(intr, pcpu_masks[cpu].pcpu_mask);
-
-	irq_set_irq_type(virq, IRQ_TYPE_EDGE_RISING);
-
-	irq_set_handler(virq, handle_percpu_irq);
-	setup_irq(virq, action);
-}
-
-static __init void gic_ipi_init(void)
-{
-	int i;
-
-	/* Use last 2 * NR_CPUS interrupts as IPIs */
-	gic_resched_int_base = gic_shared_intrs - nr_cpu_ids;
-	gic_call_int_base = gic_resched_int_base - nr_cpu_ids;
-
-	for (i = 0; i < nr_cpu_ids; i++) {
-		gic_ipi_init_one(gic_call_int_base + i, i, &irq_call);
-		gic_ipi_init_one(gic_resched_int_base + i, i, &irq_resched);
-	}
-}
-#else
-static inline void gic_ipi_init(void)
-{
-}
-#endif
-
 static void __init gic_basic_init(void)
 {
 	unsigned int i;
@@ -979,8 +902,6 @@ static void __init __gic_init(unsigned long gic_base_addr,
 	bitmap_set(ipi_resrv, gic_shared_intrs - 2 * NR_CPUS, 2 * NR_CPUS);
 
 	gic_basic_init();
-
-	gic_ipi_init();
 }
 
 void __init gic_init(unsigned long gic_base_addr,
diff --git a/include/linux/irqchip/mips-gic.h b/include/linux/irqchip/mips-gic.h
index 4e6861605050..321278767506 100644
--- a/include/linux/irqchip/mips-gic.h
+++ b/include/linux/irqchip/mips-gic.h
@@ -258,9 +258,6 @@ extern void gic_write_compare(cycle_t cnt);
 extern void gic_write_cpu_compare(cycle_t cnt, int cpu);
 extern void gic_start_count(void);
 extern void gic_stop_count(void);
-extern void gic_send_ipi(unsigned int intr);
-extern unsigned int plat_ipi_call_int_xlate(unsigned int);
-extern unsigned int plat_ipi_resched_int_xlate(unsigned int);
 extern int gic_get_c0_compare_int(void);
 extern int gic_get_c0_perfcount_int(void);
 extern int gic_get_c0_fdc_int(void);
-- 
2.1.0

Re: [RFC v2 PATCH 00/14] Implement generic IPI support mechanism

[Thomas Gleixner]

On Tue, 13 Oct 2015, Qais Yousef wrote:

> This series is based on Linus tree. I couldn't compile test it
> because MIPS compilation was broken due to other reasons. I expect
> some brokeness because of the introduction of struct irq_common_data
> which is not present on the 4.1 tree I was testing my code on before
> porting it to Linus tip. I will fix these issues and introduce
> proper accessors for accessing struct ipi_mask given that the
> concept is approved.

Please base it on 4.1-rc5 + irq/core.
 
>   irq: add new IRQ_DOMAIN_FLAGS_IPI

The proper prefix for the core parts is 'genirq:'. Please start the
sentence after the prefix with an uppercase letter

>   irq: add GENERIC_IRQ_IPI Kconfig symbol
>   irq: add new struct ipi_mask
>   irq: add a new irq_send_ipi() to irq_chip
>   irq: add struct ipi_mask to irq_data
>   irq: add struct ipi_mapping and its helper functions
>   irq: add a new generic IPI reservation code to irq core
>   irq: implement irq_send_ipi
>   MIPS: add support for generic SMP IPI support
>   MIPS: make smp CMP, CPS and MT use the new generic IPI functions
>   MIPS: delete smp-gic.c
>   irqchip: mips-gic: add a IPI hierarchy domain

Please make that

irqchip/mips-gic: Add ....

Thanks,

	tglx

Re: [RFC v2 PATCH 00/14] Implement generic IPI support mechanism

[Qais Yousef]

On 10/13/2015 02:53 PM, Thomas Gleixner wrote:
> On Tue, 13 Oct 2015, Qais Yousef wrote:
>
>> This series is based on Linus tree. I couldn't compile test it
>> because MIPS compilation was broken due to other reasons. I expect
>> some brokeness because of the introduction of struct irq_common_data
>> which is not present on the 4.1 tree I was testing my code on before
>> porting it to Linus tip. I will fix these issues and introduce
>> proper accessors for accessing struct ipi_mask given that the
>> concept is approved.
> Please base it on 4.1-rc5 + irq/core.
>   
>>    irq: add new IRQ_DOMAIN_FLAGS_IPI
> The proper prefix for the core parts is 'genirq:'. Please start the
> sentence after the prefix with an uppercase letter
>
>>    irq: add GENERIC_IRQ_IPI Kconfig symbol
>>    irq: add new struct ipi_mask
>>    irq: add a new irq_send_ipi() to irq_chip
>>    irq: add struct ipi_mask to irq_data
>>    irq: add struct ipi_mapping and its helper functions
>>    irq: add a new generic IPI reservation code to irq core
>>    irq: implement irq_send_ipi
>>    MIPS: add support for generic SMP IPI support
>>    MIPS: make smp CMP, CPS and MT use the new generic IPI functions
>>    MIPS: delete smp-gic.c
>>    irqchip: mips-gic: add a IPI hierarchy domain
> Please make that
>
> irqchip/mips-gic: Add ....
>
>


Will do. Thanks a lot for the review and all the pointers. I need to 
revive the DT binding discussion now, in the proper list this time.

Thanks,
Qais

Re: [RFC v2 PATCH 00/14] Implement generic IPI support mechanism

[Sergei Shtylyov]

Hello.

On 10/13/2015 04:53 PM, Thomas Gleixner wrote:

>> This series is based on Linus tree. I couldn't compile test it
>> because MIPS compilation was broken due to other reasons. I expect
>> some brokeness because of the introduction of struct irq_common_data
>> which is not present on the 4.1 tree I was testing my code on before
>> porting it to Linus tip. I will fix these issues and introduce
>> proper accessors for accessing struct ipi_mask given that the
>> concept is approved.
>
> Please base it on 4.1-rc5 + irq/core.

    On 4.3-rc5, you mean?

MBR, Sergei

Re: [RFC v2 PATCH 00/14] Implement generic IPI support mechanism

[Thomas Gleixner]

On Tue, 13 Oct 2015, Sergei Shtylyov wrote:
> On 10/13/2015 04:53 PM, Thomas Gleixner wrote:
> > 
> > Please base it on 4.1-rc5 + irq/core.
> 
>    On 4.3-rc5, you mean?

Indeed!

Re: [RFC v2 PATCH 00/14] Implement generic IPI support mechanism

[Davidlohr Bueso]

On Tue, 13 Oct 2015, Qais Yousef wrote:

>Qais Yousef (14):
>  irq: add new IRQ_DOMAIN_FLAGS_IPI
>  irq: add GENERIC_IRQ_IPI Kconfig symbol
>  irq: add new struct ipi_mask
>  irq: add a new irq_send_ipi() to irq_chip
>  irq: add struct ipi_mask to irq_data
>  irq: add struct ipi_mapping and its helper functions
>  irq: add a new generic IPI reservation code to irq core
>  irq: implement irq_send_ipi
>  MIPS: add support for generic SMP IPI support
>  MIPS: make smp CMP, CPS and MT use the new generic IPI functions
>  MIPS: delete smp-gic.c
>  irqchip: mips-gic: add a IPI hierarchy domain
>  irqchip: mips-gic: implement the new irq_send_ipi
>  irqchip: mips-gic: remove IPI init code
>
> arch/mips/Kconfig                |   6 --
> arch/mips/include/asm/smp-ops.h  |   5 +-
> arch/mips/kernel/Makefile        |   1 -
> arch/mips/kernel/smp-cmp.c       |   4 +-
> arch/mips/kernel/smp-cps.c       |   4 +-
> arch/mips/kernel/smp-gic.c       |  64 -----------
> arch/mips/kernel/smp-mt.c        |   2 +-
> arch/mips/kernel/smp.c           | 117 ++++++++++++++++++++
> drivers/irqchip/Kconfig          |   2 +
> drivers/irqchip/irq-mips-gic.c   | 225 ++++++++++++++++++++++++---------------
> include/linux/irq.h              |  43 ++++++++
> include/linux/irqchip/mips-gic.h |   3 -
> include/linux/irqdomain.h        |  19 ++++
> kernel/irq/Kconfig               |   4 +
> kernel/irq/irqdomain.c           |  84 +++++++++++++++
> kernel/irq/manage.c              | 103 ++++++++++++++++++
> 16 files changed, 517 insertions(+), 169 deletions(-)
> delete mode 100644 arch/mips/kernel/smp-gic.c

It strikes me that Documentation/ should at least get _some_ love. Perhaps IRQ-ipi.txt? I dunno...

Thanks,
Davidlohr

Re: [RFC v2 PATCH 00/14] Implement generic IPI support mechanism

[Qais Yousef]

On 10/14/2015 04:04 PM, Davidlohr Bueso wrote:
> On Tue, 13 Oct 2015, Qais Yousef wrote:
>
>> Qais Yousef (14):
>>  irq: add new IRQ_DOMAIN_FLAGS_IPI
>>  irq: add GENERIC_IRQ_IPI Kconfig symbol
>>  irq: add new struct ipi_mask
>>  irq: add a new irq_send_ipi() to irq_chip
>>  irq: add struct ipi_mask to irq_data
>>  irq: add struct ipi_mapping and its helper functions
>>  irq: add a new generic IPI reservation code to irq core
>>  irq: implement irq_send_ipi
>>  MIPS: add support for generic SMP IPI support
>>  MIPS: make smp CMP, CPS and MT use the new generic IPI functions
>>  MIPS: delete smp-gic.c
>>  irqchip: mips-gic: add a IPI hierarchy domain
>>  irqchip: mips-gic: implement the new irq_send_ipi
>>  irqchip: mips-gic: remove IPI init code
>>
>> arch/mips/Kconfig                |   6 --
>> arch/mips/include/asm/smp-ops.h  |   5 +-
>> arch/mips/kernel/Makefile        |   1 -
>> arch/mips/kernel/smp-cmp.c       |   4 +-
>> arch/mips/kernel/smp-cps.c       |   4 +-
>> arch/mips/kernel/smp-gic.c       |  64 -----------
>> arch/mips/kernel/smp-mt.c        |   2 +-
>> arch/mips/kernel/smp.c           | 117 ++++++++++++++++++++
>> drivers/irqchip/Kconfig          |   2 +
>> drivers/irqchip/irq-mips-gic.c   | 225 
>> ++++++++++++++++++++++++---------------
>> include/linux/irq.h              |  43 ++++++++
>> include/linux/irqchip/mips-gic.h |   3 -
>> include/linux/irqdomain.h        |  19 ++++
>> kernel/irq/Kconfig               |   4 +
>> kernel/irq/irqdomain.c           |  84 +++++++++++++++
>> kernel/irq/manage.c              | 103 ++++++++++++++++++
>> 16 files changed, 517 insertions(+), 169 deletions(-)
>> delete mode 100644 arch/mips/kernel/smp-gic.c
>
> It strikes me that Documentation/ should at least get _some_ love. 
> Perhaps IRQ-ipi.txt? I dunno...

Since this was an RFC I didn't update documentation without first making 
sure the changes are OK. In the next series I'll add the documentation 
changes.

Thanks for pointing it out though. I could have missed it in the next 
series to be honest as I'm getting more focused on the small details :)

Thanks,
Qais