[PATCH v3 0/6] genirq: irqdomain_hierarchy: fixes

[PATCH v3 0/6] genirq: irqdomain_hierarchy: fixes

[Grygorii Strashko]

Hi All,

I've had able to identify and reproduce four issues related to switching on
using IRQ domain hierarchy on TI OMAP DRA7 (dra7-evm). Most of them were
discovered during testing of Suspend to RAM and IRQ wakeup functionality.
In my opinion, most of these issue could also affect on other ARM SoC if
they are using IRQ domain hierarchy.

In case of TI OMAP DRA7 the following IRQ hierarchy is defined:
 ARM GIC <- OMAP wakeupgen <- TI CBAR

Issue 1 - patch 1: IRQ re-triggering is not working if IRQ domain
hierarchy used in combination with ARM GIC and irq_chip_retrigger_hierarchy()
is used by child IRQ domain.

Issue 2 - patches 2-4: ARM GIC IRQ triggering type is not configured
properly when IRQ is routed through IRQ domain hierarchy and
system started using DT. As result, system will start using default
ARM GIC configuration, ignore DT IRQ triggering configuration,
and value of desc->irq_data.state_use_accessors = 0.

/ # cat /proc/interrupts -- before
230:          0          0      CBAR  30 Edge    -00000000  48051000.gpio
263:          0          0      CBAR 116 Edge    -00000000  48053000.gpio
296:        246          0      CBAR  67 Edge    -00000000  OMAP UART0
311:        341          0      CBAR  51 Edge    -00000000  48070000.i2c
312:          3          0      CBAR  52 Edge    -00000000  48072000.i2c
313:          0          0      CBAR  56 Edge    -00000000  48060000.i2c
314:        255          0      CBAR  78 Edge    -00000000  mmc0
315:         40          0      CBAR  81 Edge    -00000000  mmc1
316:         13          0      CBAR  91 Edge    -00000000  mmc2
320:          0          0      CBAR  49 Edge    -00000000  4a140000.sata
324:          0          0      CBAR   1 Edge    -00000000  48078000.elm
325:          0          0      CBAR  15 Edge    -00000000  gpmc
333:          0          0      CBAR 335 Edge    -00000000  48484000.ethernet
334:          0          0      CBAR 336 Edge    -00000000  48484000.ethernet
415:          0          0   pcf857x   2 Edge    -00000000  btnUser1
416:          0          0   pcf857x   3 Edge    -00000000  btnUser2
445:          0          0   pinctrl 992 Edge    -00010000  4806a000.seria

/ # cat /proc/interrupts -- after
230:          0          0      CBAR  30 Level   -00002004  48051000.gpio
263:          0          0      CBAR 116 Level   -00002004  48053000.gpio
296:        567          0      CBAR  67 Level   -00002004  OMAP UART0
311:        363          0      CBAR  51 Level   -00002004  48070000.i2c
312:          3          0      CBAR  52 Level   -00002004  48072000.i2c
313:          0          0      CBAR  56 Level   -00002004  48060000.i2c
314:        197          0      CBAR  78 Level   -00002004  mmc0
315:         40          0      CBAR  81 Level   -00002004  mmc1
316:         13          0      CBAR  91 Level   -00002004  mmc2
320:          0          0      CBAR  49 Level   -00002004  4a140000.sata
324:          0          0      CBAR   1 Level   -00002004  48078000.elm
325:          0          0      CBAR  15 Level   -00002004  gpmc
333:          0          0      CBAR 335 Level   -00002004  48484000.ethernet
334:          0          0      CBAR 336 Level   -00002004  48484000.ethernet
415:          0          0   pcf857x   2 Edge    -00000000  btnUser1
416:          4          2   pcf857x   3 Edge    -00000000  btnUser2
445:          0          0   pinctrl 992 Edge    -00010000  4806a000.serial

-xxxxxxxx are current values of desc->irq_data.state_use_accessors

Issue 3 - patch 5: ARM GIC IRQs are not masked during suspend even
if they are not wakeup source. 
	This issue related to IRQCHIP MASK ON SUSPEND flag usage and,
honestly, it's not clear how this flag has to be processed in
case of IRQ domain hierarchy - previously there was one IRQ chip per IRQ
desc and if it has IRQCHIP MASK ON SUSPEND flag set we had to mask IRQ.
But now there are up to three (DRA7) IRQ chip per IRQ desc, and all of them
could have different IRQCHIP MASK ON SUSPEND flag's configuration.
	Here, to fix an issue, I've just added IRQCHIP MASK ON SUSPEND flag
to TI Crossbar which is the last IRQ domain in hierarchy.

Issue 4 - patch 6: On TI dra7-evm, when irq_set_irq_type() is called for
IRQ which belongs to IRQ domain hierarchy the .irq_set_wake() callback
will be executed for TI Crossbar IRQ chip. The TI Crossbar doesn't provides
any facility to configure the wakeup sources, but it implements
.irq_set_wake() callback as:
	.irq_set_wake		= irq_chip_set_wake_parent
As result, the irq_chip_set_wake_parent() will try to execute
.irq_set_wake() callback for parent IRQ domain, which is TI OMAP wakeupgen,
but TI OMAP wakeupgen has IRQCHIP_SKIP_SET_WAKE flag set and do not
implement .irq_set_wake() callback. Thus, irq_chip_set_wake_parent() will
fail with -ENOSYS, but it shouldn't.

IRQ domain hierarchy code was merged in 4.1, so it seems these patches
are material for stable 4.1+.

Tested on:
 TI OMAP DRA7, dra7-evm

Series based on:
 Linux 4.2-rc6

Changes in v3:
 - copy-past issue fixed in patch 'genirq: introduce
   irq_chip_set_type_parent() helper'
 - dropped patch 'genirq: fix irqchip_set_wake_parent if
   IRQCHIP_SKIP_SET_WAKE'
 - added patch 'irqchip: crossbar: fix set_wake functionality'

Changes in v2:
- initial patch has been split on three;
- added three more patches.

Links:
 v2: https://lkml.org/lkml/2015/8/12/759
 v1: https://lkml.org/lkml/2015/8/11/179

Cc: Sudeep Holla <sudeep.holla@arm.com>
Cc: Jiang Liu <jiang.liu@linux.intel.com>
Grygorii Strashko (6):
  genirq: fix irq_chip_retrigger_hierarchy
  genirq: introduce irq_chip_set_type_parent() helper
  irqchip: crossbar: fix arm gic irq type configuration
  ARM: OMAP: wakeupgen: fix arm gic irq type configuration
  irqchip: crossbar: fix irq masking at suspend
  irqchip: crossbar: fix set_wake functionality

 arch/arm/mach-omap2/omap-wakeupgen.c |  1 +
 drivers/irqchip/irq-crossbar.c       |  4 +++-
 include/linux/irq.h                  |  1 +
 kernel/irq/chip.c                    | 19 ++++++++++++++++++-
 4 files changed, 23 insertions(+), 2 deletions(-)

-- 
2.5.0

[PATCH v3 1/6] genirq: fix irq_chip_retrigger_hierarchy

[Grygorii Strashko]

Now irq_chip_retrigger_hierarchy() returns -ENOSYS if it
was not able to find at least one .irq_retrigger() callback
implemented in IRQ domain hierarchy. As result, IRQ
re-triggering is not working now on ARM (TI OMAP) where
ARM GIC is not implemented this callback.
The .irq_retrigger() is optional (see check_irq_resend())
and there are no reasons to fail if it was not found, hence
lets return 0 in this case.

In case of TI OMAP DRA7 the following IRQ hierarchy is defined:
ARM GIC <- OMAP wakeupgen <- TI CBAR

Failure is reproduced during resume from suspend to RAM:
- wakeup by IRQx
- suspend_enter
  + arch_suspend_enable_irqs
    + handle_fasteoi_irq
      + irq_may_run
        + irq_pm_check_wakeup
          + irq_disable(IRQx)
  + dpm_resume_noirq()
    + resume_device_irqs
      + resume_irqs
        + resume_irq
          + __enable_irq <== IRQx is not re-triggered

Cc: Sudeep Holla <sudeep.holla@arm.com>
Cc: Jiang Liu <jiang.liu@linux.intel.com>
Fixes: 85f08c17de26 ('genirq: Introduce helper functions...')
Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com>
---
 kernel/irq/chip.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/irq/chip.c b/kernel/irq/chip.c
index 27f4332..6de638b 100644
--- a/kernel/irq/chip.c
+++ b/kernel/irq/chip.c
@@ -997,7 +997,7 @@ int irq_chip_retrigger_hierarchy(struct irq_data *data)
 		if (data->chip && data->chip->irq_retrigger)
 			return data->chip->irq_retrigger(data);
 
-	return -ENOSYS;
+	return 0;
 }
 
 /**
-- 
2.5.0

[PATCH v3 2/6] genirq: introduce irq_chip_set_type_parent() helper

[Grygorii Strashko]

It's expected to use this helper when the current
domain doesn't implement .irq_set_type(),  but expect
the parent to do so.

Cc: Sudeep Holla <sudeep.holla@arm.com>
Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com>
---
 include/linux/irq.h |  1 +
 kernel/irq/chip.c   | 17 +++++++++++++++++
 2 files changed, 18 insertions(+)

diff --git a/include/linux/irq.h b/include/linux/irq.h
index 92188b0..51744bc 100644
--- a/include/linux/irq.h
+++ b/include/linux/irq.h
@@ -484,6 +484,7 @@ extern int irq_chip_set_affinity_parent(struct irq_data *data,
 extern int irq_chip_set_wake_parent(struct irq_data *data, unsigned int on);
 extern int irq_chip_set_vcpu_affinity_parent(struct irq_data *data,
 					     void *vcpu_info);
+extern int irq_chip_set_type_parent(struct irq_data *data, unsigned int type);
 #endif
 
 /* Handling of unhandled and spurious interrupts: */
diff --git a/kernel/irq/chip.c b/kernel/irq/chip.c
index 6de638b..ae21682 100644
--- a/kernel/irq/chip.c
+++ b/kernel/irq/chip.c
@@ -985,6 +985,23 @@ int irq_chip_set_affinity_parent(struct irq_data *data,
 }
 
 /**
+ * irq_chip_set_type_parent - Set IRQ type on the parent interrupt
+ * @data:	Pointer to interrupt specific data
+ * @type:	IRQ_TYPE_{LEVEL,EDGE}_* value - see include/linux/irq.h
+ *
+ * Conditional, as the underlying parent chip might not implement it.
+ */
+int irq_chip_set_type_parent(struct irq_data *data, unsigned int type)
+{
+	data = data->parent_data;
+
+	if (data->chip->irq_set_type)
+		return data->chip->irq_set_type(data, type);
+
+	return -ENOSYS;
+}
+
+/**
  * irq_chip_retrigger_hierarchy - Retrigger an interrupt in hardware
  * @data:	Pointer to interrupt specific data
  *
-- 
2.5.0

[PATCH v3 3/6] irqchip: crossbar: fix arm gic irq type configuration

[Grygorii Strashko]

It's observed that ARM GIC IRQ triggering type is not configured
properly when IRQ is routed through IRQ domain hierarchy and
system started using DT. As result, system will start using default
ARM GIC configuration, ignore DT IRQ triggering configuration,
and value of desc->irq_data.state_use_accessors = 0.

In case of TI OMAP DRA7 the following IRQ hierarchy is defined:
ARM GIC <- OMAP wakeupgen <- TI CBAR

Failed call chain:
     irq_create_of_mapping
     irq_set_irq_type
     __irq_set_trigger
     if (!chip || !chip->irq_set_type) {
        return 0; <- return here
     }
Crossbar has no .irq_set_type() defined and, so, IRQ triggering
configuration will not be propagated to parent IRQ domain.

Hence, fix it by using irq_chip_set_type_parent() for
propagation IRQ triggering type to parent IRQ domains.

Cc: Sudeep Holla <sudeep.holla@arm.com>
Fixes: 783d31863fb8 ('irqchip: crossbar: Convert dra7 crossbar...')
Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com>
---
 drivers/irqchip/irq-crossbar.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/irqchip/irq-crossbar.c b/drivers/irqchip/irq-crossbar.c
index 692fe2b..3ba58e7 100644
--- a/drivers/irqchip/irq-crossbar.c
+++ b/drivers/irqchip/irq-crossbar.c
@@ -69,6 +69,7 @@ static struct irq_chip crossbar_chip = {
 	.irq_unmask		= irq_chip_unmask_parent,
 	.irq_retrigger		= irq_chip_retrigger_hierarchy,
 	.irq_set_wake		= irq_chip_set_wake_parent,
+	.irq_set_type		= irq_chip_set_type_parent,
 #ifdef CONFIG_SMP
 	.irq_set_affinity	= irq_chip_set_affinity_parent,
 #endif
-- 
2.5.0

[PATCH v3 4/6] ARM: OMAP: wakeupgen: fix arm gic irq type configuration

[Grygorii Strashko]

It's observed that ARM GIC IRQ triggering type is not configured
properly when IRQ is routed through IRQ domain hierarchy and
system started using DT. As result, system will start using default
ARM GIC configuration, ignore DT IRQ triggering configuration,
and value of desc->irq_data.state_use_accessors = 0.

In case of TI OMAP DRA7 the following IRQ hierarchy is defined:
ARM GIC <- OMAP wakeupgen <- TI CBAR

Failed call chain:
     irq_create_of_mapping
     irq_set_irq_type
     __irq_set_trigger
     if (!chip || !chip->irq_set_type) {
        return 0; <- return here
     }
OMAP wakeupgen has no .irq_set_type() defined and, so, IRQ triggering
configuration will not be propagated to parent IRQ domain.

Hence, fix it by using irq_chip_set_type_parent() for
propagation IRQ triggering type to parent IRQ domains.

Cc: Sudeep Holla <sudeep.holla@arm.com>
Fixes: 7136d457f365 ('ARM: omap: convert wakeupgen to stacked domains')
Acked-by: Tony Lindgren <tony@atomide.com>
Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com>
---
 arch/arm/mach-omap2/omap-wakeupgen.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm/mach-omap2/omap-wakeupgen.c b/arch/arm/mach-omap2/omap-wakeupgen.c
index 8e52621..e1d2e99 100644
--- a/arch/arm/mach-omap2/omap-wakeupgen.c
+++ b/arch/arm/mach-omap2/omap-wakeupgen.c
@@ -392,6 +392,7 @@ static struct irq_chip wakeupgen_chip = {
 	.irq_mask		= wakeupgen_mask,
 	.irq_unmask		= wakeupgen_unmask,
 	.irq_retrigger		= irq_chip_retrigger_hierarchy,
+	.irq_set_type		= irq_chip_set_type_parent,
 	.flags			= IRQCHIP_SKIP_SET_WAKE | IRQCHIP_MASK_ON_SUSPEND,
 #ifdef CONFIG_SMP
 	.irq_set_affinity	= irq_chip_set_affinity_parent,
-- 
2.5.0

[PATCH v3 5/6] irqchip: crossbar: fix irq masking at suspend

[Grygorii Strashko]

All ARM GIC IRQs have to masked during suspend if they are not
wakeup source - this is expected behavior.Now this is not happen, since
switching to use IRQ domain hierarchy, because suspend_device_irq()
only checks flags in the last IRQ chip in hierarchy for
IRQCHIP_MASK_ON_SUSPEND bit set. And in the case of TI OMAP DRA7 the
last IRQ chip is TI Crossbar which do not have this flag set.

In case of TI OMAP DRA7 the following IRQ hierarchy is defined:
  ARM GIC <- OMAP wakeupgen <- TI CBAR
ARM GIC - IRQCHIP_MASK_ON_SUSPEND=n
OMAP wakeupgen - IRQCHIP_MASK_ON_SUSPEND=y
TI CBAR - IRQCHIP_MASK_ON_SUSPEND=n

Hence, fix by adding IRQCHIP_MASK_ON_SUSPEND for TI Crossbar IRQ chip.

Cc: Sudeep Holla <sudeep.holla@arm.com>
Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com>
---
 drivers/irqchip/irq-crossbar.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/irqchip/irq-crossbar.c b/drivers/irqchip/irq-crossbar.c
index 3ba58e7..f5a72cc 100644
--- a/drivers/irqchip/irq-crossbar.c
+++ b/drivers/irqchip/irq-crossbar.c
@@ -70,6 +70,7 @@ static struct irq_chip crossbar_chip = {
 	.irq_retrigger		= irq_chip_retrigger_hierarchy,
 	.irq_set_wake		= irq_chip_set_wake_parent,
 	.irq_set_type		= irq_chip_set_type_parent,
+	.flags			= IRQCHIP_MASK_ON_SUSPEND,
 #ifdef CONFIG_SMP
 	.irq_set_affinity	= irq_chip_set_affinity_parent,
 #endif
-- 
2.5.0

[PATCH v3 6/6] irqchip: crossbar: fix set_wake functionality

[Grygorii Strashko]

The TI crossbar doesn't provides any facility to configure the wakeup
sources, but it implements .irq_set_wake() callback:
	.irq_set_wake		= irq_chip_set_wake_parent

As result, the irq_chip_set_wake_parent() will try to execute
.irq_set_wake() callback for parent IRQ domain, which is TI OMAP wakeupgen,
but TI OMAP wakeupgen has IRQCHIP_SKIP_SET_WAKE flag set and do not
implement .irq_set_wake() callback. Thus, irq_chip_set_wake_parent() will
fail with -ENOSYS.

In case of TI OMAP DRA7 the issue reproduced with following
configuration:
   ARM GIC<-OMAP wakeupgen<-TI CBAR<-GPIO<-GPIO pcf857x<-gpio_key

gpio_key is wakeup source

Failure is reproduced during suspend/resume to RAM:
 suspend:
  - gpio_keys_suspend
    + enable_irq_wake
      + pcf857x_irq_set_wake
        + omap_gpio_wake_enable
          + TI CBAR irq_chip_set_wake_parent
            + OMAP wakeupgen has no .irq_set_wake() and
              -ENOSYS will be returned

 resume:
  - gpio_keys_resume
    + disable_irq_wake
      + irq_set_irq_wake
         + WARN(1, "Unbalanced IRQ %d wake disable\n", irq);

Hence, fix it by adding IRQCHIP_SKIP_SET_WAKE flag to TI Crossbar and
drop .irq_set_wake() at the same time.

Cc: Sudeep Holla <sudeep.holla@arm.com>
Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com>
---
 drivers/irqchip/irq-crossbar.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/irqchip/irq-crossbar.c b/drivers/irqchip/irq-crossbar.c
index f5a72cc..c12bb93 100644
--- a/drivers/irqchip/irq-crossbar.c
+++ b/drivers/irqchip/irq-crossbar.c
@@ -68,9 +68,9 @@ static struct irq_chip crossbar_chip = {
 	.irq_mask		= irq_chip_mask_parent,
 	.irq_unmask		= irq_chip_unmask_parent,
 	.irq_retrigger		= irq_chip_retrigger_hierarchy,
-	.irq_set_wake		= irq_chip_set_wake_parent,
 	.irq_set_type		= irq_chip_set_type_parent,
-	.flags			= IRQCHIP_MASK_ON_SUSPEND,
+	.flags			= IRQCHIP_MASK_ON_SUSPEND |
+				  IRQCHIP_SKIP_SET_WAKE,
 #ifdef CONFIG_SMP
 	.irq_set_affinity	= irq_chip_set_affinity_parent,
 #endif
-- 
2.5.0

[PATCH v3 1/6] genirq: fix irq_chip_retrigger_hierarchy

[Jiang Liu]

On 2015/8/14 20:20, Grygorii Strashko wrote:
> Now irq_chip_retrigger_hierarchy() returns -ENOSYS if it
> was not able to find at least one .irq_retrigger() callback
> implemented in IRQ domain hierarchy. As result, IRQ
> re-triggering is not working now on ARM (TI OMAP) where
> ARM GIC is not implemented this callback.
> The .irq_retrigger() is optional (see check_irq_resend())
> and there are no reasons to fail if it was not found, hence
> lets return 0 in this case.
> 
> In case of TI OMAP DRA7 the following IRQ hierarchy is defined:
> ARM GIC <- OMAP wakeupgen <- TI CBAR
> 
> Failure is reproduced during resume from suspend to RAM:
> - wakeup by IRQx
> - suspend_enter
>   + arch_suspend_enable_irqs
>     + handle_fasteoi_irq
>       + irq_may_run
>         + irq_pm_check_wakeup
>           + irq_disable(IRQx)
>   + dpm_resume_noirq()
>     + resume_device_irqs
>       + resume_irqs
>         + resume_irq
>           + __enable_irq <== IRQx is not re-triggered
> 
> Cc: Sudeep Holla <sudeep.holla@arm.com>
> Cc: Jiang Liu <jiang.liu@linux.intel.com>
> Fixes: 85f08c17de26 ('genirq: Introduce helper functions...')
> Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>
> Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com>
> ---
>  kernel/irq/chip.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/kernel/irq/chip.c b/kernel/irq/chip.c
> index 27f4332..6de638b 100644
> --- a/kernel/irq/chip.c
> +++ b/kernel/irq/chip.c
> @@ -997,7 +997,7 @@ int irq_chip_retrigger_hierarchy(struct irq_data *data)
>  		if (data->chip && data->chip->irq_retrigger)
>  			return data->chip->irq_retrigger(data);
>  
> -	return -ENOSYS;
> +	return 0;
>  }
>  
>  /**

Hi Grygorii,
	Thanks for fixing this regression:)
Reviewed-by: Jiang Liu <jiang.liu@linux.intel.com>