[PATCH 0/4] armada-370-xp irqchip updates

[PATCH 0/4] armada-370-xp irqchip updates

[Marek Behún]

Hi Thomas, Andrew, Andy, et al.

this is a resurrection of an old series of changes for the armada-370-xp
irqchip.

I have refactored these changes and updated commit messages.

The first three patches are small "cleanups".
The 4th is a little larger, it increases the number of available MSI
interrupt lines from 16 to 32, on platforms where the MPIC is not used
for IPI interrupts.

Note that this driver is in need of a major refactor, to bring it to
modern standards, but that is unfortunately currently infeasible with my
time constraints.

Marek

Pali Rohár (4):
  irqchip/armada-370-xp: Do not allow mapping IRQ 0 and 1
  irqchip/armada-370-xp: Exit ipi_resume() early if IPI is not used
  irqchip/armada-370-xp: Do not touch IPI registers on platforms without
    IPI
  irqchip/armada-370-xp: Add support for 32 MSI interrupts on non-IPI
    platforms

 drivers/irqchip/irq-armada-370-xp.c | 101 +++++++++++++++++++++++-----
 1 file changed, 86 insertions(+), 15 deletions(-)

-- 
2.44.2

[PATCH 1/4] irqchip/armada-370-xp: Do not allow mapping IRQ 0 and 1

[Marek Behún]

From: Pali Rohár <pali@kernel.org>

IRQs 0 (IPI) and 1 (MSI) are handled internally by this driver,
generic_handle_domain_irq() is never called for these IRQs.

Disallow mapping these IRQs.

Fixes: 344e873e5657 ("arm: mvebu: Add IPI support via doorbells")
Fixes: 31f614edb726 ("irqchip: armada-370-xp: implement MSI support")
Signed-off-by: Pali Rohár <pali@kernel.org>
[ changed commit message, added Fixes tags ]
Signed-off-by: Marek Behún <kabel@kernel.org>
---
 drivers/irqchip/irq-armada-370-xp.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/irqchip/irq-armada-370-xp.c b/drivers/irqchip/irq-armada-370-xp.c
index 4b021a67bdfe..f488c35d9130 100644
--- a/drivers/irqchip/irq-armada-370-xp.c
+++ b/drivers/irqchip/irq-armada-370-xp.c
@@ -566,6 +566,10 @@ static struct irq_chip armada_370_xp_irq_chip = {
 static int armada_370_xp_mpic_irq_map(struct irq_domain *h,
 				      unsigned int virq, irq_hw_number_t hw)
 {
+	/* IRQs 0 and 1 cannot be mapped, they are handled internally */
+	if (hw <= 1)
+		return -EINVAL;
+
 	armada_370_xp_irq_mask(irq_get_irq_data(virq));
 	if (!is_percpu_irq(hw))
 		writel(hw, per_cpu_int_base +
-- 
2.44.2

Re: [PATCH 1/4] irqchip/armada-370-xp: Do not allow mapping IRQ 0 and 1

[Andrew Lunn]

On Wed, Jun 19, 2024 at 04:11:31PM +0200, Marek Behún wrote:
> From: Pali Rohár <pali@kernel.org>
> 
> IRQs 0 (IPI) and 1 (MSI) are handled internally by this driver,
> generic_handle_domain_irq() is never called for these IRQs.
> 
> Disallow mapping these IRQs.
> 
> Fixes: 344e873e5657 ("arm: mvebu: Add IPI support via doorbells")
> Fixes: 31f614edb726 ("irqchip: armada-370-xp: implement MSI support")
> Signed-off-by: Pali Rohár <pali@kernel.org>
> [ changed commit message, added Fixes tags ]
> Signed-off-by: Marek Behún <kabel@kernel.org>

Is this needed in stable? Is the problem being fixed a real bug that
bothers people?

https://www.kernel.org/doc/html/latest/process/stable-kernel-rules.html

If you would like this back ported in stable, please post it as a
separate patch. Otherwise it can be kept as part of the series.  The
Fixes: might result in it being backported anyway.

	Andrew

Re: [PATCH 1/4] irqchip/armada-370-xp: Do not allow mapping IRQ 0 and 1

[Marek Behún]

On Wed, 19 Jun 2024 19:08:04 +0200
Andrew Lunn <andrew@lunn.ch> wrote:

> On Wed, Jun 19, 2024 at 04:11:31PM +0200, Marek Behún wrote:
> > From: Pali Rohár <pali@kernel.org>
> > 
> > IRQs 0 (IPI) and 1 (MSI) are handled internally by this driver,
> > generic_handle_domain_irq() is never called for these IRQs.
> > 
> > Disallow mapping these IRQs.
> > 
> > Fixes: 344e873e5657 ("arm: mvebu: Add IPI support via doorbells")
> > Fixes: 31f614edb726 ("irqchip: armada-370-xp: implement MSI support")
> > Signed-off-by: Pali Rohár <pali@kernel.org>
> > [ changed commit message, added Fixes tags ]
> > Signed-off-by: Marek Behún <kabel@kernel.org>  
> 
> Is this needed in stable? Is the problem being fixed a real bug that
> bothers people?

Not as far as I know, but I have been asked to add Fixes tag for less.

> https://www.kernel.org/doc/html/latest/process/stable-kernel-rules.html
> 
> If you would like this back ported in stable, please post it as a
> separate patch. Otherwise it can be kept as part of the series.  The
> Fixes: might result in it being backported anyway.

I don't need it backported. Semantically it is a fix, but it doesn't
bother anyone.

I will post v2 without the Fixes tags.

Marek

[PATCH 2/4] irqchip/armada-370-xp: Exit ipi_resume() early if IPI is not used

[Marek Behún]

From: Pali Rohár <pali@kernel.org>

Exit ipi_resume() early if IPI is not used.

IPI is used only on systems where the mpic controller does not have a
parent GIC IRQ (e.g. on Armada XP).

Signed-off-by: Pali Rohár <pali@kernel.org>
[ changed commit message and moved the code change into ipi_resume() ]
Signed-off-by: Marek Behún <kabel@kernel.org>
---
 drivers/irqchip/irq-armada-370-xp.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/drivers/irqchip/irq-armada-370-xp.c b/drivers/irqchip/irq-armada-370-xp.c
index f488c35d9130..65f21624263e 100644
--- a/drivers/irqchip/irq-armada-370-xp.c
+++ b/drivers/irqchip/irq-armada-370-xp.c
@@ -29,6 +29,7 @@
 #include <linux/slab.h>
 #include <linux/syscore_ops.h>
 #include <linux/msi.h>
+#include <linux/types.h>
 #include <asm/mach/arch.h>
 #include <asm/exception.h>
 #include <asm/smp_plat.h>
@@ -156,6 +157,12 @@ static DEFINE_MUTEX(msi_used_lock);
 static phys_addr_t msi_doorbell_addr;
 #endif
 
+static inline bool is_ipi_available(void)
+{
+	/* IPI is used only if we do not have parent irq */
+	return parent_irq <= 0;
+}
+
 static inline bool is_percpu_irq(irq_hw_number_t irq)
 {
 	if (irq <= ARMADA_370_XP_MAX_PER_CPU_IRQS)
@@ -429,6 +436,9 @@ static void ipi_resume(void)
 {
 	int i;
 
+	if (!is_ipi_available())
+		return;
+
 	for (i = 0; i < IPI_DOORBELL_END; i++) {
 		int irq;
 
-- 
2.44.2

Re: [PATCH 2/4] irqchip/armada-370-xp: Exit ipi_resume() early if IPI is not used

[Andrew Lunn]

On Wed, Jun 19, 2024 at 04:11:32PM +0200, Marek Behún wrote:
> From: Pali Rohár <pali@kernel.org>
> 
> Exit ipi_resume() early if IPI is not used.
> 
> IPI is used only on systems where the mpic controller does not have a
> parent GIC IRQ (e.g. on Armada XP).
> 
> Signed-off-by: Pali Rohár <pali@kernel.org>
> [ changed commit message and moved the code change into ipi_resume() ]
> Signed-off-by: Marek Behún <kabel@kernel.org>
> ---
>  drivers/irqchip/irq-armada-370-xp.c | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/drivers/irqchip/irq-armada-370-xp.c b/drivers/irqchip/irq-armada-370-xp.c
> index f488c35d9130..65f21624263e 100644
> --- a/drivers/irqchip/irq-armada-370-xp.c
> +++ b/drivers/irqchip/irq-armada-370-xp.c
> @@ -29,6 +29,7 @@
>  #include <linux/slab.h>
>  #include <linux/syscore_ops.h>
>  #include <linux/msi.h>
> +#include <linux/types.h>
>  #include <asm/mach/arch.h>
>  #include <asm/exception.h>
>  #include <asm/smp_plat.h>
> @@ -156,6 +157,12 @@ static DEFINE_MUTEX(msi_used_lock);
>  static phys_addr_t msi_doorbell_addr;
>  #endif
>  
> +static inline bool is_ipi_available(void)
> +{
> +	/* IPI is used only if we do not have parent irq */
> +	return parent_irq <= 0;
> +}
> +
>  static inline bool is_percpu_irq(irq_hw_number_t irq)
>  {
>  	if (irq <= ARMADA_370_XP_MAX_PER_CPU_IRQS)
> @@ -429,6 +436,9 @@ static void ipi_resume(void)
>  {
>  	int i;
>  
> +	if (!is_ipi_available())
> +		return;

But of a nitpick, but it seems odd calling ipi_resume() if it does not
exist. I would prefer

       
	if (is_ipi_available())
		ipi_resume();

At the two places ipi_resume() is called.

   Andrew

Re: [PATCH 2/4] irqchip/armada-370-xp: Exit ipi_resume() early if IPI is not used

[Marek Behún]

On Wed, 19 Jun 2024 19:11:44 +0200
Andrew Lunn <andrew@lunn.ch> wrote:

> On Wed, Jun 19, 2024 at 04:11:32PM +0200, Marek Behún wrote:
> > From: Pali Rohár <pali@kernel.org>
> > 
> > Exit ipi_resume() early if IPI is not used.
> > 
> > IPI is used only on systems where the mpic controller does not have a
> > parent GIC IRQ (e.g. on Armada XP).
> > 
> > Signed-off-by: Pali Rohár <pali@kernel.org>
> > [ changed commit message and moved the code change into ipi_resume() ]
> > Signed-off-by: Marek Behún <kabel@kernel.org>
> > ---
> >  drivers/irqchip/irq-armada-370-xp.c | 10 ++++++++++
> >  1 file changed, 10 insertions(+)
> > 
> > diff --git a/drivers/irqchip/irq-armada-370-xp.c b/drivers/irqchip/irq-armada-370-xp.c
> > index f488c35d9130..65f21624263e 100644
> > --- a/drivers/irqchip/irq-armada-370-xp.c
> > +++ b/drivers/irqchip/irq-armada-370-xp.c
> > @@ -29,6 +29,7 @@
> >  #include <linux/slab.h>
> >  #include <linux/syscore_ops.h>
> >  #include <linux/msi.h>
> > +#include <linux/types.h>
> >  #include <asm/mach/arch.h>
> >  #include <asm/exception.h>
> >  #include <asm/smp_plat.h>
> > @@ -156,6 +157,12 @@ static DEFINE_MUTEX(msi_used_lock);
> >  static phys_addr_t msi_doorbell_addr;
> >  #endif
> >  
> > +static inline bool is_ipi_available(void)
> > +{
> > +	/* IPI is used only if we do not have parent irq */
> > +	return parent_irq <= 0;
> > +}
> > +
> >  static inline bool is_percpu_irq(irq_hw_number_t irq)
> >  {
> >  	if (irq <= ARMADA_370_XP_MAX_PER_CPU_IRQS)
> > @@ -429,6 +436,9 @@ static void ipi_resume(void)
> >  {
> >  	int i;
> >  
> > +	if (!is_ipi_available())
> > +		return;  
> 
> But of a nitpick, but it seems odd calling ipi_resume() if it does not
> exist. I would prefer
> 
>        
> 	if (is_ipi_available())
> 		ipi_resume();
> 
> At the two places ipi_resume() is called.

That is how it was in the original patch, just hardcoded (there
was no is_ipi_available() call, instead it was

  /* comment */
  if (parent_irq <= 0)
    ...

I guess I was trying to be smart in deduplicating code, and I got too
far the other way.

I will fix this in v2.

Marek

[PATCH 3/4] irqchip/armada-370-xp: Do not touch IPI registers on platforms without IPI

[Marek Behún]

From: Pali Rohár <pali@kernel.org>

IPI is not available on platforms where MPIC has a parent irq.

On these platforms the IPI registers are used as additional set of MSI
interrupts (currently unused by the driver).

Do not touch these registers if IPI is not available.

Signed-off-by: Pali Rohár <pali@kernel.org>
[ refactored, changed commit message ]
Signed-off-by: Marek Behún <kabel@kernel.org>
---
 drivers/irqchip/irq-armada-370-xp.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/drivers/irqchip/irq-armada-370-xp.c b/drivers/irqchip/irq-armada-370-xp.c
index 65f21624263e..0097f7dd36e7 100644
--- a/drivers/irqchip/irq-armada-370-xp.c
+++ b/drivers/irqchip/irq-armada-370-xp.c
@@ -506,6 +506,9 @@ static void armada_xp_mpic_smp_cpu_init(void)
 	for (i = 0; i < nr_irqs; i++)
 		writel(i, per_cpu_int_base + ARMADA_370_XP_INT_SET_MASK_OFFS);
 
+	if (!is_ipi_available())
+		return;
+
 	/* Disable all IPIs */
 	writel(0, per_cpu_int_base + ARMADA_370_XP_IN_DRBEL_MSK_OFFS);
 
@@ -755,7 +758,7 @@ static void armada_370_xp_mpic_resume(void)
 	/* Reconfigure doorbells for IPIs and MSIs */
 	writel(doorbell_mask_reg,
 	       per_cpu_int_base + ARMADA_370_XP_IN_DRBEL_MSK_OFFS);
-	if (doorbell_mask_reg & IPI_DOORBELL_MASK)
+	if (is_ipi_available() && (doorbell_mask_reg & IPI_DOORBELL_MASK))
 		writel(0, per_cpu_int_base + ARMADA_370_XP_INT_CLEAR_MASK_OFFS);
 	if (doorbell_mask_reg & PCI_MSI_DOORBELL_MASK)
 		writel(1, per_cpu_int_base + ARMADA_370_XP_INT_CLEAR_MASK_OFFS);
@@ -805,13 +808,18 @@ static int __init armada_370_xp_mpic_of_init(struct device_node *node,
 	BUG_ON(!armada_370_xp_mpic_domain);
 	irq_domain_update_bus_token(armada_370_xp_mpic_domain, DOMAIN_BUS_WIRED);
 
+	/*
+	 * Initialize parent_irq before calling any other functions, since it is
+	 * used to distinguish between IPI and non-IPI platforms.
+	 */
+	parent_irq = irq_of_parse_and_map(node, 0);
+
 	/* Setup for the boot CPU */
 	armada_xp_mpic_perf_init();
 	armada_xp_mpic_smp_cpu_init();
 
 	armada_370_xp_msi_init(node, main_int_res.start);
 
-	parent_irq = irq_of_parse_and_map(node, 0);
 	if (parent_irq <= 0) {
 		irq_set_default_host(armada_370_xp_mpic_domain);
 		set_handle_irq(armada_370_xp_handle_irq);
-- 
2.44.2

Re: [PATCH 3/4] irqchip/armada-370-xp: Do not touch IPI registers on platforms without IPI

[Andrew Lunn]

On Wed, Jun 19, 2024 at 04:11:33PM +0200, Marek Behún wrote:
> From: Pali Rohár <pali@kernel.org>
> 
> IPI is not available on platforms where MPIC has a parent irq.
> 
> On these platforms the IPI registers are used as additional set of MSI
> interrupts (currently unused by the driver).
> 
> Do not touch these registers if IPI is not available.
> 
> Signed-off-by: Pali Rohár <pali@kernel.org>
> [ refactored, changed commit message ]
> Signed-off-by: Marek Behún <kabel@kernel.org>

Reviewed-by: Andrew Lunn <andrew@lunn.ch>

    Andrew

[PATCH 4/4] irqchip/armada-370-xp: Add support for 32 MSI interrupts on non-IPI platforms

[Marek Behún]

From: Pali Rohár <pali@kernel.org>

The doorbell interrupts have the following layout on IPI vs no-IPI
platforms:

                    |  0...7  |  8...15  |       16...31       |
  ------------------+---------+----------+---------------------+
       IPI platform |   IPI   |   n/a    |         MSI         |
  ------------------+---------+----------+---------------------+
   non-IPI platform |                   MSI                    |
  ------------------+------------------------------------------+

Currently the driver only allows for the upper 16...31 interrupts for
MSI domain (i.e. the MSI domain has only 16 interrupts).

On platforms where IPI is not available, we can use whole 32 MSI
interrupts.

Implement support also for the lower 16 MSI interrupts on non-IPI
platforms.

Signed-off-by: Pali Rohár <pali@kernel.org>
[ refactored, changed commit message ]
Signed-off-by: Marek Behún <kabel@kernel.org>
---
 drivers/irqchip/irq-armada-370-xp.c | 77 +++++++++++++++++++++++------
 1 file changed, 63 insertions(+), 14 deletions(-)

diff --git a/drivers/irqchip/irq-armada-370-xp.c b/drivers/irqchip/irq-armada-370-xp.c
index 0097f7dd36e7..f87c21cf5ec6 100644
--- a/drivers/irqchip/irq-armada-370-xp.c
+++ b/drivers/irqchip/irq-armada-370-xp.c
@@ -13,6 +13,7 @@
  * warranty of any kind, whether express or implied.
  */
 
+#include <linux/bits.h>
 #include <linux/kernel.h>
 #include <linux/module.h>
 #include <linux/init.h>
@@ -136,6 +137,7 @@
 
 #define ARMADA_370_XP_MAX_PER_CPU_IRQS		(28)
 
+/* IPI and MSI interrupt definitions for IPI platforms */
 #define IPI_DOORBELL_START                      (0)
 #define IPI_DOORBELL_END                        (8)
 #define IPI_DOORBELL_MASK                       0xFF
@@ -144,6 +146,14 @@
 #define PCI_MSI_DOORBELL_END                    (32)
 #define PCI_MSI_DOORBELL_MASK                   0xFFFF0000
 
+/* MSI interrupt definitions for non-IPI platforms */
+#define PCI_MSI_FULL_DOORBELL_START		0
+#define PCI_MSI_FULL_DOORBELL_NR		32
+#define PCI_MSI_FULL_DOORBELL_END		32
+#define PCI_MSI_FULL_DOORBELL_MASK		GENMASK(31, 0)
+#define PCI_MSI_FULL_DOORBELL_SRC0_MASK		GENMASK(15, 0)
+#define PCI_MSI_FULL_DOORBELL_SRC1_MASK		GENMASK(31, 16)
+
 static void __iomem *per_cpu_int_base;
 static void __iomem *main_int_base;
 static struct irq_domain *armada_370_xp_mpic_domain;
@@ -152,7 +162,7 @@ static int parent_irq;
 #ifdef CONFIG_PCI_MSI
 static struct irq_domain *armada_370_xp_msi_domain;
 static struct irq_domain *armada_370_xp_msi_inner_domain;
-static DECLARE_BITMAP(msi_used, PCI_MSI_DOORBELL_NR);
+static DECLARE_BITMAP(msi_used, PCI_MSI_FULL_DOORBELL_NR);
 static DEFINE_MUTEX(msi_used_lock);
 static phys_addr_t msi_doorbell_addr;
 #endif
@@ -163,6 +173,30 @@ static inline bool is_ipi_available(void)
 	return parent_irq <= 0;
 }
 
+static inline u32 msi_doorbell_mask(void)
+{
+	return is_ipi_available() ? PCI_MSI_DOORBELL_MASK :
+				    PCI_MSI_FULL_DOORBELL_MASK;
+}
+
+static inline unsigned int msi_doorbell_start(void)
+{
+	return is_ipi_available() ? PCI_MSI_DOORBELL_START :
+				    PCI_MSI_FULL_DOORBELL_START;
+}
+
+static inline unsigned int msi_doorbell_size(void)
+{
+	return is_ipi_available() ? PCI_MSI_DOORBELL_NR :
+				    PCI_MSI_FULL_DOORBELL_NR;
+}
+
+static inline unsigned int msi_doorbell_end(void)
+{
+	return is_ipi_available() ? PCI_MSI_DOORBELL_END :
+				    PCI_MSI_FULL_DOORBELL_END;
+}
+
 static inline bool is_percpu_irq(irq_hw_number_t irq)
 {
 	if (irq <= ARMADA_370_XP_MAX_PER_CPU_IRQS)
@@ -220,7 +254,7 @@ static void armada_370_xp_compose_msi_msg(struct irq_data *data, struct msi_msg
 
 	msg->address_lo = lower_32_bits(msi_doorbell_addr);
 	msg->address_hi = upper_32_bits(msi_doorbell_addr);
-	msg->data = BIT(cpu + 8) | (data->hwirq + PCI_MSI_DOORBELL_START);
+	msg->data = BIT(cpu + 8) | (data->hwirq + msi_doorbell_start());
 }
 
 static int armada_370_xp_msi_set_affinity(struct irq_data *irq_data,
@@ -253,7 +287,7 @@ static int armada_370_xp_msi_alloc(struct irq_domain *domain, unsigned int virq,
 	int hwirq, i;
 
 	mutex_lock(&msi_used_lock);
-	hwirq = bitmap_find_free_region(msi_used, PCI_MSI_DOORBELL_NR,
+	hwirq = bitmap_find_free_region(msi_used, msi_doorbell_size(),
 					order_base_2(nr_irqs));
 	mutex_unlock(&msi_used_lock);
 
@@ -290,9 +324,10 @@ static void armada_370_xp_msi_reenable_percpu(void)
 	u32 reg;
 
 	/* Enable MSI doorbell mask and combined cpu local interrupt */
-	reg = readl(per_cpu_int_base + ARMADA_370_XP_IN_DRBEL_MSK_OFFS)
-		| PCI_MSI_DOORBELL_MASK;
+	reg = readl(per_cpu_int_base + ARMADA_370_XP_IN_DRBEL_MSK_OFFS);
+	reg |= msi_doorbell_mask();
 	writel(reg, per_cpu_int_base + ARMADA_370_XP_IN_DRBEL_MSK_OFFS);
+
 	/* Unmask local doorbell interrupt */
 	writel(1, per_cpu_int_base + ARMADA_370_XP_INT_CLEAR_MASK_OFFS);
 }
@@ -304,7 +339,7 @@ static int armada_370_xp_msi_init(struct device_node *node,
 		ARMADA_370_XP_SW_TRIG_INT_OFFS;
 
 	armada_370_xp_msi_inner_domain =
-		irq_domain_add_linear(NULL, PCI_MSI_DOORBELL_NR,
+		irq_domain_add_linear(NULL, msi_doorbell_size(),
 				      &armada_370_xp_msi_domain_ops, NULL);
 	if (!armada_370_xp_msi_inner_domain)
 		return -ENOMEM;
@@ -320,6 +355,10 @@ static int armada_370_xp_msi_init(struct device_node *node,
 
 	armada_370_xp_msi_reenable_percpu();
 
+	/* Unmask low 16 MSI irqs on non-IPI platforms */
+	if (!is_ipi_available())
+		writel(0, per_cpu_int_base + ARMADA_370_XP_INT_CLEAR_MASK_OFFS);
+
 	return 0;
 }
 #else
@@ -616,20 +655,20 @@ static void armada_370_xp_handle_msi_irq(struct pt_regs *regs, bool is_chained)
 	u32 msimask, msinr;
 
 	msimask = readl_relaxed(per_cpu_int_base +
-				ARMADA_370_XP_IN_DRBEL_CAUSE_OFFS)
-		& PCI_MSI_DOORBELL_MASK;
+				ARMADA_370_XP_IN_DRBEL_CAUSE_OFFS);
+	msimask &= msi_doorbell_mask();
 
 	writel(~msimask, per_cpu_int_base +
 	       ARMADA_370_XP_IN_DRBEL_CAUSE_OFFS);
 
-	for (msinr = PCI_MSI_DOORBELL_START;
-	     msinr < PCI_MSI_DOORBELL_END; msinr++) {
+	for (msinr = msi_doorbell_start();
+	     msinr < msi_doorbell_end(); msinr++) {
 		unsigned int irq;
 
 		if (!(msimask & BIT(msinr)))
 			continue;
 
-		irq = msinr - PCI_MSI_DOORBELL_START;
+		irq = msinr - msi_doorbell_start();
 
 		generic_handle_domain_irq(armada_370_xp_msi_inner_domain, irq);
 	}
@@ -658,7 +697,7 @@ static void armada_370_xp_mpic_handle_cascade_irq(struct irq_desc *desc)
 		if (!(irqsrc & ARMADA_370_XP_INT_IRQ_FIQ_MASK(cpuid)))
 			continue;
 
-		if (irqn == 1) {
+		if (irqn == 0 || irqn == 1) {
 			armada_370_xp_handle_msi_irq(NULL, true);
 			continue;
 		}
@@ -719,6 +758,7 @@ static int armada_370_xp_mpic_suspend(void)
 
 static void armada_370_xp_mpic_resume(void)
 {
+	bool src0 = false, src1 = false;
 	int nirqs;
 	irq_hw_number_t irq;
 
@@ -758,9 +798,18 @@ static void armada_370_xp_mpic_resume(void)
 	/* Reconfigure doorbells for IPIs and MSIs */
 	writel(doorbell_mask_reg,
 	       per_cpu_int_base + ARMADA_370_XP_IN_DRBEL_MSK_OFFS);
-	if (is_ipi_available() && (doorbell_mask_reg & IPI_DOORBELL_MASK))
+
+	if (is_ipi_available()) {
+		src0 = doorbell_mask_reg & IPI_DOORBELL_MASK;
+		src1 = doorbell_mask_reg & PCI_MSI_DOORBELL_MASK;
+	} else {
+		src0 = doorbell_mask_reg & PCI_MSI_FULL_DOORBELL_SRC0_MASK;
+		src1 = doorbell_mask_reg & PCI_MSI_FULL_DOORBELL_SRC1_MASK;
+	}
+
+	if (src0)
 		writel(0, per_cpu_int_base + ARMADA_370_XP_INT_CLEAR_MASK_OFFS);
-	if (doorbell_mask_reg & PCI_MSI_DOORBELL_MASK)
+	if (src1)
 		writel(1, per_cpu_int_base + ARMADA_370_XP_INT_CLEAR_MASK_OFFS);
 
 	ipi_resume();
-- 
2.44.2

Re: [PATCH 4/4] irqchip/armada-370-xp: Add support for 32 MSI interrupts on non-IPI platforms

[Andrew Lunn]

On Wed, Jun 19, 2024 at 04:11:34PM +0200, Marek Behún wrote:
> From: Pali Rohár <pali@kernel.org>
> 
> The doorbell interrupts have the following layout on IPI vs no-IPI
> platforms:
> 
>                     |  0...7  |  8...15  |       16...31       |
>   ------------------+---------+----------+---------------------+
>        IPI platform |   IPI   |   n/a    |         MSI         |
>   ------------------+---------+----------+---------------------+
>    non-IPI platform |                   MSI                    |
>   ------------------+------------------------------------------+
> 
> Currently the driver only allows for the upper 16...31 interrupts for
> MSI domain (i.e. the MSI domain has only 16 interrupts).
> 
> On platforms where IPI is not available, we can use whole 32 MSI
> interrupts.
> 
> Implement support also for the lower 16 MSI interrupts on non-IPI
> platforms.
> 
> Signed-off-by: Pali Rohár <pali@kernel.org>
> [ refactored, changed commit message ]
> Signed-off-by: Marek Behún <kabel@kernel.org>

Reviewed-by: Andrew Lunn <andrew@lunn.ch>

    Andrew