[PATCH v3 0/2] Add support to register platform service IRQ

[PATCH v3 0/2] Add support to register platform service IRQ

[Stefan Roese]

Some platforms have dedicated IRQ lines for platform-specific System Errors
like AER/PME etc. The root complex on these platform will use these seperate
IRQ lines to report AER/PME etc., interrupts and will not generate
MSI/MSI-X/INTx interrupts for these services.

These patches will add new method for these kind of platforms to register the
platform IRQ number with respective PCIe services.

Changes in v3 (Stefan):
- Restructure patches from 4 patches in v2 to now 2 patches in v3
- Rename of functions names
- init_platform_service_irqs() now uses "struct pci_dev *" instead of
  "struct pci_host_bridge *"
- pcie_init_platform_service_irqs() is called before pcie_init_service_irqs()
- Use more PCIe spec terminology as suggested by Bjorn (hopefully enough, I
  don't have the spec at hand)

Bharat Kumar Gogada (2):
  PCI/portdrv: Add option to setup IRQs for platform-specific Service
    Errors
  PCI: xilinx-nwl: Add method to init_platform_service_irqs hook

 drivers/pci/controller/pcie-xilinx-nwl.c | 14 ++++++++
 drivers/pci/pcie/portdrv_core.c          | 43 +++++++++++++++++++++++-
 include/linux/pci.h                      |  2 ++
 3 files changed, 58 insertions(+), 1 deletion(-)

-- 
2.34.1

[PATCH v3 1/2] PCI/portdrv: Add option to setup IRQs for platform-specific Service Errors

[Stefan Roese]

From: Bharat Kumar Gogada <bharat.kumar.gogada@xilinx.com>

As per section 6.2.4.1.2, 6.2.6 in PCIe r4.0 (and later versions),
platform-specific System Errors like AER can be delivered via platform-
specific interrupt lines.

This patch adds the init_platform_service_irqs() hook to struct
pci_host_bridge, making it possible that platforms may implement this
function to hook IRQs for these platform-specific System Errors, like
AER.

If these platform-specific service IRQs have been successfully
installed via pcie_init_platform_service_irqs(),
pcie_init_service_irqs() is skipped.

Signed-off-by: Bharat Kumar Gogada <bharat.kumar.gogada@xilinx.com>
Signed-off-by: Stefan Roese <sr@denx.de>
Cc: Bjorn Helgaas <helgaas@kernel.org>
Cc: Pali Rohár <pali@kernel.org>
Cc: Michal Simek <michal.simek@xilinx.com>
---
 drivers/pci/pcie/portdrv_core.c | 43 ++++++++++++++++++++++++++++++++-
 include/linux/pci.h             |  2 ++
 2 files changed, 44 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/pcie/portdrv_core.c b/drivers/pci/pcie/portdrv_core.c
index bda630889f95..4dab74ff4368 100644
--- a/drivers/pci/pcie/portdrv_core.c
+++ b/drivers/pci/pcie/portdrv_core.c
@@ -190,6 +190,31 @@ static int pcie_init_service_irqs(struct pci_dev *dev, int *irqs, int mask)
 	return 0;
 }
 
+/**
+ * pcie_init_platform_service_irqs - initialize platform service irqs for
+ * platform-specific System Errors
+ * @dev: PCI Express port to handle
+ * @irqs: Array of irqs to populate
+ * @mask: Bitmask of capabilities
+ *
+ * Return value: true/false for platforms service irqs installed or not
+ */
+static bool pcie_init_platform_service_irqs(struct pci_dev *dev,
+					    int *irqs, int mask)
+{
+	struct pci_host_bridge *bridge;
+
+	if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) {
+		bridge = pci_find_host_bridge(dev->bus);
+		if (bridge && bridge->init_platform_service_irqs) {
+			bridge->init_platform_service_irqs(dev, irqs, mask);
+			return true;
+		}
+	}
+
+	return false;
+}
+
 /**
  * get_port_device_capability - discover capabilities of a PCI Express port
  * @dev: PCI Express port to examine
@@ -318,6 +343,7 @@ int pcie_port_device_register(struct pci_dev *dev)
 	int irqs[PCIE_PORT_DEVICE_MAXSERVICES] = {
 		[0 ... PCIE_PORT_DEVICE_MAXSERVICES-1] = -1
 	};
+	bool plat_irqs;
 
 	/* Enable PCI Express port device */
 	status = pci_enable_device(dev);
@@ -342,7 +368,22 @@ int pcie_port_device_register(struct pci_dev *dev)
 		irq_services |= PCIE_PORT_SERVICE_DPC;
 	irq_services &= capabilities;
 
-	if (irq_services) {
+	/*
+	 * Some platforms have dedicated interrupts from root complex to
+	 * interrupt controller for PCIe platform-specific System Errors
+	 * like AER/PME etc., check if the platform registered with any such
+	 * IRQ.
+	 */
+	if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) {
+		plat_irqs = pcie_init_platform_service_irqs(dev, irqs,
+							    capabilities);
+	}
+
+	/*
+	 * Only install service irqs, when the platform-specific hook was
+	 * unsuccessful
+	 */
+	if (irq_services && !plat_irqs) {
 		/*
 		 * Initialize service IRQs. Don't use service devices that
 		 * require interrupts if there is no way to generate them.
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 18a75c8e615c..a0bcc8062b91 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -554,6 +554,8 @@ struct pci_host_bridge {
 	u8 (*swizzle_irq)(struct pci_dev *, u8 *); /* Platform IRQ swizzler */
 	int (*map_irq)(const struct pci_dev *, u8, u8);
 	void (*release_fn)(struct pci_host_bridge *);
+	void (*init_platform_service_irqs)(struct pci_dev *dev, int *irqs,
+					   int plat_mask);
 	void		*release_data;
 	unsigned int	ignore_reset_delay:1;	/* For entire hierarchy */
 	unsigned int	no_ext_tags:1;		/* No Extended Tags */
-- 
2.34.1

Re: [PATCH v3 1/2] PCI/portdrv: Add option to setup IRQs for platform-specific Service Errors

[Pali Rohár]

On Thursday 13 January 2022 11:49:38 Stefan Roese wrote:
> From: Bharat Kumar Gogada <bharat.kumar.gogada@xilinx.com>
> 
> As per section 6.2.4.1.2, 6.2.6 in PCIe r4.0 (and later versions),
> platform-specific System Errors like AER can be delivered via platform-
> specific interrupt lines.
> 
> This patch adds the init_platform_service_irqs() hook to struct
> pci_host_bridge, making it possible that platforms may implement this
> function to hook IRQs for these platform-specific System Errors, like
> AER.
> 
> If these platform-specific service IRQs have been successfully
> installed via pcie_init_platform_service_irqs(),
> pcie_init_service_irqs() is skipped.
> 
> Signed-off-by: Bharat Kumar Gogada <bharat.kumar.gogada@xilinx.com>
> Signed-off-by: Stefan Roese <sr@denx.de>
> Cc: Bjorn Helgaas <helgaas@kernel.org>
> Cc: Pali Rohár <pali@kernel.org>
> Cc: Michal Simek <michal.simek@xilinx.com>
> ---
>  drivers/pci/pcie/portdrv_core.c | 43 ++++++++++++++++++++++++++++++++-
>  include/linux/pci.h             |  2 ++
>  2 files changed, 44 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/pci/pcie/portdrv_core.c b/drivers/pci/pcie/portdrv_core.c
> index bda630889f95..4dab74ff4368 100644
> --- a/drivers/pci/pcie/portdrv_core.c
> +++ b/drivers/pci/pcie/portdrv_core.c
> @@ -190,6 +190,31 @@ static int pcie_init_service_irqs(struct pci_dev *dev, int *irqs, int mask)
>  	return 0;
>  }
>  
> +/**
> + * pcie_init_platform_service_irqs - initialize platform service irqs for
> + * platform-specific System Errors
> + * @dev: PCI Express port to handle
> + * @irqs: Array of irqs to populate
> + * @mask: Bitmask of capabilities
> + *
> + * Return value: true/false for platforms service irqs installed or not
> + */
> +static bool pcie_init_platform_service_irqs(struct pci_dev *dev,
> +					    int *irqs, int mask)
> +{
> +	struct pci_host_bridge *bridge;
> +
> +	if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) {

I think that this check is not needed as it is done before calling
pcie_init_platform_service_irqs() function.

> +		bridge = pci_find_host_bridge(dev->bus);
> +		if (bridge && bridge->init_platform_service_irqs) {
> +			bridge->init_platform_service_irqs(dev, irqs, mask);
> +			return true;

Suggestion: What about "return bridge->init_platform_service_irqs(...);" ?
This could allow callback function to fail...

> +		}
> +	}
> +
> +	return false;
> +}
> +
>  /**
>   * get_port_device_capability - discover capabilities of a PCI Express port
>   * @dev: PCI Express port to examine
> @@ -318,6 +343,7 @@ int pcie_port_device_register(struct pci_dev *dev)
>  	int irqs[PCIE_PORT_DEVICE_MAXSERVICES] = {
>  		[0 ... PCIE_PORT_DEVICE_MAXSERVICES-1] = -1
>  	};
> +	bool plat_irqs;
>  
>  	/* Enable PCI Express port device */
>  	status = pci_enable_device(dev);
> @@ -342,7 +368,22 @@ int pcie_port_device_register(struct pci_dev *dev)
>  		irq_services |= PCIE_PORT_SERVICE_DPC;
>  	irq_services &= capabilities;
>  
> -	if (irq_services) {
> +	/*
> +	 * Some platforms have dedicated interrupts from root complex to
> +	 * interrupt controller for PCIe platform-specific System Errors
> +	 * like AER/PME etc., check if the platform registered with any such
> +	 * IRQ.
> +	 */
> +	if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) {
> +		plat_irqs = pcie_init_platform_service_irqs(dev, irqs,
> +							    capabilities);
> +	}
> +
> +	/*
> +	 * Only install service irqs, when the platform-specific hook was
> +	 * unsuccessful
> +	 */
> +	if (irq_services && !plat_irqs) {
>  		/*
>  		 * Initialize service IRQs. Don't use service devices that
>  		 * require interrupts if there is no way to generate them.
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index 18a75c8e615c..a0bcc8062b91 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -554,6 +554,8 @@ struct pci_host_bridge {
>  	u8 (*swizzle_irq)(struct pci_dev *, u8 *); /* Platform IRQ swizzler */
>  	int (*map_irq)(const struct pci_dev *, u8, u8);
>  	void (*release_fn)(struct pci_host_bridge *);
> +	void (*init_platform_service_irqs)(struct pci_dev *dev, int *irqs,
> +					   int plat_mask);
>  	void		*release_data;
>  	unsigned int	ignore_reset_delay:1;	/* For entire hierarchy */
>  	unsigned int	no_ext_tags:1;		/* No Extended Tags */
> -- 
> 2.34.1
> 

Re: [PATCH v3 1/2] PCI/portdrv: Add option to setup IRQs for platform-specific Service Errors

[Stefan Roese]

On 1/13/22 12:14, Pali Rohár wrote:
> On Thursday 13 January 2022 11:49:38 Stefan Roese wrote:
>> From: Bharat Kumar Gogada <bharat.kumar.gogada@xilinx.com>
>>
>> As per section 6.2.4.1.2, 6.2.6 in PCIe r4.0 (and later versions),
>> platform-specific System Errors like AER can be delivered via platform-
>> specific interrupt lines.
>>
>> This patch adds the init_platform_service_irqs() hook to struct
>> pci_host_bridge, making it possible that platforms may implement this
>> function to hook IRQs for these platform-specific System Errors, like
>> AER.
>>
>> If these platform-specific service IRQs have been successfully
>> installed via pcie_init_platform_service_irqs(),
>> pcie_init_service_irqs() is skipped.
>>
>> Signed-off-by: Bharat Kumar Gogada <bharat.kumar.gogada@xilinx.com>
>> Signed-off-by: Stefan Roese <sr@denx.de>
>> Cc: Bjorn Helgaas <helgaas@kernel.org>
>> Cc: Pali Rohár <pali@kernel.org>
>> Cc: Michal Simek <michal.simek@xilinx.com>
>> ---
>>   drivers/pci/pcie/portdrv_core.c | 43 ++++++++++++++++++++++++++++++++-
>>   include/linux/pci.h             |  2 ++
>>   2 files changed, 44 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/pci/pcie/portdrv_core.c b/drivers/pci/pcie/portdrv_core.c
>> index bda630889f95..4dab74ff4368 100644
>> --- a/drivers/pci/pcie/portdrv_core.c
>> +++ b/drivers/pci/pcie/portdrv_core.c
>> @@ -190,6 +190,31 @@ static int pcie_init_service_irqs(struct pci_dev *dev, int *irqs, int mask)
>>   	return 0;
>>   }
>>   
>> +/**
>> + * pcie_init_platform_service_irqs - initialize platform service irqs for
>> + * platform-specific System Errors
>> + * @dev: PCI Express port to handle
>> + * @irqs: Array of irqs to populate
>> + * @mask: Bitmask of capabilities
>> + *
>> + * Return value: true/false for platforms service irqs installed or not
>> + */
>> +static bool pcie_init_platform_service_irqs(struct pci_dev *dev,
>> +					    int *irqs, int mask)
>> +{
>> +	struct pci_host_bridge *bridge;
>> +
>> +	if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) {
> 
> I think that this check is not needed as it is done before calling
> pcie_init_platform_service_irqs() function.

Ah, you are correct. I'll remove this check in v4.

>> +		bridge = pci_find_host_bridge(dev->bus);
>> +		if (bridge && bridge->init_platform_service_irqs) {
>> +			bridge->init_platform_service_irqs(dev, irqs, mask);
>> +			return true;
> 
> Suggestion: What about "return bridge->init_platform_service_irqs(...);" ?
> This could allow callback function to fail...

Even better. I'll make this change as well and will wait a bit with
sending v4 to collect a few more review comments.

Thanks,
Stefan

[PATCH v3 2/2] PCI: xilinx-nwl: Add method to init_platform_service_irqs hook

[Stefan Roese]

From: Bharat Kumar Gogada <bharat.kumar.gogada@xilinx.com>

Add nwl_init_platform_service_irqs() hook to init_platform_service_irqs
to register the platform-specific Service Errors IRQs for this PCIe
controller to fully support e.g. AER on this platform.

Signed-off-by: Bharat Kumar Gogada <bharat.kumar.gogada@xilinx.com>
Signed-off-by: Stefan Roese <sr@denx.de>
Cc: Bjorn Helgaas <helgaas@kernel.org>
Cc: Pali Rohár <pali@kernel.org>
Cc: Michal Simek <michal.simek@xilinx.com>
---
 drivers/pci/controller/pcie-xilinx-nwl.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/drivers/pci/controller/pcie-xilinx-nwl.c b/drivers/pci/controller/pcie-xilinx-nwl.c
index 414b679175b3..607d26a395ec 100644
--- a/drivers/pci/controller/pcie-xilinx-nwl.c
+++ b/drivers/pci/controller/pcie-xilinx-nwl.c
@@ -24,6 +24,7 @@
 #include <linux/irqchip/chained_irq.h>
 
 #include "../pci.h"
+#include "../pcie/portdrv.h"
 
 /* Bridge core config registers */
 #define BRCFG_PCIE_RX0			0x00000000
@@ -806,6 +807,18 @@ static int nwl_pcie_parse_dt(struct nwl_pcie *pcie,
 	return 0;
 }
 
+static void nwl_init_platform_service_irqs(struct pci_dev *dev, int *irqs,
+					   int plat_mask)
+{
+	struct pci_host_bridge *bridge;
+	struct nwl_pcie *pcie;
+
+	bridge = pci_find_host_bridge(dev->bus);
+	pcie = pci_host_bridge_priv(bridge);
+	if (plat_mask & PCIE_PORT_SERVICE_AER)
+		irqs[PCIE_PORT_SERVICE_AER_SHIFT] = pcie->irq_misc;
+}
+
 static const struct of_device_id nwl_pcie_of_match[] = {
 	{ .compatible = "xlnx,nwl-pcie-2.11", },
 	{}
@@ -857,6 +870,7 @@ static int nwl_pcie_probe(struct platform_device *pdev)
 
 	bridge->sysdata = pcie;
 	bridge->ops = &nwl_pcie_ops;
+	bridge->init_platform_service_irqs = nwl_init_platform_service_irqs;
 
 	if (IS_ENABLED(CONFIG_PCI_MSI)) {
 		err = nwl_pcie_enable_msi(pcie);
-- 
2.34.1