All of lore.kernel.org
 help / color / mirror / Atom feed
From: Marc Zyngier <marc.zyngier@arm.com>
To: "suravee.suthikulpanit\@amd.com" <suravee.suthikulpanit@amd.com>
Cc: Mark Rutland <Mark.Rutland@arm.com>,
	"jason\@lakedaemon.net" <jason@lakedaemon.net>,
	Pawel Moll <Pawel.Moll@arm.com>,
	Catalin Marinas <Catalin.Marinas@arm.com>,
	Will Deacon <Will.Deacon@arm.com>,
	"tglx\@linutronix.de" <tglx@linutronix.de>,
	"Harish.Kasiviswanathan\@amd.com"
	<Harish.Kasiviswanathan@amd.com>,
	"linux-arm-kernel\@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>,
	"linux-pci\@vger.kernel.org" <linux-pci@vger.kernel.org>,
	"linux-kernel\@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-doc\@vger.kernel.org" <linux-doc@vger.kernel.org>,
	"devicetree\@vger.kernel.org" <devicetree@vger.kernel.org>
Subject: Re: [PATCH 1/2 V6] irqchip: gic: Add supports for ARM GICv2m MSI(-X)
Date: Mon, 15 Sep 2014 20:26:20 +0100	[thread overview]
Message-ID: <86iokon0rn.fsf@arm.com> (raw)
In-Reply-To: <1410676216-27953-2-git-send-email-suravee.suthikulpanit@amd.com> (suravee's message of "Sun, 14 Sep 2014 07:30:15 +0100")

On Sun, Sep 14 2014 at 07:30:15 AM, "suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com> wrote:

Hi Suravee,

> From: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
>
> ARM GICv2m specification extends GICv2 to support MSI(-X) with
> a new set of register frame. This patch introduces support for
> the non-secure GICv2m register frame. Currently, GICV2m is available
> in certain version of GIC-400.
>
> The patch introduces a new property in ARM gic binding, the v2m subnode.
> It is optional.

Overall, this looks much better than the previous versions I've
reviewed. There is still a number of points I'd like to see addressed,
see below.

>
> Signed-off-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
> Cc: Mark Rutland <Mark.Rutland@arm.com>
> Cc: Marc Zyngier <Marc.Zyngier@arm.com>
> Cc: Jason Cooper <jason@lakedaemon.net>
> Cc: Catalin Marinas <Catalin.Marinas@arm.com>
> Cc: Will Deacon <Will.Deacon@arm.com>
> ---
>  Documentation/devicetree/bindings/arm/gic.txt |  39 ++++
>  drivers/irqchip/Kconfig                       |   7 +
>  drivers/irqchip/Makefile                      |   1 +
>  drivers/irqchip/irq-gic-v2m.c                 | 261 ++++++++++++++++++++++++++
>  drivers/irqchip/irq-gic.c                     |  88 ++++++---
>  drivers/irqchip/irq-gic.h                     |  51 +++++
>  6 files changed, 419 insertions(+), 28 deletions(-)
>  create mode 100644 drivers/irqchip/irq-gic-v2m.c
>  create mode 100644 drivers/irqchip/irq-gic.h
>
> diff --git a/Documentation/devicetree/bindings/arm/gic.txt b/Documentation/devicetree/bindings/arm/gic.txt
> index c7d2fa1..a461591 100644
> --- a/Documentation/devicetree/bindings/arm/gic.txt
> +++ b/Documentation/devicetree/bindings/arm/gic.txt
> @@ -96,3 +96,42 @@ Example:
>                       <0x2c006000 0x2000>;
>                 interrupts = <1 9 0xf04>;
>         };
> +
> +
> +* GICv2m extension for MSI/MSI-x support (Optional)
> +
> +Certain revision of GIC-400 supports MSI/MSI-x via V2M register frame.
> +This is enabled by specifying v2m sub-node.
> +
> +Required properties:
> +
> +- msi-controller : Identifies the node as an MSI controller.
> +
> +- reg : GICv2m MSI interface register base and size
> +
> +Example:
> +
> +       interrupt-controller@e1101000 {
> +               compatible = "arm,gic-400";
> +               #interrupt-cells = <3>;
> +               #address-cells = <2>;
> +               #size-cells = <2>;
> +               interrupt-controller;
> +               interrupts = <1 8 0xf04>;
> +               ranges = <0 0 0 0xe1100000 0 0x100000>;
> +               reg = <0x0 0xe1110000 0 0x01000>,
> +                     <0x0 0xe112f000 0 0x02000>,
> +                     <0x0 0xe1140000 0 0x10000>,
> +                     <0x0 0xe1160000 0 0x10000>;
> +               v2m0 {

I believe the convention is to have
		v2m0 : v2m@0x80000 {

> +                       msi-controller;
> +                       reg = <0x0 0x80000 0 0x1000>;

I think having a "compatible" property would be nice, and would help
handle variants if they ever happen.

> +               };
> +
> +               ....
> +
> +               v2mN {
> +                       msi-controller;
> +                       reg = <0x0 0x90000 0 0x1000>;
> +               };
> +       };
> diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
> index b8632bf..ddb31ee 100644
> --- a/drivers/irqchip/Kconfig
> +++ b/drivers/irqchip/Kconfig
> @@ -7,6 +7,13 @@ config ARM_GIC
>         select IRQ_DOMAIN
>         select MULTI_IRQ_HANDLER
>
> +config ARM_GIC_V2M
> +       bool
> +       select IRQ_DOMAIN
> +       select MULTI_IRQ_HANDLER
> +       depends on ARM_GIC
> +       depends on PCI && PCI_MSI
> +
>  config GIC_NON_BANKED
>         bool
>
> diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
> index 73052ba..3bda951 100644
> --- a/drivers/irqchip/Makefile
> +++ b/drivers/irqchip/Makefile
> @@ -17,6 +17,7 @@ obj-$(CONFIG_ARCH_SUNXI)              += irq-sun4i.o
>  obj-$(CONFIG_ARCH_SUNXI)               += irq-sunxi-nmi.o
>  obj-$(CONFIG_ARCH_SPEAR3XX)            += spear-shirq.o
>  obj-$(CONFIG_ARM_GIC)                  += irq-gic.o irq-gic-common.o
> +obj-$(CONFIG_ARM_GIC_V2M)              += irq-gic-v2m.o
>  obj-$(CONFIG_ARM_GIC_V3)               += irq-gic-v3.o irq-gic-common.o
>  obj-$(CONFIG_ARM_NVIC)                 += irq-nvic.o
>  obj-$(CONFIG_ARM_VIC)                  += irq-vic.o
> diff --git a/drivers/irqchip/irq-gic-v2m.c b/drivers/irqchip/irq-gic-v2m.c
> new file mode 100644
> index 0000000..499bfb9
> --- /dev/null
> +++ b/drivers/irqchip/irq-gic-v2m.c
> @@ -0,0 +1,261 @@
> +/*
> + * ARM GIC v2m MSI(-X) support
> + * Support for Message Signaled Interrupts for systems that
> + * implement ARM Generic Interrupt Controller: GICv2m.
> + *
> + * Copyright (C) 2014 Advanced Micro Devices, Inc.
> + * Authors: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
> + *          Harish Kasiviswanathan <harish.kasiviswanathan@amd.com>
> + *          Brandon Anderson <brandon.anderson@amd.com>
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License version 2 as published
> + * by the Free Software Foundation.
> + */
> +
> +#define pr_fmt(fmt) "GICv2m: " fmt
> +
> +#include <linux/bitmap.h>
> +#include <linux/irq.h>
> +#include <linux/kernel.h>
> +#include <linux/module.h>
> +#include <linux/of_address.h>
> +#include <linux/of_pci.h>
> +#include <linux/pci.h>
> +#include <linux/platform_device.h>
> +#include <linux/slab.h>
> +#include <linux/spinlock.h>
> +
> +#include "irqchip.h"
> +#include "irq-gic.h"
> +
> +/*
> +* MSI_TYPER:
> +*     [31:26] Reserved
> +*     [25:16] lowest SPI assigned to MSI
> +*     [15:10] Reserved
> +*     [9:0]   Numer of SPIs assigned to MSI
> +*/
> +#define V2M_MSI_TYPER                  0x008
> +#define V2M_MSI_TYPER_BASE_SHIFT       16
> +#define V2M_MSI_TYPER_BASE_MASK                0x3FF
> +#define V2M_MSI_TYPER_NUM_MASK         0x3FF
> +#define V2M_MSI_SETSPI_NS              0x040
> +#define V2M_MIN_SPI                    32
> +#define V2M_MAX_SPI                    1019
> +
> +/*
> + * alloc_msi_irq - Allocate MSIs from available MSI bitmap.
> + * @data: Pointer to v2m_data
> + * @nvec: Number of interrupts to allocate
> + * @irq: Pointer to the allocated irq
> + *
> + * Allocates interrupts only if the contiguous range of MSIs
> + * with specified nvec are available. Otherwise return the number
> + * of available interrupts. If none are available, then returns -ENOENT.
> + */
> +static int alloc_msi_irq(struct v2m_data *data, int nvec, int *irq)
> +{
> +       int size = data->nr_spis;
> +       int next = size, i = nvec, ret;
> +
> +       /* We should never allocate more than available nr_spis */
> +       if (i >= size)
> +               i = size;
> +
> +       spin_lock(&data->msi_cnt_lock);
> +
> +       for (; i > 0; i--) {
> +               next = bitmap_find_next_zero_area(data->bm,
> +                                       size, 0, i, 0);
> +               if (next < size)
> +                       break;
> +       }
> +
> +       if (i != nvec) {
> +               ret = i ? : -ENOENT;
> +       } else {
> +               bitmap_set(data->bm, next, nvec);
> +               *irq = data->spi_start + next;
> +               ret = 0;
> +       }
> +
> +       spin_unlock(&data->msi_cnt_lock);
> +
> +       return ret;
> +}
> +
> +static void gicv2m_teardown_msi_irq(struct msi_chip *chip, unsigned int irq)
> +{
> +       int pos;
> +       struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
> +
> +       spin_lock(&data->msi_cnt_lock);
> +
> +       pos = irq - data->spi_start;
> +       if (pos >= 0 && pos < data->nr_spis)
> +               bitmap_clear(data->bm, pos, 1);
> +
> +       spin_unlock(&data->msi_cnt_lock);
> +}
> +
> +static int gicv2m_setup_msi_irq(struct msi_chip *chip,
> +                               struct pci_dev *pdev,
> +                               struct msi_desc *desc)
> +{
> +       int irq, avail;
> +       struct msi_msg msg;
> +       phys_addr_t addr;
> +       struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
> +
> +       if (!desc) {
> +               dev_err(&pdev->dev,
> +                       "GICv2m: MSI setup failed. Invalid msi descriptor\n");

Isn't the "GICv2m" header already provided by the pr_fmt at the
beginning of the file?

> +               return -EINVAL;
> +       }
> +
> +       avail = alloc_msi_irq(data, 1, &irq);
> +       if (avail != 0) {
> +               dev_err(&pdev->dev,
> +                       "GICv2m: MSI setup failed. Cannnot allocate IRQ\n");
> +               return -ENOSPC;
> +       }
> +
> +       irq_set_chip_data(irq, chip);
> +       irq_set_msi_desc(irq, desc);
> +       irq_set_irq_type(irq, IRQ_TYPE_EDGE_RISING);
> +
> +       addr = data->res.start + V2M_MSI_SETSPI_NS;
> +       msg.address_hi = (u32)(addr >> 32);
> +       msg.address_lo = (u32)(addr);
> +       msg.data = irq;
> +       write_msi_msg(irq, &msg);
> +
> +       return 0;
> +}
> +
> +static void gicv2m_mask_irq(struct irq_data *d)
> +{
> +       gic_mask_irq(d);
> +       if (d->msi_desc)
> +               mask_msi_irq(d);
> +}
> +
> +static void gicv2m_unmask_irq(struct irq_data *d)
> +{
> +       gic_unmask_irq(d);
> +       if (d->msi_desc)
> +               unmask_msi_irq(d);
> +}
> +
> +#ifdef CONFIG_OF

I think you can loose the CONFIG_OF here. There is no valid
configuration without OF on arm64.

> +
> +static int __init
> +v2m_init(struct device_node *node, struct v2m_data *v2m,
> +                 struct resource *res)
> +{
> +       int ret;
> +       unsigned int typer;
> +
> +       v2m->msi_chip.owner = THIS_MODULE;
> +       v2m->msi_chip.of_node = node;
> +       v2m->msi_chip.setup_irq = gicv2m_setup_msi_irq;
> +       v2m->msi_chip.teardown_irq = gicv2m_teardown_msi_irq;
> +       ret = of_pci_msi_chip_add(&v2m->msi_chip);
> +       if (ret) {
> +               pr_info("Failed to add msi_chip.\n");
> +               return ret;
> +       }
> +
> +       v2m->base = ioremap(res->start, resource_size(res));
> +       if (!v2m->base) {
> +               pr_err("Failed to map GICv2m resource\n");
> +               return -EINVAL;

Beware, you're leaving with a registered msi_chip. You probably want to
remove it on the error path.

> +       }
> +
> +       memcpy(&v2m->res, res, sizeof(struct resource));
> +
> +       typer = readl_relaxed(v2m->base + V2M_MSI_TYPER);
> +       if (!typer) {
> +               pr_warn("Failed to read V2M_MSI_TYPER register\n");
> +               return -EINVAL;

And here leaving the v2m widget mapped.

> +       }
> +
> +       v2m->spi_start = (typer >> V2M_MSI_TYPER_BASE_SHIFT) &
> +                               V2M_MSI_TYPER_BASE_MASK;
> +       v2m->nr_spis = typer & V2M_MSI_TYPER_NUM_MASK;
> +       if ((v2m->spi_start < V2M_MIN_SPI) || (v2m->nr_spis >= V2M_MAX_SPI)) {
> +                       pr_err("Invalid MSI_TYPER (%#x)\n", typer);
> +                       return -EINVAL;

Blah. The whole error handling of this function is b0rked.

> +       }
> +
> +       v2m->bm = kzalloc(sizeof(long) * BITS_TO_LONGS(v2m->nr_spis),
> +                         GFP_KERNEL);
> +       if (!v2m->bm) {
> +               pr_err("Failed to allocate MSI bitmap\n");
> +               return -ENOMEM;
> +       }
> +
> +       spin_lock_init(&v2m->msi_cnt_lock);
> +
> +       pr_info("SPI range [%d:%d]\n",
> +               v2m->spi_start, (v2m->spi_start + v2m->nr_spis));
> +
> +       return 0;
> +}
> +
> +int __init gicv2m_of_init(struct device_node *node,
> +                         struct gic_chip_data *gic,
> +                         struct irq_chip *v2m_chip)
> +{
> +       int ret = 0;
> +       struct v2m_data *v2m;
> +       struct resource res;
> +       struct device_node *child = NULL;
> +
> +       INIT_LIST_HEAD(&gic->v2m_list);
> +
> +       for (;;) {
> +               child = of_get_next_child(node, child);

I don't see any of_node_put() in this function. You'll need to fix
this...

> +               if (!child)
> +                       break;
> +
> +               if (!of_find_property(child, "msi-controller", NULL))
> +                       continue;
> +
> +               ret = of_address_to_resource(child, 0, &res);
> +               if (ret) {
> +                       pr_err("Failed to allocate v2m resource.\n");
> +                       break;
> +               }
> +
> +               v2m = kzalloc(sizeof(struct v2m_data), GFP_KERNEL);
> +               if (!v2m) {
> +                       pr_err("Failed to allocate struct v2m_data.\n");
> +                       return -ENOMEM;
> +               }
> +
> +               ret = v2m_init(child, v2m, &res);
> +               if (ret)
> +                       break;
> +
> +               v2m->gic = gic;
> +
> +               pr_info(" %s: [%lx : %lx]\n", child->name,
> +                       (unsigned long)res.start, (unsigned long)res.end);
> +
> +               list_add_tail(&v2m->list, &gic->v2m_list);
> +       }
> +
> +       if (ret && list_empty(&gic->v2m_list)) {
> +               pr_err("MSI resource not found\n");
> +               return -EINVAL;
> +       }
> +
> +       v2m_chip->irq_mask = gicv2m_mask_irq;
> +       v2m_chip->irq_unmask = gicv2m_unmask_irq;
> +

Again, the whole error handling is evil. You can end-up allocating a few
v2m structures, registering them, and yet have kzalloc failing at some
point. At that stage, you exit early, not setting the mas/unmask
methods. Boom.

> +       return ret;
> +}
> +
> +#endif /* CONFIG_OF */
> diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
> index 4b959e6..f490599 100644
> --- a/drivers/irqchip/irq-gic.c
> +++ b/drivers/irqchip/irq-gic.c
> @@ -46,30 +46,9 @@
>  #include <asm/smp_plat.h>
>
>  #include "irq-gic-common.h"
> +#include "irq-gic.h"
>  #include "irqchip.h"
>
> -union gic_base {
> -       void __iomem *common_base;
> -       void __percpu * __iomem *percpu_base;
> -};
> -
> -struct gic_chip_data {
> -       union gic_base dist_base;
> -       union gic_base cpu_base;
> -#ifdef CONFIG_CPU_PM
> -       u32 saved_spi_enable[DIV_ROUND_UP(1020, 32)];
> -       u32 saved_spi_conf[DIV_ROUND_UP(1020, 16)];
> -       u32 saved_spi_target[DIV_ROUND_UP(1020, 4)];
> -       u32 __percpu *saved_ppi_enable;
> -       u32 __percpu *saved_ppi_conf;
> -#endif
> -       struct irq_domain *domain;
> -       unsigned int gic_irqs;
> -#ifdef CONFIG_GIC_NON_BANKED
> -       void __iomem *(*get_base)(union gic_base *);
> -#endif
> -};
> -
>  static DEFINE_RAW_SPINLOCK(irq_controller_lock);
>
>  /*
> @@ -131,15 +110,36 @@ static inline void gic_set_base_accessor(struct gic_chip_data *data,
>  #define gic_set_base_accessor(d, f)
>  #endif
>
> +static inline
> +struct gic_chip_data *irq_data_get_gic_chip_data(struct irq_data *d)
> +{
> +       struct gic_chip_data *gic_data;
> +       struct msi_chip *mchip;
> +       struct v2m_data *v2mdat;
> +
> +       /*
> +        * For MSI, irq_data.chip_data points to struct msi_chip.
> +        * For non-MSI, irq_data.chip_data points to struct gic_chip_data.
> +        */
> +       if (d->msi_desc) {
> +               mchip = irq_data_get_irq_chip_data(d);
> +               v2mdat = container_of(mchip, struct v2m_data, msi_chip);
> +               gic_data = v2mdat->gic;
> +       } else {
> +               gic_data = irq_data_get_irq_chip_data(d);
> +       }
> +       return gic_data;
> +}
> +
>  static inline void __iomem *gic_dist_base(struct irq_data *d)
>  {
> -       struct gic_chip_data *gic_data = irq_data_get_irq_chip_data(d);
> +       struct gic_chip_data *gic_data = irq_data_get_gic_chip_data(d);
>         return gic_data_dist_base(gic_data);
>  }
>
>  static inline void __iomem *gic_cpu_base(struct irq_data *d)
>  {
> -       struct gic_chip_data *gic_data = irq_data_get_irq_chip_data(d);
> +       struct gic_chip_data *gic_data = irq_data_get_gic_chip_data(d);
>         return gic_data_cpu_base(gic_data);
>  }
>
> @@ -151,7 +151,7 @@ static inline unsigned int gic_irq(struct irq_data *d)
>  /*
>   * Routines to acknowledge, disable and enable interrupts
>   */
> -static void gic_mask_irq(struct irq_data *d)
> +void gic_mask_irq(struct irq_data *d)

I'm still very uncomfortable with exporting the mask/unmask methods
outside of this file... Maybe it is not a big deal, but I see evil
everywhere.

>  {
>         u32 mask = 1 << (gic_irq(d) % 32);
>
> @@ -162,7 +162,7 @@ static void gic_mask_irq(struct irq_data *d)
>         raw_spin_unlock(&irq_controller_lock);
>  }
>
> -static void gic_unmask_irq(struct irq_data *d)
> +void gic_unmask_irq(struct irq_data *d)
>  {
>         u32 mask = 1 << (gic_irq(d) % 32);
>
> @@ -325,6 +325,15 @@ static struct irq_chip gic_chip = {
>         .irq_set_wake           = gic_set_wake,
>  };
>
> +static struct irq_chip v2m_chip = {
> +       .name                   = "GICv2m",
> +       .irq_eoi                = gic_eoi_irq,
> +       .irq_set_type           = gic_set_type,
> +#ifdef CONFIG_SMP
> +       .irq_set_affinity       = gic_set_affinity,
> +#endif
> +};
> +
>  void __init gic_cascade_irq(unsigned int gic_nr, unsigned int irq)
>  {
>         if (gic_nr >= MAX_GIC_NR)
> @@ -767,6 +776,24 @@ void __init gic_init_physaddr(struct device_node *node)
>  static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
>                                 irq_hw_number_t hw)
>  {
> +       struct gic_chip_data *gic = d->host_data;
> +       struct v2m_data *v2m;
> +
> +       irq_set_chip_data(irq, gic);
>
> +#ifdef CONFIG_ARM_GIC_V2M
> +       list_for_each_entry(v2m, &gic->v2m_list, list) {
> +               if (hw >= v2m->spi_start &&
> +                   hw <  v2m->spi_start + v2m->nr_spis) {
> +
> +                       irq_set_chip_and_handler(irq, &v2m_chip,
> +                                        handle_fasteoi_irq);
> +                       set_irq_flags(irq, IRQF_VALID | IRQF_PROBE);
> +                       return 0;
> +               }
> +       }
> +#endif
> +
>         if (hw < 32) {
>                 irq_set_percpu_devid(irq);
>                 irq_set_chip_and_handler(irq, &gic_chip,
> @@ -779,7 +806,6 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
>
>                 gic_routable_irq_domain_ops->map(d, irq, hw);
>         }
> -       irq_set_chip_data(irq, d->host_data);

These two hunks make the code a bit hard to read:
- First, message-based SPIs,
- Then PPIs
- and finally the rest of the SPIs.

Can you come up with a construct that keeps everything in a logical
order (PPIs, and then SPIs)? Not a big deal but it would make the code
nicer.

>         return 0;
>  }
>
> @@ -996,7 +1022,7 @@ gic_of_init(struct device_node *node, struct device_node *parent)
>         void __iomem *cpu_base;
>         void __iomem *dist_base;
>         u32 percpu_offset;
> -       int irq;
> +       int irq, ret;
>
>         if (WARN_ON(!node))
>                 return -ENODEV;
> @@ -1010,6 +1036,12 @@ gic_of_init(struct device_node *node, struct device_node *parent)
>         if (of_property_read_u32(node, "cpu-offset", &percpu_offset))
>                 percpu_offset = 0;
>
> +       if (IS_ENABLED(CONFIG_ARM_GIC_V2M)) {
> +               ret = gicv2m_of_init(node, &gic_data[gic_cnt], &v2m_chip);
> +               if (ret)
> +                       pr_warn("Warning: Failed to enable GICv2m support.\n");
> +       }
> +
>         gic_init_bases(gic_cnt, -1, dist_base, cpu_base, percpu_offset, node);
>         if (!gic_cnt)
>                 gic_init_physaddr(node);
> diff --git a/drivers/irqchip/irq-gic.h b/drivers/irqchip/irq-gic.h
> new file mode 100644
> index 0000000..1f4a2b0
> --- /dev/null
> +++ b/drivers/irqchip/irq-gic.h
> @@ -0,0 +1,51 @@
> +#ifndef _IRQ_GIC_H_
> +#define _IRQ_GIC_H_
> +
> +#include <linux/msi.h>
> +#include <linux/pci.h>
> +
> +union gic_base {
> +       void __iomem *common_base;
> +       void __percpu * __iomem *percpu_base;
> +};
> +
> +struct gic_chip_data;
> +
> +struct v2m_data {
> +#ifdef CONFIG_ARM_GIC_V2M
> +       struct list_head list;
> +       spinlock_t msi_cnt_lock;
> +       struct msi_chip msi_chip;
> +       struct resource res;      /* GICv2m resource */
> +       void __iomem *base;       /* GICv2m virt address */
> +       unsigned int spi_start;   /* The SPI number that MSIs start */
> +       unsigned int nr_spis;     /* The number of SPIs for MSIs */
> +       unsigned long *bm;        /* MSI vector bitmap */
> +       struct gic_chip_data *gic;
> +#endif
> +};
> +
> +struct gic_chip_data {
> +       union gic_base dist_base;
> +       union gic_base cpu_base;
> +#ifdef CONFIG_CPU_PM
> +       u32 saved_spi_enable[DIV_ROUND_UP(1020, 32)];
> +       u32 saved_spi_conf[DIV_ROUND_UP(1020, 16)];
> +       u32 saved_spi_target[DIV_ROUND_UP(1020, 4)];
> +       u32 __percpu *saved_ppi_enable;
> +       u32 __percpu *saved_ppi_conf;
> +#endif
> +       struct irq_domain *domain;
> +       unsigned int gic_irqs;
> +#ifdef CONFIG_GIC_NON_BANKED
> +       void __iomem *(*get_base)(union gic_base *);
> +#endif
> +       struct list_head v2m_list;
> +};

Maybe wrap this list_head around #ifdef/#endif?

> +
> +void gic_mask_irq(struct irq_data *d);
> +void gic_unmask_irq(struct irq_data *d);
> +int gicv2m_of_init(struct device_node *node, struct gic_chip_data *gic,
> +                  struct irq_chip *v2m_chip) __init;
> +
> +#endif /* _IRQ_GIC_H_ */
> --
> 1.9.3
>
>
-- 
Jazz is not dead. It just smells funny.

WARNING: multiple messages have this Message-ID (diff)
From: marc.zyngier@arm.com (Marc Zyngier)
To: linux-arm-kernel@lists.infradead.org
Subject: [PATCH 1/2 V6] irqchip: gic: Add supports for ARM GICv2m MSI(-X)
Date: Mon, 15 Sep 2014 20:26:20 +0100	[thread overview]
Message-ID: <86iokon0rn.fsf@arm.com> (raw)
In-Reply-To: <1410676216-27953-2-git-send-email-suravee.suthikulpanit@amd.com> (suravee's message of "Sun, 14 Sep 2014 07:30:15 +0100")

On Sun, Sep 14 2014 at 07:30:15 AM, "suravee.suthikulpanit at amd.com" <suravee.suthikulpanit@amd.com> wrote:

Hi Suravee,

> From: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
>
> ARM GICv2m specification extends GICv2 to support MSI(-X) with
> a new set of register frame. This patch introduces support for
> the non-secure GICv2m register frame. Currently, GICV2m is available
> in certain version of GIC-400.
>
> The patch introduces a new property in ARM gic binding, the v2m subnode.
> It is optional.

Overall, this looks much better than the previous versions I've
reviewed. There is still a number of points I'd like to see addressed,
see below.

>
> Signed-off-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
> Cc: Mark Rutland <Mark.Rutland@arm.com>
> Cc: Marc Zyngier <Marc.Zyngier@arm.com>
> Cc: Jason Cooper <jason@lakedaemon.net>
> Cc: Catalin Marinas <Catalin.Marinas@arm.com>
> Cc: Will Deacon <Will.Deacon@arm.com>
> ---
>  Documentation/devicetree/bindings/arm/gic.txt |  39 ++++
>  drivers/irqchip/Kconfig                       |   7 +
>  drivers/irqchip/Makefile                      |   1 +
>  drivers/irqchip/irq-gic-v2m.c                 | 261 ++++++++++++++++++++++++++
>  drivers/irqchip/irq-gic.c                     |  88 ++++++---
>  drivers/irqchip/irq-gic.h                     |  51 +++++
>  6 files changed, 419 insertions(+), 28 deletions(-)
>  create mode 100644 drivers/irqchip/irq-gic-v2m.c
>  create mode 100644 drivers/irqchip/irq-gic.h
>
> diff --git a/Documentation/devicetree/bindings/arm/gic.txt b/Documentation/devicetree/bindings/arm/gic.txt
> index c7d2fa1..a461591 100644
> --- a/Documentation/devicetree/bindings/arm/gic.txt
> +++ b/Documentation/devicetree/bindings/arm/gic.txt
> @@ -96,3 +96,42 @@ Example:
>                       <0x2c006000 0x2000>;
>                 interrupts = <1 9 0xf04>;
>         };
> +
> +
> +* GICv2m extension for MSI/MSI-x support (Optional)
> +
> +Certain revision of GIC-400 supports MSI/MSI-x via V2M register frame.
> +This is enabled by specifying v2m sub-node.
> +
> +Required properties:
> +
> +- msi-controller : Identifies the node as an MSI controller.
> +
> +- reg : GICv2m MSI interface register base and size
> +
> +Example:
> +
> +       interrupt-controller at e1101000 {
> +               compatible = "arm,gic-400";
> +               #interrupt-cells = <3>;
> +               #address-cells = <2>;
> +               #size-cells = <2>;
> +               interrupt-controller;
> +               interrupts = <1 8 0xf04>;
> +               ranges = <0 0 0 0xe1100000 0 0x100000>;
> +               reg = <0x0 0xe1110000 0 0x01000>,
> +                     <0x0 0xe112f000 0 0x02000>,
> +                     <0x0 0xe1140000 0 0x10000>,
> +                     <0x0 0xe1160000 0 0x10000>;
> +               v2m0 {

I believe the convention is to have
		v2m0 : v2m at 0x80000 {

> +                       msi-controller;
> +                       reg = <0x0 0x80000 0 0x1000>;

I think having a "compatible" property would be nice, and would help
handle variants if they ever happen.

> +               };
> +
> +               ....
> +
> +               v2mN {
> +                       msi-controller;
> +                       reg = <0x0 0x90000 0 0x1000>;
> +               };
> +       };
> diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
> index b8632bf..ddb31ee 100644
> --- a/drivers/irqchip/Kconfig
> +++ b/drivers/irqchip/Kconfig
> @@ -7,6 +7,13 @@ config ARM_GIC
>         select IRQ_DOMAIN
>         select MULTI_IRQ_HANDLER
>
> +config ARM_GIC_V2M
> +       bool
> +       select IRQ_DOMAIN
> +       select MULTI_IRQ_HANDLER
> +       depends on ARM_GIC
> +       depends on PCI && PCI_MSI
> +
>  config GIC_NON_BANKED
>         bool
>
> diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
> index 73052ba..3bda951 100644
> --- a/drivers/irqchip/Makefile
> +++ b/drivers/irqchip/Makefile
> @@ -17,6 +17,7 @@ obj-$(CONFIG_ARCH_SUNXI)              += irq-sun4i.o
>  obj-$(CONFIG_ARCH_SUNXI)               += irq-sunxi-nmi.o
>  obj-$(CONFIG_ARCH_SPEAR3XX)            += spear-shirq.o
>  obj-$(CONFIG_ARM_GIC)                  += irq-gic.o irq-gic-common.o
> +obj-$(CONFIG_ARM_GIC_V2M)              += irq-gic-v2m.o
>  obj-$(CONFIG_ARM_GIC_V3)               += irq-gic-v3.o irq-gic-common.o
>  obj-$(CONFIG_ARM_NVIC)                 += irq-nvic.o
>  obj-$(CONFIG_ARM_VIC)                  += irq-vic.o
> diff --git a/drivers/irqchip/irq-gic-v2m.c b/drivers/irqchip/irq-gic-v2m.c
> new file mode 100644
> index 0000000..499bfb9
> --- /dev/null
> +++ b/drivers/irqchip/irq-gic-v2m.c
> @@ -0,0 +1,261 @@
> +/*
> + * ARM GIC v2m MSI(-X) support
> + * Support for Message Signaled Interrupts for systems that
> + * implement ARM Generic Interrupt Controller: GICv2m.
> + *
> + * Copyright (C) 2014 Advanced Micro Devices, Inc.
> + * Authors: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
> + *          Harish Kasiviswanathan <harish.kasiviswanathan@amd.com>
> + *          Brandon Anderson <brandon.anderson@amd.com>
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License version 2 as published
> + * by the Free Software Foundation.
> + */
> +
> +#define pr_fmt(fmt) "GICv2m: " fmt
> +
> +#include <linux/bitmap.h>
> +#include <linux/irq.h>
> +#include <linux/kernel.h>
> +#include <linux/module.h>
> +#include <linux/of_address.h>
> +#include <linux/of_pci.h>
> +#include <linux/pci.h>
> +#include <linux/platform_device.h>
> +#include <linux/slab.h>
> +#include <linux/spinlock.h>
> +
> +#include "irqchip.h"
> +#include "irq-gic.h"
> +
> +/*
> +* MSI_TYPER:
> +*     [31:26] Reserved
> +*     [25:16] lowest SPI assigned to MSI
> +*     [15:10] Reserved
> +*     [9:0]   Numer of SPIs assigned to MSI
> +*/
> +#define V2M_MSI_TYPER                  0x008
> +#define V2M_MSI_TYPER_BASE_SHIFT       16
> +#define V2M_MSI_TYPER_BASE_MASK                0x3FF
> +#define V2M_MSI_TYPER_NUM_MASK         0x3FF
> +#define V2M_MSI_SETSPI_NS              0x040
> +#define V2M_MIN_SPI                    32
> +#define V2M_MAX_SPI                    1019
> +
> +/*
> + * alloc_msi_irq - Allocate MSIs from available MSI bitmap.
> + * @data: Pointer to v2m_data
> + * @nvec: Number of interrupts to allocate
> + * @irq: Pointer to the allocated irq
> + *
> + * Allocates interrupts only if the contiguous range of MSIs
> + * with specified nvec are available. Otherwise return the number
> + * of available interrupts. If none are available, then returns -ENOENT.
> + */
> +static int alloc_msi_irq(struct v2m_data *data, int nvec, int *irq)
> +{
> +       int size = data->nr_spis;
> +       int next = size, i = nvec, ret;
> +
> +       /* We should never allocate more than available nr_spis */
> +       if (i >= size)
> +               i = size;
> +
> +       spin_lock(&data->msi_cnt_lock);
> +
> +       for (; i > 0; i--) {
> +               next = bitmap_find_next_zero_area(data->bm,
> +                                       size, 0, i, 0);
> +               if (next < size)
> +                       break;
> +       }
> +
> +       if (i != nvec) {
> +               ret = i ? : -ENOENT;
> +       } else {
> +               bitmap_set(data->bm, next, nvec);
> +               *irq = data->spi_start + next;
> +               ret = 0;
> +       }
> +
> +       spin_unlock(&data->msi_cnt_lock);
> +
> +       return ret;
> +}
> +
> +static void gicv2m_teardown_msi_irq(struct msi_chip *chip, unsigned int irq)
> +{
> +       int pos;
> +       struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
> +
> +       spin_lock(&data->msi_cnt_lock);
> +
> +       pos = irq - data->spi_start;
> +       if (pos >= 0 && pos < data->nr_spis)
> +               bitmap_clear(data->bm, pos, 1);
> +
> +       spin_unlock(&data->msi_cnt_lock);
> +}
> +
> +static int gicv2m_setup_msi_irq(struct msi_chip *chip,
> +                               struct pci_dev *pdev,
> +                               struct msi_desc *desc)
> +{
> +       int irq, avail;
> +       struct msi_msg msg;
> +       phys_addr_t addr;
> +       struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
> +
> +       if (!desc) {
> +               dev_err(&pdev->dev,
> +                       "GICv2m: MSI setup failed. Invalid msi descriptor\n");

Isn't the "GICv2m" header already provided by the pr_fmt at the
beginning of the file?

> +               return -EINVAL;
> +       }
> +
> +       avail = alloc_msi_irq(data, 1, &irq);
> +       if (avail != 0) {
> +               dev_err(&pdev->dev,
> +                       "GICv2m: MSI setup failed. Cannnot allocate IRQ\n");
> +               return -ENOSPC;
> +       }
> +
> +       irq_set_chip_data(irq, chip);
> +       irq_set_msi_desc(irq, desc);
> +       irq_set_irq_type(irq, IRQ_TYPE_EDGE_RISING);
> +
> +       addr = data->res.start + V2M_MSI_SETSPI_NS;
> +       msg.address_hi = (u32)(addr >> 32);
> +       msg.address_lo = (u32)(addr);
> +       msg.data = irq;
> +       write_msi_msg(irq, &msg);
> +
> +       return 0;
> +}
> +
> +static void gicv2m_mask_irq(struct irq_data *d)
> +{
> +       gic_mask_irq(d);
> +       if (d->msi_desc)
> +               mask_msi_irq(d);
> +}
> +
> +static void gicv2m_unmask_irq(struct irq_data *d)
> +{
> +       gic_unmask_irq(d);
> +       if (d->msi_desc)
> +               unmask_msi_irq(d);
> +}
> +
> +#ifdef CONFIG_OF

I think you can loose the CONFIG_OF here. There is no valid
configuration without OF on arm64.

> +
> +static int __init
> +v2m_init(struct device_node *node, struct v2m_data *v2m,
> +                 struct resource *res)
> +{
> +       int ret;
> +       unsigned int typer;
> +
> +       v2m->msi_chip.owner = THIS_MODULE;
> +       v2m->msi_chip.of_node = node;
> +       v2m->msi_chip.setup_irq = gicv2m_setup_msi_irq;
> +       v2m->msi_chip.teardown_irq = gicv2m_teardown_msi_irq;
> +       ret = of_pci_msi_chip_add(&v2m->msi_chip);
> +       if (ret) {
> +               pr_info("Failed to add msi_chip.\n");
> +               return ret;
> +       }
> +
> +       v2m->base = ioremap(res->start, resource_size(res));
> +       if (!v2m->base) {
> +               pr_err("Failed to map GICv2m resource\n");
> +               return -EINVAL;

Beware, you're leaving with a registered msi_chip. You probably want to
remove it on the error path.

> +       }
> +
> +       memcpy(&v2m->res, res, sizeof(struct resource));
> +
> +       typer = readl_relaxed(v2m->base + V2M_MSI_TYPER);
> +       if (!typer) {
> +               pr_warn("Failed to read V2M_MSI_TYPER register\n");
> +               return -EINVAL;

And here leaving the v2m widget mapped.

> +       }
> +
> +       v2m->spi_start = (typer >> V2M_MSI_TYPER_BASE_SHIFT) &
> +                               V2M_MSI_TYPER_BASE_MASK;
> +       v2m->nr_spis = typer & V2M_MSI_TYPER_NUM_MASK;
> +       if ((v2m->spi_start < V2M_MIN_SPI) || (v2m->nr_spis >= V2M_MAX_SPI)) {
> +                       pr_err("Invalid MSI_TYPER (%#x)\n", typer);
> +                       return -EINVAL;

Blah. The whole error handling of this function is b0rked.

> +       }
> +
> +       v2m->bm = kzalloc(sizeof(long) * BITS_TO_LONGS(v2m->nr_spis),
> +                         GFP_KERNEL);
> +       if (!v2m->bm) {
> +               pr_err("Failed to allocate MSI bitmap\n");
> +               return -ENOMEM;
> +       }
> +
> +       spin_lock_init(&v2m->msi_cnt_lock);
> +
> +       pr_info("SPI range [%d:%d]\n",
> +               v2m->spi_start, (v2m->spi_start + v2m->nr_spis));
> +
> +       return 0;
> +}
> +
> +int __init gicv2m_of_init(struct device_node *node,
> +                         struct gic_chip_data *gic,
> +                         struct irq_chip *v2m_chip)
> +{
> +       int ret = 0;
> +       struct v2m_data *v2m;
> +       struct resource res;
> +       struct device_node *child = NULL;
> +
> +       INIT_LIST_HEAD(&gic->v2m_list);
> +
> +       for (;;) {
> +               child = of_get_next_child(node, child);

I don't see any of_node_put() in this function. You'll need to fix
this...

> +               if (!child)
> +                       break;
> +
> +               if (!of_find_property(child, "msi-controller", NULL))
> +                       continue;
> +
> +               ret = of_address_to_resource(child, 0, &res);
> +               if (ret) {
> +                       pr_err("Failed to allocate v2m resource.\n");
> +                       break;
> +               }
> +
> +               v2m = kzalloc(sizeof(struct v2m_data), GFP_KERNEL);
> +               if (!v2m) {
> +                       pr_err("Failed to allocate struct v2m_data.\n");
> +                       return -ENOMEM;
> +               }
> +
> +               ret = v2m_init(child, v2m, &res);
> +               if (ret)
> +                       break;
> +
> +               v2m->gic = gic;
> +
> +               pr_info(" %s: [%lx : %lx]\n", child->name,
> +                       (unsigned long)res.start, (unsigned long)res.end);
> +
> +               list_add_tail(&v2m->list, &gic->v2m_list);
> +       }
> +
> +       if (ret && list_empty(&gic->v2m_list)) {
> +               pr_err("MSI resource not found\n");
> +               return -EINVAL;
> +       }
> +
> +       v2m_chip->irq_mask = gicv2m_mask_irq;
> +       v2m_chip->irq_unmask = gicv2m_unmask_irq;
> +

Again, the whole error handling is evil. You can end-up allocating a few
v2m structures, registering them, and yet have kzalloc failing at some
point. At that stage, you exit early, not setting the mas/unmask
methods. Boom.

> +       return ret;
> +}
> +
> +#endif /* CONFIG_OF */
> diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
> index 4b959e6..f490599 100644
> --- a/drivers/irqchip/irq-gic.c
> +++ b/drivers/irqchip/irq-gic.c
> @@ -46,30 +46,9 @@
>  #include <asm/smp_plat.h>
>
>  #include "irq-gic-common.h"
> +#include "irq-gic.h"
>  #include "irqchip.h"
>
> -union gic_base {
> -       void __iomem *common_base;
> -       void __percpu * __iomem *percpu_base;
> -};
> -
> -struct gic_chip_data {
> -       union gic_base dist_base;
> -       union gic_base cpu_base;
> -#ifdef CONFIG_CPU_PM
> -       u32 saved_spi_enable[DIV_ROUND_UP(1020, 32)];
> -       u32 saved_spi_conf[DIV_ROUND_UP(1020, 16)];
> -       u32 saved_spi_target[DIV_ROUND_UP(1020, 4)];
> -       u32 __percpu *saved_ppi_enable;
> -       u32 __percpu *saved_ppi_conf;
> -#endif
> -       struct irq_domain *domain;
> -       unsigned int gic_irqs;
> -#ifdef CONFIG_GIC_NON_BANKED
> -       void __iomem *(*get_base)(union gic_base *);
> -#endif
> -};
> -
>  static DEFINE_RAW_SPINLOCK(irq_controller_lock);
>
>  /*
> @@ -131,15 +110,36 @@ static inline void gic_set_base_accessor(struct gic_chip_data *data,
>  #define gic_set_base_accessor(d, f)
>  #endif
>
> +static inline
> +struct gic_chip_data *irq_data_get_gic_chip_data(struct irq_data *d)
> +{
> +       struct gic_chip_data *gic_data;
> +       struct msi_chip *mchip;
> +       struct v2m_data *v2mdat;
> +
> +       /*
> +        * For MSI, irq_data.chip_data points to struct msi_chip.
> +        * For non-MSI, irq_data.chip_data points to struct gic_chip_data.
> +        */
> +       if (d->msi_desc) {
> +               mchip = irq_data_get_irq_chip_data(d);
> +               v2mdat = container_of(mchip, struct v2m_data, msi_chip);
> +               gic_data = v2mdat->gic;
> +       } else {
> +               gic_data = irq_data_get_irq_chip_data(d);
> +       }
> +       return gic_data;
> +}
> +
>  static inline void __iomem *gic_dist_base(struct irq_data *d)
>  {
> -       struct gic_chip_data *gic_data = irq_data_get_irq_chip_data(d);
> +       struct gic_chip_data *gic_data = irq_data_get_gic_chip_data(d);
>         return gic_data_dist_base(gic_data);
>  }
>
>  static inline void __iomem *gic_cpu_base(struct irq_data *d)
>  {
> -       struct gic_chip_data *gic_data = irq_data_get_irq_chip_data(d);
> +       struct gic_chip_data *gic_data = irq_data_get_gic_chip_data(d);
>         return gic_data_cpu_base(gic_data);
>  }
>
> @@ -151,7 +151,7 @@ static inline unsigned int gic_irq(struct irq_data *d)
>  /*
>   * Routines to acknowledge, disable and enable interrupts
>   */
> -static void gic_mask_irq(struct irq_data *d)
> +void gic_mask_irq(struct irq_data *d)

I'm still very uncomfortable with exporting the mask/unmask methods
outside of this file... Maybe it is not a big deal, but I see evil
everywhere.

>  {
>         u32 mask = 1 << (gic_irq(d) % 32);
>
> @@ -162,7 +162,7 @@ static void gic_mask_irq(struct irq_data *d)
>         raw_spin_unlock(&irq_controller_lock);
>  }
>
> -static void gic_unmask_irq(struct irq_data *d)
> +void gic_unmask_irq(struct irq_data *d)
>  {
>         u32 mask = 1 << (gic_irq(d) % 32);
>
> @@ -325,6 +325,15 @@ static struct irq_chip gic_chip = {
>         .irq_set_wake           = gic_set_wake,
>  };
>
> +static struct irq_chip v2m_chip = {
> +       .name                   = "GICv2m",
> +       .irq_eoi                = gic_eoi_irq,
> +       .irq_set_type           = gic_set_type,
> +#ifdef CONFIG_SMP
> +       .irq_set_affinity       = gic_set_affinity,
> +#endif
> +};
> +
>  void __init gic_cascade_irq(unsigned int gic_nr, unsigned int irq)
>  {
>         if (gic_nr >= MAX_GIC_NR)
> @@ -767,6 +776,24 @@ void __init gic_init_physaddr(struct device_node *node)
>  static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
>                                 irq_hw_number_t hw)
>  {
> +       struct gic_chip_data *gic = d->host_data;
> +       struct v2m_data *v2m;
> +
> +       irq_set_chip_data(irq, gic);
>
> +#ifdef CONFIG_ARM_GIC_V2M
> +       list_for_each_entry(v2m, &gic->v2m_list, list) {
> +               if (hw >= v2m->spi_start &&
> +                   hw <  v2m->spi_start + v2m->nr_spis) {
> +
> +                       irq_set_chip_and_handler(irq, &v2m_chip,
> +                                        handle_fasteoi_irq);
> +                       set_irq_flags(irq, IRQF_VALID | IRQF_PROBE);
> +                       return 0;
> +               }
> +       }
> +#endif
> +
>         if (hw < 32) {
>                 irq_set_percpu_devid(irq);
>                 irq_set_chip_and_handler(irq, &gic_chip,
> @@ -779,7 +806,6 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
>
>                 gic_routable_irq_domain_ops->map(d, irq, hw);
>         }
> -       irq_set_chip_data(irq, d->host_data);

These two hunks make the code a bit hard to read:
- First, message-based SPIs,
- Then PPIs
- and finally the rest of the SPIs.

Can you come up with a construct that keeps everything in a logical
order (PPIs, and then SPIs)? Not a big deal but it would make the code
nicer.

>         return 0;
>  }
>
> @@ -996,7 +1022,7 @@ gic_of_init(struct device_node *node, struct device_node *parent)
>         void __iomem *cpu_base;
>         void __iomem *dist_base;
>         u32 percpu_offset;
> -       int irq;
> +       int irq, ret;
>
>         if (WARN_ON(!node))
>                 return -ENODEV;
> @@ -1010,6 +1036,12 @@ gic_of_init(struct device_node *node, struct device_node *parent)
>         if (of_property_read_u32(node, "cpu-offset", &percpu_offset))
>                 percpu_offset = 0;
>
> +       if (IS_ENABLED(CONFIG_ARM_GIC_V2M)) {
> +               ret = gicv2m_of_init(node, &gic_data[gic_cnt], &v2m_chip);
> +               if (ret)
> +                       pr_warn("Warning: Failed to enable GICv2m support.\n");
> +       }
> +
>         gic_init_bases(gic_cnt, -1, dist_base, cpu_base, percpu_offset, node);
>         if (!gic_cnt)
>                 gic_init_physaddr(node);
> diff --git a/drivers/irqchip/irq-gic.h b/drivers/irqchip/irq-gic.h
> new file mode 100644
> index 0000000..1f4a2b0
> --- /dev/null
> +++ b/drivers/irqchip/irq-gic.h
> @@ -0,0 +1,51 @@
> +#ifndef _IRQ_GIC_H_
> +#define _IRQ_GIC_H_
> +
> +#include <linux/msi.h>
> +#include <linux/pci.h>
> +
> +union gic_base {
> +       void __iomem *common_base;
> +       void __percpu * __iomem *percpu_base;
> +};
> +
> +struct gic_chip_data;
> +
> +struct v2m_data {
> +#ifdef CONFIG_ARM_GIC_V2M
> +       struct list_head list;
> +       spinlock_t msi_cnt_lock;
> +       struct msi_chip msi_chip;
> +       struct resource res;      /* GICv2m resource */
> +       void __iomem *base;       /* GICv2m virt address */
> +       unsigned int spi_start;   /* The SPI number that MSIs start */
> +       unsigned int nr_spis;     /* The number of SPIs for MSIs */
> +       unsigned long *bm;        /* MSI vector bitmap */
> +       struct gic_chip_data *gic;
> +#endif
> +};
> +
> +struct gic_chip_data {
> +       union gic_base dist_base;
> +       union gic_base cpu_base;
> +#ifdef CONFIG_CPU_PM
> +       u32 saved_spi_enable[DIV_ROUND_UP(1020, 32)];
> +       u32 saved_spi_conf[DIV_ROUND_UP(1020, 16)];
> +       u32 saved_spi_target[DIV_ROUND_UP(1020, 4)];
> +       u32 __percpu *saved_ppi_enable;
> +       u32 __percpu *saved_ppi_conf;
> +#endif
> +       struct irq_domain *domain;
> +       unsigned int gic_irqs;
> +#ifdef CONFIG_GIC_NON_BANKED
> +       void __iomem *(*get_base)(union gic_base *);
> +#endif
> +       struct list_head v2m_list;
> +};

Maybe wrap this list_head around #ifdef/#endif?

> +
> +void gic_mask_irq(struct irq_data *d);
> +void gic_unmask_irq(struct irq_data *d);
> +int gicv2m_of_init(struct device_node *node, struct gic_chip_data *gic,
> +                  struct irq_chip *v2m_chip) __init;
> +
> +#endif /* _IRQ_GIC_H_ */
> --
> 1.9.3
>
>
-- 
Jazz is not dead. It just smells funny.

WARNING: multiple messages have this Message-ID (diff)
From: Marc Zyngier <marc.zyngier@arm.com>
To: "suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com>
Cc: Mark Rutland <Mark.Rutland@arm.com>,
	"jason@lakedaemon.net" <jason@lakedaemon.net>,
	Pawel Moll <Pawel.Moll@arm.com>,
	Catalin Marinas <Catalin.Marinas@arm.com>,
	Will Deacon <Will.Deacon@arm.com>,
	"tglx@linutronix.de" <tglx@linutronix.de>,
	"Harish.Kasiviswanathan@amd.com" <Harish.Kasiviswanathan@amd.com>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>,
	"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-doc@vger.kernel.org" <linux-doc@vger.kernel.org>,
	"devicetree@vger.kernel.org" <devicetree@vger.kernel.org>
Subject: Re: [PATCH 1/2 V6] irqchip: gic: Add supports for ARM GICv2m MSI(-X)
Date: Mon, 15 Sep 2014 20:26:20 +0100	[thread overview]
Message-ID: <86iokon0rn.fsf@arm.com> (raw)
In-Reply-To: <1410676216-27953-2-git-send-email-suravee.suthikulpanit@amd.com> (suravee's message of "Sun, 14 Sep 2014 07:30:15 +0100")

On Sun, Sep 14 2014 at 07:30:15 AM, "suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com> wrote:

Hi Suravee,

> From: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
>
> ARM GICv2m specification extends GICv2 to support MSI(-X) with
> a new set of register frame. This patch introduces support for
> the non-secure GICv2m register frame. Currently, GICV2m is available
> in certain version of GIC-400.
>
> The patch introduces a new property in ARM gic binding, the v2m subnode.
> It is optional.

Overall, this looks much better than the previous versions I've
reviewed. There is still a number of points I'd like to see addressed,
see below.

>
> Signed-off-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
> Cc: Mark Rutland <Mark.Rutland@arm.com>
> Cc: Marc Zyngier <Marc.Zyngier@arm.com>
> Cc: Jason Cooper <jason@lakedaemon.net>
> Cc: Catalin Marinas <Catalin.Marinas@arm.com>
> Cc: Will Deacon <Will.Deacon@arm.com>
> ---
>  Documentation/devicetree/bindings/arm/gic.txt |  39 ++++
>  drivers/irqchip/Kconfig                       |   7 +
>  drivers/irqchip/Makefile                      |   1 +
>  drivers/irqchip/irq-gic-v2m.c                 | 261 ++++++++++++++++++++++++++
>  drivers/irqchip/irq-gic.c                     |  88 ++++++---
>  drivers/irqchip/irq-gic.h                     |  51 +++++
>  6 files changed, 419 insertions(+), 28 deletions(-)
>  create mode 100644 drivers/irqchip/irq-gic-v2m.c
>  create mode 100644 drivers/irqchip/irq-gic.h
>
> diff --git a/Documentation/devicetree/bindings/arm/gic.txt b/Documentation/devicetree/bindings/arm/gic.txt
> index c7d2fa1..a461591 100644
> --- a/Documentation/devicetree/bindings/arm/gic.txt
> +++ b/Documentation/devicetree/bindings/arm/gic.txt
> @@ -96,3 +96,42 @@ Example:
>                       <0x2c006000 0x2000>;
>                 interrupts = <1 9 0xf04>;
>         };
> +
> +
> +* GICv2m extension for MSI/MSI-x support (Optional)
> +
> +Certain revision of GIC-400 supports MSI/MSI-x via V2M register frame.
> +This is enabled by specifying v2m sub-node.
> +
> +Required properties:
> +
> +- msi-controller : Identifies the node as an MSI controller.
> +
> +- reg : GICv2m MSI interface register base and size
> +
> +Example:
> +
> +       interrupt-controller@e1101000 {
> +               compatible = "arm,gic-400";
> +               #interrupt-cells = <3>;
> +               #address-cells = <2>;
> +               #size-cells = <2>;
> +               interrupt-controller;
> +               interrupts = <1 8 0xf04>;
> +               ranges = <0 0 0 0xe1100000 0 0x100000>;
> +               reg = <0x0 0xe1110000 0 0x01000>,
> +                     <0x0 0xe112f000 0 0x02000>,
> +                     <0x0 0xe1140000 0 0x10000>,
> +                     <0x0 0xe1160000 0 0x10000>;
> +               v2m0 {

I believe the convention is to have
		v2m0 : v2m@0x80000 {

> +                       msi-controller;
> +                       reg = <0x0 0x80000 0 0x1000>;

I think having a "compatible" property would be nice, and would help
handle variants if they ever happen.

> +               };
> +
> +               ....
> +
> +               v2mN {
> +                       msi-controller;
> +                       reg = <0x0 0x90000 0 0x1000>;
> +               };
> +       };
> diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
> index b8632bf..ddb31ee 100644
> --- a/drivers/irqchip/Kconfig
> +++ b/drivers/irqchip/Kconfig
> @@ -7,6 +7,13 @@ config ARM_GIC
>         select IRQ_DOMAIN
>         select MULTI_IRQ_HANDLER
>
> +config ARM_GIC_V2M
> +       bool
> +       select IRQ_DOMAIN
> +       select MULTI_IRQ_HANDLER
> +       depends on ARM_GIC
> +       depends on PCI && PCI_MSI
> +
>  config GIC_NON_BANKED
>         bool
>
> diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
> index 73052ba..3bda951 100644
> --- a/drivers/irqchip/Makefile
> +++ b/drivers/irqchip/Makefile
> @@ -17,6 +17,7 @@ obj-$(CONFIG_ARCH_SUNXI)              += irq-sun4i.o
>  obj-$(CONFIG_ARCH_SUNXI)               += irq-sunxi-nmi.o
>  obj-$(CONFIG_ARCH_SPEAR3XX)            += spear-shirq.o
>  obj-$(CONFIG_ARM_GIC)                  += irq-gic.o irq-gic-common.o
> +obj-$(CONFIG_ARM_GIC_V2M)              += irq-gic-v2m.o
>  obj-$(CONFIG_ARM_GIC_V3)               += irq-gic-v3.o irq-gic-common.o
>  obj-$(CONFIG_ARM_NVIC)                 += irq-nvic.o
>  obj-$(CONFIG_ARM_VIC)                  += irq-vic.o
> diff --git a/drivers/irqchip/irq-gic-v2m.c b/drivers/irqchip/irq-gic-v2m.c
> new file mode 100644
> index 0000000..499bfb9
> --- /dev/null
> +++ b/drivers/irqchip/irq-gic-v2m.c
> @@ -0,0 +1,261 @@
> +/*
> + * ARM GIC v2m MSI(-X) support
> + * Support for Message Signaled Interrupts for systems that
> + * implement ARM Generic Interrupt Controller: GICv2m.
> + *
> + * Copyright (C) 2014 Advanced Micro Devices, Inc.
> + * Authors: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
> + *          Harish Kasiviswanathan <harish.kasiviswanathan@amd.com>
> + *          Brandon Anderson <brandon.anderson@amd.com>
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License version 2 as published
> + * by the Free Software Foundation.
> + */
> +
> +#define pr_fmt(fmt) "GICv2m: " fmt
> +
> +#include <linux/bitmap.h>
> +#include <linux/irq.h>
> +#include <linux/kernel.h>
> +#include <linux/module.h>
> +#include <linux/of_address.h>
> +#include <linux/of_pci.h>
> +#include <linux/pci.h>
> +#include <linux/platform_device.h>
> +#include <linux/slab.h>
> +#include <linux/spinlock.h>
> +
> +#include "irqchip.h"
> +#include "irq-gic.h"
> +
> +/*
> +* MSI_TYPER:
> +*     [31:26] Reserved
> +*     [25:16] lowest SPI assigned to MSI
> +*     [15:10] Reserved
> +*     [9:0]   Numer of SPIs assigned to MSI
> +*/
> +#define V2M_MSI_TYPER                  0x008
> +#define V2M_MSI_TYPER_BASE_SHIFT       16
> +#define V2M_MSI_TYPER_BASE_MASK                0x3FF
> +#define V2M_MSI_TYPER_NUM_MASK         0x3FF
> +#define V2M_MSI_SETSPI_NS              0x040
> +#define V2M_MIN_SPI                    32
> +#define V2M_MAX_SPI                    1019
> +
> +/*
> + * alloc_msi_irq - Allocate MSIs from available MSI bitmap.
> + * @data: Pointer to v2m_data
> + * @nvec: Number of interrupts to allocate
> + * @irq: Pointer to the allocated irq
> + *
> + * Allocates interrupts only if the contiguous range of MSIs
> + * with specified nvec are available. Otherwise return the number
> + * of available interrupts. If none are available, then returns -ENOENT.
> + */
> +static int alloc_msi_irq(struct v2m_data *data, int nvec, int *irq)
> +{
> +       int size = data->nr_spis;
> +       int next = size, i = nvec, ret;
> +
> +       /* We should never allocate more than available nr_spis */
> +       if (i >= size)
> +               i = size;
> +
> +       spin_lock(&data->msi_cnt_lock);
> +
> +       for (; i > 0; i--) {
> +               next = bitmap_find_next_zero_area(data->bm,
> +                                       size, 0, i, 0);
> +               if (next < size)
> +                       break;
> +       }
> +
> +       if (i != nvec) {
> +               ret = i ? : -ENOENT;
> +       } else {
> +               bitmap_set(data->bm, next, nvec);
> +               *irq = data->spi_start + next;
> +               ret = 0;
> +       }
> +
> +       spin_unlock(&data->msi_cnt_lock);
> +
> +       return ret;
> +}
> +
> +static void gicv2m_teardown_msi_irq(struct msi_chip *chip, unsigned int irq)
> +{
> +       int pos;
> +       struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
> +
> +       spin_lock(&data->msi_cnt_lock);
> +
> +       pos = irq - data->spi_start;
> +       if (pos >= 0 && pos < data->nr_spis)
> +               bitmap_clear(data->bm, pos, 1);
> +
> +       spin_unlock(&data->msi_cnt_lock);
> +}
> +
> +static int gicv2m_setup_msi_irq(struct msi_chip *chip,
> +                               struct pci_dev *pdev,
> +                               struct msi_desc *desc)
> +{
> +       int irq, avail;
> +       struct msi_msg msg;
> +       phys_addr_t addr;
> +       struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
> +
> +       if (!desc) {
> +               dev_err(&pdev->dev,
> +                       "GICv2m: MSI setup failed. Invalid msi descriptor\n");

Isn't the "GICv2m" header already provided by the pr_fmt at the
beginning of the file?

> +               return -EINVAL;
> +       }
> +
> +       avail = alloc_msi_irq(data, 1, &irq);
> +       if (avail != 0) {
> +               dev_err(&pdev->dev,
> +                       "GICv2m: MSI setup failed. Cannnot allocate IRQ\n");
> +               return -ENOSPC;
> +       }
> +
> +       irq_set_chip_data(irq, chip);
> +       irq_set_msi_desc(irq, desc);
> +       irq_set_irq_type(irq, IRQ_TYPE_EDGE_RISING);
> +
> +       addr = data->res.start + V2M_MSI_SETSPI_NS;
> +       msg.address_hi = (u32)(addr >> 32);
> +       msg.address_lo = (u32)(addr);
> +       msg.data = irq;
> +       write_msi_msg(irq, &msg);
> +
> +       return 0;
> +}
> +
> +static void gicv2m_mask_irq(struct irq_data *d)
> +{
> +       gic_mask_irq(d);
> +       if (d->msi_desc)
> +               mask_msi_irq(d);
> +}
> +
> +static void gicv2m_unmask_irq(struct irq_data *d)
> +{
> +       gic_unmask_irq(d);
> +       if (d->msi_desc)
> +               unmask_msi_irq(d);
> +}
> +
> +#ifdef CONFIG_OF

I think you can loose the CONFIG_OF here. There is no valid
configuration without OF on arm64.

> +
> +static int __init
> +v2m_init(struct device_node *node, struct v2m_data *v2m,
> +                 struct resource *res)
> +{
> +       int ret;
> +       unsigned int typer;
> +
> +       v2m->msi_chip.owner = THIS_MODULE;
> +       v2m->msi_chip.of_node = node;
> +       v2m->msi_chip.setup_irq = gicv2m_setup_msi_irq;
> +       v2m->msi_chip.teardown_irq = gicv2m_teardown_msi_irq;
> +       ret = of_pci_msi_chip_add(&v2m->msi_chip);
> +       if (ret) {
> +               pr_info("Failed to add msi_chip.\n");
> +               return ret;
> +       }
> +
> +       v2m->base = ioremap(res->start, resource_size(res));
> +       if (!v2m->base) {
> +               pr_err("Failed to map GICv2m resource\n");
> +               return -EINVAL;

Beware, you're leaving with a registered msi_chip. You probably want to
remove it on the error path.

> +       }
> +
> +       memcpy(&v2m->res, res, sizeof(struct resource));
> +
> +       typer = readl_relaxed(v2m->base + V2M_MSI_TYPER);
> +       if (!typer) {
> +               pr_warn("Failed to read V2M_MSI_TYPER register\n");
> +               return -EINVAL;

And here leaving the v2m widget mapped.

> +       }
> +
> +       v2m->spi_start = (typer >> V2M_MSI_TYPER_BASE_SHIFT) &
> +                               V2M_MSI_TYPER_BASE_MASK;
> +       v2m->nr_spis = typer & V2M_MSI_TYPER_NUM_MASK;
> +       if ((v2m->spi_start < V2M_MIN_SPI) || (v2m->nr_spis >= V2M_MAX_SPI)) {
> +                       pr_err("Invalid MSI_TYPER (%#x)\n", typer);
> +                       return -EINVAL;

Blah. The whole error handling of this function is b0rked.

> +       }
> +
> +       v2m->bm = kzalloc(sizeof(long) * BITS_TO_LONGS(v2m->nr_spis),
> +                         GFP_KERNEL);
> +       if (!v2m->bm) {
> +               pr_err("Failed to allocate MSI bitmap\n");
> +               return -ENOMEM;
> +       }
> +
> +       spin_lock_init(&v2m->msi_cnt_lock);
> +
> +       pr_info("SPI range [%d:%d]\n",
> +               v2m->spi_start, (v2m->spi_start + v2m->nr_spis));
> +
> +       return 0;
> +}
> +
> +int __init gicv2m_of_init(struct device_node *node,
> +                         struct gic_chip_data *gic,
> +                         struct irq_chip *v2m_chip)
> +{
> +       int ret = 0;
> +       struct v2m_data *v2m;
> +       struct resource res;
> +       struct device_node *child = NULL;
> +
> +       INIT_LIST_HEAD(&gic->v2m_list);
> +
> +       for (;;) {
> +               child = of_get_next_child(node, child);

I don't see any of_node_put() in this function. You'll need to fix
this...

> +               if (!child)
> +                       break;
> +
> +               if (!of_find_property(child, "msi-controller", NULL))
> +                       continue;
> +
> +               ret = of_address_to_resource(child, 0, &res);
> +               if (ret) {
> +                       pr_err("Failed to allocate v2m resource.\n");
> +                       break;
> +               }
> +
> +               v2m = kzalloc(sizeof(struct v2m_data), GFP_KERNEL);
> +               if (!v2m) {
> +                       pr_err("Failed to allocate struct v2m_data.\n");
> +                       return -ENOMEM;
> +               }
> +
> +               ret = v2m_init(child, v2m, &res);
> +               if (ret)
> +                       break;
> +
> +               v2m->gic = gic;
> +
> +               pr_info(" %s: [%lx : %lx]\n", child->name,
> +                       (unsigned long)res.start, (unsigned long)res.end);
> +
> +               list_add_tail(&v2m->list, &gic->v2m_list);
> +       }
> +
> +       if (ret && list_empty(&gic->v2m_list)) {
> +               pr_err("MSI resource not found\n");
> +               return -EINVAL;
> +       }
> +
> +       v2m_chip->irq_mask = gicv2m_mask_irq;
> +       v2m_chip->irq_unmask = gicv2m_unmask_irq;
> +

Again, the whole error handling is evil. You can end-up allocating a few
v2m structures, registering them, and yet have kzalloc failing at some
point. At that stage, you exit early, not setting the mas/unmask
methods. Boom.

> +       return ret;
> +}
> +
> +#endif /* CONFIG_OF */
> diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
> index 4b959e6..f490599 100644
> --- a/drivers/irqchip/irq-gic.c
> +++ b/drivers/irqchip/irq-gic.c
> @@ -46,30 +46,9 @@
>  #include <asm/smp_plat.h>
>
>  #include "irq-gic-common.h"
> +#include "irq-gic.h"
>  #include "irqchip.h"
>
> -union gic_base {
> -       void __iomem *common_base;
> -       void __percpu * __iomem *percpu_base;
> -};
> -
> -struct gic_chip_data {
> -       union gic_base dist_base;
> -       union gic_base cpu_base;
> -#ifdef CONFIG_CPU_PM
> -       u32 saved_spi_enable[DIV_ROUND_UP(1020, 32)];
> -       u32 saved_spi_conf[DIV_ROUND_UP(1020, 16)];
> -       u32 saved_spi_target[DIV_ROUND_UP(1020, 4)];
> -       u32 __percpu *saved_ppi_enable;
> -       u32 __percpu *saved_ppi_conf;
> -#endif
> -       struct irq_domain *domain;
> -       unsigned int gic_irqs;
> -#ifdef CONFIG_GIC_NON_BANKED
> -       void __iomem *(*get_base)(union gic_base *);
> -#endif
> -};
> -
>  static DEFINE_RAW_SPINLOCK(irq_controller_lock);
>
>  /*
> @@ -131,15 +110,36 @@ static inline void gic_set_base_accessor(struct gic_chip_data *data,
>  #define gic_set_base_accessor(d, f)
>  #endif
>
> +static inline
> +struct gic_chip_data *irq_data_get_gic_chip_data(struct irq_data *d)
> +{
> +       struct gic_chip_data *gic_data;
> +       struct msi_chip *mchip;
> +       struct v2m_data *v2mdat;
> +
> +       /*
> +        * For MSI, irq_data.chip_data points to struct msi_chip.
> +        * For non-MSI, irq_data.chip_data points to struct gic_chip_data.
> +        */
> +       if (d->msi_desc) {
> +               mchip = irq_data_get_irq_chip_data(d);
> +               v2mdat = container_of(mchip, struct v2m_data, msi_chip);
> +               gic_data = v2mdat->gic;
> +       } else {
> +               gic_data = irq_data_get_irq_chip_data(d);
> +       }
> +       return gic_data;
> +}
> +
>  static inline void __iomem *gic_dist_base(struct irq_data *d)
>  {
> -       struct gic_chip_data *gic_data = irq_data_get_irq_chip_data(d);
> +       struct gic_chip_data *gic_data = irq_data_get_gic_chip_data(d);
>         return gic_data_dist_base(gic_data);
>  }
>
>  static inline void __iomem *gic_cpu_base(struct irq_data *d)
>  {
> -       struct gic_chip_data *gic_data = irq_data_get_irq_chip_data(d);
> +       struct gic_chip_data *gic_data = irq_data_get_gic_chip_data(d);
>         return gic_data_cpu_base(gic_data);
>  }
>
> @@ -151,7 +151,7 @@ static inline unsigned int gic_irq(struct irq_data *d)
>  /*
>   * Routines to acknowledge, disable and enable interrupts
>   */
> -static void gic_mask_irq(struct irq_data *d)
> +void gic_mask_irq(struct irq_data *d)

I'm still very uncomfortable with exporting the mask/unmask methods
outside of this file... Maybe it is not a big deal, but I see evil
everywhere.

>  {
>         u32 mask = 1 << (gic_irq(d) % 32);
>
> @@ -162,7 +162,7 @@ static void gic_mask_irq(struct irq_data *d)
>         raw_spin_unlock(&irq_controller_lock);
>  }
>
> -static void gic_unmask_irq(struct irq_data *d)
> +void gic_unmask_irq(struct irq_data *d)
>  {
>         u32 mask = 1 << (gic_irq(d) % 32);
>
> @@ -325,6 +325,15 @@ static struct irq_chip gic_chip = {
>         .irq_set_wake           = gic_set_wake,
>  };
>
> +static struct irq_chip v2m_chip = {
> +       .name                   = "GICv2m",
> +       .irq_eoi                = gic_eoi_irq,
> +       .irq_set_type           = gic_set_type,
> +#ifdef CONFIG_SMP
> +       .irq_set_affinity       = gic_set_affinity,
> +#endif
> +};
> +
>  void __init gic_cascade_irq(unsigned int gic_nr, unsigned int irq)
>  {
>         if (gic_nr >= MAX_GIC_NR)
> @@ -767,6 +776,24 @@ void __init gic_init_physaddr(struct device_node *node)
>  static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
>                                 irq_hw_number_t hw)
>  {
> +       struct gic_chip_data *gic = d->host_data;
> +       struct v2m_data *v2m;
> +
> +       irq_set_chip_data(irq, gic);
>
> +#ifdef CONFIG_ARM_GIC_V2M
> +       list_for_each_entry(v2m, &gic->v2m_list, list) {
> +               if (hw >= v2m->spi_start &&
> +                   hw <  v2m->spi_start + v2m->nr_spis) {
> +
> +                       irq_set_chip_and_handler(irq, &v2m_chip,
> +                                        handle_fasteoi_irq);
> +                       set_irq_flags(irq, IRQF_VALID | IRQF_PROBE);
> +                       return 0;
> +               }
> +       }
> +#endif
> +
>         if (hw < 32) {
>                 irq_set_percpu_devid(irq);
>                 irq_set_chip_and_handler(irq, &gic_chip,
> @@ -779,7 +806,6 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
>
>                 gic_routable_irq_domain_ops->map(d, irq, hw);
>         }
> -       irq_set_chip_data(irq, d->host_data);

These two hunks make the code a bit hard to read:
- First, message-based SPIs,
- Then PPIs
- and finally the rest of the SPIs.

Can you come up with a construct that keeps everything in a logical
order (PPIs, and then SPIs)? Not a big deal but it would make the code
nicer.

>         return 0;
>  }
>
> @@ -996,7 +1022,7 @@ gic_of_init(struct device_node *node, struct device_node *parent)
>         void __iomem *cpu_base;
>         void __iomem *dist_base;
>         u32 percpu_offset;
> -       int irq;
> +       int irq, ret;
>
>         if (WARN_ON(!node))
>                 return -ENODEV;
> @@ -1010,6 +1036,12 @@ gic_of_init(struct device_node *node, struct device_node *parent)
>         if (of_property_read_u32(node, "cpu-offset", &percpu_offset))
>                 percpu_offset = 0;
>
> +       if (IS_ENABLED(CONFIG_ARM_GIC_V2M)) {
> +               ret = gicv2m_of_init(node, &gic_data[gic_cnt], &v2m_chip);
> +               if (ret)
> +                       pr_warn("Warning: Failed to enable GICv2m support.\n");
> +       }
> +
>         gic_init_bases(gic_cnt, -1, dist_base, cpu_base, percpu_offset, node);
>         if (!gic_cnt)
>                 gic_init_physaddr(node);
> diff --git a/drivers/irqchip/irq-gic.h b/drivers/irqchip/irq-gic.h
> new file mode 100644
> index 0000000..1f4a2b0
> --- /dev/null
> +++ b/drivers/irqchip/irq-gic.h
> @@ -0,0 +1,51 @@
> +#ifndef _IRQ_GIC_H_
> +#define _IRQ_GIC_H_
> +
> +#include <linux/msi.h>
> +#include <linux/pci.h>
> +
> +union gic_base {
> +       void __iomem *common_base;
> +       void __percpu * __iomem *percpu_base;
> +};
> +
> +struct gic_chip_data;
> +
> +struct v2m_data {
> +#ifdef CONFIG_ARM_GIC_V2M
> +       struct list_head list;
> +       spinlock_t msi_cnt_lock;
> +       struct msi_chip msi_chip;
> +       struct resource res;      /* GICv2m resource */
> +       void __iomem *base;       /* GICv2m virt address */
> +       unsigned int spi_start;   /* The SPI number that MSIs start */
> +       unsigned int nr_spis;     /* The number of SPIs for MSIs */
> +       unsigned long *bm;        /* MSI vector bitmap */
> +       struct gic_chip_data *gic;
> +#endif
> +};
> +
> +struct gic_chip_data {
> +       union gic_base dist_base;
> +       union gic_base cpu_base;
> +#ifdef CONFIG_CPU_PM
> +       u32 saved_spi_enable[DIV_ROUND_UP(1020, 32)];
> +       u32 saved_spi_conf[DIV_ROUND_UP(1020, 16)];
> +       u32 saved_spi_target[DIV_ROUND_UP(1020, 4)];
> +       u32 __percpu *saved_ppi_enable;
> +       u32 __percpu *saved_ppi_conf;
> +#endif
> +       struct irq_domain *domain;
> +       unsigned int gic_irqs;
> +#ifdef CONFIG_GIC_NON_BANKED
> +       void __iomem *(*get_base)(union gic_base *);
> +#endif
> +       struct list_head v2m_list;
> +};

Maybe wrap this list_head around #ifdef/#endif?

> +
> +void gic_mask_irq(struct irq_data *d);
> +void gic_unmask_irq(struct irq_data *d);
> +int gicv2m_of_init(struct device_node *node, struct gic_chip_data *gic,
> +                  struct irq_chip *v2m_chip) __init;
> +
> +#endif /* _IRQ_GIC_H_ */
> --
> 1.9.3
>
>
-- 
Jazz is not dead. It just smells funny.

  reply	other threads:[~2014-09-15 19:26 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-09-14  6:30 [PATCH 0/2 V6] irqchip: gic: Introduce ARM GICv2m MSI(-X) support suravee.suthikulpanit
2014-09-14  6:30 ` suravee.suthikulpanit
2014-09-14  6:30 ` suravee.suthikulpanit at amd.com
2014-09-14  6:30 ` [PATCH 1/2 V6] irqchip: gic: Add supports for ARM GICv2m MSI(-X) suravee.suthikulpanit
2014-09-14  6:30   ` suravee.suthikulpanit
2014-09-14  6:30   ` suravee.suthikulpanit at amd.com
2014-09-15 19:26   ` Marc Zyngier [this message]
2014-09-15 19:26     ` Marc Zyngier
2014-09-15 19:26     ` Marc Zyngier
2014-09-14  6:30 ` [PATCH 2/2 V6] irqchip: gicv2m: Add support for multiple MSI for ARM64 GICv2m suravee.suthikulpanit
2014-09-14  6:30   ` suravee.suthikulpanit
2014-09-14  6:30   ` suravee.suthikulpanit at amd.com
2014-09-15 19:41   ` Marc Zyngier
2014-09-15 19:41     ` Marc Zyngier
2014-09-15 19:41     ` Marc Zyngier

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=86iokon0rn.fsf@arm.com \
    --to=marc.zyngier@arm.com \
    --cc=Catalin.Marinas@arm.com \
    --cc=Harish.Kasiviswanathan@amd.com \
    --cc=Mark.Rutland@arm.com \
    --cc=Pawel.Moll@arm.com \
    --cc=Will.Deacon@arm.com \
    --cc=devicetree@vger.kernel.org \
    --cc=jason@lakedaemon.net \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=suravee.suthikulpanit@amd.com \
    --cc=tglx@linutronix.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.