[PATCH v10 0/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05

[PATCH v10 0/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05

[Zhou Wang]

This patchset adds PCIe host support for HiSilicon SoC Hip05. The PCIe hosts
use PCIe IP core from Synopsys, So this driver is based on designware PCIe driver.

Hip05 is an ARMv8 architecture SoC. It should be able to use ARM64 PCIe API in
designware PCIe driver. So this patch also adds ARM64 support for designware
pcie.

This patchset is based on v4.3-rc1.

Change from v9:
- Use syscon to get subctrl base address.
- 5/6 is based on [PATCH v3 0/2] arm64: Support Hisilicon Hip05-D02 board
  from Ding Tianhong
- Add hisi_pcie_cfg_read in pcie-hisi.c to match
  [PATCH v6 0/3] PCI: designware: change dw_pcie_cfg_write() and dw_pcie_cfg_read()
  from Gabriele.

Change from v8:
- Rebase on v4.3-rc1.
- Add Tested-by from Gabriel and Minghuan.
- Remove ITS domain parsing in msi_host_init in pcie-hisi.c, no need this as PCI
  core does related job. Add ITS base address parsing in msi_host_init.
- Change vmid/asid table configuration, previous configuration was wrong.
- Add wr_own_conf callback in pcie-hisi.c.
- Use subsys_initcall to init hisi PCIe.

Change from v7:
- Remove pp->root_bus_nr = 0 in dra7xx, exynos, imx6, keystone, layerscape,
  spear13xx. Pass pp->busn->start to pci_create_root_bus as root bus number.
- Remove bus-range parsing in pcie-hisi.c.

Change from v6:
- Add Pratyush's Acked-by for 1/6 and 2/6.
- Add James' Tested-by for 3/6.

Change from v5:
- Merge 1/6 in this series, discussion about this can be found in [1]

Change from v4:
- Change the author of 1/5 to Gabriele.
- Modify problems in 3/5 pointed by Bjorn.
- Modify spelling problems in 4/5.

Change from v3:
- Change 1/5 to what Gabriele suggested.
- Use win->__res.start to get *_mod_base in 2/5, this fix a bug in v3 series.

Change from v2:
- Move struct pci_dev *dev and struct pci_sys_data *sys in
  pcibios_align_resource in 1/5.
- Add Gabriele's codes in 2/5 which delete unnecessary information parse and
  use of_pci_get_host_bridge_resources for both ARM32 and ARM64.
- Add maintainer patch 5/5.

Change from RFC v1:
- Add 1/4 patch by Arnd which removes align_resource callback in ARM
  pcibios_align_resource.
- Change head file in pcie-designware from asm/hardirq.h to linux/hardirq.h.
- Set pp->root_bus_nr = 0 in dra7xx, exynos, imx6, keystone, layerscape,
  spear13xx.
- Remove unnecessary parentheses of some macros in pcie-hisi.
- Use macro to replace some magic values.
- Merge two loops together and add some comments about it in context_config
  function in pcie-hisi.
- Modify some value of items in pcie node example in binding document. 

Change from RFC:
- delete dw_pcie_setup, dw_pcie_scan_bus, dw_pcie_map_irq and struct hw_pci,
  merge related operations into dw_pcie_host_init.

Link of v9:
- http://www.spinics.net/lists/linux-pci/msg44545.html
Link of v8:
- http://www.spinics.net/lists/linux-pci/msg44192.html
Link of v7:
- http://www.spinics.net/lists/devicetree/msg90690.html
Link of v6:
- http://www.spinics.net/lists/linux-pci/msg43669.html
Link of v5:
- http://www.spinics.net/lists/devicetree/msg87959.html
Link of v4:
- http://www.spinics.net/lists/arm-kernel/msg433050.html
Link of v3:
- http://www.spinics.net/lists/linux-pci/msg42539.html
Link of v2:
- http://www.spinics.net/lists/linux-pci/msg41844.html
Link of RFC v1:
- http://www.spinics.net/lists/linux-pci/msg41305.html
Link of RFC:
- http://www.spinics.net/lists/linux-pci/msg40434.html

[1] http://lists.infradead.org/pipermail/linux-arm-kernel/2015-July/359741.html

Zhou Wang (4):
  PCI: designware: Add ARM64 support
  PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05
  Documentation: DT: Add HiSilicon PCIe host binding
  MAINTAINERS: Add pcie-hisi maintainer

gabriele paoloni (2):
  PCI: designware: move calculation of bus addresses to DRA7xx
  ARM/PCI: remove align_resource in pci_sys_data

 .../bindings/arm/hisilicon/hisilicon.txt           |  17 ++
 .../devicetree/bindings/pci/hisilicon-pcie.txt     |  44 +++
 MAINTAINERS                                        |   7 +
 arch/arm/include/asm/mach/pci.h                    |   6 -
 arch/arm/kernel/bios32.c                           |  12 +-
 drivers/pci/host/Kconfig                           |   8 +
 drivers/pci/host/Makefile                          |   1 +
 drivers/pci/host/pci-dra7xx.c                      |  13 +
 drivers/pci/host/pci-keystone-dw.c                 |   2 +-
 drivers/pci/host/pcie-designware.c                 | 245 +++++-----------
 drivers/pci/host/pcie-designware.h                 |  14 +-
 drivers/pci/host/pcie-hisi.c                       | 320 +++++++++++++++++++++
 12 files changed, 501 insertions(+), 188 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/pci/hisilicon-pcie.txt
 create mode 100644 drivers/pci/host/pcie-hisi.c

-- 
1.9.1

[PATCH v10 1/6] PCI: designware: move calculation of bus addresses to DRA7xx

[Zhou Wang]

From: gabriele paoloni <gabriele.paoloni@huawei.com>

Commit f4c55c5a3f7f ("PCI: designware: Program ATU with untranslated
address") added the calculation of PCI BUS addresses in designware,
storing them in new fields added in "struct pcie_port". This
calculation is done for every designware user even if is only
applicable to DRA7xx.
This patch moves the calculation of the bus addresses to the DRA7xx
driver and is needed to allow the rework of designware to use
the new DT parsing API.

Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
Acked-by: Pratyush Anand <pratyush.anand@gmail.com>
---
 drivers/pci/host/pci-dra7xx.c      | 13 +++++++++++++
 drivers/pci/host/pcie-designware.c | 15 ++++-----------
 2 files changed, 17 insertions(+), 11 deletions(-)

diff --git a/drivers/pci/host/pci-dra7xx.c b/drivers/pci/host/pci-dra7xx.c
index 199e29a..ebdffa0 100644
--- a/drivers/pci/host/pci-dra7xx.c
+++ b/drivers/pci/host/pci-dra7xx.c
@@ -62,6 +62,7 @@
 
 #define	PCIECTRL_DRA7XX_CONF_PHY_CS			0x010C
 #define	LINK_UP						BIT(16)
+#define	CPU_TO_BUS_ADDR					0x0FFFFFFF
 
 struct dra7xx_pcie {
 	void __iomem		*base;
@@ -151,6 +152,18 @@ static void dra7xx_pcie_enable_interrupts(struct pcie_port *pp)
 static void dra7xx_pcie_host_init(struct pcie_port *pp)
 {
 	dw_pcie_setup_rc(pp);
+
+	if (pp->io_mod_base)
+		pp->io_mod_base &= CPU_TO_BUS_ADDR;
+
+	if (pp->mem_mod_base)
+		pp->mem_mod_base &= CPU_TO_BUS_ADDR;
+
+	if (pp->cfg0_mod_base) {
+		pp->cfg0_mod_base &= CPU_TO_BUS_ADDR;
+		pp->cfg1_mod_base &= CPU_TO_BUS_ADDR;
+	}
+
 	dra7xx_pcie_establish_link(pp);
 	if (IS_ENABLED(CONFIG_PCI_MSI))
 		dw_pcie_msi_init(pp);
diff --git a/drivers/pci/host/pcie-designware.c b/drivers/pci/host/pcie-designware.c
index 52aa6e3..75338a6 100644
--- a/drivers/pci/host/pcie-designware.c
+++ b/drivers/pci/host/pcie-designware.c
@@ -365,14 +365,10 @@ int dw_pcie_host_init(struct pcie_port *pp)
 	struct of_pci_range range;
 	struct of_pci_range_parser parser;
 	struct resource *cfg_res;
-	u32 val, na, ns;
+	u32 val, ns;
 	const __be32 *addrp;
 	int i, index, ret;
 
-	/* Find the address cell size and the number of cells in order to get
-	 * the untranslated address.
-	 */
-	of_property_read_u32(np, "#address-cells", &na);
 	ns = of_n_size_cells(np);
 
 	cfg_res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "config");
@@ -415,8 +411,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
 			pp->io_base = range.cpu_addr;
 
 			/* Find the untranslated IO space address */
-			pp->io_mod_base = of_read_number(parser.range -
-							 parser.np + na, ns);
+			pp->io_mod_base = range.cpu_addr;
 		}
 		if (restype == IORESOURCE_MEM) {
 			of_pci_range_to_resource(&range, np, &pp->mem);
@@ -425,8 +420,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
 			pp->mem_bus_addr = range.pci_addr;
 
 			/* Find the untranslated MEM space address */
-			pp->mem_mod_base = of_read_number(parser.range -
-							  parser.np + na, ns);
+			pp->mem_mod_base = range.cpu_addr;
 		}
 		if (restype == 0) {
 			of_pci_range_to_resource(&range, np, &pp->cfg);
@@ -436,8 +430,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
 			pp->cfg1_base = pp->cfg.start + pp->cfg0_size;
 
 			/* Find the untranslated configuration space address */
-			pp->cfg0_mod_base = of_read_number(parser.range -
-							   parser.np + na, ns);
+			pp->cfg0_mod_base = range.cpu_addr;
 			pp->cfg1_mod_base = pp->cfg0_mod_base +
 					    pp->cfg0_size;
 		}
-- 
1.9.1

[PATCH v10 2/6] ARM/PCI: remove align_resource in pci_sys_data

[Zhou Wang]

From: gabriele paoloni <gabriele.paoloni@huawei.com>

This patch is needed in order to unify the PCIe designware framework for ARM and
ARM64 architectures. In the PCIe designware unification process we are calling
pci_create_root_bus() passing a "sysdata" parameter that is the same for both
ARM and ARM64 and is of type "struct pcie_port*". In the ARM case this will
cause a problem with the function pcibios_align_resource(); in fact this will
cast "dev->sysdata" to "struct pci_sys_data*", whereas designware had passed a
"struct pcie_port*" pointer.

This patch solves the issue by removing "align_resource" from "pci_sys_data"
struct and defining a static global function pointer in "bios32.c"

Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
Acked-by: Pratyush Anand <pratyush.anand@gmail.com>
---
 arch/arm/include/asm/mach/pci.h |  6 ------
 arch/arm/kernel/bios32.c        | 12 ++++++++----
 2 files changed, 8 insertions(+), 10 deletions(-)

diff --git a/arch/arm/include/asm/mach/pci.h b/arch/arm/include/asm/mach/pci.h
index 8857d28..0070e85 100644
--- a/arch/arm/include/asm/mach/pci.h
+++ b/arch/arm/include/asm/mach/pci.h
@@ -52,12 +52,6 @@ struct pci_sys_data {
 	u8		(*swizzle)(struct pci_dev *, u8 *);
 					/* IRQ mapping				*/
 	int		(*map_irq)(const struct pci_dev *, u8, u8);
-					/* Resource alignement requirements	*/
-	resource_size_t (*align_resource)(struct pci_dev *dev,
-					  const struct resource *res,
-					  resource_size_t start,
-					  resource_size_t size,
-					  resource_size_t align);
 	void		*private_data;	/* platform controller private data	*/
 };
 
diff --git a/arch/arm/kernel/bios32.c b/arch/arm/kernel/bios32.c
index 874e182..6551d28 100644
--- a/arch/arm/kernel/bios32.c
+++ b/arch/arm/kernel/bios32.c
@@ -17,6 +17,11 @@
 #include <asm/mach/pci.h>
 
 static int debug_pci;
+static resource_size_t (*align_resource)(struct pci_dev *dev,
+		  const struct resource *res,
+		  resource_size_t start,
+		  resource_size_t size,
+		  resource_size_t align) = NULL;
 
 /*
  * We can't use pci_get_device() here since we are
@@ -456,7 +461,7 @@ static void pcibios_init_hw(struct device *parent, struct hw_pci *hw,
 		sys->busnr   = busnr;
 		sys->swizzle = hw->swizzle;
 		sys->map_irq = hw->map_irq;
-		sys->align_resource = hw->align_resource;
+		align_resource = hw->align_resource;
 		INIT_LIST_HEAD(&sys->resources);
 
 		if (hw->private_data)
@@ -572,7 +577,6 @@ resource_size_t pcibios_align_resource(void *data, const struct resource *res,
 				resource_size_t size, resource_size_t align)
 {
 	struct pci_dev *dev = data;
-	struct pci_sys_data *sys = dev->sysdata;
 	resource_size_t start = res->start;
 
 	if (res->flags & IORESOURCE_IO && start & 0x300)
@@ -580,8 +584,8 @@ resource_size_t pcibios_align_resource(void *data, const struct resource *res,
 
 	start = (start + align - 1) & ~(align - 1);
 
-	if (sys->align_resource)
-		return sys->align_resource(dev, res, start, size, align);
+	if (align_resource)
+		return align_resource(dev, res, start, size, align);
 
 	return start;
 }
-- 
1.9.1

[PATCH v10 3/6] PCI: designware: Add ARM64 support

[Zhou Wang]

This patch tries to unify ARM32 and ARM64 PCIe in designware driver. Delete
function dw_pcie_setup, dw_pcie_scan_bus, dw_pcie_map_irq and struct hw_pci,
move related operations to dw_pcie_host_init.

This patch also try to use of_pci_get_host_bridge_resources for ARM32 and ARM64
according to the suggestion for Gabriele[1]

This patch reverts commit f4c55c5a3f7f ("PCI: designware: Program ATU with
untranslated address") based on 1/6 in this series. we delete *_mod_base in
pcie-designware. This was discussed in [2]

I have compiled the driver with multi_v7_defconfig. However, I don't have
ARM32 PCIe related board to do test. It will be appreciated if someone could
help to test it.

Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Tested-by: James Morse <james.morse@arm.com>
Tested-by: Gabriel Fernandez <gabriel.fernandez@st.com>
Tested-by: Minghuan Lian <Minghuan.Lian@freescale.com>
Acked-by: Pratyush Anand <pratyush.anand@gmail.com>

[1] http://www.spinics.net/lists/linux-pci/msg42194.html
[2] http://www.spinics.net/lists/arm-kernel/msg436779.html
---
 drivers/pci/host/pci-dra7xx.c      |  14 +--
 drivers/pci/host/pci-keystone-dw.c |   2 +-
 drivers/pci/host/pcie-designware.c | 238 ++++++++++++-------------------------
 drivers/pci/host/pcie-designware.h |  14 +--
 4 files changed, 90 insertions(+), 178 deletions(-)

diff --git a/drivers/pci/host/pci-dra7xx.c b/drivers/pci/host/pci-dra7xx.c
index ebdffa0..b88c248 100644
--- a/drivers/pci/host/pci-dra7xx.c
+++ b/drivers/pci/host/pci-dra7xx.c
@@ -153,15 +153,15 @@ static void dra7xx_pcie_host_init(struct pcie_port *pp)
 {
 	dw_pcie_setup_rc(pp);
 
-	if (pp->io_mod_base)
-		pp->io_mod_base &= CPU_TO_BUS_ADDR;
+	if (pp->io_base)
+		pp->io_base &= CPU_TO_BUS_ADDR;
 
-	if (pp->mem_mod_base)
-		pp->mem_mod_base &= CPU_TO_BUS_ADDR;
+	if (pp->mem_base)
+		pp->mem_base &= CPU_TO_BUS_ADDR;
 
-	if (pp->cfg0_mod_base) {
-		pp->cfg0_mod_base &= CPU_TO_BUS_ADDR;
-		pp->cfg1_mod_base &= CPU_TO_BUS_ADDR;
+	if (pp->cfg0_base) {
+		pp->cfg0_base &= CPU_TO_BUS_ADDR;
+		pp->cfg1_base &= CPU_TO_BUS_ADDR;
 	}
 
 	dra7xx_pcie_establish_link(pp);
diff --git a/drivers/pci/host/pci-keystone-dw.c b/drivers/pci/host/pci-keystone-dw.c
index e71da99..8062ddb 100644
--- a/drivers/pci/host/pci-keystone-dw.c
+++ b/drivers/pci/host/pci-keystone-dw.c
@@ -322,7 +322,7 @@ static void ks_dw_pcie_clear_dbi_mode(void __iomem *reg_virt)
 void ks_dw_pcie_setup_rc_app_regs(struct keystone_pcie *ks_pcie)
 {
 	struct pcie_port *pp = &ks_pcie->pp;
-	u32 start = pp->mem.start, end = pp->mem.end;
+	u32 start = pp->mem->start, end = pp->mem->end;
 	int i, tr_size;
 
 	/* Disable BARs for inbound access */
diff --git a/drivers/pci/host/pcie-designware.c b/drivers/pci/host/pcie-designware.c
index 75338a6..8ef74da 100644
--- a/drivers/pci/host/pcie-designware.c
+++ b/drivers/pci/host/pcie-designware.c
@@ -11,6 +11,7 @@
  * published by the Free Software Foundation.
  */
 
+#include <linux/hardirq.h>
 #include <linux/irq.h>
 #include <linux/irqdomain.h>
 #include <linux/kernel.h>
@@ -69,16 +70,7 @@
 #define PCIE_ATU_FUNC(x)		(((x) & 0x7) << 16)
 #define PCIE_ATU_UPPER_TARGET		0x91C
 
-static struct hw_pci dw_pci;
-
-static unsigned long global_io_offset;
-
-static inline struct pcie_port *sys_to_pcie(struct pci_sys_data *sys)
-{
-	BUG_ON(!sys->private_data);
-
-	return sys->private_data;
-}
+static struct pci_ops dw_pcie_ops;
 
 int dw_pcie_cfg_read(void __iomem *addr, int where, int size, u32 *val)
 {
@@ -255,7 +247,7 @@ static void dw_pcie_msi_set_irq(struct pcie_port *pp, int irq)
 static int assign_irq(int no_irqs, struct msi_desc *desc, int *pos)
 {
 	int irq, pos0, i;
-	struct pcie_port *pp = sys_to_pcie(msi_desc_to_pci_sysdata(desc));
+	struct pcie_port *pp = (struct pcie_port *) msi_desc_to_pci_sysdata(desc);
 
 	pos0 = bitmap_find_free_region(pp->msi_irq_in_use, MAX_MSI_IRQS,
 				       order_base_2(no_irqs));
@@ -298,7 +290,7 @@ static int dw_msi_setup_irq(struct msi_controller *chip, struct pci_dev *pdev,
 {
 	int irq, pos;
 	struct msi_msg msg;
-	struct pcie_port *pp = sys_to_pcie(pdev->bus->sysdata);
+	struct pcie_port *pp = pdev->bus->sysdata;
 
 	if (desc->msi_attrib.is_msix)
 		return -EINVAL;
@@ -327,7 +319,7 @@ static void dw_msi_teardown_irq(struct msi_controller *chip, unsigned int irq)
 {
 	struct irq_data *data = irq_get_irq_data(irq);
 	struct msi_desc *msi = irq_data_get_msi_desc(data);
-	struct pcie_port *pp = sys_to_pcie(msi_desc_to_pci_sysdata(msi));
+	struct pcie_port *pp = (struct pcie_port *) msi_desc_to_pci_sysdata(msi);
 
 	clear_irq_range(pp, irq, 1, data->hwirq);
 }
@@ -362,14 +354,12 @@ int dw_pcie_host_init(struct pcie_port *pp)
 {
 	struct device_node *np = pp->dev->of_node;
 	struct platform_device *pdev = to_platform_device(pp->dev);
-	struct of_pci_range range;
-	struct of_pci_range_parser parser;
+	struct pci_bus *bus;
 	struct resource *cfg_res;
-	u32 val, ns;
-	const __be32 *addrp;
-	int i, index, ret;
-
-	ns = of_n_size_cells(np);
+	LIST_HEAD(res);
+	u32 val;
+	int i, ret;
+	struct resource_entry *win;
 
 	cfg_res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "config");
 	if (cfg_res) {
@@ -377,85 +367,60 @@ int dw_pcie_host_init(struct pcie_port *pp)
 		pp->cfg1_size = resource_size(cfg_res)/2;
 		pp->cfg0_base = cfg_res->start;
 		pp->cfg1_base = cfg_res->start + pp->cfg0_size;
-
-		/* Find the untranslated configuration space address */
-		index = of_property_match_string(np, "reg-names", "config");
-		addrp = of_get_address(np, index, NULL, NULL);
-		pp->cfg0_mod_base = of_read_number(addrp, ns);
-		pp->cfg1_mod_base = pp->cfg0_mod_base + pp->cfg0_size;
 	} else if (!pp->va_cfg0_base) {
 		dev_err(pp->dev, "missing *config* reg space\n");
 	}
 
-	if (of_pci_range_parser_init(&parser, np)) {
-		dev_err(pp->dev, "missing ranges property\n");
-		return -EINVAL;
-	}
+	ret = of_pci_get_host_bridge_resources(np, 0, 0xff, &res, &pp->io_base);
+	if (ret)
+		return ret;
 
 	/* Get the I/O and memory ranges from DT */
-	for_each_of_pci_range(&parser, &range) {
-		unsigned long restype = range.flags & IORESOURCE_TYPE_BITS;
-
-		if (restype == IORESOURCE_IO) {
-			of_pci_range_to_resource(&range, np, &pp->io);
-			pp->io.name = "I/O";
-			pp->io.start = max_t(resource_size_t,
-					     PCIBIOS_MIN_IO,
-					     range.pci_addr + global_io_offset);
-			pp->io.end = min_t(resource_size_t,
-					   IO_SPACE_LIMIT,
-					   range.pci_addr + range.size
-					   + global_io_offset - 1);
-			pp->io_size = resource_size(&pp->io);
-			pp->io_bus_addr = range.pci_addr;
-			pp->io_base = range.cpu_addr;
-
-			/* Find the untranslated IO space address */
-			pp->io_mod_base = range.cpu_addr;
-		}
-		if (restype == IORESOURCE_MEM) {
-			of_pci_range_to_resource(&range, np, &pp->mem);
-			pp->mem.name = "MEM";
-			pp->mem_size = resource_size(&pp->mem);
-			pp->mem_bus_addr = range.pci_addr;
-
-			/* Find the untranslated MEM space address */
-			pp->mem_mod_base = range.cpu_addr;
-		}
-		if (restype == 0) {
-			of_pci_range_to_resource(&range, np, &pp->cfg);
-			pp->cfg0_size = resource_size(&pp->cfg)/2;
-			pp->cfg1_size = resource_size(&pp->cfg)/2;
-			pp->cfg0_base = pp->cfg.start;
-			pp->cfg1_base = pp->cfg.start + pp->cfg0_size;
-
-			/* Find the untranslated configuration space address */
-			pp->cfg0_mod_base = range.cpu_addr;
-			pp->cfg1_mod_base = pp->cfg0_mod_base +
-					    pp->cfg0_size;
+	resource_list_for_each_entry(win, &res) {
+		switch (resource_type(win->res)) {
+		case IORESOURCE_IO:
+			pp->io = win->res;
+			pp->io->name = "I/O";
+			pp->io_size = resource_size(pp->io);
+			pp->io_bus_addr = pp->io->start - win->offset;
+			ret = pci_remap_iospace(pp->io, pp->io_base);
+			if (ret) {
+				dev_warn(pp->dev, "error %d: failed to map resource %pR\n",
+					 ret, pp->io);
+				continue;
+			}
+			break;
+		case IORESOURCE_MEM:
+			pp->mem = win->res;
+			pp->mem->name = "MEM";
+			pp->mem_size = resource_size(pp->mem);
+			pp->mem_bus_addr = pp->mem->start - win->offset;
+			break;
+		case 0:
+			pp->cfg = win->res;
+			pp->cfg0_size = resource_size(pp->cfg)/2;
+			pp->cfg1_size = resource_size(pp->cfg)/2;
+			pp->cfg0_base = pp->cfg->start;
+			pp->cfg1_base = pp->cfg->start + pp->cfg0_size;
+			break;
+		case IORESOURCE_BUS:
+			pp->busn = win->res;
+			break;
+		default:
+			continue;
 		}
 	}
 
-	ret = of_pci_parse_bus_range(np, &pp->busn);
-	if (ret < 0) {
-		pp->busn.name = np->name;
-		pp->busn.start = 0;
-		pp->busn.end = 0xff;
-		pp->busn.flags = IORESOURCE_BUS;
-		dev_dbg(pp->dev, "failed to parse bus-range property: %d, using default %pR\n",
-			ret, &pp->busn);
-	}
-
 	if (!pp->dbi_base) {
-		pp->dbi_base = devm_ioremap(pp->dev, pp->cfg.start,
-					resource_size(&pp->cfg));
+		pp->dbi_base = devm_ioremap(pp->dev, pp->cfg->start,
+					resource_size(pp->cfg));
 		if (!pp->dbi_base) {
 			dev_err(pp->dev, "error with ioremap\n");
 			return -ENOMEM;
 		}
 	}
 
-	pp->mem_base = pp->mem.start;
+	pp->mem_base = pp->mem->start;
 
 	if (!pp->va_cfg0_base) {
 		pp->va_cfg0_base = devm_ioremap(pp->dev, pp->cfg0_base,
@@ -504,7 +469,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
 
 	if (!pp->ops->rd_other_conf)
 		dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX1,
-					  PCIE_ATU_TYPE_MEM, pp->mem_mod_base,
+					  PCIE_ATU_TYPE_MEM, pp->mem_base,
 					  pp->mem_bus_addr, pp->mem_size);
 
 	dw_pcie_wr_own_conf(pp, PCI_BASE_ADDRESS_0, 4, 0);
@@ -516,14 +481,28 @@ int dw_pcie_host_init(struct pcie_port *pp)
 	val |= PORT_LOGIC_SPEED_CHANGE;
 	dw_pcie_wr_own_conf(pp, PCIE_LINK_WIDTH_SPEED_CONTROL, 4, val);
 
-#ifdef CONFIG_PCI_MSI
-	dw_pcie_msi_chip.dev = pp->dev;
-#endif
+	pp->root_bus_nr = pp->busn->start;
+	if (IS_ENABLED(CONFIG_PCI_MSI)) {
+		bus = pci_scan_root_bus_msi(pp->dev, pp->root_bus_nr,
+					    &dw_pcie_ops, pp, &res,
+					    &dw_pcie_msi_chip);
+		dw_pcie_msi_chip.dev = pp->dev;
+	} else
+		bus = pci_scan_root_bus(pp->dev, pp->root_bus_nr, &dw_pcie_ops,
+					pp, &res);
+	if (!bus)
+		return -ENOMEM;
 
-	dw_pci.nr_controllers = 1;
-	dw_pci.private_data = (void **)&pp;
+	if (pp->ops->scan_bus)
+		pp->ops->scan_bus(pp);
 
-	pci_common_init_dev(pp->dev, &dw_pci);
+#ifdef CONFIG_ARM
+	/* support old dtbs that incorrectly describe IRQs */
+	pci_fixup_irqs(pci_common_swizzle, of_irq_parse_and_map_pci);
+#endif
+
+	pci_assign_unassigned_bus_resources(bus);
+	pci_bus_add_devices(bus);
 
 	return 0;
 }
@@ -542,12 +521,12 @@ static int dw_pcie_rd_other_conf(struct pcie_port *pp, struct pci_bus *bus,
 
 	if (bus->parent->number == pp->root_bus_nr) {
 		type = PCIE_ATU_TYPE_CFG0;
-		cpu_addr = pp->cfg0_mod_base;
+		cpu_addr = pp->cfg0_base;
 		cfg_size = pp->cfg0_size;
 		va_cfg_base = pp->va_cfg0_base;
 	} else {
 		type = PCIE_ATU_TYPE_CFG1;
-		cpu_addr = pp->cfg1_mod_base;
+		cpu_addr = pp->cfg1_base;
 		cfg_size = pp->cfg1_size;
 		va_cfg_base = pp->va_cfg1_base;
 	}
@@ -557,7 +536,7 @@ static int dw_pcie_rd_other_conf(struct pcie_port *pp, struct pci_bus *bus,
 				  busdev, cfg_size);
 	ret = dw_pcie_cfg_read(va_cfg_base + address, where, size, val);
 	dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX0,
-				  PCIE_ATU_TYPE_IO, pp->io_mod_base,
+				  PCIE_ATU_TYPE_IO, pp->io_base,
 				  pp->io_bus_addr, pp->io_size);
 
 	return ret;
@@ -577,12 +556,12 @@ static int dw_pcie_wr_other_conf(struct pcie_port *pp, struct pci_bus *bus,
 
 	if (bus->parent->number == pp->root_bus_nr) {
 		type = PCIE_ATU_TYPE_CFG0;
-		cpu_addr = pp->cfg0_mod_base;
+		cpu_addr = pp->cfg0_base;
 		cfg_size = pp->cfg0_size;
 		va_cfg_base = pp->va_cfg0_base;
 	} else {
 		type = PCIE_ATU_TYPE_CFG1;
-		cpu_addr = pp->cfg1_mod_base;
+		cpu_addr = pp->cfg1_base;
 		cfg_size = pp->cfg1_size;
 		va_cfg_base = pp->va_cfg1_base;
 	}
@@ -592,7 +571,7 @@ static int dw_pcie_wr_other_conf(struct pcie_port *pp, struct pci_bus *bus,
 				  busdev, cfg_size);
 	ret = dw_pcie_cfg_write(va_cfg_base + address, where, size, val);
 	dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX0,
-				  PCIE_ATU_TYPE_IO, pp->io_mod_base,
+				  PCIE_ATU_TYPE_IO, pp->io_base,
 				  pp->io_bus_addr, pp->io_size);
 
 	return ret;
@@ -624,7 +603,7 @@ static int dw_pcie_valid_config(struct pcie_port *pp,
 static int dw_pcie_rd_conf(struct pci_bus *bus, u32 devfn, int where,
 			int size, u32 *val)
 {
-	struct pcie_port *pp = sys_to_pcie(bus->sysdata);
+	struct pcie_port *pp = bus->sysdata;
 	int ret;
 
 	if (dw_pcie_valid_config(pp, bus, PCI_SLOT(devfn)) == 0) {
@@ -648,7 +627,7 @@ static int dw_pcie_rd_conf(struct pci_bus *bus, u32 devfn, int where,
 static int dw_pcie_wr_conf(struct pci_bus *bus, u32 devfn,
 			int where, int size, u32 val)
 {
-	struct pcie_port *pp = sys_to_pcie(bus->sysdata);
+	struct pcie_port *pp = bus->sysdata;
 	int ret;
 
 	if (dw_pcie_valid_config(pp, bus, PCI_SLOT(devfn)) == 0)
@@ -672,69 +651,6 @@ static struct pci_ops dw_pcie_ops = {
 	.write = dw_pcie_wr_conf,
 };
 
-static int dw_pcie_setup(int nr, struct pci_sys_data *sys)
-{
-	struct pcie_port *pp;
-
-	pp = sys_to_pcie(sys);
-
-	if (global_io_offset < SZ_1M && pp->io_size > 0) {
-		sys->io_offset = global_io_offset - pp->io_bus_addr;
-		pci_ioremap_io(global_io_offset, pp->io_base);
-		global_io_offset += SZ_64K;
-		pci_add_resource_offset(&sys->resources, &pp->io,
-					sys->io_offset);
-	}
-
-	sys->mem_offset = pp->mem.start - pp->mem_bus_addr;
-	pci_add_resource_offset(&sys->resources, &pp->mem, sys->mem_offset);
-	pci_add_resource(&sys->resources, &pp->busn);
-
-	return 1;
-}
-
-static struct pci_bus *dw_pcie_scan_bus(int nr, struct pci_sys_data *sys)
-{
-	struct pci_bus *bus;
-	struct pcie_port *pp = sys_to_pcie(sys);
-
-	pp->root_bus_nr = sys->busnr;
-
-	if (IS_ENABLED(CONFIG_PCI_MSI))
-		bus = pci_scan_root_bus_msi(pp->dev, sys->busnr, &dw_pcie_ops,
-					    sys, &sys->resources,
-					    &dw_pcie_msi_chip);
-	else
-		bus = pci_scan_root_bus(pp->dev, sys->busnr, &dw_pcie_ops,
-					sys, &sys->resources);
-
-	if (!bus)
-		return NULL;
-
-	if (bus && pp->ops->scan_bus)
-		pp->ops->scan_bus(pp);
-
-	return bus;
-}
-
-static int dw_pcie_map_irq(const struct pci_dev *dev, u8 slot, u8 pin)
-{
-	struct pcie_port *pp = sys_to_pcie(dev->bus->sysdata);
-	int irq;
-
-	irq = of_irq_parse_and_map_pci(dev, slot, pin);
-	if (!irq)
-		irq = pp->irq;
-
-	return irq;
-}
-
-static struct hw_pci dw_pci = {
-	.setup		= dw_pcie_setup,
-	.scan		= dw_pcie_scan_bus,
-	.map_irq	= dw_pcie_map_irq,
-};
-
 void dw_pcie_setup_rc(struct pcie_port *pp)
 {
 	u32 val;
diff --git a/drivers/pci/host/pcie-designware.h b/drivers/pci/host/pcie-designware.h
index d0bbd27..264c969 100644
--- a/drivers/pci/host/pcie-designware.h
+++ b/drivers/pci/host/pcie-designware.h
@@ -27,25 +27,21 @@ struct pcie_port {
 	u8			root_bus_nr;
 	void __iomem		*dbi_base;
 	u64			cfg0_base;
-	u64			cfg0_mod_base;
 	void __iomem		*va_cfg0_base;
 	u32			cfg0_size;
 	u64			cfg1_base;
-	u64			cfg1_mod_base;
 	void __iomem		*va_cfg1_base;
 	u32			cfg1_size;
-	u64			io_base;
-	u64			io_mod_base;
+	resource_size_t		io_base;
 	phys_addr_t		io_bus_addr;
 	u32			io_size;
 	u64			mem_base;
-	u64			mem_mod_base;
 	phys_addr_t		mem_bus_addr;
 	u32			mem_size;
-	struct resource		cfg;
-	struct resource		io;
-	struct resource		mem;
-	struct resource		busn;
+	struct resource		*cfg;
+	struct resource		*io;
+	struct resource		*mem;
+	struct resource		*busn;
 	int			irq;
 	u32			lanes;
 	struct pcie_host_ops	*ops;
-- 
1.9.1

[PATCH v10 4/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05

[Zhou Wang]

This patch adds PCIe host support for HiSilicon SoC Hip05.

Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
Signed-off-by: liudongdong <liudongdong3@huawei.com>
Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
Signed-off-by: qiuzhenfa <qiuzhenfa@hisilicon.com>
---
 drivers/pci/host/Kconfig     |   8 ++
 drivers/pci/host/Makefile    |   1 +
 drivers/pci/host/pcie-hisi.c | 320 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 329 insertions(+)
 create mode 100644 drivers/pci/host/pcie-hisi.c

diff --git a/drivers/pci/host/Kconfig b/drivers/pci/host/Kconfig
index d5e58ba..ae873be 100644
--- a/drivers/pci/host/Kconfig
+++ b/drivers/pci/host/Kconfig
@@ -145,4 +145,12 @@ config PCIE_IPROC_BCMA
 	  Say Y here if you want to use the Broadcom iProc PCIe controller
 	  through the BCMA bus interface
 
+config PCI_HISI
+	depends on OF && ARM64
+	bool "HiSilicon SoC HIP05 PCIe controller"
+	select PCIEPORTBUS
+	select PCIE_DW
+	help
+	  Say Y here if you want PCIe controller support on HiSilicon HIP05 SoC
+
 endmenu
diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile
index 140d66f..ea1dbf2 100644
--- a/drivers/pci/host/Makefile
+++ b/drivers/pci/host/Makefile
@@ -17,3 +17,4 @@ obj-$(CONFIG_PCI_VERSATILE) += pci-versatile.o
 obj-$(CONFIG_PCIE_IPROC) += pcie-iproc.o
 obj-$(CONFIG_PCIE_IPROC_PLATFORM) += pcie-iproc-platform.o
 obj-$(CONFIG_PCIE_IPROC_BCMA) += pcie-iproc-bcma.o
+obj-$(CONFIG_PCI_HISI) += pcie-hisi.o
diff --git a/drivers/pci/host/pcie-hisi.c b/drivers/pci/host/pcie-hisi.c
new file mode 100644
index 0000000..26aa0d9
--- /dev/null
+++ b/drivers/pci/host/pcie-hisi.c
@@ -0,0 +1,320 @@
+/*
+ * PCIe host controller driver for HiSilicon Hip05 SoC
+ *
+ * Copyright (C) 2015 HiSilicon Co., Ltd. http://www.hisilicon.com
+ *
+ * Author: Zhou Wang <wangzhou1@hisilicon.com>
+ *         Dacai Zhu <zhudacai@hisilicon.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+#include <linux/interrupt.h>
+#include <linux/irqdomain.h>
+#include <linux/module.h>
+#include <linux/mfd/syscon.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/of_pci.h>
+#include <linux/platform_device.h>
+#include <linux/regmap.h>
+
+#include "pcie-designware.h"
+
+#define PCIE_SUBCTRL_MODE_REG                           0x2800
+#define PCIE_SUBCTRL_SYS_STATE4_REG                     0x6818
+#define PCIE_SLV_DBI_MODE                               0x0
+#define PCIE_SLV_SYSCTRL_MODE                           0x1
+#define PCIE_SLV_CONTENT_MODE                           0x2
+#define PCIE_SLV_MSI_ASID                               0x10
+#define PCIE_LTSSM_LINKUP_STATE                         0x11
+#define PCIE_LTSSM_STATE_MASK                           0x3F
+#define PCIE_MSI_ASID_ENABLE                            (0x1 << 12)
+#define PCIE_MSI_ASID_VALUE                             (0x1 << 16)
+#define PCIE_MSI_TRANS_ENABLE                           (0x1 << 12)
+#define PCIE_MSI_TRANS_REG                              0x1c8
+#define PCIE_MSI_LOW_ADDRESS                            0x1b4
+#define PCIE_MSI_HIGH_ADDRESS                           0x1c4
+#define PCIE_GITS_TRANSLATER                            0x10040
+
+#define PCIE_SYS_CTRL20_REG                             0x20
+#define PCIE_RD_TAB_SEL                                 BIT(31)
+#define PCIE_RD_TAB_EN                                  BIT(30)
+
+#define to_hisi_pcie(x)	container_of(x, struct hisi_pcie, pp)
+
+struct hisi_pcie {
+	struct regmap *subctrl;
+	void __iomem *reg_base;
+	u32 port_id;
+	struct pcie_port pp;
+};
+
+static inline void hisi_pcie_apb_writel(struct hisi_pcie *pcie,
+					u32 val, u32 reg)
+{
+	writel(val, pcie->reg_base + reg);
+}
+
+static inline u32 hisi_pcie_apb_readl(struct hisi_pcie *pcie, u32 reg)
+{
+	return readl(pcie->reg_base + reg);
+}
+
+/*
+ * Change mode to indicate the same reg_base to base of PCIe host configure
+ * registers, base of RC configure space or base of vmid/asid context table
+ */
+static void hisi_pcie_change_apb_mode(struct hisi_pcie *pcie, u32 mode)
+{
+	u32 val;
+	u32 bit_mask;
+	u32 bit_shift;
+	u32 port_id = pcie->port_id;
+	u32 reg = PCIE_SUBCTRL_MODE_REG + 0x100 * port_id;
+
+	if ((port_id == 1) || (port_id == 2)) {
+		bit_mask = 0xc;
+		bit_shift = 0x2;
+	} else {
+		bit_mask = 0x6;
+		bit_shift = 0x1;
+	}
+
+	regmap_update_bits(pcie->subctrl, reg, bit_mask, mode << bit_shift);
+}
+
+/* Configure vmid/asid table in PCIe host */
+static void hisi_pcie_config_context(struct hisi_pcie *pcie)
+{
+	int i;
+	u32 val;
+
+	/* enable to clean vmid and asid tables though apb bus */
+	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_SYSCTRL_MODE);
+
+	val = hisi_pcie_apb_readl(pcie, PCIE_SYS_CTRL20_REG);
+	/* enable ar channel */
+	val |= PCIE_RD_TAB_SEL | PCIE_RD_TAB_EN;
+	hisi_pcie_apb_writel(pcie, val, PCIE_SYS_CTRL20_REG);
+
+	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_CONTENT_MODE);
+
+	/*
+	 * init vmid and asid tables for all PCIe devices as 0
+	 * vmid table: 0 ~ 0x3ff, asid table: 0x400 ~ 0x7ff
+	 */
+	for (i = 0; i < 0x800; i++)
+		hisi_pcie_apb_writel(pcie, 0x0, i * 4);
+
+	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_SYSCTRL_MODE);
+
+	/* enable aw channel */
+	val &= (~PCIE_RD_TAB_SEL);
+	val |= PCIE_RD_TAB_EN;
+	hisi_pcie_apb_writel(pcie, val, PCIE_SYS_CTRL20_REG);
+
+	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_CONTENT_MODE);
+
+	for (i = 0; i < 0x800; i++)
+		hisi_pcie_apb_writel(pcie, 0x0, i * 4);
+
+	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_SYSCTRL_MODE);
+
+	val = hisi_pcie_apb_readl(pcie, PCIE_SYS_CTRL20_REG);
+	/* disable ar channel */
+	val |= PCIE_RD_TAB_SEL;
+	val &= (~PCIE_RD_TAB_EN);
+	hisi_pcie_apb_writel(pcie, val, PCIE_SYS_CTRL20_REG);
+	/* disable aw channel */
+	val &= ((~PCIE_RD_TAB_SEL) & (~PCIE_RD_TAB_EN));
+	hisi_pcie_apb_writel(pcie, val, PCIE_SYS_CTRL20_REG);
+
+	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_DBI_MODE);
+}
+
+static int hisi_pcie_link_up(struct pcie_port *pp)
+{
+	u32 val;
+	struct hisi_pcie *hisi_pcie = to_hisi_pcie(pp);
+
+	regmap_read(hisi_pcie->subctrl, PCIE_SUBCTRL_SYS_STATE4_REG +
+		    0x100 * hisi_pcie->port_id, &val);
+
+	return ((val & PCIE_LTSSM_STATE_MASK) == PCIE_LTSSM_LINKUP_STATE);
+}
+
+static
+int hisi_pcie_msi_host_init(struct pcie_port *pp, struct msi_controller *chip)
+{
+	u64 addr;
+	struct device_node *msi_node;
+	struct resource res;
+	struct device_node *np = pp->dev->of_node;
+	struct hisi_pcie *pcie = to_hisi_pcie(pp);
+
+	msi_node = of_parse_phandle(np, "msi-parent", 0);
+	if (!msi_node) {
+		dev_err(pp->dev, "failed to find msi-parent\n");
+		return -EINVAL;
+	}
+	of_address_to_resource(msi_node, 0, &res);
+	addr = res.start + PCIE_GITS_TRANSLATER;
+
+	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_SYSCTRL_MODE);
+
+	hisi_pcie_apb_writel(pcie, addr & 0xffffffff, PCIE_MSI_LOW_ADDRESS);
+	hisi_pcie_apb_writel(pcie, addr >> 32, PCIE_MSI_HIGH_ADDRESS);
+	hisi_pcie_apb_writel(pcie, PCIE_MSI_ASID_ENABLE | PCIE_MSI_ASID_VALUE,
+			     PCIE_SLV_MSI_ASID);
+	hisi_pcie_apb_writel(pcie, PCIE_MSI_TRANS_ENABLE, PCIE_MSI_TRANS_REG);
+
+	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_DBI_MODE);
+
+	return 0;
+}
+
+static int hisi_pcie_cfg_write(struct pcie_port *pp, int where, int  size,
+				u32 val)
+{
+	u32 reg_val;
+	u32 reg;
+	struct hisi_pcie *pcie = to_hisi_pcie(pp);
+	void *walker = &reg_val;
+
+	walker += (where & 0x3);
+	reg = where & ~0x3;
+	if (size == 4)
+		hisi_pcie_apb_writel(pcie, val, reg);
+	else if (size == 2) {
+		reg_val = hisi_pcie_apb_readl(pcie, reg);
+		*(u16 __force *) walker = val;
+		hisi_pcie_apb_writel(pcie, reg_val, reg);
+	} else if (size == 1) {
+		reg_val = hisi_pcie_apb_readl(pcie, reg);
+		*(u8 __force *) walker = val;
+		hisi_pcie_apb_writel(pcie, reg_val, reg);
+	} else
+		return PCIBIOS_BAD_REGISTER_NUMBER;
+
+	return PCIBIOS_SUCCESSFUL;
+}
+
+static int hisi_pcie_cfg_read(struct pcie_port *pp, int where, int size,
+			      u32 *val)
+{
+	u32 reg;
+	u32 reg_val;
+	struct hisi_pcie *pcie = to_hisi_pcie(pp);
+	void *walker = &reg_val;
+
+	walker += (where & 0x3);
+	reg = where & ~0x3;
+	reg_val = hisi_pcie_apb_readl(pcie, reg);
+
+	if (size == 1)
+		*val = *(u8 __force *) walker;
+	else if (size == 2)
+		*val = *(u16 __force *) walker;
+	else if (size != 4)
+		return PCIBIOS_BAD_REGISTER_NUMBER;
+
+	return PCIBIOS_SUCCESSFUL;
+}
+
+static struct pcie_host_ops hisi_pcie_host_ops = {
+	.link_up = hisi_pcie_link_up,
+	.msi_host_init = hisi_pcie_msi_host_init,
+	.wr_own_conf = hisi_pcie_cfg_write,
+	.rd_own_conf = hisi_pcie_cfg_read,
+};
+
+static int __init hisi_add_pcie_port(struct pcie_port *pp,
+				     struct platform_device *pdev)
+{
+	int ret;
+	u32 port_id;
+	struct hisi_pcie *hisi_pcie = to_hisi_pcie(pp);
+
+	if (of_property_read_u32(pdev->dev.of_node, "port-id", &port_id)) {
+		dev_err(&pdev->dev, "failed to read port-id\n");
+		return -EINVAL;
+	}
+	if (port_id > 3) {
+		dev_err(&pdev->dev, "Invalid port-id: %d\n", port_id);
+		return -EINVAL;
+	}
+	hisi_pcie->port_id = port_id;
+
+	pp->ops = &hisi_pcie_host_ops;
+
+	hisi_pcie_config_context(hisi_pcie);
+
+	ret = dw_pcie_host_init(pp);
+	if (ret) {
+		dev_err(&pdev->dev, "failed to initialize host\n");
+		return ret;
+	}
+
+	return 0;
+}
+
+static int __init hisi_pcie_probe(struct platform_device *pdev)
+{
+	struct hisi_pcie *hisi_pcie;
+	struct pcie_port *pp;
+	struct resource *reg;
+	int ret;
+
+	hisi_pcie = devm_kzalloc(&pdev->dev, sizeof(*hisi_pcie), GFP_KERNEL);
+	if (!hisi_pcie)
+		return -ENOMEM;
+
+	pp = &hisi_pcie->pp;
+	pp->dev = &pdev->dev;
+
+	hisi_pcie->subctrl =
+	syscon_regmap_lookup_by_compatible("hisilicon,pcie-sas-subctrl");
+	if (IS_ERR(hisi_pcie->subctrl)) {
+		dev_err(pp->dev, "cannot get subctrl base\n");
+		return PTR_ERR(hisi_pcie->subctrl);
+	}
+
+	reg = platform_get_resource_byname(pdev, IORESOURCE_MEM, "rc_dbi");
+	hisi_pcie->reg_base = devm_ioremap_resource(&pdev->dev, reg);
+	if (IS_ERR(hisi_pcie->reg_base)) {
+		dev_err(pp->dev, "cannot get rc_dbi base\n");
+		return PTR_ERR(hisi_pcie->reg_base);
+	}
+
+	hisi_pcie->pp.dbi_base = hisi_pcie->reg_base;
+
+	ret = hisi_add_pcie_port(pp, pdev);
+	if (ret)
+		return ret;
+
+	platform_set_drvdata(pdev, hisi_pcie);
+
+	return 0;
+}
+
+static const struct of_device_id hisi_pcie_of_match[] = {
+	{.compatible = "hisilicon,hip05-pcie",},
+	{},
+};
+
+MODULE_DEVICE_TABLE(of, hisi_pcie_of_match);
+
+static struct platform_driver hisi_pcie_driver = {
+	.driver = {
+		   .name = "hisi-pcie",
+		   .of_match_table = hisi_pcie_of_match,
+	},
+};
+
+static int __init hisi_pcie_init(void)
+{
+	return platform_driver_probe(&hisi_pcie_driver, hisi_pcie_probe);
+}
+subsys_initcall(hisi_pcie_init);
-- 
1.9.1

[PATCH v10 5/6] Documentation: DT: Add HiSilicon PCIe host binding

[Zhou Wang]

This patch adds related DTS binding document for HiSilicon PCIe host driver.

Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
---
 .../bindings/arm/hisilicon/hisilicon.txt           | 17 +++++++++
 .../devicetree/bindings/pci/hisilicon-pcie.txt     | 44 ++++++++++++++++++++++
 2 files changed, 61 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/pci/hisilicon-pcie.txt

diff --git a/Documentation/devicetree/bindings/arm/hisilicon/hisilicon.txt b/Documentation/devicetree/bindings/arm/hisilicon/hisilicon.txt
index 3504dca..6ac7c00 100644
--- a/Documentation/devicetree/bindings/arm/hisilicon/hisilicon.txt
+++ b/Documentation/devicetree/bindings/arm/hisilicon/hisilicon.txt
@@ -171,6 +171,23 @@ Example:
 	};
 
 -----------------------------------------------------------------------
+Hisilicon HiP05 PCIe-SAS system controller
+
+Required properties:
+- compatible : "hisilicon,pcie-sas-subctrl", "syscon";
+- reg : Register address and size
+
+The HiP05 PCIe-SAS system controller is shared by PCIe and SAS controllers in
+HiP05 Soc to implement some basic configurations.
+
+Example:
+	/* for HiP05 PCIe-SAS system */
+	pcie_sas: system_controller@0xb0000000 {
+		compatible = "hisilicon,pcie-sas-subctrl", "syscon";
+		reg = <0xb0000000 0x10000>;
+	};
+
+-----------------------------------------------------------------------
 Hisilicon CPU controller
 
 Required properties:
diff --git a/Documentation/devicetree/bindings/pci/hisilicon-pcie.txt b/Documentation/devicetree/bindings/pci/hisilicon-pcie.txt
new file mode 100644
index 0000000..17c6ed9
--- /dev/null
+++ b/Documentation/devicetree/bindings/pci/hisilicon-pcie.txt
@@ -0,0 +1,44 @@
+HiSilicon PCIe host bridge DT description
+
+HiSilicon PCIe host controller is based on Designware PCI core.
+It shares common functions with PCIe Designware core driver and inherits
+common properties defined in
+Documentation/devicetree/bindings/pci/designware-pci.txt.
+
+Additional properties are described here:
+
+Required properties:
+- compatible: Should contain "hisilicon,hip05-pcie".
+- reg: Should contain rc_dbi, config registers location and length.
+- reg-names: Must include the following entries:
+  "rc_dbi": controller configuration registers;
+  "config": PCIe configuration space registers.
+- msi-parent: Should be its_pcie which is an ITS receiving MSI interrupts.
+- port-id: Should be 0, 1, 2 or 3.
+
+Optional properties:
+- status: Either "ok" or "disabled".
+- dma-coherent: Present if DMA operations are coherent.
+
+Example:
+	pcie@0xb0080000 {
+		compatible = "hisilicon,hip05-pcie", "snps,dw-pcie";
+		reg = <0 0xb0080000 0 0x10000>, <0x220 0x00000000 0 0x2000>;
+		reg-names = "rc_dbi", "config";
+		bus-range = <0  15>;
+		msi-parent = <&its_pcie>;
+		#address-cells = <3>;
+		#size-cells = <2>;
+		device_type = "pci";
+		dma-coherent;
+		ranges = <0x82000000 0 0x00000000 0x220 0x00000000 0 0x10000000>;
+		num-lanes = <8>;
+		port-id = <1>;
+		#interrupts-cells = <1>;
+		interrupts-map-mask = <0xf800 0 0 7>;
+		interrupts-map = <0x0 0 0 1 &mbigen_pcie 1 10
+				  0x0 0 0 2 &mbigen_pcie 2 11
+				  0x0 0 0 3 &mbigen_pcie 3 12
+				  0x0 0 0 4 &mbigen_pcie 4 13>;
+		status = "ok";
+	};
-- 
1.9.1

[PATCH v10 6/6] MAINTAINERS: Add pcie-hisi maintainer

[Zhou Wang]

Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
---
 MAINTAINERS | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 7ba7ab7..944a229 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8047,6 +8047,13 @@ S:	Maintained
 F:	Documentation/devicetree/bindings/pci/xgene-pci-msi.txt
 F:	drivers/pci/host/pci-xgene-msi.c
 
+PCIE DRIVER FOR HISILICON
+M:	Zhou Wang <wangzhou1@hisilicon.com>
+L:	linux-pci@vger.kernel.org
+S:	Maintained
+F:	Documentation/devicetree/bindings/pci/hisilicon-pcie.txt
+F:	drivers/pci/host/pcie-hisi.c
+
 PCMCIA SUBSYSTEM
 P:	Linux PCMCIA Team
 L:	linux-pcmcia@lists.infradead.org
-- 
1.9.1

Re: [PATCH v10 4/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05

[Bjorn Helgaas]

[+cc Arnd, Rob]

Hi Zhou,

I have a few minor comments and two questions: one about the fact
all the config accesses are 32 bits, and another about the use of the
"msi-parent" node.

On Sat, Oct 10, 2015 at 10:59:15AM +0800, Zhou Wang wrote:
> This patch adds PCIe host support for HiSilicon SoC Hip05.
> 
> Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
> Signed-off-by: liudongdong <liudongdong3@huawei.com>
> Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
> Signed-off-by: qiuzhenfa <qiuzhenfa@hisilicon.com>

This is a pretty long Signed-off-by chain.  If each of these people
created part of this patch, that's fine.  I'm just checking whether
that's the case, or whether some of these should be Reviewed-by,
Suggested-by, etc., per Documentation/SubmittingPatches.

> ---
>  drivers/pci/host/Kconfig     |   8 ++
>  drivers/pci/host/Makefile    |   1 +
>  drivers/pci/host/pcie-hisi.c | 320 +++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 329 insertions(+)
>  create mode 100644 drivers/pci/host/pcie-hisi.c
> 
> diff --git a/drivers/pci/host/Kconfig b/drivers/pci/host/Kconfig
> index d5e58ba..ae873be 100644
> --- a/drivers/pci/host/Kconfig
> +++ b/drivers/pci/host/Kconfig
> @@ -145,4 +145,12 @@ config PCIE_IPROC_BCMA
>  	  Say Y here if you want to use the Broadcom iProc PCIe controller
>  	  through the BCMA bus interface
>  
> +config PCI_HISI
> +	depends on OF && ARM64
> +	bool "HiSilicon SoC HIP05 PCIe controller"
> +	select PCIEPORTBUS
> +	select PCIE_DW
> +	help
> +	  Say Y here if you want PCIe controller support on HiSilicon HIP05 SoC
> +
>  endmenu
> diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile
> index 140d66f..ea1dbf2 100644
> --- a/drivers/pci/host/Makefile
> +++ b/drivers/pci/host/Makefile
> @@ -17,3 +17,4 @@ obj-$(CONFIG_PCI_VERSATILE) += pci-versatile.o
>  obj-$(CONFIG_PCIE_IPROC) += pcie-iproc.o
>  obj-$(CONFIG_PCIE_IPROC_PLATFORM) += pcie-iproc-platform.o
>  obj-$(CONFIG_PCIE_IPROC_BCMA) += pcie-iproc-bcma.o
> +obj-$(CONFIG_PCI_HISI) += pcie-hisi.o
> diff --git a/drivers/pci/host/pcie-hisi.c b/drivers/pci/host/pcie-hisi.c
> new file mode 100644
> index 0000000..26aa0d9
> --- /dev/null
> +++ b/drivers/pci/host/pcie-hisi.c
> @@ -0,0 +1,320 @@
> +/*
> + * PCIe host controller driver for HiSilicon Hip05 SoC
> + *
> + * Copyright (C) 2015 HiSilicon Co., Ltd. http://www.hisilicon.com
> + *
> + * Author: Zhou Wang <wangzhou1@hisilicon.com>
> + *         Dacai Zhu <zhudacai@hisilicon.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + */
> +#include <linux/interrupt.h>
> +#include <linux/irqdomain.h>
> +#include <linux/module.h>
> +#include <linux/mfd/syscon.h>
> +#include <linux/of.h>
> +#include <linux/of_address.h>
> +#include <linux/of_pci.h>
> +#include <linux/platform_device.h>
> +#include <linux/regmap.h>
> +
> +#include "pcie-designware.h"
> +
> +#define PCIE_SUBCTRL_MODE_REG                           0x2800
> +#define PCIE_SUBCTRL_SYS_STATE4_REG                     0x6818
> +#define PCIE_SLV_DBI_MODE                               0x0
> +#define PCIE_SLV_SYSCTRL_MODE                           0x1
> +#define PCIE_SLV_CONTENT_MODE                           0x2
> +#define PCIE_SLV_MSI_ASID                               0x10
> +#define PCIE_LTSSM_LINKUP_STATE                         0x11
> +#define PCIE_LTSSM_STATE_MASK                           0x3F
> +#define PCIE_MSI_ASID_ENABLE                            (0x1 << 12)
> +#define PCIE_MSI_ASID_VALUE                             (0x1 << 16)
> +#define PCIE_MSI_TRANS_ENABLE                           (0x1 << 12)
> +#define PCIE_MSI_TRANS_REG                              0x1c8
> +#define PCIE_MSI_LOW_ADDRESS                            0x1b4
> +#define PCIE_MSI_HIGH_ADDRESS                           0x1c4
> +#define PCIE_GITS_TRANSLATER                            0x10040

"TRANSLATOR" is the more common spelling.  But if you're matching
hardware documentation, you should follow that.

> +#define PCIE_SYS_CTRL20_REG                             0x20
> +#define PCIE_RD_TAB_SEL                                 BIT(31)
> +#define PCIE_RD_TAB_EN                                  BIT(30)
> +
> +#define to_hisi_pcie(x)	container_of(x, struct hisi_pcie, pp)
> +
> +struct hisi_pcie {
> +	struct regmap *subctrl;
> +	void __iomem *reg_base;
> +	u32 port_id;
> +	struct pcie_port pp;
> +};
> +
> +static inline void hisi_pcie_apb_writel(struct hisi_pcie *pcie,
> +					u32 val, u32 reg)
> +{
> +	writel(val, pcie->reg_base + reg);
> +}
> +
> +static inline u32 hisi_pcie_apb_readl(struct hisi_pcie *pcie, u32 reg)
> +{
> +	return readl(pcie->reg_base + reg);
> +}
> +
> +/*
> + * Change mode to indicate the same reg_base to base of PCIe host configure
> + * registers, base of RC configure space or base of vmid/asid context table

It's fine to use lower-case in the C code, but in English text, it
reads a little better to capitalize acronyms like VMID, ASID, APB, AR,
AW, etc.  

> + */
> +static void hisi_pcie_change_apb_mode(struct hisi_pcie *pcie, u32 mode)
> +{
> +	u32 val;
> +	u32 bit_mask;
> +	u32 bit_shift;
> +	u32 port_id = pcie->port_id;
> +	u32 reg = PCIE_SUBCTRL_MODE_REG + 0x100 * port_id;
> +
> +	if ((port_id == 1) || (port_id == 2)) {
> +		bit_mask = 0xc;
> +		bit_shift = 0x2;
> +	} else {
> +		bit_mask = 0x6;
> +		bit_shift = 0x1;
> +	}
> +
> +	regmap_update_bits(pcie->subctrl, reg, bit_mask, mode << bit_shift);
> +}
> +
> +/* Configure vmid/asid table in PCIe host */
> +static void hisi_pcie_config_context(struct hisi_pcie *pcie)
> +{
> +	int i;
> +	u32 val;
> +
> +	/* enable to clean vmid and asid tables though apb bus */
> +	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_SYSCTRL_MODE);
> +
> +	val = hisi_pcie_apb_readl(pcie, PCIE_SYS_CTRL20_REG);
> +	/* enable ar channel */
> +	val |= PCIE_RD_TAB_SEL | PCIE_RD_TAB_EN;
> +	hisi_pcie_apb_writel(pcie, val, PCIE_SYS_CTRL20_REG);
> +
> +	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_CONTENT_MODE);
> +
> +	/*
> +	 * init vmid and asid tables for all PCIe devices as 0

s/as 0/to 0./

> +	 * vmid table: 0 ~ 0x3ff, asid table: 0x400 ~ 0x7ff
> +	 */
> +	for (i = 0; i < 0x800; i++)
> +		hisi_pcie_apb_writel(pcie, 0x0, i * 4);
> +
> +	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_SYSCTRL_MODE);
> +
> +	/* enable aw channel */
> +	val &= (~PCIE_RD_TAB_SEL);
> +	val |= PCIE_RD_TAB_EN;
> +	hisi_pcie_apb_writel(pcie, val, PCIE_SYS_CTRL20_REG);
> +
> +	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_CONTENT_MODE);
> +
> +	for (i = 0; i < 0x800; i++)
> +		hisi_pcie_apb_writel(pcie, 0x0, i * 4);
> +
> +	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_SYSCTRL_MODE);
> +
> +	val = hisi_pcie_apb_readl(pcie, PCIE_SYS_CTRL20_REG);
> +	/* disable ar channel */
> +	val |= PCIE_RD_TAB_SEL;
> +	val &= (~PCIE_RD_TAB_EN);
> +	hisi_pcie_apb_writel(pcie, val, PCIE_SYS_CTRL20_REG);
> +	/* disable aw channel */
> +	val &= ((~PCIE_RD_TAB_SEL) & (~PCIE_RD_TAB_EN));
> +	hisi_pcie_apb_writel(pcie, val, PCIE_SYS_CTRL20_REG);
> +
> +	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_DBI_MODE);
> +}
> +
> +static int hisi_pcie_link_up(struct pcie_port *pp)
> +{
> +	u32 val;
> +	struct hisi_pcie *hisi_pcie = to_hisi_pcie(pp);
> +
> +	regmap_read(hisi_pcie->subctrl, PCIE_SUBCTRL_SYS_STATE4_REG +
> +		    0x100 * hisi_pcie->port_id, &val);
> +
> +	return ((val & PCIE_LTSSM_STATE_MASK) == PCIE_LTSSM_LINKUP_STATE);
> +}
> +
> +static
> +int hisi_pcie_msi_host_init(struct pcie_port *pp, struct msi_controller *chip)

"static int hisi_..." as in other cases (wrap before "struct
msi_controller" instead of before "int hisi_pcie_msi_host_init").

> +{
> +	u64 addr;
> +	struct device_node *msi_node;
> +	struct resource res;
> +	struct device_node *np = pp->dev->of_node;
> +	struct hisi_pcie *pcie = to_hisi_pcie(pp);
> +
> +	msi_node = of_parse_phandle(np, "msi-parent", 0);
> +	if (!msi_node) {
> +		dev_err(pp->dev, "failed to find msi-parent\n");
> +		return -EINVAL;
> +	}
> +	of_address_to_resource(msi_node, 0, &res);

Does this use the "msi-parent" node in the same way as other drivers
do?  I'm sure there must be other places where we extract struct
resource information from an "msi-parent" node, but I don't see them.

I'm trying to verify that this isn't some kind of incompatible
extension of the "msi-parent" property.  I cc'd Arnd and Rob (DT
experts).

> +	addr = res.start + PCIE_GITS_TRANSLATER;
> +
> +	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_SYSCTRL_MODE);
> +
> +	hisi_pcie_apb_writel(pcie, addr & 0xffffffff, PCIE_MSI_LOW_ADDRESS);
> +	hisi_pcie_apb_writel(pcie, addr >> 32, PCIE_MSI_HIGH_ADDRESS);
> +	hisi_pcie_apb_writel(pcie, PCIE_MSI_ASID_ENABLE | PCIE_MSI_ASID_VALUE,
> +			     PCIE_SLV_MSI_ASID);
> +	hisi_pcie_apb_writel(pcie, PCIE_MSI_TRANS_ENABLE, PCIE_MSI_TRANS_REG);
> +
> +	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_DBI_MODE);
> +
> +	return 0;
> +}
> +
> +static int hisi_pcie_cfg_write(struct pcie_port *pp, int where, int  size,
> +				u32 val)
> +{
> +	u32 reg_val;
> +	u32 reg;
> +	struct hisi_pcie *pcie = to_hisi_pcie(pp);
> +	void *walker = &reg_val;
> +
> +	walker += (where & 0x3);
> +	reg = where & ~0x3;
> +	if (size == 4)
> +		hisi_pcie_apb_writel(pcie, val, reg);
> +	else if (size == 2) {
> +		reg_val = hisi_pcie_apb_readl(pcie, reg);
> +		*(u16 __force *) walker = val;
> +		hisi_pcie_apb_writel(pcie, reg_val, reg);
> +	} else if (size == 1) {
> +		reg_val = hisi_pcie_apb_readl(pcie, reg);
> +		*(u8 __force *) walker = val;
> +		hisi_pcie_apb_writel(pcie, reg_val, reg);

This has the read/modify/write problem pointed out by Russell King:
http://lkml.kernel.org/r/E1Zenfg-0004d5-Dg@rmk-PC.arm.linux.org.uk

Is the HiSilicon hardware limited to 32-bit config accesses?  If so,
that seems like a hardware implementation defect, and this needs at
least a comment in the code.  When we figure out a good mechanism,
we'll want to taint or warn about this hardware limitation somehow.

> +	} else
> +		return PCIBIOS_BAD_REGISTER_NUMBER;
> +
> +	return PCIBIOS_SUCCESSFUL;
> +}
> +
> +static int hisi_pcie_cfg_read(struct pcie_port *pp, int where, int size,
> +			      u32 *val)
> +{
> +	u32 reg;
> +	u32 reg_val;
> +	struct hisi_pcie *pcie = to_hisi_pcie(pp);
> +	void *walker = &reg_val;
> +
> +	walker += (where & 0x3);
> +	reg = where & ~0x3;
> +	reg_val = hisi_pcie_apb_readl(pcie, reg);
> +
> +	if (size == 1)
> +		*val = *(u8 __force *) walker;
> +	else if (size == 2)
> +		*val = *(u16 __force *) walker;
> +	else if (size != 4)
> +		return PCIBIOS_BAD_REGISTER_NUMBER;
> +
> +	return PCIBIOS_SUCCESSFUL;
> +}
> +
> +static struct pcie_host_ops hisi_pcie_host_ops = {
> +	.link_up = hisi_pcie_link_up,
> +	.msi_host_init = hisi_pcie_msi_host_init,
> +	.wr_own_conf = hisi_pcie_cfg_write,
> +	.rd_own_conf = hisi_pcie_cfg_read,

Under the rule of "do it the same way unless there's a reason to be
different," please put the hisi_pcie_cfg_read() definition before the
hisi_pcie_cfg_write() definition, and change the order of these
function pointers similarly.  That matches the order in the struct
pcie_host_ops definition and in other drivers.

> +};
> +
> +static int __init hisi_add_pcie_port(struct pcie_port *pp,
> +				     struct platform_device *pdev)
> +{
> +	int ret;
> +	u32 port_id;
> +	struct hisi_pcie *hisi_pcie = to_hisi_pcie(pp);
> +
> +	if (of_property_read_u32(pdev->dev.of_node, "port-id", &port_id)) {
> +		dev_err(&pdev->dev, "failed to read port-id\n");
> +		return -EINVAL;
> +	}
> +	if (port_id > 3) {
> +		dev_err(&pdev->dev, "Invalid port-id: %d\n", port_id);
> +		return -EINVAL;
> +	}
> +	hisi_pcie->port_id = port_id;
> +
> +	pp->ops = &hisi_pcie_host_ops;
> +
> +	hisi_pcie_config_context(hisi_pcie);
> +
> +	ret = dw_pcie_host_init(pp);
> +	if (ret) {
> +		dev_err(&pdev->dev, "failed to initialize host\n");
> +		return ret;
> +	}
> +
> +	return 0;
> +}
> +
> +static int __init hisi_pcie_probe(struct platform_device *pdev)
> +{
> +	struct hisi_pcie *hisi_pcie;
> +	struct pcie_port *pp;
> +	struct resource *reg;
> +	int ret;
> +
> +	hisi_pcie = devm_kzalloc(&pdev->dev, sizeof(*hisi_pcie), GFP_KERNEL);
> +	if (!hisi_pcie)
> +		return -ENOMEM;
> +
> +	pp = &hisi_pcie->pp;
> +	pp->dev = &pdev->dev;
> +
> +	hisi_pcie->subctrl =
> +	syscon_regmap_lookup_by_compatible("hisilicon,pcie-sas-subctrl");

Indent a tab here.

> +	if (IS_ERR(hisi_pcie->subctrl)) {
> +		dev_err(pp->dev, "cannot get subctrl base\n");
> +		return PTR_ERR(hisi_pcie->subctrl);
> +	}
> +
> +	reg = platform_get_resource_byname(pdev, IORESOURCE_MEM, "rc_dbi");
> +	hisi_pcie->reg_base = devm_ioremap_resource(&pdev->dev, reg);
> +	if (IS_ERR(hisi_pcie->reg_base)) {
> +		dev_err(pp->dev, "cannot get rc_dbi base\n");
> +		return PTR_ERR(hisi_pcie->reg_base);
> +	}
> +
> +	hisi_pcie->pp.dbi_base = hisi_pcie->reg_base;
> +
> +	ret = hisi_add_pcie_port(pp, pdev);
> +	if (ret)
> +		return ret;
> +
> +	platform_set_drvdata(pdev, hisi_pcie);
> +
> +	return 0;
> +}
> +
> +static const struct of_device_id hisi_pcie_of_match[] = {
> +	{.compatible = "hisilicon,hip05-pcie",},
> +	{},
> +};
> +
> +MODULE_DEVICE_TABLE(of, hisi_pcie_of_match);
> +
> +static struct platform_driver hisi_pcie_driver = {
> +	.driver = {
> +		   .name = "hisi-pcie",
> +		   .of_match_table = hisi_pcie_of_match,
> +	},
> +};
> +
> +static int __init hisi_pcie_init(void)
> +{
> +	return platform_driver_probe(&hisi_pcie_driver, hisi_pcie_probe);
> +}
> +subsys_initcall(hisi_pcie_init);

Can you use module_platform_driver() or module_platform_driver_probe()
here instead of the subsys_initcall()?  No, I don't really know what
the difference between module_platform_driver() and
module_platform_driver_probe() is, sorry :)

Bjorn

Re: [PATCH v10 4/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05

[Zhou Wang]

On 2015/10/13 5:35, Bjorn Helgaas wrote:
> [+cc Arnd, Rob]
> 
> Hi Zhou,
> 
> I have a few minor comments and two questions: one about the fact
> all the config accesses are 32 bits, and another about the use of the
> "msi-parent" node.
> 
> On Sat, Oct 10, 2015 at 10:59:15AM +0800, Zhou Wang wrote:
>> This patch adds PCIe host support for HiSilicon SoC Hip05.
>>
>> Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
>> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
>> Signed-off-by: liudongdong <liudongdong3@huawei.com>
>> Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
>> Signed-off-by: qiuzhenfa <qiuzhenfa@hisilicon.com>
> 
> This is a pretty long Signed-off-by chain.  If each of these people
> created part of this patch, that's fine.  I'm just checking whether
> that's the case, or whether some of these should be Reviewed-by,
> Suggested-by, etc., per Documentation/SubmittingPatches.
>

Hi Bjorn,

Thanks for reminding this. I had merged patches from those colleagues,
so just add Signed-off-by from them.

>> ---
>>  drivers/pci/host/Kconfig     |   8 ++
>>  drivers/pci/host/Makefile    |   1 +
>>  drivers/pci/host/pcie-hisi.c | 320 +++++++++++++++++++++++++++++++++++++++++++
>>  3 files changed, 329 insertions(+)
>>  create mode 100644 drivers/pci/host/pcie-hisi.c
>>
>> diff --git a/drivers/pci/host/Kconfig b/drivers/pci/host/Kconfig
>> index d5e58ba..ae873be 100644
>> --- a/drivers/pci/host/Kconfig
>> +++ b/drivers/pci/host/Kconfig
>> @@ -145,4 +145,12 @@ config PCIE_IPROC_BCMA
>>  	  Say Y here if you want to use the Broadcom iProc PCIe controller
>>  	  through the BCMA bus interface
>>  
>> +config PCI_HISI
>> +	depends on OF && ARM64
>> +	bool "HiSilicon SoC HIP05 PCIe controller"
>> +	select PCIEPORTBUS
>> +	select PCIE_DW
>> +	help
>> +	  Say Y here if you want PCIe controller support on HiSilicon HIP05 SoC
>> +
>>  endmenu
>> diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile
>> index 140d66f..ea1dbf2 100644
>> --- a/drivers/pci/host/Makefile
>> +++ b/drivers/pci/host/Makefile
>> @@ -17,3 +17,4 @@ obj-$(CONFIG_PCI_VERSATILE) += pci-versatile.o
>>  obj-$(CONFIG_PCIE_IPROC) += pcie-iproc.o
>>  obj-$(CONFIG_PCIE_IPROC_PLATFORM) += pcie-iproc-platform.o
>>  obj-$(CONFIG_PCIE_IPROC_BCMA) += pcie-iproc-bcma.o
>> +obj-$(CONFIG_PCI_HISI) += pcie-hisi.o
>> diff --git a/drivers/pci/host/pcie-hisi.c b/drivers/pci/host/pcie-hisi.c
>> new file mode 100644
>> index 0000000..26aa0d9
>> --- /dev/null
>> +++ b/drivers/pci/host/pcie-hisi.c
>> @@ -0,0 +1,320 @@
>> +/*
>> + * PCIe host controller driver for HiSilicon Hip05 SoC
>> + *
>> + * Copyright (C) 2015 HiSilicon Co., Ltd. http://www.hisilicon.com
>> + *
>> + * Author: Zhou Wang <wangzhou1@hisilicon.com>
>> + *         Dacai Zhu <zhudacai@hisilicon.com>
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 as
>> + * published by the Free Software Foundation.
>> + */
>> +#include <linux/interrupt.h>
>> +#include <linux/irqdomain.h>
>> +#include <linux/module.h>
>> +#include <linux/mfd/syscon.h>
>> +#include <linux/of.h>
>> +#include <linux/of_address.h>
>> +#include <linux/of_pci.h>
>> +#include <linux/platform_device.h>
>> +#include <linux/regmap.h>
>> +
>> +#include "pcie-designware.h"
>> +
>> +#define PCIE_SUBCTRL_MODE_REG                           0x2800
>> +#define PCIE_SUBCTRL_SYS_STATE4_REG                     0x6818
>> +#define PCIE_SLV_DBI_MODE                               0x0
>> +#define PCIE_SLV_SYSCTRL_MODE                           0x1
>> +#define PCIE_SLV_CONTENT_MODE                           0x2
>> +#define PCIE_SLV_MSI_ASID                               0x10
>> +#define PCIE_LTSSM_LINKUP_STATE                         0x11
>> +#define PCIE_LTSSM_STATE_MASK                           0x3F
>> +#define PCIE_MSI_ASID_ENABLE                            (0x1 << 12)
>> +#define PCIE_MSI_ASID_VALUE                             (0x1 << 16)
>> +#define PCIE_MSI_TRANS_ENABLE                           (0x1 << 12)
>> +#define PCIE_MSI_TRANS_REG                              0x1c8
>> +#define PCIE_MSI_LOW_ADDRESS                            0x1b4
>> +#define PCIE_MSI_HIGH_ADDRESS                           0x1c4
>> +#define PCIE_GITS_TRANSLATER                            0x10040
> 
> "TRANSLATOR" is the more common spelling.  But if you're matching
> hardware documentation, you should follow that.
>

This is a copy from linux/include/linux/irqchip/arm-gic-v3.h:
#define GITS_TRANSLATER 0X10040
It indicates the offset of ITS's GITS_TRANSLATER register.

>> +#define PCIE_SYS_CTRL20_REG                             0x20
>> +#define PCIE_RD_TAB_SEL                                 BIT(31)
>> +#define PCIE_RD_TAB_EN                                  BIT(30)
>> +
>> +#define to_hisi_pcie(x)	container_of(x, struct hisi_pcie, pp)
>> +
>> +struct hisi_pcie {
>> +	struct regmap *subctrl;
>> +	void __iomem *reg_base;
>> +	u32 port_id;
>> +	struct pcie_port pp;
>> +};
>> +
>> +static inline void hisi_pcie_apb_writel(struct hisi_pcie *pcie,
>> +					u32 val, u32 reg)
>> +{
>> +	writel(val, pcie->reg_base + reg);
>> +}
>> +
>> +static inline u32 hisi_pcie_apb_readl(struct hisi_pcie *pcie, u32 reg)
>> +{
>> +	return readl(pcie->reg_base + reg);
>> +}
>> +
>> +/*
>> + * Change mode to indicate the same reg_base to base of PCIe host configure
>> + * registers, base of RC configure space or base of vmid/asid context table
> 
> It's fine to use lower-case in the C code, but in English text, it
> reads a little better to capitalize acronyms like VMID, ASID, APB, AR,
> AW, etc.  
>

Thanks for reminding this, will modify this next version.

>> + */
>> +static void hisi_pcie_change_apb_mode(struct hisi_pcie *pcie, u32 mode)
>> +{
>> +	u32 val;
>> +	u32 bit_mask;
>> +	u32 bit_shift;
>> +	u32 port_id = pcie->port_id;
>> +	u32 reg = PCIE_SUBCTRL_MODE_REG + 0x100 * port_id;
>> +
>> +	if ((port_id == 1) || (port_id == 2)) {
>> +		bit_mask = 0xc;
>> +		bit_shift = 0x2;
>> +	} else {
>> +		bit_mask = 0x6;
>> +		bit_shift = 0x1;
>> +	}
>> +
>> +	regmap_update_bits(pcie->subctrl, reg, bit_mask, mode << bit_shift);
>> +}
>> +
>> +/* Configure vmid/asid table in PCIe host */
>> +static void hisi_pcie_config_context(struct hisi_pcie *pcie)
>> +{
>> +	int i;
>> +	u32 val;
>> +
>> +	/* enable to clean vmid and asid tables though apb bus */
>> +	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_SYSCTRL_MODE);
>> +
>> +	val = hisi_pcie_apb_readl(pcie, PCIE_SYS_CTRL20_REG);
>> +	/* enable ar channel */
>> +	val |= PCIE_RD_TAB_SEL | PCIE_RD_TAB_EN;
>> +	hisi_pcie_apb_writel(pcie, val, PCIE_SYS_CTRL20_REG);
>> +
>> +	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_CONTENT_MODE);
>> +
>> +	/*
>> +	 * init vmid and asid tables for all PCIe devices as 0
> 
> s/as 0/to 0./
>

Thanks.

>> +	 * vmid table: 0 ~ 0x3ff, asid table: 0x400 ~ 0x7ff
>> +	 */
>> +	for (i = 0; i < 0x800; i++)
>> +		hisi_pcie_apb_writel(pcie, 0x0, i * 4);
>> +
>> +	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_SYSCTRL_MODE);
>> +
>> +	/* enable aw channel */
>> +	val &= (~PCIE_RD_TAB_SEL);
>> +	val |= PCIE_RD_TAB_EN;
>> +	hisi_pcie_apb_writel(pcie, val, PCIE_SYS_CTRL20_REG);
>> +
>> +	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_CONTENT_MODE);
>> +
>> +	for (i = 0; i < 0x800; i++)
>> +		hisi_pcie_apb_writel(pcie, 0x0, i * 4);
>> +
>> +	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_SYSCTRL_MODE);
>> +
>> +	val = hisi_pcie_apb_readl(pcie, PCIE_SYS_CTRL20_REG);
>> +	/* disable ar channel */
>> +	val |= PCIE_RD_TAB_SEL;
>> +	val &= (~PCIE_RD_TAB_EN);
>> +	hisi_pcie_apb_writel(pcie, val, PCIE_SYS_CTRL20_REG);
>> +	/* disable aw channel */
>> +	val &= ((~PCIE_RD_TAB_SEL) & (~PCIE_RD_TAB_EN));
>> +	hisi_pcie_apb_writel(pcie, val, PCIE_SYS_CTRL20_REG);
>> +
>> +	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_DBI_MODE);
>> +}
>> +
>> +static int hisi_pcie_link_up(struct pcie_port *pp)
>> +{
>> +	u32 val;
>> +	struct hisi_pcie *hisi_pcie = to_hisi_pcie(pp);
>> +
>> +	regmap_read(hisi_pcie->subctrl, PCIE_SUBCTRL_SYS_STATE4_REG +
>> +		    0x100 * hisi_pcie->port_id, &val);
>> +
>> +	return ((val & PCIE_LTSSM_STATE_MASK) == PCIE_LTSSM_LINKUP_STATE);
>> +}
>> +
>> +static
>> +int hisi_pcie_msi_host_init(struct pcie_port *pp, struct msi_controller *chip)
> 
> "static int hisi_..." as in other cases (wrap before "struct
> msi_controller" instead of before "int hisi_pcie_msi_host_init").
>

Thanks, will do like this.

>> +{
>> +	u64 addr;
>> +	struct device_node *msi_node;
>> +	struct resource res;
>> +	struct device_node *np = pp->dev->of_node;
>> +	struct hisi_pcie *pcie = to_hisi_pcie(pp);
>> +
>> +	msi_node = of_parse_phandle(np, "msi-parent", 0);
>> +	if (!msi_node) {
>> +		dev_err(pp->dev, "failed to find msi-parent\n");
>> +		return -EINVAL;
>> +	}
>> +	of_address_to_resource(msi_node, 0, &res);
> 
> Does this use the "msi-parent" node in the same way as other drivers
> do?  I'm sure there must be other places where we extract struct
> resource information from an "msi-parent" node, but I don't see them.
> 
> I'm trying to verify that this isn't some kind of incompatible
> extension of the "msi-parent" property.  I cc'd Arnd and Rob (DT
> experts).
>

Here I just want to get the address of related ITS' GITS_TRANSLATER register,
HiSilicon Hip05 PCIe controller need know this address to send MSI interrupt.
I think other drivers need not to do this.

struct resource above is for ITS base address which is extracted in GIC ITS
driver also: drivers/irqchip/irq-gic-v3-its.c

>> +	addr = res.start + PCIE_GITS_TRANSLATER;
>> +
>> +	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_SYSCTRL_MODE);
>> +
>> +	hisi_pcie_apb_writel(pcie, addr & 0xffffffff, PCIE_MSI_LOW_ADDRESS);
>> +	hisi_pcie_apb_writel(pcie, addr >> 32, PCIE_MSI_HIGH_ADDRESS);
>> +	hisi_pcie_apb_writel(pcie, PCIE_MSI_ASID_ENABLE | PCIE_MSI_ASID_VALUE,
>> +			     PCIE_SLV_MSI_ASID);
>> +	hisi_pcie_apb_writel(pcie, PCIE_MSI_TRANS_ENABLE, PCIE_MSI_TRANS_REG);
>> +
>> +	hisi_pcie_change_apb_mode(pcie, PCIE_SLV_DBI_MODE);
>> +
>> +	return 0;
>> +}
>> +
>> +static int hisi_pcie_cfg_write(struct pcie_port *pp, int where, int  size,
>> +				u32 val)
>> +{
>> +	u32 reg_val;
>> +	u32 reg;
>> +	struct hisi_pcie *pcie = to_hisi_pcie(pp);
>> +	void *walker = &reg_val;
>> +
>> +	walker += (where & 0x3);
>> +	reg = where & ~0x3;
>> +	if (size == 4)
>> +		hisi_pcie_apb_writel(pcie, val, reg);
>> +	else if (size == 2) {
>> +		reg_val = hisi_pcie_apb_readl(pcie, reg);
>> +		*(u16 __force *) walker = val;
>> +		hisi_pcie_apb_writel(pcie, reg_val, reg);
>> +	} else if (size == 1) {
>> +		reg_val = hisi_pcie_apb_readl(pcie, reg);
>> +		*(u8 __force *) walker = val;
>> +		hisi_pcie_apb_writel(pcie, reg_val, reg);
> 
> This has the read/modify/write problem pointed out by Russell King:
> http://lkml.kernel.org/r/E1Zenfg-0004d5-Dg@rmk-PC.arm.linux.org.uk
> 
> Is the HiSilicon hardware limited to 32-bit config accesses?  If so,
> that seems like a hardware implementation defect, and this needs at
> least a comment in the code.  When we figure out a good mechanism,
> we'll want to taint or warn about this hardware limitation somehow.
>

For config accesses of PCIe host, Hip05 PCIe only support 32-bit access
that is why we need a private callback here.

I will add a comment here in next version.

>> +	} else
>> +		return PCIBIOS_BAD_REGISTER_NUMBER;
>> +
>> +	return PCIBIOS_SUCCESSFUL;
>> +}
>> +
>> +static int hisi_pcie_cfg_read(struct pcie_port *pp, int where, int size,
>> +			      u32 *val)
>> +{
>> +	u32 reg;
>> +	u32 reg_val;
>> +	struct hisi_pcie *pcie = to_hisi_pcie(pp);
>> +	void *walker = &reg_val;
>> +
>> +	walker += (where & 0x3);
>> +	reg = where & ~0x3;
>> +	reg_val = hisi_pcie_apb_readl(pcie, reg);
>> +
>> +	if (size == 1)
>> +		*val = *(u8 __force *) walker;
>> +	else if (size == 2)
>> +		*val = *(u16 __force *) walker;
>> +	else if (size != 4)
>> +		return PCIBIOS_BAD_REGISTER_NUMBER;
>> +
>> +	return PCIBIOS_SUCCESSFUL;
>> +}
>> +
>> +static struct pcie_host_ops hisi_pcie_host_ops = {
>> +	.link_up = hisi_pcie_link_up,
>> +	.msi_host_init = hisi_pcie_msi_host_init,
>> +	.wr_own_conf = hisi_pcie_cfg_write,
>> +	.rd_own_conf = hisi_pcie_cfg_read,
> 
> Under the rule of "do it the same way unless there's a reason to be
> different," please put the hisi_pcie_cfg_read() definition before the
> hisi_pcie_cfg_write() definition, and change the order of these
> function pointers similarly.  That matches the order in the struct
> pcie_host_ops definition and in other drivers.
> 

Thanks for pointing this, will do like this.

>> +};
>> +
>> +static int __init hisi_add_pcie_port(struct pcie_port *pp,
>> +				     struct platform_device *pdev)
>> +{
>> +	int ret;
>> +	u32 port_id;
>> +	struct hisi_pcie *hisi_pcie = to_hisi_pcie(pp);
>> +
>> +	if (of_property_read_u32(pdev->dev.of_node, "port-id", &port_id)) {
>> +		dev_err(&pdev->dev, "failed to read port-id\n");
>> +		return -EINVAL;
>> +	}
>> +	if (port_id > 3) {
>> +		dev_err(&pdev->dev, "Invalid port-id: %d\n", port_id);
>> +		return -EINVAL;
>> +	}
>> +	hisi_pcie->port_id = port_id;
>> +
>> +	pp->ops = &hisi_pcie_host_ops;
>> +
>> +	hisi_pcie_config_context(hisi_pcie);
>> +
>> +	ret = dw_pcie_host_init(pp);
>> +	if (ret) {
>> +		dev_err(&pdev->dev, "failed to initialize host\n");
>> +		return ret;
>> +	}
>> +
>> +	return 0;
>> +}
>> +
>> +static int __init hisi_pcie_probe(struct platform_device *pdev)
>> +{
>> +	struct hisi_pcie *hisi_pcie;
>> +	struct pcie_port *pp;
>> +	struct resource *reg;
>> +	int ret;
>> +
>> +	hisi_pcie = devm_kzalloc(&pdev->dev, sizeof(*hisi_pcie), GFP_KERNEL);
>> +	if (!hisi_pcie)
>> +		return -ENOMEM;
>> +
>> +	pp = &hisi_pcie->pp;
>> +	pp->dev = &pdev->dev;
>> +
>> +	hisi_pcie->subctrl =
>> +	syscon_regmap_lookup_by_compatible("hisilicon,pcie-sas-subctrl");
> 
> Indent a tab here.

OK, will do like this.

> 
>> +	if (IS_ERR(hisi_pcie->subctrl)) {
>> +		dev_err(pp->dev, "cannot get subctrl base\n");
>> +		return PTR_ERR(hisi_pcie->subctrl);
>> +	}
>> +
>> +	reg = platform_get_resource_byname(pdev, IORESOURCE_MEM, "rc_dbi");
>> +	hisi_pcie->reg_base = devm_ioremap_resource(&pdev->dev, reg);
>> +	if (IS_ERR(hisi_pcie->reg_base)) {
>> +		dev_err(pp->dev, "cannot get rc_dbi base\n");
>> +		return PTR_ERR(hisi_pcie->reg_base);
>> +	}
>> +
>> +	hisi_pcie->pp.dbi_base = hisi_pcie->reg_base;
>> +
>> +	ret = hisi_add_pcie_port(pp, pdev);
>> +	if (ret)
>> +		return ret;
>> +
>> +	platform_set_drvdata(pdev, hisi_pcie);
>> +
>> +	return 0;
>> +}
>> +
>> +static const struct of_device_id hisi_pcie_of_match[] = {
>> +	{.compatible = "hisilicon,hip05-pcie",},
>> +	{},
>> +};
>> +
>> +MODULE_DEVICE_TABLE(of, hisi_pcie_of_match);
>> +
>> +static struct platform_driver hisi_pcie_driver = {
>> +	.driver = {
>> +		   .name = "hisi-pcie",
>> +		   .of_match_table = hisi_pcie_of_match,
>> +	},
>> +};
>> +
>> +static int __init hisi_pcie_init(void)
>> +{
>> +	return platform_driver_probe(&hisi_pcie_driver, hisi_pcie_probe);
>> +}
>> +subsys_initcall(hisi_pcie_init);
> 
> Can you use module_platform_driver() or module_platform_driver_probe()
> here instead of the subsys_initcall()?  No, I don't really know what
> the difference between module_platform_driver() and
> module_platform_driver_probe() is, sorry :)

In fact, I used module_platform_driver_probe in previous version, but
A PCIe VGA card of HiSilicon will use Hip05 PCIe host, so we modified
module_platform_driver_probe to subsys_initcall which will be called
before module_platform_driver_probe.

We will upstream the driver of above PCIe VGA card soon.

Many thanks for your reviewing,
Zhou

> 
> Bjorn
> 
> .
> 

RE: [PATCH v10 4/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05

[Gabriele Paoloni]

> >> +
> >> +static int __init hisi_pcie_init(void)
> >> +{
> >> +	return platform_driver_probe(&hisi_pcie_driver, hisi_pcie_probe);
> >> +}
> >> +subsys_initcall(hisi_pcie_init);
> >
> > Can you use module_platform_driver() or module_platform_driver_probe()
> > here instead of the subsys_initcall()?  No, I don't really know what
> > the difference between module_platform_driver() and
> > module_platform_driver_probe() is, sorry :)
> 
> In fact, I used module_platform_driver_probe in previous version, but
> A PCIe VGA card of HiSilicon will use Hip05 PCIe host, so we modified
> module_platform_driver_probe to subsys_initcall which will be called
> before module_platform_driver_probe.
> 
> We will upstream the driver of above PCIe VGA card soon.
> 
> Many thanks for your reviewing,
> Zhou

Hi Bjorn, firstly many thanks for looking at this.

About this last bit the reason why we use subsys_initcall() is that
our host bridge is embedded in the SoC and as such is not hot-pluggable
for instance see:
http://lxr.free-electrons.com/source/Documentation/driver-model/platform.txt#L59

Our approach is also used by Exynos:
http://lxr.free-electrons.com/source/drivers/pci/host/pci-exynos.c#L644

Cheers

Gab

> 
> >
> > Bjorn
> >
> > .
> >
> 

Re: [PATCH v10 4/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05

[Arnd Bergmann]

On Monday 12 October 2015 16:35:45 Bjorn Helgaas wrote:
> 
> > +{
> > +     u64 addr;
> > +     struct device_node *msi_node;
> > +     struct resource res;
> > +     struct device_node *np = pp->dev->of_node;
> > +     struct hisi_pcie *pcie = to_hisi_pcie(pp);
> > +
> > +     msi_node = of_parse_phandle(np, "msi-parent", 0);
> > +     if (!msi_node) {
> > +             dev_err(pp->dev, "failed to find msi-parent\n");
> > +             return -EINVAL;
> > +     }
> > +     of_address_to_resource(msi_node, 0, &res);
> 
> Does this use the "msi-parent" node in the same way as other drivers
> do?  I'm sure there must be other places where we extract struct
> resource information from an "msi-parent" node, but I don't see them.
> 
> I'm trying to verify that this isn't some kind of incompatible
> extension of the "msi-parent" property.  I cc'd Arnd and Rob (DT
> experts).

This is not ok, what this does is that it relies on a particular
implementation of the MSI controller and directly accesses its
registers.

Instead, it should reference only the msi irq domain and let the
driver for the MSI controller access the registers. Otherwise this
code has to be rewritten once the same PCI host code appears in
a machine that has a real GICv2m or GICv3.

	Arnd

Re: [PATCH v10 4/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05

[Arnd Bergmann]

On Tuesday 13 October 2015 06:58:42 Gabriele Paoloni wrote:
> 
> > >> +
> > >> +static int __init hisi_pcie_init(void)
> > >> +{
> > >> +	return platform_driver_probe(&hisi_pcie_driver, hisi_pcie_probe);
> > >> +}
> > >> +subsys_initcall(hisi_pcie_init);
> > >
> > > Can you use module_platform_driver() or module_platform_driver_probe()
> > > here instead of the subsys_initcall()?  No, I don't really know what
> > > the difference between module_platform_driver() and
> > > module_platform_driver_probe() is, sorry :)

module_platform_driver_probe() will only call the probe function once
(and fail in case of -EPROBE_DEFER), while module_platform_driver()
installs the platform driver in a way that the device can be bound
and unbound at any point.

> > In fact, I used module_platform_driver_probe in previous version, but
> > A PCIe VGA card of HiSilicon will use Hip05 PCIe host, so we modified
> > module_platform_driver_probe to subsys_initcall which will be called
> > before module_platform_driver_probe.
> > 
> > We will upstream the driver of above PCIe VGA card soon.

I don't see a reason why a VGA driver would need the PCI host to be
probed this early, unless it is the only usable console in the system.

> Hi Bjorn, firstly many thanks for looking at this.
> 
> About this last bit the reason why we use subsys_initcall() is that
> our host bridge is embedded in the SoC and as such is not hot-pluggable
> for instance see:
> http://lxr.free-electrons.com/source/Documentation/driver-model/platform.txt#L59

We should still be able to build the driver as a loadable module,
even if you don't do that on your own kernels. 

This doesn't mean that it has to be module_platform_driver, subsys_initcall
will also work in a loadable module, it just won't be as early. However,
we should try to come up with a consistent approach for all PCI host drivers,
I don't see any reason for hisi to be different from the others here.

> Our approach is also used by Exynos:
> http://lxr.free-electrons.com/source/drivers/pci/host/pci-exynos.c#L644

Maybe we should change that too.

	Arnd

RE: [PATCH v10 4/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05

[Gabriele Paoloni]

> -----Original Message-----
> From: Arnd Bergmann [mailto:arnd@arndb.de]
> Sent: Tuesday, October 13, 2015 12:12 PM
> To: Bjorn Helgaas
> Cc: Wangzhou (B); Bjorn Helgaas; jingoohan1@gmail.com;
> pratyush.anand@gmail.com; linux@arm.linux.org.uk;
> thomas.petazzoni@free-electrons.com; Gabriele Paoloni;
> lorenzo.pieralisi@arm.com; james.morse@arm.com; Liviu.Dudau@arm.com;
> jason@lakedaemon.net; robh@kernel.org; gabriel.fernandez@linaro.org;
> Minghuan.Lian@freescale.com; linux-pci@vger.kernel.org; linux-arm-
> kernel@lists.infradead.org; devicetree@vger.kernel.org; linux-
> kernel@vger.kernel.org; zhangjukuo; qiuzhenfa; liudongdong (C);
> qiujiang; xuwei (O); Liguozhu (Kenneth); Wangkefeng (Kevin); Rob
> Herring
> Subject: Re: [PATCH v10 4/6] PCI: hisi: Add PCIe host support for
> HiSilicon SoC Hip05
> 
> On Monday 12 October 2015 16:35:45 Bjorn Helgaas wrote:
> >
> > > +{
> > > +     u64 addr;
> > > +     struct device_node *msi_node;
> > > +     struct resource res;
> > > +     struct device_node *np = pp->dev->of_node;
> > > +     struct hisi_pcie *pcie = to_hisi_pcie(pp);
> > > +
> > > +     msi_node = of_parse_phandle(np, "msi-parent", 0);
> > > +     if (!msi_node) {
> > > +             dev_err(pp->dev, "failed to find msi-parent\n");
> > > +             return -EINVAL;
> > > +     }
> > > +     of_address_to_resource(msi_node, 0, &res);
> >
> > Does this use the "msi-parent" node in the same way as other drivers
> > do?  I'm sure there must be other places where we extract struct
> > resource information from an "msi-parent" node, but I don't see them.
> >
> > I'm trying to verify that this isn't some kind of incompatible
> > extension of the "msi-parent" property.  I cc'd Arnd and Rob (DT
> > experts).
> 
> This is not ok, what this does is that it relies on a particular
> implementation of the MSI controller and directly accesses its
> registers.

Hi Arnd, thanks for reviewing.

What we do is to retrieve the msi-parent physical address and we store it
in our internal PCIe register locations...

So we do not operate directly on the msi controller registers...

So I wonder if the current implementation is Ok to retrieve the
msi-parent address....

Thanks

Gab


> 
> Instead, it should reference only the msi irq domain and let the
> driver for the MSI controller access the registers. Otherwise this
> code has to be rewritten once the same PCI host code appears in
> a machine that has a real GICv2m or GICv3.
> 
> 	Arnd

Re: [PATCH v10 4/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05

[Arnd Bergmann]

On Tuesday 13 October 2015 14:49:07 Gabriele Paoloni wrote:
> > On Monday 12 October 2015 16:35:45 Bjorn Helgaas wrote:
> > >
> > > > +{
> > > > +     u64 addr;
> > > > +     struct device_node *msi_node;
> > > > +     struct resource res;
> > > > +     struct device_node *np = pp->dev->of_node;
> > > > +     struct hisi_pcie *pcie = to_hisi_pcie(pp);
> > > > +
> > > > +     msi_node = of_parse_phandle(np, "msi-parent", 0);
> > > > +     if (!msi_node) {
> > > > +             dev_err(pp->dev, "failed to find msi-parent\n");
> > > > +             return -EINVAL;
> > > > +     }
> > > > +     of_address_to_resource(msi_node, 0, &res);
> > >
> > > Does this use the "msi-parent" node in the same way as other drivers
> > > do?  I'm sure there must be other places where we extract struct
> > > resource information from an "msi-parent" node, but I don't see them.
> > >
> > > I'm trying to verify that this isn't some kind of incompatible
> > > extension of the "msi-parent" property.  I cc'd Arnd and Rob (DT
> > > experts).
> > 
> > This is not ok, what this does is that it relies on a particular
> > implementation of the MSI controller and directly accesses its
> > registers.
> 
> Hi Arnd, thanks for reviewing.
> 
> What we do is to retrieve the msi-parent physical address and we store it
> in our internal PCIe register locations...

Ah, thanks for the clarification, I missed that part.

> So we do not operate directly on the msi controller registers...
> 
> So I wonder if the current implementation is Ok to retrieve the
> msi-parent address....

Not sure. Why do you do this? Normally the PCI host does not know
or care about the address of the MSI controller, because the messages
are just passed up to the parent bus as DMA transfers.

I don't think what you do here is safe because the 'reg' property
of the MSI controller might point to the address that is used for
the message directly.

	Arnd

RE: [PATCH v10 4/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05

[Gabriele Paoloni]

> -----Original Message-----
> From: Arnd Bergmann [mailto:arnd@arndb.de]
> Sent: Tuesday, October 13, 2015 12:19 PM
> To: Gabriele Paoloni
> Cc: Wangzhou (B); Bjorn Helgaas; Bjorn Helgaas; jingoohan1@gmail.com;
> pratyush.anand@gmail.com; linux@arm.linux.org.uk;
> thomas.petazzoni@free-electrons.com; lorenzo.pieralisi@arm.com;
> james.morse@arm.com; Liviu.Dudau@arm.com; jason@lakedaemon.net;
> robh@kernel.org; gabriel.fernandez@linaro.org;
> Minghuan.Lian@freescale.com; linux-pci@vger.kernel.org; linux-arm-
> kernel@lists.infradead.org; devicetree@vger.kernel.org; linux-
> kernel@vger.kernel.org; zhangjukuo; qiuzhenfa; liudongdong (C);
> qiujiang; xuwei (O); Liguozhu (Kenneth); Wangkefeng (Kevin); Rob
> Herring
> Subject: Re: [PATCH v10 4/6] PCI: hisi: Add PCIe host support for
> HiSilicon SoC Hip05
> 
> On Tuesday 13 October 2015 06:58:42 Gabriele Paoloni wrote:
> >
> > > >> +
> > > >> +static int __init hisi_pcie_init(void)
> > > >> +{
> > > >> +	return platform_driver_probe(&hisi_pcie_driver,
> hisi_pcie_probe);
> > > >> +}
> > > >> +subsys_initcall(hisi_pcie_init);
> > > >
> > > > Can you use module_platform_driver() or
> module_platform_driver_probe()
> > > > here instead of the subsys_initcall()?  No, I don't really know
> what
> > > > the difference between module_platform_driver() and
> > > > module_platform_driver_probe() is, sorry :)
> 
> module_platform_driver_probe() will only call the probe function once
> (and fail in case of -EPROBE_DEFER), while module_platform_driver()
> installs the platform driver in a way that the device can be bound
> and unbound at any point.
> 
> > > In fact, I used module_platform_driver_probe in previous version,
> but
> > > A PCIe VGA card of HiSilicon will use Hip05 PCIe host, so we
> modified
> > > module_platform_driver_probe to subsys_initcall which will be
> called
> > > before module_platform_driver_probe.
> > >
> > > We will upstream the driver of above PCIe VGA card soon.
> 
> I don't see a reason why a VGA driver would need the PCI host to be
> probed this early, unless it is the only usable console in the system.
> 
> > Hi Bjorn, firstly many thanks for looking at this.
> >
> > About this last bit the reason why we use subsys_initcall() is that
> > our host bridge is embedded in the SoC and as such is not hot-
> pluggable
> > for instance see:
> > http://lxr.free-electrons.com/source/Documentation/driver-
> model/platform.txt#L59
> 
> We should still be able to build the driver as a loadable module,
> even if you don't do that on your own kernels.

Hi Arnd, I don't see the point of having loadable KOs for platform
devices that are integrated into SoCs (like PCIe Host Controllers...)

> 
> This doesn't mean that it has to be module_platform_driver,
> subsys_initcall
> will also work in a loadable module, it just won't be as early. However,
> we should try to come up with a consistent approach for all PCI host
> drivers,
> I don't see any reason for hisi to be different from the others here.

To me it sounds more appropriate to adopt subsys_initcall() for all the
PCI Host Bridge controllers rather than having them as loadable modules...

What is your view?

Thanks

Gab

> 
> > Our approach is also used by Exynos:
> > http://lxr.free-electrons.com/source/drivers/pci/host/pci-
> exynos.c#L644
> 
> Maybe we should change that too.
> 
> 	Arnd

Re: [PATCH v10 4/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05

[Zhou Wang]

On 2015/10/13 23:00, Arnd Bergmann wrote:
> On Tuesday 13 October 2015 14:49:07 Gabriele Paoloni wrote:
>>> On Monday 12 October 2015 16:35:45 Bjorn Helgaas wrote:
>>>>
>>>>> +{
>>>>> +     u64 addr;
>>>>> +     struct device_node *msi_node;
>>>>> +     struct resource res;
>>>>> +     struct device_node *np = pp->dev->of_node;
>>>>> +     struct hisi_pcie *pcie = to_hisi_pcie(pp);
>>>>> +
>>>>> +     msi_node = of_parse_phandle(np, "msi-parent", 0);
>>>>> +     if (!msi_node) {
>>>>> +             dev_err(pp->dev, "failed to find msi-parent\n");
>>>>> +             return -EINVAL;
>>>>> +     }
>>>>> +     of_address_to_resource(msi_node, 0, &res);
>>>>
>>>> Does this use the "msi-parent" node in the same way as other drivers
>>>> do?  I'm sure there must be other places where we extract struct
>>>> resource information from an "msi-parent" node, but I don't see them.
>>>>
>>>> I'm trying to verify that this isn't some kind of incompatible
>>>> extension of the "msi-parent" property.  I cc'd Arnd and Rob (DT
>>>> experts).
>>>
>>> This is not ok, what this does is that it relies on a particular
>>> implementation of the MSI controller and directly accesses its
>>> registers.
>>
>> Hi Arnd, thanks for reviewing.
>>
>> What we do is to retrieve the msi-parent physical address and we store it
>> in our internal PCIe register locations...
> 
> Ah, thanks for the clarification, I missed that part.
> 
>> So we do not operate directly on the msi controller registers...
>>
>> So I wonder if the current implementation is Ok to retrieve the
>> msi-parent address....
> 
> Not sure. Why do you do this? Normally the PCI host does not know
> or care about the address of the MSI controller, because the messages
> are just passed up to the parent bus as DMA transfers.
>

Hi Arnd,

In Hip05 PCIe host, it uses GITS_TRANSLATER's address to get TLP package
which contains MSI address and MSI data, and then combine BDF and MSI data
to a 32 bit data which will be writen to GITS_TRANSLATER register of ITS.

I think maybe this is a defect of our PCIe controller.

> I don't think what you do here is safe because the 'reg' property
> of the MSI controller might point to the address that is used for
> the message directly.

I see your point, however we must get address of GITS_TRANSLATER and set it to PCIe
host. How about adding necessary comments here?

Best Regards,
Zhou

>
> 	Arnd
> 
> .
> 

Re: [PATCH v10 4/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05

[Arnd Bergmann]

On Wednesday 14 October 2015 08:34:43 Gabriele Paoloni wrote:
> > -----Original Message-----
> > From: Arnd Bergmann [mailto:arnd@arndb.de]
> > Sent: Tuesday, October 13, 2015 12:19 PM
> > To: Gabriele Paoloni
> > Cc: Wangzhou (B); Bjorn Helgaas; Bjorn Helgaas; jingoohan1@gmail.com;
> > pratyush.anand@gmail.com; linux@arm.linux.org.uk;
> > thomas.petazzoni@free-electrons.com; lorenzo.pieralisi@arm.com;
> > james.morse@arm.com; Liviu.Dudau@arm.com; jason@lakedaemon.net;
> > robh@kernel.org; gabriel.fernandez@linaro.org;
> > Minghuan.Lian@freescale.com; linux-pci@vger.kernel.org; linux-arm-
> > kernel@lists.infradead.org; devicetree@vger.kernel.org; linux-
> > kernel@vger.kernel.org; zhangjukuo; qiuzhenfa; liudongdong (C);
> > qiujiang; xuwei (O); Liguozhu (Kenneth); Wangkefeng (Kevin); Rob
> > Herring
> > Subject: Re: [PATCH v10 4/6] PCI: hisi: Add PCIe host support for
> > HiSilicon SoC Hip05
> > 
> > On Tuesday 13 October 2015 06:58:42 Gabriele Paoloni wrote:
> > >
> > > > >> +
> > > > >> +static int __init hisi_pcie_init(void)
> > > > >> +{
> > > > >> +      return platform_driver_probe(&hisi_pcie_driver,
> > hisi_pcie_probe);
> > > > >> +}
> > > > >> +subsys_initcall(hisi_pcie_init);
> > > > >
> > > > > Can you use module_platform_driver() or
> > module_platform_driver_probe()
> > > > > here instead of the subsys_initcall()?  No, I don't really know
> > what
> > > > > the difference between module_platform_driver() and
> > > > > module_platform_driver_probe() is, sorry 
> > 
> > module_platform_driver_probe() will only call the probe function once
> > (and fail in case of -EPROBE_DEFER), while module_platform_driver()
> > installs the platform driver in a way that the device can be bound
> > and unbound at any point.
> > 
> > > > In fact, I used module_platform_driver_probe in previous version,
> > but
> > > > A PCIe VGA card of HiSilicon will use Hip05 PCIe host, so we
> > modified
> > > > module_platform_driver_probe to subsys_initcall which will be
> > called
> > > > before module_platform_driver_probe.
> > > >
> > > > We will upstream the driver of above PCIe VGA card soon.
> > 
> > I don't see a reason why a VGA driver would need the PCI host to be
> > probed this early, unless it is the only usable console in the system.
> > 
> > > Hi Bjorn, firstly many thanks for looking at this.
> > >
> > > About this last bit the reason why we use subsys_initcall() is that
> > > our host bridge is embedded in the SoC and as such is not hot-
> > pluggable
> > > for instance see:
> > > http://lxr.free-electrons.com/source/Documentation/driver-
> > model/platform.txt#L59
> > 
> > We should still be able to build the driver as a loadable module,
> > even if you don't do that on your own kernels.
> 
> Hi Arnd, I don't see the point of having loadable KOs for platform
> devices that are integrated into SoCs (like PCIe Host Controllers...)

Mainly we want as many drivers as possible to be loadable modules,
and there is no reason why PCI needs to be different from other
subsystems here.

> > This doesn't mean that it has to be module_platform_driver,
> > subsys_initcall
> > will also work in a loadable module, it just won't be as early. However,
> > we should try to come up with a consistent approach for all PCI host
> > drivers,
> > I don't see any reason for hisi to be different from the others here.
> 
> To me it sounds more appropriate to adopt subsys_initcall() for all the
> PCI Host Bridge controllers rather than having them as loadable modules...
> 
> What is your view?

subsys_initcall() sounds odd because it's a driver rather than a subsystem,
but I realize that most of the other levels don't fit any better.

As I said, it's not really a choice we have to make in the source code,
we can use subsys_initcall together with module_exit(), or we can
create a helper macro that is similar to module_platform_driver()
specifically for PCI that uses a particular initcall level.

	Arnd

Re: [PATCH v10 4/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05

[Arnd Bergmann]

On Wednesday 14 October 2015 16:59:03 Zhou Wang wrote:
> 
> Hi Arnd,
> 
> In Hip05 PCIe host, it uses GITS_TRANSLATER's address to get TLP package
> which contains MSI address and MSI data, and then combine BDF and MSI data
> to a 32 bit data which will be writen to GITS_TRANSLATER register of ITS.
> 
> I think maybe this is a defect of our PCIe controller.

I'd consider it a bug in the firmware if this is not set up correctly
before boot.

> > I don't think what you do here is safe because the 'reg' property
> > of the MSI controller might point to the address that is used for
> > the message directly.
> 
> I see your point, however we must get address of GITS_TRANSLATER and
> set it to PCIe host. How about adding necessary comments here?

This seems to just be static setup that should be done before Linux
is even loaded. Any reason you can't do it that way?

	Arnd

RE: [PATCH v10 4/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05

[Gabriele Paoloni]

> -----Original Message-----
> From: Arnd Bergmann [mailto:arnd@arndb.de]
> Sent: Wednesday, October 14, 2015 10:04 AM
> To: Gabriele Paoloni
> Cc: Wangzhou (B); Bjorn Helgaas; Bjorn Helgaas; jingoohan1@gmail.com;
> pratyush.anand@gmail.com; linux@arm.linux.org.uk;
> thomas.petazzoni@free-electrons.com; lorenzo.pieralisi@arm.com;
> james.morse@arm.com; Liviu.Dudau@arm.com; jason@lakedaemon.net;
> robh@kernel.org; gabriel.fernandez@linaro.org;
> Minghuan.Lian@freescale.com; linux-pci@vger.kernel.org; linux-arm-
> kernel@lists.infradead.org; devicetree@vger.kernel.org; linux-
> kernel@vger.kernel.org; zhangjukuo; qiuzhenfa; liudongdong (C);
> qiujiang; xuwei (O); Liguozhu (Kenneth); Wangkefeng (Kevin); Rob
> Herring
> Subject: Re: [PATCH v10 4/6] PCI: hisi: Add PCIe host support for
> HiSilicon SoC Hip05
> 
> On Wednesday 14 October 2015 08:34:43 Gabriele Paoloni wrote:
> > > -----Original Message-----
> > > From: Arnd Bergmann [mailto:arnd@arndb.de]
> > > Sent: Tuesday, October 13, 2015 12:19 PM
> > > To: Gabriele Paoloni
> > > Cc: Wangzhou (B); Bjorn Helgaas; Bjorn Helgaas;
> jingoohan1@gmail.com;
> > > pratyush.anand@gmail.com; linux@arm.linux.org.uk;
> > > thomas.petazzoni@free-electrons.com; lorenzo.pieralisi@arm.com;
> > > james.morse@arm.com; Liviu.Dudau@arm.com; jason@lakedaemon.net;
> > > robh@kernel.org; gabriel.fernandez@linaro.org;
> > > Minghuan.Lian@freescale.com; linux-pci@vger.kernel.org; linux-arm-
> > > kernel@lists.infradead.org; devicetree@vger.kernel.org; linux-
> > > kernel@vger.kernel.org; zhangjukuo; qiuzhenfa; liudongdong (C);
> > > qiujiang; xuwei (O); Liguozhu (Kenneth); Wangkefeng (Kevin); Rob
> > > Herring
> > > Subject: Re: [PATCH v10 4/6] PCI: hisi: Add PCIe host support for
> > > HiSilicon SoC Hip05
> > >
> > > On Tuesday 13 October 2015 06:58:42 Gabriele Paoloni wrote:
> > > >
> > > > > >> +
> > > > > >> +static int __init hisi_pcie_init(void)
> > > > > >> +{
> > > > > >> +      return platform_driver_probe(&hisi_pcie_driver,
> > > hisi_pcie_probe);
> > > > > >> +}
> > > > > >> +subsys_initcall(hisi_pcie_init);
> > > > > >
> > > > > > Can you use module_platform_driver() or
> > > module_platform_driver_probe()
> > > > > > here instead of the subsys_initcall()?  No, I don't really
> know
> > > what
> > > > > > the difference between module_platform_driver() and
> > > > > > module_platform_driver_probe() is, sorry
> > >
> > > module_platform_driver_probe() will only call the probe function
> once
> > > (and fail in case of -EPROBE_DEFER), while module_platform_driver()
> > > installs the platform driver in a way that the device can be bound
> > > and unbound at any point.
> > >
> > > > > In fact, I used module_platform_driver_probe in previous
> version,
> > > but
> > > > > A PCIe VGA card of HiSilicon will use Hip05 PCIe host, so we
> > > modified
> > > > > module_platform_driver_probe to subsys_initcall which will be
> > > called
> > > > > before module_platform_driver_probe.
> > > > >
> > > > > We will upstream the driver of above PCIe VGA card soon.
> > >
> > > I don't see a reason why a VGA driver would need the PCI host to be
> > > probed this early, unless it is the only usable console in the
> system.
> > >
> > > > Hi Bjorn, firstly many thanks for looking at this.
> > > >
> > > > About this last bit the reason why we use subsys_initcall() is
> that
> > > > our host bridge is embedded in the SoC and as such is not hot-
> > > pluggable
> > > > for instance see:
> > > > http://lxr.free-electrons.com/source/Documentation/driver-
> > > model/platform.txt#L59
> > >
> > > We should still be able to build the driver as a loadable module,
> > > even if you don't do that on your own kernels.
> >
> > Hi Arnd, I don't see the point of having loadable KOs for platform
> > devices that are integrated into SoCs (like PCIe Host Controllers...)
> 
> Mainly we want as many drivers as possible to be loadable modules,
> and there is no reason why PCI needs to be different from other
> subsystems here.

Ok I see now. Thanks

> 
> > > This doesn't mean that it has to be module_platform_driver,
> > > subsys_initcall
> > > will also work in a loadable module, it just won't be as early.
> However,
> > > we should try to come up with a consistent approach for all PCI
> host
> > > drivers,
> > > I don't see any reason for hisi to be different from the others
> here.
> >
> > To me it sounds more appropriate to adopt subsys_initcall() for all
> the
> > PCI Host Bridge controllers rather than having them as loadable
> modules...
> >
> > What is your view?
> 
> subsys_initcall() sounds odd because it's a driver rather than a
> subsystem,
> but I realize that most of the other levels don't fit any better.

Yes well I was seeing for example the vgaarb 
http://lxr.free-electrons.com/source/drivers/gpu/vga/vgaarb.c#L1357

That in the init is calling pci_get_subsys()

So I was wondering that the PCI devices may not be registered unless
we also init the PCI host bridge through subsys_initcall()...

But then maybe is the vgaarb to be buggy...

> 
> As I said, it's not really a choice we have to make in the source code,
> we can use subsys_initcall together with module_exit(), or we can
> create a helper macro that is similar to module_platform_driver()
> specifically for PCI that uses a particular initcall level.

Ok got it. But I guess this needs to be thought and applied to all
the PCI host bridge controllers...

So maybe for this driver I can use module_platform_driver_probe()
and then we can see...

> 
> 	Arnd

Re: [PATCH v10 4/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05

[Arnd Bergmann]

On Wednesday 14 October 2015 09:31:48 Gabriele Paoloni wrote:
> > 
> > > To me it sounds more appropriate to adopt subsys_initcall() for all
> > the
> > > PCI Host Bridge controllers rather than having them as loadable
> > modules...
> > >
> > > What is your view?
> > 
> > subsys_initcall() sounds odd because it's a driver rather than a
> > subsystem,
> > but I realize that most of the other levels don't fit any better.
> 
> Yes well I was seeing for example the vgaarb 
> http://lxr.free-electrons.com/source/drivers/gpu/vga/vgaarb.c#L1357
> 
> That in the init is calling pci_get_subsys()
> 
> So I was wondering that the PCI devices may not be registered unless
> we also init the PCI host bridge through subsys_initcall()...

I think this should work as is: the code first looks for devices
that are already there and then registers a notifier for devices
that show up later. This is meant to work for both devices that
are hotplugged at a later point as well as PCI buses that are
already there but not yet probed.

> But then maybe is the vgaarb to be buggy...

Possible. It may well be that the code is only tested on x86,
which always probes its PCI very early.

> > As I said, it's not really a choice we have to make in the source code,
> > we can use subsys_initcall together with module_exit(), or we can
> > create a helper macro that is similar to module_platform_driver()
> > specifically for PCI that uses a particular initcall level.
> 
> Ok got it. But I guess this needs to be thought and applied to all
> the PCI host bridge controllers...
> 
> So maybe for this driver I can use module_platform_driver_probe()
> and then we can see...

Sounds good. Let's focus on getting the driver merged first and
then follow up with a patch to get this right for all PCI hosts.

	Arnd

Re: [PATCH v10 4/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05

[Zhou Wang]

On 2015/10/14 17:06, Arnd Bergmann wrote:
> On Wednesday 14 October 2015 16:59:03 Zhou Wang wrote:
>>
>> Hi Arnd,
>>
>> In Hip05 PCIe host, it uses GITS_TRANSLATER's address to get TLP package
>> which contains MSI address and MSI data, and then combine BDF and MSI data
>> to a 32 bit data which will be writen to GITS_TRANSLATER register of ITS.
>>
>> I think maybe this is a defect of our PCIe controller.
> 
> I'd consider it a bug in the firmware if this is not set up correctly
> before boot.
> 
>>> I don't think what you do here is safe because the 'reg' property
>>> of the MSI controller might point to the address that is used for
>>> the message directly.
>>
>> I see your point, however we must get address of GITS_TRANSLATER and
>> set it to PCIe host. How about adding necessary comments here?
> 
> This seems to just be static setup that should be done before Linux
> is even loaded. Any reason you can't do it that way?
>

There are some ITSs in Hip05-D02 platform, in fact, we can use any of them
as a msi-controller,  which we can configure in dts. I am afraid that
hard-setting the value in BIOS would lead to restrictions in terms of flexibility,
as with the current implementation the same BIOS-driver can fit different
DTS structures.

Regards,
Zhou

> 	Arnd
> 
> .
> 

RE: [PATCH v10 4/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05

[Gabriele Paoloni]

> -----Original Message-----
> From: Arnd Bergmann [mailto:arnd@arndb.de]
> Sent: Wednesday, October 14, 2015 10:42 AM
> To: Gabriele Paoloni
> Cc: Wangzhou (B); Bjorn Helgaas; Bjorn Helgaas; jingoohan1@gmail.com;
> pratyush.anand@gmail.com; linux@arm.linux.org.uk;
> thomas.petazzoni@free-electrons.com; lorenzo.pieralisi@arm.com;
> james.morse@arm.com; Liviu.Dudau@arm.com; jason@lakedaemon.net;
> robh@kernel.org; gabriel.fernandez@linaro.org;
> Minghuan.Lian@freescale.com; linux-pci@vger.kernel.org; linux-arm-
> kernel@lists.infradead.org; devicetree@vger.kernel.org; linux-
> kernel@vger.kernel.org; zhangjukuo; qiuzhenfa; liudongdong (C);
> qiujiang; xuwei (O); Liguozhu (Kenneth); Wangkefeng (Kevin); Rob
> Herring
> Subject: Re: [PATCH v10 4/6] PCI: hisi: Add PCIe host support for
> HiSilicon SoC Hip05
> 
> On Wednesday 14 October 2015 09:31:48 Gabriele Paoloni wrote:
> > >
> > > > To me it sounds more appropriate to adopt subsys_initcall() for
> all
> > > the
> > > > PCI Host Bridge controllers rather than having them as loadable
> > > modules...
> > > >
> > > > What is your view?
> > >
> > > subsys_initcall() sounds odd because it's a driver rather than a
> > > subsystem,
> > > but I realize that most of the other levels don't fit any better.
> >
> > Yes well I was seeing for example the vgaarb
> > http://lxr.free-electrons.com/source/drivers/gpu/vga/vgaarb.c#L1357
> >
> > That in the init is calling pci_get_subsys()
> >
> > So I was wondering that the PCI devices may not be registered unless
> > we also init the PCI host bridge through subsys_initcall()...
> 
> I think this should work as is: the code first looks for devices
> that are already there and then registers a notifier for devices
> that show up later. This is meant to work for both devices that
> are hotplugged at a later point as well as PCI buses that are
> already there but not yet probed.
> 
> > But then maybe is the vgaarb to be buggy...
> 
> Possible. It may well be that the code is only tested on x86,
> which always probes its PCI very early.
> 
> > > As I said, it's not really a choice we have to make in the source
> code,
> > > we can use subsys_initcall together with module_exit(), or we can
> > > create a helper macro that is similar to module_platform_driver()
> > > specifically for PCI that uses a particular initcall level.
> >
> > Ok got it. But I guess this needs to be thought and applied to all
> > the PCI host bridge controllers...
> >
> > So maybe for this driver I can use module_platform_driver_probe()
> > and then we can see...
> 
> Sounds good. Let's focus on getting the driver merged first and
> then follow up with a patch to get this right for all PCI hosts.

Ok good let's do like this then

will change in v11

Again Many Thanks

Gab

> 
> 	Arnd

Re: [PATCH v10 4/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05

[Arnd Bergmann]

On Wednesday 14 October 2015 17:44:11 Zhou Wang wrote:
> On 2015/10/14 17:06, Arnd Bergmann wrote:
> > On Wednesday 14 October 2015 16:59:03 Zhou Wang wrote:
> >>
> >> Hi Arnd,
> >>
> >> In Hip05 PCIe host, it uses GITS_TRANSLATER's address to get TLP package
> >> which contains MSI address and MSI data, and then combine BDF and MSI data
> >> to a 32 bit data which will be writen to GITS_TRANSLATER register of ITS.
> >>
> >> I think maybe this is a defect of our PCIe controller.
> > 
> > I'd consider it a bug in the firmware if this is not set up correctly
> > before boot.
> > 
> >>> I don't think what you do here is safe because the 'reg' property
> >>> of the MSI controller might point to the address that is used for
> >>> the message directly.
> >>
> >> I see your point, however we must get address of GITS_TRANSLATER and
> >> set it to PCIe host. How about adding necessary comments here?
> > 
> > This seems to just be static setup that should be done before Linux
> > is even loaded. Any reason you can't do it that way?
> >
> 
> There are some ITSs in Hip05-D02 platform, in fact, we can use any of them
> as a msi-controller,  which we can configure in dts. I am afraid that
> hard-setting the value in BIOS would lead to restrictions in terms of flexibility,
> as with the current implementation the same BIOS-driver can fit different
> DTS structures.

The dtb generally should be expected to match whatever the firmware sets up,
so if there is one reasonable setting here, I see no problem with hardcoding
it that way. In particular on server systems, we usually expect the firmware
to configure almost everything in advance and just tell us how it is
configured, while on embedded systems we can't trust the bootload and
usually set it all up in the kernel from scratch.

What would be a reason to pick one ITS over another?

On a related note, don't you also need to describe in DT how PCI B/D/F
function numbers get turned into addresses in the ITS? Does that also
require configuration in the driver? I see this code here:


+       hisi_pcie_apb_writel(pcie, PCIE_MSI_ASID_ENABLE | PCIE_MSI_ASID_VALUE,
+                            PCIE_SLV_MSI_ASID);
+       hisi_pcie_apb_writel(pcie, PCIE_MSI_TRANS_ENABLE, PCIE_MSI_TRANS_REG);
+       hisi_pcie_change_apb_mode(pcie, PCIE_SLV_DBI_MODE);

plus all of hisi_pcie_config_context(). This looks like it will change
the way the MSI is interpreted. This also seems like something that
could be done in the firmware in advance, and just get reported in DT.

	Arnd

Re: [PATCH v10 4/6] PCI: hisi: Add PCIe host support for HiSilicon SoC Hip05

[Zhou Wang]

On 2015/10/15 5:56, Arnd Bergmann wrote:
> On Wednesday 14 October 2015 17:44:11 Zhou Wang wrote:
>> On 2015/10/14 17:06, Arnd Bergmann wrote:
>>> On Wednesday 14 October 2015 16:59:03 Zhou Wang wrote:
>>>>
>>>> Hi Arnd,
>>>>
>>>> In Hip05 PCIe host, it uses GITS_TRANSLATER's address to get TLP package
>>>> which contains MSI address and MSI data, and then combine BDF and MSI data
>>>> to a 32 bit data which will be writen to GITS_TRANSLATER register of ITS.
>>>>
>>>> I think maybe this is a defect of our PCIe controller.
>>>
>>> I'd consider it a bug in the firmware if this is not set up correctly
>>> before boot.
>>>
>>>>> I don't think what you do here is safe because the 'reg' property
>>>>> of the MSI controller might point to the address that is used for
>>>>> the message directly.
>>>>
>>>> I see your point, however we must get address of GITS_TRANSLATER and
>>>> set it to PCIe host. How about adding necessary comments here?
>>>
>>> This seems to just be static setup that should be done before Linux
>>> is even loaded. Any reason you can't do it that way?
>>>
>>
>> There are some ITSs in Hip05-D02 platform, in fact, we can use any of them
>> as a msi-controller,  which we can configure in dts. I am afraid that
>> hard-setting the value in BIOS would lead to restrictions in terms of flexibility,
>> as with the current implementation the same BIOS-driver can fit different
>> DTS structures.
> 
> The dtb generally should be expected to match whatever the firmware sets up,
> so if there is one reasonable setting here, I see no problem with hardcoding
> it that way. In particular on server systems, we usually expect the firmware
> to configure almost everything in advance and just tell us how it is
> configured, while on embedded systems we can't trust the bootload and
> usually set it all up in the kernel from scratch.
>

I see your point. Actually in order to support platforms without PCIe configuration
support BIOS we planned to have a further commit once the driver was upstreamed,
where we check if the link is already up, if not we would configure it in the kernel,
otherwise we would return silently.

Now about this patchset we can remove GITS_TRANSLATER address setting and do this
in BIOS together with link-up setup; then in the next commit for supporting platforms
without PCIe setup in UEFI, we can add this part back where we first check
if the link is already up (we can assume that if BIOS has configured link-up,
it has also setup the msi-parent address), if the link is up we skip reading
msi-parent address.

> What would be a reason to pick one ITS over another?

In fact, we set PCIe host and ITS binding in dts. I mean that PCIe host can
bind to any ITS nodes in system.

> 
> On a related note, don't you also need to describe in DT how PCI B/D/F
> function numbers get turned into addresses in the ITS? Does that also
> require configuration in the driver? I see this code here:

We don't need to describe this. we need only describe the relationship between
PCIe host and ITS, PCIe host will service PCIe devices which are connected to it.

> 
> 
> +       hisi_pcie_apb_writel(pcie, PCIE_MSI_ASID_ENABLE | PCIE_MSI_ASID_VALUE,
> +                            PCIE_SLV_MSI_ASID);
> +       hisi_pcie_apb_writel(pcie, PCIE_MSI_TRANS_ENABLE, PCIE_MSI_TRANS_REG);
> +       hisi_pcie_change_apb_mode(pcie, PCIE_SLV_DBI_MODE);
>

This code is to enable MSI support in PCIe host.

> plus all of hisi_pcie_config_context(). This looks like it will change
> the way the MSI is interpreted. This also seems like something that
> could be done in the firmware in advance, and just get reported in DT.

I think all hisi_pcie_config_context can be move to BIOS for this patchset,
however in order to support other BIOSs which have no PCIe setup we would plan
a future commit adding this back and working as explained above.

Thanks,
Zhou

> 
> 	Arnd
> 
> .
>