* [PATCH v5 01/12] PCI: imx6: Fix establish link failure in EP mode for iMX8MM and iMX8MP
2024-05-28 19:39 [PATCH v5 00/12] PCI: imx6: Fix\rename\clean up and add lut information for imx95 Frank Li
@ 2024-05-28 19:39 ` Frank Li
2024-05-28 19:39 ` [PATCH v5 02/12] PCI: imx6: Fix i.MX8MP PCIe EP's occasional failure to trigger MSI Frank Li
` (12 subsequent siblings)
13 siblings, 0 replies; 44+ messages in thread
From: Frank Li @ 2024-05-28 19:39 UTC (permalink / raw)
To: Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley
Cc: linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Frank Li
From: Richard Zhu <hongxing.zhu@nxp.com>
Add IMX6_PCIE_FLAG_HAS_APP_RESET flag to IMX8MM_EP and IMX8MP_EP drvdata.
This flag was overlooked during code restructuring. It is crucial to
release the app-reset from the System Reset Controller before initiating
LTSSM to rectify the issue
Fixes: 0c9651c21f2a ("PCI: imx6: Simplify reset handling by using *_FLAG_HAS_*_RESET")
Signed-off-by: Richard Zhu <hongxing.zhu@nxp.com>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
---
drivers/pci/controller/dwc/pci-imx6.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c
index 62a4994c55019..e9a16083d79d8 100644
--- a/drivers/pci/controller/dwc/pci-imx6.c
+++ b/drivers/pci/controller/dwc/pci-imx6.c
@@ -1562,7 +1562,8 @@ static const struct imx6_pcie_drvdata drvdata[] = {
},
[IMX8MM_EP] = {
.variant = IMX8MM_EP,
- .flags = IMX6_PCIE_FLAG_HAS_PHYDRV,
+ .flags = IMX6_PCIE_FLAG_HAS_APP_RESET |
+ IMX6_PCIE_FLAG_HAS_PHYDRV,
.mode = DW_PCIE_EP_TYPE,
.gpr = "fsl,imx8mm-iomuxc-gpr",
.clk_names = imx8mm_clks,
@@ -1573,7 +1574,8 @@ static const struct imx6_pcie_drvdata drvdata[] = {
},
[IMX8MP_EP] = {
.variant = IMX8MP_EP,
- .flags = IMX6_PCIE_FLAG_HAS_PHYDRV,
+ .flags = IMX6_PCIE_FLAG_HAS_APP_RESET |
+ IMX6_PCIE_FLAG_HAS_PHYDRV,
.mode = DW_PCIE_EP_TYPE,
.gpr = "fsl,imx8mp-iomuxc-gpr",
.clk_names = imx8mm_clks,
--
2.34.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 44+ messages in thread* [PATCH v5 02/12] PCI: imx6: Fix i.MX8MP PCIe EP's occasional failure to trigger MSI
2024-05-28 19:39 [PATCH v5 00/12] PCI: imx6: Fix\rename\clean up and add lut information for imx95 Frank Li
2024-05-28 19:39 ` [PATCH v5 01/12] PCI: imx6: Fix establish link failure in EP mode for iMX8MM and iMX8MP Frank Li
@ 2024-05-28 19:39 ` Frank Li
2024-05-28 19:39 ` [PATCH v5 03/12] PCI: imx6: Rename imx6_* with imx_* Frank Li
` (11 subsequent siblings)
13 siblings, 0 replies; 44+ messages in thread
From: Frank Li @ 2024-05-28 19:39 UTC (permalink / raw)
To: Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley
Cc: linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Frank Li, Jason Liu
From: Richard Zhu <hongxing.zhu@nxp.com>
Correct occasional MSI triggering failures in i.MX8MP PCIe EP by apply 64KB
hardware alignment requirement.
MSI triggering fail if the outbound MSI memory region (ep->msi_mem) is not
aligned to 64KB.
In dw_pcie_ep_init():
ep->msi_mem = pci_epc_mem_alloc_addr(epc, &ep->msi_mem_phys,
epc->mem->window.page_size);
Set ep->page_size to match drvdata::epc_features::align since different
SOCs have different alignment requirements.
Fixes: 1bd0d43dcf3b ("PCI: imx6: Clean up addr_space retrieval code")
Signed-off-by: Richard Zhu <hongxing.zhu@nxp.com>
Acked-by: Jason Liu <jason.hui.liu@nxp.com>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
---
drivers/pci/controller/dwc/pci-imx6.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c
index e9a16083d79d8..282261b2e28e5 100644
--- a/drivers/pci/controller/dwc/pci-imx6.c
+++ b/drivers/pci/controller/dwc/pci-imx6.c
@@ -1113,6 +1113,8 @@ static int imx6_add_pcie_ep(struct imx6_pcie *imx6_pcie,
if (imx6_check_flag(imx6_pcie, IMX6_PCIE_FLAG_SUPPORT_64BIT))
dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64));
+ ep->page_size = imx6_pcie->drvdata->epc_features->align;
+
ret = dw_pcie_ep_init(ep);
if (ret) {
dev_err(dev, "failed to initialize endpoint\n");
--
2.34.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 44+ messages in thread* [PATCH v5 03/12] PCI: imx6: Rename imx6_* with imx_*
2024-05-28 19:39 [PATCH v5 00/12] PCI: imx6: Fix\rename\clean up and add lut information for imx95 Frank Li
2024-05-28 19:39 ` [PATCH v5 01/12] PCI: imx6: Fix establish link failure in EP mode for iMX8MM and iMX8MP Frank Li
2024-05-28 19:39 ` [PATCH v5 02/12] PCI: imx6: Fix i.MX8MP PCIe EP's occasional failure to trigger MSI Frank Li
@ 2024-05-28 19:39 ` Frank Li
2024-05-28 19:39 ` [PATCH v5 04/12] PCI: imx6: Introduce SoC specific callbacks for controlling REFCLK Frank Li
` (10 subsequent siblings)
13 siblings, 0 replies; 44+ messages in thread
From: Frank Li @ 2024-05-28 19:39 UTC (permalink / raw)
To: Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley
Cc: linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Frank Li
Since this driver has evolved to support other i.MX SoCs such as i.MX7/8/9,
let's rename the 'imx6' prefix to 'imx' to avoid confusion. But the driver
name is left unchanged to avoid breaking userspace scripts
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
---
drivers/pci/controller/dwc/pci-imx6.c | 748 +++++++++++++++++-----------------
1 file changed, 374 insertions(+), 374 deletions(-)
diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c
index 282261b2e28e5..72e973312d203 100644
--- a/drivers/pci/controller/dwc/pci-imx6.c
+++ b/drivers/pci/controller/dwc/pci-imx6.c
@@ -54,9 +54,9 @@
#define IMX95_PE0_GEN_CTRL_3 0x1058
#define IMX95_PCIE_LTSSM_EN BIT(0)
-#define to_imx6_pcie(x) dev_get_drvdata((x)->dev)
+#define to_imx_pcie(x) dev_get_drvdata((x)->dev)
-enum imx6_pcie_variants {
+enum imx_pcie_variants {
IMX6Q,
IMX6SX,
IMX6QP,
@@ -71,25 +71,25 @@ enum imx6_pcie_variants {
IMX95_EP,
};
-#define IMX6_PCIE_FLAG_IMX6_PHY BIT(0)
-#define IMX6_PCIE_FLAG_IMX6_SPEED_CHANGE BIT(1)
-#define IMX6_PCIE_FLAG_SUPPORTS_SUSPEND BIT(2)
-#define IMX6_PCIE_FLAG_HAS_PHYDRV BIT(3)
-#define IMX6_PCIE_FLAG_HAS_APP_RESET BIT(4)
-#define IMX6_PCIE_FLAG_HAS_PHY_RESET BIT(5)
-#define IMX6_PCIE_FLAG_HAS_SERDES BIT(6)
-#define IMX6_PCIE_FLAG_SUPPORT_64BIT BIT(7)
+#define IMX_PCIE_FLAG_IMX_PHY BIT(0)
+#define IMX_PCIE_FLAG_IMX_SPEED_CHANGE BIT(1)
+#define IMX_PCIE_FLAG_SUPPORTS_SUSPEND BIT(2)
+#define IMX_PCIE_FLAG_HAS_PHYDRV BIT(3)
+#define IMX_PCIE_FLAG_HAS_APP_RESET BIT(4)
+#define IMX_PCIE_FLAG_HAS_PHY_RESET BIT(5)
+#define IMX_PCIE_FLAG_HAS_SERDES BIT(6)
+#define IMX_PCIE_FLAG_SUPPORT_64BIT BIT(7)
-#define imx6_check_flag(pci, val) (pci->drvdata->flags & val)
+#define imx_check_flag(pci, val) (pci->drvdata->flags & val)
-#define IMX6_PCIE_MAX_CLKS 6
+#define IMX_PCIE_MAX_CLKS 6
-#define IMX6_PCIE_MAX_INSTANCES 2
+#define IMX_PCIE_MAX_INSTANCES 2
-struct imx6_pcie;
+struct imx_pcie;
-struct imx6_pcie_drvdata {
- enum imx6_pcie_variants variant;
+struct imx_pcie_drvdata {
+ enum imx_pcie_variants variant;
enum dw_pcie_device_mode mode;
u32 flags;
int dbi_length;
@@ -98,17 +98,17 @@ struct imx6_pcie_drvdata {
const u32 clks_cnt;
const u32 ltssm_off;
const u32 ltssm_mask;
- const u32 mode_off[IMX6_PCIE_MAX_INSTANCES];
- const u32 mode_mask[IMX6_PCIE_MAX_INSTANCES];
+ const u32 mode_off[IMX_PCIE_MAX_INSTANCES];
+ const u32 mode_mask[IMX_PCIE_MAX_INSTANCES];
const struct pci_epc_features *epc_features;
- int (*init_phy)(struct imx6_pcie *pcie);
+ int (*init_phy)(struct imx_pcie *pcie);
};
-struct imx6_pcie {
+struct imx_pcie {
struct dw_pcie *pci;
struct gpio_desc *reset_gpiod;
bool link_is_up;
- struct clk_bulk_data clks[IMX6_PCIE_MAX_CLKS];
+ struct clk_bulk_data clks[IMX_PCIE_MAX_CLKS];
struct regmap *iomuxc_gpr;
u16 msi_ctrl;
u32 controller_id;
@@ -129,7 +129,7 @@ struct imx6_pcie {
/* power domain for pcie phy */
struct device *pd_pcie_phy;
struct phy *phy;
- const struct imx6_pcie_drvdata *drvdata;
+ const struct imx_pcie_drvdata *drvdata;
};
/* Parameters for the waiting for PCIe PHY PLL to lock on i.MX7 */
@@ -184,28 +184,28 @@ struct imx6_pcie {
#define PHY_RX_OVRD_IN_LO_RX_DATA_EN BIT(5)
#define PHY_RX_OVRD_IN_LO_RX_PLL_EN BIT(3)
-static unsigned int imx6_pcie_grp_offset(const struct imx6_pcie *imx6_pcie)
+static unsigned int imx_pcie_grp_offset(const struct imx_pcie *imx_pcie)
{
- WARN_ON(imx6_pcie->drvdata->variant != IMX8MQ &&
- imx6_pcie->drvdata->variant != IMX8MQ_EP &&
- imx6_pcie->drvdata->variant != IMX8MM &&
- imx6_pcie->drvdata->variant != IMX8MM_EP &&
- imx6_pcie->drvdata->variant != IMX8MP &&
- imx6_pcie->drvdata->variant != IMX8MP_EP);
- return imx6_pcie->controller_id == 1 ? IOMUXC_GPR16 : IOMUXC_GPR14;
+ WARN_ON(imx_pcie->drvdata->variant != IMX8MQ &&
+ imx_pcie->drvdata->variant != IMX8MQ_EP &&
+ imx_pcie->drvdata->variant != IMX8MM &&
+ imx_pcie->drvdata->variant != IMX8MM_EP &&
+ imx_pcie->drvdata->variant != IMX8MP &&
+ imx_pcie->drvdata->variant != IMX8MP_EP);
+ return imx_pcie->controller_id == 1 ? IOMUXC_GPR16 : IOMUXC_GPR14;
}
-static int imx95_pcie_init_phy(struct imx6_pcie *imx6_pcie)
+static int imx95_pcie_init_phy(struct imx_pcie *imx_pcie)
{
- regmap_update_bits(imx6_pcie->iomuxc_gpr,
+ regmap_update_bits(imx_pcie->iomuxc_gpr,
IMX95_PCIE_SS_RW_REG_0,
IMX95_PCIE_PHY_CR_PARA_SEL,
IMX95_PCIE_PHY_CR_PARA_SEL);
- regmap_update_bits(imx6_pcie->iomuxc_gpr,
+ regmap_update_bits(imx_pcie->iomuxc_gpr,
IMX95_PCIE_PHY_GEN_CTRL,
IMX95_PCIE_REF_USE_PAD, 0);
- regmap_update_bits(imx6_pcie->iomuxc_gpr,
+ regmap_update_bits(imx_pcie->iomuxc_gpr,
IMX95_PCIE_SS_RW_REG_0,
IMX95_PCIE_REF_CLKEN,
IMX95_PCIE_REF_CLKEN);
@@ -213,9 +213,9 @@ static int imx95_pcie_init_phy(struct imx6_pcie *imx6_pcie)
return 0;
}
-static void imx6_pcie_configure_type(struct imx6_pcie *imx6_pcie)
+static void imx_pcie_configure_type(struct imx_pcie *imx_pcie)
{
- const struct imx6_pcie_drvdata *drvdata = imx6_pcie->drvdata;
+ const struct imx_pcie_drvdata *drvdata = imx_pcie->drvdata;
unsigned int mask, val, mode, id;
if (drvdata->mode == DW_PCIE_EP_TYPE)
@@ -223,7 +223,7 @@ static void imx6_pcie_configure_type(struct imx6_pcie *imx6_pcie)
else
mode = PCI_EXP_TYPE_ROOT_PORT;
- id = imx6_pcie->controller_id;
+ id = imx_pcie->controller_id;
/* If mode_mask[id] is zero, means each controller have its individual gpr */
if (!drvdata->mode_mask[id])
@@ -232,12 +232,12 @@ static void imx6_pcie_configure_type(struct imx6_pcie *imx6_pcie)
mask = drvdata->mode_mask[id];
val = mode << (ffs(mask) - 1);
- regmap_update_bits(imx6_pcie->iomuxc_gpr, drvdata->mode_off[id], mask, val);
+ regmap_update_bits(imx_pcie->iomuxc_gpr, drvdata->mode_off[id], mask, val);
}
-static int pcie_phy_poll_ack(struct imx6_pcie *imx6_pcie, bool exp_val)
+static int pcie_phy_poll_ack(struct imx_pcie *imx_pcie, bool exp_val)
{
- struct dw_pcie *pci = imx6_pcie->pci;
+ struct dw_pcie *pci = imx_pcie->pci;
bool val;
u32 max_iterations = 10;
u32 wait_counter = 0;
@@ -256,9 +256,9 @@ static int pcie_phy_poll_ack(struct imx6_pcie *imx6_pcie, bool exp_val)
return -ETIMEDOUT;
}
-static int pcie_phy_wait_ack(struct imx6_pcie *imx6_pcie, int addr)
+static int pcie_phy_wait_ack(struct imx_pcie *imx_pcie, int addr)
{
- struct dw_pcie *pci = imx6_pcie->pci;
+ struct dw_pcie *pci = imx_pcie->pci;
u32 val;
int ret;
@@ -268,24 +268,24 @@ static int pcie_phy_wait_ack(struct imx6_pcie *imx6_pcie, int addr)
val |= PCIE_PHY_CTRL_CAP_ADR;
dw_pcie_writel_dbi(pci, PCIE_PHY_CTRL, val);
- ret = pcie_phy_poll_ack(imx6_pcie, true);
+ ret = pcie_phy_poll_ack(imx_pcie, true);
if (ret)
return ret;
val = PCIE_PHY_CTRL_DATA(addr);
dw_pcie_writel_dbi(pci, PCIE_PHY_CTRL, val);
- return pcie_phy_poll_ack(imx6_pcie, false);
+ return pcie_phy_poll_ack(imx_pcie, false);
}
/* Read from the 16-bit PCIe PHY control registers (not memory-mapped) */
-static int pcie_phy_read(struct imx6_pcie *imx6_pcie, int addr, u16 *data)
+static int pcie_phy_read(struct imx_pcie *imx_pcie, int addr, u16 *data)
{
- struct dw_pcie *pci = imx6_pcie->pci;
+ struct dw_pcie *pci = imx_pcie->pci;
u32 phy_ctl;
int ret;
- ret = pcie_phy_wait_ack(imx6_pcie, addr);
+ ret = pcie_phy_wait_ack(imx_pcie, addr);
if (ret)
return ret;
@@ -293,7 +293,7 @@ static int pcie_phy_read(struct imx6_pcie *imx6_pcie, int addr, u16 *data)
phy_ctl = PCIE_PHY_CTRL_RD;
dw_pcie_writel_dbi(pci, PCIE_PHY_CTRL, phy_ctl);
- ret = pcie_phy_poll_ack(imx6_pcie, true);
+ ret = pcie_phy_poll_ack(imx_pcie, true);
if (ret)
return ret;
@@ -302,18 +302,18 @@ static int pcie_phy_read(struct imx6_pcie *imx6_pcie, int addr, u16 *data)
/* deassert Read signal */
dw_pcie_writel_dbi(pci, PCIE_PHY_CTRL, 0x00);
- return pcie_phy_poll_ack(imx6_pcie, false);
+ return pcie_phy_poll_ack(imx_pcie, false);
}
-static int pcie_phy_write(struct imx6_pcie *imx6_pcie, int addr, u16 data)
+static int pcie_phy_write(struct imx_pcie *imx_pcie, int addr, u16 data)
{
- struct dw_pcie *pci = imx6_pcie->pci;
+ struct dw_pcie *pci = imx_pcie->pci;
u32 var;
int ret;
/* write addr */
/* cap addr */
- ret = pcie_phy_wait_ack(imx6_pcie, addr);
+ ret = pcie_phy_wait_ack(imx_pcie, addr);
if (ret)
return ret;
@@ -324,7 +324,7 @@ static int pcie_phy_write(struct imx6_pcie *imx6_pcie, int addr, u16 data)
var |= PCIE_PHY_CTRL_CAP_DAT;
dw_pcie_writel_dbi(pci, PCIE_PHY_CTRL, var);
- ret = pcie_phy_poll_ack(imx6_pcie, true);
+ ret = pcie_phy_poll_ack(imx_pcie, true);
if (ret)
return ret;
@@ -333,7 +333,7 @@ static int pcie_phy_write(struct imx6_pcie *imx6_pcie, int addr, u16 data)
dw_pcie_writel_dbi(pci, PCIE_PHY_CTRL, var);
/* wait for ack de-assertion */
- ret = pcie_phy_poll_ack(imx6_pcie, false);
+ ret = pcie_phy_poll_ack(imx_pcie, false);
if (ret)
return ret;
@@ -342,7 +342,7 @@ static int pcie_phy_write(struct imx6_pcie *imx6_pcie, int addr, u16 data)
dw_pcie_writel_dbi(pci, PCIE_PHY_CTRL, var);
/* wait for ack */
- ret = pcie_phy_poll_ack(imx6_pcie, true);
+ ret = pcie_phy_poll_ack(imx_pcie, true);
if (ret)
return ret;
@@ -351,7 +351,7 @@ static int pcie_phy_write(struct imx6_pcie *imx6_pcie, int addr, u16 data)
dw_pcie_writel_dbi(pci, PCIE_PHY_CTRL, var);
/* wait for ack de-assertion */
- ret = pcie_phy_poll_ack(imx6_pcie, false);
+ ret = pcie_phy_poll_ack(imx_pcie, false);
if (ret)
return ret;
@@ -360,74 +360,74 @@ static int pcie_phy_write(struct imx6_pcie *imx6_pcie, int addr, u16 data)
return 0;
}
-static int imx8mq_pcie_init_phy(struct imx6_pcie *imx6_pcie)
+static int imx8mq_pcie_init_phy(struct imx_pcie *imx_pcie)
{
/* TODO: Currently this code assumes external oscillator is being used */
- regmap_update_bits(imx6_pcie->iomuxc_gpr,
- imx6_pcie_grp_offset(imx6_pcie),
+ regmap_update_bits(imx_pcie->iomuxc_gpr,
+ imx_pcie_grp_offset(imx_pcie),
IMX8MQ_GPR_PCIE_REF_USE_PAD,
IMX8MQ_GPR_PCIE_REF_USE_PAD);
/*
* Regarding the datasheet, the PCIE_VPH is suggested to be 1.8V. If the PCIE_VPH is
* supplied by 3.3V, the VREG_BYPASS should be cleared to zero.
*/
- if (imx6_pcie->vph && regulator_get_voltage(imx6_pcie->vph) > 3000000)
- regmap_update_bits(imx6_pcie->iomuxc_gpr,
- imx6_pcie_grp_offset(imx6_pcie),
+ if (imx_pcie->vph && regulator_get_voltage(imx_pcie->vph) > 3000000)
+ regmap_update_bits(imx_pcie->iomuxc_gpr,
+ imx_pcie_grp_offset(imx_pcie),
IMX8MQ_GPR_PCIE_VREG_BYPASS,
0);
return 0;
}
-static int imx7d_pcie_init_phy(struct imx6_pcie *imx6_pcie)
+static int imx7d_pcie_init_phy(struct imx_pcie *imx_pcie)
{
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR12, IMX7D_GPR12_PCIE_PHY_REFCLK_SEL, 0);
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR12, IMX7D_GPR12_PCIE_PHY_REFCLK_SEL, 0);
return 0;
}
-static int imx6_pcie_init_phy(struct imx6_pcie *imx6_pcie)
+static int imx_pcie_init_phy(struct imx_pcie *imx_pcie)
{
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR12,
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR12,
IMX6Q_GPR12_PCIE_CTL_2, 0 << 10);
/* configure constant input signal to the pcie ctrl and phy */
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR12,
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR12,
IMX6Q_GPR12_LOS_LEVEL, 9 << 4);
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR8,
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR8,
IMX6Q_GPR8_TX_DEEMPH_GEN1,
- imx6_pcie->tx_deemph_gen1 << 0);
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR8,
+ imx_pcie->tx_deemph_gen1 << 0);
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR8,
IMX6Q_GPR8_TX_DEEMPH_GEN2_3P5DB,
- imx6_pcie->tx_deemph_gen2_3p5db << 6);
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR8,
+ imx_pcie->tx_deemph_gen2_3p5db << 6);
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR8,
IMX6Q_GPR8_TX_DEEMPH_GEN2_6DB,
- imx6_pcie->tx_deemph_gen2_6db << 12);
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR8,
+ imx_pcie->tx_deemph_gen2_6db << 12);
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR8,
IMX6Q_GPR8_TX_SWING_FULL,
- imx6_pcie->tx_swing_full << 18);
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR8,
+ imx_pcie->tx_swing_full << 18);
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR8,
IMX6Q_GPR8_TX_SWING_LOW,
- imx6_pcie->tx_swing_low << 25);
+ imx_pcie->tx_swing_low << 25);
return 0;
}
-static int imx6sx_pcie_init_phy(struct imx6_pcie *imx6_pcie)
+static int imx6sx_pcie_init_phy(struct imx_pcie *imx_pcie)
{
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR12,
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR12,
IMX6SX_GPR12_PCIE_RX_EQ_MASK, IMX6SX_GPR12_PCIE_RX_EQ_2);
- return imx6_pcie_init_phy(imx6_pcie);
+ return imx_pcie_init_phy(imx_pcie);
}
-static void imx7d_pcie_wait_for_phy_pll_lock(struct imx6_pcie *imx6_pcie)
+static void imx7d_pcie_wait_for_phy_pll_lock(struct imx_pcie *imx_pcie)
{
u32 val;
- struct device *dev = imx6_pcie->pci->dev;
+ struct device *dev = imx_pcie->pci->dev;
- if (regmap_read_poll_timeout(imx6_pcie->iomuxc_gpr,
+ if (regmap_read_poll_timeout(imx_pcie->iomuxc_gpr,
IOMUXC_GPR22, val,
val & IMX7D_GPR22_PCIE_PHY_PLL_LOCKED,
PHY_PLL_LOCK_WAIT_USLEEP_MAX,
@@ -435,19 +435,19 @@ static void imx7d_pcie_wait_for_phy_pll_lock(struct imx6_pcie *imx6_pcie)
dev_err(dev, "PCIe PLL lock timeout\n");
}
-static int imx6_setup_phy_mpll(struct imx6_pcie *imx6_pcie)
+static int imx_setup_phy_mpll(struct imx_pcie *imx_pcie)
{
unsigned long phy_rate = 0;
int mult, div;
u16 val;
int i;
- if (!(imx6_pcie->drvdata->flags & IMX6_PCIE_FLAG_IMX6_PHY))
+ if (!(imx_pcie->drvdata->flags & IMX_PCIE_FLAG_IMX_PHY))
return 0;
- for (i = 0; i < imx6_pcie->drvdata->clks_cnt; i++)
- if (strncmp(imx6_pcie->clks[i].id, "pcie_phy", 8) == 0)
- phy_rate = clk_get_rate(imx6_pcie->clks[i].clk);
+ for (i = 0; i < imx_pcie->drvdata->clks_cnt; i++)
+ if (strncmp(imx_pcie->clks[i].id, "pcie_phy", 8) == 0)
+ phy_rate = clk_get_rate(imx_pcie->clks[i].clk);
switch (phy_rate) {
case 125000000:
@@ -465,46 +465,46 @@ static int imx6_setup_phy_mpll(struct imx6_pcie *imx6_pcie)
div = 1;
break;
default:
- dev_err(imx6_pcie->pci->dev,
+ dev_err(imx_pcie->pci->dev,
"Unsupported PHY reference clock rate %lu\n", phy_rate);
return -EINVAL;
}
- pcie_phy_read(imx6_pcie, PCIE_PHY_MPLL_OVRD_IN_LO, &val);
+ pcie_phy_read(imx_pcie, PCIE_PHY_MPLL_OVRD_IN_LO, &val);
val &= ~(PCIE_PHY_MPLL_MULTIPLIER_MASK <<
PCIE_PHY_MPLL_MULTIPLIER_SHIFT);
val |= mult << PCIE_PHY_MPLL_MULTIPLIER_SHIFT;
val |= PCIE_PHY_MPLL_MULTIPLIER_OVRD;
- pcie_phy_write(imx6_pcie, PCIE_PHY_MPLL_OVRD_IN_LO, val);
+ pcie_phy_write(imx_pcie, PCIE_PHY_MPLL_OVRD_IN_LO, val);
- pcie_phy_read(imx6_pcie, PCIE_PHY_ATEOVRD, &val);
+ pcie_phy_read(imx_pcie, PCIE_PHY_ATEOVRD, &val);
val &= ~(PCIE_PHY_ATEOVRD_REF_CLKDIV_MASK <<
PCIE_PHY_ATEOVRD_REF_CLKDIV_SHIFT);
val |= div << PCIE_PHY_ATEOVRD_REF_CLKDIV_SHIFT;
val |= PCIE_PHY_ATEOVRD_EN;
- pcie_phy_write(imx6_pcie, PCIE_PHY_ATEOVRD, val);
+ pcie_phy_write(imx_pcie, PCIE_PHY_ATEOVRD, val);
return 0;
}
-static void imx6_pcie_reset_phy(struct imx6_pcie *imx6_pcie)
+static void imx_pcie_reset_phy(struct imx_pcie *imx_pcie)
{
u16 tmp;
- if (!(imx6_pcie->drvdata->flags & IMX6_PCIE_FLAG_IMX6_PHY))
+ if (!(imx_pcie->drvdata->flags & IMX_PCIE_FLAG_IMX_PHY))
return;
- pcie_phy_read(imx6_pcie, PHY_RX_OVRD_IN_LO, &tmp);
+ pcie_phy_read(imx_pcie, PHY_RX_OVRD_IN_LO, &tmp);
tmp |= (PHY_RX_OVRD_IN_LO_RX_DATA_EN |
PHY_RX_OVRD_IN_LO_RX_PLL_EN);
- pcie_phy_write(imx6_pcie, PHY_RX_OVRD_IN_LO, tmp);
+ pcie_phy_write(imx_pcie, PHY_RX_OVRD_IN_LO, tmp);
usleep_range(2000, 3000);
- pcie_phy_read(imx6_pcie, PHY_RX_OVRD_IN_LO, &tmp);
+ pcie_phy_read(imx_pcie, PHY_RX_OVRD_IN_LO, &tmp);
tmp &= ~(PHY_RX_OVRD_IN_LO_RX_DATA_EN |
PHY_RX_OVRD_IN_LO_RX_PLL_EN);
- pcie_phy_write(imx6_pcie, PHY_RX_OVRD_IN_LO, tmp);
+ pcie_phy_write(imx_pcie, PHY_RX_OVRD_IN_LO, tmp);
}
#ifdef CONFIG_ARM
@@ -543,22 +543,22 @@ static int imx6q_pcie_abort_handler(unsigned long addr,
}
#endif
-static int imx6_pcie_attach_pd(struct device *dev)
+static int imx_pcie_attach_pd(struct device *dev)
{
- struct imx6_pcie *imx6_pcie = dev_get_drvdata(dev);
+ struct imx_pcie *imx_pcie = dev_get_drvdata(dev);
struct device_link *link;
/* Do nothing when in a single power domain */
if (dev->pm_domain)
return 0;
- imx6_pcie->pd_pcie = dev_pm_domain_attach_by_name(dev, "pcie");
- if (IS_ERR(imx6_pcie->pd_pcie))
- return PTR_ERR(imx6_pcie->pd_pcie);
+ imx_pcie->pd_pcie = dev_pm_domain_attach_by_name(dev, "pcie");
+ if (IS_ERR(imx_pcie->pd_pcie))
+ return PTR_ERR(imx_pcie->pd_pcie);
/* Do nothing when power domain missing */
- if (!imx6_pcie->pd_pcie)
+ if (!imx_pcie->pd_pcie)
return 0;
- link = device_link_add(dev, imx6_pcie->pd_pcie,
+ link = device_link_add(dev, imx_pcie->pd_pcie,
DL_FLAG_STATELESS |
DL_FLAG_PM_RUNTIME |
DL_FLAG_RPM_ACTIVE);
@@ -567,11 +567,11 @@ static int imx6_pcie_attach_pd(struct device *dev)
return -EINVAL;
}
- imx6_pcie->pd_pcie_phy = dev_pm_domain_attach_by_name(dev, "pcie_phy");
- if (IS_ERR(imx6_pcie->pd_pcie_phy))
- return PTR_ERR(imx6_pcie->pd_pcie_phy);
+ imx_pcie->pd_pcie_phy = dev_pm_domain_attach_by_name(dev, "pcie_phy");
+ if (IS_ERR(imx_pcie->pd_pcie_phy))
+ return PTR_ERR(imx_pcie->pd_pcie_phy);
- link = device_link_add(dev, imx6_pcie->pd_pcie_phy,
+ link = device_link_add(dev, imx_pcie->pd_pcie_phy,
DL_FLAG_STATELESS |
DL_FLAG_PM_RUNTIME |
DL_FLAG_RPM_ACTIVE);
@@ -583,20 +583,20 @@ static int imx6_pcie_attach_pd(struct device *dev)
return 0;
}
-static int imx6_pcie_enable_ref_clk(struct imx6_pcie *imx6_pcie)
+static int imx_pcie_enable_ref_clk(struct imx_pcie *imx_pcie)
{
unsigned int offset;
int ret = 0;
- switch (imx6_pcie->drvdata->variant) {
+ switch (imx_pcie->drvdata->variant) {
case IMX6SX:
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR12,
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR12,
IMX6SX_GPR12_PCIE_TEST_POWERDOWN, 0);
break;
case IMX6QP:
case IMX6Q:
/* power up core phy and enable ref clock */
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR1,
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1,
IMX6Q_GPR1_PCIE_TEST_PD, 0 << 18);
/*
* the async reset input need ref clock to sync internally,
@@ -605,7 +605,7 @@ static int imx6_pcie_enable_ref_clk(struct imx6_pcie *imx6_pcie)
* add one ~10us delay here.
*/
usleep_range(10, 100);
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR1,
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1,
IMX6Q_GPR1_PCIE_REF_CLK_EN, 1 << 16);
break;
case IMX7D:
@@ -618,15 +618,15 @@ static int imx6_pcie_enable_ref_clk(struct imx6_pcie *imx6_pcie)
case IMX8MQ_EP:
case IMX8MP:
case IMX8MP_EP:
- offset = imx6_pcie_grp_offset(imx6_pcie);
+ offset = imx_pcie_grp_offset(imx_pcie);
/*
* Set the over ride low and enabled
* make sure that REF_CLK is turned on.
*/
- regmap_update_bits(imx6_pcie->iomuxc_gpr, offset,
+ regmap_update_bits(imx_pcie->iomuxc_gpr, offset,
IMX8MQ_GPR_PCIE_CLK_REQ_OVERRIDE,
0);
- regmap_update_bits(imx6_pcie->iomuxc_gpr, offset,
+ regmap_update_bits(imx_pcie->iomuxc_gpr, offset,
IMX8MQ_GPR_PCIE_CLK_REQ_OVERRIDE_EN,
IMX8MQ_GPR_PCIE_CLK_REQ_OVERRIDE_EN);
break;
@@ -635,19 +635,19 @@ static int imx6_pcie_enable_ref_clk(struct imx6_pcie *imx6_pcie)
return ret;
}
-static void imx6_pcie_disable_ref_clk(struct imx6_pcie *imx6_pcie)
+static void imx_pcie_disable_ref_clk(struct imx_pcie *imx_pcie)
{
- switch (imx6_pcie->drvdata->variant) {
+ switch (imx_pcie->drvdata->variant) {
case IMX6QP:
case IMX6Q:
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR1,
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1,
IMX6Q_GPR1_PCIE_REF_CLK_EN, 0);
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR1,
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1,
IMX6Q_GPR1_PCIE_TEST_PD,
IMX6Q_GPR1_PCIE_TEST_PD);
break;
case IMX7D:
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR12,
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR12,
IMX7D_GPR12_PCIE_PHY_REFCLK_SEL,
IMX7D_GPR12_PCIE_PHY_REFCLK_SEL);
break;
@@ -656,17 +656,17 @@ static void imx6_pcie_disable_ref_clk(struct imx6_pcie *imx6_pcie)
}
}
-static int imx6_pcie_clk_enable(struct imx6_pcie *imx6_pcie)
+static int imx_pcie_clk_enable(struct imx_pcie *imx_pcie)
{
- struct dw_pcie *pci = imx6_pcie->pci;
+ struct dw_pcie *pci = imx_pcie->pci;
struct device *dev = pci->dev;
int ret;
- ret = clk_bulk_prepare_enable(imx6_pcie->drvdata->clks_cnt, imx6_pcie->clks);
+ ret = clk_bulk_prepare_enable(imx_pcie->drvdata->clks_cnt, imx_pcie->clks);
if (ret)
return ret;
- ret = imx6_pcie_enable_ref_clk(imx6_pcie);
+ ret = imx_pcie_enable_ref_clk(imx_pcie);
if (ret) {
dev_err(dev, "unable to enable pcie ref clock\n");
goto err_ref_clk;
@@ -677,41 +677,41 @@ static int imx6_pcie_clk_enable(struct imx6_pcie *imx6_pcie)
return 0;
err_ref_clk:
- clk_bulk_disable_unprepare(imx6_pcie->drvdata->clks_cnt, imx6_pcie->clks);
+ clk_bulk_disable_unprepare(imx_pcie->drvdata->clks_cnt, imx_pcie->clks);
return ret;
}
-static void imx6_pcie_clk_disable(struct imx6_pcie *imx6_pcie)
+static void imx_pcie_clk_disable(struct imx_pcie *imx_pcie)
{
- imx6_pcie_disable_ref_clk(imx6_pcie);
- clk_bulk_disable_unprepare(imx6_pcie->drvdata->clks_cnt, imx6_pcie->clks);
+ imx_pcie_disable_ref_clk(imx_pcie);
+ clk_bulk_disable_unprepare(imx_pcie->drvdata->clks_cnt, imx_pcie->clks);
}
-static void imx6_pcie_assert_core_reset(struct imx6_pcie *imx6_pcie)
+static void imx_pcie_assert_core_reset(struct imx_pcie *imx_pcie)
{
- reset_control_assert(imx6_pcie->pciephy_reset);
- reset_control_assert(imx6_pcie->apps_reset);
+ reset_control_assert(imx_pcie->pciephy_reset);
+ reset_control_assert(imx_pcie->apps_reset);
- switch (imx6_pcie->drvdata->variant) {
+ switch (imx_pcie->drvdata->variant) {
case IMX6SX:
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR12,
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR12,
IMX6SX_GPR12_PCIE_TEST_POWERDOWN,
IMX6SX_GPR12_PCIE_TEST_POWERDOWN);
/* Force PCIe PHY reset */
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR5,
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR5,
IMX6SX_GPR5_PCIE_BTNRST_RESET,
IMX6SX_GPR5_PCIE_BTNRST_RESET);
break;
case IMX6QP:
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR1,
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1,
IMX6Q_GPR1_PCIE_SW_RST,
IMX6Q_GPR1_PCIE_SW_RST);
break;
case IMX6Q:
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR1,
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1,
IMX6Q_GPR1_PCIE_TEST_PD, 1 << 18);
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR1,
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1,
IMX6Q_GPR1_PCIE_REF_CLK_EN, 0 << 16);
break;
default:
@@ -719,45 +719,45 @@ static void imx6_pcie_assert_core_reset(struct imx6_pcie *imx6_pcie)
}
/* Some boards don't have PCIe reset GPIO. */
- gpiod_set_value_cansleep(imx6_pcie->reset_gpiod, 1);
+ gpiod_set_value_cansleep(imx_pcie->reset_gpiod, 1);
}
-static int imx6_pcie_deassert_core_reset(struct imx6_pcie *imx6_pcie)
+static int imx_pcie_deassert_core_reset(struct imx_pcie *imx_pcie)
{
- struct dw_pcie *pci = imx6_pcie->pci;
+ struct dw_pcie *pci = imx_pcie->pci;
struct device *dev = pci->dev;
- reset_control_deassert(imx6_pcie->pciephy_reset);
+ reset_control_deassert(imx_pcie->pciephy_reset);
- switch (imx6_pcie->drvdata->variant) {
+ switch (imx_pcie->drvdata->variant) {
case IMX7D:
/* Workaround for ERR010728, failure of PCI-e PLL VCO to
* oscillate, especially when cold. This turns off "Duty-cycle
* Corrector" and other mysterious undocumented things.
*/
- if (likely(imx6_pcie->phy_base)) {
+ if (likely(imx_pcie->phy_base)) {
/* De-assert DCC_FB_EN */
writel(PCIE_PHY_CMN_REG4_DCC_FB_EN,
- imx6_pcie->phy_base + PCIE_PHY_CMN_REG4);
+ imx_pcie->phy_base + PCIE_PHY_CMN_REG4);
/* Assert RX_EQS and RX_EQS_SEL */
writel(PCIE_PHY_CMN_REG24_RX_EQ_SEL
| PCIE_PHY_CMN_REG24_RX_EQ,
- imx6_pcie->phy_base + PCIE_PHY_CMN_REG24);
+ imx_pcie->phy_base + PCIE_PHY_CMN_REG24);
/* Assert ATT_MODE */
writel(PCIE_PHY_CMN_REG26_ATT_MODE,
- imx6_pcie->phy_base + PCIE_PHY_CMN_REG26);
+ imx_pcie->phy_base + PCIE_PHY_CMN_REG26);
} else {
dev_warn(dev, "Unable to apply ERR010728 workaround. DT missing fsl,imx7d-pcie-phy phandle ?\n");
}
- imx7d_pcie_wait_for_phy_pll_lock(imx6_pcie);
+ imx7d_pcie_wait_for_phy_pll_lock(imx_pcie);
break;
case IMX6SX:
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR5,
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR5,
IMX6SX_GPR5_PCIE_BTNRST_RESET, 0);
break;
case IMX6QP:
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR1,
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1,
IMX6Q_GPR1_PCIE_SW_RST, 0);
usleep_range(200, 500);
@@ -767,9 +767,9 @@ static int imx6_pcie_deassert_core_reset(struct imx6_pcie *imx6_pcie)
}
/* Some boards don't have PCIe reset GPIO. */
- if (imx6_pcie->reset_gpiod) {
+ if (imx_pcie->reset_gpiod) {
msleep(100);
- gpiod_set_value_cansleep(imx6_pcie->reset_gpiod, 0);
+ gpiod_set_value_cansleep(imx_pcie->reset_gpiod, 0);
/* Wait for 100ms after PERST# deassertion (PCIe r5.0, 6.6.1) */
msleep(100);
}
@@ -777,9 +777,9 @@ static int imx6_pcie_deassert_core_reset(struct imx6_pcie *imx6_pcie)
return 0;
}
-static int imx6_pcie_wait_for_speed_change(struct imx6_pcie *imx6_pcie)
+static int imx_pcie_wait_for_speed_change(struct imx_pcie *imx_pcie)
{
- struct dw_pcie *pci = imx6_pcie->pci;
+ struct dw_pcie *pci = imx_pcie->pci;
struct device *dev = pci->dev;
u32 tmp;
unsigned int retries;
@@ -796,33 +796,33 @@ static int imx6_pcie_wait_for_speed_change(struct imx6_pcie *imx6_pcie)
return -ETIMEDOUT;
}
-static void imx6_pcie_ltssm_enable(struct device *dev)
+static void imx_pcie_ltssm_enable(struct device *dev)
{
- struct imx6_pcie *imx6_pcie = dev_get_drvdata(dev);
- const struct imx6_pcie_drvdata *drvdata = imx6_pcie->drvdata;
+ struct imx_pcie *imx_pcie = dev_get_drvdata(dev);
+ const struct imx_pcie_drvdata *drvdata = imx_pcie->drvdata;
if (drvdata->ltssm_mask)
- regmap_update_bits(imx6_pcie->iomuxc_gpr, drvdata->ltssm_off, drvdata->ltssm_mask,
+ regmap_update_bits(imx_pcie->iomuxc_gpr, drvdata->ltssm_off, drvdata->ltssm_mask,
drvdata->ltssm_mask);
- reset_control_deassert(imx6_pcie->apps_reset);
+ reset_control_deassert(imx_pcie->apps_reset);
}
-static void imx6_pcie_ltssm_disable(struct device *dev)
+static void imx_pcie_ltssm_disable(struct device *dev)
{
- struct imx6_pcie *imx6_pcie = dev_get_drvdata(dev);
- const struct imx6_pcie_drvdata *drvdata = imx6_pcie->drvdata;
+ struct imx_pcie *imx_pcie = dev_get_drvdata(dev);
+ const struct imx_pcie_drvdata *drvdata = imx_pcie->drvdata;
if (drvdata->ltssm_mask)
- regmap_update_bits(imx6_pcie->iomuxc_gpr, drvdata->ltssm_off,
+ regmap_update_bits(imx_pcie->iomuxc_gpr, drvdata->ltssm_off,
drvdata->ltssm_mask, 0);
- reset_control_assert(imx6_pcie->apps_reset);
+ reset_control_assert(imx_pcie->apps_reset);
}
-static int imx6_pcie_start_link(struct dw_pcie *pci)
+static int imx_pcie_start_link(struct dw_pcie *pci)
{
- struct imx6_pcie *imx6_pcie = to_imx6_pcie(pci);
+ struct imx_pcie *imx_pcie = to_imx_pcie(pci);
struct device *dev = pci->dev;
u8 offset = dw_pcie_find_capability(pci, PCI_CAP_ID_EXP);
u32 tmp;
@@ -841,7 +841,7 @@ static int imx6_pcie_start_link(struct dw_pcie *pci)
dw_pcie_dbi_ro_wr_dis(pci);
/* Start LTSSM. */
- imx6_pcie_ltssm_enable(dev);
+ imx_pcie_ltssm_enable(dev);
ret = dw_pcie_wait_for_link(pci);
if (ret)
@@ -864,8 +864,8 @@ static int imx6_pcie_start_link(struct dw_pcie *pci)
dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, tmp);
dw_pcie_dbi_ro_wr_dis(pci);
- if (imx6_pcie->drvdata->flags &
- IMX6_PCIE_FLAG_IMX6_SPEED_CHANGE) {
+ if (imx_pcie->drvdata->flags &
+ IMX_PCIE_FLAG_IMX_SPEED_CHANGE) {
/*
* On i.MX7, DIRECT_SPEED_CHANGE behaves differently
* from i.MX6 family when no link speed transition
@@ -875,7 +875,7 @@ static int imx6_pcie_start_link(struct dw_pcie *pci)
* failure.
*/
- ret = imx6_pcie_wait_for_speed_change(imx6_pcie);
+ ret = imx_pcie_wait_for_speed_change(imx_pcie);
if (ret) {
dev_err(dev, "Failed to bring link up!\n");
goto err_reset_phy;
@@ -890,37 +890,37 @@ static int imx6_pcie_start_link(struct dw_pcie *pci)
dev_info(dev, "Link: Only Gen1 is enabled\n");
}
- imx6_pcie->link_is_up = true;
+ imx_pcie->link_is_up = true;
tmp = dw_pcie_readw_dbi(pci, offset + PCI_EXP_LNKSTA);
dev_info(dev, "Link up, Gen%i\n", tmp & PCI_EXP_LNKSTA_CLS);
return 0;
err_reset_phy:
- imx6_pcie->link_is_up = false;
+ imx_pcie->link_is_up = false;
dev_dbg(dev, "PHY DEBUG_R0=0x%08x DEBUG_R1=0x%08x\n",
dw_pcie_readl_dbi(pci, PCIE_PORT_DEBUG0),
dw_pcie_readl_dbi(pci, PCIE_PORT_DEBUG1));
- imx6_pcie_reset_phy(imx6_pcie);
+ imx_pcie_reset_phy(imx_pcie);
return 0;
}
-static void imx6_pcie_stop_link(struct dw_pcie *pci)
+static void imx_pcie_stop_link(struct dw_pcie *pci)
{
struct device *dev = pci->dev;
/* Turn off PCIe LTSSM */
- imx6_pcie_ltssm_disable(dev);
+ imx_pcie_ltssm_disable(dev);
}
-static int imx6_pcie_host_init(struct dw_pcie_rp *pp)
+static int imx_pcie_host_init(struct dw_pcie_rp *pp)
{
struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
struct device *dev = pci->dev;
- struct imx6_pcie *imx6_pcie = to_imx6_pcie(pci);
+ struct imx_pcie *imx_pcie = to_imx_pcie(pci);
int ret;
- if (imx6_pcie->vpcie) {
- ret = regulator_enable(imx6_pcie->vpcie);
+ if (imx_pcie->vpcie) {
+ ret = regulator_enable(imx_pcie->vpcie);
if (ret) {
dev_err(dev, "failed to enable vpcie regulator: %d\n",
ret);
@@ -928,83 +928,83 @@ static int imx6_pcie_host_init(struct dw_pcie_rp *pp)
}
}
- imx6_pcie_assert_core_reset(imx6_pcie);
+ imx_pcie_assert_core_reset(imx_pcie);
- if (imx6_pcie->drvdata->init_phy)
- imx6_pcie->drvdata->init_phy(imx6_pcie);
+ if (imx_pcie->drvdata->init_phy)
+ imx_pcie->drvdata->init_phy(imx_pcie);
- imx6_pcie_configure_type(imx6_pcie);
+ imx_pcie_configure_type(imx_pcie);
- ret = imx6_pcie_clk_enable(imx6_pcie);
+ ret = imx_pcie_clk_enable(imx_pcie);
if (ret) {
dev_err(dev, "unable to enable pcie clocks: %d\n", ret);
goto err_reg_disable;
}
- if (imx6_pcie->phy) {
- ret = phy_init(imx6_pcie->phy);
+ if (imx_pcie->phy) {
+ ret = phy_init(imx_pcie->phy);
if (ret) {
dev_err(dev, "pcie PHY power up failed\n");
goto err_clk_disable;
}
}
- if (imx6_pcie->phy) {
- ret = phy_power_on(imx6_pcie->phy);
+ if (imx_pcie->phy) {
+ ret = phy_power_on(imx_pcie->phy);
if (ret) {
dev_err(dev, "waiting for PHY ready timeout!\n");
goto err_phy_off;
}
}
- ret = imx6_pcie_deassert_core_reset(imx6_pcie);
+ ret = imx_pcie_deassert_core_reset(imx_pcie);
if (ret < 0) {
dev_err(dev, "pcie deassert core reset failed: %d\n", ret);
goto err_phy_off;
}
- imx6_setup_phy_mpll(imx6_pcie);
+ imx_setup_phy_mpll(imx_pcie);
return 0;
err_phy_off:
- if (imx6_pcie->phy)
- phy_exit(imx6_pcie->phy);
+ if (imx_pcie->phy)
+ phy_exit(imx_pcie->phy);
err_clk_disable:
- imx6_pcie_clk_disable(imx6_pcie);
+ imx_pcie_clk_disable(imx_pcie);
err_reg_disable:
- if (imx6_pcie->vpcie)
- regulator_disable(imx6_pcie->vpcie);
+ if (imx_pcie->vpcie)
+ regulator_disable(imx_pcie->vpcie);
return ret;
}
-static void imx6_pcie_host_exit(struct dw_pcie_rp *pp)
+static void imx_pcie_host_exit(struct dw_pcie_rp *pp)
{
struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
- struct imx6_pcie *imx6_pcie = to_imx6_pcie(pci);
+ struct imx_pcie *imx_pcie = to_imx_pcie(pci);
- if (imx6_pcie->phy) {
- if (phy_power_off(imx6_pcie->phy))
+ if (imx_pcie->phy) {
+ if (phy_power_off(imx_pcie->phy))
dev_err(pci->dev, "unable to power off PHY\n");
- phy_exit(imx6_pcie->phy);
+ phy_exit(imx_pcie->phy);
}
- imx6_pcie_clk_disable(imx6_pcie);
+ imx_pcie_clk_disable(imx_pcie);
- if (imx6_pcie->vpcie)
- regulator_disable(imx6_pcie->vpcie);
+ if (imx_pcie->vpcie)
+ regulator_disable(imx_pcie->vpcie);
}
-static const struct dw_pcie_host_ops imx6_pcie_host_ops = {
- .init = imx6_pcie_host_init,
- .deinit = imx6_pcie_host_exit,
+static const struct dw_pcie_host_ops imx_pcie_host_ops = {
+ .init = imx_pcie_host_init,
+ .deinit = imx_pcie_host_exit,
};
static const struct dw_pcie_ops dw_pcie_ops = {
- .start_link = imx6_pcie_start_link,
- .stop_link = imx6_pcie_stop_link,
+ .start_link = imx_pcie_start_link,
+ .stop_link = imx_pcie_stop_link,
};
-static void imx6_pcie_ep_init(struct dw_pcie_ep *ep)
+static void imx_pcie_ep_init(struct dw_pcie_ep *ep)
{
enum pci_barno bar;
struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
@@ -1013,7 +1013,7 @@ static void imx6_pcie_ep_init(struct dw_pcie_ep *ep)
dw_pcie_ep_reset_bar(pci, bar);
}
-static int imx6_pcie_ep_raise_irq(struct dw_pcie_ep *ep, u8 func_no,
+static int imx_pcie_ep_raise_irq(struct dw_pcie_ep *ep, u8 func_no,
unsigned int type, u16 interrupt_num)
{
struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
@@ -1060,35 +1060,35 @@ static const struct pci_epc_features imx95_pcie_epc_features = {
};
static const struct pci_epc_features*
-imx6_pcie_ep_get_features(struct dw_pcie_ep *ep)
+imx_pcie_ep_get_features(struct dw_pcie_ep *ep)
{
struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
- struct imx6_pcie *imx6_pcie = to_imx6_pcie(pci);
+ struct imx_pcie *imx_pcie = to_imx_pcie(pci);
- return imx6_pcie->drvdata->epc_features;
+ return imx_pcie->drvdata->epc_features;
}
static const struct dw_pcie_ep_ops pcie_ep_ops = {
- .init = imx6_pcie_ep_init,
- .raise_irq = imx6_pcie_ep_raise_irq,
- .get_features = imx6_pcie_ep_get_features,
+ .init = imx_pcie_ep_init,
+ .raise_irq = imx_pcie_ep_raise_irq,
+ .get_features = imx_pcie_ep_get_features,
};
-static int imx6_add_pcie_ep(struct imx6_pcie *imx6_pcie,
+static int imx_add_pcie_ep(struct imx_pcie *imx_pcie,
struct platform_device *pdev)
{
int ret;
unsigned int pcie_dbi2_offset;
struct dw_pcie_ep *ep;
- struct dw_pcie *pci = imx6_pcie->pci;
+ struct dw_pcie *pci = imx_pcie->pci;
struct dw_pcie_rp *pp = &pci->pp;
struct device *dev = pci->dev;
- imx6_pcie_host_init(pp);
+ imx_pcie_host_init(pp);
ep = &pci->ep;
ep->ops = &pcie_ep_ops;
- switch (imx6_pcie->drvdata->variant) {
+ switch (imx_pcie->drvdata->variant) {
case IMX8MQ_EP:
case IMX8MM_EP:
case IMX8MP_EP:
@@ -1110,10 +1110,10 @@ static int imx6_add_pcie_ep(struct imx6_pcie *imx6_pcie,
if (device_property_match_string(dev, "reg-names", "dbi2") >= 0)
pci->dbi_base2 = NULL;
- if (imx6_check_flag(imx6_pcie, IMX6_PCIE_FLAG_SUPPORT_64BIT))
+ if (imx_check_flag(imx_pcie, IMX_PCIE_FLAG_SUPPORT_64BIT))
dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64));
- ep->page_size = imx6_pcie->drvdata->epc_features->align;
+ ep->page_size = imx_pcie->drvdata->epc_features->align;
ret = dw_pcie_ep_init(ep);
if (ret) {
@@ -1131,30 +1131,30 @@ static int imx6_add_pcie_ep(struct imx6_pcie *imx6_pcie,
dw_pcie_ep_init_notify(ep);
/* Start LTSSM. */
- imx6_pcie_ltssm_enable(dev);
+ imx_pcie_ltssm_enable(dev);
return 0;
}
-static void imx6_pcie_pm_turnoff(struct imx6_pcie *imx6_pcie)
+static void imx_pcie_pm_turnoff(struct imx_pcie *imx_pcie)
{
- struct device *dev = imx6_pcie->pci->dev;
+ struct device *dev = imx_pcie->pci->dev;
/* Some variants have a turnoff reset in DT */
- if (imx6_pcie->turnoff_reset) {
- reset_control_assert(imx6_pcie->turnoff_reset);
- reset_control_deassert(imx6_pcie->turnoff_reset);
+ if (imx_pcie->turnoff_reset) {
+ reset_control_assert(imx_pcie->turnoff_reset);
+ reset_control_deassert(imx_pcie->turnoff_reset);
goto pm_turnoff_sleep;
}
/* Others poke directly at IOMUXC registers */
- switch (imx6_pcie->drvdata->variant) {
+ switch (imx_pcie->drvdata->variant) {
case IMX6SX:
case IMX6QP:
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR12,
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR12,
IMX6SX_GPR12_PCIE_PM_TURN_OFF,
IMX6SX_GPR12_PCIE_PM_TURN_OFF);
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR12,
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR12,
IMX6SX_GPR12_PCIE_PM_TURN_OFF, 0);
break;
default:
@@ -1173,73 +1173,73 @@ static void imx6_pcie_pm_turnoff(struct imx6_pcie *imx6_pcie)
usleep_range(1000, 10000);
}
-static void imx6_pcie_msi_save_restore(struct imx6_pcie *imx6_pcie, bool save)
+static void imx_pcie_msi_save_restore(struct imx_pcie *imx_pcie, bool save)
{
u8 offset;
u16 val;
- struct dw_pcie *pci = imx6_pcie->pci;
+ struct dw_pcie *pci = imx_pcie->pci;
if (pci_msi_enabled()) {
offset = dw_pcie_find_capability(pci, PCI_CAP_ID_MSI);
if (save) {
val = dw_pcie_readw_dbi(pci, offset + PCI_MSI_FLAGS);
- imx6_pcie->msi_ctrl = val;
+ imx_pcie->msi_ctrl = val;
} else {
dw_pcie_dbi_ro_wr_en(pci);
- val = imx6_pcie->msi_ctrl;
+ val = imx_pcie->msi_ctrl;
dw_pcie_writew_dbi(pci, offset + PCI_MSI_FLAGS, val);
dw_pcie_dbi_ro_wr_dis(pci);
}
}
}
-static int imx6_pcie_suspend_noirq(struct device *dev)
+static int imx_pcie_suspend_noirq(struct device *dev)
{
- struct imx6_pcie *imx6_pcie = dev_get_drvdata(dev);
- struct dw_pcie_rp *pp = &imx6_pcie->pci->pp;
+ struct imx_pcie *imx_pcie = dev_get_drvdata(dev);
+ struct dw_pcie_rp *pp = &imx_pcie->pci->pp;
- if (!(imx6_pcie->drvdata->flags & IMX6_PCIE_FLAG_SUPPORTS_SUSPEND))
+ if (!(imx_pcie->drvdata->flags & IMX_PCIE_FLAG_SUPPORTS_SUSPEND))
return 0;
- imx6_pcie_msi_save_restore(imx6_pcie, true);
- imx6_pcie_pm_turnoff(imx6_pcie);
- imx6_pcie_stop_link(imx6_pcie->pci);
- imx6_pcie_host_exit(pp);
+ imx_pcie_msi_save_restore(imx_pcie, true);
+ imx_pcie_pm_turnoff(imx_pcie);
+ imx_pcie_stop_link(imx_pcie->pci);
+ imx_pcie_host_exit(pp);
return 0;
}
-static int imx6_pcie_resume_noirq(struct device *dev)
+static int imx_pcie_resume_noirq(struct device *dev)
{
int ret;
- struct imx6_pcie *imx6_pcie = dev_get_drvdata(dev);
- struct dw_pcie_rp *pp = &imx6_pcie->pci->pp;
+ struct imx_pcie *imx_pcie = dev_get_drvdata(dev);
+ struct dw_pcie_rp *pp = &imx_pcie->pci->pp;
- if (!(imx6_pcie->drvdata->flags & IMX6_PCIE_FLAG_SUPPORTS_SUSPEND))
+ if (!(imx_pcie->drvdata->flags & IMX_PCIE_FLAG_SUPPORTS_SUSPEND))
return 0;
- ret = imx6_pcie_host_init(pp);
+ ret = imx_pcie_host_init(pp);
if (ret)
return ret;
- imx6_pcie_msi_save_restore(imx6_pcie, false);
+ imx_pcie_msi_save_restore(imx_pcie, false);
dw_pcie_setup_rc(pp);
- if (imx6_pcie->link_is_up)
- imx6_pcie_start_link(imx6_pcie->pci);
+ if (imx_pcie->link_is_up)
+ imx_pcie_start_link(imx_pcie->pci);
return 0;
}
-static const struct dev_pm_ops imx6_pcie_pm_ops = {
- NOIRQ_SYSTEM_SLEEP_PM_OPS(imx6_pcie_suspend_noirq,
- imx6_pcie_resume_noirq)
+static const struct dev_pm_ops imx_pcie_pm_ops = {
+ NOIRQ_SYSTEM_SLEEP_PM_OPS(imx_pcie_suspend_noirq,
+ imx_pcie_resume_noirq)
};
-static int imx6_pcie_probe(struct platform_device *pdev)
+static int imx_pcie_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
struct dw_pcie *pci;
- struct imx6_pcie *imx6_pcie;
+ struct imx_pcie *imx_pcie;
struct device_node *np;
struct resource *dbi_base;
struct device_node *node = dev->of_node;
@@ -1247,8 +1247,8 @@ static int imx6_pcie_probe(struct platform_device *pdev)
u16 val;
int i;
- imx6_pcie = devm_kzalloc(dev, sizeof(*imx6_pcie), GFP_KERNEL);
- if (!imx6_pcie)
+ imx_pcie = devm_kzalloc(dev, sizeof(*imx_pcie), GFP_KERNEL);
+ if (!imx_pcie)
return -ENOMEM;
pci = devm_kzalloc(dev, sizeof(*pci), GFP_KERNEL);
@@ -1257,10 +1257,10 @@ static int imx6_pcie_probe(struct platform_device *pdev)
pci->dev = dev;
pci->ops = &dw_pcie_ops;
- pci->pp.ops = &imx6_pcie_host_ops;
+ pci->pp.ops = &imx_pcie_host_ops;
- imx6_pcie->pci = pci;
- imx6_pcie->drvdata = of_device_get_match_data(dev);
+ imx_pcie->pci = pci;
+ imx_pcie->drvdata = of_device_get_match_data(dev);
/* Find the PHY if one is defined, only imx7d uses it */
np = of_parse_phandle(node, "fsl,imx7d-pcie-phy", 0);
@@ -1272,9 +1272,9 @@ static int imx6_pcie_probe(struct platform_device *pdev)
dev_err(dev, "Unable to map PCIe PHY\n");
return ret;
}
- imx6_pcie->phy_base = devm_ioremap_resource(dev, &res);
- if (IS_ERR(imx6_pcie->phy_base))
- return PTR_ERR(imx6_pcie->phy_base);
+ imx_pcie->phy_base = devm_ioremap_resource(dev, &res);
+ if (IS_ERR(imx_pcie->phy_base))
+ return PTR_ERR(imx_pcie->phy_base);
}
pci->dbi_base = devm_platform_get_and_ioremap_resource(pdev, 0, &dbi_base);
@@ -1282,72 +1282,72 @@ static int imx6_pcie_probe(struct platform_device *pdev)
return PTR_ERR(pci->dbi_base);
/* Fetch GPIOs */
- imx6_pcie->reset_gpiod = devm_gpiod_get_optional(dev, "reset", GPIOD_OUT_HIGH);
- if (IS_ERR(imx6_pcie->reset_gpiod))
- return dev_err_probe(dev, PTR_ERR(imx6_pcie->reset_gpiod),
+ imx_pcie->reset_gpiod = devm_gpiod_get_optional(dev, "reset", GPIOD_OUT_HIGH);
+ if (IS_ERR(imx_pcie->reset_gpiod))
+ return dev_err_probe(dev, PTR_ERR(imx_pcie->reset_gpiod),
"unable to get reset gpio\n");
- gpiod_set_consumer_name(imx6_pcie->reset_gpiod, "PCIe reset");
+ gpiod_set_consumer_name(imx_pcie->reset_gpiod, "PCIe reset");
- if (imx6_pcie->drvdata->clks_cnt >= IMX6_PCIE_MAX_CLKS)
+ if (imx_pcie->drvdata->clks_cnt >= IMX_PCIE_MAX_CLKS)
return dev_err_probe(dev, -ENOMEM, "clks_cnt is too big\n");
- for (i = 0; i < imx6_pcie->drvdata->clks_cnt; i++)
- imx6_pcie->clks[i].id = imx6_pcie->drvdata->clk_names[i];
+ for (i = 0; i < imx_pcie->drvdata->clks_cnt; i++)
+ imx_pcie->clks[i].id = imx_pcie->drvdata->clk_names[i];
/* Fetch clocks */
- ret = devm_clk_bulk_get(dev, imx6_pcie->drvdata->clks_cnt, imx6_pcie->clks);
+ ret = devm_clk_bulk_get(dev, imx_pcie->drvdata->clks_cnt, imx_pcie->clks);
if (ret)
return ret;
- if (imx6_check_flag(imx6_pcie, IMX6_PCIE_FLAG_HAS_PHYDRV)) {
- imx6_pcie->phy = devm_phy_get(dev, "pcie-phy");
- if (IS_ERR(imx6_pcie->phy))
- return dev_err_probe(dev, PTR_ERR(imx6_pcie->phy),
+ if (imx_check_flag(imx_pcie, IMX_PCIE_FLAG_HAS_PHYDRV)) {
+ imx_pcie->phy = devm_phy_get(dev, "pcie-phy");
+ if (IS_ERR(imx_pcie->phy))
+ return dev_err_probe(dev, PTR_ERR(imx_pcie->phy),
"failed to get pcie phy\n");
}
- if (imx6_check_flag(imx6_pcie, IMX6_PCIE_FLAG_HAS_APP_RESET)) {
- imx6_pcie->apps_reset = devm_reset_control_get_exclusive(dev, "apps");
- if (IS_ERR(imx6_pcie->apps_reset))
- return dev_err_probe(dev, PTR_ERR(imx6_pcie->apps_reset),
+ if (imx_check_flag(imx_pcie, IMX_PCIE_FLAG_HAS_APP_RESET)) {
+ imx_pcie->apps_reset = devm_reset_control_get_exclusive(dev, "apps");
+ if (IS_ERR(imx_pcie->apps_reset))
+ return dev_err_probe(dev, PTR_ERR(imx_pcie->apps_reset),
"failed to get pcie apps reset control\n");
}
- if (imx6_check_flag(imx6_pcie, IMX6_PCIE_FLAG_HAS_PHY_RESET)) {
- imx6_pcie->pciephy_reset = devm_reset_control_get_exclusive(dev, "pciephy");
- if (IS_ERR(imx6_pcie->pciephy_reset))
- return dev_err_probe(dev, PTR_ERR(imx6_pcie->pciephy_reset),
+ if (imx_check_flag(imx_pcie, IMX_PCIE_FLAG_HAS_PHY_RESET)) {
+ imx_pcie->pciephy_reset = devm_reset_control_get_exclusive(dev, "pciephy");
+ if (IS_ERR(imx_pcie->pciephy_reset))
+ return dev_err_probe(dev, PTR_ERR(imx_pcie->pciephy_reset),
"Failed to get PCIEPHY reset control\n");
}
- switch (imx6_pcie->drvdata->variant) {
+ switch (imx_pcie->drvdata->variant) {
case IMX8MQ:
case IMX8MQ_EP:
case IMX7D:
if (dbi_base->start == IMX8MQ_PCIE2_BASE_ADDR)
- imx6_pcie->controller_id = 1;
+ imx_pcie->controller_id = 1;
break;
default:
break;
}
/* Grab turnoff reset */
- imx6_pcie->turnoff_reset = devm_reset_control_get_optional_exclusive(dev, "turnoff");
- if (IS_ERR(imx6_pcie->turnoff_reset)) {
+ imx_pcie->turnoff_reset = devm_reset_control_get_optional_exclusive(dev, "turnoff");
+ if (IS_ERR(imx_pcie->turnoff_reset)) {
dev_err(dev, "Failed to get TURNOFF reset control\n");
- return PTR_ERR(imx6_pcie->turnoff_reset);
+ return PTR_ERR(imx_pcie->turnoff_reset);
}
- if (imx6_pcie->drvdata->gpr) {
+ if (imx_pcie->drvdata->gpr) {
/* Grab GPR config register range */
- imx6_pcie->iomuxc_gpr =
- syscon_regmap_lookup_by_compatible(imx6_pcie->drvdata->gpr);
- if (IS_ERR(imx6_pcie->iomuxc_gpr))
- return dev_err_probe(dev, PTR_ERR(imx6_pcie->iomuxc_gpr),
+ imx_pcie->iomuxc_gpr =
+ syscon_regmap_lookup_by_compatible(imx_pcie->drvdata->gpr);
+ if (IS_ERR(imx_pcie->iomuxc_gpr))
+ return dev_err_probe(dev, PTR_ERR(imx_pcie->iomuxc_gpr),
"unable to find iomuxc registers\n");
}
- if (imx6_check_flag(imx6_pcie, IMX6_PCIE_FLAG_HAS_SERDES)) {
+ if (imx_check_flag(imx_pcie, IMX_PCIE_FLAG_HAS_SERDES)) {
void __iomem *off = devm_platform_ioremap_resource_byname(pdev, "app");
if (IS_ERR(off))
@@ -1360,59 +1360,59 @@ static int imx6_pcie_probe(struct platform_device *pdev)
.reg_stride = 4,
};
- imx6_pcie->iomuxc_gpr = devm_regmap_init_mmio(dev, off, ®map_config);
- if (IS_ERR(imx6_pcie->iomuxc_gpr))
- return dev_err_probe(dev, PTR_ERR(imx6_pcie->iomuxc_gpr),
+ imx_pcie->iomuxc_gpr = devm_regmap_init_mmio(dev, off, ®map_config);
+ if (IS_ERR(imx_pcie->iomuxc_gpr))
+ return dev_err_probe(dev, PTR_ERR(imx_pcie->iomuxc_gpr),
"unable to find iomuxc registers\n");
}
/* Grab PCIe PHY Tx Settings */
if (of_property_read_u32(node, "fsl,tx-deemph-gen1",
- &imx6_pcie->tx_deemph_gen1))
- imx6_pcie->tx_deemph_gen1 = 0;
+ &imx_pcie->tx_deemph_gen1))
+ imx_pcie->tx_deemph_gen1 = 0;
if (of_property_read_u32(node, "fsl,tx-deemph-gen2-3p5db",
- &imx6_pcie->tx_deemph_gen2_3p5db))
- imx6_pcie->tx_deemph_gen2_3p5db = 0;
+ &imx_pcie->tx_deemph_gen2_3p5db))
+ imx_pcie->tx_deemph_gen2_3p5db = 0;
if (of_property_read_u32(node, "fsl,tx-deemph-gen2-6db",
- &imx6_pcie->tx_deemph_gen2_6db))
- imx6_pcie->tx_deemph_gen2_6db = 20;
+ &imx_pcie->tx_deemph_gen2_6db))
+ imx_pcie->tx_deemph_gen2_6db = 20;
if (of_property_read_u32(node, "fsl,tx-swing-full",
- &imx6_pcie->tx_swing_full))
- imx6_pcie->tx_swing_full = 127;
+ &imx_pcie->tx_swing_full))
+ imx_pcie->tx_swing_full = 127;
if (of_property_read_u32(node, "fsl,tx-swing-low",
- &imx6_pcie->tx_swing_low))
- imx6_pcie->tx_swing_low = 127;
+ &imx_pcie->tx_swing_low))
+ imx_pcie->tx_swing_low = 127;
/* Limit link speed */
pci->link_gen = 1;
of_property_read_u32(node, "fsl,max-link-speed", &pci->link_gen);
- imx6_pcie->vpcie = devm_regulator_get_optional(&pdev->dev, "vpcie");
- if (IS_ERR(imx6_pcie->vpcie)) {
- if (PTR_ERR(imx6_pcie->vpcie) != -ENODEV)
- return PTR_ERR(imx6_pcie->vpcie);
- imx6_pcie->vpcie = NULL;
+ imx_pcie->vpcie = devm_regulator_get_optional(&pdev->dev, "vpcie");
+ if (IS_ERR(imx_pcie->vpcie)) {
+ if (PTR_ERR(imx_pcie->vpcie) != -ENODEV)
+ return PTR_ERR(imx_pcie->vpcie);
+ imx_pcie->vpcie = NULL;
}
- imx6_pcie->vph = devm_regulator_get_optional(&pdev->dev, "vph");
- if (IS_ERR(imx6_pcie->vph)) {
- if (PTR_ERR(imx6_pcie->vph) != -ENODEV)
- return PTR_ERR(imx6_pcie->vph);
- imx6_pcie->vph = NULL;
+ imx_pcie->vph = devm_regulator_get_optional(&pdev->dev, "vph");
+ if (IS_ERR(imx_pcie->vph)) {
+ if (PTR_ERR(imx_pcie->vph) != -ENODEV)
+ return PTR_ERR(imx_pcie->vph);
+ imx_pcie->vph = NULL;
}
- platform_set_drvdata(pdev, imx6_pcie);
+ platform_set_drvdata(pdev, imx_pcie);
- ret = imx6_pcie_attach_pd(dev);
+ ret = imx_pcie_attach_pd(dev);
if (ret)
return ret;
- if (imx6_pcie->drvdata->mode == DW_PCIE_EP_TYPE) {
- ret = imx6_add_pcie_ep(imx6_pcie, pdev);
+ if (imx_pcie->drvdata->mode == DW_PCIE_EP_TYPE) {
+ ret = imx_add_pcie_ep(imx_pcie, pdev);
if (ret < 0)
return ret;
} else {
@@ -1432,12 +1432,12 @@ static int imx6_pcie_probe(struct platform_device *pdev)
return 0;
}
-static void imx6_pcie_shutdown(struct platform_device *pdev)
+static void imx_pcie_shutdown(struct platform_device *pdev)
{
- struct imx6_pcie *imx6_pcie = platform_get_drvdata(pdev);
+ struct imx_pcie *imx_pcie = platform_get_drvdata(pdev);
/* bring down link, so bootloader gets clean state in case of reboot */
- imx6_pcie_assert_core_reset(imx6_pcie);
+ imx_pcie_assert_core_reset(imx_pcie);
}
static const char * const imx6q_clks[] = {"pcie_bus", "pcie", "pcie_phy"};
@@ -1445,11 +1445,11 @@ static const char * const imx8mm_clks[] = {"pcie_bus", "pcie", "pcie_aux"};
static const char * const imx8mq_clks[] = {"pcie_bus", "pcie", "pcie_phy", "pcie_aux"};
static const char * const imx6sx_clks[] = {"pcie_bus", "pcie", "pcie_phy", "pcie_inbound_axi"};
-static const struct imx6_pcie_drvdata drvdata[] = {
+static const struct imx_pcie_drvdata drvdata[] = {
[IMX6Q] = {
.variant = IMX6Q,
- .flags = IMX6_PCIE_FLAG_IMX6_PHY |
- IMX6_PCIE_FLAG_IMX6_SPEED_CHANGE,
+ .flags = IMX_PCIE_FLAG_IMX_PHY |
+ IMX_PCIE_FLAG_IMX_SPEED_CHANGE,
.dbi_length = 0x200,
.gpr = "fsl,imx6q-iomuxc-gpr",
.clk_names = imx6q_clks,
@@ -1458,13 +1458,13 @@ static const struct imx6_pcie_drvdata drvdata[] = {
.ltssm_mask = IMX6Q_GPR12_PCIE_CTL_2,
.mode_off[0] = IOMUXC_GPR12,
.mode_mask[0] = IMX6Q_GPR12_DEVICE_TYPE,
- .init_phy = imx6_pcie_init_phy,
+ .init_phy = imx_pcie_init_phy,
},
[IMX6SX] = {
.variant = IMX6SX,
- .flags = IMX6_PCIE_FLAG_IMX6_PHY |
- IMX6_PCIE_FLAG_IMX6_SPEED_CHANGE |
- IMX6_PCIE_FLAG_SUPPORTS_SUSPEND,
+ .flags = IMX_PCIE_FLAG_IMX_PHY |
+ IMX_PCIE_FLAG_IMX_SPEED_CHANGE |
+ IMX_PCIE_FLAG_SUPPORTS_SUSPEND,
.gpr = "fsl,imx6q-iomuxc-gpr",
.clk_names = imx6sx_clks,
.clks_cnt = ARRAY_SIZE(imx6sx_clks),
@@ -1476,9 +1476,9 @@ static const struct imx6_pcie_drvdata drvdata[] = {
},
[IMX6QP] = {
.variant = IMX6QP,
- .flags = IMX6_PCIE_FLAG_IMX6_PHY |
- IMX6_PCIE_FLAG_IMX6_SPEED_CHANGE |
- IMX6_PCIE_FLAG_SUPPORTS_SUSPEND,
+ .flags = IMX_PCIE_FLAG_IMX_PHY |
+ IMX_PCIE_FLAG_IMX_SPEED_CHANGE |
+ IMX_PCIE_FLAG_SUPPORTS_SUSPEND,
.dbi_length = 0x200,
.gpr = "fsl,imx6q-iomuxc-gpr",
.clk_names = imx6q_clks,
@@ -1487,13 +1487,13 @@ static const struct imx6_pcie_drvdata drvdata[] = {
.ltssm_mask = IMX6Q_GPR12_PCIE_CTL_2,
.mode_off[0] = IOMUXC_GPR12,
.mode_mask[0] = IMX6Q_GPR12_DEVICE_TYPE,
- .init_phy = imx6_pcie_init_phy,
+ .init_phy = imx_pcie_init_phy,
},
[IMX7D] = {
.variant = IMX7D,
- .flags = IMX6_PCIE_FLAG_SUPPORTS_SUSPEND |
- IMX6_PCIE_FLAG_HAS_APP_RESET |
- IMX6_PCIE_FLAG_HAS_PHY_RESET,
+ .flags = IMX_PCIE_FLAG_SUPPORTS_SUSPEND |
+ IMX_PCIE_FLAG_HAS_APP_RESET |
+ IMX_PCIE_FLAG_HAS_PHY_RESET,
.gpr = "fsl,imx7d-iomuxc-gpr",
.clk_names = imx6q_clks,
.clks_cnt = ARRAY_SIZE(imx6q_clks),
@@ -1503,8 +1503,8 @@ static const struct imx6_pcie_drvdata drvdata[] = {
},
[IMX8MQ] = {
.variant = IMX8MQ,
- .flags = IMX6_PCIE_FLAG_HAS_APP_RESET |
- IMX6_PCIE_FLAG_HAS_PHY_RESET,
+ .flags = IMX_PCIE_FLAG_HAS_APP_RESET |
+ IMX_PCIE_FLAG_HAS_PHY_RESET,
.gpr = "fsl,imx8mq-iomuxc-gpr",
.clk_names = imx8mq_clks,
.clks_cnt = ARRAY_SIZE(imx8mq_clks),
@@ -1516,9 +1516,9 @@ static const struct imx6_pcie_drvdata drvdata[] = {
},
[IMX8MM] = {
.variant = IMX8MM,
- .flags = IMX6_PCIE_FLAG_SUPPORTS_SUSPEND |
- IMX6_PCIE_FLAG_HAS_PHYDRV |
- IMX6_PCIE_FLAG_HAS_APP_RESET,
+ .flags = IMX_PCIE_FLAG_SUPPORTS_SUSPEND |
+ IMX_PCIE_FLAG_HAS_PHYDRV |
+ IMX_PCIE_FLAG_HAS_APP_RESET,
.gpr = "fsl,imx8mm-iomuxc-gpr",
.clk_names = imx8mm_clks,
.clks_cnt = ARRAY_SIZE(imx8mm_clks),
@@ -1527,9 +1527,9 @@ static const struct imx6_pcie_drvdata drvdata[] = {
},
[IMX8MP] = {
.variant = IMX8MP,
- .flags = IMX6_PCIE_FLAG_SUPPORTS_SUSPEND |
- IMX6_PCIE_FLAG_HAS_PHYDRV |
- IMX6_PCIE_FLAG_HAS_APP_RESET,
+ .flags = IMX_PCIE_FLAG_SUPPORTS_SUSPEND |
+ IMX_PCIE_FLAG_HAS_PHYDRV |
+ IMX_PCIE_FLAG_HAS_APP_RESET,
.gpr = "fsl,imx8mp-iomuxc-gpr",
.clk_names = imx8mm_clks,
.clks_cnt = ARRAY_SIZE(imx8mm_clks),
@@ -1538,7 +1538,7 @@ static const struct imx6_pcie_drvdata drvdata[] = {
},
[IMX95] = {
.variant = IMX95,
- .flags = IMX6_PCIE_FLAG_HAS_SERDES,
+ .flags = IMX_PCIE_FLAG_HAS_SERDES,
.clk_names = imx8mq_clks,
.clks_cnt = ARRAY_SIZE(imx8mq_clks),
.ltssm_off = IMX95_PE0_GEN_CTRL_3,
@@ -1549,8 +1549,8 @@ static const struct imx6_pcie_drvdata drvdata[] = {
},
[IMX8MQ_EP] = {
.variant = IMX8MQ_EP,
- .flags = IMX6_PCIE_FLAG_HAS_APP_RESET |
- IMX6_PCIE_FLAG_HAS_PHY_RESET,
+ .flags = IMX_PCIE_FLAG_HAS_APP_RESET |
+ IMX_PCIE_FLAG_HAS_PHY_RESET,
.mode = DW_PCIE_EP_TYPE,
.gpr = "fsl,imx8mq-iomuxc-gpr",
.clk_names = imx8mq_clks,
@@ -1564,8 +1564,8 @@ static const struct imx6_pcie_drvdata drvdata[] = {
},
[IMX8MM_EP] = {
.variant = IMX8MM_EP,
- .flags = IMX6_PCIE_FLAG_HAS_APP_RESET |
- IMX6_PCIE_FLAG_HAS_PHYDRV,
+ .flags = IMX_PCIE_FLAG_HAS_APP_RESET |
+ IMX_PCIE_FLAG_HAS_PHYDRV,
.mode = DW_PCIE_EP_TYPE,
.gpr = "fsl,imx8mm-iomuxc-gpr",
.clk_names = imx8mm_clks,
@@ -1576,8 +1576,8 @@ static const struct imx6_pcie_drvdata drvdata[] = {
},
[IMX8MP_EP] = {
.variant = IMX8MP_EP,
- .flags = IMX6_PCIE_FLAG_HAS_APP_RESET |
- IMX6_PCIE_FLAG_HAS_PHYDRV,
+ .flags = IMX_PCIE_FLAG_HAS_APP_RESET |
+ IMX_PCIE_FLAG_HAS_PHYDRV,
.mode = DW_PCIE_EP_TYPE,
.gpr = "fsl,imx8mp-iomuxc-gpr",
.clk_names = imx8mm_clks,
@@ -1588,8 +1588,8 @@ static const struct imx6_pcie_drvdata drvdata[] = {
},
[IMX95_EP] = {
.variant = IMX95_EP,
- .flags = IMX6_PCIE_FLAG_HAS_SERDES |
- IMX6_PCIE_FLAG_SUPPORT_64BIT,
+ .flags = IMX_PCIE_FLAG_HAS_SERDES |
+ IMX_PCIE_FLAG_SUPPORT_64BIT,
.clk_names = imx8mq_clks,
.clks_cnt = ARRAY_SIZE(imx8mq_clks),
.ltssm_off = IMX95_PE0_GEN_CTRL_3,
@@ -1602,7 +1602,7 @@ static const struct imx6_pcie_drvdata drvdata[] = {
},
};
-static const struct of_device_id imx6_pcie_of_match[] = {
+static const struct of_device_id imx_pcie_of_match[] = {
{ .compatible = "fsl,imx6q-pcie", .data = &drvdata[IMX6Q], },
{ .compatible = "fsl,imx6sx-pcie", .data = &drvdata[IMX6SX], },
{ .compatible = "fsl,imx6qp-pcie", .data = &drvdata[IMX6QP], },
@@ -1618,19 +1618,19 @@ static const struct of_device_id imx6_pcie_of_match[] = {
{},
};
-static struct platform_driver imx6_pcie_driver = {
+static struct platform_driver imx_pcie_driver = {
.driver = {
.name = "imx6q-pcie",
- .of_match_table = imx6_pcie_of_match,
+ .of_match_table = imx_pcie_of_match,
.suppress_bind_attrs = true,
- .pm = &imx6_pcie_pm_ops,
+ .pm = &imx_pcie_pm_ops,
.probe_type = PROBE_PREFER_ASYNCHRONOUS,
},
- .probe = imx6_pcie_probe,
- .shutdown = imx6_pcie_shutdown,
+ .probe = imx_pcie_probe,
+ .shutdown = imx_pcie_shutdown,
};
-static void imx6_pcie_quirk(struct pci_dev *dev)
+static void imx_pcie_quirk(struct pci_dev *dev)
{
struct pci_bus *bus = dev->bus;
struct dw_pcie_rp *pp = bus->sysdata;
@@ -1640,33 +1640,33 @@ static void imx6_pcie_quirk(struct pci_dev *dev)
return;
/* Make sure we only quirk devices associated with this driver */
- if (bus->dev.parent->parent->driver != &imx6_pcie_driver.driver)
+ if (bus->dev.parent->parent->driver != &imx_pcie_driver.driver)
return;
if (pci_is_root_bus(bus)) {
struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
- struct imx6_pcie *imx6_pcie = to_imx6_pcie(pci);
+ struct imx_pcie *imx_pcie = to_imx_pcie(pci);
/*
* Limit config length to avoid the kernel reading beyond
* the register set and causing an abort on i.MX 6Quad
*/
- if (imx6_pcie->drvdata->dbi_length) {
- dev->cfg_size = imx6_pcie->drvdata->dbi_length;
+ if (imx_pcie->drvdata->dbi_length) {
+ dev->cfg_size = imx_pcie->drvdata->dbi_length;
dev_info(&dev->dev, "Limiting cfg_size to %d\n",
dev->cfg_size);
}
}
}
DECLARE_PCI_FIXUP_CLASS_HEADER(PCI_VENDOR_ID_SYNOPSYS, 0xabcd,
- PCI_CLASS_BRIDGE_PCI, 8, imx6_pcie_quirk);
+ PCI_CLASS_BRIDGE_PCI, 8, imx_pcie_quirk);
-static int __init imx6_pcie_init(void)
+static int __init imx_pcie_init(void)
{
#ifdef CONFIG_ARM
struct device_node *np;
- np = of_find_matching_node(NULL, imx6_pcie_of_match);
+ np = of_find_matching_node(NULL, imx_pcie_of_match);
if (!np)
return -ENODEV;
of_node_put(np);
@@ -1682,6 +1682,6 @@ static int __init imx6_pcie_init(void)
"external abort on non-linefetch");
#endif
- return platform_driver_register(&imx6_pcie_driver);
+ return platform_driver_register(&imx_pcie_driver);
}
-device_initcall(imx6_pcie_init);
+device_initcall(imx_pcie_init);
--
2.34.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 44+ messages in thread* [PATCH v5 04/12] PCI: imx6: Introduce SoC specific callbacks for controlling REFCLK
2024-05-28 19:39 [PATCH v5 00/12] PCI: imx6: Fix\rename\clean up and add lut information for imx95 Frank Li
` (2 preceding siblings ...)
2024-05-28 19:39 ` [PATCH v5 03/12] PCI: imx6: Rename imx6_* with imx_* Frank Li
@ 2024-05-28 19:39 ` Frank Li
2024-05-28 19:39 ` [PATCH v5 05/12] PCI: imx6: Simplify switch-case logic by involve core_reset callback Frank Li
` (9 subsequent siblings)
13 siblings, 0 replies; 44+ messages in thread
From: Frank Li @ 2024-05-28 19:39 UTC (permalink / raw)
To: Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley
Cc: linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Frank Li
Instead of using the switch case statement to enable/disable the reference
clock handled by this driver itself, let's introduce a new callback
set_ref_clk() and define it for platforms that require it. This simplifies
the code.
Signed-off-by: Frank Li <Frank.Li@nxp.com>
---
drivers/pci/controller/dwc/pci-imx6.c | 112 ++++++++++++++++------------------
1 file changed, 52 insertions(+), 60 deletions(-)
diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c
index 72e973312d203..c5d490afa981e 100644
--- a/drivers/pci/controller/dwc/pci-imx6.c
+++ b/drivers/pci/controller/dwc/pci-imx6.c
@@ -102,6 +102,7 @@ struct imx_pcie_drvdata {
const u32 mode_mask[IMX_PCIE_MAX_INSTANCES];
const struct pci_epc_features *epc_features;
int (*init_phy)(struct imx_pcie *pcie);
+ int (*set_ref_clk)(struct imx_pcie *pcie, bool enable);
};
struct imx_pcie {
@@ -583,21 +584,19 @@ static int imx_pcie_attach_pd(struct device *dev)
return 0;
}
-static int imx_pcie_enable_ref_clk(struct imx_pcie *imx_pcie)
+static int imx6sx_pcie_set_ref_clk(struct imx_pcie *imx_pcie, bool enable)
{
- unsigned int offset;
- int ret = 0;
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR12, IMX6SX_GPR12_PCIE_TEST_POWERDOWN,
+ enable ? 0 : IMX6SX_GPR12_PCIE_TEST_POWERDOWN);
- switch (imx_pcie->drvdata->variant) {
- case IMX6SX:
- regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR12,
- IMX6SX_GPR12_PCIE_TEST_POWERDOWN, 0);
- break;
- case IMX6QP:
- case IMX6Q:
+ return 0;
+}
+
+static int imx6q_pcie_set_ref_clk(struct imx_pcie *imx_pcie, bool enable)
+{
+ if (enable) {
/* power up core phy and enable ref clock */
- regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1,
- IMX6Q_GPR1_PCIE_TEST_PD, 0 << 18);
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, IMX6Q_GPR1_PCIE_TEST_PD, 0);
/*
* the async reset input need ref clock to sync internally,
* when the ref clock comes after reset, internal synced
@@ -606,54 +605,34 @@ static int imx_pcie_enable_ref_clk(struct imx_pcie *imx_pcie)
*/
usleep_range(10, 100);
regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1,
- IMX6Q_GPR1_PCIE_REF_CLK_EN, 1 << 16);
- break;
- case IMX7D:
- case IMX95:
- case IMX95_EP:
- break;
- case IMX8MM:
- case IMX8MM_EP:
- case IMX8MQ:
- case IMX8MQ_EP:
- case IMX8MP:
- case IMX8MP_EP:
- offset = imx_pcie_grp_offset(imx_pcie);
- /*
- * Set the over ride low and enabled
- * make sure that REF_CLK is turned on.
- */
- regmap_update_bits(imx_pcie->iomuxc_gpr, offset,
- IMX8MQ_GPR_PCIE_CLK_REQ_OVERRIDE,
- 0);
- regmap_update_bits(imx_pcie->iomuxc_gpr, offset,
- IMX8MQ_GPR_PCIE_CLK_REQ_OVERRIDE_EN,
- IMX8MQ_GPR_PCIE_CLK_REQ_OVERRIDE_EN);
- break;
+ IMX6Q_GPR1_PCIE_REF_CLK_EN, IMX6Q_GPR1_PCIE_REF_CLK_EN);
+ } else {
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1,
+ IMX6Q_GPR1_PCIE_REF_CLK_EN, 0);
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1,
+ IMX6Q_GPR1_PCIE_TEST_PD, IMX6Q_GPR1_PCIE_TEST_PD);
}
- return ret;
+ return 0;
}
-static void imx_pcie_disable_ref_clk(struct imx_pcie *imx_pcie)
+static int imx8mm_pcie_set_ref_clk(struct imx_pcie *imx_pcie, bool enable)
{
- switch (imx_pcie->drvdata->variant) {
- case IMX6QP:
- case IMX6Q:
- regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1,
- IMX6Q_GPR1_PCIE_REF_CLK_EN, 0);
- regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1,
- IMX6Q_GPR1_PCIE_TEST_PD,
- IMX6Q_GPR1_PCIE_TEST_PD);
- break;
- case IMX7D:
- regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR12,
- IMX7D_GPR12_PCIE_PHY_REFCLK_SEL,
- IMX7D_GPR12_PCIE_PHY_REFCLK_SEL);
- break;
- default:
- break;
- }
+ int offset = imx_pcie_grp_offset(imx_pcie);
+
+ /* Set the over ride low and enabled make sure that REF_CLK is turned on.*/
+ regmap_update_bits(imx_pcie->iomuxc_gpr, offset, IMX8MQ_GPR_PCIE_CLK_REQ_OVERRIDE,
+ enable ? 0 : IMX8MQ_GPR_PCIE_CLK_REQ_OVERRIDE);
+ regmap_update_bits(imx_pcie->iomuxc_gpr, offset, IMX8MQ_GPR_PCIE_CLK_REQ_OVERRIDE_EN,
+ enable ? IMX8MQ_GPR_PCIE_CLK_REQ_OVERRIDE_EN : 0);
+ return 0;
+}
+
+static int imx7d_pcie_set_ref_clk(struct imx_pcie *imx_pcie, bool enable)
+{
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR12, IMX7D_GPR12_PCIE_PHY_REFCLK_SEL,
+ enable ? 0 : IMX7D_GPR12_PCIE_PHY_REFCLK_SEL);
+ return 0;
}
static int imx_pcie_clk_enable(struct imx_pcie *imx_pcie)
@@ -666,10 +645,12 @@ static int imx_pcie_clk_enable(struct imx_pcie *imx_pcie)
if (ret)
return ret;
- ret = imx_pcie_enable_ref_clk(imx_pcie);
- if (ret) {
- dev_err(dev, "unable to enable pcie ref clock\n");
- goto err_ref_clk;
+ if (imx_pcie->drvdata->set_ref_clk) {
+ ret = imx_pcie->drvdata->set_ref_clk(imx_pcie, true);
+ if (ret) {
+ dev_err(dev, "Failed to enable PCIe REFCLK\n");
+ goto err_ref_clk;
+ }
}
/* allow the clocks to stabilize */
@@ -684,7 +665,8 @@ static int imx_pcie_clk_enable(struct imx_pcie *imx_pcie)
static void imx_pcie_clk_disable(struct imx_pcie *imx_pcie)
{
- imx_pcie_disable_ref_clk(imx_pcie);
+ if (imx_pcie->drvdata->set_ref_clk)
+ imx_pcie->drvdata->set_ref_clk(imx_pcie, false);
clk_bulk_disable_unprepare(imx_pcie->drvdata->clks_cnt, imx_pcie->clks);
}
@@ -1459,6 +1441,7 @@ static const struct imx_pcie_drvdata drvdata[] = {
.mode_off[0] = IOMUXC_GPR12,
.mode_mask[0] = IMX6Q_GPR12_DEVICE_TYPE,
.init_phy = imx_pcie_init_phy,
+ .set_ref_clk = imx6q_pcie_set_ref_clk,
},
[IMX6SX] = {
.variant = IMX6SX,
@@ -1473,6 +1456,7 @@ static const struct imx_pcie_drvdata drvdata[] = {
.mode_off[0] = IOMUXC_GPR12,
.mode_mask[0] = IMX6Q_GPR12_DEVICE_TYPE,
.init_phy = imx6sx_pcie_init_phy,
+ .set_ref_clk = imx6sx_pcie_set_ref_clk,
},
[IMX6QP] = {
.variant = IMX6QP,
@@ -1488,6 +1472,7 @@ static const struct imx_pcie_drvdata drvdata[] = {
.mode_off[0] = IOMUXC_GPR12,
.mode_mask[0] = IMX6Q_GPR12_DEVICE_TYPE,
.init_phy = imx_pcie_init_phy,
+ .set_ref_clk = imx6q_pcie_set_ref_clk,
},
[IMX7D] = {
.variant = IMX7D,
@@ -1500,6 +1485,7 @@ static const struct imx_pcie_drvdata drvdata[] = {
.mode_off[0] = IOMUXC_GPR12,
.mode_mask[0] = IMX6Q_GPR12_DEVICE_TYPE,
.init_phy = imx7d_pcie_init_phy,
+ .set_ref_clk = imx7d_pcie_set_ref_clk,
},
[IMX8MQ] = {
.variant = IMX8MQ,
@@ -1513,6 +1499,7 @@ static const struct imx_pcie_drvdata drvdata[] = {
.mode_off[1] = IOMUXC_GPR12,
.mode_mask[1] = IMX8MQ_GPR12_PCIE2_CTRL_DEVICE_TYPE,
.init_phy = imx8mq_pcie_init_phy,
+ .set_ref_clk = imx8mm_pcie_set_ref_clk,
},
[IMX8MM] = {
.variant = IMX8MM,
@@ -1524,6 +1511,7 @@ static const struct imx_pcie_drvdata drvdata[] = {
.clks_cnt = ARRAY_SIZE(imx8mm_clks),
.mode_off[0] = IOMUXC_GPR12,
.mode_mask[0] = IMX6Q_GPR12_DEVICE_TYPE,
+ .set_ref_clk = imx8mm_pcie_set_ref_clk,
},
[IMX8MP] = {
.variant = IMX8MP,
@@ -1535,6 +1523,7 @@ static const struct imx_pcie_drvdata drvdata[] = {
.clks_cnt = ARRAY_SIZE(imx8mm_clks),
.mode_off[0] = IOMUXC_GPR12,
.mode_mask[0] = IMX6Q_GPR12_DEVICE_TYPE,
+ .set_ref_clk = imx8mm_pcie_set_ref_clk,
},
[IMX95] = {
.variant = IMX95,
@@ -1561,6 +1550,7 @@ static const struct imx_pcie_drvdata drvdata[] = {
.mode_mask[1] = IMX8MQ_GPR12_PCIE2_CTRL_DEVICE_TYPE,
.epc_features = &imx8m_pcie_epc_features,
.init_phy = imx8mq_pcie_init_phy,
+ .set_ref_clk = imx8mm_pcie_set_ref_clk,
},
[IMX8MM_EP] = {
.variant = IMX8MM_EP,
@@ -1573,6 +1563,7 @@ static const struct imx_pcie_drvdata drvdata[] = {
.mode_off[0] = IOMUXC_GPR12,
.mode_mask[0] = IMX6Q_GPR12_DEVICE_TYPE,
.epc_features = &imx8m_pcie_epc_features,
+ .set_ref_clk = imx8mm_pcie_set_ref_clk,
},
[IMX8MP_EP] = {
.variant = IMX8MP_EP,
@@ -1585,6 +1576,7 @@ static const struct imx_pcie_drvdata drvdata[] = {
.mode_off[0] = IOMUXC_GPR12,
.mode_mask[0] = IMX6Q_GPR12_DEVICE_TYPE,
.epc_features = &imx8m_pcie_epc_features,
+ .set_ref_clk = imx8mm_pcie_set_ref_clk,
},
[IMX95_EP] = {
.variant = IMX95_EP,
--
2.34.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 44+ messages in thread* [PATCH v5 05/12] PCI: imx6: Simplify switch-case logic by involve core_reset callback
2024-05-28 19:39 [PATCH v5 00/12] PCI: imx6: Fix\rename\clean up and add lut information for imx95 Frank Li
` (3 preceding siblings ...)
2024-05-28 19:39 ` [PATCH v5 04/12] PCI: imx6: Introduce SoC specific callbacks for controlling REFCLK Frank Li
@ 2024-05-28 19:39 ` Frank Li
2024-05-28 19:39 ` [PATCH v5 06/12] PCI: imx6: Improve comment for workaround ERR010728 Frank Li
` (8 subsequent siblings)
13 siblings, 0 replies; 44+ messages in thread
From: Frank Li @ 2024-05-28 19:39 UTC (permalink / raw)
To: Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley
Cc: linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Frank Li
Instead of using the switch case statement to assert/dassert the core reset
handled by this driver itself, let's introduce a new callback core_reset()
and define it for platforms that require it. This simplifies the code.
Signed-off-by: Frank Li <Frank.Li@nxp.com>
---
drivers/pci/controller/dwc/pci-imx6.c | 134 ++++++++++++++++++----------------
1 file changed, 71 insertions(+), 63 deletions(-)
diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c
index c5d490afa981e..5e21fc942e90e 100644
--- a/drivers/pci/controller/dwc/pci-imx6.c
+++ b/drivers/pci/controller/dwc/pci-imx6.c
@@ -103,6 +103,7 @@ struct imx_pcie_drvdata {
const struct pci_epc_features *epc_features;
int (*init_phy)(struct imx_pcie *pcie);
int (*set_ref_clk)(struct imx_pcie *pcie, bool enable);
+ int (*core_reset)(struct imx_pcie *pcie, bool assert);
};
struct imx_pcie {
@@ -670,35 +671,75 @@ static void imx_pcie_clk_disable(struct imx_pcie *imx_pcie)
clk_bulk_disable_unprepare(imx_pcie->drvdata->clks_cnt, imx_pcie->clks);
}
+static int imx6sx_pcie_core_reset(struct imx_pcie *imx_pcie, bool assert)
+{
+ if (assert)
+ regmap_set_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR12,
+ IMX6SX_GPR12_PCIE_TEST_POWERDOWN);
+
+ /* Force PCIe PHY reset */
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR5, IMX6SX_GPR5_PCIE_BTNRST_RESET,
+ assert ? IMX6SX_GPR5_PCIE_BTNRST_RESET : 0);
+ return 0;
+}
+
+static int imx6qp_pcie_core_reset(struct imx_pcie *imx_pcie, bool assert)
+{
+ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, IMX6Q_GPR1_PCIE_SW_RST,
+ assert ? IMX6Q_GPR1_PCIE_SW_RST : 0);
+ if (!assert)
+ usleep_range(200, 500);
+
+ return 0;
+}
+
+static int imx6q_pcie_core_reset(struct imx_pcie *imx_pcie, bool assert)
+{
+ if (!assert)
+ return 0;
+
+ regmap_set_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, IMX6Q_GPR1_PCIE_TEST_PD);
+ regmap_set_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, IMX6Q_GPR1_PCIE_REF_CLK_EN);
+
+ return 0;
+}
+
+static int imx7d_pcie_core_reset(struct imx_pcie *imx_pcie, bool assert)
+{
+ struct dw_pcie *pci = imx_pcie->pci;
+ struct device *dev = pci->dev;
+
+ if (assert)
+ return 0;
+
+ /*
+ * Workaround for ERR010728, failure of PCI-e PLL VCO to
+ * oscillate, especially when cold. This turns off "Duty-cycle
+ * Corrector" and other mysterious undocumented things.
+ */
+
+ if (likely(imx_pcie->phy_base)) {
+ /* De-assert DCC_FB_EN */
+ writel(PCIE_PHY_CMN_REG4_DCC_FB_EN, imx_pcie->phy_base + PCIE_PHY_CMN_REG4);
+ /* Assert RX_EQS and RX_EQS_SEL */
+ writel(PCIE_PHY_CMN_REG24_RX_EQ_SEL | PCIE_PHY_CMN_REG24_RX_EQ,
+ imx_pcie->phy_base + PCIE_PHY_CMN_REG24);
+ /* Assert ATT_MODE */
+ writel(PCIE_PHY_CMN_REG26_ATT_MODE, imx_pcie->phy_base + PCIE_PHY_CMN_REG26);
+ } else {
+ dev_warn(dev, "Unable to apply ERR010728 workaround. DT missing fsl,imx7d-pcie-phy phandle ?\n");
+ }
+ imx7d_pcie_wait_for_phy_pll_lock(imx_pcie);
+ return 0;
+}
+
static void imx_pcie_assert_core_reset(struct imx_pcie *imx_pcie)
{
reset_control_assert(imx_pcie->pciephy_reset);
reset_control_assert(imx_pcie->apps_reset);
- switch (imx_pcie->drvdata->variant) {
- case IMX6SX:
- regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR12,
- IMX6SX_GPR12_PCIE_TEST_POWERDOWN,
- IMX6SX_GPR12_PCIE_TEST_POWERDOWN);
- /* Force PCIe PHY reset */
- regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR5,
- IMX6SX_GPR5_PCIE_BTNRST_RESET,
- IMX6SX_GPR5_PCIE_BTNRST_RESET);
- break;
- case IMX6QP:
- regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1,
- IMX6Q_GPR1_PCIE_SW_RST,
- IMX6Q_GPR1_PCIE_SW_RST);
- break;
- case IMX6Q:
- regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1,
- IMX6Q_GPR1_PCIE_TEST_PD, 1 << 18);
- regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1,
- IMX6Q_GPR1_PCIE_REF_CLK_EN, 0 << 16);
- break;
- default:
- break;
- }
+ if (imx_pcie->drvdata->core_reset)
+ imx_pcie->drvdata->core_reset(imx_pcie, true);
/* Some boards don't have PCIe reset GPIO. */
gpiod_set_value_cansleep(imx_pcie->reset_gpiod, 1);
@@ -706,47 +747,10 @@ static void imx_pcie_assert_core_reset(struct imx_pcie *imx_pcie)
static int imx_pcie_deassert_core_reset(struct imx_pcie *imx_pcie)
{
- struct dw_pcie *pci = imx_pcie->pci;
- struct device *dev = pci->dev;
-
reset_control_deassert(imx_pcie->pciephy_reset);
- switch (imx_pcie->drvdata->variant) {
- case IMX7D:
- /* Workaround for ERR010728, failure of PCI-e PLL VCO to
- * oscillate, especially when cold. This turns off "Duty-cycle
- * Corrector" and other mysterious undocumented things.
- */
- if (likely(imx_pcie->phy_base)) {
- /* De-assert DCC_FB_EN */
- writel(PCIE_PHY_CMN_REG4_DCC_FB_EN,
- imx_pcie->phy_base + PCIE_PHY_CMN_REG4);
- /* Assert RX_EQS and RX_EQS_SEL */
- writel(PCIE_PHY_CMN_REG24_RX_EQ_SEL
- | PCIE_PHY_CMN_REG24_RX_EQ,
- imx_pcie->phy_base + PCIE_PHY_CMN_REG24);
- /* Assert ATT_MODE */
- writel(PCIE_PHY_CMN_REG26_ATT_MODE,
- imx_pcie->phy_base + PCIE_PHY_CMN_REG26);
- } else {
- dev_warn(dev, "Unable to apply ERR010728 workaround. DT missing fsl,imx7d-pcie-phy phandle ?\n");
- }
-
- imx7d_pcie_wait_for_phy_pll_lock(imx_pcie);
- break;
- case IMX6SX:
- regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR5,
- IMX6SX_GPR5_PCIE_BTNRST_RESET, 0);
- break;
- case IMX6QP:
- regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1,
- IMX6Q_GPR1_PCIE_SW_RST, 0);
-
- usleep_range(200, 500);
- break;
- default:
- break;
- }
+ if (imx_pcie->drvdata->core_reset)
+ imx_pcie->drvdata->core_reset(imx_pcie, false);
/* Some boards don't have PCIe reset GPIO. */
if (imx_pcie->reset_gpiod) {
@@ -1442,6 +1446,7 @@ static const struct imx_pcie_drvdata drvdata[] = {
.mode_mask[0] = IMX6Q_GPR12_DEVICE_TYPE,
.init_phy = imx_pcie_init_phy,
.set_ref_clk = imx6q_pcie_set_ref_clk,
+ .core_reset = imx6q_pcie_core_reset,
},
[IMX6SX] = {
.variant = IMX6SX,
@@ -1457,6 +1462,7 @@ static const struct imx_pcie_drvdata drvdata[] = {
.mode_mask[0] = IMX6Q_GPR12_DEVICE_TYPE,
.init_phy = imx6sx_pcie_init_phy,
.set_ref_clk = imx6sx_pcie_set_ref_clk,
+ .core_reset = imx6sx_pcie_core_reset,
},
[IMX6QP] = {
.variant = IMX6QP,
@@ -1473,6 +1479,7 @@ static const struct imx_pcie_drvdata drvdata[] = {
.mode_mask[0] = IMX6Q_GPR12_DEVICE_TYPE,
.init_phy = imx_pcie_init_phy,
.set_ref_clk = imx6q_pcie_set_ref_clk,
+ .core_reset = imx6qp_pcie_core_reset,
},
[IMX7D] = {
.variant = IMX7D,
@@ -1486,6 +1493,7 @@ static const struct imx_pcie_drvdata drvdata[] = {
.mode_mask[0] = IMX6Q_GPR12_DEVICE_TYPE,
.init_phy = imx7d_pcie_init_phy,
.set_ref_clk = imx7d_pcie_set_ref_clk,
+ .core_reset = imx7d_pcie_core_reset,
},
[IMX8MQ] = {
.variant = IMX8MQ,
--
2.34.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 44+ messages in thread* [PATCH v5 06/12] PCI: imx6: Improve comment for workaround ERR010728
2024-05-28 19:39 [PATCH v5 00/12] PCI: imx6: Fix\rename\clean up and add lut information for imx95 Frank Li
` (4 preceding siblings ...)
2024-05-28 19:39 ` [PATCH v5 05/12] PCI: imx6: Simplify switch-case logic by involve core_reset callback Frank Li
@ 2024-05-28 19:39 ` Frank Li
2024-05-28 19:39 ` [PATCH v5 07/12] PCI: imx6: Add help function imx_pcie_match_device() Frank Li
` (7 subsequent siblings)
13 siblings, 0 replies; 44+ messages in thread
From: Frank Li @ 2024-05-28 19:39 UTC (permalink / raw)
To: Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley
Cc: linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Frank Li
Improve comment about workaround ERR010727 by using official errata
document content.
Signed-off-by: Frank Li <Frank.Li@nxp.com>
---
drivers/pci/controller/dwc/pci-imx6.c | 22 +++++++++++++++++++---
1 file changed, 19 insertions(+), 3 deletions(-)
diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c
index 5e21fc942e90e..5533b7ad0f092 100644
--- a/drivers/pci/controller/dwc/pci-imx6.c
+++ b/drivers/pci/controller/dwc/pci-imx6.c
@@ -713,9 +713,25 @@ static int imx7d_pcie_core_reset(struct imx_pcie *imx_pcie, bool assert)
return 0;
/*
- * Workaround for ERR010728, failure of PCI-e PLL VCO to
- * oscillate, especially when cold. This turns off "Duty-cycle
- * Corrector" and other mysterious undocumented things.
+ * Workaround for ERR010728 (IMX7DS_2N09P, Rev. 1.1, 4/2023):
+ *
+ * PCIe: PLL may fail to lock under corner conditions Initial VCO
+ * oscillation may fail under corner conditions such as cold
+ * temperature which will cause the PCIe PLL fail to lock in the
+ * initialization phase.
+ *
+ * The Duty-cycle Corrector calibration must be disabled.
+ *
+ * 1. De-assert the G_RST signal by clearing
+ * SRC_PCIEPHY_RCR[PCIEPHY_G_RST].
+ * 2. De-assert DCC_FB_EN by writing data “0x29” to the register
+ * address 0x306d0014 (PCIE_PHY_CMN_REG4).
+ * 3. Assert RX_EQS, RX_EQ_SEL by writing data “0x48” to the register
+ * address 0x306d0090 (PCIE_PHY_CMN_REG24).
+ * 4. Assert ATT_MODE by writing data “0xbc” to the register
+ * address 0x306d0098 (PCIE_PHY_CMN_REG26).
+ * 5. De-assert the CMN_RST signal by clearing register bit
+ * SRC_PCIEPHY_RCR[PCIEPHY_BTN]
*/
if (likely(imx_pcie->phy_base)) {
--
2.34.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 44+ messages in thread* [PATCH v5 07/12] PCI: imx6: Add help function imx_pcie_match_device()
2024-05-28 19:39 [PATCH v5 00/12] PCI: imx6: Fix\rename\clean up and add lut information for imx95 Frank Li
` (5 preceding siblings ...)
2024-05-28 19:39 ` [PATCH v5 06/12] PCI: imx6: Improve comment for workaround ERR010728 Frank Li
@ 2024-05-28 19:39 ` Frank Li
2024-05-28 19:39 ` [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95 Frank Li
` (6 subsequent siblings)
13 siblings, 0 replies; 44+ messages in thread
From: Frank Li @ 2024-05-28 19:39 UTC (permalink / raw)
To: Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley
Cc: linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Frank Li
Introduce the help function imx_pcie_match_device() to facilitate
imx_pcie_quirk() in verifying whether the device's bus driver corresponds
to the IMX6 PCI controller. This addition lays the groundwork for future
support of ITS and IOMMU in the IMX95.
Signed-off-by: Frank Li <Frank.Li@nxp.com>
---
drivers/pci/controller/dwc/pci-imx6.c | 18 +++++++++++++-----
1 file changed, 13 insertions(+), 5 deletions(-)
diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c
index 5533b7ad0f092..29309ad0e352b 100644
--- a/drivers/pci/controller/dwc/pci-imx6.c
+++ b/drivers/pci/controller/dwc/pci-imx6.c
@@ -1646,17 +1646,25 @@ static struct platform_driver imx_pcie_driver = {
.shutdown = imx_pcie_shutdown,
};
-static void imx_pcie_quirk(struct pci_dev *dev)
+static bool imx_pcie_match_device(struct pci_bus *bus)
{
- struct pci_bus *bus = dev->bus;
- struct dw_pcie_rp *pp = bus->sysdata;
-
/* Bus parent is the PCI bridge, its parent is this platform driver */
if (!bus->dev.parent || !bus->dev.parent->parent)
- return;
+ return false;
/* Make sure we only quirk devices associated with this driver */
if (bus->dev.parent->parent->driver != &imx_pcie_driver.driver)
+ return false;
+
+ return true;
+}
+
+static void imx_pcie_quirk(struct pci_dev *dev)
+{
+ struct pci_bus *bus = dev->bus;
+ struct dw_pcie_rp *pp = bus->sysdata;
+
+ if (!imx_pcie_match_device(bus))
return;
if (pci_is_root_bus(bus)) {
--
2.34.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 44+ messages in thread* [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-05-28 19:39 [PATCH v5 00/12] PCI: imx6: Fix\rename\clean up and add lut information for imx95 Frank Li
` (6 preceding siblings ...)
2024-05-28 19:39 ` [PATCH v5 07/12] PCI: imx6: Add help function imx_pcie_match_device() Frank Li
@ 2024-05-28 19:39 ` Frank Li
2024-05-30 23:08 ` Bjorn Helgaas
2024-06-22 4:11 ` Manivannan Sadhasivam
2024-05-28 19:39 ` [PATCH v5 09/12] PCI: imx6: Consolidate redundant if-checks Frank Li
` (5 subsequent siblings)
13 siblings, 2 replies; 44+ messages in thread
From: Frank Li @ 2024-05-28 19:39 UTC (permalink / raw)
To: Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley
Cc: linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Frank Li
For the i.MX95, configuration of a LUT is necessary to convert Bus Device
Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS.
This involves examining the msi-map and smmu-map to ensure consistent
mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related
registers are configured. In the absence of an msi-map, the built-in MSI
controller is utilized as a fallback.
Additionally, register a PCI bus notifier to trigger imx_pcie_add_device()
upon the appearance of a new PCI device and when the bus is an iMX6 PCI
controller. This function configures the correct LUT based on Device Tree
Settings (DTS).
Signed-off-by: Frank Li <Frank.Li@nxp.com>
---
drivers/pci/controller/dwc/pci-imx6.c | 175 +++++++++++++++++++++++++++++++++-
1 file changed, 174 insertions(+), 1 deletion(-)
diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c
index 29309ad0e352b..8ecc00049e20b 100644
--- a/drivers/pci/controller/dwc/pci-imx6.c
+++ b/drivers/pci/controller/dwc/pci-imx6.c
@@ -54,6 +54,22 @@
#define IMX95_PE0_GEN_CTRL_3 0x1058
#define IMX95_PCIE_LTSSM_EN BIT(0)
+#define IMX95_PE0_LUT_ACSCTRL 0x1008
+#define IMX95_PEO_LUT_RWA BIT(16)
+#define IMX95_PE0_LUT_ENLOC GENMASK(4, 0)
+
+#define IMX95_PE0_LUT_DATA1 0x100c
+#define IMX95_PE0_LUT_VLD BIT(31)
+#define IMX95_PE0_LUT_DAC_ID GENMASK(10, 8)
+#define IMX95_PE0_LUT_STREAM_ID GENMASK(5, 0)
+
+#define IMX95_PE0_LUT_DATA2 0x1010
+#define IMX95_PE0_LUT_REQID GENMASK(31, 16)
+#define IMX95_PE0_LUT_MASK GENMASK(15, 0)
+
+#define IMX95_SID_MASK GENMASK(5, 0)
+#define IMX95_MAX_LUT 32
+
#define to_imx_pcie(x) dev_get_drvdata((x)->dev)
enum imx_pcie_variants {
@@ -79,6 +95,7 @@ enum imx_pcie_variants {
#define IMX_PCIE_FLAG_HAS_PHY_RESET BIT(5)
#define IMX_PCIE_FLAG_HAS_SERDES BIT(6)
#define IMX_PCIE_FLAG_SUPPORT_64BIT BIT(7)
+#define IMX_PCIE_FLAG_MONITOR_DEV BIT(8)
#define imx_check_flag(pci, val) (pci->drvdata->flags & val)
@@ -132,6 +149,8 @@ struct imx_pcie {
struct device *pd_pcie_phy;
struct phy *phy;
const struct imx_pcie_drvdata *drvdata;
+
+ struct mutex lock;
};
/* Parameters for the waiting for PCIe PHY PLL to lock on i.MX7 */
@@ -215,6 +234,66 @@ static int imx95_pcie_init_phy(struct imx_pcie *imx_pcie)
return 0;
}
+static int imx_pcie_config_lut(struct imx_pcie *imx_pcie, u16 reqid, u8 sid)
+{
+ struct dw_pcie *pci = imx_pcie->pci;
+ struct device *dev = pci->dev;
+ u32 data1, data2;
+ int i;
+
+ if (sid >= 64) {
+ dev_err(dev, "Invalid SID for index %d\n", sid);
+ return -EINVAL;
+ }
+
+ guard(mutex)(&imx_pcie->lock);
+
+ for (i = 0; i < IMX95_MAX_LUT; i++) {
+ regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_ACSCTRL, IMX95_PEO_LUT_RWA | i);
+
+ regmap_read(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA1, &data1);
+ if (data1 & IMX95_PE0_LUT_VLD)
+ continue;
+
+ data1 = FIELD_PREP(IMX95_PE0_LUT_DAC_ID, 0);
+ data1 |= FIELD_PREP(IMX95_PE0_LUT_STREAM_ID, sid);
+ data1 |= IMX95_PE0_LUT_VLD;
+
+ regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA1, data1);
+
+ data2 = 0xffff;
+ data2 |= FIELD_PREP(IMX95_PE0_LUT_REQID, reqid);
+
+ regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA2, data2);
+
+ regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_ACSCTRL, i);
+
+ return 0;
+ }
+
+ dev_err(dev, "All lut already used\n");
+ return -EINVAL;
+}
+
+static void imx_pcie_remove_lut(struct imx_pcie *imx_pcie, u16 reqid)
+{
+ u32 data2 = 0;
+ int i;
+
+ guard(mutex)(&imx_pcie->lock);
+
+ for (i = 0; i < IMX95_MAX_LUT; i++) {
+ regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_ACSCTRL, IMX95_PEO_LUT_RWA | i);
+
+ regmap_read(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA2, &data2);
+ if (FIELD_GET(IMX95_PE0_LUT_REQID, data2) == reqid) {
+ regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA1, 0);
+ regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA2, 0);
+ regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_ACSCTRL, i);
+ }
+ }
+}
+
static void imx_pcie_configure_type(struct imx_pcie *imx_pcie)
{
const struct imx_pcie_drvdata *drvdata = imx_pcie->drvdata;
@@ -1232,6 +1311,85 @@ static int imx_pcie_resume_noirq(struct device *dev)
return 0;
}
+static bool imx_pcie_match_device(struct pci_bus *bus);
+
+static int imx_pcie_add_device(struct imx_pcie *imx_pcie, struct pci_dev *pdev)
+{
+ u32 sid_i = 0, sid_m = 0, rid = pci_dev_id(pdev);
+ struct device *dev = imx_pcie->pci->dev;
+ int err;
+
+ err = of_map_id(dev->of_node, rid, "iommu-map", "iommu-map-mask", NULL, &sid_i);
+ if (err)
+ return err;
+
+ err = of_map_id(dev->of_node, rid, "msi-map", "msi-map-mask", NULL, &sid_m);
+ if (err)
+ return err;
+
+ if (sid_i != rid && sid_m != rid)
+ if ((sid_i & IMX95_SID_MASK) != (sid_m & IMX95_SID_MASK)) {
+ dev_err(dev, "its and iommu stream id miss match, please check dts file\n");
+ return -EINVAL;
+ }
+
+ /* if iommu-map is not existed then use msi-map's stream id*/
+ if (sid_i == rid)
+ sid_i = sid_m;
+
+ sid_i &= IMX95_SID_MASK;
+
+ if (sid_i != rid)
+ return imx_pcie_config_lut(imx_pcie, rid, sid_i);
+
+ /* Use dwc built-in MSI controller */
+ return 0;
+}
+
+static void imx_pcie_del_device(struct imx_pcie *imx_pcie, struct pci_dev *pdev)
+{
+ imx_pcie_remove_lut(imx_pcie, pci_dev_id(pdev));
+}
+
+
+static int imx_pcie_bus_notifier(struct notifier_block *nb, unsigned long action, void *data)
+{
+ struct pci_host_bridge *host;
+ struct imx_pcie *imx_pcie;
+ struct pci_dev *pdev;
+ int err;
+
+ pdev = to_pci_dev(data);
+ host = pci_find_host_bridge(pdev->bus);
+
+ if (!imx_pcie_match_device(host->bus))
+ return NOTIFY_OK;
+
+ imx_pcie = to_imx_pcie(to_dw_pcie_from_pp(host->sysdata));
+
+ if (!imx_check_flag(imx_pcie, IMX_PCIE_FLAG_MONITOR_DEV))
+ return NOTIFY_OK;
+
+ switch (action) {
+ case BUS_NOTIFY_ADD_DEVICE:
+ err = imx_pcie_add_device(imx_pcie, pdev);
+ if (err)
+ return notifier_from_errno(err);
+ break;
+ case BUS_NOTIFY_DEL_DEVICE:
+ imx_pcie_del_device(imx_pcie, pdev);
+ break;
+ default:
+ return NOTIFY_DONE;
+ }
+
+ return NOTIFY_OK;
+}
+
+static struct notifier_block imx_pcie_nb = {
+ .notifier_call = imx_pcie_bus_notifier,
+};
+
static const struct dev_pm_ops imx_pcie_pm_ops = {
NOIRQ_SYSTEM_SLEEP_PM_OPS(imx_pcie_suspend_noirq,
imx_pcie_resume_noirq)
@@ -1264,6 +1422,8 @@ static int imx_pcie_probe(struct platform_device *pdev)
imx_pcie->pci = pci;
imx_pcie->drvdata = of_device_get_match_data(dev);
+ mutex_init(&imx_pcie->lock);
+
/* Find the PHY if one is defined, only imx7d uses it */
np = of_parse_phandle(node, "fsl,imx7d-pcie-phy", 0);
if (np) {
@@ -1551,7 +1711,8 @@ static const struct imx_pcie_drvdata drvdata[] = {
},
[IMX95] = {
.variant = IMX95,
- .flags = IMX_PCIE_FLAG_HAS_SERDES,
+ .flags = IMX_PCIE_FLAG_HAS_SERDES |
+ IMX_PCIE_FLAG_MONITOR_DEV,
.clk_names = imx8mq_clks,
.clks_cnt = ARRAY_SIZE(imx8mq_clks),
.ltssm_off = IMX95_PE0_GEN_CTRL_3,
@@ -1687,6 +1848,8 @@ DECLARE_PCI_FIXUP_CLASS_HEADER(PCI_VENDOR_ID_SYNOPSYS, 0xabcd,
static int __init imx_pcie_init(void)
{
+ int ret;
+
#ifdef CONFIG_ARM
struct device_node *np;
@@ -1705,7 +1868,17 @@ static int __init imx_pcie_init(void)
hook_fault_code(8, imx6q_pcie_abort_handler, SIGBUS, 0,
"external abort on non-linefetch");
#endif
+ ret = bus_register_notifier(&pci_bus_type, &imx_pcie_nb);
+ if (ret)
+ return ret;
return platform_driver_register(&imx_pcie_driver);
}
+
+static void __exit imx_pcie_exit(void)
+{
+ bus_unregister_notifier(&pci_bus_type, &imx_pcie_nb);
+}
+
device_initcall(imx_pcie_init);
+__exitcall(imx_pcie_exit);
--
2.34.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 44+ messages in thread* Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-05-28 19:39 ` [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95 Frank Li
@ 2024-05-30 23:08 ` Bjorn Helgaas
2024-05-31 13:52 ` Marc Zyngier
` (2 more replies)
2024-06-22 4:11 ` Manivannan Sadhasivam
1 sibling, 3 replies; 44+ messages in thread
From: Bjorn Helgaas @ 2024-05-30 23:08 UTC (permalink / raw)
To: Frank Li
Cc: Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley,
linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Will Deacon, Robin Murphy, Joerg Roedel, Jason Gunthorpe,
Alyssa Rosenzweig, Marc Zyngier
[+cc IOMMU and pcie-apple.c folks for comment]
On Tue, May 28, 2024 at 03:39:21PM -0400, Frank Li wrote:
> For the i.MX95, configuration of a LUT is necessary to convert Bus Device
> Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS.
> This involves examining the msi-map and smmu-map to ensure consistent
> mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related
> registers are configured. In the absence of an msi-map, the built-in MSI
> controller is utilized as a fallback.
>
> Additionally, register a PCI bus notifier to trigger imx_pcie_add_device()
> upon the appearance of a new PCI device and when the bus is an iMX6 PCI
> controller. This function configures the correct LUT based on Device Tree
> Settings (DTS).
This scheme is pretty similar to apple_pcie_bus_notifier(). If we
have to do this, I wish it were *more* similar, i.e., copy the
function names, bitmap tracking, code structure, etc.
I don't really know how stream IDs work, but I assume they are used on
most or all arm64 platforms, so I'm a little surprised that of all the
PCI host drivers used on arm64, only pcie-apple.c and pci-imx6.c need
this notifier.
There's this path, which is pretty generic and does at least the
of_map_id() part of what you're doing in imx_pcie_add_device():
__driver_probe_device
really_probe
pci_dma_configure # pci_bus_type.dma_configure
of_dma_configure
of_dma_configure_id
of_iommu_configure
of_pci_iommu_init
of_iommu_configure_dev_id
of_map_id
of_iommu_xlate
ops = iommu_ops_from_fwnode
iommu_fwspec_init
ops->of_xlate(dev, iommu_spec)
Maybe this needs to be extended somehow with a hook to do the
device-specific work like updating the LUT? Just speculating here,
the IOMMU folks will know how this is expected to work.
Some typos and minor comments below.
> Signed-off-by: Frank Li <Frank.Li@nxp.com>
> ---
> drivers/pci/controller/dwc/pci-imx6.c | 175 +++++++++++++++++++++++++++++++++-
> 1 file changed, 174 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c
> index 29309ad0e352b..8ecc00049e20b 100644
> --- a/drivers/pci/controller/dwc/pci-imx6.c
> +++ b/drivers/pci/controller/dwc/pci-imx6.c
> @@ -54,6 +54,22 @@
> #define IMX95_PE0_GEN_CTRL_3 0x1058
> #define IMX95_PCIE_LTSSM_EN BIT(0)
>
> +#define IMX95_PE0_LUT_ACSCTRL 0x1008
> +#define IMX95_PEO_LUT_RWA BIT(16)
> +#define IMX95_PE0_LUT_ENLOC GENMASK(4, 0)
> +
> +#define IMX95_PE0_LUT_DATA1 0x100c
> +#define IMX95_PE0_LUT_VLD BIT(31)
> +#define IMX95_PE0_LUT_DAC_ID GENMASK(10, 8)
> +#define IMX95_PE0_LUT_STREAM_ID GENMASK(5, 0)
> +
> +#define IMX95_PE0_LUT_DATA2 0x1010
> +#define IMX95_PE0_LUT_REQID GENMASK(31, 16)
> +#define IMX95_PE0_LUT_MASK GENMASK(15, 0)
> +
> +#define IMX95_SID_MASK GENMASK(5, 0)
> +#define IMX95_MAX_LUT 32
> +
> #define to_imx_pcie(x) dev_get_drvdata((x)->dev)
>
> enum imx_pcie_variants {
> @@ -79,6 +95,7 @@ enum imx_pcie_variants {
> #define IMX_PCIE_FLAG_HAS_PHY_RESET BIT(5)
> #define IMX_PCIE_FLAG_HAS_SERDES BIT(6)
> #define IMX_PCIE_FLAG_SUPPORT_64BIT BIT(7)
> +#define IMX_PCIE_FLAG_MONITOR_DEV BIT(8)
>
> #define imx_check_flag(pci, val) (pci->drvdata->flags & val)
>
> @@ -132,6 +149,8 @@ struct imx_pcie {
> struct device *pd_pcie_phy;
> struct phy *phy;
> const struct imx_pcie_drvdata *drvdata;
> +
> + struct mutex lock;
> };
>
> /* Parameters for the waiting for PCIe PHY PLL to lock on i.MX7 */
> @@ -215,6 +234,66 @@ static int imx95_pcie_init_phy(struct imx_pcie *imx_pcie)
> return 0;
> }
>
> +static int imx_pcie_config_lut(struct imx_pcie *imx_pcie, u16 reqid, u8 sid)
> +{
> + struct dw_pcie *pci = imx_pcie->pci;
> + struct device *dev = pci->dev;
> + u32 data1, data2;
> + int i;
> +
> + if (sid >= 64) {
> + dev_err(dev, "Invalid SID for index %d\n", sid);
> + return -EINVAL;
> + }
> +
> + guard(mutex)(&imx_pcie->lock);
> +
> + for (i = 0; i < IMX95_MAX_LUT; i++) {
> + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_ACSCTRL, IMX95_PEO_LUT_RWA | i);
> +
> + regmap_read(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA1, &data1);
> + if (data1 & IMX95_PE0_LUT_VLD)
> + continue;
> +
> + data1 = FIELD_PREP(IMX95_PE0_LUT_DAC_ID, 0);
> + data1 |= FIELD_PREP(IMX95_PE0_LUT_STREAM_ID, sid);
> + data1 |= IMX95_PE0_LUT_VLD;
> +
> + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA1, data1);
> +
> + data2 = 0xffff;
> + data2 |= FIELD_PREP(IMX95_PE0_LUT_REQID, reqid);
> +
> + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA2, data2);
> +
> + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_ACSCTRL, i);
> +
> + return 0;
> + }
> +
> + dev_err(dev, "All lut already used\n");
> + return -EINVAL;
> +}
> +
> +static void imx_pcie_remove_lut(struct imx_pcie *imx_pcie, u16 reqid)
> +{
> + u32 data2 = 0;
> + int i;
> +
> + guard(mutex)(&imx_pcie->lock);
> +
> + for (i = 0; i < IMX95_MAX_LUT; i++) {
> + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_ACSCTRL, IMX95_PEO_LUT_RWA | i);
> +
> + regmap_read(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA2, &data2);
> + if (FIELD_GET(IMX95_PE0_LUT_REQID, data2) == reqid) {
> + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA1, 0);
> + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA2, 0);
> + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_ACSCTRL, i);
> + }
> + }
> +}
> +
> static void imx_pcie_configure_type(struct imx_pcie *imx_pcie)
> {
> const struct imx_pcie_drvdata *drvdata = imx_pcie->drvdata;
> @@ -1232,6 +1311,85 @@ static int imx_pcie_resume_noirq(struct device *dev)
> return 0;
> }
>
> +static bool imx_pcie_match_device(struct pci_bus *bus);
Can you add the imx_pcie_match_device() earlier in the file so we
don't need this forward declaration?
> +static int imx_pcie_add_device(struct imx_pcie *imx_pcie, struct pci_dev *pdev)
> +{
> + u32 sid_i = 0, sid_m = 0, rid = pci_dev_id(pdev);
> + struct device *dev = imx_pcie->pci->dev;
> + int err;
> +
> + err = of_map_id(dev->of_node, rid, "iommu-map", "iommu-map-mask", NULL, &sid_i);
> + if (err)
> + return err;
> +
> + err = of_map_id(dev->of_node, rid, "msi-map", "msi-map-mask", NULL, &sid_m);
> + if (err)
> + return err;
> +
> + if (sid_i != rid && sid_m != rid)
> + if ((sid_i & IMX95_SID_MASK) != (sid_m & IMX95_SID_MASK)) {
> + dev_err(dev, "its and iommu stream id miss match, please check dts file\n");
> + return -EINVAL;
> + }
> +
> + /* if iommu-map is not existed then use msi-map's stream id*/
Capitalize consistently, e.g., the most comments in this file start
with a capital letter.
s/is not existed/does not exist/
Add space before closing */
> + if (sid_i == rid)
> + sid_i = sid_m;
> +
> + sid_i &= IMX95_SID_MASK;
> +
> + if (sid_i != rid)
> + return imx_pcie_config_lut(imx_pcie, rid, sid_i);
> +
> + /* Use dwc built-in MSI controller */
> + return 0;
> +}
> +
> +static void imx_pcie_del_device(struct imx_pcie *imx_pcie, struct pci_dev *pdev)
> +{
> + imx_pcie_remove_lut(imx_pcie, pci_dev_id(pdev));
> +}
> +
> +
> +static int imx_pcie_bus_notifier(struct notifier_block *nb, unsigned long action, void *data)
> +{
> + struct pci_host_bridge *host;
> + struct imx_pcie *imx_pcie;
> + struct pci_dev *pdev;
> + int err;
> +
> + pdev = to_pci_dev(data);
> + host = pci_find_host_bridge(pdev->bus);
> +
> + if (!imx_pcie_match_device(host->bus))
> + return NOTIFY_OK;
> +
> + imx_pcie = to_imx_pcie(to_dw_pcie_from_pp(host->sysdata));
> +
> + if (!imx_check_flag(imx_pcie, IMX_PCIE_FLAG_MONITOR_DEV))
> + return NOTIFY_OK;
> +
> + switch (action) {
> + case BUS_NOTIFY_ADD_DEVICE:
> + err = imx_pcie_add_device(imx_pcie, pdev);
> + if (err)
> + return notifier_from_errno(err);
> + break;
> + case BUS_NOTIFY_DEL_DEVICE:
> + imx_pcie_del_device(imx_pcie, pdev);
> + break;
> + default:
> + return NOTIFY_DONE;
> + }
> +
> + return NOTIFY_OK;
> +}
> +
> +static struct notifier_block imx_pcie_nb = {
> + .notifier_call = imx_pcie_bus_notifier,
> +};
> +
> static const struct dev_pm_ops imx_pcie_pm_ops = {
> NOIRQ_SYSTEM_SLEEP_PM_OPS(imx_pcie_suspend_noirq,
> imx_pcie_resume_noirq)
> @@ -1264,6 +1422,8 @@ static int imx_pcie_probe(struct platform_device *pdev)
> imx_pcie->pci = pci;
> imx_pcie->drvdata = of_device_get_match_data(dev);
>
> + mutex_init(&imx_pcie->lock);
> +
> /* Find the PHY if one is defined, only imx7d uses it */
> np = of_parse_phandle(node, "fsl,imx7d-pcie-phy", 0);
> if (np) {
> @@ -1551,7 +1711,8 @@ static const struct imx_pcie_drvdata drvdata[] = {
> },
> [IMX95] = {
> .variant = IMX95,
> - .flags = IMX_PCIE_FLAG_HAS_SERDES,
> + .flags = IMX_PCIE_FLAG_HAS_SERDES |
> + IMX_PCIE_FLAG_MONITOR_DEV,
> .clk_names = imx8mq_clks,
> .clks_cnt = ARRAY_SIZE(imx8mq_clks),
> .ltssm_off = IMX95_PE0_GEN_CTRL_3,
> @@ -1687,6 +1848,8 @@ DECLARE_PCI_FIXUP_CLASS_HEADER(PCI_VENDOR_ID_SYNOPSYS, 0xabcd,
>
> static int __init imx_pcie_init(void)
> {
> + int ret;
> +
> #ifdef CONFIG_ARM
> struct device_node *np;
>
> @@ -1705,7 +1868,17 @@ static int __init imx_pcie_init(void)
> hook_fault_code(8, imx6q_pcie_abort_handler, SIGBUS, 0,
> "external abort on non-linefetch");
> #endif
> + ret = bus_register_notifier(&pci_bus_type, &imx_pcie_nb);
> + if (ret)
> + return ret;
I think this should go in imx6_pcie_probe().
> return platform_driver_register(&imx_pcie_driver);
> }
> +
> +static void __exit imx_pcie_exit(void)
> +{
> + bus_unregister_notifier(&pci_bus_type, &imx_pcie_nb);
It looks like this driver is removable?
What happens when an external abort occurs after the
imx6q_pcie_abort_handler() text is removed?
> +}
> +
> device_initcall(imx_pcie_init);
> +__exitcall(imx_pcie_exit);
>
> --
> 2.34.1
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 44+ messages in thread* Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-05-30 23:08 ` Bjorn Helgaas
@ 2024-05-31 13:52 ` Marc Zyngier
2024-05-31 14:58 ` Robin Murphy
2024-05-31 16:14 ` Frank Li
2 siblings, 0 replies; 44+ messages in thread
From: Marc Zyngier @ 2024-05-31 13:52 UTC (permalink / raw)
To: Bjorn Helgaas
Cc: Frank Li, Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley,
linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Will Deacon, Robin Murphy, Joerg Roedel, Jason Gunthorpe,
Alyssa Rosenzweig
On Fri, 31 May 2024 00:08:32 +0100,
Bjorn Helgaas <helgaas@kernel.org> wrote:
>
> [+cc IOMMU and pcie-apple.c folks for comment]
>
> On Tue, May 28, 2024 at 03:39:21PM -0400, Frank Li wrote:
> > For the i.MX95, configuration of a LUT is necessary to convert Bus Device
> > Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS.
> > This involves examining the msi-map and smmu-map to ensure consistent
> > mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related
> > registers are configured. In the absence of an msi-map, the built-in MSI
> > controller is utilized as a fallback.
> >
> > Additionally, register a PCI bus notifier to trigger imx_pcie_add_device()
> > upon the appearance of a new PCI device and when the bus is an iMX6 PCI
> > controller. This function configures the correct LUT based on Device Tree
> > Settings (DTS).
>
> This scheme is pretty similar to apple_pcie_bus_notifier(). If we
> have to do this, I wish it were *more* similar, i.e., copy the
> function names, bitmap tracking, code structure, etc.
>
> I don't really know how stream IDs work, but I assume they are used on
> most or all arm64 platforms, so I'm a little surprised that of all the
> PCI host drivers used on arm64, only pcie-apple.c and pci-imx6.c need
> this notifier.
That's because on sane platforms, PCI's RID and the IOMMU's SID are
the same thing, and there is no need to do anything bizarre. In the
worse case, there is a static transformation that can be applied (IORT
for ACPI, or iommu-map for DT).
Some "creative" systems such as the Apple stuff require some extra
side-band configuration because they don't know what a RID is at all
(the RID isn't conveyed to the IOMMU), nor is there a static (baked in
HW) transformation between RID and SID.
So there is a widget on the side that performs the conversion, and
this widget needs to be programmed. The way it works is that the
driver parses the iommu-map to find the expected and arbitrary) SID
on the IOMMU side for a given RID, and programs the association in the
RID-to-SID mapper. It does so at device probe time in order to make
sure the widget is alive (it seems to be part of the port power
domain).
Yes, this is a terrible hack, and I wish we didn't have to deal with
this sort of crap.
>
> There's this path, which is pretty generic and does at least the
> of_map_id() part of what you're doing in imx_pcie_add_device():
>
> __driver_probe_device
> really_probe
> pci_dma_configure # pci_bus_type.dma_configure
> of_dma_configure
> of_dma_configure_id
> of_iommu_configure
> of_pci_iommu_init
> of_iommu_configure_dev_id
> of_map_id
> of_iommu_xlate
> ops = iommu_ops_from_fwnode
> iommu_fwspec_init
> ops->of_xlate(dev, iommu_spec)
>
> Maybe this needs to be extended somehow with a hook to do the
> device-specific work like updating the LUT? Just speculating here,
> the IOMMU folks will know how this is expected to work.
That'd be a possibility. But this would be adding extra complexity to
the IOMMU core, and I'm not sure it is worth it given that these
systems are thankfully "rare".
It is also something that is conceptually part of the PCIe root port,
and not really the IOMMU, so I'm a bit reluctant to shove things
there. In any case, it would still require callbacks into the PCIe
host driver, and I have the feeling that we'd be reinventing the bus
notifier wheel, only with pointier angles...
Thanks,
M.
--
Without deviation from the norm, progress is not possible.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-05-30 23:08 ` Bjorn Helgaas
2024-05-31 13:52 ` Marc Zyngier
@ 2024-05-31 14:58 ` Robin Murphy
2024-05-31 16:25 ` Frank Li
` (2 more replies)
2024-05-31 16:14 ` Frank Li
2 siblings, 3 replies; 44+ messages in thread
From: Robin Murphy @ 2024-05-31 14:58 UTC (permalink / raw)
To: Bjorn Helgaas, Frank Li
Cc: Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley,
linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Will Deacon, Joerg Roedel, Jason Gunthorpe, Alyssa Rosenzweig,
Marc Zyngier
On 2024-05-31 12:08 am, Bjorn Helgaas wrote:
> [+cc IOMMU and pcie-apple.c folks for comment]
>
> On Tue, May 28, 2024 at 03:39:21PM -0400, Frank Li wrote:
>> For the i.MX95, configuration of a LUT is necessary to convert Bus Device
>> Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS.
>> This involves examining the msi-map and smmu-map to ensure consistent
>> mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related
>> registers are configured. In the absence of an msi-map, the built-in MSI
>> controller is utilized as a fallback.
>>
>> Additionally, register a PCI bus notifier to trigger imx_pcie_add_device()
>> upon the appearance of a new PCI device and when the bus is an iMX6 PCI
>> controller. This function configures the correct LUT based on Device Tree
>> Settings (DTS).
>
> This scheme is pretty similar to apple_pcie_bus_notifier(). If we
> have to do this, I wish it were *more* similar, i.e., copy the
> function names, bitmap tracking, code structure, etc.
>
> I don't really know how stream IDs work, but I assume they are used on
> most or all arm64 platforms, so I'm a little surprised that of all the
> PCI host drivers used on arm64, only pcie-apple.c and pci-imx6.c need
> this notifier.
This is one of those things that's mostly at the mercy of the PCIe root
complex implementation. Typically the SMMU StreamID and/or GIC ITS
DeviceID is derived directly from the PCI RID, sometimes with additional
high-order bits hard-wired to disambiguate PCI segments. I believe this
RID-translation LUT is a particular feature of the the Synopsys IP - I
know there's also one on the NXP Layerscape platforms, but on those it's
programmed by the bootloader, which also generates the appropriate
"msi-map" and "iommu-map" properties to match. Ideally that's what i.MX
should do as well, but hey.
> There's this path, which is pretty generic and does at least the
> of_map_id() part of what you're doing in imx_pcie_add_device():
>
> __driver_probe_device
> really_probe
> pci_dma_configure # pci_bus_type.dma_configure
> of_dma_configure
> of_dma_configure_id
> of_iommu_configure
> of_pci_iommu_init
> of_iommu_configure_dev_id
> of_map_id
> of_iommu_xlate
> ops = iommu_ops_from_fwnode
> iommu_fwspec_init
> ops->of_xlate(dev, iommu_spec)
>
> Maybe this needs to be extended somehow with a hook to do the
> device-specific work like updating the LUT? Just speculating here,
> the IOMMU folks will know how this is expected to work.
Note that that particular code path has fundamental issues and much of
it needs to go away (I'm working on it, but it's a rich ~8-year-old pile
of technical debt...). IOMMU configuration needs to be happening at
device_add() time via the IOMMU layer's own bus notifier.
If it's really necessary to do this programming from Linux, then there's
still no point in it being dynamic - the mappings cannot ever change,
since the rest of the kernel believes that what the DT said at boot time
was already a property of the hardware. It would be a lot more logical,
and likely simpler, for the driver to just read the relevant map
property and program the entire LUT to match, all in one go at
controller probe time. Rather like what's already commonly done with the
parsing of "dma-ranges" to program address-translation LUTs for inbound
windows.
Plus that would also give a chance of safely dealing with bad DTs
specifying invalid ID mappings (by refusing to probe at all). As it is,
returning an error from a child's BUS_NOTIFY_ADD_DEVICE does nothing
except prevent any further notifiers from running at that point - the
device will still be added, allowed to bind a driver, and able to start
sending DMA/MSI traffic without the controller being correctly
programmed, which at best won't work and at worst may break the whole
system.
Thanks,
Robin.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 44+ messages in thread* Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-05-31 14:58 ` Robin Murphy
@ 2024-05-31 16:25 ` Frank Li
2024-06-03 17:19 ` Bjorn Helgaas
2024-06-03 20:29 ` Laurentiu Tudor
2 siblings, 0 replies; 44+ messages in thread
From: Frank Li @ 2024-05-31 16:25 UTC (permalink / raw)
To: Robin Murphy
Cc: Bjorn Helgaas, Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley,
linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Will Deacon, Joerg Roedel, Jason Gunthorpe, Alyssa Rosenzweig,
Marc Zyngier
On Fri, May 31, 2024 at 03:58:49PM +0100, Robin Murphy wrote:
> On 2024-05-31 12:08 am, Bjorn Helgaas wrote:
> > [+cc IOMMU and pcie-apple.c folks for comment]
> >
> > On Tue, May 28, 2024 at 03:39:21PM -0400, Frank Li wrote:
> > > For the i.MX95, configuration of a LUT is necessary to convert Bus Device
> > > Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS.
> > > This involves examining the msi-map and smmu-map to ensure consistent
> > > mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related
> > > registers are configured. In the absence of an msi-map, the built-in MSI
> > > controller is utilized as a fallback.
> > >
> > > Additionally, register a PCI bus notifier to trigger imx_pcie_add_device()
> > > upon the appearance of a new PCI device and when the bus is an iMX6 PCI
> > > controller. This function configures the correct LUT based on Device Tree
> > > Settings (DTS).
> >
> > This scheme is pretty similar to apple_pcie_bus_notifier(). If we
> > have to do this, I wish it were *more* similar, i.e., copy the
> > function names, bitmap tracking, code structure, etc.
> >
> > I don't really know how stream IDs work, but I assume they are used on
> > most or all arm64 platforms, so I'm a little surprised that of all the
> > PCI host drivers used on arm64, only pcie-apple.c and pci-imx6.c need
> > this notifier.
>
> This is one of those things that's mostly at the mercy of the PCIe root
> complex implementation. Typically the SMMU StreamID and/or GIC ITS DeviceID
> is derived directly from the PCI RID, sometimes with additional high-order
> bits hard-wired to disambiguate PCI segments. I believe this RID-translation
> LUT is a particular feature of the the Synopsys IP - I know there's also one
> on the NXP Layerscape platforms, but on those it's programmed by the
> bootloader, which also generates the appropriate "msi-map" and "iommu-map"
> properties to match. Ideally that's what i.MX should do as well, but hey.
Actually, I think it is not good for uboot config it because PCIe device is
hotplug, such as SD7.0. SD7.0 card may plug after system boot. uboot miss
config it because uboot scan nothing when it run.
>
> > There's this path, which is pretty generic and does at least the
> > of_map_id() part of what you're doing in imx_pcie_add_device():
> >
> > __driver_probe_device
> > really_probe
> > pci_dma_configure # pci_bus_type.dma_configure
> > of_dma_configure
> > of_dma_configure_id
> > of_iommu_configure
> > of_pci_iommu_init
> > of_iommu_configure_dev_id
> > of_map_id
> > of_iommu_xlate
> > ops = iommu_ops_from_fwnode
> > iommu_fwspec_init
> > ops->of_xlate(dev, iommu_spec)
> >
> > Maybe this needs to be extended somehow with a hook to do the
> > device-specific work like updating the LUT? Just speculating here,
> > the IOMMU folks will know how this is expected to work.
>
> Note that that particular code path has fundamental issues and much of it
> needs to go away (I'm working on it, but it's a rich ~8-year-old pile of
> technical debt...). IOMMU configuration needs to be happening at
> device_add() time via the IOMMU layer's own bus notifier.
>
> If it's really necessary to do this programming from Linux, then there's
> still no point in it being dynamic - the mappings cannot ever change, since
> the rest of the kernel believes that what the DT said at boot time was
> already a property of the hardware. It would be a lot more logical, and
> likely simpler, for the driver to just read the relevant map property and
> program the entire LUT to match, all in one go at controller probe time.
> Rather like what's already commonly done with the parsing of "dma-ranges" to
> program address-translation LUTs for inbound windows.
Do you means prefer v3's method?
https://lore.kernel.org/imx/20240402-pci2_upstream-v3-8-803414bdb430@nxp.com/
>
> Plus that would also give a chance of safely dealing with bad DTs specifying
> invalid ID mappings (by refusing to probe at all). As it is, returning an
> error from a child's BUS_NOTIFY_ADD_DEVICE does nothing except prevent any
> further notifiers from running at that point - the device will still be
> added, allowed to bind a driver, and able to start sending DMA/MSI traffic
> without the controller being correctly programmed, which at best won't work
> and at worst may break the whole system.
>
> Thanks,
> Robin.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-05-31 14:58 ` Robin Murphy
2024-05-31 16:25 ` Frank Li
@ 2024-06-03 17:19 ` Bjorn Helgaas
2024-06-03 18:42 ` Frank Li
2024-06-03 20:20 ` Robin Murphy
2024-06-03 20:29 ` Laurentiu Tudor
2 siblings, 2 replies; 44+ messages in thread
From: Bjorn Helgaas @ 2024-06-03 17:19 UTC (permalink / raw)
To: Robin Murphy
Cc: Frank Li, Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley,
linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Will Deacon, Joerg Roedel, Jason Gunthorpe, Alyssa Rosenzweig,
Marc Zyngier
On Fri, May 31, 2024 at 03:58:49PM +0100, Robin Murphy wrote:
> On 2024-05-31 12:08 am, Bjorn Helgaas wrote:
> > [+cc IOMMU and pcie-apple.c folks for comment]
> >
> > On Tue, May 28, 2024 at 03:39:21PM -0400, Frank Li wrote:
> > > For the i.MX95, configuration of a LUT is necessary to convert Bus Device
> > > Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS.
> > > This involves examining the msi-map and smmu-map to ensure consistent
> > > mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related
> > > registers are configured. In the absence of an msi-map, the built-in MSI
> > > controller is utilized as a fallback.
> > >
> > > Additionally, register a PCI bus notifier to trigger imx_pcie_add_device()
> > > upon the appearance of a new PCI device and when the bus is an iMX6 PCI
> > > controller. This function configures the correct LUT based on Device Tree
> > > Settings (DTS).
> >
> > This scheme is pretty similar to apple_pcie_bus_notifier(). If we
> > have to do this, I wish it were *more* similar, i.e., copy the
> > function names, bitmap tracking, code structure, etc.
> >
> > I don't really know how stream IDs work, but I assume they are used on
> > most or all arm64 platforms, so I'm a little surprised that of all the
> > PCI host drivers used on arm64, only pcie-apple.c and pci-imx6.c need
> > this notifier.
>
> This is one of those things that's mostly at the mercy of the PCIe root
> complex implementation. Typically the SMMU StreamID and/or GIC ITS DeviceID
> is derived directly from the PCI RID, sometimes with additional high-order
> bits hard-wired to disambiguate PCI segments. I believe this RID-translation
> LUT is a particular feature of the the Synopsys IP - I know there's also one
> on the NXP Layerscape platforms, but on those it's programmed by the
> bootloader, which also generates the appropriate "msi-map" and "iommu-map"
> properties to match. Ideally that's what i.MX should do as well, but hey.
Maybe this RID-translation is a feature of i.MX, not of Synopsys? I
see that the LUT CSR accesses use IMX95_* definitions.
> If it's really necessary to do this programming from Linux, then there's
> still no point in it being dynamic - the mappings cannot ever change, since
> the rest of the kernel believes that what the DT said at boot time was
> already a property of the hardware. It would be a lot more logical, and
> likely simpler, for the driver to just read the relevant map property and
> program the entire LUT to match, all in one go at controller probe time.
> Rather like what's already commonly done with the parsing of "dma-ranges" to
> program address-translation LUTs for inbound windows.
>
> Plus that would also give a chance of safely dealing with bad DTs specifying
> invalid ID mappings (by refusing to probe at all). As it is, returning an
> error from a child's BUS_NOTIFY_ADD_DEVICE does nothing except prevent any
> further notifiers from running at that point - the device will still be
> added, allowed to bind a driver, and able to start sending DMA/MSI traffic
> without the controller being correctly programmed, which at best won't work
> and at worst may break the whole system.
Frank, could the imx LUT be programmed once at boot-time instead of at
device-add time? I'm guessing maybe not because apparently there is a
risk of running out of LUT entries?
It sounds like the consequences of running out of LUT entries are
catastrophic, e.g., memory corruption from mis-directed DMA? If
that's possible, I think we need to figure out how to prevent the
device from being used, not just dev_warn() about it.
Bjorn
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-06-03 17:19 ` Bjorn Helgaas
@ 2024-06-03 18:42 ` Frank Li
2024-06-03 18:56 ` Bjorn Helgaas
2024-06-03 20:20 ` Robin Murphy
1 sibling, 1 reply; 44+ messages in thread
From: Frank Li @ 2024-06-03 18:42 UTC (permalink / raw)
To: Bjorn Helgaas
Cc: Robin Murphy, Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley,
linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Will Deacon, Joerg Roedel, Jason Gunthorpe, Alyssa Rosenzweig,
Marc Zyngier
On Mon, Jun 03, 2024 at 12:19:21PM -0500, Bjorn Helgaas wrote:
> On Fri, May 31, 2024 at 03:58:49PM +0100, Robin Murphy wrote:
> > On 2024-05-31 12:08 am, Bjorn Helgaas wrote:
> > > [+cc IOMMU and pcie-apple.c folks for comment]
> > >
> > > On Tue, May 28, 2024 at 03:39:21PM -0400, Frank Li wrote:
> > > > For the i.MX95, configuration of a LUT is necessary to convert Bus Device
> > > > Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS.
> > > > This involves examining the msi-map and smmu-map to ensure consistent
> > > > mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related
> > > > registers are configured. In the absence of an msi-map, the built-in MSI
> > > > controller is utilized as a fallback.
> > > >
> > > > Additionally, register a PCI bus notifier to trigger imx_pcie_add_device()
> > > > upon the appearance of a new PCI device and when the bus is an iMX6 PCI
> > > > controller. This function configures the correct LUT based on Device Tree
> > > > Settings (DTS).
> > >
> > > This scheme is pretty similar to apple_pcie_bus_notifier(). If we
> > > have to do this, I wish it were *more* similar, i.e., copy the
> > > function names, bitmap tracking, code structure, etc.
> > >
> > > I don't really know how stream IDs work, but I assume they are used on
> > > most or all arm64 platforms, so I'm a little surprised that of all the
> > > PCI host drivers used on arm64, only pcie-apple.c and pci-imx6.c need
> > > this notifier.
> >
> > This is one of those things that's mostly at the mercy of the PCIe root
> > complex implementation. Typically the SMMU StreamID and/or GIC ITS DeviceID
> > is derived directly from the PCI RID, sometimes with additional high-order
> > bits hard-wired to disambiguate PCI segments. I believe this RID-translation
> > LUT is a particular feature of the the Synopsys IP - I know there's also one
> > on the NXP Layerscape platforms, but on those it's programmed by the
> > bootloader, which also generates the appropriate "msi-map" and "iommu-map"
> > properties to match. Ideally that's what i.MX should do as well, but hey.
>
> Maybe this RID-translation is a feature of i.MX, not of Synopsys? I
> see that the LUT CSR accesses use IMX95_* definitions.
Yes, it convert 16bit RID to 6bit stream id.
>
> > If it's really necessary to do this programming from Linux, then there's
> > still no point in it being dynamic - the mappings cannot ever change, since
> > the rest of the kernel believes that what the DT said at boot time was
> > already a property of the hardware. It would be a lot more logical, and
> > likely simpler, for the driver to just read the relevant map property and
> > program the entire LUT to match, all in one go at controller probe time.
> > Rather like what's already commonly done with the parsing of "dma-ranges" to
> > program address-translation LUTs for inbound windows.
> >
> > Plus that would also give a chance of safely dealing with bad DTs specifying
> > invalid ID mappings (by refusing to probe at all). As it is, returning an
> > error from a child's BUS_NOTIFY_ADD_DEVICE does nothing except prevent any
> > further notifiers from running at that point - the device will still be
> > added, allowed to bind a driver, and able to start sending DMA/MSI traffic
> > without the controller being correctly programmed, which at best won't work
> > and at worst may break the whole system.
>
> Frank, could the imx LUT be programmed once at boot-time instead of at
> device-add time? I'm guessing maybe not because apparently there is a
> risk of running out of LUT entries?
It is not good idea to depend on boot loader so much. Some hot plug devics
(SD7.0) may plug after system boot. Two PCIe instances shared one set
of 6bits stream id (total 64). Assume total 16 assign to two PCIe
controllers. each have 8 stream id. If use uboot assign it static, each
PCIe controller have below 8 devices. It will be failrue one controller
connect 7, another connect 9. but if dynamtic alloc when devices add, both
controller can work.
Although we have not so much devices now, this way give us possility to
improve it in future.
>
> It sounds like the consequences of running out of LUT entries are
> catastrophic, e.g., memory corruption from mis-directed DMA? If
> that's possible, I think we need to figure out how to prevent the
> device from being used, not just dev_warn() about it.
Yes, but so far, we have not met such problem now. We can improve it when
we really face such problem.
>
> Bjorn
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-06-03 18:42 ` Frank Li
@ 2024-06-03 18:56 ` Bjorn Helgaas
2024-06-03 20:07 ` Frank Li
0 siblings, 1 reply; 44+ messages in thread
From: Bjorn Helgaas @ 2024-06-03 18:56 UTC (permalink / raw)
To: Frank Li
Cc: Robin Murphy, Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley,
linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Will Deacon, Joerg Roedel, Jason Gunthorpe, Alyssa Rosenzweig,
Marc Zyngier
On Mon, Jun 03, 2024 at 02:42:45PM -0400, Frank Li wrote:
> On Mon, Jun 03, 2024 at 12:19:21PM -0500, Bjorn Helgaas wrote:
> > On Fri, May 31, 2024 at 03:58:49PM +0100, Robin Murphy wrote:
> > > On 2024-05-31 12:08 am, Bjorn Helgaas wrote:
> > > > [+cc IOMMU and pcie-apple.c folks for comment]
> > > >
> > > > On Tue, May 28, 2024 at 03:39:21PM -0400, Frank Li wrote:
> > > > > For the i.MX95, configuration of a LUT is necessary to convert Bus Device
> > > > > Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS.
> > > > > This involves examining the msi-map and smmu-map to ensure consistent
> > > > > mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related
> > > > > registers are configured. In the absence of an msi-map, the built-in MSI
> > > > > controller is utilized as a fallback.
> > > > >
> > > > > Additionally, register a PCI bus notifier to trigger imx_pcie_add_device()
> > > > > upon the appearance of a new PCI device and when the bus is an iMX6 PCI
> > > > > controller. This function configures the correct LUT based on Device Tree
> > > > > Settings (DTS).
> > > >
> > > > This scheme is pretty similar to apple_pcie_bus_notifier(). If we
> > > > have to do this, I wish it were *more* similar, i.e., copy the
> > > > function names, bitmap tracking, code structure, etc.
> > > >
> > > > I don't really know how stream IDs work, but I assume they are used on
> > > > most or all arm64 platforms, so I'm a little surprised that of all the
> > > > PCI host drivers used on arm64, only pcie-apple.c and pci-imx6.c need
> > > > this notifier.
> > >
> > > This is one of those things that's mostly at the mercy of the PCIe root
> > > complex implementation. Typically the SMMU StreamID and/or GIC ITS DeviceID
> > > is derived directly from the PCI RID, sometimes with additional high-order
> > > bits hard-wired to disambiguate PCI segments. I believe this RID-translation
> > > LUT is a particular feature of the the Synopsys IP - I know there's also one
> > > on the NXP Layerscape platforms, but on those it's programmed by the
> > > bootloader, which also generates the appropriate "msi-map" and "iommu-map"
> > > properties to match. Ideally that's what i.MX should do as well, but hey.
> >
> > Maybe this RID-translation is a feature of i.MX, not of Synopsys? I
> > see that the LUT CSR accesses use IMX95_* definitions.
>
> Yes, it convert 16bit RID to 6bit stream id.
IIUC, you're saying this is not a Synopsys feature, it's an i.MX
feature.
> > > If it's really necessary to do this programming from Linux, then there's
> > > still no point in it being dynamic - the mappings cannot ever change, since
> > > the rest of the kernel believes that what the DT said at boot time was
> > > already a property of the hardware. It would be a lot more logical, and
> > > likely simpler, for the driver to just read the relevant map property and
> > > program the entire LUT to match, all in one go at controller probe time.
> > > Rather like what's already commonly done with the parsing of "dma-ranges" to
> > > program address-translation LUTs for inbound windows.
> > >
> > > Plus that would also give a chance of safely dealing with bad DTs specifying
> > > invalid ID mappings (by refusing to probe at all). As it is, returning an
> > > error from a child's BUS_NOTIFY_ADD_DEVICE does nothing except prevent any
> > > further notifiers from running at that point - the device will still be
> > > added, allowed to bind a driver, and able to start sending DMA/MSI traffic
> > > without the controller being correctly programmed, which at best won't work
> > > and at worst may break the whole system.
> >
> > Frank, could the imx LUT be programmed once at boot-time instead of at
> > device-add time? I'm guessing maybe not because apparently there is a
> > risk of running out of LUT entries?
>
> It is not good idea to depend on boot loader so much.
I meant "could this be programmed once when the Linux imx host
controller driver is probed?" But from the below, it sounds like
that's not possible in general because you don't have enough stream
IDs to do that.
> Some hot plug devics
> (SD7.0) may plug after system boot. Two PCIe instances shared one set
> of 6bits stream id (total 64). Assume total 16 assign to two PCIe
> controllers. each have 8 stream id. If use uboot assign it static, each
> PCIe controller have below 8 devices. It will be failrue one controller
> connect 7, another connect 9. but if dynamtic alloc when devices add, both
> controller can work.
>
> Although we have not so much devices now, this way give us possility to
> improve it in future.
>
> > It sounds like the consequences of running out of LUT entries are
> > catastrophic, e.g., memory corruption from mis-directed DMA? If
> > that's possible, I think we need to figure out how to prevent the
> > device from being used, not just dev_warn() about it.
>
> Yes, but so far, we have not met such problem now. We can improve it when
> we really face such problem.
If this controller can only support DMA from a limited number of
endpoints below it, I think we should figure out how to enforce that
directly. Maybe we can prevent drivers from enabling bus mastering or
something. I'm not happy with the idea of waiting for and debugging a
report of data corruption.
Bjorn
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-06-03 18:56 ` Bjorn Helgaas
@ 2024-06-03 20:07 ` Frank Li
2024-06-06 20:24 ` Frank Li
0 siblings, 1 reply; 44+ messages in thread
From: Frank Li @ 2024-06-03 20:07 UTC (permalink / raw)
To: Bjorn Helgaas
Cc: Robin Murphy, Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley,
linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Will Deacon, Joerg Roedel, Jason Gunthorpe, Alyssa Rosenzweig,
Marc Zyngier
On Mon, Jun 03, 2024 at 01:56:27PM -0500, Bjorn Helgaas wrote:
> On Mon, Jun 03, 2024 at 02:42:45PM -0400, Frank Li wrote:
> > On Mon, Jun 03, 2024 at 12:19:21PM -0500, Bjorn Helgaas wrote:
> > > On Fri, May 31, 2024 at 03:58:49PM +0100, Robin Murphy wrote:
> > > > On 2024-05-31 12:08 am, Bjorn Helgaas wrote:
> > > > > [+cc IOMMU and pcie-apple.c folks for comment]
> > > > >
> > > > > On Tue, May 28, 2024 at 03:39:21PM -0400, Frank Li wrote:
> > > > > > For the i.MX95, configuration of a LUT is necessary to convert Bus Device
> > > > > > Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS.
> > > > > > This involves examining the msi-map and smmu-map to ensure consistent
> > > > > > mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related
> > > > > > registers are configured. In the absence of an msi-map, the built-in MSI
> > > > > > controller is utilized as a fallback.
> > > > > >
> > > > > > Additionally, register a PCI bus notifier to trigger imx_pcie_add_device()
> > > > > > upon the appearance of a new PCI device and when the bus is an iMX6 PCI
> > > > > > controller. This function configures the correct LUT based on Device Tree
> > > > > > Settings (DTS).
> > > > >
> > > > > This scheme is pretty similar to apple_pcie_bus_notifier(). If we
> > > > > have to do this, I wish it were *more* similar, i.e., copy the
> > > > > function names, bitmap tracking, code structure, etc.
> > > > >
> > > > > I don't really know how stream IDs work, but I assume they are used on
> > > > > most or all arm64 platforms, so I'm a little surprised that of all the
> > > > > PCI host drivers used on arm64, only pcie-apple.c and pci-imx6.c need
> > > > > this notifier.
> > > >
> > > > This is one of those things that's mostly at the mercy of the PCIe root
> > > > complex implementation. Typically the SMMU StreamID and/or GIC ITS DeviceID
> > > > is derived directly from the PCI RID, sometimes with additional high-order
> > > > bits hard-wired to disambiguate PCI segments. I believe this RID-translation
> > > > LUT is a particular feature of the the Synopsys IP - I know there's also one
> > > > on the NXP Layerscape platforms, but on those it's programmed by the
> > > > bootloader, which also generates the appropriate "msi-map" and "iommu-map"
> > > > properties to match. Ideally that's what i.MX should do as well, but hey.
> > >
> > > Maybe this RID-translation is a feature of i.MX, not of Synopsys? I
> > > see that the LUT CSR accesses use IMX95_* definitions.
> >
> > Yes, it convert 16bit RID to 6bit stream id.
>
> IIUC, you're saying this is not a Synopsys feature, it's an i.MX
> feature.
Yes, it is i.MX feature. But I think other vendor should have similar
situation if use old arm smmu.
>
> > > > If it's really necessary to do this programming from Linux, then there's
> > > > still no point in it being dynamic - the mappings cannot ever change, since
> > > > the rest of the kernel believes that what the DT said at boot time was
> > > > already a property of the hardware. It would be a lot more logical, and
> > > > likely simpler, for the driver to just read the relevant map property and
> > > > program the entire LUT to match, all in one go at controller probe time.
> > > > Rather like what's already commonly done with the parsing of "dma-ranges" to
> > > > program address-translation LUTs for inbound windows.
> > > >
> > > > Plus that would also give a chance of safely dealing with bad DTs specifying
> > > > invalid ID mappings (by refusing to probe at all). As it is, returning an
> > > > error from a child's BUS_NOTIFY_ADD_DEVICE does nothing except prevent any
> > > > further notifiers from running at that point - the device will still be
> > > > added, allowed to bind a driver, and able to start sending DMA/MSI traffic
> > > > without the controller being correctly programmed, which at best won't work
> > > > and at worst may break the whole system.
> > >
> > > Frank, could the imx LUT be programmed once at boot-time instead of at
> > > device-add time? I'm guessing maybe not because apparently there is a
> > > risk of running out of LUT entries?
> >
> > It is not good idea to depend on boot loader so much.
>
> I meant "could this be programmed once when the Linux imx host
> controller driver is probed?" But from the below, it sounds like
> that's not possible in general because you don't have enough stream
> IDs to do that.
Oh! sorry miss understand what your means. It is possible like what I did
at v3 version. But I think it is not good enough.
>
> > Some hot plug devics
> > (SD7.0) may plug after system boot. Two PCIe instances shared one set
> > of 6bits stream id (total 64). Assume total 16 assign to two PCIe
> > controllers. each have 8 stream id. If use uboot assign it static, each
> > PCIe controller have below 8 devices. It will be failrue one controller
> > connect 7, another connect 9. but if dynamtic alloc when devices add, both
> > controller can work.
> >
> > Although we have not so much devices now, this way give us possility to
> > improve it in future.
> >
> > > It sounds like the consequences of running out of LUT entries are
> > > catastrophic, e.g., memory corruption from mis-directed DMA? If
> > > that's possible, I think we need to figure out how to prevent the
> > > device from being used, not just dev_warn() about it.
> >
> > Yes, but so far, we have not met such problem now. We can improve it when
> > we really face such problem.
>
> If this controller can only support DMA from a limited number of
> endpoints below it, I think we should figure out how to enforce that
> directly. Maybe we can prevent drivers from enabling bus mastering or
> something. I'm not happy with the idea of waiting for and debugging a
> report of data corruption.
It may add a pre-add hook function to pci bridge. let me do more research.
Frank
>
> Bjorn
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-06-03 20:07 ` Frank Li
@ 2024-06-06 20:24 ` Frank Li
2024-06-13 22:41 ` Bjorn Helgaas
0 siblings, 1 reply; 44+ messages in thread
From: Frank Li @ 2024-06-06 20:24 UTC (permalink / raw)
To: Bjorn Helgaas
Cc: Robin Murphy, Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley,
linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Will Deacon, Joerg Roedel, Jason Gunthorpe, Alyssa Rosenzweig,
Marc Zyngier
On Mon, Jun 03, 2024 at 04:07:55PM -0400, Frank Li wrote:
> On Mon, Jun 03, 2024 at 01:56:27PM -0500, Bjorn Helgaas wrote:
> > On Mon, Jun 03, 2024 at 02:42:45PM -0400, Frank Li wrote:
> > > On Mon, Jun 03, 2024 at 12:19:21PM -0500, Bjorn Helgaas wrote:
> > > > On Fri, May 31, 2024 at 03:58:49PM +0100, Robin Murphy wrote:
> > > > > On 2024-05-31 12:08 am, Bjorn Helgaas wrote:
> > > > > > [+cc IOMMU and pcie-apple.c folks for comment]
> > > > > >
> > > > > > On Tue, May 28, 2024 at 03:39:21PM -0400, Frank Li wrote:
> > > > > > > For the i.MX95, configuration of a LUT is necessary to convert Bus Device
> > > > > > > Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS.
> > > > > > > This involves examining the msi-map and smmu-map to ensure consistent
> > > > > > > mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related
> > > > > > > registers are configured. In the absence of an msi-map, the built-in MSI
> > > > > > > controller is utilized as a fallback.
> > > > > > >
> > > > > > > Additionally, register a PCI bus notifier to trigger imx_pcie_add_device()
> > > > > > > upon the appearance of a new PCI device and when the bus is an iMX6 PCI
> > > > > > > controller. This function configures the correct LUT based on Device Tree
> > > > > > > Settings (DTS).
> > > > > >
> > > > > > This scheme is pretty similar to apple_pcie_bus_notifier(). If we
> > > > > > have to do this, I wish it were *more* similar, i.e., copy the
> > > > > > function names, bitmap tracking, code structure, etc.
> > > > > >
> > > > > > I don't really know how stream IDs work, but I assume they are used on
> > > > > > most or all arm64 platforms, so I'm a little surprised that of all the
> > > > > > PCI host drivers used on arm64, only pcie-apple.c and pci-imx6.c need
> > > > > > this notifier.
> > > > >
> > > > > This is one of those things that's mostly at the mercy of the PCIe root
> > > > > complex implementation. Typically the SMMU StreamID and/or GIC ITS DeviceID
> > > > > is derived directly from the PCI RID, sometimes with additional high-order
> > > > > bits hard-wired to disambiguate PCI segments. I believe this RID-translation
> > > > > LUT is a particular feature of the the Synopsys IP - I know there's also one
> > > > > on the NXP Layerscape platforms, but on those it's programmed by the
> > > > > bootloader, which also generates the appropriate "msi-map" and "iommu-map"
> > > > > properties to match. Ideally that's what i.MX should do as well, but hey.
> > > >
> > > > Maybe this RID-translation is a feature of i.MX, not of Synopsys? I
> > > > see that the LUT CSR accesses use IMX95_* definitions.
> > >
> > > Yes, it convert 16bit RID to 6bit stream id.
> >
> > IIUC, you're saying this is not a Synopsys feature, it's an i.MX
> > feature.
>
> Yes, it is i.MX feature. But I think other vendor should have similar
> situation if use old arm smmu.
>
> >
> > > > > If it's really necessary to do this programming from Linux, then there's
> > > > > still no point in it being dynamic - the mappings cannot ever change, since
> > > > > the rest of the kernel believes that what the DT said at boot time was
> > > > > already a property of the hardware. It would be a lot more logical, and
> > > > > likely simpler, for the driver to just read the relevant map property and
> > > > > program the entire LUT to match, all in one go at controller probe time.
> > > > > Rather like what's already commonly done with the parsing of "dma-ranges" to
> > > > > program address-translation LUTs for inbound windows.
> > > > >
> > > > > Plus that would also give a chance of safely dealing with bad DTs specifying
> > > > > invalid ID mappings (by refusing to probe at all). As it is, returning an
> > > > > error from a child's BUS_NOTIFY_ADD_DEVICE does nothing except prevent any
> > > > > further notifiers from running at that point - the device will still be
> > > > > added, allowed to bind a driver, and able to start sending DMA/MSI traffic
> > > > > without the controller being correctly programmed, which at best won't work
> > > > > and at worst may break the whole system.
> > > >
> > > > Frank, could the imx LUT be programmed once at boot-time instead of at
> > > > device-add time? I'm guessing maybe not because apparently there is a
> > > > risk of running out of LUT entries?
> > >
> > > It is not good idea to depend on boot loader so much.
> >
> > I meant "could this be programmed once when the Linux imx host
> > controller driver is probed?" But from the below, it sounds like
> > that's not possible in general because you don't have enough stream
> > IDs to do that.
>
> Oh! sorry miss understand what your means. It is possible like what I did
> at v3 version. But I think it is not good enough.
>
> >
> > > Some hot plug devics
> > > (SD7.0) may plug after system boot. Two PCIe instances shared one set
> > > of 6bits stream id (total 64). Assume total 16 assign to two PCIe
> > > controllers. each have 8 stream id. If use uboot assign it static, each
> > > PCIe controller have below 8 devices. It will be failrue one controller
> > > connect 7, another connect 9. but if dynamtic alloc when devices add, both
> > > controller can work.
> > >
> > > Although we have not so much devices now, this way give us possility to
> > > improve it in future.
> > >
> > > > It sounds like the consequences of running out of LUT entries are
> > > > catastrophic, e.g., memory corruption from mis-directed DMA? If
> > > > that's possible, I think we need to figure out how to prevent the
> > > > device from being used, not just dev_warn() about it.
> > >
> > > Yes, but so far, we have not met such problem now. We can improve it when
> > > we really face such problem.
> >
> > If this controller can only support DMA from a limited number of
> > endpoints below it, I think we should figure out how to enforce that
> > directly. Maybe we can prevent drivers from enabling bus mastering or
> > something. I'm not happy with the idea of waiting for and debugging a
> > report of data corruption.
>
> It may add a pre-add hook function to pci bridge. let me do more research.
Hi Bjorn:
int pci_setup_device(struct pci_dev *dev)
{
dev->error_state = pci_channel_io_normal;
...
pci_fixup_device(pci_fixup_early, dev);
^^^ I can add fixup hook for pci_fixup_early. If not resource,
I can set dev->error_state to pci_channel_io_frozen or
pci_channel_io_perm_failure
And add below check here after call hook function.
if (dev->error_state != pci_channel_io_normal)
return -EIO;
}
How do you think this method? If you agree, I can continue search device
remove hook up.
Frank
>
> Frank
>
> >
> > Bjorn
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 44+ messages in thread* Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-06-06 20:24 ` Frank Li
@ 2024-06-13 22:41 ` Bjorn Helgaas
2024-06-17 14:26 ` Frank Li
0 siblings, 1 reply; 44+ messages in thread
From: Bjorn Helgaas @ 2024-06-13 22:41 UTC (permalink / raw)
To: Frank Li
Cc: Robin Murphy, Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley,
linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Will Deacon, Joerg Roedel, Jason Gunthorpe, Alyssa Rosenzweig,
Marc Zyngier
On Thu, Jun 06, 2024 at 04:24:17PM -0400, Frank Li wrote:
> On Mon, Jun 03, 2024 at 04:07:55PM -0400, Frank Li wrote:
> > On Mon, Jun 03, 2024 at 01:56:27PM -0500, Bjorn Helgaas wrote:
> > > On Mon, Jun 03, 2024 at 02:42:45PM -0400, Frank Li wrote:
> > > > On Mon, Jun 03, 2024 at 12:19:21PM -0500, Bjorn Helgaas wrote:
> > > > > On Fri, May 31, 2024 at 03:58:49PM +0100, Robin Murphy wrote:
> > > > > > On 2024-05-31 12:08 am, Bjorn Helgaas wrote:
> > > > > > > [+cc IOMMU and pcie-apple.c folks for comment]
> > > > > > >
> > > > > > > On Tue, May 28, 2024 at 03:39:21PM -0400, Frank Li wrote:
> > > > > > > > For the i.MX95, configuration of a LUT is necessary to convert Bus Device
> > > > > > > > Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS.
> > > > > > > > This involves examining the msi-map and smmu-map to ensure consistent
> > > > > > > > mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related
> > > > > > > > registers are configured. In the absence of an msi-map, the built-in MSI
> > > > > > > > controller is utilized as a fallback.
> > > > > > > >
> > > > > > > > Additionally, register a PCI bus notifier to trigger imx_pcie_add_device()
> > > > > > > > upon the appearance of a new PCI device and when the bus is an iMX6 PCI
> > > > > > > > controller. This function configures the correct LUT based on Device Tree
> > > > > > > > Settings (DTS).
> > > > > > >
> > > > > > > This scheme is pretty similar to apple_pcie_bus_notifier(). If we
> > > > > > > have to do this, I wish it were *more* similar, i.e., copy the
> > > > > > > function names, bitmap tracking, code structure, etc.
> > > > > > >
> > > > > > > I don't really know how stream IDs work, but I assume they are used on
> > > > > > > most or all arm64 platforms, so I'm a little surprised that of all the
> > > > > > > PCI host drivers used on arm64, only pcie-apple.c and pci-imx6.c need
> > > > > > > this notifier.
> > > > > >
> > > > > > This is one of those things that's mostly at the mercy of the PCIe root
> > > > > > complex implementation. Typically the SMMU StreamID and/or GIC ITS DeviceID
> > > > > > is derived directly from the PCI RID, sometimes with additional high-order
> > > > > > bits hard-wired to disambiguate PCI segments. I believe this RID-translation
> > > > > > LUT is a particular feature of the the Synopsys IP - I know there's also one
> > > > > > on the NXP Layerscape platforms, but on those it's programmed by the
> > > > > > bootloader, which also generates the appropriate "msi-map" and "iommu-map"
> > > > > > properties to match. Ideally that's what i.MX should do as well, but hey.
> > > > >
> > > > > Maybe this RID-translation is a feature of i.MX, not of Synopsys? I
> > > > > see that the LUT CSR accesses use IMX95_* definitions.
> > > >
> > > > Yes, it convert 16bit RID to 6bit stream id.
> > >
> > > IIUC, you're saying this is not a Synopsys feature, it's an i.MX
> > > feature.
> >
> > Yes, it is i.MX feature. But I think other vendor should have similar
> > situation if use old arm smmu.
> >
> > >
> > > > > > If it's really necessary to do this programming from Linux, then there's
> > > > > > still no point in it being dynamic - the mappings cannot ever change, since
> > > > > > the rest of the kernel believes that what the DT said at boot time was
> > > > > > already a property of the hardware. It would be a lot more logical, and
> > > > > > likely simpler, for the driver to just read the relevant map property and
> > > > > > program the entire LUT to match, all in one go at controller probe time.
> > > > > > Rather like what's already commonly done with the parsing of "dma-ranges" to
> > > > > > program address-translation LUTs for inbound windows.
> > > > > >
> > > > > > Plus that would also give a chance of safely dealing with bad DTs specifying
> > > > > > invalid ID mappings (by refusing to probe at all). As it is, returning an
> > > > > > error from a child's BUS_NOTIFY_ADD_DEVICE does nothing except prevent any
> > > > > > further notifiers from running at that point - the device will still be
> > > > > > added, allowed to bind a driver, and able to start sending DMA/MSI traffic
> > > > > > without the controller being correctly programmed, which at best won't work
> > > > > > and at worst may break the whole system.
> > > > >
> > > > > Frank, could the imx LUT be programmed once at boot-time instead of at
> > > > > device-add time? I'm guessing maybe not because apparently there is a
> > > > > risk of running out of LUT entries?
> > > >
> > > > It is not good idea to depend on boot loader so much.
> > >
> > > I meant "could this be programmed once when the Linux imx host
> > > controller driver is probed?" But from the below, it sounds like
> > > that's not possible in general because you don't have enough stream
> > > IDs to do that.
> >
> > Oh! sorry miss understand what your means. It is possible like what I did
> > at v3 version. But I think it is not good enough.
> >
> > >
> > > > Some hot plug devics
> > > > (SD7.0) may plug after system boot. Two PCIe instances shared one set
> > > > of 6bits stream id (total 64). Assume total 16 assign to two PCIe
> > > > controllers. each have 8 stream id. If use uboot assign it static, each
> > > > PCIe controller have below 8 devices. It will be failrue one controller
> > > > connect 7, another connect 9. but if dynamtic alloc when devices add, both
> > > > controller can work.
> > > >
> > > > Although we have not so much devices now, this way give us possility to
> > > > improve it in future.
> > > >
> > > > > It sounds like the consequences of running out of LUT entries are
> > > > > catastrophic, e.g., memory corruption from mis-directed DMA? If
> > > > > that's possible, I think we need to figure out how to prevent the
> > > > > device from being used, not just dev_warn() about it.
> > > >
> > > > Yes, but so far, we have not met such problem now. We can improve it when
> > > > we really face such problem.
> > >
> > > If this controller can only support DMA from a limited number of
> > > endpoints below it, I think we should figure out how to enforce that
> > > directly. Maybe we can prevent drivers from enabling bus mastering or
> > > something. I'm not happy with the idea of waiting for and debugging a
> > > report of data corruption.
> >
> > It may add a pre-add hook function to pci bridge. let me do more research.
>
> Hi Bjorn:
>
> int pci_setup_device(struct pci_dev *dev)
> {
> dev->error_state = pci_channel_io_normal;
> ...
> pci_fixup_device(pci_fixup_early, dev);
>
> ^^^ I can add fixup hook for pci_fixup_early. If not resource,
> I can set dev->error_state to pci_channel_io_frozen or
> pci_channel_io_perm_failure
>
> And add below check here after call hook function.
>
> if (dev->error_state != pci_channel_io_normal)
> return -EIO;
>
> }
>
> How do you think this method? If you agree, I can continue search device
> remove hook up.
I think this would mean the device would not appear to be enumerated
at all, right? I.e., it wouldn't show up in lspci? And we couldn't
use even a pure programmed IO driver with no DMA or MSI?
I wonder if we should have a function pointer in struct
pci_host_bridge, kind of like the existing ->map_irq(), where we could
do host bridge-specific setup when enumerating a PCI device.
We'd still have to solve the issue of preventing DMA, but a hook like
that might avoid the need for a quirk or the bus notifier approach.
Bjorn
^ permalink raw reply [flat|nested] 44+ messages in thread* Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-06-13 22:41 ` Bjorn Helgaas
@ 2024-06-17 14:26 ` Frank Li
2024-06-21 22:29 ` Frank Li
0 siblings, 1 reply; 44+ messages in thread
From: Frank Li @ 2024-06-17 14:26 UTC (permalink / raw)
To: Bjorn Helgaas
Cc: Robin Murphy, Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley,
linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Will Deacon, Joerg Roedel, Jason Gunthorpe, Alyssa Rosenzweig,
Marc Zyngier
On Thu, Jun 13, 2024 at 05:41:25PM -0500, Bjorn Helgaas wrote:
> On Thu, Jun 06, 2024 at 04:24:17PM -0400, Frank Li wrote:
> > On Mon, Jun 03, 2024 at 04:07:55PM -0400, Frank Li wrote:
> > > On Mon, Jun 03, 2024 at 01:56:27PM -0500, Bjorn Helgaas wrote:
> > > > On Mon, Jun 03, 2024 at 02:42:45PM -0400, Frank Li wrote:
> > > > > On Mon, Jun 03, 2024 at 12:19:21PM -0500, Bjorn Helgaas wrote:
> > > > > > On Fri, May 31, 2024 at 03:58:49PM +0100, Robin Murphy wrote:
> > > > > > > On 2024-05-31 12:08 am, Bjorn Helgaas wrote:
> > > > > > > > [+cc IOMMU and pcie-apple.c folks for comment]
> > > > > > > >
> > > > > > > > On Tue, May 28, 2024 at 03:39:21PM -0400, Frank Li wrote:
> > > > > > > > > For the i.MX95, configuration of a LUT is necessary to convert Bus Device
> > > > > > > > > Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS.
> > > > > > > > > This involves examining the msi-map and smmu-map to ensure consistent
> > > > > > > > > mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related
> > > > > > > > > registers are configured. In the absence of an msi-map, the built-in MSI
> > > > > > > > > controller is utilized as a fallback.
> > > > > > > > >
> > > > > > > > > Additionally, register a PCI bus notifier to trigger imx_pcie_add_device()
> > > > > > > > > upon the appearance of a new PCI device and when the bus is an iMX6 PCI
> > > > > > > > > controller. This function configures the correct LUT based on Device Tree
> > > > > > > > > Settings (DTS).
> > > > > > > >
> > > > > > > > This scheme is pretty similar to apple_pcie_bus_notifier(). If we
> > > > > > > > have to do this, I wish it were *more* similar, i.e., copy the
> > > > > > > > function names, bitmap tracking, code structure, etc.
> > > > > > > >
> > > > > > > > I don't really know how stream IDs work, but I assume they are used on
> > > > > > > > most or all arm64 platforms, so I'm a little surprised that of all the
> > > > > > > > PCI host drivers used on arm64, only pcie-apple.c and pci-imx6.c need
> > > > > > > > this notifier.
> > > > > > >
> > > > > > > This is one of those things that's mostly at the mercy of the PCIe root
> > > > > > > complex implementation. Typically the SMMU StreamID and/or GIC ITS DeviceID
> > > > > > > is derived directly from the PCI RID, sometimes with additional high-order
> > > > > > > bits hard-wired to disambiguate PCI segments. I believe this RID-translation
> > > > > > > LUT is a particular feature of the the Synopsys IP - I know there's also one
> > > > > > > on the NXP Layerscape platforms, but on those it's programmed by the
> > > > > > > bootloader, which also generates the appropriate "msi-map" and "iommu-map"
> > > > > > > properties to match. Ideally that's what i.MX should do as well, but hey.
> > > > > >
> > > > > > Maybe this RID-translation is a feature of i.MX, not of Synopsys? I
> > > > > > see that the LUT CSR accesses use IMX95_* definitions.
> > > > >
> > > > > Yes, it convert 16bit RID to 6bit stream id.
> > > >
> > > > IIUC, you're saying this is not a Synopsys feature, it's an i.MX
> > > > feature.
> > >
> > > Yes, it is i.MX feature. But I think other vendor should have similar
> > > situation if use old arm smmu.
> > >
> > > >
> > > > > > > If it's really necessary to do this programming from Linux, then there's
> > > > > > > still no point in it being dynamic - the mappings cannot ever change, since
> > > > > > > the rest of the kernel believes that what the DT said at boot time was
> > > > > > > already a property of the hardware. It would be a lot more logical, and
> > > > > > > likely simpler, for the driver to just read the relevant map property and
> > > > > > > program the entire LUT to match, all in one go at controller probe time.
> > > > > > > Rather like what's already commonly done with the parsing of "dma-ranges" to
> > > > > > > program address-translation LUTs for inbound windows.
> > > > > > >
> > > > > > > Plus that would also give a chance of safely dealing with bad DTs specifying
> > > > > > > invalid ID mappings (by refusing to probe at all). As it is, returning an
> > > > > > > error from a child's BUS_NOTIFY_ADD_DEVICE does nothing except prevent any
> > > > > > > further notifiers from running at that point - the device will still be
> > > > > > > added, allowed to bind a driver, and able to start sending DMA/MSI traffic
> > > > > > > without the controller being correctly programmed, which at best won't work
> > > > > > > and at worst may break the whole system.
> > > > > >
> > > > > > Frank, could the imx LUT be programmed once at boot-time instead of at
> > > > > > device-add time? I'm guessing maybe not because apparently there is a
> > > > > > risk of running out of LUT entries?
> > > > >
> > > > > It is not good idea to depend on boot loader so much.
> > > >
> > > > I meant "could this be programmed once when the Linux imx host
> > > > controller driver is probed?" But from the below, it sounds like
> > > > that's not possible in general because you don't have enough stream
> > > > IDs to do that.
> > >
> > > Oh! sorry miss understand what your means. It is possible like what I did
> > > at v3 version. But I think it is not good enough.
> > >
> > > >
> > > > > Some hot plug devics
> > > > > (SD7.0) may plug after system boot. Two PCIe instances shared one set
> > > > > of 6bits stream id (total 64). Assume total 16 assign to two PCIe
> > > > > controllers. each have 8 stream id. If use uboot assign it static, each
> > > > > PCIe controller have below 8 devices. It will be failrue one controller
> > > > > connect 7, another connect 9. but if dynamtic alloc when devices add, both
> > > > > controller can work.
> > > > >
> > > > > Although we have not so much devices now, this way give us possility to
> > > > > improve it in future.
> > > > >
> > > > > > It sounds like the consequences of running out of LUT entries are
> > > > > > catastrophic, e.g., memory corruption from mis-directed DMA? If
> > > > > > that's possible, I think we need to figure out how to prevent the
> > > > > > device from being used, not just dev_warn() about it.
> > > > >
> > > > > Yes, but so far, we have not met such problem now. We can improve it when
> > > > > we really face such problem.
> > > >
> > > > If this controller can only support DMA from a limited number of
> > > > endpoints below it, I think we should figure out how to enforce that
> > > > directly. Maybe we can prevent drivers from enabling bus mastering or
> > > > something. I'm not happy with the idea of waiting for and debugging a
> > > > report of data corruption.
> > >
> > > It may add a pre-add hook function to pci bridge. let me do more research.
> >
> > Hi Bjorn:
> >
> > int pci_setup_device(struct pci_dev *dev)
> > {
> > dev->error_state = pci_channel_io_normal;
> > ...
> > pci_fixup_device(pci_fixup_early, dev);
> >
> > ^^^ I can add fixup hook for pci_fixup_early. If not resource,
> > I can set dev->error_state to pci_channel_io_frozen or
> > pci_channel_io_perm_failure
> >
> > And add below check here after call hook function.
> >
> > if (dev->error_state != pci_channel_io_normal)
> > return -EIO;
> >
> > }
> >
> > How do you think this method? If you agree, I can continue search device
> > remove hook up.
>
> I think this would mean the device would not appear to be enumerated
> at all, right? I.e., it wouldn't show up in lspci? And we couldn't
> use even a pure programmed IO driver with no DMA or MSI?
Make sense. Let me do more research on this.
Frank
>
> I wonder if we should have a function pointer in struct
> pci_host_bridge, kind of like the existing ->map_irq(), where we could
> do host bridge-specific setup when enumerating a PCI device.
>
> We'd still have to solve the issue of preventing DMA, but a hook like
> that might avoid the need for a quirk or the bus notifier approach.
>
> Bjorn
^ permalink raw reply [flat|nested] 44+ messages in thread* Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-06-17 14:26 ` Frank Li
@ 2024-06-21 22:29 ` Frank Li
2024-06-21 22:43 ` Bjorn Helgaas
0 siblings, 1 reply; 44+ messages in thread
From: Frank Li @ 2024-06-21 22:29 UTC (permalink / raw)
To: Bjorn Helgaas
Cc: Robin Murphy, Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley,
linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Will Deacon, Joerg Roedel, Jason Gunthorpe, Alyssa Rosenzweig,
Marc Zyngier
On Mon, Jun 17, 2024 at 10:26:36AM -0400, Frank Li wrote:
> On Thu, Jun 13, 2024 at 05:41:25PM -0500, Bjorn Helgaas wrote:
> > On Thu, Jun 06, 2024 at 04:24:17PM -0400, Frank Li wrote:
> > > On Mon, Jun 03, 2024 at 04:07:55PM -0400, Frank Li wrote:
> > > > On Mon, Jun 03, 2024 at 01:56:27PM -0500, Bjorn Helgaas wrote:
> > > > > On Mon, Jun 03, 2024 at 02:42:45PM -0400, Frank Li wrote:
> > > > > > On Mon, Jun 03, 2024 at 12:19:21PM -0500, Bjorn Helgaas wrote:
> > > > > > > On Fri, May 31, 2024 at 03:58:49PM +0100, Robin Murphy wrote:
> > > > > > > > On 2024-05-31 12:08 am, Bjorn Helgaas wrote:
> > > > > > > > > [+cc IOMMU and pcie-apple.c folks for comment]
> > > > > > > > >
> > > > > > > > > On Tue, May 28, 2024 at 03:39:21PM -0400, Frank Li wrote:
> > > > > > > > > > For the i.MX95, configuration of a LUT is necessary to convert Bus Device
> > > > > > > > > > Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS.
> > > > > > > > > > This involves examining the msi-map and smmu-map to ensure consistent
> > > > > > > > > > mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related
> > > > > > > > > > registers are configured. In the absence of an msi-map, the built-in MSI
> > > > > > > > > > controller is utilized as a fallback.
> > > > > > > > > >
> > > > > > > > > > Additionally, register a PCI bus notifier to trigger imx_pcie_add_device()
> > > > > > > > > > upon the appearance of a new PCI device and when the bus is an iMX6 PCI
> > > > > > > > > > controller. This function configures the correct LUT based on Device Tree
> > > > > > > > > > Settings (DTS).
> > > > > > > > >
> > > > > > > > > This scheme is pretty similar to apple_pcie_bus_notifier(). If we
> > > > > > > > > have to do this, I wish it were *more* similar, i.e., copy the
> > > > > > > > > function names, bitmap tracking, code structure, etc.
> > > > > > > > >
> > > > > > > > > I don't really know how stream IDs work, but I assume they are used on
> > > > > > > > > most or all arm64 platforms, so I'm a little surprised that of all the
> > > > > > > > > PCI host drivers used on arm64, only pcie-apple.c and pci-imx6.c need
> > > > > > > > > this notifier.
> > > > > > > >
> > > > > > > > This is one of those things that's mostly at the mercy of the PCIe root
> > > > > > > > complex implementation. Typically the SMMU StreamID and/or GIC ITS DeviceID
> > > > > > > > is derived directly from the PCI RID, sometimes with additional high-order
> > > > > > > > bits hard-wired to disambiguate PCI segments. I believe this RID-translation
> > > > > > > > LUT is a particular feature of the the Synopsys IP - I know there's also one
> > > > > > > > on the NXP Layerscape platforms, but on those it's programmed by the
> > > > > > > > bootloader, which also generates the appropriate "msi-map" and "iommu-map"
> > > > > > > > properties to match. Ideally that's what i.MX should do as well, but hey.
> > > > > > >
> > > > > > > Maybe this RID-translation is a feature of i.MX, not of Synopsys? I
> > > > > > > see that the LUT CSR accesses use IMX95_* definitions.
> > > > > >
> > > > > > Yes, it convert 16bit RID to 6bit stream id.
> > > > >
> > > > > IIUC, you're saying this is not a Synopsys feature, it's an i.MX
> > > > > feature.
> > > >
> > > > Yes, it is i.MX feature. But I think other vendor should have similar
> > > > situation if use old arm smmu.
> > > >
> > > > >
> > > > > > > > If it's really necessary to do this programming from Linux, then there's
> > > > > > > > still no point in it being dynamic - the mappings cannot ever change, since
> > > > > > > > the rest of the kernel believes that what the DT said at boot time was
> > > > > > > > already a property of the hardware. It would be a lot more logical, and
> > > > > > > > likely simpler, for the driver to just read the relevant map property and
> > > > > > > > program the entire LUT to match, all in one go at controller probe time.
> > > > > > > > Rather like what's already commonly done with the parsing of "dma-ranges" to
> > > > > > > > program address-translation LUTs for inbound windows.
> > > > > > > >
> > > > > > > > Plus that would also give a chance of safely dealing with bad DTs specifying
> > > > > > > > invalid ID mappings (by refusing to probe at all). As it is, returning an
> > > > > > > > error from a child's BUS_NOTIFY_ADD_DEVICE does nothing except prevent any
> > > > > > > > further notifiers from running at that point - the device will still be
> > > > > > > > added, allowed to bind a driver, and able to start sending DMA/MSI traffic
> > > > > > > > without the controller being correctly programmed, which at best won't work
> > > > > > > > and at worst may break the whole system.
> > > > > > >
> > > > > > > Frank, could the imx LUT be programmed once at boot-time instead of at
> > > > > > > device-add time? I'm guessing maybe not because apparently there is a
> > > > > > > risk of running out of LUT entries?
> > > > > >
> > > > > > It is not good idea to depend on boot loader so much.
> > > > >
> > > > > I meant "could this be programmed once when the Linux imx host
> > > > > controller driver is probed?" But from the below, it sounds like
> > > > > that's not possible in general because you don't have enough stream
> > > > > IDs to do that.
> > > >
> > > > Oh! sorry miss understand what your means. It is possible like what I did
> > > > at v3 version. But I think it is not good enough.
> > > >
> > > > >
> > > > > > Some hot plug devics
> > > > > > (SD7.0) may plug after system boot. Two PCIe instances shared one set
> > > > > > of 6bits stream id (total 64). Assume total 16 assign to two PCIe
> > > > > > controllers. each have 8 stream id. If use uboot assign it static, each
> > > > > > PCIe controller have below 8 devices. It will be failrue one controller
> > > > > > connect 7, another connect 9. but if dynamtic alloc when devices add, both
> > > > > > controller can work.
> > > > > >
> > > > > > Although we have not so much devices now, this way give us possility to
> > > > > > improve it in future.
> > > > > >
> > > > > > > It sounds like the consequences of running out of LUT entries are
> > > > > > > catastrophic, e.g., memory corruption from mis-directed DMA? If
> > > > > > > that's possible, I think we need to figure out how to prevent the
> > > > > > > device from being used, not just dev_warn() about it.
> > > > > >
> > > > > > Yes, but so far, we have not met such problem now. We can improve it when
> > > > > > we really face such problem.
> > > > >
> > > > > If this controller can only support DMA from a limited number of
> > > > > endpoints below it, I think we should figure out how to enforce that
> > > > > directly. Maybe we can prevent drivers from enabling bus mastering or
> > > > > something. I'm not happy with the idea of waiting for and debugging a
> > > > > report of data corruption.
> > > >
> > > > It may add a pre-add hook function to pci bridge. let me do more research.
> > >
> > > Hi Bjorn:
> > >
> > > int pci_setup_device(struct pci_dev *dev)
> > > {
> > > dev->error_state = pci_channel_io_normal;
> > > ...
> > > pci_fixup_device(pci_fixup_early, dev);
> > >
> > > ^^^ I can add fixup hook for pci_fixup_early. If not resource,
> > > I can set dev->error_state to pci_channel_io_frozen or
> > > pci_channel_io_perm_failure
> > >
> > > And add below check here after call hook function.
> > >
> > > if (dev->error_state != pci_channel_io_normal)
> > > return -EIO;
> > >
> > > }
> > >
> > > How do you think this method? If you agree, I can continue search device
> > > remove hook up.
> >
> > I think this would mean the device would not appear to be enumerated
> > at all, right? I.e., it wouldn't show up in lspci? And we couldn't
> > use even a pure programmed IO driver with no DMA or MSI?
>
> Make sense. Let me do more research on this.
>
> Frank
> >
> > I wonder if we should have a function pointer in struct
> > pci_host_bridge, kind of like the existing ->map_irq(), where we could
> > do host bridge-specific setup when enumerating a PCI device.
Consider some device may no use MSI or DMA. It'd better set LUT when
allocate msi irq. I think insert a irq-domain in irq hierarchy.
static const struct irq_domain_ops lut_pcie_msi_domain_ops = {
.alloc = lut_pcie_irq_domain_alloc,
.free = lut_pcie_irq_domain_free,
};
int dw_pcie_allocate_domains(struct dw_pcie_rp *pp)
{
struct fwnode_handle *fwnode = of_node_to_fwnode(pci->dev->of_node);
pp->irq_domain = irq_domain_create_hierarchy(...)
pp->msi_domain = pci_msi_create_irq_domain(...);
return 0;
}
Manage lut stream id in lut_pcie_irq_domain_alloc() and
lut_pcie_irq_domain_free().
So failure happen only when driver use MSI and no-stream ID avaiable. It
should be better than failure when add devices. Some devices may not use
at all.
How do you think this method? If it is okay, I can try research how to
set LUT when do dma_mapping(unmap).
Frank Li
> >
> > We'd still have to solve the issue of preventing DMA, but a hook like
> > that might avoid the need for a quirk or the bus notifier approach.
> >
> > Bjorn
^ permalink raw reply [flat|nested] 44+ messages in thread* Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-06-21 22:29 ` Frank Li
@ 2024-06-21 22:43 ` Bjorn Helgaas
2024-06-22 17:38 ` Bjorn Helgaas
0 siblings, 1 reply; 44+ messages in thread
From: Bjorn Helgaas @ 2024-06-21 22:43 UTC (permalink / raw)
To: Frank Li
Cc: Robin Murphy, Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley,
linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Will Deacon, Joerg Roedel, Jason Gunthorpe, Alyssa Rosenzweig,
Marc Zyngier
On Fri, Jun 21, 2024 at 06:29:48PM -0400, Frank Li wrote:
> On Mon, Jun 17, 2024 at 10:26:36AM -0400, Frank Li wrote:
> > On Thu, Jun 13, 2024 at 05:41:25PM -0500, Bjorn Helgaas wrote:
> > > On Thu, Jun 06, 2024 at 04:24:17PM -0400, Frank Li wrote:
> > > > On Mon, Jun 03, 2024 at 04:07:55PM -0400, Frank Li wrote:
> > > > > On Mon, Jun 03, 2024 at 01:56:27PM -0500, Bjorn Helgaas wrote:
> > > > > > On Mon, Jun 03, 2024 at 02:42:45PM -0400, Frank Li wrote:
> > > > > > > On Mon, Jun 03, 2024 at 12:19:21PM -0500, Bjorn Helgaas wrote:
> > > > > > > > On Fri, May 31, 2024 at 03:58:49PM +0100, Robin Murphy wrote:
> > > > > > > > > On 2024-05-31 12:08 am, Bjorn Helgaas wrote:
> > > > > > > > > > [+cc IOMMU and pcie-apple.c folks for comment]
> > > > > > > > > >
> > > > > > > > > > On Tue, May 28, 2024 at 03:39:21PM -0400, Frank Li wrote:
> > > > > > > > > > > For the i.MX95, configuration of a LUT is necessary to convert Bus Device
> > > > > > > > > > > Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS.
> > > > > > > > > > > This involves examining the msi-map and smmu-map to ensure consistent
> > > > > > > > > > > mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related
> > > > > > > > > > > registers are configured. In the absence of an msi-map, the built-in MSI
> > > > > > > > > > > controller is utilized as a fallback.
> > > > > > > > > > >
> > > > > > > > > > > Additionally, register a PCI bus notifier to trigger imx_pcie_add_device()
> > > > > > > > > > > upon the appearance of a new PCI device and when the bus is an iMX6 PCI
> > > > > > > > > > > controller. This function configures the correct LUT based on Device Tree
> > > > > > > > > > > Settings (DTS).
> > > > > > > > > >
> > > > > > > > > > This scheme is pretty similar to apple_pcie_bus_notifier(). If we
> > > > > > > > > > have to do this, I wish it were *more* similar, i.e., copy the
> > > > > > > > > > function names, bitmap tracking, code structure, etc.
> > > > > > > > > >
> > > > > > > > > > I don't really know how stream IDs work, but I assume they are used on
> > > > > > > > > > most or all arm64 platforms, so I'm a little surprised that of all the
> > > > > > > > > > PCI host drivers used on arm64, only pcie-apple.c and pci-imx6.c need
> > > > > > > > > > this notifier.
> > > > > > > > >
> > > > > > > > > This is one of those things that's mostly at the mercy of the PCIe root
> > > > > > > > > complex implementation. Typically the SMMU StreamID and/or GIC ITS DeviceID
> > > > > > > > > is derived directly from the PCI RID, sometimes with additional high-order
> > > > > > > > > bits hard-wired to disambiguate PCI segments. I believe this RID-translation
> > > > > > > > > LUT is a particular feature of the the Synopsys IP - I know there's also one
> > > > > > > > > on the NXP Layerscape platforms, but on those it's programmed by the
> > > > > > > > > bootloader, which also generates the appropriate "msi-map" and "iommu-map"
> > > > > > > > > properties to match. Ideally that's what i.MX should do as well, but hey.
> > > > > > > >
> > > > > > > > Maybe this RID-translation is a feature of i.MX, not of Synopsys? I
> > > > > > > > see that the LUT CSR accesses use IMX95_* definitions.
> > > > > > >
> > > > > > > Yes, it convert 16bit RID to 6bit stream id.
> > > > > >
> > > > > > IIUC, you're saying this is not a Synopsys feature, it's an i.MX
> > > > > > feature.
> > > > >
> > > > > Yes, it is i.MX feature. But I think other vendor should have similar
> > > > > situation if use old arm smmu.
> > > > >
> > > > > >
> > > > > > > > > If it's really necessary to do this programming from Linux, then there's
> > > > > > > > > still no point in it being dynamic - the mappings cannot ever change, since
> > > > > > > > > the rest of the kernel believes that what the DT said at boot time was
> > > > > > > > > already a property of the hardware. It would be a lot more logical, and
> > > > > > > > > likely simpler, for the driver to just read the relevant map property and
> > > > > > > > > program the entire LUT to match, all in one go at controller probe time.
> > > > > > > > > Rather like what's already commonly done with the parsing of "dma-ranges" to
> > > > > > > > > program address-translation LUTs for inbound windows.
> > > > > > > > >
> > > > > > > > > Plus that would also give a chance of safely dealing with bad DTs specifying
> > > > > > > > > invalid ID mappings (by refusing to probe at all). As it is, returning an
> > > > > > > > > error from a child's BUS_NOTIFY_ADD_DEVICE does nothing except prevent any
> > > > > > > > > further notifiers from running at that point - the device will still be
> > > > > > > > > added, allowed to bind a driver, and able to start sending DMA/MSI traffic
> > > > > > > > > without the controller being correctly programmed, which at best won't work
> > > > > > > > > and at worst may break the whole system.
> > > > > > > >
> > > > > > > > Frank, could the imx LUT be programmed once at boot-time instead of at
> > > > > > > > device-add time? I'm guessing maybe not because apparently there is a
> > > > > > > > risk of running out of LUT entries?
> > > > > > >
> > > > > > > It is not good idea to depend on boot loader so much.
> > > > > >
> > > > > > I meant "could this be programmed once when the Linux imx host
> > > > > > controller driver is probed?" But from the below, it sounds like
> > > > > > that's not possible in general because you don't have enough stream
> > > > > > IDs to do that.
> > > > >
> > > > > Oh! sorry miss understand what your means. It is possible like what I did
> > > > > at v3 version. But I think it is not good enough.
> > > > >
> > > > > >
> > > > > > > Some hot plug devics
> > > > > > > (SD7.0) may plug after system boot. Two PCIe instances shared one set
> > > > > > > of 6bits stream id (total 64). Assume total 16 assign to two PCIe
> > > > > > > controllers. each have 8 stream id. If use uboot assign it static, each
> > > > > > > PCIe controller have below 8 devices. It will be failrue one controller
> > > > > > > connect 7, another connect 9. but if dynamtic alloc when devices add, both
> > > > > > > controller can work.
> > > > > > >
> > > > > > > Although we have not so much devices now, this way give us possility to
> > > > > > > improve it in future.
> > > > > > >
> > > > > > > > It sounds like the consequences of running out of LUT entries are
> > > > > > > > catastrophic, e.g., memory corruption from mis-directed DMA? If
> > > > > > > > that's possible, I think we need to figure out how to prevent the
> > > > > > > > device from being used, not just dev_warn() about it.
> > > > > > >
> > > > > > > Yes, but so far, we have not met such problem now. We can improve it when
> > > > > > > we really face such problem.
> > > > > >
> > > > > > If this controller can only support DMA from a limited number of
> > > > > > endpoints below it, I think we should figure out how to enforce that
> > > > > > directly. Maybe we can prevent drivers from enabling bus mastering or
> > > > > > something. I'm not happy with the idea of waiting for and debugging a
> > > > > > report of data corruption.
> > > > >
> > > > > It may add a pre-add hook function to pci bridge. let me do more research.
> > > >
> > > > Hi Bjorn:
> > > >
> > > > int pci_setup_device(struct pci_dev *dev)
> > > > {
> > > > dev->error_state = pci_channel_io_normal;
> > > > ...
> > > > pci_fixup_device(pci_fixup_early, dev);
> > > >
> > > > ^^^ I can add fixup hook for pci_fixup_early. If not resource,
> > > > I can set dev->error_state to pci_channel_io_frozen or
> > > > pci_channel_io_perm_failure
> > > >
> > > > And add below check here after call hook function.
> > > >
> > > > if (dev->error_state != pci_channel_io_normal)
> > > > return -EIO;
> > > >
> > > > }
> > > >
> > > > How do you think this method? If you agree, I can continue search device
> > > > remove hook up.
> > >
> > > I think this would mean the device would not appear to be enumerated
> > > at all, right? I.e., it wouldn't show up in lspci? And we couldn't
> > > use even a pure programmed IO driver with no DMA or MSI?
> >
> > Make sense. Let me do more research on this.
> >
> > Frank
> > >
> > > I wonder if we should have a function pointer in struct
> > > pci_host_bridge, kind of like the existing ->map_irq(), where we could
> > > do host bridge-specific setup when enumerating a PCI device.
>
> Consider some device may no use MSI or DMA. It'd better set LUT when
> allocate msi irq. I think insert a irq-domain in irq hierarchy.
>
> static const struct irq_domain_ops lut_pcie_msi_domain_ops = {
> .alloc = lut_pcie_irq_domain_alloc,
> .free = lut_pcie_irq_domain_free,
> };
>
> int dw_pcie_allocate_domains(struct dw_pcie_rp *pp)
> {
> struct fwnode_handle *fwnode = of_node_to_fwnode(pci->dev->of_node);
>
> pp->irq_domain = irq_domain_create_hierarchy(...)
>
> pp->msi_domain = pci_msi_create_irq_domain(...);
>
> return 0;
> }
>
> Manage lut stream id in lut_pcie_irq_domain_alloc() and
> lut_pcie_irq_domain_free().
>
> So failure happen only when driver use MSI and no-stream ID avaiable. It
> should be better than failure when add devices. Some devices may not use
> at all.
I'm not an IRQ expert, but it sounds plausible. There might even be
an opportunity to fall back to INTx if there's no stream ID available
for MSI?
^ permalink raw reply [flat|nested] 44+ messages in thread* Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-06-21 22:43 ` Bjorn Helgaas
@ 2024-06-22 17:38 ` Bjorn Helgaas
2024-06-24 15:00 ` Frank Li
0 siblings, 1 reply; 44+ messages in thread
From: Bjorn Helgaas @ 2024-06-22 17:38 UTC (permalink / raw)
To: Frank Li
Cc: Robin Murphy, Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley,
linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Will Deacon, Joerg Roedel, Jason Gunthorpe, Alyssa Rosenzweig,
Marc Zyngier
On Fri, Jun 21, 2024 at 05:43:21PM -0500, Bjorn Helgaas wrote:
> On Fri, Jun 21, 2024 at 06:29:48PM -0400, Frank Li wrote:
> > On Mon, Jun 17, 2024 at 10:26:36AM -0400, Frank Li wrote:
> > > On Thu, Jun 13, 2024 at 05:41:25PM -0500, Bjorn Helgaas wrote:
> > > > On Thu, Jun 06, 2024 at 04:24:17PM -0400, Frank Li wrote:
> > > > > On Mon, Jun 03, 2024 at 04:07:55PM -0400, Frank Li wrote:
> > > > > > On Mon, Jun 03, 2024 at 01:56:27PM -0500, Bjorn Helgaas wrote:
> > > > > > > On Mon, Jun 03, 2024 at 02:42:45PM -0400, Frank Li wrote:
> > > > > > > > On Mon, Jun 03, 2024 at 12:19:21PM -0500, Bjorn Helgaas wrote:
> > > > > > > > > On Fri, May 31, 2024 at 03:58:49PM +0100, Robin Murphy wrote:
> > > > > > > > > > On 2024-05-31 12:08 am, Bjorn Helgaas wrote:
> > > > > > > > > > > [+cc IOMMU and pcie-apple.c folks for comment]
> > > > > > > > > > >
> > > > > > > > > > > On Tue, May 28, 2024 at 03:39:21PM -0400, Frank Li wrote:
> > > > > > > > > > > > For the i.MX95, configuration of a LUT is necessary to convert Bus Device
> > > > > > > > > > > > Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS.
> > > > > > > > > > > > This involves examining the msi-map and smmu-map to ensure consistent
> > > > > > > > > > > > mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related
> > > > > > > > > > > > registers are configured. In the absence of an msi-map, the built-in MSI
> > > > > > > > > > > > controller is utilized as a fallback.
> > > > > > > > > > > >
> > > > > > > > > > > > Additionally, register a PCI bus notifier to trigger imx_pcie_add_device()
> > > > > > > > > > > > upon the appearance of a new PCI device and when the bus is an iMX6 PCI
> > > > > > > > > > > > controller. This function configures the correct LUT based on Device Tree
> > > > > > > > > > > > Settings (DTS).
> > > > > > > > > > >
> > > > > > > > > > > This scheme is pretty similar to apple_pcie_bus_notifier(). If we
> > > > > > > > > > > have to do this, I wish it were *more* similar, i.e., copy the
> > > > > > > > > > > function names, bitmap tracking, code structure, etc.
> > > > > > > > > > >
> > > > > > > > > > > I don't really know how stream IDs work, but I assume they are used on
> > > > > > > > > > > most or all arm64 platforms, so I'm a little surprised that of all the
> > > > > > > > > > > PCI host drivers used on arm64, only pcie-apple.c and pci-imx6.c need
> > > > > > > > > > > this notifier.
> > > > > > > > > >
> > > > > > > > > > This is one of those things that's mostly at the mercy of the PCIe root
> > > > > > > > > > complex implementation. Typically the SMMU StreamID and/or GIC ITS DeviceID
> > > > > > > > > > is derived directly from the PCI RID, sometimes with additional high-order
> > > > > > > > > > bits hard-wired to disambiguate PCI segments. I believe this RID-translation
> > > > > > > > > > LUT is a particular feature of the the Synopsys IP - I know there's also one
> > > > > > > > > > on the NXP Layerscape platforms, but on those it's programmed by the
> > > > > > > > > > bootloader, which also generates the appropriate "msi-map" and "iommu-map"
> > > > > > > > > > properties to match. Ideally that's what i.MX should do as well, but hey.
> > > > > > > > >
> > > > > > > > > Maybe this RID-translation is a feature of i.MX, not of Synopsys? I
> > > > > > > > > see that the LUT CSR accesses use IMX95_* definitions.
> > > > > > > >
> > > > > > > > Yes, it convert 16bit RID to 6bit stream id.
> > > > > > >
> > > > > > > IIUC, you're saying this is not a Synopsys feature, it's an i.MX
> > > > > > > feature.
> > > > > >
> > > > > > Yes, it is i.MX feature. But I think other vendor should have similar
> > > > > > situation if use old arm smmu.
> > > > > >
> > > > > > >
> > > > > > > > > > If it's really necessary to do this programming from Linux, then there's
> > > > > > > > > > still no point in it being dynamic - the mappings cannot ever change, since
> > > > > > > > > > the rest of the kernel believes that what the DT said at boot time was
> > > > > > > > > > already a property of the hardware. It would be a lot more logical, and
> > > > > > > > > > likely simpler, for the driver to just read the relevant map property and
> > > > > > > > > > program the entire LUT to match, all in one go at controller probe time.
> > > > > > > > > > Rather like what's already commonly done with the parsing of "dma-ranges" to
> > > > > > > > > > program address-translation LUTs for inbound windows.
> > > > > > > > > >
> > > > > > > > > > Plus that would also give a chance of safely dealing with bad DTs specifying
> > > > > > > > > > invalid ID mappings (by refusing to probe at all). As it is, returning an
> > > > > > > > > > error from a child's BUS_NOTIFY_ADD_DEVICE does nothing except prevent any
> > > > > > > > > > further notifiers from running at that point - the device will still be
> > > > > > > > > > added, allowed to bind a driver, and able to start sending DMA/MSI traffic
> > > > > > > > > > without the controller being correctly programmed, which at best won't work
> > > > > > > > > > and at worst may break the whole system.
> > > > > > > > >
> > > > > > > > > Frank, could the imx LUT be programmed once at boot-time instead of at
> > > > > > > > > device-add time? I'm guessing maybe not because apparently there is a
> > > > > > > > > risk of running out of LUT entries?
> > > > > > > >
> > > > > > > > It is not good idea to depend on boot loader so much.
> > > > > > >
> > > > > > > I meant "could this be programmed once when the Linux imx host
> > > > > > > controller driver is probed?" But from the below, it sounds like
> > > > > > > that's not possible in general because you don't have enough stream
> > > > > > > IDs to do that.
> > > > > >
> > > > > > Oh! sorry miss understand what your means. It is possible like what I did
> > > > > > at v3 version. But I think it is not good enough.
> > > > > >
> > > > > > >
> > > > > > > > Some hot plug devics
> > > > > > > > (SD7.0) may plug after system boot. Two PCIe instances shared one set
> > > > > > > > of 6bits stream id (total 64). Assume total 16 assign to two PCIe
> > > > > > > > controllers. each have 8 stream id. If use uboot assign it static, each
> > > > > > > > PCIe controller have below 8 devices. It will be failrue one controller
> > > > > > > > connect 7, another connect 9. but if dynamtic alloc when devices add, both
> > > > > > > > controller can work.
> > > > > > > >
> > > > > > > > Although we have not so much devices now, this way give us possility to
> > > > > > > > improve it in future.
> > > > > > > >
> > > > > > > > > It sounds like the consequences of running out of LUT entries are
> > > > > > > > > catastrophic, e.g., memory corruption from mis-directed DMA? If
> > > > > > > > > that's possible, I think we need to figure out how to prevent the
> > > > > > > > > device from being used, not just dev_warn() about it.
> > > > > > > >
> > > > > > > > Yes, but so far, we have not met such problem now. We can improve it when
> > > > > > > > we really face such problem.
> > > > > > >
> > > > > > > If this controller can only support DMA from a limited number of
> > > > > > > endpoints below it, I think we should figure out how to enforce that
> > > > > > > directly. Maybe we can prevent drivers from enabling bus mastering or
> > > > > > > something. I'm not happy with the idea of waiting for and debugging a
> > > > > > > report of data corruption.
> > > > > >
> > > > > > It may add a pre-add hook function to pci bridge. let me do more research.
> > > > >
> > > > > Hi Bjorn:
> > > > >
> > > > > int pci_setup_device(struct pci_dev *dev)
> > > > > {
> > > > > dev->error_state = pci_channel_io_normal;
> > > > > ...
> > > > > pci_fixup_device(pci_fixup_early, dev);
> > > > >
> > > > > ^^^ I can add fixup hook for pci_fixup_early. If not resource,
> > > > > I can set dev->error_state to pci_channel_io_frozen or
> > > > > pci_channel_io_perm_failure
> > > > >
> > > > > And add below check here after call hook function.
> > > > >
> > > > > if (dev->error_state != pci_channel_io_normal)
> > > > > return -EIO;
> > > > >
> > > > > }
> > > > >
> > > > > How do you think this method? If you agree, I can continue search device
> > > > > remove hook up.
> > > >
> > > > I think this would mean the device would not appear to be enumerated
> > > > at all, right? I.e., it wouldn't show up in lspci? And we couldn't
> > > > use even a pure programmed IO driver with no DMA or MSI?
> > >
> > > Make sense. Let me do more research on this.
> > >
> > > Frank
> > > >
> > > > I wonder if we should have a function pointer in struct
> > > > pci_host_bridge, kind of like the existing ->map_irq(), where we could
> > > > do host bridge-specific setup when enumerating a PCI device.
> >
> > Consider some device may no use MSI or DMA. It'd better set LUT when
> > allocate msi irq. I think insert a irq-domain in irq hierarchy.
> >
> > static const struct irq_domain_ops lut_pcie_msi_domain_ops = {
> > .alloc = lut_pcie_irq_domain_alloc,
> > .free = lut_pcie_irq_domain_free,
> > };
> >
> > int dw_pcie_allocate_domains(struct dw_pcie_rp *pp)
> > {
> > struct fwnode_handle *fwnode = of_node_to_fwnode(pci->dev->of_node);
> >
> > pp->irq_domain = irq_domain_create_hierarchy(...)
> >
> > pp->msi_domain = pci_msi_create_irq_domain(...);
> >
> > return 0;
> > }
> >
> > Manage lut stream id in lut_pcie_irq_domain_alloc() and
> > lut_pcie_irq_domain_free().
> >
> > So failure happen only when driver use MSI and no-stream ID avaiable. It
> > should be better than failure when add devices. Some devices may not use
> > at all.
>
> I'm not an IRQ expert, but it sounds plausible. There might even be
> an opportunity to fall back to INTx if there's no stream ID available
> for MSI?
Sorry, I think this was a half-baked thought. Exhaustion of stream
IDs should be an uncommon situation, and the important thing is to
prevent terrible things from happening.
I don't think it's worth bending over backwards to make everything
possible limp along. If it's easy to just make the device
inaccessible, that's fine. If there's a simple way to make it
available but keep from enabling bus mastering, we could do that too,
but only if it's really simple.
^ permalink raw reply [flat|nested] 44+ messages in thread* Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-06-22 17:38 ` Bjorn Helgaas
@ 2024-06-24 15:00 ` Frank Li
0 siblings, 0 replies; 44+ messages in thread
From: Frank Li @ 2024-06-24 15:00 UTC (permalink / raw)
To: Bjorn Helgaas
Cc: Robin Murphy, Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley,
linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Will Deacon, Joerg Roedel, Jason Gunthorpe, Alyssa Rosenzweig,
Marc Zyngier
On Sat, Jun 22, 2024 at 12:38:49PM -0500, Bjorn Helgaas wrote:
> On Fri, Jun 21, 2024 at 05:43:21PM -0500, Bjorn Helgaas wrote:
> > On Fri, Jun 21, 2024 at 06:29:48PM -0400, Frank Li wrote:
> > > On Mon, Jun 17, 2024 at 10:26:36AM -0400, Frank Li wrote:
> > > > On Thu, Jun 13, 2024 at 05:41:25PM -0500, Bjorn Helgaas wrote:
> > > > > On Thu, Jun 06, 2024 at 04:24:17PM -0400, Frank Li wrote:
> > > > > > On Mon, Jun 03, 2024 at 04:07:55PM -0400, Frank Li wrote:
> > > > > > > On Mon, Jun 03, 2024 at 01:56:27PM -0500, Bjorn Helgaas wrote:
> > > > > > > > On Mon, Jun 03, 2024 at 02:42:45PM -0400, Frank Li wrote:
> > > > > > > > > On Mon, Jun 03, 2024 at 12:19:21PM -0500, Bjorn Helgaas wrote:
> > > > > > > > > > On Fri, May 31, 2024 at 03:58:49PM +0100, Robin Murphy wrote:
> > > > > > > > > > > On 2024-05-31 12:08 am, Bjorn Helgaas wrote:
> > > > > > > > > > > > [+cc IOMMU and pcie-apple.c folks for comment]
> > > > > > > > > > > >
> > > > > > > > > > > > On Tue, May 28, 2024 at 03:39:21PM -0400, Frank Li wrote:
> > > > > > > > > > > > > For the i.MX95, configuration of a LUT is necessary to convert Bus Device
> > > > > > > > > > > > > Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS.
> > > > > > > > > > > > > This involves examining the msi-map and smmu-map to ensure consistent
> > > > > > > > > > > > > mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related
> > > > > > > > > > > > > registers are configured. In the absence of an msi-map, the built-in MSI
> > > > > > > > > > > > > controller is utilized as a fallback.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Additionally, register a PCI bus notifier to trigger imx_pcie_add_device()
> > > > > > > > > > > > > upon the appearance of a new PCI device and when the bus is an iMX6 PCI
> > > > > > > > > > > > > controller. This function configures the correct LUT based on Device Tree
> > > > > > > > > > > > > Settings (DTS).
> > > > > > > > > > > >
> > > > > > > > > > > > This scheme is pretty similar to apple_pcie_bus_notifier(). If we
> > > > > > > > > > > > have to do this, I wish it were *more* similar, i.e., copy the
> > > > > > > > > > > > function names, bitmap tracking, code structure, etc.
> > > > > > > > > > > >
> > > > > > > > > > > > I don't really know how stream IDs work, but I assume they are used on
> > > > > > > > > > > > most or all arm64 platforms, so I'm a little surprised that of all the
> > > > > > > > > > > > PCI host drivers used on arm64, only pcie-apple.c and pci-imx6.c need
> > > > > > > > > > > > this notifier.
> > > > > > > > > > >
> > > > > > > > > > > This is one of those things that's mostly at the mercy of the PCIe root
> > > > > > > > > > > complex implementation. Typically the SMMU StreamID and/or GIC ITS DeviceID
> > > > > > > > > > > is derived directly from the PCI RID, sometimes with additional high-order
> > > > > > > > > > > bits hard-wired to disambiguate PCI segments. I believe this RID-translation
> > > > > > > > > > > LUT is a particular feature of the the Synopsys IP - I know there's also one
> > > > > > > > > > > on the NXP Layerscape platforms, but on those it's programmed by the
> > > > > > > > > > > bootloader, which also generates the appropriate "msi-map" and "iommu-map"
> > > > > > > > > > > properties to match. Ideally that's what i.MX should do as well, but hey.
> > > > > > > > > >
> > > > > > > > > > Maybe this RID-translation is a feature of i.MX, not of Synopsys? I
> > > > > > > > > > see that the LUT CSR accesses use IMX95_* definitions.
> > > > > > > > >
> > > > > > > > > Yes, it convert 16bit RID to 6bit stream id.
> > > > > > > >
> > > > > > > > IIUC, you're saying this is not a Synopsys feature, it's an i.MX
> > > > > > > > feature.
> > > > > > >
> > > > > > > Yes, it is i.MX feature. But I think other vendor should have similar
> > > > > > > situation if use old arm smmu.
> > > > > > >
> > > > > > > >
> > > > > > > > > > > If it's really necessary to do this programming from Linux, then there's
> > > > > > > > > > > still no point in it being dynamic - the mappings cannot ever change, since
> > > > > > > > > > > the rest of the kernel believes that what the DT said at boot time was
> > > > > > > > > > > already a property of the hardware. It would be a lot more logical, and
> > > > > > > > > > > likely simpler, for the driver to just read the relevant map property and
> > > > > > > > > > > program the entire LUT to match, all in one go at controller probe time.
> > > > > > > > > > > Rather like what's already commonly done with the parsing of "dma-ranges" to
> > > > > > > > > > > program address-translation LUTs for inbound windows.
> > > > > > > > > > >
> > > > > > > > > > > Plus that would also give a chance of safely dealing with bad DTs specifying
> > > > > > > > > > > invalid ID mappings (by refusing to probe at all). As it is, returning an
> > > > > > > > > > > error from a child's BUS_NOTIFY_ADD_DEVICE does nothing except prevent any
> > > > > > > > > > > further notifiers from running at that point - the device will still be
> > > > > > > > > > > added, allowed to bind a driver, and able to start sending DMA/MSI traffic
> > > > > > > > > > > without the controller being correctly programmed, which at best won't work
> > > > > > > > > > > and at worst may break the whole system.
> > > > > > > > > >
> > > > > > > > > > Frank, could the imx LUT be programmed once at boot-time instead of at
> > > > > > > > > > device-add time? I'm guessing maybe not because apparently there is a
> > > > > > > > > > risk of running out of LUT entries?
> > > > > > > > >
> > > > > > > > > It is not good idea to depend on boot loader so much.
> > > > > > > >
> > > > > > > > I meant "could this be programmed once when the Linux imx host
> > > > > > > > controller driver is probed?" But from the below, it sounds like
> > > > > > > > that's not possible in general because you don't have enough stream
> > > > > > > > IDs to do that.
> > > > > > >
> > > > > > > Oh! sorry miss understand what your means. It is possible like what I did
> > > > > > > at v3 version. But I think it is not good enough.
> > > > > > >
> > > > > > > >
> > > > > > > > > Some hot plug devics
> > > > > > > > > (SD7.0) may plug after system boot. Two PCIe instances shared one set
> > > > > > > > > of 6bits stream id (total 64). Assume total 16 assign to two PCIe
> > > > > > > > > controllers. each have 8 stream id. If use uboot assign it static, each
> > > > > > > > > PCIe controller have below 8 devices. It will be failrue one controller
> > > > > > > > > connect 7, another connect 9. but if dynamtic alloc when devices add, both
> > > > > > > > > controller can work.
> > > > > > > > >
> > > > > > > > > Although we have not so much devices now, this way give us possility to
> > > > > > > > > improve it in future.
> > > > > > > > >
> > > > > > > > > > It sounds like the consequences of running out of LUT entries are
> > > > > > > > > > catastrophic, e.g., memory corruption from mis-directed DMA? If
> > > > > > > > > > that's possible, I think we need to figure out how to prevent the
> > > > > > > > > > device from being used, not just dev_warn() about it.
> > > > > > > > >
> > > > > > > > > Yes, but so far, we have not met such problem now. We can improve it when
> > > > > > > > > we really face such problem.
> > > > > > > >
> > > > > > > > If this controller can only support DMA from a limited number of
> > > > > > > > endpoints below it, I think we should figure out how to enforce that
> > > > > > > > directly. Maybe we can prevent drivers from enabling bus mastering or
> > > > > > > > something. I'm not happy with the idea of waiting for and debugging a
> > > > > > > > report of data corruption.
> > > > > > >
> > > > > > > It may add a pre-add hook function to pci bridge. let me do more research.
> > > > > >
> > > > > > Hi Bjorn:
> > > > > >
> > > > > > int pci_setup_device(struct pci_dev *dev)
> > > > > > {
> > > > > > dev->error_state = pci_channel_io_normal;
> > > > > > ...
> > > > > > pci_fixup_device(pci_fixup_early, dev);
> > > > > >
> > > > > > ^^^ I can add fixup hook for pci_fixup_early. If not resource,
> > > > > > I can set dev->error_state to pci_channel_io_frozen or
> > > > > > pci_channel_io_perm_failure
> > > > > >
> > > > > > And add below check here after call hook function.
> > > > > >
> > > > > > if (dev->error_state != pci_channel_io_normal)
> > > > > > return -EIO;
> > > > > >
> > > > > > }
> > > > > >
> > > > > > How do you think this method? If you agree, I can continue search device
> > > > > > remove hook up.
> > > > >
> > > > > I think this would mean the device would not appear to be enumerated
> > > > > at all, right? I.e., it wouldn't show up in lspci? And we couldn't
> > > > > use even a pure programmed IO driver with no DMA or MSI?
> > > >
> > > > Make sense. Let me do more research on this.
> > > >
> > > > Frank
> > > > >
> > > > > I wonder if we should have a function pointer in struct
> > > > > pci_host_bridge, kind of like the existing ->map_irq(), where we could
> > > > > do host bridge-specific setup when enumerating a PCI device.
> > >
> > > Consider some device may no use MSI or DMA. It'd better set LUT when
> > > allocate msi irq. I think insert a irq-domain in irq hierarchy.
> > >
> > > static const struct irq_domain_ops lut_pcie_msi_domain_ops = {
> > > .alloc = lut_pcie_irq_domain_alloc,
> > > .free = lut_pcie_irq_domain_free,
> > > };
> > >
> > > int dw_pcie_allocate_domains(struct dw_pcie_rp *pp)
> > > {
> > > struct fwnode_handle *fwnode = of_node_to_fwnode(pci->dev->of_node);
> > >
> > > pp->irq_domain = irq_domain_create_hierarchy(...)
> > >
> > > pp->msi_domain = pci_msi_create_irq_domain(...);
> > >
> > > return 0;
> > > }
> > >
> > > Manage lut stream id in lut_pcie_irq_domain_alloc() and
> > > lut_pcie_irq_domain_free().
> > >
> > > So failure happen only when driver use MSI and no-stream ID avaiable. It
> > > should be better than failure when add devices. Some devices may not use
> > > at all.
> >
> > I'm not an IRQ expert, but it sounds plausible. There might even be
> > an opportunity to fall back to INTx if there's no stream ID available
> > for MSI?
>
> Sorry, I think this was a half-baked thought. Exhaustion of stream
> IDs should be an uncommon situation, and the important thing is to
> prevent terrible things from happening.
>
> I don't think it's worth bending over backwards to make everything
> possible limp along. If it's easy to just make the device
> inaccessible, that's fine. If there's a simple way to make it
> available but keep from enabling bus mastering, we could do that too,
> but only if it's really simple.
Mani mentions qcom use simple method to config all lut when probe at
qcom_pcie_config_sid_1_9_0(). It is similar with my v3 version.
https://lore.kernel.org/imx/20240402-pci2_upstream-v3-8-803414bdb430@nxp.com/
Of course both have not resolved run-out sid problems. But genenerally,
one PCIE slot only connect one devices. static alloc 8/16 sid are enough
for 99.9% user case.
best regards
Frank Li
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-06-03 17:19 ` Bjorn Helgaas
2024-06-03 18:42 ` Frank Li
@ 2024-06-03 20:20 ` Robin Murphy
2024-06-03 21:04 ` Frank Li
1 sibling, 1 reply; 44+ messages in thread
From: Robin Murphy @ 2024-06-03 20:20 UTC (permalink / raw)
To: Bjorn Helgaas
Cc: Frank Li, Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley,
linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Will Deacon, Joerg Roedel, Jason Gunthorpe, Alyssa Rosenzweig,
Marc Zyngier
On 2024-06-03 6:19 pm, Bjorn Helgaas wrote:
> On Fri, May 31, 2024 at 03:58:49PM +0100, Robin Murphy wrote:
>> On 2024-05-31 12:08 am, Bjorn Helgaas wrote:
>>> [+cc IOMMU and pcie-apple.c folks for comment]
>>>
>>> On Tue, May 28, 2024 at 03:39:21PM -0400, Frank Li wrote:
>>>> For the i.MX95, configuration of a LUT is necessary to convert Bus Device
>>>> Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS.
>>>> This involves examining the msi-map and smmu-map to ensure consistent
>>>> mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related
>>>> registers are configured. In the absence of an msi-map, the built-in MSI
>>>> controller is utilized as a fallback.
>>>>
>>>> Additionally, register a PCI bus notifier to trigger imx_pcie_add_device()
>>>> upon the appearance of a new PCI device and when the bus is an iMX6 PCI
>>>> controller. This function configures the correct LUT based on Device Tree
>>>> Settings (DTS).
>>>
>>> This scheme is pretty similar to apple_pcie_bus_notifier(). If we
>>> have to do this, I wish it were *more* similar, i.e., copy the
>>> function names, bitmap tracking, code structure, etc.
>>>
>>> I don't really know how stream IDs work, but I assume they are used on
>>> most or all arm64 platforms, so I'm a little surprised that of all the
>>> PCI host drivers used on arm64, only pcie-apple.c and pci-imx6.c need
>>> this notifier.
>>
>> This is one of those things that's mostly at the mercy of the PCIe root
>> complex implementation. Typically the SMMU StreamID and/or GIC ITS DeviceID
>> is derived directly from the PCI RID, sometimes with additional high-order
>> bits hard-wired to disambiguate PCI segments. I believe this RID-translation
>> LUT is a particular feature of the the Synopsys IP - I know there's also one
>> on the NXP Layerscape platforms, but on those it's programmed by the
>> bootloader, which also generates the appropriate "msi-map" and "iommu-map"
>> properties to match. Ideally that's what i.MX should do as well, but hey.
>
> Maybe this RID-translation is a feature of i.MX, not of Synopsys? I
> see that the LUT CSR accesses use IMX95_* definitions.
Well, it's not unreasonable to call things "IMX95" in this context if
they are only relevant to the configuration used by i.MX95, and not to
the other i.MX SoCs which this driver also supports. However the data
register fields certainly look suspiciously similar to those used on
Layerscape[1], although I guess that still doesn't rule out it being
NXP's own widget either. Anyway, the exact details aren't really
significant, the point was really just to say don't expect this to
generalise much beyond what you've seen already, and that there's
precedent for bootloaders doing this for us.
>> If it's really necessary to do this programming from Linux, then there's
>> still no point in it being dynamic - the mappings cannot ever change, since
>> the rest of the kernel believes that what the DT said at boot time was
>> already a property of the hardware. It would be a lot more logical, and
>> likely simpler, for the driver to just read the relevant map property and
>> program the entire LUT to match, all in one go at controller probe time.
>> Rather like what's already commonly done with the parsing of "dma-ranges" to
>> program address-translation LUTs for inbound windows.
>>
>> Plus that would also give a chance of safely dealing with bad DTs specifying
>> invalid ID mappings (by refusing to probe at all). As it is, returning an
>> error from a child's BUS_NOTIFY_ADD_DEVICE does nothing except prevent any
>> further notifiers from running at that point - the device will still be
>> added, allowed to bind a driver, and able to start sending DMA/MSI traffic
>> without the controller being correctly programmed, which at best won't work
>> and at worst may break the whole system.
>
> Frank, could the imx LUT be programmed once at boot-time instead of at
> device-add time? I'm guessing maybe not because apparently there is a
> risk of running out of LUT entries?
The risk still exists just as much either way - if we have a bogus DT
and/or just more PCI RIDs present than we can handle, we're going to
have a bad time. There's no advantage to only finding that out once we
try to add the 33rd device and it's too late to even do anything about it.
In fact if anything, this notifier approach exacerbates that risk the
most by consuming one LUT entry per PCI RID regardless of whether an
"iommu-map-mask" is involved. Assuming the IMX95_PE0_LUT_MASK field is
the same as its Layerscape counterpart, we could support >32 RIDs if the
map and mask are constructed to squash multiple RIDs onto each StreamID
(the SMMU driver supports this), and we have the up-front information to
easily configure hardware masking in the LUT itself. It's not
necessarily possible to reconstruct such mappings from only seeing
individual input and output values one-by-one.
Thanks,
Robin.
[1]
https://source.denx.de/u-boot/u-boot/-/blob/master/drivers/pci/pcie_layerscape_fixup.c?ref_type=heads#L83
> It sounds like the consequences of running out of LUT entries are
> catastrophic, e.g., memory corruption from mis-directed DMA? If
> that's possible, I think we need to figure out how to prevent the
> device from being used, not just dev_warn() about it.
>
> Bjorn
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-06-03 20:20 ` Robin Murphy
@ 2024-06-03 21:04 ` Frank Li
2024-06-04 15:25 ` Marc Zyngier
0 siblings, 1 reply; 44+ messages in thread
From: Frank Li @ 2024-06-03 21:04 UTC (permalink / raw)
To: Robin Murphy
Cc: Bjorn Helgaas, Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley,
linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Will Deacon, Joerg Roedel, Jason Gunthorpe, Alyssa Rosenzweig,
Marc Zyngier
On Mon, Jun 03, 2024 at 09:20:03PM +0100, Robin Murphy wrote:
> On 2024-06-03 6:19 pm, Bjorn Helgaas wrote:
> > On Fri, May 31, 2024 at 03:58:49PM +0100, Robin Murphy wrote:
> > > On 2024-05-31 12:08 am, Bjorn Helgaas wrote:
> > > > [+cc IOMMU and pcie-apple.c folks for comment]
> > > >
> > > > On Tue, May 28, 2024 at 03:39:21PM -0400, Frank Li wrote:
> > > > > For the i.MX95, configuration of a LUT is necessary to convert Bus Device
> > > > > Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS.
> > > > > This involves examining the msi-map and smmu-map to ensure consistent
> > > > > mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related
> > > > > registers are configured. In the absence of an msi-map, the built-in MSI
> > > > > controller is utilized as a fallback.
> > > > >
> > > > > Additionally, register a PCI bus notifier to trigger imx_pcie_add_device()
> > > > > upon the appearance of a new PCI device and when the bus is an iMX6 PCI
> > > > > controller. This function configures the correct LUT based on Device Tree
> > > > > Settings (DTS).
> > > >
> > > > This scheme is pretty similar to apple_pcie_bus_notifier(). If we
> > > > have to do this, I wish it were *more* similar, i.e., copy the
> > > > function names, bitmap tracking, code structure, etc.
> > > >
> > > > I don't really know how stream IDs work, but I assume they are used on
> > > > most or all arm64 platforms, so I'm a little surprised that of all the
> > > > PCI host drivers used on arm64, only pcie-apple.c and pci-imx6.c need
> > > > this notifier.
> > >
> > > This is one of those things that's mostly at the mercy of the PCIe root
> > > complex implementation. Typically the SMMU StreamID and/or GIC ITS DeviceID
> > > is derived directly from the PCI RID, sometimes with additional high-order
> > > bits hard-wired to disambiguate PCI segments. I believe this RID-translation
> > > LUT is a particular feature of the the Synopsys IP - I know there's also one
> > > on the NXP Layerscape platforms, but on those it's programmed by the
> > > bootloader, which also generates the appropriate "msi-map" and "iommu-map"
> > > properties to match. Ideally that's what i.MX should do as well, but hey.
> >
> > Maybe this RID-translation is a feature of i.MX, not of Synopsys? I
> > see that the LUT CSR accesses use IMX95_* definitions.
>
> Well, it's not unreasonable to call things "IMX95" in this context if they
> are only relevant to the configuration used by i.MX95, and not to the other
> i.MX SoCs which this driver also supports. However the data register fields
> certainly look suspiciously similar to those used on Layerscape[1], although
> I guess that still doesn't rule out it being NXP's own widget either.
> Anyway, the exact details aren't really significant, the point was really
> just to say don't expect this to generalise much beyond what you've seen
> already, and that there's precedent for bootloaders doing this for us.
It is re-used layerscape design at this point. I don't know the history,
why choose use bootloader to config it, supposed it should be done at
kernel. We found some problem by using bootloader to config LUT. Most
layserscape system PCI devices are fixed during boot.
During I debug layerscape PCI, I was trapped by uboot many times. It is
quite anoise, especially using difference version uboot.
>
> > > If it's really necessary to do this programming from Linux, then there's
> > > still no point in it being dynamic - the mappings cannot ever change, since
> > > the rest of the kernel believes that what the DT said at boot time was
> > > already a property of the hardware. It would be a lot more logical, and
> > > likely simpler, for the driver to just read the relevant map property and
> > > program the entire LUT to match, all in one go at controller probe time.
> > > Rather like what's already commonly done with the parsing of "dma-ranges" to
> > > program address-translation LUTs for inbound windows.
> > >
> > > Plus that would also give a chance of safely dealing with bad DTs specifying
> > > invalid ID mappings (by refusing to probe at all). As it is, returning an
> > > error from a child's BUS_NOTIFY_ADD_DEVICE does nothing except prevent any
> > > further notifiers from running at that point - the device will still be
> > > added, allowed to bind a driver, and able to start sending DMA/MSI traffic
> > > without the controller being correctly programmed, which at best won't work
> > > and at worst may break the whole system.
> >
> > Frank, could the imx LUT be programmed once at boot-time instead of at
> > device-add time? I'm guessing maybe not because apparently there is a
> > risk of running out of LUT entries?
>
> The risk still exists just as much either way - if we have a bogus DT and/or
> just more PCI RIDs present than we can handle, we're going to have a bad
> time. There's no advantage to only finding that out once we try to add the
> 33rd device and it's too late to even do anything about it.
>
> In fact if anything, this notifier approach exacerbates that risk the most
> by consuming one LUT entry per PCI RID regardless of whether an
> "iommu-map-mask" is involved. Assuming the IMX95_PE0_LUT_MASK field is the
> same as its Layerscape counterpart, we could support >32 RIDs if the map and
> mask are constructed to squash multiple RIDs onto each StreamID (the SMMU
> driver supports this), and we have the up-front information to easily
> configure hardware masking in the LUT itself. It's not necessarily possible
> to reconstruct such mappings from only seeing individual input and output
> values one-by-one.
iommu may share one stream id for multi-devices. but ITS MSI can't. each
device's MSI index start from 0. It needs difference stream id for each
device.
>
> Thanks,
> Robin.
>
> [1] https://source.denx.de/u-boot/u-boot/-/blob/master/drivers/pci/pcie_layerscape_fixup.c?ref_type=heads#L83
Thanks, but It can't resolve device hot-plug problem.
>
> > It sounds like the consequences of running out of LUT entries are
> > catastrophic, e.g., memory corruption from mis-directed DMA? If
> > that's possible, I think we need to figure out how to prevent the
> > device from being used, not just dev_warn() about it.
> >
> > Bjorn
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-06-03 21:04 ` Frank Li
@ 2024-06-04 15:25 ` Marc Zyngier
2024-06-04 15:47 ` Frank Li
0 siblings, 1 reply; 44+ messages in thread
From: Marc Zyngier @ 2024-06-04 15:25 UTC (permalink / raw)
To: Frank Li
Cc: Robin Murphy, Bjorn Helgaas, Richard Zhu, Lucas Stach,
Lorenzo Pieralisi, Krzysztof Wilczyński, Rob Herring,
Bjorn Helgaas, Shawn Guo, Sascha Hauer, Pengutronix Kernel Team,
Fabio Estevam, NXP Linux Team, Philipp Zabel, Liam Girdwood,
Mark Brown, Manivannan Sadhasivam, Krzysztof Kozlowski,
Conor Dooley, linux-pci, imx, linux-arm-kernel, linux-kernel, bpf,
devicetree, Will Deacon, Joerg Roedel, Jason Gunthorpe,
Alyssa Rosenzweig
On Mon, 03 Jun 2024 22:04:23 +0100,
Frank Li <Frank.li@nxp.com> wrote:
>
> iommu may share one stream id for multi-devices. but ITS MSI can't. each
> device's MSI index start from 0. It needs difference stream id for each
> device.
That's not quite true. We go through all sort of hoops to find about
device aliasing on PCI and allow devices that translate into the same
DID to get MSIs.
Of course, just like the IOMMU, you lose any form of isolation, but
you get what you pay for.
M.
--
Without deviation from the norm, progress is not possible.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-06-04 15:25 ` Marc Zyngier
@ 2024-06-04 15:47 ` Frank Li
0 siblings, 0 replies; 44+ messages in thread
From: Frank Li @ 2024-06-04 15:47 UTC (permalink / raw)
To: Marc Zyngier
Cc: Robin Murphy, Bjorn Helgaas, Richard Zhu, Lucas Stach,
Lorenzo Pieralisi, Krzysztof Wilczyński, Rob Herring,
Bjorn Helgaas, Shawn Guo, Sascha Hauer, Pengutronix Kernel Team,
Fabio Estevam, NXP Linux Team, Philipp Zabel, Liam Girdwood,
Mark Brown, Manivannan Sadhasivam, Krzysztof Kozlowski,
Conor Dooley, linux-pci, imx, linux-arm-kernel, linux-kernel, bpf,
devicetree, Will Deacon, Joerg Roedel, Jason Gunthorpe,
Alyssa Rosenzweig
On Tue, Jun 04, 2024 at 04:25:51PM +0100, Marc Zyngier wrote:
> On Mon, 03 Jun 2024 22:04:23 +0100,
> Frank Li <Frank.li@nxp.com> wrote:
> >
> > iommu may share one stream id for multi-devices. but ITS MSI can't. each
> > device's MSI index start from 0. It needs difference stream id for each
> > device.
>
> That's not quite true. We go through all sort of hoops to find about
> device aliasing on PCI and allow devices that translate into the same
> DID to get MSIs.
Could you please point me the code location? I remember I met error when I
try to enable MSI for EP support one year ago.
https://lore.kernel.org/imx/20221124055036.1630573-1-Frank.Li@nxp.com/
I remember that it is failure try to allocate more irqs with the same DID.
Frank
>
> Of course, just like the IOMMU, you lose any form of isolation, but
> you get what you pay for.
Anyways, if there are one quirk in PCI system, bridge driver can detect
PCI devices add/remove. it will be better if assign stream ID as need,
compared with static allocate.
>
> M.
>
> --
> Without deviation from the norm, progress is not possible.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-05-31 14:58 ` Robin Murphy
2024-05-31 16:25 ` Frank Li
2024-06-03 17:19 ` Bjorn Helgaas
@ 2024-06-03 20:29 ` Laurentiu Tudor
2024-06-03 21:16 ` Frank Li
2 siblings, 1 reply; 44+ messages in thread
From: Laurentiu Tudor @ 2024-06-03 20:29 UTC (permalink / raw)
To: Robin Murphy, Bjorn Helgaas, Frank Li
Cc: Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley,
linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Will Deacon, Joerg Roedel, Jason Gunthorpe, Alyssa Rosenzweig,
Marc Zyngier
On 5/31/24 17:58, Robin Murphy wrote:
> On 2024-05-31 12:08 am, Bjorn Helgaas wrote:
>> [+cc IOMMU and pcie-apple.c folks for comment]
>>
>> On Tue, May 28, 2024 at 03:39:21PM -0400, Frank Li wrote:
>>> For the i.MX95, configuration of a LUT is necessary to convert Bus
>>> Device
>>> Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS.
>>> This involves examining the msi-map and smmu-map to ensure consistent
>>> mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related
>>> registers are configured. In the absence of an msi-map, the built-in MSI
>>> controller is utilized as a fallback.
>>>
>>> Additionally, register a PCI bus notifier to trigger
>>> imx_pcie_add_device()
>>> upon the appearance of a new PCI device and when the bus is an iMX6 PCI
>>> controller. This function configures the correct LUT based on Device
>>> Tree
>>> Settings (DTS).
>>
>> This scheme is pretty similar to apple_pcie_bus_notifier(). If we
>> have to do this, I wish it were *more* similar, i.e., copy the
>> function names, bitmap tracking, code structure, etc.
>>
>> I don't really know how stream IDs work, but I assume they are used on
>> most or all arm64 platforms, so I'm a little surprised that of all the
>> PCI host drivers used on arm64, only pcie-apple.c and pci-imx6.c need
>> this notifier.
>
> This is one of those things that's mostly at the mercy of the PCIe root
> complex implementation. Typically the SMMU StreamID and/or GIC ITS
> DeviceID is derived directly from the PCI RID, sometimes with additional
> high-order bits hard-wired to disambiguate PCI segments. I believe this
> RID-translation LUT is a particular feature of the the Synopsys IP - I
> know there's also one on the NXP Layerscape platforms, but on those it's
> programmed by the bootloader, which also generates the appropriate
> "msi-map" and "iommu-map" properties to match. Ideally that's what i.MX
> should do as well, but hey.
That's usually fine, except when SRIOV and/or hotplug devices (that is,
not discoverable at bootloader time) come into play. We came up with
this "solution" to cover these more dynamic scenarios.
https://source.denx.de/u-boot/u-boot/-/commit/2a5bbb13cc39102a68fcc31056925427ab44b591
---
Best Regards, Laurentiu
>> There's this path, which is pretty generic and does at least the
>> of_map_id() part of what you're doing in imx_pcie_add_device():
>>
>> __driver_probe_device
>> really_probe
>> pci_dma_configure #
>> pci_bus_type.dma_configure
>> of_dma_configure
>> of_dma_configure_id
>> of_iommu_configure
>> of_pci_iommu_init
>> of_iommu_configure_dev_id
>> of_map_id
>> of_iommu_xlate
>> ops = iommu_ops_from_fwnode
>> iommu_fwspec_init
>> ops->of_xlate(dev, iommu_spec)
>>
>> Maybe this needs to be extended somehow with a hook to do the
>> device-specific work like updating the LUT? Just speculating here,
>> the IOMMU folks will know how this is expected to work.
>
> Note that that particular code path has fundamental issues and much of
> it needs to go away (I'm working on it, but it's a rich ~8-year-old pile
> of technical debt...). IOMMU configuration needs to be happening at
> device_add() time via the IOMMU layer's own bus notifier.
>
> If it's really necessary to do this programming from Linux, then there's
> still no point in it being dynamic - the mappings cannot ever change,
> since the rest of the kernel believes that what the DT said at boot time
> was already a property of the hardware. It would be a lot more logical,
> and likely simpler, for the driver to just read the relevant map
> property and program the entire LUT to match, all in one go at
> controller probe time. Rather like what's already commonly done with the
> parsing of "dma-ranges" to program address-translation LUTs for inbound
> windows.
>
> Plus that would also give a chance of safely dealing with bad DTs
> specifying invalid ID mappings (by refusing to probe at all). As it is,
> returning an error from a child's BUS_NOTIFY_ADD_DEVICE does nothing
> except prevent any further notifiers from running at that point - the
> device will still be added, allowed to bind a driver, and able to start
> sending DMA/MSI traffic without the controller being correctly
> programmed, which at best won't work and at worst may break the whole
> system.
>
> Thanks,
> Robin.
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-06-03 20:29 ` Laurentiu Tudor
@ 2024-06-03 21:16 ` Frank Li
0 siblings, 0 replies; 44+ messages in thread
From: Frank Li @ 2024-06-03 21:16 UTC (permalink / raw)
To: Laurentiu Tudor
Cc: Robin Murphy, Bjorn Helgaas, Richard Zhu, Lucas Stach,
Lorenzo Pieralisi, Krzysztof Wilczyński, Rob Herring,
Bjorn Helgaas, Shawn Guo, Sascha Hauer, Pengutronix Kernel Team,
Fabio Estevam, NXP Linux Team, Philipp Zabel, Liam Girdwood,
Mark Brown, Manivannan Sadhasivam, Krzysztof Kozlowski,
Conor Dooley, linux-pci, imx, linux-arm-kernel, linux-kernel, bpf,
devicetree, Will Deacon, Joerg Roedel, Jason Gunthorpe,
Alyssa Rosenzweig, Marc Zyngier
On Mon, Jun 03, 2024 at 11:29:38PM +0300, Laurentiu Tudor wrote:
>
>
> On 5/31/24 17:58, Robin Murphy wrote:
> > On 2024-05-31 12:08 am, Bjorn Helgaas wrote:
> > > [+cc IOMMU and pcie-apple.c folks for comment]
> > >
> > > On Tue, May 28, 2024 at 03:39:21PM -0400, Frank Li wrote:
> > > > For the i.MX95, configuration of a LUT is necessary to convert
> > > > Bus Device
> > > > Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS.
> > > > This involves examining the msi-map and smmu-map to ensure consistent
> > > > mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related
> > > > registers are configured. In the absence of an msi-map, the built-in MSI
> > > > controller is utilized as a fallback.
> > > >
> > > > Additionally, register a PCI bus notifier to trigger
> > > > imx_pcie_add_device()
> > > > upon the appearance of a new PCI device and when the bus is an iMX6 PCI
> > > > controller. This function configures the correct LUT based on
> > > > Device Tree
> > > > Settings (DTS).
> > >
> > > This scheme is pretty similar to apple_pcie_bus_notifier(). If we
> > > have to do this, I wish it were *more* similar, i.e., copy the
> > > function names, bitmap tracking, code structure, etc.
> > >
> > > I don't really know how stream IDs work, but I assume they are used on
> > > most or all arm64 platforms, so I'm a little surprised that of all the
> > > PCI host drivers used on arm64, only pcie-apple.c and pci-imx6.c need
> > > this notifier.
> >
> > This is one of those things that's mostly at the mercy of the PCIe root
> > complex implementation. Typically the SMMU StreamID and/or GIC ITS
> > DeviceID is derived directly from the PCI RID, sometimes with additional
> > high-order bits hard-wired to disambiguate PCI segments. I believe this
> > RID-translation LUT is a particular feature of the the Synopsys IP - I
> > know there's also one on the NXP Layerscape platforms, but on those it's
> > programmed by the bootloader, which also generates the appropriate
> > "msi-map" and "iommu-map" properties to match. Ideally that's what i.MX
> > should do as well, but hey.
>
> That's usually fine, except when SRIOV and/or hotplug devices (that is, not
> discoverable at bootloader time) come into play. We came up with this
> "solution" to cover these more dynamic scenarios.
>
> https://source.denx.de/u-boot/u-boot/-/commit/2a5bbb13cc39102a68fcc31056925427ab44b591
If my understand is correct, basic it was still pre-allocate method. Just
reserve more streams id for SRIOV and hotplugs. It was not really depend
on what devices in system.
Frank
>
> ---
> Best Regards, Laurentiu
>
> > > There's this path, which is pretty generic and does at least the
> > > of_map_id() part of what you're doing in imx_pcie_add_device():
> > >
> > > __driver_probe_device
> > > really_probe
> > > pci_dma_configure #
> > > pci_bus_type.dma_configure
> > > of_dma_configure
> > > of_dma_configure_id
> > > of_iommu_configure
> > > of_pci_iommu_init
> > > of_iommu_configure_dev_id
> > > of_map_id
> > > of_iommu_xlate
> > > ops = iommu_ops_from_fwnode
> > > iommu_fwspec_init
> > > ops->of_xlate(dev, iommu_spec)
> > >
> > > Maybe this needs to be extended somehow with a hook to do the
> > > device-specific work like updating the LUT? Just speculating here,
> > > the IOMMU folks will know how this is expected to work.
> >
> > Note that that particular code path has fundamental issues and much of
> > it needs to go away (I'm working on it, but it's a rich ~8-year-old pile
> > of technical debt...). IOMMU configuration needs to be happening at
> > device_add() time via the IOMMU layer's own bus notifier.
> >
> > If it's really necessary to do this programming from Linux, then there's
> > still no point in it being dynamic - the mappings cannot ever change,
> > since the rest of the kernel believes that what the DT said at boot time
> > was already a property of the hardware. It would be a lot more logical,
> > and likely simpler, for the driver to just read the relevant map
> > property and program the entire LUT to match, all in one go at
> > controller probe time. Rather like what's already commonly done with the
> > parsing of "dma-ranges" to program address-translation LUTs for inbound
> > windows.
> >
> > Plus that would also give a chance of safely dealing with bad DTs
> > specifying invalid ID mappings (by refusing to probe at all). As it is,
> > returning an error from a child's BUS_NOTIFY_ADD_DEVICE does nothing
> > except prevent any further notifiers from running at that point - the
> > device will still be added, allowed to bind a driver, and able to start
> > sending DMA/MSI traffic without the controller being correctly
> > programmed, which at best won't work and at worst may break the whole
> > system.
> >
> > Thanks,
> > Robin.
> >
> > _______________________________________________
> > linux-arm-kernel mailing list
> > linux-arm-kernel@lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-05-30 23:08 ` Bjorn Helgaas
2024-05-31 13:52 ` Marc Zyngier
2024-05-31 14:58 ` Robin Murphy
@ 2024-05-31 16:14 ` Frank Li
2024-06-03 17:11 ` Bjorn Helgaas
2 siblings, 1 reply; 44+ messages in thread
From: Frank Li @ 2024-05-31 16:14 UTC (permalink / raw)
To: Bjorn Helgaas
Cc: Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley,
linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Will Deacon, Robin Murphy, Joerg Roedel, Jason Gunthorpe,
Alyssa Rosenzweig, Marc Zyngier
On Thu, May 30, 2024 at 06:08:32PM -0500, Bjorn Helgaas wrote:
> [+cc IOMMU and pcie-apple.c folks for comment]
>
> On Tue, May 28, 2024 at 03:39:21PM -0400, Frank Li wrote:
> > For the i.MX95, configuration of a LUT is necessary to convert Bus Device
> > Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS.
> > This involves examining the msi-map and smmu-map to ensure consistent
> > mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related
> > registers are configured. In the absence of an msi-map, the built-in MSI
> > controller is utilized as a fallback.
> >
> > Additionally, register a PCI bus notifier to trigger imx_pcie_add_device()
> > upon the appearance of a new PCI device and when the bus is an iMX6 PCI
> > controller. This function configures the correct LUT based on Device Tree
> > Settings (DTS).
>
> This scheme is pretty similar to apple_pcie_bus_notifier(). If we
> have to do this, I wish it were *more* similar, i.e., copy the
> function names, bitmap tracking, code structure, etc.
Actually, I refer apple_pcie_bus_notifier(). I can't direct use apple's
implement because in imx95 have difference PCI host controller, another one
is PCI ECAM netc controller. At lease function name should be similar with
apple.
>
> I don't really know how stream IDs work, but I assume they are used on
> most or all arm64 platforms, so I'm a little surprised that of all the
> PCI host drivers used on arm64, only pcie-apple.c and pci-imx6.c need
> this notifier.
>
> There's this path, which is pretty generic and does at least the
> of_map_id() part of what you're doing in imx_pcie_add_device():
>
> __driver_probe_device
> really_probe
> pci_dma_configure # pci_bus_type.dma_configure
> of_dma_configure
> of_dma_configure_id
> of_iommu_configure
> of_pci_iommu_init
> of_iommu_configure_dev_id
> of_map_id
> of_iommu_xlate
> ops = iommu_ops_from_fwnode
> iommu_fwspec_init
> ops->of_xlate(dev, iommu_spec)
>
> Maybe this needs to be extended somehow with a hook to do the
> device-specific work like updating the LUT? Just speculating here,
> the IOMMU folks will know how this is expected to work.
Let me do more study. But I think ITS also need stream ID, not sure if
only hook IOMMU can work. Some configuration, IOMMU have not enabled. but
ITS need it.
Ideally, pci system can provide a hook function to host bridge when new
device add and remove.
Frank
>
> Some typos and minor comments below.
>
> > Signed-off-by: Frank Li <Frank.Li@nxp.com>
> > ---
> > drivers/pci/controller/dwc/pci-imx6.c | 175 +++++++++++++++++++++++++++++++++-
> > 1 file changed, 174 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c
> > index 29309ad0e352b..8ecc00049e20b 100644
> > --- a/drivers/pci/controller/dwc/pci-imx6.c
> > +++ b/drivers/pci/controller/dwc/pci-imx6.c
> > @@ -54,6 +54,22 @@
> > #define IMX95_PE0_GEN_CTRL_3 0x1058
> > #define IMX95_PCIE_LTSSM_EN BIT(0)
> >
> > +#define IMX95_PE0_LUT_ACSCTRL 0x1008
> > +#define IMX95_PEO_LUT_RWA BIT(16)
> > +#define IMX95_PE0_LUT_ENLOC GENMASK(4, 0)
> > +
> > +#define IMX95_PE0_LUT_DATA1 0x100c
> > +#define IMX95_PE0_LUT_VLD BIT(31)
> > +#define IMX95_PE0_LUT_DAC_ID GENMASK(10, 8)
> > +#define IMX95_PE0_LUT_STREAM_ID GENMASK(5, 0)
> > +
> > +#define IMX95_PE0_LUT_DATA2 0x1010
> > +#define IMX95_PE0_LUT_REQID GENMASK(31, 16)
> > +#define IMX95_PE0_LUT_MASK GENMASK(15, 0)
> > +
> > +#define IMX95_SID_MASK GENMASK(5, 0)
> > +#define IMX95_MAX_LUT 32
> > +
> > #define to_imx_pcie(x) dev_get_drvdata((x)->dev)
> >
> > enum imx_pcie_variants {
> > @@ -79,6 +95,7 @@ enum imx_pcie_variants {
> > #define IMX_PCIE_FLAG_HAS_PHY_RESET BIT(5)
> > #define IMX_PCIE_FLAG_HAS_SERDES BIT(6)
> > #define IMX_PCIE_FLAG_SUPPORT_64BIT BIT(7)
> > +#define IMX_PCIE_FLAG_MONITOR_DEV BIT(8)
> >
> > #define imx_check_flag(pci, val) (pci->drvdata->flags & val)
> >
> > @@ -132,6 +149,8 @@ struct imx_pcie {
> > struct device *pd_pcie_phy;
> > struct phy *phy;
> > const struct imx_pcie_drvdata *drvdata;
> > +
> > + struct mutex lock;
> > };
> >
> > /* Parameters for the waiting for PCIe PHY PLL to lock on i.MX7 */
> > @@ -215,6 +234,66 @@ static int imx95_pcie_init_phy(struct imx_pcie *imx_pcie)
> > return 0;
> > }
> >
> > +static int imx_pcie_config_lut(struct imx_pcie *imx_pcie, u16 reqid, u8 sid)
> > +{
> > + struct dw_pcie *pci = imx_pcie->pci;
> > + struct device *dev = pci->dev;
> > + u32 data1, data2;
> > + int i;
> > +
> > + if (sid >= 64) {
> > + dev_err(dev, "Invalid SID for index %d\n", sid);
> > + return -EINVAL;
> > + }
> > +
> > + guard(mutex)(&imx_pcie->lock);
> > +
> > + for (i = 0; i < IMX95_MAX_LUT; i++) {
> > + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_ACSCTRL, IMX95_PEO_LUT_RWA | i);
> > +
> > + regmap_read(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA1, &data1);
> > + if (data1 & IMX95_PE0_LUT_VLD)
> > + continue;
> > +
> > + data1 = FIELD_PREP(IMX95_PE0_LUT_DAC_ID, 0);
> > + data1 |= FIELD_PREP(IMX95_PE0_LUT_STREAM_ID, sid);
> > + data1 |= IMX95_PE0_LUT_VLD;
> > +
> > + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA1, data1);
> > +
> > + data2 = 0xffff;
> > + data2 |= FIELD_PREP(IMX95_PE0_LUT_REQID, reqid);
> > +
> > + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA2, data2);
> > +
> > + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_ACSCTRL, i);
> > +
> > + return 0;
> > + }
> > +
> > + dev_err(dev, "All lut already used\n");
> > + return -EINVAL;
> > +}
> > +
> > +static void imx_pcie_remove_lut(struct imx_pcie *imx_pcie, u16 reqid)
> > +{
> > + u32 data2 = 0;
> > + int i;
> > +
> > + guard(mutex)(&imx_pcie->lock);
> > +
> > + for (i = 0; i < IMX95_MAX_LUT; i++) {
> > + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_ACSCTRL, IMX95_PEO_LUT_RWA | i);
> > +
> > + regmap_read(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA2, &data2);
> > + if (FIELD_GET(IMX95_PE0_LUT_REQID, data2) == reqid) {
> > + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA1, 0);
> > + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA2, 0);
> > + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_ACSCTRL, i);
> > + }
> > + }
> > +}
> > +
> > static void imx_pcie_configure_type(struct imx_pcie *imx_pcie)
> > {
> > const struct imx_pcie_drvdata *drvdata = imx_pcie->drvdata;
> > @@ -1232,6 +1311,85 @@ static int imx_pcie_resume_noirq(struct device *dev)
> > return 0;
> > }
> >
> > +static bool imx_pcie_match_device(struct pci_bus *bus);
>
> Can you add the imx_pcie_match_device() earlier in the file so we
> don't need this forward declaration?
imx_pcie_match_device() use global veriable imx_pcie_driver, which close to
end of the file. There will be bigger change if move imx_pcie_driver ahead.
>
> > +static int imx_pcie_add_device(struct imx_pcie *imx_pcie, struct pci_dev *pdev)
> > +{
> > + u32 sid_i = 0, sid_m = 0, rid = pci_dev_id(pdev);
> > + struct device *dev = imx_pcie->pci->dev;
> > + int err;
> > +
> > + err = of_map_id(dev->of_node, rid, "iommu-map", "iommu-map-mask", NULL, &sid_i);
> > + if (err)
> > + return err;
> > +
> > + err = of_map_id(dev->of_node, rid, "msi-map", "msi-map-mask", NULL, &sid_m);
> > + if (err)
> > + return err;
> > +
> > + if (sid_i != rid && sid_m != rid)
> > + if ((sid_i & IMX95_SID_MASK) != (sid_m & IMX95_SID_MASK)) {
> > + dev_err(dev, "its and iommu stream id miss match, please check dts file\n");
> > + return -EINVAL;
> > + }
> > +
> > + /* if iommu-map is not existed then use msi-map's stream id*/
>
> Capitalize consistently, e.g., the most comments in this file start
> with a capital letter.
>
> s/is not existed/does not exist/
>
> Add space before closing */
>
> > + if (sid_i == rid)
> > + sid_i = sid_m;
> > +
> > + sid_i &= IMX95_SID_MASK;
> > +
> > + if (sid_i != rid)
> > + return imx_pcie_config_lut(imx_pcie, rid, sid_i);
> > +
> > + /* Use dwc built-in MSI controller */
> > + return 0;
> > +}
> > +
> > +static void imx_pcie_del_device(struct imx_pcie *imx_pcie, struct pci_dev *pdev)
> > +{
> > + imx_pcie_remove_lut(imx_pcie, pci_dev_id(pdev));
> > +}
> > +
> > +
> > +static int imx_pcie_bus_notifier(struct notifier_block *nb, unsigned long action, void *data)
> > +{
> > + struct pci_host_bridge *host;
> > + struct imx_pcie *imx_pcie;
> > + struct pci_dev *pdev;
> > + int err;
> > +
> > + pdev = to_pci_dev(data);
> > + host = pci_find_host_bridge(pdev->bus);
> > +
> > + if (!imx_pcie_match_device(host->bus))
> > + return NOTIFY_OK;
> > +
> > + imx_pcie = to_imx_pcie(to_dw_pcie_from_pp(host->sysdata));
> > +
> > + if (!imx_check_flag(imx_pcie, IMX_PCIE_FLAG_MONITOR_DEV))
> > + return NOTIFY_OK;
> > +
> > + switch (action) {
> > + case BUS_NOTIFY_ADD_DEVICE:
> > + err = imx_pcie_add_device(imx_pcie, pdev);
> > + if (err)
> > + return notifier_from_errno(err);
> > + break;
> > + case BUS_NOTIFY_DEL_DEVICE:
> > + imx_pcie_del_device(imx_pcie, pdev);
> > + break;
> > + default:
> > + return NOTIFY_DONE;
> > + }
> > +
> > + return NOTIFY_OK;
> > +}
> > +
> > +static struct notifier_block imx_pcie_nb = {
> > + .notifier_call = imx_pcie_bus_notifier,
> > +};
> > +
> > static const struct dev_pm_ops imx_pcie_pm_ops = {
> > NOIRQ_SYSTEM_SLEEP_PM_OPS(imx_pcie_suspend_noirq,
> > imx_pcie_resume_noirq)
> > @@ -1264,6 +1422,8 @@ static int imx_pcie_probe(struct platform_device *pdev)
> > imx_pcie->pci = pci;
> > imx_pcie->drvdata = of_device_get_match_data(dev);
> >
> > + mutex_init(&imx_pcie->lock);
> > +
> > /* Find the PHY if one is defined, only imx7d uses it */
> > np = of_parse_phandle(node, "fsl,imx7d-pcie-phy", 0);
> > if (np) {
> > @@ -1551,7 +1711,8 @@ static const struct imx_pcie_drvdata drvdata[] = {
> > },
> > [IMX95] = {
> > .variant = IMX95,
> > - .flags = IMX_PCIE_FLAG_HAS_SERDES,
> > + .flags = IMX_PCIE_FLAG_HAS_SERDES |
> > + IMX_PCIE_FLAG_MONITOR_DEV,
> > .clk_names = imx8mq_clks,
> > .clks_cnt = ARRAY_SIZE(imx8mq_clks),
> > .ltssm_off = IMX95_PE0_GEN_CTRL_3,
> > @@ -1687,6 +1848,8 @@ DECLARE_PCI_FIXUP_CLASS_HEADER(PCI_VENDOR_ID_SYNOPSYS, 0xabcd,
> >
> > static int __init imx_pcie_init(void)
> > {
> > + int ret;
> > +
> > #ifdef CONFIG_ARM
> > struct device_node *np;
> >
> > @@ -1705,7 +1868,17 @@ static int __init imx_pcie_init(void)
> > hook_fault_code(8, imx6q_pcie_abort_handler, SIGBUS, 0,
> > "external abort on non-linefetch");
> > #endif
> > + ret = bus_register_notifier(&pci_bus_type, &imx_pcie_nb);
> > + if (ret)
> > + return ret;
>
> I think this should go in imx6_pcie_probe().
The same nb only register once. If move to probe, bus_register_notifier()
will return error and dump error message when second pci controller
instance probe.
>
> > return platform_driver_register(&imx_pcie_driver);
> > }
> > +
> > +static void __exit imx_pcie_exit(void)
> > +{
> > + bus_unregister_notifier(&pci_bus_type, &imx_pcie_nb);
>
> It looks like this driver is removable?
Actually, I think not. I am not sure how to prevent driver remove. There
are raise condition when driver remove and a pci device hot plug at the
same time although hot plug pci have not implement yet in imx platform.
>
> What happens when an external abort occurs after the
> imx6q_pcie_abort_handler() text is removed?
>
> > +}
> > +
> > device_initcall(imx_pcie_init);
> > +__exitcall(imx_pcie_exit);
> >
> > --
> > 2.34.1
> >
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 44+ messages in thread* Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-05-31 16:14 ` Frank Li
@ 2024-06-03 17:11 ` Bjorn Helgaas
2024-06-03 18:21 ` Frank Li
0 siblings, 1 reply; 44+ messages in thread
From: Bjorn Helgaas @ 2024-06-03 17:11 UTC (permalink / raw)
To: Frank Li
Cc: Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley,
linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Will Deacon, Robin Murphy, Joerg Roedel, Jason Gunthorpe,
Alyssa Rosenzweig, Marc Zyngier
On Fri, May 31, 2024 at 12:14:18PM -0400, Frank Li wrote:
> On Thu, May 30, 2024 at 06:08:32PM -0500, Bjorn Helgaas wrote:
> > On Tue, May 28, 2024 at 03:39:21PM -0400, Frank Li wrote:
> > > For the i.MX95, configuration of a LUT is necessary to convert Bus Device
> > > Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS.
> > > This involves examining the msi-map and smmu-map to ensure consistent
> > > mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related
> > > registers are configured. In the absence of an msi-map, the built-in MSI
> > > controller is utilized as a fallback.
> > >
> > > Additionally, register a PCI bus notifier to trigger imx_pcie_add_device()
> > > upon the appearance of a new PCI device and when the bus is an iMX6 PCI
> > > controller. This function configures the correct LUT based on Device Tree
> > > Settings (DTS).
> >
> > This scheme is pretty similar to apple_pcie_bus_notifier(). If we
> > have to do this, I wish it were *more* similar, i.e., copy the
> > function names, bitmap tracking, code structure, etc.
>
> Actually, I refer apple_pcie_bus_notifier(). I can't direct use apple's
> implement because in imx95 have difference PCI host controller, another one
> is PCI ECAM netc controller. At lease function name should be similar with
> apple.
I know it's different hardware, so obviously it can't be exactly the
same. These are the differences that looked possibly unnecessary:
- registering from initcall instead of .probe():
apple_pcie_probe # .probe() method
bus_register_notifier(&pci_bus_type, &apple_pcie_nb)
imx_pcie_init # device_initcall()
bus_register_notifier(&pci_bus_type, &imx_pcie_nb)
- naming BUS_NOTIFY_DEL_DEVICE function:
apple_pcie_release_device()
imx_pcie_del_device()
- tracking entries in use via bitmap vs scanning hardware for
invalid entries:
bitmap_find_free_region # apple
imx_pcie_config_lut # imx
for (i = 0; i < IMX95_MAX_LUT; i++)
regmap_read(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA1, &data1)
if (data1 & IMX95_PE0_LUT_VLD)
continue
When we fix a bug in one driver, it's easier to check whether other
drivers also need the fix if they use the same structure and names.
Bjorn
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 44+ messages in thread* Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-06-03 17:11 ` Bjorn Helgaas
@ 2024-06-03 18:21 ` Frank Li
0 siblings, 0 replies; 44+ messages in thread
From: Frank Li @ 2024-06-03 18:21 UTC (permalink / raw)
To: Bjorn Helgaas
Cc: Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley,
linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Will Deacon, Robin Murphy, Joerg Roedel, Jason Gunthorpe,
Alyssa Rosenzweig, Marc Zyngier
On Mon, Jun 03, 2024 at 12:11:49PM -0500, Bjorn Helgaas wrote:
> On Fri, May 31, 2024 at 12:14:18PM -0400, Frank Li wrote:
> > On Thu, May 30, 2024 at 06:08:32PM -0500, Bjorn Helgaas wrote:
> > > On Tue, May 28, 2024 at 03:39:21PM -0400, Frank Li wrote:
> > > > For the i.MX95, configuration of a LUT is necessary to convert Bus Device
> > > > Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS.
> > > > This involves examining the msi-map and smmu-map to ensure consistent
> > > > mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related
> > > > registers are configured. In the absence of an msi-map, the built-in MSI
> > > > controller is utilized as a fallback.
> > > >
> > > > Additionally, register a PCI bus notifier to trigger imx_pcie_add_device()
> > > > upon the appearance of a new PCI device and when the bus is an iMX6 PCI
> > > > controller. This function configures the correct LUT based on Device Tree
> > > > Settings (DTS).
> > >
> > > This scheme is pretty similar to apple_pcie_bus_notifier(). If we
> > > have to do this, I wish it were *more* similar, i.e., copy the
> > > function names, bitmap tracking, code structure, etc.
> >
> > Actually, I refer apple_pcie_bus_notifier(). I can't direct use apple's
> > implement because in imx95 have difference PCI host controller, another one
> > is PCI ECAM netc controller. At lease function name should be similar with
> > apple.
>
> I know it's different hardware, so obviously it can't be exactly the
> same. These are the differences that looked possibly unnecessary:
>
> - registering from initcall instead of .probe():
>
> apple_pcie_probe # .probe() method
> bus_register_notifier(&pci_bus_type, &apple_pcie_nb)
>
> imx_pcie_init # device_initcall()
> bus_register_notifier(&pci_bus_type, &imx_pcie_nb)
Maybe apple only one PCIe controller with multi ports. One nb just register
once.
If there are multi PCIe controller instance, second bus_register_notifier()
report error.
Of course, if you like, I can use global reference counter to handle this.
>
> - naming BUS_NOTIFY_DEL_DEVICE function:
>
> apple_pcie_release_device()
> imx_pcie_del_device()
Okay, I can change to release_device().
>
> - tracking entries in use via bitmap vs scanning hardware for
> invalid entries:
>
> bitmap_find_free_region # apple
>
> imx_pcie_config_lut # imx
> for (i = 0; i < IMX95_MAX_LUT; i++)
> regmap_read(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA1, &data1)
> if (data1 & IMX95_PE0_LUT_VLD)
> continue
Okay, I can change to use bitmap.
>
> When we fix a bug in one driver, it's easier to check whether other
> drivers also need the fix if they use the same structure and names.
>
> Bjorn
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-05-28 19:39 ` [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95 Frank Li
2024-05-30 23:08 ` Bjorn Helgaas
@ 2024-06-22 4:11 ` Manivannan Sadhasivam
2024-06-24 15:06 ` Frank Li
1 sibling, 1 reply; 44+ messages in thread
From: Manivannan Sadhasivam @ 2024-06-22 4:11 UTC (permalink / raw)
To: Frank Li
Cc: Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Krzysztof Kozlowski, Conor Dooley, linux-pci, imx,
linux-arm-kernel, linux-kernel, bpf, devicetree
On Tue, May 28, 2024 at 03:39:21PM -0400, Frank Li wrote:
> For the i.MX95, configuration of a LUT is necessary to convert Bus Device
> Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS.
> This involves examining the msi-map and smmu-map to ensure consistent
> mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related
> registers are configured. In the absence of an msi-map, the built-in MSI
> controller is utilized as a fallback.
>
> Additionally, register a PCI bus notifier to trigger imx_pcie_add_device()
> upon the appearance of a new PCI device and when the bus is an iMX6 PCI
> controller. This function configures the correct LUT based on Device Tree
> Settings (DTS).
>
Sorry for jumping the ship very late... But why can't you configure the LUT
during probe() itself? Anyway, you are going to use the 'iommu-map' and
'msi-map' which are static info provided in DT. I don't see a necessity to do it
during device addition time.
Qcom RC driver also uses a similar configuration in
qcom_pcie_config_sid_1_9_0().
- Mani
> Signed-off-by: Frank Li <Frank.Li@nxp.com>
> ---
> drivers/pci/controller/dwc/pci-imx6.c | 175 +++++++++++++++++++++++++++++++++-
> 1 file changed, 174 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c
> index 29309ad0e352b..8ecc00049e20b 100644
> --- a/drivers/pci/controller/dwc/pci-imx6.c
> +++ b/drivers/pci/controller/dwc/pci-imx6.c
> @@ -54,6 +54,22 @@
> #define IMX95_PE0_GEN_CTRL_3 0x1058
> #define IMX95_PCIE_LTSSM_EN BIT(0)
>
> +#define IMX95_PE0_LUT_ACSCTRL 0x1008
> +#define IMX95_PEO_LUT_RWA BIT(16)
> +#define IMX95_PE0_LUT_ENLOC GENMASK(4, 0)
> +
> +#define IMX95_PE0_LUT_DATA1 0x100c
> +#define IMX95_PE0_LUT_VLD BIT(31)
> +#define IMX95_PE0_LUT_DAC_ID GENMASK(10, 8)
> +#define IMX95_PE0_LUT_STREAM_ID GENMASK(5, 0)
> +
> +#define IMX95_PE0_LUT_DATA2 0x1010
> +#define IMX95_PE0_LUT_REQID GENMASK(31, 16)
> +#define IMX95_PE0_LUT_MASK GENMASK(15, 0)
> +
> +#define IMX95_SID_MASK GENMASK(5, 0)
> +#define IMX95_MAX_LUT 32
> +
> #define to_imx_pcie(x) dev_get_drvdata((x)->dev)
>
> enum imx_pcie_variants {
> @@ -79,6 +95,7 @@ enum imx_pcie_variants {
> #define IMX_PCIE_FLAG_HAS_PHY_RESET BIT(5)
> #define IMX_PCIE_FLAG_HAS_SERDES BIT(6)
> #define IMX_PCIE_FLAG_SUPPORT_64BIT BIT(7)
> +#define IMX_PCIE_FLAG_MONITOR_DEV BIT(8)
>
> #define imx_check_flag(pci, val) (pci->drvdata->flags & val)
>
> @@ -132,6 +149,8 @@ struct imx_pcie {
> struct device *pd_pcie_phy;
> struct phy *phy;
> const struct imx_pcie_drvdata *drvdata;
> +
> + struct mutex lock;
> };
>
> /* Parameters for the waiting for PCIe PHY PLL to lock on i.MX7 */
> @@ -215,6 +234,66 @@ static int imx95_pcie_init_phy(struct imx_pcie *imx_pcie)
> return 0;
> }
>
> +static int imx_pcie_config_lut(struct imx_pcie *imx_pcie, u16 reqid, u8 sid)
> +{
> + struct dw_pcie *pci = imx_pcie->pci;
> + struct device *dev = pci->dev;
> + u32 data1, data2;
> + int i;
> +
> + if (sid >= 64) {
> + dev_err(dev, "Invalid SID for index %d\n", sid);
> + return -EINVAL;
> + }
> +
> + guard(mutex)(&imx_pcie->lock);
> +
> + for (i = 0; i < IMX95_MAX_LUT; i++) {
> + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_ACSCTRL, IMX95_PEO_LUT_RWA | i);
> +
> + regmap_read(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA1, &data1);
> + if (data1 & IMX95_PE0_LUT_VLD)
> + continue;
> +
> + data1 = FIELD_PREP(IMX95_PE0_LUT_DAC_ID, 0);
> + data1 |= FIELD_PREP(IMX95_PE0_LUT_STREAM_ID, sid);
> + data1 |= IMX95_PE0_LUT_VLD;
> +
> + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA1, data1);
> +
> + data2 = 0xffff;
> + data2 |= FIELD_PREP(IMX95_PE0_LUT_REQID, reqid);
> +
> + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA2, data2);
> +
> + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_ACSCTRL, i);
> +
> + return 0;
> + }
> +
> + dev_err(dev, "All lut already used\n");
> + return -EINVAL;
> +}
> +
> +static void imx_pcie_remove_lut(struct imx_pcie *imx_pcie, u16 reqid)
> +{
> + u32 data2 = 0;
> + int i;
> +
> + guard(mutex)(&imx_pcie->lock);
> +
> + for (i = 0; i < IMX95_MAX_LUT; i++) {
> + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_ACSCTRL, IMX95_PEO_LUT_RWA | i);
> +
> + regmap_read(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA2, &data2);
> + if (FIELD_GET(IMX95_PE0_LUT_REQID, data2) == reqid) {
> + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA1, 0);
> + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA2, 0);
> + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_ACSCTRL, i);
> + }
> + }
> +}
> +
> static void imx_pcie_configure_type(struct imx_pcie *imx_pcie)
> {
> const struct imx_pcie_drvdata *drvdata = imx_pcie->drvdata;
> @@ -1232,6 +1311,85 @@ static int imx_pcie_resume_noirq(struct device *dev)
> return 0;
> }
>
> +static bool imx_pcie_match_device(struct pci_bus *bus);
> +
> +static int imx_pcie_add_device(struct imx_pcie *imx_pcie, struct pci_dev *pdev)
> +{
> + u32 sid_i = 0, sid_m = 0, rid = pci_dev_id(pdev);
> + struct device *dev = imx_pcie->pci->dev;
> + int err;
> +
> + err = of_map_id(dev->of_node, rid, "iommu-map", "iommu-map-mask", NULL, &sid_i);
> + if (err)
> + return err;
> +
> + err = of_map_id(dev->of_node, rid, "msi-map", "msi-map-mask", NULL, &sid_m);
> + if (err)
> + return err;
> +
> + if (sid_i != rid && sid_m != rid)
> + if ((sid_i & IMX95_SID_MASK) != (sid_m & IMX95_SID_MASK)) {
> + dev_err(dev, "its and iommu stream id miss match, please check dts file\n");
> + return -EINVAL;
> + }
> +
> + /* if iommu-map is not existed then use msi-map's stream id*/
> + if (sid_i == rid)
> + sid_i = sid_m;
> +
> + sid_i &= IMX95_SID_MASK;
> +
> + if (sid_i != rid)
> + return imx_pcie_config_lut(imx_pcie, rid, sid_i);
> +
> + /* Use dwc built-in MSI controller */
> + return 0;
> +}
> +
> +static void imx_pcie_del_device(struct imx_pcie *imx_pcie, struct pci_dev *pdev)
> +{
> + imx_pcie_remove_lut(imx_pcie, pci_dev_id(pdev));
> +}
> +
> +
> +static int imx_pcie_bus_notifier(struct notifier_block *nb, unsigned long action, void *data)
> +{
> + struct pci_host_bridge *host;
> + struct imx_pcie *imx_pcie;
> + struct pci_dev *pdev;
> + int err;
> +
> + pdev = to_pci_dev(data);
> + host = pci_find_host_bridge(pdev->bus);
> +
> + if (!imx_pcie_match_device(host->bus))
> + return NOTIFY_OK;
> +
> + imx_pcie = to_imx_pcie(to_dw_pcie_from_pp(host->sysdata));
> +
> + if (!imx_check_flag(imx_pcie, IMX_PCIE_FLAG_MONITOR_DEV))
> + return NOTIFY_OK;
> +
> + switch (action) {
> + case BUS_NOTIFY_ADD_DEVICE:
> + err = imx_pcie_add_device(imx_pcie, pdev);
> + if (err)
> + return notifier_from_errno(err);
> + break;
> + case BUS_NOTIFY_DEL_DEVICE:
> + imx_pcie_del_device(imx_pcie, pdev);
> + break;
> + default:
> + return NOTIFY_DONE;
> + }
> +
> + return NOTIFY_OK;
> +}
> +
> +static struct notifier_block imx_pcie_nb = {
> + .notifier_call = imx_pcie_bus_notifier,
> +};
> +
> static const struct dev_pm_ops imx_pcie_pm_ops = {
> NOIRQ_SYSTEM_SLEEP_PM_OPS(imx_pcie_suspend_noirq,
> imx_pcie_resume_noirq)
> @@ -1264,6 +1422,8 @@ static int imx_pcie_probe(struct platform_device *pdev)
> imx_pcie->pci = pci;
> imx_pcie->drvdata = of_device_get_match_data(dev);
>
> + mutex_init(&imx_pcie->lock);
> +
> /* Find the PHY if one is defined, only imx7d uses it */
> np = of_parse_phandle(node, "fsl,imx7d-pcie-phy", 0);
> if (np) {
> @@ -1551,7 +1711,8 @@ static const struct imx_pcie_drvdata drvdata[] = {
> },
> [IMX95] = {
> .variant = IMX95,
> - .flags = IMX_PCIE_FLAG_HAS_SERDES,
> + .flags = IMX_PCIE_FLAG_HAS_SERDES |
> + IMX_PCIE_FLAG_MONITOR_DEV,
> .clk_names = imx8mq_clks,
> .clks_cnt = ARRAY_SIZE(imx8mq_clks),
> .ltssm_off = IMX95_PE0_GEN_CTRL_3,
> @@ -1687,6 +1848,8 @@ DECLARE_PCI_FIXUP_CLASS_HEADER(PCI_VENDOR_ID_SYNOPSYS, 0xabcd,
>
> static int __init imx_pcie_init(void)
> {
> + int ret;
> +
> #ifdef CONFIG_ARM
> struct device_node *np;
>
> @@ -1705,7 +1868,17 @@ static int __init imx_pcie_init(void)
> hook_fault_code(8, imx6q_pcie_abort_handler, SIGBUS, 0,
> "external abort on non-linefetch");
> #endif
> + ret = bus_register_notifier(&pci_bus_type, &imx_pcie_nb);
> + if (ret)
> + return ret;
>
> return platform_driver_register(&imx_pcie_driver);
> }
> +
> +static void __exit imx_pcie_exit(void)
> +{
> + bus_unregister_notifier(&pci_bus_type, &imx_pcie_nb);
> +}
> +
> device_initcall(imx_pcie_init);
> +__exitcall(imx_pcie_exit);
>
> --
> 2.34.1
>
--
மணிவண்ணன் சதாசிவம்
^ permalink raw reply [flat|nested] 44+ messages in thread* Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
2024-06-22 4:11 ` Manivannan Sadhasivam
@ 2024-06-24 15:06 ` Frank Li
0 siblings, 0 replies; 44+ messages in thread
From: Frank Li @ 2024-06-24 15:06 UTC (permalink / raw)
To: Manivannan Sadhasivam
Cc: Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Krzysztof Kozlowski, Conor Dooley, linux-pci, imx,
linux-arm-kernel, linux-kernel, bpf, devicetree
On Sat, Jun 22, 2024 at 09:41:15AM +0530, Manivannan Sadhasivam wrote:
> On Tue, May 28, 2024 at 03:39:21PM -0400, Frank Li wrote:
> > For the i.MX95, configuration of a LUT is necessary to convert Bus Device
> > Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS.
> > This involves examining the msi-map and smmu-map to ensure consistent
> > mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related
> > registers are configured. In the absence of an msi-map, the built-in MSI
> > controller is utilized as a fallback.
> >
> > Additionally, register a PCI bus notifier to trigger imx_pcie_add_device()
> > upon the appearance of a new PCI device and when the bus is an iMX6 PCI
> > controller. This function configures the correct LUT based on Device Tree
> > Settings (DTS).
> >
>
> Sorry for jumping the ship very late... But why can't you configure the LUT
> during probe() itself? Anyway, you are going to use the 'iommu-map' and
> 'msi-map' which are static info provided in DT. I don't see a necessity to do it
> during device addition time.
>
> Qcom RC driver also uses a similar configuration in
> qcom_pcie_config_sid_1_9_0().
It is similar with my v3 version.
https://lore.kernel.org/imx/20240402-pci2_upstream-v3-8-803414bdb430@nxp.com/
Rob don't like parse these property by individial driver.
https://lore.kernel.org/imx/20240429150842.GC1709920-robh@kernel.org/#t
But if use standarnd of_map_xxx() API to get sid, it needs RID information,
which only get when new PCI device added.
SID problem may take more long time to discuss. Can you help review v6
version:
https://lore.kernel.org/imx/20240617-pci2_upstream-v6-0-e0821238f997@nxp.com/T/#t
I want make bug fixes and 8qxp support to be merged firstly.
Frank
>
> - Mani
>
> > Signed-off-by: Frank Li <Frank.Li@nxp.com>
> > ---
> > drivers/pci/controller/dwc/pci-imx6.c | 175 +++++++++++++++++++++++++++++++++-
> > 1 file changed, 174 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c
> > index 29309ad0e352b..8ecc00049e20b 100644
> > --- a/drivers/pci/controller/dwc/pci-imx6.c
> > +++ b/drivers/pci/controller/dwc/pci-imx6.c
> > @@ -54,6 +54,22 @@
> > #define IMX95_PE0_GEN_CTRL_3 0x1058
> > #define IMX95_PCIE_LTSSM_EN BIT(0)
> >
> > +#define IMX95_PE0_LUT_ACSCTRL 0x1008
> > +#define IMX95_PEO_LUT_RWA BIT(16)
> > +#define IMX95_PE0_LUT_ENLOC GENMASK(4, 0)
> > +
> > +#define IMX95_PE0_LUT_DATA1 0x100c
> > +#define IMX95_PE0_LUT_VLD BIT(31)
> > +#define IMX95_PE0_LUT_DAC_ID GENMASK(10, 8)
> > +#define IMX95_PE0_LUT_STREAM_ID GENMASK(5, 0)
> > +
> > +#define IMX95_PE0_LUT_DATA2 0x1010
> > +#define IMX95_PE0_LUT_REQID GENMASK(31, 16)
> > +#define IMX95_PE0_LUT_MASK GENMASK(15, 0)
> > +
> > +#define IMX95_SID_MASK GENMASK(5, 0)
> > +#define IMX95_MAX_LUT 32
> > +
> > #define to_imx_pcie(x) dev_get_drvdata((x)->dev)
> >
> > enum imx_pcie_variants {
> > @@ -79,6 +95,7 @@ enum imx_pcie_variants {
> > #define IMX_PCIE_FLAG_HAS_PHY_RESET BIT(5)
> > #define IMX_PCIE_FLAG_HAS_SERDES BIT(6)
> > #define IMX_PCIE_FLAG_SUPPORT_64BIT BIT(7)
> > +#define IMX_PCIE_FLAG_MONITOR_DEV BIT(8)
> >
> > #define imx_check_flag(pci, val) (pci->drvdata->flags & val)
> >
> > @@ -132,6 +149,8 @@ struct imx_pcie {
> > struct device *pd_pcie_phy;
> > struct phy *phy;
> > const struct imx_pcie_drvdata *drvdata;
> > +
> > + struct mutex lock;
> > };
> >
> > /* Parameters for the waiting for PCIe PHY PLL to lock on i.MX7 */
> > @@ -215,6 +234,66 @@ static int imx95_pcie_init_phy(struct imx_pcie *imx_pcie)
> > return 0;
> > }
> >
> > +static int imx_pcie_config_lut(struct imx_pcie *imx_pcie, u16 reqid, u8 sid)
> > +{
> > + struct dw_pcie *pci = imx_pcie->pci;
> > + struct device *dev = pci->dev;
> > + u32 data1, data2;
> > + int i;
> > +
> > + if (sid >= 64) {
> > + dev_err(dev, "Invalid SID for index %d\n", sid);
> > + return -EINVAL;
> > + }
> > +
> > + guard(mutex)(&imx_pcie->lock);
> > +
> > + for (i = 0; i < IMX95_MAX_LUT; i++) {
> > + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_ACSCTRL, IMX95_PEO_LUT_RWA | i);
> > +
> > + regmap_read(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA1, &data1);
> > + if (data1 & IMX95_PE0_LUT_VLD)
> > + continue;
> > +
> > + data1 = FIELD_PREP(IMX95_PE0_LUT_DAC_ID, 0);
> > + data1 |= FIELD_PREP(IMX95_PE0_LUT_STREAM_ID, sid);
> > + data1 |= IMX95_PE0_LUT_VLD;
> > +
> > + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA1, data1);
> > +
> > + data2 = 0xffff;
> > + data2 |= FIELD_PREP(IMX95_PE0_LUT_REQID, reqid);
> > +
> > + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA2, data2);
> > +
> > + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_ACSCTRL, i);
> > +
> > + return 0;
> > + }
> > +
> > + dev_err(dev, "All lut already used\n");
> > + return -EINVAL;
> > +}
> > +
> > +static void imx_pcie_remove_lut(struct imx_pcie *imx_pcie, u16 reqid)
> > +{
> > + u32 data2 = 0;
> > + int i;
> > +
> > + guard(mutex)(&imx_pcie->lock);
> > +
> > + for (i = 0; i < IMX95_MAX_LUT; i++) {
> > + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_ACSCTRL, IMX95_PEO_LUT_RWA | i);
> > +
> > + regmap_read(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA2, &data2);
> > + if (FIELD_GET(IMX95_PE0_LUT_REQID, data2) == reqid) {
> > + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA1, 0);
> > + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA2, 0);
> > + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_ACSCTRL, i);
> > + }
> > + }
> > +}
> > +
> > static void imx_pcie_configure_type(struct imx_pcie *imx_pcie)
> > {
> > const struct imx_pcie_drvdata *drvdata = imx_pcie->drvdata;
> > @@ -1232,6 +1311,85 @@ static int imx_pcie_resume_noirq(struct device *dev)
> > return 0;
> > }
> >
> > +static bool imx_pcie_match_device(struct pci_bus *bus);
> > +
> > +static int imx_pcie_add_device(struct imx_pcie *imx_pcie, struct pci_dev *pdev)
> > +{
> > + u32 sid_i = 0, sid_m = 0, rid = pci_dev_id(pdev);
> > + struct device *dev = imx_pcie->pci->dev;
> > + int err;
> > +
> > + err = of_map_id(dev->of_node, rid, "iommu-map", "iommu-map-mask", NULL, &sid_i);
> > + if (err)
> > + return err;
> > +
> > + err = of_map_id(dev->of_node, rid, "msi-map", "msi-map-mask", NULL, &sid_m);
> > + if (err)
> > + return err;
> > +
> > + if (sid_i != rid && sid_m != rid)
> > + if ((sid_i & IMX95_SID_MASK) != (sid_m & IMX95_SID_MASK)) {
> > + dev_err(dev, "its and iommu stream id miss match, please check dts file\n");
> > + return -EINVAL;
> > + }
> > +
> > + /* if iommu-map is not existed then use msi-map's stream id*/
> > + if (sid_i == rid)
> > + sid_i = sid_m;
> > +
> > + sid_i &= IMX95_SID_MASK;
> > +
> > + if (sid_i != rid)
> > + return imx_pcie_config_lut(imx_pcie, rid, sid_i);
> > +
> > + /* Use dwc built-in MSI controller */
> > + return 0;
> > +}
> > +
> > +static void imx_pcie_del_device(struct imx_pcie *imx_pcie, struct pci_dev *pdev)
> > +{
> > + imx_pcie_remove_lut(imx_pcie, pci_dev_id(pdev));
> > +}
> > +
> > +
> > +static int imx_pcie_bus_notifier(struct notifier_block *nb, unsigned long action, void *data)
> > +{
> > + struct pci_host_bridge *host;
> > + struct imx_pcie *imx_pcie;
> > + struct pci_dev *pdev;
> > + int err;
> > +
> > + pdev = to_pci_dev(data);
> > + host = pci_find_host_bridge(pdev->bus);
> > +
> > + if (!imx_pcie_match_device(host->bus))
> > + return NOTIFY_OK;
> > +
> > + imx_pcie = to_imx_pcie(to_dw_pcie_from_pp(host->sysdata));
> > +
> > + if (!imx_check_flag(imx_pcie, IMX_PCIE_FLAG_MONITOR_DEV))
> > + return NOTIFY_OK;
> > +
> > + switch (action) {
> > + case BUS_NOTIFY_ADD_DEVICE:
> > + err = imx_pcie_add_device(imx_pcie, pdev);
> > + if (err)
> > + return notifier_from_errno(err);
> > + break;
> > + case BUS_NOTIFY_DEL_DEVICE:
> > + imx_pcie_del_device(imx_pcie, pdev);
> > + break;
> > + default:
> > + return NOTIFY_DONE;
> > + }
> > +
> > + return NOTIFY_OK;
> > +}
> > +
> > +static struct notifier_block imx_pcie_nb = {
> > + .notifier_call = imx_pcie_bus_notifier,
> > +};
> > +
> > static const struct dev_pm_ops imx_pcie_pm_ops = {
> > NOIRQ_SYSTEM_SLEEP_PM_OPS(imx_pcie_suspend_noirq,
> > imx_pcie_resume_noirq)
> > @@ -1264,6 +1422,8 @@ static int imx_pcie_probe(struct platform_device *pdev)
> > imx_pcie->pci = pci;
> > imx_pcie->drvdata = of_device_get_match_data(dev);
> >
> > + mutex_init(&imx_pcie->lock);
> > +
> > /* Find the PHY if one is defined, only imx7d uses it */
> > np = of_parse_phandle(node, "fsl,imx7d-pcie-phy", 0);
> > if (np) {
> > @@ -1551,7 +1711,8 @@ static const struct imx_pcie_drvdata drvdata[] = {
> > },
> > [IMX95] = {
> > .variant = IMX95,
> > - .flags = IMX_PCIE_FLAG_HAS_SERDES,
> > + .flags = IMX_PCIE_FLAG_HAS_SERDES |
> > + IMX_PCIE_FLAG_MONITOR_DEV,
> > .clk_names = imx8mq_clks,
> > .clks_cnt = ARRAY_SIZE(imx8mq_clks),
> > .ltssm_off = IMX95_PE0_GEN_CTRL_3,
> > @@ -1687,6 +1848,8 @@ DECLARE_PCI_FIXUP_CLASS_HEADER(PCI_VENDOR_ID_SYNOPSYS, 0xabcd,
> >
> > static int __init imx_pcie_init(void)
> > {
> > + int ret;
> > +
> > #ifdef CONFIG_ARM
> > struct device_node *np;
> >
> > @@ -1705,7 +1868,17 @@ static int __init imx_pcie_init(void)
> > hook_fault_code(8, imx6q_pcie_abort_handler, SIGBUS, 0,
> > "external abort on non-linefetch");
> > #endif
> > + ret = bus_register_notifier(&pci_bus_type, &imx_pcie_nb);
> > + if (ret)
> > + return ret;
> >
> > return platform_driver_register(&imx_pcie_driver);
> > }
> > +
> > +static void __exit imx_pcie_exit(void)
> > +{
> > + bus_unregister_notifier(&pci_bus_type, &imx_pcie_nb);
> > +}
> > +
> > device_initcall(imx_pcie_init);
> > +__exitcall(imx_pcie_exit);
> >
> > --
> > 2.34.1
> >
>
> --
> மணிவண்ணன் சதாசிவம்
^ permalink raw reply [flat|nested] 44+ messages in thread
* [PATCH v5 09/12] PCI: imx6: Consolidate redundant if-checks
2024-05-28 19:39 [PATCH v5 00/12] PCI: imx6: Fix\rename\clean up and add lut information for imx95 Frank Li
` (7 preceding siblings ...)
2024-05-28 19:39 ` [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95 Frank Li
@ 2024-05-28 19:39 ` Frank Li
2024-05-28 19:39 ` [PATCH v5 10/12] dt-bindings: imx6q-pcie: Add i.MX8Q pcie compatible string Frank Li
` (4 subsequent siblings)
13 siblings, 0 replies; 44+ messages in thread
From: Frank Li @ 2024-05-28 19:39 UTC (permalink / raw)
To: Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley
Cc: linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Frank Li
Consolidated redundant if-checks pertaining to imx_pcie->phy. Instead of
two separate checks, merged them into one to improve code readability.
if (imx_pcie->phy) {
... code 1
}
if (imx_pcie->phy) {
... code 2
}
Merge into one if block.
if (imx_pcie->phy) {
... code 1
... code 2
}
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
---
drivers/pci/controller/dwc/pci-imx6.c | 2 --
1 file changed, 2 deletions(-)
diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c
index 8ecc00049e20b..c8d58481f80dc 100644
--- a/drivers/pci/controller/dwc/pci-imx6.c
+++ b/drivers/pci/controller/dwc/pci-imx6.c
@@ -1028,9 +1028,7 @@ static int imx_pcie_host_init(struct dw_pcie_rp *pp)
dev_err(dev, "pcie PHY power up failed\n");
goto err_clk_disable;
}
- }
- if (imx_pcie->phy) {
ret = phy_power_on(imx_pcie->phy);
if (ret) {
dev_err(dev, "waiting for PHY ready timeout!\n");
--
2.34.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 44+ messages in thread* [PATCH v5 10/12] dt-bindings: imx6q-pcie: Add i.MX8Q pcie compatible string
2024-05-28 19:39 [PATCH v5 00/12] PCI: imx6: Fix\rename\clean up and add lut information for imx95 Frank Li
` (8 preceding siblings ...)
2024-05-28 19:39 ` [PATCH v5 09/12] PCI: imx6: Consolidate redundant if-checks Frank Li
@ 2024-05-28 19:39 ` Frank Li
2024-05-28 19:39 ` [PATCH v5 11/12] PCI: imx6: Call: Common PHY API to set mode, speed, and submode Frank Li
` (3 subsequent siblings)
13 siblings, 0 replies; 44+ messages in thread
From: Frank Li @ 2024-05-28 19:39 UTC (permalink / raw)
To: Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley
Cc: linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Frank Li, Conor Dooley
From: Richard Zhu <hongxing.zhu@nxp.com>
Add i.MX8Q PCIe "fsl,imx8q-pcie" compatible strings. clock-names align dwc
common naming convension.
Signed-off-by: Richard Zhu <hongxing.zhu@nxp.com>
Acked-by: Conor Dooley <conor.dooley@microchip.com>
Reviewed-by: Rob Herring (Arm) <robh@kernel.org>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
---
.../devicetree/bindings/pci/fsl,imx6q-pcie.yaml | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/Documentation/devicetree/bindings/pci/fsl,imx6q-pcie.yaml b/Documentation/devicetree/bindings/pci/fsl,imx6q-pcie.yaml
index 8b8d77b1154b5..1e05c560d7975 100644
--- a/Documentation/devicetree/bindings/pci/fsl,imx6q-pcie.yaml
+++ b/Documentation/devicetree/bindings/pci/fsl,imx6q-pcie.yaml
@@ -30,6 +30,7 @@ properties:
- fsl,imx8mm-pcie
- fsl,imx8mp-pcie
- fsl,imx95-pcie
+ - fsl,imx8q-pcie
clocks:
minItems: 3
@@ -184,6 +185,21 @@ allOf:
- const: pcie_bus
- const: pcie_aux
+ - if:
+ properties:
+ compatible:
+ enum:
+ - fsl,imx8q-pcie
+ then:
+ properties:
+ clocks:
+ maxItems: 3
+ clock-names:
+ items:
+ - const: dbi
+ - const: mstr
+ - const: slv
+
unevaluatedProperties: false
examples:
--
2.34.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 44+ messages in thread* [PATCH v5 11/12] PCI: imx6: Call: Common PHY API to set mode, speed, and submode
2024-05-28 19:39 [PATCH v5 00/12] PCI: imx6: Fix\rename\clean up and add lut information for imx95 Frank Li
` (9 preceding siblings ...)
2024-05-28 19:39 ` [PATCH v5 10/12] dt-bindings: imx6q-pcie: Add i.MX8Q pcie compatible string Frank Li
@ 2024-05-28 19:39 ` Frank Li
2024-05-28 19:39 ` [PATCH v5 12/12] PCI: imx6: Add i.MX8Q PCIe root complex (RC) support Frank Li
` (2 subsequent siblings)
13 siblings, 0 replies; 44+ messages in thread
From: Frank Li @ 2024-05-28 19:39 UTC (permalink / raw)
To: Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley
Cc: linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Frank Li
Invoke the common PHY API to configure mode, speed, and submode. While
these functions are optional in the PHY interface, they are necessary for
certain PHY drivers. Lack of support for these functions in a PHY driver
does not cause harm.
Signed-off-by: Frank Li <Frank.Li@nxp.com>
---
drivers/pci/controller/dwc/pci-imx6.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c
index c8d58481f80dc..5a725ef6ed0cb 100644
--- a/drivers/pci/controller/dwc/pci-imx6.c
+++ b/drivers/pci/controller/dwc/pci-imx6.c
@@ -29,6 +29,7 @@
#include <linux/interrupt.h>
#include <linux/reset.h>
#include <linux/phy/phy.h>
+#include <linux/phy/pcie.h>
#include <linux/pm_domain.h>
#include <linux/pm_runtime.h>
@@ -306,6 +307,10 @@ static void imx_pcie_configure_type(struct imx_pcie *imx_pcie)
id = imx_pcie->controller_id;
+ /* If mode_mask[0] is 0, means use phy driver to set mode */
+ if (!drvdata->mode_mask[0])
+ return;
+
/* If mode_mask[id] is zero, means each controller have its individual gpr */
if (!drvdata->mode_mask[id])
id = 0;
@@ -882,6 +887,7 @@ static void imx_pcie_ltssm_enable(struct device *dev)
struct imx_pcie *imx_pcie = dev_get_drvdata(dev);
const struct imx_pcie_drvdata *drvdata = imx_pcie->drvdata;
+ phy_set_speed(imx_pcie->phy, PCI_EXP_LNKCAP_SLS_2_5GB);
if (drvdata->ltssm_mask)
regmap_update_bits(imx_pcie->iomuxc_gpr, drvdata->ltssm_off, drvdata->ltssm_mask,
drvdata->ltssm_mask);
@@ -894,6 +900,7 @@ static void imx_pcie_ltssm_disable(struct device *dev)
struct imx_pcie *imx_pcie = dev_get_drvdata(dev);
const struct imx_pcie_drvdata *drvdata = imx_pcie->drvdata;
+ phy_set_speed(imx_pcie->phy, 0);
if (drvdata->ltssm_mask)
regmap_update_bits(imx_pcie->iomuxc_gpr, drvdata->ltssm_off,
drvdata->ltssm_mask, 0);
@@ -1029,6 +1036,12 @@ static int imx_pcie_host_init(struct dw_pcie_rp *pp)
goto err_clk_disable;
}
+ ret = phy_set_mode_ext(imx_pcie->phy, PHY_MODE_PCIE, PHY_MODE_PCIE_RC);
+ if (ret) {
+ dev_err(dev, "unable to set pcie PHY mode\n");
+ goto err_phy_off;
+ }
+
ret = phy_power_on(imx_pcie->phy);
if (ret) {
dev_err(dev, "waiting for PHY ready timeout!\n");
--
2.34.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 44+ messages in thread* [PATCH v5 12/12] PCI: imx6: Add i.MX8Q PCIe root complex (RC) support
2024-05-28 19:39 [PATCH v5 00/12] PCI: imx6: Fix\rename\clean up and add lut information for imx95 Frank Li
` (10 preceding siblings ...)
2024-05-28 19:39 ` [PATCH v5 11/12] PCI: imx6: Call: Common PHY API to set mode, speed, and submode Frank Li
@ 2024-05-28 19:39 ` Frank Li
2024-05-28 22:31 ` [PATCH v5 00/12] PCI: imx6: Fix\rename\clean up and add lut information for imx95 Bjorn Helgaas
2024-06-06 21:24 ` Frank Li
13 siblings, 0 replies; 44+ messages in thread
From: Frank Li @ 2024-05-28 19:39 UTC (permalink / raw)
To: Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley
Cc: linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Frank Li
From: Richard Zhu <hongxing.zhu@nxp.com>
Implement i.MX8Q (i.MX8QM, i.MX8QXP, and i.MX8DXL) PCIe RC support. While
the controller resembles that of iMX8MP, the PHY differs significantly.
Notably, there's a distinction between PCI bus addresses and CPU addresses.
Introduce IMX_PCIE_FLAG_CPU_ADDR_FIXUP in drvdata::flags to indicate driver
need the cpu_addr_fixup() callback to facilitate CPU address to PCI bus
address conversion according to "range" property.
Signed-off-by: Richard Zhu <hongxing.zhu@nxp.com>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
---
drivers/pci/controller/dwc/pci-imx6.c | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)
diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c
index 5a725ef6ed0cb..62713a0e381fc 100644
--- a/drivers/pci/controller/dwc/pci-imx6.c
+++ b/drivers/pci/controller/dwc/pci-imx6.c
@@ -81,6 +81,7 @@ enum imx_pcie_variants {
IMX8MQ,
IMX8MM,
IMX8MP,
+ IMX8Q,
IMX95,
IMX8MQ_EP,
IMX8MM_EP,
@@ -97,6 +98,7 @@ enum imx_pcie_variants {
#define IMX_PCIE_FLAG_HAS_SERDES BIT(6)
#define IMX_PCIE_FLAG_SUPPORT_64BIT BIT(7)
#define IMX_PCIE_FLAG_MONITOR_DEV BIT(8)
+#define IMX_PCIE_FLAG_CPU_ADDR_FIXUP BIT(9)
#define imx_check_flag(pci, val) (pci->drvdata->flags & val)
@@ -1086,6 +1088,22 @@ static void imx_pcie_host_exit(struct dw_pcie_rp *pp)
regulator_disable(imx_pcie->vpcie);
}
+static u64 imx_pcie_cpu_addr_fixup(struct dw_pcie *pcie, u64 cpu_addr)
+{
+ struct imx_pcie *imx_pcie = to_imx_pcie(pcie);
+ struct dw_pcie_rp *pp = &pcie->pp;
+ struct resource_entry *entry;
+ unsigned int offset;
+
+ if (!(imx_pcie->drvdata->flags & IMX_PCIE_FLAG_CPU_ADDR_FIXUP))
+ return cpu_addr;
+
+ entry = resource_list_first_type(&pp->bridge->windows, IORESOURCE_MEM);
+ offset = entry->offset;
+
+ return (cpu_addr - offset);
+}
+
static const struct dw_pcie_host_ops imx_pcie_host_ops = {
.init = imx_pcie_host_init,
.deinit = imx_pcie_host_exit,
@@ -1094,6 +1112,7 @@ static const struct dw_pcie_host_ops imx_pcie_host_ops = {
static const struct dw_pcie_ops dw_pcie_ops = {
.start_link = imx_pcie_start_link,
.stop_link = imx_pcie_stop_link,
+ .cpu_addr_fixup = imx_pcie_cpu_addr_fixup,
};
static void imx_pcie_ep_init(struct dw_pcie_ep *ep)
@@ -1593,6 +1612,13 @@ static int imx_pcie_probe(struct platform_device *pdev)
if (ret < 0)
return ret;
+ if (imx_check_flag(imx_pcie, IMX_PCIE_FLAG_CPU_ADDR_FIXUP)) {
+ if (!resource_list_first_type(&pci->pp.bridge->windows, IORESOURCE_MEM)) {
+ dw_pcie_host_deinit(&pci->pp);
+ return dev_err_probe(dev, -EINVAL, "DTS Miss PCI memory range");
+ }
+ }
+
if (pci_msi_enabled()) {
u8 offset = dw_pcie_find_capability(pci, PCI_CAP_ID_MSI);
@@ -1617,6 +1643,7 @@ static const char * const imx6q_clks[] = {"pcie_bus", "pcie", "pcie_phy"};
static const char * const imx8mm_clks[] = {"pcie_bus", "pcie", "pcie_aux"};
static const char * const imx8mq_clks[] = {"pcie_bus", "pcie", "pcie_phy", "pcie_aux"};
static const char * const imx6sx_clks[] = {"pcie_bus", "pcie", "pcie_phy", "pcie_inbound_axi"};
+static const char * const imx8q_clks[] = {"mstr", "slv", "dbi"};
static const struct imx_pcie_drvdata drvdata[] = {
[IMX6Q] = {
@@ -1720,6 +1747,13 @@ static const struct imx_pcie_drvdata drvdata[] = {
.mode_mask[0] = IMX6Q_GPR12_DEVICE_TYPE,
.set_ref_clk = imx8mm_pcie_set_ref_clk,
},
+ [IMX8Q] = {
+ .variant = IMX8Q,
+ .flags = IMX_PCIE_FLAG_HAS_PHYDRV |
+ IMX_PCIE_FLAG_CPU_ADDR_FIXUP,
+ .clk_names = imx8q_clks,
+ .clks_cnt = ARRAY_SIZE(imx8q_clks),
+ },
[IMX95] = {
.variant = IMX95,
.flags = IMX_PCIE_FLAG_HAS_SERDES |
@@ -1798,6 +1832,7 @@ static const struct of_device_id imx_pcie_of_match[] = {
{ .compatible = "fsl,imx8mq-pcie", .data = &drvdata[IMX8MQ], },
{ .compatible = "fsl,imx8mm-pcie", .data = &drvdata[IMX8MM], },
{ .compatible = "fsl,imx8mp-pcie", .data = &drvdata[IMX8MP], },
+ { .compatible = "fsl,imx8q-pcie", .data = &drvdata[IMX8Q], },
{ .compatible = "fsl,imx95-pcie", .data = &drvdata[IMX95], },
{ .compatible = "fsl,imx8mq-pcie-ep", .data = &drvdata[IMX8MQ_EP], },
{ .compatible = "fsl,imx8mm-pcie-ep", .data = &drvdata[IMX8MM_EP], },
--
2.34.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 44+ messages in thread* Re: [PATCH v5 00/12] PCI: imx6: Fix\rename\clean up and add lut information for imx95
2024-05-28 19:39 [PATCH v5 00/12] PCI: imx6: Fix\rename\clean up and add lut information for imx95 Frank Li
` (11 preceding siblings ...)
2024-05-28 19:39 ` [PATCH v5 12/12] PCI: imx6: Add i.MX8Q PCIe root complex (RC) support Frank Li
@ 2024-05-28 22:31 ` Bjorn Helgaas
2024-05-29 15:00 ` Frank Li
2024-06-06 21:24 ` Frank Li
13 siblings, 1 reply; 44+ messages in thread
From: Bjorn Helgaas @ 2024-05-28 22:31 UTC (permalink / raw)
To: Frank Li
Cc: Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley,
linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Jason Liu, Conor Dooley
On Tue, May 28, 2024 at 03:39:13PM -0400, Frank Li wrote:
> Fixed 8mp EP mode problem.
>
> imx6 actaully for all imx chips (imx6*, imx7*, imx8*, imx9*). To avoid
> confuse, rename all imx6_* to imx_*, IMX6_* to IMX_*. pci-imx6.c to
> pci-imx.c to avoid confuse.
>
> Using callback to reduce switch case for core reset and refclk.
>
> Add imx95 iommux and its stream id information.
>
> Base on linux-pci/controller/imx
This applies cleanly to the pci/controller/gpio branch, which has some
minor rework in pci-imx6.c.
When we apply this, I think we should do it on a a pci/controller/imx6
branch that is based on "main" (v6.10-rc1).
I can resolve the conflicts with pci/controller/gpio when building
pci/next.
Bjorn
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 44+ messages in thread* Re: [PATCH v5 00/12] PCI: imx6: Fix\rename\clean up and add lut information for imx95
2024-05-28 22:31 ` [PATCH v5 00/12] PCI: imx6: Fix\rename\clean up and add lut information for imx95 Bjorn Helgaas
@ 2024-05-29 15:00 ` Frank Li
2024-05-30 17:27 ` Bjorn Helgaas
0 siblings, 1 reply; 44+ messages in thread
From: Frank Li @ 2024-05-29 15:00 UTC (permalink / raw)
To: Bjorn Helgaas
Cc: Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley,
linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Jason Liu, Conor Dooley
On Tue, May 28, 2024 at 05:31:36PM -0500, Bjorn Helgaas wrote:
> On Tue, May 28, 2024 at 03:39:13PM -0400, Frank Li wrote:
> > Fixed 8mp EP mode problem.
> >
> > imx6 actaully for all imx chips (imx6*, imx7*, imx8*, imx9*). To avoid
> > confuse, rename all imx6_* to imx_*, IMX6_* to IMX_*. pci-imx6.c to
> > pci-imx.c to avoid confuse.
> >
> > Using callback to reduce switch case for core reset and refclk.
> >
> > Add imx95 iommux and its stream id information.
> >
> > Base on linux-pci/controller/imx
>
> This applies cleanly to the pci/controller/gpio branch, which has some
> minor rework in pci-imx6.c.
>
> When we apply this, I think we should do it on a a pci/controller/imx6
> branch that is based on "main" (v6.10-rc1).
>
> I can resolve the conflicts with pci/controller/gpio when building
> pci/next.
Sorry, I forget update this. It should be base on linux-pci/next
(e3fca37312892122d73f8c5293c0d1cc8c34500b).
commit e3fca37312892122d73f8c5293c0d1cc8c34500b (pci/next, linux-pci/next)
Merge: 86e0cd3da71b5 d19a86d584e04
Author: Bjorn Helgaas <bhelgaas@google.com>
Date: Tue May 28 12:34:12 2024 -0500
Merge branch 'pci/controller/tegra194'
- Ensure Tegra194 and Tegra234 inbound ATU entries are 64KB-aligned to
match the hardware restriction (Jon Hunter)
* pci/controller/tegra194:
PCI: tegra194: Set EP alignment restriction for inbound ATU
>
> Bjorn
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 44+ messages in thread* Re: [PATCH v5 00/12] PCI: imx6: Fix\rename\clean up and add lut information for imx95
2024-05-29 15:00 ` Frank Li
@ 2024-05-30 17:27 ` Bjorn Helgaas
2024-05-30 17:56 ` Frank Li
0 siblings, 1 reply; 44+ messages in thread
From: Bjorn Helgaas @ 2024-05-30 17:27 UTC (permalink / raw)
To: Frank Li
Cc: Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley,
linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Jason Liu, Conor Dooley
On Wed, May 29, 2024 at 11:00:49AM -0400, Frank Li wrote:
> On Tue, May 28, 2024 at 05:31:36PM -0500, Bjorn Helgaas wrote:
> > On Tue, May 28, 2024 at 03:39:13PM -0400, Frank Li wrote:
> ...
> > > Base on linux-pci/controller/imx
> >
> > This applies cleanly to the pci/controller/gpio branch, which has some
> > minor rework in pci-imx6.c.
> >
> > When we apply this, I think we should do it on a a pci/controller/imx6
> > branch that is based on "main" (v6.10-rc1).
> >
> > I can resolve the conflicts with pci/controller/gpio when building
> > pci/next.
>
> Sorry, I forget update this. It should be base on linux-pci/next
> (e3fca37312892122d73f8c5293c0d1cc8c34500b).
I prefer patches that are based on -rc1, i.e., the pci/main branch,
not on the pci/next branch.
If a series *requires* functionality that is already on a topic
branch, you can base it on that branch instead of on pci/main.
This series happens to touch some of the same code as
pci/controller/gpio, but it doesn't require those gpio changes, so it
does not need to be based on pci/controller/gpio.
Having this series based on pci/main means that if we update or drop
the gpio branch for some reason, this series will still make sense.
Bjorn
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v5 00/12] PCI: imx6: Fix\rename\clean up and add lut information for imx95
2024-05-30 17:27 ` Bjorn Helgaas
@ 2024-05-30 17:56 ` Frank Li
0 siblings, 0 replies; 44+ messages in thread
From: Frank Li @ 2024-05-30 17:56 UTC (permalink / raw)
To: Bjorn Helgaas
Cc: Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley,
linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Jason Liu, Conor Dooley
On Thu, May 30, 2024 at 12:27:49PM -0500, Bjorn Helgaas wrote:
> On Wed, May 29, 2024 at 11:00:49AM -0400, Frank Li wrote:
> > On Tue, May 28, 2024 at 05:31:36PM -0500, Bjorn Helgaas wrote:
> > > On Tue, May 28, 2024 at 03:39:13PM -0400, Frank Li wrote:
> > ...
>
> > > > Base on linux-pci/controller/imx
> > >
> > > This applies cleanly to the pci/controller/gpio branch, which has some
> > > minor rework in pci-imx6.c.
> > >
> > > When we apply this, I think we should do it on a a pci/controller/imx6
> > > branch that is based on "main" (v6.10-rc1).
> > >
> > > I can resolve the conflicts with pci/controller/gpio when building
> > > pci/next.
> >
> > Sorry, I forget update this. It should be base on linux-pci/next
> > (e3fca37312892122d73f8c5293c0d1cc8c34500b).
>
> I prefer patches that are based on -rc1, i.e., the pci/main branch,
> not on the pci/next branch.
>
> If a series *requires* functionality that is already on a topic
> branch, you can base it on that branch instead of on pci/main.
>
> This series happens to touch some of the same code as
> pci/controller/gpio, but it doesn't require those gpio changes, so it
> does not need to be based on pci/controller/gpio.
>
> Having this series based on pci/main means that if we update or drop
> the gpio branch for some reason, this series will still make sense.
v4 should cleanly apply to pci/main. It is not big deal. Do you have any
additional comments for this series?
We still have some other works (such as simply suspend/resume by using
common PME turn off method), which base on this series.
Frank
>
> Bjorn
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v5 00/12] PCI: imx6: Fix\rename\clean up and add lut information for imx95
2024-05-28 19:39 [PATCH v5 00/12] PCI: imx6: Fix\rename\clean up and add lut information for imx95 Frank Li
` (12 preceding siblings ...)
2024-05-28 22:31 ` [PATCH v5 00/12] PCI: imx6: Fix\rename\clean up and add lut information for imx95 Bjorn Helgaas
@ 2024-06-06 21:24 ` Frank Li
13 siblings, 0 replies; 44+ messages in thread
From: Frank Li @ 2024-06-06 21:24 UTC (permalink / raw)
To: Richard Zhu, Lucas Stach, Lorenzo Pieralisi,
Krzysztof Wilczyński, Rob Herring, Bjorn Helgaas, Shawn Guo,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
NXP Linux Team, Philipp Zabel, Liam Girdwood, Mark Brown,
Manivannan Sadhasivam, Krzysztof Kozlowski, Conor Dooley
Cc: linux-pci, imx, linux-arm-kernel, linux-kernel, bpf, devicetree,
Jason Liu, Conor Dooley
On Tue, May 28, 2024 at 03:39:13PM -0400, Frank Li wrote:
> Fixed 8mp EP mode problem.
hi Manivannan:
Do you have any additional comments except for the patch
Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95?
Frank Li
>
> imx6 actaully for all imx chips (imx6*, imx7*, imx8*, imx9*). To avoid
> confuse, rename all imx6_* to imx_*, IMX6_* to IMX_*. pci-imx6.c to
> pci-imx.c to avoid confuse.
>
> Using callback to reduce switch case for core reset and refclk.
>
> Add imx95 iommux and its stream id information.
>
> Base on linux-pci/controller/imx
>
> To: Richard Zhu <hongxing.zhu@nxp.com>
> To: Lucas Stach <l.stach@pengutronix.de>
> To: Lorenzo Pieralisi <lpieralisi@kernel.org>
> To: Krzysztof Wilczyński <kw@linux.com>
> To: Rob Herring <robh@kernel.org>
> To: Bjorn Helgaas <bhelgaas@google.com>
> To: Shawn Guo <shawnguo@kernel.org>
> To: Sascha Hauer <s.hauer@pengutronix.de>
> To: Pengutronix Kernel Team <kernel@pengutronix.de>
> To: Fabio Estevam <festevam@gmail.com>
> To: NXP Linux Team <linux-imx@nxp.com>
> To: Philipp Zabel <p.zabel@pengutronix.de>
> To: Liam Girdwood <lgirdwood@gmail.com>
> To: Mark Brown <broonie@kernel.org>
> To: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> To: Krzysztof Kozlowski <krzysztof.kozlowski+dt@linaro.org>
> To: Conor Dooley <conor+dt@kernel.org>
> Cc: linux-pci@vger.kernel.org
> Cc: imx@lists.linux.dev
> Cc: linux-arm-kernel@lists.infradead.org
> Cc: linux-kernel@vger.kernel.org
> Cc: bpf@vger.kernel.org
> Cc: devicetree@vger.kernel.org
> Signed-off-by: Frank Li <Frank.Li@nxp.com>
>
> Changes in v5:
> - Rebase to linux-pci next. fix conflict with gpiod change
> - Add rob and cornor's review tag
> - Link to v4: https://lore.kernel.org/r/20240507-pci2_upstream-v4-0-e8c80d874057@nxp.com
>
> Changes in v4:
> - Improve comment message for patch 1 and 2.
> - Rework commit message for patch 3 and add mani's review tag
> - Remove file rename patch and update maintainer patch
> - [PATCH v3 06/11] PCI: imx: Simplify switch-case logic by involve set_ref_clk callback
> remove extra space.
> keep original comments format (wrap at 80 column width)
> update error message "'Failed to enable PCIe REFCLK'"
> - PATCH v3 07/11] PCI: imx: Simplify switch-case logic by involve core_reset callback
> keep exact the logic as original code
> - Add patch to update comment about workaround ERR010728
> - Add patch about help function imx_pcie_match_device()
> - Using bus device notify to update LUT information for imx95 to avoid
> parse iommu-map and msi-map in driver code. Bus notify will better and
> only update lut when device added.
> - split patch call PHY interface function.
> - Improve commit message for imx8q. remove local-address dts proptery. and
> use standard "range" to convert cpu address to bus address.
> - Check entry in cpu_fix function is too late. Check it at probe
> - Link to v3: https://lore.kernel.org/r/20240402-pci2_upstream-v3-0-803414bdb430@nxp.com
>
> Changes in v3:
> - Add an EP fixed patch
> PCI: imx6: Fix PCIe link down when i.MX8MM and i.MX8MP PCIe is EP mode
> PCI: imx6: Fix i.MX8MP PCIe EP can not trigger MSI
> - Add 8qxp rc support
> dt-bing yaml pass binding check
> make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j8 dt_binding_check DT_SCHEMA_FILES=fsl,imx6q-pcie.yaml
> LINT Documentation/devicetree/bindings
> DTEX Documentation/devicetree/bindings/pci/fsl,imx6q-pcie.example.dts
> CHKDT Documentation/devicetree/bindings/processed-schema.json
> SCHEMA Documentation/devicetree/bindings/processed-schema.json
> DTC_CHK Documentation/devicetree/bindings/pci/fsl,imx6q-pcie.example.dtb
>
> - Link to v2: https://lore.kernel.org/r/20240304-pci2_upstream-v2-0-ad07c5eb6d67@nxp.com
>
> Changes in v2:
> - remove file to 'pcie-imx.c'
> - keep CONFIG unchange.
> - Link to v1: https://lore.kernel.org/r/20240227-pci2_upstream-v1-0-b952f8333606@nxp.com
>
> ---
> Frank Li (8):
> PCI: imx6: Rename imx6_* with imx_*
> PCI: imx6: Introduce SoC specific callbacks for controlling REFCLK
> PCI: imx6: Simplify switch-case logic by involve core_reset callback
> PCI: imx6: Improve comment for workaround ERR010728
> PCI: imx6: Add help function imx_pcie_match_device()
> PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95
> PCI: imx6: Consolidate redundant if-checks
> PCI: imx6: Call: Common PHY API to set mode, speed, and submode
>
> Richard Zhu (4):
> PCI: imx6: Fix establish link failure in EP mode for iMX8MM and iMX8MP
> PCI: imx6: Fix i.MX8MP PCIe EP's occasional failure to trigger MSI
> dt-bindings: imx6q-pcie: Add i.MX8Q pcie compatible string
> PCI: imx6: Add i.MX8Q PCIe root complex (RC) support
>
> .../devicetree/bindings/pci/fsl,imx6q-pcie.yaml | 16 +
> drivers/pci/controller/dwc/pci-imx6.c | 1181 ++++++++++++--------
> 2 files changed, 730 insertions(+), 467 deletions(-)
> ---
> base-commit: 50d96936b7b1be01bcc7b9ff898191214ee72697
> change-id: 20240227-pci2_upstream-0cdd19a15163
>
> Best regards,
> ---
> Frank Li <Frank.Li@nxp.com>
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 44+ messages in thread