Re: [PATCH v5] PCI: keystone: Fix pci_ops for AM654x SoC

[Niklas Cassel <cassel@kernel.org>]

On Tue, Mar 26, 2024 at 04:49:05PM +0530, Siddharth Vadapalli wrote:
> In the process of converting .scan_bus() callbacks to .add_bus(), the
> ks_pcie_v3_65_scan_bus() function was changed to ks_pcie_v3_65_add_bus().
> The .scan_bus() method belonged to ks_pcie_host_ops which was specific
> to controller version 3.65a, while the .add_bus() method had been added
> to ks_pcie_ops which is shared between the controller versions 3.65a and
> 4.90a. Neither the older ks_pcie_v3_65_scan_bus() method, nor the newer
> ks_pcie_v3_65_add_bus() method are applicable to the controller version
> 4.90a which is present in AM654x SoCs.
> 
> Thus, as a fix, move the contents of "ks_pcie_v3_65_add_bus()" to the
> .msi_init callback "ks_pcie_msi_host_init()" which is specific to the
> 3.65a controller. Also, move the definitions of ks_pcie_set_dbi_mode()
> and ks_pcie_clear_dbi_mode() above ks_pcie_msi_host_init() in order to
> avoid forward declaration.
> 
> Fixes: 6ab15b5e7057 ("PCI: dwc: keystone: Convert .scan_bus() callback to use add_bus")
> Suggested-by: Serge Semin <fancer.lancer@gmail.com>
> Suggested-by: Bjorn Helgaas <helgaas@kernel.org>
> Suggested-by: Niklas Cassel <cassel@kernel.org>
> Signed-off-by: Siddharth Vadapalli <s-vadapalli@ti.com>
> ---
> 
> Hello,
> 
> This patch is based on linux-next tagged next-20240326.
> 
> v4:
> https://lore.kernel.org/r/20240325053722.1955433-1-s-vadapalli@ti.com/
> Changes since v4:
> - As suggested by Niklas Cassel <cassel@kernel.org> at:
>   https://lore.kernel.org/r/ZgF_5fYsI5lOFjOv@ryzen/
>   the contents of "ks_pcie_v3_65_add_bus()" have been moved to
>   "ks_pcie_msi_host_init()" instead of "ks_pcie_host_init()". This
>   avoids unnecessary checks for "!ks_pcie->is_am6" since
>   "ks_pcie_msi_host_init()" is specific to the v3.65a controller version
>   which corresponds to "!ks_pcie->is_am6".
> - Updated commit message to match the change in implementation.
> - Added "Suggested-by" tag of Niklas Cassel <cassel@kernel.org> based on:
>   https://lore.kernel.org/r/ZgKaNrhoReJ0A525@x1-carbon/
> - Moved the definitions for ks_pcie_set_dbi_mode() and
>   ks_pcie_clear_dbi_mode() above ks_pcie_msi_host_init().
> 
> Regards,
> Siddharth.
> 
>  drivers/pci/controller/dwc/pci-keystone.c | 136 ++++++++++------------
>  1 file changed, 60 insertions(+), 76 deletions(-)
> 
> diff --git a/drivers/pci/controller/dwc/pci-keystone.c b/drivers/pci/controller/dwc/pci-keystone.c
> index 844de4418724..c2252448d9e8 100644
> --- a/drivers/pci/controller/dwc/pci-keystone.c
> +++ b/drivers/pci/controller/dwc/pci-keystone.c
> @@ -245,8 +245,68 @@ static struct irq_chip ks_pcie_msi_irq_chip = {
>  	.irq_unmask = ks_pcie_msi_unmask,
>  };
>  
> +/**
> + * ks_pcie_set_dbi_mode() - Set DBI mode to access overlaid BAR mask registers
> + * @ks_pcie: A pointer to the keystone_pcie structure which holds the KeyStone
> + *	     PCIe host controller driver information.
> + *
> + * Since modification of dbi_cs2 involves different clock domain, read the
> + * status back to ensure the transition is complete.
> + */
> +static void ks_pcie_set_dbi_mode(struct keystone_pcie *ks_pcie)
> +{
> +	u32 val;
> +
> +	val = ks_pcie_app_readl(ks_pcie, CMD_STATUS);
> +	val |= DBI_CS2;
> +	ks_pcie_app_writel(ks_pcie, CMD_STATUS, val);
> +
> +	do {
> +		val = ks_pcie_app_readl(ks_pcie, CMD_STATUS);
> +	} while (!(val & DBI_CS2));
> +}
> +
> +/**
> + * ks_pcie_clear_dbi_mode() - Disable DBI mode
> + * @ks_pcie: A pointer to the keystone_pcie structure which holds the KeyStone
> + *	     PCIe host controller driver information.
> + *
> + * Since modification of dbi_cs2 involves different clock domain, read the
> + * status back to ensure the transition is complete.
> + */
> +static void ks_pcie_clear_dbi_mode(struct keystone_pcie *ks_pcie)
> +{
> +	u32 val;
> +
> +	val = ks_pcie_app_readl(ks_pcie, CMD_STATUS);
> +	val &= ~DBI_CS2;
> +	ks_pcie_app_writel(ks_pcie, CMD_STATUS, val);
> +
> +	do {
> +		val = ks_pcie_app_readl(ks_pcie, CMD_STATUS);
> +	} while (val & DBI_CS2);
> +}
> +
>  static int ks_pcie_msi_host_init(struct dw_pcie_rp *pp)
>  {
> +	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
> +	struct keystone_pcie *ks_pcie = to_keystone_pcie(pci);
> +
> +	/* Configure and set up BAR0 */
> +	ks_pcie_set_dbi_mode(ks_pcie);
> +
> +	/* Enable BAR0 */
> +	dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, 1);
> +	dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, SZ_4K - 1);
> +
> +	ks_pcie_clear_dbi_mode(ks_pcie);
> +
> +	 /*
> +	  * For BAR0, just setting bus address for inbound writes (MSI) should
> +	  * be sufficient.  Use physical address to avoid any conflicts.
> +	  */

This comment seems to have wrong indentation.
With that fixed:

Reviewed-by: Niklas Cassel <cassel@kernel.org>

> +	dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, ks_pcie->app.start);
> +
>  	pp->msi_irq_chip = &ks_pcie_msi_irq_chip;
>  	return dw_pcie_allocate_domains(pp);
>  }