From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0240AC61DA4 for ; Thu, 2 Feb 2023 11:45:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ELGS2apq+F3YB7zc3DvmP+Io+ypoOsKNi6+/ZiaapbQ=; b=j9QaNU+QGjjijg HNREYYdY7H2sZ15Xd0VzmfRhujlC88V3dQL0/sYkkeUyPnZbbx+yfLZKyKFAWiB7NbrGmEz8LTkKE 6Qk4xYz7b1ZDFr1FTyRN3xDOQxhiIXu4qPlhHU1OxPx+JsdFd5sqrSaMsOgbsRghLnjHXV6ImJjC9 031ugxQT0E1gKDfuME90LsBnxv7FsVnm/gSOv7tEKPoO5EKjjKBxqIFw532Hf/AHYp21OSmwKy8Jz 5GLzlyTDJmNp+uBZIUyrM5o4x5CVe+B28xiCmaYopoEolym7dTLlXf9gfLjKaEg3rDPbmMoVNZ21j iOr7JnWzoKwzb40KcAag==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pNY28-00FeTS-Tt; Thu, 02 Feb 2023 11:45:40 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pNY25-00FeSe-Ju for linux-riscv@lists.infradead.org; Thu, 02 Feb 2023 11:45:39 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id CB16561A9C; Thu, 2 Feb 2023 11:45:36 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3B7CBC433D2; Thu, 2 Feb 2023 11:45:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1675338336; bh=m87/DE9fpIUi8cZc8zFaS3+uraBQbfVmUU3Jv/n9q3s=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=rPxSt+Dqpis7qHMsDQaD7dOjyTgkbczeo3rPxrPIgpnJn9uFmqhF7tU8uAoV0ErJg /hWdiHl4mQ1Hky+a1k4ewQdPm6vyXE7Ogpq0YDrd49UGrQgRRTzcSsDzBH6LF3yeeN 81eNtVruaN7b5vmLqcUGmR4Xt2qcGAiyS6ALtyvUxmFlDnA/WQNosMdDwIzPpMUUvb FjMFxKK6bgK2X8llRTBuT6AmMO1+fKi5WBxU4U0/j657SwwIqXp9z+tdJs91ZgQ31X xSzynawvGJGgHspKSV+2C17zggK3ACwLpy+wYAHRkjs5I/btP1kaf+Z1TwnHKJ3Vgb 4TdTITgnhs+cA== Date: Thu, 2 Feb 2023 12:45:30 +0100 From: Lorenzo Pieralisi To: daire.mcnamara@microchip.com Cc: conor.dooley@microchip.com, robh+dt@kernel.org, krzysztof.kozlowski+dt@linaro.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, kw@linux.com, bhelgaas@google.com, linux-riscv@lists.infradead.org, devicetree@vger.kernel.org, linux-pci@vger.kernel.org Subject: Re: [PATCH v3 07/11] PCI: microchip: Gather MSI information from hardware config registers Message-ID: References: <20230111125323.1911373-1-daire.mcnamara@microchip.com> <20230111125323.1911373-8-daire.mcnamara@microchip.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20230111125323.1911373-8-daire.mcnamara@microchip.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230202_034537_758483_A415786F X-CRM114-Status: GOOD ( 27.60 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org On Wed, Jan 11, 2023 at 12:53:19PM +0000, daire.mcnamara@microchip.com wrote: > From: Daire McNamara > > The PCIe Root Complex on PolarFire SoC is configured at bitstream creation > time using Libero. Key MSI-related parameters include the number of > MSIs (1/2/4/8/16/32) and the MSI address. In the device driver, extract > this information from hw registers at init time, and use it to configure > MSI system, including configuring MSI capability structure correctly in > configuration space. I don't understand whether this is backward compatible with all existing host controllers in the field. I assume MSI parameters can always been probed and that has been always the case for all controllers supported based on this design, please confirm before I proceed with the review. Thanks, Lorenzo > Signed-off-by: Daire McNamara > Reviewed-by: Conor Dooley > --- > drivers/pci/controller/pcie-microchip-host.c | 73 +++++++++++--------- > 1 file changed, 40 insertions(+), 33 deletions(-) > > diff --git a/drivers/pci/controller/pcie-microchip-host.c b/drivers/pci/controller/pcie-microchip-host.c > index 751f0243deb4..9ff0fb04b953 100644 > --- a/drivers/pci/controller/pcie-microchip-host.c > +++ b/drivers/pci/controller/pcie-microchip-host.c > @@ -20,8 +20,7 @@ > #include "../pci.h" > > /* Number of MSI IRQs */ > -#define MC_NUM_MSI_IRQS 32 > -#define MC_NUM_MSI_IRQS_CODED 5 > +#define MC_MAX_NUM_MSI_IRQS 32 > > /* PCIe Bridge Phy and Controller Phy offsets */ > #define MC_PCIE1_BRIDGE_ADDR 0x00008000u > @@ -31,6 +30,11 @@ > #define MC_PCIE_CTRL_ADDR (MC_PCIE1_CTRL_ADDR) > > /* PCIe Bridge Phy Regs */ > +#define PCIE_PCI_IRQ_DW0 0xa8 > +#define MSIX_CAP_MASK BIT(31) > +#define NUM_MSI_MSGS_MASK GENMASK(6, 4) > +#define NUM_MSI_MSGS_SHIFT 4 > + > #define IMASK_LOCAL 0x180 > #define DMA_END_ENGINE_0_MASK 0x00000000u > #define DMA_END_ENGINE_0_SHIFT 0 > @@ -79,7 +83,6 @@ > #define IMASK_HOST 0x188 > #define ISTATUS_HOST 0x18c > #define IMSI_ADDR 0x190 > -#define MSI_ADDR 0x190 > #define ISTATUS_MSI 0x194 > > /* PCIe Master table init defines */ > @@ -158,8 +161,6 @@ > > /* PCIe Config space MSI capability structure */ > #define MC_MSI_CAP_CTRL_OFFSET 0xe0u > -#define MC_MSI_MAX_Q_AVAIL (MC_NUM_MSI_IRQS_CODED << 1) > -#define MC_MSI_Q_SIZE (MC_NUM_MSI_IRQS_CODED << 4) > > /* Events */ > #define EVENT_PCIE_L2_EXIT 0 > @@ -259,7 +260,7 @@ struct mc_msi { > struct irq_domain *dev_domain; > u32 num_vectors; > u64 vector_phy; > - DECLARE_BITMAP(used, MC_NUM_MSI_IRQS); > + DECLARE_BITMAP(used, MC_MAX_NUM_MSI_IRQS); > }; > > struct mc_pcie { > @@ -382,25 +383,29 @@ static struct { > > static char poss_clks[][5] = { "fic0", "fic1", "fic2", "fic3" }; > > -static void mc_pcie_enable_msi(struct mc_pcie *port, void __iomem *base) > +static void mc_pcie_fixup_ecam(struct mc_pcie *port, void __iomem *ecam) > { > struct mc_msi *msi = &port->msi; > - u32 cap_offset = MC_MSI_CAP_CTRL_OFFSET; > - u16 msg_ctrl = readw_relaxed(base + cap_offset + PCI_MSI_FLAGS); > - > - msg_ctrl |= PCI_MSI_FLAGS_ENABLE; > - msg_ctrl &= ~PCI_MSI_FLAGS_QMASK; > - msg_ctrl |= MC_MSI_MAX_Q_AVAIL; > - msg_ctrl &= ~PCI_MSI_FLAGS_QSIZE; > - msg_ctrl |= MC_MSI_Q_SIZE; > - msg_ctrl |= PCI_MSI_FLAGS_64BIT; > - > - writew_relaxed(msg_ctrl, base + cap_offset + PCI_MSI_FLAGS); > - > + u16 reg; > + u8 queue_size; > + > + /* Fixup MSI enable flag */ > + reg = readw_relaxed(ecam + MC_MSI_CAP_CTRL_OFFSET + PCI_MSI_FLAGS); > + reg |= PCI_MSI_FLAGS_ENABLE; > + writew_relaxed(reg, ecam + MC_MSI_CAP_CTRL_OFFSET + PCI_MSI_FLAGS); > + > + /* Fixup PCI MSI queue flags */ > + queue_size = reg & PCI_MSI_FLAGS_QMASK; > + queue_size >>= 1; > + reg &= ~PCI_MSI_FLAGS_QSIZE; > + reg |= queue_size << 4; > + writew_relaxed(reg, ecam + MC_MSI_CAP_CTRL_OFFSET + PCI_MSI_FLAGS); > + > + /* Fixup MSI addr fields */ > writel_relaxed(lower_32_bits(msi->vector_phy), > - base + cap_offset + PCI_MSI_ADDRESS_LO); > + ecam + MC_MSI_CAP_CTRL_OFFSET + PCI_MSI_ADDRESS_LO); > writel_relaxed(upper_32_bits(msi->vector_phy), > - base + cap_offset + PCI_MSI_ADDRESS_HI); > + ecam + MC_MSI_CAP_CTRL_OFFSET + PCI_MSI_ADDRESS_HI); > } > > static void mc_handle_msi(struct irq_desc *desc) > @@ -473,10 +478,7 @@ static int mc_irq_msi_domain_alloc(struct irq_domain *domain, unsigned int virq, > { > struct mc_pcie *port = domain->host_data; > struct mc_msi *msi = &port->msi; > - void __iomem *bridge_base_addr = > - port->axi_base_addr + MC_PCIE_BRIDGE_ADDR; > unsigned long bit; > - u32 val; > > mutex_lock(&msi->lock); > bit = find_first_zero_bit(msi->used, msi->num_vectors); > @@ -490,11 +492,6 @@ static int mc_irq_msi_domain_alloc(struct irq_domain *domain, unsigned int virq, > irq_domain_set_info(domain, virq, bit, &mc_msi_bottom_irq_chip, > domain->host_data, handle_edge_irq, NULL, NULL); > > - /* Enable MSI interrupts */ > - val = readl_relaxed(bridge_base_addr + IMASK_LOCAL); > - val |= PM_MSI_INT_MSI_MASK; > - writel_relaxed(val, bridge_base_addr + IMASK_LOCAL); > - > mutex_unlock(&msi->lock); > > return 0; > @@ -1117,6 +1114,7 @@ static int mc_platform_init(struct pci_config_window *cfg) > struct mc_pcie *port; > void __iomem *bridge_base_addr; > int ret; > + u32 val; > > port = devm_kzalloc(dev, sizeof(*port), GFP_KERNEL); > if (!port) > @@ -1137,11 +1135,20 @@ static int mc_platform_init(struct pci_config_window *cfg) > > bridge_base_addr = port->axi_base_addr + MC_PCIE_BRIDGE_ADDR; > > - port->msi.vector_phy = MSI_ADDR; > - port->msi.num_vectors = MC_NUM_MSI_IRQS; > + /* Allow enabling MSI by disabling MSI-X */ > + val = readl(bridge_base_addr + PCIE_PCI_IRQ_DW0); > + val &= ~MSIX_CAP_MASK; > + writel(val, bridge_base_addr + PCIE_PCI_IRQ_DW0); > + > + /* Pick num vectors from bitfile programmed onto FPGA fabric */ > + val = readl(bridge_base_addr + PCIE_PCI_IRQ_DW0); > + val &= NUM_MSI_MSGS_MASK; > + val >>= NUM_MSI_MSGS_SHIFT; > + > + port->msi.num_vectors = 1 << val; > > - /* Hardware doesn't setup MSI by default */ > - mc_pcie_enable_msi(port, cfg->win); > + /* Pick vector address from design */ > + port->msi.vector_phy = readl_relaxed(bridge_base_addr + IMSI_ADDR); > > /* Configure Address Translation Table 0 for PCIe config space */ > mc_pcie_setup_window(bridge_base_addr, 0, cfg->res.start & 0xffffffff, > -- > 2.25.1 > > > _______________________________________________ > linux-riscv mailing list > linux-riscv@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-riscv _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv