From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4AC2C433E1 for ; Wed, 24 Mar 2021 13:16:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7B0A261A13 for ; Wed, 24 Mar 2021 13:16:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234762AbhCXNQC (ORCPT ); Wed, 24 Mar 2021 09:16:02 -0400 Received: from mail.kernel.org ([198.145.29.99]:44522 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234750AbhCXNPn (ORCPT ); Wed, 24 Mar 2021 09:15:43 -0400 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 51C1060C3D; Wed, 24 Mar 2021 13:15:42 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from ) id 1lP3MK-003Wd7-2d; Wed, 24 Mar 2021 13:15:40 +0000 Date: Wed, 24 Mar 2021 13:15:38 +0000 Message-ID: <877dlwk805.wl-maz@kernel.org> From: Marc Zyngier To: Bharat Kumar Gogada Cc: "lorenzo.pieralisi@arm.com" , Bjorn Helgaas , Frank Wunderlich , Thierry Reding , Thomas Gleixner , Rob Herring , Will Deacon , "K. Y. Srinivasan" , Haiyang Zhang , Stephen Hemminger , Michael Kelley , Wei Liu , Thierry Reding , Jonathan Hunter , Ryder Lee , Marek Vasut , Yoshihiro Shimoda , Michal Simek , Paul Walmsley , "linux-pci@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "linux-arm-kernel@lists.infradead.org" , "linux-hyperv@vger.kernel.org" , "linux-tegra@vger.kernel.org" , "linux-mediatek@lists.infradead.org" , "linux-renesas-soc@vger.kernel.org" , "kernel-team@android.com" Subject: Re: [PATCH v2 05/15] PCI: xilinx: Convert to MSI domains In-Reply-To: References: <20210322184614.802565-1-maz@kernel.org> <20210322184614.802565-6-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: bharatku@xilinx.com, lorenzo.pieralisi@arm.com, bhelgaas@google.com, frank-w@public-files.de, treding@nvidia.com, tglx@linutronix.de, robh@kernel.org, will@kernel.org, kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, mikelley@microsoft.com, wei.liu@kernel.org, thierry.reding@gmail.com, jonathanh@nvidia.com, ryder.lee@mediatek.com, marek.vasut+renesas@gmail.com, yoshihiro.shimoda.uh@renesas.com, michals@xilinx.com, paul.walmsley@sifive.com, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-hyperv@vger.kernel.org, linux-tegra@vger.kernel.org, linux-mediatek@lists.infradead.org, linux-renesas-soc@vger.kernel.org, kernel-team@android.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org On Wed, 24 Mar 2021 12:42:24 +0000, Bharat Kumar Gogada wrote: > > Hi Marc, > > Thanks for the patch. > > > Subject: [PATCH v2 05/15] PCI: xilinx: Convert to MSI domains > > > > In anticipation of the removal of the msi_controller structure, convert the > > ancient xilinx host controller driver to MSI domains. > > > > We end-up with the usual two domain structure, the top one being a generic > > PCI/MSI domain, the bottom one being xilinx-specific and handling the > > actual HW interrupt allocation. > > > > This allows us to fix some of the most appalling MSI programming, where the > > message programmed in the device is the virtual IRQ number instead of the > > allocated vector number. The allocator is also made safe with a mutex. This > > should allow support for MultiMSI, but I decided not to even try, since I > > cannot test it. > > > > Acked-by: Bjorn Helgaas > > Signed-off-by: Marc Zyngier > > --- > > drivers/pci/controller/Kconfig | 2 +- > > drivers/pci/controller/pcie-xilinx.c | 234 +++++++++++---------------- > > 2 files changed, 97 insertions(+), 139 deletions(-) > > > > diff --git a/drivers/pci/controller/Kconfig b/drivers/pci/controller/Kconfig > > index 5cc07d28a3a0..60045f7aafc5 100644 > ... > > > > +static struct irq_chip xilinx_msi_bottom_chip = { > > + .name = "Xilinx MSI", > > + .irq_set_affinity = xilinx_msi_set_affinity, > > + .irq_compose_msi_msg = xilinx_compose_msi_msg, > > +}; > > > I see a crash while testing MSI in handle_edge_irq > [] (handle_edge_irq) from [] (generic_handle_irq+0x28/0x38) > [] (generic_handle_irq) from [] (xilinx_pcie_intr_handler+0x17c/0x2b0) > [] (xilinx_pcie_intr_handler) from [] (__handle_irq_event_percpu+0x3c/0xc0) > [] (__handle_irq_event_percpu) from [] (handle_irq_event_percpu+0x2c/0x7c) > [] (handle_irq_event_percpu) from [] (handle_irq_event+0x38/0x5c) > [] (handle_irq_event) from [] (handle_fasteoi_irq+0x9c/0x114) Thanks for that. Can you please try the following patch and let me know if it helps? Thanks, M. diff --git a/drivers/pci/controller/pcie-xilinx.c b/drivers/pci/controller/pcie-xilinx.c index ad9abf405167..14001febf59a 100644 --- a/drivers/pci/controller/pcie-xilinx.c +++ b/drivers/pci/controller/pcie-xilinx.c @@ -194,8 +194,18 @@ static struct pci_ops xilinx_pcie_ops = { /* MSI functions */ +static void xilinx_msi_top_irq_ack(struct irq_data *d) +{ + /* + * xilinx_pcie_intr_handler() will have performed the Ack. + * Eventually, this should be fixed and the Ack be moved in + * the respective callbacks for INTx and MSI. + */ +} + static struct irq_chip xilinx_msi_top_chip = { .name = "PCIe MSI", + .irq_ack = xilinx_msi_top_irq_ack, }; static int xilinx_msi_set_affinity(struct irq_data *d, const struct cpumask *mask, bool force) @@ -206,7 +216,7 @@ static int xilinx_msi_set_affinity(struct irq_data *d, const struct cpumask *mas static void xilinx_compose_msi_msg(struct irq_data *data, struct msi_msg *msg) { struct xilinx_pcie_port *pcie = irq_data_get_irq_chip_data(data); - phys_addr_t pa = virt_to_phys(pcie); + phys_addr_t pa = ALIGN_DOWN(virt_to_phys(pcie), SZ_4K); msg->address_lo = lower_32_bits(pa); msg->address_hi = upper_32_bits(pa); @@ -468,7 +478,7 @@ static int xilinx_pcie_init_irq_domain(struct xilinx_pcie_port *port) /* Setup MSI */ if (IS_ENABLED(CONFIG_PCI_MSI)) { - phys_addr_t pa = virt_to_phys(port); + phys_addr_t pa = ALIGN_DOWN(virt_to_phys(port), SZ_4K); ret = xilinx_allocate_msi_domains(port); if (ret) -- Without deviation from the norm, progress is not possible.