From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1DECDC25B75 for ; Tue, 14 May 2024 12:12:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:CC:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Nq/wS/WfwX2LNjF+jNCLH9sQiQUXfJ+Nz0LbkFgkYkk=; b=tUKk0mTfZ7Y+Wa NskWS5YjkZ0ags5n+lX+NYzU4ZmpAXTsU978ShUJupX5jkcnTOyB86YE4O8/Z5wLsElFLpXAojdTd vHHGdZ+FXakOaFIPbUKRoYnVFB2lG50OkYILtxZiluZIxbgZjhm+11uYumJkQonB7zPuiI4r9HNgY ZC2CNxZkqGSok7XGgPCsgHQCD7pSoxdlbXLR4nIAR50UJge3/MzHXT+NzY21hHx5jUeUmJX9p2Djb +fq9YRM6/5adZ7Mb3vAxkMAaYfEiaCsyBSWJ2gawU1LO/99+TtvNzhj6V6ifUHRLSOSZ59DQpRktE xDAwLmdFtSbsbijg4v4Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s6r0y-0000000Fpz6-3Udt; Tue, 14 May 2024 12:12:16 +0000 Received: from lelv0142.ext.ti.com ([198.47.23.249]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s6r0t-0000000Fpxw-2wB8 for linux-arm-kernel@lists.infradead.org; Tue, 14 May 2024 12:12:14 +0000 Received: from fllv0035.itg.ti.com ([10.64.41.0]) by lelv0142.ext.ti.com (8.15.2/8.15.2) with ESMTP id 44ECBo3G115948; Tue, 14 May 2024 07:11:50 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ti.com; s=ti-com-17Q1; t=1715688710; bh=OsXgZbCTz/uuKLSGsGQZvWLyWdVYSJ4fAsjKQWfCgac=; h=Date:From:To:CC:Subject:References:In-Reply-To; b=RSU5IvbSkiCst0Z228ER9IBFdciijquqVWZfTKVGX1ee0iIlHHlsagsqJOcixTQ7D hhNHtB5B8skwhev606unk2HF9CYAzk9c0NOeKc0q/KAXdZFSk7qGSS2xNAQNMPDfJc UN6eaNPGYAJjGZQWLALBYzvrIeqKp4fexuoorAy0= Received: from DFLE109.ent.ti.com (dfle109.ent.ti.com [10.64.6.30]) by fllv0035.itg.ti.com (8.15.2/8.15.2) with ESMTPS id 44ECBo39105112 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=FAIL); Tue, 14 May 2024 07:11:50 -0500 Received: from DFLE103.ent.ti.com (10.64.6.24) by DFLE109.ent.ti.com (10.64.6.30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2507.23; Tue, 14 May 2024 07:11:50 -0500 Received: from lelvsmtp5.itg.ti.com (10.180.75.250) by DFLE103.ent.ti.com (10.64.6.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2507.23 via Frontend Transport; Tue, 14 May 2024 07:11:50 -0500 Received: from localhost (uda0492258.dhcp.ti.com [172.24.227.9]) by lelvsmtp5.itg.ti.com (8.15.2/8.15.2) with ESMTP id 44ECBnnY006071; Tue, 14 May 2024 07:11:50 -0500 Date: Tue, 14 May 2024 17:41:48 +0530 From: Siddharth Vadapalli To: Bjorn Helgaas CC: Siddharth Vadapalli , , , , , , , , , , , , , , Subject: Re: [PATCH v7 2/2] PCI: keystone: Fix pci_ops for AM654x SoC Message-ID: <8b56604d-a2b8-4227-8a6f-c477332416b4@ti.com> References: <20240328085041.2916899-3-s-vadapalli@ti.com> <20240513215350.GA1996021@bhelgaas> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240513215350.GA1996021@bhelgaas> X-EXCLAIMER-MD-CONFIG: e1e8a2fd-e40a-4ac6-ac9b-f7e9cc9ee180 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240514_051211_984108_87C91C6E X-CRM114-Status: GOOD ( 30.14 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, May 13, 2024 at 04:53:50PM -0500, Bjorn Helgaas wrote: > On Thu, Mar 28, 2024 at 02:20:41PM +0530, Siddharth Vadapalli wrote: > > In the process of converting .scan_bus() callbacks to .add_bus(), the > > ks_pcie_v3_65_scan_bus() function was changed to ks_pcie_v3_65_add_bus(). > > The .scan_bus() method belonged to ks_pcie_host_ops which was specific > > to controller version 3.65a, while the .add_bus() method had been added > > to ks_pcie_ops which is shared between the controller versions 3.65a and > > 4.90a. Neither the older ks_pcie_v3_65_scan_bus() method, nor the newer > > ks_pcie_v3_65_add_bus() method is applicable to the controller version > > 4.90a which is present in AM654x SoCs. > > > > Thus, as a fix, remove "ks_pcie_v3_65_add_bus()" and move its contents > > to the .msi_init callback "ks_pcie_msi_host_init()" which is specific to > > the 3.65a controller. > > > > Fixes: 6ab15b5e7057 ("PCI: dwc: keystone: Convert .scan_bus() callback to use add_bus") > > Suggested-by: Serge Semin > > Suggested-by: Bjorn Helgaas > > Suggested-by: Niklas Cassel > > Reviewed-by: Niklas Cassel > > Signed-off-by: Siddharth Vadapalli > > Thanks for splitting this into two patches. Krzysztof has applied > both to pci/controller/keystone and we hope to merge them for v6.10. > > I *would* like the commit log to be at a little higher level if > possible. Right now it's a detailed description at the level of the > code edits, but it doesn't say *why* we want this change. > > I think the first cut at this was > https://lore.kernel.org/linux-pci/20231011123451.34827-1-s-vadapalli@ti.com/t/#u, > which mentioned Completion Timeouts during MSI-X configuration and 45 > second delays during boot. > > IIUC, prior to 6ab15b5e7057, ks_pcie_v3_65_scan_bus() initialized BAR > 0 and was only used for v3.65a devices. 6ab15b5e7057 renamed it to > ks_pcie_v3_65_add_bus() and called it for both v3.65a and v4.90a. > > I think the problem is that in the current code, the > ks_pcie_ops.add_bus() method (ks_pcie_v3_65_add_bus()) is used for all > devices (both v3.65a and v4.90a). So I guess doing the BAR 0 setup on > v4.90a broke something there? BAR0 was set to a different value on AM654x SoC which has the v4.90a controller, which is identical to what is set even for the v3.65a controller. The difference is that BAR0 is programmed to a different value for enabling inbound MSI writes on top of the common configuration performed for BAR0. Common configuration for BAR0: ks_pcie_probe dw_pcie_host_init dw_pcie_setup_rc ... /* Setup RC BARs */ dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, 0x00000004); dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_1, 0x00000000); ... dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, 0); ... MSI specific configuration of BAR0 performed after the common configuration via the ks_pcie_v3_65_scan_bus() callback: /* Configure and set up BAR0 */ ks_pcie_set_dbi_mode(ks_pcie); /* Enable BAR0 */ dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, 1); dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, SZ_4K - 1); ks_pcie_clear_dbi_mode(ks_pcie); /* * For BAR0, just setting bus address for inbound writes (MSI) should * be sufficient. Use physical address to avoid any conflicts. */ dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, ks_pcie->app.start); The above configuration of BAR0 shouldn't be performed for AM654x SoC. While I am not certain, the timeouts are probably a result of the BAR being programmed to a wrong value which results in a "no match" outcome. > > I'm not quite clear on the mechanism, but it would be helpful to at > least know what's wrong and on what platform. E.g., currently v4.90 > suffers Completion Timeouts and 45 second boot delays? And this patch > fixes that? Yes, the Completion Timeouts cause the 45 second boot delays and this patch fixes that. Regards, Siddharth. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel