From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2FD8FC433EF for ; Wed, 13 Apr 2022 09:17:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=J9ATC/7zDvTH3JQ02zTFHi4HLYWvz6x+Z7xkRqMw3W4=; b=Vb5v4nDe79rNHu AjfgZIeYz7jAeB2AT0Y3AOGjAQz33Is+rzGHJGBhEuaWRxoq6RBIDftZf5gFMZ1HKHjB5tMFs9eKA TAiz8yERaBzC3ntuUHxRhpVbnOohlIPDerCwm7HpbyGtDgXL53G02dd+vrdFeDnFLqDOH2K9txMHy Rp30EXPz8iUjGlkRSfqBHcVCLR/SUCUe3BzU4lu+jZCmHkbJg/AJOL+7VmKJnhlD8ZOYf0+BF6iHd f/9GdZBSrkDC4P/VmsgvqwR1x2gxjasi6JuWOVgkqDg9VWdatq4lm76JnwGh33QvqvZJT3XL6ifXC pSjjiieGEqClUPKXiUkw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1neZ6c-000H6L-3b; Wed, 13 Apr 2022 09:16:06 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1neZ6Y-000H4u-36 for linux-arm-kernel@lists.infradead.org; Wed, 13 Apr 2022 09:16:04 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D45A113D5; Wed, 13 Apr 2022 02:15:59 -0700 (PDT) Received: from lpieralisi (unknown [10.57.8.17]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C47CC3F5A1; Wed, 13 Apr 2022 02:15:57 -0700 (PDT) Date: Wed, 13 Apr 2022 10:16:03 +0100 From: Lorenzo Pieralisi To: Pali =?iso-8859-1?Q?Roh=E1r?= Cc: Marek =?iso-8859-1?Q?Beh=FAn?= , Bjorn Helgaas , Krzysztof =?utf-8?Q?Wilczy=C5=84ski?= , Marc Zyngier , linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Gregory CLEMENT Subject: Re: [PATCH 17/18] PCI: aardvark: Run link training in separate worker Message-ID: <20220413091603.GA19821@lpieralisi> References: <20220220193346.23789-1-kabel@kernel.org> <20220220193346.23789-18-kabel@kernel.org> <20220412152524.GB20749@lpieralisi> <20220412175541.gnrynogn3s2llari@pali> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20220412175541.gnrynogn3s2llari@pali> User-Agent: Mutt/1.9.4 (2018-02-28) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220413_021602_266679_26F2591C X-CRM114-Status: GOOD ( 41.12 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, Apr 12, 2022 at 07:55:41PM +0200, Pali Roh=E1r wrote: > On Tuesday 12 April 2022 16:25:24 Lorenzo Pieralisi wrote: > > On Sun, Feb 20, 2022 at 08:33:45PM +0100, Marek Beh=FAn wrote: > > > From: Pali Roh=E1r > > > = > > > Link training and PCIe card reset routines in Aardvark contain several > > > delays, resulting in rather slow PCIe card probing. The worst case is > > > when there is no card connected: the driver tries link training at all > > > possible speeds and waits until all timers expire. > > > = > > > Since probe methods for all system devices are called sequentially, t= his > > > results in noticeably longer boot time. > > > = > > > Move card reset and link training code from driver probe function into > > > a separate worker, so that kernel can do something different while the > > > driver is waiting during reset or training. > > > = > > > On ESPRESSObin and Turris MOX this decreases boot time by 0.4s with > > > plugged PCIe card and by 2.2s if no card is connected. > > = > > I believe this is what the PROBE_PREFER_ASYNCHRONOUS flag in > > struct device_driver.probe_type flag is there for unless I am > > missing something obvious here. > > = > > Can you give it a try and report back please ? > = > Hello Lorenzo. > = > During testing patches 17 and 18 I saw that following race condition > https://lore.kernel.org/linux-pci/20210407144146.rl7x2h5l2cc3escy@pali/ > (which cause kernel oops) was triggered more often. > = > I'm not sure if above race condition was fully fixed by the last > Krzysztof's patches or there is also other issue which cause oops. > = > As both patches 17 and 18 are just optimizations, I would suggest to > skip it for now, until all these issues are resolved or verified that > they are not triggered anymore. > = > I guess that at this time we can look at PROBE_PREFER_ASYNCHRONOUS flag > and decide how to implement this optimization. > = > Do you agree, or do you have other opinion? It is fine by me - I will consider other patches in the series. Lorenzo > > Thanks, > > Lorenzo > > = > > > Signed-off-by: Pali Roh=E1r > > > Signed-off-by: Marek Beh=FAn > > > --- > > > drivers/pci/controller/pci-aardvark.c | 42 ++++++++++++++++++-------= -- > > > 1 file changed, 28 insertions(+), 14 deletions(-) > > > = > > > diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/cont= roller/pci-aardvark.c > > > index 8c9ac7766ac7..056f49d0e3a4 100644 > > > --- a/drivers/pci/controller/pci-aardvark.c > > > +++ b/drivers/pci/controller/pci-aardvark.c > > > @@ -26,6 +26,7 @@ > > > #include > > > #include > > > #include > > > +#include > > > = > > > #include "../pci.h" > > > #include "../pci-bridge-emul.h" > > > @@ -296,6 +297,8 @@ struct advk_pcie { > > > int link_gen; > > > bool link_was_up; > > > struct timer_list link_irq_timer; > > > + struct delayed_work probe_card_work; > > > + bool host_bridge_probed; > > > struct pci_bridge_emul bridge; > > > struct gpio_desc *reset_gpio; > > > struct clk *clk; > > > @@ -497,6 +500,21 @@ static void advk_pcie_train_link(struct advk_pci= e *pcie) > > > dev_err(dev, "link never came up\n"); > > > } > > > = > > > +static void advk_pcie_probe_card_work(struct work_struct *work) > > > +{ > > > + struct delayed_work *dwork =3D container_of(work, struct delayed_wo= rk, > > > + work); > > > + struct advk_pcie *pcie =3D container_of(dwork, struct advk_pcie, > > > + probe_card_work); > > > + struct pci_host_bridge *bridge =3D pci_host_bridge_from_priv(pcie); > > > + int ret; > > > + > > > + advk_pcie_train_link(pcie); > > > + ret =3D pci_host_probe(bridge); > > > + if (!ret) > > > + pcie->host_bridge_probed =3D true; > > > +} > > > + > > > /* > > > * Set PCIe address window register which could be used for memory > > > * mapping. > > > @@ -701,8 +719,6 @@ static void advk_pcie_setup_hw(struct advk_pcie *= pcie) > > > /* Disable remaining PCIe outbound windows */ > > > for (i =3D pcie->wins_count; i < OB_WIN_COUNT; i++) > > > advk_pcie_disable_ob_win(pcie, i); > > > - > > > - advk_pcie_train_link(pcie); > > > } > > > = > > > static int advk_pcie_check_pio_status(struct advk_pcie *pcie, bool a= llow_crs, u32 *val) > > > @@ -2112,14 +2128,8 @@ static int advk_pcie_probe(struct platform_dev= ice *pdev) > > > bridge->ops =3D &advk_pcie_ops; > > > bridge->map_irq =3D advk_pcie_map_irq; > > > = > > > - ret =3D pci_host_probe(bridge); > > > - if (ret < 0) { > > > - irq_set_chained_handler_and_data(pcie->irq, NULL, NULL); > > > - advk_pcie_remove_rp_irq_domain(pcie); > > > - advk_pcie_remove_msi_irq_domain(pcie); > > > - advk_pcie_remove_irq_domain(pcie); > > > - return ret; > > > - } > > > + INIT_DELAYED_WORK(&pcie->probe_card_work, advk_pcie_probe_card_work= ); > > > + schedule_delayed_work(&pcie->probe_card_work, 1); > > > = > > > return 0; > > > } > > > @@ -2131,11 +2141,15 @@ static int advk_pcie_remove(struct platform_d= evice *pdev) > > > u32 val; > > > int i; > > > = > > > + cancel_delayed_work_sync(&pcie->probe_card_work); > > > + > > > /* Remove PCI bus with all devices */ > > > - pci_lock_rescan_remove(); > > > - pci_stop_root_bus(bridge->bus); > > > - pci_remove_root_bus(bridge->bus); > > > - pci_unlock_rescan_remove(); > > > + if (pcie->host_bridge_probed) { > > > + pci_lock_rescan_remove(); > > > + pci_stop_root_bus(bridge->bus); > > > + pci_remove_root_bus(bridge->bus); > > > + pci_unlock_rescan_remove(); > > > + } > > > = > > > /* Disable Root Bridge I/O space, memory space and bus mastering */ > > > val =3D advk_readl(pcie, PCIE_CORE_CMD_STATUS_REG); > > > -- = > > > 2.34.1 > > > = _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel