From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA71EC00140 for ; Tue, 2 Aug 2022 12:39:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236081AbiHBMjT (ORCPT ); Tue, 2 Aug 2022 08:39:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56290 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233181AbiHBMjS (ORCPT ); Tue, 2 Aug 2022 08:39:18 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8C64E1FCDD; Tue, 2 Aug 2022 05:39:17 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 2A166B81EF7; Tue, 2 Aug 2022 12:39:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3B945C433D6; Tue, 2 Aug 2022 12:39:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1659443954; bh=czjqSbGtpIYmswV/Rh3E0u4EPRbeo8kro8V9wiWdp5k=; h=From:To:Cc:Subject:Date:From; b=GzvWjGshRogL5da+jhHTw7kSXC/ItA43Eelv5Fg5fhWHujny4CORKbWe3G8cl6dmv rWDUl1TZNX6WUGK3WmTrBjdB0J0lGa/AWWNQbJsBqmgHDiAMb+ULVxWfmtDnZO/NhY mONuZthsFIplbOHBEv6BspaNrSEQTKlZ8YXYJ+xDGL54Bmzg8tzaEpWzPFnf3cS8HH ykbPOMNvbi6SXS5gkp/kv/9cembG4Nk0MpWeFo3ghK6CDKZsffLJM8T7yqTmuRPLll oJMxIGznROtPB7+wi3zpKTh9xPVZCFnPG60O7he/CRWBUUhjNE5eioQjhGudIdGDAV o1uy/9X1hpKAQ== Received: by pali.im (Postfix) id F224DF81; Tue, 2 Aug 2022 14:39:10 +0200 (CEST) From: =?UTF-8?q?Pali=20Roh=C3=A1r?= To: Thomas Petazzoni , Lorenzo Pieralisi , Bjorn Helgaas , Elad Nachman Cc: =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Rob Herring , linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Gregory Clement , =?UTF-8?q?Marek=20Beh=C3=BAn?= , Remi Pommarel , Xogium , Tomasz Maciej Nowak Subject: [PATCH v2] PCI: aardvark: Implement workaround for PCIe Completion Timeout Date: Tue, 2 Aug 2022 14:38:16 +0200 Message-Id: <20220802123816.21817-1-pali@kernel.org> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Marvell Armada 3700 Functional Errata, Guidelines, and Restrictions document describes in erratum 3.12 PCIe Completion Timeout (Ref #: 251), that PCIe IP does not support a strong-ordered model for inbound posted vs. outbound completion. As a workaround for this erratum, DIS_ORD_CHK flag in Debug Mux Control register must be set. It disables the ordering check in the core between Completions and Posted requests received from the link. Marvell also suggests to do full memory barrier at the beginning of aardvark summary interrupt handler before calling interrupt handlers of endpoint drivers in order to minimize the risk for the race condition documented in the Erratum between the DMA done status reading and the completion of writing to the host memory. More details about this issue and suggested workarounds are in discussion: https://lore.kernel.org/linux-pci/BN9PR18MB425154FE5019DCAF2028A1D5DB8D9@BN9PR18MB4251.namprd18.prod.outlook.com/t/#u It was reported that enabling this workaround fixes instability issues and "Unhandled fault" errors when using 60 GHz WiFi 802.11ad card with Qualcomm QCA6335 chip under significant load which were caused by interrupt status stuck in the outbound CMPLT queue traced back to this erratum. This workaround fixes also kernel panic triggered after some minutes of usage 5 GHz WiFi 802.11ax card with Mediatek MT7915 chip: Internal error: synchronous external abort: 96000210 [#1] SMP Kernel panic - not syncing: Fatal exception in interrupt Signed-off-by: Thomas Petazzoni Signed-off-by: Pali Rohár Fixes: 8c39d710363c ("PCI: aardvark: Add Aardvark PCI host controller driver") Cc: stable@vger.kernel.org --- drivers/pci/controller/pci-aardvark.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c index 060936ef01fe..3ae8a85ec72e 100644 --- a/drivers/pci/controller/pci-aardvark.c +++ b/drivers/pci/controller/pci-aardvark.c @@ -210,6 +210,8 @@ enum { }; #define VENDOR_ID_REG (LMI_BASE_ADDR + 0x44) +#define DEBUG_MUX_CTRL_REG (LMI_BASE_ADDR + 0x208) +#define DIS_ORD_CHK BIT(30) /* PCIe core controller registers */ #define CTRL_CORE_BASE_ADDR 0x18000 @@ -558,6 +560,11 @@ static void advk_pcie_setup_hw(struct advk_pcie *pcie) PCIE_CORE_CTRL2_TD_ENABLE; advk_writel(pcie, reg, PCIE_CORE_CTRL2_REG); + /* Disable ordering checks, workaround for erratum 3.12 "PCIe completion timeout" */ + reg = advk_readl(pcie, DEBUG_MUX_CTRL_REG); + reg |= DIS_ORD_CHK; + advk_writel(pcie, reg, DEBUG_MUX_CTRL_REG); + /* Set lane X1 */ reg = advk_readl(pcie, PCIE_CORE_CTRL0_REG); reg &= ~LANE_CNT_MSK; @@ -1581,6 +1588,9 @@ static irqreturn_t advk_pcie_irq_handler(int irq, void *arg) struct advk_pcie *pcie = arg; u32 status; + /* Full memory barrier (ARM dsb sy), workaround for erratum 3.12 "PCIe completion timeout" */ + mb(); + status = advk_readl(pcie, HOST_CTRL_INT_STATUS_REG); if (!(status & PCIE_IRQ_CORE_INT)) return IRQ_NONE; -- 2.20.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D3032C00140 for ; Tue, 2 Aug 2022 12:40:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=ZTehhH1t8Diql/BCw1vUAgnpbDtgyr5bAizdgqvWntI=; b=4PD6IZGsZjomkJ J7muGrC41uwTILNHnlP7zp1DCyGq+KYJ4DpR1qPuq5R+5fe79NK/dQqUzFNsyxRw/Scx0HK7G+BPz IdCMoX18Xbd6o3a7R13mh0R2wx36kCVQl/Vuz7mXSgd2nBHvPnqbEbBov45ISN5HjL3kzOXjaQN9O F9jx8VdlmcMYuYkhXRQr18pjRi8qErs2eCK23o+583KnIinWa3IFkfo96/REDXYWP+33bxrT9SVLN uluNdVJNmQ+7qJcYggROKxREWFYZNsDN/tdGkAV3RBWdeP3neDjKWFl/Tp1ei2rbS/rC4csqaq/ES cW/E6lsYbbh430vIpYZQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oIrB9-00E4Bm-JF; Tue, 02 Aug 2022 12:39:19 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oIrB6-00E4AG-Ga for linux-arm-kernel@lists.infradead.org; Tue, 02 Aug 2022 12:39:18 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 3A618612DA; Tue, 2 Aug 2022 12:39:15 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3B945C433D6; Tue, 2 Aug 2022 12:39:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1659443954; bh=czjqSbGtpIYmswV/Rh3E0u4EPRbeo8kro8V9wiWdp5k=; h=From:To:Cc:Subject:Date:From; b=GzvWjGshRogL5da+jhHTw7kSXC/ItA43Eelv5Fg5fhWHujny4CORKbWe3G8cl6dmv rWDUl1TZNX6WUGK3WmTrBjdB0J0lGa/AWWNQbJsBqmgHDiAMb+ULVxWfmtDnZO/NhY mONuZthsFIplbOHBEv6BspaNrSEQTKlZ8YXYJ+xDGL54Bmzg8tzaEpWzPFnf3cS8HH ykbPOMNvbi6SXS5gkp/kv/9cembG4Nk0MpWeFo3ghK6CDKZsffLJM8T7yqTmuRPLll oJMxIGznROtPB7+wi3zpKTh9xPVZCFnPG60O7he/CRWBUUhjNE5eioQjhGudIdGDAV o1uy/9X1hpKAQ== Received: by pali.im (Postfix) id F224DF81; Tue, 2 Aug 2022 14:39:10 +0200 (CEST) From: =?UTF-8?q?Pali=20Roh=C3=A1r?= To: Thomas Petazzoni , Lorenzo Pieralisi , Bjorn Helgaas , Elad Nachman Cc: =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Rob Herring , linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Gregory Clement , =?UTF-8?q?Marek=20Beh=C3=BAn?= , Remi Pommarel , Xogium , Tomasz Maciej Nowak Subject: [PATCH v2] PCI: aardvark: Implement workaround for PCIe Completion Timeout Date: Tue, 2 Aug 2022 14:38:16 +0200 Message-Id: <20220802123816.21817-1-pali@kernel.org> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220802_053916_663448_51465BE9 X-CRM114-Status: GOOD ( 15.66 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org TWFydmVsbCBBcm1hZGEgMzcwMCBGdW5jdGlvbmFsIEVycmF0YSwgR3VpZGVsaW5lcywgYW5kIFJl c3RyaWN0aW9ucwpkb2N1bWVudCBkZXNjcmliZXMgaW4gZXJyYXR1bSAzLjEyIFBDSWUgQ29tcGxl dGlvbiBUaW1lb3V0IChSZWYgIzogMjUxKSwKdGhhdCBQQ0llIElQIGRvZXMgbm90IHN1cHBvcnQg YSBzdHJvbmctb3JkZXJlZCBtb2RlbCBmb3IgaW5ib3VuZCBwb3N0ZWQgdnMuCm91dGJvdW5kIGNv bXBsZXRpb24uCgpBcyBhIHdvcmthcm91bmQgZm9yIHRoaXMgZXJyYXR1bSwgRElTX09SRF9DSEsg ZmxhZyBpbiBEZWJ1ZyBNdXggQ29udHJvbApyZWdpc3RlciBtdXN0IGJlIHNldC4gSXQgZGlzYWJs ZXMgdGhlIG9yZGVyaW5nIGNoZWNrIGluIHRoZSBjb3JlIGJldHdlZW4KQ29tcGxldGlvbnMgYW5k IFBvc3RlZCByZXF1ZXN0cyByZWNlaXZlZCBmcm9tIHRoZSBsaW5rLgoKTWFydmVsbCBhbHNvIHN1 Z2dlc3RzIHRvIGRvIGZ1bGwgbWVtb3J5IGJhcnJpZXIgYXQgdGhlIGJlZ2lubmluZyBvZgphYXJk dmFyayBzdW1tYXJ5IGludGVycnVwdCBoYW5kbGVyIGJlZm9yZSBjYWxsaW5nIGludGVycnVwdCBo YW5kbGVycyBvZgplbmRwb2ludCBkcml2ZXJzIGluIG9yZGVyIHRvIG1pbmltaXplIHRoZSByaXNr IGZvciB0aGUgcmFjZSBjb25kaXRpb24KZG9jdW1lbnRlZCBpbiB0aGUgRXJyYXR1bSBiZXR3ZWVu IHRoZSBETUEgZG9uZSBzdGF0dXMgcmVhZGluZyBhbmQgdGhlCmNvbXBsZXRpb24gb2Ygd3JpdGlu ZyB0byB0aGUgaG9zdCBtZW1vcnkuCgpNb3JlIGRldGFpbHMgYWJvdXQgdGhpcyBpc3N1ZSBhbmQg c3VnZ2VzdGVkIHdvcmthcm91bmRzIGFyZSBpbiBkaXNjdXNzaW9uOgpodHRwczovL2xvcmUua2Vy bmVsLm9yZy9saW51eC1wY2kvQk45UFIxOE1CNDI1MTU0RkU1MDE5RENBRjIwMjhBMUQ1REI4RDlA Qk45UFIxOE1CNDI1MS5uYW1wcmQxOC5wcm9kLm91dGxvb2suY29tL3QvI3UKCkl0IHdhcyByZXBv cnRlZCB0aGF0IGVuYWJsaW5nIHRoaXMgd29ya2Fyb3VuZCBmaXhlcyBpbnN0YWJpbGl0eSBpc3N1 ZXMgYW5kCiJVbmhhbmRsZWQgZmF1bHQiIGVycm9ycyB3aGVuIHVzaW5nIDYwIEdIeiBXaUZpIDgw Mi4xMWFkIGNhcmQgd2l0aCBRdWFsY29tbQpRQ0E2MzM1IGNoaXAgdW5kZXIgc2lnbmlmaWNhbnQg bG9hZCB3aGljaCB3ZXJlIGNhdXNlZCBieSBpbnRlcnJ1cHQgc3RhdHVzCnN0dWNrIGluIHRoZSBv dXRib3VuZCBDTVBMVCBxdWV1ZSB0cmFjZWQgYmFjayB0byB0aGlzIGVycmF0dW0uCgpUaGlzIHdv cmthcm91bmQgZml4ZXMgYWxzbyBrZXJuZWwgcGFuaWMgdHJpZ2dlcmVkIGFmdGVyIHNvbWUgbWlu dXRlcyBvZgp1c2FnZSA1IEdIeiBXaUZpIDgwMi4xMWF4IGNhcmQgd2l0aCBNZWRpYXRlayBNVDc5 MTUgY2hpcDoKCiAgICBJbnRlcm5hbCBlcnJvcjogc3luY2hyb25vdXMgZXh0ZXJuYWwgYWJvcnQ6 IDk2MDAwMjEwIFsjMV0gU01QCiAgICBLZXJuZWwgcGFuaWMgLSBub3Qgc3luY2luZzogRmF0YWwg ZXhjZXB0aW9uIGluIGludGVycnVwdAoKU2lnbmVkLW9mZi1ieTogVGhvbWFzIFBldGF6em9uaSA8 dGhvbWFzLnBldGF6em9uaUBib290bGluLmNvbT4KU2lnbmVkLW9mZi1ieTogUGFsaSBSb2jDoXIg PHBhbGlAa2VybmVsLm9yZz4KRml4ZXM6IDhjMzlkNzEwMzYzYyAoIlBDSTogYWFyZHZhcms6IEFk ZCBBYXJkdmFyayBQQ0kgaG9zdCBjb250cm9sbGVyIGRyaXZlciIpCkNjOiBzdGFibGVAdmdlci5r ZXJuZWwub3JnCi0tLQogZHJpdmVycy9wY2kvY29udHJvbGxlci9wY2ktYWFyZHZhcmsuYyB8IDEw ICsrKysrKysrKysKIDEgZmlsZSBjaGFuZ2VkLCAxMCBpbnNlcnRpb25zKCspCgpkaWZmIC0tZ2l0 IGEvZHJpdmVycy9wY2kvY29udHJvbGxlci9wY2ktYWFyZHZhcmsuYyBiL2RyaXZlcnMvcGNpL2Nv bnRyb2xsZXIvcGNpLWFhcmR2YXJrLmMKaW5kZXggMDYwOTM2ZWYwMWZlLi4zYWU4YTg1ZWM3MmUg MTAwNjQ0Ci0tLSBhL2RyaXZlcnMvcGNpL2NvbnRyb2xsZXIvcGNpLWFhcmR2YXJrLmMKKysrIGIv ZHJpdmVycy9wY2kvY29udHJvbGxlci9wY2ktYWFyZHZhcmsuYwpAQCAtMjEwLDYgKzIxMCw4IEBA IGVudW0gewogfTsKIAogI2RlZmluZSBWRU5ET1JfSURfUkVHCQkJCShMTUlfQkFTRV9BRERSICsg MHg0NCkKKyNkZWZpbmUgREVCVUdfTVVYX0NUUkxfUkVHCQkJKExNSV9CQVNFX0FERFIgKyAweDIw OCkKKyNkZWZpbmUgICAgIERJU19PUkRfQ0hLCQkJCUJJVCgzMCkKIAogLyogUENJZSBjb3JlIGNv bnRyb2xsZXIgcmVnaXN0ZXJzICovCiAjZGVmaW5lIENUUkxfQ09SRV9CQVNFX0FERFIJCQkweDE4 MDAwCkBAIC01NTgsNiArNTYwLDExIEBAIHN0YXRpYyB2b2lkIGFkdmtfcGNpZV9zZXR1cF9odyhz dHJ1Y3QgYWR2a19wY2llICpwY2llKQogCQlQQ0lFX0NPUkVfQ1RSTDJfVERfRU5BQkxFOwogCWFk dmtfd3JpdGVsKHBjaWUsIHJlZywgUENJRV9DT1JFX0NUUkwyX1JFRyk7CiAKKwkvKiBEaXNhYmxl IG9yZGVyaW5nIGNoZWNrcywgd29ya2Fyb3VuZCBmb3IgZXJyYXR1bSAzLjEyICJQQ0llIGNvbXBs ZXRpb24gdGltZW91dCIgKi8KKwlyZWcgPSBhZHZrX3JlYWRsKHBjaWUsIERFQlVHX01VWF9DVFJM X1JFRyk7CisJcmVnIHw9IERJU19PUkRfQ0hLOworCWFkdmtfd3JpdGVsKHBjaWUsIHJlZywgREVC VUdfTVVYX0NUUkxfUkVHKTsKKwogCS8qIFNldCBsYW5lIFgxICovCiAJcmVnID0gYWR2a19yZWFk bChwY2llLCBQQ0lFX0NPUkVfQ1RSTDBfUkVHKTsKIAlyZWcgJj0gfkxBTkVfQ05UX01TSzsKQEAg LTE1ODEsNiArMTU4OCw5IEBAIHN0YXRpYyBpcnFyZXR1cm5fdCBhZHZrX3BjaWVfaXJxX2hhbmRs ZXIoaW50IGlycSwgdm9pZCAqYXJnKQogCXN0cnVjdCBhZHZrX3BjaWUgKnBjaWUgPSBhcmc7CiAJ dTMyIHN0YXR1czsKIAorCS8qIEZ1bGwgbWVtb3J5IGJhcnJpZXIgKEFSTSBkc2Igc3kpLCB3b3Jr YXJvdW5kIGZvciBlcnJhdHVtIDMuMTIgIlBDSWUgY29tcGxldGlvbiB0aW1lb3V0IiAqLworCW1i KCk7CisKIAlzdGF0dXMgPSBhZHZrX3JlYWRsKHBjaWUsIEhPU1RfQ1RSTF9JTlRfU1RBVFVTX1JF Ryk7CiAJaWYgKCEoc3RhdHVzICYgUENJRV9JUlFfQ09SRV9JTlQpKQogCQlyZXR1cm4gSVJRX05P TkU7Ci0tIAoyLjIwLjEKCgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fXwpsaW51eC1hcm0ta2VybmVsIG1haWxpbmcgbGlzdApsaW51eC1hcm0ta2VybmVsQGxp c3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVhZC5vcmcvbWFpbG1hbi9saXN0 aW5mby9saW51eC1hcm0ta2VybmVsCg==