From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4161ECCA471 for ; Fri, 3 Oct 2025 19:56:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=8l4grpSFSjWNM4iqEUy0sH3xfwBR0H8/rIl0Jd6fkMU=; b=hMPjGejK5d6+VQeoYT/lwpPgFQ VyjO3665NsmqtIxo1nswEG06oDXyTMxDP952CWoSdZjv1tLldjoLVy2pMr9Izao0DqwP+mt/2uSHA BlXNghTk3N7s8lCS118yoa0jWg/t/TEPT5PLVJbPeaWb/r8QMhK8lP6hVZMKvMHno7wGNmKswarA9 Pw77sbyWFqF62hw7gsgOMlo8IVqZWUU3ATrP4JcNFCE+c8FI4y+1+3r6pVUdf9dU4e2gpyHfiKllo vprfnGvFoW1i5H2/blCtkxOUIk7ORq2UXHLOXZop9JU+SsLCvgjdHgcB/7Mlb1dYCF0K8m4megx6e LDazVu/w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.2 #2 (Red Hat Linux)) id 1v4ltA-0000000D1fd-036l; Fri, 03 Oct 2025 19:56:24 +0000 Received: from mail-qk1-x762.google.com ([2607:f8b0:4864:20::762]) by bombadil.infradead.org with esmtps (Exim 4.98.2 #2 (Red Hat Linux)) id 1v4lt6-0000000D1dv-3d03 for linux-arm-kernel@lists.infradead.org; Fri, 03 Oct 2025 19:56:22 +0000 Received: by mail-qk1-x762.google.com with SMTP id af79cd13be357-8582a34639aso169330285a.3 for ; Fri, 03 Oct 2025 12:56:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759521380; x=1760126180; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=8l4grpSFSjWNM4iqEUy0sH3xfwBR0H8/rIl0Jd6fkMU=; b=vYQ2qdkEkVDREKRnCunuJRzOhwG4hbGPfeiuOGB3jutxqho7jYtmYJcEV7TDQaKQqb MPeN7L2kaBRv0ZI/0CWxSEQ8FFFLI9vId8af3+O/3ZKGBPGWLARCoi1EWfcZymggyiFf lv94wcfzBE0QqCxMYVgOznBz1wI5FXvjjIWKPCtDcOqXUmXfpTIIc7Rgbmo3D6+iCnvT vuW3+gLCKtyxXLBWat51ouBpzsafg4gJHGFocYJHYV7qdKxPBLdxKNUa53Ubiap5S3gF jWrUysP2QiMgNQlLPdmnIHgmsYi6moZbeAZwPH+65RQqPT8d0GTqCsRhLeBckF7qMvP2 wT+w== X-Forwarded-Encrypted: i=1; AJvYcCWCu4UDWQ5SkuExHsFi90iM/lWb3myALvCpSNJPdIZ61o1SqltkFJvnr7ur1K5OrNZUrnutyu3gOPOl1yaOssmZ@lists.infradead.org X-Gm-Message-State: AOJu0YwS+tWxm0V9ykpudCo18C6FYU88JAqtmMbC1y0ns6+/npLuPLOs woiQNbg3W4sO2ncU4MVvg8Yv7AIQB+UscPOSIo1PRFBG3ov9pdDeG5wjQ+cGfcPOgdm53eWDkOm k0aUbGcoxq2Ks0yAD4D32poRLBjA9p9eYSRGE21qXvkNg3wwOuPrBpzk56kKlOT8kkeM4iLZAXF oCLaoKrhixhmV6/x/lzUGVYf0aFoLJXnTXk7VbKIxNvXP9w0ErYvkgvBpz8pySW3ZfcD8lUTim7 9jpKlIZO3Bp8gS8nMPA3DSo3DrK7AU= X-Gm-Gg: ASbGncsHsg5CaqiOS9GEIeOv8tEUQ034hWsjxqGq5cIaooa38knaMfTWR3PXjjtCa0/ GGju/UkreZptxG93rjnJcUjVbFojzup2IL88EuC6WstV2/30T9k6FVFOtGioOSztVNAThs2gZQq ZCjSsWfNt94WMMS8exRC6podmb2EyHeeDWZUMMzVQvDhmPxHiQ+HqPinfb0l4oUB9rIywzFEY2p rtrNj+n5IwKHDH4GiBdEbM9UWYpH6gqmTHhHaekCd/5XAEfuUiMm1oUX8+qFYshMPAc6FWFO630 FBfQTHBnyFVrlokTa22tMc5KArCeqT4YsEKNrfrZigBEeW/s0PYyXAY/oamsYOJXK/YCoivRzcL 7jRxqISxyodmsicbFXLKDzLQr5j6oufQOxIsiTHP9QbreB2FJVTtLZAUMgfQ7NDxNrZBnThJACb xtJAgXzy3+ X-Google-Smtp-Source: AGHT+IElIOO6lzmuOzOko1LEPuIRAZs0/mqMJ3F1/b0UOSfNyVOlaNQxPk0IljwSUXFGQyff4hXeBwthYv76 X-Received: by 2002:a05:620a:454d:b0:870:141b:754c with SMTP id af79cd13be357-87a3410c137mr699196085a.22.1759521379465; Fri, 03 Oct 2025 12:56:19 -0700 (PDT) Received: from smtp-us-east1-p01-i01-si01.dlp.protect.broadcom.com (address-144-49-247-126.dlp.protect.broadcom.com. [144.49.247.126]) by smtp-relay.gmail.com with ESMTPS id af79cd13be357-87768069bf9sm39753985a.0.2025.10.03.12.56.19 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Oct 2025 12:56:19 -0700 (PDT) X-Relaying-Domain: broadcom.com X-CFilter-Loop: Reflected Received: by mail-pf1-f200.google.com with SMTP id d2e1a72fcca58-781253de15aso4674453b3a.2 for ; Fri, 03 Oct 2025 12:56:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1759521378; x=1760126178; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8l4grpSFSjWNM4iqEUy0sH3xfwBR0H8/rIl0Jd6fkMU=; b=MSRkWkPZBGbjCAno0zmArezfZ0rc6AWAurtwGf9Jx+4l4232fiB3kwi9wrUet/mYbp JlLF2jl1atHAK8XnMoZ/jZq1jXCz2YgHJy/FKa1VSzaKDNJyishV9qCoTA2jPTS6Dznk uwJWkepQpkwcOM7rD7ywKEMJnfighvP30imoU= X-Forwarded-Encrypted: i=1; AJvYcCWwNZ0RDOMSxEdPx7p4SNYr/GhzorC3JOLZaVrDSFCHx4VtW2GHupNCBKOA185ew1axTkXDlS/ke6g9wNQGI7zB@lists.infradead.org X-Received: by 2002:a05:6a20:9154:b0:301:5784:8cf3 with SMTP id adf61e73a8af0-32b620d6861mr5267920637.49.1759521378104; Fri, 03 Oct 2025 12:56:18 -0700 (PDT) X-Received: by 2002:a05:6a20:9154:b0:301:5784:8cf3 with SMTP id adf61e73a8af0-32b620d6861mr5267896637.49.1759521377723; Fri, 03 Oct 2025 12:56:17 -0700 (PDT) Received: from stband-bld-1.and.broadcom.net ([192.19.144.250]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-b6099add13esm5371162a12.8.2025.10.03.12.56.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Oct 2025 12:56:17 -0700 (PDT) From: Jim Quinlan To: linux-pci@vger.kernel.org, Nicolas Saenz Julienne , Bjorn Helgaas , Lorenzo Pieralisi , Cyril Brulebois , bcm-kernel-feedback-list@broadcom.com, jim2101024@gmail.com, james.quinlan@broadcom.com Cc: Florian Fainelli , Lorenzo Pieralisi , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Manivannan Sadhasivam , Rob Herring , linux-rpi-kernel@lists.infradead.org (moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE), linux-arm-kernel@lists.infradead.org (moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE), linux-kernel@vger.kernel.org (open list) Subject: [PATCH v3 2/2] PCI: brcmstb: Add panic/die handler to driver Date: Fri, 3 Oct 2025 15:56:07 -0400 Message-Id: <20251003195607.2009785-3-james.quinlan@broadcom.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20251003195607.2009785-1-james.quinlan@broadcom.com> References: <20251003195607.2009785-1-james.quinlan@broadcom.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-DetectorID-Processed: b00c1d49-9d2e-4205-b15f-d015386d3d5e X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20251003_125620_931250_3D987445 X-CRM114-Status: GOOD ( 24.01 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Whereas most PCIe HW returns 0xffffffff on illegal accesses and the like, by default Broadcom's STB PCIe controller effects an abort. Some SoCs -- 7216 and its descendants -- have new HW that identifies error details. This simple handler determines if the PCIe controller was the cause of the abort and if so, prints out diagnostic info. Unfortunately, an abort still occurs. Care is taken to read the error registers only when the PCIe bridge is active and the PCIe registers are acceptable. Otherwise, a "die" event caused by something other than the PCIe could cause an abort if the PCIe "die" handler tried to access registers when the bridge is off. Example error output: brcm-pcie 8b20000.pcie: Error: Mem Acc: 32bit, Read, @0x38000000 brcm-pcie 8b20000.pcie: Type: TO=0 Abt=0 UnspReq=1 AccDsble=0 BadAddr=0 Signed-off-by: Jim Quinlan --- drivers/pci/controller/pcie-brcmstb.c | 155 +++++++++++++++++++++++++- 1 file changed, 154 insertions(+), 1 deletion(-) diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c index 9f1f746091be..326155c9ce52 100644 --- a/drivers/pci/controller/pcie-brcmstb.c +++ b/drivers/pci/controller/pcie-brcmstb.c @@ -14,15 +14,18 @@ #include #include #include +#include #include #include #include #include #include +#include #include #include #include #include +#include #include #include #include @@ -156,6 +159,39 @@ #define MSI_INT_MASK_SET 0x10 #define MSI_INT_MASK_CLR 0x14 +/* Error report registers */ +#define PCIE_OUTB_ERR_TREAT 0x6000 +#define PCIE_OUTB_ERR_TREAT_CONFIG_MASK 0x1 +#define PCIE_OUTB_ERR_TREAT_MEM_MASK 0x2 +#define PCIE_OUTB_ERR_VALID 0x6004 +#define PCIE_OUTB_ERR_CLEAR 0x6008 +#define PCIE_OUTB_ERR_ACC_INFO 0x600c +#define PCIE_OUTB_ERR_ACC_INFO_CFG_ERR_MASK 0x01 +#define PCIE_OUTB_ERR_ACC_INFO_MEM_ERR_MASK 0x02 +#define PCIE_OUTB_ERR_ACC_INFO_TYPE_64_MASK 0x04 +#define PCIE_OUTB_ERR_ACC_INFO_DIR_WRITE_MASK 0x10 +#define PCIE_OUTB_ERR_ACC_INFO_BYTE_LANES_MASK 0xff00 +#define PCIE_OUTB_ERR_ACC_ADDR 0x6010 +#define PCIE_OUTB_ERR_ACC_ADDR_BUS_MASK 0xff00000 +#define PCIE_OUTB_ERR_ACC_ADDR_DEV_MASK 0xf8000 +#define PCIE_OUTB_ERR_ACC_ADDR_FUNC_MASK 0x7000 +#define PCIE_OUTB_ERR_ACC_ADDR_REG_MASK 0xfff +#define PCIE_OUTB_ERR_CFG_CAUSE 0x6014 +#define PCIE_OUTB_ERR_CFG_CAUSE_TIMEOUT_MASK 0x40 +#define PCIE_OUTB_ERR_CFG_CAUSE_ABORT_MASK 0x20 +#define PCIE_OUTB_ERR_CFG_CAUSE_UNSUPP_REQ_MASK 0x10 +#define PCIE_OUTB_ERR_CFG_CAUSE_ACC_TIMEOUT_MASK 0x4 +#define PCIE_OUTB_ERR_CFG_CAUSE_ACC_DISABLED_MASK 0x2 +#define PCIE_OUTB_ERR_CFG_CAUSE_ACC_64BIT__MASK 0x1 +#define PCIE_OUTB_ERR_MEM_ADDR_LO 0x6018 +#define PCIE_OUTB_ERR_MEM_ADDR_HI 0x601c +#define PCIE_OUTB_ERR_MEM_CAUSE 0x6020 +#define PCIE_OUTB_ERR_MEM_CAUSE_TIMEOUT_MASK 0x40 +#define PCIE_OUTB_ERR_MEM_CAUSE_ABORT_MASK 0x20 +#define PCIE_OUTB_ERR_MEM_CAUSE_UNSUPP_REQ_MASK 0x10 +#define PCIE_OUTB_ERR_MEM_CAUSE_ACC_DISABLED_MASK 0x2 +#define PCIE_OUTB_ERR_MEM_CAUSE_BAD_ADDR_MASK 0x1 + #define PCIE_RGR1_SW_INIT_1_PERST_MASK 0x1 #define PCIE_RGR1_SW_INIT_1_PERST_SHIFT 0x0 @@ -306,6 +342,8 @@ struct brcm_pcie { bool ep_wakeup_capable; const struct pcie_cfg_data *cfg; bool bridge_in_reset; + struct notifier_block die_notifier; + struct notifier_block panic_notifier; spinlock_t bridge_lock; }; @@ -1731,6 +1769,115 @@ static int brcm_pcie_resume_noirq(struct device *dev) return ret; } +/* Dump out PCIe errors on die or panic */ +static int _brcm_pcie_dump_err(struct brcm_pcie *pcie, + const char *type) +{ + void __iomem *base = pcie->base; + int i, is_cfg_err, is_mem_err, lanes; + char *width_str, *direction_str, lanes_str[9]; + u32 info, cfg_addr, cfg_cause, mem_cause, lo, hi; + unsigned long flags; + + spin_lock_irqsave(&pcie->bridge_lock, flags); + /* Don't access registers when the bridge is off */ + if (pcie->bridge_in_reset || readl(base + PCIE_OUTB_ERR_VALID) == 0) { + spin_unlock_irqrestore(&pcie->bridge_lock, flags); + return NOTIFY_DONE; + } + + /* Read all necessary registers so we can release the spinlock ASAP */ + info = readl(base + PCIE_OUTB_ERR_ACC_INFO); + is_cfg_err = !!(info & PCIE_OUTB_ERR_ACC_INFO_CFG_ERR_MASK); + is_mem_err = !!(info & PCIE_OUTB_ERR_ACC_INFO_MEM_ERR_MASK); + if (is_cfg_err) { + cfg_addr = readl(base + PCIE_OUTB_ERR_ACC_ADDR); + cfg_cause = readl(base + PCIE_OUTB_ERR_CFG_CAUSE); + } + if (is_mem_err) { + mem_cause = readl(base + PCIE_OUTB_ERR_MEM_CAUSE); + lo = readl(base + PCIE_OUTB_ERR_MEM_ADDR_LO); + hi = readl(base + PCIE_OUTB_ERR_MEM_ADDR_HI); + } + /* We've got all of the info, clear the error */ + writel(1, base + PCIE_OUTB_ERR_CLEAR); + spin_unlock_irqrestore(&pcie->bridge_lock, flags); + + dev_err(pcie->dev, "reporting data on PCIe %s error\n", type); + width_str = (info & PCIE_OUTB_ERR_ACC_INFO_TYPE_64_MASK) ? "64bit" : "32bit"; + direction_str = (info & PCIE_OUTB_ERR_ACC_INFO_DIR_WRITE_MASK) ? "Write" : "Read"; + lanes = FIELD_GET(PCIE_OUTB_ERR_ACC_INFO_BYTE_LANES_MASK, info); + for (i = 0, lanes_str[8] = 0; i < 8; i++) + lanes_str[i] = (lanes & (1 << i)) ? '1' : '0'; + + if (is_cfg_err) { + int bus = FIELD_GET(PCIE_OUTB_ERR_ACC_ADDR_BUS_MASK, cfg_addr); + int dev = FIELD_GET(PCIE_OUTB_ERR_ACC_ADDR_DEV_MASK, cfg_addr); + int func = FIELD_GET(PCIE_OUTB_ERR_ACC_ADDR_FUNC_MASK, cfg_addr); + int reg = FIELD_GET(PCIE_OUTB_ERR_ACC_ADDR_REG_MASK, cfg_addr); + + dev_err(pcie->dev, "Error: CFG Acc, %s, %s, Bus=%d, Dev=%d, Fun=%d, Reg=0x%x, lanes=%s\n", + width_str, direction_str, bus, dev, func, reg, lanes_str); + dev_err(pcie->dev, " Type: TO=%d Abt=%d UnsupReq=%d AccTO=%d AccDsbld=%d Acc64bit=%d\n", + !!(cfg_cause & PCIE_OUTB_ERR_CFG_CAUSE_TIMEOUT_MASK), + !!(cfg_cause & PCIE_OUTB_ERR_CFG_CAUSE_ABORT_MASK), + !!(cfg_cause & PCIE_OUTB_ERR_CFG_CAUSE_UNSUPP_REQ_MASK), + !!(cfg_cause & PCIE_OUTB_ERR_CFG_CAUSE_ACC_TIMEOUT_MASK), + !!(cfg_cause & PCIE_OUTB_ERR_CFG_CAUSE_ACC_DISABLED_MASK), + !!(cfg_cause & PCIE_OUTB_ERR_CFG_CAUSE_ACC_64BIT__MASK)); + } + + if (is_mem_err) { + u64 addr = ((u64)hi << 32) | (u64)lo; + + dev_err(pcie->dev, "Error: Mem Acc, %s, %s, @0x%llx, lanes=%s\n", + width_str, direction_str, addr, lanes_str); + dev_err(pcie->dev, " Type: TO=%d Abt=%d UnsupReq=%d AccDsble=%d BadAddr=%d\n", + !!(mem_cause & PCIE_OUTB_ERR_MEM_CAUSE_TIMEOUT_MASK), + !!(mem_cause & PCIE_OUTB_ERR_MEM_CAUSE_ABORT_MASK), + !!(mem_cause & PCIE_OUTB_ERR_MEM_CAUSE_UNSUPP_REQ_MASK), + !!(mem_cause & PCIE_OUTB_ERR_MEM_CAUSE_ACC_DISABLED_MASK), + !!(mem_cause & PCIE_OUTB_ERR_MEM_CAUSE_BAD_ADDR_MASK)); + } + + return NOTIFY_OK; +} + +static int brcm_pcie_die_notify_cb(struct notifier_block *self, + unsigned long v, void *p) +{ + struct brcm_pcie *pcie = + container_of(self, struct brcm_pcie, die_notifier); + + return _brcm_pcie_dump_err(pcie, "Die"); +} + +static int brcm_pcie_panic_notify_cb(struct notifier_block *self, + unsigned long v, void *p) +{ + struct brcm_pcie *pcie = + container_of(self, struct brcm_pcie, panic_notifier); + + return _brcm_pcie_dump_err(pcie, "Panic"); +} + +static void brcm_register_die_notifiers(struct brcm_pcie *pcie) +{ + pcie->panic_notifier.notifier_call = brcm_pcie_panic_notify_cb; + atomic_notifier_chain_register(&panic_notifier_list, + &pcie->panic_notifier); + + pcie->die_notifier.notifier_call = brcm_pcie_die_notify_cb; + register_die_notifier(&pcie->die_notifier); +} + +static void brcm_unregister_die_notifiers(struct brcm_pcie *pcie) +{ + unregister_die_notifier(&pcie->die_notifier); + atomic_notifier_chain_unregister(&panic_notifier_list, + &pcie->panic_notifier); +} + static void __brcm_pcie_remove(struct brcm_pcie *pcie) { brcm_msi_remove(pcie); @@ -1749,6 +1896,9 @@ static void brcm_pcie_remove(struct platform_device *pdev) pci_stop_root_bus(bridge->bus); pci_remove_root_bus(bridge->bus); + if (pcie->cfg->has_err_report) + brcm_unregister_die_notifiers(pcie); + __brcm_pcie_remove(pcie); } @@ -1849,6 +1999,7 @@ static const struct pcie_cfg_data bcm7216_cfg = { .bridge_sw_init_set = brcm_pcie_bridge_sw_init_set_7278, .has_phy = true, .num_inbound_wins = 3, + .has_err_report = true, }; static const struct pcie_cfg_data bcm7712_cfg = { @@ -2023,8 +2174,10 @@ static int brcm_pcie_probe(struct platform_device *pdev) return ret; } - if (pcie->cfg->has_err_report) + if (pcie->cfg->has_err_report) { spin_lock_init(&pcie->bridge_lock); + brcm_register_die_notifiers(pcie); + } return 0; -- 2.34.1