From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from shelob.surriel.com (shelob.surriel.com [96.67.55.147]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 86F54C021A4 for ; Mon, 24 Feb 2025 19:54:58 +0000 (UTC) Received: from localhost ([::1] helo=shelob.surriel.com) by shelob.surriel.com with esmtp (Exim 4.97.1) (envelope-from ) id 1tmeXQ-000000006Wh-39K2; Mon, 24 Feb 2025 14:54:48 -0500 Received: from tor.source.kernel.org ([2600:3c04::f03c:95ff:fe5e:7468]) by shelob.surriel.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.97.1) (envelope-from ) id 1tmeXO-000000006WX-2dPj for kernelnewbies@kernelnewbies.org; Mon, 24 Feb 2025 14:54:46 -0500 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by tor.source.kernel.org (Postfix) with ESMTP id AE1826112C; Mon, 24 Feb 2025 19:54:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3AA40C4CED6; Mon, 24 Feb 2025 19:54:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1740426865; bh=eqo9h/PAZIFf5dMdH1i9KPDuTNmee9ed3YQZ/4tZnaY=; h=Date:From:To:Cc:Subject:In-Reply-To:From; b=cpglDdjU1KMY82rxuH8gT0zRoyKc2n6kTkm4p5aWOIjwBB9C+4g0BTTXUfU6lTvX9 sJy+i/PsStFGOxxqxdmxOyOTdtxE8jd0INf4I6XWYP6hs21zmKRR1lL9z08MiNVza7 wipiYzFN9H1yQLjdXFqSG3oLbcu5PtdNqccPC69oAlC0kkjK5vZSj4QtVuHTjzWc5Z CX3RBOaSXlmCcuLNhj4xcuNpZiSABVcq2A/elCjGyYu+puvSFDJ1X3f0x5HVKVVzTt 718WO/Me3Xn2STtbtb3mcg6V3E2VpiQkLIfKEpN2tpb7AaqMzOMYXpli8YjNm0OkvJ 4CzmON7cRxb3w== Date: Mon, 24 Feb 2025 13:54:23 -0600 From: Bjorn Helgaas To: Naveen Kumar P Subject: Re: PCI: hotplug_event: PCIe PLDA Device BAR Reset Message-ID: <20250224195423.GA473540@bhelgaas> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Cc: linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org, linux-kernel@vger.kernel.org, kernelnewbies X-BeenThere: kernelnewbies@kernelnewbies.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Learn about the Linux kernel List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: kernelnewbies-bounces@kernelnewbies.org T24gVHVlLCBGZWIgMjUsIDIwMjUgYXQgMTI6Mjk6MDBBTSArMDUzMCwgTmF2ZWVuIEt1bWFyIFAg d3JvdGU6Cj4gT24gTW9uLCBGZWIgMjQsIDIwMjUgYXQgMTE6MDPigK9QTSBCam9ybiBIZWxnYWFz IDxoZWxnYWFzQGtlcm5lbC5vcmc+IHdyb3RlOgo+ID4gT24gTW9uLCBGZWIgMjQsIDIwMjUgYXQg MDU6NDU6MzVQTSArMDUzMCwgTmF2ZWVuIEt1bWFyIFAgd3JvdGU6Cj4gPiA+IE9uIFdlZCwgRmVi IDE5LCAyMDI1IGF0IDEwOjM24oCvUE0gQmpvcm4gSGVsZ2FhcyA8aGVsZ2Fhc0BrZXJuZWwub3Jn PiB3cm90ZToKPiA+ID4gPiBPbiBXZWQsIEZlYiAxOSwgMjAyNSBhdCAwNTo1Mjo0N1BNICswNTMw LCBOYXZlZW4gS3VtYXIgUCB3cm90ZToKPiA+ID4gPiA+IEhpIGFsbCwKPiA+ID4gPiA+Cj4gPiA+ ID4gPiBJIGFtIHdyaXRpbmcgdG8gc2VlayBhc3Npc3RhbmNlIHdpdGggYW4gaXNzdWUgd2UgYXJl IGV4cGVyaWVuY2luZyB3aXRoCj4gPiA+ID4gPiBhIFBDSWUgZGV2aWNlIChQTERBIERldmljZSA1 NTU1KSBjb25uZWN0ZWQgdGhyb3VnaCBQQ0kgRXhwcmVzcyBSb290Cj4gPiA+ID4gPiBQb3J0IDEg dG8gdGhlIGhvc3QgYnJpZGdlLgo+ID4gPiA+ID4KPiA+ID4gPiA+IFdlIGhhdmUgb2JzZXJ2ZWQg dGhhdCBhZnRlciBib290aW5nIHRoZSBzeXN0ZW0sIHRoZSBCYXNlIEFkZHJlc3MKPiA+ID4gPiA+ IFJlZ2lzdGVyIChCQVIwKSBtZW1vcnkgb2YgdGhpcyBkZXZpY2UgZ2V0cyByZXNldCB0byAweDAg YWZ0ZXIKPiA+ID4gPiA+IGFwcHJveGltYXRlbHkgb25lIGhvdXIgb3IgbW9yZSAodGhlIHRpbWlu ZyBpcyBpbmNvbnNpc3RlbnQpLiBUaGlzIHdhcwo+ID4gPiA+ID4gdmVyaWZpZWQgdXNpbmcgdGhl IGxzcGNpIG91dHB1dCBhbmQgdGhlIHNldHBjaSAtcyAwMTowMC4wCj4gPiA+ID4gPiBCQVNFX0FE RFJFU1NfMCBjb21tYW5kLgoKPiAuLi4KPiBJIGJvb3RlZCB3aXRoIHRoZSBwY2llX2FzcG09b2Zm IGtlcm5lbCBwYXJhbWV0ZXIsIHdoaWNoIG1lYW5zIHRoYXQKPiBQQ0llIEFjdGl2ZSBTdGF0ZSBQ b3dlciBNYW5hZ2VtZW50IChBU1BNKSBpcyBkaXNhYmxlZC4gR2l2ZW4gdGhpcwo+IGNvbnRleHQs IHNob3VsZCBJIGNvbnNpZGVyIHJlbW92aW5nIHRoaXMgc2V0dGluZyB0byBzZWUgaWYgaXQgYWZm ZWN0cwo+IHRoZSBvY2N1cnJlbmNlIG9mIHRoZSBCdXMgQ2hlY2sgbm90aWZpY2F0aW9ucyBhbmQg dGhlIEJBUjAgcmVzZXQKPiBpc3N1ZT8KCkRvZXNuJ3Qgc2VlbSBsaWtlbHkgdG8gYmUgcmVsYXRl ZC4gIE9uY2UgY29uZmlndXJlZCwgQVNQTSBvcGVyYXRlcwp3aXRob3V0IGFueSBzb2Z0d2FyZSBp bnRlcnZlbnRpb24uICBCdXQgbm90ZSB0aGF0ICJwY2llX2FzcG09b2ZmIgptZWFucyB0aGUga2Vy bmVsIGRvZXNuJ3QgdG91Y2ggQVNQTSBjb25maWd1cmF0aW9uIGF0IGFsbCwgYW5kIGFueQpjb25m aWd1cmF0aW9uIGRvbmUgYnkgZmlybXdhcmUgcmVtYWlucyBpbiBlZmZlY3QuCgpZb3UgY2FuIHRl bGwgd2hldGhlciBBU1BNIGhhcyBiZWVuIGVuYWJsZWQgYnkgZmlybXdhcmUgd2l0aCAic3Vkbwps c3BjaSAtdnYiIGJlZm9yZSB0aGUgcHJvYmxlbSBvY2N1cnMuCgo+ID4gPiBEdXJpbmcgdGhlIEFD UElfTk9USUZZX0JVU19DSEVDSyBldmVudCwgdGhlIGxzcGNpIG91dHB1dCBpbml0aWFsbHkKPiA+ ID4gc2hvd2VkIGFsbCBGRidzLCBhbmQgdGhlbiB0aGUgbmV4dCBydW4gb2YgdGhlIHNhbWUgY29t bWFuZCBzaG93ZWQKPiA+ID4gQkFTRV9BRERSRVNTXzAgcmVzZXQgdG8gemVybzoKPiA+ID4gJCBz dWRvIGxzcGNpIC14eHggLXMgMDE6MDAuMCB8IGdyZXAgIjEwOiIKPiA+ID4gMTA6IGZmIGZmIGZm IGZmIGZmIGZmIGZmIGZmIGZmIGZmIGZmIGZmIGZmIGZmIGZmIGZmCj4gPgo+ID4gTG9va3MgbGlr ZSB0aGUgZGV2aWNlIGlzbid0IHJlc3BvbmRpbmcgYXQgYWxsIGhlcmUuICBDb3VsZCBoYXBwZW4g aWYKPiA+IHRoZSBkZXZpY2UgaXMgcmVzZXQgb3IgcG93ZXJlZCBkb3duLgo+Cj4gRnJvbSB0aGUg a2VybmVsIGRyaXZlciBvciB1c2VyIHNwYWNlIHRvb2xzLCBpcyBpdCBwb3NzaWJsZSB0bwo+IGRl dGVybWluZSB3aGV0aGVyIHRoZSBkZXZpY2UgaGFzIGJlZW4gcmVzZXQgb3IgcG93ZXJlZCBkb3du PyAgQXJlCj4gdGhlcmUgYW55IHBvd2VyIG1hbmFnZW1lbnQgc2V0dGluZ3Mgb3IgY29uZmlndXJh dGlvbnMgdGhhdCBjb3VsZCBiZQo+IGNhdXNpbmcgdGhlIGRldmljZSB0byByZXNldCBvciBwb3dl ciBkb3duIHVuZXhwZWN0ZWRseT8KCk5vdCByZWFsbHkuICBCeSAicG93ZXJlZCBkb3duIiwgSSBt ZWFudCBEM2NvbGQsIHdoZXJlIHRoZSBtYWluIHBvd2VyCmlzIHJlbW92ZWQuICBDb25maWcgc3Bh Y2UgaXMgcmVhZGFibGUgaW4gYWxsIG90aGVyIHBvd2VyIHN0YXRlcy4KCj4gPiBXaGF0IGlzIHRo aXMgZGV2aWNlPyAgV2hhdCBkcml2ZXIgaXMgYm91bmQgdG8gaXQ/ICBJIGRvbid0IHNlZQo+ID4g YW55dGhpbmcgaW4gZG1lc2cgdGhhdCBpZGVudGlmaWVzIGEgZHJpdmVyLgo+Cj4gVGhlIFBDSWUg ZGV2aWNlIGluIHF1ZXN0aW9uIGlzIGEgWGlsaW54IEZQR0EgZW5kcG9pbnQsIHdoaWNoIGlzCj4g Zmxhc2hlZCB3aXRoIFJUTCBjb2RlIHRvIGV4cG9zZSBzZXZlcmFsIGhvc3QgaW50ZXJmYWNlcyB0 byB0aGUgc3lzdGVtCj4gdmlhIHRoZSBQQ0llIGxpbmsuCj4gCj4gV2UgaGF2ZSBhbiBvdXQtb2Yt dHJlZSBkcml2ZXIgZm9yIHRoaXMgZGV2aWNlLCBidXQgdG8gZWxpbWluYXRlIHRoZQo+IGRyaXZl cidzIHJvbGUgaW4gdGhpcyBpc3N1ZSwgSSByZW5hbWVkIHRoZSBkcml2ZXIgdG8gcHJldmVudCBp dCBmcm9tCj4gbG9hZGluZyBhdXRvbWF0aWNhbGx5IGFmdGVyIHJlYm9vdGluZyB0aGUgbWFjaGlu ZS4gRGVzcGl0ZSBub3QgdXNpbmcKPiB0aGUgZHJpdmVyLCB0aGUgaXNzdWUgc3RpbGwgb2NjdXJy ZWQuCgpPaCwgcmlnaHQsIEkgZm9yZ290IHRoYXQgeW91IG1lbnRpb25lZCB0aGlzIGJlZm9yZS4K Cj4gPiBZb3UncmUgc2VlaW5nIHRoZSBwcm9ibGVtIG9uIHY1LjQgKE5vdiAyMDE5KSwgd2hpY2gg aXMgbXVjaCBuZXdlciB0aGFuCj4gPiB2NC40IChKYW4gMjAxNikuICBCdXQgdjUuNCBpcyBzdGls bCByZWFsbHkgdG9vIG9sZCB0byBzcGVuZCBhIGxvdCBvZgo+ID4gdGltZSBvbiB1bmxlc3MgdGhl IHByb2JsZW0gc3RpbGwgaGFwcGVucyBvbiBhIGN1cnJlbnQga2VybmVsLgoKVGhpcyBwYXJ0IGlz IGltcG9ydGFudC4gIFdlIGRvbid0IHdhbnQgdG8gc3BlbmQgYSBsb3Qgb2YgdGltZQpkZWJ1Z2dp bmcgYW4gaXNzdWUgdGhhdCBtYXkgaGF2ZSBhbHJlYWR5IGJlZW4gZml4ZWQgdXBzdHJlYW0uCgpC am9ybgoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KS2Vy bmVsbmV3YmllcyBtYWlsaW5nIGxpc3QKS2VybmVsbmV3Ymllc0BrZXJuZWxuZXdiaWVzLm9yZwpo dHRwczovL2xpc3RzLmtlcm5lbG5ld2JpZXMub3JnL21haWxtYW4vbGlzdGluZm8va2VybmVsbmV3 Ymllcwo= From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E0C871A5B8B; Mon, 24 Feb 2025 19:54:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740426866; cv=none; b=UJyIdS7tTFCQ4PmlCcgaBo6bB1qI/23kC50frpNpLJ0C01byyzIf/tQSSeAyDiTw5LLB8u21sQGOWba6p38TQ4bKqm6DQoG/brop3Z1BhS9uyTP/Ebwb3h+S11llS8kqS6unYfaahUsRBgEoU89767i62zhRmBBmOf3dqVT8gZ4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740426866; c=relaxed/simple; bh=eqo9h/PAZIFf5dMdH1i9KPDuTNmee9ed3YQZ/4tZnaY=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition:In-Reply-To; b=lUS5jhql8AwYzz8RNnK8HQ+6kuWx8EkDMmcSUR6koke6x9xsdcVNKStzxBkgeIOyAGoe3lvxwZMAvIO5sD2XMxfMmNVCp53cG8rRwL7DxVtC0B6ZKGSZs/xy7ShDsPuvACPChKFZzlpoNWxgmwP3UA7OqmJeSI/uwbgkJE6vQ6E= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=cpglDdjU; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="cpglDdjU" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3AA40C4CED6; Mon, 24 Feb 2025 19:54:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1740426865; bh=eqo9h/PAZIFf5dMdH1i9KPDuTNmee9ed3YQZ/4tZnaY=; h=Date:From:To:Cc:Subject:In-Reply-To:From; b=cpglDdjU1KMY82rxuH8gT0zRoyKc2n6kTkm4p5aWOIjwBB9C+4g0BTTXUfU6lTvX9 sJy+i/PsStFGOxxqxdmxOyOTdtxE8jd0INf4I6XWYP6hs21zmKRR1lL9z08MiNVza7 wipiYzFN9H1yQLjdXFqSG3oLbcu5PtdNqccPC69oAlC0kkjK5vZSj4QtVuHTjzWc5Z CX3RBOaSXlmCcuLNhj4xcuNpZiSABVcq2A/elCjGyYu+puvSFDJ1X3f0x5HVKVVzTt 718WO/Me3Xn2STtbtb3mcg6V3E2VpiQkLIfKEpN2tpb7AaqMzOMYXpli8YjNm0OkvJ 4CzmON7cRxb3w== Date: Mon, 24 Feb 2025 13:54:23 -0600 From: Bjorn Helgaas To: Naveen Kumar P Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, kernelnewbies , linux-acpi@vger.kernel.org Subject: Re: PCI: hotplug_event: PCIe PLDA Device BAR Reset Message-ID: <20250224195423.GA473540@bhelgaas> Precedence: bulk X-Mailing-List: linux-acpi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: On Tue, Feb 25, 2025 at 12:29:00AM +0530, Naveen Kumar P wrote: > On Mon, Feb 24, 2025 at 11:03 PM Bjorn Helgaas wrote: > > On Mon, Feb 24, 2025 at 05:45:35PM +0530, Naveen Kumar P wrote: > > > On Wed, Feb 19, 2025 at 10:36 PM Bjorn Helgaas wrote: > > > > On Wed, Feb 19, 2025 at 05:52:47PM +0530, Naveen Kumar P wrote: > > > > > Hi all, > > > > > > > > > > I am writing to seek assistance with an issue we are experiencing with > > > > > a PCIe device (PLDA Device 5555) connected through PCI Express Root > > > > > Port 1 to the host bridge. > > > > > > > > > > We have observed that after booting the system, the Base Address > > > > > Register (BAR0) memory of this device gets reset to 0x0 after > > > > > approximately one hour or more (the timing is inconsistent). This was > > > > > verified using the lspci output and the setpci -s 01:00.0 > > > > > BASE_ADDRESS_0 command. > ... > I booted with the pcie_aspm=off kernel parameter, which means that > PCIe Active State Power Management (ASPM) is disabled. Given this > context, should I consider removing this setting to see if it affects > the occurrence of the Bus Check notifications and the BAR0 reset > issue? Doesn't seem likely to be related. Once configured, ASPM operates without any software intervention. But note that "pcie_aspm=off" means the kernel doesn't touch ASPM configuration at all, and any configuration done by firmware remains in effect. You can tell whether ASPM has been enabled by firmware with "sudo lspci -vv" before the problem occurs. > > > During the ACPI_NOTIFY_BUS_CHECK event, the lspci output initially > > > showed all FF's, and then the next run of the same command showed > > > BASE_ADDRESS_0 reset to zero: > > > $ sudo lspci -xxx -s 01:00.0 | grep "10:" > > > 10: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff > > > > Looks like the device isn't responding at all here. Could happen if > > the device is reset or powered down. > > From the kernel driver or user space tools, is it possible to > determine whether the device has been reset or powered down? Are > there any power management settings or configurations that could be > causing the device to reset or power down unexpectedly? Not really. By "powered down", I meant D3cold, where the main power is removed. Config space is readable in all other power states. > > What is this device? What driver is bound to it? I don't see > > anything in dmesg that identifies a driver. > > The PCIe device in question is a Xilinx FPGA endpoint, which is > flashed with RTL code to expose several host interfaces to the system > via the PCIe link. > > We have an out-of-tree driver for this device, but to eliminate the > driver's role in this issue, I renamed the driver to prevent it from > loading automatically after rebooting the machine. Despite not using > the driver, the issue still occurred. Oh, right, I forgot that you mentioned this before. > > You're seeing the problem on v5.4 (Nov 2019), which is much newer than > > v4.4 (Jan 2016). But v5.4 is still really too old to spend a lot of > > time on unless the problem still happens on a current kernel. This part is important. We don't want to spend a lot of time debugging an issue that may have already been fixed upstream. Bjorn