From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 431F618C32 for ; Thu, 20 Jul 2023 16:07:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 07552C433C8; Thu, 20 Jul 2023 16:07:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1689869275; bh=Ou2Mh5nraWXlzNwKruRqfrLeoeDKp8beV04BRuHhYwo=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=QDPRzplSB7rDPnYbyMPv7JcU43dmdgPKHwR+inxmoJlN/LC+4kmsPvpf8Knct6shU sCOH6u11Hx9i/jQ2yC4biJjDAzRXm5N8sqvIwdD4VEWrj/C4+YWp7EsyQpdfxpT6ET tzeaLBWvvINiKVnxRMtgpdevhG222aUIR+jswm5aaKFNKFPVn0GiCjEoHjmol1oReF I+gfEBo9fvPHIYRqyQdoo26D7Hm05LZYqdQctEY8AmxDINlc1Ndus9EtZSsuf2xrcd zWHtOCKOf76wMwe+Fm2eio8zTnhLH4lMCKp6zTr3VlFwhJ88+I6HjXJjBmaL7m6z5x ddQTciqQJmiKg== Date: Thu, 20 Jul 2023 21:37:38 +0530 From: Manivannan Sadhasivam To: Frank Li Cc: Manivannan Sadhasivam , helgaas@kernel.org, imx@lists.linux.dev, bhelgaas@google.com, devicetree@vger.kernel.org, gustavo.pimentel@synopsys.com, kw@linux.com, leoyang.li@nxp.com, linux-arm-kernel@lists.infradead.org, linux-imx@nxp.com, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, lorenzo.pieralisi@arm.com, minghuan.lian@nxp.com, mingkai.hu@nxp.com, robh+dt@kernel.org, roy.zang@nxp.com, shawnguo@kernel.org, zhiqiang.hou@nxp.com Subject: Re: [PATCH v3 1/2] PCI: dwc: Implement general suspend/resume functionality for L2/L3 transitions Message-ID: <20230720160738.GC48270@thinkpad> References: <20230419164118.596300-1-Frank.Li@nxp.com> <20230717164526.GC35455@thinkpad> <20230718100400.GB4771@thinkpad> <20230720142509.GB48270@thinkpad> Precedence: bulk X-Mailing-List: imx@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: On Thu, Jul 20, 2023 at 10:37:36AM -0400, Frank Li wrote: > On Thu, Jul 20, 2023 at 07:55:09PM +0530, Manivannan Sadhasivam wrote: > > On Tue, Jul 18, 2023 at 03:34:26PM +0530, Manivannan Sadhasivam wrote: > > > On Mon, Jul 17, 2023 at 02:36:19PM -0400, Frank Li wrote: > > > > On Mon, Jul 17, 2023 at 10:15:26PM +0530, Manivannan Sadhasivam wrote: > > > > > On Wed, Apr 19, 2023 at 12:41:17PM -0400, Frank Li wrote: > > > > > > Introduced helper function dw_pcie_get_ltssm to retrieve SMLH_LTSS_STATE. > > > > > > Added API pme_turn_off and exit_from_l2 for managing L2/L3 state transitions. > > > > > > > > > > > > Typical L2 entry workflow: > > > > > > > > > > > > 1. Transmit PME turn off signal to PCI devices. > > > > > > 2. Await link entering L2_IDLE state. > > > > > > > > > > AFAIK, typical workflow is to wait for PME_To_Ack. > > > > > > > > 1 Already wait for PME_to_ACK, 2, just wait for link actual enter L2. > > > > I think PCI RC needs some time to set link enter L2 after get ACK from > > > > PME. > > > > > > > > One more comment. If you transition the device to L2/L3, then it can loose power > > if Vaux was not provided. In that case, can all the devices work after resume? > > Most notably NVMe? > > I have not hardware to do such test, NVMe driver will reinit everything after > resume if no L1.1\L1.2 support. If there are L1.1\L1.2, NVME expect it leave > at L1.2 at suspend to get better resume latency. > To be precise, NVMe driver will shutdown the device if there is no ASPM support and keep it in low power mode otherwise (there are other cases as well but we do not need to worry). But here you are not checking for ASPM state in the suspend path, and just forcing the link to be in L2/L3 (thereby D3Cold) even though NVMe driver may expect it to be in low power state like ASPM/APST. So you should only put the link to L2/L3 if there is no ASPM support. Otherwise, you'll ending up with bug reports when users connect NVMe to it. - Mani > This API help remove duplicate codes and it can be improved gradually. > > > > > > - Mani > > > > > > -- > > மணிவண்ணன் சதாசிவம் -- மணிவண்ணன் சதாசிவம் From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 24F86EB64DC for ; Thu, 20 Jul 2023 16:08:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=P9QkTmJD4+bY4dWUUYVGrMPFYjH3ThQ/1ci0QZp9Ozk=; b=mc5cCqVl7kX20w vMsGsL2ZXLexmilHS+Ztu/EPY7eXvHTNbXV9LSmPGyh5OOtWdGlRsTukYw6VBBSZopt18Y8hALiGp 1V3QC8ANhakTMqVLg8bpQ/Q8iGicm4ryNEESCGGPkSdEyM93XO8vb2B2L0BvGiYt8ODLmyXbbPfxX ivZKO7vdvAhGBTcnp7DGxGBnjrKxsx3ILanu8Drw8+MwuUlWm/ICUrCEDnY81tm7LqLnw2WLdSrxh RuhlNXoyblBAxaASMq2aC2W/OqhI/2sP8ruxm21QMV89jqP3jIVYYhK6XeUC7E9Ur9bEoHpPPwjrX JsMQpk86vCbsoNVs69Eg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qMWC8-00BdZh-1q; Thu, 20 Jul 2023 16:08:00 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qMWC5-00BdZ7-0f for linux-arm-kernel@lists.infradead.org; Thu, 20 Jul 2023 16:07:58 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 655BA61B4E; Thu, 20 Jul 2023 16:07:56 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 07552C433C8; Thu, 20 Jul 2023 16:07:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1689869275; bh=Ou2Mh5nraWXlzNwKruRqfrLeoeDKp8beV04BRuHhYwo=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=QDPRzplSB7rDPnYbyMPv7JcU43dmdgPKHwR+inxmoJlN/LC+4kmsPvpf8Knct6shU sCOH6u11Hx9i/jQ2yC4biJjDAzRXm5N8sqvIwdD4VEWrj/C4+YWp7EsyQpdfxpT6ET tzeaLBWvvINiKVnxRMtgpdevhG222aUIR+jswm5aaKFNKFPVn0GiCjEoHjmol1oReF I+gfEBo9fvPHIYRqyQdoo26D7Hm05LZYqdQctEY8AmxDINlc1Ndus9EtZSsuf2xrcd zWHtOCKOf76wMwe+Fm2eio8zTnhLH4lMCKp6zTr3VlFwhJ88+I6HjXJjBmaL7m6z5x ddQTciqQJmiKg== Date: Thu, 20 Jul 2023 21:37:38 +0530 From: Manivannan Sadhasivam To: Frank Li Cc: Manivannan Sadhasivam , helgaas@kernel.org, imx@lists.linux.dev, bhelgaas@google.com, devicetree@vger.kernel.org, gustavo.pimentel@synopsys.com, kw@linux.com, leoyang.li@nxp.com, linux-arm-kernel@lists.infradead.org, linux-imx@nxp.com, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, lorenzo.pieralisi@arm.com, minghuan.lian@nxp.com, mingkai.hu@nxp.com, robh+dt@kernel.org, roy.zang@nxp.com, shawnguo@kernel.org, zhiqiang.hou@nxp.com Subject: Re: [PATCH v3 1/2] PCI: dwc: Implement general suspend/resume functionality for L2/L3 transitions Message-ID: <20230720160738.GC48270@thinkpad> References: <20230419164118.596300-1-Frank.Li@nxp.com> <20230717164526.GC35455@thinkpad> <20230718100400.GB4771@thinkpad> <20230720142509.GB48270@thinkpad> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230720_090757_348064_6011409C X-CRM114-Status: GOOD ( 21.83 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gVGh1LCBKdWwgMjAsIDIwMjMgYXQgMTA6Mzc6MzZBTSAtMDQwMCwgRnJhbmsgTGkgd3JvdGU6 Cj4gT24gVGh1LCBKdWwgMjAsIDIwMjMgYXQgMDc6NTU6MDlQTSArMDUzMCwgTWFuaXZhbm5hbiBT YWRoYXNpdmFtIHdyb3RlOgo+ID4gT24gVHVlLCBKdWwgMTgsIDIwMjMgYXQgMDM6MzQ6MjZQTSAr MDUzMCwgTWFuaXZhbm5hbiBTYWRoYXNpdmFtIHdyb3RlOgo+ID4gPiBPbiBNb24sIEp1bCAxNywg MjAyMyBhdCAwMjozNjoxOVBNIC0wNDAwLCBGcmFuayBMaSB3cm90ZToKPiA+ID4gPiBPbiBNb24s IEp1bCAxNywgMjAyMyBhdCAxMDoxNToyNlBNICswNTMwLCBNYW5pdmFubmFuIFNhZGhhc2l2YW0g d3JvdGU6Cj4gPiA+ID4gPiBPbiBXZWQsIEFwciAxOSwgMjAyMyBhdCAxMjo0MToxN1BNIC0wNDAw LCBGcmFuayBMaSB3cm90ZToKPiA+ID4gPiA+ID4gSW50cm9kdWNlZCBoZWxwZXIgZnVuY3Rpb24g ZHdfcGNpZV9nZXRfbHRzc20gdG8gcmV0cmlldmUgU01MSF9MVFNTX1NUQVRFLgo+ID4gPiA+ID4g PiBBZGRlZCBBUEkgcG1lX3R1cm5fb2ZmIGFuZCBleGl0X2Zyb21fbDIgZm9yIG1hbmFnaW5nIEwy L0wzIHN0YXRlIHRyYW5zaXRpb25zLgo+ID4gPiA+ID4gPiAKPiA+ID4gPiA+ID4gVHlwaWNhbCBM MiBlbnRyeSB3b3JrZmxvdzoKPiA+ID4gPiA+ID4gCj4gPiA+ID4gPiA+IDEuIFRyYW5zbWl0IFBN RSB0dXJuIG9mZiBzaWduYWwgdG8gUENJIGRldmljZXMuCj4gPiA+ID4gPiA+IDIuIEF3YWl0IGxp bmsgZW50ZXJpbmcgTDJfSURMRSBzdGF0ZS4KPiA+ID4gPiA+IAo+ID4gPiA+ID4gQUZBSUssIHR5 cGljYWwgd29ya2Zsb3cgaXMgdG8gd2FpdCBmb3IgUE1FX1RvX0Fjay4KPiA+ID4gPiAKPiA+ID4g PiAxIEFscmVhZHkgd2FpdCBmb3IgUE1FX3RvX0FDSywgIDIsIGp1c3Qgd2FpdCBmb3IgbGluayBh Y3R1YWwgZW50ZXIgTDIuCj4gPiA+ID4gSSB0aGluayBQQ0kgUkMgbmVlZHMgc29tZSB0aW1lIHRv IHNldCBsaW5rIGVudGVyIEwyIGFmdGVyIGdldCBBQ0sgZnJvbQo+ID4gPiA+IFBNRS4KPiA+ID4g PiAKPiA+IAo+ID4gT25lIG1vcmUgY29tbWVudC4gSWYgeW91IHRyYW5zaXRpb24gdGhlIGRldmlj ZSB0byBMMi9MMywgdGhlbiBpdCBjYW4gbG9vc2UgcG93ZXIKPiA+IGlmIFZhdXggd2FzIG5vdCBw cm92aWRlZC4gSW4gdGhhdCBjYXNlLCBjYW4gYWxsIHRoZSBkZXZpY2VzIHdvcmsgYWZ0ZXIgcmVz dW1lPwo+ID4gTW9zdCBub3RhYmx5IE5WTWU/Cj4gCj4gSSBoYXZlIG5vdCBoYXJkd2FyZSB0byBk byBzdWNoIHRlc3QsIE5WTWUgZHJpdmVyIHdpbGwgcmVpbml0IGV2ZXJ5dGhpbmcgYWZ0ZXIKPiBy ZXN1bWUgaWYgbm8gTDEuMVxMMS4yIHN1cHBvcnQuIElmIHRoZXJlIGFyZSBMMS4xXEwxLjIsIE5W TUUgZXhwZWN0IGl0IGxlYXZlCj4gYXQgTDEuMiBhdCBzdXNwZW5kIHRvIGdldCBiZXR0ZXIgcmVz dW1lIGxhdGVuY3kuCj4gCgpUbyBiZSBwcmVjaXNlLCBOVk1lIGRyaXZlciB3aWxsIHNodXRkb3du IHRoZSBkZXZpY2UgaWYgdGhlcmUgaXMgbm8gQVNQTSBzdXBwb3J0CmFuZCBrZWVwIGl0IGluIGxv dyBwb3dlciBtb2RlIG90aGVyd2lzZSAodGhlcmUgYXJlIG90aGVyIGNhc2VzIGFzIHdlbGwgYnV0 IHdlIGRvCm5vdCBuZWVkIHRvIHdvcnJ5KS4KCkJ1dCBoZXJlIHlvdSBhcmUgbm90IGNoZWNraW5n IGZvciBBU1BNIHN0YXRlIGluIHRoZSBzdXNwZW5kIHBhdGgsIGFuZCBqdXN0CmZvcmNpbmcgdGhl IGxpbmsgdG8gYmUgaW4gTDIvTDMgKHRoZXJlYnkgRDNDb2xkKSBldmVuIHRob3VnaCBOVk1lIGRy aXZlciBtYXkKZXhwZWN0IGl0IHRvIGJlIGluIGxvdyBwb3dlciBzdGF0ZSBsaWtlIEFTUE0vQVBT VC4KClNvIHlvdSBzaG91bGQgb25seSBwdXQgdGhlIGxpbmsgdG8gTDIvTDMgaWYgdGhlcmUgaXMg bm8gQVNQTSBzdXBwb3J0LiBPdGhlcndpc2UsCnlvdSdsbCBlbmRpbmcgdXAgd2l0aCBidWcgcmVw b3J0cyB3aGVuIHVzZXJzIGNvbm5lY3QgTlZNZSB0byBpdC4KCi0gTWFuaQoKPiBUaGlzIEFQSSBo ZWxwIHJlbW92ZSBkdXBsaWNhdGUgY29kZXMgYW5kIGl0IGNhbiBiZSBpbXByb3ZlZCBncmFkdWFs bHkuCj4gCj4gCj4gPiAKPiA+IC0gTWFuaQo+ID4gCj4gPiAKPiA+IC0tIAo+ID4g4K6u4K6j4K6/ 4K614K6j4K+N4K6j4K6p4K+NIOCumuCupOCuvuCumuCuv+CuteCuruCvjQoKLS0gCuCuruCuo+Cu v+CuteCuo+CvjeCuo+CuqeCvjSDgrprgrqTgrr7grprgrr/grrXgrq7gr40KCl9fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCmxpbnV4LWFybS1rZXJuZWwgbWFp bGluZyBsaXN0CmxpbnV4LWFybS1rZXJuZWxAbGlzdHMuaW5mcmFkZWFkLm9yZwpodHRwOi8vbGlz dHMuaW5mcmFkZWFkLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xpbnV4LWFybS1rZXJuZWwK