From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AF0971F93B for ; Fri, 21 Jul 2023 16:08:13 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EF9A4C433D9; Fri, 21 Jul 2023 16:08:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1689955693; bh=LGA32XouZSWJqq0j7YFFUr/XUGfKOFro0mhkNduNZf4=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=T4smuBCWpK0nupF6c3zfXOgsB0Hls9yafxK5mozt4nuL2Isfdn8woAcjtta73YWSV b9YsrIK03VZUT0uL3keo9H3LtdoaP1l/KQwupReDFA7dxcBdLUAlZ3lCduBxxcmo1m NUbMimf41jozFjiONXnwjA9mXuXCYv1w6A9hAvfi0He8dQDt4AEyWMbZcI11DPnE4d FQuVIWNGuWvB7L1rjxLgIGUq1QCi0APMFZR7SOsS14djYbML3h5wVLaAZ1+xclChPD X1wdRUSiXHiSj9ukafEfcKnmw9Xzlk3L7Mdzbacx6YlXopsiiRLz5oSWC0Jwz/D66d b+2LHaYbLqE0A== Date: Fri, 21 Jul 2023 21:37:55 +0530 From: Manivannan Sadhasivam To: Frank Li Cc: Shawn Lin , Manivannan Sadhasivam , helgaas@kernel.org, imx@lists.linux.dev, bhelgaas@google.com, devicetree@vger.kernel.org, gustavo.pimentel@synopsys.com, kw@linux.com, leoyang.li@nxp.com, linux-arm-kernel@lists.infradead.org, linux-imx@nxp.com, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, lorenzo.pieralisi@arm.com, minghuan.lian@nxp.com, mingkai.hu@nxp.com, robh+dt@kernel.org, roy.zang@nxp.com, shawnguo@kernel.org, zhiqiang.hou@nxp.com Subject: Re: [PATCH v3 1/2] PCI: dwc: Implement general suspend/resume functionality for L2/L3 transitions Message-ID: <20230721160755.GD2536@thinkpad> References: <20230419164118.596300-1-Frank.Li@nxp.com> <20230717164526.GC35455@thinkpad> <20230718100400.GB4771@thinkpad> <20230720142509.GB48270@thinkpad> <20230720160738.GC48270@thinkpad> <6f1eb449-5609-0b17-1323-0d114c38d969@rock-chips.com> Precedence: bulk X-Mailing-List: imx@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: On Fri, Jul 21, 2023 at 10:10:11AM -0400, Frank Li wrote: > On Fri, Jul 21, 2023 at 10:09:18AM +0800, Shawn Lin wrote: > > > > On 2023/7/21 0:07, Manivannan Sadhasivam wrote: > > > On Thu, Jul 20, 2023 at 10:37:36AM -0400, Frank Li wrote: > > > > On Thu, Jul 20, 2023 at 07:55:09PM +0530, Manivannan Sadhasivam wrote: > > > > > On Tue, Jul 18, 2023 at 03:34:26PM +0530, Manivannan Sadhasivam wrote: > > > > > > On Mon, Jul 17, 2023 at 02:36:19PM -0400, Frank Li wrote: > > > > > > > On Mon, Jul 17, 2023 at 10:15:26PM +0530, Manivannan Sadhasivam wrote: > > > > > > > > On Wed, Apr 19, 2023 at 12:41:17PM -0400, Frank Li wrote: > > > > > > > > > Introduced helper function dw_pcie_get_ltssm to retrieve SMLH_LTSS_STATE. > > > > > > > > > Added API pme_turn_off and exit_from_l2 for managing L2/L3 state transitions. > > > > > > > > > > > > > > > > > > Typical L2 entry workflow: > > > > > > > > > > > > > > > > > > 1. Transmit PME turn off signal to PCI devices. > > > > > > > > > 2. Await link entering L2_IDLE state. > > > > > > > > > > > > > > > > AFAIK, typical workflow is to wait for PME_To_Ack. > > > > > > > > > > > > > > 1 Already wait for PME_to_ACK, 2, just wait for link actual enter L2. > > > > > > > I think PCI RC needs some time to set link enter L2 after get ACK from > > > > > > > PME. > > > > > > > > > > > > > > > > > One more comment. If you transition the device to L2/L3, then it can loose power > > > > > if Vaux was not provided. In that case, can all the devices work after resume? > > > > > Most notably NVMe? > > > > > > > > I have not hardware to do such test, NVMe driver will reinit everything after > > > > resume if no L1.1\L1.2 support. If there are L1.1\L1.2, NVME expect it leave > > > > at L1.2 at suspend to get better resume latency. > > > > > > > > > > To be precise, NVMe driver will shutdown the device if there is no ASPM support > > > and keep it in low power mode otherwise (there are other cases as well but we do > > > not need to worry). > > > > > > But here you are not checking for ASPM state in the suspend path, and just > > > forcing the link to be in L2/L3 (thereby D3Cold) even though NVMe driver may > > > expect it to be in low power state like ASPM/APST. > > > > > > So you should only put the link to L2/L3 if there is no ASPM support. Otherwise, > > > you'll ending up with bug reports when users connect NVMe to it. > > > > > > > > > At this topic, it's very interesting to look at > > > > drivers/pci/controller/dwc/pcie-tegra194.c > > > > > > static int tegra_pcie_dw_suspend_noirq(struct device *dev) > > { > > struct tegra_pcie_dw *pcie = dev_get_drvdata(dev); > > > > if (!pcie->link_state) > > return 0; > > > > tegra_pcie_downstream_dev_to_D0(pcie); > > tegra_pcie_dw_pme_turnoff(pcie); > > tegra_pcie_unconfig_controller(pcie); > > > > return 0; > > } > > > > It brings back all the downstream components to D0, as I assumed it was L0 > > indeed, before sending PME aiming to enter L2. > > If current state is L1.1 or L1.2, hardware can auto enter to D0\L0 when > there are any PCI bus activity, include PME. I supposed > tegra_pcie_downstream_dev_to_D0() just make sure come back from L2/L3, > which may enter by runtime PM previously, or other reason. > > NVME ASPM problem is (at least when I debug at other platform about 1 year > ago): > > 1. NVME will not release MSI interrupt during suspsend. > 2. PCI controler enter L2 at suspned_noirq(); > 3. CPU hot plug try to down second core (CORE1, CORE2, ...) > 4. GIC try to disable MSI irq by write config space. Just for the record, this will only happen during deep sleep (s2ram) where the CPUs are powered down (including the boot CPU). > 5. panic here because config space can't be access at L2. > > I suposed tegra should have problem when ASPM enable with NVME devices. > NVMe suspend issue has several faces: If NVMe is powered down during suspend, it will result in considerable power savings. But at the same time, the suspend should not happen too frequently as it may deteriorate the lifetime of the device. Most of the recent NVMe devices have 2M power cycles (only). We can workaround the above lifetime issue by powering down the device only during s2ram. It will work great for Laptop use cases if s2ram is supported. Unfortunately, not all Laptops support s2ram though. And if the device is powered down during s2idle, it will hit the above life time issue when it is used in Android platforms such as Tablets (even future mobile phones?) which doesn't support s2ram. So I'm thinking of the following options to address the issue(s): 1. Modify the NVMe driver to power down the device during s2ram (the driver can use the global pm_suspend_target_state flag to detect the suspend state) and use the same logic to put the link into L2/L3 state in the PCIe controller drivers. For s2idle, maintain both the device and link in low power states. 2. Get the power management decision from the userspace and use that to decide the power down logic in s2idle for both NVMe and PCIe drivers. This will ensure that the NVMe device will be powered down in suitable usecases like Laptop without s2ram and kept in low power states for Android platforms. But this is quite an involved task and I don't know if it is possible at all. I'm just dumping my thoughts here. And I plan to intiate a discussion with NVMe/ power folks soon. Maybe during Plumbers? - Mani > Frank > > > > > - Mani > > > > > > > This API help remove duplicate codes and it can be improved gradually. > > > > > > > > > > > > > > > > > > - Mani > > > > > > > > > > > > > > > -- > > > > > மணிவண்ணன் சதாசிவம் > > > -- மணிவண்ணன் சதாசிவம் From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C6A42EB64DD for ; Fri, 21 Jul 2023 16:08:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=IMHYpQwKfIwH32ejhLDRQvTi0pBwaApFDis0chDWW4w=; b=1UFv1LrXPGk9qo GLdw/WmWuRzFifuL/idIoEVuRIQPXz5XVOiqIPi4umRok1DQjWVENQ4CLQt1ifLO10JYkX4J8JkaD W3Y82izE0LgrncdGc6TQuLurGRlYJkJfeo9DvZIcZLF8JbNxA2+JJEsYz+pLmIcyniozM/H4HgK5I 66oAuC3yUOZejZiH7fY0ADP85PnSimIcQTViZwU67wBvhOSGLAWf6jPWmU/BK3hiSae+VWi8CCXPD 6YWR27ROURqLgJVSp3/lh2499XnVDyE4eVtE26663ppj7I5CxdF6SY5XrCcVBEtgdvgBJHfHAtbBO 4SPHT+LFpgurTsWaXLiA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qMsfy-00EWyj-0N; Fri, 21 Jul 2023 16:08:18 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qMsfu-00EWxf-2V for linux-arm-kernel@lists.infradead.org; Fri, 21 Jul 2023 16:08:16 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 0C88061D2D; Fri, 21 Jul 2023 16:08:14 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EF9A4C433D9; Fri, 21 Jul 2023 16:08:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1689955693; bh=LGA32XouZSWJqq0j7YFFUr/XUGfKOFro0mhkNduNZf4=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=T4smuBCWpK0nupF6c3zfXOgsB0Hls9yafxK5mozt4nuL2Isfdn8woAcjtta73YWSV b9YsrIK03VZUT0uL3keo9H3LtdoaP1l/KQwupReDFA7dxcBdLUAlZ3lCduBxxcmo1m NUbMimf41jozFjiONXnwjA9mXuXCYv1w6A9hAvfi0He8dQDt4AEyWMbZcI11DPnE4d FQuVIWNGuWvB7L1rjxLgIGUq1QCi0APMFZR7SOsS14djYbML3h5wVLaAZ1+xclChPD X1wdRUSiXHiSj9ukafEfcKnmw9Xzlk3L7Mdzbacx6YlXopsiiRLz5oSWC0Jwz/D66d b+2LHaYbLqE0A== Date: Fri, 21 Jul 2023 21:37:55 +0530 From: Manivannan Sadhasivam To: Frank Li Cc: Shawn Lin , Manivannan Sadhasivam , helgaas@kernel.org, imx@lists.linux.dev, bhelgaas@google.com, devicetree@vger.kernel.org, gustavo.pimentel@synopsys.com, kw@linux.com, leoyang.li@nxp.com, linux-arm-kernel@lists.infradead.org, linux-imx@nxp.com, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, lorenzo.pieralisi@arm.com, minghuan.lian@nxp.com, mingkai.hu@nxp.com, robh+dt@kernel.org, roy.zang@nxp.com, shawnguo@kernel.org, zhiqiang.hou@nxp.com Subject: Re: [PATCH v3 1/2] PCI: dwc: Implement general suspend/resume functionality for L2/L3 transitions Message-ID: <20230721160755.GD2536@thinkpad> References: <20230419164118.596300-1-Frank.Li@nxp.com> <20230717164526.GC35455@thinkpad> <20230718100400.GB4771@thinkpad> <20230720142509.GB48270@thinkpad> <20230720160738.GC48270@thinkpad> <6f1eb449-5609-0b17-1323-0d114c38d969@rock-chips.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230721_090814_897405_4D621872 X-CRM114-Status: GOOD ( 48.35 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gRnJpLCBKdWwgMjEsIDIwMjMgYXQgMTA6MTA6MTFBTSAtMDQwMCwgRnJhbmsgTGkgd3JvdGU6 Cj4gT24gRnJpLCBKdWwgMjEsIDIwMjMgYXQgMTA6MDk6MThBTSArMDgwMCwgU2hhd24gTGluIHdy b3RlOgo+ID4gCj4gPiBPbiAyMDIzLzcvMjEgMDowNywgTWFuaXZhbm5hbiBTYWRoYXNpdmFtIHdy b3RlOgo+ID4gPiBPbiBUaHUsIEp1bCAyMCwgMjAyMyBhdCAxMDozNzozNkFNIC0wNDAwLCBGcmFu ayBMaSB3cm90ZToKPiA+ID4gPiBPbiBUaHUsIEp1bCAyMCwgMjAyMyBhdCAwNzo1NTowOVBNICsw NTMwLCBNYW5pdmFubmFuIFNhZGhhc2l2YW0gd3JvdGU6Cj4gPiA+ID4gPiBPbiBUdWUsIEp1bCAx OCwgMjAyMyBhdCAwMzozNDoyNlBNICswNTMwLCBNYW5pdmFubmFuIFNhZGhhc2l2YW0gd3JvdGU6 Cj4gPiA+ID4gPiA+IE9uIE1vbiwgSnVsIDE3LCAyMDIzIGF0IDAyOjM2OjE5UE0gLTA0MDAsIEZy YW5rIExpIHdyb3RlOgo+ID4gPiA+ID4gPiA+IE9uIE1vbiwgSnVsIDE3LCAyMDIzIGF0IDEwOjE1 OjI2UE0gKzA1MzAsIE1hbml2YW5uYW4gU2FkaGFzaXZhbSB3cm90ZToKPiA+ID4gPiA+ID4gPiA+ IE9uIFdlZCwgQXByIDE5LCAyMDIzIGF0IDEyOjQxOjE3UE0gLTA0MDAsIEZyYW5rIExpIHdyb3Rl Ogo+ID4gPiA+ID4gPiA+ID4gPiBJbnRyb2R1Y2VkIGhlbHBlciBmdW5jdGlvbiBkd19wY2llX2dl dF9sdHNzbSB0byByZXRyaWV2ZSBTTUxIX0xUU1NfU1RBVEUuCj4gPiA+ID4gPiA+ID4gPiA+IEFk ZGVkIEFQSSBwbWVfdHVybl9vZmYgYW5kIGV4aXRfZnJvbV9sMiBmb3IgbWFuYWdpbmcgTDIvTDMg c3RhdGUgdHJhbnNpdGlvbnMuCj4gPiA+ID4gPiA+ID4gPiA+IAo+ID4gPiA+ID4gPiA+ID4gPiBU eXBpY2FsIEwyIGVudHJ5IHdvcmtmbG93Ogo+ID4gPiA+ID4gPiA+ID4gPiAKPiA+ID4gPiA+ID4g PiA+ID4gMS4gVHJhbnNtaXQgUE1FIHR1cm4gb2ZmIHNpZ25hbCB0byBQQ0kgZGV2aWNlcy4KPiA+ ID4gPiA+ID4gPiA+ID4gMi4gQXdhaXQgbGluayBlbnRlcmluZyBMMl9JRExFIHN0YXRlLgo+ID4g PiA+ID4gPiA+ID4gCj4gPiA+ID4gPiA+ID4gPiBBRkFJSywgdHlwaWNhbCB3b3JrZmxvdyBpcyB0 byB3YWl0IGZvciBQTUVfVG9fQWNrLgo+ID4gPiA+ID4gPiA+IAo+ID4gPiA+ID4gPiA+IDEgQWxy ZWFkeSB3YWl0IGZvciBQTUVfdG9fQUNLLCAgMiwganVzdCB3YWl0IGZvciBsaW5rIGFjdHVhbCBl bnRlciBMMi4KPiA+ID4gPiA+ID4gPiBJIHRoaW5rIFBDSSBSQyBuZWVkcyBzb21lIHRpbWUgdG8g c2V0IGxpbmsgZW50ZXIgTDIgYWZ0ZXIgZ2V0IEFDSyBmcm9tCj4gPiA+ID4gPiA+ID4gUE1FLgo+ ID4gPiA+ID4gPiA+IAo+ID4gPiA+ID4gCj4gPiA+ID4gPiBPbmUgbW9yZSBjb21tZW50LiBJZiB5 b3UgdHJhbnNpdGlvbiB0aGUgZGV2aWNlIHRvIEwyL0wzLCB0aGVuIGl0IGNhbiBsb29zZSBwb3dl cgo+ID4gPiA+ID4gaWYgVmF1eCB3YXMgbm90IHByb3ZpZGVkLiBJbiB0aGF0IGNhc2UsIGNhbiBh bGwgdGhlIGRldmljZXMgd29yayBhZnRlciByZXN1bWU/Cj4gPiA+ID4gPiBNb3N0IG5vdGFibHkg TlZNZT8KPiA+ID4gPiAKPiA+ID4gPiBJIGhhdmUgbm90IGhhcmR3YXJlIHRvIGRvIHN1Y2ggdGVz dCwgTlZNZSBkcml2ZXIgd2lsbCByZWluaXQgZXZlcnl0aGluZyBhZnRlcgo+ID4gPiA+IHJlc3Vt ZSBpZiBubyBMMS4xXEwxLjIgc3VwcG9ydC4gSWYgdGhlcmUgYXJlIEwxLjFcTDEuMiwgTlZNRSBl eHBlY3QgaXQgbGVhdmUKPiA+ID4gPiBhdCBMMS4yIGF0IHN1c3BlbmQgdG8gZ2V0IGJldHRlciBy ZXN1bWUgbGF0ZW5jeS4KPiA+ID4gPiAKPiA+ID4gCj4gPiA+IFRvIGJlIHByZWNpc2UsIE5WTWUg ZHJpdmVyIHdpbGwgc2h1dGRvd24gdGhlIGRldmljZSBpZiB0aGVyZSBpcyBubyBBU1BNIHN1cHBv cnQKPiA+ID4gYW5kIGtlZXAgaXQgaW4gbG93IHBvd2VyIG1vZGUgb3RoZXJ3aXNlICh0aGVyZSBh cmUgb3RoZXIgY2FzZXMgYXMgd2VsbCBidXQgd2UgZG8KPiA+ID4gbm90IG5lZWQgdG8gd29ycnkp Lgo+ID4gPiAKPiA+ID4gQnV0IGhlcmUgeW91IGFyZSBub3QgY2hlY2tpbmcgZm9yIEFTUE0gc3Rh dGUgaW4gdGhlIHN1c3BlbmQgcGF0aCwgYW5kIGp1c3QKPiA+ID4gZm9yY2luZyB0aGUgbGluayB0 byBiZSBpbiBMMi9MMyAodGhlcmVieSBEM0NvbGQpIGV2ZW4gdGhvdWdoIE5WTWUgZHJpdmVyIG1h eQo+ID4gPiBleHBlY3QgaXQgdG8gYmUgaW4gbG93IHBvd2VyIHN0YXRlIGxpa2UgQVNQTS9BUFNU Lgo+ID4gPiAKPiA+ID4gU28geW91IHNob3VsZCBvbmx5IHB1dCB0aGUgbGluayB0byBMMi9MMyBp ZiB0aGVyZSBpcyBubyBBU1BNIHN1cHBvcnQuIE90aGVyd2lzZSwKPiA+ID4geW91J2xsIGVuZGlu ZyB1cCB3aXRoIGJ1ZyByZXBvcnRzIHdoZW4gdXNlcnMgY29ubmVjdCBOVk1lIHRvIGl0Lgo+ID4g PiAKPiA+IAo+ID4gCj4gPiBBdCB0aGlzIHRvcGljLCBpdCdzIHZlcnkgaW50ZXJlc3RpbmcgdG8g bG9vayBhdAo+ID4gCj4gPiBkcml2ZXJzL3BjaS9jb250cm9sbGVyL2R3Yy9wY2llLXRlZ3JhMTk0 LmMKPiA+IAo+ID4gCj4gPiBzdGF0aWMgaW50IHRlZ3JhX3BjaWVfZHdfc3VzcGVuZF9ub2lycShz dHJ1Y3QgZGV2aWNlICpkZXYpCj4gPiB7Cj4gPiAgICAgICAgIHN0cnVjdCB0ZWdyYV9wY2llX2R3 ICpwY2llID0gZGV2X2dldF9kcnZkYXRhKGRldik7Cj4gPiAKPiA+ICAgICAgICAgaWYgKCFwY2ll LT5saW5rX3N0YXRlKQo+ID4gICAgICAgICAgICAgICAgIHJldHVybiAwOwo+ID4gCj4gPiAgICAg ICAgIHRlZ3JhX3BjaWVfZG93bnN0cmVhbV9kZXZfdG9fRDAocGNpZSk7Cj4gPiAgICAgICAgIHRl Z3JhX3BjaWVfZHdfcG1lX3R1cm5vZmYocGNpZSk7Cj4gPiAgICAgICAgIHRlZ3JhX3BjaWVfdW5j b25maWdfY29udHJvbGxlcihwY2llKTsKPiA+IAo+ID4gICAgICAgICByZXR1cm4gMDsKPiA+IH0K PiA+IAo+ID4gSXQgYnJpbmdzIGJhY2sgYWxsIHRoZSBkb3duc3RyZWFtIGNvbXBvbmVudHMgdG8g RDAsIGFzIEkgYXNzdW1lZCBpdCB3YXMgTDAKPiA+IGluZGVlZCwgYmVmb3JlIHNlbmRpbmcgUE1F IGFpbWluZyB0byBlbnRlciBMMi4KPiAKPiBJZiBjdXJyZW50IHN0YXRlIGlzIEwxLjEgb3IgTDEu MiwgaGFyZHdhcmUgY2FuIGF1dG8gZW50ZXIgdG8gRDBcTDAgd2hlbgo+IHRoZXJlIGFyZSBhbnkg UENJIGJ1cyBhY3Rpdml0eSwgaW5jbHVkZSBQTUUuIEkgc3VwcG9zZWQKPiB0ZWdyYV9wY2llX2Rv d25zdHJlYW1fZGV2X3RvX0QwKCkganVzdCBtYWtlIHN1cmUgY29tZSBiYWNrIGZyb20gTDIvTDMs Cj4gd2hpY2ggbWF5IGVudGVyIGJ5IHJ1bnRpbWUgUE0gcHJldmlvdXNseSwgb3Igb3RoZXIgcmVh c29uLgo+IAo+IE5WTUUgQVNQTSBwcm9ibGVtIGlzIChhdCBsZWFzdCB3aGVuIEkgZGVidWcgYXQg b3RoZXIgcGxhdGZvcm0gYWJvdXQgMSB5ZWFyCj4gYWdvKTogCj4gCj4gICAgIDEuIE5WTUUgd2ls bCBub3QgcmVsZWFzZSBNU0kgaW50ZXJydXB0IGR1cmluZyBzdXNwc2VuZC4KPiAgICAgMi4gUENJ IGNvbnRyb2xlciBlbnRlciBMMiBhdCBzdXNwbmVkX25vaXJxKCk7Cj4gICAgIDMuIENQVSBob3Qg cGx1ZyB0cnkgdG8gZG93biBzZWNvbmQgY29yZSAoQ09SRTEsIENPUkUyLCAuLi4pCj4gICAgIDQu IEdJQyB0cnkgdG8gZGlzYWJsZSBNU0kgaXJxIGJ5IHdyaXRlIGNvbmZpZyBzcGFjZS4KCkp1c3Qg Zm9yIHRoZSByZWNvcmQsIHRoaXMgd2lsbCBvbmx5IGhhcHBlbiBkdXJpbmcgZGVlcCBzbGVlcCAo czJyYW0pIHdoZXJlIHRoZQpDUFVzIGFyZSBwb3dlcmVkIGRvd24gKGluY2x1ZGluZyB0aGUgYm9v dCBDUFUpLgoKPiAgICAgNS4gcGFuaWMgaGVyZSBiZWNhdXNlIGNvbmZpZyBzcGFjZSBjYW4ndCBi ZSBhY2Nlc3MgYXQgTDIuCj4gCj4gSSBzdXBvc2VkIHRlZ3JhIHNob3VsZCBoYXZlIHByb2JsZW0g d2hlbiBBU1BNIGVuYWJsZSB3aXRoIE5WTUUgZGV2aWNlcy4KPiAKCk5WTWUgc3VzcGVuZCBpc3N1 ZSBoYXMgc2V2ZXJhbCBmYWNlczoKCklmIE5WTWUgaXMgcG93ZXJlZCBkb3duIGR1cmluZyBzdXNw ZW5kLCBpdCB3aWxsIHJlc3VsdCBpbiBjb25zaWRlcmFibGUgcG93ZXIKc2F2aW5ncy4gQnV0IGF0 IHRoZSBzYW1lIHRpbWUsIHRoZSBzdXNwZW5kIHNob3VsZCBub3QgaGFwcGVuIHRvbyBmcmVxdWVu dGx5IGFzCml0IG1heSBkZXRlcmlvcmF0ZSB0aGUgbGlmZXRpbWUgb2YgdGhlIGRldmljZS4gTW9z dCBvZiB0aGUgcmVjZW50IE5WTWUgZGV2aWNlcwpoYXZlIDJNIHBvd2VyIGN5Y2xlcyAob25seSku CgpXZSBjYW4gd29ya2Fyb3VuZCB0aGUgYWJvdmUgbGlmZXRpbWUgaXNzdWUgYnkgcG93ZXJpbmcg ZG93biB0aGUgZGV2aWNlIG9ubHkKZHVyaW5nIHMycmFtLiBJdCB3aWxsIHdvcmsgZ3JlYXQgZm9y IExhcHRvcCB1c2UgY2FzZXMgaWYgczJyYW0gaXMgc3VwcG9ydGVkLgpVbmZvcnR1bmF0ZWx5LCBu b3QgYWxsIExhcHRvcHMgc3VwcG9ydCBzMnJhbSB0aG91Z2guIEFuZCBpZiB0aGUgZGV2aWNlIGlz CnBvd2VyZWQgZG93biBkdXJpbmcgczJpZGxlLCBpdCB3aWxsIGhpdCB0aGUgYWJvdmUgbGlmZSB0 aW1lIGlzc3VlIHdoZW4gaXQgaXMKdXNlZCBpbiBBbmRyb2lkIHBsYXRmb3JtcyBzdWNoIGFzIFRh YmxldHMgKGV2ZW4gZnV0dXJlIG1vYmlsZSBwaG9uZXM/KSB3aGljaApkb2Vzbid0IHN1cHBvcnQg czJyYW0uCgpTbyBJJ20gdGhpbmtpbmcgb2YgdGhlIGZvbGxvd2luZyBvcHRpb25zIHRvIGFkZHJl c3MgdGhlIGlzc3VlKHMpOgoKMS4gTW9kaWZ5IHRoZSBOVk1lIGRyaXZlciB0byBwb3dlciBkb3du IHRoZSBkZXZpY2UgZHVyaW5nIHMycmFtICh0aGUgZHJpdmVyIGNhbgp1c2UgdGhlIGdsb2JhbCBw bV9zdXNwZW5kX3RhcmdldF9zdGF0ZSBmbGFnIHRvIGRldGVjdCB0aGUgc3VzcGVuZCBzdGF0ZSkg YW5kIHVzZQp0aGUgc2FtZSBsb2dpYyB0byBwdXQgdGhlIGxpbmsgaW50byBMMi9MMyBzdGF0ZSBp biB0aGUgUENJZSBjb250cm9sbGVyIGRyaXZlcnMuCkZvciBzMmlkbGUsIG1haW50YWluIGJvdGgg dGhlIGRldmljZSBhbmQgbGluayBpbiBsb3cgcG93ZXIgc3RhdGVzLgoKMi4gR2V0IHRoZSBwb3dl ciBtYW5hZ2VtZW50IGRlY2lzaW9uIGZyb20gdGhlIHVzZXJzcGFjZSBhbmQgdXNlIHRoYXQgdG8g ZGVjaWRlCnRoZSBwb3dlciBkb3duIGxvZ2ljIGluIHMyaWRsZSBmb3IgYm90aCBOVk1lIGFuZCBQ Q0llIGRyaXZlcnMuIFRoaXMgd2lsbCBlbnN1cmUKdGhhdCB0aGUgTlZNZSBkZXZpY2Ugd2lsbCBi ZSBwb3dlcmVkIGRvd24gaW4gc3VpdGFibGUgdXNlY2FzZXMgbGlrZSBMYXB0b3AKd2l0aG91dCBz MnJhbSBhbmQga2VwdCBpbiBsb3cgcG93ZXIgc3RhdGVzIGZvciBBbmRyb2lkIHBsYXRmb3Jtcy4g QnV0IHRoaXMgaXMKcXVpdGUgYW4gaW52b2x2ZWQgdGFzayBhbmQgSSBkb24ndCBrbm93IGlmIGl0 IGlzIHBvc3NpYmxlIGF0IGFsbC4KCkknbSBqdXN0IGR1bXBpbmcgbXkgdGhvdWdodHMgaGVyZS4g QW5kIEkgcGxhbiB0byBpbnRpYXRlIGEgZGlzY3Vzc2lvbiB3aXRoIE5WTWUvCnBvd2VyIGZvbGtz IHNvb24uIE1heWJlIGR1cmluZyBQbHVtYmVycz8KCi0gTWFuaQoKPiBGcmFuawo+ID4gCj4gPiA+ IC0gTWFuaQo+ID4gPiAKPiA+ID4gPiBUaGlzIEFQSSBoZWxwIHJlbW92ZSBkdXBsaWNhdGUgY29k ZXMgYW5kIGl0IGNhbiBiZSBpbXByb3ZlZCBncmFkdWFsbHkuCj4gPiA+ID4gCj4gPiA+ID4gCj4g PiA+ID4gPiAKPiA+ID4gPiA+IC0gTWFuaQo+ID4gPiA+ID4gCj4gPiA+ID4gPiAKPiA+ID4gPiA+ IC0tIAo+ID4gPiA+ID4g4K6u4K6j4K6/4K614K6j4K+N4K6j4K6p4K+NIOCumuCupOCuvuCumuCu v+CuteCuruCvjQo+ID4gPiAKCi0tIArgrq7grqPgrr/grrXgrqPgr43grqPgrqngr40g4K6a4K6k 4K6+4K6a4K6/4K614K6u4K+NCgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fXwpsaW51eC1hcm0ta2VybmVsIG1haWxpbmcgbGlzdApsaW51eC1hcm0ta2VybmVs QGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVhZC5vcmcvbWFpbG1hbi9s aXN0aW5mby9saW51eC1hcm0ta2VybmVsCg==