From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; dkim=none Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 704DCD7F; Wed, 29 Nov 2023 08:35:08 -0800 (PST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0DED0C15; Wed, 29 Nov 2023 08:35:55 -0800 (PST) Received: from FVFF77S0Q05N.cambridge.arm.com (FVFF77S0Q05N.cambridge.arm.com [10.1.31.168]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 16EB33F73F; Wed, 29 Nov 2023 08:35:06 -0800 (PST) Date: Wed, 29 Nov 2023 16:35:01 +0000 From: Mark Rutland To: "Ashley, William" Cc: "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , "linux-perf-users@vger.kernel.org" , will@kernel.org Subject: Re: armv8pmu: Pending overflow interrupt is discarded when perf event is disabled Message-ID: References: <950001BD-490C-4BAC-8EEA-CDB9F7C4ADFC@amazon.com> Precedence: bulk X-Mailing-List: linux-perf-users@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Hi William, On Mon, Nov 20, 2023 at 10:32:10PM +0000, Ashley, William wrote: > Adding linux-arm-kernel@lists.infradead.org and linux-kernel@vger.kernel.org, > sorry for the noise. Thanks for this! For the benefit of others, the original mail (and attachment) can be found at: https://lore.kernel.org/linux-perf-users/950001BD-490C-4BAC-8EEA-CDB9F7C4ADFC@amazon.com/ For some reason, links and whitespace seem to have been mangled in the resend; I'm not sure what happened there. I've added Will Deacon, as Will and I co-maintain the ARM PMU drivers. > On 11/20/23, 12:36 PM, "Ashley, William" > wrote: > > > An issue [1] was opened in the rr-debugger project reporting occasional missed > perf event overflow signals on arm64. I've been digging into this and think I > understand what's happening, but wanted to confirm my understanding. > > The attached example application, derived from an rr-debugger test case, reports > when the value of a counter doesn't increase by the expected period +/- some > tolerance. When it is ping-ponged between cores (e.g. with taskset) at a high > frequency, it frequently reports increases of ~2x the expected. I've confirmed > this same behavior on kernels 5.4, 5.10, 6.1 and 6.5. > > > I found armv8pmu_disable_intens [2] that is called as part of event > de-scheduling and contains > /* Clear the overflow flag in case an interrupt is pending. */ > write_pmovsclr(mask); > which results in any pending overflow interrupt being dropped. I added some > debug output here and indeed there is a correlation of this bit being high at > the point of the above code and the reproducer identifying a missed signal. I think you're right that if we had an overflow asserted at this point, we'll throw away the occurrence of the overflow (and so not call perf_event_overflow() and generate a sample, etc). It looks like we only lose the occurrence of the overflow; the actual counts will get sampled correctly and when we next reprogram the event, armpmu_event_set_period() should set up the next overflow period. > This behavior does not occur with pseudo-NMIs (irqchip.gicv3_pseudo_nmi=1) > enabled. That's interesting, because it implies that the PMU overflow interrupt is being recognised by the CPU while regular interrupts are disabled. There are some narrow races where that could occur (e.g. taking a timer or scheduler IRQ *just* as an overflow occurs), and some other cases I'd expect RR to avoid by construction (e.g. if RR isn't using mode exclusion and also counts kernel events). It's also worth noting that this means there are races even with pseudo-NMI where overflows could be lost. How often do you see the overflow being lost? Does RR set any of the perf_event_attr::exclude_* bits? If not, does RR intentionally count events that occur within the kernel? > When an event is not being explicitly torn down (e.g. being closed), this seems > like an undesirable behavior. I agree it's undesirable, though my understanding is that for most other users this isn't any worse than losing samples for other reasons (e.g. the perf ring buffer being full), and for the general case of sample-based profiling, losing a sample every so often isn't the end of the world. > I haven't attempted to demo it yet, but I suspect > an application disabling an event temporarily could occasionally see the same > missed overflow signals. Is my understanding here correct? That sounds right to me, though I haven't checked that end-to-end yet. > Does anyone have thoughts on how this could be addressed without creating > other issues? We should be able to detect overflow from the counter value alone, so we might be able to account for that when we actually read the event out, or when we schedule it back in and reprogram the period. I'm not sure if we can reasonably do that when scheduling the event out, since if we're switching tasks, that'll queue up IRQ work which will be triggered in the context of the next task. We might be able to figure out that we have an overflow when we schedule the event in under armpmu_start(), but I'll need to go digging to see if there are any free-running counters as the comment above the call to armpmu_event_set_period() says, or whether that's a historical artifact. I suspect we might need to restructure the code somewhat to be able to catch overflows more consistently. I'll see if I can come up with something, but we might not be able to guarantee this in all cases. Mark. > [1] https://github.com/rr-debugger/rr/issues/3607 > [2] https://github.com/torvalds/linux/blob/c42d9eeef8e5ba9292eda36fd8e3c11f35ee065c/drivers/perf/arm_pmuv3.c#L652C20-L652C43 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 09AE4C4167B for ; Wed, 29 Nov 2023 16:35:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=tZ4OlJkpPmnTceb1gH/3VE3uG7ARIIfaZztopIuEw74=; b=Bf8KP0yFqBkD0R qJwOgisB8KkDWx55nv7/IRsHbm4XxubqjibyN/9btmFrWi8a+2nX+92vFlBMSA7Twh2pUEcheRScI SO29COcwoFgmlGed2xkt+IN+xfiTZnLCDmKMGtFWEWG9DZHIKwGxMlbJNmHLvqjFjNc0Y1SJqsRnp pMgBIJbCXFrK5KQfR6vzuG4HKehm7UKR4kHCAwyb8rzgkzeZylZ8Qrnacf8OxgBo4ACFc2RWEGJLh XBpPEXUAGM/jxgals2wcOkiDn5DXl525XuTlgE8E4Z544Bvd2TszEJsxG/Hyp5+8pIFXX5OuadeW8 cJLDxbOaK4S518B3OI3w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1r8NWx-008uJR-0W; Wed, 29 Nov 2023 16:35:19 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1r8NWr-008uEv-03 for linux-arm-kernel@lists.infradead.org; Wed, 29 Nov 2023 16:35:14 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0DED0C15; Wed, 29 Nov 2023 08:35:55 -0800 (PST) Received: from FVFF77S0Q05N.cambridge.arm.com (FVFF77S0Q05N.cambridge.arm.com [10.1.31.168]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 16EB33F73F; Wed, 29 Nov 2023 08:35:06 -0800 (PST) Date: Wed, 29 Nov 2023 16:35:01 +0000 From: Mark Rutland To: "Ashley, William" Cc: "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , "linux-perf-users@vger.kernel.org" , will@kernel.org Subject: Re: armv8pmu: Pending overflow interrupt is discarded when perf event is disabled Message-ID: References: <950001BD-490C-4BAC-8EEA-CDB9F7C4ADFC@amazon.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231129_083513_153366_BD41CE40 X-CRM114-Status: GOOD ( 35.39 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org SGkgV2lsbGlhbSwKCk9uIE1vbiwgTm92IDIwLCAyMDIzIGF0IDEwOjMyOjEwUE0gKzAwMDAsIEFz aGxleSwgV2lsbGlhbSB3cm90ZToKPiBBZGRpbmcgbGludXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZy YWRlYWQub3JnIGFuZCBsaW51eC1rZXJuZWxAdmdlci5rZXJuZWwub3JnLAo+IHNvcnJ5IGZvciB0 aGUgbm9pc2UuCgpUaGFua3MgZm9yIHRoaXMhCgpGb3IgdGhlIGJlbmVmaXQgb2Ygb3RoZXJzLCB0 aGUgb3JpZ2luYWwgbWFpbCAoYW5kIGF0dGFjaG1lbnQpIGNhbiBiZSBmb3VuZCBhdDoKCiAgaHR0 cHM6Ly9sb3JlLmtlcm5lbC5vcmcvbGludXgtcGVyZi11c2Vycy85NTAwMDFCRC00OTBDLTRCQUMt OEVFQS1DREI5RjdDNEFERkNAYW1hem9uLmNvbS8KCkZvciBzb21lIHJlYXNvbiwgbGlua3MgYW5k IHdoaXRlc3BhY2Ugc2VlbSB0byBoYXZlIGJlZW4gbWFuZ2xlZCBpbiB0aGUgcmVzZW5kOwpJJ20g bm90IHN1cmUgd2hhdCBoYXBwZW5lZCB0aGVyZS4KCkkndmUgYWRkZWQgV2lsbCBEZWFjb24sIGFz IFdpbGwgYW5kIEkgY28tbWFpbnRhaW4gdGhlIEFSTSBQTVUgZHJpdmVycy4KCj4g77u/T24gMTEv MjAvMjMsIDEyOjM2IFBNLCAiQXNobGV5LCBXaWxsaWFtIiA8d2FzaEBhbWF6b24uY29tIDxtYWls dG86d2FzaEBhbWF6b24uY29tPj4gd3JvdGU6Cj4gCj4gCj4gQW4gaXNzdWUgWzFdIHdhcyBvcGVu ZWQgaW4gdGhlIHJyLWRlYnVnZ2VyIHByb2plY3QgcmVwb3J0aW5nIG9jY2FzaW9uYWwgbWlzc2Vk Cj4gcGVyZiBldmVudCBvdmVyZmxvdyBzaWduYWxzIG9uIGFybTY0LiBJJ3ZlIGJlZW4gZGlnZ2lu ZyBpbnRvIHRoaXMgYW5kIHRoaW5rIEkKPiB1bmRlcnN0YW5kIHdoYXQncyBoYXBwZW5pbmcsIGJ1 dCB3YW50ZWQgdG8gY29uZmlybSBteSB1bmRlcnN0YW5kaW5nLgo+IAo+IFRoZSBhdHRhY2hlZCBl eGFtcGxlIGFwcGxpY2F0aW9uLCBkZXJpdmVkIGZyb20gYW4gcnItZGVidWdnZXIgdGVzdCBjYXNl LCByZXBvcnRzCj4gd2hlbiB0aGUgdmFsdWUgb2YgYSBjb3VudGVyIGRvZXNuJ3QgaW5jcmVhc2Ug YnkgdGhlIGV4cGVjdGVkIHBlcmlvZCArLy0gc29tZQo+IHRvbGVyYW5jZS4gV2hlbiBpdCBpcyBw aW5nLXBvbmdlZCBiZXR3ZWVuIGNvcmVzIChlLmcuIHdpdGggdGFza3NldCkgYXQgYSBoaWdoCj4g ZnJlcXVlbmN5LCBpdCBmcmVxdWVudGx5IHJlcG9ydHMgaW5jcmVhc2VzIG9mIH4yeCB0aGUgZXhw ZWN0ZWQuIEkndmUgY29uZmlybWVkCj4gdGhpcyBzYW1lIGJlaGF2aW9yIG9uIGtlcm5lbHMgNS40 LCA1LjEwLCA2LjEgYW5kIDYuNS4KPiAKPiAKPiBJIGZvdW5kIGFybXY4cG11X2Rpc2FibGVfaW50 ZW5zIFsyXSB0aGF0IGlzIGNhbGxlZCBhcyBwYXJ0IG9mIGV2ZW50Cj4gZGUtc2NoZWR1bGluZyBh bmQgY29udGFpbnMKPiAvKiBDbGVhciB0aGUgb3ZlcmZsb3cgZmxhZyBpbiBjYXNlIGFuIGludGVy cnVwdCBpcyBwZW5kaW5nLiAqLwo+IHdyaXRlX3Btb3ZzY2xyKG1hc2spOwo+IHdoaWNoIHJlc3Vs dHMgaW4gYW55IHBlbmRpbmcgb3ZlcmZsb3cgaW50ZXJydXB0IGJlaW5nIGRyb3BwZWQuIEkgYWRk ZWQgc29tZQo+IGRlYnVnIG91dHB1dCBoZXJlIGFuZCBpbmRlZWQgdGhlcmUgaXMgYSBjb3JyZWxh dGlvbiBvZiB0aGlzIGJpdCBiZWluZyBoaWdoIGF0Cj4gdGhlIHBvaW50IG9mIHRoZSBhYm92ZSBj b2RlIGFuZCB0aGUgcmVwcm9kdWNlciBpZGVudGlmeWluZyBhIG1pc3NlZCBzaWduYWwuCgpJIHRo aW5rIHlvdSdyZSByaWdodCB0aGF0IGlmIHdlIGhhZCBhbiBvdmVyZmxvdyBhc3NlcnRlZCBhdCB0 aGlzIHBvaW50LCB3ZSdsbAp0aHJvdyBhd2F5IHRoZSBvY2N1cnJlbmNlIG9mIHRoZSBvdmVyZmxv dyAoYW5kIHNvIG5vdCBjYWxsCnBlcmZfZXZlbnRfb3ZlcmZsb3coKSBhbmQgZ2VuZXJhdGUgYSBz YW1wbGUsIGV0YykuCgpJdCBsb29rcyBsaWtlIHdlIG9ubHkgbG9zZSB0aGUgb2NjdXJyZW5jZSBv ZiB0aGUgb3ZlcmZsb3c7IHRoZSBhY3R1YWwgY291bnRzCndpbGwgZ2V0IHNhbXBsZWQgY29ycmVj dGx5IGFuZCB3aGVuIHdlIG5leHQgcmVwcm9ncmFtIHRoZSBldmVudCwKYXJtcG11X2V2ZW50X3Nl dF9wZXJpb2QoKSBzaG91bGQgc2V0IHVwIHRoZSBuZXh0IG92ZXJmbG93IHBlcmlvZC4KCj4gVGhp cyBiZWhhdmlvciBkb2VzIG5vdCBvY2N1ciB3aXRoIHBzZXVkby1OTUlzIChpcnFjaGlwLmdpY3Yz X3BzZXVkb19ubWk9MSkKPiBlbmFibGVkLgoKVGhhdCdzIGludGVyZXN0aW5nLCBiZWNhdXNlIGl0 IGltcGxpZXMgdGhhdCB0aGUgUE1VIG92ZXJmbG93IGludGVycnVwdCBpcyBiZWluZwpyZWNvZ25p c2VkIGJ5IHRoZSBDUFUgd2hpbGUgcmVndWxhciBpbnRlcnJ1cHRzIGFyZSBkaXNhYmxlZC4gVGhl cmUgYXJlIHNvbWUKbmFycm93IHJhY2VzIHdoZXJlIHRoYXQgY291bGQgb2NjdXIgKGUuZy4gdGFr aW5nIGEgdGltZXIgb3Igc2NoZWR1bGVyIElSUQoqanVzdCogYXMgYW4gb3ZlcmZsb3cgb2NjdXJz KSwgYW5kIHNvbWUgb3RoZXIgY2FzZXMgSSdkIGV4cGVjdCBSUiB0byBhdm9pZCBieQpjb25zdHJ1 Y3Rpb24gKGUuZy4gaWYgUlIgaXNuJ3QgdXNpbmcgbW9kZSBleGNsdXNpb24gYW5kIGFsc28gY291 bnRzIGtlcm5lbApldmVudHMpLiBJdCdzIGFsc28gd29ydGggbm90aW5nIHRoYXQgdGhpcyBtZWFu cyB0aGVyZSBhcmUgcmFjZXMgZXZlbiB3aXRoCnBzZXVkby1OTUkgd2hlcmUgb3ZlcmZsb3dzIGNv dWxkIGJlIGxvc3QuCgpIb3cgb2Z0ZW4gZG8geW91IHNlZSB0aGUgb3ZlcmZsb3cgYmVpbmcgbG9z dD8KCkRvZXMgUlIgc2V0IGFueSBvZiB0aGUgcGVyZl9ldmVudF9hdHRyOjpleGNsdWRlXyogYml0 cz8gSWYgbm90LCBkb2VzIFJSCmludGVudGlvbmFsbHkgY291bnQgZXZlbnRzIHRoYXQgb2NjdXIg d2l0aGluIHRoZSBrZXJuZWw/Cgo+IFdoZW4gYW4gZXZlbnQgaXMgbm90IGJlaW5nIGV4cGxpY2l0 bHkgdG9ybiBkb3duIChlLmcuIGJlaW5nIGNsb3NlZCksIHRoaXMgc2VlbXMKPiBsaWtlIGFuIHVu ZGVzaXJhYmxlIGJlaGF2aW9yLgoKSSBhZ3JlZSBpdCdzIHVuZGVzaXJhYmxlLCB0aG91Z2ggbXkg dW5kZXJzdGFuZGluZyBpcyB0aGF0IGZvciBtb3N0IG90aGVyIHVzZXJzCnRoaXMgaXNuJ3QgYW55 IHdvcnNlIHRoYW4gbG9zaW5nIHNhbXBsZXMgZm9yIG90aGVyIHJlYXNvbnMgKGUuZy4gdGhlIHBl cmYgcmluZwpidWZmZXIgYmVpbmcgZnVsbCksIGFuZCBmb3IgdGhlIGdlbmVyYWwgY2FzZSBvZiBz YW1wbGUtYmFzZWQgcHJvZmlsaW5nLCBsb3NpbmcKYSBzYW1wbGUgZXZlcnkgc28gb2Z0ZW4gaXNu J3QgdGhlIGVuZCBvZiB0aGUgd29ybGQuCgo+IEkgaGF2ZW4ndCBhdHRlbXB0ZWQgdG8gZGVtbyBp dCB5ZXQsIGJ1dCBJIHN1c3BlY3QKPiBhbiBhcHBsaWNhdGlvbiBkaXNhYmxpbmcgYW4gZXZlbnQg dGVtcG9yYXJpbHkgY291bGQgb2NjYXNpb25hbGx5IHNlZSB0aGUgc2FtZQo+IG1pc3NlZCBvdmVy ZmxvdyBzaWduYWxzLiBJcyBteSB1bmRlcnN0YW5kaW5nIGhlcmUgY29ycmVjdD8KClRoYXQgc291 bmRzIHJpZ2h0IHRvIG1lLCB0aG91Z2ggSSBoYXZlbid0IGNoZWNrZWQgdGhhdCBlbmQtdG8tZW5k IHlldC4KCj4gRG9lcyBhbnlvbmUgaGF2ZSB0aG91Z2h0cyBvbiBob3cgdGhpcyBjb3VsZCBiZSBh ZGRyZXNzZWQgd2l0aG91dCBjcmVhdGluZwo+IG90aGVyIGlzc3Vlcz8KCldlIHNob3VsZCBiZSBh YmxlIHRvIGRldGVjdCBvdmVyZmxvdyBmcm9tIHRoZSBjb3VudGVyIHZhbHVlIGFsb25lLCBzbyB3 ZSBtaWdodApiZSBhYmxlIHRvIGFjY291bnQgZm9yIHRoYXQgd2hlbiB3ZSBhY3R1YWxseSByZWFk IHRoZSBldmVudCBvdXQsIG9yIHdoZW4gd2UKc2NoZWR1bGUgaXQgYmFjayBpbiBhbmQgcmVwcm9n cmFtIHRoZSBwZXJpb2QuCgpJJ20gbm90IHN1cmUgaWYgd2UgY2FuIHJlYXNvbmFibHkgZG8gdGhh dCB3aGVuIHNjaGVkdWxpbmcgdGhlIGV2ZW50IG91dCwgc2luY2UKaWYgd2UncmUgc3dpdGNoaW5n IHRhc2tzLCB0aGF0J2xsIHF1ZXVlIHVwIElSUSB3b3JrIHdoaWNoIHdpbGwgYmUgdHJpZ2dlcmVk IGluCnRoZSBjb250ZXh0IG9mIHRoZSBuZXh0IHRhc2suCgpXZSBtaWdodCBiZSBhYmxlIHRvIGZp Z3VyZSBvdXQgdGhhdCB3ZSBoYXZlIGFuIG92ZXJmbG93IHdoZW4gd2Ugc2NoZWR1bGUgdGhlCmV2 ZW50IGluIHVuZGVyIGFybXBtdV9zdGFydCgpLCBidXQgSSdsbCBuZWVkIHRvIGdvIGRpZ2dpbmcg dG8gc2VlIGlmIHRoZXJlIGFyZQphbnkgZnJlZS1ydW5uaW5nIGNvdW50ZXJzIGFzIHRoZSBjb21t ZW50IGFib3ZlIHRoZSBjYWxsIHRvCmFybXBtdV9ldmVudF9zZXRfcGVyaW9kKCkgc2F5cywgb3Ig d2hldGhlciB0aGF0J3MgYSBoaXN0b3JpY2FsIGFydGlmYWN0LgoKSSBzdXNwZWN0IHdlIG1pZ2h0 IG5lZWQgdG8gcmVzdHJ1Y3R1cmUgdGhlIGNvZGUgc29tZXdoYXQgdG8gYmUgYWJsZSB0byBjYXRj aApvdmVyZmxvd3MgbW9yZSBjb25zaXN0ZW50bHkuIEknbGwgc2VlIGlmIEkgY2FuIGNvbWUgdXAg d2l0aCBzb21ldGhpbmcsIGJ1dCB3ZQptaWdodCBub3QgYmUgYWJsZSB0byBndWFyYW50ZWUgdGhp cyBpbiBhbGwgY2FzZXMuCgpNYXJrLgoKPiBbMV0gaHR0cHM6Ly9naXRodWIuY29tL3JyLWRlYnVn Z2VyL3JyL2lzc3Vlcy8zNjA3IDxodHRwczovL2dpdGh1Yi5jb20vcnItZGVidWdnZXIvcnIvaXNz dWVzLzM2MDc+Cj4gWzJdIGh0dHBzOi8vZ2l0aHViLmNvbS90b3J2YWxkcy9saW51eC9ibG9iL2M0 MmQ5ZWVlZjhlNWJhOTI5MmVkYTM2ZmQ4ZTNjMTFmMzVlZTA2NWMvZHJpdmVycy9wZXJmL2FybV9w bXV2My5jI0w2NTJDMjAtTDY1MkM0MyA8aHR0cHM6Ly9naXRodWIuY29tL3RvcnZhbGRzL2xpbnV4 L2Jsb2IvYzQyZDllZWVmOGU1YmE5MjkyZWRhMzZmZDhlM2MxMWYzNWVlMDY1Yy9kcml2ZXJzL3Bl cmYvYXJtX3BtdXYzLmMjTDY1MkMyMC1MNjUyQzQzPgoKX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX18KbGludXgtYXJtLWtlcm5lbCBtYWlsaW5nIGxpc3QKbGlu dXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRlYWQub3JnCmh0dHA6Ly9saXN0cy5pbmZyYWRlYWQu b3JnL21haWxtYW4vbGlzdGluZm8vbGludXgtYXJtLWtlcm5lbAo=