From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C203B1FBA for ; Fri, 11 Aug 2023 07:42:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 17D0BC433C8; Fri, 11 Aug 2023 07:42:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1691739753; bh=HK9lIOLHKgtJgCEH4ivD1Xd/zI856yixYQWZXP3lJxU=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=Qs89yHqrf5/+V/ZKYwr07+SfX1N2DCzqf6d6RKoNzhWW+kQp87N3wKEGMysq7QWRq 9KHUC02x8HlaspgMuNNBBUnlhKuYTVLdHTs2ehvedcIBEYcFcxUJhU1TYdMCOMe6jY FIkXneXbPx3P0OUmGChR5zGhH1q+Jk9DPz1zGu+/eHU6JfWjQlYPtqW50di4qR4RFe md1bWa6TutMBA45nytHvEdXPUBDvmVsoYXgU/gDpKHGxrNsrqe+xLso5CN9BZzvXrO FE1bIxr2CCe2rOwVgScegOg51X0Gn4YldBoniv7T5eNsMz+ffGTF7/cvKrBRfjKvQQ Zst0o+u6CqTGA== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1qUMn0-0044Wu-QO; Fri, 11 Aug 2023 08:42:30 +0100 Date: Fri, 11 Aug 2023 08:42:30 +0100 Message-ID: <86zg2yf2jd.wl-maz@kernel.org> From: Marc Zyngier To: Shijie Huang Cc: Huang Shijie , oliver.upton@linux.dev, james.morse@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, patches@amperecomputing.com, zwang@amperecomputing.com, Mark Rutland Subject: Re: [PATCH v2] KVM/arm64: reconfigurate the event filters for guest context In-Reply-To: References: <20230810072906.4007-1-shijie@os.amperecomputing.com> <87sf8qq5o0.wl-maz@kernel.org> <95726705-765d-020b-8c85-62fb917f2c14@amperemail.onmicrosoft.com> <87r0oap0s4.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/28.2 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: shijie@amperemail.onmicrosoft.com, shijie@os.amperecomputing.com, oliver.upton@linux.dev, james.morse@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, patches@amperecomputing.com, zwang@amperecomputing.com, mark.rutland@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false On Fri, 11 Aug 2023 08:10:26 +0100, Shijie Huang wrote: >=20 > Hi Marc, >=20 > =E5=9C=A8 2023/8/11 14:10, Marc Zyngier =E5=86=99=E9=81=93: > > On Fri, 11 Aug 2023 02:46:49 +0100, > > Shijie Huang wrote: > >> Hi Marc, > >>=20 > >> =E5=9C=A8 2023/8/10 23:27, Marc Zyngier =E5=86=99=E9=81=93: > >>> Huang, > >>>=20 > >>> Please make sure you add everyone who commented on v1 (I've Cc'd Mark > >>> so that he can shime need as needed). > >> thanks. > >>> On Thu, 10 Aug 2023 08:29:06 +0100, > >>> Huang Shijie wrote: > >>>> 1.) Background. > >>>> 1.1) In arm64, start a guest with Qemu which is running as a VM= M of KVM, > >>>> and bind the guest to core 33 and run program "a" in guest. > >>>> The code of "a" shows below: > >>>> ---------------------------------------------------------- > >>>> #include > >>>>=20 > >>>> int main() > >>>> { > >>>> unsigned long i =3D 0; > >>>>=20 > >>>> for (;;) { > >>>> i++; > >>>> } > >>>>=20 > >>>> printf("i:%ld\n", i); > >>>> return 0; > >>>> } > >>>> ---------------------------------------------------------- > >>>>=20 > >>>> 1.2) Use the following perf command in host: > >>>> #perf stat -e cycles:G,cycles:H -C 33 -I 1000 sleep 1 > >>>> # time counts unit events > >>>> 1.000817400 3,299,471,572 cycles:G > >>>> 1.000817400 3,240,586 cycles:H > >>>>=20 > >>>> This result is correct, my cpu's frequency is 3.3G. > >>>>=20 > >>>> 1.3) Use the following perf command in host: > >>>> #perf stat -e cycles:G,cycles:H -C 33 -d -d -I 1000 sleep 1 > >>>> time counts unit events > >>>> 1.000831480 153,634,097 cycles:G = (70.03%) > >>>> 1.000831480 3,147,940,599 cycles:H = (70.03%) > >>>> 1.000831480 1,143,598,527 L1-dcache-loads = (70.03%) > >>>> 1.000831480 9,986 L1-dcache-load-misses = # 0.00% of all L1-dcache accesses (70.03%) > >>>> 1.000831480 LLC-loads > >>>> 1.000831480 LLC-load-misses > >>>> 1.000831480 580,887,696 L1-icache-loads = (70.03%) > >>>> 1.000831480 77,855 L1-icache-load-misses = # 0.01% of all L1-icache accesses (70.03%) > >>>> 1.000831480 6,112,224,612 dTLB-loads = (70.03%) > >>>> 1.000831480 16,222 dTLB-load-misses = # 0.00% of all dTLB cache accesses (69.94%) > >>>> 1.000831480 590,015,996 iTLB-loads = (59.95%) > >>>> 1.000831480 505 iTLB-load-misses = # 0.00% of all iTLB cache accesses (59.95%) > >>>>=20 > >>>> This result is wrong. The "cycle:G" should be nearly 3.3G. > >>>>=20 > >>>> 2.) Root cause. > >>>> There is only 7 counters in my arm64 platform: > >>>> (one cycle counter) + (6 normal counters) > >>>>=20 > >>>> In 1.3 above, we will use 10 event counters. > >>>> Since we only have 7 counters, the perf core will trigger > >>>> multiplexing in hrtimer: > >>>> perf_mux_hrtimer_restart() --> perf_rotate_context(). > >>>>=20 > >>>> If the hrtimer occurs when the host is running, it's fine. > >>>> If the hrtimer occurs when the guest is running, > >>>> the perf_rotate_context() will program the PMU with filters= for > >>>> host context. The KVM does not have a chance to restore > >>>> PMU registers with kvm_vcpu_pmu_restore_guest(). > >>>> The PMU does not work correctly, so we got wrong result. > >>>>=20 > >>>> 3.) About this patch. > >>>> Make a KVM_REQ_RELOAD_PMU request before reentering the > >>>> guest. The request will call kvm_vcpu_pmu_restore_guest() > >>>> to reconfigurate the filters for guest context. > >>>>=20 > >>>> 4.) Test result of this patch: > >>>> #perf stat -e cycles:G,cycles:H -C 33 -d -d -I 1000 sleep 1 > >>>> time counts unit events > >>>> 1.001006400 3,298,348,656 cycles:G = (70.03%) > >>>> 1.001006400 3,144,532 cycles:H = (70.03%) > >>>> 1.001006400 941,149 L1-dcache-loads = (70.03%) > >>>> 1.001006400 17,937 L1-dcache-load-misses = # 1.91% of all L1-dcache accesses (70.03%) > >>>> 1.001006400 LLC-loads > >>>> 1.001006400 LLC-load-misses > >>>> 1.001006400 1,101,889 L1-icache-loads = (70.03%) > >>>> 1.001006400 121,638 L1-icache-load-misses = # 11.04% of all L1-icache accesses (70.03%) > >>>> 1.001006400 1,031,228 dTLB-loads = (70.03%) > >>>> 1.001006400 26,952 dTLB-load-misses = # 2.61% of all dTLB cache accesses (69.93%) > >>>> 1.001006400 1,030,678 iTLB-loads = (59.94%) > >>>> 1.001006400 338 iTLB-load-misses = # 0.03% of all iTLB cache accesses (59.94%) > >>>>=20 > >>>> The result is correct. The "cycle:G" is nearly 3.3G now. > >>>>=20 > >>>> Signed-off-by: Huang Shijie > >>>> --- > >>>> v1 --> v2: > >>>> Do not change perf/core code, only change the ARM64 kvm code. > >>>> v1: https://lkml.org/lkml/2023/8/8/1465 > >>>>=20 > >>>> --- > >>>> arch/arm64/kvm/arm.c | 11 ++++++++++- > >>>> 1 file changed, 10 insertions(+), 1 deletion(-) > >>>>=20 > >>>> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c > >>>> index c2c14059f6a8..475a2f0e0e40 100644 > >>>> --- a/arch/arm64/kvm/arm.c > >>>> +++ b/arch/arm64/kvm/arm.c > >>>> @@ -919,8 +919,17 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vc= pu) > >>>> if (!ret) > >>>> ret =3D 1; > >>>> - if (ret > 0) > >>>> + if (ret > 0) { > >>>> + /* > >>>> + * The perf_rotate_context() may rotate the events and > >>>> + * reprogram PMU with filters for host context. > >>>> + * So make a request before reentering the guest to > >>>> + * reconfigurate the event filters for guest context. > >>>> + */ > >>>> + kvm_make_request(KVM_REQ_RELOAD_PMU, vcpu); > >>>> + > >>>> ret =3D check_vcpu_requests(vcpu); > >>>> + } > >>> This looks extremely heavy handed. You're performing the reload on > >>> *every* entry, and I don't think this is right (exit-heavy workloads > >>> will suffer from it) > >>>=20 > >>> Furthermore, you're also reloading the virtual state of the PMU > >>> (recreating guest events and other things), all of which looks pretty > >>> pointless, as all we're interested in is what is being counted on the > >>> *host*. > >> okay. What about to add a _new_ request, such as KVM_REQ_RESTROE_PMU_G= UEST? > >>=20 > >>=20 > >>> Instead, we can restrict the reload of the host state (and only that) > >>> to situations where: > >>>=20 > >>> - we're running on a VHE system > >>>=20 > >>> - we have a host PMUv3 (not everybody does), as that's the only way we > >>> can profile a guest > >> okay. No problem. > >>=20 > >>=20 > >>> and ideally we would have a way to detect that a rotation happened > >>> (which may requires some help from the low-level PMU code). > >> I will check it, hope we can find a better way. > > I came up with the following patch, completely untested. Let me know > > how that fares for you. > >=20 > > Thanks, > >=20 > > M. > >=20 > > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm= /kvm_host.h > > index 93c541111dea..fb875c5c0347 100644 > > --- a/arch/arm64/include/asm/kvm_host.h > > +++ b/arch/arm64/include/asm/kvm_host.h > > @@ -49,6 +49,7 @@ > > #define KVM_REQ_RELOAD_GICv4 KVM_ARCH_REQ(4) > > #define KVM_REQ_RELOAD_PMU KVM_ARCH_REQ(5) > > #define KVM_REQ_SUSPEND KVM_ARCH_REQ(6) > > +#define KVM_REQ_RELOAD_GUEST_PMU_EVENTS KVM_ARCH_REQ(7) > > #define KVM_DIRTY_LOG_MANUAL_CAPS > > (KVM_DIRTY_LOG_MANUAL_PROTECT_ENABLE | \ > > KVM_DIRTY_LOG_INITIALLY_SET) > > diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c > > index 8b51570a76f8..b40db24f1f0b 100644 > > --- a/arch/arm64/kvm/arm.c > > +++ b/arch/arm64/kvm/arm.c > > @@ -804,6 +804,9 @@ static int check_vcpu_requests(struct kvm_vcpu *vcp= u) > > kvm_pmu_handle_pmcr(vcpu, > > __vcpu_sys_reg(vcpu, PMCR_EL0)); > > + if (kvm_check_request(KVM_REQ_RELOAD_GUEST_PMU_EVENTS, > > vcpu)) > > + kvm_vcpu_pmu_restore_guest(vcpu); > > + > > if (kvm_check_request(KVM_REQ_SUSPEND, vcpu)) > > return kvm_vcpu_suspend(vcpu); > > diff --git a/drivers/perf/arm_pmuv3.c b/drivers/perf/arm_pmuv3.c > > index 08b3a1bf0ef6..7012de417092 100644 > > --- a/drivers/perf/arm_pmuv3.c > > +++ b/drivers/perf/arm_pmuv3.c > > @@ -772,6 +772,9 @@ static void armv8pmu_start(struct arm_pmu *cpu_pmu) > > /* Enable all counters */ > > armv8pmu_pmcr_write(armv8pmu_pmcr_read() | ARMV8_PMU_PMCR_E); > > + > > + if (in_interrupt()) > > + kvm_resync_guest_context(); >=20 > I currently added a similiar check in armv8pmu_get_event_idx(). >=20 > The event multiplexing will call armv8pmu_get_event_idx(), and will > definitely fail at least one time. >=20 > +++ b/drivers/perf/arm_pmuv3.c > @@ -882,6 +882,8 @@ static int armv8pmu_get_event_idx(struct > pmu_hw_events *cpuc, > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 struct arm_pmu *cpu_pmu =3D to= _arm_pmu(event->pmu); > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 struct hw_perf_event *hwc =3D = &event->hw; > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 unsigned long evtype =3D hwc->= config_base & ARMV8_PMU_EVTYPE_EVENT; > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 struct kvm_vcpu *vcpu; > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 int index; > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 struct arm_pmu *cpu_pmu =3D to= _arm_pmu(event->pmu); > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 struct hw_perf_event *hwc =3D = &event->hw; > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 unsigned long evtype =3D hwc->= config_base & ARMV8_PMU_EVTYPE_EVENT; > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 struct kvm_vcpu *vcpu; > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 int index; >=20 > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /* Always prefer to place a cy= cle counter into the cycle > counter. */ > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (evtype =3D=3D ARMV8_PMUV3_= PERFCTR_CPU_CYCLES) { > @@ -897,9 +899,22 @@ static int armv8pmu_get_event_idx(struct > pmu_hw_events *cpuc, > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 * Otherwise use events c= ounters > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 */ > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (armv8pmu_event_is_chained(= event)) > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 return=C2=A0 armv8pmu_get_chain_idx(cpuc, cpu_pmu); > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 index =3D armv8pmu_get_chain_idx(cpuc, cpu_pmu); > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 else > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 return armv8pmu_get_single_idx(cpuc, cpu_pmu); > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 index =3D armv8pmu_get_single_idx(cpuc, cpu_pmu); > + > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /* > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 * If we are in pmu multiplexi= ng, we will definitely meet a failure. > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 * Please see perf_rotate_cont= ext(). > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 * If we are in the guest cont= ext, we can mark it. > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 */ > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (index < 0) { > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 vcpu =3D kvm_get_running_vcpu(); > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 if (vcpu && in_interrupt() && !event->attr.pinned) { > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 kvm_resync_gue= st_context(); > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 } > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 } > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return index; > =C2=A0} >=20 > IMHO, it's better to change armv8pmu_get_event_idx(). >=20 > But if you think it is also okay to change armv8pmu_start() to fix the bu= g, >=20 > I am okay too. But that's doing work each time you rotate an event. And if you rotate a bunch of them, you'll hit this path multiple times, reloading the stuff again. What's the point? My take is that we can hook at the point where the PMU gets re-enabled, and have the full context once and for all. Unless of course I miss something, which is very likely as the whole perf subsystem generally escapes me altogether. In any case, I'd welcome your testing the proposed patch. Thanks, M. --=20 Without deviation from the norm, progress is not possible. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 509A0C001B0 for ; Fri, 11 Aug 2023 07:43:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=gDB3KLbsMDYeE+OTEBkvvLCY2nOeYN7MknKaYJchS7I=; b=tN+x9fa9PU9ePd Qw2rhfsMkgAmI99GKY4T9qkUGdGaayei8T4vtMbaHsKxfwzJ1SbM+QoZlSNiqx79vex4evQaBCQiK Vxq1HAU2KQLTvAKCNlalpbMh4rsHqgbifkEAlwZ8ZbSHWHZN0krIeWVjBYjXPH3k/bUeFRk8RFnd5 dK+BN3RfBYI6YElYFu2l4EaczejDs2C7je2IfkGFHAg2nsdK3tzma+vLLc7/gXD2zMcfv6FDbxvgB QpS4jdRSDSgdQLbXusIR33iBet+Aok/3UgFCDJun8TSgiqXmTGh0KrVMFgCilzw6/YjHwt8tYUhcj 6lgZs9Adzuj787m7E+xw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qUMn8-009lui-2L; Fri, 11 Aug 2023 07:42:38 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qUMn4-009ltV-2N for linux-arm-kernel@lists.infradead.org; Fri, 11 Aug 2023 07:42:37 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id CDAB5649FC; Fri, 11 Aug 2023 07:42:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 17D0BC433C8; Fri, 11 Aug 2023 07:42:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1691739753; bh=HK9lIOLHKgtJgCEH4ivD1Xd/zI856yixYQWZXP3lJxU=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=Qs89yHqrf5/+V/ZKYwr07+SfX1N2DCzqf6d6RKoNzhWW+kQp87N3wKEGMysq7QWRq 9KHUC02x8HlaspgMuNNBBUnlhKuYTVLdHTs2ehvedcIBEYcFcxUJhU1TYdMCOMe6jY FIkXneXbPx3P0OUmGChR5zGhH1q+Jk9DPz1zGu+/eHU6JfWjQlYPtqW50di4qR4RFe md1bWa6TutMBA45nytHvEdXPUBDvmVsoYXgU/gDpKHGxrNsrqe+xLso5CN9BZzvXrO FE1bIxr2CCe2rOwVgScegOg51X0Gn4YldBoniv7T5eNsMz+ffGTF7/cvKrBRfjKvQQ Zst0o+u6CqTGA== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1qUMn0-0044Wu-QO; Fri, 11 Aug 2023 08:42:30 +0100 Date: Fri, 11 Aug 2023 08:42:30 +0100 Message-ID: <86zg2yf2jd.wl-maz@kernel.org> From: Marc Zyngier To: Shijie Huang Cc: Huang Shijie , oliver.upton@linux.dev, james.morse@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, patches@amperecomputing.com, zwang@amperecomputing.com, Mark Rutland Subject: Re: [PATCH v2] KVM/arm64: reconfigurate the event filters for guest context In-Reply-To: References: <20230810072906.4007-1-shijie@os.amperecomputing.com> <87sf8qq5o0.wl-maz@kernel.org> <95726705-765d-020b-8c85-62fb917f2c14@amperemail.onmicrosoft.com> <87r0oap0s4.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/28.2 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: shijie@amperemail.onmicrosoft.com, shijie@os.amperecomputing.com, oliver.upton@linux.dev, james.morse@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, patches@amperecomputing.com, zwang@amperecomputing.com, mark.rutland@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230811_004234_860038_A6A46EC5 X-CRM114-Status: GOOD ( 47.08 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gRnJpLCAxMSBBdWcgMjAyMyAwODoxMDoyNiArMDEwMCwKU2hpamllIEh1YW5nIDxzaGlqaWVA YW1wZXJlbWFpbC5vbm1pY3Jvc29mdC5jb20+IHdyb3RlOgo+IAo+IEhpIE1hcmMsCj4gCj4g5Zyo IDIwMjMvOC8xMSAxNDoxMCwgTWFyYyBaeW5naWVyIOWGmemBkzoKPiA+IE9uIEZyaSwgMTEgQXVn IDIwMjMgMDI6NDY6NDkgKzAxMDAsCj4gPiBTaGlqaWUgSHVhbmcgPHNoaWppZUBhbXBlcmVtYWls Lm9ubWljcm9zb2Z0LmNvbT4gd3JvdGU6Cj4gPj4gSGkgTWFyYywKPiA+PiAKPiA+PiDlnKggMjAy My84LzEwIDIzOjI3LCBNYXJjIFp5bmdpZXIg5YaZ6YGTOgo+ID4+PiBIdWFuZywKPiA+Pj4gCj4g Pj4+IFBsZWFzZSBtYWtlIHN1cmUgeW91IGFkZCBldmVyeW9uZSB3aG8gY29tbWVudGVkIG9uIHYx IChJJ3ZlIENjJ2QgTWFyawo+ID4+PiBzbyB0aGF0IGhlIGNhbiBzaGltZSBuZWVkIGFzIG5lZWRl ZCkuCj4gPj4gdGhhbmtzLgo+ID4+PiBPbiBUaHUsIDEwIEF1ZyAyMDIzIDA4OjI5OjA2ICswMTAw LAo+ID4+PiBIdWFuZyBTaGlqaWUgPHNoaWppZUBvcy5hbXBlcmVjb21wdXRpbmcuY29tPiB3cm90 ZToKPiA+Pj4+IDEuKSBCYWNrZ3JvdW5kLgo+ID4+Pj4gICAgICAxLjEpIEluIGFybTY0LCBzdGFy dCBhIGd1ZXN0IHdpdGggUWVtdSB3aGljaCBpcyBydW5uaW5nIGFzIGEgVk1NIG9mIEtWTSwKPiA+ Pj4+ICAgICAgICAgICBhbmQgYmluZCB0aGUgZ3Vlc3QgdG8gY29yZSAzMyBhbmQgcnVuIHByb2dy YW0gImEiIGluIGd1ZXN0Lgo+ID4+Pj4gICAgICAgICAgIFRoZSBjb2RlIG9mICJhIiBzaG93cyBi ZWxvdzoKPiA+Pj4+ICAgICAgCS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0KPiA+Pj4+IAkJI2luY2x1ZGUgPHN0ZGlvLmg+Cj4gPj4+PiAK PiA+Pj4+IAkJaW50IG1haW4oKQo+ID4+Pj4gCQl7Cj4gPj4+PiAJCQl1bnNpZ25lZCBsb25nIGkg PSAwOwo+ID4+Pj4gCj4gPj4+PiAJCQlmb3IgKDs7KSB7Cj4gPj4+PiAJCQkJaSsrOwo+ID4+Pj4g CQkJfQo+ID4+Pj4gCj4gPj4+PiAJCQlwcmludGYoImk6JWxkXG4iLCBpKTsKPiA+Pj4+IAkJCXJl dHVybiAwOwo+ID4+Pj4gCQl9Cj4gPj4+PiAgICAgIAktLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCj4gPj4+PiAKPiA+Pj4+ICAgICAgMS4y KSBVc2UgdGhlIGZvbGxvd2luZyBwZXJmIGNvbW1hbmQgaW4gaG9zdDoKPiA+Pj4+ICAgICAgICAg I3BlcmYgc3RhdCAtZSBjeWNsZXM6RyxjeWNsZXM6SCAtQyAzMyAtSSAxMDAwIHNsZWVwIDEKPiA+ Pj4+ICAgICAgICAgICAgICMgICAgICAgICAgIHRpbWUgICAgICAgICAgICAgY291bnRzIHVuaXQg ZXZlbnRzCj4gPj4+PiAgICAgICAgICAgICAgICAgIDEuMDAwODE3NDAwICAgICAgMywyOTksNDcx LDU3MiAgICAgIGN5Y2xlczpHCj4gPj4+PiAgICAgICAgICAgICAgICAgIDEuMDAwODE3NDAwICAg ICAgICAgIDMsMjQwLDU4NiAgICAgIGN5Y2xlczpICj4gPj4+PiAKPiA+Pj4+ICAgICAgICAgIFRo aXMgcmVzdWx0IGlzIGNvcnJlY3QsIG15IGNwdSdzIGZyZXF1ZW5jeSBpcyAzLjNHLgo+ID4+Pj4g Cj4gPj4+PiAgICAgIDEuMykgVXNlIHRoZSBmb2xsb3dpbmcgcGVyZiBjb21tYW5kIGluIGhvc3Q6 Cj4gPj4+PiAgICAgICAgICNwZXJmIHN0YXQgLWUgY3ljbGVzOkcsY3ljbGVzOkggLUMgMzMgLWQg LWQgIC1JIDEwMDAgc2xlZXAgMQo+ID4+Pj4gICAgICAgICAgICAgICB0aW1lICAgICAgICAgICAg IGNvdW50cyB1bml0IGV2ZW50cwo+ID4+Pj4gICAgICAgIDEuMDAwODMxNDgwICAgICAgICAxNTMs NjM0LDA5NyAgICAgIGN5Y2xlczpHICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICg3MC4wMyUpCj4gPj4+PiAgICAgICAgMS4wMDA4 MzE0ODAgICAgICAzLDE0Nyw5NDAsNTk5ICAgICAgY3ljbGVzOkggICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgKDcwLjAzJSkKPiA+ Pj4+ICAgICAgICAxLjAwMDgzMTQ4MCAgICAgIDEsMTQzLDU5OCw1MjcgICAgICBMMS1kY2FjaGUt bG9hZHMgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAoNzAuMDMlKQo+ID4+Pj4gICAgICAgIDEuMDAwODMxNDgwICAgICAgICAgICAgICA5LDk4 NiAgICAgIEwxLWRjYWNoZS1sb2FkLW1pc3NlcyAgICAgICAgICAgICMgICAgMC4wMCUgb2YgYWxs IEwxLWRjYWNoZSBhY2Nlc3NlcyAgICg3MC4wMyUpCj4gPj4+PiAgICAgICAgMS4wMDA4MzE0ODAg ICAgPG5vdCBzdXBwb3J0ZWQ+ICAgICAgTExDLWxvYWRzCj4gPj4+PiAgICAgICAgMS4wMDA4MzE0 ODAgICAgPG5vdCBzdXBwb3J0ZWQ+ICAgICAgTExDLWxvYWQtbWlzc2VzCj4gPj4+PiAgICAgICAg MS4wMDA4MzE0ODAgICAgICAgIDU4MCw4ODcsNjk2ICAgICAgTDEtaWNhY2hlLWxvYWRzICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgKDcwLjAz JSkKPiA+Pj4+ICAgICAgICAxLjAwMDgzMTQ4MCAgICAgICAgICAgICA3Nyw4NTUgICAgICBMMS1p Y2FjaGUtbG9hZC1taXNzZXMgICAgICAgICAgICAjICAgIDAuMDElIG9mIGFsbCBMMS1pY2FjaGUg YWNjZXNzZXMgICAoNzAuMDMlKQo+ID4+Pj4gICAgICAgIDEuMDAwODMxNDgwICAgICAgNiwxMTIs MjI0LDYxMiAgICAgIGRUTEItbG9hZHMgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICg3MC4wMyUpCj4gPj4+PiAgICAgICAgMS4wMDA4 MzE0ODAgICAgICAgICAgICAgMTYsMjIyICAgICAgZFRMQi1sb2FkLW1pc3NlcyAgICAgICAgICAg ICAgICAgIyAgICAwLjAwJSBvZiBhbGwgZFRMQiBjYWNoZSBhY2Nlc3NlcyAgKDY5Ljk0JSkKPiA+ Pj4+ICAgICAgICAxLjAwMDgzMTQ4MCAgICAgICAgNTkwLDAxNSw5OTYgICAgICBpVExCLWxvYWRz ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAoNTkuOTUlKQo+ID4+Pj4gICAgICAgIDEuMDAwODMxNDgwICAgICAgICAgICAgICAgIDUw NSAgICAgIGlUTEItbG9hZC1taXNzZXMgICAgICAgICAgICAgICAgICMgICAgMC4wMCUgb2YgYWxs IGlUTEIgY2FjaGUgYWNjZXNzZXMgICg1OS45NSUpCj4gPj4+PiAKPiA+Pj4+ICAgICAgICAgIFRo aXMgcmVzdWx0IGlzIHdyb25nLiBUaGUgImN5Y2xlOkciIHNob3VsZCBiZSBuZWFybHkgMy4zRy4K PiA+Pj4+IAo+ID4+Pj4gMi4pIFJvb3QgY2F1c2UuCj4gPj4+PiAJVGhlcmUgaXMgb25seSA3IGNv dW50ZXJzIGluIG15IGFybTY0IHBsYXRmb3JtOgo+ID4+Pj4gCSAgKG9uZSBjeWNsZSBjb3VudGVy KSArICg2IG5vcm1hbCBjb3VudGVycykKPiA+Pj4+IAo+ID4+Pj4gCUluIDEuMyBhYm92ZSwgd2Ug d2lsbCB1c2UgMTAgZXZlbnQgY291bnRlcnMuCj4gPj4+PiAJU2luY2Ugd2Ugb25seSBoYXZlIDcg Y291bnRlcnMsIHRoZSBwZXJmIGNvcmUgd2lsbCB0cmlnZ2VyCj4gPj4+PiAgICAgICAgICAJbXVs dGlwbGV4aW5nIGluIGhydGltZXI6Cj4gPj4+PiAJICAgICBwZXJmX211eF9ocnRpbWVyX3Jlc3Rh cnQoKSAtLT4gcGVyZl9yb3RhdGVfY29udGV4dCgpLgo+ID4+Pj4gCj4gPj4+PiAgICAgICAgICBJ ZiB0aGUgaHJ0aW1lciBvY2N1cnMgd2hlbiB0aGUgaG9zdCBpcyBydW5uaW5nLCBpdCdzIGZpbmUu Cj4gPj4+PiAgICAgICAgICBJZiB0aGUgaHJ0aW1lciBvY2N1cnMgd2hlbiB0aGUgZ3Vlc3QgaXMg cnVubmluZywKPiA+Pj4+ICAgICAgICAgIHRoZSBwZXJmX3JvdGF0ZV9jb250ZXh0KCkgd2lsbCBw cm9ncmFtIHRoZSBQTVUgd2l0aCBmaWx0ZXJzIGZvcgo+ID4+Pj4gICAgICAgICAgaG9zdCBjb250 ZXh0LiBUaGUgS1ZNIGRvZXMgbm90IGhhdmUgYSBjaGFuY2UgdG8gcmVzdG9yZQo+ID4+Pj4gICAg ICAgICAgUE1VIHJlZ2lzdGVycyB3aXRoIGt2bV92Y3B1X3BtdV9yZXN0b3JlX2d1ZXN0KCkuCj4g Pj4+PiAgICAgICAgICBUaGUgUE1VIGRvZXMgbm90IHdvcmsgY29ycmVjdGx5LCBzbyB3ZSBnb3Qg d3JvbmcgcmVzdWx0Lgo+ID4+Pj4gCj4gPj4+PiAzLikgQWJvdXQgdGhpcyBwYXRjaC4KPiA+Pj4+ IAlNYWtlIGEgS1ZNX1JFUV9SRUxPQURfUE1VIHJlcXVlc3QgYmVmb3JlIHJlZW50ZXJpbmcgdGhl Cj4gPj4+PiAJZ3Vlc3QuIFRoZSByZXF1ZXN0IHdpbGwgY2FsbCBrdm1fdmNwdV9wbXVfcmVzdG9y ZV9ndWVzdCgpCj4gPj4+PiAJdG8gcmVjb25maWd1cmF0ZSB0aGUgZmlsdGVycyBmb3IgZ3Vlc3Qg Y29udGV4dC4KPiA+Pj4+IAo+ID4+Pj4gNC4pIFRlc3QgcmVzdWx0IG9mIHRoaXMgcGF0Y2g6Cj4g Pj4+PiAgICAgICAgICNwZXJmIHN0YXQgLWUgY3ljbGVzOkcsY3ljbGVzOkggLUMgMzMgLWQgLWQg IC1JIDEwMDAgc2xlZXAgMQo+ID4+Pj4gICAgICAgICAgICAgICB0aW1lICAgICAgICAgICAgIGNv dW50cyB1bml0IGV2ZW50cwo+ID4+Pj4gICAgICAgIDEuMDAxMDA2NDAwICAgICAgMywyOTgsMzQ4 LDY1NiAgICAgIGN5Y2xlczpHICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICg3MC4wMyUpCj4gPj4+PiAgICAgICAgMS4wMDEwMDY0 MDAgICAgICAgICAgMywxNDQsNTMyICAgICAgY3ljbGVzOkggICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgKDcwLjAzJSkKPiA+Pj4+ ICAgICAgICAxLjAwMTAwNjQwMCAgICAgICAgICAgIDk0MSwxNDkgICAgICBMMS1kY2FjaGUtbG9h ZHMgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAoNzAuMDMlKQo+ID4+Pj4gICAgICAgIDEuMDAxMDA2NDAwICAgICAgICAgICAgIDE3LDkzNyAg ICAgIEwxLWRjYWNoZS1sb2FkLW1pc3NlcyAgICAgICAgICAgICMgICAgMS45MSUgb2YgYWxsIEwx LWRjYWNoZSBhY2Nlc3NlcyAgICg3MC4wMyUpCj4gPj4+PiAgICAgICAgMS4wMDEwMDY0MDAgICAg PG5vdCBzdXBwb3J0ZWQ+ICAgICAgTExDLWxvYWRzCj4gPj4+PiAgICAgICAgMS4wMDEwMDY0MDAg ICAgPG5vdCBzdXBwb3J0ZWQ+ICAgICAgTExDLWxvYWQtbWlzc2VzCj4gPj4+PiAgICAgICAgMS4w MDEwMDY0MDAgICAgICAgICAgMSwxMDEsODg5ICAgICAgTDEtaWNhY2hlLWxvYWRzICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgKDcwLjAzJSkK PiA+Pj4+ICAgICAgICAxLjAwMTAwNjQwMCAgICAgICAgICAgIDEyMSw2MzggICAgICBMMS1pY2Fj aGUtbG9hZC1taXNzZXMgICAgICAgICAgICAjICAgMTEuMDQlIG9mIGFsbCBMMS1pY2FjaGUgYWNj ZXNzZXMgICAoNzAuMDMlKQo+ID4+Pj4gICAgICAgIDEuMDAxMDA2NDAwICAgICAgICAgIDEsMDMx LDIyOCAgICAgIGRUTEItbG9hZHMgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICg3MC4wMyUpCj4gPj4+PiAgICAgICAgMS4wMDEwMDY0 MDAgICAgICAgICAgICAgMjYsOTUyICAgICAgZFRMQi1sb2FkLW1pc3NlcyAgICAgICAgICAgICAg ICAgIyAgICAyLjYxJSBvZiBhbGwgZFRMQiBjYWNoZSBhY2Nlc3NlcyAgKDY5LjkzJSkKPiA+Pj4+ ICAgICAgICAxLjAwMTAwNjQwMCAgICAgICAgICAxLDAzMCw2NzggICAgICBpVExCLWxvYWRzICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAoNTkuOTQlKQo+ID4+Pj4gICAgICAgIDEuMDAxMDA2NDAwICAgICAgICAgICAgICAgIDMzOCAg ICAgIGlUTEItbG9hZC1taXNzZXMgICAgICAgICAgICAgICAgICMgICAgMC4wMyUgb2YgYWxsIGlU TEIgY2FjaGUgYWNjZXNzZXMgICg1OS45NCUpCj4gPj4+PiAKPiA+Pj4+ICAgICAgIFRoZSByZXN1 bHQgaXMgY29ycmVjdC4gVGhlICJjeWNsZTpHIiBpcyBuZWFybHkgMy4zRyBub3cuCj4gPj4+PiAK PiA+Pj4+IFNpZ25lZC1vZmYtYnk6IEh1YW5nIFNoaWppZSA8c2hpamllQG9zLmFtcGVyZWNvbXB1 dGluZy5jb20+Cj4gPj4+PiAtLS0KPiA+Pj4+IHYxIC0tPiB2MjoKPiA+Pj4+IAlEbyBub3QgY2hh bmdlIHBlcmYvY29yZSBjb2RlLCBvbmx5IGNoYW5nZSB0aGUgQVJNNjQga3ZtIGNvZGUuCj4gPj4+ PiAJdjE6IGh0dHBzOi8vbGttbC5vcmcvbGttbC8yMDIzLzgvOC8xNDY1Cj4gPj4+PiAKPiA+Pj4+ IC0tLQo+ID4+Pj4gICAgYXJjaC9hcm02NC9rdm0vYXJtLmMgfCAxMSArKysrKysrKysrLQo+ID4+ Pj4gICAgMSBmaWxlIGNoYW5nZWQsIDEwIGluc2VydGlvbnMoKyksIDEgZGVsZXRpb24oLSkKPiA+ Pj4+IAo+ID4+Pj4gZGlmZiAtLWdpdCBhL2FyY2gvYXJtNjQva3ZtL2FybS5jIGIvYXJjaC9hcm02 NC9rdm0vYXJtLmMKPiA+Pj4+IGluZGV4IGMyYzE0MDU5ZjZhOC4uNDc1YTJmMGUwZTQwIDEwMDY0 NAo+ID4+Pj4gLS0tIGEvYXJjaC9hcm02NC9rdm0vYXJtLmMKPiA+Pj4+ICsrKyBiL2FyY2gvYXJt NjQva3ZtL2FybS5jCj4gPj4+PiBAQCAtOTE5LDggKzkxOSwxNyBAQCBpbnQga3ZtX2FyY2hfdmNw dV9pb2N0bF9ydW4oc3RydWN0IGt2bV92Y3B1ICp2Y3B1KQo+ID4+Pj4gICAgCQlpZiAoIXJldCkK PiA+Pj4+ICAgIAkJCXJldCA9IDE7Cj4gPj4+PiAgICAtCQlpZiAocmV0ID4gMCkKPiA+Pj4+ICsJ CWlmIChyZXQgPiAwKSB7Cj4gPj4+PiArCQkJLyoKPiA+Pj4+ICsJCQkgKiBUaGUgcGVyZl9yb3Rh dGVfY29udGV4dCgpIG1heSByb3RhdGUgdGhlIGV2ZW50cyBhbmQKPiA+Pj4+ICsJCQkgKiByZXBy b2dyYW0gUE1VIHdpdGggZmlsdGVycyBmb3IgaG9zdCBjb250ZXh0Lgo+ID4+Pj4gKwkJCSAqIFNv IG1ha2UgYSByZXF1ZXN0IGJlZm9yZSByZWVudGVyaW5nIHRoZSBndWVzdCB0bwo+ID4+Pj4gKwkJ CSAqIHJlY29uZmlndXJhdGUgdGhlIGV2ZW50IGZpbHRlcnMgZm9yIGd1ZXN0IGNvbnRleHQuCj4g Pj4+PiArCQkJICovCj4gPj4+PiArCQkJa3ZtX21ha2VfcmVxdWVzdChLVk1fUkVRX1JFTE9BRF9Q TVUsIHZjcHUpOwo+ID4+Pj4gKwo+ID4+Pj4gICAgCQkJcmV0ID0gY2hlY2tfdmNwdV9yZXF1ZXN0 cyh2Y3B1KTsKPiA+Pj4+ICsJCX0KPiA+Pj4gVGhpcyBsb29rcyBleHRyZW1lbHkgaGVhdnkgaGFu ZGVkLiBZb3UncmUgcGVyZm9ybWluZyB0aGUgcmVsb2FkIG9uCj4gPj4+ICpldmVyeSogZW50cnks IGFuZCBJIGRvbid0IHRoaW5rIHRoaXMgaXMgcmlnaHQgKGV4aXQtaGVhdnkgd29ya2xvYWRzCj4g Pj4+IHdpbGwgc3VmZmVyIGZyb20gaXQpCj4gPj4+IAo+ID4+PiBGdXJ0aGVybW9yZSwgeW91J3Jl IGFsc28gcmVsb2FkaW5nIHRoZSB2aXJ0dWFsIHN0YXRlIG9mIHRoZSBQTVUKPiA+Pj4gKHJlY3Jl YXRpbmcgZ3Vlc3QgZXZlbnRzIGFuZCBvdGhlciB0aGluZ3MpLCBhbGwgb2Ygd2hpY2ggbG9va3Mg cHJldHR5Cj4gPj4+IHBvaW50bGVzcywgYXMgYWxsIHdlJ3JlIGludGVyZXN0ZWQgaW4gaXMgd2hh dCBpcyBiZWluZyBjb3VudGVkIG9uIHRoZQo+ID4+PiAqaG9zdCouCj4gPj4gb2theS4gV2hhdCBh Ym91dCB0byBhZGQgYSBfbmV3XyByZXF1ZXN0LCBzdWNoIGFzIEtWTV9SRVFfUkVTVFJPRV9QTVVf R1VFU1Q/Cj4gPj4gCj4gPj4gCj4gPj4+IEluc3RlYWQsIHdlIGNhbiByZXN0cmljdCB0aGUgcmVs b2FkIG9mIHRoZSBob3N0IHN0YXRlIChhbmQgb25seSB0aGF0KQo+ID4+PiB0byBzaXR1YXRpb25z IHdoZXJlOgo+ID4+PiAKPiA+Pj4gLSB3ZSdyZSBydW5uaW5nIG9uIGEgVkhFIHN5c3RlbQo+ID4+ PiAKPiA+Pj4gLSB3ZSBoYXZlIGEgaG9zdCBQTVV2MyAobm90IGV2ZXJ5Ym9keSBkb2VzKSwgYXMg dGhhdCdzIHRoZSBvbmx5IHdheSB3ZQo+ID4+PiAgICAgY2FuIHByb2ZpbGUgYSBndWVzdAo+ID4+ IG9rYXkuIE5vIHByb2JsZW0uCj4gPj4gCj4gPj4gCj4gPj4+IGFuZCBpZGVhbGx5IHdlIHdvdWxk IGhhdmUgYSB3YXkgdG8gZGV0ZWN0IHRoYXQgYSByb3RhdGlvbiBoYXBwZW5lZAo+ID4+PiAod2hp Y2ggbWF5IHJlcXVpcmVzIHNvbWUgaGVscCBmcm9tIHRoZSBsb3ctbGV2ZWwgUE1VIGNvZGUpLgo+ ID4+IEkgd2lsbCBjaGVjayBpdCwgaG9wZSB3ZSBjYW4gZmluZCBhIGJldHRlciB3YXkuCj4gPiBJ IGNhbWUgdXAgd2l0aCB0aGUgZm9sbG93aW5nIHBhdGNoLCBjb21wbGV0ZWx5IHVudGVzdGVkLiBM ZXQgbWUga25vdwo+ID4gaG93IHRoYXQgZmFyZXMgZm9yIHlvdS4KPiA+IAo+ID4gVGhhbmtzLAo+ ID4gCj4gPiAJTS4KPiA+IAo+ID4gZGlmZiAtLWdpdCBhL2FyY2gvYXJtNjQvaW5jbHVkZS9hc20v a3ZtX2hvc3QuaCBiL2FyY2gvYXJtNjQvaW5jbHVkZS9hc20va3ZtX2hvc3QuaAo+ID4gaW5kZXgg OTNjNTQxMTExZGVhLi5mYjg3NWM1YzAzNDcgMTAwNjQ0Cj4gPiAtLS0gYS9hcmNoL2FybTY0L2lu Y2x1ZGUvYXNtL2t2bV9ob3N0LmgKPiA+ICsrKyBiL2FyY2gvYXJtNjQvaW5jbHVkZS9hc20va3Zt X2hvc3QuaAo+ID4gQEAgLTQ5LDYgKzQ5LDcgQEAKPiA+ICAgI2RlZmluZSBLVk1fUkVRX1JFTE9B RF9HSUN2NAlLVk1fQVJDSF9SRVEoNCkKPiA+ICAgI2RlZmluZSBLVk1fUkVRX1JFTE9BRF9QTVUJ S1ZNX0FSQ0hfUkVRKDUpCj4gPiAgICNkZWZpbmUgS1ZNX1JFUV9TVVNQRU5ECQlLVk1fQVJDSF9S RVEoNikKPiA+ICsjZGVmaW5lIEtWTV9SRVFfUkVMT0FEX0dVRVNUX1BNVV9FVkVOVFMJS1ZNX0FS Q0hfUkVRKDcpCj4gPiAgICAgI2RlZmluZSBLVk1fRElSVFlfTE9HX01BTlVBTF9DQVBTCj4gPiAo S1ZNX0RJUlRZX0xPR19NQU5VQUxfUFJPVEVDVF9FTkFCTEUgfCBcCj4gPiAgIAkJCQkgICAgIEtW TV9ESVJUWV9MT0dfSU5JVElBTExZX1NFVCkKPiA+IGRpZmYgLS1naXQgYS9hcmNoL2FybTY0L2t2 bS9hcm0uYyBiL2FyY2gvYXJtNjQva3ZtL2FybS5jCj4gPiBpbmRleCA4YjUxNTcwYTc2ZjguLmI0 MGRiMjRmMWYwYiAxMDA2NDQKPiA+IC0tLSBhL2FyY2gvYXJtNjQva3ZtL2FybS5jCj4gPiArKysg Yi9hcmNoL2FybTY0L2t2bS9hcm0uYwo+ID4gQEAgLTgwNCw2ICs4MDQsOSBAQCBzdGF0aWMgaW50 IGNoZWNrX3ZjcHVfcmVxdWVzdHMoc3RydWN0IGt2bV92Y3B1ICp2Y3B1KQo+ID4gICAJCQlrdm1f cG11X2hhbmRsZV9wbWNyKHZjcHUsCj4gPiAgIAkJCQkJICAgIF9fdmNwdV9zeXNfcmVnKHZjcHUs IFBNQ1JfRUwwKSk7Cj4gPiAgICsJCWlmIChrdm1fY2hlY2tfcmVxdWVzdChLVk1fUkVRX1JFTE9B RF9HVUVTVF9QTVVfRVZFTlRTLAo+ID4gdmNwdSkpCj4gPiArCQkJa3ZtX3ZjcHVfcG11X3Jlc3Rv cmVfZ3Vlc3QodmNwdSk7Cj4gPiArCj4gPiAgIAkJaWYgKGt2bV9jaGVja19yZXF1ZXN0KEtWTV9S RVFfU1VTUEVORCwgdmNwdSkpCj4gPiAgIAkJCXJldHVybiBrdm1fdmNwdV9zdXNwZW5kKHZjcHUp Owo+ID4gICBkaWZmIC0tZ2l0IGEvZHJpdmVycy9wZXJmL2FybV9wbXV2My5jIGIvZHJpdmVycy9w ZXJmL2FybV9wbXV2My5jCj4gPiBpbmRleCAwOGIzYTFiZjBlZjYuLjcwMTJkZTQxNzA5MiAxMDA2 NDQKPiA+IC0tLSBhL2RyaXZlcnMvcGVyZi9hcm1fcG11djMuYwo+ID4gKysrIGIvZHJpdmVycy9w ZXJmL2FybV9wbXV2My5jCj4gPiBAQCAtNzcyLDYgKzc3Miw5IEBAIHN0YXRpYyB2b2lkIGFybXY4 cG11X3N0YXJ0KHN0cnVjdCBhcm1fcG11ICpjcHVfcG11KQo+ID4gICAgIAkvKiBFbmFibGUgYWxs IGNvdW50ZXJzICovCj4gPiAgIAlhcm12OHBtdV9wbWNyX3dyaXRlKGFybXY4cG11X3BtY3JfcmVh ZCgpIHwgQVJNVjhfUE1VX1BNQ1JfRSk7Cj4gPiArCj4gPiArCWlmIChpbl9pbnRlcnJ1cHQoKSkK PiA+ICsJCWt2bV9yZXN5bmNfZ3Vlc3RfY29udGV4dCgpOwo+IAo+IEkgY3VycmVudGx5IGFkZGVk IGEgc2ltaWxpYXIgY2hlY2sgaW4gYXJtdjhwbXVfZ2V0X2V2ZW50X2lkeCgpLgo+IAo+IFRoZSBl dmVudCBtdWx0aXBsZXhpbmcgd2lsbCBjYWxsIGFybXY4cG11X2dldF9ldmVudF9pZHgoKSwgYW5k IHdpbGwKPiBkZWZpbml0ZWx5IGZhaWwgYXQgbGVhc3Qgb25lIHRpbWUuCj4gCj4gKysrIGIvZHJp dmVycy9wZXJmL2FybV9wbXV2My5jCj4gQEAgLTg4Miw2ICs4ODIsOCBAQCBzdGF0aWMgaW50IGFy bXY4cG11X2dldF9ldmVudF9pZHgoc3RydWN0Cj4gcG11X2h3X2V2ZW50cyAqY3B1YywKPiDCoMKg wqDCoMKgwqDCoCBzdHJ1Y3QgYXJtX3BtdSAqY3B1X3BtdSA9IHRvX2FybV9wbXUoZXZlbnQtPnBt dSk7Cj4gwqDCoMKgwqDCoMKgwqAgc3RydWN0IGh3X3BlcmZfZXZlbnQgKmh3YyA9ICZldmVudC0+ aHc7Cj4gwqDCoMKgwqDCoMKgwqAgdW5zaWduZWQgbG9uZyBldnR5cGUgPSBod2MtPmNvbmZpZ19i YXNlICYgQVJNVjhfUE1VX0VWVFlQRV9FVkVOVDsKPiArwqDCoMKgwqDCoMKgIHN0cnVjdCBrdm1f dmNwdSAqdmNwdTsKPiArwqDCoMKgwqDCoMKgIGludCBpbmRleDsKPiDCoMKgwqDCoMKgwqDCoCBz dHJ1Y3QgYXJtX3BtdSAqY3B1X3BtdSA9IHRvX2FybV9wbXUoZXZlbnQtPnBtdSk7Cj4gwqDCoMKg wqDCoMKgwqAgc3RydWN0IGh3X3BlcmZfZXZlbnQgKmh3YyA9ICZldmVudC0+aHc7Cj4gwqDCoMKg wqDCoMKgwqAgdW5zaWduZWQgbG9uZyBldnR5cGUgPSBod2MtPmNvbmZpZ19iYXNlICYgQVJNVjhf UE1VX0VWVFlQRV9FVkVOVDsKPiArwqDCoMKgwqDCoMKgIHN0cnVjdCBrdm1fdmNwdSAqdmNwdTsK PiArwqDCoMKgwqDCoMKgIGludCBpbmRleDsKPiAKPiDCoMKgwqDCoMKgwqDCoCAvKiBBbHdheXMg cHJlZmVyIHRvIHBsYWNlIGEgY3ljbGUgY291bnRlciBpbnRvIHRoZSBjeWNsZQo+IGNvdW50ZXIu ICovCj4gwqDCoMKgwqDCoMKgwqAgaWYgKGV2dHlwZSA9PSBBUk1WOF9QTVVWM19QRVJGQ1RSX0NQ VV9DWUNMRVMpIHsKPiBAQCAtODk3LDkgKzg5OSwyMiBAQCBzdGF0aWMgaW50IGFybXY4cG11X2dl dF9ldmVudF9pZHgoc3RydWN0Cj4gcG11X2h3X2V2ZW50cyAqY3B1YywKPiDCoMKgwqDCoMKgwqDC oMKgICogT3RoZXJ3aXNlIHVzZSBldmVudHMgY291bnRlcnMKPiDCoMKgwqDCoMKgwqDCoMKgICov Cj4gwqDCoMKgwqDCoMKgwqAgaWYgKGFybXY4cG11X2V2ZW50X2lzX2NoYWluZWQoZXZlbnQpKQo+ IC3CoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHJldHVybsKgIGFybXY4cG11X2dldF9jaGFp bl9pZHgoY3B1YywgY3B1X3BtdSk7Cj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgaW5k ZXggPSBhcm12OHBtdV9nZXRfY2hhaW5faWR4KGNwdWMsIGNwdV9wbXUpOwo+IMKgwqDCoMKgwqDC oMKgIGVsc2UKPiAtwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCByZXR1cm4gYXJtdjhwbXVf Z2V0X3NpbmdsZV9pZHgoY3B1YywgY3B1X3BtdSk7Cj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqAgaW5kZXggPSBhcm12OHBtdV9nZXRfc2luZ2xlX2lkeChjcHVjLCBjcHVfcG11KTsKPiAr Cj4gK8KgwqDCoMKgwqDCoCAvKgo+ICvCoMKgwqDCoMKgwqDCoCAqIElmIHdlIGFyZSBpbiBwbXUg bXVsdGlwbGV4aW5nLCB3ZSB3aWxsIGRlZmluaXRlbHkgbWVldCBhIGZhaWx1cmUuCj4gK8KgwqDC oMKgwqDCoMKgICogUGxlYXNlIHNlZSBwZXJmX3JvdGF0ZV9jb250ZXh0KCkuCj4gK8KgwqDCoMKg wqDCoMKgICogSWYgd2UgYXJlIGluIHRoZSBndWVzdCBjb250ZXh0LCB3ZSBjYW4gbWFyayBpdC4K PiArwqDCoMKgwqDCoMKgwqAgKi8KPiArwqDCoMKgwqDCoMKgIGlmIChpbmRleCA8IDApIHsKPiAr wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB2Y3B1ID0ga3ZtX2dldF9ydW5uaW5nX3ZjcHUo KTsKPiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBpZiAodmNwdSAmJiBpbl9pbnRlcnJ1 cHQoKSAmJiAhZXZlbnQtPmF0dHIucGlubmVkKSB7Cj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgIGt2bV9yZXN5bmNfZ3Vlc3RfY29udGV4dCgpOwo+ICvCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIH0KPiArwqDCoMKgwqDCoMKgIH0KPiArwqDCoMKgwqDC oMKgIHJldHVybiBpbmRleDsKPiDCoH0KPiAKPiBJTUhPLCBpdCdzIGJldHRlciB0byBjaGFuZ2Ug YXJtdjhwbXVfZ2V0X2V2ZW50X2lkeCgpLgo+IAo+IEJ1dCBpZiB5b3UgdGhpbmsgaXQgaXMgYWxz byBva2F5IHRvIGNoYW5nZSBhcm12OHBtdV9zdGFydCgpIHRvIGZpeCB0aGUgYnVnLAo+IAo+IEkg YW0gb2theSB0b28uCgpCdXQgdGhhdCdzIGRvaW5nIHdvcmsgZWFjaCB0aW1lIHlvdSByb3RhdGUg YW4gZXZlbnQuIEFuZCBpZiB5b3Ugcm90YXRlCmEgYnVuY2ggb2YgdGhlbSwgeW91J2xsIGhpdCB0 aGlzIHBhdGggbXVsdGlwbGUgdGltZXMsIHJlbG9hZGluZyB0aGUKc3R1ZmYgYWdhaW4uIFdoYXQn cyB0aGUgcG9pbnQ/CgpNeSB0YWtlIGlzIHRoYXQgd2UgY2FuIGhvb2sgYXQgdGhlIHBvaW50IHdo ZXJlIHRoZSBQTVUgZ2V0cwpyZS1lbmFibGVkLCBhbmQgaGF2ZSB0aGUgZnVsbCBjb250ZXh0IG9u Y2UgYW5kIGZvciBhbGwuCgpVbmxlc3Mgb2YgY291cnNlIEkgbWlzcyBzb21ldGhpbmcsIHdoaWNo IGlzIHZlcnkgbGlrZWx5IGFzIHRoZSB3aG9sZQpwZXJmIHN1YnN5c3RlbSBnZW5lcmFsbHkgZXNj YXBlcyBtZSBhbHRvZ2V0aGVyLgoKSW4gYW55IGNhc2UsIEknZCB3ZWxjb21lIHlvdXIgdGVzdGlu ZyB0aGUgcHJvcG9zZWQgcGF0Y2guCgpUaGFua3MsCgoJTS4KCi0tIApXaXRob3V0IGRldmlhdGlv biBmcm9tIHRoZSBub3JtLCBwcm9ncmVzcyBpcyBub3QgcG9zc2libGUuCgpfX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpsaW51eC1hcm0ta2VybmVsIG1haWxp bmcgbGlzdApsaW51eC1hcm0ta2VybmVsQGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3Rz LmluZnJhZGVhZC5vcmcvbWFpbG1hbi9saXN0aW5mby9saW51eC1hcm0ta2VybmVsCg==