From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ot1-f53.google.com (mail-ot1-f53.google.com [209.85.210.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6CA6A10F2 for ; Wed, 21 Jun 2023 02:51:50 +0000 (UTC) Received: by mail-ot1-f53.google.com with SMTP id 46e09a7af769-6b58351327eso2362091a34.1 for ; Tue, 20 Jun 2023 19:51:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687315909; x=1689907909; h=in-reply-to:references:subject:cc:to:from:message-id:date :content-transfer-encoding:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=qJNr1vyPvHbMEk8O8XldE8NbrEXH2idSMvV3eZDgIT4=; b=J9rhQVz1R8lKDLiKwNIPca+zvG5Tv3NsIAj1Ji/wwJWZ5NGPo67uNji7TMbGfymVWO GMr37w69IFJ1a/xO2A5DmceFjN0O32RpzdAIssyO44XqTHGckAF25peyXSxvQDN3Hon3 nm7yK6UQvcoGWP40qbNg1er6nj4r5UQsT0ym99rNBvogtZMf2i5gafTMS+lgcgzNQGKq aoUmdiGQ3/vjF0fTi0Y3GwgVLDETdD79PdEDbQpGXiviLmWKyCtsFqxqvYfIKO+j7cmH 1dwMi6f30X6ti1+jJ2sfUBHAIRnjtvhicwwWnFeGYhp7QveaAM4i9Fm9K8L5dJwflH0T aPMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687315909; x=1689907909; h=in-reply-to:references:subject:cc:to:from:message-id:date :content-transfer-encoding:mime-version:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=qJNr1vyPvHbMEk8O8XldE8NbrEXH2idSMvV3eZDgIT4=; b=GCdQW0LhFDX5aC6YtuxWQW3iaje5541V3eQwbIoRvliHuHBioqTlZRigBmowKxnImW yNPXimdGGgrp2NCeto/Bh3Ux0kOOSFAPlWwwI0ih0EfuS6CDRz+xLdkAVdaQ0sS5z10P lmmJWHKV7udaKGUwcwCQe+6CdWPhw5B+hD0dg5sjsUvEBBWF0gDLFD4qMhzhzjBESfgD /XdeXYViQSBt6Dxd/mS5Zl8NnsfMn0CsEwWwbZa08gNe/FBU3h0jyrhLBkkIQy/tV/vT tNjhmJSq5KKTaxT4ZKC6ln8WAWLkd2j+2uxlp916cSL/cZK2sN8bV5m/VGZQQYxPxWLq UzUw== X-Gm-Message-State: AC+VfDzvmKHhsrW+KexcABplp0067hPAMgn013KG2BzMMoCFdgeaViC+ loFmtF6iG011k4mKLlIzQTA= X-Google-Smtp-Source: ACHHUZ4vmFcBnDcvLrBXtT15BOX4k0uY1EIRYfrVxGT80rydywp/6szlGIEPs7lB1lYg1S65G5/lLQ== X-Received: by 2002:a05:6359:a26:b0:12c:9960:ec1e with SMTP id el38-20020a0563590a2600b0012c9960ec1emr8123207rwb.19.1687315909090; Tue, 20 Jun 2023 19:51:49 -0700 (PDT) Received: from localhost (193-116-203-37.tpgi.com.au. [193.116.203.37]) by smtp.gmail.com with ESMTPSA id q15-20020a170902bd8f00b001a5fccab02dsm2246983pls.177.2023.06.20.19.51.33 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 20 Jun 2023 19:51:48 -0700 (PDT) Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=UTF-8 Date: Wed, 21 Jun 2023 12:51:31 +1000 Message-Id: From: "Nicholas Piggin" To: "Yu Zhao" Cc: "Andrew Morton" , "Paolo Bonzini" , "Alistair Popple" , "Anup Patel" , "Ben Gardon" , "Borislav Petkov" , "Catalin Marinas" , "Chao Peng" , "Christophe Leroy" , "Dave Hansen" , "Fabiano Rosas" , "Gaosheng Cui" , "Gavin Shan" , "H. Peter Anvin" , "Ingo Molnar" , "James Morse" , "Jason A. Donenfeld" , "Jason Gunthorpe" , "Jonathan Corbet" , "Marc Zyngier" , "Masami Hiramatsu" , "Michael Ellerman" , "Michael Larabel" , "Mike Rapoport" , "Oliver Upton" , "Paul Mackerras" , "Peter Xu" , "Sean Christopherson" , "Steven Rostedt" , "Suzuki K Poulose" , "Thomas Gleixner" , "Thomas Huth" , "Will Deacon" , "Zenghui Yu" , , , , , , , , , , Subject: Re: [PATCH mm-unstable v2 07/10] kvm/powerpc: add kvm_arch_test_clear_young() X-Mailer: aerc 0.14.0 References: <20230526234435.662652-1-yuzhao@google.com> <20230526234435.662652-8-yuzhao@google.com> In-Reply-To: On Wed Jun 21, 2023 at 10:38 AM AEST, Yu Zhao wrote: > On Tue, Jun 20, 2023 at 1:48=E2=80=AFAM Nicholas Piggin wrote: > > > > On Sat May 27, 2023 at 9:44 AM AEST, Yu Zhao wrote: > > > Implement kvm_arch_test_clear_young() to support the fast path in > > > mmu_notifier_ops->test_clear_young(). > > > > > > It focuses on a simple case, i.e., radix MMU sets the accessed bit in > > > KVM PTEs and VMs are not nested, where it can rely on RCU and > > > pte_xchg() to safely clear the accessed bit without taking > > > kvm->mmu_lock. Complex cases fall back to the existing slow path > > > where kvm->mmu_lock is then taken. > > > > > > Signed-off-by: Yu Zhao > > > --- > > > arch/powerpc/include/asm/kvm_host.h | 8 ++++ > > > arch/powerpc/include/asm/kvm_ppc.h | 1 + > > > arch/powerpc/kvm/book3s.c | 6 +++ > > > arch/powerpc/kvm/book3s.h | 1 + > > > arch/powerpc/kvm/book3s_64_mmu_radix.c | 59 ++++++++++++++++++++++++= ++ > > > arch/powerpc/kvm/book3s_hv.c | 5 +++ > > > 6 files changed, 80 insertions(+) > > > > > > diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/inclu= de/asm/kvm_host.h > > > index 14ee0dece853..75c260ea8a9e 100644 > > > --- a/arch/powerpc/include/asm/kvm_host.h > > > +++ b/arch/powerpc/include/asm/kvm_host.h > > > @@ -883,4 +883,12 @@ static inline void kvm_arch_sched_in(struct kvm_= vcpu *vcpu, int cpu) {} > > > static inline void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu) {} > > > static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {= } > > > > > > +#define kvm_arch_has_test_clear_young kvm_arch_has_test_clear_young > > > +static inline bool kvm_arch_has_test_clear_young(void) > > > +{ > > > + return IS_ENABLED(CONFIG_KVM_BOOK3S_HV_POSSIBLE) && > > > + cpu_has_feature(CPU_FTR_HVMODE) && cpu_has_feature(CPU_F= TR_ARCH_300) && > > > + radix_enabled(); > > > > This could probably be radix_enabled() && !kvmhv_on_pseries(). > > Will do. (I used !kvmhv_on_pseries() in v1 but had second thoughts on > moving kvmhv_on_pseries() into this file.) That should be okay. kvmhv_on_pseries is a property of the host so it seems reasonable to move it here if needed. > > Although unclear why not nested hypervisor... I'd have to think about i= t a bit > > more. Do you have any idea what might go wrong, or just didn't have the > > time to consider it? (Not just powerpc nested but any others). > > Yes, this series excludes nested KVM support on all architures. The > common reason for such a decision on powerpc and x86 (aarch64 doesn't > support nested yet) is that it's quite challenging to make the rmap, a > complex data structure that maps one PFN to multiple GFNs, lockless. > (See kvmhv_insert_nest_rmap().) It might be possible, however, the > potential ROI would be in question. Okay just wondering. rmap (at least the powerpc one) is just a list I think, with a few details. If that is all it is, it might not be so hard to make that lock-free or a fine-grained lock on the rmap chains maybe. But fine to ignore it to start with. > > > +} > > > + > > > #endif /* __POWERPC_KVM_HOST_H__ */ > > > diff --git a/arch/powerpc/include/asm/kvm_ppc.h b/arch/powerpc/includ= e/asm/kvm_ppc.h > > > index 79a9c0bb8bba..ff1af6a7b44f 100644 > > > --- a/arch/powerpc/include/asm/kvm_ppc.h > > > +++ b/arch/powerpc/include/asm/kvm_ppc.h > > > @@ -287,6 +287,7 @@ struct kvmppc_ops { > > > bool (*unmap_gfn_range)(struct kvm *kvm, struct kvm_gfn_range *= range); > > > bool (*age_gfn)(struct kvm *kvm, struct kvm_gfn_range *range); > > > bool (*test_age_gfn)(struct kvm *kvm, struct kvm_gfn_range *ran= ge); > > > + bool (*test_clear_young)(struct kvm *kvm, struct kvm_gfn_range = *range); > > > bool (*set_spte_gfn)(struct kvm *kvm, struct kvm_gfn_range *ran= ge); > > > void (*free_memslot)(struct kvm_memory_slot *slot); > > > int (*init_vm)(struct kvm *kvm); > > > diff --git a/arch/powerpc/kvm/book3s.c b/arch/powerpc/kvm/book3s.c > > > index 686d8d9eda3e..37bf40b0c4ff 100644 > > > --- a/arch/powerpc/kvm/book3s.c > > > +++ b/arch/powerpc/kvm/book3s.c > > > @@ -899,6 +899,12 @@ bool kvm_test_age_gfn(struct kvm *kvm, struct kv= m_gfn_range *range) > > > return kvm->arch.kvm_ops->test_age_gfn(kvm, range); > > > } > > > > > > +bool kvm_arch_test_clear_young(struct kvm *kvm, struct kvm_gfn_range= *range) > > > +{ > > > + return !kvm->arch.kvm_ops->test_clear_young || > > > + kvm->arch.kvm_ops->test_clear_young(kvm, range); > > > +} > > > + > > > bool kvm_set_spte_gfn(struct kvm *kvm, struct kvm_gfn_range *range) > > > { > > > return kvm->arch.kvm_ops->set_spte_gfn(kvm, range); > > > diff --git a/arch/powerpc/kvm/book3s.h b/arch/powerpc/kvm/book3s.h > > > index 58391b4b32ed..fa2659e21ccc 100644 > > > --- a/arch/powerpc/kvm/book3s.h > > > +++ b/arch/powerpc/kvm/book3s.h > > > @@ -12,6 +12,7 @@ extern void kvmppc_core_flush_memslot_hv(struct kvm= *kvm, > > > extern bool kvm_unmap_gfn_range_hv(struct kvm *kvm, struct kvm_gfn_r= ange *range); > > > extern bool kvm_age_gfn_hv(struct kvm *kvm, struct kvm_gfn_range *ra= nge); > > > extern bool kvm_test_age_gfn_hv(struct kvm *kvm, struct kvm_gfn_rang= e *range); > > > +extern bool kvm_test_clear_young_hv(struct kvm *kvm, struct kvm_gfn_= range *range); > > > extern bool kvm_set_spte_gfn_hv(struct kvm *kvm, struct kvm_gfn_rang= e *range); > > > > > > extern int kvmppc_mmu_init_pr(struct kvm_vcpu *vcpu); > > > diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kv= m/book3s_64_mmu_radix.c > > > index 3b65b3b11041..0a392e9a100a 100644 > > > --- a/arch/powerpc/kvm/book3s_64_mmu_radix.c > > > +++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c > > > @@ -1088,6 +1088,65 @@ bool kvm_test_age_radix(struct kvm *kvm, struc= t kvm_memory_slot *memslot, > > > return ref; > > > } > > > > > > +bool kvm_test_clear_young_hv(struct kvm *kvm, struct kvm_gfn_range *= range) > > > +{ > > > + bool err; > > > + gfn_t gfn =3D range->start; > > > + > > > + rcu_read_lock(); > > > + > > > + err =3D !kvm_is_radix(kvm); > > > + if (err) > > > + goto unlock; > > > + > > > + /* > > > + * Case 1: This function kvmppc_switch_mmu_to_hpt() > > > + * > > > + * rcu_read_lock() > > > + * Test kvm_is_radix() kvm->arch.radix =3D 0 > > > + * Use kvm->arch.pgtable synchronize_rcu() > > > + * rcu_read_unlock() > > > + * kvmppc_free_radix() > > > + * > > > + * > > > + * Case 2: This function kvmppc_switch_mmu_to_radix() > > > + * > > > + * kvmppc_init_vm_radix() > > > + * smp_wmb() > > > + * Test kvm_is_radix() kvm->arch.radix =3D 1 > > > + * smp_rmb() > > > + * Use kvm->arch.pgtable > > > + */ > > > + smp_rmb(); > > > > Comment could stand to expand slightly on what you are solving, in > > words. > > Will do. > > > If you use synchronize_rcu() on both sides, you wouldn't need the > > barrier, right? > > Case 2 is about memory ordering, which is orthogonal to case 1 (RCU > freeing). So we need the r/w barrier regardless. RCU can take care of memory ordering too though. If you had synchronize_rcu() where smp_wmb() is, then no smp_rmb() neeed here. > > > > + while (gfn < range->end) { > > > + pte_t *ptep; > > > + pte_t old, new; > > > + unsigned int shift; > > > + > > > + ptep =3D find_kvm_secondary_pte_unlocked(kvm, gfn * PAG= E_SIZE, &shift); > > > + if (!ptep) > > > + goto next; > > > + > > > + VM_WARN_ON_ONCE(!page_count(virt_to_page(ptep))); > > > > Not really appropriate at the KVM level. mm enforces this kind of > > thing (with notifiers). > > Will remove this. > > > > + > > > + old =3D READ_ONCE(*ptep); > > > + if (!pte_present(old) || !pte_young(old)) > > > + goto next; > > > + > > > + new =3D pte_mkold(old); > > > + > > > + if (kvm_should_clear_young(range, gfn)) > > > + pte_xchg(ptep, old, new); > > > > *Probably* will work. I can't think of a reason why not at the > > moment anyway :) > > My reasoning: > * It should work if we only change the dedicated A bit, i.e., not > shared for other purposes, because races are safe (the case here). > * It may not work for x86 EPT without the A bit (excluded in this > series) where accesses are trapped by the R/X bits, because races in > changing the R/X bits can be unsafe. (For the benefit of others reading, it works because powerpc's pte_xchg is actually a cmpxchg, for some reason which we really should fix). Although it can fail to clear the bit if the cmpxchg fails. I think pte_xchg is only used on with hash MMU in Linux before this change. I think we may want to keep it that way and use something like kvmppc_radix_update_pte() here to clear out the bit. But don't worry too much about fine details so much before sorting out the core changes I will have a better look after that. Thanks, Nick From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0680DEB64D7 for ; Wed, 21 Jun 2023 02:52:50 +0000 (UTC) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20221208 header.b=J9rhQVz1; dkim-atps=neutral Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4Qm7N11ymVz3bW1 for ; Wed, 21 Jun 2023 12:52:49 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20221208 header.b=J9rhQVz1; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::32a; helo=mail-ot1-x32a.google.com; envelope-from=npiggin@gmail.com; receiver=lists.ozlabs.org) Received: from mail-ot1-x32a.google.com (mail-ot1-x32a.google.com [IPv6:2607:f8b0:4864:20::32a]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4Qm7Ly3KXSz305g for ; Wed, 21 Jun 2023 12:51:53 +1000 (AEST) Received: by mail-ot1-x32a.google.com with SMTP id 46e09a7af769-6b46e540326so2699082a34.2 for ; Tue, 20 Jun 2023 19:51:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687315909; x=1689907909; h=in-reply-to:references:subject:cc:to:from:message-id:date :content-transfer-encoding:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=qJNr1vyPvHbMEk8O8XldE8NbrEXH2idSMvV3eZDgIT4=; b=J9rhQVz1R8lKDLiKwNIPca+zvG5Tv3NsIAj1Ji/wwJWZ5NGPo67uNji7TMbGfymVWO GMr37w69IFJ1a/xO2A5DmceFjN0O32RpzdAIssyO44XqTHGckAF25peyXSxvQDN3Hon3 nm7yK6UQvcoGWP40qbNg1er6nj4r5UQsT0ym99rNBvogtZMf2i5gafTMS+lgcgzNQGKq aoUmdiGQ3/vjF0fTi0Y3GwgVLDETdD79PdEDbQpGXiviLmWKyCtsFqxqvYfIKO+j7cmH 1dwMi6f30X6ti1+jJ2sfUBHAIRnjtvhicwwWnFeGYhp7QveaAM4i9Fm9K8L5dJwflH0T aPMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687315909; x=1689907909; h=in-reply-to:references:subject:cc:to:from:message-id:date :content-transfer-encoding:mime-version:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=qJNr1vyPvHbMEk8O8XldE8NbrEXH2idSMvV3eZDgIT4=; b=F2bmMQ8n9sKSj5i8kuL9CWBEL4hbF3fkbEq9yunImzK9+bn9pj0lkFcOapcdzlo1Nf LyiweDru8/mnlTkxuirsoOiYJNS15v4yfTqMwnYb1oHd03D6Yasu2aM+DNTj5cJu4sg/ I5s+gwvGnRnwBj4IkwzlUEkIYF9joeq1tXVqyuaM/laKnLM2zwe7++Bg+c0OY26mPhDW 6s1l1WescXjiiwLFEuwLXXvUgjFgS7dLYNoCD3LwGZtYUHdZ6mxA3qUdTP7ZRT6yVV3O NQr1WHnt7oDPlx8QK/75P4COK4+DBvKTUNH+SNstvzsss05CCNdZmyM/Fud11d7GI0Ud 5xmg== X-Gm-Message-State: AC+VfDwoOWgMjmqLqg26hc44x7QA9exT6CzRSNUvBw/IwQ4yTOzTW4kr bHHklogZa/XKsa8AiMCYk/g= X-Google-Smtp-Source: ACHHUZ4vmFcBnDcvLrBXtT15BOX4k0uY1EIRYfrVxGT80rydywp/6szlGIEPs7lB1lYg1S65G5/lLQ== X-Received: by 2002:a05:6359:a26:b0:12c:9960:ec1e with SMTP id el38-20020a0563590a2600b0012c9960ec1emr8123207rwb.19.1687315909090; Tue, 20 Jun 2023 19:51:49 -0700 (PDT) Received: from localhost (193-116-203-37.tpgi.com.au. [193.116.203.37]) by smtp.gmail.com with ESMTPSA id q15-20020a170902bd8f00b001a5fccab02dsm2246983pls.177.2023.06.20.19.51.33 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 20 Jun 2023 19:51:48 -0700 (PDT) Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=UTF-8 Date: Wed, 21 Jun 2023 12:51:31 +1000 Message-Id: From: "Nicholas Piggin" To: "Yu Zhao" Subject: Re: [PATCH mm-unstable v2 07/10] kvm/powerpc: add kvm_arch_test_clear_young() X-Mailer: aerc 0.14.0 References: <20230526234435.662652-1-yuzhao@google.com> <20230526234435.662652-8-yuzhao@google.com> In-Reply-To: X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Jason A. Donenfeld" , x86@kernel.org, Gavin Shan , kvm@vger.kernel.org, linux-doc@vger.kernel.org, Catalin Marinas , Dave Hansen , Peter Xu , linux-mm@kvack.org, Ben Gardon , Chao Peng , Will Deacon , Gaosheng Cui , Marc Zyngier , "H. Peter Anvin" , Jonathan Corbet , Alistair Popple , Jason Gunthorpe , Ingo Molnar , Zenghui Yu , linux-trace-kernel@vger.kernel.org, linux-mm@google.com, Thomas Huth , Suzuki K Poulose , Steven Rostedt , Borislav Petkov , kvmarm@lists.linux.dev, Thomas Gleixner , linux-arm-kernel@lists.infradead.org, Fabiano Rosas , Michael Larabel , Sean Christopherson , linux-kernel@vger.kernel.org, Oliver Upton , James Morse , Masami Hiramatsu , Anup Patel , Paolo Bonzini , Andrew Morton , linuxppc-dev@lists.ozlabs.org, Mike Rapoport Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Wed Jun 21, 2023 at 10:38 AM AEST, Yu Zhao wrote: > On Tue, Jun 20, 2023 at 1:48=E2=80=AFAM Nicholas Piggin wrote: > > > > On Sat May 27, 2023 at 9:44 AM AEST, Yu Zhao wrote: > > > Implement kvm_arch_test_clear_young() to support the fast path in > > > mmu_notifier_ops->test_clear_young(). > > > > > > It focuses on a simple case, i.e., radix MMU sets the accessed bit in > > > KVM PTEs and VMs are not nested, where it can rely on RCU and > > > pte_xchg() to safely clear the accessed bit without taking > > > kvm->mmu_lock. Complex cases fall back to the existing slow path > > > where kvm->mmu_lock is then taken. > > > > > > Signed-off-by: Yu Zhao > > > --- > > > arch/powerpc/include/asm/kvm_host.h | 8 ++++ > > > arch/powerpc/include/asm/kvm_ppc.h | 1 + > > > arch/powerpc/kvm/book3s.c | 6 +++ > > > arch/powerpc/kvm/book3s.h | 1 + > > > arch/powerpc/kvm/book3s_64_mmu_radix.c | 59 ++++++++++++++++++++++++= ++ > > > arch/powerpc/kvm/book3s_hv.c | 5 +++ > > > 6 files changed, 80 insertions(+) > > > > > > diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/inclu= de/asm/kvm_host.h > > > index 14ee0dece853..75c260ea8a9e 100644 > > > --- a/arch/powerpc/include/asm/kvm_host.h > > > +++ b/arch/powerpc/include/asm/kvm_host.h > > > @@ -883,4 +883,12 @@ static inline void kvm_arch_sched_in(struct kvm_= vcpu *vcpu, int cpu) {} > > > static inline void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu) {} > > > static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {= } > > > > > > +#define kvm_arch_has_test_clear_young kvm_arch_has_test_clear_young > > > +static inline bool kvm_arch_has_test_clear_young(void) > > > +{ > > > + return IS_ENABLED(CONFIG_KVM_BOOK3S_HV_POSSIBLE) && > > > + cpu_has_feature(CPU_FTR_HVMODE) && cpu_has_feature(CPU_F= TR_ARCH_300) && > > > + radix_enabled(); > > > > This could probably be radix_enabled() && !kvmhv_on_pseries(). > > Will do. (I used !kvmhv_on_pseries() in v1 but had second thoughts on > moving kvmhv_on_pseries() into this file.) That should be okay. kvmhv_on_pseries is a property of the host so it seems reasonable to move it here if needed. > > Although unclear why not nested hypervisor... I'd have to think about i= t a bit > > more. Do you have any idea what might go wrong, or just didn't have the > > time to consider it? (Not just powerpc nested but any others). > > Yes, this series excludes nested KVM support on all architures. The > common reason for such a decision on powerpc and x86 (aarch64 doesn't > support nested yet) is that it's quite challenging to make the rmap, a > complex data structure that maps one PFN to multiple GFNs, lockless. > (See kvmhv_insert_nest_rmap().) It might be possible, however, the > potential ROI would be in question. Okay just wondering. rmap (at least the powerpc one) is just a list I think, with a few details. If that is all it is, it might not be so hard to make that lock-free or a fine-grained lock on the rmap chains maybe. But fine to ignore it to start with. > > > +} > > > + > > > #endif /* __POWERPC_KVM_HOST_H__ */ > > > diff --git a/arch/powerpc/include/asm/kvm_ppc.h b/arch/powerpc/includ= e/asm/kvm_ppc.h > > > index 79a9c0bb8bba..ff1af6a7b44f 100644 > > > --- a/arch/powerpc/include/asm/kvm_ppc.h > > > +++ b/arch/powerpc/include/asm/kvm_ppc.h > > > @@ -287,6 +287,7 @@ struct kvmppc_ops { > > > bool (*unmap_gfn_range)(struct kvm *kvm, struct kvm_gfn_range *= range); > > > bool (*age_gfn)(struct kvm *kvm, struct kvm_gfn_range *range); > > > bool (*test_age_gfn)(struct kvm *kvm, struct kvm_gfn_range *ran= ge); > > > + bool (*test_clear_young)(struct kvm *kvm, struct kvm_gfn_range = *range); > > > bool (*set_spte_gfn)(struct kvm *kvm, struct kvm_gfn_range *ran= ge); > > > void (*free_memslot)(struct kvm_memory_slot *slot); > > > int (*init_vm)(struct kvm *kvm); > > > diff --git a/arch/powerpc/kvm/book3s.c b/arch/powerpc/kvm/book3s.c > > > index 686d8d9eda3e..37bf40b0c4ff 100644 > > > --- a/arch/powerpc/kvm/book3s.c > > > +++ b/arch/powerpc/kvm/book3s.c > > > @@ -899,6 +899,12 @@ bool kvm_test_age_gfn(struct kvm *kvm, struct kv= m_gfn_range *range) > > > return kvm->arch.kvm_ops->test_age_gfn(kvm, range); > > > } > > > > > > +bool kvm_arch_test_clear_young(struct kvm *kvm, struct kvm_gfn_range= *range) > > > +{ > > > + return !kvm->arch.kvm_ops->test_clear_young || > > > + kvm->arch.kvm_ops->test_clear_young(kvm, range); > > > +} > > > + > > > bool kvm_set_spte_gfn(struct kvm *kvm, struct kvm_gfn_range *range) > > > { > > > return kvm->arch.kvm_ops->set_spte_gfn(kvm, range); > > > diff --git a/arch/powerpc/kvm/book3s.h b/arch/powerpc/kvm/book3s.h > > > index 58391b4b32ed..fa2659e21ccc 100644 > > > --- a/arch/powerpc/kvm/book3s.h > > > +++ b/arch/powerpc/kvm/book3s.h > > > @@ -12,6 +12,7 @@ extern void kvmppc_core_flush_memslot_hv(struct kvm= *kvm, > > > extern bool kvm_unmap_gfn_range_hv(struct kvm *kvm, struct kvm_gfn_r= ange *range); > > > extern bool kvm_age_gfn_hv(struct kvm *kvm, struct kvm_gfn_range *ra= nge); > > > extern bool kvm_test_age_gfn_hv(struct kvm *kvm, struct kvm_gfn_rang= e *range); > > > +extern bool kvm_test_clear_young_hv(struct kvm *kvm, struct kvm_gfn_= range *range); > > > extern bool kvm_set_spte_gfn_hv(struct kvm *kvm, struct kvm_gfn_rang= e *range); > > > > > > extern int kvmppc_mmu_init_pr(struct kvm_vcpu *vcpu); > > > diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kv= m/book3s_64_mmu_radix.c > > > index 3b65b3b11041..0a392e9a100a 100644 > > > --- a/arch/powerpc/kvm/book3s_64_mmu_radix.c > > > +++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c > > > @@ -1088,6 +1088,65 @@ bool kvm_test_age_radix(struct kvm *kvm, struc= t kvm_memory_slot *memslot, > > > return ref; > > > } > > > > > > +bool kvm_test_clear_young_hv(struct kvm *kvm, struct kvm_gfn_range *= range) > > > +{ > > > + bool err; > > > + gfn_t gfn =3D range->start; > > > + > > > + rcu_read_lock(); > > > + > > > + err =3D !kvm_is_radix(kvm); > > > + if (err) > > > + goto unlock; > > > + > > > + /* > > > + * Case 1: This function kvmppc_switch_mmu_to_hpt() > > > + * > > > + * rcu_read_lock() > > > + * Test kvm_is_radix() kvm->arch.radix =3D 0 > > > + * Use kvm->arch.pgtable synchronize_rcu() > > > + * rcu_read_unlock() > > > + * kvmppc_free_radix() > > > + * > > > + * > > > + * Case 2: This function kvmppc_switch_mmu_to_radix() > > > + * > > > + * kvmppc_init_vm_radix() > > > + * smp_wmb() > > > + * Test kvm_is_radix() kvm->arch.radix =3D 1 > > > + * smp_rmb() > > > + * Use kvm->arch.pgtable > > > + */ > > > + smp_rmb(); > > > > Comment could stand to expand slightly on what you are solving, in > > words. > > Will do. > > > If you use synchronize_rcu() on both sides, you wouldn't need the > > barrier, right? > > Case 2 is about memory ordering, which is orthogonal to case 1 (RCU > freeing). So we need the r/w barrier regardless. RCU can take care of memory ordering too though. If you had synchronize_rcu() where smp_wmb() is, then no smp_rmb() neeed here. > > > > + while (gfn < range->end) { > > > + pte_t *ptep; > > > + pte_t old, new; > > > + unsigned int shift; > > > + > > > + ptep =3D find_kvm_secondary_pte_unlocked(kvm, gfn * PAG= E_SIZE, &shift); > > > + if (!ptep) > > > + goto next; > > > + > > > + VM_WARN_ON_ONCE(!page_count(virt_to_page(ptep))); > > > > Not really appropriate at the KVM level. mm enforces this kind of > > thing (with notifiers). > > Will remove this. > > > > + > > > + old =3D READ_ONCE(*ptep); > > > + if (!pte_present(old) || !pte_young(old)) > > > + goto next; > > > + > > > + new =3D pte_mkold(old); > > > + > > > + if (kvm_should_clear_young(range, gfn)) > > > + pte_xchg(ptep, old, new); > > > > *Probably* will work. I can't think of a reason why not at the > > moment anyway :) > > My reasoning: > * It should work if we only change the dedicated A bit, i.e., not > shared for other purposes, because races are safe (the case here). > * It may not work for x86 EPT without the A bit (excluded in this > series) where accesses are trapped by the R/X bits, because races in > changing the R/X bits can be unsafe. (For the benefit of others reading, it works because powerpc's pte_xchg is actually a cmpxchg, for some reason which we really should fix). Although it can fail to clear the bit if the cmpxchg fails. I think pte_xchg is only used on with hash MMU in Linux before this change. I think we may want to keep it that way and use something like kvmppc_radix_update_pte() here to clear out the bit. But don't worry too much about fine details so much before sorting out the core changes I will have a better look after that. Thanks, Nick From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B0A25EB64D7 for ; Wed, 21 Jun 2023 02:52:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:References:Subject:Cc:To: From:Message-Id:Date:Mime-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=lh29dgYh3L7EdPGbWXtsJVEdlnni/fXqpXMZKM/MHiE=; b=eyNSZUXeuJiDfT RgBgkyB8hKduXl2EKVjhYOpyJfYQCTTFFI+y2ZpvYI4peUYvSFGACUn8ktAc9Qz3/u2s/HWoSiOB5 +5IRFdjrAW2PTRgdEAOi2as5Yg4NSwRZqZYCSgQRqjC23Mgeu+ycR7KhkAidbslntIuMhfXD0+zQ+ mWTcP52qPCTM6SxL5uQWsuQSsJjouYTZyXF+xPoH+p+7+QwLNqk8MzsIKrZU7NFNbpYGrtA6HNkAC OaJ/xhEwflwGQQgAzByNh4LWROEdkd3j8XLc1qjvYQmMIeI3mRIo9mhuerTYJOuIdxFS8n9F6FpND uxNpUwezANKf3GUfeXKA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qBnwo-00Czas-10; Wed, 21 Jun 2023 02:51:54 +0000 Received: from mail-ot1-x332.google.com ([2607:f8b0:4864:20::332]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qBnwl-00CzaC-1T for linux-arm-kernel@lists.infradead.org; Wed, 21 Jun 2023 02:51:53 +0000 Received: by mail-ot1-x332.google.com with SMTP id 46e09a7af769-6b46e540326so2699078a34.2 for ; Tue, 20 Jun 2023 19:51:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687315909; x=1689907909; h=in-reply-to:references:subject:cc:to:from:message-id:date :content-transfer-encoding:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=qJNr1vyPvHbMEk8O8XldE8NbrEXH2idSMvV3eZDgIT4=; b=J9rhQVz1R8lKDLiKwNIPca+zvG5Tv3NsIAj1Ji/wwJWZ5NGPo67uNji7TMbGfymVWO GMr37w69IFJ1a/xO2A5DmceFjN0O32RpzdAIssyO44XqTHGckAF25peyXSxvQDN3Hon3 nm7yK6UQvcoGWP40qbNg1er6nj4r5UQsT0ym99rNBvogtZMf2i5gafTMS+lgcgzNQGKq aoUmdiGQ3/vjF0fTi0Y3GwgVLDETdD79PdEDbQpGXiviLmWKyCtsFqxqvYfIKO+j7cmH 1dwMi6f30X6ti1+jJ2sfUBHAIRnjtvhicwwWnFeGYhp7QveaAM4i9Fm9K8L5dJwflH0T aPMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687315909; x=1689907909; h=in-reply-to:references:subject:cc:to:from:message-id:date :content-transfer-encoding:mime-version:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=qJNr1vyPvHbMEk8O8XldE8NbrEXH2idSMvV3eZDgIT4=; b=HCo8NIlq6PWq3UJkjJcl/JRM+/XJuo3gkTfkoIz4AUCBhlw33/HV+QH7oz86djz4yr 0VwpC0GS111IeuCd0TwJBc/D2pgFVPGcNFgKuVWdGUJ4/jnHsjLG5CfSZY0xpRrlhdav MaXIJ+2vUCKtrqsyy1foS5s6oCAHNTJkQ1na36JZp+eqd1YTVvsWzfeqOFAOrYv9kTpx gON8sCtlf5cNIV1tyKpxmWmq3pokqAbcnh2oeaNT6t568iX3puNxK0Yo3E8l5FFKTaLS AFCcC4fuEGDQ8Ht8Wr0bXEF6vzqcLKBeVRO11lI0LLGsHAXhXbZChInax1EZHHcr2km6 DESg== X-Gm-Message-State: AC+VfDwpqf1ceJSeaecYS/faAzpHL0tWvHXA1fPUKrCsPSgfa/H3hcyU 0AwL6bwXU+v9URTKLRl69EM= X-Google-Smtp-Source: ACHHUZ4vmFcBnDcvLrBXtT15BOX4k0uY1EIRYfrVxGT80rydywp/6szlGIEPs7lB1lYg1S65G5/lLQ== X-Received: by 2002:a05:6359:a26:b0:12c:9960:ec1e with SMTP id el38-20020a0563590a2600b0012c9960ec1emr8123207rwb.19.1687315909090; Tue, 20 Jun 2023 19:51:49 -0700 (PDT) Received: from localhost (193-116-203-37.tpgi.com.au. [193.116.203.37]) by smtp.gmail.com with ESMTPSA id q15-20020a170902bd8f00b001a5fccab02dsm2246983pls.177.2023.06.20.19.51.33 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 20 Jun 2023 19:51:48 -0700 (PDT) Mime-Version: 1.0 Date: Wed, 21 Jun 2023 12:51:31 +1000 Message-Id: From: "Nicholas Piggin" To: "Yu Zhao" Cc: "Andrew Morton" , "Paolo Bonzini" , "Alistair Popple" , "Anup Patel" , "Ben Gardon" , "Borislav Petkov" , "Catalin Marinas" , "Chao Peng" , "Christophe Leroy" , "Dave Hansen" , "Fabiano Rosas" , "Gaosheng Cui" , "Gavin Shan" , "H. Peter Anvin" , "Ingo Molnar" , "James Morse" , "Jason A. Donenfeld" , "Jason Gunthorpe" , "Jonathan Corbet" , "Marc Zyngier" , "Masami Hiramatsu" , "Michael Ellerman" , "Michael Larabel" , "Mike Rapoport" , "Oliver Upton" , "Paul Mackerras" , "Peter Xu" , "Sean Christopherson" , "Steven Rostedt" , "Suzuki K Poulose" , "Thomas Gleixner" , "Thomas Huth" , "Will Deacon" , "Zenghui Yu" , , , , , , , , , , Subject: Re: [PATCH mm-unstable v2 07/10] kvm/powerpc: add kvm_arch_test_clear_young() X-Mailer: aerc 0.14.0 References: <20230526234435.662652-1-yuzhao@google.com> <20230526234435.662652-8-yuzhao@google.com> In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230620_195151_501365_C7882E15 X-CRM114-Status: GOOD ( 45.38 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gV2VkIEp1biAyMSwgMjAyMyBhdCAxMDozOCBBTSBBRVNULCBZdSBaaGFvIHdyb3RlOgo+IE9u IFR1ZSwgSnVuIDIwLCAyMDIzIGF0IDE6NDjigK9BTSBOaWNob2xhcyBQaWdnaW4gPG5waWdnaW5A Z21haWwuY29tPiB3cm90ZToKPiA+Cj4gPiBPbiBTYXQgTWF5IDI3LCAyMDIzIGF0IDk6NDQgQU0g QUVTVCwgWXUgWmhhbyB3cm90ZToKPiA+ID4gSW1wbGVtZW50IGt2bV9hcmNoX3Rlc3RfY2xlYXJf eW91bmcoKSB0byBzdXBwb3J0IHRoZSBmYXN0IHBhdGggaW4KPiA+ID4gbW11X25vdGlmaWVyX29w cy0+dGVzdF9jbGVhcl95b3VuZygpLgo+ID4gPgo+ID4gPiBJdCBmb2N1c2VzIG9uIGEgc2ltcGxl IGNhc2UsIGkuZS4sIHJhZGl4IE1NVSBzZXRzIHRoZSBhY2Nlc3NlZCBiaXQgaW4KPiA+ID4gS1ZN IFBURXMgYW5kIFZNcyBhcmUgbm90IG5lc3RlZCwgd2hlcmUgaXQgY2FuIHJlbHkgb24gUkNVIGFu ZAo+ID4gPiBwdGVfeGNoZygpIHRvIHNhZmVseSBjbGVhciB0aGUgYWNjZXNzZWQgYml0IHdpdGhv dXQgdGFraW5nCj4gPiA+IGt2bS0+bW11X2xvY2suIENvbXBsZXggY2FzZXMgZmFsbCBiYWNrIHRv IHRoZSBleGlzdGluZyBzbG93IHBhdGgKPiA+ID4gd2hlcmUga3ZtLT5tbXVfbG9jayBpcyB0aGVu IHRha2VuLgo+ID4gPgo+ID4gPiBTaWduZWQtb2ZmLWJ5OiBZdSBaaGFvIDx5dXpoYW9AZ29vZ2xl LmNvbT4KPiA+ID4gLS0tCj4gPiA+ICBhcmNoL3Bvd2VycGMvaW5jbHVkZS9hc20va3ZtX2hvc3Qu aCAgICB8ICA4ICsrKysKPiA+ID4gIGFyY2gvcG93ZXJwYy9pbmNsdWRlL2FzbS9rdm1fcHBjLmgg ICAgIHwgIDEgKwo+ID4gPiAgYXJjaC9wb3dlcnBjL2t2bS9ib29rM3MuYyAgICAgICAgICAgICAg fCAgNiArKysKPiA+ID4gIGFyY2gvcG93ZXJwYy9rdm0vYm9vazNzLmggICAgICAgICAgICAgIHwg IDEgKwo+ID4gPiAgYXJjaC9wb3dlcnBjL2t2bS9ib29rM3NfNjRfbW11X3JhZGl4LmMgfCA1OSAr KysrKysrKysrKysrKysrKysrKysrKysrKwo+ID4gPiAgYXJjaC9wb3dlcnBjL2t2bS9ib29rM3Nf aHYuYyAgICAgICAgICAgfCAgNSArKysKPiA+ID4gIDYgZmlsZXMgY2hhbmdlZCwgODAgaW5zZXJ0 aW9ucygrKQo+ID4gPgo+ID4gPiBkaWZmIC0tZ2l0IGEvYXJjaC9wb3dlcnBjL2luY2x1ZGUvYXNt L2t2bV9ob3N0LmggYi9hcmNoL3Bvd2VycGMvaW5jbHVkZS9hc20va3ZtX2hvc3QuaAo+ID4gPiBp bmRleCAxNGVlMGRlY2U4NTMuLjc1YzI2MGVhOGE5ZSAxMDA2NDQKPiA+ID4gLS0tIGEvYXJjaC9w b3dlcnBjL2luY2x1ZGUvYXNtL2t2bV9ob3N0LmgKPiA+ID4gKysrIGIvYXJjaC9wb3dlcnBjL2lu Y2x1ZGUvYXNtL2t2bV9ob3N0LmgKPiA+ID4gQEAgLTg4Myw0ICs4ODMsMTIgQEAgc3RhdGljIGlu bGluZSB2b2lkIGt2bV9hcmNoX3NjaGVkX2luKHN0cnVjdCBrdm1fdmNwdSAqdmNwdSwgaW50IGNw dSkge30KPiA+ID4gIHN0YXRpYyBpbmxpbmUgdm9pZCBrdm1fYXJjaF92Y3B1X2Jsb2NraW5nKHN0 cnVjdCBrdm1fdmNwdSAqdmNwdSkge30KPiA+ID4gIHN0YXRpYyBpbmxpbmUgdm9pZCBrdm1fYXJj aF92Y3B1X3VuYmxvY2tpbmcoc3RydWN0IGt2bV92Y3B1ICp2Y3B1KSB7fQo+ID4gPgo+ID4gPiAr I2RlZmluZSBrdm1fYXJjaF9oYXNfdGVzdF9jbGVhcl95b3VuZyBrdm1fYXJjaF9oYXNfdGVzdF9j bGVhcl95b3VuZwo+ID4gPiArc3RhdGljIGlubGluZSBib29sIGt2bV9hcmNoX2hhc190ZXN0X2Ns ZWFyX3lvdW5nKHZvaWQpCj4gPiA+ICt7Cj4gPiA+ICsgICAgIHJldHVybiBJU19FTkFCTEVEKENP TkZJR19LVk1fQk9PSzNTX0hWX1BPU1NJQkxFKSAmJgo+ID4gPiArICAgICAgICAgICAgY3B1X2hh c19mZWF0dXJlKENQVV9GVFJfSFZNT0RFKSAmJiBjcHVfaGFzX2ZlYXR1cmUoQ1BVX0ZUUl9BUkNI XzMwMCkgJiYKPiA+ID4gKyAgICAgICAgICAgIHJhZGl4X2VuYWJsZWQoKTsKPiA+Cj4gPiBUaGlz IGNvdWxkIHByb2JhYmx5IGJlIHJhZGl4X2VuYWJsZWQoKSAmJiAha3ZtaHZfb25fcHNlcmllcygp Lgo+Cj4gV2lsbCBkby4gKEkgdXNlZCAha3ZtaHZfb25fcHNlcmllcygpIGluIHYxIGJ1dCBoYWQg c2Vjb25kIHRob3VnaHRzIG9uCj4gbW92aW5nIGt2bWh2X29uX3BzZXJpZXMoKSBpbnRvIHRoaXMg ZmlsZS4pCgpUaGF0IHNob3VsZCBiZSBva2F5LiBrdm1odl9vbl9wc2VyaWVzIGlzIGEgcHJvcGVy dHkgb2YgdGhlIGhvc3Qgc28gaXQKc2VlbXMgcmVhc29uYWJsZSB0byBtb3ZlIGl0IGhlcmUgaWYg bmVlZGVkLgoKPiA+IEFsdGhvdWdoIHVuY2xlYXIgd2h5IG5vdCBuZXN0ZWQgaHlwZXJ2aXNvci4u LiBJJ2QgaGF2ZSB0byB0aGluayBhYm91dCBpdCBhIGJpdAo+ID4gbW9yZS4gRG8geW91IGhhdmUg YW55IGlkZWEgd2hhdCBtaWdodCBnbyB3cm9uZywgb3IganVzdCBkaWRuJ3QgaGF2ZSB0aGUKPiA+ IHRpbWUgdG8gY29uc2lkZXIgaXQ/IChOb3QganVzdCBwb3dlcnBjIG5lc3RlZCBidXQgYW55IG90 aGVycykuCj4KPiBZZXMsIHRoaXMgc2VyaWVzIGV4Y2x1ZGVzIG5lc3RlZCBLVk0gc3VwcG9ydCBv biBhbGwgYXJjaGl0dXJlcy4gVGhlCj4gY29tbW9uIHJlYXNvbiBmb3Igc3VjaCBhIGRlY2lzaW9u IG9uIHBvd2VycGMgYW5kIHg4NiAoYWFyY2g2NCBkb2Vzbid0Cj4gc3VwcG9ydCBuZXN0ZWQgeWV0 KSBpcyB0aGF0IGl0J3MgcXVpdGUgY2hhbGxlbmdpbmcgdG8gbWFrZSB0aGUgcm1hcCwgYQo+IGNv bXBsZXggZGF0YSBzdHJ1Y3R1cmUgdGhhdCBtYXBzIG9uZSBQRk4gdG8gbXVsdGlwbGUgR0ZOcywg bG9ja2xlc3MuCj4gKFNlZSBrdm1odl9pbnNlcnRfbmVzdF9ybWFwKCkuKSBJdCBtaWdodCBiZSBw b3NzaWJsZSwgaG93ZXZlciwgdGhlCj4gcG90ZW50aWFsIFJPSSB3b3VsZCBiZSBpbiBxdWVzdGlv bi4KCk9rYXkganVzdCB3b25kZXJpbmcuIHJtYXAgKGF0IGxlYXN0IHRoZSBwb3dlcnBjIG9uZSkg aXMganVzdCBhIGxpc3QKSSB0aGluaywgd2l0aCBhIGZldyBkZXRhaWxzLiBJZiB0aGF0IGlzIGFs bCBpdCBpcywgaXQgbWlnaHQgbm90IGJlCnNvIGhhcmQgdG8gbWFrZSB0aGF0IGxvY2stZnJlZSBv ciBhIGZpbmUtZ3JhaW5lZCBsb2NrIG9uIHRoZSBybWFwCmNoYWlucyBtYXliZS4gQnV0IGZpbmUg dG8gaWdub3JlIGl0IHRvIHN0YXJ0IHdpdGguCgo+ID4gPiArfQo+ID4gPiArCj4gPiA+ICAjZW5k aWYgLyogX19QT1dFUlBDX0tWTV9IT1NUX0hfXyAqLwo+ID4gPiBkaWZmIC0tZ2l0IGEvYXJjaC9w b3dlcnBjL2luY2x1ZGUvYXNtL2t2bV9wcGMuaCBiL2FyY2gvcG93ZXJwYy9pbmNsdWRlL2FzbS9r dm1fcHBjLmgKPiA+ID4gaW5kZXggNzlhOWMwYmI4YmJhLi5mZjFhZjZhN2I0NGYgMTAwNjQ0Cj4g PiA+IC0tLSBhL2FyY2gvcG93ZXJwYy9pbmNsdWRlL2FzbS9rdm1fcHBjLmgKPiA+ID4gKysrIGIv YXJjaC9wb3dlcnBjL2luY2x1ZGUvYXNtL2t2bV9wcGMuaAo+ID4gPiBAQCAtMjg3LDYgKzI4Nyw3 IEBAIHN0cnVjdCBrdm1wcGNfb3BzIHsKPiA+ID4gICAgICAgYm9vbCAoKnVubWFwX2dmbl9yYW5n ZSkoc3RydWN0IGt2bSAqa3ZtLCBzdHJ1Y3Qga3ZtX2dmbl9yYW5nZSAqcmFuZ2UpOwo+ID4gPiAg ICAgICBib29sICgqYWdlX2dmbikoc3RydWN0IGt2bSAqa3ZtLCBzdHJ1Y3Qga3ZtX2dmbl9yYW5n ZSAqcmFuZ2UpOwo+ID4gPiAgICAgICBib29sICgqdGVzdF9hZ2VfZ2ZuKShzdHJ1Y3Qga3ZtICpr dm0sIHN0cnVjdCBrdm1fZ2ZuX3JhbmdlICpyYW5nZSk7Cj4gPiA+ICsgICAgIGJvb2wgKCp0ZXN0 X2NsZWFyX3lvdW5nKShzdHJ1Y3Qga3ZtICprdm0sIHN0cnVjdCBrdm1fZ2ZuX3JhbmdlICpyYW5n ZSk7Cj4gPiA+ICAgICAgIGJvb2wgKCpzZXRfc3B0ZV9nZm4pKHN0cnVjdCBrdm0gKmt2bSwgc3Ry dWN0IGt2bV9nZm5fcmFuZ2UgKnJhbmdlKTsKPiA+ID4gICAgICAgdm9pZCAoKmZyZWVfbWVtc2xv dCkoc3RydWN0IGt2bV9tZW1vcnlfc2xvdCAqc2xvdCk7Cj4gPiA+ICAgICAgIGludCAoKmluaXRf dm0pKHN0cnVjdCBrdm0gKmt2bSk7Cj4gPiA+IGRpZmYgLS1naXQgYS9hcmNoL3Bvd2VycGMva3Zt L2Jvb2szcy5jIGIvYXJjaC9wb3dlcnBjL2t2bS9ib29rM3MuYwo+ID4gPiBpbmRleCA2ODZkOGQ5 ZWRhM2UuLjM3YmY0MGIwYzRmZiAxMDA2NDQKPiA+ID4gLS0tIGEvYXJjaC9wb3dlcnBjL2t2bS9i b29rM3MuYwo+ID4gPiArKysgYi9hcmNoL3Bvd2VycGMva3ZtL2Jvb2szcy5jCj4gPiA+IEBAIC04 OTksNiArODk5LDEyIEBAIGJvb2wga3ZtX3Rlc3RfYWdlX2dmbihzdHJ1Y3Qga3ZtICprdm0sIHN0 cnVjdCBrdm1fZ2ZuX3JhbmdlICpyYW5nZSkKPiA+ID4gICAgICAgcmV0dXJuIGt2bS0+YXJjaC5r dm1fb3BzLT50ZXN0X2FnZV9nZm4oa3ZtLCByYW5nZSk7Cj4gPiA+ICB9Cj4gPiA+Cj4gPiA+ICti b29sIGt2bV9hcmNoX3Rlc3RfY2xlYXJfeW91bmcoc3RydWN0IGt2bSAqa3ZtLCBzdHJ1Y3Qga3Zt X2dmbl9yYW5nZSAqcmFuZ2UpCj4gPiA+ICt7Cj4gPiA+ICsgICAgIHJldHVybiAha3ZtLT5hcmNo Lmt2bV9vcHMtPnRlc3RfY2xlYXJfeW91bmcgfHwKPiA+ID4gKyAgICAgICAgICAgIGt2bS0+YXJj aC5rdm1fb3BzLT50ZXN0X2NsZWFyX3lvdW5nKGt2bSwgcmFuZ2UpOwo+ID4gPiArfQo+ID4gPiAr Cj4gPiA+ICBib29sIGt2bV9zZXRfc3B0ZV9nZm4oc3RydWN0IGt2bSAqa3ZtLCBzdHJ1Y3Qga3Zt X2dmbl9yYW5nZSAqcmFuZ2UpCj4gPiA+ICB7Cj4gPiA+ICAgICAgIHJldHVybiBrdm0tPmFyY2gu a3ZtX29wcy0+c2V0X3NwdGVfZ2ZuKGt2bSwgcmFuZ2UpOwo+ID4gPiBkaWZmIC0tZ2l0IGEvYXJj aC9wb3dlcnBjL2t2bS9ib29rM3MuaCBiL2FyY2gvcG93ZXJwYy9rdm0vYm9vazNzLmgKPiA+ID4g aW5kZXggNTgzOTFiNGIzMmVkLi5mYTI2NTllMjFjY2MgMTAwNjQ0Cj4gPiA+IC0tLSBhL2FyY2gv cG93ZXJwYy9rdm0vYm9vazNzLmgKPiA+ID4gKysrIGIvYXJjaC9wb3dlcnBjL2t2bS9ib29rM3Mu aAo+ID4gPiBAQCAtMTIsNiArMTIsNyBAQCBleHRlcm4gdm9pZCBrdm1wcGNfY29yZV9mbHVzaF9t ZW1zbG90X2h2KHN0cnVjdCBrdm0gKmt2bSwKPiA+ID4gIGV4dGVybiBib29sIGt2bV91bm1hcF9n Zm5fcmFuZ2VfaHYoc3RydWN0IGt2bSAqa3ZtLCBzdHJ1Y3Qga3ZtX2dmbl9yYW5nZSAqcmFuZ2Up Owo+ID4gPiAgZXh0ZXJuIGJvb2wga3ZtX2FnZV9nZm5faHYoc3RydWN0IGt2bSAqa3ZtLCBzdHJ1 Y3Qga3ZtX2dmbl9yYW5nZSAqcmFuZ2UpOwo+ID4gPiAgZXh0ZXJuIGJvb2wga3ZtX3Rlc3RfYWdl X2dmbl9odihzdHJ1Y3Qga3ZtICprdm0sIHN0cnVjdCBrdm1fZ2ZuX3JhbmdlICpyYW5nZSk7Cj4g PiA+ICtleHRlcm4gYm9vbCBrdm1fdGVzdF9jbGVhcl95b3VuZ19odihzdHJ1Y3Qga3ZtICprdm0s IHN0cnVjdCBrdm1fZ2ZuX3JhbmdlICpyYW5nZSk7Cj4gPiA+ICBleHRlcm4gYm9vbCBrdm1fc2V0 X3NwdGVfZ2ZuX2h2KHN0cnVjdCBrdm0gKmt2bSwgc3RydWN0IGt2bV9nZm5fcmFuZ2UgKnJhbmdl KTsKPiA+ID4KPiA+ID4gIGV4dGVybiBpbnQga3ZtcHBjX21tdV9pbml0X3ByKHN0cnVjdCBrdm1f dmNwdSAqdmNwdSk7Cj4gPiA+IGRpZmYgLS1naXQgYS9hcmNoL3Bvd2VycGMva3ZtL2Jvb2szc182 NF9tbXVfcmFkaXguYyBiL2FyY2gvcG93ZXJwYy9rdm0vYm9vazNzXzY0X21tdV9yYWRpeC5jCj4g PiA+IGluZGV4IDNiNjViM2IxMTA0MS4uMGEzOTJlOWExMDBhIDEwMDY0NAo+ID4gPiAtLS0gYS9h cmNoL3Bvd2VycGMva3ZtL2Jvb2szc182NF9tbXVfcmFkaXguYwo+ID4gPiArKysgYi9hcmNoL3Bv d2VycGMva3ZtL2Jvb2szc182NF9tbXVfcmFkaXguYwo+ID4gPiBAQCAtMTA4OCw2ICsxMDg4LDY1 IEBAIGJvb2wga3ZtX3Rlc3RfYWdlX3JhZGl4KHN0cnVjdCBrdm0gKmt2bSwgc3RydWN0IGt2bV9t ZW1vcnlfc2xvdCAqbWVtc2xvdCwKPiA+ID4gICAgICAgcmV0dXJuIHJlZjsKPiA+ID4gIH0KPiA+ ID4KPiA+ID4gK2Jvb2wga3ZtX3Rlc3RfY2xlYXJfeW91bmdfaHYoc3RydWN0IGt2bSAqa3ZtLCBz dHJ1Y3Qga3ZtX2dmbl9yYW5nZSAqcmFuZ2UpCj4gPiA+ICt7Cj4gPiA+ICsgICAgIGJvb2wgZXJy Owo+ID4gPiArICAgICBnZm5fdCBnZm4gPSByYW5nZS0+c3RhcnQ7Cj4gPiA+ICsKPiA+ID4gKyAg ICAgcmN1X3JlYWRfbG9jaygpOwo+ID4gPiArCj4gPiA+ICsgICAgIGVyciA9ICFrdm1faXNfcmFk aXgoa3ZtKTsKPiA+ID4gKyAgICAgaWYgKGVycikKPiA+ID4gKyAgICAgICAgICAgICBnb3RvIHVu bG9jazsKPiA+ID4gKwo+ID4gPiArICAgICAvKgo+ID4gPiArICAgICAgKiBDYXNlIDE6ICBUaGlz IGZ1bmN0aW9uICAgICAgICAgIGt2bXBwY19zd2l0Y2hfbW11X3RvX2hwdCgpCj4gPiA+ICsgICAg ICAqCj4gPiA+ICsgICAgICAqICAgICAgICAgIHJjdV9yZWFkX2xvY2soKQo+ID4gPiArICAgICAg KiAgICAgICAgICBUZXN0IGt2bV9pc19yYWRpeCgpICAgIGt2bS0+YXJjaC5yYWRpeCA9IDAKPiA+ ID4gKyAgICAgICogICAgICAgICAgVXNlIGt2bS0+YXJjaC5wZ3RhYmxlICBzeW5jaHJvbml6ZV9y Y3UoKQo+ID4gPiArICAgICAgKiAgICAgICAgICByY3VfcmVhZF91bmxvY2soKQo+ID4gPiArICAg ICAgKiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGt2bXBwY19mcmVlX3JhZGl4KCkK PiA+ID4gKyAgICAgICoKPiA+ID4gKyAgICAgICoKPiA+ID4gKyAgICAgICogQ2FzZSAyOiAgVGhp cyBmdW5jdGlvbiAgICAgICAgICBrdm1wcGNfc3dpdGNoX21tdV90b19yYWRpeCgpCj4gPiA+ICsg ICAgICAqCj4gPiA+ICsgICAgICAqICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAga3Zt cHBjX2luaXRfdm1fcmFkaXgoKQo+ID4gPiArICAgICAgKiAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgIHNtcF93bWIoKQo+ID4gPiArICAgICAgKiAgICAgICAgICBUZXN0IGt2bV9pc19y YWRpeCgpICAgIGt2bS0+YXJjaC5yYWRpeCA9IDEKPiA+ID4gKyAgICAgICogICAgICAgICAgc21w X3JtYigpCj4gPiA+ICsgICAgICAqICAgICAgICAgIFVzZSBrdm0tPmFyY2gucGd0YWJsZQo+ID4g PiArICAgICAgKi8KPiA+ID4gKyAgICAgc21wX3JtYigpOwo+ID4KPiA+IENvbW1lbnQgY291bGQg c3RhbmQgdG8gZXhwYW5kIHNsaWdodGx5IG9uIHdoYXQgeW91IGFyZSBzb2x2aW5nLCBpbgo+ID4g d29yZHMuCj4KPiBXaWxsIGRvLgo+Cj4gPiBJZiB5b3UgdXNlIHN5bmNocm9uaXplX3JjdSgpIG9u IGJvdGggc2lkZXMsIHlvdSB3b3VsZG4ndCBuZWVkIHRoZQo+ID4gYmFycmllciwgcmlnaHQ/Cj4K PiBDYXNlIDIgaXMgYWJvdXQgbWVtb3J5IG9yZGVyaW5nLCB3aGljaCBpcyBvcnRob2dvbmFsIHRv IGNhc2UgMSAoUkNVCj4gZnJlZWluZykuIFNvIHdlIG5lZWQgdGhlIHIvdyBiYXJyaWVyIHJlZ2Fy ZGxlc3MuCgpSQ1UgY2FuIHRha2UgY2FyZSBvZiBtZW1vcnkgb3JkZXJpbmcgdG9vIHRob3VnaC4g SWYgeW91IGhhZApzeW5jaHJvbml6ZV9yY3UoKSB3aGVyZSBzbXBfd21iKCkgaXMsIHRoZW4gbm8g c21wX3JtYigpIG5lZWVkIGhlcmUuCgo+Cj4gPiA+ICsgICAgIHdoaWxlIChnZm4gPCByYW5nZS0+ ZW5kKSB7Cj4gPiA+ICsgICAgICAgICAgICAgcHRlX3QgKnB0ZXA7Cj4gPiA+ICsgICAgICAgICAg ICAgcHRlX3Qgb2xkLCBuZXc7Cj4gPiA+ICsgICAgICAgICAgICAgdW5zaWduZWQgaW50IHNoaWZ0 Owo+ID4gPiArCj4gPiA+ICsgICAgICAgICAgICAgcHRlcCA9IGZpbmRfa3ZtX3NlY29uZGFyeV9w dGVfdW5sb2NrZWQoa3ZtLCBnZm4gKiBQQUdFX1NJWkUsICZzaGlmdCk7Cj4gPiA+ICsgICAgICAg ICAgICAgaWYgKCFwdGVwKQo+ID4gPiArICAgICAgICAgICAgICAgICAgICAgZ290byBuZXh0Owo+ ID4gPiArCj4gPiA+ICsgICAgICAgICAgICAgVk1fV0FSTl9PTl9PTkNFKCFwYWdlX2NvdW50KHZp cnRfdG9fcGFnZShwdGVwKSkpOwo+ID4KPiA+IE5vdCByZWFsbHkgYXBwcm9wcmlhdGUgYXQgdGhl IEtWTSBsZXZlbC4gbW0gZW5mb3JjZXMgdGhpcyBraW5kIG9mCj4gPiB0aGluZyAod2l0aCBub3Rp ZmllcnMpLgo+Cj4gV2lsbCByZW1vdmUgdGhpcy4KPgo+ID4gPiArCj4gPiA+ICsgICAgICAgICAg ICAgb2xkID0gUkVBRF9PTkNFKCpwdGVwKTsKPiA+ID4gKyAgICAgICAgICAgICBpZiAoIXB0ZV9w cmVzZW50KG9sZCkgfHwgIXB0ZV95b3VuZyhvbGQpKQo+ID4gPiArICAgICAgICAgICAgICAgICAg ICAgZ290byBuZXh0Owo+ID4gPiArCj4gPiA+ICsgICAgICAgICAgICAgbmV3ID0gcHRlX21rb2xk KG9sZCk7Cj4gPiA+ICsKPiA+ID4gKyAgICAgICAgICAgICBpZiAoa3ZtX3Nob3VsZF9jbGVhcl95 b3VuZyhyYW5nZSwgZ2ZuKSkKPiA+ID4gKyAgICAgICAgICAgICAgICAgICAgIHB0ZV94Y2hnKHB0 ZXAsIG9sZCwgbmV3KTsKPiA+Cj4gPiAqUHJvYmFibHkqIHdpbGwgd29yay4gSSBjYW4ndCB0aGlu ayBvZiBhIHJlYXNvbiB3aHkgbm90IGF0IHRoZQo+ID4gbW9tZW50IGFueXdheSA6KQo+Cj4gTXkg cmVhc29uaW5nOgo+ICogSXQgc2hvdWxkIHdvcmsgaWYgd2Ugb25seSBjaGFuZ2UgdGhlIGRlZGlj YXRlZCBBIGJpdCwgaS5lLiwgbm90Cj4gc2hhcmVkIGZvciBvdGhlciBwdXJwb3NlcywgYmVjYXVz ZSByYWNlcyBhcmUgc2FmZSAodGhlIGNhc2UgaGVyZSkuCj4gKiBJdCBtYXkgbm90IHdvcmsgZm9y IHg4NiBFUFQgd2l0aG91dCB0aGUgQSBiaXQgKGV4Y2x1ZGVkIGluIHRoaXMKPiBzZXJpZXMpIHdo ZXJlIGFjY2Vzc2VzIGFyZSB0cmFwcGVkIGJ5IHRoZSBSL1ggYml0cywgYmVjYXVzZSByYWNlcyBp bgo+IGNoYW5naW5nIHRoZSBSL1ggYml0cyBjYW4gYmUgdW5zYWZlLgoKKEZvciB0aGUgYmVuZWZp dCBvZiBvdGhlcnMgcmVhZGluZywgaXQgd29ya3MgYmVjYXVzZSBwb3dlcnBjJ3MgcHRlX3hjaGcK aXMgYWN0dWFsbHkgYSBjbXB4Y2hnLCBmb3Igc29tZSByZWFzb24gd2hpY2ggd2UgcmVhbGx5IHNo b3VsZCBmaXgpLgpBbHRob3VnaCBpdCBjYW4gZmFpbCB0byBjbGVhciB0aGUgYml0IGlmIHRoZSBj bXB4Y2hnIGZhaWxzLgoKSSB0aGluayBwdGVfeGNoZyBpcyBvbmx5IHVzZWQgb24gd2l0aCBoYXNo IE1NVSBpbiBMaW51eCBiZWZvcmUgdGhpcwpjaGFuZ2UuIEkgdGhpbmsgd2UgbWF5IHdhbnQgdG8g a2VlcCBpdCB0aGF0IHdheSBhbmQgdXNlIHNvbWV0aGluZwpsaWtlIGt2bXBwY19yYWRpeF91cGRh dGVfcHRlKCkgaGVyZSB0byBjbGVhciBvdXQgdGhlIGJpdC4gQnV0IGRvbid0CndvcnJ5IHRvbyBt dWNoIGFib3V0IGZpbmUgZGV0YWlscyBzbyBtdWNoIGJlZm9yZSBzb3J0aW5nIG91dCB0aGUKY29y ZSBjaGFuZ2VzIEkgd2lsbCBoYXZlIGEgYmV0dGVyIGxvb2sgYWZ0ZXIgdGhhdC4KClRoYW5rcywK TmljawoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGlu dXgtYXJtLWtlcm5lbCBtYWlsaW5nIGxpc3QKbGludXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRl YWQub3JnCmh0dHA6Ly9saXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8vbGludXgt YXJtLWtlcm5lbAo=