From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from out-11.mta1.migadu.com (out-11.mta1.migadu.com [95.215.58.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 48462C942 for ; Tue, 4 Apr 2023 19:09:25 +0000 (UTC) Date: Tue, 4 Apr 2023 19:09:17 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1680635363; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=S1gwK2nwRYjccyYSDh2DfgE6FA3Ietx2jBphXAwjNbk=; b=Yu2TDQQadJpNt4mwGvQ9bzOiNHxLFFVeynE9Zm4Z/2zV4EpNmeVQ0aosrF7mNhuNrIwjzT gapdoPi3cKVwSWnN3/qKJohgqXd7/RWoQXLNr/W+t79INkjFQMYcwXRQ4AKeiSxDFLeBKr b8zbHPJJu5vXodgmHwXHnG2L+fKTZPM= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Oliver Upton To: Raghavendra Rao Ananta Cc: Oliver Upton , Marc Zyngier , Ricardo Koller , Reiji Watanabe , James Morse , Alexandru Elisei , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Catalin Marinas , Jing Zhang , Colton Lewis , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Subject: Re: [PATCH v2 4/7] KVM: arm64: Implement kvm_arch_flush_remote_tlbs_range() Message-ID: References: <20230206172340.2639971-1-rananta@google.com> <20230206172340.2639971-5-rananta@google.com> Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Migadu-Flow: FLOW_OUT On Mon, Apr 03, 2023 at 02:23:17PM -0700, Raghavendra Rao Ananta wrote: > On Wed, Mar 29, 2023 at 5:53 PM Oliver Upton wrote: > > > > On Mon, Feb 06, 2023 at 05:23:37PM +0000, Raghavendra Rao Ananta wrote: > > > Implement kvm_arch_flush_remote_tlbs_range() for arm64, > > > such that it can utilize the TLBI range based instructions > > > if supported. > > > > > > Signed-off-by: Raghavendra Rao Ananta > > > --- > > > arch/arm64/include/asm/kvm_host.h | 3 +++ > > > arch/arm64/kvm/mmu.c | 15 +++++++++++++++ > > > 2 files changed, 18 insertions(+) > > > > > > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h > > > index dee530d75b957..211fab0c1de74 100644 > > > --- a/arch/arm64/include/asm/kvm_host.h > > > +++ b/arch/arm64/include/asm/kvm_host.h > > > @@ -1002,6 +1002,9 @@ struct kvm *kvm_arch_alloc_vm(void); > > > #define __KVM_HAVE_ARCH_FLUSH_REMOTE_TLBS > > > int kvm_arch_flush_remote_tlbs(struct kvm *kvm); > > > > > > +#define __KVM_HAVE_ARCH_FLUSH_REMOTE_TLBS_RANGE > > > +int kvm_arch_flush_remote_tlbs_range(struct kvm *kvm, gfn_t start_gfn, u64 pages); > > > + > > > static inline bool kvm_vm_is_protected(struct kvm *kvm) > > > { > > > return false; > > > diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c > > > index e98910a8d0af6..409cb187f4911 100644 > > > --- a/arch/arm64/kvm/mmu.c > > > +++ b/arch/arm64/kvm/mmu.c > > > @@ -91,6 +91,21 @@ int kvm_arch_flush_remote_tlbs(struct kvm *kvm) > > > return 0; > > > } > > > > > > +int kvm_arch_flush_remote_tlbs_range(struct kvm *kvm, gfn_t start_gfn, u64 pages) > > > +{ > > > + phys_addr_t start, end; > > > + > > > + if (!system_supports_tlb_range()) > > > + return -EOPNOTSUPP; > > > > There's multiple layers of fallback throughout this series, as it would > > appear that deep in __kvm_tlb_flush_range() you're blasting the whole > > VMID if either the range is too large or the feature isn't supported. > > > > Is it possible to just normalize on a single spot to gate the use of > > range-based invalidations? I have a slight preference for doing it deep > > in the handler, as it keeps the upper layers of code a bit more > > readable. > > > I was a little skeptical on this part, since the > kvm_arch_flush_remote_tlbs_range() expects to return -EOPNOTSUPP if > indeed there's no support. Well, the arch-neutral code can expect whatever it wants :) The only real contract we have with it is to return 0 iff the specified range has been invalidated, even if that comes with over-invalidating. > But I see your point. The if-else in kvm_pgtable_stage2_flush_range() > seems redundant and I can simply manage this conditions inside > __kvm_tlb_flush_range_vmid_ipa() itself, but I'll leave the > kvm_arch_flush_remote_tlbs_range()'s implementation as is. Thoughts? The largest concern I had is that the series is testing for FEAT_TLBIRANGE all over the shop and I just raised that concern on this patch. AFAICT, the iterative approach to invalidating a range of IPAs is effectively dead code, as all flows into __kvm_tlb_flush_range_vmid_ipa() are gated by system_supports_tlb_range() somewhere. Personally, I prefer keeping the higher level software models making aggressive use of range-based interfaces and letting the actual implementation under the hood select the appropriate instruction. That helps readability, as it directly communicates the expected outcome of the invalidation. So, if you want to make use of the iterative approach to TLB invalidations on !TLBIRANGE systems, then this function should _not_ return EOPNOTSUPP. -- Thanks, Oliver From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 10E79C6FD1D for ; Tue, 4 Apr 2023 19:10:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=w6kkXVzBgjYuIghe9n64lD71VrH14XgKDZOAhB/PT+k=; b=4HRaYEN+pt1MY4 p/VU4s6RwbwArm3yerUYzq6BVpz169bU3VZij63Q9HmFYkr/ysOjSPzNEoBVxEtAFiuLJKs+OajPD RVsyyeKmBIrQQxXHJC7kvRNrcf2PvPF+SQr+mRu3R3BWjZQUVrO5F8MUyT+J70CPZOXgWBdeyQP0Y 8bJ0FdvSbhBadSipQMHIUZh8RhEt1AGAo/cw2JflE/MLt6r3Ydjs3h6F6NqkPl+CmJCPvbu9hvJGd PGJrIyTZH2NbxlfnVvX1dt5dwoRke+gbAsRrZBFA6G/9NxpGbUiUBS+uyuO8xX7AIMz3mpqHaPIgh WnCEKVqqlpb5nII4gAeQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1pjm26-002a4s-1e; Tue, 04 Apr 2023 19:09:30 +0000 Received: from out-22.mta1.migadu.com ([95.215.58.22]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1pjm22-002a3l-1u for linux-arm-kernel@lists.infradead.org; Tue, 04 Apr 2023 19:09:28 +0000 Date: Tue, 4 Apr 2023 19:09:17 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1680635363; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=S1gwK2nwRYjccyYSDh2DfgE6FA3Ietx2jBphXAwjNbk=; b=Yu2TDQQadJpNt4mwGvQ9bzOiNHxLFFVeynE9Zm4Z/2zV4EpNmeVQ0aosrF7mNhuNrIwjzT gapdoPi3cKVwSWnN3/qKJohgqXd7/RWoQXLNr/W+t79INkjFQMYcwXRQ4AKeiSxDFLeBKr b8zbHPJJu5vXodgmHwXHnG2L+fKTZPM= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Oliver Upton To: Raghavendra Rao Ananta Cc: Oliver Upton , Marc Zyngier , Ricardo Koller , Reiji Watanabe , James Morse , Alexandru Elisei , Suzuki K Poulose , Will Deacon , Paolo Bonzini , Catalin Marinas , Jing Zhang , Colton Lewis , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Subject: Re: [PATCH v2 4/7] KVM: arm64: Implement kvm_arch_flush_remote_tlbs_range() Message-ID: References: <20230206172340.2639971-1-rananta@google.com> <20230206172340.2639971-5-rananta@google.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Migadu-Flow: FLOW_OUT X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230404_120927_046602_A64EB021 X-CRM114-Status: GOOD ( 31.35 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gTW9uLCBBcHIgMDMsIDIwMjMgYXQgMDI6MjM6MTdQTSAtMDcwMCwgUmFnaGF2ZW5kcmEgUmFv IEFuYW50YSB3cm90ZToKPiBPbiBXZWQsIE1hciAyOSwgMjAyMyBhdCA1OjUz4oCvUE0gT2xpdmVy IFVwdG9uIDxvbGl2ZXIudXB0b25AbGludXguZGV2PiB3cm90ZToKPiA+Cj4gPiBPbiBNb24sIEZl YiAwNiwgMjAyMyBhdCAwNToyMzozN1BNICswMDAwLCBSYWdoYXZlbmRyYSBSYW8gQW5hbnRhIHdy b3RlOgo+ID4gPiBJbXBsZW1lbnQga3ZtX2FyY2hfZmx1c2hfcmVtb3RlX3RsYnNfcmFuZ2UoKSBm b3IgYXJtNjQsCj4gPiA+IHN1Y2ggdGhhdCBpdCBjYW4gdXRpbGl6ZSB0aGUgVExCSSByYW5nZSBi YXNlZCBpbnN0cnVjdGlvbnMKPiA+ID4gaWYgc3VwcG9ydGVkLgo+ID4gPgo+ID4gPiBTaWduZWQt b2ZmLWJ5OiBSYWdoYXZlbmRyYSBSYW8gQW5hbnRhIDxyYW5hbnRhQGdvb2dsZS5jb20+Cj4gPiA+ IC0tLQo+ID4gPiAgYXJjaC9hcm02NC9pbmNsdWRlL2FzbS9rdm1faG9zdC5oIHwgIDMgKysrCj4g PiA+ICBhcmNoL2FybTY0L2t2bS9tbXUuYyAgICAgICAgICAgICAgfCAxNSArKysrKysrKysrKysr KysKPiA+ID4gIDIgZmlsZXMgY2hhbmdlZCwgMTggaW5zZXJ0aW9ucygrKQo+ID4gPgo+ID4gPiBk aWZmIC0tZ2l0IGEvYXJjaC9hcm02NC9pbmNsdWRlL2FzbS9rdm1faG9zdC5oIGIvYXJjaC9hcm02 NC9pbmNsdWRlL2FzbS9rdm1faG9zdC5oCj4gPiA+IGluZGV4IGRlZTUzMGQ3NWI5NTcuLjIxMWZh YjBjMWRlNzQgMTAwNjQ0Cj4gPiA+IC0tLSBhL2FyY2gvYXJtNjQvaW5jbHVkZS9hc20va3ZtX2hv c3QuaAo+ID4gPiArKysgYi9hcmNoL2FybTY0L2luY2x1ZGUvYXNtL2t2bV9ob3N0LmgKPiA+ID4g QEAgLTEwMDIsNiArMTAwMiw5IEBAIHN0cnVjdCBrdm0gKmt2bV9hcmNoX2FsbG9jX3ZtKHZvaWQp Owo+ID4gPiAgI2RlZmluZSBfX0tWTV9IQVZFX0FSQ0hfRkxVU0hfUkVNT1RFX1RMQlMKPiA+ID4g IGludCBrdm1fYXJjaF9mbHVzaF9yZW1vdGVfdGxicyhzdHJ1Y3Qga3ZtICprdm0pOwo+ID4gPgo+ ID4gPiArI2RlZmluZSBfX0tWTV9IQVZFX0FSQ0hfRkxVU0hfUkVNT1RFX1RMQlNfUkFOR0UKPiA+ ID4gK2ludCBrdm1fYXJjaF9mbHVzaF9yZW1vdGVfdGxic19yYW5nZShzdHJ1Y3Qga3ZtICprdm0s IGdmbl90IHN0YXJ0X2dmbiwgdTY0IHBhZ2VzKTsKPiA+ID4gKwo+ID4gPiAgc3RhdGljIGlubGlu ZSBib29sIGt2bV92bV9pc19wcm90ZWN0ZWQoc3RydWN0IGt2bSAqa3ZtKQo+ID4gPiAgewo+ID4g PiAgICAgICByZXR1cm4gZmFsc2U7Cj4gPiA+IGRpZmYgLS1naXQgYS9hcmNoL2FybTY0L2t2bS9t bXUuYyBiL2FyY2gvYXJtNjQva3ZtL21tdS5jCj4gPiA+IGluZGV4IGU5ODkxMGE4ZDBhZjYuLjQw OWNiMTg3ZjQ5MTEgMTAwNjQ0Cj4gPiA+IC0tLSBhL2FyY2gvYXJtNjQva3ZtL21tdS5jCj4gPiA+ ICsrKyBiL2FyY2gvYXJtNjQva3ZtL21tdS5jCj4gPiA+IEBAIC05MSw2ICs5MSwyMSBAQCBpbnQg a3ZtX2FyY2hfZmx1c2hfcmVtb3RlX3RsYnMoc3RydWN0IGt2bSAqa3ZtKQo+ID4gPiAgICAgICBy ZXR1cm4gMDsKPiA+ID4gIH0KPiA+ID4KPiA+ID4gK2ludCBrdm1fYXJjaF9mbHVzaF9yZW1vdGVf dGxic19yYW5nZShzdHJ1Y3Qga3ZtICprdm0sIGdmbl90IHN0YXJ0X2dmbiwgdTY0IHBhZ2VzKQo+ ID4gPiArewo+ID4gPiArICAgICBwaHlzX2FkZHJfdCBzdGFydCwgZW5kOwo+ID4gPiArCj4gPiA+ ICsgICAgIGlmICghc3lzdGVtX3N1cHBvcnRzX3RsYl9yYW5nZSgpKQo+ID4gPiArICAgICAgICAg ICAgIHJldHVybiAtRU9QTk9UU1VQUDsKPiA+Cj4gPiBUaGVyZSdzIG11bHRpcGxlIGxheWVycyBv ZiBmYWxsYmFjayB0aHJvdWdob3V0IHRoaXMgc2VyaWVzLCBhcyBpdCB3b3VsZAo+ID4gYXBwZWFy IHRoYXQgZGVlcCBpbiBfX2t2bV90bGJfZmx1c2hfcmFuZ2UoKSB5b3UncmUgYmxhc3RpbmcgdGhl IHdob2xlCj4gPiBWTUlEIGlmIGVpdGhlciB0aGUgcmFuZ2UgaXMgdG9vIGxhcmdlIG9yIHRoZSBm ZWF0dXJlIGlzbid0IHN1cHBvcnRlZC4KPiA+Cj4gPiBJcyBpdCBwb3NzaWJsZSB0byBqdXN0IG5v cm1hbGl6ZSBvbiBhIHNpbmdsZSBzcG90IHRvIGdhdGUgdGhlIHVzZSBvZgo+ID4gcmFuZ2UtYmFz ZWQgaW52YWxpZGF0aW9ucz8gSSBoYXZlIGEgc2xpZ2h0IHByZWZlcmVuY2UgZm9yIGRvaW5nIGl0 IGRlZXAKPiA+IGluIHRoZSBoYW5kbGVyLCBhcyBpdCBrZWVwcyB0aGUgdXBwZXIgbGF5ZXJzIG9m IGNvZGUgYSBiaXQgbW9yZQo+ID4gcmVhZGFibGUuCj4gPgo+IEkgd2FzIGEgbGl0dGxlIHNrZXB0 aWNhbCBvbiB0aGlzIHBhcnQsIHNpbmNlIHRoZQo+IGt2bV9hcmNoX2ZsdXNoX3JlbW90ZV90bGJz X3JhbmdlKCkgZXhwZWN0cyB0byByZXR1cm4gLUVPUE5PVFNVUFAgaWYKPiBpbmRlZWQgdGhlcmUn cyBubyBzdXBwb3J0LgoKV2VsbCwgdGhlIGFyY2gtbmV1dHJhbCBjb2RlIGNhbiBleHBlY3Qgd2hh dGV2ZXIgaXQgd2FudHMgOikgVGhlIG9ubHkKcmVhbCBjb250cmFjdCB3ZSBoYXZlIHdpdGggaXQg aXMgdG8gcmV0dXJuIDAgaWZmIHRoZSBzcGVjaWZpZWQgcmFuZ2UgaGFzCmJlZW4gaW52YWxpZGF0 ZWQsIGV2ZW4gaWYgdGhhdCBjb21lcyB3aXRoIG92ZXItaW52YWxpZGF0aW5nLgoKPiBCdXQgSSBz ZWUgeW91ciBwb2ludC4gVGhlIGlmLWVsc2UgaW4ga3ZtX3BndGFibGVfc3RhZ2UyX2ZsdXNoX3Jh bmdlKCkKPiBzZWVtcyByZWR1bmRhbnQgYW5kIEkgY2FuIHNpbXBseSBtYW5hZ2UgdGhpcyBjb25k aXRpb25zIGluc2lkZQo+IF9fa3ZtX3RsYl9mbHVzaF9yYW5nZV92bWlkX2lwYSgpIGl0c2VsZiwg YnV0IEknbGwgbGVhdmUgdGhlCj4ga3ZtX2FyY2hfZmx1c2hfcmVtb3RlX3RsYnNfcmFuZ2UoKSdz IGltcGxlbWVudGF0aW9uIGFzIGlzLiBUaG91Z2h0cz8KClRoZSBsYXJnZXN0IGNvbmNlcm4gSSBo YWQgaXMgdGhhdCB0aGUgc2VyaWVzIGlzIHRlc3RpbmcgZm9yIEZFQVRfVExCSVJBTkdFCmFsbCBv dmVyIHRoZSBzaG9wIGFuZCBJIGp1c3QgcmFpc2VkIHRoYXQgY29uY2VybiBvbiB0aGlzIHBhdGNo LiBBRkFJQ1QsCnRoZSBpdGVyYXRpdmUgYXBwcm9hY2ggdG8gaW52YWxpZGF0aW5nIGEgcmFuZ2Ug b2YgSVBBcyBpcyBlZmZlY3RpdmVseQpkZWFkIGNvZGUsIGFzIGFsbCBmbG93cyBpbnRvIF9fa3Zt X3RsYl9mbHVzaF9yYW5nZV92bWlkX2lwYSgpIGFyZSBnYXRlZApieSBzeXN0ZW1fc3VwcG9ydHNf dGxiX3JhbmdlKCkgc29tZXdoZXJlLgoKUGVyc29uYWxseSwgSSBwcmVmZXIga2VlcGluZyB0aGUg aGlnaGVyIGxldmVsIHNvZnR3YXJlIG1vZGVscyBtYWtpbmcKYWdncmVzc2l2ZSB1c2Ugb2YgcmFu Z2UtYmFzZWQgaW50ZXJmYWNlcyBhbmQgbGV0dGluZyB0aGUgYWN0dWFsCmltcGxlbWVudGF0aW9u IHVuZGVyIHRoZSBob29kIHNlbGVjdCB0aGUgYXBwcm9wcmlhdGUgaW5zdHJ1Y3Rpb24uIFRoYXQK aGVscHMgcmVhZGFiaWxpdHksIGFzIGl0IGRpcmVjdGx5IGNvbW11bmljYXRlcyB0aGUgZXhwZWN0 ZWQgb3V0Y29tZSBvZgp0aGUgaW52YWxpZGF0aW9uLgoKU28sIGlmIHlvdSB3YW50IHRvIG1ha2Ug dXNlIG9mIHRoZSBpdGVyYXRpdmUgYXBwcm9hY2ggdG8gVExCIGludmFsaWRhdGlvbnMgb24KIVRM QklSQU5HRSBzeXN0ZW1zLCB0aGVuIHRoaXMgZnVuY3Rpb24gc2hvdWxkIF9ub3RfIHJldHVybiBF T1BOT1RTVVBQLgoKLS0gClRoYW5rcywKT2xpdmVyCgpfX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fXwpsaW51eC1hcm0ta2VybmVsIG1haWxpbmcgbGlzdApsaW51 eC1hcm0ta2VybmVsQGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVhZC5v cmcvbWFpbG1hbi9saXN0aW5mby9saW51eC1hcm0ta2VybmVsCg==