From mboxrd@z Thu Jan 1 00:00:00 1970 Received: by 2002:a19:ee12:0:0:0:0:0 with SMTP id g18csp1920693lfb; Tue, 14 Jun 2022 01:41:17 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzScMR2q3Pg6RQwGCyHFSudmTlcFh9cWiRqLd+7mukb5N9xzt2y1I/30/hoOi0ur0Rsj79c X-Received: by 2002:a05:620a:410d:b0:6a7:571d:8615 with SMTP id j13-20020a05620a410d00b006a7571d8615mr3104918qko.259.1655196077113; Tue, 14 Jun 2022 01:41:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1655196077; cv=none; d=google.com; s=arc-20160816; b=oKC9dH7XLDEN9pUdP6tnErZDZxu/N4n5B8YzzRGG09SWpRxeyC8xqrVvaEcTsz20tz Bbr+jv3WAxzD7LaXKLk0JXt0df2WHSQO3ESkWIWaAcvKe5Y55W8VueqIhT2a118f7jcf RNGtzPz8j/q/zbn8YTovhaOhweREnOZcanzsPvrkkloLYrKwZTymQJOSduNPC07iAnnJ s/H/ZAThHcMgZSRvIl3dsPKSMa/O8QkTMLr3uN/MjDlQmtX2HN41U2B+vz+tAHQaOBw6 iJAFieLrPLuJJZjXAmPqHHzid4j/qQdNPNMXvkvlsBGDuCP6k5LRXEFAzeFJjQR1IFFZ oBDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:mime-version:message-id:date :user-agent:references:organization:in-reply-to:subject:cc:to:from :dkim-signature; bh=7JqiAgsVYLwS4+FFvNHU6GMxusP/+pOciDPNZhIjWYw=; b=QRu2SGbBKNdqyRLM9Kl1aBeKSl+8cD+B5T3AX2n8id4DPb/taguOlOpeJ4QJ8OLjc1 pi9YqgMvNVjf9YqMh9U27Irkih6pIajIW8Ofrc2PaJRkPppkKaH34qX7tncZXnwMZjUR MlfTQBdAZF4cOfaR5+Dj63OvXxB9w6fg0+ixbOlC9V+0h7UqATx7zanSjdl8cU4/V2k7 lAbtZqmKt0Bn+8QboBwUg5S3tuDdTpcuR5RQCl8xkI4THJlrHvxN2DAYdXBFWCfe1HH4 zE6GBG14O/XOIThYJEucaDp38CIuukv6FYMTtTPJYSosYtCcp047qTQATJDHhPWbt9c6 iY/A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b="IrqYeh/3"; spf=pass (google.com: domain of qemu-arm-bounces+alex.bennee=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-arm-bounces+alex.bennee=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id q8-20020a05622a030800b00304d3b60ff2si5237680qtw.175.2022.06.14.01.41.16 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 14 Jun 2022 01:41:17 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-arm-bounces+alex.bennee=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b="IrqYeh/3"; spf=pass (google.com: domain of qemu-arm-bounces+alex.bennee=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-arm-bounces+alex.bennee=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from localhost ([::1]:48964 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o126u-0007CU-Hf for alex.bennee@linaro.org; Tue, 14 Jun 2022 04:41:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37230) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o126d-0007AH-Sl for qemu-arm@nongnu.org; Tue, 14 Jun 2022 04:41:00 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:23985) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o126b-0007UN-Md for qemu-arm@nongnu.org; Tue, 14 Jun 2022 04:40:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1655196057; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=7JqiAgsVYLwS4+FFvNHU6GMxusP/+pOciDPNZhIjWYw=; b=IrqYeh/31r0j/vr1SA0outgUNfGsEvBDLdtG/CjX6X+eRug+5LAS0geSCwER6hvFAKaaYI GgmU33zQn2NTBD7hWrk+yrB7BvHP3eR3mxv2m6cOlmgKrvyQAzfkQ0/0gqQzSsWXUx+e7V EOtXCQdXTMrDC3OWbAkIcb5Cj9QMvkA= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-111-mA8d-KHJNYa1q9rSGXm01g-1; Tue, 14 Jun 2022 04:40:42 -0400 X-MC-Unique: mA8d-KHJNYa1q9rSGXm01g-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9796A811E81; Tue, 14 Jun 2022 08:40:41 +0000 (UTC) Received: from localhost (unknown [10.39.193.235]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 312E69D7F; Tue, 14 Jun 2022 08:40:41 +0000 (UTC) From: Cornelia Huck To: Eric Auger , Peter Maydell , Thomas Huth , Laurent Vivier Cc: Andrew Jones , qemu-arm@nongnu.org, qemu-devel@nongnu.org, kvm@vger.kernel.org Subject: Re: [PATCH RFC 1/2] arm/kvm: enable MTE if available In-Reply-To: Organization: Red Hat GmbH References: <20220512131146.78457-1-cohuck@redhat.com> <20220512131146.78457-2-cohuck@redhat.com> User-Agent: Notmuch/0.36 (https://notmuchmail.org) Date: Tue, 14 Jun 2022 10:40:39 +0200 Message-ID: <877d5jskmw.fsf@redhat.com> MIME-Version: 1.0 Content-Type: text/plain X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 Received-SPF: pass client-ip=170.10.129.124; envelope-from=cohuck@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-arm@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-arm-bounces+alex.bennee=linaro.org@nongnu.org Sender: "Qemu-arm" X-TUID: EUPvjy2eP4M4 On Fri, Jun 10 2022, Eric Auger wrote: > Hi Connie, > On 5/12/22 15:11, Cornelia Huck wrote: >> We need to disable migration, as we do not yet have a way to migrate >> the tags as well. > > This patch does much more than adding a migration blocker ;-) you may > describe the new cpu option and how it works. I admit this is a bit terse ;) The idea is to control mte at the cpu level directly (and not indirectly via tag memory at the machine level). I.e. the user gets whatever is available given the constraints (host support etc.) if they don't specify anything, and they can explicitly turn it off/on. >> >> Signed-off-by: Cornelia Huck >> --- >> target/arm/cpu.c | 18 ++++------ >> target/arm/cpu.h | 4 +++ >> target/arm/cpu64.c | 78 ++++++++++++++++++++++++++++++++++++++++++++ >> target/arm/kvm64.c | 5 +++ >> target/arm/kvm_arm.h | 12 +++++++ >> target/arm/monitor.c | 1 + >> 6 files changed, 106 insertions(+), 12 deletions(-) >> >> diff --git a/target/arm/cpu.c b/target/arm/cpu.c >> index 029f644768b1..f0505815b1e7 100644 >> --- a/target/arm/cpu.c >> +++ b/target/arm/cpu.c >> @@ -1435,6 +1435,11 @@ void arm_cpu_finalize_features(ARMCPU *cpu, Error **errp) >> error_propagate(errp, local_err); >> return; >> } >> + arm_cpu_mte_finalize(cpu, &local_err); >> + if (local_err != NULL) { >> + error_propagate(errp, local_err); >> + return; >> + } >> } >> >> if (kvm_enabled()) { >> @@ -1504,7 +1509,7 @@ static void arm_cpu_realizefn(DeviceState *dev, Error **errp) >> } >> if (cpu->tag_memory) { >> error_setg(errp, >> - "Cannot enable KVM when guest CPUs has MTE enabled"); >> + "Cannot enable KVM when guest CPUs has tag memory enabled"); > before this series, tag_memory was used to detect MTE was enabled at > machine level. And this was not compatible with KVM. > > Hasn't it changed now with this series? Sorry I don't know much about > that tag_memory along with the KVM use case? Can you describe it as well > in the cover letter. IIU the current code correctly, the purpose of tag_memory is twofold: - control whether mte should be available in the first place - provide a place where a memory region used by the tcg implemtation can be linked The latter part (extra memory region) is not compatible with kvm. "Presence of extra memory for the implementation" as the knob to configure mte for tcg makes sense, but it didn't seem right to me to use it for kvm while controlling something which is basically a cpu property. >> return; >> } >> } (...) >> +void aarch64_add_mte_properties(Object *obj) >> +{ >> + ARMCPU *cpu = ARM_CPU(obj); >> + >> + /* >> + * For tcg, the machine type may provide tag memory for MTE emulation. > s/machine type/machine? Either, I guess, as only the virt machine type provides tag memory in the first place. >> + * We do not know whether that is the case at this point in time, so >> + * default MTE to on and check later. >> + * This preserves pre-existing behaviour, but is really a bit awkward. >> + */ >> + qdev_property_add_static(DEVICE(obj), &arm_cpu_mte_property); >> + if (kvm_enabled()) { >> + /* >> + * Default MTE to off, as long as migration support is not >> + * yet implemented. >> + * TODO: implement migration support for kvm >> + */ >> + cpu->prop_mte = false; >> + } >> +} >> + >> +void arm_cpu_mte_finalize(ARMCPU *cpu, Error **errp) >> +{ >> + if (!cpu->prop_mte) { >> + /* Disable MTE feature bits. */ >> + cpu->isar.id_aa64pfr1 = >> + FIELD_DP64(cpu->isar.id_aa64pfr1, ID_AA64PFR1, MTE, 0); >> + return; >> + } >> +#ifndef CONFIG_USER_ONLY >> + if (!kvm_enabled()) { >> + if (cpu_isar_feature(aa64_mte, cpu) && !cpu->tag_memory) { >> + /* >> + * Disable the MTE feature bits, unless we have tag-memory >> + * provided by the machine. >> + * This silent downgrade is not really nice if the user had >> + * explicitly requested MTE to be enabled by the cpu, but it >> + * preserves pre-existing behaviour. In an ideal world, we > > > Can't we "simply" prevent the end-user from using the prop_mte option > with a TCG CPU? and have something like > > For TCG, MTE depends on the CPU feature availability + machine tag memory > For KVM, MTE depends on the user opt-in + CPU feature avail (if > relevant) + host VM capability (?) I don't like kvm and tcg cpus behaving too differently... but then, tcg is already different as it needs tag_memory. Thinking about it, maybe we could repurpose tag_memory in the kvm case (e.g. for a temporary buffer for migration purposes) and require it in all cases (making kvm fail if the user specified tag memory, but the host doesn't support it). A cpu feature still looks more natural to me, but I'm not yet quite used to how things are done in arm :) The big elefant in the room is how migration will end up working... after reading the disscussions in https://lore.kernel.org/all/CAJc+Z1FZxSYB_zJit4+0uTR-88VqQL+-01XNMSEfua-dXDy6Wg@mail.gmail.com/ I don't think it will be as "easy" as I thought, and we probably require some further fiddling on the kernel side. > > But maybe I miss the point ... >> + * would fail if MTE was requested, but no tag memory has >> + * been provided. >> + */ >> + cpu->isar.id_aa64pfr1 = >> + FIELD_DP64(cpu->isar.id_aa64pfr1, ID_AA64PFR1, MTE, 0); >> + } >> + if (!cpu_isar_feature(aa64_mte, cpu)) { >> + cpu->prop_mte = false; >> + } >> + return; >> + } >> + if (kvm_arm_mte_supported()) { >> +#ifdef CONFIG_KVM >> + if (kvm_vm_enable_cap(kvm_state, KVM_CAP_ARM_MTE, 0)) { >> + error_setg(errp, "Failed to enable KVM_CAP_ARM_MTE"); >> + } else { >> + /* TODO: add proper migration support with MTE enabled */ >> + if (!mte_migration_blocker) { >> + error_setg(&mte_migration_blocker, >> + "Live migration disabled due to MTE enabled"); >> + if (migrate_add_blocker(mte_migration_blocker, NULL)) { > error_free(mte_migration_blocker); > mte_migration_blocker = NULL; Ah, I missed that, thanks. >> + error_setg(errp, "Failed to add MTE migration blocker"); >> + } >> + } >> + } >> +#endif >> + } >> + /* When HVF provides support for MTE, add it here */ >> +#endif >> +} >> +