From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76737C433FF for ; Fri, 2 Aug 2019 12:32:35 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 33E9021841 for ; Fri, 2 Aug 2019 12:32:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 33E9021841 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:34470 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1htWja-00084R-Fs for qemu-devel@archiver.kernel.org; Fri, 02 Aug 2019 08:32:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37853) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1htWdp-0005a5-Lp for qemu-devel@nongnu.org; Fri, 02 Aug 2019 08:26:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1htWdl-0001yS-MD for qemu-devel@nongnu.org; Fri, 02 Aug 2019 08:26:37 -0400 Received: from mx1.redhat.com ([209.132.183.28]:34894) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1htWdb-0001t9-QZ; Fri, 02 Aug 2019 08:26:24 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0D8643082129; Fri, 2 Aug 2019 12:26:22 +0000 (UTC) Received: from kamzik.brq.redhat.com (unknown [10.43.2.160]) by smtp.corp.redhat.com (Postfix) with ESMTP id A02CD5C205; Fri, 2 Aug 2019 12:26:19 +0000 (UTC) From: Andrew Jones To: qemu-devel@nongnu.org, qemu-arm@nongnu.org Date: Fri, 2 Aug 2019 14:25:33 +0200 Message-Id: <20190802122540.26385-9-drjones@redhat.com> In-Reply-To: <20190802122540.26385-1-drjones@redhat.com> References: <20190802122540.26385-1-drjones@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.42]); Fri, 02 Aug 2019 12:26:22 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 08/15] target/arm/cpu64: max cpu: Introduce sve properties X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, richard.henderson@linaro.org, armbru@redhat.com, eric.auger@redhat.com, imammedo@redhat.com, alex.bennee@linaro.org, Dave.Martin@arm.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Introduce cpu properties to give fine control over SVE vector lengths. We introduce a property for each valid length up to the current maximum supported, which is 2048-bits. The properties are named, e.g. sve128, sve256, sve384, sve512, ..., where the number is the number of bits. See the updates to docs/arm-cpu-features.rst for a description of the semantics and for example uses. Note, as sve-max-vq is still present and we'd like to be able to support qmp_query_cpu_model_expansion with guests launched with e.g. -cpu max,sve-max-vq=3D8 on their command lines, then we do allow sve-max-vq and sve to be provided at the same time, but this is not recommended, and is why sve-max-vq is not mentioned in the document. If sve-max-vq is provided then it enables all lengths smaller than and including the max and disables all lengths larger. It also has the side-effect that no larger lengths may be enabled and that the max itself cannot be disabled. Smaller non-power-of-2 lengths may, however, be disabled, e.g. -cpu max,sve-max-vq=3D4,sve384=3Doff provides a guest the vector lengths 128, 256, and 512 bits. Signed-off-by: Andrew Jones --- docs/arm-cpu-features.rst | 157 +++++++++++++++++- target/arm/cpu.c | 4 + target/arm/cpu.h | 14 ++ target/arm/cpu64.c | 340 +++++++++++++++++++++++++++++++++++++- target/arm/helper.c | 13 +- target/arm/monitor.c | 16 ++ tests/arm-cpu-features.c | 197 ++++++++++++++++++++++ 7 files changed, 726 insertions(+), 15 deletions(-) diff --git a/docs/arm-cpu-features.rst b/docs/arm-cpu-features.rst index c79dcffb5556..aaaed861cda2 100644 --- a/docs/arm-cpu-features.rst +++ b/docs/arm-cpu-features.rst @@ -48,18 +48,28 @@ block in the script for usage) is used to issue the Q= MP commands. (QEMU) query-cpu-model-expansion type=3Dfull model=3D{"name":"max"= } { "return": { "model": { "name": "max", "props": { - "pmu": true, "aarch64": true + "sve1664": true, "pmu": true, "sve1792": true, "sve1920": true, + "sve128": true, "aarch64": true, "sve1024": true, "sve": true, + "sve640": true, "sve768": true, "sve1408": true, "sve256": true, + "sve1152": true, "sve512": true, "sve384": true, "sve1536": true= , + "sve896": true, "sve1280": true, "sve2048": true }}}} =20 -We see that the `max` CPU type has the `pmu` and `aarch64` CPU features. -We also see that the CPU features are enabled, as they are all `true`. +We see that the `max` CPU type has the `pmu`, `aarch64`, `sve`, and many +`sve` CPU features. We also see that all the CPU features are +enabled, as they are all `true`. (The `sve` CPU features are all +optional SVE vector lengths. See "SVE CPU Properties".) =20 (2) Let's try to disable the PMU:: =20 (QEMU) query-cpu-model-expansion type=3Dfull model=3D{"name":"max"= ,"props":{"pmu":false}} { "return": { "model": { "name": "max", "props": { - "pmu": false, "aarch64": true + "sve1664": true, "pmu": false, "sve1792": true, "sve1920": true, + "sve128": true, "aarch64": true, "sve1024": true, "sve": true, + "sve640": true, "sve768": true, "sve1408": true, "sve256": true, + "sve1152": true, "sve512": true, "sve384": true, "sve1536": true= , + "sve896": true, "sve1280": true, "sve2048": true }}}} =20 We see it worked, as `pmu` is now `false`. @@ -75,7 +85,22 @@ We see it worked, as `pmu` is now `false`. It looks like this feature is limited to a configuration we do not currently have. =20 -(4) Let's try probing CPU features for the Cortex-A15 CPU type:: +(4) Let's disable `sve` and see what happens to all the optional SVE + vector lengths:: + + (QEMU) query-cpu-model-expansion type=3Dfull model=3D{"name":"max"= ,"props":{"sve":false}} + { "return": { + "model": { "name": "max", "props": { + "sve1664": false, "pmu": true, "sve1792": false, "sve1920": fals= e, + "sve128": false, "aarch64": true, "sve1024": false, "sve": false= , + "sve640": false, "sve768": false, "sve1408": false, "sve256": fa= lse, + "sve1152": false, "sve512": false, "sve384": false, "sve1536": f= alse, + "sve896": false, "sve1280": false, "sve2048": false + }}}} + +As expected they are now all `false`. + +(5) Let's try probing CPU features for the Cortex-A15 CPU type:: =20 (QEMU) query-cpu-model-expansion type=3Dfull model=3D{"name":"cort= ex-a15"} {"return": {"model": {"name": "cortex-a15", "props": {"pmu": true}= }}} @@ -131,7 +156,125 @@ After determining which CPU features are available = and supported for a given CPU type, then they may be selectively enabled or disabled on the QEMU command line with that CPU type:: =20 - $ qemu-system-aarch64 -M virt -cpu max,pmu=3Doff + $ qemu-system-aarch64 -M virt -cpu max,pmu=3Doff,sve=3Don,sve128=3Don,= sve256=3Don =20 -The example above disables the PMU for the `max` CPU type. +The example above disables the PMU and enables the first two SVE vector +lengths for the `max` CPU type. Note, the `sve=3Don` isn't actually +necessary, because, as we observed above with our probe of the `max` CPU +type, `sve` is already on by default. Also, based on our probe of +defaults, it would seem we need to disable many SVE vector lengths, rath= er +than only enabling the two we want. This isn't the case, because, as +disabling many SVE vector lengths would be quite verbose, the `sve` C= PU +properties have special semantics (see "SVE CPU Property Parsing +Semantics"). + +SVE CPU Properties +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +There are two types of SVE CPU properties: `sve` and `sve`. The firs= t +is used to enable or disable the entire SVE feature, just as the `pmu` +CPU property completely enables or disables the PMU. The second type +is used to enable or disable specific vector lengths, where `N` is the +number of bits of the length. The `sve` CPU properties have special +dependencies and constraints, see "SVE CPU Property Dependencies and +Constraints" below. Additionally, as we want all supported vector lengt= hs +to be enabled by default, then, in order to avoid overly verbose command +lines (command lines full of `sve=3Doff`, for all `N` not wanted), we +provide the parsing semantics listed in "SVE CPU Property Parsing +Semantics". + +SVE CPU Property Dependencies and Constraints +--------------------------------------------- + + 1) At least one vector length must be enabled when `sve` is enabled. + + 2) If a vector length `N` is enabled, then all power-of-2 vector + lengths smaller than `N` must also be enabled. E.g. if `sve512` + is enabled, then `sve128` and `sve256` must also be enabled, + but `sve384` is not required. + +SVE CPU Property Parsing Semantics +---------------------------------- + + 1) If SVE is disabled (`sve=3Doff`), then which SVE vector lengths + are enabled or disabled is irrelevant to the guest, as the entire + SVE feature is disabled and that disables all vector lengths for + the guest. However QEMU will still track any `sve` CPU + properties provided by the user. If later an `sve=3Don` is provide= d, + then the guest will get only the enabled lengths. + + 2) If SVE is enabled (`sve=3Don`), but no `sve` CPU properties are + provided, then all supported vector lengths are enabled. + + 3) If SVE is enabled, then an error is generated when attempting to + disable the last enabled vector length (see constraint (1) of "SVE + CPU Property Dependencies and Constraints"). + + 4) If one or more `sve` CPU properties are set `off`, but no `sve`, + CPU properties are set `on`, then the specified vector lengths are + disabled but the default for any unspecified lengths remains enable= d. + Disabling a power-of-2 vector length also disables all vector lengt= hs + larger than the power-of-2 length (see constraint (2) of "SVE CPU + Property Dependencies and Constraints"). + + 5) If one or more `sve` CPU properties are set to `on`, then they + are enabled and all unspecified lengths default to disabled, except + for the required lengths per constraint (2) of "SVE CPU Property + Dependencies and Constraints", which will even be auto-enabled if + they were not explicitly enabled. + + 6) If SVE was disabled (`sve=3Doff`), allowing all vector lengths to b= e + explicitly disabled (i.e. avoiding the error specified in (3) of + "SVE CPU Property Parsing Semantics"), then if later an `sve=3Don` = is + provided an error will be generated. To avoid this error, one must + enable at least one vector length prior to enabling SVE. + +SVE CPU Property Examples +------------------------- + + 1) Disable SVE:: + + $ qemu-system-aarch64 -M virt -cpu max,sve=3Doff + + 2) Implicitly enable all vector lengths for the `max` CPU type:: + + $ qemu-system-aarch64 -M virt -cpu max + + 3) Only enable the 128-bit vector length:: + + $ qemu-system-aarch64 -M virt -cpu max,sve128=3Don + + 4) Disable the 256-bit vector length and all larger vector lengths + since 256 is a power-of-2 (this results in only the 128-bit length + being enabled):: + + $ qemu-system-aarch64 -M virt -cpu max,sve256=3Doff + + 5) Enable the 128-bit, 256-bit, and 512-bit vector lengths:: + + $ qemu-system-aarch64 -M virt -cpu max,sve128=3Don,sve256=3Don,sve5= 12=3Don + + 6) The same as (5), but since the 128-bit and 256-bit vector + lengths are required for the 512-bit vector length to be enabled, + then allow them to be auto-enabled:: + + $ qemu-system-aarch64 -M virt -cpu max,sve512=3Don + + 7) Do the same as (6), but by first disabling SVE and then re-enabling= it:: + + $ qemu-system-aarch64 -M virt -cpu max,sve=3Doff,sve512=3Don,sve=3D= on + + 8) Force errors regarding the last vector length:: + + $ qemu-system-aarch64 -M virt -cpu max,sve128=3Doff + $ qemu-system-aarch64 -M virt -cpu max,sve=3Doff,sve128=3Doff,sve=3D= on + +SVE CPU Property Recommendations +-------------------------------- + +The examples in "SVE CPU Property Examples" exhibit many ways to select +vector lengths which developers may find useful in order to avoid overly +verbose command lines. However, the recommended way to select vector +lengths is to explicitly enable each desired length. Therefore only +example's (1), (3), and (5) exhibit recommended uses of the properties. =20 diff --git a/target/arm/cpu.c b/target/arm/cpu.c index 930f4579ccae..842637ae0c49 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -1247,6 +1247,10 @@ static void arm_cpu_realizefn(DeviceState *dev, Er= ror **errp) return; } =20 + if (cpu_isar_feature(aa64_sve, cpu)) { + arm_cpu_sve_finalize(cpu); + } + if (arm_feature(env, ARM_FEATURE_AARCH64) && cpu->has_vfp !=3D cpu->has_neon) { /* diff --git a/target/arm/cpu.h b/target/arm/cpu.h index a3300f3ee7d7..c8b96293c5e2 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -184,8 +184,13 @@ typedef struct { =20 #ifdef TARGET_AARCH64 # define ARM_MAX_VQ 16 +void arm_cpu_sve_finalize(ARMCPU *cpu); +uint32_t arm_cpu_vq_map_next_smaller(ARMCPU *cpu, uint32_t vq); #else # define ARM_MAX_VQ 1 +static inline void arm_cpu_sve_finalize(ARMCPU *cpu) { } +static inline uint32_t arm_cpu_vq_map_next_smaller(ARMCPU *cpu, uint32_t= vq) +{ return 0; } #endif =20 typedef struct ARMVectorReg { @@ -915,6 +920,15 @@ struct ARMCPU { =20 /* Used to set the maximum vector length the cpu will support. */ uint32_t sve_max_vq; + + /* + * In sve_vq_map each set bit is a supported vector length of + * (bit-number + 1) * 16 bytes, i.e. each bit number + 1 is the vect= or + * length in quadwords. We need a map size twice the maximum + * quadword length though because we use two bits for each vector + * length in order to track three states: uninitialized, off, and on= . + */ + DECLARE_BITMAP(sve_vq_map, ARM_MAX_VQ * 2); }; =20 void arm_cpu_post_init(Object *obj); diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c index e470c34f89ac..c9412a0b71af 100644 --- a/target/arm/cpu64.c +++ b/target/arm/cpu64.c @@ -257,6 +257,150 @@ static void aarch64_a72_initfn(Object *obj) define_arm_cp_regs(cpu, cortex_a72_a57_a53_cp_reginfo); } =20 +/* + * While we eventually use cpu->sve_vq_map as a typical bitmap, where ea= ch vq + * has only two states (off/on), until we've finalized the map at realiz= e time + * we use an extra bit, at the vq - 1 + ARM_MAX_VQ bit number, to also a= llow + * tracking of the uninitialized state. The arm_vq_state typedef and fol= lowing + * functions allow us to more easily work with the bitmap. + */ +typedef enum arm_vq_state { + ARM_VQ_OFF, + ARM_VQ_ON, + ARM_VQ_UNINITIALIZED, +} arm_vq_state; + +static arm_vq_state arm_cpu_vq_map_get(ARMCPU *cpu, uint32_t vq) +{ + assert(vq <=3D ARM_MAX_VQ); + + return test_bit(vq - 1, cpu->sve_vq_map) | + test_bit(vq - 1 + ARM_MAX_VQ, cpu->sve_vq_map) << 1; +} + +static void arm_cpu_vq_map_set(ARMCPU *cpu, uint32_t vq, arm_vq_state st= ate) +{ + assert(vq <=3D ARM_MAX_VQ); + assert(state =3D=3D ARM_VQ_OFF || state =3D=3D ARM_VQ_ON); + + clear_bit(vq - 1 + ARM_MAX_VQ, cpu->sve_vq_map); + + if (state =3D=3D ARM_VQ_ON) { + set_bit(vq - 1, cpu->sve_vq_map); + } else { + clear_bit(vq - 1, cpu->sve_vq_map); + } +} + +/* + * Uninitialized vector lengths need a default value to use in case we n= eed + * to query their value prior to map finalization. Additionally map fina= lizing + * itself needs to determine what value to assign uninitialized vector l= engths. + * The default is determined as follows: + * + * * When no vector lengths have been explicitly enabled, i.e. either n= o + * input has been provided by the user at all, or vector lengths have + * only been disabled, then all uninitialized vector lengths default = 'ON'. + * + * * When one or more vector lengths have been enabled, then all uninit= ialized + * vector lengths default 'OFF'. Note, when checking for enabled vect= or + * lengths we do not discriminate between user-enabled vector lengths= and + * auto-enabled vector lengths (which were auto-enabled in order to s= atisfy + * the user-enabled vector lengths). This implies the default can nev= er + * transition back to 'ON', even if the user-enabled and auto-enabled= vector + * lengths that initially transitioned it to 'OFF' are later disabled= , as at + * least one vector length must remain enabled unless the SVE feature= is + * completely disabled. If SVE is completely disabled then all vector + * lengths are effectively 'OFF'. + */ +static bool arm_cpu_vq_map_get_default(ARMCPU *cpu) +{ + uint32_t vq; + + for (vq =3D 1; vq <=3D ARM_MAX_VQ; ++vq) { + if (arm_cpu_vq_map_get(cpu, vq) =3D=3D ARM_VQ_ON) { + return false; + } + } + + return true; +} + +/* + * We need to be able to track the number of enabled (or will-be enabled= ) + * vector lengths in order to ensure we never drop to zero. If the defau= lt + * is 'ON', then we count enabled and uninitialized vector lengths. Othe= rwise, + * if the default is 'OFF', then we only count enabled vector lengths. + */ +static int arm_cpu_num_vqs_available(ARMCPU *cpu) +{ + uint32_t vq; + bool defval; + int num =3D 0; + + defval =3D arm_cpu_vq_map_get_default(cpu); + + for (vq =3D 1; vq <=3D ARM_MAX_VQ; ++vq) { + arm_vq_state vq_state =3D arm_cpu_vq_map_get(cpu, vq); + + if (vq_state =3D=3D ARM_VQ_ON) { + ++num; + } else if (defval && vq_state =3D=3D ARM_VQ_UNINITIALIZED) { + ++num; + } + } + + return num; +} + +static void arm_cpu_vq_map_init(ARMCPU *cpu) +{ + /* Set all vq's to 0b10 (ARM_VQ_UNINITIALIZED) */ + bitmap_clear(cpu->sve_vq_map, 0, ARM_MAX_VQ); + bitmap_set(cpu->sve_vq_map, ARM_MAX_VQ, ARM_MAX_VQ); +} + +void arm_cpu_sve_finalize(ARMCPU *cpu) +{ + bool defval =3D arm_cpu_vq_map_get_default(cpu); + uint32_t vq, max_vq; + + for (vq =3D 1; vq <=3D ARM_MAX_VQ; ++vq) { + arm_vq_state vq_state =3D arm_cpu_vq_map_get(cpu, vq); + + if (vq_state =3D=3D ARM_VQ_UNINITIALIZED) { + if (defval) { + arm_cpu_vq_map_set(cpu, vq, ARM_VQ_ON); + } else { + arm_cpu_vq_map_set(cpu, vq, ARM_VQ_OFF); + } + } + } + + max_vq =3D arm_cpu_vq_map_next_smaller(cpu, ARM_MAX_VQ + 1); + + if (!cpu->sve_max_vq) { + cpu->sve_max_vq =3D max_vq; + } + + assert(max_vq && cpu->sve_max_vq =3D=3D max_vq); +} + +uint32_t arm_cpu_vq_map_next_smaller(ARMCPU *cpu, uint32_t vq) +{ + uint32_t bitnum; + + /* + * We allow vq =3D=3D ARM_MAX_VQ + 1 to be input because the caller = may want + * to find the maximum vq enabled, which may be ARM_MAX_VQ, but this + * function always returns the next smaller than the input. + */ + assert(vq <=3D ARM_MAX_VQ + 1); + + bitnum =3D find_last_bit(cpu->sve_vq_map, vq - 1); + return bitnum =3D=3D vq - 1 ? 0 : bitnum + 1; +} + static void cpu_max_get_sve_max_vq(Object *obj, Visitor *v, const char *= name, void *opaque, Error **errp) { @@ -269,15 +413,178 @@ static void cpu_max_set_sve_max_vq(Object *obj, Vi= sitor *v, const char *name, { ARMCPU *cpu =3D ARM_CPU(obj); Error *err =3D NULL; + uint32_t vq; =20 visit_type_uint32(v, name, &cpu->sve_max_vq, &err); + if (err) { + error_propagate(errp, err); + return; + } =20 - if (!err && (cpu->sve_max_vq =3D=3D 0 || cpu->sve_max_vq > ARM_MAX_V= Q)) { - error_setg(&err, "unsupported SVE vector length"); - error_append_hint(&err, "Valid sve-max-vq in range [1-%d]\n", + if (cpu->sve_max_vq =3D=3D 0 || cpu->sve_max_vq > ARM_MAX_VQ) { + error_setg(errp, "unsupported SVE vector length"); + error_append_hint(errp, "Valid sve-max-vq in range [1-%d]\n", ARM_MAX_VQ); + return; + } + + for (vq =3D 1; vq <=3D ARM_MAX_VQ; ++vq) { + if (vq <=3D cpu->sve_max_vq) { + char tmp[8]; + + sprintf(tmp, "sve%d", vq * 128); + object_property_set_bool(obj, true, tmp, &err); + if (err) { + error_propagate(errp, err); + return; + } + } else if (arm_cpu_vq_map_get(cpu, vq) =3D=3D ARM_VQ_ON) { + arm_cpu_vq_map_set(cpu, vq, ARM_VQ_OFF); + } + } +} + +static void cpu_arm_get_sve_vq(Object *obj, Visitor *v, const char *name= , + void *opaque, Error **errp) +{ + ARMCPU *cpu =3D ARM_CPU(obj); + uint32_t vq =3D atoi(&name[3]) / 128; + arm_vq_state vq_state =3D arm_cpu_vq_map_get(cpu, vq); + bool value; + + if (!cpu_isar_feature(aa64_sve, cpu)) { + /* All vector lengths are disabled when SVE is off. */ + value =3D false; + } else if (vq_state =3D=3D ARM_VQ_ON) { + value =3D true; + } else if (vq_state =3D=3D ARM_VQ_OFF) { + value =3D false; + } else if (arm_cpu_vq_map_get_default(cpu)) { + value =3D true; + } else { + value =3D false; + } + + visit_type_bool(v, name, &value, errp); +} + +static void cpu_arm_set_sve_vq(Object *obj, Visitor *v, const char *name= , + void *opaque, Error **errp) +{ + ARMCPU *cpu =3D ARM_CPU(obj); + uint32_t vq =3D atoi(&name[3]) / 128; + uint32_t max_vq =3D 0; + Error *err =3D NULL; + bool value; + + visit_type_bool(v, name, &value, &err); + if (err) { + error_propagate(errp, err); + return; + } + + /* + * We need to know the maximum vector length, which may just current= ly + * be the maximum length, in order to validate the enabling/disablin= g + * of this vector length. + */ + if (!cpu->sve_max_vq) { + for (max_vq =3D ARM_MAX_VQ; max_vq >=3D 1; --max_vq) { + if (arm_cpu_vq_map_get(cpu, max_vq) =3D=3D ARM_VQ_ON) { + break; + } + } + } + + if (cpu->sve_max_vq && value && vq > cpu->sve_max_vq) { + error_setg(errp, "cannot enable %s", name); + error_append_hint(errp, "vq=3D%d (%d bits) is larger than the " + "maximum vector length, sve-max-vq=3D%d " + "(%d bits)\n", vq, vq * 128, + cpu->sve_max_vq, cpu->sve_max_vq * 128); + } else if (cpu->sve_max_vq && !value && vq =3D=3D cpu->sve_max_vq) { + error_setg(errp, "cannot disable %s", name); + error_append_hint(errp, "The maximum vector length must be " + "enabled, sve-max-vq=3D%d (%d bits)\n", + cpu->sve_max_vq, cpu->sve_max_vq * 128); + } else if (cpu->sve_max_vq && !value && vq < cpu->sve_max_vq && + is_power_of_2(vq)) { + error_setg(errp, "cannot disable %s", name); + error_append_hint(errp, "vq=3D%d (%d bits) is required as it is = a " + "power-of-2 length smaller than the maximum, " + "sve-max-vq=3D%d (%d bits)\n", vq, vq * 128, + cpu->sve_max_vq, cpu->sve_max_vq * 128); + } else if (max_vq && !value && vq < max_vq && is_power_of_2(vq)) { + error_setg(errp, "cannot disable %s", name); + error_append_hint(errp, "Vector length %d-bits is required as it= " + "is a power-of-2 length smaller than another " + "enabled vector length. Disable all larger vec= tor " + "lengths first.\n", vq * 128); + } else { + uint32_t s; + + if (value) { + arm_vq_state vq_state; + bool fail =3D false; + + /* + * Enabling a vector length automatically enables all + * uninitialized power-of-2 lengths smaller than it, as + * per the architecture. + */ + for (s =3D 1; s < vq; ++s) { + if (is_power_of_2(s)) { + vq_state =3D arm_cpu_vq_map_get(cpu, s); + if (vq_state =3D=3D ARM_VQ_UNINITIALIZED) { + arm_cpu_vq_map_set(cpu, s, ARM_VQ_ON); + } else if (vq_state =3D=3D ARM_VQ_OFF) { + fail =3D true; + break; + } + } + } + + if (fail) { + error_setg(errp, "cannot enable %s", name); + error_append_hint(errp, "Vector length %d-bits is disabl= ed " + "and is a power-of-2 length smaller th= an " + "%s. All power-of-2 vector lengths sma= ller " + "than the maximum length are required.= \n", + s * 128, name); + } else { + arm_cpu_vq_map_set(cpu, vq, ARM_VQ_ON); + } + } else { + /* + * We would have errored-out already if we were attempting t= o + * disable a power-of-2 vector length less than another enab= led + * vector length, but there may be uninitialized vector leng= ths + * larger than a power-of-2 vector length that we're disabli= ng. + * We disable all of those lengths now too, as they can no l= onger + * be enabled. + */ + if (is_power_of_2(vq)) { + for (s =3D vq + 1; s <=3D ARM_MAX_VQ; ++s) { + arm_cpu_vq_map_set(cpu, s, ARM_VQ_OFF); + } + } + + arm_cpu_vq_map_set(cpu, vq, ARM_VQ_OFF); + + /* + * We just disabled one or more vector lengths. We need to m= ake + * sure we didn't disable them all when SVE is enabled. + */ + if (cpu_isar_feature(aa64_sve, cpu) && + !arm_cpu_num_vqs_available(cpu)) { + error_setg(errp, "cannot disable %s", name); + error_append_hint(errp, "Disabling %s results in all vec= tor " + "lengths being disabled.\n", name); + error_append_hint(errp, "With SVE enabled, at least one = vector " + "length must be enabled.\n"); + } + } } - error_propagate(errp, err); } =20 static void cpu_arm_get_sve(Object *obj, Visitor *v, const char *name, @@ -306,6 +613,16 @@ static void cpu_arm_set_sve(Object *obj, Visitor *v,= const char *name, t =3D cpu->isar.id_aa64pfr0; t =3D FIELD_DP64(t, ID_AA64PFR0, SVE, value); cpu->isar.id_aa64pfr0 =3D t; + + /* + * When SVE is enabled ensure that we have at least one vector + * length available. + */ + if (cpu_isar_feature(aa64_sve, cpu) && !arm_cpu_num_vqs_available(cp= u)) { + error_setg(errp, "cannot enable SVE"); + error_append_hint(errp, "All possible SVE vector lengths have " + "been disabled.\n"); + } } =20 /* -cpu max: if KVM is enabled, like -cpu host (best possible with this = host); @@ -316,6 +633,7 @@ static void cpu_arm_set_sve(Object *obj, Visitor *v, = const char *name, static void aarch64_max_initfn(Object *obj) { ARMCPU *cpu =3D ARM_CPU(obj); + uint32_t vq; =20 if (kvm_enabled()) { kvm_arm_set_cpu_features_from_host(cpu); @@ -400,11 +718,23 @@ static void aarch64_max_initfn(Object *obj) cpu->dcz_blocksize =3D 7; /* 512 bytes */ #endif =20 - cpu->sve_max_vq =3D ARM_MAX_VQ; object_property_add(obj, "sve-max-vq", "uint32", cpu_max_get_sve= _max_vq, cpu_max_set_sve_max_vq, NULL, NULL, &error_f= atal); object_property_add(obj, "sve", "bool", cpu_arm_get_sve, cpu_arm_set_sve, NULL, NULL, &error_fatal); + + /* + * sve_vq_map uses a special state while setting properties, so + * we initialize it here with its init function and finalize it + * in arm_cpu_realizefn(). + */ + arm_cpu_vq_map_init(cpu); + for (vq =3D 1; vq <=3D ARM_MAX_VQ; ++vq) { + char name[8]; + sprintf(name, "sve%d", vq * 128); + object_property_add(obj, name, "bool", cpu_arm_get_sve_vq, + cpu_arm_set_sve_vq, NULL, NULL, &error_f= atal); + } } } =20 diff --git a/target/arm/helper.c b/target/arm/helper.c index 3064067c69a6..5d38f92eebef 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -5272,6 +5272,13 @@ int sve_exception_el(CPUARMState *env, int el) return 0; } =20 +static uint32_t sve_zcr_get_valid_len(ARMCPU *cpu, uint32_t start_len) +{ + uint32_t start_vq =3D (start_len & 0xf) + 1; + + return arm_cpu_vq_map_next_smaller(cpu, start_vq + 1) - 1; +} + /* * Given that SVE is enabled, return the vector length for EL. */ @@ -5281,13 +5288,13 @@ uint32_t sve_zcr_len_for_el(CPUARMState *env, int= el) uint32_t zcr_len =3D cpu->sve_max_vq - 1; =20 if (el <=3D 1) { - zcr_len =3D MIN(zcr_len, 0xf & (uint32_t)env->vfp.zcr_el[1]); + zcr_len =3D sve_zcr_get_valid_len(cpu, env->vfp.zcr_el[1]); } if (el <=3D 2 && arm_feature(env, ARM_FEATURE_EL2)) { - zcr_len =3D MIN(zcr_len, 0xf & (uint32_t)env->vfp.zcr_el[2]); + zcr_len =3D sve_zcr_get_valid_len(cpu, env->vfp.zcr_el[2]); } if (arm_feature(env, ARM_FEATURE_EL3)) { - zcr_len =3D MIN(zcr_len, 0xf & (uint32_t)env->vfp.zcr_el[3]); + zcr_len =3D sve_zcr_get_valid_len(cpu, env->vfp.zcr_el[3]); } return zcr_len; } diff --git a/target/arm/monitor.c b/target/arm/monitor.c index 36a86cb83ce5..f7099917e137 100644 --- a/target/arm/monitor.c +++ b/target/arm/monitor.c @@ -90,8 +90,24 @@ GICCapabilityList *qmp_query_gic_capabilities(Error **= errp) return head; } =20 +QEMU_BUILD_BUG_ON(ARM_MAX_VQ > 16); + +/* + * These are cpu model features we want to advertise. The order here + * matters as this is the order in which qmp_query_cpu_model_expansion + * will attempt to set them. If there are dependencies between features, + * as there are with the sve features, then the order that + * considers those dependencies must be used. + * + * The sve features need to be in reverse order in order to + * enable/disable the largest vector lengths first, ensuring all + * power-of-2 vector lengths smaller can also be enabled/disabled. + */ static const char *cpu_model_advertised_features[] =3D { "aarch64", "pmu", "sve", + "sve2048", "sve1920", "sve1792", "sve1664", "sve1536", "sve1408", + "sve1280", "sve1152", "sve1024", "sve896", "sve768", "sve640", + "sve512", "sve384", "sve256", "sve128", NULL }; =20 diff --git a/tests/arm-cpu-features.c b/tests/arm-cpu-features.c index 202bc0e3e823..a0752d000c08 100644 --- a/tests/arm-cpu-features.c +++ b/tests/arm-cpu-features.c @@ -13,6 +13,18 @@ #include "qapi/qmp/qdict.h" #include "qapi/qmp/qjson.h" =20 +#if __SIZEOF_LONG__ =3D=3D 8 +#define BIT(n) (1UL << (n)) +#else +#define BIT(n) (1ULL << (n)) +#endif + +/* + * We expect the SVE max-vq to be 16. Also it must be <=3D 64 + * for our test code, otherwise 'vls' can't just be a uint64_t. + */ +#define SVE_MAX_VQ 16 + #define MACHINE "-machine virt,gic-version=3Dmax " #define QUERY_HEAD "{ 'execute': 'query-cpu-model-expansion', " \ "'arguments': { 'type': 'full', " @@ -157,6 +169,181 @@ static void assert_bad_props(QTestState *qts, const= char *cpu_type) qobject_unref(resp); } =20 +static uint64_t resp_get_sve_vls(QDict *resp) +{ + QDict *props; + const QDictEntry *e; + uint64_t vls =3D 0; + int n =3D 0; + + g_assert(resp); + g_assert(resp_has_props(resp)); + + props =3D resp_get_props(resp); + + for (e =3D qdict_first(props); e; e =3D qdict_next(props, e)) { + if (strlen(e->key) > 3 && !strncmp(e->key, "sve", 3) && + g_ascii_isdigit(e->key[3])) { + char *endptr; + int bits; + + bits =3D g_ascii_strtoll(&e->key[3], &endptr, 10); + if (!bits || *endptr !=3D '\0') { + continue; + } + + if (qdict_get_bool(props, e->key)) { + vls |=3D BIT((bits / 128) - 1); + } + ++n; + } + } + + g_assert(n =3D=3D SVE_MAX_VQ); + + return vls; +} + +#define assert_sve_vls(qts, cpu_type, expected_vls, fmt, ...) \ +({ \ + QDict *_resp =3D do_query(qts, cpu_type, fmt, ##__VA_ARGS__); = \ + g_assert(_resp); \ + g_assert(resp_has_props(_resp)); \ + g_assert(resp_get_sve_vls(_resp) =3D=3D expected_vls); = \ + qobject_unref(_resp); \ +}) + +static void sve_tests_default(QTestState *qts, const char *cpu_type) +{ + /* + * With no sve-max-vq or sve properties on the command line + * the default is to have all vector lengths enabled. This also + * tests that 'sve' is 'on' by default. + */ + assert_sve_vls(qts, cpu_type, BIT(SVE_MAX_VQ) - 1, NULL); + + /* With SVE off, all vector lengths should also be off. */ + assert_sve_vls(qts, cpu_type, 0, "{ 'sve': false }"); + + /* With SVE on, we must have at least one vector length enabled. */ + assert_error(qts, cpu_type, "cannot disable sve128", "{ 'sve128': fa= lse }"); + + /* + * -----------------------------------------------------------------= -- + * power-of-2(vq) all-power- can can + * of-2(< vq) enable disab= le + * -----------------------------------------------------------------= -- + * vq < max_vq no MUST* yes yes + * vq < max_vq yes MUST* yes no + * -----------------------------------------------------------------= -- + * vq =3D=3D max_vq n/a MUST* yes** = yes** + * -----------------------------------------------------------------= -- + * vq > max_vq n/a no no yes + * vq > max_vq n/a yes yes yes + * -----------------------------------------------------------------= -- + * + * [*] "MUST" means this requirement must already be satisfied, + * otherwise 'max_vq' couldn't itself be enabled. + * + * [**] Not testable with the QMP interface, only with the command l= ine. + */ + + /* max_vq :=3D 8 */ + assert_sve_vls(qts, cpu_type, 0x8b, "{ 'sve1024': true }"); + + /* max_vq :=3D 8, vq < max_vq, !power-of-2(vq) */ + assert_sve_vls(qts, cpu_type, 0x8f, + "{ 'sve1024': true, 'sve384': true }"); + assert_sve_vls(qts, cpu_type, 0x8b, + "{ 'sve1024': true, 'sve384': false }"); + + /* max_vq :=3D 8, vq < max_vq, power-of-2(vq) */ + assert_sve_vls(qts, cpu_type, 0x8b, + "{ 'sve1024': true, 'sve256': true }"); + assert_error(qts, cpu_type, "cannot disable sve256", + "{ 'sve1024': true, 'sve256': false }"); + + /* + * max_vq :=3D 3, vq > max_vq, !all-power-of-2(< vq) + * + * If given sve384=3Don,sve512=3Doff,sve640=3Don the command line er= ror would be + * "cannot enable sve640", but QMP visits the vector lengths in reve= rse + * order, so we get "cannot disable sve512" instead. The command lin= e would + * also give that error if given sve384=3Don,sve640=3Don,sve512=3Dof= f, so this is + * all fine. The important thing is that we get an error. + */ + assert_error(qts, cpu_type, "cannot disable sve512", + "{ 'sve384': true, 'sve512': false, 'sve640': true }"); + + /* + * We can disable power-of-2 vector lengths when all larger lengths + * are also disabled. We only need to disable the power-of-2 length, + * as all non-enabled larger lengths will then be auto-disabled. + */ + assert_sve_vls(qts, cpu_type, 0x7, "{ 'sve512': false }"); + + /* max_vq :=3D 3, vq > max_vq, all-power-of-2(< vq) */ + assert_sve_vls(qts, cpu_type, 0x1f, + "{ 'sve384': true, 'sve512': true, 'sve640': true }")= ; + assert_sve_vls(qts, cpu_type, 0xf, + "{ 'sve384': true, 'sve512': true, 'sve640': false }"= ); +} + +static void sve_tests_sve_max_vq_8(const void *data) +{ + QTestState *qts; + + qts =3D qtest_init(MACHINE "-cpu max,sve-max-vq=3D8"); + + assert_sve_vls(qts, "max", BIT(8) - 1, NULL); + + /* + * Disabling the max-vq set by sve-max-vq is not allowed, but + * of course enabling it is OK. + */ + assert_error(qts, "max", "cannot disable sve1024", "{ 'sve1024': fal= se }"); + assert_sve_vls(qts, "max", 0xff, "{ 'sve1024': true }"); + + /* + * Enabling anything larger than max-vq set by sve-max-vq is not + * allowed, but of course disabling everything larger is OK. + */ + assert_error(qts, "max", "cannot enable sve1152", "{ 'sve1152': true= }"); + assert_sve_vls(qts, "max", 0xff, "{ 'sve1152': false }"); + + /* + * We can disable non power-of-2 lengths smaller than the max-vq + * set by sve-max-vq, but not power-of-2 lengths. + */ + assert_sve_vls(qts, "max", 0xfb, "{ 'sve384': false }"); + assert_error(qts, "max", "cannot disable sve256", "{ 'sve256': false= }"); + + qtest_quit(qts); +} + +static void sve_tests_sve_off(const void *data) +{ + QTestState *qts; + + qts =3D qtest_init(MACHINE "-cpu max,sve=3Doff"); + + /* SVE is off, so the map should be empty. */ + assert_sve_vls(qts, "max", 0, NULL); + + /* The map stays empty even if we turn lengths on or off. */ + assert_sve_vls(qts, "max", 0, "{ 'sve128': true }"); + assert_sve_vls(qts, "max", 0, "{ 'sve128': false }"); + + /* With SVE re-enabled we should get all vector lengths enabled. */ + assert_sve_vls(qts, "max", BIT(SVE_MAX_VQ) - 1, "{ 'sve': true }"); + + /* Or enable SVE with just specific vector lengths. */ + assert_sve_vls(qts, "max", 0x3, + "{ 'sve': true, 'sve128': true, 'sve256': true }"); + + qtest_quit(qts); +} + static void test_query_cpu_model_expansion(const void *data) { QTestState *qts; @@ -180,9 +367,12 @@ static void test_query_cpu_model_expansion(const voi= d *data) if (g_str_equal(qtest_get_arch(), "aarch64")) { assert_has_feature(qts, "max", "aarch64"); assert_has_feature(qts, "max", "sve"); + assert_has_feature(qts, "max", "sve128"); assert_has_feature(qts, "cortex-a57", "pmu"); assert_has_feature(qts, "cortex-a57", "aarch64"); =20 + sve_tests_default(qts, "max"); + /* Test that features that depend on KVM generate errors without= . */ assert_error(qts, "max", "'aarch64' feature cannot be disabled " @@ -234,6 +424,13 @@ int main(int argc, char **argv) qtest_add_data_func("/arm/query-cpu-model-expansion", NULL, test_query_cpu_model_expansion); =20 + if (g_str_equal(qtest_get_arch(), "aarch64")) { + qtest_add_data_func("/arm/max/query-cpu-model-expansion/sve-max-= vq-8", + NULL, sve_tests_sve_max_vq_8); + qtest_add_data_func("/arm/max/query-cpu-model-expansion/sve-off"= , + NULL, sve_tests_sve_off); + } + if (kvm_available) { qtest_add_data_func("/arm/kvm/query-cpu-model-expansion", NULL, test_query_cpu_model_expansion_kvm); --=20 2.20.1