* [kvm-unit-tests PATCH v2 0/5] arm64: Change the default QEMU CPU type to "max"
@ 2025-03-14 15:49 Jean-Philippe Brucker
2025-03-14 15:49 ` [kvm-unit-tests PATCH v2 1/5] configure: arm64: Don't display 'aarch64' as the default architecture Jean-Philippe Brucker
` (4 more replies)
0 siblings, 5 replies; 26+ messages in thread
From: Jean-Philippe Brucker @ 2025-03-14 15:49 UTC (permalink / raw)
To: andrew.jones, alexandru.elisei
Cc: eric.auger, kvmarm, kvm, kvm-riscv, vladimir.murzin,
Jean-Philippe Brucker
This is v2 of the series that cleans up the configure flags and sets the
default CPU type to "max" on arm64. Since v1 [1] (sent by Alexandru) we
addressed the review comments and distinguished "--processor" from the new
"--qemu-cpu", to configure build and run flags respectively.
The default TCG CPU on arm64 was the ancient Cortex-A57. With this
change, we can test the latest features of arm64 QEMU, and for example
run Vladimir's MTE test [2].
[1] https://lore.kernel.org/all/20250110135848.35465-1-alexandru.elisei@arm.com/
[2] https://lore.kernel.org/all/20250227152240.118721-1-vladimir.murzin@arm.com/
Alexandru Elisei (3):
configure: arm64: Don't display 'aarch64' as the default architecture
configure: arm/arm64: Display the correct default processor
arm64: Implement the ./configure --processor option
Jean-Philippe Brucker (2):
configure: Add --qemu-cpu option
arm64: Use -cpu max as the default for TCG
scripts/mkstandalone.sh | 2 +-
arm/run | 17 +++++++++++------
riscv/run | 8 ++++----
configure | 42 +++++++++++++++++++++++++++++++----------
arm/Makefile.arm | 1 -
arm/Makefile.common | 1 +
6 files changed, 49 insertions(+), 22 deletions(-)
base-commit: 0cc3a351b925928827baa4b69cf0e46ff5837083
--
2.48.1
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply [flat|nested] 26+ messages in thread* [kvm-unit-tests PATCH v2 1/5] configure: arm64: Don't display 'aarch64' as the default architecture 2025-03-14 15:49 [kvm-unit-tests PATCH v2 0/5] arm64: Change the default QEMU CPU type to "max" Jean-Philippe Brucker @ 2025-03-14 15:49 ` Jean-Philippe Brucker 2025-03-17 8:19 ` Eric Auger 2025-03-22 11:04 ` Andrew Jones 2025-03-14 15:49 ` [kvm-unit-tests PATCH v2 2/5] configure: arm/arm64: Display the correct default processor Jean-Philippe Brucker ` (3 subsequent siblings) 4 siblings, 2 replies; 26+ messages in thread From: Jean-Philippe Brucker @ 2025-03-14 15:49 UTC (permalink / raw) To: andrew.jones, alexandru.elisei Cc: eric.auger, kvmarm, kvm, kvm-riscv, vladimir.murzin From: Alexandru Elisei <alexandru.elisei@arm.com> --arch=aarch64, intentional or not, has been supported since the initial arm64 support, commit 39ac3f8494be ("arm64: initial drop"). However, "aarch64" does not show up in the list of supported architectures, but it's displayed as the default architecture if doing ./configure --help on an arm64 machine. Keep everything consistent and make sure that the default value for $arch is "arm64", but still allow --arch=aarch64, in case they are users that use this configuration for kvm-unit-tests. The help text for --arch changes from: --arch=ARCH architecture to compile for (aarch64). ARCH can be one of: arm, arm64, i386, ppc64, riscv32, riscv64, s390x, x86_64 to: --arch=ARCH architecture to compile for (arm64). ARCH can be one of: arm, arm64, i386, ppc64, riscv32, riscv64, s390x, x86_64 Signed-off-by: Alexandru Elisei <alexandru.elisei@arm.com> --- configure | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/configure b/configure index 06532a89..dc3413fc 100755 --- a/configure +++ b/configure @@ -15,8 +15,9 @@ objdump=objdump readelf=readelf ar=ar addr2line=addr2line -arch=$(uname -m | sed -e 's/i.86/i386/;s/arm64/aarch64/;s/arm.*/arm/;s/ppc64.*/ppc64/') -host=$arch +host=$(uname -m | sed -e 's/i.86/i386/;s/arm64/aarch64/;s/arm.*/arm/;s/ppc64.*/ppc64/') +arch=$host +[ "$arch" = "aarch64" ] && arch="arm64" cross_prefix= endian="" pretty_print_stacks=yes -- 2.48.1 -- kvm-riscv mailing list kvm-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kvm-riscv ^ permalink raw reply related [flat|nested] 26+ messages in thread
* Re: [kvm-unit-tests PATCH v2 1/5] configure: arm64: Don't display 'aarch64' as the default architecture 2025-03-14 15:49 ` [kvm-unit-tests PATCH v2 1/5] configure: arm64: Don't display 'aarch64' as the default architecture Jean-Philippe Brucker @ 2025-03-17 8:19 ` Eric Auger 2025-03-17 8:27 ` Eric Auger 2025-03-22 11:04 ` Andrew Jones 1 sibling, 1 reply; 26+ messages in thread From: Eric Auger @ 2025-03-17 8:19 UTC (permalink / raw) To: Jean-Philippe Brucker, andrew.jones, alexandru.elisei Cc: kvmarm, kvm, kvm-riscv, vladimir.murzin Hi Jean-Philippe, On 3/14/25 4:49 PM, Jean-Philippe Brucker wrote: > From: Alexandru Elisei <alexandru.elisei@arm.com> > > --arch=aarch64, intentional or not, has been supported since the initial > arm64 support, commit 39ac3f8494be ("arm64: initial drop"). However, > "aarch64" does not show up in the list of supported architectures, but > it's displayed as the default architecture if doing ./configure --help > on an arm64 machine. > > Keep everything consistent and make sure that the default value for > $arch is "arm64", but still allow --arch=aarch64, in case they are users there > that use this configuration for kvm-unit-tests. > > The help text for --arch changes from: > > --arch=ARCH architecture to compile for (aarch64). ARCH can be one of: > arm, arm64, i386, ppc64, riscv32, riscv64, s390x, x86_64 > > to: > > --arch=ARCH architecture to compile for (arm64). ARCH can be one of: > arm, arm64, i386, ppc64, riscv32, riscv64, s390x, x86_64 > > Signed-off-by: Alexandru Elisei <alexandru.elisei@arm.com> > --- > configure | 5 +++-- > 1 file changed, 3 insertions(+), 2 deletions(-) > > diff --git a/configure b/configure > index 06532a89..dc3413fc 100755 > --- a/configure > +++ b/configure > @@ -15,8 +15,9 @@ objdump=objdump > readelf=readelf > ar=ar > addr2line=addr2line > -arch=$(uname -m | sed -e 's/i.86/i386/;s/arm64/aarch64/;s/arm.*/arm/;s/ppc64.*/ppc64/') > -host=$arch > +host=$(uname -m | sed -e 's/i.86/i386/;s/arm64/aarch64/;s/arm.*/arm/;s/ppc64.*/ppc64/') > +arch=$host > +[ "$arch" = "aarch64" ] && arch="arm64" Looks the same it done again below arch_name=$arch [ "$arch" = "aarch64" ] && arch="arm64" [ "$arch_name" = "arm64" ] && arch_name="aarch64" <--- arch_libdir=$arch maybe we could move all arch settings at the same location so that it becomes clearer? Thanks Eric > cross_prefix= > endian="" > pretty_print_stacks=yes -- kvm-riscv mailing list kvm-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kvm-riscv ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [kvm-unit-tests PATCH v2 1/5] configure: arm64: Don't display 'aarch64' as the default architecture 2025-03-17 8:19 ` Eric Auger @ 2025-03-17 8:27 ` Eric Auger 2025-03-24 15:48 ` Jean-Philippe Brucker 0 siblings, 1 reply; 26+ messages in thread From: Eric Auger @ 2025-03-17 8:27 UTC (permalink / raw) To: Jean-Philippe Brucker, andrew.jones, alexandru.elisei Cc: kvmarm, kvm, kvm-riscv, vladimir.murzin On 3/17/25 9:19 AM, Eric Auger wrote: > Hi Jean-Philippe, > > > On 3/14/25 4:49 PM, Jean-Philippe Brucker wrote: >> From: Alexandru Elisei <alexandru.elisei@arm.com> >> >> --arch=aarch64, intentional or not, has been supported since the initial >> arm64 support, commit 39ac3f8494be ("arm64: initial drop"). However, >> "aarch64" does not show up in the list of supported architectures, but >> it's displayed as the default architecture if doing ./configure --help >> on an arm64 machine. >> >> Keep everything consistent and make sure that the default value for >> $arch is "arm64", but still allow --arch=aarch64, in case they are users > there >> that use this configuration for kvm-unit-tests. >> >> The help text for --arch changes from: >> >> --arch=ARCH architecture to compile for (aarch64). ARCH can be one of: >> arm, arm64, i386, ppc64, riscv32, riscv64, s390x, x86_64 >> >> to: >> >> --arch=ARCH architecture to compile for (arm64). ARCH can be one of: >> arm, arm64, i386, ppc64, riscv32, riscv64, s390x, x86_64 >> >> Signed-off-by: Alexandru Elisei <alexandru.elisei@arm.com> >> --- >> configure | 5 +++-- >> 1 file changed, 3 insertions(+), 2 deletions(-) >> >> diff --git a/configure b/configure >> index 06532a89..dc3413fc 100755 >> --- a/configure >> +++ b/configure >> @@ -15,8 +15,9 @@ objdump=objdump >> readelf=readelf >> ar=ar >> addr2line=addr2line >> -arch=$(uname -m | sed -e 's/i.86/i386/;s/arm64/aarch64/;s/arm.*/arm/;s/ppc64.*/ppc64/') >> -host=$arch >> +host=$(uname -m | sed -e 's/i.86/i386/;s/arm64/aarch64/;s/arm.*/arm/;s/ppc64.*/ppc64/') >> +arch=$host >> +[ "$arch" = "aarch64" ] && arch="arm64" > Looks the same it done again below Ignore this. This is done again after explicit arch setting :-/ Need another coffee So looks good to me Reviewed-by: Eric Auger <eric.auger@redhat.com> Eric > > arch_name=$arch > [ "$arch" = "aarch64" ] && arch="arm64" > [ "$arch_name" = "arm64" ] && arch_name="aarch64" <--- > arch_libdir=$arch > > maybe we could move all arch settings at the same location so that it > becomes clearer? > > Thanks > > Eric > >> cross_prefix= >> endian="" >> pretty_print_stacks=yes > -- kvm-riscv mailing list kvm-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kvm-riscv ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [kvm-unit-tests PATCH v2 1/5] configure: arm64: Don't display 'aarch64' as the default architecture 2025-03-17 8:27 ` Eric Auger @ 2025-03-24 15:48 ` Jean-Philippe Brucker 0 siblings, 0 replies; 26+ messages in thread From: Jean-Philippe Brucker @ 2025-03-24 15:48 UTC (permalink / raw) To: eric.auger Cc: andrew.jones, alexandru.elisei, kvmarm, kvm, kvm-riscv, vladimir.murzin On Mon, Mar 17, 2025 at 09:27:32AM +0100, Eric Auger wrote: > > > On 3/17/25 9:19 AM, Eric Auger wrote: > > Hi Jean-Philippe, > > > > > > On 3/14/25 4:49 PM, Jean-Philippe Brucker wrote: > >> From: Alexandru Elisei <alexandru.elisei@arm.com> > >> > >> --arch=aarch64, intentional or not, has been supported since the initial > >> arm64 support, commit 39ac3f8494be ("arm64: initial drop"). However, > >> "aarch64" does not show up in the list of supported architectures, but > >> it's displayed as the default architecture if doing ./configure --help > >> on an arm64 machine. > >> > >> Keep everything consistent and make sure that the default value for > >> $arch is "arm64", but still allow --arch=aarch64, in case they are users > > there > >> that use this configuration for kvm-unit-tests. > >> > >> The help text for --arch changes from: > >> > >> --arch=ARCH architecture to compile for (aarch64). ARCH can be one of: > >> arm, arm64, i386, ppc64, riscv32, riscv64, s390x, x86_64 > >> > >> to: > >> > >> --arch=ARCH architecture to compile for (arm64). ARCH can be one of: > >> arm, arm64, i386, ppc64, riscv32, riscv64, s390x, x86_64 > >> > >> Signed-off-by: Alexandru Elisei <alexandru.elisei@arm.com> > >> --- > >> configure | 5 +++-- > >> 1 file changed, 3 insertions(+), 2 deletions(-) > >> > >> diff --git a/configure b/configure > >> index 06532a89..dc3413fc 100755 > >> --- a/configure > >> +++ b/configure > >> @@ -15,8 +15,9 @@ objdump=objdump > >> readelf=readelf > >> ar=ar > >> addr2line=addr2line > >> -arch=$(uname -m | sed -e 's/i.86/i386/;s/arm64/aarch64/;s/arm.*/arm/;s/ppc64.*/ppc64/') > >> -host=$arch > >> +host=$(uname -m | sed -e 's/i.86/i386/;s/arm64/aarch64/;s/arm.*/arm/;s/ppc64.*/ppc64/') > >> +arch=$host > >> +[ "$arch" = "aarch64" ] && arch="arm64" > > Looks the same it done again below > > Ignore this. This is done again after explicit arch setting :-/ Need > another coffee > > So looks good to me > Reviewed-by: Eric Auger <eric.auger@redhat.com> Thanks for the review! Jean -- kvm-riscv mailing list kvm-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kvm-riscv ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [kvm-unit-tests PATCH v2 1/5] configure: arm64: Don't display 'aarch64' as the default architecture 2025-03-14 15:49 ` [kvm-unit-tests PATCH v2 1/5] configure: arm64: Don't display 'aarch64' as the default architecture Jean-Philippe Brucker 2025-03-17 8:19 ` Eric Auger @ 2025-03-22 11:04 ` Andrew Jones 1 sibling, 0 replies; 26+ messages in thread From: Andrew Jones @ 2025-03-22 11:04 UTC (permalink / raw) To: Jean-Philippe Brucker Cc: alexandru.elisei, eric.auger, kvmarm, kvm, kvm-riscv, vladimir.murzin On Fri, Mar 14, 2025 at 03:49:01PM +0000, Jean-Philippe Brucker wrote: > From: Alexandru Elisei <alexandru.elisei@arm.com> > > --arch=aarch64, intentional or not, has been supported since the initial > arm64 support, commit 39ac3f8494be ("arm64: initial drop"). However, > "aarch64" does not show up in the list of supported architectures, but > it's displayed as the default architecture if doing ./configure --help > on an arm64 machine. > > Keep everything consistent and make sure that the default value for > $arch is "arm64", but still allow --arch=aarch64, in case they are users > that use this configuration for kvm-unit-tests. > > The help text for --arch changes from: > > --arch=ARCH architecture to compile for (aarch64). ARCH can be one of: > arm, arm64, i386, ppc64, riscv32, riscv64, s390x, x86_64 > > to: > > --arch=ARCH architecture to compile for (arm64). ARCH can be one of: > arm, arm64, i386, ppc64, riscv32, riscv64, s390x, x86_64 > > Signed-off-by: Alexandru Elisei <alexandru.elisei@arm.com> > --- > configure | 5 +++-- > 1 file changed, 3 insertions(+), 2 deletions(-) > > diff --git a/configure b/configure > index 06532a89..dc3413fc 100755 > --- a/configure > +++ b/configure > @@ -15,8 +15,9 @@ objdump=objdump > readelf=readelf > ar=ar > addr2line=addr2line > -arch=$(uname -m | sed -e 's/i.86/i386/;s/arm64/aarch64/;s/arm.*/arm/;s/ppc64.*/ppc64/') > -host=$arch > +host=$(uname -m | sed -e 's/i.86/i386/;s/arm64/aarch64/;s/arm.*/arm/;s/ppc64.*/ppc64/') > +arch=$host > +[ "$arch" = "aarch64" ] && arch="arm64" I'd prefer we keep our block of assignments a block of assignments. We can put this at the bottom of the block, or, since the whole point of this is to make sure help text looks right, then just put it in usage() at the top. Thanks, drew > cross_prefix= > endian="" > pretty_print_stacks=yes > -- > 2.48.1 > > > -- > kvm-riscv mailing list > kvm-riscv@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/kvm-riscv -- kvm-riscv mailing list kvm-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kvm-riscv ^ permalink raw reply [flat|nested] 26+ messages in thread
* [kvm-unit-tests PATCH v2 2/5] configure: arm/arm64: Display the correct default processor 2025-03-14 15:49 [kvm-unit-tests PATCH v2 0/5] arm64: Change the default QEMU CPU type to "max" Jean-Philippe Brucker 2025-03-14 15:49 ` [kvm-unit-tests PATCH v2 1/5] configure: arm64: Don't display 'aarch64' as the default architecture Jean-Philippe Brucker @ 2025-03-14 15:49 ` Jean-Philippe Brucker 2025-03-17 8:30 ` Eric Auger 2025-03-22 11:07 ` Andrew Jones 2025-03-14 15:49 ` [kvm-unit-tests PATCH v2 3/5] arm64: Implement the ./configure --processor option Jean-Philippe Brucker ` (2 subsequent siblings) 4 siblings, 2 replies; 26+ messages in thread From: Jean-Philippe Brucker @ 2025-03-14 15:49 UTC (permalink / raw) To: andrew.jones, alexandru.elisei Cc: eric.auger, kvmarm, kvm, kvm-riscv, vladimir.murzin From: Alexandru Elisei <alexandru.elisei@arm.com> The help text for the --processor option displays the architecture name as the default processor type. But the default for arm is cortex-a15, and for arm64 is cortex-a57. Teach configure to display the correct default processor type for these two architectures. Signed-off-by: Alexandru Elisei <alexandru.elisei@arm.com> --- configure | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/configure b/configure index dc3413fc..5306bad3 100755 --- a/configure +++ b/configure @@ -5,6 +5,24 @@ if [ -z "${BASH_VERSINFO[0]}" ] || [ "${BASH_VERSINFO[0]}" -lt 4 ] ; then exit 1 fi +function get_default_processor() +{ + local arch="$1" + + case "$arch" in + "arm") + default_processor="cortex-a15" + ;; + "arm64") + default_processor="cortex-a57" + ;; + *) + default_processor=$arch + esac + + echo "$default_processor" +} + srcdir=$(cd "$(dirname "$0")"; pwd) prefix=/usr/local cc=gcc @@ -43,13 +61,14 @@ else fi usage() { + [ -z "$processor" ] && processor=$(get_default_processor $arch) cat <<-EOF Usage: $0 [options] Options include: --arch=ARCH architecture to compile for ($arch). ARCH can be one of: arm, arm64, i386, ppc64, riscv32, riscv64, s390x, x86_64 - --processor=PROCESSOR processor to compile for ($arch) + --processor=PROCESSOR processor to compile for ($processor) --target=TARGET target platform that the tests will be running on (qemu or kvmtool, default is qemu) (arm/arm64 only) --cross-prefix=PREFIX cross compiler prefix @@ -319,13 +338,8 @@ if [ "$earlycon" ]; then fi fi -[ -z "$processor" ] && processor="$arch" - -if [ "$processor" = "arm64" ]; then - processor="cortex-a57" -elif [ "$processor" = "arm" ]; then - processor="cortex-a15" -fi +# $arch will have changed when cross-compiling. +[ -z "$processor" ] && processor=$(get_default_processor $arch) if [ "$arch" = "i386" ] || [ "$arch" = "x86_64" ]; then testdir=x86 -- 2.48.1 -- kvm-riscv mailing list kvm-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kvm-riscv ^ permalink raw reply related [flat|nested] 26+ messages in thread
* Re: [kvm-unit-tests PATCH v2 2/5] configure: arm/arm64: Display the correct default processor 2025-03-14 15:49 ` [kvm-unit-tests PATCH v2 2/5] configure: arm/arm64: Display the correct default processor Jean-Philippe Brucker @ 2025-03-17 8:30 ` Eric Auger 2025-03-22 11:07 ` Andrew Jones 1 sibling, 0 replies; 26+ messages in thread From: Eric Auger @ 2025-03-17 8:30 UTC (permalink / raw) To: Jean-Philippe Brucker, andrew.jones, alexandru.elisei Cc: kvmarm, kvm, kvm-riscv, vladimir.murzin On 3/14/25 4:49 PM, Jean-Philippe Brucker wrote: > From: Alexandru Elisei <alexandru.elisei@arm.com> > > The help text for the --processor option displays the architecture name as > the default processor type. But the default for arm is cortex-a15, and for > arm64 is cortex-a57. Teach configure to display the correct default > processor type for these two architectures. > > Signed-off-by: Alexandru Elisei <alexandru.elisei@arm.com> * Reviewed-by: Eric Auger <eric.auger@redhat.com> Eric * > --- > configure | 30 ++++++++++++++++++++++-------- > 1 file changed, 22 insertions(+), 8 deletions(-) > > diff --git a/configure b/configure > index dc3413fc..5306bad3 100755 > --- a/configure > +++ b/configure > @@ -5,6 +5,24 @@ if [ -z "${BASH_VERSINFO[0]}" ] || [ "${BASH_VERSINFO[0]}" -lt 4 ] ; then > exit 1 > fi > > +function get_default_processor() > +{ > + local arch="$1" > + > + case "$arch" in > + "arm") > + default_processor="cortex-a15" > + ;; > + "arm64") > + default_processor="cortex-a57" > + ;; > + *) > + default_processor=$arch > + esac > + > + echo "$default_processor" > +} > + > srcdir=$(cd "$(dirname "$0")"; pwd) > prefix=/usr/local > cc=gcc > @@ -43,13 +61,14 @@ else > fi > > usage() { > + [ -z "$processor" ] && processor=$(get_default_processor $arch) > cat <<-EOF > Usage: $0 [options] > > Options include: > --arch=ARCH architecture to compile for ($arch). ARCH can be one of: > arm, arm64, i386, ppc64, riscv32, riscv64, s390x, x86_64 > - --processor=PROCESSOR processor to compile for ($arch) > + --processor=PROCESSOR processor to compile for ($processor) > --target=TARGET target platform that the tests will be running on (qemu or > kvmtool, default is qemu) (arm/arm64 only) > --cross-prefix=PREFIX cross compiler prefix > @@ -319,13 +338,8 @@ if [ "$earlycon" ]; then > fi > fi > > -[ -z "$processor" ] && processor="$arch" > - > -if [ "$processor" = "arm64" ]; then > - processor="cortex-a57" > -elif [ "$processor" = "arm" ]; then > - processor="cortex-a15" > -fi > +# $arch will have changed when cross-compiling. > +[ -z "$processor" ] && processor=$(get_default_processor $arch) > > if [ "$arch" = "i386" ] || [ "$arch" = "x86_64" ]; then > testdir=x86 -- kvm-riscv mailing list kvm-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kvm-riscv ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [kvm-unit-tests PATCH v2 2/5] configure: arm/arm64: Display the correct default processor 2025-03-14 15:49 ` [kvm-unit-tests PATCH v2 2/5] configure: arm/arm64: Display the correct default processor Jean-Philippe Brucker 2025-03-17 8:30 ` Eric Auger @ 2025-03-22 11:07 ` Andrew Jones 1 sibling, 0 replies; 26+ messages in thread From: Andrew Jones @ 2025-03-22 11:07 UTC (permalink / raw) To: Jean-Philippe Brucker Cc: alexandru.elisei, eric.auger, kvmarm, kvm, kvm-riscv, vladimir.murzin On Fri, Mar 14, 2025 at 03:49:02PM +0000, Jean-Philippe Brucker wrote: > From: Alexandru Elisei <alexandru.elisei@arm.com> > > The help text for the --processor option displays the architecture name as > the default processor type. But the default for arm is cortex-a15, and for > arm64 is cortex-a57. Teach configure to display the correct default > processor type for these two architectures. > > Signed-off-by: Alexandru Elisei <alexandru.elisei@arm.com> > --- > configure | 30 ++++++++++++++++++++++-------- > 1 file changed, 22 insertions(+), 8 deletions(-) > > diff --git a/configure b/configure > index dc3413fc..5306bad3 100755 > --- a/configure > +++ b/configure > @@ -5,6 +5,24 @@ if [ -z "${BASH_VERSINFO[0]}" ] || [ "${BASH_VERSINFO[0]}" -lt 4 ] ; then > exit 1 > fi > > +function get_default_processor() > +{ > + local arch="$1" > + > + case "$arch" in > + "arm") > + default_processor="cortex-a15" > + ;; > + "arm64") > + default_processor="cortex-a57" > + ;; > + *) > + default_processor=$arch Missing ';;' > + esac > + > + echo "$default_processor" > +} > + > srcdir=$(cd "$(dirname "$0")"; pwd) > prefix=/usr/local > cc=gcc > @@ -43,13 +61,14 @@ else > fi > > usage() { > + [ -z "$processor" ] && processor=$(get_default_processor $arch) Seeing this convinces me that the additional arch=arm64 from the last patch should be done here above this. > cat <<-EOF > Usage: $0 [options] > > Options include: > --arch=ARCH architecture to compile for ($arch). ARCH can be one of: > arm, arm64, i386, ppc64, riscv32, riscv64, s390x, x86_64 > - --processor=PROCESSOR processor to compile for ($arch) > + --processor=PROCESSOR processor to compile for ($processor) > --target=TARGET target platform that the tests will be running on (qemu or > kvmtool, default is qemu) (arm/arm64 only) > --cross-prefix=PREFIX cross compiler prefix > @@ -319,13 +338,8 @@ if [ "$earlycon" ]; then > fi > fi > > -[ -z "$processor" ] && processor="$arch" > - > -if [ "$processor" = "arm64" ]; then > - processor="cortex-a57" > -elif [ "$processor" = "arm" ]; then > - processor="cortex-a15" > -fi > +# $arch will have changed when cross-compiling. > +[ -z "$processor" ] && processor=$(get_default_processor $arch) > > if [ "$arch" = "i386" ] || [ "$arch" = "x86_64" ]; then > testdir=x86 > -- > 2.48.1 > Otherwise, Reviewed-by: Andrew Jones <andrew.jones@linux.dev> -- kvm-riscv mailing list kvm-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kvm-riscv ^ permalink raw reply [flat|nested] 26+ messages in thread
* [kvm-unit-tests PATCH v2 3/5] arm64: Implement the ./configure --processor option 2025-03-14 15:49 [kvm-unit-tests PATCH v2 0/5] arm64: Change the default QEMU CPU type to "max" Jean-Philippe Brucker 2025-03-14 15:49 ` [kvm-unit-tests PATCH v2 1/5] configure: arm64: Don't display 'aarch64' as the default architecture Jean-Philippe Brucker 2025-03-14 15:49 ` [kvm-unit-tests PATCH v2 2/5] configure: arm/arm64: Display the correct default processor Jean-Philippe Brucker @ 2025-03-14 15:49 ` Jean-Philippe Brucker 2025-03-17 8:34 ` Eric Auger 2025-03-14 15:49 ` [kvm-unit-tests PATCH v2 4/5] configure: Add --qemu-cpu option Jean-Philippe Brucker 2025-03-14 15:49 ` [kvm-unit-tests PATCH v2 5/5] arm64: Use -cpu max as the default for TCG Jean-Philippe Brucker 4 siblings, 1 reply; 26+ messages in thread From: Jean-Philippe Brucker @ 2025-03-14 15:49 UTC (permalink / raw) To: andrew.jones, alexandru.elisei Cc: eric.auger, kvmarm, kvm, kvm-riscv, vladimir.murzin From: Alexandru Elisei <alexandru.elisei@arm.com> The help text for the ./configure --processor option says: --processor=PROCESSOR processor to compile for (cortex-a57) but, unlike arm, the build system does not pass a -mcpu argument to the compiler. Fix it, and bring arm64 at parity with arm. Note that this introduces a regression, which is also present on arm: if the --processor argument is something that the compiler doesn't understand, but qemu does (like 'max'), then compilation fails. This will be fixed in a following patch; another fix is to specify a CPU model that gcc implements by using --cflags=-mcpu=<cpu>. Reviewed-by: Andrew Jones <andrew.jones@linux.dev> Signed-off-by: Alexandru Elisei <alexandru.elisei@arm.com> --- arm/Makefile.arm | 1 - arm/Makefile.common | 1 + 2 files changed, 1 insertion(+), 1 deletion(-) diff --git a/arm/Makefile.arm b/arm/Makefile.arm index 7fd39f3a..d6250b7f 100644 --- a/arm/Makefile.arm +++ b/arm/Makefile.arm @@ -12,7 +12,6 @@ $(error Cannot build arm32 tests as EFI apps) endif CFLAGS += $(machine) -CFLAGS += -mcpu=$(PROCESSOR) CFLAGS += -mno-unaligned-access ifeq ($(TARGET),qemu) diff --git a/arm/Makefile.common b/arm/Makefile.common index f828dbe0..a5d97bcf 100644 --- a/arm/Makefile.common +++ b/arm/Makefile.common @@ -25,6 +25,7 @@ AUXFLAGS ?= 0x0 # stack.o relies on frame pointers. KEEP_FRAME_POINTER := y +CFLAGS += -mcpu=$(PROCESSOR) CFLAGS += -std=gnu99 CFLAGS += -ffreestanding CFLAGS += -O2 -- 2.48.1 -- kvm-riscv mailing list kvm-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kvm-riscv ^ permalink raw reply related [flat|nested] 26+ messages in thread
* Re: [kvm-unit-tests PATCH v2 3/5] arm64: Implement the ./configure --processor option 2025-03-14 15:49 ` [kvm-unit-tests PATCH v2 3/5] arm64: Implement the ./configure --processor option Jean-Philippe Brucker @ 2025-03-17 8:34 ` Eric Auger 0 siblings, 0 replies; 26+ messages in thread From: Eric Auger @ 2025-03-17 8:34 UTC (permalink / raw) To: Jean-Philippe Brucker, andrew.jones, alexandru.elisei Cc: kvmarm, kvm, kvm-riscv, vladimir.murzin On 3/14/25 4:49 PM, Jean-Philippe Brucker wrote: > From: Alexandru Elisei <alexandru.elisei@arm.com> > > The help text for the ./configure --processor option says: > > --processor=PROCESSOR processor to compile for (cortex-a57) > > but, unlike arm, the build system does not pass a -mcpu argument to the > compiler. Fix it, and bring arm64 at parity with arm. > > Note that this introduces a regression, which is also present on arm: if > the --processor argument is something that the compiler doesn't understand, > but qemu does (like 'max'), then compilation fails. This will be fixed in a > following patch; another fix is to specify a CPU model that gcc implements > by using --cflags=-mcpu=<cpu>. > > Reviewed-by: Andrew Jones <andrew.jones@linux.dev> > Signed-off-by: Alexandru Elisei <alexandru.elisei@arm.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Eric > --- > arm/Makefile.arm | 1 - > arm/Makefile.common | 1 + > 2 files changed, 1 insertion(+), 1 deletion(-) > > diff --git a/arm/Makefile.arm b/arm/Makefile.arm > index 7fd39f3a..d6250b7f 100644 > --- a/arm/Makefile.arm > +++ b/arm/Makefile.arm > @@ -12,7 +12,6 @@ $(error Cannot build arm32 tests as EFI apps) > endif > > CFLAGS += $(machine) > -CFLAGS += -mcpu=$(PROCESSOR) > CFLAGS += -mno-unaligned-access > > ifeq ($(TARGET),qemu) > diff --git a/arm/Makefile.common b/arm/Makefile.common > index f828dbe0..a5d97bcf 100644 > --- a/arm/Makefile.common > +++ b/arm/Makefile.common > @@ -25,6 +25,7 @@ AUXFLAGS ?= 0x0 > # stack.o relies on frame pointers. > KEEP_FRAME_POINTER := y > > +CFLAGS += -mcpu=$(PROCESSOR) > CFLAGS += -std=gnu99 > CFLAGS += -ffreestanding > CFLAGS += -O2 -- kvm-riscv mailing list kvm-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kvm-riscv ^ permalink raw reply [flat|nested] 26+ messages in thread
* [kvm-unit-tests PATCH v2 4/5] configure: Add --qemu-cpu option 2025-03-14 15:49 [kvm-unit-tests PATCH v2 0/5] arm64: Change the default QEMU CPU type to "max" Jean-Philippe Brucker ` (2 preceding siblings ...) 2025-03-14 15:49 ` [kvm-unit-tests PATCH v2 3/5] arm64: Implement the ./configure --processor option Jean-Philippe Brucker @ 2025-03-14 15:49 ` Jean-Philippe Brucker 2025-03-17 8:53 ` Eric Auger ` (2 more replies) 2025-03-14 15:49 ` [kvm-unit-tests PATCH v2 5/5] arm64: Use -cpu max as the default for TCG Jean-Philippe Brucker 4 siblings, 3 replies; 26+ messages in thread From: Jean-Philippe Brucker @ 2025-03-14 15:49 UTC (permalink / raw) To: andrew.jones, alexandru.elisei Cc: eric.auger, kvmarm, kvm, kvm-riscv, vladimir.murzin, Jean-Philippe Brucker Add the --qemu-cpu option to let users set the CPU type to run on. At the moment --processor allows to set both GCC -mcpu flag and QEMU -cpu. On Arm we'd like to pass `-cpu max` to QEMU in order to enable all the TCG features by default, and it could also be nice to let users modify the CPU capabilities by setting extra -cpu options. Since GCC -mcpu doesn't accept "max" or "host", separate the compiler and QEMU arguments. `--processor` is now exclusively for compiler options, as indicated by its documentation ("processor to compile for"). So use $QEMU_CPU on RISC-V as well. Suggested-by: Andrew Jones <andrew.jones@linux.dev> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> --- scripts/mkstandalone.sh | 2 +- arm/run | 17 +++++++++++------ riscv/run | 8 ++++---- configure | 7 +++++++ 4 files changed, 23 insertions(+), 11 deletions(-) diff --git a/scripts/mkstandalone.sh b/scripts/mkstandalone.sh index 2318a85f..6b5f725d 100755 --- a/scripts/mkstandalone.sh +++ b/scripts/mkstandalone.sh @@ -42,7 +42,7 @@ generate_test () config_export ARCH config_export ARCH_NAME - config_export PROCESSOR + config_export QEMU_CPU echo "echo BUILD_HEAD=$(cat build-head)" diff --git a/arm/run b/arm/run index efdd44ce..561bafab 100755 --- a/arm/run +++ b/arm/run @@ -8,7 +8,7 @@ if [ -z "$KUT_STANDALONE" ]; then source config.mak source scripts/arch-run.bash fi -processor="$PROCESSOR" +qemu_cpu="$QEMU_CPU" if [ "$QEMU" ] && [ -z "$ACCEL" ] && [ "$HOST" = "aarch64" ] && [ "$ARCH" = "arm" ] && @@ -37,12 +37,17 @@ if [ "$ACCEL" = "kvm" ]; then fi fi -if [ "$ACCEL" = "kvm" ] || [ "$ACCEL" = "hvf" ]; then - if [ "$HOST" = "aarch64" ] || [ "$HOST" = "arm" ]; then - processor="host" +if [ -z "$qemu_cpu" ]; then + if ( [ "$ACCEL" = "kvm" ] || [ "$ACCEL" = "hvf" ] ) && + ( [ "$HOST" = "aarch64" ] || [ "$HOST" = "arm" ] ); then + qemu_cpu="host" if [ "$ARCH" = "arm" ] && [ "$HOST" = "aarch64" ]; then - processor+=",aarch64=off" + qemu_cpu+=",aarch64=off" fi + elif [ "$ARCH" = "arm64" ]; then + qemu_cpu="cortex-a57" + else + qemu_cpu="cortex-a15" fi fi @@ -71,7 +76,7 @@ if $qemu $M -device '?' | grep -q pci-testdev; then fi A="-accel $ACCEL$ACCEL_PROPS" -command="$qemu -nodefaults $M $A -cpu $processor $chr_testdev $pci_testdev" +command="$qemu -nodefaults $M $A -cpu $qemu_cpu $chr_testdev $pci_testdev" command+=" -display none -serial stdio" command="$(migration_cmd) $(timeout_cmd) $command" diff --git a/riscv/run b/riscv/run index e2f5a922..02fcf0c0 100755 --- a/riscv/run +++ b/riscv/run @@ -11,12 +11,12 @@ fi # Allow user overrides of some config.mak variables mach=$MACHINE_OVERRIDE -processor=$PROCESSOR_OVERRIDE +qemu_cpu=$QEMU_CPU_OVERRIDE firmware=$FIRMWARE_OVERRIDE -[ "$PROCESSOR" = "$ARCH" ] && PROCESSOR="max" +[ -z "$QEMU_CPU" ] && QEMU_CPU="max" : "${mach:=virt}" -: "${processor:=$PROCESSOR}" +: "${qemu_cpu:=$QEMU_CPU}" : "${firmware:=$FIRMWARE}" [ "$firmware" ] && firmware="-bios $firmware" @@ -32,7 +32,7 @@ fi mach="-machine $mach" command="$qemu -nodefaults -nographic -serial mon:stdio" -command+=" $mach $acc $firmware -cpu $processor " +command+=" $mach $acc $firmware -cpu $qemu_cpu " command="$(migration_cmd) $(timeout_cmd) $command" if [ "$UEFI_SHELL_RUN" = "y" ]; then diff --git a/configure b/configure index 5306bad3..d25bd23e 100755 --- a/configure +++ b/configure @@ -52,6 +52,7 @@ page_size= earlycon= efi= efi_direct= +qemu_cpu= # Enable -Werror by default for git repositories only (i.e. developer builds) if [ -e "$srcdir"/.git ]; then @@ -69,6 +70,8 @@ usage() { --arch=ARCH architecture to compile for ($arch). ARCH can be one of: arm, arm64, i386, ppc64, riscv32, riscv64, s390x, x86_64 --processor=PROCESSOR processor to compile for ($processor) + --qemu-cpu=CPU the CPU model to run on. The default depends on + the configuration, usually it is "host" or "max". --target=TARGET target platform that the tests will be running on (qemu or kvmtool, default is qemu) (arm/arm64 only) --cross-prefix=PREFIX cross compiler prefix @@ -142,6 +145,9 @@ while [[ $optno -le $argc ]]; do --processor) processor="$arg" ;; + --qemu-cpu) + qemu_cpu="$arg" + ;; --target) target="$arg" ;; @@ -464,6 +470,7 @@ ARCH=$arch ARCH_NAME=$arch_name ARCH_LIBDIR=$arch_libdir PROCESSOR=$processor +QEMU_CPU=$qemu_cpu CC=$cc CFLAGS=$cflags LD=$cross_prefix$ld -- 2.48.1 -- kvm-riscv mailing list kvm-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kvm-riscv ^ permalink raw reply related [flat|nested] 26+ messages in thread
* Re: [kvm-unit-tests PATCH v2 4/5] configure: Add --qemu-cpu option 2025-03-14 15:49 ` [kvm-unit-tests PATCH v2 4/5] configure: Add --qemu-cpu option Jean-Philippe Brucker @ 2025-03-17 8:53 ` Eric Auger 2025-03-20 13:42 ` Alexandru Elisei 2025-03-22 11:26 ` Andrew Jones 2 siblings, 0 replies; 26+ messages in thread From: Eric Auger @ 2025-03-17 8:53 UTC (permalink / raw) To: Jean-Philippe Brucker, andrew.jones, alexandru.elisei Cc: kvmarm, kvm, kvm-riscv, vladimir.murzin Hi Jean-Philippe, On 3/14/25 4:49 PM, Jean-Philippe Brucker wrote: > Add the --qemu-cpu option to let users set the CPU type to run on. > At the moment --processor allows to set both GCC -mcpu flag and QEMU > -cpu. On Arm we'd like to pass `-cpu max` to QEMU in order to enable all > the TCG features by default, and it could also be nice to let users > modify the CPU capabilities by setting extra -cpu options. > Since GCC -mcpu doesn't accept "max" or "host", separate the compiler > and QEMU arguments. > > `--processor` is now exclusively for compiler options, as indicated by > its documentation ("processor to compile for"). So use $QEMU_CPU on > RISC-V as well. > > Suggested-by: Andrew Jones <andrew.jones@linux.dev> > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> > --- > scripts/mkstandalone.sh | 2 +- > arm/run | 17 +++++++++++------ > riscv/run | 8 ++++---- > configure | 7 +++++++ > 4 files changed, 23 insertions(+), 11 deletions(-) > > diff --git a/scripts/mkstandalone.sh b/scripts/mkstandalone.sh > index 2318a85f..6b5f725d 100755 > --- a/scripts/mkstandalone.sh > +++ b/scripts/mkstandalone.sh > @@ -42,7 +42,7 @@ generate_test () > > config_export ARCH > config_export ARCH_NAME > - config_export PROCESSOR > + config_export QEMU_CPU > > echo "echo BUILD_HEAD=$(cat build-head)" > > diff --git a/arm/run b/arm/run > index efdd44ce..561bafab 100755 > --- a/arm/run > +++ b/arm/run > @@ -8,7 +8,7 @@ if [ -z "$KUT_STANDALONE" ]; then > source config.mak > source scripts/arch-run.bash > fi > -processor="$PROCESSOR" > +qemu_cpu="$QEMU_CPU" > > if [ "$QEMU" ] && [ -z "$ACCEL" ] && > [ "$HOST" = "aarch64" ] && [ "$ARCH" = "arm" ] && > @@ -37,12 +37,17 @@ if [ "$ACCEL" = "kvm" ]; then > fi > fi > > -if [ "$ACCEL" = "kvm" ] || [ "$ACCEL" = "hvf" ]; then > - if [ "$HOST" = "aarch64" ] || [ "$HOST" = "arm" ]; then > - processor="host" > +if [ -z "$qemu_cpu" ]; then > + if ( [ "$ACCEL" = "kvm" ] || [ "$ACCEL" = "hvf" ] ) && > + ( [ "$HOST" = "aarch64" ] || [ "$HOST" = "arm" ] ); then > + qemu_cpu="host" > if [ "$ARCH" = "arm" ] && [ "$HOST" = "aarch64" ]; then > - processor+=",aarch64=off" > + qemu_cpu+=",aarch64=off" > fi > + elif [ "$ARCH" = "arm64" ]; then > + qemu_cpu="cortex-a57" > + else > + qemu_cpu="cortex-a15" > fi > fi > > @@ -71,7 +76,7 @@ if $qemu $M -device '?' | grep -q pci-testdev; then > fi > > A="-accel $ACCEL$ACCEL_PROPS" > -command="$qemu -nodefaults $M $A -cpu $processor $chr_testdev $pci_testdev" > +command="$qemu -nodefaults $M $A -cpu $qemu_cpu $chr_testdev $pci_testdev" > command+=" -display none -serial stdio" > command="$(migration_cmd) $(timeout_cmd) $command" > > diff --git a/riscv/run b/riscv/run > index e2f5a922..02fcf0c0 100755 > --- a/riscv/run > +++ b/riscv/run > @@ -11,12 +11,12 @@ fi > > # Allow user overrides of some config.mak variables > mach=$MACHINE_OVERRIDE > -processor=$PROCESSOR_OVERRIDE > +qemu_cpu=$QEMU_CPU_OVERRIDE > firmware=$FIRMWARE_OVERRIDE > > -[ "$PROCESSOR" = "$ARCH" ] && PROCESSOR="max" > +[ -z "$QEMU_CPU" ] && QEMU_CPU="max" > : "${mach:=virt}" > -: "${processor:=$PROCESSOR}" > +: "${qemu_cpu:=$QEMU_CPU}" > : "${firmware:=$FIRMWARE}" > [ "$firmware" ] && firmware="-bios $firmware" > > @@ -32,7 +32,7 @@ fi > mach="-machine $mach" > > command="$qemu -nodefaults -nographic -serial mon:stdio" > -command+=" $mach $acc $firmware -cpu $processor " > +command+=" $mach $acc $firmware -cpu $qemu_cpu " > command="$(migration_cmd) $(timeout_cmd) $command" > > if [ "$UEFI_SHELL_RUN" = "y" ]; then > diff --git a/configure b/configure > index 5306bad3..d25bd23e 100755 > --- a/configure > +++ b/configure > @@ -52,6 +52,7 @@ page_size= > earlycon= > efi= > efi_direct= > +qemu_cpu= > > # Enable -Werror by default for git repositories only (i.e. developer builds) > if [ -e "$srcdir"/.git ]; then > @@ -69,6 +70,8 @@ usage() { > --arch=ARCH architecture to compile for ($arch). ARCH can be one of: > arm, arm64, i386, ppc64, riscv32, riscv64, s390x, x86_64 > --processor=PROCESSOR processor to compile for ($processor) > + --qemu-cpu=CPU the CPU model to run on. The default depends on > + the configuration, usually it is "host" or "max". nit: I would remove 'usually it is "host" or "max"'. I don't see any consistency check between qemu_cpu and processor in case it is explicitly set? Do we care? Eric > --target=TARGET target platform that the tests will be running on (qemu or > kvmtool, default is qemu) (arm/arm64 only) > --cross-prefix=PREFIX cross compiler prefix > @@ -142,6 +145,9 @@ while [[ $optno -le $argc ]]; do > --processor) > processor="$arg" > ;; > + --qemu-cpu) > + qemu_cpu="$arg" > + ;; > --target) > target="$arg" > ;; > @@ -464,6 +470,7 @@ ARCH=$arch > ARCH_NAME=$arch_name > ARCH_LIBDIR=$arch_libdir > PROCESSOR=$processor > +QEMU_CPU=$qemu_cpu > CC=$cc > CFLAGS=$cflags > LD=$cross_prefix$ld Reviewed-by: Eric Auger <eric.auger@redhat.com> Eric -- kvm-riscv mailing list kvm-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kvm-riscv ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [kvm-unit-tests PATCH v2 4/5] configure: Add --qemu-cpu option 2025-03-14 15:49 ` [kvm-unit-tests PATCH v2 4/5] configure: Add --qemu-cpu option Jean-Philippe Brucker 2025-03-17 8:53 ` Eric Auger @ 2025-03-20 13:42 ` Alexandru Elisei 2025-03-22 11:25 ` Andrew Jones 2025-03-22 11:26 ` Andrew Jones 2 siblings, 1 reply; 26+ messages in thread From: Alexandru Elisei @ 2025-03-20 13:42 UTC (permalink / raw) To: Jean-Philippe Brucker Cc: andrew.jones, eric.auger, kvmarm, kvm, kvm-riscv, vladimir.murzin Hi Jean-Philippe, Thank you for picking up this work, much appreciated. I like this approach a lot: the value for -cpu, if the user hasn't specified one with --qemu-cpu, depends on the qemu accelerator, which is either in the test specification, or an environment variable, making it unknown at configure time. So it makes more sense to have arm/run choose the correct -cpu value at runtime, than having configure put a default value for QEMU_CPU in config.mak. On Fri, Mar 14, 2025 at 03:49:04PM +0000, Jean-Philippe Brucker wrote: > Add the --qemu-cpu option to let users set the CPU type to run on. > At the moment --processor allows to set both GCC -mcpu flag and QEMU > -cpu. On Arm we'd like to pass `-cpu max` to QEMU in order to enable all > the TCG features by default, and it could also be nice to let users > modify the CPU capabilities by setting extra -cpu options. > Since GCC -mcpu doesn't accept "max" or "host", separate the compiler > and QEMU arguments. > > `--processor` is now exclusively for compiler options, as indicated by > its documentation ("processor to compile for"). So use $QEMU_CPU on > RISC-V as well. > > Suggested-by: Andrew Jones <andrew.jones@linux.dev> > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> > --- > scripts/mkstandalone.sh | 2 +- > arm/run | 17 +++++++++++------ > riscv/run | 8 ++++---- > configure | 7 +++++++ > 4 files changed, 23 insertions(+), 11 deletions(-) > > diff --git a/scripts/mkstandalone.sh b/scripts/mkstandalone.sh > index 2318a85f..6b5f725d 100755 > --- a/scripts/mkstandalone.sh > +++ b/scripts/mkstandalone.sh > @@ -42,7 +42,7 @@ generate_test () > > config_export ARCH > config_export ARCH_NAME > - config_export PROCESSOR > + config_export QEMU_CPU > > echo "echo BUILD_HEAD=$(cat build-head)" > > diff --git a/arm/run b/arm/run > index efdd44ce..561bafab 100755 > --- a/arm/run > +++ b/arm/run > @@ -8,7 +8,7 @@ if [ -z "$KUT_STANDALONE" ]; then > source config.mak > source scripts/arch-run.bash > fi > -processor="$PROCESSOR" > +qemu_cpu="$QEMU_CPU" > > if [ "$QEMU" ] && [ -z "$ACCEL" ] && > [ "$HOST" = "aarch64" ] && [ "$ARCH" = "arm" ] && > @@ -37,12 +37,17 @@ if [ "$ACCEL" = "kvm" ]; then > fi > fi > > -if [ "$ACCEL" = "kvm" ] || [ "$ACCEL" = "hvf" ]; then > - if [ "$HOST" = "aarch64" ] || [ "$HOST" = "arm" ]; then > - processor="host" > +if [ -z "$qemu_cpu" ]; then > + if ( [ "$ACCEL" = "kvm" ] || [ "$ACCEL" = "hvf" ] ) && > + ( [ "$HOST" = "aarch64" ] || [ "$HOST" = "arm" ] ); then > + qemu_cpu="host" > if [ "$ARCH" = "arm" ] && [ "$HOST" = "aarch64" ]; then > - processor+=",aarch64=off" > + qemu_cpu+=",aarch64=off" > fi > + elif [ "$ARCH" = "arm64" ]; then > + qemu_cpu="cortex-a57" > + else > + qemu_cpu="cortex-a15" > fi > fi > > @@ -71,7 +76,7 @@ if $qemu $M -device '?' | grep -q pci-testdev; then > fi > > A="-accel $ACCEL$ACCEL_PROPS" > -command="$qemu -nodefaults $M $A -cpu $processor $chr_testdev $pci_testdev" > +command="$qemu -nodefaults $M $A -cpu $qemu_cpu $chr_testdev $pci_testdev" > command+=" -display none -serial stdio" > command="$(migration_cmd) $(timeout_cmd) $command" > > diff --git a/riscv/run b/riscv/run > index e2f5a922..02fcf0c0 100755 > --- a/riscv/run > +++ b/riscv/run > @@ -11,12 +11,12 @@ fi > > # Allow user overrides of some config.mak variables > mach=$MACHINE_OVERRIDE > -processor=$PROCESSOR_OVERRIDE > +qemu_cpu=$QEMU_CPU_OVERRIDE The name QEMU_CPU_OVERRIDE makes more sense, but I think this will break existing setups where people use PROCESSOR_OVERRIDE to pass a particular -cpu value to qemu. If I were to guess, the environment variable was added to pass a value to -cpu different than what it can be specified with ./configure --processor, which is exactly what --qemu-cpu does. But I'll let Drew comment on that. > firmware=$FIRMWARE_OVERRIDE > > -[ "$PROCESSOR" = "$ARCH" ] && PROCESSOR="max" > +[ -z "$QEMU_CPU" ] && QEMU_CPU="max" > : "${mach:=virt}" > -: "${processor:=$PROCESSOR}" > +: "${qemu_cpu:=$QEMU_CPU}" > : "${firmware:=$FIRMWARE}" > [ "$firmware" ] && firmware="-bios $firmware" > > @@ -32,7 +32,7 @@ fi > mach="-machine $mach" > > command="$qemu -nodefaults -nographic -serial mon:stdio" > -command+=" $mach $acc $firmware -cpu $processor " > +command+=" $mach $acc $firmware -cpu $qemu_cpu " > command="$(migration_cmd) $(timeout_cmd) $command" > > if [ "$UEFI_SHELL_RUN" = "y" ]; then > diff --git a/configure b/configure > index 5306bad3..d25bd23e 100755 > --- a/configure > +++ b/configure > @@ -52,6 +52,7 @@ page_size= > earlycon= > efi= > efi_direct= > +qemu_cpu= > > # Enable -Werror by default for git repositories only (i.e. developer builds) > if [ -e "$srcdir"/.git ]; then > @@ -69,6 +70,8 @@ usage() { > --arch=ARCH architecture to compile for ($arch). ARCH can be one of: > arm, arm64, i386, ppc64, riscv32, riscv64, s390x, x86_64 > --processor=PROCESSOR processor to compile for ($processor) > + --qemu-cpu=CPU the CPU model to run on. The default depends on > + the configuration, usually it is "host" or "max". Nitpick here, would you mind changing this to "The default value depends on the host system and the test configuration [..]", to make it clear that it also depends on the machine the tests are being run on? Thanks, Alex > --target=TARGET target platform that the tests will be running on (qemu or > kvmtool, default is qemu) (arm/arm64 only) > --cross-prefix=PREFIX cross compiler prefix > @@ -142,6 +145,9 @@ while [[ $optno -le $argc ]]; do > --processor) > processor="$arg" > ;; > + --qemu-cpu) > + qemu_cpu="$arg" > + ;; > --target) > target="$arg" > ;; > @@ -464,6 +470,7 @@ ARCH=$arch > ARCH_NAME=$arch_name > ARCH_LIBDIR=$arch_libdir > PROCESSOR=$processor > +QEMU_CPU=$qemu_cpu > CC=$cc > CFLAGS=$cflags > LD=$cross_prefix$ld > -- > 2.48.1 > -- kvm-riscv mailing list kvm-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kvm-riscv ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [kvm-unit-tests PATCH v2 4/5] configure: Add --qemu-cpu option 2025-03-20 13:42 ` Alexandru Elisei @ 2025-03-22 11:25 ` Andrew Jones 0 siblings, 0 replies; 26+ messages in thread From: Andrew Jones @ 2025-03-22 11:25 UTC (permalink / raw) To: Alexandru Elisei Cc: Jean-Philippe Brucker, eric.auger, kvmarm, kvm, kvm-riscv, vladimir.murzin On Thu, Mar 20, 2025 at 01:42:01PM +0000, Alexandru Elisei wrote: ... > > mach=$MACHINE_OVERRIDE > > -processor=$PROCESSOR_OVERRIDE > > +qemu_cpu=$QEMU_CPU_OVERRIDE > > The name QEMU_CPU_OVERRIDE makes more sense, but I think this will break > existing setups where people use PROCESSOR_OVERRIDE to pass a particular > -cpu value to qemu. > > If I were to guess, the environment variable was added to pass a value to > -cpu different than what it can be specified with ./configure --processor, > which is exactly what --qemu-cpu does. But I'll let Drew comment on that. Yeah, ideally we wouldn't rename environment variables, but I'll cross my fingers that nobody will notice. Indeed, it was created to avoid needing to reconfigure for experimenting with different QEMU cpu models, so, even with --qemu-cpu we'll want something and the renaming here makes sense. > > > firmware=$FIRMWARE_OVERRIDE > > > > -[ "$PROCESSOR" = "$ARCH" ] && PROCESSOR="max" > > +[ -z "$QEMU_CPU" ] && QEMU_CPU="max" > > : "${mach:=virt}" > > -: "${processor:=$PROCESSOR}" > > +: "${qemu_cpu:=$QEMU_CPU}" > > : "${firmware:=$FIRMWARE}" > > [ "$firmware" ] && firmware="-bios $firmware" > > > > @@ -32,7 +32,7 @@ fi > > mach="-machine $mach" > > > > command="$qemu -nodefaults -nographic -serial mon:stdio" > > -command+=" $mach $acc $firmware -cpu $processor " > > +command+=" $mach $acc $firmware -cpu $qemu_cpu " > > command="$(migration_cmd) $(timeout_cmd) $command" > > > > if [ "$UEFI_SHELL_RUN" = "y" ]; then > > diff --git a/configure b/configure > > index 5306bad3..d25bd23e 100755 > > --- a/configure > > +++ b/configure > > @@ -52,6 +52,7 @@ page_size= > > earlycon= > > efi= > > efi_direct= > > +qemu_cpu= > > > > # Enable -Werror by default for git repositories only (i.e. developer builds) > > if [ -e "$srcdir"/.git ]; then > > @@ -69,6 +70,8 @@ usage() { > > --arch=ARCH architecture to compile for ($arch). ARCH can be one of: > > arm, arm64, i386, ppc64, riscv32, riscv64, s390x, x86_64 > > --processor=PROCESSOR processor to compile for ($processor) > > + --qemu-cpu=CPU the CPU model to run on. The default depends on > > + the configuration, usually it is "host" or "max". > > Nitpick here, would you mind changing this to "The default value depends on > the host system and the test configuration [..]", to make it clear that it also > depends on the machine the tests are being run on? I agree. Thanks, drew -- kvm-riscv mailing list kvm-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kvm-riscv ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [kvm-unit-tests PATCH v2 4/5] configure: Add --qemu-cpu option 2025-03-14 15:49 ` [kvm-unit-tests PATCH v2 4/5] configure: Add --qemu-cpu option Jean-Philippe Brucker 2025-03-17 8:53 ` Eric Auger 2025-03-20 13:42 ` Alexandru Elisei @ 2025-03-22 11:26 ` Andrew Jones 2025-03-23 11:16 ` Alexandru Elisei 2 siblings, 1 reply; 26+ messages in thread From: Andrew Jones @ 2025-03-22 11:26 UTC (permalink / raw) To: Jean-Philippe Brucker Cc: alexandru.elisei, eric.auger, kvmarm, kvm, kvm-riscv, vladimir.murzin On Fri, Mar 14, 2025 at 03:49:04PM +0000, Jean-Philippe Brucker wrote: > Add the --qemu-cpu option to let users set the CPU type to run on. > At the moment --processor allows to set both GCC -mcpu flag and QEMU > -cpu. On Arm we'd like to pass `-cpu max` to QEMU in order to enable all > the TCG features by default, and it could also be nice to let users > modify the CPU capabilities by setting extra -cpu options. > Since GCC -mcpu doesn't accept "max" or "host", separate the compiler > and QEMU arguments. > > `--processor` is now exclusively for compiler options, as indicated by > its documentation ("processor to compile for"). So use $QEMU_CPU on > RISC-V as well. > > Suggested-by: Andrew Jones <andrew.jones@linux.dev> > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> > --- > scripts/mkstandalone.sh | 2 +- > arm/run | 17 +++++++++++------ > riscv/run | 8 ++++---- > configure | 7 +++++++ > 4 files changed, 23 insertions(+), 11 deletions(-) > > diff --git a/scripts/mkstandalone.sh b/scripts/mkstandalone.sh > index 2318a85f..6b5f725d 100755 > --- a/scripts/mkstandalone.sh > +++ b/scripts/mkstandalone.sh > @@ -42,7 +42,7 @@ generate_test () > > config_export ARCH > config_export ARCH_NAME > - config_export PROCESSOR > + config_export QEMU_CPU > > echo "echo BUILD_HEAD=$(cat build-head)" > > diff --git a/arm/run b/arm/run > index efdd44ce..561bafab 100755 > --- a/arm/run > +++ b/arm/run > @@ -8,7 +8,7 @@ if [ -z "$KUT_STANDALONE" ]; then > source config.mak > source scripts/arch-run.bash > fi > -processor="$PROCESSOR" > +qemu_cpu="$QEMU_CPU" > > if [ "$QEMU" ] && [ -z "$ACCEL" ] && > [ "$HOST" = "aarch64" ] && [ "$ARCH" = "arm" ] && > @@ -37,12 +37,17 @@ if [ "$ACCEL" = "kvm" ]; then > fi > fi > > -if [ "$ACCEL" = "kvm" ] || [ "$ACCEL" = "hvf" ]; then > - if [ "$HOST" = "aarch64" ] || [ "$HOST" = "arm" ]; then > - processor="host" > +if [ -z "$qemu_cpu" ]; then > + if ( [ "$ACCEL" = "kvm" ] || [ "$ACCEL" = "hvf" ] ) && > + ( [ "$HOST" = "aarch64" ] || [ "$HOST" = "arm" ] ); then > + qemu_cpu="host" > if [ "$ARCH" = "arm" ] && [ "$HOST" = "aarch64" ]; then > - processor+=",aarch64=off" > + qemu_cpu+=",aarch64=off" > fi > + elif [ "$ARCH" = "arm64" ]; then > + qemu_cpu="cortex-a57" > + else > + qemu_cpu="cortex-a15" configure could set this in config.mak as DEFAULT_PROCESSOR, avoiding the need to duplicate it here. > fi > fi > > @@ -71,7 +76,7 @@ if $qemu $M -device '?' | grep -q pci-testdev; then > fi > > A="-accel $ACCEL$ACCEL_PROPS" > -command="$qemu -nodefaults $M $A -cpu $processor $chr_testdev $pci_testdev" > +command="$qemu -nodefaults $M $A -cpu $qemu_cpu $chr_testdev $pci_testdev" > command+=" -display none -serial stdio" > command="$(migration_cmd) $(timeout_cmd) $command" > > diff --git a/riscv/run b/riscv/run > index e2f5a922..02fcf0c0 100755 > --- a/riscv/run > +++ b/riscv/run > @@ -11,12 +11,12 @@ fi > > # Allow user overrides of some config.mak variables > mach=$MACHINE_OVERRIDE > -processor=$PROCESSOR_OVERRIDE > +qemu_cpu=$QEMU_CPU_OVERRIDE > firmware=$FIRMWARE_OVERRIDE > > -[ "$PROCESSOR" = "$ARCH" ] && PROCESSOR="max" > +[ -z "$QEMU_CPU" ] && QEMU_CPU="max" > : "${mach:=virt}" > -: "${processor:=$PROCESSOR}" > +: "${qemu_cpu:=$QEMU_CPU}" > : "${firmware:=$FIRMWARE}" > [ "$firmware" ] && firmware="-bios $firmware" > > @@ -32,7 +32,7 @@ fi > mach="-machine $mach" > > command="$qemu -nodefaults -nographic -serial mon:stdio" > -command+=" $mach $acc $firmware -cpu $processor " > +command+=" $mach $acc $firmware -cpu $qemu_cpu " > command="$(migration_cmd) $(timeout_cmd) $command" > > if [ "$UEFI_SHELL_RUN" = "y" ]; then > diff --git a/configure b/configure > index 5306bad3..d25bd23e 100755 > --- a/configure > +++ b/configure > @@ -52,6 +52,7 @@ page_size= > earlycon= > efi= > efi_direct= > +qemu_cpu= > > # Enable -Werror by default for git repositories only (i.e. developer builds) > if [ -e "$srcdir"/.git ]; then > @@ -69,6 +70,8 @@ usage() { > --arch=ARCH architecture to compile for ($arch). ARCH can be one of: > arm, arm64, i386, ppc64, riscv32, riscv64, s390x, x86_64 > --processor=PROCESSOR processor to compile for ($processor) > + --qemu-cpu=CPU the CPU model to run on. The default depends on > + the configuration, usually it is "host" or "max". > --target=TARGET target platform that the tests will be running on (qemu or > kvmtool, default is qemu) (arm/arm64 only) > --cross-prefix=PREFIX cross compiler prefix > @@ -142,6 +145,9 @@ while [[ $optno -le $argc ]]; do > --processor) > processor="$arg" > ;; > + --qemu-cpu) > + qemu_cpu="$arg" > + ;; > --target) > target="$arg" > ;; > @@ -464,6 +470,7 @@ ARCH=$arch > ARCH_NAME=$arch_name > ARCH_LIBDIR=$arch_libdir > PROCESSOR=$processor > +QEMU_CPU=$qemu_cpu > CC=$cc > CFLAGS=$cflags > LD=$cross_prefix$ld > -- > 2.48.1 > With the Alex's and Eric's requested changes to the help text, Reviewed-by: Andrew Jones <andrew.jones@linux.dev> -- kvm-riscv mailing list kvm-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kvm-riscv ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [kvm-unit-tests PATCH v2 4/5] configure: Add --qemu-cpu option 2025-03-22 11:26 ` Andrew Jones @ 2025-03-23 11:16 ` Alexandru Elisei 2025-03-24 8:19 ` Andrew Jones 0 siblings, 1 reply; 26+ messages in thread From: Alexandru Elisei @ 2025-03-23 11:16 UTC (permalink / raw) To: Andrew Jones Cc: Jean-Philippe Brucker, eric.auger, kvmarm, kvm, kvm-riscv, vladimir.murzin Hi Drew, On Sat, Mar 22, 2025 at 12:26:59PM +0100, Andrew Jones wrote: > On Fri, Mar 14, 2025 at 03:49:04PM +0000, Jean-Philippe Brucker wrote: > > Add the --qemu-cpu option to let users set the CPU type to run on. > > At the moment --processor allows to set both GCC -mcpu flag and QEMU > > -cpu. On Arm we'd like to pass `-cpu max` to QEMU in order to enable all > > the TCG features by default, and it could also be nice to let users > > modify the CPU capabilities by setting extra -cpu options. > > Since GCC -mcpu doesn't accept "max" or "host", separate the compiler > > and QEMU arguments. > > > > `--processor` is now exclusively for compiler options, as indicated by > > its documentation ("processor to compile for"). So use $QEMU_CPU on > > RISC-V as well. > > > > Suggested-by: Andrew Jones <andrew.jones@linux.dev> > > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> > > --- > > scripts/mkstandalone.sh | 2 +- > > arm/run | 17 +++++++++++------ > > riscv/run | 8 ++++---- > > configure | 7 +++++++ > > 4 files changed, 23 insertions(+), 11 deletions(-) > > > > diff --git a/scripts/mkstandalone.sh b/scripts/mkstandalone.sh > > index 2318a85f..6b5f725d 100755 > > --- a/scripts/mkstandalone.sh > > +++ b/scripts/mkstandalone.sh > > @@ -42,7 +42,7 @@ generate_test () > > > > config_export ARCH > > config_export ARCH_NAME > > - config_export PROCESSOR > > + config_export QEMU_CPU > > > > echo "echo BUILD_HEAD=$(cat build-head)" > > > > diff --git a/arm/run b/arm/run > > index efdd44ce..561bafab 100755 > > --- a/arm/run > > +++ b/arm/run > > @@ -8,7 +8,7 @@ if [ -z "$KUT_STANDALONE" ]; then > > source config.mak > > source scripts/arch-run.bash > > fi > > -processor="$PROCESSOR" > > +qemu_cpu="$QEMU_CPU" > > > > if [ "$QEMU" ] && [ -z "$ACCEL" ] && > > [ "$HOST" = "aarch64" ] && [ "$ARCH" = "arm" ] && > > @@ -37,12 +37,17 @@ if [ "$ACCEL" = "kvm" ]; then > > fi > > fi > > > > -if [ "$ACCEL" = "kvm" ] || [ "$ACCEL" = "hvf" ]; then > > - if [ "$HOST" = "aarch64" ] || [ "$HOST" = "arm" ]; then > > - processor="host" > > +if [ -z "$qemu_cpu" ]; then > > + if ( [ "$ACCEL" = "kvm" ] || [ "$ACCEL" = "hvf" ] ) && > > + ( [ "$HOST" = "aarch64" ] || [ "$HOST" = "arm" ] ); then > > + qemu_cpu="host" > > if [ "$ARCH" = "arm" ] && [ "$HOST" = "aarch64" ]; then > > - processor+=",aarch64=off" > > + qemu_cpu+=",aarch64=off" > > fi > > + elif [ "$ARCH" = "arm64" ]; then > > + qemu_cpu="cortex-a57" > > + else > > + qemu_cpu="cortex-a15" > > configure could set this in config.mak as DEFAULT_PROCESSOR, avoiding the > need to duplicate it here. That was my first instinct too, having the default value in config.mak seemed like the correct solution. But the problem with this is that the default -cpu type depends on -accel (set via unittests.cfg or as an environment variable), host and test architecture combination. All of these variables are known only at runtime. Let's say we have DEFAULT_QEMU_CPU=cortex-a57 in config.mak. If we keep the above heuristic, arm/run will override it with host,aarch64=off. IMO, having it in config.mak, but arm/run using it only under certain conditions is worse than not having it at all. arm/run choosing the default value **all the time** is at least consistent. We could modify the help text for --qemu-cpu to say something like "If left unset, the $ARCH/run script will choose a best value based on the host system and test configuration." Thanks, Alex > > > fi > > fi > > > > @@ -71,7 +76,7 @@ if $qemu $M -device '?' | grep -q pci-testdev; then > > fi > > > > A="-accel $ACCEL$ACCEL_PROPS" > > -command="$qemu -nodefaults $M $A -cpu $processor $chr_testdev $pci_testdev" > > +command="$qemu -nodefaults $M $A -cpu $qemu_cpu $chr_testdev $pci_testdev" > > command+=" -display none -serial stdio" > > command="$(migration_cmd) $(timeout_cmd) $command" > > > > diff --git a/riscv/run b/riscv/run > > index e2f5a922..02fcf0c0 100755 > > --- a/riscv/run > > +++ b/riscv/run > > @@ -11,12 +11,12 @@ fi > > > > # Allow user overrides of some config.mak variables > > mach=$MACHINE_OVERRIDE > > -processor=$PROCESSOR_OVERRIDE > > +qemu_cpu=$QEMU_CPU_OVERRIDE > > firmware=$FIRMWARE_OVERRIDE > > > > -[ "$PROCESSOR" = "$ARCH" ] && PROCESSOR="max" > > +[ -z "$QEMU_CPU" ] && QEMU_CPU="max" > > : "${mach:=virt}" > > -: "${processor:=$PROCESSOR}" > > +: "${qemu_cpu:=$QEMU_CPU}" > > : "${firmware:=$FIRMWARE}" > > [ "$firmware" ] && firmware="-bios $firmware" > > > > @@ -32,7 +32,7 @@ fi > > mach="-machine $mach" > > > > command="$qemu -nodefaults -nographic -serial mon:stdio" > > -command+=" $mach $acc $firmware -cpu $processor " > > +command+=" $mach $acc $firmware -cpu $qemu_cpu " > > command="$(migration_cmd) $(timeout_cmd) $command" > > > > if [ "$UEFI_SHELL_RUN" = "y" ]; then > > diff --git a/configure b/configure > > index 5306bad3..d25bd23e 100755 > > --- a/configure > > +++ b/configure > > @@ -52,6 +52,7 @@ page_size= > > earlycon= > > efi= > > efi_direct= > > +qemu_cpu= > > > > # Enable -Werror by default for git repositories only (i.e. developer builds) > > if [ -e "$srcdir"/.git ]; then > > @@ -69,6 +70,8 @@ usage() { > > --arch=ARCH architecture to compile for ($arch). ARCH can be one of: > > arm, arm64, i386, ppc64, riscv32, riscv64, s390x, x86_64 > > --processor=PROCESSOR processor to compile for ($processor) > > + --qemu-cpu=CPU the CPU model to run on. The default depends on > > + the configuration, usually it is "host" or "max". > > --target=TARGET target platform that the tests will be running on (qemu or > > kvmtool, default is qemu) (arm/arm64 only) > > --cross-prefix=PREFIX cross compiler prefix > > @@ -142,6 +145,9 @@ while [[ $optno -le $argc ]]; do > > --processor) > > processor="$arg" > > ;; > > + --qemu-cpu) > > + qemu_cpu="$arg" > > + ;; > > --target) > > target="$arg" > > ;; > > @@ -464,6 +470,7 @@ ARCH=$arch > > ARCH_NAME=$arch_name > > ARCH_LIBDIR=$arch_libdir > > PROCESSOR=$processor > > +QEMU_CPU=$qemu_cpu > > CC=$cc > > CFLAGS=$cflags > > LD=$cross_prefix$ld > > -- > > 2.48.1 > > > > With the Alex's and Eric's requested changes to the help text, > > Reviewed-by: Andrew Jones <andrew.jones@linux.dev> -- kvm-riscv mailing list kvm-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kvm-riscv ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [kvm-unit-tests PATCH v2 4/5] configure: Add --qemu-cpu option 2025-03-23 11:16 ` Alexandru Elisei @ 2025-03-24 8:19 ` Andrew Jones 2025-03-24 10:41 ` Alexandru Elisei 0 siblings, 1 reply; 26+ messages in thread From: Andrew Jones @ 2025-03-24 8:19 UTC (permalink / raw) To: Alexandru Elisei Cc: Jean-Philippe Brucker, eric.auger, kvmarm, kvm, kvm-riscv, vladimir.murzin On Sun, Mar 23, 2025 at 11:16:19AM +0000, Alexandru Elisei wrote: ... > > > +if [ -z "$qemu_cpu" ]; then > > > + if ( [ "$ACCEL" = "kvm" ] || [ "$ACCEL" = "hvf" ] ) && > > > + ( [ "$HOST" = "aarch64" ] || [ "$HOST" = "arm" ] ); then > > > + qemu_cpu="host" > > > if [ "$ARCH" = "arm" ] && [ "$HOST" = "aarch64" ]; then > > > - processor+=",aarch64=off" > > > + qemu_cpu+=",aarch64=off" > > > fi > > > + elif [ "$ARCH" = "arm64" ]; then > > > + qemu_cpu="cortex-a57" > > > + else > > > + qemu_cpu="cortex-a15" > > > > configure could set this in config.mak as DEFAULT_PROCESSOR, avoiding the > > need to duplicate it here. > > That was my first instinct too, having the default value in config.mak seemed > like the correct solution. > > But the problem with this is that the default -cpu type depends on -accel (set > via unittests.cfg or as an environment variable), host and test architecture > combination. All of these variables are known only at runtime. > > Let's say we have DEFAULT_QEMU_CPU=cortex-a57 in config.mak. If we keep the > above heuristic, arm/run will override it with host,aarch64=off. IMO, having it > in config.mak, but arm/run using it only under certain conditions is worse than > not having it at all. arm/run choosing the default value **all the time** is at > least consistent. I think having 'DEFAULT' in the name implies that it will only be used if there's nothing better, and we don't require everything in config.mak to be used (there's even some s390x-specific stuff in there for all architectures...) > > We could modify the help text for --qemu-cpu to say something like "If left > unset, the $ARCH/run script will choose a best value based on the host system > and test configuration." This is helpful, so we should add it regardless. Thanks, drew -- kvm-riscv mailing list kvm-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kvm-riscv ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [kvm-unit-tests PATCH v2 4/5] configure: Add --qemu-cpu option 2025-03-24 8:19 ` Andrew Jones @ 2025-03-24 10:41 ` Alexandru Elisei 2025-03-24 13:13 ` Andrew Jones 0 siblings, 1 reply; 26+ messages in thread From: Alexandru Elisei @ 2025-03-24 10:41 UTC (permalink / raw) To: Andrew Jones Cc: Jean-Philippe Brucker, eric.auger, kvmarm, kvm, kvm-riscv, vladimir.murzin Hi Drew, On Mon, Mar 24, 2025 at 09:19:27AM +0100, Andrew Jones wrote: > On Sun, Mar 23, 2025 at 11:16:19AM +0000, Alexandru Elisei wrote: > ... > > > > +if [ -z "$qemu_cpu" ]; then > > > > + if ( [ "$ACCEL" = "kvm" ] || [ "$ACCEL" = "hvf" ] ) && > > > > + ( [ "$HOST" = "aarch64" ] || [ "$HOST" = "arm" ] ); then > > > > + qemu_cpu="host" > > > > if [ "$ARCH" = "arm" ] && [ "$HOST" = "aarch64" ]; then > > > > - processor+=",aarch64=off" > > > > + qemu_cpu+=",aarch64=off" > > > > fi > > > > + elif [ "$ARCH" = "arm64" ]; then > > > > + qemu_cpu="cortex-a57" > > > > + else > > > > + qemu_cpu="cortex-a15" > > > > > > configure could set this in config.mak as DEFAULT_PROCESSOR, avoiding the > > > need to duplicate it here. > > > > That was my first instinct too, having the default value in config.mak seemed > > like the correct solution. > > > > But the problem with this is that the default -cpu type depends on -accel (set > > via unittests.cfg or as an environment variable), host and test architecture > > combination. All of these variables are known only at runtime. > > > > Let's say we have DEFAULT_QEMU_CPU=cortex-a57 in config.mak. If we keep the > > above heuristic, arm/run will override it with host,aarch64=off. IMO, having it > > in config.mak, but arm/run using it only under certain conditions is worse than > > not having it at all. arm/run choosing the default value **all the time** is at > > least consistent. > > I think having 'DEFAULT' in the name implies that it will only be used if > there's nothing better, and we don't require everything in config.mak to > be used (there's even some s390x-specific stuff in there for all > architectures...) I'm still leaning towards having the default value and the heuristics for when to pick it in one place ($ARCH/run) as being more convenient, but I can certainly see your point of view. So yeah, up to you :) > > > > > We could modify the help text for --qemu-cpu to say something like "If left > > unset, the $ARCH/run script will choose a best value based on the host system > > and test configuration." > > This is helpful, so we should add it regardless. Thanks, Alex -- kvm-riscv mailing list kvm-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kvm-riscv ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [kvm-unit-tests PATCH v2 4/5] configure: Add --qemu-cpu option 2025-03-24 10:41 ` Alexandru Elisei @ 2025-03-24 13:13 ` Andrew Jones 2025-03-24 15:52 ` Jean-Philippe Brucker 0 siblings, 1 reply; 26+ messages in thread From: Andrew Jones @ 2025-03-24 13:13 UTC (permalink / raw) To: Alexandru Elisei Cc: Jean-Philippe Brucker, eric.auger, kvmarm, kvm, kvm-riscv, vladimir.murzin On Mon, Mar 24, 2025 at 10:41:03AM +0000, Alexandru Elisei wrote: > Hi Drew, > > On Mon, Mar 24, 2025 at 09:19:27AM +0100, Andrew Jones wrote: > > On Sun, Mar 23, 2025 at 11:16:19AM +0000, Alexandru Elisei wrote: > > ... > > > > > +if [ -z "$qemu_cpu" ]; then > > > > > + if ( [ "$ACCEL" = "kvm" ] || [ "$ACCEL" = "hvf" ] ) && > > > > > + ( [ "$HOST" = "aarch64" ] || [ "$HOST" = "arm" ] ); then > > > > > + qemu_cpu="host" > > > > > if [ "$ARCH" = "arm" ] && [ "$HOST" = "aarch64" ]; then > > > > > - processor+=",aarch64=off" > > > > > + qemu_cpu+=",aarch64=off" > > > > > fi > > > > > + elif [ "$ARCH" = "arm64" ]; then > > > > > + qemu_cpu="cortex-a57" > > > > > + else > > > > > + qemu_cpu="cortex-a15" > > > > > > > > configure could set this in config.mak as DEFAULT_PROCESSOR, avoiding the > > > > need to duplicate it here. > > > > > > That was my first instinct too, having the default value in config.mak seemed > > > like the correct solution. > > > > > > But the problem with this is that the default -cpu type depends on -accel (set > > > via unittests.cfg or as an environment variable), host and test architecture > > > combination. All of these variables are known only at runtime. > > > > > > Let's say we have DEFAULT_QEMU_CPU=cortex-a57 in config.mak. If we keep the > > > above heuristic, arm/run will override it with host,aarch64=off. IMO, having it > > > in config.mak, but arm/run using it only under certain conditions is worse than > > > not having it at all. arm/run choosing the default value **all the time** is at > > > least consistent. > > > > I think having 'DEFAULT' in the name implies that it will only be used if > > there's nothing better, and we don't require everything in config.mak to > > be used (there's even some s390x-specific stuff in there for all > > architectures...) > > I'm still leaning towards having the default value and the heuristics for > when to pick it in one place ($ARCH/run) as being more convenient, but I > can certainly see your point of view. I wouldn't mind it only being in $ARCH/run, but this series adds the same logic to $ARCH/run and to ./configure. I think an additional, potentially unused, variable in config.mak is better than code duplication. Thanks, drew -- kvm-riscv mailing list kvm-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kvm-riscv ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [kvm-unit-tests PATCH v2 4/5] configure: Add --qemu-cpu option 2025-03-24 13:13 ` Andrew Jones @ 2025-03-24 15:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 26+ messages in thread From: Jean-Philippe Brucker @ 2025-03-24 15:52 UTC (permalink / raw) To: Andrew Jones Cc: Alexandru Elisei, eric.auger, kvmarm, kvm, kvm-riscv, vladimir.murzin On Mon, Mar 24, 2025 at 02:13:13PM +0100, Andrew Jones wrote: > On Mon, Mar 24, 2025 at 10:41:03AM +0000, Alexandru Elisei wrote: > > Hi Drew, > > > > On Mon, Mar 24, 2025 at 09:19:27AM +0100, Andrew Jones wrote: > > > On Sun, Mar 23, 2025 at 11:16:19AM +0000, Alexandru Elisei wrote: > > > ... > > > > > > +if [ -z "$qemu_cpu" ]; then > > > > > > + if ( [ "$ACCEL" = "kvm" ] || [ "$ACCEL" = "hvf" ] ) && > > > > > > + ( [ "$HOST" = "aarch64" ] || [ "$HOST" = "arm" ] ); then > > > > > > + qemu_cpu="host" > > > > > > if [ "$ARCH" = "arm" ] && [ "$HOST" = "aarch64" ]; then > > > > > > - processor+=",aarch64=off" > > > > > > + qemu_cpu+=",aarch64=off" > > > > > > fi > > > > > > + elif [ "$ARCH" = "arm64" ]; then > > > > > > + qemu_cpu="cortex-a57" > > > > > > + else > > > > > > + qemu_cpu="cortex-a15" > > > > > > > > > > configure could set this in config.mak as DEFAULT_PROCESSOR, avoiding the > > > > > need to duplicate it here. > > > > > > > > That was my first instinct too, having the default value in config.mak seemed > > > > like the correct solution. > > > > > > > > But the problem with this is that the default -cpu type depends on -accel (set > > > > via unittests.cfg or as an environment variable), host and test architecture > > > > combination. All of these variables are known only at runtime. > > > > > > > > Let's say we have DEFAULT_QEMU_CPU=cortex-a57 in config.mak. If we keep the > > > > above heuristic, arm/run will override it with host,aarch64=off. IMO, having it > > > > in config.mak, but arm/run using it only under certain conditions is worse than > > > > not having it at all. arm/run choosing the default value **all the time** is at > > > > least consistent. > > > > > > I think having 'DEFAULT' in the name implies that it will only be used if > > > there's nothing better, and we don't require everything in config.mak to > > > be used (there's even some s390x-specific stuff in there for all > > > architectures...) > > > > I'm still leaning towards having the default value and the heuristics for > > when to pick it in one place ($ARCH/run) as being more convenient, but I > > can certainly see your point of view. > > I wouldn't mind it only being in $ARCH/run, but this series adds the same > logic to $ARCH/run and to ./configure. I think an additional, potentially > unused, variable in config.mak is better than code duplication. I agree with this. However the next version ends up replacing both cortex-* types here with "max", so we won't need to pass these values in the end. The "processor" selection in configure will only be used by the Makefile for the compiler flag. Thanks, Jean -- kvm-riscv mailing list kvm-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kvm-riscv ^ permalink raw reply [flat|nested] 26+ messages in thread
* [kvm-unit-tests PATCH v2 5/5] arm64: Use -cpu max as the default for TCG 2025-03-14 15:49 [kvm-unit-tests PATCH v2 0/5] arm64: Change the default QEMU CPU type to "max" Jean-Philippe Brucker ` (3 preceding siblings ...) 2025-03-14 15:49 ` [kvm-unit-tests PATCH v2 4/5] configure: Add --qemu-cpu option Jean-Philippe Brucker @ 2025-03-14 15:49 ` Jean-Philippe Brucker 2025-03-17 10:13 ` Eric Auger 2025-03-22 11:27 ` Andrew Jones 4 siblings, 2 replies; 26+ messages in thread From: Jean-Philippe Brucker @ 2025-03-14 15:49 UTC (permalink / raw) To: andrew.jones, alexandru.elisei Cc: eric.auger, kvmarm, kvm, kvm-riscv, vladimir.murzin, Jean-Philippe Brucker In order to test all the latest features, default to "max" as the QEMU CPU type on arm64. Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> --- arm/run | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arm/run b/arm/run index 561bafab..84232e28 100755 --- a/arm/run +++ b/arm/run @@ -45,7 +45,7 @@ if [ -z "$qemu_cpu" ]; then qemu_cpu+=",aarch64=off" fi elif [ "$ARCH" = "arm64" ]; then - qemu_cpu="cortex-a57" + qemu_cpu="max" else qemu_cpu="cortex-a15" fi -- 2.48.1 -- kvm-riscv mailing list kvm-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kvm-riscv ^ permalink raw reply related [flat|nested] 26+ messages in thread
* Re: [kvm-unit-tests PATCH v2 5/5] arm64: Use -cpu max as the default for TCG 2025-03-14 15:49 ` [kvm-unit-tests PATCH v2 5/5] arm64: Use -cpu max as the default for TCG Jean-Philippe Brucker @ 2025-03-17 10:13 ` Eric Auger 2025-03-22 11:27 ` Andrew Jones 1 sibling, 0 replies; 26+ messages in thread From: Eric Auger @ 2025-03-17 10:13 UTC (permalink / raw) To: Jean-Philippe Brucker, andrew.jones, alexandru.elisei Cc: kvmarm, kvm, kvm-riscv, vladimir.murzin Hi Jean-Philippe, On 3/14/25 4:49 PM, Jean-Philippe Brucker wrote: > In order to test all the latest features, default to "max" as the QEMU > CPU type on arm64. > > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Reviewed-by: Eric Auger <eric.auger@redhat.com> Eric > --- > arm/run | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/arm/run b/arm/run > index 561bafab..84232e28 100755 > --- a/arm/run > +++ b/arm/run > @@ -45,7 +45,7 @@ if [ -z "$qemu_cpu" ]; then > qemu_cpu+=",aarch64=off" > fi > elif [ "$ARCH" = "arm64" ]; then > - qemu_cpu="cortex-a57" > + qemu_cpu="max" > else > qemu_cpu="cortex-a15" > fi -- kvm-riscv mailing list kvm-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kvm-riscv ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [kvm-unit-tests PATCH v2 5/5] arm64: Use -cpu max as the default for TCG 2025-03-14 15:49 ` [kvm-unit-tests PATCH v2 5/5] arm64: Use -cpu max as the default for TCG Jean-Philippe Brucker 2025-03-17 10:13 ` Eric Auger @ 2025-03-22 11:27 ` Andrew Jones 2025-03-24 15:50 ` Jean-Philippe Brucker 1 sibling, 1 reply; 26+ messages in thread From: Andrew Jones @ 2025-03-22 11:27 UTC (permalink / raw) To: Jean-Philippe Brucker Cc: alexandru.elisei, eric.auger, kvmarm, kvm, kvm-riscv, vladimir.murzin On Fri, Mar 14, 2025 at 03:49:05PM +0000, Jean-Philippe Brucker wrote: > In order to test all the latest features, default to "max" as the QEMU > CPU type on arm64. > > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> > --- > arm/run | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/arm/run b/arm/run > index 561bafab..84232e28 100755 > --- a/arm/run > +++ b/arm/run > @@ -45,7 +45,7 @@ if [ -z "$qemu_cpu" ]; then > qemu_cpu+=",aarch64=off" > fi > elif [ "$ARCH" = "arm64" ]; then > - qemu_cpu="cortex-a57" > + qemu_cpu="max" > else > qemu_cpu="cortex-a15" arm should also be able to default to 'max', right? Thanks, drew > fi > -- > 2.48.1 > > > -- > kvm-riscv mailing list > kvm-riscv@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/kvm-riscv -- kvm-riscv mailing list kvm-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kvm-riscv ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [kvm-unit-tests PATCH v2 5/5] arm64: Use -cpu max as the default for TCG 2025-03-22 11:27 ` Andrew Jones @ 2025-03-24 15:50 ` Jean-Philippe Brucker 2025-03-24 16:33 ` Andrew Jones 0 siblings, 1 reply; 26+ messages in thread From: Jean-Philippe Brucker @ 2025-03-24 15:50 UTC (permalink / raw) To: Andrew Jones Cc: alexandru.elisei, eric.auger, kvmarm, kvm, kvm-riscv, vladimir.murzin On Sat, Mar 22, 2025 at 12:27:56PM +0100, Andrew Jones wrote: > On Fri, Mar 14, 2025 at 03:49:05PM +0000, Jean-Philippe Brucker wrote: > > In order to test all the latest features, default to "max" as the QEMU > > CPU type on arm64. > > > > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> > > --- > > arm/run | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/arm/run b/arm/run > > index 561bafab..84232e28 100755 > > --- a/arm/run > > +++ b/arm/run > > @@ -45,7 +45,7 @@ if [ -z "$qemu_cpu" ]; then > > qemu_cpu+=",aarch64=off" > > fi > > elif [ "$ARCH" = "arm64" ]; then > > - qemu_cpu="cortex-a57" > > + qemu_cpu="max" > > else > > qemu_cpu="cortex-a15" > > arm should also be able to default to 'max', right? Yes I'll change this. I didn't earlier because it failed when I tried it, but it looks like I had QEMU=.../qemu-system-aarch64 in my environment variables, overriding the default qemu-system-arm (32-bit only). "qemu-system-aarch64 -cpu max" doesn't boot 32-bit code, but "qemu-system-aarch64 -cpu cortex-a15" does. Anyway, without explicitly setting the wrong QEMU, "-cpu max" works for 32-bit. Thanks, Jean -- kvm-riscv mailing list kvm-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kvm-riscv ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [kvm-unit-tests PATCH v2 5/5] arm64: Use -cpu max as the default for TCG 2025-03-24 15:50 ` Jean-Philippe Brucker @ 2025-03-24 16:33 ` Andrew Jones 0 siblings, 0 replies; 26+ messages in thread From: Andrew Jones @ 2025-03-24 16:33 UTC (permalink / raw) To: Jean-Philippe Brucker Cc: alexandru.elisei, eric.auger, kvmarm, kvm, kvm-riscv, vladimir.murzin On Mon, Mar 24, 2025 at 03:50:45PM +0000, Jean-Philippe Brucker wrote: > On Sat, Mar 22, 2025 at 12:27:56PM +0100, Andrew Jones wrote: > > On Fri, Mar 14, 2025 at 03:49:05PM +0000, Jean-Philippe Brucker wrote: > > > In order to test all the latest features, default to "max" as the QEMU > > > CPU type on arm64. > > > > > > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> > > > --- > > > arm/run | 2 +- > > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > > > diff --git a/arm/run b/arm/run > > > index 561bafab..84232e28 100755 > > > --- a/arm/run > > > +++ b/arm/run > > > @@ -45,7 +45,7 @@ if [ -z "$qemu_cpu" ]; then > > > qemu_cpu+=",aarch64=off" > > > fi > > > elif [ "$ARCH" = "arm64" ]; then > > > - qemu_cpu="cortex-a57" > > > + qemu_cpu="max" > > > else > > > qemu_cpu="cortex-a15" > > > > arm should also be able to default to 'max', right? > > Yes I'll change this. > > I didn't earlier because it failed when I tried it, but it looks like I > had QEMU=.../qemu-system-aarch64 in my environment variables, overriding > the default qemu-system-arm (32-bit only). "qemu-system-aarch64 -cpu max" > doesn't boot 32-bit code, but "qemu-system-aarch64 -cpu cortex-a15" does. > Anyway, without explicitly setting the wrong QEMU, "-cpu max" works for > 32-bit. Hmm, so now I'm not sure we want to change arm to max. Maybe? People have certainly gotten used to only needing qemu-system-aarch64 to run both arm64 and arm, so this change would break that. It seems like qemu-system-aarch64 should also have a 'max32' cpu model, but it doesn't. So, let's hold off on changing arm to max for now. Users who want it will have to both change their QEMU binary and specify -qemu-cpu. Thanks, drew -- kvm-riscv mailing list kvm-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kvm-riscv ^ permalink raw reply [flat|nested] 26+ messages in thread
end of thread, other threads:[~2025-03-24 16:44 UTC | newest] Thread overview: 26+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2025-03-14 15:49 [kvm-unit-tests PATCH v2 0/5] arm64: Change the default QEMU CPU type to "max" Jean-Philippe Brucker 2025-03-14 15:49 ` [kvm-unit-tests PATCH v2 1/5] configure: arm64: Don't display 'aarch64' as the default architecture Jean-Philippe Brucker 2025-03-17 8:19 ` Eric Auger 2025-03-17 8:27 ` Eric Auger 2025-03-24 15:48 ` Jean-Philippe Brucker 2025-03-22 11:04 ` Andrew Jones 2025-03-14 15:49 ` [kvm-unit-tests PATCH v2 2/5] configure: arm/arm64: Display the correct default processor Jean-Philippe Brucker 2025-03-17 8:30 ` Eric Auger 2025-03-22 11:07 ` Andrew Jones 2025-03-14 15:49 ` [kvm-unit-tests PATCH v2 3/5] arm64: Implement the ./configure --processor option Jean-Philippe Brucker 2025-03-17 8:34 ` Eric Auger 2025-03-14 15:49 ` [kvm-unit-tests PATCH v2 4/5] configure: Add --qemu-cpu option Jean-Philippe Brucker 2025-03-17 8:53 ` Eric Auger 2025-03-20 13:42 ` Alexandru Elisei 2025-03-22 11:25 ` Andrew Jones 2025-03-22 11:26 ` Andrew Jones 2025-03-23 11:16 ` Alexandru Elisei 2025-03-24 8:19 ` Andrew Jones 2025-03-24 10:41 ` Alexandru Elisei 2025-03-24 13:13 ` Andrew Jones 2025-03-24 15:52 ` Jean-Philippe Brucker 2025-03-14 15:49 ` [kvm-unit-tests PATCH v2 5/5] arm64: Use -cpu max as the default for TCG Jean-Philippe Brucker 2025-03-17 10:13 ` Eric Auger 2025-03-22 11:27 ` Andrew Jones 2025-03-24 15:50 ` Jean-Philippe Brucker 2025-03-24 16:33 ` Andrew Jones
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).