[kvm-unit-tests PATCH 0/3] tcg is becoming a first class citizen

[kvm-unit-tests PATCH 0/3] tcg is becoming a first class citizen

[Andrew Jones]

At least on the ARM side of things we're making sure unit tests can
be used for [MT]TCG, as well as for KVM. This series adds support to
unittests.cfg to allow us to specify kvm vs. tcg, as not all tests
will be able to run on both. The first patch is a repost of one of
Alex's pending patches, as the 3rd patch depends on it. The 3rd patch
obsoletes/replaces Alex's pending "arm/run: introduce usingkvm var
and use it" patch.

Alex Bennée (1):
  configure: emit HOST=$host to config.mak

Andrew Jones (2):
  run_tests: pass test name to run script
  arm/run: use ACCEL to choose between kvm and tcg

 arm/run                | 36 ++++++++++++++++++++++++++++++------
 arm/unittests.cfg      |  4 +++-
 configure              |  2 ++
 run_tests.sh           |  3 ++-
 scripts/functions.bash |  8 ++++++--
 5 files changed, 43 insertions(+), 10 deletions(-)

-- 
2.4.3

[kvm-unit-tests PATCH 1/3] configure: emit HOST=$host to config.mak

[Andrew Jones]

From: Alex Bennée <alex.bennee@linaro.org>

From: Alex Bennée <alex.bennee@linaro.org>

This is useful information for the run scripts to know, especially if
they want to drop to using TCG.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Andrew Jones <drjones@redhat.com>
---
 configure | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/configure b/configure
index b2ad32a3e3a52..078b70ce096a6 100755
--- a/configure
+++ b/configure
@@ -7,6 +7,7 @@ ld=ld
 objcopy=objcopy
 ar=ar
 arch=`uname -m | sed -e s/i.86/i386/ | sed -e 's/arm.*/arm/'`
+host=$arch
 cross_prefix=
 
 usage() {
@@ -122,6 +123,7 @@ ln -s $asm lib/asm
 cat <<EOF > config.mak
 PREFIX=$prefix
 KERNELDIR=$(readlink -f $kerneldir)
+HOST=$host
 ARCH=$arch
 ARCH_NAME=$arch_name
 PROCESSOR=$processor
-- 
2.4.3

[kvm-unit-tests PATCH 2/3] run_tests: pass test name to run script

[Andrew Jones]

With this $TEST_DIR/run can output test specific error messages.

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 run_tests.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/run_tests.sh b/run_tests.sh
index ebb7e9fe6fdfc..80b87823c3358 100755
--- a/run_tests.sh
+++ b/run_tests.sh
@@ -46,7 +46,7 @@ function run()
         fi
     done
 
-    cmdline="./$TEST_DIR-run $kernel -smp $smp $opts"
+    cmdline="TESTNAME=$testname ./$TEST_DIR-run $kernel -smp $smp $opts"
     if [ $verbose != 0 ]; then
         echo $cmdline
     fi
-- 
2.4.3

[kvm-unit-tests PATCH 3/3] arm/run: use ACCEL to choose between kvm and tcg

[Andrew Jones]

Inspired by a patch by Alex Bennée. This version uses a new
unittests.cfg variable and includes support for DRYRUN.

Signed-off-by: Andrew Jones <drjones@redhat.com>
---

Another difference with Alex's patch is we no longer output
'Running with TCG', as I don't think it's necessary. The
command line captures that, and the whole point of the patch
is to silence the '"kvm" accelerator not found.' messages
anyway.

 arm/run                | 36 ++++++++++++++++++++++++++++++------
 arm/unittests.cfg      |  4 +++-
 run_tests.sh           |  3 ++-
 scripts/functions.bash |  8 ++++++--
 4 files changed, 41 insertions(+), 10 deletions(-)

diff --git a/arm/run b/arm/run
index 8cc2fa2571967..1208a22b776d9 100755
--- a/arm/run
+++ b/arm/run
@@ -7,6 +7,35 @@ fi
 source config.mak
 processor="$PROCESSOR"
 
+if [ -c /dev/kvm ]; then
+	if [ "$HOST" = "arm" ] && [ "$ARCH" = "arm" ]; then
+		kvm_available=yes
+	elif [ "$HOST" = "aarch64" ]; then
+		kvm_available=yes
+	fi
+fi
+
+if [ "$kvm_available" != "yes" ] && [ "$ACCEL" = "kvm" ]; then
+	printf "skip $TESTNAME (kvm only)\n\n"
+	exit 2
+fi
+
+if [ "$kvm_available" != "yes" ] || [ "$ACCEL" = "tcg" ]; then
+	accel=",accel=tcg"
+else
+	accel=",accel=kvm"
+	if [ "$ARCH" = "arm64" ]; then
+		# arm64 must use '-cpu host' with kvm
+		processor="host"
+	fi
+fi
+
+if [ "$DRYRUN" = "yes" ]; then
+	# Output kvm with tcg fallback for dryrun, since the
+	# command line we output may get used elsewhere.
+	accel=",accel=kvm:tcg"
+fi
+
 qemu="${QEMU:-qemu-system-$ARCH_NAME}"
 qpath=$(which $qemu 2>/dev/null)
 
@@ -33,15 +62,10 @@ if $qemu $M -chardev testdev,id=id -initrd . 2>&1 \
 	exit 2
 fi
 
-M='-machine virt,accel=kvm:tcg'
 chr_testdev='-device virtio-serial-device'
 chr_testdev+=' -device virtconsole,chardev=ctd -chardev testdev,id=ctd'
 
-# arm64 must use '-cpu host' with kvm
-if [ "$(arch)" = "aarch64" ] && [ "$ARCH" = "arm64" ] && [ -c /dev/kvm ]; then
-	processor="host"
-fi
-
+M+=$accel
 command="$qemu $M -cpu $processor $chr_testdev"
 command+=" -display none -serial stdio -kernel"
 echo $command "$@"
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index e068a0cdd9c1f..243c13301811b 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -3,8 +3,10 @@
 # file = foo.flat # Name of the flat file to be used
 # smp  = 2        # Number of processors the VM will use during this test
 # extra_params = -append <params...> # Additional parameters used
-# arch = arm/arm64                   # Only if test case is specific to one
+# arch = arm|arm64                   # Only if test case is specific to one
 # groups = group1 group2 # Used to identify test cases with run_tests -g ...
+# accel = kvm|tcg # Optionally specify if test must run with kvm or tcg.
+#                 # If not specified, then kvm will be used when available.
 
 #
 # Test that the configured number of processors (smp = <num>), and
diff --git a/run_tests.sh b/run_tests.sh
index 80b87823c3358..b1b4c541ecaea 100755
--- a/run_tests.sh
+++ b/run_tests.sh
@@ -20,6 +20,7 @@ function run()
     local opts="$5"
     local arch="$6"
     local check="$7"
+    local accel="$8"
 
     if [ -z "$testname" ]; then
         return
@@ -46,7 +47,7 @@ function run()
         fi
     done
 
-    cmdline="TESTNAME=$testname ./$TEST_DIR-run $kernel -smp $smp $opts"
+    cmdline="TESTNAME=$testname ACCEL=$accel ./$TEST_DIR-run $kernel -smp $smp $opts"
     if [ $verbose != 0 ]; then
         echo $cmdline
     fi
diff --git a/scripts/functions.bash b/scripts/functions.bash
index 7ed5a517250bc..f13fe6f88f23d 100644
--- a/scripts/functions.bash
+++ b/scripts/functions.bash
@@ -10,12 +10,13 @@ function for_each_unittest()
 	local groups
 	local arch
 	local check
+	local accel
 
 	exec {fd}<"$unittests"
 
 	while read -u $fd line; do
 		if [[ "$line" =~ ^\[(.*)\]$ ]]; then
-			"$cmd" "$testname" "$groups" "$smp" "$kernel" "$opts" "$arch" "$check"
+			"$cmd" "$testname" "$groups" "$smp" "$kernel" "$opts" "$arch" "$check" "$accel"
 			testname=${BASH_REMATCH[1]}
 			smp=1
 			kernel=""
@@ -23,6 +24,7 @@ function for_each_unittest()
 			groups=""
 			arch=""
 			check=""
+			accel=""
 		elif [[ $line =~ ^file\ *=\ *(.*)$ ]]; then
 			kernel=$TEST_DIR/${BASH_REMATCH[1]}
 		elif [[ $line =~ ^smp\ *=\ *(.*)$ ]]; then
@@ -35,8 +37,10 @@ function for_each_unittest()
 			arch=${BASH_REMATCH[1]}
 		elif [[ $line =~ ^check\ *=\ *(.*)$ ]]; then
 			check=${BASH_REMATCH[1]}
+		elif [[ $line =~ ^accel\ *=\ *(.*)$ ]]; then
+			accel=${BASH_REMATCH[1]}
 		fi
 	done
-	"$cmd" "$testname" "$groups" "$smp" "$kernel" "$opts" "$arch" "$check"
+	"$cmd" "$testname" "$groups" "$smp" "$kernel" "$opts" "$arch" "$check" "$accel"
 	exec {fd}<&-
 }
-- 
2.4.3

Re: [kvm-unit-tests PATCH 3/3] arm/run: use ACCEL to choose between kvm and tcg

[Andrew Jones]

On Mon, Aug 03, 2015 at 07:51:51PM +0200, Andrew Jones wrote:
> Inspired by a patch by Alex Bennée. This version uses a new
> unittests.cfg variable and includes support for DRYRUN.
> 
> Signed-off-by: Andrew Jones <drjones@redhat.com>
> ---
> 
> Another difference with Alex's patch is we no longer output
> 'Running with TCG', as I don't think it's necessary. The
> command line captures that, and the whole point of the patch
> is to silence the '"kvm" accelerator not found.' messages
> anyway.

I forgot to actually test mkstandlone... The dryrun stuff
in this patch isn't sufficient for the new unittests variable.
I've worked up a v2 and will post the series again shortly.

drew

> 
>  arm/run                | 36 ++++++++++++++++++++++++++++++------
>  arm/unittests.cfg      |  4 +++-
>  run_tests.sh           |  3 ++-
>  scripts/functions.bash |  8 ++++++--
>  4 files changed, 41 insertions(+), 10 deletions(-)
> 
> diff --git a/arm/run b/arm/run
> index 8cc2fa2571967..1208a22b776d9 100755
> --- a/arm/run
> +++ b/arm/run
> @@ -7,6 +7,35 @@ fi
>  source config.mak
>  processor="$PROCESSOR"
>  
> +if [ -c /dev/kvm ]; then
> +	if [ "$HOST" = "arm" ] && [ "$ARCH" = "arm" ]; then
> +		kvm_available=yes
> +	elif [ "$HOST" = "aarch64" ]; then
> +		kvm_available=yes
> +	fi
> +fi
> +
> +if [ "$kvm_available" != "yes" ] && [ "$ACCEL" = "kvm" ]; then
> +	printf "skip $TESTNAME (kvm only)\n\n"
> +	exit 2
> +fi
> +
> +if [ "$kvm_available" != "yes" ] || [ "$ACCEL" = "tcg" ]; then
> +	accel=",accel=tcg"
> +else
> +	accel=",accel=kvm"
> +	if [ "$ARCH" = "arm64" ]; then
> +		# arm64 must use '-cpu host' with kvm
> +		processor="host"
> +	fi
> +fi
> +
> +if [ "$DRYRUN" = "yes" ]; then
> +	# Output kvm with tcg fallback for dryrun, since the
> +	# command line we output may get used elsewhere.
> +	accel=",accel=kvm:tcg"
> +fi
> +
>  qemu="${QEMU:-qemu-system-$ARCH_NAME}"
>  qpath=$(which $qemu 2>/dev/null)
>  
> @@ -33,15 +62,10 @@ if $qemu $M -chardev testdev,id=id -initrd . 2>&1 \
>  	exit 2
>  fi
>  
> -M='-machine virt,accel=kvm:tcg'
>  chr_testdev='-device virtio-serial-device'
>  chr_testdev+=' -device virtconsole,chardev=ctd -chardev testdev,id=ctd'
>  
> -# arm64 must use '-cpu host' with kvm
> -if [ "$(arch)" = "aarch64" ] && [ "$ARCH" = "arm64" ] && [ -c /dev/kvm ]; then
> -	processor="host"
> -fi
> -
> +M+=$accel
>  command="$qemu $M -cpu $processor $chr_testdev"
>  command+=" -display none -serial stdio -kernel"
>  echo $command "$@"
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index e068a0cdd9c1f..243c13301811b 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -3,8 +3,10 @@
>  # file = foo.flat # Name of the flat file to be used
>  # smp  = 2        # Number of processors the VM will use during this test
>  # extra_params = -append <params...> # Additional parameters used
> -# arch = arm/arm64                   # Only if test case is specific to one
> +# arch = arm|arm64                   # Only if test case is specific to one
>  # groups = group1 group2 # Used to identify test cases with run_tests -g ...
> +# accel = kvm|tcg # Optionally specify if test must run with kvm or tcg.
> +#                 # If not specified, then kvm will be used when available.
>  
>  #
>  # Test that the configured number of processors (smp = <num>), and
> diff --git a/run_tests.sh b/run_tests.sh
> index 80b87823c3358..b1b4c541ecaea 100755
> --- a/run_tests.sh
> +++ b/run_tests.sh
> @@ -20,6 +20,7 @@ function run()
>      local opts="$5"
>      local arch="$6"
>      local check="$7"
> +    local accel="$8"
>  
>      if [ -z "$testname" ]; then
>          return
> @@ -46,7 +47,7 @@ function run()
>          fi
>      done
>  
> -    cmdline="TESTNAME=$testname ./$TEST_DIR-run $kernel -smp $smp $opts"
> +    cmdline="TESTNAME=$testname ACCEL=$accel ./$TEST_DIR-run $kernel -smp $smp $opts"
>      if [ $verbose != 0 ]; then
>          echo $cmdline
>      fi
> diff --git a/scripts/functions.bash b/scripts/functions.bash
> index 7ed5a517250bc..f13fe6f88f23d 100644
> --- a/scripts/functions.bash
> +++ b/scripts/functions.bash
> @@ -10,12 +10,13 @@ function for_each_unittest()
>  	local groups
>  	local arch
>  	local check
> +	local accel
>  
>  	exec {fd}<"$unittests"
>  
>  	while read -u $fd line; do
>  		if [[ "$line" =~ ^\[(.*)\]$ ]]; then
> -			"$cmd" "$testname" "$groups" "$smp" "$kernel" "$opts" "$arch" "$check"
> +			"$cmd" "$testname" "$groups" "$smp" "$kernel" "$opts" "$arch" "$check" "$accel"
>  			testname=${BASH_REMATCH[1]}
>  			smp=1
>  			kernel=""
> @@ -23,6 +24,7 @@ function for_each_unittest()
>  			groups=""
>  			arch=""
>  			check=""
> +			accel=""
>  		elif [[ $line =~ ^file\ *=\ *(.*)$ ]]; then
>  			kernel=$TEST_DIR/${BASH_REMATCH[1]}
>  		elif [[ $line =~ ^smp\ *=\ *(.*)$ ]]; then
> @@ -35,8 +37,10 @@ function for_each_unittest()
>  			arch=${BASH_REMATCH[1]}
>  		elif [[ $line =~ ^check\ *=\ *(.*)$ ]]; then
>  			check=${BASH_REMATCH[1]}
> +		elif [[ $line =~ ^accel\ *=\ *(.*)$ ]]; then
> +			accel=${BASH_REMATCH[1]}
>  		fi
>  	done
> -	"$cmd" "$testname" "$groups" "$smp" "$kernel" "$opts" "$arch" "$check"
> +	"$cmd" "$testname" "$groups" "$smp" "$kernel" "$opts" "$arch" "$check" "$accel"
>  	exec {fd}<&-
>  }
> -- 
> 2.4.3
> 
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [kvm-unit-tests PATCH 3/3] arm/run: use ACCEL to choose between kvm and tcg

[Alex Bennée]

Andrew Jones <drjones@redhat.com> writes:

> Inspired by a patch by Alex Bennée. This version uses a new
> unittests.cfg variable and includes support for DRYRUN.
>
> Signed-off-by: Andrew Jones <drjones@redhat.com>
> ---
>
> Another difference with Alex's patch is we no longer output
> 'Running with TCG', as I don't think it's necessary. The
> command line captures that, and the whole point of the patch
> is to silence the '"kvm" accelerator not found.' messages
> anyway.

Yeah that makes sense.

>
>  arm/run                | 36 ++++++++++++++++++++++++++++++------
>  arm/unittests.cfg      |  4 +++-
>  run_tests.sh           |  3 ++-
>  scripts/functions.bash |  8 ++++++--
>  4 files changed, 41 insertions(+), 10 deletions(-)
<snip>

I see your re-posting so I'll do a full review then. Is there a branch I
can re-base my stuff on top of for now?

-- 
Alex Bennée

Re: [kvm-unit-tests PATCH 3/3] arm/run: use ACCEL to choose between kvm and tcg

[Andrew Jones]

On Thu, Aug 06, 2015 at 07:29:00PM +0100, Alex Bennée wrote:
> 
> Andrew Jones <drjones@redhat.com> writes:
> 
> > Inspired by a patch by Alex Bennée. This version uses a new
> > unittests.cfg variable and includes support for DRYRUN.
> >
> > Signed-off-by: Andrew Jones <drjones@redhat.com>
> > ---
> >
> > Another difference with Alex's patch is we no longer output
> > 'Running with TCG', as I don't think it's necessary. The
> > command line captures that, and the whole point of the patch
> > is to silence the '"kvm" accelerator not found.' messages
> > anyway.
> 
> Yeah that makes sense.
> 
> >
> >  arm/run                | 36 ++++++++++++++++++++++++++++++------
> >  arm/unittests.cfg      |  4 +++-
> >  run_tests.sh           |  3 ++-
> >  scripts/functions.bash |  8 ++++++--
> >  4 files changed, 41 insertions(+), 10 deletions(-)
> <snip>
> 
> I see your re-posting so I'll do a full review then. Is there a branch I
> can re-base my stuff on top of for now?

OK, https://github.com/rhdrjones/kvm-unit-tests/commits/staging has been
updated. BTW, that branch is anything but stable. It'll get a rebase
anytime upstream master or next is updated, and iff neither of those are
a sufficient base (so not only is it unstable, but it could go stale...)
Hopefully atm it's useful for you to rebase on though.

Thanks,
drew

> 
> -- 
> Alex Bennée
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[kvm-unit-tests PATCH 0/3] tcg is becoming a first class citizen

[Andrew Jones]

At least on the ARM side of things we're making sure unit tests can
be used for [MT]TCG, as well as for KVM. This series adds support to
unittests.cfg to allow us to specify kvm vs. tcg, as not all tests
will be able (should/need) to run on both. The first patch is a repost
of one of Alex's pending patches, as the 3rd patch depends on it. The
3rd patch obsoletes/replaces Alex's pending "arm/run: introduce
usingkvm var and use it" patch.

Alex Bennée (1):
  configure: emit HOST=$host to config.mak

Andrew Jones (2):
  run_tests: pass test name to run script
  arm/run: use ACCEL to choose between kvm and tcg

 arm/run                 | 43 +++++++++++++++++++++++++++++++++++++------
 arm/unittests.cfg       |  4 +++-
 configure               |  2 ++
 run_tests.sh            |  3 ++-
 scripts/functions.bash  |  8 ++++++--
 scripts/mkstandalone.sh | 15 +++++++++++----
 6 files changed, 61 insertions(+), 14 deletions(-)

-- 
2.4.3

Re: [kvm-unit-tests PATCH 0/3] tcg is becoming a first class citizen

[Andrew Jones]

Crap. Forgot -v2 on my format-patch command line...
Paolo, should I repost?

drew

On Tue, Aug 04, 2015 at 09:25:52AM +0200, Andrew Jones wrote:
> At least on the ARM side of things we're making sure unit tests can
> be used for [MT]TCG, as well as for KVM. This series adds support to
> unittests.cfg to allow us to specify kvm vs. tcg, as not all tests
> will be able (should/need) to run on both. The first patch is a repost
> of one of Alex's pending patches, as the 3rd patch depends on it. The
> 3rd patch obsoletes/replaces Alex's pending "arm/run: introduce
> usingkvm var and use it" patch.
> 
> Alex Bennée (1):
>   configure: emit HOST=$host to config.mak
> 
> Andrew Jones (2):
>   run_tests: pass test name to run script
>   arm/run: use ACCEL to choose between kvm and tcg
> 
>  arm/run                 | 43 +++++++++++++++++++++++++++++++++++++------
>  arm/unittests.cfg       |  4 +++-
>  configure               |  2 ++
>  run_tests.sh            |  3 ++-
>  scripts/functions.bash  |  8 ++++++--
>  scripts/mkstandalone.sh | 15 +++++++++++----
>  6 files changed, 61 insertions(+), 14 deletions(-)
> 
> -- 
> 2.4.3
> 
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html