* [Qemu-devel] [PATCH v10 01/13] gitignore: Ignore vm test images
2017-09-20 3:25 [Qemu-devel] [PATCH v10 00/13] tests: Add VM based build tests (for non-x86_64 and/or non-Linux) Fam Zheng
@ 2017-09-20 3:25 ` Fam Zheng
2017-09-20 13:13 ` Eric Blake
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 02/13] qemu.py: Add "wait()" method Fam Zheng
` (11 subsequent siblings)
12 siblings, 1 reply; 20+ messages in thread
From: Fam Zheng @ 2017-09-20 3:25 UTC (permalink / raw)
To: qemu-devel
Cc: berrange, Alex Bennée, Fam Zheng,
Philippe Mathieu-Daudé, Peter Maydell, stefanha, Cleber Rosa,
pbonzini, eblake, Kamil Rytarowski
Signed-off-by: Fam Zheng <famz@redhat.com>
---
.gitignore | 1 +
tests/.gitignore | 1 +
2 files changed, 2 insertions(+)
diff --git a/.gitignore b/.gitignore
index cf65316863..40acfcb9e2 100644
--- a/.gitignore
+++ b/.gitignore
@@ -52,6 +52,7 @@
/vscclient
/vhost-user-scsi
/fsdev/virtfs-proxy-helper
+*.tmp
*.[1-9]
*.a
*.aux
diff --git a/tests/.gitignore b/tests/.gitignore
index fed0189a5a..cf6d99c91e 100644
--- a/tests/.gitignore
+++ b/tests/.gitignore
@@ -95,3 +95,4 @@ test-filter-mirror
test-filter-redirector
*-test
qapi-schema/*.test.*
+vm/*.img
--
2.13.5
^ permalink raw reply related [flat|nested] 20+ messages in thread* Re: [Qemu-devel] [PATCH v10 01/13] gitignore: Ignore vm test images
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 01/13] gitignore: Ignore vm test images Fam Zheng
@ 2017-09-20 13:13 ` Eric Blake
0 siblings, 0 replies; 20+ messages in thread
From: Eric Blake @ 2017-09-20 13:13 UTC (permalink / raw)
To: Fam Zheng, qemu-devel
Cc: berrange, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, stefanha, Cleber Rosa, pbonzini, Kamil Rytarowski
[-- Attachment #1: Type: text/plain, Size: 369 bytes --]
On 09/19/2017 10:25 PM, Fam Zheng wrote:
> Signed-off-by: Fam Zheng <famz@redhat.com>
> ---
> .gitignore | 1 +
> tests/.gitignore | 1 +
> 2 files changed, 2 insertions(+)
>
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]
^ permalink raw reply [flat|nested] 20+ messages in thread
* [Qemu-devel] [PATCH v10 02/13] qemu.py: Add "wait()" method
2017-09-20 3:25 [Qemu-devel] [PATCH v10 00/13] tests: Add VM based build tests (for non-x86_64 and/or non-Linux) Fam Zheng
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 01/13] gitignore: Ignore vm test images Fam Zheng
@ 2017-09-20 3:25 ` Fam Zheng
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 03/13] scripts: Add archive-source.sh Fam Zheng
` (10 subsequent siblings)
12 siblings, 0 replies; 20+ messages in thread
From: Fam Zheng @ 2017-09-20 3:25 UTC (permalink / raw)
To: qemu-devel
Cc: berrange, Alex Bennée, Fam Zheng,
Philippe Mathieu-Daudé, Peter Maydell, stefanha, Cleber Rosa,
pbonzini, eblake, Kamil Rytarowski
Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
scripts/qemu.py | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/scripts/qemu.py b/scripts/qemu.py
index 5e02dd8e78..2b5c2accb5 100644
--- a/scripts/qemu.py
+++ b/scripts/qemu.py
@@ -215,6 +215,13 @@ class QEMUMachine(object):
LOG.debug('Output: %r', self._iolog)
raise
+ def wait(self):
+ '''Wait for the VM to power off'''
+ self._popen.wait()
+ self._qmp.close()
+ self._load_io_log()
+ self._post_shutdown()
+
def shutdown(self):
'''Terminate the VM and clean up'''
if self.is_running():
--
2.13.5
^ permalink raw reply related [flat|nested] 20+ messages in thread* [Qemu-devel] [PATCH v10 03/13] scripts: Add archive-source.sh
2017-09-20 3:25 [Qemu-devel] [PATCH v10 00/13] tests: Add VM based build tests (for non-x86_64 and/or non-Linux) Fam Zheng
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 01/13] gitignore: Ignore vm test images Fam Zheng
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 02/13] qemu.py: Add "wait()" method Fam Zheng
@ 2017-09-20 3:25 ` Fam Zheng
2017-09-20 13:20 ` Eric Blake
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 04/13] tests: Add a test key pair Fam Zheng
` (9 subsequent siblings)
12 siblings, 1 reply; 20+ messages in thread
From: Fam Zheng @ 2017-09-20 3:25 UTC (permalink / raw)
To: qemu-devel
Cc: berrange, Alex Bennée, Fam Zheng,
Philippe Mathieu-Daudé, Peter Maydell, stefanha, Cleber Rosa,
pbonzini, eblake, Kamil Rytarowski
Signed-off-by: Fam Zheng <famz@redhat.com>
---
scripts/archive-source.sh | 51 +++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 51 insertions(+)
create mode 100755 scripts/archive-source.sh
diff --git a/scripts/archive-source.sh b/scripts/archive-source.sh
new file mode 100755
index 0000000000..81e559cb8a
--- /dev/null
+++ b/scripts/archive-source.sh
@@ -0,0 +1,51 @@
+#!/bin/bash
+#
+# Author: Fam Zheng <famz@redhat.com>
+#
+# Archive source tree, including submodules. This is created for test code to
+# export the source files, in order to be built in a different enviornment,
+# such as in a docker instance or VM.
+#
+# This code is licensed under the GPL version 2 or later. See
+# the COPYING file in the top-level directory.
+
+error() {
+ echo "$@" >&2
+ exit 1
+}
+
+if test $# -lt 1; then
+ error "Usage: $0 <output tarball>"
+fi
+
+submodules=$(git submodule foreach --recursive --quiet 'echo $name')
+
+if test $? -ne 0; then
+ error "git submodule command failed"
+fi
+
+if test -n "$submodules"; then
+ {
+ git ls-files || error "git ls-files failed"
+ for sm in $submodules; do
+ (cd $sm; git ls-files) | sed "s:^:$sm/:"
+ if test ${PIPESTATUS[0]} -ne 0 -o $? -ne 0; then
+ error "git ls-files in submodule $sm failed"
+ fi
+ done
+ } | grep -x -v $(for sm in $submodules; do echo "-e $sm"; done) > "$1".list
+else
+ git ls-files > "$1".list
+fi
+
+if test $? -ne 0; then
+ error "failed to generate list file"
+fi
+
+tar -cf "$1" -T "$1".list
+status=$?
+rm "$1".list
+if test $statue -ne 0; then
+ error "failed to create tar file"
+fi
+exit 0
--
2.13.5
^ permalink raw reply related [flat|nested] 20+ messages in thread* Re: [Qemu-devel] [PATCH v10 03/13] scripts: Add archive-source.sh
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 03/13] scripts: Add archive-source.sh Fam Zheng
@ 2017-09-20 13:20 ` Eric Blake
2017-09-21 0:33 ` Fam Zheng
2017-09-21 0:45 ` Fam Zheng
0 siblings, 2 replies; 20+ messages in thread
From: Eric Blake @ 2017-09-20 13:20 UTC (permalink / raw)
To: Fam Zheng, qemu-devel
Cc: berrange, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, stefanha, Cleber Rosa, pbonzini, Kamil Rytarowski
[-- Attachment #1: Type: text/plain, Size: 2055 bytes --]
On 09/19/2017 10:25 PM, Fam Zheng wrote:
> Signed-off-by: Fam Zheng <famz@redhat.com>
> ---
> scripts/archive-source.sh | 51 +++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 51 insertions(+)
> create mode 100755 scripts/archive-source.sh
>
> +
> +if test -n "$submodules"; then
> + {
> + git ls-files || error "git ls-files failed"
> + for sm in $submodules; do
> + (cd $sm; git ls-files) | sed "s:^:$sm/:"
> + if test ${PIPESTATUS[0]} -ne 0 -o $? -ne 0; then
This relies on 'test ... -o ...' which is non-portable. It "works"
because there is no possible ambiguity in the contents of $PIPESTATUS
that could cause a different parse of the test arguments, but I tend to
discourage any use of -a/-o inside test on principle. Sadly, writing:
if test ${PIPESTATUS[0]} -ne 0 || test $? -ne 0
has a flaw that $? is no longer what you want, at which point you would
have to introduce a temporary variable. But we're using bash, so you
can instead write this as:
if test "${PIPESTATUS[@]}" != "0 0"; then
> + error "git ls-files in submodule $sm failed"
> + fi
> + done
> + } | grep -x -v $(for sm in $submodules; do echo "-e $sm"; done) > "$1".list
> +else
> + git ls-files > "$1".list
> +fi
At this point, $1.list has been created, even if commands failed...
> +
> +if test $? -ne 0; then
> + error "failed to generate list file"
> +fi
...but this exits without cleanup. If we really want it cleaned no
matter what, it's probably better to do:
trap "status=$?; rm -f "$1".list; exit \$status" 0 1 2 3 15
earlier than anything that can create the file.
> +
> +tar -cf "$1" -T "$1".list
> +status=$?
> +rm "$1".list
> +if test $statue -ne 0; then
Umm, $statue is not the same as $status.
> + error "failed to create tar file"
> +fi
> +exit 0
>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]
^ permalink raw reply [flat|nested] 20+ messages in thread* Re: [Qemu-devel] [PATCH v10 03/13] scripts: Add archive-source.sh
2017-09-20 13:20 ` Eric Blake
@ 2017-09-21 0:33 ` Fam Zheng
2017-09-21 0:45 ` Fam Zheng
1 sibling, 0 replies; 20+ messages in thread
From: Fam Zheng @ 2017-09-21 0:33 UTC (permalink / raw)
To: Eric Blake
Cc: qemu-devel, berrange, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, stefanha, Cleber Rosa,
pbonzini, Kamil Rytarowski
On Wed, 09/20 08:20, Eric Blake wrote:
> On 09/19/2017 10:25 PM, Fam Zheng wrote:
> > Signed-off-by: Fam Zheng <famz@redhat.com>
> > ---
> > scripts/archive-source.sh | 51 +++++++++++++++++++++++++++++++++++++++++++++++
> > 1 file changed, 51 insertions(+)
> > create mode 100755 scripts/archive-source.sh
> >
>
> > +
> > +if test -n "$submodules"; then
> > + {
> > + git ls-files || error "git ls-files failed"
> > + for sm in $submodules; do
> > + (cd $sm; git ls-files) | sed "s:^:$sm/:"
> > + if test ${PIPESTATUS[0]} -ne 0 -o $? -ne 0; then
>
> This relies on 'test ... -o ...' which is non-portable. It "works"
> because there is no possible ambiguity in the contents of $PIPESTATUS
> that could cause a different parse of the test arguments, but I tend to
> discourage any use of -a/-o inside test on principle. Sadly, writing:
>
> if test ${PIPESTATUS[0]} -ne 0 || test $? -ne 0
>
> has a flaw that $? is no longer what you want, at which point you would
> have to introduce a temporary variable. But we're using bash, so you
> can instead write this as:
>
> if test "${PIPESTATUS[@]}" != "0 0"; then
Okay.
>
> > + error "git ls-files in submodule $sm failed"
> > + fi
> > + done
> > + } | grep -x -v $(for sm in $submodules; do echo "-e $sm"; done) > "$1".list
> > +else
> > + git ls-files > "$1".list
> > +fi
>
> At this point, $1.list has been created, even if commands failed...
>
> > +
> > +if test $? -ne 0; then
> > + error "failed to generate list file"
> > +fi
>
> ...but this exits without cleanup. If we really want it cleaned no
> matter what, it's probably better to do:
>
> trap "status=$?; rm -f "$1".list; exit \$status" 0 1 2 3 15
Sounds good, will do.
>
> earlier than anything that can create the file.
>
> > +
> > +tar -cf "$1" -T "$1".list
> > +status=$?
> > +rm "$1".list
> > +if test $statue -ne 0; then
>
> Umm, $statue is not the same as $status.
Oops, will fix.
Fam
^ permalink raw reply [flat|nested] 20+ messages in thread* Re: [Qemu-devel] [PATCH v10 03/13] scripts: Add archive-source.sh
2017-09-20 13:20 ` Eric Blake
2017-09-21 0:33 ` Fam Zheng
@ 2017-09-21 0:45 ` Fam Zheng
2017-09-21 13:36 ` Eric Blake
1 sibling, 1 reply; 20+ messages in thread
From: Fam Zheng @ 2017-09-21 0:45 UTC (permalink / raw)
To: Eric Blake
Cc: qemu-devel, berrange, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, stefanha, Cleber Rosa,
pbonzini, Kamil Rytarowski
On Wed, 09/20 08:20, Eric Blake wrote:
> On 09/19/2017 10:25 PM, Fam Zheng wrote:
> > Signed-off-by: Fam Zheng <famz@redhat.com>
> > ---
> > scripts/archive-source.sh | 51 +++++++++++++++++++++++++++++++++++++++++++++++
> > 1 file changed, 51 insertions(+)
> > create mode 100755 scripts/archive-source.sh
> >
>
> > +
> > +if test -n "$submodules"; then
> > + {
> > + git ls-files || error "git ls-files failed"
> > + for sm in $submodules; do
> > + (cd $sm; git ls-files) | sed "s:^:$sm/:"
> > + if test ${PIPESTATUS[0]} -ne 0 -o $? -ne 0; then
>
> This relies on 'test ... -o ...' which is non-portable. It "works"
> because there is no possible ambiguity in the contents of $PIPESTATUS
> that could cause a different parse of the test arguments, but I tend to
> discourage any use of -a/-o inside test on principle. Sadly, writing:
>
> if test ${PIPESTATUS[0]} -ne 0 || test $? -ne 0
>
> has a flaw that $? is no longer what you want, at which point you would
> have to introduce a temporary variable. But we're using bash, so you
> can instead write this as:
>
> if test "${PIPESTATUS[@]}" != "0 0"; then
Hmm, with exactly this line here I get something like:
./scripts/archive-source.sh: line 36: test: too many arguments
But with
if test "${PIPESTATUS[0]} ${PIPESTATUS[1]}" != "0 0"; then
it seems to work fine. What is the magic here?
Fam
^ permalink raw reply [flat|nested] 20+ messages in thread* Re: [Qemu-devel] [PATCH v10 03/13] scripts: Add archive-source.sh
2017-09-21 0:45 ` Fam Zheng
@ 2017-09-21 13:36 ` Eric Blake
0 siblings, 0 replies; 20+ messages in thread
From: Eric Blake @ 2017-09-21 13:36 UTC (permalink / raw)
To: Fam Zheng
Cc: qemu-devel, berrange, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, stefanha, Cleber Rosa,
pbonzini, Kamil Rytarowski
[-- Attachment #1: Type: text/plain, Size: 904 bytes --]
On 09/20/2017 07:45 PM, Fam Zheng wrote:
>>
>> has a flaw that $? is no longer what you want, at which point you would
>> have to introduce a temporary variable. But we're using bash, so you
>> can instead write this as:
>>
>> if test "${PIPESTATUS[@]}" != "0 0"; then
>
> Hmm, with exactly this line here I get something like:
>
> ./scripts/archive-source.sh: line 36: test: too many arguments
D'oh - [@] causes word splitting (same as "$@"); we want [*] instead:
if test "${PIPESTATUS[*]}" != "0 0";
>
> But with
>
> if test "${PIPESTATUS[0]} ${PIPESTATUS[1]}" != "0 0"; then
or that's a manual way of spelling the auto-array-concatenation that you
get with [*].
>
> it seems to work fine. What is the magic here?
>
> Fam
>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 615 bytes --]
^ permalink raw reply [flat|nested] 20+ messages in thread
* [Qemu-devel] [PATCH v10 04/13] tests: Add a test key pair
2017-09-20 3:25 [Qemu-devel] [PATCH v10 00/13] tests: Add VM based build tests (for non-x86_64 and/or non-Linux) Fam Zheng
` (2 preceding siblings ...)
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 03/13] scripts: Add archive-source.sh Fam Zheng
@ 2017-09-20 3:25 ` Fam Zheng
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 05/13] tests: Add vm test lib Fam Zheng
` (8 subsequent siblings)
12 siblings, 0 replies; 20+ messages in thread
From: Fam Zheng @ 2017-09-20 3:25 UTC (permalink / raw)
To: qemu-devel
Cc: berrange, Alex Bennée, Fam Zheng,
Philippe Mathieu-Daudé, Peter Maydell, stefanha, Cleber Rosa,
pbonzini, eblake, Kamil Rytarowski
This will be used by setup test user ssh.
Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
tests/keys/README | 6 ++++++
tests/keys/id_rsa | 27 +++++++++++++++++++++++++++
tests/keys/id_rsa.pub | 1 +
3 files changed, 34 insertions(+)
create mode 100644 tests/keys/README
create mode 100644 tests/keys/id_rsa
create mode 100644 tests/keys/id_rsa.pub
diff --git a/tests/keys/README b/tests/keys/README
new file mode 100644
index 0000000000..b995268f06
--- /dev/null
+++ b/tests/keys/README
@@ -0,0 +1,6 @@
+This folder contains a well-known ssh key pair used in QEMU tests.
+
+Some guests require the key to exist prior to provisioning the guest; also,
+reusing a pre-built key avoids consuming entropy every time the testsuite is
+run. Because the private key is well-known, care must be taken to use the key
+ONLY in situations that cannot be compromised by external network clients.
diff --git a/tests/keys/id_rsa b/tests/keys/id_rsa
new file mode 100644
index 0000000000..2933eac3db
--- /dev/null
+++ b/tests/keys/id_rsa
@@ -0,0 +1,27 @@
+-----BEGIN RSA PRIVATE KEY-----
+MIIEowIBAAKCAQEAopAuOlmLV6LVHdFBj8/eeOwI9CqguIJPp7eAQSZvOiB4Ag/R
+coEhl/RBbrV5Yc/SmSD4PTpJO/iM10RwliNjDb4a3I8q3sykRJu9c9PI/YsH8WN9
++NH2NjKPtJIcKTu287IM5JYxyB6nDoOzILbTyJ1TDR/xH6qYEfBAyiblggdjcvhA
+RTf93QIn39F/xLypXvT1K2O9BJEsnJ8lEUvB2UXhKo/JTfSeZF8wPBeowaP9EONk
+7b+nuJOWHGg68Ji6wVi62tjwl2Szch6lxIhZBpnV7QNRKMfYHP6eIyF4pusazzZq
+Telsq6xI2ghecWLzb/MF5A+rklsGx2FNuJSAJwIDAQABAoIBAHHi4o/8VZNivz0x
+cWXn8erzKV6tUoWQvW85Lj/2RiwJvSlsnYZDkx5af1CpEE2HA/pFT8PNRqsd+MWC
+7AEy710cVsM4BYerBFYQaYxwzblaoojo88LSjVPw3h5Z0iLM8+IMVd36nwuc9dpE
+R8TecMZ1+U4Tl6BgqkK+9xToZRdPKdjS8L5MoFhGN+xY0vRbbJbGaV9Q0IHxLBkB
+rEBV7T1mUynneCHRUQlJQEwJmKpT8MH3IjsUXlG5YvnuuvcQJSNTaW2iDLxuOKp8
+cxW8+qL88zpb1D5dppoIu6rlrugN0azSq70ruFJQPc/A8GQrDKoGgRQiagxNY3u+
+vHZzXlECgYEA0dKO3gfkSxsDBb94sQwskMScqLhcKhztEa8kPxTx6Yqh+x8/scx3
+XhJyOt669P8U1v8a/2Al+s81oZzzfQSzO1Q7gEwSrgBcRMSIoRBUw9uYcy02ngb/
+j/ng3DGivfJztjjiSJwb46FHkJ2JR8mF2UisC6UMXk3NgFY/3vWQx78CgYEAxlcG
+T3hfSWSmTgKRczMJuHQOX9ULfTBIqwP5VqkkkiavzigGRirzb5lgnmuTSPTpF0LB
+XVPjR2M4q+7gzP0Dca3pocrvLEoxjwIKnCbYKnyyvnUoE9qHv4Kr+vDbgWpa2LXG
+JbLmE7tgTCIp20jOPPT4xuDvlbzQZBJ5qCQSoZkCgYEAgrotSSihlCnAOFSTXbu4
+CHp3IKe8xIBBNENq0eK61kcJpOxTQvOha3sSsJsU4JAM6+cFaxb8kseHIqonCj1j
+bhOM/uJmwQJ4el/4wGDsbxriYOBKpyq1D38gGhDS1IW6kk3erl6VAb36WJ/OaGum
+eTpN9vNeQWM4Jj2WjdNx4QECgYAwTdd6mU1TmZCrJRL5ZG+0nYc2rbMrnQvFoqUi
+BvWiJovggHzur90zy73tNzPaq9Ls2FQxf5G1vCN8NCRJqEEjeYCR59OSDMu/EXc2
+CnvQ9SevHOdS1oEDEjcCWZCMFzPi3XpRih1gptzQDe31uuiHjf3cqcGPzTlPdfRt
+D8P92QKBgC4UaBvIRwREVJsdZzpIzm224Bpe8LOmA7DeTnjlT0b3lkGiBJ36/Q0p
+VhYh/6cjX4/iuIs7gJbGon7B+YPB8scmOi3fj0+nkJAONue1mMfBNkba6qQTc6Y2
+5mEKw2/O7/JpND7ucU3OK9plcw/qnrWDgHxl0Iz95+OzUIIagxne
+-----END RSA PRIVATE KEY-----
diff --git a/tests/keys/id_rsa.pub b/tests/keys/id_rsa.pub
new file mode 100644
index 0000000000..b5ec6d4712
--- /dev/null
+++ b/tests/keys/id_rsa.pub
@@ -0,0 +1 @@
+ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCikC46WYtXotUd0UGPz9547Aj0KqC4gk+nt4BBJm86IHgCD9FygSGX9EFutXlhz9KZIPg9Okk7+IzXRHCWI2MNvhrcjyrezKREm71z08j9iwfxY3340fY2Mo+0khwpO7bzsgzkljHIHqcOg7MgttPInVMNH/EfqpgR8EDKJuWCB2Ny+EBFN/3dAiff0X/EvKle9PUrY70EkSycnyURS8HZReEqj8lN9J5kXzA8F6jBo/0Q42Ttv6e4k5YcaDrwmLrBWLra2PCXZLNyHqXEiFkGmdXtA1Eox9gc/p4jIXim6xrPNmpN6WyrrEjaCF5xYvNv8wXkD6uSWwbHYU24lIAn well-known key for qemu-test, do not use on any machine exposed to an external network
--
2.13.5
^ permalink raw reply related [flat|nested] 20+ messages in thread* [Qemu-devel] [PATCH v10 05/13] tests: Add vm test lib
2017-09-20 3:25 [Qemu-devel] [PATCH v10 00/13] tests: Add VM based build tests (for non-x86_64 and/or non-Linux) Fam Zheng
` (3 preceding siblings ...)
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 04/13] tests: Add a test key pair Fam Zheng
@ 2017-09-20 3:25 ` Fam Zheng
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 06/13] tests: Add ubuntu.i386 image Fam Zheng
` (7 subsequent siblings)
12 siblings, 0 replies; 20+ messages in thread
From: Fam Zheng @ 2017-09-20 3:25 UTC (permalink / raw)
To: qemu-devel
Cc: berrange, Alex Bennée, Fam Zheng,
Philippe Mathieu-Daudé, Peter Maydell, stefanha, Cleber Rosa,
pbonzini, eblake, Kamil Rytarowski
This is the common code to implement a "VM test" to
1) Download and initialize a pre-defined VM that has necessary
dependencies to build QEMU and SSH access.
2) Archive $SRC_PATH to a .tar file.
3) Boot the VM, and pass the source tar file to the guest.
4) SSH into the VM, untar the source tarball, build from the source.
Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
tests/vm/basevm.py | 262 +++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 262 insertions(+)
create mode 100755 tests/vm/basevm.py
diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
new file mode 100755
index 0000000000..3c863bc237
--- /dev/null
+++ b/tests/vm/basevm.py
@@ -0,0 +1,262 @@
+#!/usr/bin/env python
+#
+# VM testing base class
+#
+# Copyright 2017 Red Hat Inc.
+#
+# Authors:
+# Fam Zheng <famz@redhat.com>
+#
+# This code is licensed under the GPL version 2 or later. See
+# the COPYING file in the top-level directory.
+#
+
+import os
+import sys
+import logging
+import time
+import datetime
+sys.path.append(os.path.join(os.path.dirname(__file__), "..", "..", "scripts"))
+from qemu import QEMUMachine
+import subprocess
+import hashlib
+import optparse
+import atexit
+import tempfile
+import shutil
+import multiprocessing
+import traceback
+
+SSH_KEY = open(os.path.join(os.path.dirname(__file__),
+ "..", "keys", "id_rsa")).read()
+SSH_PUB_KEY = open(os.path.join(os.path.dirname(__file__),
+ "..", "keys", "id_rsa.pub")).read()
+
+class BaseVM(object):
+ GUEST_USER = "qemu"
+ GUEST_PASS = "qemupass"
+ ROOT_PASS = "qemupass"
+
+ # The script to run in the guest that builds QEMU
+ BUILD_SCRIPT = ""
+ # The guest name, to be overridden by subclasses
+ name = "#base"
+ def __init__(self, debug=False, vcpus=None):
+ self._guest = None
+ self._tmpdir = os.path.realpath(tempfile.mkdtemp(prefix="vm-test-",
+ suffix=".tmp",
+ dir="."))
+ atexit.register(shutil.rmtree, self._tmpdir)
+
+ self._ssh_key_file = os.path.join(self._tmpdir, "id_rsa")
+ open(self._ssh_key_file, "w").write(SSH_KEY)
+ subprocess.check_call(["chmod", "600", self._ssh_key_file])
+
+ self._ssh_pub_key_file = os.path.join(self._tmpdir, "id_rsa.pub")
+ open(self._ssh_pub_key_file, "w").write(SSH_PUB_KEY)
+
+ self.debug = debug
+ self._stderr = sys.stderr
+ self._devnull = open(os.devnull, "w")
+ if self.debug:
+ self._stdout = sys.stdout
+ else:
+ self._stdout = self._devnull
+ self._args = [ \
+ "-nodefaults", "-m", "2G",
+ "-cpu", "host",
+ "-netdev", "user,id=vnet,hostfwd=:127.0.0.1:0-:22",
+ "-device", "virtio-net-pci,netdev=vnet",
+ "-vnc", "127.0.0.1:0,to=20",
+ "-serial", "file:%s" % os.path.join(self._tmpdir, "serial.out")]
+ if vcpus:
+ self._args += ["-smp", str(vcpus)]
+ if os.access("/dev/kvm", os.R_OK | os.W_OK):
+ self._args += ["-enable-kvm"]
+ else:
+ logging.info("KVM not available, not using -enable-kvm")
+ self._data_args = []
+
+ def _download_with_cache(self, url, sha256sum=None):
+ def check_sha256sum(fname):
+ if not sha256sum:
+ return True
+ checksum = subprocess.check_output(["sha256sum", fname]).split()[0]
+ return sha256sum == checksum
+
+ cache_dir = os.path.expanduser("~/.cache/qemu-vm/download")
+ if not os.path.exists(cache_dir):
+ os.makedirs(cache_dir)
+ fname = os.path.join(cache_dir, hashlib.sha1(url).hexdigest())
+ if os.path.exists(fname) and check_sha256sum(fname):
+ return fname
+ logging.debug("Downloading %s to %s...", url, fname)
+ subprocess.check_call(["wget", "-c", url, "-O", fname + ".download"],
+ stdout=self._stdout, stderr=self._stderr)
+ os.rename(fname + ".download", fname)
+ return fname
+
+ def _ssh_do(self, user, cmd, check, interactive=False):
+ ssh_cmd = ["ssh", "-q",
+ "-o", "StrictHostKeyChecking=no",
+ "-o", "UserKnownHostsFile=" + os.devnull,
+ "-o", "ConnectTimeout=1",
+ "-p", self.ssh_port, "-i", self._ssh_key_file]
+ if interactive:
+ ssh_cmd += ['-t']
+ assert not isinstance(cmd, str)
+ ssh_cmd += ["%s@127.0.0.1" % user] + list(cmd)
+ logging.debug("ssh_cmd: %s", " ".join(ssh_cmd))
+ r = subprocess.call(ssh_cmd,
+ stdin=sys.stdin if interactive else self._devnull,
+ stdout=sys.stdout if interactive else self._stdout,
+ stderr=sys.stderr if interactive else self._stderr)
+ if check and r != 0:
+ raise Exception("SSH command failed: %s" % cmd)
+ return r
+
+ def ssh(self, *cmd):
+ return self._ssh_do(self.GUEST_USER, cmd, False)
+
+ def ssh_interactive(self, *cmd):
+ return self._ssh_do(self.GUEST_USER, cmd, False, True)
+
+ def ssh_root(self, *cmd):
+ return self._ssh_do("root", cmd, False)
+
+ def ssh_check(self, *cmd):
+ self._ssh_do(self.GUEST_USER, cmd, True)
+
+ def ssh_root_check(self, *cmd):
+ self._ssh_do("root", cmd, True)
+
+ def build_image(self, img):
+ raise NotImplementedError
+
+ def add_source_dir(self, src_dir):
+ name = "data-" + hashlib.sha1(src_dir).hexdigest()[:5]
+ tarfile = os.path.join(self._tmpdir, name + ".tar")
+ logging.debug("Creating archive %s for src_dir dir: %s", tarfile, src_dir)
+ subprocess.check_call(["./scripts/archive-source.sh", tarfile],
+ cwd=src_dir, stdin=self._devnull,
+ stdout=self._stdout, stderr=self._stderr)
+ self._data_args += ["-drive",
+ "file=%s,if=none,id=%s,cache=writeback,format=raw" % \
+ (tarfile, name),
+ "-device",
+ "virtio-blk,drive=%s,serial=%s,bootindex=1" % (name, name)]
+
+ def boot(self, img, extra_args=[]):
+ args = self._args + [
+ "-device", "VGA",
+ "-drive", "file=%s,if=none,id=drive0,cache=writeback" % img,
+ "-device", "virtio-blk,drive=drive0,bootindex=0"]
+ args += self._data_args + extra_args
+ logging.debug("QEMU args: %s", " ".join(args))
+ qemu_bin = os.environ.get("QEMU", "qemu-system-x86_64")
+ guest = QEMUMachine(binary=qemu_bin, args=args)
+ try:
+ guest.launch()
+ except:
+ logging.error("Failed to launch QEMU, command line:")
+ logging.error(" ".join([qemu_bin] + args))
+ logging.error("Log:")
+ logging.error(guest.get_log())
+ logging.error("QEMU version >= 2.10 is required")
+ raise
+ atexit.register(self.shutdown)
+ self._guest = guest
+ usernet_info = guest.qmp("human-monitor-command",
+ command_line="info usernet")
+ self.ssh_port = None
+ for l in usernet_info["return"].splitlines():
+ fields = l.split()
+ if "TCP[HOST_FORWARD]" in fields and "22" in fields:
+ self.ssh_port = l.split()[3]
+ if not self.ssh_port:
+ raise Exception("Cannot find ssh port from 'info usernet':\n%s" % \
+ usernet_info)
+
+ def wait_ssh(self, seconds=120):
+ starttime = datetime.datetime.now()
+ guest_up = False
+ while (datetime.datetime.now() - starttime).total_seconds() < seconds:
+ if self.ssh("exit 0") == 0:
+ guest_up = True
+ break
+ time.sleep(1)
+ if not guest_up:
+ raise Exception("Timeout while waiting for guest ssh")
+
+ def shutdown(self):
+ self._guest.shutdown()
+
+ def wait(self):
+ self._guest.wait()
+
+ def qmp(self, *args, **kwargs):
+ return self._guest.qmp(*args, **kwargs)
+
+def parse_args(vm_name):
+ parser = optparse.OptionParser(
+ description="VM test utility. Exit codes: "
+ "0 = success, "
+ "1 = command line error, "
+ "2 = environment initialization failed, "
+ "3 = test command failed")
+ parser.add_option("--debug", "-D", action="store_true",
+ help="enable debug output")
+ parser.add_option("--image", "-i", default="%s.img" % vm_name,
+ help="image file name")
+ parser.add_option("--force", "-f", action="store_true",
+ help="force build image even if image exists")
+ parser.add_option("--jobs", type=int, default=multiprocessing.cpu_count() / 2,
+ help="number of virtual CPUs")
+ parser.add_option("--build-image", "-b", action="store_true",
+ help="build image")
+ parser.add_option("--build-qemu",
+ help="build QEMU from source in guest")
+ parser.add_option("--interactive", "-I", action="store_true",
+ help="Interactively run command")
+ parser.disable_interspersed_args()
+ return parser.parse_args()
+
+def main(vmcls):
+ try:
+ args, argv = parse_args(vmcls.name)
+ if not argv and not args.build_qemu and not args.build_image:
+ print "Nothing to do?"
+ return 1
+ if args.debug:
+ logging.getLogger().setLevel(logging.DEBUG)
+ vm = vmcls(debug=args.debug, vcpus=args.jobs)
+ if args.build_image:
+ if os.path.exists(args.image) and not args.force:
+ sys.stderr.writelines(["Image file exists: %s\n" % args.image,
+ "Use --force option to overwrite\n"])
+ return 1
+ return vm.build_image(args.image)
+ if args.build_qemu:
+ vm.add_source_dir(args.build_qemu)
+ cmd = [vm.BUILD_SCRIPT.format(
+ configure_opts = " ".join(argv),
+ jobs=args.jobs)]
+ else:
+ cmd = argv
+ vm.boot(args.image + ",snapshot=on")
+ vm.wait_ssh()
+ except Exception as e:
+ if isinstance(e, SystemExit) and e.code == 0:
+ return 0
+ sys.stderr.write("Failed to prepare guest environment\n")
+ traceback.print_exc()
+ return 2
+
+ if args.interactive:
+ if vm.ssh_interactive(*cmd) == 0:
+ return 0
+ vm.ssh_interactive()
+ return 3
+ else:
+ if vm.ssh(*cmd) != 0:
+ return 3
--
2.13.5
^ permalink raw reply related [flat|nested] 20+ messages in thread* [Qemu-devel] [PATCH v10 06/13] tests: Add ubuntu.i386 image
2017-09-20 3:25 [Qemu-devel] [PATCH v10 00/13] tests: Add VM based build tests (for non-x86_64 and/or non-Linux) Fam Zheng
` (4 preceding siblings ...)
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 05/13] tests: Add vm test lib Fam Zheng
@ 2017-09-20 3:25 ` Fam Zheng
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 07/13] tests: Add FreeBSD image Fam Zheng
` (6 subsequent siblings)
12 siblings, 0 replies; 20+ messages in thread
From: Fam Zheng @ 2017-09-20 3:25 UTC (permalink / raw)
To: qemu-devel
Cc: berrange, Alex Bennée, Fam Zheng,
Philippe Mathieu-Daudé, Peter Maydell, stefanha, Cleber Rosa,
pbonzini, eblake, Kamil Rytarowski
This adds a 32bit guest.
The official LTS cloud image is downloaded and initialized with
cloud-init.
Signed-off-by: Fam Zheng <famz@redhat.com>
---
tests/vm/ubuntu.i386 | 89 ++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 89 insertions(+)
create mode 100755 tests/vm/ubuntu.i386
diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
new file mode 100755
index 0000000000..fc319e0e6e
--- /dev/null
+++ b/tests/vm/ubuntu.i386
@@ -0,0 +1,89 @@
+#!/usr/bin/env python
+#
+# Ubuntu i386 image
+#
+# Copyright 2017 Red Hat Inc.
+#
+# Authors:
+# Fam Zheng <famz@redhat.com>
+#
+# This code is licensed under the GPL version 2 or later. See
+# the COPYING file in the top-level directory.
+#
+
+import os
+import sys
+import subprocess
+import basevm
+import time
+
+class UbuntuX86VM(basevm.BaseVM):
+ name = "ubuntu.i386"
+ BUILD_SCRIPT = """
+ set -e;
+ cd $(mktemp -d);
+ sudo chmod a+r /dev/vdb;
+ tar -xf /dev/vdb;
+ ./configure {configure_opts};
+ make -j{jobs};
+ make check;
+ """
+
+ def _gen_cloud_init_iso(self):
+ cidir = self._tmpdir
+ mdata = open(os.path.join(cidir, "meta-data"), "w")
+ mdata.writelines(["instance-id: ubuntu-vm-0\n",
+ "local-hostname: ubuntu-guest\n"])
+ mdata.close()
+ udata = open(os.path.join(cidir, "user-data"), "w")
+ udata.writelines(["#cloud-config\n",
+ "chpasswd:\n",
+ " list: |\n",
+ " root:%s\n" % self.ROOT_PASS,
+ " %s:%s\n" % (self.GUEST_USER, self.GUEST_PASS),
+ " expire: False\n",
+ "users:\n",
+ " - name: %s\n" % self.GUEST_USER,
+ " sudo: ALL=(ALL) NOPASSWD:ALL\n",
+ " ssh-authorized-keys:\n",
+ " - %s\n" % basevm.SSH_PUB_KEY,
+ " - name: root\n",
+ " ssh-authorized-keys:\n",
+ " - %s\n" % basevm.SSH_PUB_KEY,
+ "locale: en_US.UTF-8\n"])
+ udata.close()
+ subprocess.check_call(["genisoimage", "-output", "cloud-init.iso",
+ "-volid", "cidata", "-joliet", "-rock",
+ "user-data", "meta-data"],
+ cwd=cidir,
+ stdin=self._devnull, stdout=self._stdout,
+ stderr=self._stdout)
+ return os.path.join(cidir, "cloud-init.iso")
+
+ def build_image(self, img):
+ cimg = self._download_with_cache("https://cloud-images.ubuntu.com/releases/16.04/release/ubuntu-16.04-server-cloudimg-i386-disk1.img")
+ img_tmp = img + ".tmp"
+ subprocess.check_call(["cp", "-f", cimg, img_tmp])
+ subprocess.check_call(["qemu-img", "resize", img_tmp, "50G"])
+ self.boot(img_tmp, extra_args = ["-cdrom", self._gen_cloud_init_iso()])
+ self.wait_ssh()
+ self.ssh_root_check("touch /etc/cloud/cloud-init.disabled")
+ self.ssh_root_check("apt-get update")
+ self.ssh_root_check("apt-get install -y cloud-initramfs-growroot")
+ # Don't check the status in case the guest hang up too quickly
+ self.ssh_root("sync && reboot")
+ time.sleep(5)
+ self.wait_ssh()
+ # The previous update sometimes doesn't survive a reboot, so do it again
+ self.ssh_root_check("apt-get update")
+ self.ssh_root_check("apt-get build-dep -y qemu")
+ self.ssh_root_check("apt-get install -y libfdt-dev")
+ self.ssh_root("poweroff")
+ self.wait()
+ if os.path.exists(img):
+ os.remove(img)
+ os.rename(img_tmp, img)
+ return 0
+
+if __name__ == "__main__":
+ sys.exit(basevm.main(UbuntuX86VM))
--
2.13.5
^ permalink raw reply related [flat|nested] 20+ messages in thread* [Qemu-devel] [PATCH v10 07/13] tests: Add FreeBSD image
2017-09-20 3:25 [Qemu-devel] [PATCH v10 00/13] tests: Add VM based build tests (for non-x86_64 and/or non-Linux) Fam Zheng
` (5 preceding siblings ...)
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 06/13] tests: Add ubuntu.i386 image Fam Zheng
@ 2017-09-20 3:25 ` Fam Zheng
2017-09-20 20:15 ` Eric Blake
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 08/13] tests: Add NetBSD image Fam Zheng
` (5 subsequent siblings)
12 siblings, 1 reply; 20+ messages in thread
From: Fam Zheng @ 2017-09-20 3:25 UTC (permalink / raw)
To: qemu-devel
Cc: berrange, Alex Bennée, Fam Zheng,
Philippe Mathieu-Daudé, Peter Maydell, stefanha, Cleber Rosa,
pbonzini, eblake, Kamil Rytarowski
The image is prepared following instructions as in:
https://wiki.qemu.org/Hosts/BSD
Signed-off-by: Fam Zheng <famz@redhat.com>
---
tests/vm/freebsd | 42 ++++++++++++++++++++++++++++++++++++++++++
1 file changed, 42 insertions(+)
create mode 100755 tests/vm/freebsd
diff --git a/tests/vm/freebsd b/tests/vm/freebsd
new file mode 100755
index 0000000000..039dad8f69
--- /dev/null
+++ b/tests/vm/freebsd
@@ -0,0 +1,42 @@
+#!/usr/bin/env python
+#
+# FreeBSD VM image
+#
+# Copyright 2017 Red Hat Inc.
+#
+# Authors:
+# Fam Zheng <famz@redhat.com>
+#
+# This code is licensed under the GPL version 2 or later. See
+# the COPYING file in the top-level directory.
+#
+
+import os
+import sys
+import subprocess
+import basevm
+
+class FreeBSDVM(basevm.BaseVM):
+ name = "freebsd"
+ BUILD_SCRIPT = """
+ set -e;
+ cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
+ tar -xf /dev/vtbd1;
+ ./configure {configure_opts};
+ gmake -j{jobs};
+ gmake check;
+ """
+
+ def build_image(self, img):
+ cimg = self._download_with_cache("http://download.patchew.org/freebsd-11.1-amd64.img.xz",
+ sha256sum='adcb771549b37bc63826c501f05121a206ed3d9f55f49145908f7e1432d65891')
+ img_tmp_xz = img + ".tmp.xz"
+ img_tmp = img + ".tmp"
+ subprocess.check_call(["cp", "-f", cimg, img_tmp_xz])
+ subprocess.check_call(["xz", "-df", img_tmp_xz])
+ if os.path.exists(img):
+ os.remove(img)
+ os.rename(img_tmp, img)
+
+if __name__ == "__main__":
+ sys.exit(basevm.main(FreeBSDVM))
--
2.13.5
^ permalink raw reply related [flat|nested] 20+ messages in thread* Re: [Qemu-devel] [PATCH v10 07/13] tests: Add FreeBSD image
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 07/13] tests: Add FreeBSD image Fam Zheng
@ 2017-09-20 20:15 ` Eric Blake
0 siblings, 0 replies; 20+ messages in thread
From: Eric Blake @ 2017-09-20 20:15 UTC (permalink / raw)
To: Fam Zheng, qemu-devel
Cc: berrange, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, stefanha, Cleber Rosa, pbonzini, Kamil Rytarowski
[-- Attachment #1: Type: text/plain, Size: 758 bytes --]
On 09/19/2017 10:25 PM, Fam Zheng wrote:
> The image is prepared following instructions as in:
>
> https://wiki.qemu.org/Hosts/BSD
>
> Signed-off-by: Fam Zheng <famz@redhat.com>
> ---
> tests/vm/freebsd | 42 ++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 42 insertions(+)
> create mode 100755 tests/vm/freebsd
>
While testing v10, I got:
$ make vm-build-freebsd
...
VM-BUILD freebsd
./scripts/archive-source.sh: line 48: test: -ne: unary operator expected
ar: warning: creating libfdt/libfdt.a
At least the archive-source.sh warning is part of your series, and worth
fixing.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]
^ permalink raw reply [flat|nested] 20+ messages in thread
* [Qemu-devel] [PATCH v10 08/13] tests: Add NetBSD image
2017-09-20 3:25 [Qemu-devel] [PATCH v10 00/13] tests: Add VM based build tests (for non-x86_64 and/or non-Linux) Fam Zheng
` (6 preceding siblings ...)
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 07/13] tests: Add FreeBSD image Fam Zheng
@ 2017-09-20 3:25 ` Fam Zheng
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 09/13] tests: Add OpenBSD image Fam Zheng
` (4 subsequent siblings)
12 siblings, 0 replies; 20+ messages in thread
From: Fam Zheng @ 2017-09-20 3:25 UTC (permalink / raw)
To: qemu-devel
Cc: berrange, Alex Bennée, Fam Zheng,
Philippe Mathieu-Daudé, Peter Maydell, stefanha, Cleber Rosa,
pbonzini, eblake, Kamil Rytarowski
The image is prepared following instructions as in:
https://wiki.qemu.org/Hosts/BSD
Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Kamil Rytarowski <n54@gmx.com>
---
tests/vm/netbsd | 42 ++++++++++++++++++++++++++++++++++++++++++
1 file changed, 42 insertions(+)
create mode 100755 tests/vm/netbsd
diff --git a/tests/vm/netbsd b/tests/vm/netbsd
new file mode 100755
index 0000000000..3972d8b45c
--- /dev/null
+++ b/tests/vm/netbsd
@@ -0,0 +1,42 @@
+#!/usr/bin/env python
+#
+# NetBSD VM image
+#
+# Copyright 2017 Red Hat Inc.
+#
+# Authors:
+# Fam Zheng <famz@redhat.com>
+#
+# This code is licensed under the GPL version 2 or later. See
+# the COPYING file in the top-level directory.
+#
+
+import os
+import sys
+import subprocess
+import basevm
+
+class NetBSDVM(basevm.BaseVM):
+ name = "netbsd"
+ BUILD_SCRIPT = """
+ set -e;
+ cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
+ tar -xf /dev/rld1a;
+ ./configure --python=python2.7 {configure_opts};
+ gmake -j{jobs};
+ gmake check;
+ """
+
+ def build_image(self, img):
+ cimg = self._download_with_cache("http://download.patchew.org/netbsd-7.1-amd64.img.xz",
+ sha256sum='b633d565b0eac3d02015cd0c81440bd8a7a8df8512615ac1ee05d318be015732')
+ img_tmp_xz = img + ".tmp.xz"
+ img_tmp = img + ".tmp"
+ subprocess.check_call(["cp", "-f", cimg, img_tmp_xz])
+ subprocess.check_call(["xz", "-df", img_tmp_xz])
+ if os.path.exists(img):
+ os.remove(img)
+ os.rename(img_tmp, img)
+
+if __name__ == "__main__":
+ sys.exit(basevm.main(NetBSDVM))
--
2.13.5
^ permalink raw reply related [flat|nested] 20+ messages in thread* [Qemu-devel] [PATCH v10 09/13] tests: Add OpenBSD image
2017-09-20 3:25 [Qemu-devel] [PATCH v10 00/13] tests: Add VM based build tests (for non-x86_64 and/or non-Linux) Fam Zheng
` (7 preceding siblings ...)
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 08/13] tests: Add NetBSD image Fam Zheng
@ 2017-09-20 3:25 ` Fam Zheng
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 10/13] Makefile: Add rules to run vm tests Fam Zheng
` (3 subsequent siblings)
12 siblings, 0 replies; 20+ messages in thread
From: Fam Zheng @ 2017-09-20 3:25 UTC (permalink / raw)
To: qemu-devel
Cc: berrange, Alex Bennée, Fam Zheng,
Philippe Mathieu-Daudé, Peter Maydell, stefanha, Cleber Rosa,
pbonzini, eblake, Kamil Rytarowski
The image is prepared following instructions as in:
https://wiki.qemu.org/Hosts/BSD
Signed-off-by: Fam Zheng <famz@redhat.com>
---
tests/vm/openbsd | 43 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 43 insertions(+)
create mode 100755 tests/vm/openbsd
diff --git a/tests/vm/openbsd b/tests/vm/openbsd
new file mode 100755
index 0000000000..6ae16d97fd
--- /dev/null
+++ b/tests/vm/openbsd
@@ -0,0 +1,43 @@
+#!/usr/bin/env python
+#
+# OpenBSD VM image
+#
+# Copyright 2017 Red Hat Inc.
+#
+# Authors:
+# Fam Zheng <famz@redhat.com>
+#
+# This code is licensed under the GPL version 2 or later. See
+# the COPYING file in the top-level directory.
+#
+
+import os
+import sys
+import subprocess
+import basevm
+
+class OpenBSDVM(basevm.BaseVM):
+ name = "openbsd"
+ BUILD_SCRIPT = """
+ set -e;
+ cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
+ tar -xf /dev/rsd1c;
+ ./configure --cc=x86_64-unknown-openbsd6.1-gcc-4.9.4 --python=python2.7 {configure_opts};
+ gmake -j{jobs};
+ # XXX: "gmake check" seems to always hang or fail
+ #gmake check;
+ """
+
+ def build_image(self, img):
+ cimg = self._download_with_cache("http://download.patchew.org/openbsd-6.1-amd64.img.xz",
+ sha256sum='8c6cedc483e602cfee5e04f0406c64eb99138495e8ca580bc0293bcf0640c1bf')
+ img_tmp_xz = img + ".tmp.xz"
+ img_tmp = img + ".tmp"
+ subprocess.check_call(["cp", "-f", cimg, img_tmp_xz])
+ subprocess.check_call(["xz", "-df", img_tmp_xz])
+ if os.path.exists(img):
+ os.remove(img)
+ os.rename(img_tmp, img)
+
+if __name__ == "__main__":
+ sys.exit(basevm.main(OpenBSDVM))
--
2.13.5
^ permalink raw reply related [flat|nested] 20+ messages in thread* [Qemu-devel] [PATCH v10 10/13] Makefile: Add rules to run vm tests
2017-09-20 3:25 [Qemu-devel] [PATCH v10 00/13] tests: Add VM based build tests (for non-x86_64 and/or non-Linux) Fam Zheng
` (8 preceding siblings ...)
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 09/13] tests: Add OpenBSD image Fam Zheng
@ 2017-09-20 3:25 ` Fam Zheng
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 11/13] MAINTAINERS: Add tests/vm entry Fam Zheng
` (2 subsequent siblings)
12 siblings, 0 replies; 20+ messages in thread
From: Fam Zheng @ 2017-09-20 3:25 UTC (permalink / raw)
To: qemu-devel
Cc: berrange, Alex Bennée, Fam Zheng,
Philippe Mathieu-Daudé, Peter Maydell, stefanha, Cleber Rosa,
pbonzini, eblake, Kamil Rytarowski
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
---
Makefile | 2 ++
configure | 2 +-
tests/vm/Makefile.include | 42 ++++++++++++++++++++++++++++++++++++++++++
3 files changed, 45 insertions(+), 1 deletion(-)
create mode 100644 tests/vm/Makefile.include
diff --git a/Makefile b/Makefile
index eb831b98d1..d1501a07ee 100644
--- a/Makefile
+++ b/Makefile
@@ -811,6 +811,7 @@ endif
-include $(wildcard *.d tests/*.d)
include $(SRC_PATH)/tests/docker/Makefile.include
+include $(SRC_PATH)/tests/vm/Makefile.include
.PHONY: help
help:
@@ -834,6 +835,7 @@ help:
@echo 'Test targets:'
@echo ' check - Run all tests (check-help for details)'
@echo ' docker - Help about targets running tests inside Docker containers'
+ @echo ' vm-test - Help about targets running tests inside VM'
@echo ''
@echo 'Documentation targets:'
@echo ' html info pdf txt'
diff --git a/configure b/configure
index cb0f7ed0ab..dd84bb413a 100755
--- a/configure
+++ b/configure
@@ -6505,7 +6505,7 @@ if test "$ccache_cpp2" = "yes"; then
fi
# build tree in object directory in case the source is not in the current directory
-DIRS="tests tests/tcg tests/tcg/cris tests/tcg/lm32 tests/libqos tests/qapi-schema tests/tcg/xtensa tests/qemu-iotests"
+DIRS="tests tests/tcg tests/tcg/cris tests/tcg/lm32 tests/libqos tests/qapi-schema tests/tcg/xtensa tests/qemu-iotests tests/vm"
DIRS="$DIRS docs docs/interop fsdev"
DIRS="$DIRS pc-bios/optionrom pc-bios/spapr-rtas pc-bios/s390-ccw"
DIRS="$DIRS roms/seabios roms/vgabios"
diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
new file mode 100644
index 0000000000..5daa2a3b73
--- /dev/null
+++ b/tests/vm/Makefile.include
@@ -0,0 +1,42 @@
+# Makefile for VM tests
+
+.PHONY: vm-build-all
+
+IMAGES := ubuntu.i386 freebsd netbsd openbsd
+IMAGE_FILES := $(patsubst %, tests/vm/%.img, $(IMAGES))
+
+.PRECIOUS: $(IMAGE_FILES)
+
+vm-test:
+ @echo "vm-test: Test QEMU in preconfigured virtual machines"
+ @echo
+ @echo " vm-build-ubuntu.i386 - Build QEMU in ubuntu i386 VM"
+ @echo " vm-build-freebsd - Build QEMU in FreeBSD VM"
+ @echo " vm-build-netbsd - Build QEMU in NetBSD VM"
+ @echo " vm-build-openbsd - Build QEMU in OpenBSD VM"
+
+vm-build-all: $(addprefix vm-build-, $(IMAGES))
+
+tests/vm/%.img: $(SRC_PATH)/tests/vm/% \
+ $(SRC_PATH)/tests/vm/basevm.py \
+ $(SRC_PATH)/tests/vm/Makefile.include
+ $(call quiet-command, \
+ $< \
+ $(if $(V)$(DEBUG), --debug) \
+ --image "$@" \
+ --force \
+ --build-image $@, \
+ " VM-IMAGE $*")
+
+
+# Build in VM $(IMAGE)
+vm-build-%: tests/vm/%.img
+ $(call quiet-command, \
+ $(SRC_PATH)/tests/vm/$* \
+ $(if $(V)$(DEBUG), --debug) \
+ $(if $(DEBUG), --interactive) \
+ $(if $(J),--jobs $(J)) \
+ --image "$<" \
+ --build-qemu $(SRC_PATH), \
+ " VM-BUILD $*")
+
--
2.13.5
^ permalink raw reply related [flat|nested] 20+ messages in thread* [Qemu-devel] [PATCH v10 11/13] MAINTAINERS: Add tests/vm entry
2017-09-20 3:25 [Qemu-devel] [PATCH v10 00/13] tests: Add VM based build tests (for non-x86_64 and/or non-Linux) Fam Zheng
` (9 preceding siblings ...)
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 10/13] Makefile: Add rules to run vm tests Fam Zheng
@ 2017-09-20 3:25 ` Fam Zheng
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 12/13] tests: Add README for vm tests Fam Zheng
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 13/13] docker: Use archive-source.py Fam Zheng
12 siblings, 0 replies; 20+ messages in thread
From: Fam Zheng @ 2017-09-20 3:25 UTC (permalink / raw)
To: qemu-devel
Cc: berrange, Alex Bennée, Fam Zheng,
Philippe Mathieu-Daudé, Peter Maydell, stefanha, Cleber Rosa,
pbonzini, eblake, Kamil Rytarowski
Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
---
MAINTAINERS | 1 +
1 file changed, 1 insertion(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index 3d735979ef..0f8f1db332 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1901,6 +1901,7 @@ S: Maintained
F: .travis.yml
F: .shippable.yml
F: tests/docker/
+F: tests/vm/
W: https://travis-ci.org/qemu/qemu
W: https://app.shippable.com/github/qemu/qemu
W: http://patchew.org/QEMU/
--
2.13.5
^ permalink raw reply related [flat|nested] 20+ messages in thread* [Qemu-devel] [PATCH v10 12/13] tests: Add README for vm tests
2017-09-20 3:25 [Qemu-devel] [PATCH v10 00/13] tests: Add VM based build tests (for non-x86_64 and/or non-Linux) Fam Zheng
` (10 preceding siblings ...)
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 11/13] MAINTAINERS: Add tests/vm entry Fam Zheng
@ 2017-09-20 3:25 ` Fam Zheng
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 13/13] docker: Use archive-source.py Fam Zheng
12 siblings, 0 replies; 20+ messages in thread
From: Fam Zheng @ 2017-09-20 3:25 UTC (permalink / raw)
To: qemu-devel
Cc: berrange, Alex Bennée, Fam Zheng,
Philippe Mathieu-Daudé, Peter Maydell, stefanha, Cleber Rosa,
pbonzini, eblake, Kamil Rytarowski
Signed-off-by: Fam Zheng <famz@redhat.com>
---
tests/vm/README | 89 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 89 insertions(+)
create mode 100644 tests/vm/README
diff --git a/tests/vm/README b/tests/vm/README
new file mode 100644
index 0000000000..ae53dce6ee
--- /dev/null
+++ b/tests/vm/README
@@ -0,0 +1,89 @@
+=== VM test suite to run build in guests ===
+
+== Intro ==
+
+This test suite contains scripts that bootstrap various guest images that have
+necessary packages to build QEMU. The basic usage is documented in Makefile
+help which is displayed with "make vm-test".
+
+== Quick start ==
+
+Run "make vm-test" to list available make targets. Invoke a specific make
+command to run build test in an image. For example, "make vm-build-freebsd"
+will build the source tree in the FreeBSD image. The command can be executed
+from either the source tree or the build dir; if the former, ./configure is not
+needed. The command will then generate the test image in ./tests/vm/ under the
+working directory.
+
+Note: images created by the scripts accept a well-known RSA key pair for SSH
+access, so they SHOULD NOT be exposed to external interfaces if you are
+concerned about attackers taking control of the guest and potentially
+exploiting a QEMU security bug to compromise the host.
+
+== QEMU binary ==
+
+By default, qemu-system-x86_64 is searched in $PATH to run the guest. If there
+isn't one, or if it is older than 2.10, the test won't work. In this case,
+provide the QEMU binary in env var: QEMU=/path/to/qemu-2.10+.
+
+== Make jobs ==
+
+The "-j$X" option in the make command line is not propagated into the VM,
+specify "J=$X" to control the make jobs in the guest.
+
+== Debugging ==
+
+Add "DEBUG=1" and/or "V=1" to the make command to allow interactive debugging
+and verbose output. If this is not enough, see the next section.
+
+== Manual invocation ==
+
+Each guest script is an executable script with the same command line options.
+For example to work with the netbsd guest, use $QEMU_SRC/tests/vm/netbsd:
+
+ $ cd $QEMU_SRC/tests/vm
+
+ # To bootstrap the image
+ $ ./netbsd --build-image --image /var/tmp/netbsd.img
+ <...>
+
+ # To run an arbitrary command in guest (the output will not be echoed unless
+ # --debug is added)
+ $ ./netbsd --debug --image /var/tmp/netbsd.img uname -a
+
+ # To build QEMU in guest
+ $ ./netbsd --debug --image /var/tmp/netbsd.img --build-qemu $QEMU_SRC
+
+ # To get to an interactive shell
+ $ ./netbsd --interactive --image /var/tmp/netbsd.img sh
+
+== Adding new guests ==
+
+Please look at existing guest scripts for how to add new guests.
+
+Most importantly, create a subclass of BaseVM and implement build_image()
+method and define BUILD_SCRIPT, then finally call basevm.main() from the
+script's main().
+
+ - Usually in build_image(), a template image is downloaded from a predefined
+ URL. BaseVM._download_with_cache() takes care of the cache and the
+ checksum, so consider using it.
+
+ - Once the image is downloaded, users, SSH server and QEMU build deps should
+ be set up:
+
+ * Root password set to BaseVM.ROOT_PASS
+ * User BaseVM.GUEST_USER is created, and password set to BaseVM.GUEST_PASS
+ * SSH service is enabled and started on boot,
+ $QEMU_SRC/tests/keys/id_rsa.pub is added to ssh's "authorized_keys" file
+ of both root and the normal user
+ * DHCP client service is enabled and started on boot, so that it can
+ automatically configure the virtio-net-pci NIC and communicate with QEMU
+ user net (10.0.2.2)
+ * Necessary packages are installed to untar the source tarball and build
+ QEMU
+
+ - Write a proper BUILD_SCRIPT template, which should be a shell script that
+ untars a raw virtio-blk block device, which is the tarball data blob of the
+ QEMU source tree, then configure/build it. Running "make check" is also
+ recommended.
--
2.13.5
^ permalink raw reply related [flat|nested] 20+ messages in thread* [Qemu-devel] [PATCH v10 13/13] docker: Use archive-source.py
2017-09-20 3:25 [Qemu-devel] [PATCH v10 00/13] tests: Add VM based build tests (for non-x86_64 and/or non-Linux) Fam Zheng
` (11 preceding siblings ...)
2017-09-20 3:25 ` [Qemu-devel] [PATCH v10 12/13] tests: Add README for vm tests Fam Zheng
@ 2017-09-20 3:25 ` Fam Zheng
12 siblings, 0 replies; 20+ messages in thread
From: Fam Zheng @ 2017-09-20 3:25 UTC (permalink / raw)
To: qemu-devel
Cc: berrange, Alex Bennée, Fam Zheng,
Philippe Mathieu-Daudé, Peter Maydell, stefanha, Cleber Rosa,
pbonzini, eblake, Kamil Rytarowski
Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
tests/docker/Makefile.include | 14 ++------------
tests/docker/run | 8 +-------
2 files changed, 3 insertions(+), 19 deletions(-)
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index aa566aa223..7a027d5bd6 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -17,23 +17,13 @@ DOCKER_TOOLS := travis
TESTS ?= %
IMAGES ?= %
-# Make archive from git repo $1 to tar.gz $2
-make-archive-maybe = $(if $(wildcard $1/*), \
- $(call quiet-command, \
- (cd $1; if git diff-index --quiet HEAD -- &>/dev/null; then \
- git archive -1 HEAD --format=tar.gz; \
- else \
- git archive -1 $$(git stash create) --format=tar.gz; \
- fi) > $2, \
- "ARCHIVE","$(notdir $2)"))
-
CUR_TIME := $(shell date +%Y-%m-%d-%H.%M.%S.$$$$)
DOCKER_SRC_COPY := docker-src.$(CUR_TIME)
$(DOCKER_SRC_COPY):
@mkdir $@
- $(call make-archive-maybe, $(SRC_PATH), $@/qemu.tgz)
- $(call make-archive-maybe, $(SRC_PATH)/dtc, $@/dtc.tgz)
+ $(call quiet-command, $(SRC_PATH)/scripts/archive-source.sh $@/qemu.tar, \
+ "GEN", "$@/qemu.tar")
$(call quiet-command, cp $(SRC_PATH)/tests/docker/run $@/run, \
"COPY","RUNNER")
diff --git a/tests/docker/run b/tests/docker/run
index c1e4513bce..9eb9165f76 100755
--- a/tests/docker/run
+++ b/tests/docker/run
@@ -32,13 +32,7 @@ export TEST_DIR=/tmp/qemu-test
mkdir -p $TEST_DIR/{src,build,install}
# Extract the source tarballs
-tar -C $TEST_DIR/src -xzf $BASE/qemu.tgz
-for p in dtc pixman; do
- if test -f $BASE/$p.tgz; then
- tar -C $TEST_DIR/src/$p -xzf $BASE/$p.tgz
- export FEATURES="$FEATURES $p"
- fi
-done
+tar -C $TEST_DIR/src -xf $BASE/qemu.tar
if test -n "$SHOW_ENV"; then
if test -f /packages.txt; then
--
2.13.5
^ permalink raw reply related [flat|nested] 20+ messages in thread