[KVM-AUTOTEST PATCH 1/2] KVM test: add shutdown test

[KVM-AUTOTEST PATCH 1/2] KVM test: add shutdown test

[Michael Goldish]

The shutdown test logs into a VM and sends a shutdown command.
It serves two purposes:
- Test KVM's ability to shut down.
- Clean up after the other tests:
Currently VMs of the last test remain alive when Autotest finishes running.
When one guest finishes testing and another begins, the VM is automatically
shut down by the preprocessor because the QEMU command required for the next
guest differs from that of the guest that just finished.  In the case of the
final guest this doesn't happen because no guest follows it, so the preprocessor
must be explicitly instructed to kill the VM.
However, we have no easy way of knowing which test runs last because the user
usually selects a subset of the tests/guests.
The addition of a shutdown test can be a decent solution to this small problem:
by convention the shutdown test will always be the last to run, and if users
wish to clean up after the tests, they must select the shutdown test.

Note: it is beneficial to allow users to leave the VMs of the last test running
because it saves time when developing and testing tests. A test writer can run
the test once on a VM, and when the test exits, make some modifications to its
code and re-run it on the same living VM, and repeat this procedure without
having to shutdown/boot the VM every time.

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm.py       |    1 +
 client/tests/kvm/kvm_tests.py |   37 +++++++++++++++++++++++++++++++++++++
 2 files changed, 38 insertions(+), 0 deletions(-)

diff --git a/client/tests/kvm/kvm.py b/client/tests/kvm/kvm.py
index 9428162..aa727da 100644
--- a/client/tests/kvm/kvm.py
+++ b/client/tests/kvm/kvm.py
@@ -48,6 +48,7 @@ class kvm(test.test):
                 "steps":        test_routine("kvm_guest_wizard", "run_steps"),
                 "stepmaker":    test_routine("stepmaker", "run_stepmaker"),
                 "boot":         test_routine("kvm_tests", "run_boot"),
+                "shutdown":     test_routine("kvm_tests", "run_shutdown"),
                 "migration":    test_routine("kvm_tests", "run_migration"),
                 "yum_update":   test_routine("kvm_tests", "run_yum_update"),
                 "autotest":     test_routine("kvm_tests", "run_autotest"),
diff --git a/client/tests/kvm/kvm_tests.py b/client/tests/kvm/kvm_tests.py
index ffe9116..4c9653f 100644
--- a/client/tests/kvm/kvm_tests.py
+++ b/client/tests/kvm/kvm_tests.py
@@ -57,6 +57,43 @@ def run_boot(test, params, env):
     session.close()
 
 
+def run_shutdown(test, params, env):
+    """
+    KVM shutdown test:
+    1) Log into a guest
+    2) Send a shutdown command to the guest
+    3) Wait until it's down
+
+    @param test: kvm test object
+    @param params: Dictionary with the test parameters
+    @param env: Dictionary with test environment
+    """
+    vm = kvm_utils.env_get_vm(env, params.get("main_vm"))
+    if not vm:
+        raise error.TestError("VM object not found in environment")
+    if not vm.is_alive():
+        raise error.TestError("VM seems to be dead; Test requires a living VM")
+
+    logging.info("Waiting for guest to be up...")
+
+    session = kvm_utils.wait_for(vm.ssh_login, 240, 0, 2)
+    if not session:
+        raise error.TestFail("Could not log into guest")
+
+    logging.info("Logged in")
+
+    # Send the VM's shutdown command
+    session.sendline(vm.get_params().get("cmd_shutdown"))
+    session.close()
+
+    logging.info("Shutdown command sent; waiting for guest to go down...")
+
+    if not kvm_utils.wait_for(vm.is_dead, 120, 0, 1):
+        raise error.TestFail("Guest refuses to go down")
+
+    logging.info("Guest is down")
+
+
 def run_migration(test, params, env):
     """
     KVM migration test:
-- 
1.5.4.1

[KVM-AUTOTEST PATCH 2/2] KVM test: include shutdown test in kvm_tests.cfg.sample

[Michael Goldish]

The shutdown test should always be last -- new tests will be added above it.

If the user chooses to run all tests on a guest, the guest will be shut down
by the final shutdown test.  If the user selects a specific subset of the tests,
the guest will shut down if the shutdown test is included in this subset.
Note that this only applies to the last guest of the job; since currently all
guests use the same VM object (vm1), the VM is shut down automatically when
it's passed from one guest to the next.

In short, standard test sets (weekly, daily, per-release) should include the
shutdown test.  In particular, tests running from the server should include it.
Custom test sets run in client mode, intended for testing/developing a specific
test or a set of tests, may run without the shutdown test.

The shutdown test is defined with 'kill_vm = yes', so if the test fails to shut
the guest down nicely, the postprocessor will take care of it.

Also added 'kill_vm_on_error = yes' to 'setup' tests, because if a setup test
fails, shutdown will not run, so the setup test must take care of shutting the
VM down.  Install tests already have this defined.

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm_tests.cfg.sample |    9 ++++++++-
 1 files changed, 8 insertions(+), 1 deletions(-)

diff --git a/client/tests/kvm/kvm_tests.cfg.sample b/client/tests/kvm/kvm_tests.cfg.sample
index 54c9975..412c4b0 100644
--- a/client/tests/kvm/kvm_tests.cfg.sample
+++ b/client/tests/kvm/kvm_tests.cfg.sample
@@ -41,6 +41,7 @@ variants:
         fail_if_stuck_for = 300
         stuck_detection_history = 2
         keep_screendump_history = yes
+        kill_vm_on_error = yes
 
     - boot:         install setup
         type = boot
@@ -83,6 +84,12 @@ variants:
     - linux_s3:      install setup
         type = linux_s3
 
+    - shutdown:      install setup
+        type = shutdown
+        kill_vm = yes
+        kill_vm_gracefully = no
+
+
 # NICs
 variants:
     - @rtl8139:
@@ -544,7 +551,7 @@ variants:
         only default
         only up
         only Fedora.8.32
-        only install setup boot
+        only install setup boot shutdown
         only rtl8139
     - @sample1:
         only qcow2
-- 
1.5.4.1

Re: [KVM-AUTOTEST PATCH 1/2] KVM test: add shutdown test

[Alexey Eromenko]

Michael I don't fully understand why shutdown test is needed; Shutdown is tested during reboot, where GuestOS must de-init itself.

Re: [KVM-AUTOTEST PATCH 1/2] KVM test: add shutdown test

[Michael Goldish]

----- "Alexey Eromenko" <aeromenk@redhat.com> wrote:

> Michael I don't fully understand why shutdown test is needed; Shutdown
> is tested during reboot, where GuestOS must de-init itself.

The main motivation for a shutdown test is that it allows us to choose
whether VMs should be kept alive after the last test.  Including this test
in a job takes no more time than running the job without it, because
normally the preprocessor automatically shuts down guests (except for the
last one).

There's also a small difference between a complete shutdown and a reboot --
a shutdown is supposed to close the qemu process.  I'm not sure this
difference justifies the test, but the reason mentioned in the previous
paragraph does, in my opinion.

Re: [KVM-AUTOTEST PATCH 1/2] KVM test: add shutdown test

[jason wang]

Hello Michael:

The shutdown case is useful but the patch does really similar work to
vm.destroy(). Maybe we could simple let the preprocess progress to
shutdown all the vms just like:

    - shutdown:      install setup
        vms = ''

Michael Goldish 写道:
> The shutdown test logs into a VM and sends a shutdown command.
> It serves two purposes:
> - Test KVM's ability to shut down.
> - Clean up after the other tests:
> Currently VMs of the last test remain alive when Autotest finishes running.
> When one guest finishes testing and another begins, the VM is automatically
> shut down by the preprocessor because the QEMU command required for the next
> guest differs from that of the guest that just finished.  In the case of the
> final guest this doesn't happen because no guest follows it, so the preprocessor
> must be explicitly instructed to kill the VM.
> However, we have no easy way of knowing which test runs last because the user
> usually selects a subset of the tests/guests.
> The addition of a shutdown test can be a decent solution to this small problem:
> by convention the shutdown test will always be the last to run, and if users
> wish to clean up after the tests, they must select the shutdown test.
>
> Note: it is beneficial to allow users to leave the VMs of the last test running
> because it saves time when developing and testing tests. A test writer can run
> the test once on a VM, and when the test exits, make some modifications to its
> code and re-run it on the same living VM, and repeat this procedure without
> having to shutdown/boot the VM every time.
>
> Signed-off-by: Michael Goldish <mgoldish@redhat.com>
> ---
>  client/tests/kvm/kvm.py       |    1 +
>  client/tests/kvm/kvm_tests.py |   37 +++++++++++++++++++++++++++++++++++++
>  2 files changed, 38 insertions(+), 0 deletions(-)
>
> diff --git a/client/tests/kvm/kvm.py b/client/tests/kvm/kvm.py
> index 9428162..aa727da 100644
> --- a/client/tests/kvm/kvm.py
> +++ b/client/tests/kvm/kvm.py
> @@ -48,6 +48,7 @@ class kvm(test.test):
>                  "steps":        test_routine("kvm_guest_wizard", "run_steps"),
>                  "stepmaker":    test_routine("stepmaker", "run_stepmaker"),
>                  "boot":         test_routine("kvm_tests", "run_boot"),
> +                "shutdown":     test_routine("kvm_tests", "run_shutdown"),
>                  "migration":    test_routine("kvm_tests", "run_migration"),
>                  "yum_update":   test_routine("kvm_tests", "run_yum_update"),
>                  "autotest":     test_routine("kvm_tests", "run_autotest"),
> diff --git a/client/tests/kvm/kvm_tests.py b/client/tests/kvm/kvm_tests.py
> index ffe9116..4c9653f 100644
> --- a/client/tests/kvm/kvm_tests.py
> +++ b/client/tests/kvm/kvm_tests.py
> @@ -57,6 +57,43 @@ def run_boot(test, params, env):
>      session.close()
>  
>  
> +def run_shutdown(test, params, env):
> +    """
> +    KVM shutdown test:
> +    1) Log into a guest
> +    2) Send a shutdown command to the guest
> +    3) Wait until it's down
> +
> +    @param test: kvm test object
> +    @param params: Dictionary with the test parameters
> +    @param env: Dictionary with test environment
> +    """
> +    vm = kvm_utils.env_get_vm(env, params.get("main_vm"))
> +    if not vm:
> +        raise error.TestError("VM object not found in environment")
> +    if not vm.is_alive():
> +        raise error.TestError("VM seems to be dead; Test requires a living VM")
> +
> +    logging.info("Waiting for guest to be up...")
> +
> +    session = kvm_utils.wait_for(vm.ssh_login, 240, 0, 2)
> +    if not session:
> +        raise error.TestFail("Could not log into guest")
> +
> +    logging.info("Logged in")
> +
> +    # Send the VM's shutdown command
> +    session.sendline(vm.get_params().get("cmd_shutdown"))
> +    session.close()
> +
> +    logging.info("Shutdown command sent; waiting for guest to go down...")
> +
> +    if not kvm_utils.wait_for(vm.is_dead, 120, 0, 1):
> +        raise error.TestFail("Guest refuses to go down")
> +
> +    logging.info("Guest is down")
> +
> +
>  def run_migration(test, params, env):
>      """
>      KVM migration test:
>   

Re: [Autotest] [KVM-AUTOTEST PATCH 1/2] KVM test: add shutdown test

[Lucas Meneghel Rodrigues]

On Tue, 2009-06-16 at 07:05 -0400, Michael Goldish wrote:
> ----- "Alexey Eromenko" <aeromenk@redhat.com> wrote:
> 
> > Michael I don't fully understand why shutdown test is needed; Shutdown
> > is tested during reboot, where GuestOS must de-init itself.
> 
> The main motivation for a shutdown test is that it allows us to choose
> whether VMs should be kept alive after the last test.  Including this test
> in a job takes no more time than running the job without it, because
> normally the preprocessor automatically shuts down guests (except for the
> last one).
> 
> There's also a small difference between a complete shutdown and a reboot --
> a shutdown is supposed to close the qemu process.  I'm not sure this
> difference justifies the test, but the reason mentioned in the previous
> paragraph does, in my opinion.

Fair enough, I agree. Will apply the patch.

-- 
Lucas Meneghel Rodrigues
Software Engineer (QE)
Red Hat - Emerging Technologies

Re: [Autotest] [KVM-AUTOTEST PATCH 1/2] KVM test: add shutdown test

[Lucas Meneghel Rodrigues]

On Tue, 2009-06-16 at 10:37 +0300, Michael Goldish wrote:
> The shutdown test logs into a VM and sends a shutdown command.
> It serves two purposes:
> - Test KVM's ability to shut down.
> - Clean up after the other tests:
> Currently VMs of the last test remain alive when Autotest finishes running.
> When one guest finishes testing and another begins, the VM is automatically
> shut down by the preprocessor because the QEMU command required for the next
> guest differs from that of the guest that just finished.  In the case of the
> final guest this doesn't happen because no guest follows it, so the preprocessor
> must be explicitly instructed to kill the VM.
> However, we have no easy way of knowing which test runs last because the user
> usually selects a subset of the tests/guests.
> The addition of a shutdown test can be a decent solution to this small problem:
> by convention the shutdown test will always be the last to run, and if users
> wish to clean up after the tests, they must select the shutdown test.
> 
> Note: it is beneficial to allow users to leave the VMs of the last test running
> because it saves time when developing and testing tests. A test writer can run
> the test once on a VM, and when the test exits, make some modifications to its
> code and re-run it on the same living VM, and repeat this procedure without
> having to shutdown/boot the VM every time.

Ok, patch set applied. Thank you!

> Signed-off-by: Michael Goldish <mgoldish@redhat.com>
> ---
>  client/tests/kvm/kvm.py       |    1 +
>  client/tests/kvm/kvm_tests.py |   37 +++++++++++++++++++++++++++++++++++++
>  2 files changed, 38 insertions(+), 0 deletions(-)
> 
> diff --git a/client/tests/kvm/kvm.py b/client/tests/kvm/kvm.py
> index 9428162..aa727da 100644
> --- a/client/tests/kvm/kvm.py
> +++ b/client/tests/kvm/kvm.py
> @@ -48,6 +48,7 @@ class kvm(test.test):
>                  "steps":        test_routine("kvm_guest_wizard", "run_steps"),
>                  "stepmaker":    test_routine("stepmaker", "run_stepmaker"),
>                  "boot":         test_routine("kvm_tests", "run_boot"),
> +                "shutdown":     test_routine("kvm_tests", "run_shutdown"),
>                  "migration":    test_routine("kvm_tests", "run_migration"),
>                  "yum_update":   test_routine("kvm_tests", "run_yum_update"),
>                  "autotest":     test_routine("kvm_tests", "run_autotest"),
> diff --git a/client/tests/kvm/kvm_tests.py b/client/tests/kvm/kvm_tests.py
> index ffe9116..4c9653f 100644
> --- a/client/tests/kvm/kvm_tests.py
> +++ b/client/tests/kvm/kvm_tests.py
> @@ -57,6 +57,43 @@ def run_boot(test, params, env):
>      session.close()
>  
> 
> +def run_shutdown(test, params, env):
> +    """
> +    KVM shutdown test:
> +    1) Log into a guest
> +    2) Send a shutdown command to the guest
> +    3) Wait until it's down
> +
> +    @param test: kvm test object
> +    @param params: Dictionary with the test parameters
> +    @param env: Dictionary with test environment
> +    """
> +    vm = kvm_utils.env_get_vm(env, params.get("main_vm"))
> +    if not vm:
> +        raise error.TestError("VM object not found in environment")
> +    if not vm.is_alive():
> +        raise error.TestError("VM seems to be dead; Test requires a living VM")
> +
> +    logging.info("Waiting for guest to be up...")
> +
> +    session = kvm_utils.wait_for(vm.ssh_login, 240, 0, 2)
> +    if not session:
> +        raise error.TestFail("Could not log into guest")
> +
> +    logging.info("Logged in")
> +
> +    # Send the VM's shutdown command
> +    session.sendline(vm.get_params().get("cmd_shutdown"))
> +    session.close()
> +
> +    logging.info("Shutdown command sent; waiting for guest to go down...")
> +
> +    if not kvm_utils.wait_for(vm.is_dead, 120, 0, 1):
> +        raise error.TestFail("Guest refuses to go down")
> +
> +    logging.info("Guest is down")
> +
> +
>  def run_migration(test, params, env):
>      """
>      KVM migration test:
-- 
Lucas Meneghel Rodrigues
Software Engineer (QE)
Red Hat - Emerging Technologies