[KVM-AUTOTEST PATCH v2 0/3] KVM test: corrections to the AutoIt patchset

[KVM-AUTOTEST PATCH v2 0/3] KVM test: corrections to the AutoIt patchset

[Michael Goldish]

This is the AutoIt patch set with some minor corrections:
- Use read_up_to_prompt() instead of read_nonblocking() before running the
  AutoIt command
- autoit_script_params defaults to "" in the test code as well
- Indentation fix in notepad1.au3

[KVM-AUTOTEST PATCH v2 1/3] KVM test: add AutoIt test

[Michael Goldish]

Currently the test only logs in, runs a given script and fails if the script
takes too long to exit or if its exit status is nonzero.

The test expects these parameters:
autoit_binary: Path to AutoIt binary in the guest.
autoit_script: Path to script in the host.
autoit_script_params: Command line parameters to send to the script.
autoit_script_timeout: The time duration (in seconds) to wait for the script to
exit.

The test code can be extended later to add more features.

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm.py       |    1 +
 client/tests/kvm/kvm_tests.py |   66 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 67 insertions(+), 0 deletions(-)

diff --git a/client/tests/kvm/kvm.py b/client/tests/kvm/kvm.py
index 070e463..4930e80 100644
--- a/client/tests/kvm/kvm.py
+++ b/client/tests/kvm/kvm.py
@@ -56,6 +56,7 @@ class kvm(test.test):
                 "linux_s3":     test_routine("kvm_tests", "run_linux_s3"),
                 "stress_boot":  test_routine("kvm_tests", "run_stress_boot"),
                 "timedrift":    test_routine("kvm_tests", "run_timedrift"),
+                "autoit":       test_routine("kvm_tests", "run_autoit"),
                 }
 
         # Make it possible to import modules from the test's bindir
diff --git a/client/tests/kvm/kvm_tests.py b/client/tests/kvm/kvm_tests.py
index 9cd01e2..749c1fd 100644
--- a/client/tests/kvm/kvm_tests.py
+++ b/client/tests/kvm/kvm_tests.py
@@ -776,3 +776,69 @@ def run_timedrift(test, params, env):
     if drift > drift_threshold_after_rest:
         raise error.TestFail("Time drift too large after rest period: %.2f%%"
                              % drift_total)
+
+
+def run_autoit(test, params, env):
+    """
+    A wrapper for AutoIt scripts.
+
+    1) Log into a guest.
+    2) Run AutoIt script.
+    3) Wait for script execution to complete.
+    4) Pass/fail according to exit status of script.
+
+    @param test: KVM test object.
+    @param params: Dictionary with test parameters.
+    @param env: Dictionary with the test environment.
+    """
+    vm = kvm_utils.env_get_vm(env, params.get("main_vm"))
+    if not vm:
+        raise error.TestError("VM object not found in environment")
+    if not vm.is_alive():
+        raise error.TestError("VM seems to be dead; Test requires a living VM")
+
+    logging.info("Waiting for guest to be up...")
+
+    session = kvm_utils.wait_for(vm.remote_login, 240, 0, 2)
+    if not session:
+        raise error.TestFail("Could not log into guest")
+
+    try:
+        logging.info("Logged in; starting script...")
+
+        # Collect test parameters
+        binary = params.get("autoit_binary")
+        script = params.get("autoit_script")
+        script_params = params.get("autoit_script_params", "")
+        timeout = float(params.get("autoit_script_timeout", 600))
+
+        # Send AutoIt script to guest (this code will be replaced once we
+        # support sending files to Windows guests)
+        session.sendline("del script.au3")
+        file = open(kvm_utils.get_path(test.bindir, script))
+        for line in file.readlines():
+            # Insert a '^' before each character
+            line = "".join("^" + c for c in line.rstrip())
+            if line:
+                # Append line to the file
+                session.sendline("echo %s>>script.au3" % line)
+        file.close()
+
+        session.read_up_to_prompt()
+
+        command = "cmd /c %s script.au3 %s" % (binary, script_params)
+
+        logging.info("---------------- Script output ----------------")
+        status = session.get_command_status(command,
+                                            print_func=logging.info,
+                                            timeout=timeout)
+        logging.info("---------------- End of script output ----------------")
+
+        if status is None:
+            raise error.TestFail("Timeout expired before script execution "
+                                 "completed (or something weird happened)")
+        if status != 0:
+            raise error.TestFail("Script execution failed")
+
+    finally:
+        session.close()
-- 
1.5.4.1

[KVM-AUTOTEST PATCH v2 2/3] KVM test: add sample AutoIt script

[Michael Goldish]

Add a sample autoit script under autoit/.
The script is a modified version of the notepad1.au3 example script that ships
with AutoIt.

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/autoit/notepad1.au3 |   44 ++++++++++++++++++++++++++++++++++
 1 files changed, 44 insertions(+), 0 deletions(-)
 create mode 100644 client/tests/kvm/autoit/notepad1.au3

diff --git a/client/tests/kvm/autoit/notepad1.au3 b/client/tests/kvm/autoit/notepad1.au3
new file mode 100644
index 0000000..af1d417
--- /dev/null
+++ b/client/tests/kvm/autoit/notepad1.au3
@@ -0,0 +1,44 @@
+; This is a sample AutoIt script, based on the notepad1 sample script by Jonathan Bennett.
+; It runs notepad, enters some text and exits.
+
+
+; Exit with a nonzero exit status if the parameter equals 0.
+; This is useful for functions that return 0 upon failure.
+Func Assert($n)
+    If $n = 0 Then Exit(1)
+EndFunc
+
+; Wait for a window to exist, activate it, and wait for it to become active.
+; If timeout expires while waiting, exit with a nonzero exit status.
+Func WaitForWindow($title, $text="", $timeout=60)
+    Assert(WinWait($title, $text, $timeout))
+    WinActivate($title, $text)
+    Assert(WinWaitActive($title, $text, $timeout))
+EndFunc
+
+; Run Notepad
+Assert(Run("notepad.exe"))
+
+; Wait up to 10 seconds for Notepad to become active --
+; it is titled "Untitled - Notepad" on English systems
+WaitForWindow("Untitled - Notepad", "", 10)
+
+; Now that the Notepad window is active type some text
+Send("Hello from Notepad.{ENTER}1 2 3 4 5 6 7 8 9 10{ENTER}")
+Sleep(500)
+Send("+{UP 2}")
+Sleep(500)
+
+; Now quit by pressing Alt-f and then x (File menu -> Exit)
+Send("!f")
+Send("x")
+
+; Now a screen will pop up and ask to save the changes, the window is called 
+; "Notepad" and has some text "Yes" and "No"
+WaitForWindow("Notepad", "", 10)
+Send("n")
+
+; Now wait for Notepad to close before continuing
+WinWaitClose("Untitled - Notepad", "", 10)
+
+; Finished!
-- 
1.5.4.1

[KVM-AUTOTEST PATCH v2 3/3] KVM test: add AutoIt test to kvm_tests.cfg.sample

[Michael Goldish]

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm_tests.cfg.sample |   12 +++++++++++-
 1 files changed, 11 insertions(+), 1 deletions(-)

diff --git a/client/tests/kvm/kvm_tests.cfg.sample b/client/tests/kvm/kvm_tests.cfg.sample
index 12e907b..05a1ca7 100644
--- a/client/tests/kvm/kvm_tests.cfg.sample
+++ b/client/tests/kvm/kvm_tests.cfg.sample
@@ -126,6 +126,16 @@ variants:
         kill_vm_gracefully = no
         extra_params += " -snapshot"
 
+    - autoit:       install setup
+        type = autoit
+        cdrom = windows/autoit.iso
+        autoit_binary = D:\AutoIt3.exe
+        autoit_script_timeout = 600
+        autoit_script_params =
+        variants:
+            - notepad:
+                autoit_script = autoit/notepad1.au3
+
     - shutdown:     install setup
         type = shutdown
         kill_vm = yes
@@ -147,7 +157,7 @@ variants:
 variants:
     # Linux section
     - @Linux:
-        no timedrift
+        no timedrift autoit
         shutdown_command = shutdown -h now
         reboot_command = shutdown -r now
         status_test_command = echo $?
-- 
1.5.4.1

Re: [KVM-AUTOTEST PATCH v2 3/3] KVM test: add AutoIt test to kvm_tests.cfg.sample

[Cao, Chen]

On Tue, Aug 11, 2009 at 03:10:44PM +0300, Michael Goldish wrote:
> Signed-off-by: Michael Goldish <mgoldish@redhat.com>
> ---
>  client/tests/kvm/kvm_tests.cfg.sample |   12 +++++++++++-
>  1 files changed, 11 insertions(+), 1 deletions(-)
> 
> diff --git a/client/tests/kvm/kvm_tests.cfg.sample b/client/tests/kvm/kvm_tests.cfg.sample
> index 12e907b..05a1ca7 100644
> --- a/client/tests/kvm/kvm_tests.cfg.sample
> +++ b/client/tests/kvm/kvm_tests.cfg.sample
> @@ -126,6 +126,16 @@ variants:
>          kill_vm_gracefully = no
>          extra_params += " -snapshot"
>  
> +    - autoit:       install setup
> +        type = autoit
> +        cdrom = windows/autoit.iso
I am wondering what is the advantage of using .iso to store the binary
files. I think it might be hard to maintain the (scripts/binary)
files.
is it possible to pass a directory to run_autoit()?

> +        autoit_binary = D:\AutoIt3.exe
> +        autoit_script_timeout = 600
> +        autoit_script_params =
> +        variants:
> +            - notepad:
> +                autoit_script = autoit/notepad1.au3
> +... blahblah...

Besides, I think we may use staf framework to send command to MS
Windows
to run autoit scripts, it would be easier to control the status of the
autoit process.

what is your suggestion?

Thank you.


Regards,

Cao, Chen
2009/08/12

Re: [Autotest] [KVM-AUTOTEST PATCH v2 1/3] KVM test: add AutoIt test

[Yolkfull Chow]

On Tue, Aug 11, 2009 at 03:10:42PM +0300, Michael Goldish wrote:
> Currently the test only logs in, runs a given script and fails if the script
> takes too long to exit or if its exit status is nonzero.
> 
> The test expects these parameters:
> autoit_binary: Path to AutoIt binary in the guest.
> autoit_script: Path to script in the host.
> autoit_script_params: Command line parameters to send to the script.
> autoit_script_timeout: The time duration (in seconds) to wait for the script to
> exit.
> 
> The test code can be extended later to add more features.
> 
> Signed-off-by: Michael Goldish <mgoldish@redhat.com>
> ---
>  client/tests/kvm/kvm.py       |    1 +
>  client/tests/kvm/kvm_tests.py |   66 +++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 67 insertions(+), 0 deletions(-)
> 
> diff --git a/client/tests/kvm/kvm.py b/client/tests/kvm/kvm.py
> index 070e463..4930e80 100644
> --- a/client/tests/kvm/kvm.py
> +++ b/client/tests/kvm/kvm.py
> @@ -56,6 +56,7 @@ class kvm(test.test):
>                  "linux_s3":     test_routine("kvm_tests", "run_linux_s3"),
>                  "stress_boot":  test_routine("kvm_tests", "run_stress_boot"),
>                  "timedrift":    test_routine("kvm_tests", "run_timedrift"),
> +                "autoit":       test_routine("kvm_tests", "run_autoit"),
>                  }
>  
>          # Make it possible to import modules from the test's bindir
> diff --git a/client/tests/kvm/kvm_tests.py b/client/tests/kvm/kvm_tests.py
> index 9cd01e2..749c1fd 100644
> --- a/client/tests/kvm/kvm_tests.py
> +++ b/client/tests/kvm/kvm_tests.py
> @@ -776,3 +776,69 @@ def run_timedrift(test, params, env):
>      if drift > drift_threshold_after_rest:
>          raise error.TestFail("Time drift too large after rest period: %.2f%%"
>                               % drift_total)
> +
> +
> +def run_autoit(test, params, env):
> +    """
> +    A wrapper for AutoIt scripts.
> +
> +    1) Log into a guest.
> +    2) Run AutoIt script.
> +    3) Wait for script execution to complete.
> +    4) Pass/fail according to exit status of script.
> +
> +    @param test: KVM test object.
> +    @param params: Dictionary with test parameters.
> +    @param env: Dictionary with the test environment.
> +    """
> +    vm = kvm_utils.env_get_vm(env, params.get("main_vm"))
> +    if not vm:
> +        raise error.TestError("VM object not found in environment")
> +    if not vm.is_alive():
> +        raise error.TestError("VM seems to be dead; Test requires a living VM")
> +
> +    logging.info("Waiting for guest to be up...")
> +
> +    session = kvm_utils.wait_for(vm.remote_login, 240, 0, 2)
> +    if not session:
> +        raise error.TestFail("Could not log into guest")
> +
> +    try:
> +        logging.info("Logged in; starting script...")
> +
> +        # Collect test parameters
> +        binary = params.get("autoit_binary")
> +        script = params.get("autoit_script")
> +        script_params = params.get("autoit_script_params", "")
> +        timeout = float(params.get("autoit_script_timeout", 600))
> +
> +        # Send AutoIt script to guest (this code will be replaced once we
> +        # support sending files to Windows guests)
> +        session.sendline("del script.au3")
> +        file = open(kvm_utils.get_path(test.bindir, script))
> +        for line in file.readlines():
> +            # Insert a '^' before each character
> +            line = "".join("^" + c for c in line.rstrip())
> +            if line:
> +                # Append line to the file
> +                session.sendline("echo %s>>script.au3" % line)
> +        file.close()
> +
> +        session.read_up_to_prompt()
> +
> +        command = "cmd /c %s script.au3 %s" % (binary, script_params)

Hi Michael, for the problem that execute script in Windows cmd shell, I
have some information share with you:

Guys in our team had found that the value which `echo %errorlevel%`
returns is not always right. It just reflects whether the action to
execute the script has been implemented successfully and it ALWAYS return
even if errors occur. That means as soon as the script has been started
successfully it will return 0 even if error occurred during script running.

One solution could be use command 'start /wait script.au3' which will
let program wait for it to terminate:
http://ss64.com/nt/start.html

I have not investigated it enough as well, if any mistake made, please
just ignore this reply. ;-)


> +
> +        logging.info("---------------- Script output ----------------")
> +        status = session.get_command_status(command,
> +                                            print_func=logging.info,
> +                                            timeout=timeout)
> +        logging.info("---------------- End of script output ----------------")
> +
> +        if status is None:
> +            raise error.TestFail("Timeout expired before script execution "
> +                                 "completed (or something weird happened)")
> +        if status != 0:
> +            raise error.TestFail("Script execution failed")
> +
> +    finally:
> +        session.close()
> -- 
> 1.5.4.1
> 
> _______________________________________________
> Autotest mailing list
> Autotest@test.kernel.org
> http://test.kernel.org/cgi-bin/mailman/listinfo/autotest

Re: [Autotest] [KVM-AUTOTEST PATCH v2 0/3] KVM test: corrections to the AutoIt patchset

[Lucas Meneghel Rodrigues]

On Tue, Aug 11, 2009 at 9:10 AM, Michael Goldish<mgoldish@redhat.com> wrote:
>
> This is the AutoIt patch set with some minor corrections:
> - Use read_up_to_prompt() instead of read_nonblocking() before running the
>  AutoIt command
> - autoit_script_params defaults to "" in the test code as well
> - Indentation fix in notepad1.au3

Ok, test looks good, I tested the simple notepad script, works ok. Applied.

With this the patch queue is significantly smaller, and I have no
pending patches from you Michael (I believe). I will work on an
updated patch queue document and follow up with other important
patchsets.

Lucas