[KVM-AUTOTEST PATCH v2 0/3] KVM test: corrections to the AutoIt patchset

[KVM-AUTOTEST PATCH v2 0/3] KVM test: corrections to the AutoIt patchset

[Michael Goldish]

This is the AutoIt patch set with some minor corrections:
- Use read_up_to_prompt() instead of read_nonblocking() before running the
  AutoIt command
- autoit_script_params defaults to "" in the test code as well
- Indentation fix in notepad1.au3

[KVM-AUTOTEST PATCH v2 1/3] KVM test: add AutoIt test

[Michael Goldish]

Currently the test only logs in, runs a given script and fails if the script
takes too long to exit or if its exit status is nonzero.

The test expects these parameters:
autoit_binary: Path to AutoIt binary in the guest.
autoit_script: Path to script in the host.
autoit_script_params: Command line parameters to send to the script.
autoit_script_timeout: The time duration (in seconds) to wait for the script to
exit.

The test code can be extended later to add more features.

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm.py       |    1 +
 client/tests/kvm/kvm_tests.py |   66 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 67 insertions(+), 0 deletions(-)

diff --git a/client/tests/kvm/kvm.py b/client/tests/kvm/kvm.py
index 070e463..4930e80 100644
--- a/client/tests/kvm/kvm.py
+++ b/client/tests/kvm/kvm.py
@@ -56,6 +56,7 @@ class kvm(test.test):
                 "linux_s3":     test_routine("kvm_tests", "run_linux_s3"),
                 "stress_boot":  test_routine("kvm_tests", "run_stress_boot"),
                 "timedrift":    test_routine("kvm_tests", "run_timedrift"),
+                "autoit":       test_routine("kvm_tests", "run_autoit"),
                 }
 
         # Make it possible to import modules from the test's bindir
diff --git a/client/tests/kvm/kvm_tests.py b/client/tests/kvm/kvm_tests.py
index 9cd01e2..749c1fd 100644
--- a/client/tests/kvm/kvm_tests.py
+++ b/client/tests/kvm/kvm_tests.py
@@ -776,3 +776,69 @@ def run_timedrift(test, params, env):
     if drift > drift_threshold_after_rest:
         raise error.TestFail("Time drift too large after rest period: %.2f%%"
                              % drift_total)
+
+
+def run_autoit(test, params, env):
+    """
+    A wrapper for AutoIt scripts.
+
+    1) Log into a guest.
+    2) Run AutoIt script.
+    3) Wait for script execution to complete.
+    4) Pass/fail according to exit status of script.
+
+    @param test: KVM test object.
+    @param params: Dictionary with test parameters.
+    @param env: Dictionary with the test environment.
+    """
+    vm = kvm_utils.env_get_vm(env, params.get("main_vm"))
+    if not vm:
+        raise error.TestError("VM object not found in environment")
+    if not vm.is_alive():
+        raise error.TestError("VM seems to be dead; Test requires a living VM")
+
+    logging.info("Waiting for guest to be up...")
+
+    session = kvm_utils.wait_for(vm.remote_login, 240, 0, 2)
+    if not session:
+        raise error.TestFail("Could not log into guest")
+
+    try:
+        logging.info("Logged in; starting script...")
+
+        # Collect test parameters
+        binary = params.get("autoit_binary")
+        script = params.get("autoit_script")
+        script_params = params.get("autoit_script_params", "")
+        timeout = float(params.get("autoit_script_timeout", 600))
+
+        # Send AutoIt script to guest (this code will be replaced once we
+        # support sending files to Windows guests)
+        session.sendline("del script.au3")
+        file = open(kvm_utils.get_path(test.bindir, script))
+        for line in file.readlines():
+            # Insert a '^' before each character
+            line = "".join("^" + c for c in line.rstrip())
+            if line:
+                # Append line to the file
+                session.sendline("echo %s>>script.au3" % line)
+        file.close()
+
+        session.read_up_to_prompt()
+
+        command = "cmd /c %s script.au3 %s" % (binary, script_params)
+
+        logging.info("---------------- Script output ----------------")
+        status = session.get_command_status(command,
+                                            print_func=logging.info,
+                                            timeout=timeout)
+        logging.info("---------------- End of script output ----------------")
+
+        if status is None:
+            raise error.TestFail("Timeout expired before script execution "
+                                 "completed (or something weird happened)")
+        if status != 0:
+            raise error.TestFail("Script execution failed")
+
+    finally:
+        session.close()
-- 
1.5.4.1

[KVM-AUTOTEST PATCH v2 2/3] KVM test: add sample AutoIt script

[Michael Goldish]

Add a sample autoit script under autoit/.
The script is a modified version of the notepad1.au3 example script that ships
with AutoIt.

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/autoit/notepad1.au3 |   44 ++++++++++++++++++++++++++++++++++
 1 files changed, 44 insertions(+), 0 deletions(-)
 create mode 100644 client/tests/kvm/autoit/notepad1.au3

diff --git a/client/tests/kvm/autoit/notepad1.au3 b/client/tests/kvm/autoit/notepad1.au3
new file mode 100644
index 0000000..af1d417
--- /dev/null
+++ b/client/tests/kvm/autoit/notepad1.au3
@@ -0,0 +1,44 @@
+; This is a sample AutoIt script, based on the notepad1 sample script by Jonathan Bennett.
+; It runs notepad, enters some text and exits.
+
+
+; Exit with a nonzero exit status if the parameter equals 0.
+; This is useful for functions that return 0 upon failure.
+Func Assert($n)
+    If $n = 0 Then Exit(1)
+EndFunc
+
+; Wait for a window to exist, activate it, and wait for it to become active.
+; If timeout expires while waiting, exit with a nonzero exit status.
+Func WaitForWindow($title, $text="", $timeout=60)
+    Assert(WinWait($title, $text, $timeout))
+    WinActivate($title, $text)
+    Assert(WinWaitActive($title, $text, $timeout))
+EndFunc
+
+; Run Notepad
+Assert(Run("notepad.exe"))
+
+; Wait up to 10 seconds for Notepad to become active --
+; it is titled "Untitled - Notepad" on English systems
+WaitForWindow("Untitled - Notepad", "", 10)
+
+; Now that the Notepad window is active type some text
+Send("Hello from Notepad.{ENTER}1 2 3 4 5 6 7 8 9 10{ENTER}")
+Sleep(500)
+Send("+{UP 2}")
+Sleep(500)
+
+; Now quit by pressing Alt-f and then x (File menu -> Exit)
+Send("!f")
+Send("x")
+
+; Now a screen will pop up and ask to save the changes, the window is called 
+; "Notepad" and has some text "Yes" and "No"
+WaitForWindow("Notepad", "", 10)
+Send("n")
+
+; Now wait for Notepad to close before continuing
+WinWaitClose("Untitled - Notepad", "", 10)
+
+; Finished!
-- 
1.5.4.1

[KVM-AUTOTEST PATCH v2 3/3] KVM test: add AutoIt test to kvm_tests.cfg.sample

[Michael Goldish]

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm_tests.cfg.sample |   12 +++++++++++-
 1 files changed, 11 insertions(+), 1 deletions(-)

diff --git a/client/tests/kvm/kvm_tests.cfg.sample b/client/tests/kvm/kvm_tests.cfg.sample
index 12e907b..05a1ca7 100644
--- a/client/tests/kvm/kvm_tests.cfg.sample
+++ b/client/tests/kvm/kvm_tests.cfg.sample
@@ -126,6 +126,16 @@ variants:
         kill_vm_gracefully = no
         extra_params += " -snapshot"
 
+    - autoit:       install setup
+        type = autoit
+        cdrom = windows/autoit.iso
+        autoit_binary = D:\AutoIt3.exe
+        autoit_script_timeout = 600
+        autoit_script_params =
+        variants:
+            - notepad:
+                autoit_script = autoit/notepad1.au3
+
     - shutdown:     install setup
         type = shutdown
         kill_vm = yes
@@ -147,7 +157,7 @@ variants:
 variants:
     # Linux section
     - @Linux:
-        no timedrift
+        no timedrift autoit
         shutdown_command = shutdown -h now
         reboot_command = shutdown -r now
         status_test_command = echo $?
-- 
1.5.4.1

Re: [Autotest] [KVM-AUTOTEST PATCH v2 1/3] KVM test: add AutoIt test

[Yolkfull Chow]

On Tue, Aug 11, 2009 at 03:10:42PM +0300, Michael Goldish wrote:
> Currently the test only logs in, runs a given script and fails if the script
> takes too long to exit or if its exit status is nonzero.
> 
> The test expects these parameters:
> autoit_binary: Path to AutoIt binary in the guest.
> autoit_script: Path to script in the host.
> autoit_script_params: Command line parameters to send to the script.
> autoit_script_timeout: The time duration (in seconds) to wait for the script to
> exit.
> 
> The test code can be extended later to add more features.
> 
> Signed-off-by: Michael Goldish <mgoldish@redhat.com>
> ---
>  client/tests/kvm/kvm.py       |    1 +
>  client/tests/kvm/kvm_tests.py |   66 +++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 67 insertions(+), 0 deletions(-)
> 
> diff --git a/client/tests/kvm/kvm.py b/client/tests/kvm/kvm.py
> index 070e463..4930e80 100644
> --- a/client/tests/kvm/kvm.py
> +++ b/client/tests/kvm/kvm.py
> @@ -56,6 +56,7 @@ class kvm(test.test):
>                  "linux_s3":     test_routine("kvm_tests", "run_linux_s3"),
>                  "stress_boot":  test_routine("kvm_tests", "run_stress_boot"),
>                  "timedrift":    test_routine("kvm_tests", "run_timedrift"),
> +                "autoit":       test_routine("kvm_tests", "run_autoit"),
>                  }
>  
>          # Make it possible to import modules from the test's bindir
> diff --git a/client/tests/kvm/kvm_tests.py b/client/tests/kvm/kvm_tests.py
> index 9cd01e2..749c1fd 100644
> --- a/client/tests/kvm/kvm_tests.py
> +++ b/client/tests/kvm/kvm_tests.py
> @@ -776,3 +776,69 @@ def run_timedrift(test, params, env):
>      if drift > drift_threshold_after_rest:
>          raise error.TestFail("Time drift too large after rest period: %.2f%%"
>                               % drift_total)
> +
> +
> +def run_autoit(test, params, env):
> +    """
> +    A wrapper for AutoIt scripts.
> +
> +    1) Log into a guest.
> +    2) Run AutoIt script.
> +    3) Wait for script execution to complete.
> +    4) Pass/fail according to exit status of script.
> +
> +    @param test: KVM test object.
> +    @param params: Dictionary with test parameters.
> +    @param env: Dictionary with the test environment.
> +    """
> +    vm = kvm_utils.env_get_vm(env, params.get("main_vm"))
> +    if not vm:
> +        raise error.TestError("VM object not found in environment")
> +    if not vm.is_alive():
> +        raise error.TestError("VM seems to be dead; Test requires a living VM")
> +
> +    logging.info("Waiting for guest to be up...")
> +
> +    session = kvm_utils.wait_for(vm.remote_login, 240, 0, 2)
> +    if not session:
> +        raise error.TestFail("Could not log into guest")
> +
> +    try:
> +        logging.info("Logged in; starting script...")
> +
> +        # Collect test parameters
> +        binary = params.get("autoit_binary")
> +        script = params.get("autoit_script")
> +        script_params = params.get("autoit_script_params", "")
> +        timeout = float(params.get("autoit_script_timeout", 600))
> +
> +        # Send AutoIt script to guest (this code will be replaced once we
> +        # support sending files to Windows guests)
> +        session.sendline("del script.au3")
> +        file = open(kvm_utils.get_path(test.bindir, script))
> +        for line in file.readlines():
> +            # Insert a '^' before each character
> +            line = "".join("^" + c for c in line.rstrip())
> +            if line:
> +                # Append line to the file
> +                session.sendline("echo %s>>script.au3" % line)
> +        file.close()
> +
> +        session.read_up_to_prompt()
> +
> +        command = "cmd /c %s script.au3 %s" % (binary, script_params)

Hi Michael, for the problem that execute script in Windows cmd shell, I
have some information share with you:

Guys in our team had found that the value which `echo %errorlevel%`
returns is not always right. It just reflects whether the action to
execute the script has been implemented successfully and it ALWAYS return
even if errors occur. That means as soon as the script has been started
successfully it will return 0 even if error occurred during script running.

One solution could be use command 'start /wait script.au3' which will
let program wait for it to terminate:
http://ss64.com/nt/start.html

I have not investigated it enough as well, if any mistake made, please
just ignore this reply. ;-)


> +
> +        logging.info("---------------- Script output ----------------")
> +        status = session.get_command_status(command,
> +                                            print_func=logging.info,
> +                                            timeout=timeout)
> +        logging.info("---------------- End of script output ----------------")
> +
> +        if status is None:
> +            raise error.TestFail("Timeout expired before script execution "
> +                                 "completed (or something weird happened)")
> +        if status != 0:
> +            raise error.TestFail("Script execution failed")
> +
> +    finally:
> +        session.close()
> -- 
> 1.5.4.1
> 
> _______________________________________________
> Autotest mailing list
> Autotest@test.kernel.org
> http://test.kernel.org/cgi-bin/mailman/listinfo/autotest

Re: [KVM-AUTOTEST PATCH v2 3/3] KVM test: add AutoIt test to kvm_tests.cfg.sample

[Cao, Chen]

On Tue, Aug 11, 2009 at 03:10:44PM +0300, Michael Goldish wrote:
> Signed-off-by: Michael Goldish <mgoldish@redhat.com>
> ---
>  client/tests/kvm/kvm_tests.cfg.sample |   12 +++++++++++-
>  1 files changed, 11 insertions(+), 1 deletions(-)
> 
> diff --git a/client/tests/kvm/kvm_tests.cfg.sample b/client/tests/kvm/kvm_tests.cfg.sample
> index 12e907b..05a1ca7 100644
> --- a/client/tests/kvm/kvm_tests.cfg.sample
> +++ b/client/tests/kvm/kvm_tests.cfg.sample
> @@ -126,6 +126,16 @@ variants:
>          kill_vm_gracefully = no
>          extra_params += " -snapshot"
>  
> +    - autoit:       install setup
> +        type = autoit
> +        cdrom = windows/autoit.iso
I am wondering what is the advantage of using .iso to store the binary
files. I think it might be hard to maintain the (scripts/binary)
files.
is it possible to pass a directory to run_autoit()?

> +        autoit_binary = D:\AutoIt3.exe
> +        autoit_script_timeout = 600
> +        autoit_script_params =
> +        variants:
> +            - notepad:
> +                autoit_script = autoit/notepad1.au3
> +... blahblah...

Besides, I think we may use staf framework to send command to MS
Windows
to run autoit scripts, it would be easier to control the status of the
autoit process.

what is your suggestion?

Thank you.


Regards,

Cao, Chen
2009/08/12

Re: [KVM-AUTOTEST PATCH v2 3/3] KVM test: add AutoIt test to kvm_tests.cfg.sample

[Michael Goldish]

----- "Chen Cao" <kcao@redhat.com> wrote:

> On Tue, Aug 11, 2009 at 03:10:44PM +0300, Michael Goldish wrote:
> > Signed-off-by: Michael Goldish <mgoldish@redhat.com>
> > ---
> >  client/tests/kvm/kvm_tests.cfg.sample |   12 +++++++++++-
> >  1 files changed, 11 insertions(+), 1 deletions(-)
> > 
> > diff --git a/client/tests/kvm/kvm_tests.cfg.sample
> b/client/tests/kvm/kvm_tests.cfg.sample
> > index 12e907b..05a1ca7 100644
> > --- a/client/tests/kvm/kvm_tests.cfg.sample
> > +++ b/client/tests/kvm/kvm_tests.cfg.sample
> > @@ -126,6 +126,16 @@ variants:
> >          kill_vm_gracefully = no
> >          extra_params += " -snapshot"
> >  
> > +    - autoit:       install setup
> > +        type = autoit
> > +        cdrom = windows/autoit.iso
> I am wondering what is the advantage of using .iso to store the
> binary
> files. I think it might be hard to maintain the (scripts/binary)
> files.
> is it possible to pass a directory to run_autoit()?

- I don't think it's too much trouble to maintain one big ISO that
  will serve all Windows tests.  I think we should start documenting
  its contents in the wiki.
  (Currently, by default, each test that requires a binary uses a
  different ISO file (rss.iso, vlc.iso, autoit.iso) but I think it's
  a good idea to put everything in a single ISO.)

- qemu doesn't directly read from directories, as far as I know.
  It requires an image file (ISO).

- We currently don't support sending files to all Windows guests.
  (We have no solution for Vista and 2008, which are quite important,
  and the solution I'm currently aiming for is adding send/receive
  functionality to rss.exe.)

> > +        autoit_binary = D:\AutoIt3.exe
> > +        autoit_script_timeout = 600
> > +        autoit_script_params =
> > +        variants:
> > +            - notepad:
> > +                autoit_script = autoit/notepad1.au3
> > +... blahblah...
> 
> Besides, I think we may use staf framework to send command to MS
> Windows
> to run autoit scripts, it would be easier to control the status of
> the
> autoit process.

What advantages does STAF have?  I don't think we lack control over the
AutoIt process, especially since by default, rss.exe kills all processes
(AutoIt in this case) started by a session when that session exits.
This can also be done manually, in case rss.exe isn't used, but I left
that to a future patch.

STAF is non-interactive, which means working with it will be different
from working with SSH, Telnet or rss.exe.  Do you suggest that we use
STAF for everything, or just for the AutoIt tests?
Using it for everything means we wouldn't have shell interactivity;
using it for AutoIt makes sense only if STAF is really that much better
for AutoIt tests than regular interactive shells.

> what is your suggestion?
> 
> Thank you.
> 
> 
> Regards,
> 
> Cao, Chen
> 2009/08/12

Thanks,
Michael

Re: [KVM-AUTOTEST PATCH v2 3/3] KVM test: add AutoIt test to kvm_tests.cfg.sample

[Cao, Chen]

On Wed, Aug 12, 2009 at 05:44:34AM -0400, Michael Goldish wrote:
> 
> ----- "Chen Cao" <kcao@redhat.com> wrote:
> 
> > On Tue, Aug 11, 2009 at 03:10:44PM +0300, Michael Goldish wrote:
> > > Signed-off-by: Michael Goldish <mgoldish@redhat.com>
> > > ---
> > >  client/tests/kvm/kvm_tests.cfg.sample |   12 +++++++++++-
> > >  1 files changed, 11 insertions(+), 1 deletions(-)
> > > 
> > > diff --git a/client/tests/kvm/kvm_tests.cfg.sample
> > b/client/tests/kvm/kvm_tests.cfg.sample
> > > index 12e907b..05a1ca7 100644
> > > --- a/client/tests/kvm/kvm_tests.cfg.sample
> > > +++ b/client/tests/kvm/kvm_tests.cfg.sample
> > > @@ -126,6 +126,16 @@ variants:
> > >          kill_vm_gracefully = no
> > >          extra_params += " -snapshot"
> > >  
> > > +    - autoit:       install setup
> > > +        type = autoit
> > > +        cdrom = windows/autoit.iso
> > I am wondering what is the advantage of using .iso to store the
> > binary
> > files. I think it might be hard to maintain the (scripts/binary)
> > files.
> > is it possible to pass a directory to run_autoit()?
> 
> - I don't think it's too much trouble to maintain one big ISO that
>   will serve all Windows tests.  I think we should start documenting
>   its contents in the wiki.
>   (Currently, by default, each test that requires a binary uses a
>   different ISO file (rss.iso, vlc.iso, autoit.iso) but I think it's
>   a good idea to put everything in a single ISO.)
> 
if the autoit scripts are changed, the iso has to be updated everytime
before testing, have you already got a good solution for this?

> - qemu doesn't directly read from directories, as far as I know.
>   It requires an image file (ISO).
> 
> - We currently don't support sending files to all Windows guests.
>   (We have no solution for Vista and 2008, which are quite important,
>   and the solution I'm currently aiming for is adding send/receive
>   functionality to rss.exe.)
> 
> > > +        autoit_binary = D:\AutoIt3.exe
> > > +        autoit_script_timeout = 600
> > > +        autoit_script_params =
> > > +        variants:
> > > +            - notepad:
> > > +                autoit_script = autoit/notepad1.au3
> > > +... blahblah...
> > 
> > Besides, I think we may use staf framework to send command to MS
> > Windows
> > to run autoit scripts, it would be easier to control the status of
> > the
> > autoit process.
> 
> What advantages does STAF have?  I don't think we lack control over the
> AutoIt process, especially since by default, rss.exe kills all processes
> (AutoIt in this case) started by a session when that session exits.
> This can also be done manually, in case rss.exe isn't used, but I left
> that to a future patch.
> 
> STAF is non-interactive, which means working with it will be different
> from working with SSH, Telnet or rss.exe.  Do you suggest that we use
> STAF for everything, or just for the AutoIt tests?
> Using it for everything means we wouldn't have shell interactivity;
> using it for AutoIt makes sense only if STAF is really that much better
> for AutoIt tests than regular interactive shells.
> 
I meant STAF for AutoIt only.
An advanced rss.exe may be the ultimate solution for MS Windows,
and life would be much easier.

but before that, we may also use some other tools to send file to guests,
and STAF to control the autoit tests. we just need to launch the test
and query the status, STAF is enough.

Nevertheless, I'm eager to see the advanced rss.exe.

Thank you.
 

Regards,

Cao, Chen
2009/08/12

Re: [KVM-AUTOTEST PATCH v2 3/3] KVM test: add AutoIt test to kvm_tests.cfg.sample

[Michael Goldish]

----- "Chen Cao" <kcao@redhat.com> wrote:

> On Wed, Aug 12, 2009 at 05:44:34AM -0400, Michael Goldish wrote:
> > 
> > ----- "Chen Cao" <kcao@redhat.com> wrote:
> > 
> > > On Tue, Aug 11, 2009 at 03:10:44PM +0300, Michael Goldish wrote:
> > > > Signed-off-by: Michael Goldish <mgoldish@redhat.com>
> > > > ---
> > > >  client/tests/kvm/kvm_tests.cfg.sample |   12 +++++++++++-
> > > >  1 files changed, 11 insertions(+), 1 deletions(-)
> > > > 
> > > > diff --git a/client/tests/kvm/kvm_tests.cfg.sample
> > > b/client/tests/kvm/kvm_tests.cfg.sample
> > > > index 12e907b..05a1ca7 100644
> > > > --- a/client/tests/kvm/kvm_tests.cfg.sample
> > > > +++ b/client/tests/kvm/kvm_tests.cfg.sample
> > > > @@ -126,6 +126,16 @@ variants:
> > > >          kill_vm_gracefully = no
> > > >          extra_params += " -snapshot"
> > > >  
> > > > +    - autoit:       install setup
> > > > +        type = autoit
> > > > +        cdrom = windows/autoit.iso
> > > I am wondering what is the advantage of using .iso to store the
> > > binary
> > > files. I think it might be hard to maintain the (scripts/binary)
> > > files.
> > > is it possible to pass a directory to run_autoit()?
> > 
> > - I don't think it's too much trouble to maintain one big ISO that
> >   will serve all Windows tests.  I think we should start
> documenting
> >   its contents in the wiki.
> >   (Currently, by default, each test that requires a binary uses a
> >   different ISO file (rss.iso, vlc.iso, autoit.iso) but I think
> it's
> >   a good idea to put everything in a single ISO.)
> > 
> if the autoit scripts are changed, the iso has to be updated
> everytime
> before testing, have you already got a good solution for this?

The AutoIt test already sends the scripts to the guest at runtime.
Only the binaries need to be in an ISO.
The method of sending the scripts is slightly hackish (using echo),
but in my opinion it is robust and I don't expect it to cause any
trouble.

> > - qemu doesn't directly read from directories, as far as I know.
> >   It requires an image file (ISO).
> > 
> > - We currently don't support sending files to all Windows guests.
> >   (We have no solution for Vista and 2008, which are quite
> important,
> >   and the solution I'm currently aiming for is adding send/receive
> >   functionality to rss.exe.)
> > 
> > > > +        autoit_binary = D:\AutoIt3.exe
> > > > +        autoit_script_timeout = 600
> > > > +        autoit_script_params =
> > > > +        variants:
> > > > +            - notepad:
> > > > +                autoit_script = autoit/notepad1.au3
> > > > +... blahblah...
> > > 
> > > Besides, I think we may use staf framework to send command to MS
> > > Windows
> > > to run autoit scripts, it would be easier to control the status
> of
> > > the
> > > autoit process.
> > 
> > What advantages does STAF have?  I don't think we lack control over
> the
> > AutoIt process, especially since by default, rss.exe kills all
> processes
> > (AutoIt in this case) started by a session when that session exits.
> > This can also be done manually, in case rss.exe isn't used, but I
> left
> > that to a future patch.
> > 
> > STAF is non-interactive, which means working with it will be
> different
> > from working with SSH, Telnet or rss.exe.  Do you suggest that we
> use
> > STAF for everything, or just for the AutoIt tests?
> > Using it for everything means we wouldn't have shell interactivity;
> > using it for AutoIt makes sense only if STAF is really that much
> better
> > for AutoIt tests than regular interactive shells.
> > 
> I meant STAF for AutoIt only.
> An advanced rss.exe may be the ultimate solution for MS Windows,
> and life would be much easier.
> 
> but before that, we may also use some other tools to send file to
> guests,
> and STAF to control the autoit tests. we just need to launch the test
> and query the status, STAF is enough.

Everything you mentioned is already being done using rss.exe, except
for sending binaries to the guest.
The AutoIt test sends the script, launches it and queries the exit
status.
I can see why STAF is enough to control the tests, I just don't see how
it is better.

> Nevertheless, I'm eager to see the advanced rss.exe.
> 
> Thank you.
>  
> 
> Regards,
> 
> Cao, Chen
> 2009/08/12

Re: [KVM-AUTOTEST PATCH v2 3/3] KVM test: add AutoIt test to kvm_tests.cfg.sample

[Cao, Chen]

On Wed, Aug 12, 2009 at 07:29:25AM -0400, Michael Goldish wrote:
> 
> ----- "Chen Cao" <kcao@redhat.com> wrote:
> 
> > On Wed, Aug 12, 2009 at 05:44:34AM -0400, Michael Goldish wrote:
> > > 
> > > ----- "Chen Cao" <kcao@redhat.com> wrote:
> > > 
> > > > On Tue, Aug 11, 2009 at 03:10:44PM +0300, Michael Goldish wrote:
> > > > > Signed-off-by: Michael Goldish <mgoldish@redhat.com>
> > > > > ---
> > > > >  client/tests/kvm/kvm_tests.cfg.sample |   12 +++++++++++-
> > > > >  1 files changed, 11 insertions(+), 1 deletions(-)
> > > > > 
> > > > > diff --git a/client/tests/kvm/kvm_tests.cfg.sample
> > > > b/client/tests/kvm/kvm_tests.cfg.sample
> > > > > index 12e907b..05a1ca7 100644
> > > > > --- a/client/tests/kvm/kvm_tests.cfg.sample
> > > > > +++ b/client/tests/kvm/kvm_tests.cfg.sample
> > > > > @@ -126,6 +126,16 @@ variants:
> > > > >          kill_vm_gracefully = no
> > > > >          extra_params += " -snapshot"
> > > > >  
> > > > > +    - autoit:       install setup
> > > > > +        type = autoit
> > > > > +        cdrom = windows/autoit.iso
> > > > I am wondering what is the advantage of using .iso to store the
> > > > binary
> > > > files. I think it might be hard to maintain the (scripts/binary)
> > > > files.
> > > > is it possible to pass a directory to run_autoit()?
> > > 
> > > - I don't think it's too much trouble to maintain one big ISO that
> > >   will serve all Windows tests.  I think we should start
> > documenting
> > >   its contents in the wiki.
> > >   (Currently, by default, each test that requires a binary uses a
> > >   different ISO file (rss.iso, vlc.iso, autoit.iso) but I think
> > it's
> > >   a good idea to put everything in a single ISO.)
> > > 
> > if the autoit scripts are changed, the iso has to be updated
> > everytime
> > before testing, have you already got a good solution for this?
> 
> The AutoIt test already sends the scripts to the guest at runtime.
> Only the binaries need to be in an ISO.
> The method of sending the scripts is slightly hackish (using echo),
> but in my opinion it is robust and I don't expect it to cause any
> trouble.

it seems that this testcase cannot handle more one one .au3 files.
and i think it quite possible that there are testcases that require
two or more .au3 files, for reusing code or some other reasons.

> 
> > > - qemu doesn't directly read from directories, as far as I know.
> > >   It requires an image file (ISO).
> > > 
> > > - We currently don't support sending files to all Windows guests.
> > >   (We have no solution for Vista and 2008, which are quite
> > important,
> > >   and the solution I'm currently aiming for is adding send/receive
> > >   functionality to rss.exe.)
> > > 
> > > > > +        autoit_binary = D:\AutoIt3.exe
> > > > > +        autoit_script_timeout = 600
> > > > > +        autoit_script_params =
> > > > > +        variants:
> > > > > +            - notepad:
> > > > > +                autoit_script = autoit/notepad1.au3
> > > > > +... blahblah...
> > > > 
> > > > Besides, I think we may use staf framework to send command to MS
> > > > Windows
> > > > to run autoit scripts, it would be easier to control the status
> > of
> > > > the
> > > > autoit process.
> > > 
> > > What advantages does STAF have?  I don't think we lack control over
> > the
> > > AutoIt process, especially since by default, rss.exe kills all
> > processes
> > > (AutoIt in this case) started by a session when that session exits.
> > > This can also be done manually, in case rss.exe isn't used, but I
> > left
> > > that to a future patch.
> > > 
> > > STAF is non-interactive, which means working with it will be
> > different
> > > from working with SSH, Telnet or rss.exe.  Do you suggest that we
> > use
> > > STAF for everything, or just for the AutoIt tests?
> > > Using it for everything means we wouldn't have shell interactivity;
> > > using it for AutoIt makes sense only if STAF is really that much
> > better
> > > for AutoIt tests than regular interactive shells.
> > > 
> > I meant STAF for AutoIt only.
> > An advanced rss.exe may be the ultimate solution for MS Windows,
> > and life would be much easier.
> > 
> > but before that, we may also use some other tools to send file to
> > guests,
> > and STAF to control the autoit tests. we just need to launch the test
> > and query the status, STAF is enough.
> 
> Everything you mentioned is already being done using rss.exe, except
> for sending binaries to the guest.
> The AutoIt test sends the script, launches it and queries the exit
> status.
> I can see why STAF is enough to control the tests, I just don't see how
> it is better.
> 

Since there is no further demand, I cannot tell which is better now.
for now, we do no need to speed time to maintain STAF, while we can get
more features, if needed, with rss.exe. However, they are just auxiliary
tools, and, as a user, i just choose the one that makes things run.

And, Thank you for the detailed explaination.


Regards,
 
Cao, Chen
2009/08/13

Re: [KVM-AUTOTEST PATCH v2 3/3] KVM test: add AutoIt test to kvm_tests.cfg.sample

[Michael Goldish]

----- "Chen Cao" <kcao@redhat.com> wrote:

> On Wed, Aug 12, 2009 at 07:29:25AM -0400, Michael Goldish wrote:
> > 
> > ----- "Chen Cao" <kcao@redhat.com> wrote:
> > 
> > > On Wed, Aug 12, 2009 at 05:44:34AM -0400, Michael Goldish wrote:
> > > > 
> > > > ----- "Chen Cao" <kcao@redhat.com> wrote:
> > > > 
> > > > > On Tue, Aug 11, 2009 at 03:10:44PM +0300, Michael Goldish
> wrote:
> > > > > > Signed-off-by: Michael Goldish <mgoldish@redhat.com>
> > > > > > ---
> > > > > >  client/tests/kvm/kvm_tests.cfg.sample |   12 +++++++++++-
> > > > > >  1 files changed, 11 insertions(+), 1 deletions(-)
> > > > > > 
> > > > > > diff --git a/client/tests/kvm/kvm_tests.cfg.sample
> > > > > b/client/tests/kvm/kvm_tests.cfg.sample
> > > > > > index 12e907b..05a1ca7 100644
> > > > > > --- a/client/tests/kvm/kvm_tests.cfg.sample
> > > > > > +++ b/client/tests/kvm/kvm_tests.cfg.sample
> > > > > > @@ -126,6 +126,16 @@ variants:
> > > > > >          kill_vm_gracefully = no
> > > > > >          extra_params += " -snapshot"
> > > > > >  
> > > > > > +    - autoit:       install setup
> > > > > > +        type = autoit
> > > > > > +        cdrom = windows/autoit.iso
> > > > > I am wondering what is the advantage of using .iso to store
> the
> > > > > binary
> > > > > files. I think it might be hard to maintain the
> (scripts/binary)
> > > > > files.
> > > > > is it possible to pass a directory to run_autoit()?
> > > > 
> > > > - I don't think it's too much trouble to maintain one big ISO
> that
> > > >   will serve all Windows tests.  I think we should start
> > > documenting
> > > >   its contents in the wiki.
> > > >   (Currently, by default, each test that requires a binary uses
> a
> > > >   different ISO file (rss.iso, vlc.iso, autoit.iso) but I think
> > > it's
> > > >   a good idea to put everything in a single ISO.)
> > > > 
> > > if the autoit scripts are changed, the iso has to be updated
> > > everytime
> > > before testing, have you already got a good solution for this?
> > 
> > The AutoIt test already sends the scripts to the guest at runtime.
> > Only the binaries need to be in an ISO.
> > The method of sending the scripts is slightly hackish (using echo),
> > but in my opinion it is robust and I don't expect it to cause any
> > trouble.
> 
> it seems that this testcase cannot handle more one one .au3 files.
> and i think it quite possible that there are testcases that require
> two or more .au3 files, for reusing code or some other reasons.

OK, that makes sense.

I'll send a patch that lets users specify additional files to send in a
parameter named 'autoit_script_deps' or something like that.

In any case, there's plenty that can be done with single file scripts,
and I hope to see us get to the point where we actually need the multi-
file functionality.  We still have a long way to go from the sample
notepad1.au3 script.

> > 
> > > > - qemu doesn't directly read from directories, as far as I
> know.
> > > >   It requires an image file (ISO).
> > > > 
> > > > - We currently don't support sending files to all Windows
> guests.
> > > >   (We have no solution for Vista and 2008, which are quite
> > > important,
> > > >   and the solution I'm currently aiming for is adding
> send/receive
> > > >   functionality to rss.exe.)
> > > > 
> > > > > > +        autoit_binary = D:\AutoIt3.exe
> > > > > > +        autoit_script_timeout = 600
> > > > > > +        autoit_script_params =
> > > > > > +        variants:
> > > > > > +            - notepad:
> > > > > > +                autoit_script = autoit/notepad1.au3
> > > > > > +... blahblah...
> > > > > 
> > > > > Besides, I think we may use staf framework to send command to
> MS
> > > > > Windows
> > > > > to run autoit scripts, it would be easier to control the
> status
> > > of
> > > > > the
> > > > > autoit process.
> > > > 
> > > > What advantages does STAF have?  I don't think we lack control
> over
> > > the
> > > > AutoIt process, especially since by default, rss.exe kills all
> > > processes
> > > > (AutoIt in this case) started by a session when that session
> exits.
> > > > This can also be done manually, in case rss.exe isn't used, but
> I
> > > left
> > > > that to a future patch.
> > > > 
> > > > STAF is non-interactive, which means working with it will be
> > > different
> > > > from working with SSH, Telnet or rss.exe.  Do you suggest that
> we
> > > use
> > > > STAF for everything, or just for the AutoIt tests?
> > > > Using it for everything means we wouldn't have shell
> interactivity;
> > > > using it for AutoIt makes sense only if STAF is really that
> much
> > > better
> > > > for AutoIt tests than regular interactive shells.
> > > > 
> > > I meant STAF for AutoIt only.
> > > An advanced rss.exe may be the ultimate solution for MS Windows,
> > > and life would be much easier.
> > > 
> > > but before that, we may also use some other tools to send file to
> > > guests,
> > > and STAF to control the autoit tests. we just need to launch the
> test
> > > and query the status, STAF is enough.
> > 
> > Everything you mentioned is already being done using rss.exe,
> except
> > for sending binaries to the guest.
> > The AutoIt test sends the script, launches it and queries the exit
> > status.
> > I can see why STAF is enough to control the tests, I just don't see
> how
> > it is better.
> > 
> 
> Since there is no further demand, I cannot tell which is better now.
> for now, we do no need to speed time to maintain STAF, while we can
> get
> more features, if needed, with rss.exe. However, they are just
> auxiliary
> tools, and, as a user, i just choose the one that makes things run.
> 
> And, Thank you for the detailed explaination.
> 
> 
> Regards,
>  
> Cao, Chen
> 2009/08/13

Re: [Autotest] [KVM-AUTOTEST PATCH v2 0/3] KVM test: corrections to the AutoIt patchset

[Lucas Meneghel Rodrigues]

On Tue, Aug 11, 2009 at 9:10 AM, Michael Goldish<mgoldish@redhat.com> wrote:
>
> This is the AutoIt patch set with some minor corrections:
> - Use read_up_to_prompt() instead of read_nonblocking() before running the
>  AutoIt command
> - autoit_script_params defaults to "" in the test code as well
> - Indentation fix in notepad1.au3

Ok, test looks good, I tested the simple notepad script, works ok. Applied.

With this the patch queue is significantly smaller, and I have no
pending patches from you Michael (I believe). I will work on an
updated patch queue document and follow up with other important
patchsets.

Lucas