[PATCH 01/19] KVM test: kvm_utils.py: make verify_ip_address_ownership() more robust

[PATCH 01/19] KVM test: kvm_utils.py: make verify_ip_address_ownership() more robust

[Michael Goldish]

Use arping in addition to querying the arp cache.
Under certain circumstances a TCP connection may not trigger an ARP request so
it must be triggered explicitly using arping.

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm_utils.py |   40 ++++++++++++++++++----------------------
 1 files changed, 18 insertions(+), 22 deletions(-)

diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py
index dfca938..f046810 100644
--- a/client/tests/kvm/kvm_utils.py
+++ b/client/tests/kvm/kvm_utils.py
@@ -152,39 +152,35 @@ def get_mac_ip_pair_from_dict(dict):
     return (None, None)
 
 
-def verify_ip_address_ownership(ip, macs, timeout=3.0):
+def verify_ip_address_ownership(ip, macs, timeout=10.0):
     """
-    Connect to a given IP address and make sure its MAC address equals one of
-    the given MAC address.
+    Use arping and the ARP cache to make sure a given IP address belongs to one
+    of the given MAC addresses.
 
     @param ip: An IP address.
     @param macs: A list or tuple of MAC addresses.
     @return: True iff ip is assigned to a MAC address in macs.
     """
-    def check_arp_cache(regex):
-        o = commands.getoutput("/sbin/arp -n")
-        return bool(re.search(regex, o, re.IGNORECASE))
-
+    # Compile a regex that matches the given IP address and any of the given
+    # MAC addresses
     mac_regex = "|".join("(%s)" % mac for mac in macs)
     regex = re.compile(r"\b%s\b.*\b(%s)\b" % (ip, mac_regex))
 
-    if check_arp_cache(regex):
+    # Check the ARP cache
+    o = commands.getoutput("/sbin/arp -n")
+    if re.search(regex, o, re.IGNORECASE):
         return True
 
-    s = socket.socket()
-    s.setblocking(False)
-    try:
-        s.connect((ip, 55555))
-    except socket.error:
-        pass
-    end_time = time.time() + timeout
-    while time.time() < end_time:
-        time.sleep(0.2)
-        if check_arp_cache(regex):
-            s.close()
-            return True
-    s.close()
-    return False
+    # Get the name of the bridge device for arping
+    o = commands.getoutput("/sbin/ip route get %s" % ip)
+    dev = re.findall("dev\s+\S+", o, re.IGNORECASE)
+    if not dev:
+        return False
+    dev = dev[0].split()[-1]
+
+    # Send an ARP request
+    o = commands.getoutput("/sbin/arping -f -c 3 -I %s %s" % (dev, ip))
+    return bool(re.search(regex, o, re.IGNORECASE))
 
 
 # Functions for working with the environment (a dict-like object)
-- 
1.5.4.1

[PATCH 02/19] KVM test: Migration test cleanup

[Michael Goldish]

This should hopefully make the test code a little more readable.

The test is now also more similar to other tests in that it uses a single
'main_vm', which gets cloned, instead of a source and a destination VM.
(This will allow to easily run multiple migration iterations.)

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm_tests.cfg.sample |   11 +----
 client/tests/kvm/kvm_tests.py         |   86 ++++++++++----------------------
 2 files changed, 28 insertions(+), 69 deletions(-)

diff --git a/client/tests/kvm/kvm_tests.cfg.sample b/client/tests/kvm/kvm_tests.cfg.sample
index fdf2963..73e929f 100644
--- a/client/tests/kvm/kvm_tests.cfg.sample
+++ b/client/tests/kvm/kvm_tests.cfg.sample
@@ -63,18 +63,9 @@ variants:
 
     - migrate:      install setup
         type = migration
-        vms += " dst"
         migration_test_command = help
         kill_vm_on_error = yes
-        variants:
-            - 1:
-                start_vm_for_migration_dst = yes
-                migration_src = vm1
-                migration_dst = dst
-            - 2: 1
-                start_vm_for_migration_vm1 = yes
-                migration_src = dst
-                migration_dst = vm1
+        iterations = 2
 
     - autotest:     install setup
         type = autotest
diff --git a/client/tests/kvm/kvm_tests.py b/client/tests/kvm/kvm_tests.py
index 4e3391e..1ee7775 100644
--- a/client/tests/kvm/kvm_tests.py
+++ b/client/tests/kvm/kvm_tests.py
@@ -99,45 +99,33 @@ def run_shutdown(test, params, env):
 def run_migration(test, params, env):
     """
     KVM migration test:
-
-    1) Get two live VMs. One will be the 'source', the other will be the
-    'destination'.
-    2) Verify if the source VM supports migration. If it does, proceed with
-    the test
-    3) Send a migration command to the source vm and wait until it's finished.
-    4) Kill off the source vm
-    3) Log into the destination vm after the migration is finished.
+    1) Get a live VM and clone it.
+    2) Verify that the source VM supports migration.  If it does, proceed with
+            the test.
+    3) Send a migration command to the source VM and wait until it's finished.
+    4) Kill off the source VM.
+    3) Log into the destination VM after the migration is finished.
     4) Compare the output of a reference command executed on the source with
-    the output of the same command on the destination machine
+            the output of the same command on the destination machine.
 
     @param test: kvm test object.
     @param params: Dictionary with test parameters.
     @param env: Dictionary with the test environment.
     """
-    src_vm_name = params.get("migration_src")
-    vm = kvm_utils.env_get_vm(env, src_vm_name)
+    vm = kvm_utils.env_get_vm(env, params.get("main_vm"))
     if not vm:
-        raise error.TestError("VM '%s' not found in environment" % src_vm_name)
+        raise error.TestError("VM object not found in environment")
     if not vm.is_alive():
-        raise error.TestError("VM '%s' seems to be dead; Test requires a"
-                              " living VM" % src_vm_name)
-
-    dest_vm_name = params.get("migration_dst")
-    dest_vm = kvm_utils.env_get_vm(env, dest_vm_name)
-    if not dest_vm:
-        raise error.TestError("VM '%s' not found in environment" % dest_vm_name)
-    if not dest_vm.is_alive():
-        raise error.TestError("VM '%s' seems to be dead; Test requires a"
-                              " living VM" % dest_vm_name)
-
-    pre_scrdump_filename = os.path.join(test.debugdir, "migration_pre.ppm")
-    post_scrdump_filename = os.path.join(test.debugdir, "migration_post.ppm")
+        raise error.TestError("VM seems to be dead; Test requires a living VM")
 
     # See if migration is supported
     s, o = vm.send_monitor_cmd("help info")
     if not "info migrate" in o:
         raise error.TestError("Migration is not supported")
 
+    dest_vm = vm.clone()
+    dest_vm.create(for_migration=True)
+
     # Log into guest and get the output of migration_test_command
     logging.info("Waiting for guest to be up...")
 
@@ -165,54 +153,32 @@ def run_migration(test, params, env):
     # Define some helper functions
     def mig_finished():
         s, o = vm.send_monitor_cmd("info migrate")
-        if s:
-            return False
-        if "Migration status: active" in o:
-            return False
-        return True
+        return s == 0 and not "Migration status: active" in o
 
     def mig_succeeded():
         s, o = vm.send_monitor_cmd("info migrate")
-        if s == 0 and "Migration status: completed" in o:
-            return True
-        return False
+        return s == 0 and "Migration status: completed" in o
 
     def mig_failed():
         s, o = vm.send_monitor_cmd("info migrate")
-        if s == 0 and "Migration status: failed" in o:
-            return True
-        return False
+        return s == 0 and "Migration status: failed" in o
 
     # Wait for migration to finish
     if not kvm_utils.wait_for(mig_finished, 90, 2, 2,
                               "Waiting for migration to finish..."):
-        raise error.TestFail("Timeout elapsed while waiting for migration to"
+        raise error.TestFail("Timeout elapsed while waiting for migration to "
                              "finish")
 
     # Report migration status
     if mig_succeeded():
         logging.info("Migration finished successfully")
+    elif mig_failed():
+        raise error.TestFail("Migration failed")
     else:
-        if mig_failed():
-            message = "Migration failed"
-        else:
-            message = "Migration ended with unknown status"
-        raise error.TestFail(message)
-
-    # Get 'post' screendump
-    dest_vm.send_monitor_cmd("screendump %s" % post_scrdump_filename)
-
-    # Get 'pre' screendump
-    vm.send_monitor_cmd("screendump %s" % pre_scrdump_filename)
+        raise error.TestFail("Migration ended with unknown status")
 
     # Kill the source VM
-    vm.send_monitor_cmd("quit", block=False)
-
-    # Hack: it seems that the first attempt to communicate with the SSH port
-    # following migration always fails (or succeeds after a very long time).
-    # So just connect to the port once so the following call to remote_login
-    # succeeds.
-    dest_vm.is_sshd_running(timeout=0.0)
+    vm.destroy(gracefully=False)
 
     # Log into guest and get the output of migration_test_command
     logging.info("Logging into guest after migration...")
@@ -228,14 +194,16 @@ def run_migration(test, params, env):
 
     # Compare output to reference output
     if output != reference_output:
-        logging.info("Command output before migration differs from command"
-                     " output after migration")
+        logging.info("Command output before migration differs from command "
+                     "output after migration")
         logging.info("Command: %s" % params.get("migration_test_command"))
         logging.info("Output before:" +
                      kvm_utils.format_str_for_message(reference_output))
         logging.info("Output after:" + kvm_utils.format_str_for_message(output))
-        raise error.TestFail("Command produced different output before and"
-                             " after migration")
+        raise error.TestFail("Command produced different output before and "
+                             "after migration")
+
+    kvm_utils.env_register_vm(env, params.get("main_vm"), dest_vm)
 
 
 def run_autotest(test, params, env):
-- 
1.5.4.1

[PATCH 03/19] KVM test: scan_results.py: allow parsing and printing of multiple result files

[Michael Goldish]

This is useful for printing results of several jobs executing in parallel
(using job.parallel()).

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/scan_results.py |   42 +++++++++++++++++++------------------
 1 files changed, 22 insertions(+), 20 deletions(-)

diff --git a/client/tests/kvm/scan_results.py b/client/tests/kvm/scan_results.py
index 95507bd..a22634e 100755
--- a/client/tests/kvm/scan_results.py
+++ b/client/tests/kvm/scan_results.py
@@ -51,28 +51,30 @@ def print_result(result):
         print '%-48s\t\t%s\t%s\t%s' % tuple(map(str, result))
 
 
-def main(resfile):
-    print_result(('test', 'status', 'seconds', 'info'))
+def main(resfiles):
+    print_result(('Test', 'Status', 'Seconds', 'Info'))
     print_result(('----', '------', '-------', '----'))
 
-    f = file(resfile)
-    text = f.read()
-    f.close()
-
-    results = parse_results(text)
-    map(print_result, results)
+    for resfile in resfiles:
+        print '        (Result file: %s)' % resfile
+        try:
+            f = file(resfile)
+            text = f.read()
+            f.close()
+        except IOError:
+            print 'Bad result file: %s' % resfile
+            return
+        results = parse_results(text)
+        map(print_result, results)
 
 
 if __name__ == '__main__':
-    import sys, os
-
-    resfile = '../../results/default/status'
-    if len(sys.argv) == 2:
-        resfile = sys.argv[1]
-    if resfile == '-h' or resfile == '--help' or len(sys.argv) > 2:
-        print 'usage: %s [ <resfile> ]' % sys.argv[0]
-        sys.exit(0)
-    if not os.path.exists(resfile):
-        print 'Bad result file: %s' % resfile
-        sys.exit(1)
-    main(resfile)
+    import sys, os, glob
+
+    resfiles = glob.glob('../../results/default/status*')
+    if len(sys.argv) > 1:
+        if sys.argv[1] == '-h' or sys.argv[1] == '--help':
+            print 'Usage: %s [result files]' % sys.argv[0]
+            sys.exit(0)
+        resfiles = sys.argv[1:]
+    main(resfiles)
-- 
1.5.4.1

[PATCH 04/19] KVM test: kvm_utils.py: add kill_process_tree()

[Michael Goldish]

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm_utils.py |   17 +++++++++++++++++
 1 files changed, 17 insertions(+), 0 deletions(-)

diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py
index f046810..f686a53 100644
--- a/client/tests/kvm/kvm_utils.py
+++ b/client/tests/kvm/kvm_utils.py
@@ -265,6 +265,23 @@ def safe_kill(pid, signal):
         return False
 
 
+def kill_process_tree(pid, sig=signal.SIGKILL):
+    """Signal a process and all of its children.
+
+    If the process does not exist -- return.
+
+    @param pid: The pid of the process to signal.
+    @param sig: The signal to send to the processes.
+    """
+    if not safe_kill(pid, signal.SIGSTOP):
+        return
+    children = commands.getoutput("ps --ppid=%d -o pid=" % pid).split()
+    for child in children:
+        kill_process_tree(int(child), sig)
+    safe_kill(pid, sig)
+    safe_kill(pid, signal.SIGCONT)
+
+
 def get_latest_kvm_release_tag(release_dir):
     """
     Fetches the latest release tag for KVM.
-- 
1.5.4.1

[PATCH 05/19] KVM test: kvm_subprocess: use kill_process_tree() to close child processes

[Michael Goldish]

This is safer than just killing the session leader because some processes seem
to remain alive after their session leader has exited.

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm_subprocess.py |    7 ++-----
 client/tests/kvm/kvm_vm.py         |    2 +-
 2 files changed, 3 insertions(+), 6 deletions(-)

diff --git a/client/tests/kvm/kvm_subprocess.py b/client/tests/kvm/kvm_subprocess.py
index 07303a8..ba61a8c 100755
--- a/client/tests/kvm/kvm_subprocess.py
+++ b/client/tests/kvm/kvm_subprocess.py
@@ -367,7 +367,7 @@ class kvm_spawn:
         return _locked(self.lock_server_running_filename)
 
 
-    def close(self, sig=signal.SIGTERM):
+    def close(self, sig=signal.SIGKILL):
         """
         Kill the child process if it's alive and remove temporary files.
 
@@ -375,10 +375,7 @@ class kvm_spawn:
         """
         # Kill it if it's alive
         if self.is_alive():
-            try:
-                os.kill(self.get_shell_pid(), sig)
-            except:
-                pass
+            kvm_utils.kill_process_tree(self.get_shell_pid(), sig)
         # Wait for the server to exit
         _wait(self.lock_server_running_filename)
         # Call all cleanup routines
diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
index bd7e0fc..79ca81d 100755
--- a/client/tests/kvm/kvm_vm.py
+++ b/client/tests/kvm/kvm_vm.py
@@ -541,7 +541,7 @@ class VM:
         # If the VM isn't dead yet...
         logging.debug("Cannot quit normally; sending a kill to close the "
                       "deal...")
-        kvm_utils.safe_kill(self.process.get_pid(), 9)
+        kvm_utils.kill_process_tree(self.process.get_pid(), 9)
         # Wait for the VM to be really dead
         if kvm_utils.wait_for(self.is_dead, 5, 0.5, 0.5):
             logging.debug("VM is down")
-- 
1.5.4.1

[PATCH 06/19] KVM test: timedrift test: set CPU affinity recursively for all children

[Michael Goldish]

In addition to setting the CPU affinity of the given process, set it for all of
its children too.

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm_tests.py |   10 +++++-----
 1 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/client/tests/kvm/kvm_tests.py b/client/tests/kvm/kvm_tests.py
index 1ee7775..deab59e 100644
--- a/client/tests/kvm/kvm_tests.py
+++ b/client/tests/kvm/kvm_tests.py
@@ -578,6 +578,7 @@ def run_timedrift(test, params, env):
     def set_cpu_affinity(pid, mask):
         """
         Set the CPU affinity of all threads of the process with PID pid.
+        Do this recursively for all child processes as well.
 
         @param pid: The process ID.
         @param mask: The CPU affinity mask.
@@ -589,6 +590,9 @@ def run_timedrift(test, params, env):
             prev_mask = commands.getoutput("taskset -p %s" % tid).split()[-1]
             prev_masks[tid] = prev_mask
             commands.getoutput("taskset -p %s %s" % (mask, tid))
+        children = commands.getoutput("ps --ppid=%s -o pid=" % pid).split()
+        for child in children:
+            prev_masks.update(set_cpu_affinity(child, mask))
         return prev_masks
 
     def restore_cpu_affinity(prev_masks):
@@ -683,13 +687,9 @@ def run_timedrift(test, params, env):
                                       output_func=logging.debug,
                                       output_prefix="(host load %d) " % i,
                                       timeout=0.5))
-            # Set the CPU affinity of the shell running the load process
+            # Set the CPU affinity of the load process
             pid = host_load_sessions[-1].get_shell_pid()
             set_cpu_affinity(pid, cpu_mask)
-            # Try setting the CPU affinity of the load process itself
-            pid = host_load_sessions[-1].get_pid()
-            if pid:
-                set_cpu_affinity(pid, cpu_mask)
 
         # Sleep for a while (during load)
         logging.info("Sleeping for %s seconds..." % load_duration)
-- 
1.5.4.1

[PATCH 07/19] KVM test: kvm_subprocess: get rid of get_pid() (replace it with get_shell_pid())

[Michael Goldish]

Don't differentiate between the PID of the subprocess and the PID of the shell
running it.

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm_subprocess.py |   38 +++++------------------------------
 client/tests/kvm/kvm_tests.py      |    2 +-
 2 files changed, 7 insertions(+), 33 deletions(-)

diff --git a/client/tests/kvm/kvm_subprocess.py b/client/tests/kvm/kvm_subprocess.py
index ba61a8c..d142ed9 100755
--- a/client/tests/kvm/kvm_subprocess.py
+++ b/client/tests/kvm/kvm_subprocess.py
@@ -290,10 +290,12 @@ class kvm_spawn:
         return self.id
 
 
-    def get_shell_pid(self):
+    def get_pid(self):
         """
-        Return the PID of the subshell process, or None if not available.
-        The subshell is the shell that runs the command.
+        Return the PID of the process.
+
+        Note: this may be the PID of the shell process running the user given
+        command.
         """
         try:
             file = open(self.shell_pid_filename, "r")
@@ -304,34 +306,6 @@ class kvm_spawn:
             return None
 
 
-    def get_pid(self, index=0):
-        """
-        Try to get and return the PID of a child process of the subshell.
-        This is usually the PID of the process executed in the subshell.
-        There are 3 exceptions:
-            - If the subshell couldn't start the process for some reason, no
-              PID can be returned.
-            - If the subshell is running several processes in parallel,
-              multiple PIDs can be returned.  Use the index parameter in this
-              case.
-            - Before starting the process, after the process has terminated,
-              or while running shell code that doesn't start any processes --
-              no PID can be returned.
-
-        @param index: The index of the child process whose PID is requested.
-                Normally this should remain 0.
-        @return: The PID of the child process, or None if none could be found.
-        """
-        parent_pid = self.get_shell_pid()
-        if not parent_pid:
-            return None
-        pids = commands.getoutput("ps --ppid %d -o pid=" % parent_pid).split()
-        try:
-            return int(pids[index])
-        except:
-            return None
-
-
     def get_status(self):
         """
         Wait for the process to exit and return its exit status, or None
@@ -375,7 +349,7 @@ class kvm_spawn:
         """
         # Kill it if it's alive
         if self.is_alive():
-            kvm_utils.kill_process_tree(self.get_shell_pid(), sig)
+            kvm_utils.kill_process_tree(self.get_pid(), sig)
         # Wait for the server to exit
         _wait(self.lock_server_running_filename)
         # Call all cleanup routines
diff --git a/client/tests/kvm/kvm_tests.py b/client/tests/kvm/kvm_tests.py
index deab59e..c2bfb8a 100644
--- a/client/tests/kvm/kvm_tests.py
+++ b/client/tests/kvm/kvm_tests.py
@@ -688,7 +688,7 @@ def run_timedrift(test, params, env):
                                       output_prefix="(host load %d) " % i,
                                       timeout=0.5))
             # Set the CPU affinity of the load process
-            pid = host_load_sessions[-1].get_shell_pid()
+            pid = host_load_sessions[-1].get_pid()
             set_cpu_affinity(pid, cpu_mask)
 
         # Sleep for a while (during load)
-- 
1.5.4.1

[PATCH 08/19] KVM test: remove unused function is_sshd_running()

[Michael Goldish]

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm_utils.py |   46 -----------------------------------------
 client/tests/kvm/kvm_vm.py    |   15 -------------
 2 files changed, 0 insertions(+), 61 deletions(-)

diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py
index f686a53..ac9ede7 100644
--- a/client/tests/kvm/kvm_utils.py
+++ b/client/tests/kvm/kvm_utils.py
@@ -605,52 +605,6 @@ def netcat(host, port, username, password, prompt, timeout=10):
 
 # The following are utility functions related to ports.
 
-def is_sshd_running(host, port, timeout=10.0):
-    """
-    Connect to the given host and port and wait for output.
-    Return True if the given host and port are responsive.
-
-    @param host: Host's hostname
-    @param port: Host's port
-    @param timeout: Time (seconds) before we giving up on checking the SSH
-            daemon.
-
-    @return: If output is available, return True. If timeout expires and no
-            output was available, return False.
-    """
-    try:
-        # Try to connect
-        #s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-        s = socket.socket()
-        s.connect((host, port))
-    except socket.error:
-        # Can't connect -- return False
-        s.close()
-        logging.debug("Could not connect")
-        return False
-    s.setblocking(False)
-    # Wait up to 'timeout' seconds
-    end_time = time.time() + timeout
-    while time.time() < end_time:
-        try:
-            time.sleep(0.1)
-            # Try to receive some text
-            str = s.recv(1024)
-            if len(str) > 0:
-                s.shutdown(socket.SHUT_RDWR)
-                s.close()
-                logging.debug("Success! got string %r" % str)
-                return True
-        except socket.error:
-            # No text was available; try again
-            pass
-    # Timeout elapsed and no text was received
-    s.shutdown(socket.SHUT_RDWR)
-    s.close()
-    logging.debug("Timeout")
-    return False
-
-
 def is_port_free(port):
     """
     Return True if the given port is available for use.
diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
index 79ca81d..d360d8e 100755
--- a/client/tests/kvm/kvm_vm.py
+++ b/client/tests/kvm/kvm_vm.py
@@ -654,21 +654,6 @@ class VM:
         return self.process.get_pid()
 
 
-    def is_sshd_running(self, timeout=10):
-        """
-        Return True iff the guest's remote shell port is responsive.
-
-        @param timeout: Time (seconds) before giving up checking the daemon's
-                responsiveness.
-        """
-        address = self.get_address()
-        port = self.get_port(int(self.params.get("shell_port")))
-        if not address or not port:
-            logging.debug("IP address or port unavailable")
-            return False
-        return kvm_utils.is_sshd_running(address, port, timeout=timeout)
-
-
     def remote_login(self, nic_index=0, timeout=10):
         """
         Log into the guest via SSH/Telnet/Netcat.
-- 
1.5.4.1

[PATCH 09/19] KVM test: kvm_config.py: remove unused function get_match_block_indices()

[Michael Goldish]

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm_config.py |   27 ---------------------------
 1 files changed, 0 insertions(+), 27 deletions(-)

diff --git a/client/tests/kvm/kvm_config.py b/client/tests/kvm/kvm_config.py
index da7988b..405de4b 100755
--- a/client/tests/kvm/kvm_config.py
+++ b/client/tests/kvm/kvm_config.py
@@ -99,33 +99,6 @@ class config:
         return [dict for dict in list if self.match(filter, dict)]
 
 
-    # Currently unused, will be removed if it remains unused
-    def get_match_block_indices(self, filter, list=None):
-        """
-        Get the indexes of a list that match a given filter.
-
-        @param filter: A regular expression that will filter the list.
-        @param list: List which we want to know the indexes that match a filter.
-        """
-        if list is None:
-            list = self.list
-        block_list = []
-        prev_match = False
-        for index in range(len(list)):
-            dict = list[index]
-            if self.match(filter, dict):
-                if not prev_match:
-                    block_list.append([index])
-                prev_match = True
-            else:
-                if prev_match:
-                    block_list[-1].append(index)
-                prev_match = False
-        if prev_match:
-            block_list[-1].append(len(list))
-        return block_list
-
-
     def split_and_strip(self, str, sep="="):
         """
         Split str and strip quotes from the resulting parts.
-- 
1.5.4.1

[PATCH 10/19] KVM test: boot test: add option to reboot using system_reset

[Michael Goldish]

The boot test now behaves as follows:
- Normally it just waits for the guest to go up and logs into it.
- If reboot_method is specified and equals "shell", the guest is sent a reboot
  shell command, and the test waits until the guest comes back up.
- If reboot method is specified and equals "system_reset", the VM is sent a
  system_reset monitor command, and the test waits until the guest comes back
  up.
  Before sending the system_reset command the test sleeps for a given duration.
  The duration is controlled by the parameter 'sleep_before_reset'.

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm_tests.cfg.sample |    8 +++++++-
 client/tests/kvm/kvm_tests.py         |   32 ++++++++++++++++++++++----------
 2 files changed, 29 insertions(+), 11 deletions(-)

diff --git a/client/tests/kvm/kvm_tests.cfg.sample b/client/tests/kvm/kvm_tests.cfg.sample
index 73e929f..0703e64 100644
--- a/client/tests/kvm/kvm_tests.cfg.sample
+++ b/client/tests/kvm/kvm_tests.cfg.sample
@@ -58,7 +58,7 @@ variants:
 
     - reboot:       install setup
         type = boot
-        reboot = yes
+        reboot_method = shell
         kill_vm_on_error = yes
 
     - migrate:      install setup
@@ -127,6 +127,12 @@ variants:
             - notepad:
                 autoit_script = autoit/notepad1.au3
 
+    - system_reset: install setup
+        type = boot
+        reboot_method = system_reset
+        sleep_before_reset = 20
+        kill_vm_on_error = yes
+
     - shutdown:     install setup
         type = shutdown
         kill_vm = yes
diff --git a/client/tests/kvm/kvm_tests.py b/client/tests/kvm/kvm_tests.py
index c2bfb8a..801b84e 100644
--- a/client/tests/kvm/kvm_tests.py
+++ b/client/tests/kvm/kvm_tests.py
@@ -13,9 +13,9 @@ def run_boot(test, params, env):
     """
     KVM reboot test:
     1) Log into a guest
-    2) Send a reboot command to the guest
-    3) Wait until it's up.
-    4) Log into the guest to verify it's up again.
+    2) Send a reboot command or a system_reset monitor command (optional)
+    3) Wait until the guest is up again
+    4) Log into the guest to verify it's up again
 
     @param test: kvm test object
     @param params: Dictionary with the test parameters
@@ -33,13 +33,24 @@ def run_boot(test, params, env):
     if not session:
         raise error.TestFail("Could not log into guest")
 
-    logging.info("Logged in")
-
-    if params.get("reboot") == "yes":
-        # Send the VM's reboot command
-        session.sendline(vm.get_params().get("reboot_command"))
-        logging.info("Reboot command sent; waiting for guest to go down...")
+    try:
+        logging.info("Logged in")
 
+        if params.get("reboot_method") == "shell":
+            # Send a reboot command to the guest's shell
+            session.sendline(vm.get_params().get("reboot_command"))
+            logging.info("Reboot command sent; waiting for guest to go "
+                         "down...")
+        elif params.get("reboot_method") == "system_reset":
+            # Sleep for a while -- give the guest a chance to finish booting
+            time.sleep(float(params.get("sleep_before_reset", 10)))
+            # Send a system_reset monitor command
+            vm.send_monitor_cmd("system_reset")
+            logging.info("system_reset monitor command sent; waiting for "
+                         "guest to go down...")
+        else: return
+
+        # Wait for the session to become unresponsive
         if not kvm_utils.wait_for(lambda: not session.is_responsive(),
                                   120, 0, 1):
             raise error.TestFail("Guest refuses to go down")
@@ -54,7 +65,8 @@ def run_boot(test, params, env):
 
         logging.info("Guest is up again")
 
-    session.close()
+    finally:
+        session.close()
 
 
 def run_shutdown(test, params, env):
-- 
1.5.4.1

[PATCH 11/19] KVM test: shutdown test: allow shutting down using system_powerdown

[Michael Goldish]

In addition to using the shutdown command/program, allow using the
system_powerdown monitor command.  The behavior is controlled using the
shutdown_method parameter, which may equal either "shell" or "system_powerdown".

If shutdown_method equals "system_powerdown", the test sleeps a given amount of
time (controlled by the parameter sleep_before_powerdown) and then sends the
system_powerdown monitor command.

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm_tests.cfg.sample |    8 ++++++++
 client/tests/kvm/kvm_tests.py         |   22 ++++++++++++++++------
 2 files changed, 24 insertions(+), 6 deletions(-)

diff --git a/client/tests/kvm/kvm_tests.cfg.sample b/client/tests/kvm/kvm_tests.cfg.sample
index 0703e64..201b0e1 100644
--- a/client/tests/kvm/kvm_tests.cfg.sample
+++ b/client/tests/kvm/kvm_tests.cfg.sample
@@ -133,8 +133,16 @@ variants:
         sleep_before_reset = 20
         kill_vm_on_error = yes
 
+    - system_powerdown: install setup
+        type = shutdown
+        shutdown_method = system_powerdown
+        sleep_before_powerdown = 20
+        kill_vm = yes
+        kill_vm_gracefully = no
+
     - shutdown:     install setup
         type = shutdown
+        shutdown_method = shell
         kill_vm = yes
         kill_vm_gracefully = no
 
diff --git a/client/tests/kvm/kvm_tests.py b/client/tests/kvm/kvm_tests.py
index 801b84e..863b863 100644
--- a/client/tests/kvm/kvm_tests.py
+++ b/client/tests/kvm/kvm_tests.py
@@ -73,8 +73,9 @@ def run_shutdown(test, params, env):
     """
     KVM shutdown test:
     1) Log into a guest
-    2) Send a shutdown command to the guest
-    3) Wait until it's down
+    2) Send a shutdown command to the guest, or issue a system_powerdown
+       monitor command (depending on the value of shutdown_method)
+    3) Wait until the guest is down
 
     @param test: kvm test object
     @param params: Dictionary with the test parameters
@@ -95,15 +96,24 @@ def run_shutdown(test, params, env):
     try:
         logging.info("Logged in")
 
-        # Send the VM's shutdown command
-        session.sendline(vm.get_params().get("shutdown_command"))
-
-        logging.info("Shutdown command sent; waiting for guest to go down...")
+        if params.get("shutdown_method") == "shell":
+            # Send a shutdown command to the guest's shell
+            session.sendline(vm.get_params().get("shutdown_command"))
+            logging.info("Shutdown command sent; waiting for guest to go "
+                         "down...")
+        elif params.get("shutdown_method") == "system_powerdown":
+            # Sleep for a while -- give the guest a chance to finish booting
+            time.sleep(float(params.get("sleep_before_powerdown", 10)))
+            # Send a system_powerdown monitor command
+            vm.send_monitor_cmd("system_powerdown")
+            logging.info("system_powerdown monitor command sent; waiting for "
+                         "guest to go down...")
 
         if not kvm_utils.wait_for(vm.is_dead, 240, 0, 1):
             raise error.TestFail("Guest refuses to go down")
 
         logging.info("Guest is down")
+
     finally:
         session.close()
 
-- 
1.5.4.1

[PATCH 12/19] KVM test: Add new module kvm_test_utils.py

[Michael Goldish]

This module is meant to reduce code size by performing common test procedures.
Generally, code here should look like test code.
More specifically:
    - Functions in this module should raise exceptions if things go wrong
      (unlike functions in kvm_utils.py and kvm_vm.py which report failure via
      their returned values).
    - Functions in this module may use logging.info(), in addition to
      logging.debug() and logging.error(), to log messages the user may be
      interested in (unlike kvm_utils.py and kvm_vm.py which use
      logging.debug() for everything that isn't an error).
    - Functions in this module typically use functions and classes from
      lower-level modules (e.g. kvm_utils.py, kvm_vm.py, kvm_subprocess.py).
    - Functions in this module should not be used by lower-level modules.
    - Functions in this module should be used in the right context.
      For example, a function should not be used where it may display
      misleading or inaccurate info or debug messages.

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm_test_utils.py |   61 ++++++++++++++++++++++++++++++++++++
 1 files changed, 61 insertions(+), 0 deletions(-)
 create mode 100644 client/tests/kvm/kvm_test_utils.py

diff --git a/client/tests/kvm/kvm_test_utils.py b/client/tests/kvm/kvm_test_utils.py
new file mode 100644
index 0000000..39e92b9
--- /dev/null
+++ b/client/tests/kvm/kvm_test_utils.py
@@ -0,0 +1,61 @@
+import time, os, logging, re, commands
+from autotest_lib.client.common_lib import utils, error
+import kvm_utils, kvm_vm, kvm_subprocess
+
+"""
+High-level KVM test utility functions.
+
+This module is meant to reduce code size by performing common test procedures.
+Generally, code here should look like test code.
+More specifically:
+    - Functions in this module should raise exceptions if things go wrong
+      (unlike functions in kvm_utils.py and kvm_vm.py which report failure via
+      their returned values).
+    - Functions in this module may use logging.info(), in addition to
+      logging.debug() and logging.error(), to log messages the user may be
+      interested in (unlike kvm_utils.py and kvm_vm.py which use
+      logging.debug() for anything that isn't an error).
+    - Functions in this module typically use functions and classes from
+      lower-level modules (e.g. kvm_utils.py, kvm_vm.py, kvm_subprocess.py).
+    - Functions in this module should not be used by lower-level modules.
+    - Functions in this module should be used in the right context.
+      For example, a function should not be used where it may display
+      misleading or inaccurate info or debug messages.
+
+@copyright: 2008-2009 Red Hat Inc.
+"""
+
+
+def get_living_vm(env, vm_name):
+    """
+    Get a VM object from the environment and make sure it's alive.
+
+    @param env: Dictionary with test environment.
+    @param vm_name: Name of the desired VM object.
+    @return: A VM object.
+    """
+    vm = kvm_utils.env_get_vm(env, vm_name)
+    if not vm:
+        raise error.TestError("VM '%s' not found in environment" % vm_name)
+    if not vm.is_alive():
+        raise error.TestError("VM '%s' seems to be dead; test requires a "
+                              "living VM" % vm_name)
+    return vm
+
+
+def wait_for_login(vm, nic_index=0, timeout=240):
+    """
+    Try logging into a VM repeatedly.  Stop on success or when timeout expires.
+
+    @param vm: VM object.
+    @param nic_index: Index of NIC to access in the VM.
+    @param timeout: Time to wait before giving up.
+    @return: A shell session object.
+    """
+    logging.info("Waiting for guest to be up...")
+    session = kvm_utils.wait_for(lambda: vm.remote_login(nic_index=nic_index),
+                                 timeout, 0, 2)
+    if not session:
+        raise error.TestFail("Could not log into guest")
+    logging.info("Logged in")
+    return session
-- 
1.5.4.1

[PATCH 13/19] KVM test: kvm_tests.py: save test code by using utilities in kvm_test_utils.py

[Michael Goldish]

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm_tests.py |  148 ++++++++---------------------------------
 1 files changed, 27 insertions(+), 121 deletions(-)

diff --git a/client/tests/kvm/kvm_tests.py b/client/tests/kvm/kvm_tests.py
index 863b863..35666cf 100644
--- a/client/tests/kvm/kvm_tests.py
+++ b/client/tests/kvm/kvm_tests.py
@@ -1,6 +1,6 @@
 import time, os, logging, re, commands
 from autotest_lib.client.common_lib import utils, error
-import kvm_utils, kvm_subprocess, ppm_utils, scan_results
+import kvm_utils, kvm_subprocess, ppm_utils, scan_results, kvm_test_utils
 
 """
 KVM test definitions.
@@ -21,21 +21,10 @@ def run_boot(test, params, env):
     @param params: Dictionary with the test parameters
     @param env: Dictionary with test environment.
     """
-    vm = kvm_utils.env_get_vm(env, params.get("main_vm"))
-    if not vm:
-        raise error.TestError("VM object not found in environment")
-    if not vm.is_alive():
-        raise error.TestError("VM seems to be dead; Test requires a living VM")
-
-    logging.info("Waiting for guest to be up...")
-
-    session = kvm_utils.wait_for(vm.remote_login, 240, 0, 2)
-    if not session:
-        raise error.TestFail("Could not log into guest")
+    vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+    session = kvm_test_utils.wait_for_login(vm)
 
     try:
-        logging.info("Logged in")
-
         if params.get("reboot_method") == "shell":
             # Send a reboot command to the guest's shell
             session.sendline(vm.get_params().get("reboot_command"))
@@ -81,21 +70,10 @@ def run_shutdown(test, params, env):
     @param params: Dictionary with the test parameters
     @param env: Dictionary with test environment
     """
-    vm = kvm_utils.env_get_vm(env, params.get("main_vm"))
-    if not vm:
-        raise error.TestError("VM object not found in environment")
-    if not vm.is_alive():
-        raise error.TestError("VM seems to be dead; Test requires a living VM")
-
-    logging.info("Waiting for guest to be up...")
-
-    session = kvm_utils.wait_for(vm.remote_login, 240, 0, 2)
-    if not session:
-        raise error.TestFail("Could not log into guest")
+    vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+    session = kvm_test_utils.wait_for_login(vm)
 
     try:
-        logging.info("Logged in")
-
         if params.get("shutdown_method") == "shell":
             # Send a shutdown command to the guest's shell
             session.sendline(vm.get_params().get("shutdown_command"))
@@ -134,33 +112,23 @@ def run_migration(test, params, env):
     @param params: Dictionary with test parameters.
     @param env: Dictionary with the test environment.
     """
-    vm = kvm_utils.env_get_vm(env, params.get("main_vm"))
-    if not vm:
-        raise error.TestError("VM object not found in environment")
-    if not vm.is_alive():
-        raise error.TestError("VM seems to be dead; Test requires a living VM")
+    vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
 
     # See if migration is supported
     s, o = vm.send_monitor_cmd("help info")
     if not "info migrate" in o:
         raise error.TestError("Migration is not supported")
 
-    dest_vm = vm.clone()
-    dest_vm.create(for_migration=True)
-
     # Log into guest and get the output of migration_test_command
-    logging.info("Waiting for guest to be up...")
-
-    session = kvm_utils.wait_for(vm.remote_login, 240, 0, 2)
-    if not session:
-        raise error.TestFail("Could not log into guest")
-
-    logging.info("Logged in")
-
-    reference_output = session.get_command_output(params.get("migration_test_"
-                                                             "command"))
+    session = kvm_test_utils.wait_for_login(vm)
+    migration_test_command = params.get("migration_test_command")
+    reference_output = session.get_command_output(migration_test_command)
     session.close()
 
+    # Clone the main VM and ask it to wait for incoming migration
+    dest_vm = vm.clone()
+    dest_vm.create(for_migration=True)
+
     # Define the migration command
     cmd = "migrate -d tcp:localhost:%d" % dest_vm.migration_port
     logging.debug("Migration command: %s" % cmd)
@@ -211,7 +179,7 @@ def run_migration(test, params, env):
 
     logging.info("Logged in after migration")
 
-    output = session.get_command_output(params.get("migration_test_command"))
+    output = session.get_command_output(migration_test_command)
     session.close()
 
     # Compare output to reference output
@@ -225,6 +193,7 @@ def run_migration(test, params, env):
         raise error.TestFail("Command produced different output before and "
                              "after migration")
 
+    # Replace the main VM with the new cloned VM
     kvm_utils.env_register_vm(env, params.get("main_vm"), dest_vm)
 
 
@@ -236,19 +205,8 @@ def run_autotest(test, params, env):
     @param params: Dictionary with test parameters.
     @param env: Dictionary with the test environment.
     """
-    vm = kvm_utils.env_get_vm(env, params.get("main_vm"))
-    if not vm:
-        raise error.TestError("VM object not found in environment")
-    if not vm.is_alive():
-        raise error.TestError("VM seems to be dead; Test requires a living VM")
-
-    logging.info("Logging into guest...")
-
-    session = kvm_utils.wait_for(vm.remote_login, 240, 0, 2)
-    if not session:
-        raise error.TestFail("Could not log into guest")
-
-    logging.info("Logged in")
+    vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+    session = kvm_test_utils.wait_for_login(vm)
 
     # Collect some info
     test_name = params.get("test_name")
@@ -426,25 +384,8 @@ def run_yum_update(test, params, env):
     @param params: Dictionary with test parameters.
     @param env: Dictionary with the test environment.
     """
-    vm = kvm_utils.env_get_vm(env, params.get("main_vm"))
-    if not vm:
-        message = "VM object not found in environment"
-        logging.error(message)
-        raise error.TestError(message)
-    if not vm.is_alive():
-        message = "VM seems to be dead; Test requires a living VM"
-        logging.error(message)
-        raise error.TestError(message)
-
-    logging.info("Logging into guest...")
-
-    session = kvm_utils.wait_for(vm.remote_login, 240, 0, 2)
-    if not session:
-        message = "Could not log into guest"
-        logging.error(message)
-        raise error.TestFail(message)
-
-    logging.info("Logged in")
+    vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+    session = kvm_test_utils.wait_for_login(vm)
 
     internal_yum_update(session, "yum update", params.get("shell_prompt"), 600)
     internal_yum_update(session, "yum update kernel",
@@ -461,21 +402,10 @@ def run_linux_s3(test, params, env):
     @param params: Dictionary with test parameters.
     @param env: Dictionary with the test environment.
     """
-    vm = kvm_utils.env_get_vm(env, params.get("main_vm"))
-    if not vm:
-        raise error.TestError("VM object not found in environment")
-    if not vm.is_alive():
-        raise error.TestError("VM seems to be dead; Test requires a living VM")
-
-    logging.info("Waiting for guest to be up...")
-
-    session = kvm_utils.wait_for(vm.remote_login, 240, 0, 2)
-    if not session:
-        raise error.TestFail("Could not log into guest")
+    vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+    session = kvm_test_utils.wait_for_login(vm)
 
-    logging.info("Logged in")
     logging.info("Checking that VM supports S3")
-
     status = session.get_command_status("grep -q mem /sys/power/state")
     if status == None:
         logging.error("Failed to check if S3 exists")
@@ -520,11 +450,7 @@ def run_stress_boot(tests, params, env):
     @param env:    Dictionary with test environment.
     """
     # boot the first vm
-    vm = kvm_utils.env_get_vm(env, params.get("main_vm"))
-    if not vm:
-        raise error.TestError("VM object not found in environment")
-    if not vm.is_alive():
-        raise error.TestError("VM seems to be dead; Test requires a living VM")
+    vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
 
     logging.info("Waiting for first guest to be up...")
 
@@ -645,19 +571,8 @@ def run_timedrift(test, params, env):
         guest_time = time.mktime(time.strptime(s, time_format))
         return (host_time, guest_time)
 
-    vm = kvm_utils.env_get_vm(env, params.get("main_vm"))
-    if not vm:
-        raise error.TestError("VM object not found in environment")
-    if not vm.is_alive():
-        raise error.TestError("VM seems to be dead; Test requires a living VM")
-
-    logging.info("Waiting for guest to be up...")
-
-    session = kvm_utils.wait_for(vm.remote_login, 240, 0, 2)
-    if not session:
-        raise error.TestFail("Could not log into guest")
-
-    logging.info("Logged in")
+    vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+    session = kvm_test_utils.wait_for_login(vm)
 
     # Collect test parameters:
     # Command to run to get the current time
@@ -781,20 +696,11 @@ def run_autoit(test, params, env):
     @param params: Dictionary with test parameters.
     @param env: Dictionary with the test environment.
     """
-    vm = kvm_utils.env_get_vm(env, params.get("main_vm"))
-    if not vm:
-        raise error.TestError("VM object not found in environment")
-    if not vm.is_alive():
-        raise error.TestError("VM seems to be dead; Test requires a living VM")
-
-    logging.info("Waiting for guest to be up...")
-
-    session = kvm_utils.wait_for(vm.remote_login, 240, 0, 2)
-    if not session:
-        raise error.TestFail("Could not log into guest")
+    vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
+    session = kvm_test_utils.wait_for_login(vm)
 
     try:
-        logging.info("Logged in; starting script...")
+        logging.info("Starting script...")
 
         # Collect test parameters
         binary = params.get("autoit_binary")
-- 
1.5.4.1

[PATCH 14/19] KVM test: kvm_preprocessing.py: don't explicitly print failure messages

[Michael Goldish]

This is taken care of elsewhere, so currently when something fails during
pre/post-processing, messages get logged twice.

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm_preprocessing.py |   14 ++++----------
 1 files changed, 4 insertions(+), 10 deletions(-)

diff --git a/client/tests/kvm/kvm_preprocessing.py b/client/tests/kvm/kvm_preprocessing.py
index 8fa74cf..729e1d4 100644
--- a/client/tests/kvm/kvm_preprocessing.py
+++ b/client/tests/kvm/kvm_preprocessing.py
@@ -31,11 +31,8 @@ def preprocess_image(test, params):
         logging.debug("Creating image...")
         create_image = True
 
-    if create_image:
-        if not kvm_vm.create_image(params, test.bindir):
-            message = "Could not create image"
-            logging.error(message)
-            raise error.TestError(message)
+    if create_image and not kvm_vm.create_image(params, test.bindir):
+        raise error.TestError("Could not create image")
 
 
 def preprocess_vm(test, params, env, name):
@@ -78,11 +75,8 @@ def preprocess_vm(test, params, env, name):
                           "restarting it...")
             start_vm = True
 
-    if start_vm:
-        if not vm.create(name, params, test.bindir, for_migration):
-            message = "Could not start VM"
-            logging.error(message)
-            raise error.TestError(message)
+    if start_vm and not vm.create(name, params, test.bindir, for_migration):
+        raise error.TestError("Could not start VM")
 
     scrdump_filename = os.path.join(test.debugdir, "pre_%s.ppm" % name)
     vm.send_monitor_cmd("screendump %s" % scrdump_filename)
-- 
1.5.4.1

[PATCH 15/19] KVM test: Autotest test wrapper cleanup

[Michael Goldish]

Make the code shorter and simpler.

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm_tests.py |  153 +++++++++++++++++++----------------------
 1 files changed, 72 insertions(+), 81 deletions(-)

diff --git a/client/tests/kvm/kvm_tests.py b/client/tests/kvm/kvm_tests.py
index 35666cf..8c785c1 100644
--- a/client/tests/kvm/kvm_tests.py
+++ b/client/tests/kvm/kvm_tests.py
@@ -205,10 +205,44 @@ def run_autotest(test, params, env):
     @param params: Dictionary with test parameters.
     @param env: Dictionary with the test environment.
     """
+    # Helper functions
+    def copy_if_size_differs(vm, local_path, remote_path):
+        """
+        Copy a file to a guest if it doesn't exist or if its size differs.
+
+        @param vm: VM object
+        @param local_path: Local path
+        @param remote_path: Remote path
+        """
+        copy = False
+        output = session.get_command_output("ls -l %s" % remote_path)
+        if ("such file" in output or
+            int(output.split()[4]) != os.path.getsize(local_path)):
+            basename = os.path.basename(local_path)
+            logging.info("Copying %s to guest (file is missing or has a "
+                         "different size)..." % basename)
+            if not vm.copy_files_to(local_path, remote_path):
+                raise error.TestFail("Could not copy %s to guest" % basename)
+
+    def extract(vm, remote_path, dest_dir="."):
+        """
+        Extract a .tar.bz2 file on the guest.
+
+        @param vm: VM object
+        @param remote_path: Remote file path
+        @param dest_dir: Destination dir for the contents
+        """
+        basename = os.path.basename(remote_path)
+        logging.info("Extracting %s..." % basename)
+        status = session.get_command_status("tar xfj %s -C %s" %
+                                            (remote_path, dest_dir))
+        if status != 0:
+            raise error.TestFail("Could not extract %s" % basename)
+
     vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
     session = kvm_test_utils.wait_for_login(vm)
 
-    # Collect some info
+    # Collect test parameters
     test_name = params.get("test_name")
     test_timeout = int(params.get("test_timeout", 300))
     test_control_file = params.get("test_control_file", "control")
@@ -235,104 +269,47 @@ def run_autotest(test, params, env):
                           (os.path.join(test.bindir, "autotest", "tests"),
                            tarred_test_path, test_name), timeout=30)
 
-    # Check if we need to copy autotest.tar.bz2
-    copy = False
-    output = session.get_command_output("ls -l autotest.tar.bz2")
-    if "such file" in output:
-        copy = True
-    else:
-        size = int(output.split()[4])
-        if size != os.path.getsize(tarred_autotest_path):
-            copy = True
-    # Perform the copy
-    if copy:
-        logging.info("Copying autotest.tar.bz2 to guest"
-                     " (file is missing or has a different size)...")
-        if not vm.copy_files_to(tarred_autotest_path, ""):
-            raise error.TestFail("Could not copy autotest.tar.bz2 to guest")
-
-    # Check if we need to copy <test_name>.tar.bz2
-    copy = False
-    output = session.get_command_output("ls -l %s.tar.bz2" % test_name)
-    if "such file" in output:
-        copy = True
-    else:
-        size = int(output.split()[4])
-        if size != os.path.getsize(tarred_test_path):
-            copy = True
-    # Perform the copy
-    if copy:
-        logging.info("Copying %s.tar.bz2 to guest (file is missing or has a"
-                     " different size)..." % test_name)
-        if not vm.copy_files_to(tarred_test_path, ""):
-            raise error.TestFail("Could not copy %s.tar.bz2 to guest" %
-                                 test_name)
+    # Copy autotest.tar.bz2
+    copy_if_size_differs(vm, tarred_autotest_path, "autotest.tar.bz2")
+
+    # Copy <test_name>.tar.bz2
+    copy_if_size_differs(vm, tarred_test_path, test_name + ".tar.bz2")
 
     # Extract autotest.tar.bz2
-    logging.info("Extracting autotest.tar.bz2...")
-    status = session.get_command_status("tar xvfj autotest.tar.bz2")
-    if status != 0:
-        raise error.TestFail("Could not extract autotest.tar.bz2")
+    extract(vm, "autotest.tar.bz2")
 
     # mkdir autotest/tests
     session.sendline("mkdir autotest/tests")
 
     # Extract <test_name>.tar.bz2 into autotest/tests
-    logging.info("Extracting %s.tar.bz2..." % test_name)
-    status = session.get_command_status("tar xvfj %s.tar.bz2 -C "
-                                        "autotest/tests" % test_name)
-    if status != 0:
-        raise error.TestFail("Could not extract %s.tar.bz2" % test_name)
+    extract(vm, test_name + ".tar.bz2", "autotest/tests")
 
-    # Cleaning up old remaining results
-    session.sendline("rm -rf autotest/results/*")
-    # Copying the selected control file (located inside
-    # test.bindir/autotest_control to the autotest dir
+    # Copy the selected control file (located inside
+    # test.bindir/autotest_control) to the autotest dir
     control_file_path = os.path.join(test.bindir, "autotest_control",
                                      test_control_file)
     if not vm.copy_files_to(control_file_path, "autotest/control"):
         raise error.TestFail("Could not copy the test control file to guest")
+
     # Run the test
     logging.info("Running test '%s'..." % test_name)
     session.sendline("cd autotest")
     session.sendline("rm -f control.state")
+    session.sendline("rm -rf results/*")
     session.read_up_to_prompt()
-    session.sendline("bin/autotest control")
     logging.info("---------------- Test output ----------------")
-    match = session.read_up_to_prompt(timeout=test_timeout,
-                                      print_func=logging.info)[0]
+    status = session.get_command_status("bin/autotest control",
+                                        timeout=test_timeout,
+                                        print_func=logging.info)
     logging.info("---------------- End of test output ----------------")
-    if not match:
+    if status is None:
         raise error.TestFail("Timeout elapsed while waiting for test to "
                              "complete")
+
     # Get the results generated by autotest
     output = session.get_command_output("cat results/*/status")
-
-    # Parse test results
-    result_list = scan_results.parse_results(output)
-
-    # Report test results and check for FAIL/ERROR status
-    logging.info("Results (test, status, duration, info):")
-    status_error = False
-    status_fail = False
-    if result_list == []:
-        status_fail = True
-        message_fail = ("Test '%s' did not produce any recognizable "
-                        "results" % test_name)
-    for result in result_list:
-        logging.info(str(result))
-        if result[1] == "FAIL":
-            status_fail = True
-            message_fail = ("Test '%s' ended with FAIL "
-                            "(info: '%s')" % (result[0], result[3]))
-        if result[1] == "ERROR":
-            status_error = True
-            message_error = ("Test '%s' ended with ERROR "
-                             "(info: '%s')" % (result[0], result[3]))
-        if result[1] == "ABORT":
-            status_error = True
-            message_error = ("Test '%s' ended with ABORT "
-                             "(info: '%s')" % (result[0], result[3]))
+    results = scan_results.parse_results(output)
+    session.close
 
     # Copy test results to the local bindir/guest_results
     logging.info("Copying results back from guest...")
@@ -342,11 +319,25 @@ def run_autotest(test, params, env):
     if not vm.copy_files_from("autotest/results/default/*", guest_results_dir):
         logging.error("Could not copy results back from guest")
 
+    # Report test results
+    logging.info("Results (test, status, duration, info):")
+    for result in results:
+        logging.info(str(result))
+
+    # Make a list of FAIL/ERROR/ABORT results (make sure FAIL results appear
+    # before ERROR results, and ERROR results appear before ABORT results)
+    bad_results = [r for r in results if r[1] == "FAIL"]
+    bad_results += [r for r in results if r[1] == "ERROR"]
+    bad_results += [r for r in results if r[1] == "ABORT"]
+
     # Fail the test if necessary
-    if status_fail:
-        raise error.TestFail(message_fail)
-    elif status_error:
-        raise error.TestError(message_error)
+    if not results:
+        raise Error.TestFail("Test '%s' did not produce any recognizable "
+                             "results" % test_name)
+    if bad_results:
+        result = bad_results[0]
+        raise Error.TestFail("Test '%s' ended with %s (reason: '%s')"
+                             % (result[0], result[1], result[3]))
 
 
 def internal_yum_update(session, command, prompt, timeout):
-- 
1.5.4.1

[PATCH 16/19] KVM test: kvm_subprocess: robustify the test for child process termination

[Michael Goldish]

Don't rely on os.read() raising an exception; use waitpid() to check for child
termination:
- every 0.5 seconds, or
- when os.read() raises an exception, or
- when os.read() returns a zero-length string.

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm_subprocess.py |   17 ++++++++++++-----
 1 files changed, 12 insertions(+), 5 deletions(-)

diff --git a/client/tests/kvm/kvm_subprocess.py b/client/tests/kvm/kvm_subprocess.py
index d142ed9..c3c48cd 100755
--- a/client/tests/kvm/kvm_subprocess.py
+++ b/client/tests/kvm/kvm_subprocess.py
@@ -1102,10 +1102,11 @@ def _server_main():
 
         # Read from child and write to files/pipes
         while True:
+            check_termination = False
             # Make a list of reader pipes whose buffers are not empty
             fds = [fd for (i, fd) in enumerate(reader_fds) if buffers[i]]
             # Wait until there's something to do
-            r, w, x = select.select([shell_fd, inpipe_fd], fds, [])
+            r, w, x = select.select([shell_fd, inpipe_fd], fds, [], 0.5)
             # If a reader pipe is ready for writing --
             for (i, fd) in enumerate(reader_fds):
                 if fd in w:
@@ -1116,7 +1117,9 @@ def _server_main():
                 try:
                     data = os.read(shell_fd, 16384)
                 except OSError:
-                    break
+                    data = ""
+                if not data:
+                    check_termination = True
                 # Remove carriage returns from the data -- they often cause
                 # trouble and are normally not needed
                 data = data.replace("\r", "")
@@ -1124,14 +1127,18 @@ def _server_main():
                 output_file.flush()
                 for i in range(len(readers)):
                     buffers[i] += data
+            # If os.read() raised an exception or there was nothing to read --
+            if check_termination or shell_fd not in r:
+                pid, status = os.waitpid(shell_pid, os.WNOHANG)
+                if pid:
+                    status = os.WEXITSTATUS(status)
+                    break
             # If there's data to read from the client --
             if inpipe_fd in r:
                 data = os.read(inpipe_fd, 1024)
                 os.write(shell_fd, data)
 
-        # Wait for the shell process to exit and get its exit status
-        status = os.waitpid(shell_pid, 0)[1]
-        status = os.WEXITSTATUS(status)
+        # Write the exit status to a file
         file = open(status_filename, "w")
         file.write(str(status))
         file.close()
-- 
1.5.4.1

[PATCH 17/19] KVM test: kvm_vm.py: add macaddr= to command line only if a MAC address is given

[Michael Goldish]

(Avoid adding weird things like 'macaddr=None' to the command line.)

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm_vm.py |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
index d360d8e..f728104 100755
--- a/client/tests/kvm/kvm_vm.py
+++ b/client/tests/kvm/kvm_vm.py
@@ -234,7 +234,8 @@ class VM:
                 qemu_cmd += ",model=%s" % nic_params.get("nic_model")
             if nic_params.has_key("address_index"):
                 mac, ip = kvm_utils.get_mac_ip_pair_from_dict(nic_params)
-                qemu_cmd += ",macaddr=%s" % mac
+                if mac:
+                    qemu_cmd += ",macaddr=%s" % mac
             # Handle the '-net tap' or '-net user' part
             mode = nic_params.get("nic_mode", "user")
             qemu_cmd += " -net %s,vlan=%d" % (mode, vlan)
-- 
1.5.4.1

[PATCH 18/19] KVM test: kvm_tests.cfg.sample: get all Windows test utilities from a single ISO

[Michael Goldish]

Instead of rss.iso, vlc.iso and autoit.iso -- look for everything in winutils.iso
by default.
This make maintenance a little easier, and eliminates the need to restart VMs
between tests just in order to change the cdrom.
Also, expect vlc.exe in D:\vlc\vlc.exe instead of D:\vlc.exe.

'install' tests still use the usual ISO files.

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm_tests.cfg.sample |   18 +++++-------------
 1 files changed, 5 insertions(+), 13 deletions(-)

diff --git a/client/tests/kvm/kvm_tests.cfg.sample b/client/tests/kvm/kvm_tests.cfg.sample
index 201b0e1..285a38f 100644
--- a/client/tests/kvm/kvm_tests.cfg.sample
+++ b/client/tests/kvm/kvm_tests.cfg.sample
@@ -119,7 +119,6 @@ variants:
 
     - autoit:       install setup
         type = autoit
-        cdrom = windows/autoit.iso
         autoit_binary = D:\AutoIt3.exe
         autoit_script_timeout = 600
         autoit_script_params =
@@ -362,17 +361,19 @@ variants:
         # file_transfer_client = scp
         # file_transfer_port = 22
 
+        # This ISO will be used for all tests except install:
+        cdrom = windows/winutils.iso
+
         migrate:
             migration_test_command = ver && vol
         stress_boot:
             alive_test_cmd = systeminfo
         timedrift:
-            # For this to work, the ISO should contain vlc (vlc.exe) and a video (ED_1024.avi)
-            cdrom = windows/vlc.iso
             time_command = "echo TIME: %date% %time%"
             time_filter_re = "(?<=TIME: \w\w\w ).{19}(?=\.\d\d)"
             time_format = "%m/%d/%Y %H:%M:%S"
-            guest_load_command = 'cmd /c "d:\vlc -f --loop --no-qt-privacy-ask --no-qt-system-tray d:\ED_1024.avi"'
+            # For this to work, the cdrom at d: should contain vlc (d:\vlc\vlc.exe) and a video (d:\ED_1024.avi)
+            guest_load_command = 'cmd /c "d:\vlc\vlc -f --loop --no-qt-privacy-ask --no-qt-system-tray d:\ED_1024.avi"'
             # Alternative guest load:
             #guest_load_command = "(dir /s && dir /s && dir /s && dir /s) > nul"
             guest_load_stop_command = "taskkill /F /IM vlc.exe"
@@ -395,7 +396,6 @@ variants:
                     user = user
                 setup:
                     steps = Win2000-32-rss.steps
-                    cdrom = windows/rss.iso
 
             - WinXP.32:
                 image_name = winXP-32
@@ -407,7 +407,6 @@ variants:
                     user = user
                 setup:
                     steps = WinXP-32-rss.steps
-                    cdrom = windows/rss.iso
 
             - WinXP.64:
                 no reboot
@@ -420,7 +419,6 @@ variants:
                     user = user
                 setup:
                     steps = WinXP-64-rss.steps
-                    cdrom = windows/rss.iso
 
             - Win2003:
                 image_size = 20G
@@ -438,7 +436,6 @@ variants:
                             user = user
                         setup:
                             steps = Win2003-32-rss.steps
-                            cdrom = windows/rss.iso
 
                     - 64:
                         image_name = win2003-64
@@ -450,7 +447,6 @@ variants:
                             user = user
                         setup:
                             steps = Win2003-64-rss.steps
-                            cdrom = windows/rss.iso
 
             - WinVista:
                 image_name = winvista
@@ -466,7 +462,6 @@ variants:
                             md5sum_1m = c724e9695da483bc0fd59e426eaefc72
                         setup:
                             steps = WinVista-32-rss.steps
-                            cdrom = windows/rss.iso
 
                     - 64:
                         image_name += -64
@@ -477,7 +472,6 @@ variants:
                             md5sum_1m = 0947bcd5390546139e25f25217d6f165
                         setup:
                             steps = WinVista-64-rss.steps
-                            cdrom = windows/rss.iso
 
             - Win2008:
                 image_name = win2008
@@ -497,7 +491,6 @@ variants:
                             md5sum_1m=07d7f5006393f74dc76e6e2e943e2440
                         setup:
                             steps = Win2008-32-rss.steps
-                            cdrom = windows/rss.iso
 
                     - 64:
                         image_name += -64
@@ -511,7 +504,6 @@ variants:
                             passwd = 1q2w3eP
                         setup:
                             steps = Win2008-64-rss.steps
-                            cdrom = windows/rss.iso
 
     # Unix/BSD section
     - @Unix:
-- 
1.5.4.1

[PATCH 19/19] KVM test: kvm_preprocessing.py: verify PPM file validity before passing to PIL

[Michael Goldish]

Passing PIL an invalid PPM file makes it throw an IOError.

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm_preprocessing.py |    7 ++++---
 1 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/client/tests/kvm/kvm_preprocessing.py b/client/tests/kvm/kvm_preprocessing.py
index 729e1d4..8809f31 100644
--- a/client/tests/kvm/kvm_preprocessing.py
+++ b/client/tests/kvm/kvm_preprocessing.py
@@ -273,9 +273,10 @@ def postprocess(test, params, env):
                       " files to PNG format...")
         try:
             for f in glob.glob(os.path.join(test.debugdir, "*.ppm")):
-                new_path = f.replace(".ppm", ".png")
-                image = PIL.Image.open(f)
-                image.save(new_path, format = 'PNG')
+                if ppm_utils.image_verify_ppm_file(f):
+                    new_path = f.replace(".ppm", ".png")
+                    image = PIL.Image.open(f)
+                    image.save(new_path, format='PNG')
         except NameError:
             pass
 
-- 
1.5.4.1

Re: [Autotest] [PATCH 19/19] KVM test: kvm_preprocessing.py: verify PPM file validity before passing to PIL

[Lucas Meneghel Rodrigues]

On Wed, Sep 9, 2009 at 3:12 PM, Michael Goldish <mgoldish@redhat.com> wrote:
> Passing PIL an invalid PPM file makes it throw an IOError.

This one was missing a ppm_utils import. Applied with minimal corrections.

> Signed-off-by: Michael Goldish <mgoldish@redhat.com>
> ---
>  client/tests/kvm/kvm_preprocessing.py |    7 ++++---
>  1 files changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/client/tests/kvm/kvm_preprocessing.py b/client/tests/kvm/kvm_preprocessing.py
> index 729e1d4..8809f31 100644
> --- a/client/tests/kvm/kvm_preprocessing.py
> +++ b/client/tests/kvm/kvm_preprocessing.py
> @@ -273,9 +273,10 @@ def postprocess(test, params, env):
>                       " files to PNG format...")
>         try:
>             for f in glob.glob(os.path.join(test.debugdir, "*.ppm")):
> -                new_path = f.replace(".ppm", ".png")
> -                image = PIL.Image.open(f)
> -                image.save(new_path, format = 'PNG')
> +                if ppm_utils.image_verify_ppm_file(f):
> +                    new_path = f.replace(".ppm", ".png")
> +                    image = PIL.Image.open(f)
> +                    image.save(new_path, format='PNG')
>         except NameError:
>             pass
>
> --
> 1.5.4.1
>
> _______________________________________________
> Autotest mailing list
> Autotest@test.kernel.org
> http://test.kernel.org/cgi-bin/mailman/listinfo/autotest
>



-- 
Lucas

Re: [Autotest] [PATCH 18/19] KVM test: kvm_tests.cfg.sample: get all Windows test utilities from a single ISO

[Lucas Meneghel Rodrigues]

On Wed, Sep 9, 2009 at 3:12 PM, Michael Goldish <mgoldish@redhat.com> wrote:
> Instead of rss.iso, vlc.iso and autoit.iso -- look for everything in winutils.iso
> by default.
> This make maintenance a little easier, and eliminates the need to restart VMs
> between tests just in order to change the cdrom.
> Also, expect vlc.exe in D:\vlc\vlc.exe instead of D:\vlc.exe.

Ok, makes a lot of sense. In order to make it easier to people to
build those CDs, I am going to provide a build isos scrpit under the
deps/ folder, that can download the extraneous components from the
internet. I believe this will be a nice touch for people trying the
tests.

> 'install' tests still use the usual ISO files.
>
> Signed-off-by: Michael Goldish <mgoldish@redhat.com>
> ---
>  client/tests/kvm/kvm_tests.cfg.sample |   18 +++++-------------
>  1 files changed, 5 insertions(+), 13 deletions(-)
>
> diff --git a/client/tests/kvm/kvm_tests.cfg.sample b/client/tests/kvm/kvm_tests.cfg.sample
> index 201b0e1..285a38f 100644
> --- a/client/tests/kvm/kvm_tests.cfg.sample
> +++ b/client/tests/kvm/kvm_tests.cfg.sample
> @@ -119,7 +119,6 @@ variants:
>
>     - autoit:       install setup
>         type = autoit
> -        cdrom = windows/autoit.iso
>         autoit_binary = D:\AutoIt3.exe
>         autoit_script_timeout = 600
>         autoit_script_params =
> @@ -362,17 +361,19 @@ variants:
>         # file_transfer_client = scp
>         # file_transfer_port = 22
>
> +        # This ISO will be used for all tests except install:
> +        cdrom = windows/winutils.iso
> +
>         migrate:
>             migration_test_command = ver && vol
>         stress_boot:
>             alive_test_cmd = systeminfo
>         timedrift:
> -            # For this to work, the ISO should contain vlc (vlc.exe) and a video (ED_1024.avi)
> -            cdrom = windows/vlc.iso
>             time_command = "echo TIME: %date% %time%"
>             time_filter_re = "(?<=TIME: \w\w\w ).{19}(?=\.\d\d)"
>             time_format = "%m/%d/%Y %H:%M:%S"
> -            guest_load_command = 'cmd /c "d:\vlc -f --loop --no-qt-privacy-ask --no-qt-system-tray d:\ED_1024.avi"'
> +            # For this to work, the cdrom at d: should contain vlc (d:\vlc\vlc.exe) and a video (d:\ED_1024.avi)
> +            guest_load_command = 'cmd /c "d:\vlc\vlc -f --loop --no-qt-privacy-ask --no-qt-system-tray d:\ED_1024.avi"'
>             # Alternative guest load:
>             #guest_load_command = "(dir /s && dir /s && dir /s && dir /s) > nul"
>             guest_load_stop_command = "taskkill /F /IM vlc.exe"
> @@ -395,7 +396,6 @@ variants:
>                     user = user
>                 setup:
>                     steps = Win2000-32-rss.steps
> -                    cdrom = windows/rss.iso
>
>             - WinXP.32:
>                 image_name = winXP-32
> @@ -407,7 +407,6 @@ variants:
>                     user = user
>                 setup:
>                     steps = WinXP-32-rss.steps
> -                    cdrom = windows/rss.iso
>
>             - WinXP.64:
>                 no reboot
> @@ -420,7 +419,6 @@ variants:
>                     user = user
>                 setup:
>                     steps = WinXP-64-rss.steps
> -                    cdrom = windows/rss.iso
>
>             - Win2003:
>                 image_size = 20G
> @@ -438,7 +436,6 @@ variants:
>                             user = user
>                         setup:
>                             steps = Win2003-32-rss.steps
> -                            cdrom = windows/rss.iso
>
>                     - 64:
>                         image_name = win2003-64
> @@ -450,7 +447,6 @@ variants:
>                             user = user
>                         setup:
>                             steps = Win2003-64-rss.steps
> -                            cdrom = windows/rss.iso
>
>             - WinVista:
>                 image_name = winvista
> @@ -466,7 +462,6 @@ variants:
>                             md5sum_1m = c724e9695da483bc0fd59e426eaefc72
>                         setup:
>                             steps = WinVista-32-rss.steps
> -                            cdrom = windows/rss.iso
>
>                     - 64:
>                         image_name += -64
> @@ -477,7 +472,6 @@ variants:
>                             md5sum_1m = 0947bcd5390546139e25f25217d6f165
>                         setup:
>                             steps = WinVista-64-rss.steps
> -                            cdrom = windows/rss.iso
>
>             - Win2008:
>                 image_name = win2008
> @@ -497,7 +491,6 @@ variants:
>                             md5sum_1m=07d7f5006393f74dc76e6e2e943e2440
>                         setup:
>                             steps = Win2008-32-rss.steps
> -                            cdrom = windows/rss.iso
>
>                     - 64:
>                         image_name += -64
> @@ -511,7 +504,6 @@ variants:
>                             passwd = 1q2w3eP
>                         setup:
>                             steps = Win2008-64-rss.steps
> -                            cdrom = windows/rss.iso
>
>     # Unix/BSD section
>     - @Unix:
> --
> 1.5.4.1
>
> _______________________________________________
> Autotest mailing list
> Autotest@test.kernel.org
> http://test.kernel.org/cgi-bin/mailman/listinfo/autotest
>



-- 
Lucas

Re: [Autotest] [PATCH 18/19] KVM test: kvm_tests.cfg.sample: get all Windows test utilities from a single ISO

[sudhir kumar]

On Thu, Sep 10, 2009 at 8:49 AM, Lucas Meneghel Rodrigues
<lmr@redhat.com> wrote:
> On Wed, Sep 9, 2009 at 3:12 PM, Michael Goldish <mgoldish@redhat.com> wrote:
>> Instead of rss.iso, vlc.iso and autoit.iso -- look for everything in winutils.iso
>> by default.
>> This make maintenance a little easier, and eliminates the need to restart VMs
>> between tests just in order to change the cdrom.
>> Also, expect vlc.exe in D:\vlc\vlc.exe instead of D:\vlc.exe.
>
> Ok, makes a lot of sense. In order to make it easier to people to
> build those CDs, I am going to provide a build isos scrpit under the
> deps/ folder, that can download the extraneous components from the
> internet. I believe this will be a nice touch for people trying the
> tests.
Definitely yes.
But I do not see any such script yet? Also it will be better to put a
readme under deps so that the user knows all the dependency in
advance. Otherwise one will come to know only after the test throws an
exception.
>
>> 'install' tests still use the usual ISO files.
>>
>> Signed-off-by: Michael Goldish <mgoldish@redhat.com>
>> ---
>>  client/tests/kvm/kvm_tests.cfg.sample |   18 +++++-------------
>>  1 files changed, 5 insertions(+), 13 deletions(-)
>>
>> diff --git a/client/tests/kvm/kvm_tests.cfg.sample b/client/tests/kvm/kvm_tests.cfg.sample
>> index 201b0e1..285a38f 100644
>> --- a/client/tests/kvm/kvm_tests.cfg.sample
>> +++ b/client/tests/kvm/kvm_tests.cfg.sample
>> @@ -119,7 +119,6 @@ variants:
>>
>>     - autoit:       install setup
>>         type = autoit
>> -        cdrom = windows/autoit.iso
>>         autoit_binary = D:\AutoIt3.exe
>>         autoit_script_timeout = 600
>>         autoit_script_params =
>> @@ -362,17 +361,19 @@ variants:
>>         # file_transfer_client = scp
>>         # file_transfer_port = 22
>>
>> +        # This ISO will be used for all tests except install:
>> +        cdrom = windows/winutils.iso
>> +
>>         migrate:
>>             migration_test_command = ver && vol
>>         stress_boot:
>>             alive_test_cmd = systeminfo
>>         timedrift:
>> -            # For this to work, the ISO should contain vlc (vlc.exe) and a video (ED_1024.avi)
>> -            cdrom = windows/vlc.iso
>>             time_command = "echo TIME: %date% %time%"
>>             time_filter_re = "(?<=TIME: \w\w\w ).{19}(?=\.\d\d)"
>>             time_format = "%m/%d/%Y %H:%M:%S"
>> -            guest_load_command = 'cmd /c "d:\vlc -f --loop --no-qt-privacy-ask --no-qt-system-tray d:\ED_1024.avi"'
>> +            # For this to work, the cdrom at d: should contain vlc (d:\vlc\vlc.exe) and a video (d:\ED_1024.avi)
>> +            guest_load_command = 'cmd /c "d:\vlc\vlc -f --loop --no-qt-privacy-ask --no-qt-system-tray d:\ED_1024.avi"'
>>             # Alternative guest load:
>>             #guest_load_command = "(dir /s && dir /s && dir /s && dir /s) > nul"
>>             guest_load_stop_command = "taskkill /F /IM vlc.exe"
>> @@ -395,7 +396,6 @@ variants:
>>                     user = user
>>                 setup:
>>                     steps = Win2000-32-rss.steps
>> -                    cdrom = windows/rss.iso
>>
>>             - WinXP.32:
>>                 image_name = winXP-32
>> @@ -407,7 +407,6 @@ variants:
>>                     user = user
>>                 setup:
>>                     steps = WinXP-32-rss.steps
>> -                    cdrom = windows/rss.iso
>>
>>             - WinXP.64:
>>                 no reboot
>> @@ -420,7 +419,6 @@ variants:
>>                     user = user
>>                 setup:
>>                     steps = WinXP-64-rss.steps
>> -                    cdrom = windows/rss.iso
>>
>>             - Win2003:
>>                 image_size = 20G
>> @@ -438,7 +436,6 @@ variants:
>>                             user = user
>>                         setup:
>>                             steps = Win2003-32-rss.steps
>> -                            cdrom = windows/rss.iso
>>
>>                     - 64:
>>                         image_name = win2003-64
>> @@ -450,7 +447,6 @@ variants:
>>                             user = user
>>                         setup:
>>                             steps = Win2003-64-rss.steps
>> -                            cdrom = windows/rss.iso
>>
>>             - WinVista:
>>                 image_name = winvista
>> @@ -466,7 +462,6 @@ variants:
>>                             md5sum_1m = c724e9695da483bc0fd59e426eaefc72
>>                         setup:
>>                             steps = WinVista-32-rss.steps
>> -                            cdrom = windows/rss.iso
>>
>>                     - 64:
>>                         image_name += -64
>> @@ -477,7 +472,6 @@ variants:
>>                             md5sum_1m = 0947bcd5390546139e25f25217d6f165
>>                         setup:
>>                             steps = WinVista-64-rss.steps
>> -                            cdrom = windows/rss.iso
>>
>>             - Win2008:
>>                 image_name = win2008
>> @@ -497,7 +491,6 @@ variants:
>>                             md5sum_1m=07d7f5006393f74dc76e6e2e943e2440
>>                         setup:
>>                             steps = Win2008-32-rss.steps
>> -                            cdrom = windows/rss.iso
>>
>>                     - 64:
>>                         image_name += -64
>> @@ -511,7 +504,6 @@ variants:
>>                             passwd = 1q2w3eP
>>                         setup:
>>                             steps = Win2008-64-rss.steps
>> -                            cdrom = windows/rss.iso
>>
>>     # Unix/BSD section
>>     - @Unix:
>> --
>> 1.5.4.1
>>
>> _______________________________________________
>> Autotest mailing list
>> Autotest@test.kernel.org
>> http://test.kernel.org/cgi-bin/mailman/listinfo/autotest
>>
>
>
>
> --
> Lucas
> _______________________________________________
> Autotest mailing list
> Autotest@test.kernel.org
> http://test.kernel.org/cgi-bin/mailman/listinfo/autotest
>



-- 
Sudhir Kumar

Re: [Autotest] [PATCH 15/19] KVM test: Autotest test wrapper cleanup

[Lucas Meneghel Rodrigues]

On Wed, Sep 9, 2009 at 3:12 PM, Michael Goldish <mgoldish@redhat.com> wrote:
> Make the code shorter and simpler.
>
> Signed-off-by: Michael Goldish <mgoldish@redhat.com>
> ---
>  client/tests/kvm/kvm_tests.py |  153 +++++++++++++++++++----------------------
>  1 files changed, 72 insertions(+), 81 deletions(-)
>
> diff --git a/client/tests/kvm/kvm_tests.py b/client/tests/kvm/kvm_tests.py
> index 35666cf..8c785c1 100644
> --- a/client/tests/kvm/kvm_tests.py
> +++ b/client/tests/kvm/kvm_tests.py
> @@ -205,10 +205,44 @@ def run_autotest(test, params, env):
>     @param params: Dictionary with test parameters.
>     @param env: Dictionary with the test environment.
>     """
> +    # Helper functions
> +    def copy_if_size_differs(vm, local_path, remote_path):
> +        """
> +        Copy a file to a guest if it doesn't exist or if its size differs.
> +
> +        @param vm: VM object
> +        @param local_path: Local path
> +        @param remote_path: Remote path
> +        """
> +        copy = False
> +        output = session.get_command_output("ls -l %s" % remote_path)
> +        if ("such file" in output or
> +            int(output.split()[4]) != os.path.getsize(local_path)):
> +            basename = os.path.basename(local_path)
> +            logging.info("Copying %s to guest (file is missing or has a "
> +                         "different size)..." % basename)
> +            if not vm.copy_files_to(local_path, remote_path):
> +                raise error.TestFail("Could not copy %s to guest" % basename)
> +
> +    def extract(vm, remote_path, dest_dir="."):
> +        """
> +        Extract a .tar.bz2 file on the guest.
> +
> +        @param vm: VM object
> +        @param remote_path: Remote file path
> +        @param dest_dir: Destination dir for the contents
> +        """
> +        basename = os.path.basename(remote_path)
> +        logging.info("Extracting %s..." % basename)
> +        status = session.get_command_status("tar xfj %s -C %s" %
> +                                            (remote_path, dest_dir))
> +        if status != 0:
> +            raise error.TestFail("Could not extract %s" % basename)
> +
>     vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
>     session = kvm_test_utils.wait_for_login(vm)
>
> -    # Collect some info
> +    # Collect test parameters
>     test_name = params.get("test_name")
>     test_timeout = int(params.get("test_timeout", 300))
>     test_control_file = params.get("test_control_file", "control")
> @@ -235,104 +269,47 @@ def run_autotest(test, params, env):
>                           (os.path.join(test.bindir, "autotest", "tests"),
>                            tarred_test_path, test_name), timeout=30)
>
> -    # Check if we need to copy autotest.tar.bz2
> -    copy = False
> -    output = session.get_command_output("ls -l autotest.tar.bz2")
> -    if "such file" in output:
> -        copy = True
> -    else:
> -        size = int(output.split()[4])
> -        if size != os.path.getsize(tarred_autotest_path):
> -            copy = True
> -    # Perform the copy
> -    if copy:
> -        logging.info("Copying autotest.tar.bz2 to guest"
> -                     " (file is missing or has a different size)...")
> -        if not vm.copy_files_to(tarred_autotest_path, ""):
> -            raise error.TestFail("Could not copy autotest.tar.bz2 to guest")
> -
> -    # Check if we need to copy <test_name>.tar.bz2
> -    copy = False
> -    output = session.get_command_output("ls -l %s.tar.bz2" % test_name)
> -    if "such file" in output:
> -        copy = True
> -    else:
> -        size = int(output.split()[4])
> -        if size != os.path.getsize(tarred_test_path):
> -            copy = True
> -    # Perform the copy
> -    if copy:
> -        logging.info("Copying %s.tar.bz2 to guest (file is missing or has a"
> -                     " different size)..." % test_name)
> -        if not vm.copy_files_to(tarred_test_path, ""):
> -            raise error.TestFail("Could not copy %s.tar.bz2 to guest" %
> -                                 test_name)
> +    # Copy autotest.tar.bz2
> +    copy_if_size_differs(vm, tarred_autotest_path, "autotest.tar.bz2")
> +
> +    # Copy <test_name>.tar.bz2
> +    copy_if_size_differs(vm, tarred_test_path, test_name + ".tar.bz2")
>
>     # Extract autotest.tar.bz2
> -    logging.info("Extracting autotest.tar.bz2...")
> -    status = session.get_command_status("tar xvfj autotest.tar.bz2")
> -    if status != 0:
> -        raise error.TestFail("Could not extract autotest.tar.bz2")
> +    extract(vm, "autotest.tar.bz2")
>
>     # mkdir autotest/tests
>     session.sendline("mkdir autotest/tests")
>
>     # Extract <test_name>.tar.bz2 into autotest/tests
> -    logging.info("Extracting %s.tar.bz2..." % test_name)
> -    status = session.get_command_status("tar xvfj %s.tar.bz2 -C "
> -                                        "autotest/tests" % test_name)
> -    if status != 0:
> -        raise error.TestFail("Could not extract %s.tar.bz2" % test_name)
> +    extract(vm, test_name + ".tar.bz2", "autotest/tests")
>
> -    # Cleaning up old remaining results
> -    session.sendline("rm -rf autotest/results/*")
> -    # Copying the selected control file (located inside
> -    # test.bindir/autotest_control to the autotest dir
> +    # Copy the selected control file (located inside
> +    # test.bindir/autotest_control) to the autotest dir
>     control_file_path = os.path.join(test.bindir, "autotest_control",
>                                      test_control_file)
>     if not vm.copy_files_to(control_file_path, "autotest/control"):
>         raise error.TestFail("Could not copy the test control file to guest")
> +
>     # Run the test
>     logging.info("Running test '%s'..." % test_name)
>     session.sendline("cd autotest")
>     session.sendline("rm -f control.state")
> +    session.sendline("rm -rf results/*")
>     session.read_up_to_prompt()
> -    session.sendline("bin/autotest control")
>     logging.info("---------------- Test output ----------------")
> -    match = session.read_up_to_prompt(timeout=test_timeout,
> -                                      print_func=logging.info)[0]
> +    status = session.get_command_status("bin/autotest control",
> +                                        timeout=test_timeout,
> +                                        print_func=logging.info)
>     logging.info("---------------- End of test output ----------------")
> -    if not match:
> +    if status is None:
>         raise error.TestFail("Timeout elapsed while waiting for test to "
>                              "complete")
> +
>     # Get the results generated by autotest
>     output = session.get_command_output("cat results/*/status")
> -
> -    # Parse test results
> -    result_list = scan_results.parse_results(output)
> -
> -    # Report test results and check for FAIL/ERROR status
> -    logging.info("Results (test, status, duration, info):")
> -    status_error = False
> -    status_fail = False
> -    if result_list == []:
> -        status_fail = True
> -        message_fail = ("Test '%s' did not produce any recognizable "
> -                        "results" % test_name)
> -    for result in result_list:
> -        logging.info(str(result))
> -        if result[1] == "FAIL":
> -            status_fail = True
> -            message_fail = ("Test '%s' ended with FAIL "
> -                            "(info: '%s')" % (result[0], result[3]))
> -        if result[1] == "ERROR":
> -            status_error = True
> -            message_error = ("Test '%s' ended with ERROR "
> -                             "(info: '%s')" % (result[0], result[3]))
> -        if result[1] == "ABORT":
> -            status_error = True
> -            message_error = ("Test '%s' ended with ABORT "
> -                             "(info: '%s')" % (result[0], result[3]))
> +    results = scan_results.parse_results(output)
> +    session.close
>
>     # Copy test results to the local bindir/guest_results
>     logging.info("Copying results back from guest...")
> @@ -342,11 +319,25 @@ def run_autotest(test, params, env):
>     if not vm.copy_files_from("autotest/results/default/*", guest_results_dir):
>         logging.error("Could not copy results back from guest")
>
> +    # Report test results
> +    logging.info("Results (test, status, duration, info):")
> +    for result in results:
> +        logging.info(str(result))
> +
> +    # Make a list of FAIL/ERROR/ABORT results (make sure FAIL results appear
> +    # before ERROR results, and ERROR results appear before ABORT results)
> +    bad_results = [r for r in results if r[1] == "FAIL"]
> +    bad_results += [r for r in results if r[1] == "ERROR"]
> +    bad_results += [r for r in results if r[1] == "ABORT"]
> +
>     # Fail the test if necessary
> -    if status_fail:
> -        raise error.TestFail(message_fail)
> -    elif status_error:
> -        raise error.TestError(message_error)

Caught 2 typos below, should be error.TestFail, not Error.TestFail.
Nevertheless, excellent cleanup. Applied with those minimal changes.

> +    if not results:
> +        raise Error.TestFail("Test '%s' did not produce any recognizable "
> +                             "results" % test_name)
> +    if bad_results:
> +        result = bad_results[0]
> +        raise Error.TestFail("Test '%s' ended with %s (reason: '%s')"
> +                             % (result[0], result[1], result[3]))
>
>
>  def internal_yum_update(session, command, prompt, timeout):
> --
> 1.5.4.1
>
> _______________________________________________
> Autotest mailing list
> Autotest@test.kernel.org
> http://test.kernel.org/cgi-bin/mailman/listinfo/autotest
>



-- 
Lucas

Re: [Autotest] [PATCH 12/19] KVM test: Add new module kvm_test_utils.py

[Yolkfull Chow]

On Wed, Sep 09, 2009 at 09:12:05PM +0300, Michael Goldish wrote:
> This module is meant to reduce code size by performing common test procedures.
> Generally, code here should look like test code.
> More specifically:
>     - Functions in this module should raise exceptions if things go wrong
>       (unlike functions in kvm_utils.py and kvm_vm.py which report failure via
>       their returned values).
>     - Functions in this module may use logging.info(), in addition to
>       logging.debug() and logging.error(), to log messages the user may be
>       interested in (unlike kvm_utils.py and kvm_vm.py which use
>       logging.debug() for everything that isn't an error).
>     - Functions in this module typically use functions and classes from
>       lower-level modules (e.g. kvm_utils.py, kvm_vm.py, kvm_subprocess.py).
>     - Functions in this module should not be used by lower-level modules.
>     - Functions in this module should be used in the right context.
>       For example, a function should not be used where it may display
>       misleading or inaccurate info or debug messages.
> 
> Signed-off-by: Michael Goldish <mgoldish@redhat.com>
> ---
>  client/tests/kvm/kvm_test_utils.py |   61 ++++++++++++++++++++++++++++++++++++
>  1 files changed, 61 insertions(+), 0 deletions(-)
>  create mode 100644 client/tests/kvm/kvm_test_utils.py
> 
> diff --git a/client/tests/kvm/kvm_test_utils.py b/client/tests/kvm/kvm_test_utils.py
> new file mode 100644
> index 0000000..39e92b9
> --- /dev/null
> +++ b/client/tests/kvm/kvm_test_utils.py
> @@ -0,0 +1,61 @@
> +import time, os, logging, re, commands
> +from autotest_lib.client.common_lib import utils, error
> +import kvm_utils, kvm_vm, kvm_subprocess
> +
> +"""
> +High-level KVM test utility functions.
> +
> +This module is meant to reduce code size by performing common test procedures.
> +Generally, code here should look like test code.
> +More specifically:
> +    - Functions in this module should raise exceptions if things go wrong
> +      (unlike functions in kvm_utils.py and kvm_vm.py which report failure via
> +      their returned values).
> +    - Functions in this module may use logging.info(), in addition to
> +      logging.debug() and logging.error(), to log messages the user may be
> +      interested in (unlike kvm_utils.py and kvm_vm.py which use
> +      logging.debug() for anything that isn't an error).
> +    - Functions in this module typically use functions and classes from
> +      lower-level modules (e.g. kvm_utils.py, kvm_vm.py, kvm_subprocess.py).
> +    - Functions in this module should not be used by lower-level modules.
> +    - Functions in this module should be used in the right context.
> +      For example, a function should not be used where it may display
> +      misleading or inaccurate info or debug messages.
> +
> +@copyright: 2008-2009 Red Hat Inc.
> +"""
> +
> +
> +def get_living_vm(env, vm_name):
> +    """
> +    Get a VM object from the environment and make sure it's alive.
> +
> +    @param env: Dictionary with test environment.
> +    @param vm_name: Name of the desired VM object.
> +    @return: A VM object.
> +    """
> +    vm = kvm_utils.env_get_vm(env, vm_name)
> +    if not vm:
> +        raise error.TestError("VM '%s' not found in environment" % vm_name)
> +    if not vm.is_alive():
> +        raise error.TestError("VM '%s' seems to be dead; test requires a "
> +                              "living VM" % vm_name)
> +    return vm
> +
> +
> +def wait_for_login(vm, nic_index=0, timeout=240):
> +    """
> +    Try logging into a VM repeatedly.  Stop on success or when timeout expires.
> +
> +    @param vm: VM object.
> +    @param nic_index: Index of NIC to access in the VM.
> +    @param timeout: Time to wait before giving up.
> +    @return: A shell session object.
> +    """
> +    logging.info("Waiting for guest to be up...")
> +    session = kvm_utils.wait_for(lambda: vm.remote_login(nic_index=nic_index),
> +                                 timeout, 0, 2)
> +    if not session:
> +        raise error.TestFail("Could not log into guest")

Hi Michael, I think we should also add a parameter 'vm_name' for
wait_for_login(). On the assumption that we boot more than one VMs, it's
hard to know which guest failed to login according to message above.
What do you think? :-)

> +    logging.info("Logged in")
> +    return session
> -- 
> 1.5.4.1
> 
> _______________________________________________
> Autotest mailing list
> Autotest@test.kernel.org
> http://test.kernel.org/cgi-bin/mailman/listinfo/autotest

Re: [Autotest] [PATCH 12/19] KVM test: Add new module kvm_test_utils.py

[Uri Lublin]

On 09/14/2009 08:26 AM, Yolkfull Chow wrote:
> On Wed, Sep 09, 2009 at 09:12:05PM +0300, Michael Goldish wrote:
>> This module is meant to reduce code size by performing common test procedures.
>> Generally, code here should look like test code.

>> +def wait_for_login(vm, nic_index=0, timeout=240):
>> +    """
>> +    Try logging into a VM repeatedly.  Stop on success or when timeout expires.
>> +
>> +    @param vm: VM object.
>> +    @param nic_index: Index of NIC to access in the VM.
>> +    @param timeout: Time to wait before giving up.
>> +    @return: A shell session object.
>> +    """
>> +    logging.info("Waiting for guest to be up...")
>> +    session = kvm_utils.wait_for(lambda: vm.remote_login(nic_index=nic_index),
>> +                                 timeout, 0, 2)
>> +    if not session:
>> +        raise error.TestFail("Could not log into guest")
>
> Hi Michael, I think we should also add a parameter 'vm_name' for
> wait_for_login(). On the assumption that we boot more than one VMs, it's
> hard to know which guest failed to login according to message above.
> What do you think? :-)
>

The VM object ("vm" parameter) "knows" its own name.
It is a good idea to add that name to log/error messages, since we do want to 
run different tests (VMs) in parallel (although the logs should be also saved in 
different directories/files).

Regards,
     Uri.

Re: [Autotest] [PATCH 12/19] KVM test: Add new module kvm_test_utils.py

[Yolkfull Chow]

On Mon, Sep 14, 2009 at 10:58:01AM +0300, Uri Lublin wrote:
> On 09/14/2009 08:26 AM, Yolkfull Chow wrote:
>> On Wed, Sep 09, 2009 at 09:12:05PM +0300, Michael Goldish wrote:
>>> This module is meant to reduce code size by performing common test procedures.
>>> Generally, code here should look like test code.
>
>>> +def wait_for_login(vm, nic_index=0, timeout=240):
>>> +    """
>>> +    Try logging into a VM repeatedly.  Stop on success or when timeout expires.
>>> +
>>> +    @param vm: VM object.
>>> +    @param nic_index: Index of NIC to access in the VM.
>>> +    @param timeout: Time to wait before giving up.
>>> +    @return: A shell session object.
>>> +    """
>>> +    logging.info("Waiting for guest to be up...")
>>> +    session = kvm_utils.wait_for(lambda: vm.remote_login(nic_index=nic_index),
>>> +                                 timeout, 0, 2)
>>> +    if not session:
>>> +        raise error.TestFail("Could not log into guest")
>>
>> Hi Michael, I think we should also add a parameter 'vm_name' for
>> wait_for_login(). On the assumption that we boot more than one VMs, it's
>> hard to know which guest failed to login according to message above.
>> What do you think? :-)
>>
>
> The VM object ("vm" parameter) "knows" its own name.
> It is a good idea to add that name to log/error messages, since we do 
> want to run different tests (VMs) in parallel (although the logs should 
> be also saved in different directories/files).
>

Yes, I did ignore that we could use 'vm.name'
instead of adding a parameter for wait_for_login(). Therefore those
log/error messages could be added something like this:

if not session:
    raise error.TestFail("Could not log into guest %s" vm.name)

Thanks for pointing out that. :-)

> Regards,
>     Uri.

> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Autotest] [PATCH 01/19] KVM test: kvm_utils.py: make verify_ip_address_ownership() more robust

[Lucas Meneghel Rodrigues]

As the changes were mostly bugfixes and short, I managed to review and
apply them all. Thank you very much Michael!

On Wed, Sep 9, 2009 at 3:11 PM, Michael Goldish <mgoldish@redhat.com> wrote:
> Use arping in addition to querying the arp cache.
> Under certain circumstances a TCP connection may not trigger an ARP request so
> it must be triggered explicitly using arping.
>
> Signed-off-by: Michael Goldish <mgoldish@redhat.com>
> ---
>  client/tests/kvm/kvm_utils.py |   40 ++++++++++++++++++----------------------
>  1 files changed, 18 insertions(+), 22 deletions(-)
>
> diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py
> index dfca938..f046810 100644
> --- a/client/tests/kvm/kvm_utils.py
> +++ b/client/tests/kvm/kvm_utils.py
> @@ -152,39 +152,35 @@ def get_mac_ip_pair_from_dict(dict):
>     return (None, None)
>
>
> -def verify_ip_address_ownership(ip, macs, timeout=3.0):
> +def verify_ip_address_ownership(ip, macs, timeout=10.0):
>     """
> -    Connect to a given IP address and make sure its MAC address equals one of
> -    the given MAC address.
> +    Use arping and the ARP cache to make sure a given IP address belongs to one
> +    of the given MAC addresses.
>
>     @param ip: An IP address.
>     @param macs: A list or tuple of MAC addresses.
>     @return: True iff ip is assigned to a MAC address in macs.
>     """
> -    def check_arp_cache(regex):
> -        o = commands.getoutput("/sbin/arp -n")
> -        return bool(re.search(regex, o, re.IGNORECASE))
> -
> +    # Compile a regex that matches the given IP address and any of the given
> +    # MAC addresses
>     mac_regex = "|".join("(%s)" % mac for mac in macs)
>     regex = re.compile(r"\b%s\b.*\b(%s)\b" % (ip, mac_regex))
>
> -    if check_arp_cache(regex):
> +    # Check the ARP cache
> +    o = commands.getoutput("/sbin/arp -n")
> +    if re.search(regex, o, re.IGNORECASE):
>         return True
>
> -    s = socket.socket()
> -    s.setblocking(False)
> -    try:
> -        s.connect((ip, 55555))
> -    except socket.error:
> -        pass
> -    end_time = time.time() + timeout
> -    while time.time() < end_time:
> -        time.sleep(0.2)
> -        if check_arp_cache(regex):
> -            s.close()
> -            return True
> -    s.close()
> -    return False
> +    # Get the name of the bridge device for arping
> +    o = commands.getoutput("/sbin/ip route get %s" % ip)
> +    dev = re.findall("dev\s+\S+", o, re.IGNORECASE)
> +    if not dev:
> +        return False
> +    dev = dev[0].split()[-1]
> +
> +    # Send an ARP request
> +    o = commands.getoutput("/sbin/arping -f -c 3 -I %s %s" % (dev, ip))
> +    return bool(re.search(regex, o, re.IGNORECASE))
>
>
>  # Functions for working with the environment (a dict-like object)
> --
> 1.5.4.1
>
> _______________________________________________
> Autotest mailing list
> Autotest@test.kernel.org
> http://test.kernel.org/cgi-bin/mailman/listinfo/autotest
>



-- 
Lucas

Re: [Autotest] [PATCH 12/19] KVM test: Add new module kvm_test_utils.py

[Michael Goldish]

----- "Uri Lublin" <uril@redhat.com> wrote:

> On 09/14/2009 08:26 AM, Yolkfull Chow wrote:
> > On Wed, Sep 09, 2009 at 09:12:05PM +0300, Michael Goldish wrote:
> >> This module is meant to reduce code size by performing common test
> procedures.
> >> Generally, code here should look like test code.
> 
> >> +def wait_for_login(vm, nic_index=0, timeout=240):
> >> +    """
> >> +    Try logging into a VM repeatedly.  Stop on success or when
> timeout expires.
> >> +
> >> +    @param vm: VM object.
> >> +    @param nic_index: Index of NIC to access in the VM.
> >> +    @param timeout: Time to wait before giving up.
> >> +    @return: A shell session object.
> >> +    """
> >> +    logging.info("Waiting for guest to be up...")
> >> +    session = kvm_utils.wait_for(lambda:
> vm.remote_login(nic_index=nic_index),
> >> +                                 timeout, 0, 2)
> >> +    if not session:
> >> +        raise error.TestFail("Could not log into guest")
> >
> > Hi Michael, I think we should also add a parameter 'vm_name' for
> > wait_for_login(). On the assumption that we boot more than one VMs,
> it's
> > hard to know which guest failed to login according to message
> above.
> > What do you think? :-)
> >
> 
> The VM object ("vm" parameter) "knows" its own name.
> It is a good idea to add that name to log/error messages, since we do
> want to 
> run different tests (VMs) in parallel (although the logs should be
> also saved in 
> different directories/files).
> 
> Regards,
>      Uri.

Currently it would be useful for tests that use multiple VMs, not for
multiple tests running in parallel, because all tests call their main VM
'vm1'.  That can be changed though.  In any case I agree that it's a
good idea.