[PATCH 0/9] Make use of the redirection of guest serial

[PATCH 0/9] Make use of the redirection of guest serial

[Jason Wang]

The guest console is useful for troubleshooting the failure cases
especially for the one who has calltrace. And the session through it
is also useful for the networking related test cases ( consider the
test which could shutdown or crash the guest network ). so this
patchset first redirects the guest serial to unix domain socket and
then could log the guest serial through a thread or make it as a
method of session. The last patch makes all linux guests to use serial as
their consoles through unattended files. 

---

Jason Wang (9):
      KVM test: Introduce the prompt assist
      KVM test: Add the ability to send the username in remote_login()
      KVM test: Make the login re suitable for serial console
      KVM test: Redirect the serial to the unix domain socket
      KVM test: Log the content from guest serial console
      KVM test: Raise error when met unknown type in kvm_vm.remote_login().
      KVM test: Introduce the local_login()
      KVM test: Create the background threads before calling process()
      KVM test: Redirect the console to serial for all linux guests


 client/tests/kvm/kvm_preprocessing.py        |   64 ++++++++++++++++++++++++--
 client/tests/kvm/kvm_utils.py                |   23 +++++++--
 client/tests/kvm/kvm_vm.py                   |   48 ++++++++++++++++----
 client/tests/kvm/tests_base.cfg.sample       |    1 
 client/tests/kvm/unattended/Fedora-10.ks     |    2 -
 client/tests/kvm/unattended/Fedora-11.ks     |    2 -
 client/tests/kvm/unattended/Fedora-12.ks     |    2 -
 client/tests/kvm/unattended/Fedora-8.ks      |    2 -
 client/tests/kvm/unattended/Fedora-9.ks      |    2 -
 client/tests/kvm/unattended/OpenSUSE-11.xml  |    1 
 client/tests/kvm/unattended/RHEL-3-series.ks |    2 -
 client/tests/kvm/unattended/RHEL-4-series.ks |    2 -
 client/tests/kvm/unattended/RHEL-5-series.ks |    2 -
 client/tests/kvm/unattended/SLES-11.xml      |    1 
 14 files changed, 128 insertions(+), 26 deletions(-)

-- 
Signature

[PATCH 1/9] KVM test: Introduce the prompt assist

[Jason Wang]

Sometimes we need to send an assist string to a session in order to
get the prompt especially when re-connecting to an already logged
serial session. This patch send the assist string before doing the
pattern matching of remote_login.

Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 client/tests/kvm/kvm_utils.py |    9 +++++++--
 1 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py
index 25f3c8c..9adbaee 100644
--- a/client/tests/kvm/kvm_utils.py
+++ b/client/tests/kvm/kvm_utils.py
@@ -451,7 +451,8 @@ def check_kvm_source_dir(source_dir):
 # The following are functions used for SSH, SCP and Telnet communication with
 # guests.
 
-def remote_login(command, password, prompt, linesep="\n", timeout=10):
+def remote_login(command, password, prompt, linesep="\n", timeout=10,
+                 prompt_assist = None):
     """
     Log into a remote host (guest) using SSH or Telnet. Run the given command
     using kvm_spawn and provide answers to the questions asked. If timeout
@@ -468,7 +469,8 @@ def remote_login(command, password, prompt, linesep="\n", timeout=10):
     @param timeout: The maximal time duration (in seconds) to wait for each
             step of the login procedure (i.e. the "Are you sure" prompt, the
             password prompt, the shell prompt, etc)
-
+    @prarm prompt_assist: An assistant string sent before the pattern
+            matching in order to get the prompt for some kinds of shell_client.
     @return Return the kvm_spawn object on success and None on failure.
     """
     sub = kvm_subprocess.kvm_shell_session(command,
@@ -479,6 +481,9 @@ def remote_login(command, password, prompt, linesep="\n", timeout=10):
 
     logging.debug("Trying to login with command '%s'" % command)
 
+    if prompt_assist is not None:
+        sub.sendline(prompt_assist)
+
     while True:
         (match, text) = sub.read_until_last_line_matches(
                 [r"[Aa]re you sure", r"[Pp]assword:\s*$", r"^\s*[Ll]ogin:\s*$",

[PATCH 2/9] KVM test: Add the ability to send the username in remote_login()

[Jason Wang]

In order to let the serial console work, we must let the
remote_login() send the username when needed.

Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 client/tests/kvm/kvm_utils.py |   14 ++++++++++----
 1 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py
index 9adbaee..1ea0852 100644
--- a/client/tests/kvm/kvm_utils.py
+++ b/client/tests/kvm/kvm_utils.py
@@ -452,7 +452,7 @@ def check_kvm_source_dir(source_dir):
 # guests.
 
 def remote_login(command, password, prompt, linesep="\n", timeout=10,
-                 prompt_assist = None):
+                 prompt_assist = None, username = None):
     """
     Log into a remote host (guest) using SSH or Telnet. Run the given command
     using kvm_spawn and provide answers to the questions asked. If timeout
@@ -471,6 +471,8 @@ def remote_login(command, password, prompt, linesep="\n", timeout=10,
             password prompt, the shell prompt, etc)
     @prarm prompt_assist: An assistant string sent before the pattern
             matching in order to get the prompt for some kinds of shell_client.
+    @param username: The user name used to log into the session
+            
     @return Return the kvm_spawn object on success and None on failure.
     """
     sub = kvm_subprocess.kvm_shell_session(command,
@@ -505,9 +507,13 @@ def remote_login(command, password, prompt, linesep="\n", timeout=10,
                 sub.close()
                 return None
         elif match == 2:  # "login:"
-            logging.debug("Got unexpected login prompt")
-            sub.close()
-            return None
+            if username is None:
+                logging.debug("Got unexpected login prompt")
+                sub.close()
+                return None
+            else:
+                sub.sendline(username)
+                continue
         elif match == 3:  # "Connection closed"
             logging.debug("Got 'Connection closed'")
             sub.close()

[PATCH 3/9] KVM test: Make the login re suitable for serial console

[Jason Wang]

Current matching re ^\s*[Ll]ogin:\s*$ is not suitable for the serial
console, so change it to [Ll]ogin:.

Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 client/tests/kvm/kvm_utils.py |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py
index 1ea0852..bb42314 100644
--- a/client/tests/kvm/kvm_utils.py
+++ b/client/tests/kvm/kvm_utils.py
@@ -488,7 +488,7 @@ def remote_login(command, password, prompt, linesep="\n", timeout=10,
 
     while True:
         (match, text) = sub.read_until_last_line_matches(
-                [r"[Aa]re you sure", r"[Pp]assword:\s*$", r"^\s*[Ll]ogin:\s*$",
+                [r"[Aa]re you sure", r"[Pp]assword:\s*$", r"[Ll]ogin:",
                  r"[Cc]onnection.*closed", r"[Cc]onnection.*refused",
                  r"[Pp]lease wait", prompt],
                  timeout=timeout, internal_timeout=0.5)

[PATCH 4/9] KVM test: Redirect the serial to the unix domain socket

[Jason Wang]

This patch redirect the guest serial to the unix domain socket which
would be used by the following patches to dump its content or use it
as a session.

Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 client/tests/kvm/kvm_vm.py |   19 ++++++++++++-------
 1 files changed, 12 insertions(+), 7 deletions(-)

diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
index 6bc7987..8f4753f 100755
--- a/client/tests/kvm/kvm_vm.py
+++ b/client/tests/kvm/kvm_vm.py
@@ -116,17 +116,20 @@ class VM:
         self.address_cache = address_cache
         self.pci_assignable = None
 
-        # Find available monitor filename
+        # Find available filenames for monitor and guest serial redirection
         while True:
-            # The monitor filename should be unique
+            # The filenames should be unique
             self.instance = (time.strftime("%Y%m%d-%H%M%S-") +
                              kvm_utils.generate_random_string(4))
-            self.monitor_file_name = os.path.join("/tmp",
-                                                  "monitor-" + self.instance)
-            if not os.path.exists(self.monitor_file_name):
-                break
-
 
+            names = [os.path.join("/tmp", type + self.instance) for type in
+                     "monitor-", "serial-"]
+            if True in [os.path.exists(file) for file in names]:
+                continue
+            else:
+                [self.monitor_file_name, self.serial_file_name] = names
+                break
+                                                         
     def clone(self, name=None, params=None, root_dir=None, address_cache=None):
         """
         Return a clone of the VM object with optionally modified parameters.
@@ -316,6 +319,8 @@ class VM:
             for pci_id in self.pa_pci_ids:
                 qemu_cmd += " -pcidevice host=%s" % pci_id
 
+        qemu_cmd += " -serial unix:%s,server,nowait" % self.serial_file_name
+
         return qemu_cmd

[PATCH 5/9] KVM test: Log the content from guest serial console

[Jason Wang]

This patch tries to get the content of guest serial and log it into
the debug directoy of the testcase through a dedicated thread which is
created in the preprocessing and ended in the postprocessing. The
params of serial_mode must be set to "dump" in order to make use of
this feature.

Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 client/tests/kvm/kvm_preprocessing.py  |   59 +++++++++++++++++++++++++++++++-
 client/tests/kvm/tests_base.cfg.sample |    1 +
 2 files changed, 59 insertions(+), 1 deletions(-)

diff --git a/client/tests/kvm/kvm_preprocessing.py b/client/tests/kvm/kvm_preprocessing.py
index 4b9290c..50d0e35 100644
--- a/client/tests/kvm/kvm_preprocessing.py
+++ b/client/tests/kvm/kvm_preprocessing.py
@@ -1,4 +1,5 @@
 import sys, os, time, commands, re, logging, signal, glob, threading, shutil
+import socket, select
 from autotest_lib.client.bin import test, utils
 from autotest_lib.client.common_lib import error
 import kvm_vm, kvm_utils, kvm_subprocess, ppm_utils
@@ -13,7 +14,8 @@ except ImportError:
 
 _screendump_thread = None
 _screendump_thread_termination_event = None
-
+_serialdump_thread = None
+_serialdump_thread_termination_event = None
 
 def preprocess_image(test, params):
     """
@@ -267,6 +269,16 @@ def preprocess(test, params, env):
                                               args=(test, params, env))
         _screendump_thread.start()
 
+    # Start the serial dump thread
+    if params.get("serial_mode") == "dump":
+        logging.debug("Starting serialdump thread")
+        global _serialdump_thread, _serialdump_thread_termination_event
+        _serialdump_thread_termination_event = threading.Event()
+        _serialdump_thread = threading.Thread(target=_dump_serial_console,
+                                              args=(test, params, env))
+        _serialdump_thread.start()
+
+
 
 def postprocess(test, params, env):
     """
@@ -286,6 +298,13 @@ def postprocess(test, params, env):
         _screendump_thread_termination_event.set()
         _screendump_thread.join(10)
 
+    # Terminate the serialdump thread
+    global _serialdump_thread, _serialdump_thread_termination_event
+    if _serialdump_thread:
+        logging.debug("Terminating serialdump thread...")
+        _serialdump_thread_termination_event.set()
+        _serialdump_thread.join(10)
+
     # Warn about corrupt PPM files
     for f in glob.glob(os.path.join(test.debugdir, "*.ppm")):
         if not ppm_utils.image_verify_ppm_file(f):
@@ -450,3 +469,41 @@ def _take_screendumps(test, params, env):
         if _screendump_thread_termination_event.isSet():
             break
         _screendump_thread_termination_event.wait(delay)
+
+def _dump_serial_console(test, params, env):
+    global _serialdump_thread_termination_event
+    rs = []
+    files = {}
+
+    while True:
+        for vm in kvm_utils.env_get_all_vms(env):
+            if not files.has_key(vm):
+                try:
+                    serial_socket = socket.socket(socket.AF_UNIX,
+                                                  socket.SOCK_STREAM)
+                    serial_socket.setblocking(False)
+                    serial_socket.connect(vm.serial_file_name)
+                except:
+                    logging.debug("Could not connect to serial socket for %s" %
+                                  vm.name)
+                    continue
+                rs.append(serial_socket)
+                serial_dump_filename = os.path.join(test.debugdir,
+                                                    "serial-%s" % vm.name)
+                files[vm] = [serial_socket, file(serial_dump_filename, "a+")]
+
+        r, w, x = select.select(rs, [], [], 0.5)
+        for vm in files.keys():
+            [s ,d] = files[vm]
+            if s in r:
+                data = s.recv(16384)
+                if len(data) == 0:
+                    rs.remove(s)
+                    files.pop(vm)
+                else:
+                    d.write(data)
+                
+        if _serialdump_thread_termination_event.isSet():
+            break
+
+            
diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
index 9f82ffb..169a69e 100644
--- a/client/tests/kvm/tests_base.cfg.sample
+++ b/client/tests/kvm/tests_base.cfg.sample
@@ -13,6 +13,7 @@ start_vm = yes
 kill_vm = no
 kill_vm_gracefully = yes
 kill_unresponsive_vms = yes
+serial_mode = dump
 
 # Screendump specific stuff
 convert_ppm_files_to_png_on_error = yes

[PATCH 6/9] KVM test: Raise error when met unknown type in kvm_vm.remote_login().

[Jason Wang]

Need to raise the error when met the unknown type of shell_client in
kvm_vm.remote_login() in order to avoid the traceback.

Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 client/tests/kvm/kvm_vm.py |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
index 8f4753f..0cdf925 100755
--- a/client/tests/kvm/kvm_vm.py
+++ b/client/tests/kvm/kvm_vm.py
@@ -806,7 +806,9 @@ class VM:
         elif client == "nc":
             session = kvm_utils.netcat(address, port, username, password,
                                        prompt, linesep, timeout)
-
+        else:
+            raise error.TestError("Unknown shell_client type %s" % client)
+            
         if session:
             session.set_status_test_command(self.params.get("status_test_"
                                                             "command", ""))

[PATCH 7/9] KVM test: Introduce the local_login()

[Jason Wang]

This patch introduces a new method which is used to log into the guest
through the guest serial console. The serial_mode must be set to
"session" in order to make use of this patch.

Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 client/tests/kvm/kvm_vm.py |   25 +++++++++++++++++++++++++
 1 files changed, 25 insertions(+), 0 deletions(-)

diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
index 0cdf925..a22893b 100755
--- a/client/tests/kvm/kvm_vm.py
+++ b/client/tests/kvm/kvm_vm.py
@@ -814,7 +814,32 @@ class VM:
                                                             "command", ""))
         return session
 
+    def local_login(self, timeout=240):
+        """
+        Log into the guest via serial console
+        If timeout expires while waiting for output from the guest (e.g. a
+        password prompt or a shell prompt) -- fail.
+        """
+
+        serial_mode = self.params.get("serial_mode")
+        username = self.params.get("username", "")
+        password = self.params.get("password", "")
+        prompt = self.params.get("shell_prompt", "[\#\$]")
+        linesep = eval("'%s'" % self.params.get("shell_linesep", r"\n"))
 
+        if serial_mode != "session":
+            logging.debug("serial_mode is not session")
+            return None
+        else:
+            command = "nc -U %s"  % self.serial_file_name
+            assist = self.params.get("prompt_assist")
+            session = kvm_utils.remote_login(command, password, prompt, linesep,
+                                             timeout, "", username)
+            if session:
+                session.set_status_test_command(self.params.get("status_test_"
+                                                                "command", ""))
+            return session
+            
     def copy_files_to(self, local_path, remote_path, nic_index=0, timeout=300):
         """
         Transfer files to the guest.

[PATCH 8/9] KVM test: Create the background threads before calling process()

[Jason Wang]

If the screendump and scrialdump threads are created after the
process(), we may lose the progress tracking of guest shutting
down. So this patch creates them before calling process() in preprocess.

Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 client/tests/kvm/kvm_preprocessing.py |    5 ++---
 1 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/client/tests/kvm/kvm_preprocessing.py b/client/tests/kvm/kvm_preprocessing.py
index 50d0e35..73e835a 100644
--- a/client/tests/kvm/kvm_preprocessing.py
+++ b/client/tests/kvm/kvm_preprocessing.py
@@ -257,9 +257,6 @@ def preprocess(test, params, env):
                         int(params.get("pre_command_timeout", "600")),
                         params.get("pre_command_noncritical") == "yes")
 
-    # Preprocess all VMs and images
-    process(test, params, env, preprocess_image, preprocess_vm)
-
     # Start the screendump thread
     if params.get("take_regular_screendumps") == "yes":
         logging.debug("Starting screendump thread")
@@ -278,6 +275,8 @@ def preprocess(test, params, env):
                                               args=(test, params, env))
         _serialdump_thread.start()
 
+    # Preprocess all VMs and images
+    process(test, params, env, preprocess_image, preprocess_vm)
 
 
 def postprocess(test, params, env):

[PATCH 9/9] KVM test: Redirect the console to serial for all linux guests

[Jason Wang]

As we have the ability to dump the content from serial console or use
a session through it, we need to redirect the console to serial
through unattended files to make use of it.

Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 client/tests/kvm/unattended/Fedora-10.ks     |    2 +-
 client/tests/kvm/unattended/Fedora-11.ks     |    2 +-
 client/tests/kvm/unattended/Fedora-12.ks     |    2 +-
 client/tests/kvm/unattended/Fedora-8.ks      |    2 +-
 client/tests/kvm/unattended/Fedora-9.ks      |    2 +-
 client/tests/kvm/unattended/OpenSUSE-11.xml  |    1 +
 client/tests/kvm/unattended/RHEL-3-series.ks |    2 +-
 client/tests/kvm/unattended/RHEL-4-series.ks |    2 +-
 client/tests/kvm/unattended/RHEL-5-series.ks |    2 +-
 client/tests/kvm/unattended/SLES-11.xml      |    1 +
 10 files changed, 10 insertions(+), 8 deletions(-)

diff --git a/client/tests/kvm/unattended/Fedora-10.ks b/client/tests/kvm/unattended/Fedora-10.ks
index 61e59d7..8628036 100644
--- a/client/tests/kvm/unattended/Fedora-10.ks
+++ b/client/tests/kvm/unattended/Fedora-10.ks
@@ -11,7 +11,7 @@ firewall --enabled --ssh
 selinux --enforcing
 timezone --utc America/New_York
 firstboot --disable
-bootloader --location=mbr
+bootloader --location=mbr --append="console=ttyS0,115200"
 zerombr
 clearpart --all --initlabel
 autopart
diff --git a/client/tests/kvm/unattended/Fedora-11.ks b/client/tests/kvm/unattended/Fedora-11.ks
index 0be7d06..5ce8ee2 100644
--- a/client/tests/kvm/unattended/Fedora-11.ks
+++ b/client/tests/kvm/unattended/Fedora-11.ks
@@ -10,7 +10,7 @@ firewall --enabled --ssh
 selinux --enforcing
 timezone --utc America/New_York
 firstboot --disable
-bootloader --location=mbr
+bootloader --location=mbr --append="console=ttyS0,115200"
 zerombr
 
 clearpart --all --initlabel
diff --git a/client/tests/kvm/unattended/Fedora-12.ks b/client/tests/kvm/unattended/Fedora-12.ks
index 0be7d06..5ce8ee2 100644
--- a/client/tests/kvm/unattended/Fedora-12.ks
+++ b/client/tests/kvm/unattended/Fedora-12.ks
@@ -10,7 +10,7 @@ firewall --enabled --ssh
 selinux --enforcing
 timezone --utc America/New_York
 firstboot --disable
-bootloader --location=mbr
+bootloader --location=mbr --append="console=ttyS0,115200"
 zerombr
 
 clearpart --all --initlabel
diff --git a/client/tests/kvm/unattended/Fedora-8.ks b/client/tests/kvm/unattended/Fedora-8.ks
index f4a872d..884d556 100644
--- a/client/tests/kvm/unattended/Fedora-8.ks
+++ b/client/tests/kvm/unattended/Fedora-8.ks
@@ -11,7 +11,7 @@ firewall --enabled --ssh
 selinux --enforcing
 timezone --utc America/New_York
 firstboot --disable
-bootloader --location=mbr
+bootloader --location=mbr --append="console=ttyS0,115200"
 zerombr
 clearpart --all --initlabel
 autopart
diff --git a/client/tests/kvm/unattended/Fedora-9.ks b/client/tests/kvm/unattended/Fedora-9.ks
index f4a872d..884d556 100644
--- a/client/tests/kvm/unattended/Fedora-9.ks
+++ b/client/tests/kvm/unattended/Fedora-9.ks
@@ -11,7 +11,7 @@ firewall --enabled --ssh
 selinux --enforcing
 timezone --utc America/New_York
 firstboot --disable
-bootloader --location=mbr
+bootloader --location=mbr --append="console=ttyS0,115200"
 zerombr
 clearpart --all --initlabel
 autopart
diff --git a/client/tests/kvm/unattended/OpenSUSE-11.xml b/client/tests/kvm/unattended/OpenSUSE-11.xml
index 7dd44fa..2a4fec0 100644
--- a/client/tests/kvm/unattended/OpenSUSE-11.xml
+++ b/client/tests/kvm/unattended/OpenSUSE-11.xml
@@ -50,6 +50,7 @@
         <module>edd</module>
       </initrd_module>
     </initrd_modules>
+    <append>console=ttyS0,115200</append>
     <loader_type>grub</loader_type>
     <sections config:type="list"/>
   </bootloader>
diff --git a/client/tests/kvm/unattended/RHEL-3-series.ks b/client/tests/kvm/unattended/RHEL-3-series.ks
index 884b386..26b1130 100644
--- a/client/tests/kvm/unattended/RHEL-3-series.ks
+++ b/client/tests/kvm/unattended/RHEL-3-series.ks
@@ -10,7 +10,7 @@ rootpw 123456
 firewall --enabled --ssh
 timezone America/New_York
 firstboot --disable
-bootloader --location=mbr
+bootloader --location=mbr --append="console=ttyS0,115200"
 clearpart --all --initlabel
 autopart
 reboot
diff --git a/client/tests/kvm/unattended/RHEL-4-series.ks b/client/tests/kvm/unattended/RHEL-4-series.ks
index ce4a430..f2f934f 100644
--- a/client/tests/kvm/unattended/RHEL-4-series.ks
+++ b/client/tests/kvm/unattended/RHEL-4-series.ks
@@ -11,7 +11,7 @@ firewall --enabled --ssh
 selinux --enforcing
 timezone --utc America/New_York
 firstboot --disable
-bootloader --location=mbr
+bootloader --location=mbr --append="console=ttyS0,115200"
 zerombr
 clearpart --all --initlabel
 autopart
diff --git a/client/tests/kvm/unattended/RHEL-5-series.ks b/client/tests/kvm/unattended/RHEL-5-series.ks
index f4a872d..884d556 100644
--- a/client/tests/kvm/unattended/RHEL-5-series.ks
+++ b/client/tests/kvm/unattended/RHEL-5-series.ks
@@ -11,7 +11,7 @@ firewall --enabled --ssh
 selinux --enforcing
 timezone --utc America/New_York
 firstboot --disable
-bootloader --location=mbr
+bootloader --location=mbr --append="console=ttyS0,115200"
 zerombr
 clearpart --all --initlabel
 autopart
diff --git a/client/tests/kvm/unattended/SLES-11.xml b/client/tests/kvm/unattended/SLES-11.xml
index 93e5685..7656891 100644
--- a/client/tests/kvm/unattended/SLES-11.xml
+++ b/client/tests/kvm/unattended/SLES-11.xml
@@ -49,6 +49,7 @@
         <module>edd</module>
       </initrd_module>
     </initrd_modules>
+    <append>console=ttyS0,115200</append>
     <loader_type>grub</loader_type>
     <sections config:type="list"/>
   </bootloader>

Re: [PATCH 3/9] KVM test: Make the login re suitable for serial console

[Michael Goldish]

On 04/26/2010 01:03 PM, Jason Wang wrote:
> Current matching re ^\s*[Ll]ogin:\s*$ is not suitable for the serial
> console, so change it to [Ll]ogin:.
> 
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> ---
>  client/tests/kvm/kvm_utils.py |    2 +-
>  1 files changed, 1 insertions(+), 1 deletions(-)
> 
> diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py
> index 1ea0852..bb42314 100644
> --- a/client/tests/kvm/kvm_utils.py
> +++ b/client/tests/kvm/kvm_utils.py
> @@ -488,7 +488,7 @@ def remote_login(command, password, prompt, linesep="\n", timeout=10,
>  
>      while True:
>          (match, text) = sub.read_until_last_line_matches(
> -                [r"[Aa]re you sure", r"[Pp]assword:\s*$", r"^\s*[Ll]ogin:\s*$",
> +                [r"[Aa]re you sure", r"[Pp]assword:\s*$", r"[Ll]ogin:",

This RE will also match

"Last login: Wed Apr 28 14:07:28 2010 from localhost.localdomain"

so we should probably settle for [Ll]ogin:\s*$.

>                   r"[Cc]onnection.*closed", r"[Cc]onnection.*refused",
>                   r"[Pp]lease wait", prompt],
>                   timeout=timeout, internal_timeout=0.5)
> 
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/9] KVM test: Introduce the prompt assist

[Michael Goldish]

On 04/26/2010 01:03 PM, Jason Wang wrote:
> Sometimes we need to send an assist string to a session in order to
> get the prompt especially when re-connecting to an already logged
> serial session. This patch send the assist string before doing the
> pattern matching of remote_login.

Can you give an example of a prompt assist string, and a typical usage
example?  What guests require prompt assist strings?

> Signed-off-by: Jason Wang <jasowang@redhat.com>
> ---
>  client/tests/kvm/kvm_utils.py |    9 +++++++--
>  1 files changed, 7 insertions(+), 2 deletions(-)
> 
> diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py
> index 25f3c8c..9adbaee 100644
> --- a/client/tests/kvm/kvm_utils.py
> +++ b/client/tests/kvm/kvm_utils.py
> @@ -451,7 +451,8 @@ def check_kvm_source_dir(source_dir):
>  # The following are functions used for SSH, SCP and Telnet communication with
>  # guests.
>  
> -def remote_login(command, password, prompt, linesep="\n", timeout=10):
> +def remote_login(command, password, prompt, linesep="\n", timeout=10,
> +                 prompt_assist = None):
                                 ^ ^
These spaces do not conform with PEP 8.

>      """
>      Log into a remote host (guest) using SSH or Telnet. Run the given command
>      using kvm_spawn and provide answers to the questions asked. If timeout
> @@ -468,7 +469,8 @@ def remote_login(command, password, prompt, linesep="\n", timeout=10):
>      @param timeout: The maximal time duration (in seconds) to wait for each
>              step of the login procedure (i.e. the "Are you sure" prompt, the
>              password prompt, the shell prompt, etc)
> -
> +    @prarm prompt_assist: An assistant string sent before the pattern

Typo    ^

> +            matching in order to get the prompt for some kinds of shell_client.
>      @return Return the kvm_spawn object on success and None on failure.
>      """
>      sub = kvm_subprocess.kvm_shell_session(command,
> @@ -479,6 +481,9 @@ def remote_login(command, password, prompt, linesep="\n", timeout=10):
>  
>      logging.debug("Trying to login with command '%s'" % command)
>  
> +    if prompt_assist is not None:
> +        sub.sendline(prompt_assist)
> +
>      while True:
>          (match, text) = sub.read_until_last_line_matches(
>                  [r"[Aa]re you sure", r"[Pp]assword:\s*$", r"^\s*[Ll]ogin:\s*$",
> 
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/9] KVM test: Add the ability to send the username in remote_login()

[Michael Goldish]

On 04/26/2010 01:03 PM, Jason Wang wrote:
> In order to let the serial console work, we must let the
> remote_login() send the username when needed.
> 
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> ---
>  client/tests/kvm/kvm_utils.py |   14 ++++++++++----
>  1 files changed, 10 insertions(+), 4 deletions(-)
> 
> diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py
> index 9adbaee..1ea0852 100644
> --- a/client/tests/kvm/kvm_utils.py
> +++ b/client/tests/kvm/kvm_utils.py
> @@ -452,7 +452,7 @@ def check_kvm_source_dir(source_dir):
>  # guests.
>  
>  def remote_login(command, password, prompt, linesep="\n", timeout=10,
> -                 prompt_assist = None):
> +                 prompt_assist = None, username = None):
                                 ^ ^              ^ ^
Same here: PEP 8 says no spaces around keyword arguments or default
values, and pretty much all of the KVM test code conforms with PEP 8, so
let's be consistent.

Re: [PATCH 8/9] KVM test: Create the background threads before calling process()

[Michael Goldish]

On 04/26/2010 01:04 PM, Jason Wang wrote:
> If the screendump and scrialdump threads are created after the
> process(), we may lose the progress tracking of guest shutting
> down. So this patch creates them before calling process() in preprocess.
> 
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> ---
>  client/tests/kvm/kvm_preprocessing.py |    5 ++---
>  1 files changed, 2 insertions(+), 3 deletions(-)
> 
> diff --git a/client/tests/kvm/kvm_preprocessing.py b/client/tests/kvm/kvm_preprocessing.py
> index 50d0e35..73e835a 100644
> --- a/client/tests/kvm/kvm_preprocessing.py
> +++ b/client/tests/kvm/kvm_preprocessing.py
> @@ -257,9 +257,6 @@ def preprocess(test, params, env):
>                          int(params.get("pre_command_timeout", "600")),
>                          params.get("pre_command_noncritical") == "yes")
>  
> -    # Preprocess all VMs and images
> -    process(test, params, env, preprocess_image, preprocess_vm)
> -
>      # Start the screendump thread
>      if params.get("take_regular_screendumps") == "yes":
>          logging.debug("Starting screendump thread")
> @@ -278,6 +275,8 @@ def preprocess(test, params, env):
>                                                args=(test, params, env))
>          _serialdump_thread.start()
>  
> +    # Preprocess all VMs and images
> +    process(test, params, env, preprocess_image, preprocess_vm)
>  
>  
>  def postprocess(test, params, env):

The initial shutdown procedure is carried out automatically by the
preprocessor in order to prepare the VMs for the current test, and is
not part of the test.  During the procedure VMs from a previous test are
shut down and/or restarted.  I think it'll be confusing (or at least
irrelevant) for the user to see a Fedora guest shutting down at the
beginning of a Windows test.  Also, it will be inconsistent with the
pre_*.ppm screendumps which are taken after the new VMs are started.

Re: [PATCH 7/9] KVM test: Introduce the local_login()

[Michael Goldish]

On 04/26/2010 01:04 PM, Jason Wang wrote:
> This patch introduces a new method which is used to log into the guest
> through the guest serial console. The serial_mode must be set to
> "session" in order to make use of this patch.

In what cases would we want to use this feature?  The serial console is
not supported by all guests and I'm not sure it supports multiple
concurrent sessions (does it?), so it's probably not possible to use it
reliably as a replacement for the regular remote shell servers, or even
as an alternative variant.

> Signed-off-by: Jason Wang <jasowang@redhat.com>
> ---
>  client/tests/kvm/kvm_vm.py |   25 +++++++++++++++++++++++++
>  1 files changed, 25 insertions(+), 0 deletions(-)
> 
> diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
> index 0cdf925..a22893b 100755
> --- a/client/tests/kvm/kvm_vm.py
> +++ b/client/tests/kvm/kvm_vm.py
> @@ -814,7 +814,32 @@ class VM:
>                                                              "command", ""))
>          return session
>  
> +    def local_login(self, timeout=240):
> +        """
> +        Log into the guest via serial console
> +        If timeout expires while waiting for output from the guest (e.g. a
> +        password prompt or a shell prompt) -- fail.
> +        """
> +
> +        serial_mode = self.params.get("serial_mode")
> +        username = self.params.get("username", "")
> +        password = self.params.get("password", "")
> +        prompt = self.params.get("shell_prompt", "[\#\$]")
> +        linesep = eval("'%s'" % self.params.get("shell_linesep", r"\n"))
>  
> +        if serial_mode != "session":
> +            logging.debug("serial_mode is not session")
> +            return None
> +        else:
> +            command = "nc -U %s"  % self.serial_file_name
> +            assist = self.params.get("prompt_assist")
> +            session = kvm_utils.remote_login(command, password, prompt, linesep,
> +                                             timeout, "", username)
                                                         ^
You probably meant to pass the prompt assist string to remote_login()
but instead you're passing "".

> +            if session:
> +                session.set_status_test_command(self.params.get("status_test_"
> +                                                                "command", ""))
> +            return session
> +            
>      def copy_files_to(self, local_path, remote_path, nic_index=0, timeout=300):
>          """
>          Transfer files to the guest.
> 
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 9/9] KVM test: Redirect the console to serial for all linux guests

[Michael Goldish]

On 04/26/2010 01:04 PM, Jason Wang wrote:
> As we have the ability to dump the content from serial console or use
> a session through it, we need to redirect the console to serial
> through unattended files to make use of it.
> 
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> ---
>  client/tests/kvm/unattended/Fedora-10.ks     |    2 +-
>  client/tests/kvm/unattended/Fedora-11.ks     |    2 +-
>  client/tests/kvm/unattended/Fedora-12.ks     |    2 +-
>  client/tests/kvm/unattended/Fedora-8.ks      |    2 +-
>  client/tests/kvm/unattended/Fedora-9.ks      |    2 +-
>  client/tests/kvm/unattended/OpenSUSE-11.xml  |    1 +
>  client/tests/kvm/unattended/RHEL-3-series.ks |    2 +-
>  client/tests/kvm/unattended/RHEL-4-series.ks |    2 +-
>  client/tests/kvm/unattended/RHEL-5-series.ks |    2 +-
>  client/tests/kvm/unattended/SLES-11.xml      |    1 +
>  10 files changed, 10 insertions(+), 8 deletions(-)
> 
> diff --git a/client/tests/kvm/unattended/Fedora-10.ks b/client/tests/kvm/unattended/Fedora-10.ks
> index 61e59d7..8628036 100644
> --- a/client/tests/kvm/unattended/Fedora-10.ks
> +++ b/client/tests/kvm/unattended/Fedora-10.ks
> @@ -11,7 +11,7 @@ firewall --enabled --ssh
>  selinux --enforcing
>  timezone --utc America/New_York
>  firstboot --disable
> -bootloader --location=mbr
> +bootloader --location=mbr --append="console=ttyS0,115200"

I see no reason to completely silence the regular console.  We can let
some output go to the regular console as well as the serial one by using

--append="console=tty0 console=ttyS0"

>  zerombr
>  clearpart --all --initlabel
>  autopart
> diff --git a/client/tests/kvm/unattended/Fedora-11.ks b/client/tests/kvm/unattended/Fedora-11.ks
> index 0be7d06..5ce8ee2 100644
> --- a/client/tests/kvm/unattended/Fedora-11.ks
> +++ b/client/tests/kvm/unattended/Fedora-11.ks
> @@ -10,7 +10,7 @@ firewall --enabled --ssh
>  selinux --enforcing
>  timezone --utc America/New_York
>  firstboot --disable
> -bootloader --location=mbr
> +bootloader --location=mbr --append="console=ttyS0,115200"
>  zerombr
>  
>  clearpart --all --initlabel
> diff --git a/client/tests/kvm/unattended/Fedora-12.ks b/client/tests/kvm/unattended/Fedora-12.ks
> index 0be7d06..5ce8ee2 100644
> --- a/client/tests/kvm/unattended/Fedora-12.ks
> +++ b/client/tests/kvm/unattended/Fedora-12.ks
> @@ -10,7 +10,7 @@ firewall --enabled --ssh
>  selinux --enforcing
>  timezone --utc America/New_York
>  firstboot --disable
> -bootloader --location=mbr
> +bootloader --location=mbr --append="console=ttyS0,115200"
>  zerombr
>  
>  clearpart --all --initlabel
> diff --git a/client/tests/kvm/unattended/Fedora-8.ks b/client/tests/kvm/unattended/Fedora-8.ks
> index f4a872d..884d556 100644
> --- a/client/tests/kvm/unattended/Fedora-8.ks
> +++ b/client/tests/kvm/unattended/Fedora-8.ks
> @@ -11,7 +11,7 @@ firewall --enabled --ssh
>  selinux --enforcing
>  timezone --utc America/New_York
>  firstboot --disable
> -bootloader --location=mbr
> +bootloader --location=mbr --append="console=ttyS0,115200"
>  zerombr
>  clearpart --all --initlabel
>  autopart
> diff --git a/client/tests/kvm/unattended/Fedora-9.ks b/client/tests/kvm/unattended/Fedora-9.ks
> index f4a872d..884d556 100644
> --- a/client/tests/kvm/unattended/Fedora-9.ks
> +++ b/client/tests/kvm/unattended/Fedora-9.ks
> @@ -11,7 +11,7 @@ firewall --enabled --ssh
>  selinux --enforcing
>  timezone --utc America/New_York
>  firstboot --disable
> -bootloader --location=mbr
> +bootloader --location=mbr --append="console=ttyS0,115200"
>  zerombr
>  clearpart --all --initlabel
>  autopart
> diff --git a/client/tests/kvm/unattended/OpenSUSE-11.xml b/client/tests/kvm/unattended/OpenSUSE-11.xml
> index 7dd44fa..2a4fec0 100644
> --- a/client/tests/kvm/unattended/OpenSUSE-11.xml
> +++ b/client/tests/kvm/unattended/OpenSUSE-11.xml
> @@ -50,6 +50,7 @@
>          <module>edd</module>
>        </initrd_module>
>      </initrd_modules>
> +    <append>console=ttyS0,115200</append>
>      <loader_type>grub</loader_type>
>      <sections config:type="list"/>
>    </bootloader>
> diff --git a/client/tests/kvm/unattended/RHEL-3-series.ks b/client/tests/kvm/unattended/RHEL-3-series.ks
> index 884b386..26b1130 100644
> --- a/client/tests/kvm/unattended/RHEL-3-series.ks
> +++ b/client/tests/kvm/unattended/RHEL-3-series.ks
> @@ -10,7 +10,7 @@ rootpw 123456
>  firewall --enabled --ssh
>  timezone America/New_York
>  firstboot --disable
> -bootloader --location=mbr
> +bootloader --location=mbr --append="console=ttyS0,115200"
>  clearpart --all --initlabel
>  autopart
>  reboot
> diff --git a/client/tests/kvm/unattended/RHEL-4-series.ks b/client/tests/kvm/unattended/RHEL-4-series.ks
> index ce4a430..f2f934f 100644
> --- a/client/tests/kvm/unattended/RHEL-4-series.ks
> +++ b/client/tests/kvm/unattended/RHEL-4-series.ks
> @@ -11,7 +11,7 @@ firewall --enabled --ssh
>  selinux --enforcing
>  timezone --utc America/New_York
>  firstboot --disable
> -bootloader --location=mbr
> +bootloader --location=mbr --append="console=ttyS0,115200"
>  zerombr
>  clearpart --all --initlabel
>  autopart
> diff --git a/client/tests/kvm/unattended/RHEL-5-series.ks b/client/tests/kvm/unattended/RHEL-5-series.ks
> index f4a872d..884d556 100644
> --- a/client/tests/kvm/unattended/RHEL-5-series.ks
> +++ b/client/tests/kvm/unattended/RHEL-5-series.ks
> @@ -11,7 +11,7 @@ firewall --enabled --ssh
>  selinux --enforcing
>  timezone --utc America/New_York
>  firstboot --disable
> -bootloader --location=mbr
> +bootloader --location=mbr --append="console=ttyS0,115200"
>  zerombr
>  clearpart --all --initlabel
>  autopart
> diff --git a/client/tests/kvm/unattended/SLES-11.xml b/client/tests/kvm/unattended/SLES-11.xml
> index 93e5685..7656891 100644
> --- a/client/tests/kvm/unattended/SLES-11.xml
> +++ b/client/tests/kvm/unattended/SLES-11.xml
> @@ -49,6 +49,7 @@
>          <module>edd</module>
>        </initrd_module>
>      </initrd_modules>
> +    <append>console=ttyS0,115200</append>
>      <loader_type>grub</loader_type>
>      <sections config:type="list"/>
>    </bootloader>
> 
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 5/9] KVM test: Log the content from guest serial console

[Michael Goldish]

On 04/26/2010 01:04 PM, Jason Wang wrote:
> This patch tries to get the content of guest serial and log it into
> the debug directoy of the testcase through a dedicated thread which is
> created in the preprocessing and ended in the postprocessing. The
> params of serial_mode must be set to "dump" in order to make use of
> this feature.
> 
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> ---
>  client/tests/kvm/kvm_preprocessing.py  |   59 +++++++++++++++++++++++++++++++-
>  client/tests/kvm/tests_base.cfg.sample |    1 +
>  2 files changed, 59 insertions(+), 1 deletions(-)
> 
> diff --git a/client/tests/kvm/kvm_preprocessing.py b/client/tests/kvm/kvm_preprocessing.py
> index 4b9290c..50d0e35 100644
> --- a/client/tests/kvm/kvm_preprocessing.py
> +++ b/client/tests/kvm/kvm_preprocessing.py
> @@ -1,4 +1,5 @@
>  import sys, os, time, commands, re, logging, signal, glob, threading, shutil
> +import socket, select
>  from autotest_lib.client.bin import test, utils
>  from autotest_lib.client.common_lib import error
>  import kvm_vm, kvm_utils, kvm_subprocess, ppm_utils
> @@ -13,7 +14,8 @@ except ImportError:
>  
>  _screendump_thread = None
>  _screendump_thread_termination_event = None
> -
> +_serialdump_thread = None
> +_serialdump_thread_termination_event = None
>  
>  def preprocess_image(test, params):
>      """
> @@ -267,6 +269,16 @@ def preprocess(test, params, env):
>                                                args=(test, params, env))
>          _screendump_thread.start()
>  
> +    # Start the serial dump thread
> +    if params.get("serial_mode") == "dump":
> +        logging.debug("Starting serialdump thread")
> +        global _serialdump_thread, _serialdump_thread_termination_event
> +        _serialdump_thread_termination_event = threading.Event()
> +        _serialdump_thread = threading.Thread(target=_dump_serial_console,
> +                                              args=(test, params, env))
> +        _serialdump_thread.start()
> +
> +
>  
>  def postprocess(test, params, env):
>      """
> @@ -286,6 +298,13 @@ def postprocess(test, params, env):
>          _screendump_thread_termination_event.set()
>          _screendump_thread.join(10)
>  
> +    # Terminate the serialdump thread
> +    global _serialdump_thread, _serialdump_thread_termination_event
> +    if _serialdump_thread:
> +        logging.debug("Terminating serialdump thread...")
> +        _serialdump_thread_termination_event.set()
> +        _serialdump_thread.join(10)
> +
>      # Warn about corrupt PPM files
>      for f in glob.glob(os.path.join(test.debugdir, "*.ppm")):
>          if not ppm_utils.image_verify_ppm_file(f):
> @@ -450,3 +469,41 @@ def _take_screendumps(test, params, env):
>          if _screendump_thread_termination_event.isSet():
>              break
>          _screendump_thread_termination_event.wait(delay)
> +
> +def _dump_serial_console(test, params, env):
> +    global _serialdump_thread_termination_event
> +    rs = []
> +    files = {}
> +
> +    while True:
> +        for vm in kvm_utils.env_get_all_vms(env):
> +            if not files.has_key(vm):

You should probably add "and not vm.is_dead()" to this condition.
Otherwise we'll get lots of "could not connect to serial socket"
messages for dead VMs.

Style note: AFAIK in the Autotest project the form "if not vm in files"
is preferred.

> +                try:
> +                    serial_socket = socket.socket(socket.AF_UNIX,
> +                                                  socket.SOCK_STREAM)
> +                    serial_socket.setblocking(False)
> +                    serial_socket.connect(vm.serial_file_name)
> +                except:
> +                    logging.debug("Could not connect to serial socket for %s" %
> +                                  vm.name)
> +                    continue
> +                rs.append(serial_socket)
> +                serial_dump_filename = os.path.join(test.debugdir,
> +                                                    "serial-%s" % vm.name)
> +                files[vm] = [serial_socket, file(serial_dump_filename, "a+")]
> +
> +        r, w, x = select.select(rs, [], [], 0.5)
> +        for vm in files.keys():
> +            [s ,d] = files[vm]

For consistency, please consider changing this list to a tuple, i.e.
s, d = files[vm] or (s, d) = files[vm].

> +            if s in r:
> +                data = s.recv(16384)
> +                if len(data) == 0:

Style note: AFAIK the preferred form is "if not data".

Sorry for the petty comments.  Overall the patch looks good.

> +                    rs.remove(s)
> +                    files.pop(vm)
> +                else:
> +                    d.write(data)
> +                
> +        if _serialdump_thread_termination_event.isSet():
> +            break
> +
> +            
> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
> index 9f82ffb..169a69e 100644
> --- a/client/tests/kvm/tests_base.cfg.sample
> +++ b/client/tests/kvm/tests_base.cfg.sample
> @@ -13,6 +13,7 @@ start_vm = yes
>  kill_vm = no
>  kill_vm_gracefully = yes
>  kill_unresponsive_vms = yes
> +serial_mode = dump
>  
>  # Screendump specific stuff
>  convert_ppm_files_to_png_on_error = yes
> 
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 7/9] KVM test: Introduce the local_login()

[Amos Kong]

On Wed, Apr 28, 2010 at 03:01:40PM +0300, Michael Goldish wrote:
> On 04/26/2010 01:04 PM, Jason Wang wrote:
> > This patch introduces a new method which is used to log into the guest
> > through the guest serial console. The serial_mode must be set to
> > "session" in order to make use of this patch.
> 
> In what cases would we want to use this feature?  The serial console is
> not supported by all guests and I'm not sure it supports multiple
> concurrent sessions (does it?), so it's probably not possible to use it
> reliably as a replacement for the regular remote shell servers, or even
> as an alternative variant.

We could not get system log by ssh session when network doesn't work(haven't
launched, down, unstable, ...) Using serial console can get more useful info.
Control guest by ssh in some network related testcases isn't credible. It should
be independent.
 

> > Signed-off-by: Jason Wang <jasowang@redhat.com>
> > ---
> >  client/tests/kvm/kvm_vm.py |   25 +++++++++++++++++++++++++
> >  1 files changed, 25 insertions(+), 0 deletions(-)
> > 
> > diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
> > index 0cdf925..a22893b 100755
> > --- a/client/tests/kvm/kvm_vm.py
> > +++ b/client/tests/kvm/kvm_vm.py
> > @@ -814,7 +814,32 @@ class VM:
> >                                                              "command", ""))
> >          return session
> >  
> > +    def local_login(self, timeout=240):
> > +        """
> > +        Log into the guest via serial console
> > +        If timeout expires while waiting for output from the guest (e.g. a
> > +        password prompt or a shell prompt) -- fail.
> > +        """
> > +
> > +        serial_mode = self.params.get("serial_mode")
> > +        username = self.params.get("username", "")
> > +        password = self.params.get("password", "")
> > +        prompt = self.params.get("shell_prompt", "[\#\$]")
> > +        linesep = eval("'%s'" % self.params.get("shell_linesep", r"\n"))
> >  
> > +        if serial_mode != "session":
> > +            logging.debug("serial_mode is not session")
> > +            return None
> > +        else:
> > +            command = "nc -U %s"  % self.serial_file_name
> > +            assist = self.params.get("prompt_assist")
> > +            session = kvm_utils.remote_login(command, password, prompt, linesep,
> > +                                             timeout, "", username)
>                                                          ^
> You probably meant to pass the prompt assist string to remote_login()
> but instead you're passing "".
> 
> > +            if session:
> > +                session.set_status_test_command(self.params.get("status_test_"
> > +                                                                "command", ""))
> > +            return session
> > +            
> >      def copy_files_to(self, local_path, remote_path, nic_index=0, timeout=300):
> >          """
> >          Transfer files to the guest.
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe kvm" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> _______________________________________________
> Autotest mailing list
> Autotest@test.kernel.org
> http://test.kernel.org/cgi-bin/mailman/listinfo/autotest

Re: [Autotest] [PATCH 7/9] KVM test: Introduce the local_login()

[Michael Goldish]

On 04/29/2010 02:44 AM, Amos Kong wrote:
> On Wed, Apr 28, 2010 at 03:01:40PM +0300, Michael Goldish wrote:
>> On 04/26/2010 01:04 PM, Jason Wang wrote:
>>> This patch introduces a new method which is used to log into the guest
>>> through the guest serial console. The serial_mode must be set to
>>> "session" in order to make use of this patch.
>>
>> In what cases would we want to use this feature?  The serial console is
>> not supported by all guests and I'm not sure it supports multiple
>> concurrent sessions (does it?), so it's probably not possible to use it
>> reliably as a replacement for the regular remote shell servers, or even
>> as an alternative variant.
> 
> We could not get system log by ssh session when network doesn't work(haven't
> launched, down, unstable, ...) Using serial console can get more useful info.
> Control guest by ssh in some network related testcases isn't credible. It should
> be independent.

Can you provide a usage example?  Which test is going to use this and
how?  Do you think it should be used in existing tests or in new tests only?

>  
> 
>>> Signed-off-by: Jason Wang <jasowang@redhat.com>
>>> ---
>>>  client/tests/kvm/kvm_vm.py |   25 +++++++++++++++++++++++++
>>>  1 files changed, 25 insertions(+), 0 deletions(-)
>>>
>>> diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
>>> index 0cdf925..a22893b 100755
>>> --- a/client/tests/kvm/kvm_vm.py
>>> +++ b/client/tests/kvm/kvm_vm.py
>>> @@ -814,7 +814,32 @@ class VM:
>>>                                                              "command", ""))
>>>          return session
>>>  
>>> +    def local_login(self, timeout=240):
>>> +        """
>>> +        Log into the guest via serial console
>>> +        If timeout expires while waiting for output from the guest (e.g. a
>>> +        password prompt or a shell prompt) -- fail.
>>> +        """
>>> +
>>> +        serial_mode = self.params.get("serial_mode")
>>> +        username = self.params.get("username", "")
>>> +        password = self.params.get("password", "")
>>> +        prompt = self.params.get("shell_prompt", "[\#\$]")
>>> +        linesep = eval("'%s'" % self.params.get("shell_linesep", r"\n"))
>>>  
>>> +        if serial_mode != "session":
>>> +            logging.debug("serial_mode is not session")
>>> +            return None
>>> +        else:
>>> +            command = "nc -U %s"  % self.serial_file_name
>>> +            assist = self.params.get("prompt_assist")
>>> +            session = kvm_utils.remote_login(command, password, prompt, linesep,
>>> +                                             timeout, "", username)
>>                                                          ^
>> You probably meant to pass the prompt assist string to remote_login()
>> but instead you're passing "".
>>
>>> +            if session:
>>> +                session.set_status_test_command(self.params.get("status_test_"
>>> +                                                                "command", ""))
>>> +            return session
>>> +            
>>>      def copy_files_to(self, local_path, remote_path, nic_index=0, timeout=300):
>>>          """
>>>          Transfer files to the guest.
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>> _______________________________________________
>> Autotest mailing list
>> Autotest@test.kernel.org
>> http://test.kernel.org/cgi-bin/mailman/listinfo/autotest

Re: [PATCH 1/9] KVM test: Introduce the prompt assist

[Jason Wang]

Michael Goldish wrote:
> On 04/26/2010 01:03 PM, Jason Wang wrote:
>   
>> Sometimes we need to send an assist string to a session in order to
>> get the prompt especially when re-connecting to an already logged
>> serial session. This patch send the assist string before doing the
>> pattern matching of remote_login.
>>     
>
> Can you give an example of a prompt assist string, and a typical usage
> example?  What guests require prompt assist strings?
>
>   
It was just used by serial console, consider when the first test case 
have already connected to the serial console, so the second test must 
send something in order to get the prompt string. But it may be better 
to log out during when the session is closed.
>> Signed-off-by: Jason Wang <jasowang@redhat.com>
>> ---
>>  client/tests/kvm/kvm_utils.py |    9 +++++++--
>>  1 files changed, 7 insertions(+), 2 deletions(-)
>>
>> diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py
>> index 25f3c8c..9adbaee 100644
>> --- a/client/tests/kvm/kvm_utils.py
>> +++ b/client/tests/kvm/kvm_utils.py
>> @@ -451,7 +451,8 @@ def check_kvm_source_dir(source_dir):
>>  # The following are functions used for SSH, SCP and Telnet communication with
>>  # guests.
>>  
>> -def remote_login(command, password, prompt, linesep="\n", timeout=10):
>> +def remote_login(command, password, prompt, linesep="\n", timeout=10,
>> +                 prompt_assist = None):
>>     
>                                  ^ ^
> These spaces do not conform with PEP 8.
>   
Would change them.
>   
>>      """
>>      Log into a remote host (guest) using SSH or Telnet. Run the given command
>>      using kvm_spawn and provide answers to the questions asked. If timeout
>> @@ -468,7 +469,8 @@ def remote_login(command, password, prompt, linesep="\n", timeout=10):
>>      @param timeout: The maximal time duration (in seconds) to wait for each
>>              step of the login procedure (i.e. the "Are you sure" prompt, the
>>              password prompt, the shell prompt, etc)
>> -
>> +    @prarm prompt_assist: An assistant string sent before the pattern
>>     
>
> Typo    ^
>
>   
>> +            matching in order to get the prompt for some kinds of shell_client.
>>      @return Return the kvm_spawn object on success and None on failure.
>>      """
>>      sub = kvm_subprocess.kvm_shell_session(command,
>> @@ -479,6 +481,9 @@ def remote_login(command, password, prompt, linesep="\n", timeout=10):
>>  
>>      logging.debug("Trying to login with command '%s'" % command)
>>  
>> +    if prompt_assist is not None:
>> +        sub.sendline(prompt_assist)
>> +
>>      while True:
>>          (match, text) = sub.read_until_last_line_matches(
>>                  [r"[Aa]re you sure", r"[Pp]assword:\s*$", r"^\s*[Ll]ogin:\s*$",
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>     
>
>   

Re: [PATCH 3/9] KVM test: Make the login re suitable for serial console

[Jason Wang]

Michael Goldish wrote:
> On 04/26/2010 01:03 PM, Jason Wang wrote:
>   
>> Current matching re ^\s*[Ll]ogin:\s*$ is not suitable for the serial
>> console, so change it to [Ll]ogin:.
>>
>> Signed-off-by: Jason Wang <jasowang@redhat.com>
>> ---
>>  client/tests/kvm/kvm_utils.py |    2 +-
>>  1 files changed, 1 insertions(+), 1 deletions(-)
>>
>> diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py
>> index 1ea0852..bb42314 100644
>> --- a/client/tests/kvm/kvm_utils.py
>> +++ b/client/tests/kvm/kvm_utils.py
>> @@ -488,7 +488,7 @@ def remote_login(command, password, prompt, linesep="\n", timeout=10,
>>  
>>      while True:
>>          (match, text) = sub.read_until_last_line_matches(
>> -                [r"[Aa]re you sure", r"[Pp]assword:\s*$", r"^\s*[Ll]ogin:\s*$",
>> +                [r"[Aa]re you sure", r"[Pp]assword:\s*$", r"[Ll]ogin:",
>>     
>
> This RE will also match
>
> "Last login: Wed Apr 28 14:07:28 2010 from localhost.localdomain"
>
> so we should probably settle for [Ll]ogin:\s*$.
>
>   
Yes, you are right.
>>                   r"[Cc]onnection.*closed", r"[Cc]onnection.*refused",
>>                   r"[Pp]lease wait", prompt],
>>                   timeout=timeout, internal_timeout=0.5)
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>     
>
>   

Re: [PATCH 5/9] KVM test: Log the content from guest serial console

[Jason Wang]

Michael Goldish wrote:
> On 04/26/2010 01:04 PM, Jason Wang wrote:
>   
>> This patch tries to get the content of guest serial and log it into
>> the debug directoy of the testcase through a dedicated thread which is
>> created in the preprocessing and ended in the postprocessing. The
>> params of serial_mode must be set to "dump" in order to make use of
>> this feature.
>>
>> Signed-off-by: Jason Wang <jasowang@redhat.com>
>> ---
>>  client/tests/kvm/kvm_preprocessing.py  |   59 +++++++++++++++++++++++++++++++-
>>  client/tests/kvm/tests_base.cfg.sample |    1 +
>>  2 files changed, 59 insertions(+), 1 deletions(-)
>>
>> diff --git a/client/tests/kvm/kvm_preprocessing.py b/client/tests/kvm/kvm_preprocessing.py
>> index 4b9290c..50d0e35 100644
>> --- a/client/tests/kvm/kvm_preprocessing.py
>> +++ b/client/tests/kvm/kvm_preprocessing.py
>> @@ -1,4 +1,5 @@
>>  import sys, os, time, commands, re, logging, signal, glob, threading, shutil
>> +import socket, select
>>  from autotest_lib.client.bin import test, utils
>>  from autotest_lib.client.common_lib import error
>>  import kvm_vm, kvm_utils, kvm_subprocess, ppm_utils
>> @@ -13,7 +14,8 @@ except ImportError:
>>  
>>  _screendump_thread = None
>>  _screendump_thread_termination_event = None
>> -
>> +_serialdump_thread = None
>> +_serialdump_thread_termination_event = None
>>  
>>  def preprocess_image(test, params):
>>      """
>> @@ -267,6 +269,16 @@ def preprocess(test, params, env):
>>                                                args=(test, params, env))
>>          _screendump_thread.start()
>>  
>> +    # Start the serial dump thread
>> +    if params.get("serial_mode") == "dump":
>> +        logging.debug("Starting serialdump thread")
>> +        global _serialdump_thread, _serialdump_thread_termination_event
>> +        _serialdump_thread_termination_event = threading.Event()
>> +        _serialdump_thread = threading.Thread(target=_dump_serial_console,
>> +                                              args=(test, params, env))
>> +        _serialdump_thread.start()
>> +
>> +
>>  
>>  def postprocess(test, params, env):
>>      """
>> @@ -286,6 +298,13 @@ def postprocess(test, params, env):
>>          _screendump_thread_termination_event.set()
>>          _screendump_thread.join(10)
>>  
>> +    # Terminate the serialdump thread
>> +    global _serialdump_thread, _serialdump_thread_termination_event
>> +    if _serialdump_thread:
>> +        logging.debug("Terminating serialdump thread...")
>> +        _serialdump_thread_termination_event.set()
>> +        _serialdump_thread.join(10)
>> +
>>      # Warn about corrupt PPM files
>>      for f in glob.glob(os.path.join(test.debugdir, "*.ppm")):
>>          if not ppm_utils.image_verify_ppm_file(f):
>> @@ -450,3 +469,41 @@ def _take_screendumps(test, params, env):
>>          if _screendump_thread_termination_event.isSet():
>>              break
>>          _screendump_thread_termination_event.wait(delay)
>> +
>> +def _dump_serial_console(test, params, env):
>> +    global _serialdump_thread_termination_event
>> +    rs = []
>> +    files = {}
>> +
>> +    while True:
>> +        for vm in kvm_utils.env_get_all_vms(env):
>> +            if not files.has_key(vm):
>>     
>
> You should probably add "and not vm.is_dead()" to this condition.
> Otherwise we'll get lots of "could not connect to serial socket"
> messages for dead VMs.
>   
Nice catch, thanks.
> Style note: AFAIK in the Autotest project the form "if not vm in files"
> is preferred.
>
>   
>> +                try:
>> +                    serial_socket = socket.socket(socket.AF_UNIX,
>> +                                                  socket.SOCK_STREAM)
>> +                    serial_socket.setblocking(False)
>> +                    serial_socket.connect(vm.serial_file_name)
>> +                except:
>> +                    logging.debug("Could not connect to serial socket for %s" %
>> +                                  vm.name)
>> +                    continue
>> +                rs.append(serial_socket)
>> +                serial_dump_filename = os.path.join(test.debugdir,
>> +                                                    "serial-%s" % vm.name)
>> +                files[vm] = [serial_socket, file(serial_dump_filename, "a+")]
>> +
>> +        r, w, x = select.select(rs, [], [], 0.5)
>> +        for vm in files.keys():
>> +            [s ,d] = files[vm]
>>     
>
> For consistency, please consider changing this list to a tuple, i.e.
> s, d = files[vm] or (s, d) = files[vm].
>
>   
Yes, tuple is better.
>> +            if s in r:
>> +                data = s.recv(16384)
>> +                if len(data) == 0:
>>     
>
> Style note: AFAIK the preferred form is "if not data".
>
> Sorry for the petty comments.  Overall the patch looks good.
>   
Thanks for the comments and please do not hesitate to point the defects.
>   
>> +                    rs.remove(s)
>> +                    files.pop(vm)
>> +                else:
>> +                    d.write(data)
>> +                
>> +        if _serialdump_thread_termination_event.isSet():
>> +            break
>> +
>> +            
>> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
>> index 9f82ffb..169a69e 100644
>> --- a/client/tests/kvm/tests_base.cfg.sample
>> +++ b/client/tests/kvm/tests_base.cfg.sample
>> @@ -13,6 +13,7 @@ start_vm = yes
>>  kill_vm = no
>>  kill_vm_gracefully = yes
>>  kill_unresponsive_vms = yes
>> +serial_mode = dump
>>  
>>  # Screendump specific stuff
>>  convert_ppm_files_to_png_on_error = yes
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>     
>
>   

Re: [PATCH 7/9] KVM test: Introduce the local_login()

[Jason Wang]

Michael Goldish wrote:
> On 04/26/2010 01:04 PM, Jason Wang wrote:
>   
>> This patch introduces a new method which is used to log into the guest
>> through the guest serial console. The serial_mode must be set to
>> "session" in order to make use of this patch.
>>     
>
> In what cases would we want to use this feature?  The serial console is
> not supported by all guests and I'm not sure it supports multiple
> concurrent sessions (does it?), so it's probably not possible to use it
> reliably as a replacement for the regular remote shell servers, or even
> as an alternative variant.
>
>   
We plan to push some network related tests in the next few weeks, so 
it's important to use a serial based console rather than network based 
one ( our test s may load/unload the nic driver, do the stress testing 
which may cause the session created by remote_login() not responsible)
>> Signed-off-by: Jason Wang <jasowang@redhat.com>
>> ---
>>  client/tests/kvm/kvm_vm.py |   25 +++++++++++++++++++++++++
>>  1 files changed, 25 insertions(+), 0 deletions(-)
>>
>> diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
>> index 0cdf925..a22893b 100755
>> --- a/client/tests/kvm/kvm_vm.py
>> +++ b/client/tests/kvm/kvm_vm.py
>> @@ -814,7 +814,32 @@ class VM:
>>                                                              "command", ""))
>>          return session
>>  
>> +    def local_login(self, timeout=240):
>> +        """
>> +        Log into the guest via serial console
>> +        If timeout expires while waiting for output from the guest (e.g. a
>> +        password prompt or a shell prompt) -- fail.
>> +        """
>> +
>> +        serial_mode = self.params.get("serial_mode")
>> +        username = self.params.get("username", "")
>> +        password = self.params.get("password", "")
>> +        prompt = self.params.get("shell_prompt", "[\#\$]")
>> +        linesep = eval("'%s'" % self.params.get("shell_linesep", r"\n"))
>>  
>> +        if serial_mode != "session":
>> +            logging.debug("serial_mode is not session")
>> +            return None
>> +        else:
>> +            command = "nc -U %s"  % self.serial_file_name
>> +            assist = self.params.get("prompt_assist")
>> +            session = kvm_utils.remote_login(command, password, prompt, linesep,
>> +                                             timeout, "", username)
>>     
>                                                          ^
> You probably meant to pass the prompt assist string to remote_login()
> but instead you're passing "".
>
>   
As I said, I just send a empty line in order to get the the prompt 
string, but this may be removed if I logout the console during when 
close the session.
>> +            if session:
>> +                session.set_status_test_command(self.params.get("status_test_"
>> +                                                                "command", ""))
>> +            return session
>> +            
>>      def copy_files_to(self, local_path, remote_path, nic_index=0, timeout=300):
>>          """
>>          Transfer files to the guest.
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>     
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>   

Re: [PATCH 9/9] KVM test: Redirect the console to serial for all linux guests

[Jason Wang]

Michael Goldish wrote:
> On 04/26/2010 01:04 PM, Jason Wang wrote:
>   
>> As we have the ability to dump the content from serial console or use
>> a session through it, we need to redirect the console to serial
>> through unattended files to make use of it.
>>
>> Signed-off-by: Jason Wang <jasowang@redhat.com>
>> ---
>>  client/tests/kvm/unattended/Fedora-10.ks     |    2 +-
>>  client/tests/kvm/unattended/Fedora-11.ks     |    2 +-
>>  client/tests/kvm/unattended/Fedora-12.ks     |    2 +-
>>  client/tests/kvm/unattended/Fedora-8.ks      |    2 +-
>>  client/tests/kvm/unattended/Fedora-9.ks      |    2 +-
>>  client/tests/kvm/unattended/OpenSUSE-11.xml  |    1 +
>>  client/tests/kvm/unattended/RHEL-3-series.ks |    2 +-
>>  client/tests/kvm/unattended/RHEL-4-series.ks |    2 +-
>>  client/tests/kvm/unattended/RHEL-5-series.ks |    2 +-
>>  client/tests/kvm/unattended/SLES-11.xml      |    1 +
>>  10 files changed, 10 insertions(+), 8 deletions(-)
>>
>> diff --git a/client/tests/kvm/unattended/Fedora-10.ks b/client/tests/kvm/unattended/Fedora-10.ks
>> index 61e59d7..8628036 100644
>> --- a/client/tests/kvm/unattended/Fedora-10.ks
>> +++ b/client/tests/kvm/unattended/Fedora-10.ks
>> @@ -11,7 +11,7 @@ firewall --enabled --ssh
>>  selinux --enforcing
>>  timezone --utc America/New_York
>>  firstboot --disable
>> -bootloader --location=mbr
>> +bootloader --location=mbr --append="console=ttyS0,115200"
>>     
>
> I see no reason to completely silence the regular console.  We can let
> some output go to the regular console as well as the serial one by using
>
> --append="console=tty0 console=ttyS0"
>
>   
Yes, we should always keep the tty0. Thanks for pointing that.
>>  zerombr
>>  clearpart --all --initlabel
>>  autopart
>> diff --git a/client/tests/kvm/unattended/Fedora-11.ks b/client/tests/kvm/unattended/Fedora-11.ks
>> index 0be7d06..5ce8ee2 100644
>> --- a/client/tests/kvm/unattended/Fedora-11.ks
>> +++ b/client/tests/kvm/unattended/Fedora-11.ks
>> @@ -10,7 +10,7 @@ firewall --enabled --ssh
>>  selinux --enforcing
>>  timezone --utc America/New_York
>>  firstboot --disable
>> -bootloader --location=mbr
>> +bootloader --location=mbr --append="console=ttyS0,115200"
>>  zerombr
>>  
>>  clearpart --all --initlabel
>> diff --git a/client/tests/kvm/unattended/Fedora-12.ks b/client/tests/kvm/unattended/Fedora-12.ks
>> index 0be7d06..5ce8ee2 100644
>> --- a/client/tests/kvm/unattended/Fedora-12.ks
>> +++ b/client/tests/kvm/unattended/Fedora-12.ks
>> @@ -10,7 +10,7 @@ firewall --enabled --ssh
>>  selinux --enforcing
>>  timezone --utc America/New_York
>>  firstboot --disable
>> -bootloader --location=mbr
>> +bootloader --location=mbr --append="console=ttyS0,115200"
>>  zerombr
>>  
>>  clearpart --all --initlabel
>> diff --git a/client/tests/kvm/unattended/Fedora-8.ks b/client/tests/kvm/unattended/Fedora-8.ks
>> index f4a872d..884d556 100644
>> --- a/client/tests/kvm/unattended/Fedora-8.ks
>> +++ b/client/tests/kvm/unattended/Fedora-8.ks
>> @@ -11,7 +11,7 @@ firewall --enabled --ssh
>>  selinux --enforcing
>>  timezone --utc America/New_York
>>  firstboot --disable
>> -bootloader --location=mbr
>> +bootloader --location=mbr --append="console=ttyS0,115200"
>>  zerombr
>>  clearpart --all --initlabel
>>  autopart
>> diff --git a/client/tests/kvm/unattended/Fedora-9.ks b/client/tests/kvm/unattended/Fedora-9.ks
>> index f4a872d..884d556 100644
>> --- a/client/tests/kvm/unattended/Fedora-9.ks
>> +++ b/client/tests/kvm/unattended/Fedora-9.ks
>> @@ -11,7 +11,7 @@ firewall --enabled --ssh
>>  selinux --enforcing
>>  timezone --utc America/New_York
>>  firstboot --disable
>> -bootloader --location=mbr
>> +bootloader --location=mbr --append="console=ttyS0,115200"
>>  zerombr
>>  clearpart --all --initlabel
>>  autopart
>> diff --git a/client/tests/kvm/unattended/OpenSUSE-11.xml b/client/tests/kvm/unattended/OpenSUSE-11.xml
>> index 7dd44fa..2a4fec0 100644
>> --- a/client/tests/kvm/unattended/OpenSUSE-11.xml
>> +++ b/client/tests/kvm/unattended/OpenSUSE-11.xml
>> @@ -50,6 +50,7 @@
>>          <module>edd</module>
>>        </initrd_module>
>>      </initrd_modules>
>> +    <append>console=ttyS0,115200</append>
>>      <loader_type>grub</loader_type>
>>      <sections config:type="list"/>
>>    </bootloader>
>> diff --git a/client/tests/kvm/unattended/RHEL-3-series.ks b/client/tests/kvm/unattended/RHEL-3-series.ks
>> index 884b386..26b1130 100644
>> --- a/client/tests/kvm/unattended/RHEL-3-series.ks
>> +++ b/client/tests/kvm/unattended/RHEL-3-series.ks
>> @@ -10,7 +10,7 @@ rootpw 123456
>>  firewall --enabled --ssh
>>  timezone America/New_York
>>  firstboot --disable
>> -bootloader --location=mbr
>> +bootloader --location=mbr --append="console=ttyS0,115200"
>>  clearpart --all --initlabel
>>  autopart
>>  reboot
>> diff --git a/client/tests/kvm/unattended/RHEL-4-series.ks b/client/tests/kvm/unattended/RHEL-4-series.ks
>> index ce4a430..f2f934f 100644
>> --- a/client/tests/kvm/unattended/RHEL-4-series.ks
>> +++ b/client/tests/kvm/unattended/RHEL-4-series.ks
>> @@ -11,7 +11,7 @@ firewall --enabled --ssh
>>  selinux --enforcing
>>  timezone --utc America/New_York
>>  firstboot --disable
>> -bootloader --location=mbr
>> +bootloader --location=mbr --append="console=ttyS0,115200"
>>  zerombr
>>  clearpart --all --initlabel
>>  autopart
>> diff --git a/client/tests/kvm/unattended/RHEL-5-series.ks b/client/tests/kvm/unattended/RHEL-5-series.ks
>> index f4a872d..884d556 100644
>> --- a/client/tests/kvm/unattended/RHEL-5-series.ks
>> +++ b/client/tests/kvm/unattended/RHEL-5-series.ks
>> @@ -11,7 +11,7 @@ firewall --enabled --ssh
>>  selinux --enforcing
>>  timezone --utc America/New_York
>>  firstboot --disable
>> -bootloader --location=mbr
>> +bootloader --location=mbr --append="console=ttyS0,115200"
>>  zerombr
>>  clearpart --all --initlabel
>>  autopart
>> diff --git a/client/tests/kvm/unattended/SLES-11.xml b/client/tests/kvm/unattended/SLES-11.xml
>> index 93e5685..7656891 100644
>> --- a/client/tests/kvm/unattended/SLES-11.xml
>> +++ b/client/tests/kvm/unattended/SLES-11.xml
>> @@ -49,6 +49,7 @@
>>          <module>edd</module>
>>        </initrd_module>
>>      </initrd_modules>
>> +    <append>console=ttyS0,115200</append>
>>      <loader_type>grub</loader_type>
>>      <sections config:type="list"/>
>>    </bootloader>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>     
>
>   

Re: [Autotest] [PATCH 6/9] KVM test: Raise error when met unknown type in kvm_vm.remote_login().

[Lucas Meneghel Rodrigues]

On Mon, Apr 26, 2010 at 7:04 AM, Jason Wang <jasowang@redhat.com> wrote:
> Need to raise the error when met the unknown type of shell_client in
> kvm_vm.remote_login() in order to avoid the traceback.

In order to keep consistency, please make the function return None
instead of throwing an exception. You might log the message as a
logging.error() record.

> Signed-off-by: Jason Wang <jasowang@redhat.com>
> ---
>  client/tests/kvm/kvm_vm.py |    4 +++-
>  1 files changed, 3 insertions(+), 1 deletions(-)
>
> diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
> index 8f4753f..0cdf925 100755
> --- a/client/tests/kvm/kvm_vm.py
> +++ b/client/tests/kvm/kvm_vm.py
> @@ -806,7 +806,9 @@ class VM:
>         elif client == "nc":
>             session = kvm_utils.netcat(address, port, username, password,
>                                        prompt, linesep, timeout)
> -
> +        else:
> +            raise error.TestError("Unknown shell_client type %s" % client)
> +
>         if session:
>             session.set_status_test_command(self.params.get("status_test_"
>                                                             "command", ""))
>
> _______________________________________________
> Autotest mailing list
> Autotest@test.kernel.org
> http://test.kernel.org/cgi-bin/mailman/listinfo/autotest
>



-- 
Lucas

Re: [PATCH 1/9] KVM test: Introduce the prompt assist

[Lucas Meneghel Rodrigues]

On Thu, 2010-05-06 at 10:55 +0800, Jason Wang wrote:
> Michael Goldish wrote:
> > On 04/26/2010 01:03 PM, Jason Wang wrote:
> >   
> >> Sometimes we need to send an assist string to a session in order to
> >> get the prompt especially when re-connecting to an already logged
> >> serial session. This patch send the assist string before doing the
> >> pattern matching of remote_login.
> >>     
> >
> > Can you give an example of a prompt assist string, and a typical usage
> > example?  What guests require prompt assist strings?
> >
> >   
> It was just used by serial console, consider when the first test case 
> have already connected to the serial console, so the second test must 
> send something in order to get the prompt string. But it may be better 
> to log out during when the session is closed.

This has a upside of making things cleaner, I'd consider that.

> >> Signed-off-by: Jason Wang <jasowang@redhat.com>
> >> ---
> >>  client/tests/kvm/kvm_utils.py |    9 +++++++--
> >>  1 files changed, 7 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py
> >> index 25f3c8c..9adbaee 100644
> >> --- a/client/tests/kvm/kvm_utils.py
> >> +++ b/client/tests/kvm/kvm_utils.py
> >> @@ -451,7 +451,8 @@ def check_kvm_source_dir(source_dir):
> >>  # The following are functions used for SSH, SCP and Telnet communication with
> >>  # guests.
> >>  
> >> -def remote_login(command, password, prompt, linesep="\n", timeout=10):
> >> +def remote_login(command, password, prompt, linesep="\n", timeout=10,
> >> +                 prompt_assist = None):
> >>     
> >                                  ^ ^
> > These spaces do not conform with PEP 8.
> >   
> Would change them.
> >   
> >>      """
> >>      Log into a remote host (guest) using SSH or Telnet. Run the given command
> >>      using kvm_spawn and provide answers to the questions asked. If timeout
> >> @@ -468,7 +469,8 @@ def remote_login(command, password, prompt, linesep="\n", timeout=10):
> >>      @param timeout: The maximal time duration (in seconds) to wait for each
> >>              step of the login procedure (i.e. the "Are you sure" prompt, the
> >>              password prompt, the shell prompt, etc)
> >> -
> >> +    @prarm prompt_assist: An assistant string sent before the pattern
> >>     
> >
> > Typo    ^
> >
> >   
> >> +            matching in order to get the prompt for some kinds of shell_client.
> >>      @return Return the kvm_spawn object on success and None on failure.
> >>      """
> >>      sub = kvm_subprocess.kvm_shell_session(command,
> >> @@ -479,6 +481,9 @@ def remote_login(command, password, prompt, linesep="\n", timeout=10):
> >>  
> >>      logging.debug("Trying to login with command '%s'" % command)
> >>  
> >> +    if prompt_assist is not None:
> >> +        sub.sendline(prompt_assist)
> >> +
> >>      while True:
> >>          (match, text) = sub.read_until_last_line_matches(
> >>                  [r"[Aa]re you sure", r"[Pp]assword:\s*$", r"^\s*[Ll]ogin:\s*$",
> >>
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe kvm" in
> >> the body of a message to majordomo@vger.kernel.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >>     
> >
> >   
> 
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Autotest] [PATCH 8/9] KVM test: Create the background threads before calling process()

[Lucas Meneghel Rodrigues]

On Wed, Apr 28, 2010 at 8:55 AM, Michael Goldish <mgoldish@redhat.com> wrote:
> On 04/26/2010 01:04 PM, Jason Wang wrote:
>> If the screendump and scrialdump threads are created after the
>> process(), we may lose the progress tracking of guest shutting
>> down. So this patch creates them before calling process() in preprocess.
>>
>> Signed-off-by: Jason Wang <jasowang@redhat.com>
>> ---
>>  client/tests/kvm/kvm_preprocessing.py |    5 ++---
>>  1 files changed, 2 insertions(+), 3 deletions(-)
>>
>> diff --git a/client/tests/kvm/kvm_preprocessing.py b/client/tests/kvm/kvm_preprocessing.py
>> index 50d0e35..73e835a 100644
>> --- a/client/tests/kvm/kvm_preprocessing.py
>> +++ b/client/tests/kvm/kvm_preprocessing.py
>> @@ -257,9 +257,6 @@ def preprocess(test, params, env):
>>                          int(params.get("pre_command_timeout", "600")),
>>                          params.get("pre_command_noncritical") == "yes")
>>
>> -    # Preprocess all VMs and images
>> -    process(test, params, env, preprocess_image, preprocess_vm)
>> -
>>      # Start the screendump thread
>>      if params.get("take_regular_screendumps") == "yes":
>>          logging.debug("Starting screendump thread")
>> @@ -278,6 +275,8 @@ def preprocess(test, params, env):
>>                                                args=(test, params, env))
>>          _serialdump_thread.start()
>>
>> +    # Preprocess all VMs and images
>> +    process(test, params, env, preprocess_image, preprocess_vm)
>>
>>
>>  def postprocess(test, params, env):
>
> The initial shutdown procedure is carried out automatically by the
> preprocessor in order to prepare the VMs for the current test, and is
> not part of the test.  During the procedure VMs from a previous test are
> shut down and/or restarted.  I think it'll be confusing (or at least
> irrelevant) for the user to see a Fedora guest shutting down at the
> beginning of a Windows test.  Also, it will be inconsistent with the
> pre_*.ppm screendumps which are taken after the new VMs are started.

Agreed. Besides, if we didn't specify kill_vm for that particular
test, it means that we are not terribly interested on what happens
with the VM if it has to be shutdown (worst case scenario is, the
postprocessor will quit the VM). Of course, it is allways scary to
loose information, but I can't think on why we'd need that info anyway
(correct me if I am wrong).



-- 
Lucas

Re: [PATCH 9/9] KVM test: Redirect the console to serial for all linux guests

[Lucas Meneghel Rodrigues]

On Thu, 2010-05-06 at 11:08 +0800, Jason Wang wrote:
> Michael Goldish wrote:
> > On 04/26/2010 01:04 PM, Jason Wang wrote:
> >   
> >> As we have the ability to dump the content from serial console or use
> >> a session through it, we need to redirect the console to serial
> >> through unattended files to make use of it.
> >>
> >> Signed-off-by: Jason Wang <jasowang@redhat.com>
> >> ---
> >>  client/tests/kvm/unattended/Fedora-10.ks     |    2 +-
> >>  client/tests/kvm/unattended/Fedora-11.ks     |    2 +-
> >>  client/tests/kvm/unattended/Fedora-12.ks     |    2 +-
> >>  client/tests/kvm/unattended/Fedora-8.ks      |    2 +-
> >>  client/tests/kvm/unattended/Fedora-9.ks      |    2 +-
> >>  client/tests/kvm/unattended/OpenSUSE-11.xml  |    1 +
> >>  client/tests/kvm/unattended/RHEL-3-series.ks |    2 +-
> >>  client/tests/kvm/unattended/RHEL-4-series.ks |    2 +-
> >>  client/tests/kvm/unattended/RHEL-5-series.ks |    2 +-
> >>  client/tests/kvm/unattended/SLES-11.xml      |    1 +
> >>  10 files changed, 10 insertions(+), 8 deletions(-)
> >>
> >> diff --git a/client/tests/kvm/unattended/Fedora-10.ks b/client/tests/kvm/unattended/Fedora-10.ks
> >> index 61e59d7..8628036 100644
> >> --- a/client/tests/kvm/unattended/Fedora-10.ks
> >> +++ b/client/tests/kvm/unattended/Fedora-10.ks
> >> @@ -11,7 +11,7 @@ firewall --enabled --ssh
> >>  selinux --enforcing
> >>  timezone --utc America/New_York
> >>  firstboot --disable
> >> -bootloader --location=mbr
> >> +bootloader --location=mbr --append="console=ttyS0,115200"
> >>     
> >
> > I see no reason to completely silence the regular console.  We can let
> > some output go to the regular console as well as the serial one by using
> >
> > --append="console=tty0 console=ttyS0"
> >
> >   
> Yes, we should always keep the tty0. Thanks for pointing that.

It seems like we are opening possibilities on windows land - it seems
that there is an administration console running on windows' serial ports
by default:

http://technet.microsoft.com/en-us/library/cc785873%28WS.10%29.aspx

Need to check that out.

Excellent. Jason, the patchset looks mostly good, please implement the
suggestions pointed out on the comments and re-send it.