[PATCH] KVM test: Create a verify_kernel_crash() VM method

[PATCH] KVM test: Create a verify_kernel_crash() VM method

[Lucas Meneghel Rodrigues]

A method to verify guest kernel panics can be very
useful for a number of tests. Adapted from a function
present on virtio_console test, create VM.verify_kernel_crash()
and use it on unattended_install.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
---
 client/tests/kvm/kvm_vm.py                   |   32 ++++++++++++++++++++++++++
 client/tests/kvm/tests/unattended_install.py |    1 +
 2 files changed, 33 insertions(+), 0 deletions(-)

diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
index 41f7491..ab60f71 100755
--- a/client/tests/kvm/kvm_vm.py
+++ b/client/tests/kvm/kvm_vm.py
@@ -105,6 +105,15 @@ class VMDeadError(VMError):
                 (self.status, self.output))
 
 
+class VMDeadKernelCrashError(VMError):
+    def __init__(self, kernel_crash):
+        VMError.__init__(self, kernel_crash)
+        self.kernel_crash = kernel_crash
+
+    def __str__(self):
+        return ("VM is dead due to a kernel crash: %s" % self.kernel_crash)
+
+
 class VMAddressError(VMError):
     pass
 
@@ -1471,6 +1480,29 @@ class VM:
         return self.serial_login(internal_timeout)
 
 
+    def verify_kernel_crash(self, timeout=2):
+        """
+        Find kernel crash message on serial console.
+
+        @param timeout: Timeout used to verify expected output.
+
+        @raise: VMDeadKernelCrashError, in case a kernel crash message was
+                found.
+        """
+        data = self.serial_console.read_nonblocking()
+        match = re.search("BUG:", data, re.MULTILINE)
+        if match is not None:
+            match = re.search(r"BUG:.*---\[ end trace .* \]---",
+                              data, re.DOTALL |re.MULTILINE)
+            if match is None:
+                data += self.serial_console.read_until_last_line_matches(
+                                               ["---\[ end trace .* \]---"],
+                                               timeout)
+            match = re.search(r"(BUG:.*---\[ end trace .* \]---)",
+                              data, re.DOTALL |re.MULTILINE)
+            raise VMDeadKernelCrashError(match.group(0))
+
+
     @error.context_aware
     def migrate(self, timeout=3600, protocol="tcp", cancel_delay=None,
                 offline=False, stable_check=False, clean=True,
diff --git a/client/tests/kvm/tests/unattended_install.py b/client/tests/kvm/tests/unattended_install.py
index 7c6d845..955f8d6 100644
--- a/client/tests/kvm/tests/unattended_install.py
+++ b/client/tests/kvm/tests/unattended_install.py
@@ -33,6 +33,7 @@ def run_unattended_install(test, params, env):
     start_time = time.time()
     while (time.time() - start_time) < install_timeout:
         vm.verify_alive()
+        vm.verify_kernel_crash()
         client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
         try:
             client.connect((vm.get_address(), port))
-- 
1.7.4

[PATCH] KVM test: Create a verify_kernel_crash() VM method

[Jason Wang]

Lucas Meneghel Rodrigues writes:
 > A method to verify guest kernel panics can be very
 > useful for a number of tests. Adapted from a function
 > present on virtio_console test, create VM.verify_kernel_crash()
 > and use it on unattended_install.
 > 

How about using a thread to monitor the serial console, so other test can also
benefit from this?

 > Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
 > ---
 >  client/tests/kvm/kvm_vm.py                   |   32 ++++++++++++++++++++++++++
 >  client/tests/kvm/tests/unattended_install.py |    1 +
 >  2 files changed, 33 insertions(+), 0 deletions(-)
 > 
 > diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
 > index 41f7491..ab60f71 100755
 > --- a/client/tests/kvm/kvm_vm.py
 > +++ b/client/tests/kvm/kvm_vm.py
 > @@ -105,6 +105,15 @@ class VMDeadError(VMError):
 >                  (self.status, self.output))
 >  
 >  
 > +class VMDeadKernelCrashError(VMError):
 > +    def __init__(self, kernel_crash):
 > +        VMError.__init__(self, kernel_crash)
 > +        self.kernel_crash = kernel_crash
 > +
 > +    def __str__(self):
 > +        return ("VM is dead due to a kernel crash: %s" % self.kernel_crash)
 > +
 > +
 >  class VMAddressError(VMError):
 >      pass
 >  
 > @@ -1471,6 +1480,29 @@ class VM:
 >          return self.serial_login(internal_timeout)
 >  
 >  
 > +    def verify_kernel_crash(self, timeout=2):
 > +        """
 > +        Find kernel crash message on serial console.
 > +
 > +        @param timeout: Timeout used to verify expected output.
 > +
 > +        @raise: VMDeadKernelCrashError, in case a kernel crash message was
 > +                found.
 > +        """
 > +        data = self.serial_console.read_nonblocking()
 > +        match = re.search("BUG:", data, re.MULTILINE)
 > +        if match is not None:
 > +            match = re.search(r"BUG:.*---\[ end trace .* \]---",
 > +                              data, re.DOTALL |re.MULTILINE)
 > +            if match is None:
 > +                data += self.serial_console.read_until_last_line_matches(
 > +                                               ["---\[ end trace .* \]---"],
 > +                                               timeout)
 > +            match = re.search(r"(BUG:.*---\[ end trace .* \]---)",
 > +                              data, re.DOTALL |re.MULTILINE)
 > +            raise VMDeadKernelCrashError(match.group(0))
 > +
 > +
 >      @error.context_aware
 >      def migrate(self, timeout=3600, protocol="tcp", cancel_delay=None,
 >                  offline=False, stable_check=False, clean=True,
 > diff --git a/client/tests/kvm/tests/unattended_install.py b/client/tests/kvm/tests/unattended_install.py
 > index 7c6d845..955f8d6 100644
 > --- a/client/tests/kvm/tests/unattended_install.py
 > +++ b/client/tests/kvm/tests/unattended_install.py
 > @@ -33,6 +33,7 @@ def run_unattended_install(test, params, env):
 >      start_time = time.time()
 >      while (time.time() - start_time) < install_timeout:
 >          vm.verify_alive()
 > +        vm.verify_kernel_crash()
 >          client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
 >          try:
 >              client.connect((vm.get_address(), port))
 > -- 
 > 1.7.4
 > 
 > --
 > To unsubscribe from this list: send the line "unsubscribe kvm" in
 > the body of a message to majordomo@vger.kernel.org
 > More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] KVM test: Create a verify_kernel_crash() VM method

[Michael Goldish]

On 03/02/2011 07:30 AM, Jason Wang wrote:
> Lucas Meneghel Rodrigues writes:
>  > A method to verify guest kernel panics can be very
>  > useful for a number of tests. Adapted from a function
>  > present on virtio_console test, create VM.verify_kernel_crash()
>  > and use it on unattended_install.
>  > 
> 
> How about using a thread to monitor the serial console, so other test can also
> benefit from this?

If an exception is raised in a background thread, it won't kill the main
thread, so it's not obvious how a background thread can help here.  Or
maybe I misunderstood your suggestion?

>  > Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
>  > ---
>  >  client/tests/kvm/kvm_vm.py                   |   32 ++++++++++++++++++++++++++
>  >  client/tests/kvm/tests/unattended_install.py |    1 +
>  >  2 files changed, 33 insertions(+), 0 deletions(-)
>  > 
>  > diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
>  > index 41f7491..ab60f71 100755
>  > --- a/client/tests/kvm/kvm_vm.py
>  > +++ b/client/tests/kvm/kvm_vm.py
>  > @@ -105,6 +105,15 @@ class VMDeadError(VMError):
>  >                  (self.status, self.output))
>  >  
>  >  
>  > +class VMDeadKernelCrashError(VMError):
>  > +    def __init__(self, kernel_crash):
>  > +        VMError.__init__(self, kernel_crash)
>  > +        self.kernel_crash = kernel_crash
>  > +
>  > +    def __str__(self):
>  > +        return ("VM is dead due to a kernel crash: %s" % self.kernel_crash)
>  > +
>  > +
>  >  class VMAddressError(VMError):
>  >      pass
>  >  
>  > @@ -1471,6 +1480,29 @@ class VM:
>  >          return self.serial_login(internal_timeout)
>  >  
>  >  
>  > +    def verify_kernel_crash(self, timeout=2):
>  > +        """
>  > +        Find kernel crash message on serial console.
>  > +
>  > +        @param timeout: Timeout used to verify expected output.
>  > +
>  > +        @raise: VMDeadKernelCrashError, in case a kernel crash message was
>  > +                found.
>  > +        """
>  > +        data = self.serial_console.read_nonblocking()
>  > +        match = re.search("BUG:", data, re.MULTILINE)
>  > +        if match is not None:
>  > +            match = re.search(r"BUG:.*---\[ end trace .* \]---",
>  > +                              data, re.DOTALL |re.MULTILINE)
>  > +            if match is None:
>  > +                data += self.serial_console.read_until_last_line_matches(
>  > +                                               ["---\[ end trace .* \]---"],
>  > +                                               timeout)
>  > +            match = re.search(r"(BUG:.*---\[ end trace .* \]---)",
>  > +                              data, re.DOTALL |re.MULTILINE)
>  > +            raise VMDeadKernelCrashError(match.group(0))
>  > +
>  > +
>  >      @error.context_aware
>  >      def migrate(self, timeout=3600, protocol="tcp", cancel_delay=None,
>  >                  offline=False, stable_check=False, clean=True,
>  > diff --git a/client/tests/kvm/tests/unattended_install.py b/client/tests/kvm/tests/unattended_install.py
>  > index 7c6d845..955f8d6 100644
>  > --- a/client/tests/kvm/tests/unattended_install.py
>  > +++ b/client/tests/kvm/tests/unattended_install.py
>  > @@ -33,6 +33,7 @@ def run_unattended_install(test, params, env):
>  >      start_time = time.time()
>  >      while (time.time() - start_time) < install_timeout:
>  >          vm.verify_alive()
>  > +        vm.verify_kernel_crash()
>  >          client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>  >          try:
>  >              client.connect((vm.get_address(), port))
>  > -- 
>  > 1.7.4
>  > 
>  > --
>  > To unsubscribe from this list: send the line "unsubscribe kvm" in
>  > the body of a message to majordomo@vger.kernel.org
>  > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] KVM test: Create a verify_kernel_crash() VM method

[Michael Goldish]

On 03/02/2011 05:46 AM, Lucas Meneghel Rodrigues wrote:
> A method to verify guest kernel panics can be very
> useful for a number of tests. Adapted from a function
> present on virtio_console test, create VM.verify_kernel_crash()
> and use it on unattended_install.
> 
> Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
> ---
>  client/tests/kvm/kvm_vm.py                   |   32 ++++++++++++++++++++++++++
>  client/tests/kvm/tests/unattended_install.py |    1 +
>  2 files changed, 33 insertions(+), 0 deletions(-)
> 
> diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
> index 41f7491..ab60f71 100755
> --- a/client/tests/kvm/kvm_vm.py
> +++ b/client/tests/kvm/kvm_vm.py
> @@ -105,6 +105,15 @@ class VMDeadError(VMError):
>                  (self.status, self.output))
>  
>  
> +class VMDeadKernelCrashError(VMError):
> +    def __init__(self, kernel_crash):
> +        VMError.__init__(self, kernel_crash)
> +        self.kernel_crash = kernel_crash
> +
> +    def __str__(self):
> +        return ("VM is dead due to a kernel crash: %s" % self.kernel_crash)
> +
> +
>  class VMAddressError(VMError):
>      pass
>  
> @@ -1471,6 +1480,29 @@ class VM:
>          return self.serial_login(internal_timeout)
>  
>  
> +    def verify_kernel_crash(self, timeout=2):
> +        """
> +        Find kernel crash message on serial console.
> +
> +        @param timeout: Timeout used to verify expected output.
> +
> +        @raise: VMDeadKernelCrashError, in case a kernel crash message was
> +                found.
> +        """
> +        data = self.serial_console.read_nonblocking()

I'm not sure using these methdos (read_nonblocking,
read_until_last_line_matches) is safe if verify_kernel_crash() is to be
used in tests that already use the serial console.  If a test uses the
serial console for interactive shell sessions, it might read the BUG:
message unintentionally (while waiting for a shell prompt, for example)
and then verify_kernel_crash() will not read the BUG: message because
data can't be read twice.  It's safer to use
self.serial_console.get_output(), which returns all the output seen so
far from the moment the process was started, including the output
already read by read_until_*().  However, it just returns a snapshot of
the output, so if you happen to call it while a trace is being printed,
you'll get a partial trace.  But if the intended usage for the function
is to be called periodically, that's fine, because the next time it's
called you'll get the full trace.  Does this make sense?

> +        match = re.search("BUG:", data, re.MULTILINE)
> +        if match is not None:
> +            match = re.search(r"BUG:.*---\[ end trace .* \]---",
> +                              data, re.DOTALL |re.MULTILINE)
> +            if match is None:
> +                data += self.serial_console.read_until_last_line_matches(
> +                                               ["---\[ end trace .* \]---"],
> +                                               timeout)
> +            match = re.search(r"(BUG:.*---\[ end trace .* \]---)",
> +                              data, re.DOTALL |re.MULTILINE)
> +            raise VMDeadKernelCrashError(match.group(0))
> +
> +
>      @error.context_aware
>      def migrate(self, timeout=3600, protocol="tcp", cancel_delay=None,
>                  offline=False, stable_check=False, clean=True,
> diff --git a/client/tests/kvm/tests/unattended_install.py b/client/tests/kvm/tests/unattended_install.py
> index 7c6d845..955f8d6 100644
> --- a/client/tests/kvm/tests/unattended_install.py
> +++ b/client/tests/kvm/tests/unattended_install.py
> @@ -33,6 +33,7 @@ def run_unattended_install(test, params, env):
>      start_time = time.time()
>      while (time.time() - start_time) < install_timeout:
>          vm.verify_alive()
> +        vm.verify_kernel_crash()
>          client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>          try:
>              client.connect((vm.get_address(), port))

Re: [PATCH] KVM test: Create a verify_kernel_crash() VM method

[Lucas Meneghel Rodrigues]

On Wed, 2011-03-02 at 13:05 +0200, Michael Goldish wrote:
> On 03/02/2011 05:46 AM, Lucas Meneghel Rodrigues wrote:
> > +    def verify_kernel_crash(self, timeout=2):
> > +        """
> > +        Find kernel crash message on serial console.
> > +
> > +        @param timeout: Timeout used to verify expected output.
> > +
> > +        @raise: VMDeadKernelCrashError, in case a kernel crash message was
> > +                found.
> > +        """
> > +        data = self.serial_console.read_nonblocking()
> 
> I'm not sure using these methdos (read_nonblocking,
> read_until_last_line_matches) is safe if verify_kernel_crash() is to be
> used in tests that already use the serial console.  If a test uses the
> serial console for interactive shell sessions, it might read the BUG:
> message unintentionally (while waiting for a shell prompt, for example)
> and then verify_kernel_crash() will not read the BUG: message because
> data can't be read twice.  It's safer to use
> self.serial_console.get_output(), which returns all the output seen so
> far from the moment the process was started, including the output
> already read by read_until_*().  However, it just returns a snapshot of
> the output, so if you happen to call it while a trace is being printed,
> you'll get a partial trace.  But if the intended usage for the function
> is to be called periodically, that's fine, because the next time it's
> called you'll get the full trace.  Does this make sense?
> 

It does, certainly. It's a matter of changing the code to use
get_output() then, thanks for the suggestion, awesome!

Lucas