[KVM-AUTOTEST PATCH 0/8] Re-submitting some of the patches on the patch queue

[KVM-AUTOTEST PATCH 0/8] Re-submitting some of the patches on the patch queue

[Lucas Meneghel Rodrigues]

I have rebased some of the patches on the patch queue I sent earlier to the
list. I am sending them to your appreciation, the other will follow.

[KVM-AUTOTEST PATCH 1/8] kvm_config: Allow for "=" in the value of a config parameter

[Lucas Meneghel Rodrigues]

fix modifies kvm_config.split_and_strip so it will only split once per
line.

example: kernel_args = "ks=floppy console=ttyS0 noacpi"

Signed-off-by: David Huff <dhuff@redhat.com>
---
 client/tests/kvm/kvm_config.py |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/client/tests/kvm/kvm_config.py b/client/tests/kvm/kvm_config.py
index 8b6ab15..40f16f1 100755
--- a/client/tests/kvm/kvm_config.py
+++ b/client/tests/kvm/kvm_config.py
@@ -136,7 +136,7 @@ class config:
             @param str: String that will be processed
             @param sep: Separator that will be used to split the string
         """
-        temp = str.split(sep)
+        temp = str.split(sep, 1)
         for i in range(len(temp)):
             temp[i] = temp[i].strip()
             temp[i] = temp[i].strip("\"\'")
-- 
1.6.2.2

[KVM-AUTOTEST PATCH 2/8] RHEL-4.7 step files: fix the initial boot barriers

[Lucas Meneghel Rodrigues]

Fix the first barrier to include the 'boot:' prompt. This is crucial
because the guest accepts keyboard input only after this prompt
appears.

Visibility: Small (Data file on the kvm test)
Risk: Low (Small, tested change to a data file)

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/steps/RHEL-4.7-i386.steps   |    2 +-
 client/tests/kvm/steps/RHEL-4.7-x86_64.steps |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/client/tests/kvm/steps/RHEL-4.7-i386.steps b/client/tests/kvm/steps/RHEL-4.7-i386.steps
index 3816c15..763d473 100644
--- a/client/tests/kvm/steps/RHEL-4.7-i386.steps
+++ b/client/tests/kvm/steps/RHEL-4.7-i386.steps
@@ -4,7 +4,7 @@
 # --------------------------------
 step unknown
 screendump 20090413_013526_868fe81019ae64a0b066c4c0d4ebc4e1.ppm
-barrier_2 188 30 354 265 48ef114b5a42ba0d5bebfaee47dce498 50
+barrier_2 44 38 0 363 d9ca61811a10b33cc95515d4796541e7 50
 # Sending keys: ret
 key ret
 # --------------------------------
diff --git a/client/tests/kvm/steps/RHEL-4.7-x86_64.steps b/client/tests/kvm/steps/RHEL-4.7-x86_64.steps
index 644446f..36f0109 100644
--- a/client/tests/kvm/steps/RHEL-4.7-x86_64.steps
+++ b/client/tests/kvm/steps/RHEL-4.7-x86_64.steps
@@ -6,7 +6,7 @@ step 8.84
 screendump 20080101_000001_868fe81019ae64a0b066c4c0d4ebc4e1.ppm
 # boot options
 sleep 5
-barrier_2 194 59 101 59 8c4f6b29e4087e1bed13c40e4a6a904f 44 optional
+barrier_2 44 36 0 365 ea4c08daabe1f982b243fce9c5b542a0 44 optional
 # Sending keys: ret
 key ret
 # --------------------------------
-- 
1.6.2.2

[KVM-AUTOTEST PATCH 3/8] WinXP step file fixes

[Lucas Meneghel Rodrigues]

Add an optional barrier to deal with a closed start menu.

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/steps/WinXP-32-setupssh.steps |   10 ++++++++--
 client/tests/kvm/steps/WinXP-32.steps          |    4 +++-
 client/tests/kvm/steps/WinXP-64.steps          |    3 +--
 3 files changed, 12 insertions(+), 5 deletions(-)

diff --git a/client/tests/kvm/steps/WinXP-32-setupssh.steps b/client/tests/kvm/steps/WinXP-32-setupssh.steps
index 729d9df..ebb665f 100644
--- a/client/tests/kvm/steps/WinXP-32-setupssh.steps
+++ b/client/tests/kvm/steps/WinXP-32-setupssh.steps
@@ -4,8 +4,14 @@
 # --------------------------------
 step 24.72
 screendump 20080101_000001_5965948293222a6d6f3e545db40c23c1.ppm
-# open start menu
-barrier_2 125 79 342 270 368b3d82c870dbcdc4dfc2a49660e798 124
+# desktop reached
+barrier_2 36 32 392 292 3828d3a9587b3a9766a567a2b7570e42 124
+# --------------------------------
+step 24.72
+screendump 20080101_000001_5965948293222a6d6f3e545db40c23c1.ppm
+# open start menu if not already open
+sleep 10
+barrier_2 84 48 0 552 082462ce890968a264b9b13cddda8ae3 10 optional
 # Sending keys: ctrl-esc
 key ctrl-esc
 # --------------------------------
diff --git a/client/tests/kvm/steps/WinXP-32.steps b/client/tests/kvm/steps/WinXP-32.steps
index b0c6e35..f52fd0e 100644
--- a/client/tests/kvm/steps/WinXP-32.steps
+++ b/client/tests/kvm/steps/WinXP-32.steps
@@ -136,7 +136,8 @@ key alt-n
 step 2251.56
 screendump 20080101_000022_dcdc2fe9606c044ce648422afe42e23d.ppm
 # User
-barrier_2 409 35 64 188 3d71d4d7a9364c1e6415b3d554ce6e5b 9
+barrier_2 161 37 312 187 a941ecbeb73f9d73e3e9c38da9a4b743 9
+# Sending keys: $user alt-n
 var user
 key alt-n
 # --------------------------------
@@ -154,6 +155,7 @@ barrier_2 48 51 391 288 bbac8a522510d7c8d6e515f6a3fbd4c3 240
 step 2279.61
 screendump 20090416_150641_b72ad5c48ec2dbc9814d569e38cbb4cc.ppm
 # Win XP Start Menu (closed)
+sleep 20
 barrier_2 104 41 0 559 a7cc02cecff2cb495f300aefbb99d9ae 5 optional
 # Sending keys: ctrl-esc
 key ctrl-esc
diff --git a/client/tests/kvm/steps/WinXP-64.steps b/client/tests/kvm/steps/WinXP-64.steps
index 20bac81..91e6d0f 100644
--- a/client/tests/kvm/steps/WinXP-64.steps
+++ b/client/tests/kvm/steps/WinXP-64.steps
@@ -74,7 +74,6 @@ key ret
 # --------------------------------
 step 286.86
 screendump 20080101_000010_bb878343930f948c0346f103a387157a.ppm
-barrier_2 69 15 179 8 93889bdbe5351e61a6d9c7d00bb1c971 10
 # --------------------------------
 step 409.46
 screendump 20080101_000011_30db9777a7883a07e6e65bff74e1d98f.ppm
@@ -100,7 +99,7 @@ key 0xdc
 step 978.02
 screendump 20080101_000014_213fbe6fa13bf32dfac6a00bf4205e45.ppm
 # Windows XP Start Menu Opened
-barrier_2 48 20 274 420 c4a9620d84508013050e5a37a0d9e4ef 15
+barrier_2 129 30 196 72 aae68af7e05e2312c707f2f4bd73f024 15
 # Sending keys: u
 key u
 # --------------------------------
-- 
1.6.2.2

[KVM-AUTOTEST PATCH 4/8] RHEL 5.3 step file fixes

[Lucas Meneghel Rodrigues]

Fix the initial boot barriers to include the 'boot:' prompt.
Also fix a barrier in 5.3 of a dialog that sometimes appears at an
unexpected location.

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/steps/RHEL-5.3-i386.steps   |   12 ++++++++++--
 client/tests/kvm/steps/RHEL-5.3-x86_64.steps |    2 +-
 2 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/client/tests/kvm/steps/RHEL-5.3-i386.steps b/client/tests/kvm/steps/RHEL-5.3-i386.steps
index 86066d8..0964f47 100644
--- a/client/tests/kvm/steps/RHEL-5.3-i386.steps
+++ b/client/tests/kvm/steps/RHEL-5.3-i386.steps
@@ -7,7 +7,7 @@ step 10.33
 screendump 20080101_000001_8333460cfad39ef04d6dbbf7d35fdcba.ppm
 # boot options
 sleep 10
-barrier_2 459 65 5 223 a817e977c049abaaab9a391d3cbeb1ab 52
+barrier_2 44 40 0 410 2f6c4cea4cf5b03bec757893e4982897 52
 # Sending keys: ret
 key ret
 # --------------------------------
@@ -187,10 +187,18 @@ key alt-f
 step 1518.52
 screendump 20080101_000024_3e641443dd4ea558467e517a4be68517.ppm
 # confirm no rhn
-barrier_2 97 18 146 77 a6a767b46d079c6879ebd5aec00cda46 43
+barrier_2 97 18 146 77 a6a767b46d079c6879ebd5aec00cda46 15 optional
 # Sending keys: alt-n
 key alt-n
 # --------------------------------
+step 1518.52
+screendump 20090506_091708_e5436357421502fccb3df89a830fd2f4.ppm
+# confirm no rhn (2)
+barrier_2 60 37 59 15 d94ff141696970e06545ce9854306970 5 optional
+# Sending keys: alt-tab alt-n
+key alt-tab
+key alt-n
+# --------------------------------
 step 1526.55
 screendump 20080101_000025_e10ec4a12e28baa41af798cbdbf308a1.ppm
 # finish update setup
diff --git a/client/tests/kvm/steps/RHEL-5.3-x86_64.steps b/client/tests/kvm/steps/RHEL-5.3-x86_64.steps
index 96ba87e..fc0db20 100644
--- a/client/tests/kvm/steps/RHEL-5.3-x86_64.steps
+++ b/client/tests/kvm/steps/RHEL-5.3-x86_64.steps
@@ -7,7 +7,7 @@ step 9.22
 screendump 20080101_000001_8333460cfad39ef04d6dbbf7d35fdcba.ppm
 # boot options
 sleep 10
-barrier_2 441 88 18 213 881883a4fa307a918f40804c70b9880d 46
+barrier_2 43 39 0 411 3c4a56d04a8ea1c0c5c1edfe10d472a0 46
 # Sending keys: ret
 key ret
 # --------------------------------
-- 
1.6.2.2

[KVM-AUTOTEST PATCH 5/8] stepeditor.py: get rid of some shortcuts

[Lucas Meneghel Rodrigues]

Disable a few keyboard shortcuts that were initially assumed to be useful,
but apparently override the default functionality of the involved keys
(e.g. 'home', 'end', 'delete') regardless of the widget that has the keyboard
focus.

Also make some indentation changes to the UI description.

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/stepeditor.py |   63 +++++++++++++++++++---------------------
 1 files changed, 30 insertions(+), 33 deletions(-)

diff --git a/client/tests/kvm/stepeditor.py b/client/tests/kvm/stepeditor.py
index f2ef1aa..9669200 100755
--- a/client/tests/kvm/stepeditor.py
+++ b/client/tests/kvm/stepeditor.py
@@ -858,27 +858,27 @@ class StepMakerWindow:
 class StepEditor(StepMakerWindow):
     ui = '''<ui>
     <menubar name="MenuBar">
-      <menu action="File">
-        <menuitem action="Open"/>
-        <separator/>
-        <menuitem action="Quit"/>
-      </menu>
-      <menu action="Edit">
-        <menuitem action="CopyStep"/>
-        <menuitem action="DeleteStep"/>
-      </menu>
-      <menu action="Insert">
-        <menuitem action="InsertNewBefore"/>
-        <menuitem action="InsertNewAfter"/>
-        <separator/>
-        <menuitem action="InsertStepsBefore"/>
-        <menuitem action="InsertStepsAfter"/>
-      </menu>
-      <menu action="Tools">
-        <menuitem action="CleanUp"/>
-      </menu>
+        <menu action="File">
+            <menuitem action="Open"/>
+            <separator/>
+            <menuitem action="Quit"/>
+        </menu>
+        <menu action="Edit">
+            <menuitem action="CopyStep"/>
+            <menuitem action="DeleteStep"/>
+        </menu>
+        <menu action="Insert">
+            <menuitem action="InsertNewBefore"/>
+            <menuitem action="InsertNewAfter"/>
+            <separator/>
+            <menuitem action="InsertStepsBefore"/>
+            <menuitem action="InsertStepsAfter"/>
+        </menu>
+        <menu action="Tools">
+            <menuitem action="CleanUp"/>
+        </menu>
     </menubar>
-    </ui>'''
+</ui>'''
 
     # Constructor
 
@@ -896,7 +896,7 @@ class StepEditor(StepMakerWindow):
         self.window.add_accel_group(accelgroup)
 
         # Create an ActionGroup
-        actiongroup = gtk.ActionGroup('UIManagerExample')
+        actiongroup = gtk.ActionGroup('StepEditor')
 
         # Create actions
         actiongroup.add_actions([
@@ -904,22 +904,22 @@ class StepEditor(StepMakerWindow):
              self.quit),
             ('Open', gtk.STOCK_OPEN, '_Open', None, 'Open steps file',
              self.open_steps_file),
-            ('CopyStep', gtk.STOCK_COPY, '_Copy current step...', None,
+            ('CopyStep', gtk.STOCK_COPY, '_Copy current step...', "",
              'Copy current step to user specified position', self.copy_step),
-            ('DeleteStep', gtk.STOCK_DELETE, '_Delete current step', None,
+            ('DeleteStep', gtk.STOCK_DELETE, '_Delete current step', "",
              'Delete current step', self.event_remove_clicked),
-            ('InsertNewBefore', gtk.STOCK_ADD, '_New step before current', None,
+            ('InsertNewBefore', gtk.STOCK_ADD, '_New step before current', "",
              'Insert new step before current step', self.insert_before),
-            ('InsertNewAfter', gtk.STOCK_ADD, 'N_ew step after current', None,
+            ('InsertNewAfter', gtk.STOCK_ADD, 'N_ew step after current', "",
              'Insert new step after current step', self.insert_after),
             ('InsertStepsBefore', gtk.STOCK_ADD, '_Steps before current...',
-             None, 'Insert steps (from file) before current step',
+             "", 'Insert steps (from file) before current step',
              self.insert_steps_before),
-            ('InsertStepsAfter', gtk.STOCK_ADD, 'Steps _after current...',
-             None, 'Insert steps (from file) after current step',
+            ('InsertStepsAfter', gtk.STOCK_ADD, 'Steps _after current...', "",
+             'Insert steps (from file) after current step',
              self.insert_steps_after),
-            ('CleanUp', gtk.STOCK_DELETE, '_Clean up data directory',
-             None, 'Move unused PPM files to a backup directory', self.cleanup),
+            ('CleanUp', gtk.STOCK_DELETE, '_Clean up data directory', "",
+             'Move unused PPM files to a backup directory', self.cleanup),
             ('File', None, '_File'),
             ('Edit', None, '_Edit'),
             ('Insert', None, '_Insert'),
@@ -939,9 +939,6 @@ class StepEditor(StepMakerWindow):
 
         create_shortcut("Next", self.event_next_clicked, "Page_Down")
         create_shortcut("Previous", self.event_prev_clicked, "Page_Up")
-        create_shortcut("First", self.event_first_clicked, "Home")
-        create_shortcut("Last", self.event_last_clicked, "End")
-        create_shortcut("Delete", self.event_remove_clicked, "Delete")
 
         # Add the actiongroup to the uimanager
         uimanager.insert_action_group(actiongroup, 0)
-- 
1.6.2.2

[KVM-AUTOTEST PATCH 6/8] Choose a monitor filename in the constructor of VM class

[Lucas Meneghel Rodrigues]

Choose a monitor filename in the VM class constructor instead of the
VM.create() method. This will reduce the number of monitor files left
in /tmp, because the constructor is called fewer times than VM.create().

Risk: Low (comprehensible change, just moving a block of code).
Visibility: Small (users of kvm test).

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm_vm.py |   21 +++++++++++----------
 1 files changed, 11 insertions(+), 10 deletions(-)

diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
index 76b0251..3001648 100644
--- a/client/tests/kvm/kvm_vm.py
+++ b/client/tests/kvm/kvm_vm.py
@@ -115,6 +115,17 @@ class VM:
         self.iso_dir = iso_dir
 
 
+        # Find available monitor filename
+        while True:
+            # The monitor filename should be unique
+            self.instance = time.strftime("%Y%m%d-%H%M%S-") + \
+            kvm_utils.generate_random_string(4)
+            self.monitor_file_name = os.path.join("/tmp",
+                                                  "monitor-" + self.instance)
+            if not os.path.exists(self.monitor_file_name):
+                break
+
+
     def verify_process_identity(self):
         """
         Make sure .pid really points to the original qemu process. If .pid
@@ -297,16 +308,6 @@ class VM:
                     logging.error("Actual MD5 sum differs from expected one")
                     return False
 
-        # Find available monitor filename
-        while True:
-            # The monitor filename should be unique
-            self.instance = time.strftime("%Y%m%d-%H%M%S-") + \
-            kvm_utils.generate_random_string(4)
-            self.monitor_file_name = os.path.join("/tmp",
-                                                  "monitor-" + self.instance)
-            if not os.path.exists(self.monitor_file_name):
-                break
-
         # Handle port redirections
         redir_names = kvm_utils.get_sub_dict_names(params, "redirs")
         host_ports = kvm_utils.find_free_ports(5000, 6000, len(redir_names))
-- 
1.6.2.2

[KVM-AUTOTEST PATCH 7/8] Add new function, VM.clone() to clone an existing VM

[Lucas Meneghel Rodrigues]

This method will be useful when writing functional tests.

Risk: Low (new method of the kvm test API)
Visibility: Low (kvm test developers will have an additional
API to use)

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm_vm.py |   19 +++++++++++++++++++
 1 files changed, 19 insertions(+), 0 deletions(-)

diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
index 3001648..b81c0df 100644
--- a/client/tests/kvm/kvm_vm.py
+++ b/client/tests/kvm/kvm_vm.py
@@ -126,6 +126,25 @@ class VM:
                 break
 
 
+    def clone(self, name=None, params=None, qemu_path=None, image_dir=None, iso_dir=None):
+        """Return a clone of the VM object with optionally modified parameters.
+
+        The clone is initially not alive and needs to be started using create().
+        Any parameters not passed to this function are copied from the source VM.
+        """
+        if name == None:
+            name = self.name
+        if params == None:
+            params = self.params.copy()
+        if qemu_path == None:
+            qemu_path = self.qemu_path
+        if image_dir == None:
+            image_dir = self.image_dir
+        if iso_dir == None:
+            iso_dir = self.iso_dir
+        return VM(name, params, qemu_path, image_dir, iso_dir)
+
+
     def verify_process_identity(self):
         """
         Make sure .pid really points to the original qemu process. If .pid
-- 
1.6.2.2

[KVM-AUTOTEST PATCH 8/8] kvm_runtest_2.py: use pickle instead of shelve when loading/saving env

[Lucas Meneghel Rodrigues]

pickle allows more control over the load/save process. Specifically, it
enables us to dump the contents of an object to disk without having to
unpickle it.

shelve, which uses pickle, seems to pickle and unpickle every time sync()
is called. This is bad for classes that need to be unpickled only once
per test (such a class will be introduced in a future patch).

Signed-off-by: Michael Goldish <mgoldish@redhat.com>
---
 client/tests/kvm/kvm.py |   35 +++++++++++++++++++++++++----------
 1 files changed, 25 insertions(+), 10 deletions(-)

diff --git a/client/tests/kvm/kvm.py b/client/tests/kvm/kvm.py
index 1b9013c..a658425 100644
--- a/client/tests/kvm/kvm.py
+++ b/client/tests/kvm/kvm.py
@@ -1,4 +1,4 @@
-import sys, os, time, shelve, random, resource, logging
+import sys, os, time, shelve, random, resource, logging, cPickle
 from autotest_lib.client.bin import test
 from autotest_lib.client.common_lib import error
 
@@ -10,6 +10,22 @@ class test_routine:
         self.routine = None
 
 
+def dump_env(obj, filename):
+    file = open(filename, "w")
+    cPickle.dump(obj, file)
+    file.close()
+
+
+def load_env(filename, default=None):
+    try:
+        file = open(filename, "r")
+    except:
+        return default
+    obj = cPickle.load(file)
+    file.close()
+    return obj
+
+
 class kvm(test.test):
     """
     Suite of KVM virtualization functional tests.
@@ -62,12 +78,12 @@ class kvm(test.test):
         keys = params.keys()
         keys.sort()
         for key in keys:
-            logging.debug("    %s = %s" % (key, params[key]))
+            logging.debug("    %s = %s", key, params[key])
             self.write_test_keyval({key: params[key]})
 
         # Open the environment file
         env_filename = os.path.join(self.bindir, "env")
-        env = shelve.open(env_filename, writeback=True)
+        env = load_env(env_filename, {})
         logging.debug("Contents of environment: %s" % str(env))
 
         try:
@@ -90,21 +106,20 @@ class kvm(test.test):
 
                 # Preprocess
                 kvm_preprocessing.preprocess(self, params, env)
-                env.sync()
+                dump_env(env, env_filename)
                 # Run the test function
                 routine_obj.routine(self, params, env)
-                env.sync()
+                dump_env(env, env_filename)
 
             except Exception, e:
-                logging.error("Test failed: %s" % e)
+                logging.error("Test failed: %s", e)
                 logging.debug("Postprocessing on error...")
                 kvm_preprocessing.postprocess_on_error(self, params, env)
-                env.sync()
+                dump_env(env, env_filename)
                 raise
 
         finally:
             # Postprocess
             kvm_preprocessing.postprocess(self, params, env)
-            logging.debug("Contents of environment: %s" % str(env))
-            env.sync()
-            env.close()
+            logging.debug("Contents of environment: %s", str(env))
+            dump_env(env, env_filename)
-- 
1.6.2.2

Re: [KVM-AUTOTEST PATCH 8/8] kvm_runtest_2.py: use pickle instead of shelve when loading/saving env

[Lucas Meneghel Rodrigues]

On Fri, 2009-06-05 at 17:46 -0300, Lucas Meneghel Rodrigues wrote:
> pickle allows more control over the load/save process. Specifically, it
> enables us to dump the contents of an object to disk without having to
> unpickle it.
> 
> shelve, which uses pickle, seems to pickle and unpickle every time sync()
> is called. This is bad for classes that need to be unpickled only once
> per test (such a class will be introduced in a future patch).

Applied.

> Signed-off-by: Michael Goldish <mgoldish@redhat.com>
> ---
>  client/tests/kvm/kvm.py |   35 +++++++++++++++++++++++++----------
>  1 files changed, 25 insertions(+), 10 deletions(-)
> 
> diff --git a/client/tests/kvm/kvm.py b/client/tests/kvm/kvm.py
> index 1b9013c..a658425 100644
> --- a/client/tests/kvm/kvm.py
> +++ b/client/tests/kvm/kvm.py
> @@ -1,4 +1,4 @@
> -import sys, os, time, shelve, random, resource, logging
> +import sys, os, time, shelve, random, resource, logging, cPickle
>  from autotest_lib.client.bin import test
>  from autotest_lib.client.common_lib import error
>  
> @@ -10,6 +10,22 @@ class test_routine:
>          self.routine = None
>  
> 
> +def dump_env(obj, filename):
> +    file = open(filename, "w")
> +    cPickle.dump(obj, file)
> +    file.close()
> +
> +
> +def load_env(filename, default=None):
> +    try:
> +        file = open(filename, "r")
> +    except:
> +        return default
> +    obj = cPickle.load(file)
> +    file.close()
> +    return obj
> +
> +
>  class kvm(test.test):
>      """
>      Suite of KVM virtualization functional tests.
> @@ -62,12 +78,12 @@ class kvm(test.test):
>          keys = params.keys()
>          keys.sort()
>          for key in keys:
> -            logging.debug("    %s = %s" % (key, params[key]))
> +            logging.debug("    %s = %s", key, params[key])
>              self.write_test_keyval({key: params[key]})
>  
>          # Open the environment file
>          env_filename = os.path.join(self.bindir, "env")
> -        env = shelve.open(env_filename, writeback=True)
> +        env = load_env(env_filename, {})
>          logging.debug("Contents of environment: %s" % str(env))
>  
>          try:
> @@ -90,21 +106,20 @@ class kvm(test.test):
>  
>                  # Preprocess
>                  kvm_preprocessing.preprocess(self, params, env)
> -                env.sync()
> +                dump_env(env, env_filename)
>                  # Run the test function
>                  routine_obj.routine(self, params, env)
> -                env.sync()
> +                dump_env(env, env_filename)
>  
>              except Exception, e:
> -                logging.error("Test failed: %s" % e)
> +                logging.error("Test failed: %s", e)
>                  logging.debug("Postprocessing on error...")
>                  kvm_preprocessing.postprocess_on_error(self, params, env)
> -                env.sync()
> +                dump_env(env, env_filename)
>                  raise
>  
>          finally:
>              # Postprocess
>              kvm_preprocessing.postprocess(self, params, env)
> -            logging.debug("Contents of environment: %s" % str(env))
> -            env.sync()
> -            env.close()
> +            logging.debug("Contents of environment: %s", str(env))
> +            dump_env(env, env_filename)
-- 
Lucas Meneghel Rodrigues
Software Engineer (QE)
Red Hat - Emerging Technologies

Re: [Autotest] [KVM-AUTOTEST PATCH 8/8] kvm_runtest_2.py: use pickle instead of shelve when loading/saving env

[Martin Bligh]

On Fri, Jun 5, 2009 at 1:46 PM, Lucas Meneghel Rodrigues<lmr@redhat.com> wrote:
> pickle allows more control over the load/save process. Specifically, it
> enables us to dump the contents of an object to disk without having to
> unpickle it.
>
> shelve, which uses pickle, seems to pickle and unpickle every time sync()
> is called. This is bad for classes that need to be unpickled only once
> per test (such a class will be introduced in a future patch).
>

> +def dump_env(obj, filename):
> +    file = open(filename, "w")
> +    cPickle.dump(obj, file)
> +    file.close()

This seems like a strange function name - it's really pickling any
object, nothing specific to do with the environment?

> +
> +def load_env(filename, default=None):
> +    try:
> +        file = open(filename, "r")
> +    except:
> +        return default
> +    obj = cPickle.load(file)
> +    file.close()
> +    return obj
> +
> +
>  class kvm(test.test):
>     """
>     Suite of KVM virtualization functional tests.
> @@ -62,12 +78,12 @@ class kvm(test.test):
>         keys = params.keys()
>         keys.sort()
>         for key in keys:
> -            logging.debug("    %s = %s" % (key, params[key]))
> +            logging.debug("    %s = %s", key, params[key])
>             self.write_test_keyval({key: params[key]})
>
>         # Open the environment file
>         env_filename = os.path.join(self.bindir, "env")
> -        env = shelve.open(env_filename, writeback=True)
> +        env = load_env(env_filename, {})
>         logging.debug("Contents of environment: %s" % str(env))
>
>         try:
> @@ -90,21 +106,20 @@ class kvm(test.test):
>
>                 # Preprocess
>                 kvm_preprocessing.preprocess(self, params, env)
> -                env.sync()
> +                dump_env(env, env_filename)
>                 # Run the test function
>                 routine_obj.routine(self, params, env)
> -                env.sync()
> +                dump_env(env, env_filename)
>
>             except Exception, e:
> -                logging.error("Test failed: %s" % e)
> +                logging.error("Test failed: %s", e)
>                 logging.debug("Postprocessing on error...")
>                 kvm_preprocessing.postprocess_on_error(self, params, env)
> -                env.sync()
> +                dump_env(env, env_filename)
>                 raise
>
>         finally:
>             # Postprocess
>             kvm_preprocessing.postprocess(self, params, env)
> -            logging.debug("Contents of environment: %s" % str(env))
> -            env.sync()
> -            env.close()
> +            logging.debug("Contents of environment: %s", str(env))
> +            dump_env(env, env_filename)
> --
> 1.6.2.2
>
> _______________________________________________
> Autotest mailing list
> Autotest@test.kernel.org
> http://test.kernel.org/cgi-bin/mailman/listinfo/autotest
>

Re: [Autotest] [KVM-AUTOTEST PATCH 8/8] kvm_runtest_2.py: use pickle instead of shelve when loading/saving env

[Lucas Meneghel Rodrigues]

On Mon, 2009-06-08 at 09:10 -0700, Martin Bligh wrote:
> On Fri, Jun 5, 2009 at 1:46 PM, Lucas Meneghel Rodrigues<lmr@redhat.com> wrote:
> > pickle allows more control over the load/save process. Specifically, it
> > enables us to dump the contents of an object to disk without having to
> > unpickle it.
> >
> > shelve, which uses pickle, seems to pickle and unpickle every time sync()
> > is called. This is bad for classes that need to be unpickled only once
> > per test (such a class will be introduced in a future patch).
> >
> 
> > +def dump_env(obj, filename):
> > +    file = open(filename, "w")
> > +    cPickle.dump(obj, file)
> > +    file.close()
> 
> This seems like a strange function name - it's really pickling any
> object, nothing specific to do with the environment?

Yes, I agree it's pickling any object, but in the end, we are using this
file to unpickle data from our environment file, hence Michael choose
that name.

> > +
> > +def load_env(filename, default=None):
> > +    try:
> > +        file = open(filename, "r")
> > +    except:
> > +        return default
> > +    obj = cPickle.load(file)
> > +    file.close()
> > +    return obj
> > +
> > +
> >  class kvm(test.test):
> >     """
> >     Suite of KVM virtualization functional tests.
> > @@ -62,12 +78,12 @@ class kvm(test.test):
> >         keys = params.keys()
> >         keys.sort()
> >         for key in keys:
> > -            logging.debug("    %s = %s" % (key, params[key]))
> > +            logging.debug("    %s = %s", key, params[key])
> >             self.write_test_keyval({key: params[key]})
> >
> >         # Open the environment file
> >         env_filename = os.path.join(self.bindir, "env")
> > -        env = shelve.open(env_filename, writeback=True)
> > +        env = load_env(env_filename, {})
> >         logging.debug("Contents of environment: %s" % str(env))
> >
> >         try:
> > @@ -90,21 +106,20 @@ class kvm(test.test):
> >
> >                 # Preprocess
> >                 kvm_preprocessing.preprocess(self, params, env)
> > -                env.sync()
> > +                dump_env(env, env_filename)
> >                 # Run the test function
> >                 routine_obj.routine(self, params, env)
> > -                env.sync()
> > +                dump_env(env, env_filename)
> >
> >             except Exception, e:
> > -                logging.error("Test failed: %s" % e)
> > +                logging.error("Test failed: %s", e)
> >                 logging.debug("Postprocessing on error...")
> >                 kvm_preprocessing.postprocess_on_error(self, params, env)
> > -                env.sync()
> > +                dump_env(env, env_filename)
> >                 raise
> >
> >         finally:
> >             # Postprocess
> >             kvm_preprocessing.postprocess(self, params, env)
> > -            logging.debug("Contents of environment: %s" % str(env))
> > -            env.sync()
> > -            env.close()
> > +            logging.debug("Contents of environment: %s", str(env))
> > +            dump_env(env, env_filename)
> > --
> > 1.6.2.2
> >
> > _______________________________________________
> > Autotest mailing list
> > Autotest@test.kernel.org
> > http://test.kernel.org/cgi-bin/mailman/listinfo/autotest
> >
-- 
Lucas Meneghel Rodrigues
Software Engineer (QE)
Red Hat - Emerging Technologies

Re: [KVM-AUTOTEST PATCH 7/8] Add new function, VM.clone() to clone an existing VM

[Lucas Meneghel Rodrigues]

On Fri, 2009-06-05 at 17:46 -0300, Lucas Meneghel Rodrigues wrote:
> This method will be useful when writing functional tests.
> 
> Risk: Low (new method of the kvm test API)
> Visibility: Low (kvm test developers will have an additional
> API to use)

Applied, but the commit was ammended to follow the coding style. Sorry I
have overlooked it when I rebased the patch:

diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
index 3001648..eb9717b 100644
--- a/client/tests/kvm/kvm_vm.py
+++ b/client/tests/kvm/kvm_vm.py
@@ -126,6 +126,33 @@ class VM:
                 break
 
 
+    def clone(self, name=None, params=None, qemu_path=None, image_dir=None,
+              iso_dir=None):
+        """
+        Return a clone of the VM object with optionally modified parameters.
+        The clone is initially not alive and needs to be started using create().
+        Any parameters not passed to this function are copied from the source
+        VM.
+
+        @param name: Optional new VM name
+        @param params: Optional new VM creation parameters
+        @param qemu_path: Optional new path to qemu
+        @param image_dir: Optional new image dir
+        @param iso_dir: Optional new iso directory
+        """
+        if name == None:
+            name = self.name
+        if params == None:
+            params = self.params.copy()
+        if qemu_path == None:
+            qemu_path = self.qemu_path
+        if image_dir == None:
+            image_dir = self.image_dir
+        if iso_dir == None:
+            iso_dir = self.iso_dir
+        return VM(name, params, qemu_path, image_dir, iso_dir)
+
+
     def verify_process_identity(self):
         """
         Make sure .pid really points to the original qemu process. If .pid

> Signed-off-by: Michael Goldish <mgoldish@redhat.com>
> ---
>  client/tests/kvm/kvm_vm.py |   19 +++++++++++++++++++
>  1 files changed, 19 insertions(+), 0 deletions(-)
> 
> diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py
> index 3001648..b81c0df 100644
> --- a/client/tests/kvm/kvm_vm.py
> +++ b/client/tests/kvm/kvm_vm.py
> @@ -126,6 +126,25 @@ class VM:
>                  break
>  
> 
> +    def clone(self, name=None, params=None, qemu_path=None, image_dir=None, iso_dir=None):
> +        """Return a clone of the VM object with optionally modified parameters.
> +
> +        The clone is initially not alive and needs to be started using create().
> +        Any parameters not passed to this function are copied from the source VM.
> +        """
> +        if name == None:
> +            name = self.name
> +        if params == None:
> +            params = self.params.copy()
> +        if qemu_path == None:
> +            qemu_path = self.qemu_path
> +        if image_dir == None:
> +            image_dir = self.image_dir
> +        if iso_dir == None:
> +            iso_dir = self.iso_dir
> +        return VM(name, params, qemu_path, image_dir, iso_dir)
> +
> +
>      def verify_process_identity(self):
>          """
>          Make sure .pid really points to the original qemu process. If .pid
-- 
Lucas Meneghel Rodrigues
Software Engineer (QE)
Red Hat - Emerging Technologies