Re: [Autotest PATCH] KVM-test: Add a subtest image_copy

[Lucas Meneghel Rodrigues <lmr@redhat.com>]

Yolkfull, I am copying Michael and Lawrence on the e-mail so they can
comment on the points I am going to present.

I've been meaning to comment on this test for a while, but refrained to
do so because I wanted to think a little about using image copy as an
alternate install method. Here are my thoughts:

1) Provided that one group has the ready images, indeed an explicit
image_copy test is a useful test, because it's virtually free of
failures.

2) However, operating system install is a fairly complex activity, and
it might reveal KVM bugs, so I believe the actual install step is *very*
valuable for KVM testing.

3) I do understand that the whole point of having this image_copy test
is to make sure we always execute the subsequent functional tests. But
that could be better done, considering 2), if we execute image_copy only
if unattended_install fails. But then we'd have to figure out a way to
do reverse dependency on the framework.

4) Instead of having an explicit image_copy test we could do image copy
as a postprocessing step, in case of failure of unattended install, and
remove the unattended install dependency from the subsequent tests.

My conclusion is that having this as an extra test won't be a problem
(after all we can allways choose to not run it on our test sets), but
your team could look at the possibility of 1st trying to do the actual
OS install, then resorting to image copy only if the test fails.

That said, the actual review of the test follows:

On Wed, 2010-01-06 at 11:32 +0800, Yolkfull Chow wrote:
> Add image_copy subtest for convenient KVM functional testing.
> 
> The target image will be copied into the linked directory if link 'images'
> is created, and copied to the directory specified in config file otherwise.
> 
> Signed-off-by: Yolkfull Chow <yzhou@redhat.com>
> ---
>  client/tests/kvm/kvm_utils.py          |   64 ++++++++++++++++++++++++++++++++
>  client/tests/kvm/tests/image_copy.py   |   42 +++++++++++++++++++++
>  client/tests/kvm/tests_base.cfg.sample |    6 +++
>  3 files changed, 112 insertions(+), 0 deletions(-)
>  create mode 100644 client/tests/kvm/tests/image_copy.py
> 
> diff --git a/client/tests/kvm/kvm_utils.py b/client/tests/kvm/kvm_utils.py
> index 2bbbe22..3944b2b 100644
> --- a/client/tests/kvm/kvm_utils.py
> +++ b/client/tests/kvm/kvm_utils.py
> @@ -924,3 +924,67 @@ def create_report(report_dir, results_dir):
>      reporter = os.path.join(report_dir, 'html_report.py')
>      html_file = os.path.join(results_dir, 'results.html')
>      os.system('%s -r %s -f %s -R' % (reporter, results_dir, html_file))
> +
> +
> +def is_dir_mounted(source, dest, type, perm):
> +    """
> +    Check whether `source' is mounted on `dest' with right permission.
> +
> +    @source: mount source
> +    @dest:   mount point
> +    @type:   file system type
> +    """
> +    match_string = "%s %s %s %s" % (source, dest, type, perm)
> +    try:
> +        f = open("/etc/mtab", "r")
> +        mounted = f.read()
> +        f.close()
> +    except IOError:
> +        mounted = commands.getoutput("mount")
> +    if match_string in mounted: 
> +        return True
> +    return False

Except for checking permissions, the above could be saved by just using
os.path.ismount() instead. Considering that we are only going to use the
nfs share for copying images from there, it really doesn't matter if it
is read only or read write, so perhaps we could just use
os.path.ismount() instead.

> +
> +
> +def umount(mount_point):
> +    """
> +    Umount `mount_point'.
> +
> +    @mount_point: mount point
> +    """
> +    cmd = "umount %s" % mount_point
> +    s, o = commands.getstatusoutput(cmd)
> +    if s != 0:
> +        logging.error("Fail to umount: %s" % o)
> +        return False
> +    return True

Let's avoid to use the commands api when we have utils.run or
utils.system instead. trap it on a try/except block like:

try:
    utils.run("umount %s" % mount_point)
    return True
except CmdError, e:
    logging.error("Fail to unmount %s: %s", mount_point, str(e))
    return False

> +
> +def mount(src, mount_point, type, perm = "rw"):
> +    """
> +    Mount the src into mount_point of the host.
> +
> +    @src: mount source
> +    @mount_point: mount point
> +    @type: file system type
> +    @perm: mount permission
> +    """
> +    if is_dir_mounted(src, mount_point, type, perm):
> +        return True

Please see comment on os.path.ismount().

> +    umount(mount_point)
> +
> +    cmd = "mount -t %s %s %s -o %s" % (type, src, mount_point, perm)
> +    logging.debug("Issue mount command: %s" % cmd)
> +    s, o = commands.getstatusoutput(cmd)
> +    if s != 0:
> +        logging.error("Fail to mount: %s " % o)
> +        return False

Please use utils.system() and a try/except block.

> +    if is_dir_mounted(src, mount_point, type, perm):
> +        logging.info("Successfully mounted %s" % src)
> +        return True
> +    else:
> +        logging.error("Mount verification failed; currently mounted: %s" %
> +                                                 file('/etc/mtab').read())
> +        return False
> diff --git a/client/tests/kvm/tests/image_copy.py b/client/tests/kvm/tests/image_copy.py
> new file mode 100644
> index 0000000..800fb90
> --- /dev/null
> +++ b/client/tests/kvm/tests/image_copy.py
> @@ -0,0 +1,42 @@
> +import os, logging, commands
> +from autotest_lib.client.common_lib import error
> +import kvm_utils
> +
> +def run_image_copy(test, params, env):
> +    """
> +    Copy guest images from NFS server.
> +    1) Mount the NFS directory
> +    2) Check the existence of source image
> +    3) If existence copy the image from NFS
> +
> +    @param test: kvm test object
> +    @param params: Dictionary with the test parameters
> +    @param env: Dictionary with test environment.
> +    """
> +    mount_dest_dir = params.get("dst_dir",'/mnt/images')
> +    if not os.path.exists(mount_dest_dir):
> +        os.mkdir(mount_dest_dir)
> +
> +    src_dir = params.get('nfs_images_dir')
> +    image_dir = os.path.join(os.environ['AUTODIR'],'tests/kvm/images')

A better way of doing this would use the test.bindir attribute that is
already available on the present namespace

      image_dir = os.path.join(test.bindir, 'images')

> +    if not os.path.exists(image_dir):
> +        image_dir = os.path.dirname(params.get("image_name"))
> +
> +    image = os.path.split(params['image_name'])[1]+'.'+params['image_format']

Please fix the spacing on the attribution above (around the minus sign).

> +    src_path = os.path.join(mount_dest_dir, image)
> +    dst_path = os.path.join(image_dir, image)
> +
> +    if not kvm_utils.mount(src_dir, mount_dest_dir, "nfs", "ro"):
> +        raise error.TestError("Fail to mount the %s to %s" %
> +                                  (src_dir, mount_dest_dir))

The above should be "Failed to mount..."

> +    # Check the existence of source image
> +    if not os.path.exists(src_path):
> +        raise error.TestError("Could not found %s in src directory" % src_path)

The above should be "Could not find..."

> +    logging.info("Copying image '%s'..." % image)
> +    cmd = "cp %s %s" % (src_path, dst_path)
> +    s, o = commands.getstatusoutput(cmd)
> +    if s != 0:
> +        raise error.TestFail("Failed to copy image %s: %s" % (cmd, o))

See comments on using utils.run or utils.system instead.

> diff --git a/client/tests/kvm/tests_base.cfg.sample b/client/tests/kvm/tests_base.cfg.sample
> index b8f25f4..bdeac19 100644
> --- a/client/tests/kvm/tests_base.cfg.sample
> +++ b/client/tests/kvm/tests_base.cfg.sample
> @@ -61,6 +61,12 @@ variants:
>          floppy = "images/floppy.img"
>          extra_params += " -boot d"
>  
> +    - image_copy:
> +        type = image_copy
> +        vms = ''
> +        # Here specify the NFS directory that contains all images
> +        nfs_images_dir = 
> +
>      - setup:        install unattended_install
>          type = steps
>          fail_if_stuck_for = 300