From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:44425) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XFi1l-0007r1-DQ for qemu-devel@nongnu.org; Fri, 08 Aug 2014 07:08:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XFi1h-00075k-8I for qemu-devel@nongnu.org; Fri, 08 Aug 2014 07:08:05 -0400 Received: from mx1.redhat.com ([209.132.183.28]:14680) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XFi1h-00075a-0F for qemu-devel@nongnu.org; Fri, 08 Aug 2014 07:08:01 -0400 Date: Fri, 8 Aug 2014 19:08:17 +0800 From: Fam Zheng Message-ID: <20140808110817.GB3897@T430.nay.redhat.com> References: <8ab8a1dd18656589fb9d7f17c6aa3a53da4e3f61.1407328960.git.maria.k@catit.be> <20140808085030.GA3897@T430.nay.redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Subject: Re: [Qemu-devel] [PATCH V5 2/5] runner: Tool for fuzz tests execution List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "M.Kustova" Cc: Kevin Wolf , qemu-devel@nongnu.org, Stefan Hajnoczi On Fri, 08/08 12:58, M.Kustova wrote: > On Fri, Aug 8, 2014 at 12:50 PM, Fam Zheng wrote: > > On Wed, 08/06 17:12, Maria Kustova wrote: > >> The purpose of the test runner is to prepare the test environment (e.g. create > >> a work directory, a test image, etc), execute a program under test with > >> parameters, indicate a test failure if the program was killed during the test > >> execution and collect core dumps, logs and other test artifacts. > >> > >> The test runner doesn't depend on an image format or a program will be tested, > >> so it can be used with any external image generator and program under test. > >> > >> Signed-off-by: Maria Kustova > >> --- > >> tests/image-fuzzer/runner/runner.py | 405 ++++++++++++++++++++++++++++++++++++ > >> 1 file changed, 405 insertions(+) > >> create mode 100755 tests/image-fuzzer/runner/runner.py > >> > >> diff --git a/tests/image-fuzzer/runner/runner.py b/tests/image-fuzzer/runner/runner.py > >> new file mode 100755 > >> index 0000000..3fa7fca > >> --- /dev/null > >> +++ b/tests/image-fuzzer/runner/runner.py > >> @@ -0,0 +1,405 @@ > >> +#!/usr/bin/env python > >> + > >> +# Tool for running fuzz tests > >> +# > >> +# Copyright (C) 2014 Maria Kustova > >> +# > >> +# This program is free software: you can redistribute it and/or modify > >> +# it under the terms of the GNU General Public License as published by > >> +# the Free Software Foundation, either version 2 of the License, or > >> +# (at your option) any later version. > >> +# > >> +# This program is distributed in the hope that it will be useful, > >> +# but WITHOUT ANY WARRANTY; without even the implied warranty of > >> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > >> +# GNU General Public License for more details. > >> +# > >> +# You should have received a copy of the GNU General Public License > >> +# along with this program. If not, see . > >> +# > >> + > >> +import sys > >> +import os > >> +import signal > >> +import subprocess > >> +import random > >> +import shutil > >> +from itertools import count > >> +import getopt > >> +import StringIO > >> +import resource > >> + > >> +try: > >> + import json > >> +except ImportError: > >> + try: > >> + import simplejson as json > >> + except ImportError: > >> + print >>sys.stderr, \ > >> + "Warning: Module for JSON processing is not found.\n" \ > >> + "'--config' and '--command' options are not supported." > >> + > >> +# Backing file sizes in MB > >> +MAX_BACKING_FILE_SIZE = 10 > >> +MIN_BACKING_FILE_SIZE = 1 > >> + > >> + > >> +def multilog(msg, *output): > >> + """ Write an object to all of specified file descriptors.""" > >> + for fd in output: > >> + fd.write(msg) > >> + fd.flush() > >> + > >> + > >> +def str_signal(sig): > >> + """ Convert a numeric value of a system signal to the string one > >> + defined by the current operational system. > >> + """ > >> + for k, v in signal.__dict__.items(): > >> + if v == sig: > >> + return k > >> + > >> + > >> +def run_app(fd, q_args): > >> + """Start an application with specified arguments and return its exit code > >> + or kill signal depending on the result of execution. > >> + """ > >> + devnull = open('/dev/null', 'r+') > >> + process = subprocess.Popen(q_args, stdin=devnull, > >> + stdout=subprocess.PIPE, > >> + stderr=subprocess.PIPE) > >> + out, err = process.communicate() > >> + fd.write(out) > >> + fd.write(err) > >> + return process.returncode > >> + > >> + > >> +class TestException(Exception): > >> + """Exception for errors risen by TestEnv objects.""" > >> + pass > >> + > >> + > >> +class TestEnv(object): > >> + > >> + """Test object. > >> + > >> + The class sets up test environment, generates backing and test images > >> + and executes application under tests with specified arguments and a test > >> + image provided. > >> + > >> + All logs are collected. > >> + > >> + The summary log will contain short descriptions and statuses of tests in > >> + a run. > >> + > >> + The test log will include application (e.g. 'qemu-img') logs besides info > >> + sent to the summary log. > >> + """ > >> + > >> + def __init__(self, test_id, seed, work_dir, run_log, > >> + cleanup=True, log_all=False): > >> + """Set test environment in a specified work directory. > >> + > >> + Path to qemu-img and qemu-io will be retrieved from 'QEMU_IMG' and > >> + 'QEMU_IO' environment variables. > >> + """ > >> + if seed is not None: > >> + self.seed = seed > >> + else: > >> + self.seed = str(random.randint(0, sys.maxint)) > >> + random.seed(self.seed) > >> + > >> + self.init_path = os.getcwd() > >> + self.work_dir = work_dir > >> + self.current_dir = os.path.join(work_dir, 'test-' + test_id) > >> + self.qemu_img = os.environ.get('QEMU_IMG', 'qemu-img')\ > >> + .strip().split(' ') > > > > Nitpicking. I think split(' ') doesn't make sense, this could instead be: > > > > self.qemu_img = [os.environ.get('QEMU_IMG', 'qemu-img').strip()] > > > > Otherwise user won't be able to pass in a QEMU_IMG path with space. > > > > Corner case, though. Otherwise looks good, > > This functionality was inherited from qemu-iotests/iotests.py. > Rationale for this splitting was that environment variables can > contain call arguments as well. > Ruining paths with white spaces was a reasonable trade-off. OK, thanks for explanation. Fam