From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58851) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1asbPb-0006yI-TE for qemu-devel@nongnu.org; Tue, 19 Apr 2016 15:34:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1asbPW-0001Zd-Sz for qemu-devel@nongnu.org; Tue, 19 Apr 2016 15:34:15 -0400 Received: from e06smtp11.uk.ibm.com ([195.75.94.107]:50936) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1asbPW-0001ZU-Kr for qemu-devel@nongnu.org; Tue, 19 Apr 2016 15:34:10 -0400 Received: from localhost by e06smtp11.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 19 Apr 2016 20:34:09 +0100 From: Sascha Silbe Date: Tue, 19 Apr 2016 21:34:02 +0200 Message-Id: <1461094442-16014-1-git-send-email-silbe@linux.vnet.ibm.com> In-Reply-To: <1460633543-7366-1-git-send-email-silbe@linux.vnet.ibm.com> References: <1460633543-7366-1-git-send-email-silbe@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Subject: [Qemu-devel] [PATCH for-2.6? v2] qemu-iotests: iotests: fail hard if not run via "check" List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org, qemu-block@nongnu.org, Kevin Wolf , Max Reitz Cc: Tu Bo Running an iotests-based Python test directly might appear to work, but may fail in subtle ways and is insecure: - It creates files with predictable file names in a world-writable location (/var/tmp). - Tests expect the environment to be set up by check. E.g. 041 and 055 may take the wrong code paths if QEMU_DEFAULT_MACHINE is not set. This can lead to false negatives. Instead fail hard and tell the user we want to be run via "check". The actual environment expected by the tests is currently only defined by the implementation of "check". We use two of the environment variables set by "check" as indication of whether we're being run via "check". Anyone writing their own test runner (replacing "check") will need to replicate the full environment (in a broader sense, not just environment variables) provided by "check" anyway, including setting the two environment variables we check. Whereas a regular developer just trying to invoke the tests usually won't have both of these defined in their environment so we can catch their mistake and give out useful advice. Signed-off-by: Sascha Silbe Reviewed-by: Bo Tu --- v1→v2: - Add comment indicating that it's not just TEST_DIR and QEMU_DEFAULT_MACHINE that need to be set. Amend commit message in a similar way. @Tu Bo: I'm assuming your Reviewed-By: still stands as I only added more information on the background of the change and the check. --- tests/qemu-iotests/iotests.py | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 0c0b533..054b482 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -46,7 +46,7 @@ if os.environ.get('QEMU_OPTIONS'): imgfmt = os.environ.get('IMGFMT', 'raw') imgproto = os.environ.get('IMGPROTO', 'file') -test_dir = os.environ.get('TEST_DIR', '/var/tmp') +test_dir = os.environ.get('TEST_DIR') output_dir = os.environ.get('OUTPUT_DIR', '.') cachemode = os.environ.get('CACHEMODE') qemu_default_machine = os.environ.get('QEMU_DEFAULT_MACHINE') @@ -441,6 +441,14 @@ def verify_quorum(): def main(supported_fmts=[], supported_oses=['linux']): '''Run tests''' + # We are using TEST_DIR and QEMU_DEFAULT_MACHINE as proxies to + # indicate that we're not being run via "check". There may be + # other things set up by "check" that individual test cases rely + # on. + if test_dir is None or qemu_default_machine is None: + sys.stderr.write('Please run this test via the "check" script\n') + sys.exit(os.EX_USAGE) + debug = '-d' in sys.argv verbosity = 1 verify_image_format(supported_fmts) -- 1.9.1