* [PATCH XTF v2 1/4] xtf-runner: split into logical components
2018-12-28 18:42 [PATCH XTF v2 0/4] Add monitor tests to XTF Petre Pircalabu
@ 2018-12-28 18:42 ` Petre Pircalabu
2018-12-28 18:42 ` [PATCH XTF v2 2/4] xtf: Add executable test class Petre Pircalabu
` (3 subsequent siblings)
4 siblings, 0 replies; 11+ messages in thread
From: Petre Pircalabu @ 2018-12-28 18:42 UTC (permalink / raw)
To: xen-devel
Cc: Petre Pircalabu, Stefano Stabellini, Wei Liu,
Konrad Rzeszutek Wilk, George Dunlap, Andrew Cooper, Ian Jackson,
Tim Deegan, Julien Grall, Jan Beulich
Split the xtf-runner script file into multiple modules in order to
support multiple test types.
Features:
- 2 abstract types (TestInfo and TestInstance) to represent the
test information (info.json) and, respectively to implement the test
execution.
TestInfo has to implement the "all_instances" method to create the
list of TestInstance objects.
TestInstance has to implement "set_up", "run", and "clean-up"
methods.
- TestResult - represents an XTF test result (SUCCESS, SKIP, ERROR,
FAILURE, CRASH). The values should be kept in sync with the C code
from report.h
- Dynamic test class loading. Each info.json shoudl contain a
"class_name" field which specifies the test info class describing the
test. This value defaults to "xtf.domu_test.DomuTestInfo"
- custom test info parameters. info.json can have the "extra"
field, implemented as a dictionary, which contains parameters
specific for a certain test info class.
e.g. TEST-EXTRA-INFO := arg1='--address=0x80000000 --id=4' arg2=42
- logger class (print depending on the quiet field)
- DomuTestInfo/DomuTest instance. Simple test which loads a XEN DomU
and checks the output for a specific pattern.
- toolstack abstraction using a wrapper class (e.g.
(xtf.xl_domu.XLDomU)
Signed-off-by: Petre Pircalabu <ppircalabu@bitdefender.com>
---
build/gen.mk | 13 ++-
build/mkinfo.py | 84 +++++++++++---
xtf-runner | 334 +++++-------------------------------------------------
xtf/__init__.py | 12 ++
xtf/domu_test.py | 179 +++++++++++++++++++++++++++++
xtf/exceptions.py | 6 +
xtf/logger.py | 23 ++++
xtf/suite.py | 97 ++++++++++++++++
xtf/test.py | 139 +++++++++++++++++++++++
xtf/xl_domu.py | 121 ++++++++++++++++++++
10 files changed, 687 insertions(+), 321 deletions(-)
create mode 100644 xtf/__init__.py
create mode 100644 xtf/domu_test.py
create mode 100644 xtf/exceptions.py
create mode 100644 xtf/logger.py
create mode 100644 xtf/suite.py
create mode 100644 xtf/test.py
create mode 100644 xtf/xl_domu.py
diff --git a/build/gen.mk b/build/gen.mk
index 8d7a6bf..c19ca6a 100644
--- a/build/gen.mk
+++ b/build/gen.mk
@@ -27,12 +27,23 @@ else
TEST-CFGS := $(foreach env,$(TEST-ENVS),test-$(env)-$(NAME).cfg)
endif
+CLASS ?= "xtf.domu_test.DomuTestInfo"
+
.PHONY: build
build: $(foreach env,$(TEST-ENVS),test-$(env)-$(NAME)) $(TEST-CFGS)
build: info.json
+MKINFO-OPTS := -n "$(NAME)"
+MKINFO-OPTS += -c "$(CLASS)"
+MKINFO-OPTS += -t "$(CATEGORY)"
+MKINFO-OPTS += -e "$(TEST-ENVS)"
+MKINFO-OPTS += -v "$(VARY-CFG)"
+ifneq (x$(TEST-EXTRA-INFO), x)
+MKINFO-OPTS += -x "$(TEST-EXTRA-INFO)"
+endif
+
info.json: $(ROOT)/build/mkinfo.py FORCE
- @$(PYTHON) $< $@.tmp "$(NAME)" "$(CATEGORY)" "$(TEST-ENVS)" "$(VARY-CFG)"
+ @$(PYTHON) $< $(MKINFO-OPTS) $@.tmp
@$(call move-if-changed,$@.tmp,$@)
.PHONY: install install-each-env
diff --git a/build/mkinfo.py b/build/mkinfo.py
index 94891a9..afa355c 100644
--- a/build/mkinfo.py
+++ b/build/mkinfo.py
@@ -1,24 +1,74 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
+""" mkinfo.py
-import sys, os, json
+ Generates a test info json file.
+ The script is ran at build stage using the parameters specified
+ in the test's Makefile.
+"""
-# Usage: mkcfg.py $OUT $NAME $CATEGORY $ENVS $VARIATIONS
-_, out, name, cat, envs, variations = sys.argv
+import json
+import sys
+import shlex
+from optparse import OptionParser
-template = {
- "name": name,
- "category": cat,
- "environments": [],
- "variations": [],
- }
+def main():
+ """ Main entrypoint """
+ # Avoid wrapping the epilog text
+ OptionParser.format_epilog = lambda self, formatter: self.epilog
-if envs:
- template["environments"] = envs.split(" ")
-if variations:
- template["variations"] = variations.split(" ")
+ parser = OptionParser(
+ usage = "%prog [OPTIONS] out_file",
+ description = "Xen Test Framework json generation tool",
+ )
-open(out, "w").write(
- json.dumps(template, indent=4, separators=(',', ': '))
- + "\n"
- )
+ parser.add_option("-n", "--name", action = "store",
+ dest = "name",
+ help = "Test name",
+ )
+ parser.add_option("-c", "--class", action = "store",
+ dest = "class_name",
+ help = "Test class name",
+ )
+ parser.add_option("-t", "--category", action = "store",
+ dest = "cat",
+ help = "Test category",
+ )
+ parser.add_option("-e", "--environments", action = "store",
+ dest = "envs",
+ help = "Test environments (e.g hvm64, pv64 ...)",
+ )
+ parser.add_option("-v", "--variations", action = "store",
+ dest = "variations",
+ help = "Test variations",
+ )
+ parser.add_option("-x", "--extra", action = "store",
+ dest = "extra",
+ help = "Test specific parameters",
+ )
+
+ opts, args = parser.parse_args()
+ template = {
+ "name": opts.name,
+ "class_name": opts.class_name,
+ "category": opts.cat,
+ "environments": [],
+ "variations": [],
+ "extra": {}
+ }
+
+ if opts.envs:
+ template["environments"] = opts.envs.split(" ")
+ if opts.variations:
+ template["variations"] = opts.variations.split(" ")
+ if opts.extra:
+ template["extra"] = dict([(e.split('=',1))
+ for e in shlex.split(opts.extra)])
+
+ open(args[0], "w").write(
+ json.dumps(template, indent=4, separators=(',', ': '))
+ + "\n"
+ )
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/xtf-runner b/xtf-runner
index 172cb1d..1a4901a 100755
--- a/xtf-runner
+++ b/xtf-runner
@@ -7,154 +7,30 @@
Currently assumes the presence and availability of the `xl` toolstack.
"""
-import sys, os, os.path as path
+import os
+import sys
from optparse import OptionParser
-from subprocess import Popen, PIPE, call as subproc_call
+from subprocess import Popen, PIPE
-try:
- import json
-except ImportError:
- import simplejson as json
-
-# All results of a test, keep in sync with C code report.h.
-# Notes:
-# - WARNING is not a result on its own.
-# - CRASH isn't known to the C code, but covers all cases where a valid
-# result was not found.
-all_results = ['SUCCESS', 'SKIP', 'ERROR', 'FAILURE', 'CRASH']
+from xtf import default_categories, non_default_categories, all_categories
+from xtf import pv_environments, hvm_environments, all_environments
+from xtf.exceptions import RunnerError
+from xtf.logger import Logger
+from xtf.suite import get_all_test_info, gather_all_test_info
+from xtf.test import TestResult
# Return the exit code for different states. Avoid using 1 and 2 because
# python interpreter uses them -- see document for sys.exit.
def exit_code(state):
""" Convert a test result to an xtf-runner exit code. """
- return { "SUCCESS": 0,
- "SKIP": 3,
- "ERROR": 4,
- "FAILURE": 5,
- "CRASH": 6,
+ return { TestResult.SUCCESS: 0,
+ TestResult.SKIP: 3,
+ TestResult.ERROR: 4,
+ TestResult.FAILURE: 5,
+ TestResult.CRASH: 6,
}[state]
-# All test categories
-default_categories = set(("functional", "xsa"))
-non_default_categories = set(("special", "utility", "in-development"))
-all_categories = default_categories | non_default_categories
-
-# All test environments
-pv_environments = set(("pv64", "pv32pae"))
-hvm_environments = set(("hvm64", "hvm32pae", "hvm32pse", "hvm32"))
-all_environments = pv_environments | hvm_environments
-
-
-class RunnerError(Exception):
- """ Errors relating to xtf-runner itself """
-
-class TestInstance(object):
- """ Object representing a single test. """
-
- def __init__(self, arg):
- """ Parse and verify 'arg' as a test instance. """
- self.env, self.name, self.variation = parse_test_instance_string(arg)
-
- if self.env is None:
- raise RunnerError("No environment for '%s'" % (arg, ))
-
- if self.variation is None and get_all_test_info()[self.name].variations:
- raise RunnerError("Test '%s' has variations, but none specified"
- % (self.name, ))
-
- def vm_name(self):
- """ Return the VM name as `xl` expects it. """
- return repr(self)
-
- def cfg_path(self):
- """ Return the path to the `xl` config file for this test. """
- return path.join("tests", self.name, repr(self) + ".cfg")
-
- def __repr__(self):
- if not self.variation:
- return "test-%s-%s" % (self.env, self.name)
- else:
- return "test-%s-%s~%s" % (self.env, self.name, self.variation)
-
- def __hash__(self):
- return hash(repr(self))
-
- def __cmp__(self, other):
- return cmp(repr(self), repr(other))
-
-
-class TestInfo(object):
- """ Object representing a tests info.json, in a more convenient form. """
-
- def __init__(self, test_json):
- """Parse and verify 'test_json'.
-
- May raise KeyError, TypeError or ValueError.
- """
-
- name = test_json["name"]
- if not isinstance(name, basestring):
- raise TypeError("Expected string for 'name', got '%s'"
- % (type(name), ))
- self.name = name
-
- cat = test_json["category"]
- if not isinstance(cat, basestring):
- raise TypeError("Expected string for 'category', got '%s'"
- % (type(cat), ))
- if not cat in all_categories:
- raise ValueError("Unknown category '%s'" % (cat, ))
- self.cat = cat
-
- envs = test_json["environments"]
- if not isinstance(envs, list):
- raise TypeError("Expected list for 'environments', got '%s'"
- % (type(envs), ))
- if not envs:
- raise ValueError("Expected at least one environment")
- for env in envs:
- if not env in all_environments:
- raise ValueError("Unknown environments '%s'" % (env, ))
- self.envs = envs
-
- variations = test_json["variations"]
- if not isinstance(variations, list):
- raise TypeError("Expected list for 'variations', got '%s'"
- % (type(variations), ))
- self.variations = variations
-
- def all_instances(self, env_filter = None, vary_filter = None):
- """Return a list of TestInstances, for each supported environment.
- Optionally filtered by env_filter. May return an empty list if
- the filter doesn't match any supported environment.
- """
-
- if env_filter:
- envs = set(env_filter).intersection(self.envs)
- else:
- envs = self.envs
-
- if vary_filter:
- variations = set(vary_filter).intersection(self.variations)
- else:
- variations = self.variations
-
- res = []
- if variations:
- for env in envs:
- for vary in variations:
- res.append(TestInstance("test-%s-%s~%s"
- % (env, self.name, vary)))
- else:
- res = [ TestInstance("test-%s-%s" % (env, self.name))
- for env in envs ]
- return res
-
- def __repr__(self):
- return "TestInfo(%s)" % (self.name, )
-
-
def parse_test_instance_string(arg):
"""Parse a test instance string.
@@ -221,47 +97,6 @@ def parse_test_instance_string(arg):
return env, name, variation
-
-# Cached test json from disk
-_all_test_info = {}
-
-def get_all_test_info():
- """ Open and collate each info.json """
-
- # Short circuit if already cached
- if _all_test_info:
- return _all_test_info
-
- for test in os.listdir("tests"):
-
- info_file = None
- try:
-
- # Ignore directories which don't have a info.json inside them
- try:
- info_file = open(path.join("tests", test, "info.json"))
- except IOError:
- continue
-
- # Ignore tests which have bad JSON
- try:
- test_info = TestInfo(json.load(info_file))
-
- if test_info.name != test:
- continue
-
- except (ValueError, KeyError, TypeError):
- continue
-
- _all_test_info[test] = test_info
-
- finally:
- if info_file:
- info_file.close()
-
- return _all_test_info
-
-
def tests_from_selection(cats, envs, tests):
"""Given a selection of possible categories, environment and tests, return
all tests within the provided parameters.
@@ -433,136 +268,25 @@ def list_tests(opts):
for sel in opts.selection:
print sel
-
-def interpret_result(logline):
- """ Interpret the final log line of a guest for a result """
-
- if not "Test result:" in logline:
- return "CRASH"
-
- for res in all_results:
- if res in logline:
- return res
-
- return "CRASH"
-
-
-def run_test_console(opts, test):
- """ Run a specific, obtaining results via xenconsole """
-
- cmd = ['xl', 'create', '-p', test.cfg_path()]
- if not opts.quiet:
- print "Executing '%s'" % (" ".join(cmd), )
-
- create = Popen(cmd, stdout = PIPE, stderr = PIPE)
- _, stderr = create.communicate()
-
- if create.returncode:
- if opts.quiet:
- print "Executing '%s'" % (" ".join(cmd), )
- print stderr
- raise RunnerError("Failed to create VM")
-
- cmd = ['xl', 'console', test.vm_name()]
- if not opts.quiet:
- print "Executing '%s'" % (" ".join(cmd), )
-
- console = Popen(cmd, stdout = PIPE)
-
- cmd = ['xl', 'unpause', test.vm_name()]
- if not opts.quiet:
- print "Executing '%s'" % (" ".join(cmd), )
-
- rc = subproc_call(cmd)
- if rc:
- if opts.quiet:
- print "Executing '%s'" % (" ".join(cmd), )
- raise RunnerError("Failed to unpause VM")
-
- stdout, _ = console.communicate()
-
- if console.returncode:
- raise RunnerError("Failed to obtain VM console")
-
- lines = stdout.splitlines()
-
- if lines:
- if not opts.quiet:
- print "\n".join(lines)
- print ""
-
- else:
- return "CRASH"
-
- return interpret_result(lines[-1])
-
-
-def run_test_logfile(opts, test):
- """ Run a specific test, obtaining results from a logfile """
-
- logpath = path.join(opts.logfile_dir,
- opts.logfile_pattern.replace("%s", str(test)))
-
- if not opts.quiet:
- print "Using logfile '%s'" % (logpath, )
-
- fd = os.open(logpath, os.O_CREAT | os.O_RDONLY, 0644)
- logfile = os.fdopen(fd)
- logfile.seek(0, os.SEEK_END)
-
- cmd = ['xl', 'create', '-F', test.cfg_path()]
- if not opts.quiet:
- print "Executing '%s'" % (" ".join(cmd), )
-
- guest = Popen(cmd, stdout = PIPE, stderr = PIPE)
-
- _, stderr = guest.communicate()
-
- if guest.returncode:
- if opts.quiet:
- print "Executing '%s'" % (" ".join(cmd), )
- print stderr
- raise RunnerError("Failed to run test")
-
- line = ""
- for line in logfile.readlines():
-
- line = line.rstrip()
- if not opts.quiet:
- print line
-
- if "Test result:" in line:
- print ""
- break
-
- logfile.close()
-
- return interpret_result(line)
-
-
def run_tests(opts):
""" Run tests """
tests = opts.selection
- if not len(tests):
+ if not tests:
raise RunnerError("No tests to run")
- run_test = { "console": run_test_console,
- "logfile": run_test_logfile,
- }.get(opts.results_mode, None)
-
- if run_test is None:
- raise RunnerError("Unknown mode '%s'" % (opts.mode, ))
-
- rc = all_results.index('SUCCESS')
+ rc = TestResult()
results = []
for test in tests:
+ res = TestResult()
+ test.set_up(opts, res)
+ if res == TestResult.SUCCESS:
+ test.run(res)
+ test.clean_up(res)
- res = run_test(opts, test)
- res_idx = all_results.index(res)
- if res_idx > rc:
- rc = res_idx
+ if res > rc:
+ rc = res
results.append(res)
@@ -571,7 +295,7 @@ def run_tests(opts):
for test, res in zip(tests, results):
print "%-40s %s" % (test, res)
- return exit_code(all_results[rc])
+ return exit_code(rc)
def main():
@@ -581,7 +305,7 @@ def main():
sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 1)
# Normalise $CWD to the directory this script is in
- os.chdir(path.dirname(path.abspath(sys.argv[0])))
+ os.chdir(os.path.dirname(os.path.abspath(sys.argv[0])))
# Avoid wrapping the epilog text
OptionParser.format_epilog = lambda self, formatter: self.epilog
@@ -715,12 +439,16 @@ def main():
opts, args = parser.parse_args()
opts.args = args
+ Logger().initialize(opts)
+
+ gather_all_test_info()
+
opts.selection = interpret_selection(opts)
if opts.list_tests:
return list_tests(opts)
- else:
- return run_tests(opts)
+
+ return run_tests(opts)
if __name__ == "__main__":
diff --git a/xtf/__init__.py b/xtf/__init__.py
new file mode 100644
index 0000000..889c1d5
--- /dev/null
+++ b/xtf/__init__.py
@@ -0,0 +1,12 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+# All test categories
+default_categories = set(("functional", "xsa"))
+non_default_categories = set(("special", "utility", "in-development"))
+all_categories = default_categories | non_default_categories
+
+# All test environments
+pv_environments = set(("pv64", "pv32pae"))
+hvm_environments = set(("hvm64", "hvm32pae", "hvm32pse", "hvm32"))
+all_environments = pv_environments | hvm_environments
diff --git a/xtf/domu_test.py b/xtf/domu_test.py
new file mode 100644
index 0000000..4052167
--- /dev/null
+++ b/xtf/domu_test.py
@@ -0,0 +1,179 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+"""
+Basic DomU test
+Runs a domain and checks the output for a spcific pattern.
+"""
+
+import os
+import StringIO
+
+from xtf import all_environments
+from xtf.exceptions import RunnerError
+from xtf.logger import Logger
+from xtf.test import TestInstance, TestInfo, TestResult
+from xtf.xl_domu import XLDomU
+
+class DomuTestInstance(TestInstance):
+ """ Object representing a single DOMU test. """
+
+ def __init__(self, env, name, variation):
+ super(DomuTestInstance, self).__init__(name)
+
+ self.env, self.variation = env, variation
+
+ if self.env is None:
+ raise RunnerError("No environment for '%s'" % (self.name, ))
+
+ self.domu = XLDomU(self.cfg_path())
+ self.results_mode = 'console'
+ self.logpath = None
+ if not Logger().quiet:
+ self.output = StringIO.StringIO()
+ else:
+ self.output = None
+
+ def vm_name(self):
+ """ Return the VM name as `xl` expects it. """
+ return repr(self)
+
+ def cfg_path(self):
+ """ Return the path to the `xl` config file for this test. """
+ return os.path.join("tests", self.name, repr(self) + ".cfg")
+
+ def __repr__(self):
+ if self.variation:
+ return "test-%s-%s~%s" % (self.env, self.name, self.variation)
+ return "test-%s-%s" % (self.env, self.name)
+
+ def set_up(self, opts, result):
+ self.results_mode = opts.results_mode
+ if self.results_mode not in ['console', 'logfile']:
+ raise RunnerError("Unknown mode '%s'" % (opts.results_mode, ))
+
+ self.logpath = os.path.join(opts.logfile_dir,
+ opts.logfile_pattern.replace("%s", str(self)))
+ self.domu.create()
+
+ def run(self, result):
+ """Executes the test instance"""
+ run_test = { "console": self._run_test_console,
+ "logfile": self._run_test_logfile,
+ }.get(self.results_mode, None)
+
+ run_test(result)
+
+ def clean_up(self, result):
+ if self.output:
+ self.output.close()
+
+ # wait for completion
+ if not self.domu.cleanup():
+ result.set(TestResult.CRASH)
+
+ def _run_test_console(self, result):
+ """ Run a specific, obtaining results via xenconsole """
+
+ console = self.domu.console(self.output)
+
+ # start the domain
+ self.domu.unpause()
+ value = console.expect(self.result_pattern())
+
+ if self.output is not None:
+ Logger().log(self.output.getvalue())
+
+ result.set(value)
+
+ def _run_test_logfile(self, result):
+ """ Run a specific test, obtaining results from a logfile """
+
+ Logger().log("Using logfile '%s'" % (self.logpath, ))
+
+ fd = os.open(self.logpath, os.O_CREAT | os.O_RDONLY, 0644)
+ logfile = os.fdopen(fd)
+ logfile.seek(0, os.SEEK_END)
+
+ self.domu.unpause()
+
+ # wait for completion
+ if not self.domu.cleanup():
+ result.set(TestResult.CRASH)
+
+ line = ""
+ for line in logfile.readlines():
+ line = line.rstrip()
+ Logger().log(line)
+
+ if "Test result:" in line:
+ print ""
+ break
+
+ logfile.close()
+
+ result.set(TestInstance.parse_result(line))
+
+
+class DomuTestInfo(TestInfo):
+ """ Object representing a tests info.json, in a more convenient form. """
+
+ def __init__(self, test_json):
+ """Parse and verify 'test_json'.
+
+ May raise KeyError, TypeError or ValueError.
+ """
+
+ super(DomuTestInfo, self).__init__(test_json)
+ self.instance_class = DomuTestInstance
+
+ envs = test_json["environments"]
+ if not isinstance(envs, list):
+ raise TypeError("Expected list for 'environments', got '%s'"
+ % (type(envs), ))
+ if not envs:
+ raise ValueError("Expected at least one environment")
+ for env in envs:
+ if env not in all_environments:
+ raise ValueError("Unknown environments '%s'" % (env, ))
+ self.envs = envs
+
+ variations = test_json["variations"]
+ if not isinstance(variations, list):
+ raise TypeError("Expected list for 'variations', got '%s'"
+ % (type(variations), ))
+ self.variations = variations
+
+ extra = test_json["extra"]
+ if not isinstance(extra, dict):
+ raise TypeError("Expected dict for 'extra', got '%s'"
+ % (type(extra), ))
+ self.extra = extra
+
+ def all_instances(self, env_filter = None, vary_filter = None):
+ """Return a list of TestInstances, for each supported environment.
+ Optionally filtered by env_filter. May return an empty list if
+ the filter doesn't match any supported environment.
+ """
+
+ if env_filter:
+ envs = set(env_filter).intersection(self.envs)
+ else:
+ envs = self.envs
+
+ if vary_filter:
+ variations = set(vary_filter).intersection(self.variations)
+ else:
+ variations = self.variations
+
+ res = []
+ if variations:
+ for env in envs:
+ for vary in variations:
+ res.append(self.instance_class(env, self.name, vary))
+ else:
+ res = [ self.instance_class(env, self.name, None)
+ for env in envs ]
+ return res
+
+ def __repr__(self):
+ return "%s(%s)" % (self.__class__.__name__, self.name, )
diff --git a/xtf/exceptions.py b/xtf/exceptions.py
new file mode 100644
index 0000000..26801a2
--- /dev/null
+++ b/xtf/exceptions.py
@@ -0,0 +1,6 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+class RunnerError(Exception):
+ """ Errors relating to xtf-runner itself """
+
diff --git a/xtf/logger.py b/xtf/logger.py
new file mode 100644
index 0000000..ec279e5
--- /dev/null
+++ b/xtf/logger.py
@@ -0,0 +1,23 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+class Singleton(type):
+ """Singleton meta class"""
+ _instances = {}
+ def __call__(cls, *args, **kwargs):
+ if cls not in cls._instances:
+ cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwargs)
+ return cls._instances[cls]
+
+class Logger(object):
+ """Logger class for XTF."""
+ __metaclass__ = Singleton
+
+ def initialize(self, opts):
+ """Initialize logger"""
+ self.quiet = opts.quiet
+
+ def log(self, message):
+ """Display the message"""
+ if not self.quiet:
+ print message
diff --git a/xtf/suite.py b/xtf/suite.py
new file mode 100644
index 0000000..ad7d30f
--- /dev/null
+++ b/xtf/suite.py
@@ -0,0 +1,97 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+import os, os.path as path
+import sys
+import imp
+
+try:
+ import json
+except ImportError:
+ import simplejson as json
+
+from xtf.exceptions import RunnerError
+
+# Cached test json from disk
+_all_test_info = {}
+
+def _load_module(name):
+ """Loads module dynamically"""
+ components = name.split(".")
+ module_path = sys.path
+
+ for index in xrange(len(components)):
+ module_name = components[index]
+ module = sys.modules.get(module_name)
+ if module:
+ if hasattr(module, '__path__'):
+ module_path = module.__path__
+ continue
+
+ try:
+ mod_file, filename, description = imp.find_module(module_name,
+ module_path)
+ module = imp.load_module(module_name, mod_file, filename,
+ description)
+ if hasattr(module, '__path__'):
+ module_path = module.__path__
+ finally:
+ if mod_file:
+ mod_file.close()
+
+ return module
+
+def _load_class(name):
+ """Loads python class dynamically"""
+ components = name.split(".")
+ class_name = components[-1]
+ module = _load_module(".".join(components[:-1]))
+
+ try:
+ cls = module.__dict__[class_name]
+ return cls
+ except KeyError:
+ return None
+
+
+def get_all_test_info():
+ """ Returns all available test info instances """
+
+ if not _all_test_info:
+ raise RunnerError("No available test info")
+
+ return _all_test_info
+
+
+def gather_all_test_info():
+ """ Open and collate each info.json """
+
+ for test in os.listdir("tests"):
+
+ info_file = None
+ try:
+
+ # Ignore directories which don't have a info.json inside them
+ try:
+ info_file = open(path.join("tests", test, "info.json"))
+ except IOError:
+ continue
+
+ # Ignore tests which have bad JSON
+ try:
+ json_info = json.load(info_file)
+ test_class = _load_class(json_info["class_name"])
+ test_info = test_class(json_info)
+
+ if test_info.name != test:
+ continue
+
+ except (ValueError, KeyError, TypeError):
+ continue
+
+ _all_test_info[test] = test_info
+
+ finally:
+ if info_file:
+ info_file.close()
+
diff --git a/xtf/test.py b/xtf/test.py
new file mode 100644
index 0000000..4440b47
--- /dev/null
+++ b/xtf/test.py
@@ -0,0 +1,139 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+"""
+Base XTF Test Classess
+"""
+import pexpect
+from xtf import all_categories
+
+class TestResult(object):
+ """
+ Test result wrapper class
+ All results of a test, keep in sync with C code report.h.
+ Notes:
+ - WARNING is not a result on its own.
+ - CRASH isn't known to the C code, but covers all cases where a valid
+ result was not found.
+ """
+
+ SUCCESS = 'SUCCESS'
+ SKIP = 'SKIP'
+ ERROR = 'ERROR'
+ FAILURE = 'FAILURE'
+ CRASH = 'CRASH'
+
+ all_results = [SUCCESS, SKIP, ERROR, FAILURE, CRASH]
+
+ def __init__(self, value=SUCCESS):
+ self.set(value)
+
+ def __cmp__(self, other):
+ if isinstance(other, TestResult):
+ return cmp(TestResult.all_results.index(self._value),
+ TestResult.all_results.index(repr(other)))
+ elif isinstance(other, (str, unicode)):
+ if other in TestResult.all_results:
+ return cmp(TestResult.all_results.index(self._value),
+ TestResult.all_results.index(other))
+
+ raise ValueError
+
+ def __repr__(self):
+ return self._value
+
+ def __hash__(self):
+ return hash(repr(self))
+
+ def set(self, value):
+ """
+ The result can be set using both a string value or an index
+ if the index used is out-of-bounds the result will be initialized
+ to CRASH
+ """
+ if isinstance(value, (int, long)):
+ try:
+ self._value = TestResult.all_results[value]
+ except IndexError:
+ self._value = TestResult.CRASH
+ else:
+ if value in TestResult.all_results:
+ self._value = value
+ else:
+ self._value = TestResult.CRASH
+
+
+class TestInstance(object):
+ """Base class for a XTF Test Instance object"""
+
+ @staticmethod
+ def parse_result(logline):
+ """ Interpret the final log line of a guest for a result """
+
+ if "Test result:" not in logline:
+ return TestResult.CRASH
+
+ for res in TestResult.all_results:
+ if res in logline:
+ return res
+
+ return TestResult.CRASH
+
+ @staticmethod
+ def result_pattern():
+ """the test result pattern."""
+ return ['Test result: ' + x for x in TestResult.all_results] + \
+ [pexpect.TIMEOUT, pexpect.EOF]
+
+ def __init__(self, name):
+ self.name = name
+
+ def __hash__(self):
+ return hash(repr(self))
+
+ def __cmp__(self, other):
+ return cmp(repr(self), repr(other))
+
+ def set_up(self, opts, result):
+ """Sets up the necessary resources needed to run the test."""
+ raise NotImplementedError
+
+ def run(self, result):
+ """Runs the Test Instance."""
+ raise NotImplementedError
+
+ def clean_up(self, result):
+ """Cleans up the test data."""
+ raise NotImplementedError
+
+
+class TestInfo(object):
+ """Base class for a XTF Test Info object.
+ It represents a tests info.json, in a more convenient form.
+ """
+
+ def __init__(self, test_json):
+ """Parse and verify 'test_json'.
+
+ May raise KeyError, TypeError or ValueError.
+ """
+ name = test_json["name"]
+ if not isinstance(name, basestring):
+ raise TypeError("Expected string for 'name', got '%s'"
+ % (type(name), ))
+ self.name = name
+
+ cat = test_json["category"]
+ if not isinstance(cat, basestring):
+ raise TypeError("Expected string for 'category', got '%s'"
+ % (type(cat), ))
+ if cat not in all_categories:
+ raise ValueError("Unknown category '%s'" % (cat, ))
+ self.cat = cat
+
+ def all_instances(self, env_filter = None, vary_filter = None):
+ """Return a list of TestInstances, for each supported environment.
+ Optionally filtered by env_filter. May return an empty list if
+ the filter doesn't match any supported environment.
+ """
+ raise NotImplementedError
diff --git a/xtf/xl_domu.py b/xtf/xl_domu.py
new file mode 100644
index 0000000..f76dbfe
--- /dev/null
+++ b/xtf/xl_domu.py
@@ -0,0 +1,121 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+"""XL DomU class"""
+########################################################################
+# Imports
+########################################################################
+
+import imp
+import os.path
+import time
+
+from subprocess import Popen, PIPE
+
+import pexpect
+
+from xtf.exceptions import RunnerError
+from xtf.logger import Logger
+
+########################################################################
+# Functions
+########################################################################
+
+def _run_cmd(args, quiet=False):
+ """Execute command using Popen"""
+ proc = Popen(args, stdout = PIPE, stderr = PIPE)
+ if not quiet:
+ Logger().log("Executing '%s'" % (" ".join(args), ))
+ _, stderr = proc.communicate()
+ return proc.returncode, _, stderr
+
+def _xl_create(xl_conf_file, paused, fg):
+ """Creates a XEN Domain using the XL toolstack"""
+ args = ['xl', 'create']
+ if paused:
+ args.append('-p')
+ if fg:
+ args.append('-F')
+ args.append(xl_conf_file)
+ ret, _, stderr = _run_cmd(args)
+ if ret:
+ raise RunnerError("_xl_create", ret, _, stderr)
+
+def _xl_dom_id(xl_dom_name):
+ """Returns the ID of a XEN domain specified by name"""
+ args = ['xl', 'domid', xl_dom_name]
+ ret, _, stderr = _run_cmd(args)
+ if ret:
+ raise RunnerError("_xl_dom_id", ret, _, stderr)
+ return long(_)
+
+def _xl_destroy(domid):
+ """Destroy the domain specified by domid"""
+ args = ['xl', 'destroy', str(domid)]
+ ret, _, stderr = _run_cmd(args)
+ if ret:
+ raise RunnerError("_xl_destroy", ret, _, stderr)
+
+def _xl_unpause(domid):
+ """Unpauses the domain specified by domid"""
+ args = ['xl', 'unpause', str(domid)]
+ ret, _, stderr = _run_cmd(args)
+ if ret:
+ raise RunnerError("_xl_unpause", ret, _, stderr)
+
+def _is_alive(domid):
+ """Checks if the domain is alive using xenstore."""
+ args = ['xenstore-exists', os.path.join('/local/domain', str(domid))]
+ ret = _run_cmd(args, True)[0]
+ return ret == 0
+
+
+########################################################################
+# Classes
+########################################################################
+
+class XLDomU(object):
+ """XEN DomU implementation using the XL toolstack"""
+
+ def __init__(self, conf):
+ super(XLDomU, self).__init__()
+ self.__xl_conf_file = conf
+ self.dom_id = 0
+ code = open(conf)
+ self.__config = imp.new_module(conf)
+ exec code in self.__config.__dict__
+ self.__console = None
+
+ def create(self, paused=True, fg=False):
+ """Creates the XEN domain."""
+ _xl_create(self.__xl_conf_file, paused, fg)
+ self.dom_id = _xl_dom_id(self.__config.name)
+
+ def cleanup(self, timeout=10):
+ """Destroys the domain."""
+
+ if self.dom_id == 0:
+ return True
+
+ for _ in xrange(timeout):
+ if not _is_alive(self.dom_id):
+ return True
+ time.sleep(1)
+
+ if _is_alive(self.dom_id):
+ _xl_destroy(self.dom_id)
+ self.dom_id = 0
+ return False
+
+ return True
+
+ def unpause(self):
+ """Unpauses the domain."""
+ _xl_unpause(self.dom_id)
+
+ def console(self, logfile=None):
+ """Creates the domain_console handler."""
+ if self.__console is None:
+ self.__console = pexpect.spawn('xl', ['console', str(self.dom_id)],
+ logfile=logfile)
+ return self.__console
--
2.7.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH XTF v2 2/4] xtf: Add executable test class
2018-12-28 18:42 [PATCH XTF v2 0/4] Add monitor tests to XTF Petre Pircalabu
2018-12-28 18:42 ` [PATCH XTF v2 1/4] xtf-runner: split into logical components Petre Pircalabu
@ 2018-12-28 18:42 ` Petre Pircalabu
2018-12-28 18:42 ` [PATCH XTF v2 3/4] xtf: Add monitor " Petre Pircalabu
` (2 subsequent siblings)
4 siblings, 0 replies; 11+ messages in thread
From: Petre Pircalabu @ 2018-12-28 18:42 UTC (permalink / raw)
To: xen-devel
Cc: Petre Pircalabu, Stefano Stabellini, Wei Liu,
Konrad Rzeszutek Wilk, George Dunlap, Andrew Cooper, Ian Jackson,
Tim Deegan, Julien Grall, Jan Beulich
The Executable test class runs on host (dom0). The class spawns a
process and searches the program output(stdio) for a specific pattern.
Signed-off-by: Petre Pircalabu <ppircalabu@bitdefender.com>
---
xtf/__init__.py | 2 +-
xtf/executable_test.py | 83 ++++++++++++++++++++++++++++++++++++++++++++++++++
xtf/suite.py | 5 ++-
3 files changed, 88 insertions(+), 2 deletions(-)
create mode 100644 xtf/executable_test.py
diff --git a/xtf/__init__.py b/xtf/__init__.py
index 889c1d5..07c269a 100644
--- a/xtf/__init__.py
+++ b/xtf/__init__.py
@@ -3,7 +3,7 @@
# All test categories
default_categories = set(("functional", "xsa"))
-non_default_categories = set(("special", "utility", "in-development"))
+non_default_categories = set(("special", "utility", "in-development", "host"))
all_categories = default_categories | non_default_categories
# All test environments
diff --git a/xtf/executable_test.py b/xtf/executable_test.py
new file mode 100644
index 0000000..31aa6e4
--- /dev/null
+++ b/xtf/executable_test.py
@@ -0,0 +1,83 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+"""
+Executable test classes
+
+Spawns a process and waits for a specific pattern
+"""
+
+import StringIO
+import pexpect
+
+from xtf.logger import Logger
+from xtf.test import TestInstance, TestInfo, TestResult
+
+class ExecutableTestInstance(TestInstance):
+ """Executable Test Instance"""
+ def __init__(self, name, cmd, args, pattern):
+ super(ExecutableTestInstance, self).__init__(name)
+
+ self._cmd = cmd
+ self._args = [x.encode('utf-8') for x in args]
+ self._pattern = [x.encode('utf-8') for x in pattern]
+ self._proc = None
+ self.env = "dom0"
+ self.output = StringIO.StringIO()
+
+ def __repr__(self):
+ return "test-%s-%s" %(self.env, self.name)
+
+ def wait_pattern(self, pattern):
+ """Expect the pattern given as parameter."""
+ return self._proc.expect(pattern + [pexpect.TIMEOUT, pexpect.EOF])
+
+ def set_up(self, opts, result):
+ self._proc = pexpect.spawn(self._cmd, self._args, logfile = self.output)
+ print self._cmd, self._args
+
+ if self._proc is None:
+ result.set(TestResult.ERROR)
+
+ def run(self, result):
+ """Executes the test instance"""
+ if self.wait_pattern(self._pattern) > len(self._pattern):
+ result.set(TestResult.FAILURE)
+ return
+
+ result.set(TestResult.SUCCESS)
+
+ def clean_up(self, result):
+ if self.output:
+ Logger().log(self.output.getvalue())
+ self.output.close()
+
+class ExecutableTestInfo(TestInfo):
+ """ Object representing a tests info.json, in a more convenient form. """
+
+ def __init__(self, test_json):
+ super(ExecutableTestInfo, self).__init__(test_json)
+ self.instance_class = ExecutableTestInstance
+
+ cmd = test_json["cmd"]
+ if not isinstance(cmd, (str, unicode)):
+ raise TypeError("Expected string for 'cmd', got '%s')"
+ % (type(cmd), ))
+ self.cmd = cmd
+
+ args = test_json["args"]
+ if not isinstance(args, list):
+ raise TypeError("Expected list for 'args', got '%s')"
+ % (type(args), ))
+ self.args = args
+
+ pattern = test_json["pattern"]
+ if not isinstance(pattern, list):
+ raise TypeError("Expected list for 'pattern', got '%s')"
+ % (type(pattern), ))
+ self.pattern = pattern
+
+ def all_instances(self, env_filter = None, vary_filter = None):
+ """Returns an ExecutableTestInstance object"""
+ return [self.instance_class(self.name, self.cmd, self.args,
+ self.pattern),]
diff --git a/xtf/suite.py b/xtf/suite.py
index ad7d30f..2e0727c 100644
--- a/xtf/suite.py
+++ b/xtf/suite.py
@@ -75,7 +75,10 @@ def gather_all_test_info():
try:
info_file = open(path.join("tests", test, "info.json"))
except IOError:
- continue
+ try:
+ info_file = open(path.join("tests", test, "host.json"))
+ except IOError:
+ continue
# Ignore tests which have bad JSON
try:
--
2.7.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH XTF v2 3/4] xtf: Add monitor test class
2018-12-28 18:42 [PATCH XTF v2 0/4] Add monitor tests to XTF Petre Pircalabu
2018-12-28 18:42 ` [PATCH XTF v2 1/4] xtf-runner: split into logical components Petre Pircalabu
2018-12-28 18:42 ` [PATCH XTF v2 2/4] xtf: Add executable test class Petre Pircalabu
@ 2018-12-28 18:42 ` Petre Pircalabu
2019-01-02 13:43 ` Wei Liu
2018-12-28 18:42 ` [PATCH XTF v2 4/4] xtf: Add monitor mem_access test Petre Pircalabu
2019-01-02 13:24 ` [PATCH XTF v2 0/4] Add monitor tests to XTF Wei Liu
4 siblings, 1 reply; 11+ messages in thread
From: Petre Pircalabu @ 2018-12-28 18:42 UTC (permalink / raw)
To: xen-devel
Cc: Petre Pircalabu, Stefano Stabellini, Wei Liu,
Konrad Rzeszutek Wilk, George Dunlap, Andrew Cooper, Ian Jackson,
Tim Deegan, Julien Grall, Jan Beulich
This class starts alongside the domain a monitor application which opens
an event channel corresponding to that domain and handles the received
requests.
Use the "monitor_args" key to pass test specific arguments to the
monitor application.
The arguments will be added in the test's Makefile using the
TEST-EXTRA-INFO variable.
Signed-off-by: Petre Pircalabu <ppircalabu@bitdefender.com>
---
Makefile | 6 +-
build/common.mk | 22 ++-
build/files.mk | 3 +
build/gen.mk | 12 ++
common/report.c | 8 -
docs/all-tests.dox | 3 +
include/monitor/monitor.h | 136 +++++++++++++
include/xtf/report.h | 8 +
monitor/Makefile | 20 ++
monitor/monitor.c | 481 ++++++++++++++++++++++++++++++++++++++++++++++
xtf/__init__.py | 2 +-
xtf/monitor_test.py | 132 +++++++++++++
xtf/utils.py | 17 ++
13 files changed, 838 insertions(+), 12 deletions(-)
create mode 100644 include/monitor/monitor.h
create mode 100644 monitor/Makefile
create mode 100644 monitor/monitor.c
create mode 100644 xtf/monitor_test.py
create mode 100644 xtf/utils.py
diff --git a/Makefile b/Makefile
index 15a865f..db28075 100644
--- a/Makefile
+++ b/Makefile
@@ -32,7 +32,9 @@ INSTALL_PROGRAM := $(INSTALL) -p
OBJCOPY := $(CROSS_COMPILE)objcopy
PYTHON := python
-export CC CPP INSTALL INSTALL_DATA INSTALL_DIR INSTALL_PROGRAM OBJCOPY PYTHON
+HOSTCC := gcc
+
+export CC CPP INSTALL INSTALL_DATA INSTALL_DIR INSTALL_PROGRAM OBJCOPY PYTHON HOSTCC
.PHONY: all
all:
@@ -51,7 +53,7 @@ install:
done
define all_sources
- find include/ arch/ common/ tests/ -name "*.[hcsS]"
+ find include/ arch/ common/ tests/ monitor/ -name "*.[hcsS]"
endef
.PHONY: cscope
diff --git a/build/common.mk b/build/common.mk
index b786ddf..1ec0fa4 100644
--- a/build/common.mk
+++ b/build/common.mk
@@ -1,4 +1,4 @@
-ALL_CATEGORIES := special functional xsa utility in-development
+ALL_CATEGORIES := special functional xsa utility in-development monitor
ALL_ENVIRONMENTS := pv64 pv32pae hvm64 hvm32pae hvm32pse hvm32
@@ -35,11 +35,20 @@ COMMON_AFLAGS-x86_64 := -m64
COMMON_CFLAGS-x86_32 := -m32
COMMON_CFLAGS-x86_64 := -m64
+#HOSTCFLAGS := -Wall -Werror
+HOSTCFLAGS :=
+HOSTLDFLAGS :=
+HOSTLDLIBS :=
+HOSTCFLAGS += -D__XEN_TOOLS__ -g -O3 -I$(ROOT)/include/monitor
+HOSTCFLAGS += -DXC_WANT_COMPAT_DEVICEMODEL_API -DXC_WANT_COMPAT_MAP_FOREIGN_API
+HOSTLDLIBS += -lxenctrl -lxenstore -lxenevtchn
+
defcfg-pv := $(ROOT)/config/default-pv.cfg.in
defcfg-hvm := $(ROOT)/config/default-hvm.cfg.in
obj-perarch :=
obj-perenv :=
+obj-monitor :=
include $(ROOT)/build/files.mk
@@ -90,8 +99,19 @@ DEPS-$(1) = $$(head-$(1)) \
endef
+# Setup monitor rules
+define MONITOR_setup
+DEPS-MONITOR = \
+ $$(obj-monitor:%.o=%-monitor.o)
+
+%-monitor.o: %.c
+ $$(HOSTCC) $$(HOSTCFLAGS) -c $$< -o $$@
+endef
+
$(foreach env,$(ALL_ENVIRONMENTS),$(eval $(call PERENV_setup,$(env))))
+$(eval $(call MONITOR_setup))
+
define move-if-changed
if ! cmp -s $(1) $(2); then mv -f $(1) $(2); else rm -f $(1); fi
endef
diff --git a/build/files.mk b/build/files.mk
index dfa27e4..972c797 100644
--- a/build/files.mk
+++ b/build/files.mk
@@ -54,3 +54,6 @@ $(foreach env,$(32BIT_ENVIRONMENTS),$(eval obj-$(env) += $(obj-32)))
# 64bit specific objects
obj-64 += $(ROOT)/arch/x86/entry_64.o
$(foreach env,$(64BIT_ENVIRONMENTS),$(eval obj-$(env) += $(obj-64)))
+
+# Monitor common objects
+obj-monitor += $(ROOT)/monitor/monitor.o
diff --git a/build/gen.mk b/build/gen.mk
index c19ca6a..1e6773a 100644
--- a/build/gen.mk
+++ b/build/gen.mk
@@ -32,6 +32,9 @@ CLASS ?= "xtf.domu_test.DomuTestInfo"
.PHONY: build
build: $(foreach env,$(TEST-ENVS),test-$(env)-$(NAME)) $(TEST-CFGS)
build: info.json
+ifeq (x$(CATEGORY),xmonitor)
+build: test-monitor-$(NAME)
+endif
MKINFO-OPTS := -n "$(NAME)"
MKINFO-OPTS += -c "$(CLASS)"
@@ -100,6 +103,15 @@ install-each-env: install-$(1) install-$(1).cfg
endef
$(foreach env,$(TEST-ENVS),$(eval $(call PERENV_build,$(env))))
+define MONITOR_build
+test-monitor-$(NAME): $(DEPS-MONITOR)
+ @echo $(obj-monitor)
+ @echo $(DEPS-MONITOR)
+ $(HOSTCC) $(HOSTLDFLAGS) $(DEPS-MONITOR) $(HOSTLDLIBS) -o $$@
+endef
+
+$(eval $(call MONITOR_build))
+
.PHONY: clean
clean:
find $(ROOT) \( -name "*.o" -o -name "*.d" \) -delete
diff --git a/common/report.c b/common/report.c
index ffdf098..745713a 100644
--- a/common/report.c
+++ b/common/report.c
@@ -2,14 +2,6 @@
#include <xtf/report.h>
#include <xtf/hypercall.h>
-enum test_status {
- STATUS_RUNNING, /**< Test not yet completed. */
- STATUS_SUCCESS, /**< Test was successful. */
- STATUS_SKIP, /**< Test cannot be completed. */
- STATUS_ERROR, /**< Issue with the test itself. */
- STATUS_FAILURE, /**< Issue with the tested matter. */
-};
-
/** Current status of this test. */
static enum test_status status;
diff --git a/docs/all-tests.dox b/docs/all-tests.dox
index 732d44c..11b7f41 100644
--- a/docs/all-tests.dox
+++ b/docs/all-tests.dox
@@ -145,4 +145,7 @@ enable BTS.
@subpage test-nested-svm - Nested SVM tests.
@subpage test-nested-vmx - Nested VT-x tests.
+
+
+@section index-monitor Monitor
*/
diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h
new file mode 100644
index 0000000..a8e13a8
--- /dev/null
+++ b/include/monitor/monitor.h
@@ -0,0 +1,136 @@
+/*
+ * XTF Monitor interface
+ */
+
+#ifndef XTF_MONITOR_H
+#define XTF_MONITOR_H
+
+#include <inttypes.h>
+#include <xenctrl.h>
+#include <xenevtchn.h>
+#include <xenstore.h>
+#include <xen/vm_event.h>
+
+/**
+ * The value was chosen to be greater than any VM_EVENT_REASON_*
+ */
+#define VM_EVENT_REASON_MAX 20
+
+/* Should be in sync with "test_status" from common/report.c */
+enum xtf_mon_status
+{
+ XTF_MON_STATUS_RUNNING,
+ XTF_MON_STATUS_SUCCESS,
+ XTF_MON_STATUS_SKIP,
+ XTF_MON_STATUS_ERROR,
+ XTF_MON_STATUS_FAILURE,
+};
+
+enum xtf_mon_log_level
+{
+ XTF_MON_LOG_LEVEL_FATAL,
+ XTF_MON_LOG_LEVEL_ERROR,
+ XTF_MON_LOG_LEVEL_WARNING,
+ XTF_MON_LOG_LEVEL_INFO,
+ XTF_MON_LOG_LEVEL_DEBUG,
+ XTF_MON_LOG_LEVEL_TRACE,
+};
+
+void xtf_log(enum xtf_mon_log_level lvl, const char *fmt, ...) __attribute__((__format__(__printf__, 2, 3)));
+
+#define XTF_MON_FATAL(format...) xtf_log(XTF_MON_LOG_LEVEL_FATAL, format)
+#define XTF_MON_ERROR(format...) xtf_log(XTF_MON_LOG_LEVEL_ERROR, format)
+#define XTF_MON_WARNING(format...) xtf_log(XTF_MON_LOG_LEVEL_WARNING, format)
+#define XTF_MON_INFO(format...) xtf_log(XTF_MON_LOG_LEVEL_INFO, format)
+#define XTF_MON_DEBUG(format...) xtf_log(XTF_MON_LOG_LEVEL_DEBUG, format)
+#define XTF_MON_TRACE(format...) xtf_log(XTF_MON_LOG_LEVEL_TRACE, format)
+
+typedef int (*vm_event_handler_t)(vm_event_request_t *, vm_event_response_t *);
+
+/** XTF VM Event Ring interface */
+typedef struct xtf_vm_event_ring
+{
+ /* Event channel handle */
+ xenevtchn_handle *xce_handle;
+
+ /* Event channel remote port */
+ xenevtchn_port_or_error_t remote_port;
+
+ /* Event channel local port */
+ evtchn_port_t local_port;
+
+ /* vm_event back ring */
+ vm_event_back_ring_t back_ring;
+
+ /* Shared ring page */
+ void *ring_page;
+} xtf_vm_event_ring_t;
+
+typedef struct xtf_monitor_ops
+{
+ /* Test specific setup */
+ int (*setup)(int, char*[]);
+
+ /* Test specific initialization */
+ int (*init)(void);
+
+ /* Test specific cleanup */
+ int (*cleanup)(void);
+
+ /* Returns the test's result */
+ int (*get_result)(void);
+} xtf_monitor_ops_t;
+
+/* XTF Monitor Driver */
+typedef struct xtf_monitor
+{
+ /* Domain ID */
+ domid_t domain_id;
+
+ /* LibXC intreface handle */
+ xc_interface *xch;
+
+ /* XEN store handle */
+ struct xs_handle *xsh;
+
+ /* XTF VM_EVENT ring */
+ xtf_vm_event_ring_t ring;
+
+ /* Log Level */
+ enum xtf_mon_log_level log_lvl;
+
+ /* Test status */
+ enum xtf_mon_status status;
+
+ /* Test Help message*/
+ const char * help_message;
+
+ /* Test specific operations */
+ xtf_monitor_ops_t ops;
+
+ /* Test specific VM_EVENT request handlers */
+ vm_event_handler_t handlers[VM_EVENT_REASON_MAX];
+
+} xtf_monitor_t;
+
+void usage();
+
+extern xtf_monitor_t *monitor;
+
+#define XTF_MONITOR(param) \
+static void __attribute__((constructor)) register_monitor_##param() \
+{ \
+ monitor = (xtf_monitor_t *)¶m; \
+}
+
+#endif /* XTF_MONITOR_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/include/xtf/report.h b/include/xtf/report.h
index 36517bd..08a9b53 100644
--- a/include/xtf/report.h
+++ b/include/xtf/report.h
@@ -24,6 +24,14 @@
* kept.
*/
+enum test_status {
+ STATUS_RUNNING, /**< Test not yet completed. */
+ STATUS_SUCCESS, /**< Test was successful. */
+ STATUS_SKIP, /**< Test cannot be completed. */
+ STATUS_ERROR, /**< Issue with the test itself. */
+ STATUS_FAILURE, /**< Issue with the tested matter. */
+};
+
/**
* Report test success.
*/
diff --git a/monitor/Makefile b/monitor/Makefile
new file mode 100644
index 0000000..64d4f8a
--- /dev/null
+++ b/monitor/Makefile
@@ -0,0 +1,20 @@
+.PHONY: all
+
+all: monitor
+
+HOSTCC ?= gcc
+
+OBJS = monitor.o
+
+#HOSTCFLAGS += -Wall -Werror
+HOSTCFLAGS += -D__XEN_TOOLS__ -g -O0
+HOSTCFLAGS += -DXC_WANT_COMPAT_DEVICEMODEL_API -DXC_WANT_COMPAT_MAP_FOREIGN_API
+
+%.o : %.c
+ $(HOSTCC) -c $(HOSTCFLAGS) $(HOSTCPPFLAGS) $< -o $@
+
+monitor: $(OBJS)
+ $(HOSTCC) -o $@ $^ -lxenctrl -lxenstore -lxenevtchn
+
+clean:
+ $(RM) $(OBJS) monitor
diff --git a/monitor/monitor.c b/monitor/monitor.c
new file mode 100644
index 0000000..d1741bc
--- /dev/null
+++ b/monitor/monitor.c
@@ -0,0 +1,481 @@
+/**
+ * @file monitor/monitor.c
+ *
+ * Common functions for test specific monitor applications.
+ */
+
+#include <errno.h>
+#include <monitor.h>
+#include <poll.h>
+#include <stdio.h>
+#include <string.h>
+#include <sys/mman.h>
+
+#define call_helper(func, ... ) ( (func) ? func(__VA_ARGS__) : 0 )
+
+static const char *status_to_str[] =
+{
+#define STA(x) [XTF_MON_STATUS_ ## x] = #x
+
+ STA(RUNNING),
+ STA(SUCCESS),
+ STA(SKIP),
+ STA(ERROR),
+ STA(FAILURE),
+
+#undef STA
+};
+
+static const char *log_level_to_str[] =
+{
+#define XTFMLL(x) [ XTF_MON_LOG_LEVEL_ ## x] = #x
+
+ XTFMLL(FATAL),
+ XTFMLL(ERROR),
+ XTFMLL(WARNING),
+ XTFMLL(INFO),
+ XTFMLL(DEBUG),
+ XTFMLL(TRACE),
+
+#undef XTFMLL
+};
+
+void xtf_log(enum xtf_mon_log_level lvl, const char *fmt, ...)
+{
+ va_list argptr;
+
+ if ( lvl < 0 || lvl > monitor->log_lvl )
+ return;
+
+ fprintf(stderr, "[%s]\t", log_level_to_str[lvl]);
+ va_start(argptr, fmt);
+ vfprintf(stderr, fmt, argptr);
+ va_end(argptr);
+}
+
+static void xtf_print_status(enum xtf_mon_status s)
+{
+ if ( s > XTF_MON_STATUS_RUNNING && s <= XTF_MON_STATUS_FAILURE )
+ printf("Test result: %s\n", status_to_str[s]);
+}
+
+void usage()
+{
+ fprintf(stderr, "%s", monitor->help_message);
+}
+
+xtf_monitor_t *monitor;
+
+static int xtf_monitor_init()
+{
+ int rc;
+
+ monitor->ring.ring_page = xc_monitor_enable(monitor->xch,
+ monitor->domain_id,
+ &monitor->ring.remote_port);
+ if ( !monitor->ring.ring_page )
+ {
+ XTF_MON_ERROR("Error enabling monitor\n");
+ return -1;
+ }
+
+ monitor->ring.xce_handle = xenevtchn_open(NULL, 0);
+ if ( !monitor->ring.xce_handle )
+ {
+ XTF_MON_ERROR("Failed to open XEN event channel\n");
+ return -1;
+ }
+
+ rc = xenevtchn_bind_interdomain(monitor->ring.xce_handle,
+ monitor->domain_id,
+ monitor->ring.remote_port);
+ if ( rc < 0 )
+ {
+ XTF_MON_ERROR("Failed to bind XEN event channel\n");
+ return rc;
+ }
+ monitor->ring.local_port = rc;
+
+ /* Initialise ring */
+ SHARED_RING_INIT((vm_event_sring_t *)monitor->ring.ring_page);
+ BACK_RING_INIT(&monitor->ring.back_ring,
+ (vm_event_sring_t *)monitor->ring.ring_page,
+ XC_PAGE_SIZE);
+
+ return 0;
+}
+
+static int xtf_monitor_cleanup()
+{
+ int rc;
+
+ if ( monitor->ring.ring_page )
+ munmap(monitor->ring.ring_page, XC_PAGE_SIZE);
+
+ rc = xc_monitor_disable(monitor->xch, monitor->domain_id);
+ if ( rc != 0 )
+ {
+ XTF_MON_INFO("Error disabling monitor\n");
+ return rc;
+ }
+
+ rc = xenevtchn_unbind(monitor->ring.xce_handle, monitor->ring.local_port);
+ if ( rc != 0 )
+ {
+ XTF_MON_INFO("Failed to unbind XEN event channel\n");
+ return rc;
+ }
+
+ rc = xenevtchn_close(monitor->ring.xce_handle);
+ if ( rc != 0 )
+ {
+ XTF_MON_INFO("Failed to close XEN event channel\n");
+ return rc;
+ }
+
+ return 0;
+}
+
+static int xtf_monitor_wait_for_event(unsigned long ms)
+{
+ int rc;
+ struct pollfd fds[2] = {
+ { .fd = xenevtchn_fd(monitor->ring.xce_handle), .events = POLLIN | POLLERR },
+ { .fd = xs_fileno(monitor->xsh), .events = POLLIN | POLLERR },
+ };
+
+ rc = poll(fds, 2, ms);
+
+ if ( rc < 0 )
+ return -errno;
+
+ if ( rc == 0 )
+ return 0;
+
+ if ( fds[0].revents )
+ {
+ int port = xenevtchn_pending(monitor->ring.xce_handle);
+ if ( port == -1 )
+ return -errno;
+
+ rc = xenevtchn_unmask(monitor->ring.xce_handle, port);
+ if ( rc != 0 )
+ return -errno;
+
+ return port;
+ }
+
+ if ( fds[1].revents )
+ {
+ if ( !xs_is_domain_introduced(monitor->xsh, monitor->domain_id) )
+ {
+ return 1;
+ }
+
+ return 0;
+ }
+
+ return -2; /* Error */
+}
+
+/*
+ * X86 control register names
+ */
+static const char* get_x86_ctrl_reg_name(uint32_t index)
+{
+ switch (index)
+ {
+ case VM_EVENT_X86_CR0:
+ return "CR0";
+ case VM_EVENT_X86_CR3:
+ return "CR3";
+ case VM_EVENT_X86_CR4:
+ return "CR4";
+ case VM_EVENT_X86_XCR0:
+ return "XCR0";
+ }
+ return "";
+}
+
+static void xtf_monitor_dump_request(enum xtf_mon_log_level lvl, vm_event_request_t *req)
+{
+ switch ( req->reason )
+ {
+ case VM_EVENT_REASON_MEM_ACCESS:
+ xtf_log(lvl, "PAGE ACCESS: %c%c%c for GFN %"PRIx64" (offset %06"
+ PRIx64") gla %016"PRIx64" (valid: %c; fault in gpt: %c; fault with gla: %c) (vcpu %u [%c], altp2m view %u)\n",
+ (req->u.mem_access.flags & MEM_ACCESS_R) ? 'r' : '-',
+ (req->u.mem_access.flags & MEM_ACCESS_W) ? 'w' : '-',
+ (req->u.mem_access.flags & MEM_ACCESS_X) ? 'x' : '-',
+ req->u.mem_access.gfn,
+ req->u.mem_access.offset,
+ req->u.mem_access.gla,
+ (req->u.mem_access.flags & MEM_ACCESS_GLA_VALID) ? 'y' : 'n',
+ (req->u.mem_access.flags & MEM_ACCESS_FAULT_IN_GPT) ? 'y' : 'n',
+ (req->u.mem_access.flags & MEM_ACCESS_FAULT_WITH_GLA) ? 'y': 'n',
+ req->vcpu_id,
+ (req->flags & VM_EVENT_FLAG_VCPU_PAUSED) ? 'p' : 'r',
+ req->altp2m_idx);
+ break;
+
+ case VM_EVENT_REASON_SOFTWARE_BREAKPOINT:
+ xtf_log(lvl, "Breakpoint: rip=%016"PRIx64", gfn=%"PRIx64" (vcpu %d)\n",
+ req->data.regs.x86.rip,
+ req->u.software_breakpoint.gfn,
+ req->vcpu_id);
+ break;
+
+ case VM_EVENT_REASON_PRIVILEGED_CALL:
+ xtf_log(lvl, "Privileged call: pc=%"PRIx64" (vcpu %d)\n",
+ req->data.regs.arm.pc,
+ req->vcpu_id);
+
+ case VM_EVENT_REASON_SINGLESTEP:
+ xtf_log(lvl, "Singlestep: rip=%016lx, vcpu %d, altp2m %u\n",
+ req->data.regs.x86.rip,
+ req->vcpu_id,
+ req->altp2m_idx);
+ break;
+
+ case VM_EVENT_REASON_DEBUG_EXCEPTION:
+ printf("Debug exception: rip=%016"PRIx64", vcpu %d. Type: %u. Length: %u\n",
+ req->data.regs.x86.rip,
+ req->vcpu_id,
+ req->u.debug_exception.type,
+ req->u.debug_exception.insn_length);
+ break;
+
+ case VM_EVENT_REASON_CPUID:
+ xtf_log(lvl, "CPUID executed: rip=%016"PRIx64", vcpu %d. Insn length: %"PRIu32" " \
+ "0x%"PRIx32" 0x%"PRIx32": EAX=0x%"PRIx64" EBX=0x%"PRIx64" ECX=0x%"PRIx64" EDX=0x%"PRIx64"\n",
+ req->data.regs.x86.rip,
+ req->vcpu_id,
+ req->u.cpuid.insn_length,
+ req->u.cpuid.leaf,
+ req->u.cpuid.subleaf,
+ req->data.regs.x86.rax,
+ req->data.regs.x86.rbx,
+ req->data.regs.x86.rcx,
+ req->data.regs.x86.rdx);
+ break;
+
+ case VM_EVENT_REASON_DESCRIPTOR_ACCESS:
+ xtf_log(lvl, "Descriptor access: rip=%016"PRIx64", vcpu %d: "\
+ "VMExit info=0x%"PRIx32", descriptor=%d, is write=%d\n",
+ req->data.regs.x86.rip,
+ req->vcpu_id,
+ req->u.desc_access.arch.vmx.instr_info,
+ req->u.desc_access.descriptor,
+ req->u.desc_access.is_write);
+ break;
+
+ case VM_EVENT_REASON_WRITE_CTRLREG:
+ xtf_log(lvl,"Control register written: rip=%016"PRIx64", vcpu %d: "
+ "reg=%s, old_value=%016"PRIx64", new_value=%016"PRIx64"\n",
+ req->data.regs.x86.rip,
+ req->vcpu_id,
+ get_x86_ctrl_reg_name(req->u.write_ctrlreg.index),
+ req->u.write_ctrlreg.old_value,
+ req->u.write_ctrlreg.new_value);
+ break;
+
+ case VM_EVENT_REASON_EMUL_UNIMPLEMENTED:
+ xtf_log(lvl, "Emulation unimplemented: rip=%016lx, vcpu %d:\n",
+ req->data.regs.x86.rip,
+ req->vcpu_id);
+ break;
+ }
+}
+
+static void xtf_vm_event_ring_get_request(xtf_vm_event_ring_t *evt, vm_event_request_t *req)
+{
+ vm_event_back_ring_t *back_ring;
+ RING_IDX req_cons;
+
+ back_ring = &evt->back_ring;
+ req_cons = back_ring->req_cons;
+
+ /* Copy request */
+ memcpy(req, RING_GET_REQUEST(back_ring, req_cons), sizeof(*req));
+ req_cons++;
+
+ /* Update ring */
+ back_ring->req_cons = req_cons;
+ back_ring->sring->req_event = req_cons + 1;
+}
+
+static void xtf_vm_event_ring_put_response(xtf_vm_event_ring_t *evt, vm_event_response_t *rsp)
+{
+ vm_event_back_ring_t *back_ring;
+ RING_IDX rsp_prod;
+
+ back_ring = &evt->back_ring;
+ rsp_prod = back_ring->rsp_prod_pvt;
+
+ /* Copy response */
+ memcpy(RING_GET_RESPONSE(back_ring, rsp_prod), rsp, sizeof(*rsp));
+ rsp_prod++;
+
+ /* Update ring */
+ back_ring->rsp_prod_pvt = rsp_prod;
+ RING_PUSH_RESPONSES(back_ring);
+}
+
+static int xtf_monitor_loop()
+{
+ vm_event_request_t req;
+ vm_event_response_t rsp;
+ int rc;
+
+ /*
+ * NOTE: The test harness waits for this message to unpause
+ * the monitored DOMU.
+ */
+ printf("Monitor initialization complete.\n");
+
+ for (;;)
+ {
+ rc = xtf_monitor_wait_for_event(100);
+ if ( rc < -1 )
+ {
+ XTF_MON_ERROR("Error getting event");
+ return rc;
+ }
+ else if ( rc == 1 )
+ {
+ XTF_MON_INFO("Domain %d exited\n", monitor->domain_id);
+ return 0;
+ }
+
+ while ( RING_HAS_UNCONSUMED_REQUESTS(&monitor->ring.back_ring) )
+ {
+ xtf_vm_event_ring_get_request(&monitor->ring, &req);
+
+ if ( req.version != VM_EVENT_INTERFACE_VERSION )
+ {
+ XTF_MON_ERROR("Error: vm_event interface version mismatch!\n");
+ return -1;
+ }
+
+ memset( &rsp, 0, sizeof (rsp) );
+ rsp.version = VM_EVENT_INTERFACE_VERSION;
+ rsp.vcpu_id = req.vcpu_id;
+ rsp.flags = (req.flags & VM_EVENT_FLAG_VCPU_PAUSED);
+ rsp.reason = req.reason;
+
+ rc = 0;
+
+ xtf_monitor_dump_request(XTF_MON_LOG_LEVEL_DEBUG, &req);
+
+ if ( req.reason >= VM_EVENT_REASON_MAX || !monitor->handlers[req.reason] )
+ XTF_MON_ERROR("Unhandled request: reason = %d\n", req.reason);
+ else
+ {
+ rc = monitor->handlers[req.reason](&req, &rsp);
+ if (rc)
+ return rc;
+
+ /* Put the response on the ring */
+ xtf_vm_event_ring_put_response(&monitor->ring, &rsp);
+ }
+ }
+ /* Tell Xen page is ready */
+ rc = xenevtchn_notify(monitor->ring.xce_handle, monitor->ring.local_port);
+ if ( rc )
+ {
+ XTF_MON_ERROR("Error resuming page");
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+int main(int argc, char* argv[])
+{
+ int rc;
+
+ monitor->status = XTF_MON_STATUS_RUNNING;
+ monitor->log_lvl = XTF_MON_LOG_LEVEL_ERROR;
+
+ /* test specific setup sequence */
+ rc = call_helper(monitor->ops.setup, argc, argv);
+ if ( rc )
+ {
+ monitor->status = XTF_MON_STATUS_ERROR;
+ goto e_exit;
+ }
+
+ monitor->xch = xc_interface_open(NULL, NULL, 0);
+ if ( !monitor->xch )
+ {
+ XTF_MON_FATAL("Error initialising xenaccess\n");
+ rc = -EINVAL;
+ monitor->status = XTF_MON_STATUS_ERROR;
+ goto e_exit;
+ }
+
+ monitor->xsh = xs_open(XS_OPEN_READONLY);
+ if ( !monitor->xsh )
+ {
+ XTF_MON_FATAL("Error opening XEN store\n");
+ rc = -EINVAL;
+ monitor->status = XTF_MON_STATUS_ERROR;
+ goto cleanup;
+ }
+
+ if ( !xs_watch( monitor->xsh, "@releaseDomain", "RELEASE_TOKEN") )
+ {
+ XTF_MON_FATAL("Error monitoring releaseDomain\n");
+ rc = -EINVAL;
+ monitor->status = XTF_MON_STATUS_ERROR;
+ goto cleanup;
+ }
+
+ /* test specific initialization sequence */
+ rc = xtf_monitor_init();
+ if ( !rc )
+ rc = call_helper(monitor->ops.init);
+ if ( rc )
+ {
+ monitor->status = XTF_MON_STATUS_ERROR;
+ goto cleanup;
+ }
+
+ /* Run test */
+ rc = xtf_monitor_loop();
+ if ( rc )
+ {
+ XTF_MON_ERROR("Error running test\n");
+ monitor->status = XTF_MON_STATUS_ERROR;
+ }
+ else
+ monitor->status = call_helper(monitor->ops.get_result);
+
+cleanup:
+ /* test specific cleanup sequence */
+ call_helper(monitor->ops.cleanup);
+ xtf_monitor_cleanup();
+ if ( monitor->xsh )
+ {
+ xs_unwatch(monitor->xsh, "@releaseDomain", "RELEASE_TOKEN");
+ xs_close(monitor->xsh);
+ monitor->xsh = NULL;
+ }
+
+ xc_interface_close(monitor->xch);
+
+e_exit:
+ xtf_print_status(monitor->status);
+ return rc;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xtf/__init__.py b/xtf/__init__.py
index 07c269a..e405013 100644
--- a/xtf/__init__.py
+++ b/xtf/__init__.py
@@ -3,7 +3,7 @@
# All test categories
default_categories = set(("functional", "xsa"))
-non_default_categories = set(("special", "utility", "in-development", "host"))
+non_default_categories = set(("special", "utility", "in-development", "host", "monitor"))
all_categories = default_categories | non_default_categories
# All test environments
diff --git a/xtf/monitor_test.py b/xtf/monitor_test.py
new file mode 100644
index 0000000..b9b010e
--- /dev/null
+++ b/xtf/monitor_test.py
@@ -0,0 +1,132 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+""" Monitor test classes.
+
+ The monitor test spawns an test monitor (event channel handler application)
+ instance and runs a DomU image which interacts with it.
+"""
+
+import os
+from subprocess import Popen
+
+from xtf.exceptions import RunnerError
+from xtf.domu_test import DomuTestInstance, DomuTestInfo
+from xtf.executable_test import ExecutableTestInstance
+from xtf.logger import Logger
+from xtf.test import TestResult, TestInstance
+from xtf.utils import XTFAsyncCall
+
+class MonitorTestInstance(TestInstance):
+ """Monitor test instance"""
+
+ def __init__(self, env, name, variation, monitor_args):
+ super(MonitorTestInstance, self).__init__(name)
+
+ self.env, self.variation = env, variation
+
+ if self.env is None:
+ raise RunnerError("No environment for '%s'" % (self.name, ))
+
+ self.monitor_args = monitor_args.replace("@@VM_PATH@@", self.vm_path())
+
+ self.domu_test = None
+ self.monitor_test = None
+
+ def vm_name(self):
+ """ Return the VM name as `xl` expects it. """
+ return repr(self)
+
+ def cfg_path(self):
+ """ Return the path to the `xl` config file for this test. """
+ return os.path.join("tests", self.name, repr(self) + ".cfg")
+
+ def __repr__(self):
+ if self.variation:
+ return "test-%s-%s~%s" % (self.env, self.name, self.variation)
+ return "test-%s-%s" % (self.env, self.name)
+
+ def vm_path(self):
+ """ Return the VM path. """
+ return os.path.join("tests", self.name, repr(self))
+
+ def monitor_path(self):
+ """ Return the path to the test's monitor app if applicable. """
+ return os.path.join("tests", self.name, "test-monitor-" + self.name)
+
+ def start_monitor(self, dom_id):
+ """ Starts the monitor application. """
+ cmd = [" ".join([self.monitor_path(), self.monitor_args, str(dom_id)])]
+ Logger().log("Executing '%s'" % (" ".join(cmd), ))
+ return Popen(cmd, shell=True)
+
+ def set_up(self, opts, result):
+ self.domu_test = DomuTestInstance(self.env, self.name, self.variation)
+ self.domu_test.set_up(opts, result)
+ if result != TestResult.SUCCESS:
+ return
+
+ monitor_cmd = ' '.join([self.monitor_path(), self.monitor_args,
+ str(self.domu_test.domu.dom_id)])
+
+ self.monitor_test = ExecutableTestInstance(self.name, '/bin/sh',
+ ['-c', monitor_cmd], "")
+ self.monitor_test.set_up(opts, result)
+ match = self.monitor_test.wait_pattern(['Monitor initialization complete.'])
+ if match != 0:
+ result.set(TestResult.CRASH)
+
+ def run(self, result):
+ t1 = XTFAsyncCall(target=self.domu_test.run, args=(result,))
+ t2 = XTFAsyncCall(target=self.monitor_test.wait_pattern,
+ args=(self.result_pattern(), ))
+
+ for th in (t1, t2):
+ th.start()
+
+ t1.join()
+ res = TestResult(t2.join())
+ if res > result:
+ result.set(str(res))
+
+
+ def clean_up(self, result):
+ if self.domu_test:
+ self.domu_test.clean_up(result)
+
+ if self.monitor_test:
+ self.monitor_test.clean_up(result)
+
+class MonitorTestInfo(DomuTestInfo):
+ """Monitor test info"""
+
+ def __init__(self, test_json):
+ super(MonitorTestInfo, self).__init__(test_json)
+ self.instance_class = MonitorTestInstance
+ self.monitor_args = self.extra['monitor_args']
+
+ def all_instances(self, env_filter = None, vary_filter = None):
+ """Return a list of TestInstances, for each supported environment.
+ Optionally filtered by env_filter. May return an empty list if
+ the filter doesn't match any supported environment.
+ """
+
+ if env_filter:
+ envs = set(env_filter).intersection(self.envs)
+ else:
+ envs = self.envs
+
+ if vary_filter:
+ variations = set(vary_filter).intersection(self.variations)
+ else:
+ variations = self.variations
+
+ res = []
+ if variations:
+ for env in envs:
+ for vary in variations:
+ res.append(self.instance_class(env, self.name, vary,
+ self.monitor_args))
+ else:
+ res = [ self.instance_class(env, self.name, None, self.monitor_args)
+ for env in envs ]
+ return res
diff --git a/xtf/utils.py b/xtf/utils.py
new file mode 100644
index 0000000..96c570b
--- /dev/null
+++ b/xtf/utils.py
@@ -0,0 +1,17 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+""" XTF utils """
+
+import threading
+
+class XTFAsyncCall(threading.Thread):
+ def __init__(self, group=None, target=None, name=None, args=(), kwargs={}):
+ super(XTFAsyncCall, self).__init__(group, target, name, args, kwargs)
+ self._return = None
+ def run(self):
+ if self._Thread__target is not None:
+ self._return = self._Thread__target(*self._Thread__args,
+ **self._Thread__kwargs)
+ def join(self):
+ threading.Thread.join(self)
+ return self._return
--
2.7.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 11+ messages in thread* Re: [PATCH XTF v2 3/4] xtf: Add monitor test class
2018-12-28 18:42 ` [PATCH XTF v2 3/4] xtf: Add monitor " Petre Pircalabu
@ 2019-01-02 13:43 ` Wei Liu
2019-01-03 11:47 ` Petre Ovidiu PIRCALABU
0 siblings, 1 reply; 11+ messages in thread
From: Wei Liu @ 2019-01-02 13:43 UTC (permalink / raw)
To: Petre Pircalabu
Cc: Stefano Stabellini, Wei Liu, Konrad Rzeszutek Wilk, George Dunlap,
Andrew Cooper, Ian Jackson, Tim Deegan, Julien Grall, Jan Beulich,
xen-devel
On Fri, Dec 28, 2018 at 08:42:41PM +0200, Petre Pircalabu wrote:
> This class starts alongside the domain a monitor application which opens
> an event channel corresponding to that domain and handles the received
> requests.
> Use the "monitor_args" key to pass test specific arguments to the
> monitor application.
> The arguments will be added in the test's Makefile using the
> TEST-EXTRA-INFO variable.
>
> Signed-off-by: Petre Pircalabu <ppircalabu@bitdefender.com>
> ---
> Makefile | 6 +-
> build/common.mk | 22 ++-
> build/files.mk | 3 +
> build/gen.mk | 12 ++
> common/report.c | 8 -
> docs/all-tests.dox | 3 +
> include/monitor/monitor.h | 136 +++++++++++++
> include/xtf/report.h | 8 +
> monitor/Makefile | 20 ++
> monitor/monitor.c | 481 ++++++++++++++++++++++++++++++++++++++++++++++
> xtf/__init__.py | 2 +-
> xtf/monitor_test.py | 132 +++++++++++++
> xtf/utils.py | 17 ++
> 13 files changed, 838 insertions(+), 12 deletions(-)
> create mode 100644 include/monitor/monitor.h
> create mode 100644 monitor/Makefile
> create mode 100644 monitor/monitor.c
> create mode 100644 xtf/monitor_test.py
> create mode 100644 xtf/utils.py
>
> diff --git a/Makefile b/Makefile
> index 15a865f..db28075 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -32,7 +32,9 @@ INSTALL_PROGRAM := $(INSTALL) -p
> OBJCOPY := $(CROSS_COMPILE)objcopy
> PYTHON := python
>
> -export CC CPP INSTALL INSTALL_DATA INSTALL_DIR INSTALL_PROGRAM OBJCOPY PYTHON
> +HOSTCC := gcc
> +
> +export CC CPP INSTALL INSTALL_DATA INSTALL_DIR INSTALL_PROGRAM OBJCOPY PYTHON HOSTCC
>
> .PHONY: all
> all:
> @@ -51,7 +53,7 @@ install:
> done
>
> define all_sources
> - find include/ arch/ common/ tests/ -name "*.[hcsS]"
> + find include/ arch/ common/ tests/ monitor/ -name "*.[hcsS]"
> endef
Not a maintainer of xtf, but I find this code placement strange. If
"monitor" is a just yet one category of tests, why is it not placed
under tests/ ?
>
> .PHONY: cscope
> diff --git a/build/common.mk b/build/common.mk
> index b786ddf..1ec0fa4 100644
> --- a/build/common.mk
> +++ b/build/common.mk
> @@ -1,4 +1,4 @@
> -ALL_CATEGORIES := special functional xsa utility in-development
> +ALL_CATEGORIES := special functional xsa utility in-development monitor
You use "monitor" here but "host" in previous patch. They don't match.
Wei.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH XTF v2 3/4] xtf: Add monitor test class
2019-01-02 13:43 ` Wei Liu
@ 2019-01-03 11:47 ` Petre Ovidiu PIRCALABU
0 siblings, 0 replies; 11+ messages in thread
From: Petre Ovidiu PIRCALABU @ 2019-01-03 11:47 UTC (permalink / raw)
To: Wei Liu
Cc: Stefano Stabellini, Konrad Rzeszutek Wilk, George Dunlap,
Andrew Cooper, Ian Jackson, Tim Deegan, Julien Grall, Jan Beulich,
xen-devel@lists.xenproject.org
On Wed, 2019-01-02 at 13:43 +0000, Wei Liu wrote:
> On Fri, Dec 28, 2018 at 08:42:41PM +0200, Petre Pircalabu wrote:
> >
> >
> > define all_sources
> > - find include/ arch/ common/ tests/ -name "*.[hcsS]"
> > + find include/ arch/ common/ tests/ monitor/ -name "*.[hcsS]"
> > endef
>
> Not a maintainer of xtf, but I find this code placement strange. If
> "monitor" is a just yet one category of tests, why is it not placed
> under tests/ ?
Here (in the src root dir) the monitor subdirectory contains the common
code which will be reused for all tests.
>
> >
> > .PHONY: cscope
> > diff --git a/build/common.mk b/build/common.mk
> > index b786ddf..1ec0fa4 100644
> > --- a/build/common.mk
> > +++ b/build/common.mk
> > @@ -1,4 +1,4 @@
> > -ALL_CATEGORIES := special functional xsa utility in-
> > development
> > +ALL_CATEGORIES := special functional xsa utility in-
> > development monitor
>
> You use "monitor" here but "host" in previous patch. They don't
> match.
>
> Wei.
The "monitor" test class is a compound test which executes in paralel a
"clasic" xtf test (loads DomU and runs to completion) and a "host" test
which runs an executable on Dom0. The final result is computed by
examining both exit codes.
In build/common.mk only the "monitor" class is added to ALL_CATEGORIES,
because, at least for now, I don't have an use-case for a "host" only
test.
Hope it clarifies the difference,
Petre
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH XTF v2 4/4] xtf: Add monitor mem_access test
2018-12-28 18:42 [PATCH XTF v2 0/4] Add monitor tests to XTF Petre Pircalabu
` (2 preceding siblings ...)
2018-12-28 18:42 ` [PATCH XTF v2 3/4] xtf: Add monitor " Petre Pircalabu
@ 2018-12-28 18:42 ` Petre Pircalabu
2019-01-02 14:27 ` Tamas K Lengyel
2019-01-02 13:24 ` [PATCH XTF v2 0/4] Add monitor tests to XTF Wei Liu
4 siblings, 1 reply; 11+ messages in thread
From: Petre Pircalabu @ 2018-12-28 18:42 UTC (permalink / raw)
To: xen-devel
Cc: Petre Pircalabu, Stefano Stabellini, Wei Liu,
Konrad Rzeszutek Wilk, George Dunlap, Andrew Cooper, Ian Jackson,
Tim Deegan, Julien Grall, Jan Beulich
The monitor application resets the execute permisions on a specific page
of the DOMU and handles the generated vm_event request.
Signed-off-by: Petre Pircalabu <ppircalabu@bitdefender.com>
---
docs/all-tests.dox | 1 +
tests/monitor-mem-access/Makefile | 14 ++++
tests/monitor-mem-access/main.c | 37 +++++++++
tests/monitor-mem-access/monitor.c | 159 +++++++++++++++++++++++++++++++++++++
4 files changed, 211 insertions(+)
create mode 100644 tests/monitor-mem-access/Makefile
create mode 100644 tests/monitor-mem-access/main.c
create mode 100644 tests/monitor-mem-access/monitor.c
diff --git a/docs/all-tests.dox b/docs/all-tests.dox
index 11b7f41..644cc71 100644
--- a/docs/all-tests.dox
+++ b/docs/all-tests.dox
@@ -148,4 +148,5 @@ enable BTS.
@section index-monitor Monitor
+@subpage test-monitor-mem-access - memory access monitor test
*/
diff --git a/tests/monitor-mem-access/Makefile b/tests/monitor-mem-access/Makefile
new file mode 100644
index 0000000..45aaedb
--- /dev/null
+++ b/tests/monitor-mem-access/Makefile
@@ -0,0 +1,14 @@
+include $(ROOT)/build/common.mk
+
+NAME := monitor-mem-access
+CATEGORY := monitor
+TEST-ENVS := hvm64
+CLASS := xtf.monitor_test.MonitorTestInfo
+TEST-EXTRA-INFO := monitor_args='--address=0x\$$(nm --defined-only @@VM_PATH@@ | grep test_fn | cut -d \ -f 1)'
+
+obj-perenv += main.o
+
+obj-monitor += monitor.o
+
+include $(ROOT)/build/gen.mk
+
diff --git a/tests/monitor-mem-access/main.c b/tests/monitor-mem-access/main.c
new file mode 100644
index 0000000..f5ef795
--- /dev/null
+++ b/tests/monitor-mem-access/main.c
@@ -0,0 +1,37 @@
+/**
+ * @file tests/monitor-mem-access/main.c
+ * @ref test-monitor-mem-access
+ *
+ * @page test-monitor-mem-access monitor-mem-access
+ */
+
+#include <xtf.h>
+
+const char test_title[] = "Test monitor mem-access event";
+
+static int count;
+
+static void __attribute__((section(".text.protected"))) __attribute__((noinline)) test_fn(void)
+{
+ count++;
+}
+
+void test_main(void)
+{
+ test_fn();
+
+ if ( count )
+ xtf_success(NULL);
+ else
+ xtf_failure(NULL);
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/tests/monitor-mem-access/monitor.c b/tests/monitor-mem-access/monitor.c
new file mode 100644
index 0000000..a008117
--- /dev/null
+++ b/tests/monitor-mem-access/monitor.c
@@ -0,0 +1,159 @@
+/**
+ * @file tests/emul-unimpl/monitor.c
+ */
+#include <errno.h>
+#include <getopt.h>
+#include <inttypes.h>
+#include <monitor.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/mman.h>
+#include <unistd.h>
+
+typedef struct mem_access_monitor
+{
+ xtf_monitor_t mon;
+ uint64_t address;
+ unsigned long gfn;
+} mem_access_monitor_t;
+
+static const char mem_access_test_help[] = \
+ "Usage: test-monitor-mem_access [options] <domid>\n"
+ "\t -a <address>: the start address of the execution protected page\n"
+ ;
+
+static int mem_access_setup(int argc, char *argv[]);
+static int mem_access_init();
+static int mem_access_get_result();
+
+static mem_access_monitor_t monitor_instance =
+{
+ .mon =
+ {
+ .help_message = mem_access_test_help,
+ .ops =
+ {
+ .setup = mem_access_setup,
+ .init = mem_access_init,
+ .get_result = mem_access_get_result,
+ }
+ }
+};
+
+static int test_mem_access(vm_event_request_t *req, vm_event_response_t *rsp);
+
+static int mem_access_setup(int argc, char *argv[])
+{
+ int ret, c;
+ static struct option long_options[] = {
+ {"help", no_argument, 0, 'h'},
+ {"address", required_argument, 0, 'a'},
+ {0, 0, 0, 0}
+ };
+ mem_access_monitor_t *pmon = (mem_access_monitor_t *)monitor;
+
+ if ( !pmon )
+ return -EINVAL;
+
+ if ( argc == 1 )
+ {
+ usage();
+ return -EINVAL;
+ }
+ while ( 1 )
+ {
+ int option_index = 0;
+ c = getopt_long(argc, argv, "ha:", long_options, &option_index);
+ if ( c == -1 ) break;
+
+ switch ( c )
+ {
+ case 'h':
+ usage();
+ exit(0);
+ break;
+ case 'a':
+ pmon->address = strtoul(optarg, NULL, 0);
+ break;
+ default:
+ XTF_MON_ERROR("%s: Invalid option %s\n", argv[0], optarg);
+ return -EINVAL;
+ }
+
+ if ( !pmon->address )
+ {
+ XTF_MON_ERROR("%s: Please specify a valid instruction injection address\n", argv[0]);
+ return -EINVAL;
+ }
+
+ if ( optind != argc - 1 )
+ {
+ XTF_MON_ERROR("%s: Please specify the domain id\n", argv[0]);
+ return -EINVAL;
+ }
+ }
+
+ monitor->domain_id = atoi(argv[optind]);
+
+ if ( monitor->domain_id <= 0 )
+ {
+ XTF_MON_ERROR("%s: Invalid domain id\n", argv[0]);
+ return -EINVAL;
+ }
+
+ monitor->handlers[VM_EVENT_REASON_MEM_ACCESS] = test_mem_access;
+
+ return 0;
+}
+
+static int mem_access_init()
+{
+ int rc = 0;
+ mem_access_monitor_t *pmon = (mem_access_monitor_t *)monitor;
+
+ if ( !pmon )
+ return -EINVAL;
+
+ rc = xc_domain_set_access_required(monitor->xch, monitor->domain_id, 1);
+ if ( rc < 0 )
+ {
+ XTF_MON_ERROR("Error %d setting mem_access listener required\n", rc);
+ return rc;
+ }
+
+ pmon->gfn = xc_translate_foreign_address(monitor->xch, monitor->domain_id,
+ 0, pmon->address);
+
+ rc = xc_set_mem_access(monitor->xch, monitor->domain_id, XENMEM_access_rw,
+ pmon->gfn, 1);
+ if ( rc < 0 )
+ {
+ XTF_MON_ERROR("Error %d setting memory access!\n", rc);
+ return rc;
+ }
+
+ return 0;
+}
+
+static int mem_access_get_result()
+{
+ return XTF_MON_STATUS_SUCCESS;
+}
+
+static int test_mem_access(vm_event_request_t *req, vm_event_response_t *rsp)
+{
+ return xc_set_mem_access(monitor->xch, monitor->domain_id,
+ XENMEM_access_rwx, req->u.mem_access.gfn, 1);
+}
+
+XTF_MONITOR(monitor_instance);
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
--
2.7.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 11+ messages in thread* Re: [PATCH XTF v2 4/4] xtf: Add monitor mem_access test
2018-12-28 18:42 ` [PATCH XTF v2 4/4] xtf: Add monitor mem_access test Petre Pircalabu
@ 2019-01-02 14:27 ` Tamas K Lengyel
2019-01-03 11:08 ` Petre Ovidiu PIRCALABU
0 siblings, 1 reply; 11+ messages in thread
From: Tamas K Lengyel @ 2019-01-02 14:27 UTC (permalink / raw)
To: Petre Pircalabu; +Cc: Xen-devel
> +static int mem_access_init()
> +{
> + int rc = 0;
> + mem_access_monitor_t *pmon = (mem_access_monitor_t *)monitor;
> +
> + if ( !pmon )
> + return -EINVAL;
> +
> + rc = xc_domain_set_access_required(monitor->xch, monitor->domain_id, 1);
> + if ( rc < 0 )
> + {
> + XTF_MON_ERROR("Error %d setting mem_access listener required\n", rc);
> + return rc;
> + }
> +
> + pmon->gfn = xc_translate_foreign_address(monitor->xch, monitor->domain_id,
> + 0, pmon->address);
Check that the gfn returned is not 0 before continuing.
> +
> + rc = xc_set_mem_access(monitor->xch, monitor->domain_id, XENMEM_access_rw,
> + pmon->gfn, 1);
> + if ( rc < 0 )
> + {
> + XTF_MON_ERROR("Error %d setting memory access!\n", rc);
> + return rc;
> + }
> +
> + return 0;
> +}
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [PATCH XTF v2 4/4] xtf: Add monitor mem_access test
2019-01-02 14:27 ` Tamas K Lengyel
@ 2019-01-03 11:08 ` Petre Ovidiu PIRCALABU
0 siblings, 0 replies; 11+ messages in thread
From: Petre Ovidiu PIRCALABU @ 2019-01-03 11:08 UTC (permalink / raw)
To: Tamas K Lengyel; +Cc: Xen-devel
On Wed, 2019-01-02 at 07:27 -0700, Tamas K Lengyel wrote:
> > +static int mem_access_init()
> > +{
> > + int rc = 0;
> > + mem_access_monitor_t *pmon = (mem_access_monitor_t *)monitor;
> > +
> > + if ( !pmon )
> > + return -EINVAL;
> > +
> > + rc = xc_domain_set_access_required(monitor->xch, monitor-
> > >domain_id, 1);
> > + if ( rc < 0 )
> > + {
> > + XTF_MON_ERROR("Error %d setting mem_access listener
> > required\n", rc);
> > + return rc;
> > + }
> > +
> > + pmon->gfn = xc_translate_foreign_address(monitor->xch,
> > monitor->domain_id,
> > + 0, pmon->address);
>
> Check that the gfn returned is not 0 before continuing.
I will correct it in the next iteration.
Thanks for noticing it,
Petre
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH XTF v2 0/4] Add monitor tests to XTF
2018-12-28 18:42 [PATCH XTF v2 0/4] Add monitor tests to XTF Petre Pircalabu
` (3 preceding siblings ...)
2018-12-28 18:42 ` [PATCH XTF v2 4/4] xtf: Add monitor mem_access test Petre Pircalabu
@ 2019-01-02 13:24 ` Wei Liu
2019-01-03 10:59 ` Petre Ovidiu PIRCALABU
4 siblings, 1 reply; 11+ messages in thread
From: Wei Liu @ 2019-01-02 13:24 UTC (permalink / raw)
To: Petre Pircalabu
Cc: Stefano Stabellini, Wei Liu, Konrad Rzeszutek Wilk, George Dunlap,
Andrew Cooper, Ian Jackson, Tim Deegan, Julien Grall, Jan Beulich,
xen-devel
I think you've called get_maintainers.pl on the wrong files -- it is not
applicable to xtf.git.
Wei.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [PATCH XTF v2 0/4] Add monitor tests to XTF
2019-01-02 13:24 ` [PATCH XTF v2 0/4] Add monitor tests to XTF Wei Liu
@ 2019-01-03 10:59 ` Petre Ovidiu PIRCALABU
0 siblings, 0 replies; 11+ messages in thread
From: Petre Ovidiu PIRCALABU @ 2019-01-03 10:59 UTC (permalink / raw)
To: Wei Liu
Cc: Stefano Stabellini, Konrad Rzeszutek Wilk, George Dunlap,
Andrew Cooper, Ian Jackson, Tim Deegan, Julien Grall, Jan Beulich,
xen-devel@lists.xenproject.org
On Wed, 2019-01-02 at 13:24 +0000, Wei Liu wrote:
> I think you've called get_maintainers.pl on the wrong files -- it is
> not
> applicable to xtf.git.
>
> Wei.
Oh, sorry about that. I will be more careful in the future.
//Petre
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 11+ messages in thread