* [XTF PATCH v2 1/4] gitignore: ignore vim swp file
2016-07-22 9:43 [XTF PATCH v2 0/4] Runner exit code clean up Wei Liu
@ 2016-07-22 9:43 ` Wei Liu
2016-07-22 9:43 ` [XTF PATCH v2 2/4] xtf-runner: sync all test results Wei Liu
` (2 subsequent siblings)
3 siblings, 0 replies; 9+ messages in thread
From: Wei Liu @ 2016-07-22 9:43 UTC (permalink / raw)
To: Xen-devel; +Cc: Wei Liu, Ian Jackson, andrew.cooper3
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
.gitignore | 1 +
1 file changed, 1 insertion(+)
diff --git a/.gitignore b/.gitignore
index 01243f0..56e1884 100644
--- a/.gitignore
+++ b/.gitignore
@@ -3,6 +3,7 @@
*.d
*.pyc
*.pyo
+*.swp
/arch/x86/*.lds
/cscope.*
/dist/
--
2.1.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 9+ messages in thread* [XTF PATCH v2 2/4] xtf-runner: sync all test results
2016-07-22 9:43 [XTF PATCH v2 0/4] Runner exit code clean up Wei Liu
2016-07-22 9:43 ` [XTF PATCH v2 1/4] gitignore: ignore vim swp file Wei Liu
@ 2016-07-22 9:43 ` Wei Liu
2016-07-22 9:43 ` [XTF PATCH v2 3/4] xtf-runner: provide a set of exit codes for different results Wei Liu
2016-07-22 9:43 ` [XTF PATCH v2 4/4] xtf-runner: regularise runner exit code Wei Liu
3 siblings, 0 replies; 9+ messages in thread
From: Wei Liu @ 2016-07-22 9:43 UTC (permalink / raw)
To: Xen-devel; +Cc: Wei Liu, Ian Jackson, andrew.cooper3
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
xtf-runner | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/xtf-runner b/xtf-runner
index 50a5e96..c938f28 100755
--- a/xtf-runner
+++ b/xtf-runner
@@ -17,6 +17,9 @@ try:
except ImportError:
import simplejson as json
+# All results of a test, keep in sync with C code report.h.
+# Note that warning is not a result on its own.
+all_results = ('SUCCESS', 'SKIP', 'ERROR', 'FAILURE')
# All test categories and configurations
all_categories = ("special", "functional", "xsa", "utility")
@@ -161,7 +164,7 @@ def run_test(test):
if not "Test result:" in test_result:
return "ERROR"
- for res in ("SUCCESS", "SKIP", "FAILURE"):
+ for res in all_results:
if res in test_result:
return res
--
2.1.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 9+ messages in thread* [XTF PATCH v2 3/4] xtf-runner: provide a set of exit codes for different results
2016-07-22 9:43 [XTF PATCH v2 0/4] Runner exit code clean up Wei Liu
2016-07-22 9:43 ` [XTF PATCH v2 1/4] gitignore: ignore vim swp file Wei Liu
2016-07-22 9:43 ` [XTF PATCH v2 2/4] xtf-runner: sync all test results Wei Liu
@ 2016-07-22 9:43 ` Wei Liu
2016-07-22 9:43 ` [XTF PATCH v2 4/4] xtf-runner: regularise runner exit code Wei Liu
3 siblings, 0 replies; 9+ messages in thread
From: Wei Liu @ 2016-07-22 9:43 UTC (permalink / raw)
To: Xen-devel; +Cc: Wei Liu, Ian Jackson, andrew.cooper3
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
xtf-runner | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/xtf-runner b/xtf-runner
index c938f28..1c96750 100755
--- a/xtf-runner
+++ b/xtf-runner
@@ -21,6 +21,16 @@ except ImportError:
# Note that warning is not a result on its own.
all_results = ('SUCCESS', 'SKIP', 'ERROR', 'FAILURE')
+# Return the exit code for different states. Avoid using 1 and 2 because
+# python interpreter uses them -- see document for sys.exit.
+def exit_code(state):
+ """ Convert a test result to an xtf-runner exit code. """
+ return { "SUCCESS": 0,
+ "SKIP": 3,
+ "ERROR": 4,
+ "FAILURE": 5,
+ }.get(state, 4)
+
# All test categories and configurations
all_categories = ("special", "functional", "xsa", "utility")
pv_environments = ("pv64", "pv32pae")
--
2.1.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 9+ messages in thread* [XTF PATCH v2 4/4] xtf-runner: regularise runner exit code
2016-07-22 9:43 [XTF PATCH v2 0/4] Runner exit code clean up Wei Liu
` (2 preceding siblings ...)
2016-07-22 9:43 ` [XTF PATCH v2 3/4] xtf-runner: provide a set of exit codes for different results Wei Liu
@ 2016-07-22 9:43 ` Wei Liu
2016-07-22 9:49 ` Andrew Cooper
3 siblings, 1 reply; 9+ messages in thread
From: Wei Liu @ 2016-07-22 9:43 UTC (permalink / raw)
To: Xen-devel; +Cc: Wei Liu, Ian Jackson, andrew.cooper3
The script now returns the most severe result. Document the exit code in
help string.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
xtf-runner | 22 ++++++++++++++++++----
1 file changed, 18 insertions(+), 4 deletions(-)
diff --git a/xtf-runner b/xtf-runner
index 1c96750..15b98c6 100755
--- a/xtf-runner
+++ b/xtf-runner
@@ -251,23 +251,30 @@ def run_tests(args):
if not len(tests):
raise RunnerError("No tests to run")
- rc = 0
+ rc = all_results.index('SUCCESS')
results = []
for test in tests:
res = run_test(test)
- if res != "SUCCESS":
- rc = 1
+ res_idx = all_results.index(res);
+ if res_idx > rc:
+ rc = res_idx
results.append(res)
+ if rc == exit_code('SUCCESS'):
+ for res in results:
+ if res == 'SKIP':
+ rc = exit_code('SKIP')
+ break
+
print "\nCombined test results:"
for test, res in zip(tests, results):
print "%-40s %s" % (test, res)
- return rc
+ return exit_code(all_results[rc])
def main():
@@ -308,6 +315,13 @@ def main():
" List all 'functional' or 'special' tests\n"
" ./xtf-runner --list hvm64\n"
" List all 'hvm64' tests\n"
+ "\n"
+ " Exit code for this script:\n"
+ " 0: everythin is ok\n"
+ " 1,2: reserved for python interpreter\n"
+ " 3: test(s) are skipped\n"
+ " 4: test(s) report error\n"
+ " 5: test(s) report failure\n"
),
)
--
2.1.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 9+ messages in thread* Re: [XTF PATCH v2 4/4] xtf-runner: regularise runner exit code
2016-07-22 9:43 ` [XTF PATCH v2 4/4] xtf-runner: regularise runner exit code Wei Liu
@ 2016-07-22 9:49 ` Andrew Cooper
2016-07-22 9:51 ` Wei Liu
0 siblings, 1 reply; 9+ messages in thread
From: Andrew Cooper @ 2016-07-22 9:49 UTC (permalink / raw)
To: Wei Liu, Xen-devel; +Cc: Ian Jackson
On 22/07/16 10:43, Wei Liu wrote:
> The script now returns the most severe result. Document the exit code in
> help string.
>
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> ---
> xtf-runner | 22 ++++++++++++++++++----
> 1 file changed, 18 insertions(+), 4 deletions(-)
>
> diff --git a/xtf-runner b/xtf-runner
> index 1c96750..15b98c6 100755
> --- a/xtf-runner
> +++ b/xtf-runner
> @@ -251,23 +251,30 @@ def run_tests(args):
> if not len(tests):
> raise RunnerError("No tests to run")
>
> - rc = 0
> + rc = all_results.index('SUCCESS')
> results = []
>
> for test in tests:
>
> res = run_test(test)
> - if res != "SUCCESS":
> - rc = 1
> + res_idx = all_results.index(res);
> + if res_idx > rc:
> + rc = res_idx
>
> results.append(res)
>
> + if rc == exit_code('SUCCESS'):
> + for res in results:
> + if res == 'SKIP':
> + rc = exit_code('SKIP')
> + break
Why is this conditional needed? SKIP has index 1 so will automatically
displace SUCCESS in the change above.
> +
> print "\nCombined test results:"
>
> for test, res in zip(tests, results):
> print "%-40s %s" % (test, res)
>
> - return rc
> + return exit_code(all_results[rc])
>
>
> def main():
> @@ -308,6 +315,13 @@ def main():
> " List all 'functional' or 'special' tests\n"
> " ./xtf-runner --list hvm64\n"
> " List all 'hvm64' tests\n"
> + "\n"
> + " Exit code for this script:\n"
> + " 0: everythin is ok\n"
everything
~Andrew
> + " 1,2: reserved for python interpreter\n"
> + " 3: test(s) are skipped\n"
> + " 4: test(s) report error\n"
> + " 5: test(s) report failure\n"
> ),
> )
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [XTF PATCH v2 4/4] xtf-runner: regularise runner exit code
2016-07-22 9:49 ` Andrew Cooper
@ 2016-07-22 9:51 ` Wei Liu
2016-07-22 9:58 ` Wei Liu
0 siblings, 1 reply; 9+ messages in thread
From: Wei Liu @ 2016-07-22 9:51 UTC (permalink / raw)
To: Andrew Cooper; +Cc: Xen-devel, Wei Liu, Ian Jackson
On Fri, Jul 22, 2016 at 10:49:18AM +0100, Andrew Cooper wrote:
> On 22/07/16 10:43, Wei Liu wrote:
>
> >The script now returns the most severe result. Document the exit code in
> >help string.
> >
> >Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> >---
> > xtf-runner | 22 ++++++++++++++++++----
> > 1 file changed, 18 insertions(+), 4 deletions(-)
> >
> >diff --git a/xtf-runner b/xtf-runner
> >index 1c96750..15b98c6 100755
> >--- a/xtf-runner
> >+++ b/xtf-runner
> >@@ -251,23 +251,30 @@ def run_tests(args):
> > if not len(tests):
> > raise RunnerError("No tests to run")
> >- rc = 0
> >+ rc = all_results.index('SUCCESS')
> > results = []
> > for test in tests:
> > res = run_test(test)
> >- if res != "SUCCESS":
> >- rc = 1
> >+ res_idx = all_results.index(res);
> >+ if res_idx > rc:
> >+ rc = res_idx
> > results.append(res)
> >+ if rc == exit_code('SUCCESS'):
> >+ for res in results:
> >+ if res == 'SKIP':
> >+ rc = exit_code('SKIP')
> >+ break
>
> Why is this conditional needed? SKIP has index 1 so will automatically
> displace SUCCESS in the change above.
>
Forgot to delete that hunk. I will resend.
> >+
> > print "\nCombined test results:"
> > for test, res in zip(tests, results):
> > print "%-40s %s" % (test, res)
> >- return rc
> >+ return exit_code(all_results[rc])
> > def main():
> >@@ -308,6 +315,13 @@ def main():
> > " List all 'functional' or 'special' tests\n"
> > " ./xtf-runner --list hvm64\n"
> > " List all 'hvm64' tests\n"
> >+ "\n"
> >+ " Exit code for this script:\n"
> >+ " 0: everythin is ok\n"
>
> everything
Fixed.
Wei.
>
> ~Andrew
>
> >+ " 1,2: reserved for python interpreter\n"
> >+ " 3: test(s) are skipped\n"
> >+ " 4: test(s) report error\n"
> >+ " 5: test(s) report failure\n"
> > ),
> > )
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [XTF PATCH v2 4/4] xtf-runner: regularise runner exit code
2016-07-22 9:51 ` Wei Liu
@ 2016-07-22 9:58 ` Wei Liu
2016-07-22 10:14 ` Andrew Cooper
0 siblings, 1 reply; 9+ messages in thread
From: Wei Liu @ 2016-07-22 9:58 UTC (permalink / raw)
To: Andrew Cooper; +Cc: Xen-devel, Wei Liu, Ian Jackson
Updated this patch
---8<---
From ef1f9ddfa797bb4095d48b90efc3c92a0a8fd1b6 Mon Sep 17 00:00:00 2001
From: Wei Liu <wei.liu2@citrix.com>
Date: Mon, 13 Jun 2016 15:06:48 +0100
Subject: [XTF PATCH] xtf-runner: regularise runner exit code
Cc: andrew.cooper3@citrix.com
The script now returns the most severe result. Document the exit code in
help string.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
xtf-runner | 16 ++++++++++++----
1 file changed, 12 insertions(+), 4 deletions(-)
diff --git a/xtf-runner b/xtf-runner
index 1c96750..7743316 100755
--- a/xtf-runner
+++ b/xtf-runner
@@ -251,14 +251,15 @@ def run_tests(args):
if not len(tests):
raise RunnerError("No tests to run")
- rc = 0
+ rc = all_results.index('SUCCESS')
results = []
for test in tests:
res = run_test(test)
- if res != "SUCCESS":
- rc = 1
+ res_idx = all_results.index(res);
+ if res_idx > rc:
+ rc = res_idx
results.append(res)
@@ -267,7 +268,7 @@ def run_tests(args):
for test, res in zip(tests, results):
print "%-40s %s" % (test, res)
- return rc
+ return exit_code(all_results[rc])
def main():
@@ -308,6 +309,13 @@ def main():
" List all 'functional' or 'special' tests\n"
" ./xtf-runner --list hvm64\n"
" List all 'hvm64' tests\n"
+ "\n"
+ " Exit code for this script:\n"
+ " 0: everything is ok\n"
+ " 1,2: reserved for python interpreter\n"
+ " 3: test(s) are skipped\n"
+ " 4: test(s) report error\n"
+ " 5: test(s) report failure\n"
),
)
--
2.1.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 9+ messages in thread