* [PATCH 01/16] testsuite: make the '%.t' rule depends on PROGRAMS too
2017-12-08 13:14 [PATCH 00/16] testsuite improvements Luc Van Oostenryck
@ 2017-12-08 13:14 ` Luc Van Oostenryck
2017-12-08 13:14 ` [PATCH 02/16] testsuite: fix a few more incorrect check-commands Luc Van Oostenryck
` (15 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: Luc Van Oostenryck @ 2017-12-08 13:14 UTC (permalink / raw)
To: linux-sparse; +Cc: Luc Van Oostenryck
The testsuite can be run on a specific testcase directly
via the Makefile via a 'validation/%.t' pattern rule but
this rule had no dependency on the programs being tested
and thus could be run on a previous version.
Fix this by adding the needed dependencies.
Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
---
Makefile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Makefile b/Makefile
index 1ae0a33f0..dab38aacd 100644
--- a/Makefile
+++ b/Makefile
@@ -214,7 +214,7 @@ version.h: FORCE
check: all
$(Q)cd validation && ./test-suite
-validation/%.t: FORCE
+validation/%.t: $(PROGRAMS)
@validation/test-suite single $*.c
--
2.15.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH 02/16] testsuite: fix a few more incorrect check-commands
2017-12-08 13:14 [PATCH 00/16] testsuite improvements Luc Van Oostenryck
2017-12-08 13:14 ` [PATCH 01/16] testsuite: make the '%.t' rule depends on PROGRAMS too Luc Van Oostenryck
@ 2017-12-08 13:14 ` Luc Van Oostenryck
2017-12-08 13:14 ` [PATCH 03/16] testsuite: convert to the new pattern syntax Luc Van Oostenryck
` (14 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: Luc Van Oostenryck @ 2017-12-08 13:14 UTC (permalink / raw)
To: linux-sparse; +Cc: Luc Van Oostenryck
The exact syntax for commands is:
'check-command: ' <command> <args>...
and the command itself must *not* be prefixed with './'.
Fix the last three that had it wrong.
Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
---
validation/fp-ops.c | 2 +-
validation/option-parsing-00.c | 2 +-
validation/option-parsing-01.c | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/validation/fp-ops.c b/validation/fp-ops.c
index 7f58a72fc..71cb9848b 100644
--- a/validation/fp-ops.c
+++ b/validation/fp-ops.c
@@ -7,7 +7,7 @@ _Bool ftst(double x) { return !x; }
/*
* check-name: floating-point ops
- * check-command: ./test-linearize -Wno-decl $file
+ * check-command: test-linearize -Wno-decl $file
* check-output-start
fadd:
diff --git a/validation/option-parsing-00.c b/validation/option-parsing-00.c
index 9b85943c1..9dceab7fa 100644
--- a/validation/option-parsing-00.c
+++ b/validation/option-parsing-00.c
@@ -1,5 +1,5 @@
/*
* check-name: option parsing 00
- * check-command sparse -foptimize-xyz $file
+ * check-command: sparse -foptimize-xyz $file
*/
diff --git a/validation/option-parsing-01.c b/validation/option-parsing-01.c
index e33a2ef0b..a2875bd15 100644
--- a/validation/option-parsing-01.c
+++ b/validation/option-parsing-01.c
@@ -1,5 +1,5 @@
/*
* check-name: option parsing 01
- * check-command sparse -fno-optimize-xyz $file
+ * check-command: sparse -fno-optimize-xyz $file
*/
--
2.15.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH 03/16] testsuite: convert to the new pattern syntax
2017-12-08 13:14 [PATCH 00/16] testsuite improvements Luc Van Oostenryck
2017-12-08 13:14 ` [PATCH 01/16] testsuite: make the '%.t' rule depends on PROGRAMS too Luc Van Oostenryck
2017-12-08 13:14 ` [PATCH 02/16] testsuite: fix a few more incorrect check-commands Luc Van Oostenryck
@ 2017-12-08 13:14 ` Luc Van Oostenryck
2017-12-08 13:14 ` [PATCH 04/16] testsuite: remove old ugly " Luc Van Oostenryck
` (13 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: Luc Van Oostenryck @ 2017-12-08 13:14 UTC (permalink / raw)
To: linux-sparse; +Cc: Luc Van Oostenryck
The old one is too ugly and has to die.
Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
---
validation/fp2i-cast.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/validation/fp2i-cast.c b/validation/fp2i-cast.c
index 08f8c9252..98d7453e5 100644
--- a/validation/fp2i-cast.c
+++ b/validation/fp2i-cast.c
@@ -26,5 +26,5 @@ ul d2ul(double a) { return a; }
* check-command: test-linearize -Wno-decl $file
*
* check-output-ignore
- * check-output-pattern-8-times: cast\\.
+ * check-output-pattern(8): cast\\.
*/
--
2.15.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH 04/16] testsuite: remove old ugly pattern syntax
2017-12-08 13:14 [PATCH 00/16] testsuite improvements Luc Van Oostenryck
` (2 preceding siblings ...)
2017-12-08 13:14 ` [PATCH 03/16] testsuite: convert to the new pattern syntax Luc Van Oostenryck
@ 2017-12-08 13:14 ` Luc Van Oostenryck
2017-12-08 13:14 ` [PATCH 05/16] testsuite: move verbose/error() before get_tag_value() Luc Van Oostenryck
` (12 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: Luc Van Oostenryck @ 2017-12-08 13:14 UTC (permalink / raw)
To: linux-sparse; +Cc: Luc Van Oostenryck
It was too ugly (and a bit longish).
Remove it.
Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
---
validation/test-suite | 32 --------------------------------
1 file changed, 32 deletions(-)
diff --git a/validation/test-suite b/validation/test-suite
index 903a2b908..07d9e7fc9 100755
--- a/validation/test-suite
+++ b/validation/test-suite
@@ -55,7 +55,6 @@ get_tag_value()
check_output_ignore=0
check_output_contains=0
check_output_excludes=0
- check_output_pattern_obsolete=0
check_output_pattern=0
check_arch_ignore=""
check_arch_only=""
@@ -77,7 +76,6 @@ get_tag_value()
check-output-ignore) check_output_ignore=1 ;;
check-output-contains:) check_output_contains=1 ;;
check-output-excludes:) check_output_excludes=1 ;;
- check-output-pattern-) check_output_pattern_obsolete=1 ;;
check-output-pattern) check_output_pattern=1 ;;
check-arch-ignore:) arch=$(uname -m)
check_arch_ignore="$val" ;;
@@ -149,29 +147,6 @@ has_none_patterns()
has_patterns "$1" "$2" "$4" -eq "$3"
}
-##
-# nbr_patterns(ifile tag ofile) - does ofile contains the
-# the patterns given by ifile's tags
-# the right number of time?
-nbr_patterns()
-{
- ifile="$1"
- patt="$2"
- ofile="$3"
- grep "$patt-[0-9][0-9]*-times:" "$ifile" | \
- sed -e "s/^.*$patt-\([0-9][0-9]*\)-times: *\(.*\)/\1 \2/" | \
- while read nbr pat; do
- n=$(grep -s "$pat" "$ofile" | wc -l)
- if [ "$n" -ne "$nbr" ]; then
- error "test '$ifile' failed"
- error " Pattern '$pat' expected $nbr times but got $n times"
- return 1
- fi
- done
-
- return $?
-}
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH 05/16] testsuite: move verbose/error() before get_tag_value()
2017-12-08 13:14 [PATCH 00/16] testsuite improvements Luc Van Oostenryck
` (3 preceding siblings ...)
2017-12-08 13:14 ` [PATCH 04/16] testsuite: remove old ugly " Luc Van Oostenryck
@ 2017-12-08 13:14 ` Luc Van Oostenryck
2017-12-08 13:14 ` [PATCH 06/16] testsuite: add & use warning() Luc Van Oostenryck
` (11 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: Luc Van Oostenryck @ 2017-12-08 13:14 UTC (permalink / raw)
To: linux-sparse; +Cc: Luc Van Oostenryck
So, we can use them inside get_tag_value().
Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
---
validation/test-suite | 36 +++++++++++++++++++-----------------
1 file changed, 19 insertions(+), 17 deletions(-)
diff --git a/validation/test-suite b/validation/test-suite
index 07d9e7fc9..f04d81779 100755
--- a/validation/test-suite
+++ b/validation/test-suite
@@ -42,6 +42,25 @@ vquiet=""
quiet=0
abort=0
+
+##
+# verbose(string) - prints string if we are in verbose mode
+verbose()
+{
+ [ "$V" -eq "1" ] && echo " $1"
+ return 0
+}
+
+##
+# error(string[, die]) - prints an error and exits with value die if given
+error()
+{
+ [ "$quiet" -ne 1 ] && echo "error: $1"
+ [ -n "$2" ] && exit $2
+ return 0
+}
+
+
##
# get_tag_value(file) - get the 'check-<...>' tags & values
get_tag_value()
@@ -87,23 +106,6 @@ get_tag_value()
EOT
}
-##
-# verbose(string) - prints string if we are in verbose mode
-verbose()
-{
- [ "$V" -eq "1" ] && echo " $1"
- return 0
-}
-
-##
-# error(string[, die]) - prints an error and exits with value die if given
-error()
-{
- [ "$quiet" -ne 1 ] && echo "error: $1"
- [ -n "$2" ] && exit $2
- return 0
-}
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH 06/16] testsuite: add & use warning()
2017-12-08 13:14 [PATCH 00/16] testsuite improvements Luc Van Oostenryck
` (4 preceding siblings ...)
2017-12-08 13:14 ` [PATCH 05/16] testsuite: move verbose/error() before get_tag_value() Luc Van Oostenryck
@ 2017-12-08 13:14 ` Luc Van Oostenryck
2017-12-08 13:14 ` [PATCH 07/16] testsuite: reset 'quiet' at the start of each testcase Luc Van Oostenryck
` (10 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: Luc Van Oostenryck @ 2017-12-08 13:14 UTC (permalink / raw)
To: linux-sparse; +Cc: Luc Van Oostenryck
Allow this new helper to indicate wich file trigger the
warning and replace the existing call to 'echo "warning: ...'.
Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
---
validation/test-suite | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/validation/test-suite b/validation/test-suite
index f04d81779..420214c65 100755
--- a/validation/test-suite
+++ b/validation/test-suite
@@ -51,6 +51,14 @@ verbose()
return 0
}
+##
+# warning(string) - prints a warning
+warning()
+{
+ [ "$quiet" -ne 1 ] && echo "warning: $1"
+ return 0
+}
+
##
# error(string[, die]) - prints an error and exits with value die if given
error()
@@ -253,7 +261,7 @@ do_test()
# can this test be handled by test-suite ?
# (it has to have a check-name key in it)
if [ "$check_name" = "" ]; then
- echo "warning: test '$file' unhandled"
+ warning "$file: test unhandled"
unhandled_tests=$(($unhandled_tests + 1))
return 2
fi
--
2.15.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH 07/16] testsuite: reset 'quiet' at the start of each testcase
2017-12-08 13:14 [PATCH 00/16] testsuite improvements Luc Van Oostenryck
` (5 preceding siblings ...)
2017-12-08 13:14 ` [PATCH 06/16] testsuite: add & use warning() Luc Van Oostenryck
@ 2017-12-08 13:14 ` Luc Van Oostenryck
2017-12-08 13:14 ` [PATCH 08/16] testsuite: fix invalid 'check-...' tags Luc Van Oostenryck
` (9 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: Luc Van Oostenryck @ 2017-12-08 13:14 UTC (permalink / raw)
To: linux-sparse; +Cc: Luc Van Oostenryck
The flag 'quiet' is used to quiets unwanted error messages,
for example for testcases known to fail, but this flag is reset
too late so that the beginning of the next testcases will run
with the value for the previous case.
Fix this by reseting the flag at the begining of each testcase.
Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
---
validation/test-suite | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/validation/test-suite b/validation/test-suite
index 420214c65..6a939846f 100755
--- a/validation/test-suite
+++ b/validation/test-suite
@@ -255,6 +255,7 @@ do_test()
{
test_failed=0
file="$1"
+ quiet=0
get_tag_value $file
@@ -316,7 +317,6 @@ do_test()
actual_exit_value=$?
must_fail=$check_known_to_fail
- quiet=0
[ $must_fail -eq 1 ] && [ $V -eq 0 ] && quiet=1
known_ko_tests=$(($known_ko_tests + $must_fail))
--
2.15.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH 08/16] testsuite: fix invalid 'check-...' tags
2017-12-08 13:14 [PATCH 00/16] testsuite improvements Luc Van Oostenryck
` (6 preceding siblings ...)
2017-12-08 13:14 ` [PATCH 07/16] testsuite: reset 'quiet' at the start of each testcase Luc Van Oostenryck
@ 2017-12-08 13:14 ` Luc Van Oostenryck
2017-12-08 13:14 ` [PATCH 09/16] testsuite: validate the " Luc Van Oostenryck
` (8 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: Luc Van Oostenryck @ 2017-12-08 13:14 UTC (permalink / raw)
To: linux-sparse; +Cc: Luc Van Oostenryck
A few testcases had typos in their 'check-...' tags or
the tag was plainly invalid.
Fix them in accordance to the doc.
Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
---
validation/cond-address.c | 2 +-
validation/cond-err-expand.c | 2 +-
validation/nested-declarator.c | 4 ++--
validation/nested-declarator2.c | 4 ++--
validation/optim/canonical-cmp.c | 2 +-
validation/optim/canonical-fcmp.c | 2 +-
validation/reserved.c | 4 ++--
validation/typedef_shadow.c | 4 ++--
8 files changed, 12 insertions(+), 12 deletions(-)
diff --git a/validation/cond-address.c b/validation/cond-address.c
index 2a69f4b92..c9253ad04 100644
--- a/validation/cond-address.c
+++ b/validation/cond-address.c
@@ -10,5 +10,5 @@ int qux(void) { if (f && a) return 1; return 0; }
* check-command: test-linearize -Wno-decl $file
* check-output-ignore
*
- * check-excludes: VOID
+ * check-output-excludes: VOID
*/
diff --git a/validation/cond-err-expand.c b/validation/cond-err-expand.c
index 93bbac153..7936a2d00 100644
--- a/validation/cond-err-expand.c
+++ b/validation/cond-err-expand.c
@@ -23,5 +23,5 @@ cond-err-expand.c:13:11: error: incompatible types in conditional expression (di
* check-error-end
*
* check-output-ignore
- * check-excludes: call.* __builtin_constant_p
+ * check-output-excludes: call.* __builtin_constant_p
*/
diff --git a/validation/nested-declarator.c b/validation/nested-declarator.c
index 1efe20ce4..f258d457b 100644
--- a/validation/nested-declarator.c
+++ b/validation/nested-declarator.c
@@ -15,7 +15,7 @@ int i(void (void)(*f));
int j(int [2](*));
/*
* check-name: nested declarator vs. parameters
- * check-error-start:
+ * check-error-start
nested-declarator.c:11:23: warning: missing identifier in declaration
nested-declarator.c:11:23: error: Expected ; at the end of type declaration
nested-declarator.c:11:23: error: got (
@@ -25,5 +25,5 @@ nested-declarator.c:14:18: error: Expected ) in function declarator
nested-declarator.c:14:18: error: got (
nested-declarator.c:15:14: error: Expected ) in function declarator
nested-declarator.c:15:14: error: got (
- * check-error-end:
+ * check-error-end
*/
diff --git a/validation/nested-declarator2.c b/validation/nested-declarator2.c
index 345a04b06..983b7ef5c 100644
--- a/validation/nested-declarator2.c
+++ b/validation/nested-declarator2.c
@@ -27,7 +27,7 @@ static int (-bad2);
static void [2](*bad3);
/*
* check-name: more on handling of ( in direct-declarator
- * check-error-start:
+ * check-error-start
nested-declarator2.c:17:1: warning: non-ANSI definition of function 'w1'
nested-declarator2.c:21:21: warning: non-ANSI function declaration of function '<noident>'
nested-declarator2.c:22:16: warning: variadic functions must have one named argument
@@ -37,5 +37,5 @@ nested-declarator2.c:26:13: error: Expected ) in nested declarator
nested-declarator2.c:26:13: error: got -
nested-declarator2.c:27:16: error: Expected ; at the end of type declaration
nested-declarator2.c:27:16: error: got (
- * check-error-end:
+ * check-error-end
*/
diff --git a/validation/optim/canonical-cmp.c b/validation/optim/canonical-cmp.c
index 19b416310..6f3b5180e 100644
--- a/validation/optim/canonical-cmp.c
+++ b/validation/optim/canonical-cmp.c
@@ -21,7 +21,7 @@ uint uat(uint p, uint a) { return (123 < p) ? a : 0; }
* check-name: canonical-cmp
* check-command: test-linearize -Wno-decl $file
*
- * check-output-exclude: \$123,
+ * check-output-excludes: \$123,
*
* check-output-start
seq:
diff --git a/validation/optim/canonical-fcmp.c b/validation/optim/canonical-fcmp.c
index e3e758a9b..d098da64f 100644
--- a/validation/optim/canonical-fcmp.c
+++ b/validation/optim/canonical-fcmp.c
@@ -20,7 +20,7 @@ int nfcmp_ge(double a) { return !(g < a); }
* check-name: canonical-cmp
* check-command: test-linearize -Wno-decl $file
*
- * check-output-exclude: \$123,
+ * check-output-excludes: \$123,
*
* check-output-start
fcmp_eq:
diff --git a/validation/reserved.c b/validation/reserved.c
index 29554560b..6a2163e9a 100644
--- a/validation/reserved.c
+++ b/validation/reserved.c
@@ -81,7 +81,7 @@ static int (__builtin_va_list);
/*
* check-name: const et.al. are reserved identifiers
- * check-error-start:
+ * check-error-start
reserved.c:1:12: error: Trying to use reserved word 'auto' as identifier
reserved.c:2:12: error: Trying to use reserved word 'break' as identifier
reserved.c:3:12: error: Trying to use reserved word 'case' as identifier
@@ -154,5 +154,5 @@ reserved.c:77:12: error: Trying to use reserved word '__builtin_ms_va_list' as i
reserved.c:78:12: error: Trying to use reserved word '__builtin_offsetof' as identifier
reserved.c:79:12: error: Trying to use reserved word '__builtin_types_compatible_p' as identifier
reserved.c:80:12: error: Trying to use reserved word '__builtin_va_list' as identifier
- * check-error-end:
+ * check-error-end
*/
diff --git a/validation/typedef_shadow.c b/validation/typedef_shadow.c
index e52de80f2..e8e4c229b 100644
--- a/validation/typedef_shadow.c
+++ b/validation/typedef_shadow.c
@@ -5,9 +5,9 @@ static void f(int T)
}
/*
* check-name: typedef shadowing
- * check-error-start:
+ * check-error-start
typedef_shadow.c:4:16: warning: 'T' has implicit type
typedef_shadow.c:4:18: error: Expected ; at end of declaration
typedef_shadow.c:4:18: error: got a
- * check-error-end:
+ * check-error-end
*/
--
2.15.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH 09/16] testsuite: validate the 'check-...' tags
2017-12-08 13:14 [PATCH 00/16] testsuite improvements Luc Van Oostenryck
` (7 preceding siblings ...)
2017-12-08 13:14 ` [PATCH 08/16] testsuite: fix invalid 'check-...' tags Luc Van Oostenryck
@ 2017-12-08 13:14 ` Luc Van Oostenryck
2017-12-08 13:14 ` [PATCH 10/16] testsuite: early return in getopt loop Luc Van Oostenryck
` (7 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: Luc Van Oostenryck @ 2017-12-08 13:14 UTC (permalink / raw)
To: linux-sparse; +Cc: Luc Van Oostenryck
Making a typo in one of the 'check-...' tags can make
a testcase useless and thus incapable of detecting a regression.
Add some validation to these tags in order to detect wrong
tags.
Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
---
validation/test-suite | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/validation/test-suite b/validation/test-suite
index 6a939846f..d50a3bfa2 100755
--- a/validation/test-suite
+++ b/validation/test-suite
@@ -108,6 +108,18 @@ get_tag_value()
check_arch_ignore="$val" ;;
check-arch-only:) arch=$(uname -m)
check_arch_only="$val" ;;
+
+ check-description:) ;; # ignore
+ check-note:) ;; # ignore
+ check-warning:) ;; # ignore
+ check-error-start) ;; # ignore
+ check-error-end) ;; # ignore
+ check-output-start) ;; # ignore
+ check-output-end) ;; # ignore
+ check-should-pass) ;; # ignore, unused annotation
+ check-should-fail) ;; # ignore, unused annotation
+ check-should-warn) ;; # ignore, unused annotation
+ check-*) error "$1: unknown tag '$tag'" 1 ;;
esac
done << EOT
$lines
--
2.15.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH 10/16] testsuite: early return in getopt loop
2017-12-08 13:14 [PATCH 00/16] testsuite improvements Luc Van Oostenryck
` (8 preceding siblings ...)
2017-12-08 13:14 ` [PATCH 09/16] testsuite: validate the " Luc Van Oostenryck
@ 2017-12-08 13:14 ` Luc Van Oostenryck
2017-12-08 13:14 ` [PATCH 11/16] testsuite: move do_test_suite out of the " Luc Van Oostenryck
` (6 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: Luc Van Oostenryck @ 2017-12-08 13:14 UTC (permalink / raw)
To: linux-sparse; +Cc: Luc Van Oostenryck
This is a preparatory step to allow to run only a part
of the testsuite (a subdir).
Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
---
validation/test-suite | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/validation/test-suite b/validation/test-suite
index d50a3bfa2..c16c7c843 100755
--- a/validation/test-suite
+++ b/validation/test-suite
@@ -521,10 +521,12 @@ while true; do
'')
tests_list=`find . -name '*.c' | sed -e 's#^\./\(.*\)#\1#' | sort`
do_test_suite
+ break
;;
*.c)
tests_list="$@"
do_test_suite
+ break
;;
single|--single)
@@ -535,10 +537,12 @@ while true; do
1) echo "$2 failed !";;
2) echo "$2 can't be handled by $prog_name";;
esac
+ exit $failed
;;
format|--format)
shift
do_format "$@"
+ exit 0
;;
help | *)
do_usage
--
2.15.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH 11/16] testsuite: move do_test_suite out of the getopt loop
2017-12-08 13:14 [PATCH 00/16] testsuite improvements Luc Van Oostenryck
` (9 preceding siblings ...)
2017-12-08 13:14 ` [PATCH 10/16] testsuite: early return in getopt loop Luc Van Oostenryck
@ 2017-12-08 13:14 ` Luc Van Oostenryck
2017-12-08 13:14 ` [PATCH 12/16] testsuite: move no-arg " Luc Van Oostenryck
` (5 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: Luc Van Oostenryck @ 2017-12-08 13:14 UTC (permalink / raw)
To: linux-sparse; +Cc: Luc Van Oostenryck
This is a preparatory step to allow to run only a part
of the testsuite (a subdir).
Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
---
validation/test-suite | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/validation/test-suite b/validation/test-suite
index c16c7c843..72bee9dce 100755
--- a/validation/test-suite
+++ b/validation/test-suite
@@ -520,12 +520,10 @@ while true; do
'')
tests_list=`find . -name '*.c' | sed -e 's#^\./\(.*\)#\1#' | sort`
- do_test_suite
break
;;
*.c)
tests_list="$@"
- do_test_suite
break
;;
@@ -552,5 +550,6 @@ while true; do
break
done
+do_test_suite
exit $failed
--
2.15.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH 12/16] testsuite: move no-arg out of the getopt loop
2017-12-08 13:14 [PATCH 00/16] testsuite improvements Luc Van Oostenryck
` (10 preceding siblings ...)
2017-12-08 13:14 ` [PATCH 11/16] testsuite: move do_test_suite out of the " Luc Van Oostenryck
@ 2017-12-08 13:14 ` Luc Van Oostenryck
2017-12-08 13:14 ` [PATCH 13/16] testsuite: change do_usage text Luc Van Oostenryck
` (4 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: Luc Van Oostenryck @ 2017-12-08 13:14 UTC (permalink / raw)
To: linux-sparse; +Cc: Luc Van Oostenryck
This is a preparatory step to allow to run only a part
of the testsuite (a subdir).
Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
---
validation/test-suite | 11 +++++------
1 file changed, 5 insertions(+), 6 deletions(-)
diff --git a/validation/test-suite b/validation/test-suite
index 72bee9dce..410190bd3 100755
--- a/validation/test-suite
+++ b/validation/test-suite
@@ -505,7 +505,7 @@ _EOF
return 0
}
-while true; do
+while [ "$#" -gt "0" ]; do
case "$1" in
-a|--abort)
abort=1
@@ -517,11 +517,6 @@ while true; do
shift
continue
;;
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH 13/16] testsuite: change do_usage text
2017-12-08 13:14 [PATCH 00/16] testsuite improvements Luc Van Oostenryck
` (11 preceding siblings ...)
2017-12-08 13:14 ` [PATCH 12/16] testsuite: move no-arg " Luc Van Oostenryck
@ 2017-12-08 13:14 ` Luc Van Oostenryck
2017-12-08 13:14 ` [PATCH 14/16] testsuite: allow to test only a subdir Luc Van Oostenryck
` (3 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: Luc Van Oostenryck @ 2017-12-08 13:14 UTC (permalink / raw)
To: linux-sparse; +Cc: Luc Van Oostenryck
The text for the testsuite usage used 'none' as if it was an
option/keyword while it only meant the absence of arguments.
Make the text clearer by removing the 'none' and being explicit
about the absence of arguments.
Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
---
validation/test-suite | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/validation/test-suite b/validation/test-suite
index 410190bd3..854ad8add 100755
--- a/validation/test-suite
+++ b/validation/test-suite
@@ -233,16 +233,16 @@ echo "$prog_name - a tiny automatic testing script"
echo "Usage: $prog_name [option(s)] [command] [arguments]"
echo
echo "options:"
-echo " -a|--abort abort the tests as soon as one fails"
-echo " -q|--quiet be extra quiet while running the tests"
+echo " -a|--abort Abort the tests as soon as one fails."
+echo " -q|--quiet Be extra quiet while running the tests."
echo
echo "commands:"
-echo " none runs the whole test suite"
-echo " file ... runs the test suite on the given file(s)"
-echo " single file runs the test in 'file'"
-echo " format file [name [cmd]] helps writing a new test case using cmd"
+echo " [file ...] Runs the test suite on the given file(s)."
+echo " If no file is given, run the whole testsuite."
+echo " single file Run the test in 'file'."
+echo " format file [name [cmd]] Help writing a new test case using cmd."
echo
-echo " [command] help prints usage"
+echo " [command] help Print usage."
}
disable()
--
2.15.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH 14/16] testsuite: allow to test only a subdir
2017-12-08 13:14 [PATCH 00/16] testsuite improvements Luc Van Oostenryck
` (12 preceding siblings ...)
2017-12-08 13:14 ` [PATCH 13/16] testsuite: change do_usage text Luc Van Oostenryck
@ 2017-12-08 13:14 ` Luc Van Oostenryck
2017-12-08 13:14 ` [PATCH 15/16] testsuite: default to shift in the getopt loop Luc Van Oostenryck
` (2 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: Luc Van Oostenryck @ 2017-12-08 13:14 UTC (permalink / raw)
To: linux-sparse; +Cc: Luc Van Oostenryck
During development, it is very useful to be able to run only
some of the tests, maybe a whole class.
Help this by allowing to run the testsuite on only a subdir
of the 'validation/' directory.
Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
---
validation/test-suite | 22 +++++++++++++++++-----
1 file changed, 17 insertions(+), 5 deletions(-)
diff --git a/validation/test-suite b/validation/test-suite
index 854ad8add..098cc7869 100755
--- a/validation/test-suite
+++ b/validation/test-suite
@@ -238,6 +238,7 @@ echo " -q|--quiet Be extra quiet while running the tests."
echo
echo "commands:"
echo " [file ...] Runs the test suite on the given file(s)."
+echo " If a directory is given, run only those files."
echo " If no file is given, run the whole testsuite."
echo " single file Run the test in 'file'."
echo " format file [name [cmd]] Help writing a new test case using cmd."
@@ -517,10 +518,6 @@ while [ "$#" -gt "0" ]; do
shift
continue
;;
- *.c)
- tests_list="$@"
- break
- ;;
single|--single)
arg_file "$2"
@@ -537,10 +534,25 @@ while [ "$#" -gt "0" ]; do
do_format "$@"
exit 0
;;
- help | *)
+ help)
do_usage
exit 1
;;
+
+ *.c)
+ tests_list="$tests_list $1"
+ shift
+ continue
+ ;;
+ *)
+ if [ ! -d "$1" ]; then
+ do_usage
+ exit 1
+ fi
+ tests_list="$tests_list $(find "$1" -name '*.c' | sort)"
+ shift
+ continue
+ ;;
esac
break
done
--
2.15.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH 15/16] testsuite: default to shift in the getopt loop
2017-12-08 13:14 [PATCH 00/16] testsuite improvements Luc Van Oostenryck
` (13 preceding siblings ...)
2017-12-08 13:14 ` [PATCH 14/16] testsuite: allow to test only a subdir Luc Van Oostenryck
@ 2017-12-08 13:14 ` Luc Van Oostenryck
2017-12-08 13:14 ` [PATCH 16/16] testsuite: add support for 'format -a' Luc Van Oostenryck
2017-12-09 2:09 ` [PATCH 00/16] testsuite improvements Christopher Li
16 siblings, 0 replies; 19+ messages in thread
From: Luc Van Oostenryck @ 2017-12-08 13:14 UTC (permalink / raw)
To: linux-sparse; +Cc: Luc Van Oostenryck
The getopt loop used to bear by default and only some
options had to explicitly call 'shift' and 'continue'
to process further elements.
Change this to a 'normal' loop, shifting the next arg by default
and breaking of the loop when needed.
Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
---
validation/test-suite | 10 +---------
1 file changed, 1 insertion(+), 9 deletions(-)
diff --git a/validation/test-suite b/validation/test-suite
index 098cc7869..8c0453590 100755
--- a/validation/test-suite
+++ b/validation/test-suite
@@ -510,13 +510,9 @@ while [ "$#" -gt "0" ]; do
case "$1" in
-a|--abort)
abort=1
- shift
- continue
;;
-q|--quiet)
vquiet=1
- shift
- continue
;;
single|--single)
@@ -541,8 +537,6 @@ while [ "$#" -gt "0" ]; do
*.c)
tests_list="$tests_list $1"
- shift
- continue
;;
*)
if [ ! -d "$1" ]; then
@@ -550,11 +544,9 @@ while [ "$#" -gt "0" ]; do
exit 1
fi
tests_list="$tests_list $(find "$1" -name '*.c' | sort)"
- shift
- continue
;;
esac
- break
+ shift
done
if [ -z "$tests_list" ]; then
--
2.15.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* [PATCH 16/16] testsuite: add support for 'format -a'
2017-12-08 13:14 [PATCH 00/16] testsuite improvements Luc Van Oostenryck
` (14 preceding siblings ...)
2017-12-08 13:14 ` [PATCH 15/16] testsuite: default to shift in the getopt loop Luc Van Oostenryck
@ 2017-12-08 13:14 ` Luc Van Oostenryck
2017-12-09 2:09 ` [PATCH 00/16] testsuite improvements Christopher Li
16 siblings, 0 replies; 19+ messages in thread
From: Luc Van Oostenryck @ 2017-12-08 13:14 UTC (permalink / raw)
To: linux-sparse; +Cc: Luc Van Oostenryck
The 'format' command create the information needed for the testcase
from the input file and output this on stdout. The developper must
then add this to the input file.
Let's do this automatically by adding an option '-a' to the 'format'
command to directly append the infos to the input file.
Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
---
validation/test-suite | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/validation/test-suite b/validation/test-suite
index 8c0453590..192fba300 100755
--- a/validation/test-suite
+++ b/validation/test-suite
@@ -429,6 +429,7 @@ do_format_help() {
echo "Usage: $prog_name [option(s)] [--]format file [name [cmd]]"
echo
echo "options:"
+echo " -a append the created test to the input file"
echo " -f write a test known to fail"
echo " -l write a test for linearized code"
echo
@@ -443,11 +444,14 @@ echo " cmd command to be used (defaults to 'sparse \$f
do_format()
{
def_cmd="$default_cmd"
+ append=0
linear=0
fail=0
while [ $# -gt 1 ] ; do
case "$1" in
+ -a)
+ append=1 ;;
-f)
fail=1 ;;
-l)
@@ -474,6 +478,7 @@ do_format()
cmd=`eval echo $default_path/$fcmd`
$cmd 1> $file.output.got 2> $file.error.got
fexit_value=$?
+ [ "append" != 0 ] && exec >> $file
cat <<_EOF
/*
--
2.15.0
^ permalink raw reply related [flat|nested] 19+ messages in thread* Re: [PATCH 00/16] testsuite improvements
2017-12-08 13:14 [PATCH 00/16] testsuite improvements Luc Van Oostenryck
` (15 preceding siblings ...)
2017-12-08 13:14 ` [PATCH 16/16] testsuite: add support for 'format -a' Luc Van Oostenryck
@ 2017-12-09 2:09 ` Christopher Li
2017-12-10 17:25 ` Luc Van Oostenryck
16 siblings, 1 reply; 19+ messages in thread
From: Christopher Li @ 2017-12-09 2:09 UTC (permalink / raw)
To: Luc Van Oostenryck; +Cc: Linux-Sparse
On Fri, Dec 8, 2017 at 9:14 PM, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:
> This series contains a few fixes & improvements to the
> testsuite; mainly:
> - allow to run the testsuite on all the tests of a subdir
> - teach 'format' to directly append to the testcase
> - validate the 'check-...' tags
>
> Luc Van Oostenryck (16):
> testsuite: make the '%.t' rule depends on PROGRAMS too
> testsuite: fix a few more incorrect check-commands
> testsuite: convert to the new pattern syntax
> testsuite: remove old ugly pattern syntax
> testsuite: move verbose/error() before get_tag_value()
> testsuite: add & use warning()
> testsuite: reset 'quiet' at the start of each testcase
> testsuite: fix invalid 'check-...' tags
> testsuite: validate the 'check-...' tags
> testsuite: early return in getopt loop
> testsuite: move do_test_suite out of the getopt loop
> testsuite: move no-arg out of the getopt loop
> testsuite: change do_usage text
> testsuite: allow to test only a subdir
> testsuite: default to shift in the getopt loop
> testsuite: add support for 'format -a'
I haven't have a chance to do the full review as I did before
applying them. I just briefly go over the patches. From the
distance I look, the series looks fine for me.
Usually as long as the make check was not broken,
I am not too worry about the testsuite changes.
Chris
^ permalink raw reply [flat|nested] 19+ messages in thread* Re: [PATCH 00/16] testsuite improvements
2017-12-09 2:09 ` [PATCH 00/16] testsuite improvements Christopher Li
@ 2017-12-10 17:25 ` Luc Van Oostenryck
0 siblings, 0 replies; 19+ messages in thread
From: Luc Van Oostenryck @ 2017-12-10 17:25 UTC (permalink / raw)
To: Christopher Li; +Cc: Linux-Sparse
On Sat, Dec 09, 2017 at 10:09:05AM +0800, Christopher Li wrote:
>
> I haven't have a chance to do the full review as I did before
> applying them. I just briefly go over the patches. From the
> distance I look, the series looks fine for me.
>
> Usually as long as the make check was not broken,
> I am not too worry about the testsuite changes.
>
> Chris
Thanks,
This has now been pushed to:
git://github.com/lucvoo/sparse.git
-- Luc Van Oostenryck
^ permalink raw reply [flat|nested] 19+ messages in thread