* [RFC PATCH v3 1/4] ftracetest: Initial commit for ftracetest
2014-09-02 11:06 [RFC PATCH v3 0/4] ftrace: Add a ftrace test collection Masami Hiramatsu
@ 2014-09-02 11:06 ` Masami Hiramatsu
2014-09-04 1:41 ` Namhyung Kim
2014-09-16 22:21 ` Steven Rostedt
2014-09-02 11:06 ` [RFC PATCH v3 2/4] ftracetest: Add ftrace basic testcases Masami Hiramatsu
` (3 subsequent siblings)
4 siblings, 2 replies; 14+ messages in thread
From: Masami Hiramatsu @ 2014-09-02 11:06 UTC (permalink / raw)
To: Shuah Khan, Tom Zanussi, Yoshihiro YUNOMAE, Oleg Nesterov,
Steven Rostedt, Namhyung Kim, Ingo Molnar
Cc: Linux Kernel Mailing List
ftracetest is a collection of testcase shell-scripts for ftrace.
To avoid regressions of ftrace, these testcases check correct
ftrace behaviors. If someone would like to add any features on
ftrace, the patch series should have at least one testcase for
checking the new behavior.
Changes in v3:
- Use "." instead of "source".
- Don't use -e option for echo since dash doesn't support it.
Changes in v2:
- Remove unneeded 'function' keyword.
- Add abspath and find_testcases.
- Make OPT_TEST_CASES a local var.
- Ensure given testcase ended with .tc.
- Accept a directory option which has multiple testcases.
- Change [PASSED]/[FAILED] to [PASS]/[FAIL]
- Change the basic shell to sh (dash).
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
---
tools/testing/ftrace/README | 45 ++++++++++
tools/testing/ftrace/ftracetest | 152 ++++++++++++++++++++++++++++++++++
tools/testing/ftrace/test.d/template | 4 +
3 files changed, 201 insertions(+)
create mode 100644 tools/testing/ftrace/README
create mode 100755 tools/testing/ftrace/ftracetest
create mode 100644 tools/testing/ftrace/test.d/template
diff --git a/tools/testing/ftrace/README b/tools/testing/ftrace/README
new file mode 100644
index 0000000..5d5de16
--- /dev/null
+++ b/tools/testing/ftrace/README
@@ -0,0 +1,45 @@
+Linux Ftrace Testcases
+
+This is a collection of testcases for ftrace tracing feature in the Linux
+kernel. Since ftrace exports interfaces via the debugfs, we just need
+shell scripts for testing. Feel free to add new test cases.
+
+Running the ftrace testcases
+============================
+
+At first, you need to be the root user to run this script.
+To run all testcases:
+
+ $ sudo ./ftracetest
+
+To run specific testcases:
+
+ # ./ftracetest test.d/basic3.tc
+
+Or you can also run testcases under given directory:
+
+ # ./ftracetest test.d/kprobe/
+
+Contributing new testcases
+==========================
+
+Copy test.d/template to your testcase (whose filename must has *.tc extension)
+and rewrite test description line.
+
+ * The working directory of the script is <debugfs>/tracing/.
+
+ * Take care about the side effect, because that is run with root privilege.
+
+ * Don't take too long. It's a kind of unit test.
+
+ * You can add a directory for your testcases under test.d/ if needed.
+
+ * The test cases should run on dash (busybox shell) for testing on
+ minimal cross-build environments.
+
+TODO
+====
+
+ * Fancy colored output :)
+
+ * Integrate with selftest?
diff --git a/tools/testing/ftrace/ftracetest b/tools/testing/ftrace/ftracetest
new file mode 100755
index 0000000..8d92e74
--- /dev/null
+++ b/tools/testing/ftrace/ftracetest
@@ -0,0 +1,152 @@
+#!/bin/sh
+# ftracetest - Ftrace test shell scripts
+# Written by Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
+
+usage() { # errno [message]
+[ "$2" ] && echo $2
+echo "Usage: ftracetest [options] [testcase(s)] [testcase-directory(s)]"
+echo " Options:"
+echo " -h|--help Show help message"
+echo " -k|--keep Keep passed test logs"
+echo " -d|--debug Debug mode (trace all shell commands)"
+exit $1
+}
+
+errexit() { # message
+ echo "Error: $1" 1>&2
+ exit 1
+}
+
+# Ensuring user privilege
+if [ `id -u` -ne 0 ]; then
+ errexit "this must be run by root user"
+fi
+
+# Utilities
+absdir() { # file_path
+ (cd `dirname $1`; pwd)
+}
+
+abspath() {
+ echo `absdir $1`/`basename $1`
+}
+
+find_testcases() { #directory
+ echo `find $1 -name \*.tc`
+}
+
+parse_opts() { # opts
+ local OPT_TEST_CASES=
+ local OPT_TEST_DIR=
+
+ while [ "$1" ]; do
+ case "$1" in
+ --help|-h)
+ usage 0
+ ;;
+ --keep|-k)
+ KEEP_LOG=1
+ shift 1
+ ;;
+ --debug|-d)
+ DEBUG=1
+ shift 1
+ ;;
+ *.tc)
+ if [ -f "$1" ]; then
+ OPT_TEST_CASES="$OPT_TEST_CASES `abspath $1`"
+ shift 1
+ else
+ usage 1 "$1 is not a testcase"
+ fi
+ ;;
+ *)
+ if [ -d "$1" ]; then
+ OPT_TEST_DIR=`abspath $1`
+ OPT_TEST_CASES="$OPT_TEST_CASES `find_testcases $OPT_TEST_DIR`"
+ shift 1
+ else
+ usage 1 "Invalid option ($1)"
+ fi
+ ;;
+ esac
+ done
+ if [ "$OPT_TEST_CASES" ]; then
+ TEST_CASES=$OPT_TEST_CASES
+ fi
+}
+
+# Parameters
+DEBUGFS_DIR=`grep debugfs /proc/mounts | cut -f2 -d' '`
+TRACING_DIR=$DEBUGFS_DIR/tracing
+TOP_DIR=`absdir $0`
+TEST_DIR=$TOP_DIR/test.d
+TEST_CASES=`find_testcases $TEST_DIR`
+LOG_DIR=$TOP_DIR/logs/`date +%Y%m%d-%H%M%S`/
+KEEP_LOG=0
+DEBUG=0
+# Parse command-line options
+parse_opts $*
+
+[ $DEBUG -ne 0 ] && set -x
+
+# Verify parameters
+if [ -z "$DEBUGFS_DIR" -o ! -d "$TRACING_DIR" ]; then
+ errexit "No ftrace directory found"
+fi
+
+# Preparing logs
+LOG_FILE=$LOG_DIR/ftracetest.log
+mkdir -p $LOG_DIR || errexit "Failed to make a log directory: $LOG_DIR"
+date > $LOG_FILE
+prlog() { # messages
+ echo "$@" | tee -a $LOG_FILE
+}
+catlog() { #file
+ cat $1 | tee -a $LOG_FILE
+}
+
+# Testcase management
+PASSED_CASES=
+FAILED_CASES=
+CASENO=0
+testcase() { # testfile
+ CASENO=$((CASENO+1))
+ prlog -n "[$CASENO]"`grep "^#[ \t]*description:" $1 | cut -f2 -d:`
+}
+failed() {
+ prlog " [FAIL]"
+ FAILED_CASES="$FAILED_CASES $CASENO"
+}
+passed() {
+ prlog " [PASS]"
+ PASSED_CASES="$PASSED_CASES $CASENO"
+}
+
+
+# Run one test case
+run_test() { # testfile
+ local testname=`basename $1`
+ local testlog=`mktemp --tmpdir=$LOG_DIR ${testname}-XXXXXX.log`
+ testcase $1
+ echo "execute: "$1 > $testlog
+ (cd $TRACING_DIR; set -x ; . $t) >> $testlog 2>&1
+ ret=$?
+ if [ $ret -ne 0 ]; then
+ failed
+ catlog $testlog
+ else
+ passed
+ [ $KEEP_LOG -eq 0 ] && rm $testlog
+ fi
+}
+
+# Main loop
+for t in $TEST_CASES; do
+ run_test $t
+done
+prlog ""
+prlog "# of passed: " `echo $PASSED_CASES | wc -w`
+prlog "# of failed: " `echo $FAILED_CASES | wc -w`
+
+test -z "$FAILED_CASES" # if no error, return 0
diff --git a/tools/testing/ftrace/test.d/template b/tools/testing/ftrace/test.d/template
new file mode 100644
index 0000000..ce5f735
--- /dev/null
+++ b/tools/testing/ftrace/test.d/template
@@ -0,0 +1,4 @@
+#!/bin/sh
+# description: %HERE DESCRIBE WHAT THIS DOES%
+# you have to add ".tc" extention for your testcase file
+exit 0 # Return 0 if the test is passed, otherwise return !0
^ permalink raw reply related [flat|nested] 14+ messages in thread* Re: [RFC PATCH v3 1/4] ftracetest: Initial commit for ftracetest
2014-09-02 11:06 ` [RFC PATCH v3 1/4] ftracetest: Initial commit for ftracetest Masami Hiramatsu
@ 2014-09-04 1:41 ` Namhyung Kim
2014-09-04 1:48 ` Masami Hiramatsu
2014-09-16 22:21 ` Steven Rostedt
1 sibling, 1 reply; 14+ messages in thread
From: Namhyung Kim @ 2014-09-04 1:41 UTC (permalink / raw)
To: Masami Hiramatsu
Cc: Shuah Khan, Tom Zanussi, Yoshihiro YUNOMAE, Oleg Nesterov,
Steven Rostedt, Ingo Molnar, Linux Kernel Mailing List
Hi Masami,
Just a nit..
On Tue, 02 Sep 2014 11:06:39 +0000, Masami Hiramatsu wrote:
> ftracetest is a collection of testcase shell-scripts for ftrace.
> To avoid regressions of ftrace, these testcases check correct
> ftrace behaviors. If someone would like to add any features on
> ftrace, the patch series should have at least one testcase for
> checking the new behavior.
[SNIP]
> +# Run one test case
> +run_test() { # testfile
> + local testname=`basename $1`
> + local testlog=`mktemp --tmpdir=$LOG_DIR ${testname}-XXXXXX.log`
> + testcase $1
> + echo "execute: "$1 > $testlog
> + (cd $TRACING_DIR; set -x ; . $t) >> $testlog 2>&1
s/$t/$1/ ?
Thanks,
Namhyung
> + ret=$?
> + if [ $ret -ne 0 ]; then
> + failed
> + catlog $testlog
> + else
> + passed
> + [ $KEEP_LOG -eq 0 ] && rm $testlog
> + fi
> +}
> +
> +# Main loop
> +for t in $TEST_CASES; do
> + run_test $t
> +done
> +prlog ""
> +prlog "# of passed: " `echo $PASSED_CASES | wc -w`
> +prlog "# of failed: " `echo $FAILED_CASES | wc -w`
> +
> +test -z "$FAILED_CASES" # if no error, return 0
> diff --git a/tools/testing/ftrace/test.d/template b/tools/testing/ftrace/test.d/template
> new file mode 100644
> index 0000000..ce5f735
> --- /dev/null
> +++ b/tools/testing/ftrace/test.d/template
> @@ -0,0 +1,4 @@
> +#!/bin/sh
> +# description: %HERE DESCRIBE WHAT THIS DOES%
> +# you have to add ".tc" extention for your testcase file
> +exit 0 # Return 0 if the test is passed, otherwise return !0
^ permalink raw reply [flat|nested] 14+ messages in thread* Re: [RFC PATCH v3 1/4] ftracetest: Initial commit for ftracetest
2014-09-04 1:41 ` Namhyung Kim
@ 2014-09-04 1:48 ` Masami Hiramatsu
0 siblings, 0 replies; 14+ messages in thread
From: Masami Hiramatsu @ 2014-09-04 1:48 UTC (permalink / raw)
To: Namhyung Kim
Cc: Shuah Khan, Tom Zanussi, Yoshihiro YUNOMAE, Oleg Nesterov,
Steven Rostedt, Ingo Molnar, Linux Kernel Mailing List
(2014/09/04 10:41), Namhyung Kim wrote:
> Hi Masami,
>
> Just a nit..
>
>
> On Tue, 02 Sep 2014 11:06:39 +0000, Masami Hiramatsu wrote:
>> ftracetest is a collection of testcase shell-scripts for ftrace.
>> To avoid regressions of ftrace, these testcases check correct
>> ftrace behaviors. If someone would like to add any features on
>> ftrace, the patch series should have at least one testcase for
>> checking the new behavior.
>
>
> [SNIP]
>> +# Run one test case
>> +run_test() { # testfile
>> + local testname=`basename $1`
>> + local testlog=`mktemp --tmpdir=$LOG_DIR ${testname}-XXXXXX.log`
>> + testcase $1
>> + echo "execute: "$1 > $testlog
>> + (cd $TRACING_DIR; set -x ; . $t) >> $testlog 2>&1
>
> s/$t/$1/ ?
Oops, right! I missed that. Since the $1 is same as $t, the
test itself works, but it should be fixed.
Thank you!
>
> Thanks,
> Namhyung
>
>
>> + ret=$?
>> + if [ $ret -ne 0 ]; then
>> + failed
>> + catlog $testlog
>> + else
>> + passed
>> + [ $KEEP_LOG -eq 0 ] && rm $testlog
>> + fi
>> +}
>> +
>> +# Main loop
>> +for t in $TEST_CASES; do
>> + run_test $t
>> +done
>> +prlog ""
>> +prlog "# of passed: " `echo $PASSED_CASES | wc -w`
>> +prlog "# of failed: " `echo $FAILED_CASES | wc -w`
>> +
>> +test -z "$FAILED_CASES" # if no error, return 0
>> diff --git a/tools/testing/ftrace/test.d/template b/tools/testing/ftrace/test.d/template
>> new file mode 100644
>> index 0000000..ce5f735
>> --- /dev/null
>> +++ b/tools/testing/ftrace/test.d/template
>> @@ -0,0 +1,4 @@
>> +#!/bin/sh
>> +# description: %HERE DESCRIBE WHAT THIS DOES%
>> +# you have to add ".tc" extention for your testcase file
>> +exit 0 # Return 0 if the test is passed, otherwise return !0
>
--
Masami HIRAMATSU
Software Platform Research Dept. Linux Technology Research Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [RFC PATCH v3 1/4] ftracetest: Initial commit for ftracetest
2014-09-02 11:06 ` [RFC PATCH v3 1/4] ftracetest: Initial commit for ftracetest Masami Hiramatsu
2014-09-04 1:41 ` Namhyung Kim
@ 2014-09-16 22:21 ` Steven Rostedt
2014-09-17 5:40 ` Masami Hiramatsu
1 sibling, 1 reply; 14+ messages in thread
From: Steven Rostedt @ 2014-09-16 22:21 UTC (permalink / raw)
To: Masami Hiramatsu
Cc: Shuah Khan, Tom Zanussi, Yoshihiro YUNOMAE, Oleg Nesterov,
Namhyung Kim, Ingo Molnar, Linux Kernel Mailing List
> diff --git a/tools/testing/ftrace/README b/tools/testing/ftrace/README
> new file mode 100644
> index 0000000..5d5de16
> --- /dev/null
> +++ b/tools/testing/ftrace/README
> @@ -0,0 +1,45 @@
> +Linux Ftrace Testcases
> +
> +This is a collection of testcases for ftrace tracing feature in the Linux
> +kernel. Since ftrace exports interfaces via the debugfs, we just need
> +shell scripts for testing. Feel free to add new test cases.
> +
> +Running the ftrace testcases
> +============================
> +
> +At first, you need to be the root user to run this script.
> +To run all testcases:
> +
> + $ sudo ./ftracetest
> +
> +To run specific testcases:
> +
> + # ./ftracetest test.d/basic3.tc
> +
> +Or you can also run testcases under given directory:
> +
> + # ./ftracetest test.d/kprobe/
> +
> +Contributing new testcases
> +==========================
> +
> +Copy test.d/template to your testcase (whose filename must has *.tc extension)
"whose filename must have *.tc extension"
> +and rewrite test description line.
"and rewrite the test description line"
> +
> + * The working directory of the script is <debugfs>/tracing/.
> +
> + * Take care about the side effect, because that is run with root privilege.
"Take care with side effects as the tests are run with root privilege"
> +
> + * Don't take too long. It's a kind of unit test.
"The tests should not run for a long period of time. These are to be
unit tests"
We should probably specify what the time should be. Is 20 minutes too
long?
> +
> + * You can add a directory for your testcases under test.d/ if needed.
> +
> + * The test cases should run on dash (busybox shell) for testing on
> + minimal cross-build environments.
> +
> +TODO
> +====
> +
> + * Fancy colored output :)
> +
> + * Integrate with selftest?
> diff --git a/tools/testing/ftrace/ftracetest b/tools/testing/ftrace/ftracetest
> new file mode 100755
> index 0000000..8d92e74
> --- /dev/null
> +++ b/tools/testing/ftrace/ftracetest
> @@ -0,0 +1,152 @@
> +#!/bin/sh
> +# ftracetest - Ftrace test shell scripts
> +# Written by Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Maybe add your copyright here, with year?
Also, we should probably state that this is under the GPL v2?
-- Steve
> +
> +usage() { # errno [message]
> +[ "$2" ] && echo $2
> +echo "Usage: ftracetest [options] [testcase(s)] [testcase-directory(s)]"
> +echo " Options:"
> +echo " -h|--help Show help message"
> +echo " -k|--keep Keep passed test logs"
> +echo " -d|--debug Debug mode (trace all shell commands)"
> +exit $1
> +}
^ permalink raw reply [flat|nested] 14+ messages in thread* Re: [RFC PATCH v3 1/4] ftracetest: Initial commit for ftracetest
2014-09-16 22:21 ` Steven Rostedt
@ 2014-09-17 5:40 ` Masami Hiramatsu
0 siblings, 0 replies; 14+ messages in thread
From: Masami Hiramatsu @ 2014-09-17 5:40 UTC (permalink / raw)
To: Steven Rostedt
Cc: Shuah Khan, Tom Zanussi, Yoshihiro YUNOMAE, Oleg Nesterov,
Namhyung Kim, Ingo Molnar, Linux Kernel Mailing List
Hi Steve,
Thank you for your comments.
(2014/09/17 7:21), Steven Rostedt wrote:
>
>> diff --git a/tools/testing/ftrace/README b/tools/testing/ftrace/README
>> new file mode 100644
>> index 0000000..5d5de16
>> --- /dev/null
>> +++ b/tools/testing/ftrace/README
>> @@ -0,0 +1,45 @@
>> +Linux Ftrace Testcases
>> +
>> +This is a collection of testcases for ftrace tracing feature in the Linux
>> +kernel. Since ftrace exports interfaces via the debugfs, we just need
>> +shell scripts for testing. Feel free to add new test cases.
>> +
>> +Running the ftrace testcases
>> +============================
>> +
>> +At first, you need to be the root user to run this script.
>> +To run all testcases:
>> +
>> + $ sudo ./ftracetest
>> +
>> +To run specific testcases:
>> +
>> + # ./ftracetest test.d/basic3.tc
>> +
>> +Or you can also run testcases under given directory:
>> +
>> + # ./ftracetest test.d/kprobe/
>> +
>> +Contributing new testcases
>> +==========================
>> +
>> +Copy test.d/template to your testcase (whose filename must has *.tc extension)
>
> "whose filename must have *.tc extension"
>
>> +and rewrite test description line.
>
> "and rewrite the test description line"
Thanks, I'll update it.
>
>> +
>> + * The working directory of the script is <debugfs>/tracing/.
>> +
>> + * Take care about the side effect, because that is run with root privilege.
>
> "Take care with side effects as the tests are run with root privilege"
>
>> +
>> + * Don't take too long. It's a kind of unit test.
>
> "The tests should not run for a long period of time. These are to be
> unit tests"
>
> We should probably specify what the time should be. Is 20 minutes too
> long?
I guess more than 1 minute is enough long to doubt something
goes wrong. What unit test could be so long?
I think we need something like progress bar for such long tests.
>> +
>> + * You can add a directory for your testcases under test.d/ if needed.
>> +
>> + * The test cases should run on dash (busybox shell) for testing on
>> + minimal cross-build environments.
>> +
>> +TODO
>> +====
>> +
>> + * Fancy colored output :)
>> +
>> + * Integrate with selftest?
>> diff --git a/tools/testing/ftrace/ftracetest b/tools/testing/ftrace/ftracetest
>> new file mode 100755
>> index 0000000..8d92e74
>> --- /dev/null
>> +++ b/tools/testing/ftrace/ftracetest
>> @@ -0,0 +1,152 @@
>> +#!/bin/sh
>> +# ftracetest - Ftrace test shell scripts
>> +# Written by Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
>
> Maybe add your copyright here, with year?
Ah, right.
>
> Also, we should probably state that this is under the GPL v2?
OK, I'll add it. :)
Thank you,
>
> -- Steve
>
>> +
>> +usage() { # errno [message]
>> +[ "$2" ] && echo $2
>> +echo "Usage: ftracetest [options] [testcase(s)] [testcase-directory(s)]"
>> +echo " Options:"
>> +echo " -h|--help Show help message"
>> +echo " -k|--keep Keep passed test logs"
>> +echo " -d|--debug Debug mode (trace all shell commands)"
>> +exit $1
>> +}
>
--
Masami HIRAMATSU
Software Platform Research Dept. Linux Technology Research Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com
^ permalink raw reply [flat|nested] 14+ messages in thread
* [RFC PATCH v3 2/4] ftracetest: Add ftrace basic testcases
2014-09-02 11:06 [RFC PATCH v3 0/4] ftrace: Add a ftrace test collection Masami Hiramatsu
2014-09-02 11:06 ` [RFC PATCH v3 1/4] ftracetest: Initial commit for ftracetest Masami Hiramatsu
@ 2014-09-02 11:06 ` Masami Hiramatsu
2014-09-02 11:06 ` [RFC PATCH v3 3/4] ftracetest: Add kprobe " Masami Hiramatsu
` (2 subsequent siblings)
4 siblings, 0 replies; 14+ messages in thread
From: Masami Hiramatsu @ 2014-09-02 11:06 UTC (permalink / raw)
To: Shuah Khan, Tom Zanussi, Yoshihiro YUNOMAE, Oleg Nesterov,
Steven Rostedt, Namhyung Kim, Ingo Molnar
Cc: Linux Kernel Mailing List
Add ftrace basic testcases. This just checks ftrace debugfs
interface works as it is designed.
Changes in v2:
- Change shell to sh instead of bash.
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
---
tools/testing/ftrace/test.d/basic1.tc | 3 +++
tools/testing/ftrace/test.d/basic2.tc | 6 ++++++
tools/testing/ftrace/test.d/basic3.tc | 8 ++++++++
3 files changed, 17 insertions(+)
create mode 100644 tools/testing/ftrace/test.d/basic1.tc
create mode 100644 tools/testing/ftrace/test.d/basic2.tc
create mode 100644 tools/testing/ftrace/test.d/basic3.tc
diff --git a/tools/testing/ftrace/test.d/basic1.tc b/tools/testing/ftrace/test.d/basic1.tc
new file mode 100644
index 0000000..9980ff1
--- /dev/null
+++ b/tools/testing/ftrace/test.d/basic1.tc
@@ -0,0 +1,3 @@
+#!/bin/sh
+# description: Basic trace file check
+test -f README -a -f trace -a -f tracing_on -a -f trace_pipe
diff --git a/tools/testing/ftrace/test.d/basic2.tc b/tools/testing/ftrace/test.d/basic2.tc
new file mode 100644
index 0000000..b04f30d
--- /dev/null
+++ b/tools/testing/ftrace/test.d/basic2.tc
@@ -0,0 +1,6 @@
+#!/bin/sh
+# description: Basic test for tracers
+for t in `cat available_tracers`; do
+ echo $t > current_tracer || exit 1
+done
+echo nop > current_tracer
diff --git a/tools/testing/ftrace/test.d/basic3.tc b/tools/testing/ftrace/test.d/basic3.tc
new file mode 100644
index 0000000..0c1a3a2
--- /dev/null
+++ b/tools/testing/ftrace/test.d/basic3.tc
@@ -0,0 +1,8 @@
+#!/bin/sh
+# description: Basic trace clock test
+[ -f trace_clock ] || exit 1
+for c in `cat trace_clock | tr -d \[\]`; do
+ echo $c > trace_clock || exit 1
+ grep '\['$c'\]' trace_clock || exit 1
+done
+echo local > trace_clock
^ permalink raw reply related [flat|nested] 14+ messages in thread* [RFC PATCH v3 3/4] ftracetest: Add kprobe basic testcases
2014-09-02 11:06 [RFC PATCH v3 0/4] ftrace: Add a ftrace test collection Masami Hiramatsu
2014-09-02 11:06 ` [RFC PATCH v3 1/4] ftracetest: Initial commit for ftracetest Masami Hiramatsu
2014-09-02 11:06 ` [RFC PATCH v3 2/4] ftracetest: Add ftrace basic testcases Masami Hiramatsu
@ 2014-09-02 11:06 ` Masami Hiramatsu
2014-09-02 11:06 ` [RFC PATCH v3 4/4] ftracetest: Add XFAIL/XPASS and POSIX.3 std. result codes Masami Hiramatsu
2014-09-04 2:42 ` [RFC PATCH v3 0/4] ftrace: Add a ftrace test collection Namhyung Kim
4 siblings, 0 replies; 14+ messages in thread
From: Masami Hiramatsu @ 2014-09-02 11:06 UTC (permalink / raw)
To: Shuah Khan, Tom Zanussi, Yoshihiro YUNOMAE, Oleg Nesterov,
Steven Rostedt, Namhyung Kim, Ingo Molnar
Cc: Linux Kernel Mailing List
Add basic testcases for kprobe dynamic events.
This also shows that the ftracetest accepts sub-directory
for new testcases.
Changes in v2:
- Change shell to sh instead of bash.
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
---
.../testing/ftrace/test.d/kprobe/add_and_remove.tc | 11 +++++++++++
tools/testing/ftrace/test.d/kprobe/busy_check.tc | 14 ++++++++++++++
2 files changed, 25 insertions(+)
create mode 100644 tools/testing/ftrace/test.d/kprobe/add_and_remove.tc
create mode 100644 tools/testing/ftrace/test.d/kprobe/busy_check.tc
diff --git a/tools/testing/ftrace/test.d/kprobe/add_and_remove.tc b/tools/testing/ftrace/test.d/kprobe/add_and_remove.tc
new file mode 100644
index 0000000..5ddfb47
--- /dev/null
+++ b/tools/testing/ftrace/test.d/kprobe/add_and_remove.tc
@@ -0,0 +1,11 @@
+#!/bin/sh
+# description: Kprobe dynamic event - adding and removing
+
+[ -f kprobe_events ] || exit 1
+
+echo 0 > events/enable || exit 1
+echo > kprobe_events || exit 1
+echo p:myevent do_fork > kprobe_events || exit 1
+grep myevent kprobe_events || exit 1
+[ -d events/kprobes/myevent ] || exit 1
+echo > kprobe_events
diff --git a/tools/testing/ftrace/test.d/kprobe/busy_check.tc b/tools/testing/ftrace/test.d/kprobe/busy_check.tc
new file mode 100644
index 0000000..588fde97
--- /dev/null
+++ b/tools/testing/ftrace/test.d/kprobe/busy_check.tc
@@ -0,0 +1,14 @@
+#!/bin/sh
+# description: Kprobe dynamic event - busy event check
+
+[ -f kprobe_events ] || exit 1
+
+echo 0 > events/enable || exit 1
+echo > kprobe_events || exit 1
+echo p:myevent do_fork > kprobe_events || exit 1
+[ -d events/kprobes/myevent ] || exit 1
+echo 1 > events/kprobes/myevent/enable || exit 1
+echo > kprobe_events && exit 1 # this must fail
+echo 0 > events/kprobes/myevent/enable || exit 1
+echo > kprobe_events # this must succeed
+
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [RFC PATCH v3 4/4] ftracetest: Add XFAIL/XPASS and POSIX.3 std. result codes
2014-09-02 11:06 [RFC PATCH v3 0/4] ftrace: Add a ftrace test collection Masami Hiramatsu
` (2 preceding siblings ...)
2014-09-02 11:06 ` [RFC PATCH v3 3/4] ftracetest: Add kprobe " Masami Hiramatsu
@ 2014-09-02 11:06 ` Masami Hiramatsu
2014-09-04 2:29 ` Namhyung Kim
2014-09-04 2:42 ` [RFC PATCH v3 0/4] ftrace: Add a ftrace test collection Namhyung Kim
4 siblings, 1 reply; 14+ messages in thread
From: Masami Hiramatsu @ 2014-09-02 11:06 UTC (permalink / raw)
To: Shuah Khan, Tom Zanussi, Yoshihiro YUNOMAE, Oleg Nesterov,
Steven Rostedt, Namhyung Kim, Ingo Molnar
Cc: Linux Kernel Mailing List
Add XFAIL, XPASS and POSIX 1003.3 std. codes (UNRESOLVED/
UNTESTED/UNSUPPORTED) as a result code. These are
used for the results that test case is expected to fail or
unsupported feature (by config).
This also introduces PASS/FAIL/XFAIL/XPASS/UNSUP/UNTST/UNRES
result codes for each testcase. Since the results are not
binary, each testcase must use these codes to return the
test result.
Changes in v3:
- Add UNRESOLVED/UNTESTED codes.
- Fix to handle undefined codes.
- Add a document about return codes.
- Fix to show failure logs.
- Don't use -e option for echo since dash doesn't support it.
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
---
tools/testing/ftrace/README | 33 +++++++
tools/testing/ftrace/ftracetest | 95 +++++++++++++++++---
tools/testing/ftrace/test.d/basic1.tc | 6 +
tools/testing/ftrace/test.d/basic2.tc | 6 +
tools/testing/ftrace/test.d/basic3.tc | 9 +-
.../testing/ftrace/test.d/kprobe/add_and_remove.tc | 15 ++-
tools/testing/ftrace/test.d/kprobe/busy_check.tc | 20 ++--
tools/testing/ftrace/test.d/template | 8 +-
8 files changed, 152 insertions(+), 40 deletions(-)
diff --git a/tools/testing/ftrace/README b/tools/testing/ftrace/README
index 5d5de16..758c0c6 100644
--- a/tools/testing/ftrace/README
+++ b/tools/testing/ftrace/README
@@ -37,6 +37,39 @@ and rewrite test description line.
* The test cases should run on dash (busybox shell) for testing on
minimal cross-build environments.
+ * The test cases must return one of result codes described below.
+
+Return codes
+============
+
+Ftracetest supports following return codes, which are defined as shell
+variables (do not overwrite them.)
+
+ * PASS: The test succeeded as expected.
+
+ * FAIL: The test failed, but was expected to succeed.
+
+ * UNRESOLVED: The test produced unclear or intermidiate results.
+ for example, the test was interrupted
+ or the test depends on a previous test, which failed.
+ or the test was set up incorrectly
+
+ * UNTESTED: The test was not run, currently just a placeholder.
+
+ * UNSUPPORTED: The test failed because of lack of feature.
+
+ * XFAIL: The test failed, and was expected to fail.
+
+ * XPASS: The test succeeded, but was expected to fail.
+
+Below shortened valiables are also available.
+
+ * UNRES: means UNRESOLVED.
+
+ * UNTST: means UNTESTED.
+
+ * UNSUP: means UNSUPPORTED.
+
TODO
====
diff --git a/tools/testing/ftrace/ftracetest b/tools/testing/ftrace/ftracetest
index 8d92e74..3db77c5 100755
--- a/tools/testing/ftrace/ftracetest
+++ b/tools/testing/ftrace/ftracetest
@@ -107,22 +107,79 @@ catlog() { #file
}
# Testcase management
+# Test result codes - Dejagnu extended code
+PASS=0 # The test succeeded.
+FAIL=1 # The test failed, but was expected to succeed.
+UNRESOLVED=2 # The test produced indeterminate results. (e.g. interrupted)
+UNTESTED=3 # The test was not run, currently just a placeholder.
+UNSUPPORTED=4 # The test failed because of lack of feature.
+XFAIL=5 # The test failed, and was expected to fail.
+XPASS=6 # The test succeeded, but was expected to fail.
+# Shortened codes
+UNRES=$UNRESOLVED
+UNTST=$UNTESTED
+UNSUP=$UNSUPPORTED
+
+# Accumulations
PASSED_CASES=
FAILED_CASES=
+UNRESOLVED_CASES=
+UNTESTED_CASES=
+UNSUPPORTED_CASES=
+XFAILED_CASES=
+XPASSED_CASES=
+UNDEFINED_CASES=
+
CASENO=0
testcase() { # testfile
CASENO=$((CASENO+1))
prlog -n "[$CASENO]"`grep "^#[ \t]*description:" $1 | cut -f2 -d:`
}
-failed() {
- prlog " [FAIL]"
- FAILED_CASES="$FAILED_CASES $CASENO"
-}
-passed() {
- prlog " [PASS]"
- PASSED_CASES="$PASSED_CASES $CASENO"
-}
+eval_result() { # retval
+ case $1 in
+ $PASS)
+ prlog " [PASS]"
+ PASSED_CASES="$PASSED_CASES $CASENO"
+ return 0
+ ;;
+ $FAIL)
+ prlog " [FAIL]"
+ FAILED_CASES="$FAILED_CASES $CASENO"
+ return 1 # this is a bug.
+ ;;
+ $UNRESOLVED)
+ prlog " [UNRESOLVED]"
+ UNRESOLVED_CASES="$UNRESOLVED_CASES $CASENO"
+ return 1 # this is a kind of bug.. something happened.
+ ;;
+ $UNTESTED)
+ prlog " [UNTESTED]"
+ UNTESTED_CASES="$UNTESTED_CASES $CASENO"
+ return 0
+ ;;
+ $UNSUPPORTED)
+ prlog " [UNSUPPORTED]"
+ UNSUPPORTED_CASES="$UNSUPPORTED_CASES $CASENO"
+ return 0 # this is not a bug
+ ;;
+ $XFAIL)
+ prlog " [XFAIL]"
+ XFAILED_CASES="$XFAILED_CASES $CASENO"
+ return 0
+ ;;
+ $XPASS)
+ prlog " [XPASS]"
+ XPASSED_CASES="$XPASSED_CASES $CASENO"
+ return 1 # this can be a bug
+ ;;
+ *)
+ prlog " [UNDEFINED]"
+ UNDEFINED_CASES="$UNDEFINED_CASES $CASENO"
+ return 1 # this must be a test bug
+ ;;
+ esac
+}
# Run one test case
run_test() { # testfile
@@ -132,12 +189,12 @@ run_test() { # testfile
echo "execute: "$1 > $testlog
(cd $TRACING_DIR; set -x ; . $t) >> $testlog 2>&1
ret=$?
- if [ $ret -ne 0 ]; then
- failed
- catlog $testlog
- else
- passed
+ eval_result $ret
+ if [ $? -eq 0 ]; then
+ # Remove test log if the test was done as it was expected.
[ $KEEP_LOG -eq 0 ] && rm $testlog
+ else
+ catlog $testlog
fi
}
@@ -145,8 +202,16 @@ run_test() { # testfile
for t in $TEST_CASES; do
run_test $t
done
+
prlog ""
prlog "# of passed: " `echo $PASSED_CASES | wc -w`
prlog "# of failed: " `echo $FAILED_CASES | wc -w`
-
-test -z "$FAILED_CASES" # if no error, return 0
+prlog "# of unresolved: " `echo $UNRESOLVED_CASES | wc -w`
+prlog "# of untested: " `echo $UNTESTED_CASES | wc -w`
+prlog "# of unsupported: " `echo $UNSUPPORTED_CASES | wc -w`
+prlog "# of xfailed: " `echo $XFAILED_CASES | wc -w`
+prlog "# of xpassed: " `echo $XPASSED_CASES | wc -w`
+prlog "# of undefined(test bug): " `echo $UNDEFINED_CASES | wc -w`
+
+# if no error, return 0
+test -z "$FAILED_CASES$XPASSED_CASES$UNDEFINED_CASES$UNRESOLVED_CASES"
diff --git a/tools/testing/ftrace/test.d/basic1.tc b/tools/testing/ftrace/test.d/basic1.tc
index 9980ff1..1e95209 100644
--- a/tools/testing/ftrace/test.d/basic1.tc
+++ b/tools/testing/ftrace/test.d/basic1.tc
@@ -1,3 +1,7 @@
#!/bin/sh
# description: Basic trace file check
-test -f README -a -f trace -a -f tracing_on -a -f trace_pipe
+if test -f README -a -f trace -a -f tracing_on -a -f trace_pipe; then
+ exit $PASS
+else
+ exit $FAIL
+fi
diff --git a/tools/testing/ftrace/test.d/basic2.tc b/tools/testing/ftrace/test.d/basic2.tc
index b04f30d..44949e9 100644
--- a/tools/testing/ftrace/test.d/basic2.tc
+++ b/tools/testing/ftrace/test.d/basic2.tc
@@ -1,6 +1,8 @@
#!/bin/sh
# description: Basic test for tracers
+test -f available_tracers || exit $FAIL # this is basic feature, must be there.
for t in `cat available_tracers`; do
- echo $t > current_tracer || exit 1
+ echo $t > current_tracer || exit $FAIL
done
-echo nop > current_tracer
+echo nop > current_tracer || exit $FAIL
+exit $PASS
diff --git a/tools/testing/ftrace/test.d/basic3.tc b/tools/testing/ftrace/test.d/basic3.tc
index 0c1a3a2..7bc5a53 100644
--- a/tools/testing/ftrace/test.d/basic3.tc
+++ b/tools/testing/ftrace/test.d/basic3.tc
@@ -1,8 +1,9 @@
#!/bin/sh
# description: Basic trace clock test
-[ -f trace_clock ] || exit 1
+[ -f trace_clock ] || exit $FAIL # this is basic feature, must be there
for c in `cat trace_clock | tr -d \[\]`; do
- echo $c > trace_clock || exit 1
- grep '\['$c'\]' trace_clock || exit 1
+ echo $c > trace_clock || exit $FAIL
+ grep '\['$c'\]' trace_clock || exit $FAIL
done
-echo local > trace_clock
+echo local > trace_clock || exit $FAIL
+exit $PASS
diff --git a/tools/testing/ftrace/test.d/kprobe/add_and_remove.tc b/tools/testing/ftrace/test.d/kprobe/add_and_remove.tc
index 5ddfb47..fa3034b 100644
--- a/tools/testing/ftrace/test.d/kprobe/add_and_remove.tc
+++ b/tools/testing/ftrace/test.d/kprobe/add_and_remove.tc
@@ -1,11 +1,12 @@
#!/bin/sh
# description: Kprobe dynamic event - adding and removing
-[ -f kprobe_events ] || exit 1
+[ -f kprobe_events ] || exit $UNSUP # this is configurable
-echo 0 > events/enable || exit 1
-echo > kprobe_events || exit 1
-echo p:myevent do_fork > kprobe_events || exit 1
-grep myevent kprobe_events || exit 1
-[ -d events/kprobes/myevent ] || exit 1
-echo > kprobe_events
+echo 0 > events/enable || exit $FAIL
+echo > kprobe_events || exit $FAIL
+echo p:myevent do_fork > kprobe_events || exit $FAIL
+grep myevent kprobe_events || exit $FAIL
+[ -d events/kprobes/myevent ] || exit $FAIL
+echo > kprobe_events || exit $FAIL
+exit $PASS
diff --git a/tools/testing/ftrace/test.d/kprobe/busy_check.tc b/tools/testing/ftrace/test.d/kprobe/busy_check.tc
index 588fde97..6f47784 100644
--- a/tools/testing/ftrace/test.d/kprobe/busy_check.tc
+++ b/tools/testing/ftrace/test.d/kprobe/busy_check.tc
@@ -1,14 +1,14 @@
#!/bin/sh
# description: Kprobe dynamic event - busy event check
-[ -f kprobe_events ] || exit 1
-
-echo 0 > events/enable || exit 1
-echo > kprobe_events || exit 1
-echo p:myevent do_fork > kprobe_events || exit 1
-[ -d events/kprobes/myevent ] || exit 1
-echo 1 > events/kprobes/myevent/enable || exit 1
-echo > kprobe_events && exit 1 # this must fail
-echo 0 > events/kprobes/myevent/enable || exit 1
-echo > kprobe_events # this must succeed
+[ -f kprobe_events ] || exit $UNSUP
+echo 0 > events/enable || exit $FAIL
+echo > kprobe_events || exit $FAIL
+echo p:myevent do_fork > kprobe_events || exit $FAIL
+[ -d events/kprobes/myevent ] || exit $FAIL
+echo 1 > events/kprobes/myevent/enable || exit $FAIL
+echo > kprobe_events && exit $FAIL # this must fail
+echo 0 > events/kprobes/myevent/enable || exit $FAIL
+echo > kprobe_events || exit $FAIL # this must succeed
+exit $PASS
diff --git a/tools/testing/ftrace/test.d/template b/tools/testing/ftrace/test.d/template
index ce5f735..02b97af 100644
--- a/tools/testing/ftrace/test.d/template
+++ b/tools/testing/ftrace/test.d/template
@@ -1,4 +1,10 @@
#!/bin/sh
# description: %HERE DESCRIBE WHAT THIS DOES%
# you have to add ".tc" extention for your testcase file
-exit 0 # Return 0 if the test is passed, otherwise return !0
+exit $PASS # Return $PASS if the test succeeded.
+# If the test failed, return $FAIL.
+# If the test failed as expected, return $XFAIL.
+# If the test passed, but it was expected to fail, return $XPASS.
+# If the test could not run because of lack of feature, return $UNSUPPORTED
+# If the test returned unclear results, return $UNRESOLVED
+# If the test is a dummy, or a placeholder, return $UNTESTED
^ permalink raw reply related [flat|nested] 14+ messages in thread* Re: [RFC PATCH v3 4/4] ftracetest: Add XFAIL/XPASS and POSIX.3 std. result codes
2014-09-02 11:06 ` [RFC PATCH v3 4/4] ftracetest: Add XFAIL/XPASS and POSIX.3 std. result codes Masami Hiramatsu
@ 2014-09-04 2:29 ` Namhyung Kim
2014-09-04 4:39 ` Masami Hiramatsu
0 siblings, 1 reply; 14+ messages in thread
From: Namhyung Kim @ 2014-09-04 2:29 UTC (permalink / raw)
To: Masami Hiramatsu
Cc: Shuah Khan, Tom Zanussi, Yoshihiro YUNOMAE, Oleg Nesterov,
Steven Rostedt, Ingo Molnar, Linux Kernel Mailing List
On Tue, 02 Sep 2014 11:06:59 +0000, Masami Hiramatsu wrote:
> Add XFAIL, XPASS and POSIX 1003.3 std. codes (UNRESOLVED/
> UNTESTED/UNSUPPORTED) as a result code. These are
> used for the results that test case is expected to fail or
> unsupported feature (by config).
> This also introduces PASS/FAIL/XFAIL/XPASS/UNSUP/UNTST/UNRES
> result codes for each testcase. Since the results are not
> binary, each testcase must use these codes to return the
> test result.
>
> Changes in v3:
> - Add UNRESOLVED/UNTESTED codes.
> - Fix to handle undefined codes.
> - Add a document about return codes.
> - Fix to show failure logs.
> - Don't use -e option for echo since dash doesn't support it.
>
> Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
> ---
[SNIP]
> diff --git a/tools/testing/ftrace/test.d/basic3.tc b/tools/testing/ftrace/test.d/basic3.tc
> index 0c1a3a2..7bc5a53 100644
> --- a/tools/testing/ftrace/test.d/basic3.tc
> +++ b/tools/testing/ftrace/test.d/basic3.tc
> @@ -1,8 +1,9 @@
> #!/bin/sh
> # description: Basic trace clock test
> -[ -f trace_clock ] || exit 1
> +[ -f trace_clock ] || exit $FAIL # this is basic feature, must be there
> for c in `cat trace_clock | tr -d \[\]`; do
> - echo $c > trace_clock || exit 1
> - grep '\['$c'\]' trace_clock || exit 1
> + echo $c > trace_clock || exit $FAIL
> + grep '\['$c'\]' trace_clock || exit $FAIL
> done
> -echo local > trace_clock
> +echo local > trace_clock || exit $FAIL
> +exit $PASS
IMHO it's a bit sad that we need to write every test in this way -
adding '|| exit $FAIL' for every command. It'd be better if only
exceptional cases require that explicitly and normal cases live in
simple.
Can we do better - like using trap or something?
Thanks,
Namhyung
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Re: [RFC PATCH v3 4/4] ftracetest: Add XFAIL/XPASS and POSIX.3 std. result codes
2014-09-04 2:29 ` Namhyung Kim
@ 2014-09-04 4:39 ` Masami Hiramatsu
2014-09-08 10:41 ` Masami Hiramatsu
0 siblings, 1 reply; 14+ messages in thread
From: Masami Hiramatsu @ 2014-09-04 4:39 UTC (permalink / raw)
To: Namhyung Kim
Cc: Shuah Khan, Tom Zanussi, Yoshihiro YUNOMAE, Oleg Nesterov,
Steven Rostedt, Ingo Molnar, Linux Kernel Mailing List
(2014/09/04 11:29), Namhyung Kim wrote:
>> diff --git a/tools/testing/ftrace/test.d/basic3.tc b/tools/testing/ftrace/test.d/basic3.tc
>> index 0c1a3a2..7bc5a53 100644
>> --- a/tools/testing/ftrace/test.d/basic3.tc
>> +++ b/tools/testing/ftrace/test.d/basic3.tc
>> @@ -1,8 +1,9 @@
>> #!/bin/sh
>> # description: Basic trace clock test
>> -[ -f trace_clock ] || exit 1
>> +[ -f trace_clock ] || exit $FAIL # this is basic feature, must be there
>> for c in `cat trace_clock | tr -d \[\]`; do
>> - echo $c > trace_clock || exit 1
>> - grep '\['$c'\]' trace_clock || exit 1
>> + echo $c > trace_clock || exit $FAIL
>> + grep '\['$c'\]' trace_clock || exit $FAIL
>> done
>> -echo local > trace_clock
>> +echo local > trace_clock || exit $FAIL
>> +exit $PASS
>
> IMHO it's a bit sad that we need to write every test in this way -
> adding '|| exit $FAIL' for every command. It'd be better if only
> exceptional cases require that explicitly and normal cases live in
> simple.
>
> Can we do better - like using trap or something?
Hmm, no, trap is only for the signal trapping, and using signal
means run "kill" command for special errors. That may not be what
you want... :(
Perhaps, "set -e" may help you to simplify the script. But
since it also returns various exit code, you can not control it.
(I hope to have the "atexit" for dash...)
Anyway, IMHO, the test cases should express what happened clearly
since the test is a kind of verification process.
Thank you,
--
Masami HIRAMATSU
Software Platform Research Dept. Linux Technology Research Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Re: Re: [RFC PATCH v3 4/4] ftracetest: Add XFAIL/XPASS and POSIX.3 std. result codes
2014-09-04 4:39 ` Masami Hiramatsu
@ 2014-09-08 10:41 ` Masami Hiramatsu
0 siblings, 0 replies; 14+ messages in thread
From: Masami Hiramatsu @ 2014-09-08 10:41 UTC (permalink / raw)
To: Namhyung Kim
Cc: Shuah Khan, Tom Zanussi, Yoshihiro YUNOMAE, Oleg Nesterov,
Steven Rostedt, Ingo Molnar, Linux Kernel Mailing List
(2014/09/04 13:39), Masami Hiramatsu wrote:
> (2014/09/04 11:29), Namhyung Kim wrote:
>>> diff --git a/tools/testing/ftrace/test.d/basic3.tc b/tools/testing/ftrace/test.d/basic3.tc
>>> index 0c1a3a2..7bc5a53 100644
>>> --- a/tools/testing/ftrace/test.d/basic3.tc
>>> +++ b/tools/testing/ftrace/test.d/basic3.tc
>>> @@ -1,8 +1,9 @@
>>> #!/bin/sh
>>> # description: Basic trace clock test
>>> -[ -f trace_clock ] || exit 1
>>> +[ -f trace_clock ] || exit $FAIL # this is basic feature, must be there
>>> for c in `cat trace_clock | tr -d \[\]`; do
>>> - echo $c > trace_clock || exit 1
>>> - grep '\['$c'\]' trace_clock || exit 1
>>> + echo $c > trace_clock || exit $FAIL
>>> + grep '\['$c'\]' trace_clock || exit $FAIL
>>> done
>>> -echo local > trace_clock
>>> +echo local > trace_clock || exit $FAIL
>>> +exit $PASS
>>
>> IMHO it's a bit sad that we need to write every test in this way -
>> adding '|| exit $FAIL' for every command. It'd be better if only
>> exceptional cases require that explicitly and normal cases live in
>> simple.
>>
>> Can we do better - like using trap or something?
>
> Hmm, no, trap is only for the signal trapping, and using signal
> means run "kill" command for special errors. That may not be what
> you want... :(
I've changed my mind. I've checked that the possibility of simplification
of the testcases with signal traps. :)
Here I've added the following functions,
-----
exit_unsupported () {
kill -s $SIG_UNSUPPORTED $SIG_PID
exit 0
}
trap_unsupported () {
SIG_RESULT=$UNSUPPORTED
}
trap trap_unsupported $SIG_UNSUPPORTED
-----
And add "set -e;"(errexit) for sub-shell testcases.
-----
#!/bin/sh
# description: Kprobe dynamic event - adding and removing
[ -f kprobe_events ] || exit_unsupported
# If we have any error, sub shell exits soon with error code.
echo 0 > events/enable
echo > kprobe_events
echo p:myevent do_fork > kprobe_events
grep myevent kprobe_events
test -d events/kprobes/myevent
echo > kprobe_events
exit 0
-----
So, we'll have no return code in the testcases.
>
> Perhaps, "set -e" may help you to simplify the script. But
> since it also returns various exit code, you can not control it.
> (I hope to have the "atexit" for dash...)
I've found that trap with 0 can catch the exit of subshell, but
it can not know the exit code...
BTW, I've found that $PPID and $$ in subshell doesn't updated automatically.
PPID can be updated, but $$ cannnot. I'll pass $PID so that the testcases
can use it instead of $$. Or, should I use $SHELL to run testcases?
Thank you,
--
Masami HIRAMATSU
Software Platform Research Dept. Linux Technology Research Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [RFC PATCH v3 0/4] ftrace: Add a ftrace test collection
2014-09-02 11:06 [RFC PATCH v3 0/4] ftrace: Add a ftrace test collection Masami Hiramatsu
` (3 preceding siblings ...)
2014-09-02 11:06 ` [RFC PATCH v3 4/4] ftracetest: Add XFAIL/XPASS and POSIX.3 std. result codes Masami Hiramatsu
@ 2014-09-04 2:42 ` Namhyung Kim
2014-09-04 3:57 ` Masami Hiramatsu
4 siblings, 1 reply; 14+ messages in thread
From: Namhyung Kim @ 2014-09-04 2:42 UTC (permalink / raw)
To: Masami Hiramatsu
Cc: Shuah Khan, Tom Zanussi, Yoshihiro YUNOMAE, Oleg Nesterov,
Steven Rostedt, Ingo Molnar, Linux Kernel Mailing List
On Tue, 02 Sep 2014 11:06:32 +0000, Masami Hiramatsu wrote:
> Hi,
>
> Here is the collection of testcases for ftrace version 3.
> This is just some updates and fixes according to the
> discussion at previous version.
> https://lkml.org/lkml/2014/8/26/304
>
> The major updates are:
> - Fix some expressions to work on dash (Thanks Luis!)
> - Support UNRESOLVED/UNTESTED/UNSUPPORTED results for testcase.
> Those are POSIX 1003.3 standard.
> (see http://www.gnu.org/software/dejagnu/manual/x47.html for
> more details)
> - Fix to show error logs.
>
> ftracetest is a small dash script for testing ftrace features.
> It will be required to add a unit test for each new feature
> after this is merged, because of preventing feature regressions.
>
> This version still be out of the selftest. However, at the kernel
> summit, I talked with Shuah about that. And since the kselftest
> will be the standard kernel unit test framework, I agreed to
> move this under the selftest. But I also would like to ask Steven
> and Namhyung to agree with it before moving.
>
> Actually, since we've already has similar shell-script base
> testcase under kselftest, we can easily move this under
> selftests.
By [k]selftest, did you mean tools/testing/selftests/ ?
Anyway I have no objection to the move.
Thanks for your work!
Namhyung
^ permalink raw reply [flat|nested] 14+ messages in thread* Re: Re: [RFC PATCH v3 0/4] ftrace: Add a ftrace test collection
2014-09-04 2:42 ` [RFC PATCH v3 0/4] ftrace: Add a ftrace test collection Namhyung Kim
@ 2014-09-04 3:57 ` Masami Hiramatsu
0 siblings, 0 replies; 14+ messages in thread
From: Masami Hiramatsu @ 2014-09-04 3:57 UTC (permalink / raw)
To: Namhyung Kim
Cc: Shuah Khan, Tom Zanussi, Yoshihiro YUNOMAE, Oleg Nesterov,
Steven Rostedt, Ingo Molnar, Linux Kernel Mailing List
(2014/09/04 11:42), Namhyung Kim wrote:
> On Tue, 02 Sep 2014 11:06:32 +0000, Masami Hiramatsu wrote:
>> Hi,
>>
>> Here is the collection of testcases for ftrace version 3.
>> This is just some updates and fixes according to the
>> discussion at previous version.
>> https://lkml.org/lkml/2014/8/26/304
>>
>> The major updates are:
>> - Fix some expressions to work on dash (Thanks Luis!)
>> - Support UNRESOLVED/UNTESTED/UNSUPPORTED results for testcase.
>> Those are POSIX 1003.3 standard.
>> (see http://www.gnu.org/software/dejagnu/manual/x47.html for
>> more details)
>> - Fix to show error logs.
>>
>> ftracetest is a small dash script for testing ftrace features.
>> It will be required to add a unit test for each new feature
>> after this is merged, because of preventing feature regressions.
>>
>> This version still be out of the selftest. However, at the kernel
>> summit, I talked with Shuah about that. And since the kselftest
>> will be the standard kernel unit test framework, I agreed to
>> move this under the selftest. But I also would like to ask Steven
>> and Namhyung to agree with it before moving.
>>
>> Actually, since we've already has similar shell-script base
>> testcase under kselftest, we can easily move this under
>> selftests.
>
> By [k]selftest, did you mean tools/testing/selftests/ ?
Yes, and I will just move tools/testing/ftrace to
tools/testing/selftests/ftrace and add a Makefile.
> Anyway I have no objection to the move.
Thank you!
>
> Thanks for your work!
> Namhyung
>
--
Masami HIRAMATSU
Software Platform Research Dept. Linux Technology Research Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com
^ permalink raw reply [flat|nested] 14+ messages in thread