[PATCH 0/2] ftracetest: ftracetest updates

[PATCH 0/2] ftracetest: ftracetest updates

[Masami Hiramatsu]

Hi,

Here are two patches to improve ftracetest. The first one fixes
a bug in dash environment, and the second one adds --verbose to
show all stdout messages of the testcases when running ftracetest.

Thank you,

---

Masami Hiramatsu (2):
      ftracetest: Fix to show descriptions on dash
      ftracetest: Add --verbose option for showing echo output


 tools/testing/selftests/ftrace/ftracetest |   36 ++++++++++++++++++++---------
 1 file changed, 25 insertions(+), 11 deletions(-)

--

[PATCH 1/2] ftracetest: Fix to show descriptions on dash

[Masami Hiramatsu]

The ftracetest doesn't show testcase's descriptions when
it is executed on dash. This fixes that to show the
descriptions on dash correctly by passing it via a variable
instead of directly passing the grep command output.

Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
---
 tools/testing/selftests/ftrace/ftracetest |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
index 535b98b..e8b4021 100755
--- a/tools/testing/selftests/ftrace/ftracetest
+++ b/tools/testing/selftests/ftrace/ftracetest
@@ -135,7 +135,8 @@ TOTAL_RESULT=0
 CASENO=0
 testcase() { # testfile
   CASENO=$((CASENO+1))
-  prlog -n "[$CASENO]"`grep "^#[ \t]*description:" $1 | cut -f2 -d:`
+  desc=`grep "^#[ \t]*description:" $1 | cut -f2 -d:`
+  prlog -n "[$CASENO]$desc"
 }
 
 eval_result() { # retval sigval

[PATCH 2/2] ftracetest: Add --verbose option for showing echo output

[Masami Hiramatsu]

Add --verbose/-v option for showing echo output in testcases.
This is good for checking the progress of testcases which
take a longer time to run.

To implement this feature, all the testcase failures are
captured in ftracetest and send signal to set SIG_RESULT=FAIL.

Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
---
 tools/testing/selftests/ftrace/ftracetest |   33 ++++++++++++++++++++---------
 1 file changed, 23 insertions(+), 10 deletions(-)

diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
index e8b4021..da48812 100755
--- a/tools/testing/selftests/ftrace/ftracetest
+++ b/tools/testing/selftests/ftrace/ftracetest
@@ -13,6 +13,7 @@ echo "Usage: ftracetest [options] [testcase(s)] [testcase-directory(s)]"
 echo " Options:"
 echo "		-h|--help  Show help message"
 echo "		-k|--keep  Keep passed test logs"
+echo "		-v|--verbose Show all stdout messages in testcases"
 echo "		-d|--debug Debug mode (trace all shell commands)"
 exit $1
 }
@@ -53,6 +54,10 @@ parse_opts() { # opts
       KEEP_LOG=1
       shift 1
     ;;
+    --verbose|-v)
+      VERBOSE=1
+      shift 1
+    ;;
     --debug|-d)
       DEBUG=1
       shift 1
@@ -90,6 +95,7 @@ TEST_CASES=`find_testcases $TEST_DIR`
 LOG_DIR=$TOP_DIR/logs/`date +%Y%m%d-%H%M%S`/
 KEEP_LOG=0
 DEBUG=0
+VERBOSE=0
 # Parse command-line options
 parse_opts $*
 
@@ -139,12 +145,8 @@ testcase() { # testfile
   prlog -n "[$CASENO]$desc"
 }
 
-eval_result() { # retval sigval
-  local retval=$2
-  if [ $2 -eq 0 ]; then
-    test $1 -ne 0 && retval=$FAIL
-  fi
-  case $retval in
+eval_result() { # sigval
+  case $1 in
     $PASS)
       prlog "	[PASS]"
       PASSED_CASES="$PASSED_CASES $CASENO"
@@ -188,6 +190,9 @@ SIG_RESULT=
 SIG_BASE=36	# Use realtime signals
 SIG_PID=$$
 
+SIG_FAIL=$((SIG_BASE + FAIL))
+trap 'SIG_RESULT=$FAIL' $SIG_FAIL
+
 SIG_UNRESOLVED=$((SIG_BASE + UNRESOLVED))
 exit_unresolved () {
   kill -s $SIG_UNRESOLVED $SIG_PID
@@ -216,6 +221,12 @@ exit_xfail () {
 }
 trap 'SIG_RESULT=$XFAIL' $SIG_XFAIL
 
+__run_test() { # testfile
+  # setup PID and PPID, $$ is not updated.
+  (cd $TRACING_DIR; read PID _ < /proc/self/stat ; set -e; set -x; . $1)
+  [ $? -ne 0 ] && kill -s $SIG_FAIL $SIG_PID
+}
+
 # Run one test case
 run_test() { # testfile
   local testname=`basename $1`
@@ -223,10 +234,12 @@ run_test() { # testfile
   testcase $1
   echo "execute: "$1 > $testlog
   SIG_RESULT=0
-  # setup PID and PPID, $$ is not updated.
-  (cd $TRACING_DIR; read PID _ < /proc/self/stat ;
-   set -e; set -x; . $1) >> $testlog 2>&1
-  eval_result $? $SIG_RESULT
+  if [ $VERBOSE -ne 0 ]; then
+    __run_test $1 2>> $testlog | tee -a $testlog
+  else
+    __run_test $1 >> $testlog 2>&1
+  fi
+  eval_result $SIG_RESULT
   if [ $? -eq 0 ]; then
     # Remove test log if the test was done as it was expected.
     [ $KEEP_LOG -eq 0 ] && rm $testlog

Re: [PATCH 2/2] ftracetest: Add --verbose option for showing echo output

[Namhyung Kim]

On Thu, Dec 04, 2014 at 02:41:23PM -0500, Masami Hiramatsu wrote:
> Add --verbose/-v option for showing echo output in testcases.
> This is good for checking the progress of testcases which
> take a longer time to run.
> 
> To implement this feature, all the testcase failures are
> captured in ftracetest and send signal to set SIG_RESULT=FAIL.
> 
> Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
> ---

[SNIP]
> -  # setup PID and PPID, $$ is not updated.
> -  (cd $TRACING_DIR; read PID _ < /proc/self/stat ;
> -   set -e; set -x; . $1) >> $testlog 2>&1
> -  eval_result $? $SIG_RESULT
> +  if [ $VERBOSE -ne 0 ]; then
> +    __run_test $1 2>> $testlog | tee -a $testlog

Shouldn't it be

       __run_test $1 2>&1 | tee -a $testlog

?

Thanks,
Namhyung


> +  else
> +    __run_test $1 >> $testlog 2>&1
> +  fi
> +  eval_result $SIG_RESULT
>    if [ $? -eq 0 ]; then
>      # Remove test log if the test was done as it was expected.
>      [ $KEEP_LOG -eq 0 ] && rm $testlog

Re: [PATCH 2/2] ftracetest: Add --verbose option for showing echo output

[Masami Hiramatsu]

(2014/12/05 14:08), Namhyung Kim wrote:
> On Thu, Dec 04, 2014 at 02:41:23PM -0500, Masami Hiramatsu wrote:
>> Add --verbose/-v option for showing echo output in testcases.
>> This is good for checking the progress of testcases which
>> take a longer time to run.
>>
>> To implement this feature, all the testcase failures are
>> captured in ftracetest and send signal to set SIG_RESULT=FAIL.
>>
>> Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
>> ---
> 
> [SNIP]
>> -  # setup PID and PPID, $$ is not updated.
>> -  (cd $TRACING_DIR; read PID _ < /proc/self/stat ;
>> -   set -e; set -x; . $1) >> $testlog 2>&1
>> -  eval_result $? $SIG_RESULT
>> +  if [ $VERBOSE -ne 0 ]; then
>> +    __run_test $1 2>> $testlog | tee -a $testlog
> 
> Shouldn't it be
> 
>        __run_test $1 2>&1 | tee -a $testlog
> 
> ?

No, that outputs both stdout and stderr to $testlog and console.
What I'd like to do above is only stdout to $testlog and console and
stderr goes only to $testlog. (Note that __run_test set -x which outputs
every executed command to stderr)

Thank you,

> 
> Thanks,
> Namhyung
> 
> 
>> +  else
>> +    __run_test $1 >> $testlog 2>&1
>> +  fi
>> +  eval_result $SIG_RESULT
>>    if [ $? -eq 0 ]; then
>>      # Remove test log if the test was done as it was expected.
>>      [ $KEEP_LOG -eq 0 ] && rm $testlog
> 


-- 
Masami HIRAMATSU
Software Platform Research Dept. Linux Technology Research Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com

Re: [PATCH 2/2] ftracetest: Add --verbose option for showing echo output

[Namhyung Kim]

On Fri, Dec 05, 2014 at 02:23:23PM +0900, Masami Hiramatsu wrote:
> (2014/12/05 14:08), Namhyung Kim wrote:
> > On Thu, Dec 04, 2014 at 02:41:23PM -0500, Masami Hiramatsu wrote:
> >> Add --verbose/-v option for showing echo output in testcases.
> >> This is good for checking the progress of testcases which
> >> take a longer time to run.
> >>
> >> To implement this feature, all the testcase failures are
> >> captured in ftracetest and send signal to set SIG_RESULT=FAIL.
> >>
> >> Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
> >> ---
> > 
> > [SNIP]
> >> -  # setup PID and PPID, $$ is not updated.
> >> -  (cd $TRACING_DIR; read PID _ < /proc/self/stat ;
> >> -   set -e; set -x; . $1) >> $testlog 2>&1
> >> -  eval_result $? $SIG_RESULT
> >> +  if [ $VERBOSE -ne 0 ]; then
> >> +    __run_test $1 2>> $testlog | tee -a $testlog
> > 
> > Shouldn't it be
> > 
> >        __run_test $1 2>&1 | tee -a $testlog
> > 
> > ?
> 
> No, that outputs both stdout and stderr to $testlog and console.
> What I'd like to do above is only stdout to $testlog and console and
> stderr goes only to $testlog. (Note that __run_test set -x which outputs
> every executed command to stderr)

Hmm.. so the actual output of the failing command is not shown on
console even when -v option is given, right?

Anyway I worried about that the order of messages might be mixed..

  echo msg1 > /dev/stdout
  echo msg2 > /dev/stderr
  echo msg3 > /dev/stdout
  echo msg4 > /dev/stderr

So the output in this case can be like msg2, msg4, msg1, and msg3..?

Thanks,
Namhyung

Re: [PATCH 2/2] ftracetest: Add --verbose option for showing echo output

[Masami Hiramatsu]

(2014/12/05 14:52), Namhyung Kim wrote:
> On Fri, Dec 05, 2014 at 02:23:23PM +0900, Masami Hiramatsu wrote:
>> (2014/12/05 14:08), Namhyung Kim wrote:
>>> On Thu, Dec 04, 2014 at 02:41:23PM -0500, Masami Hiramatsu wrote:
>>>> Add --verbose/-v option for showing echo output in testcases.
>>>> This is good for checking the progress of testcases which
>>>> take a longer time to run.
>>>>
>>>> To implement this feature, all the testcase failures are
>>>> captured in ftracetest and send signal to set SIG_RESULT=FAIL.
>>>>
>>>> Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
>>>> ---
>>>
>>> [SNIP]
>>>> -  # setup PID and PPID, $$ is not updated.
>>>> -  (cd $TRACING_DIR; read PID _ < /proc/self/stat ;
>>>> -   set -e; set -x; . $1) >> $testlog 2>&1
>>>> -  eval_result $? $SIG_RESULT
>>>> +  if [ $VERBOSE -ne 0 ]; then
>>>> +    __run_test $1 2>> $testlog | tee -a $testlog
>>>
>>> Shouldn't it be
>>>
>>>        __run_test $1 2>&1 | tee -a $testlog
>>>
>>> ?
>>
>> No, that outputs both stdout and stderr to $testlog and console.
>> What I'd like to do above is only stdout to $testlog and console and
>> stderr goes only to $testlog. (Note that __run_test set -x which outputs
>> every executed command to stderr)
> 
> Hmm.. so the actual output of the failing command is not shown on
> console even when -v option is given, right?

No, if a command fails, the testcase is automatically terminated
(by set -e) and $testlog is dumped on console, so that tester can
trace back the log.

> Anyway I worried about that the order of messages might be mixed..
> 
>   echo msg1 > /dev/stdout
>   echo msg2 > /dev/stderr
>   echo msg3 > /dev/stdout
>   echo msg4 > /dev/stderr
> 
> So the output in this case can be like msg2, msg4, msg1, and msg3..?

I'm not sure this can happen... Would you have any other good way to
solve this?

Thank you,

-- 
Masami HIRAMATSU
Software Platform Research Dept. Linux Technology Research Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com