[PATCH net-next 1/1] tools: tc-testing: Can pause just before post-suite

[PATCH net-next 1/1] tools: tc-testing: Can pause just before post-suite

[Brenda J. Butler]

At debug level 5 or above, the test script will pause just before
the post_suite functions are called.  This allows the tester to
inspect the system before it is torn down.

Signed-off-by: Brenda J. Butler <bjb@mojatatu.com>
---
 tools/testing/selftests/tc-testing/tdc.py | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/tools/testing/selftests/tc-testing/tdc.py b/tools/testing/selftests/tc-testing/tdc.py
index 241eea37e4a4..3f5b0a696df2 100755
--- a/tools/testing/selftests/tc-testing/tdc.py
+++ b/tools/testing/selftests/tc-testing/tdc.py
@@ -308,6 +308,7 @@ def test_runner(pm, args, filtered_tests):
     # if we failed in setup or teardown,
     # fill in the remaining tests with ok-skipped
     count = index
+
     if not args.notap:
         tap += 'about to flush the tap output if tests need to be skipped\n'
         if tcount + 1 != index:
@@ -318,6 +319,12 @@ def test_runner(pm, args, filtered_tests):
                 count += 1
 
         tap += 'done flushing skipped test tap output\n'
+
+    if args.verbose > 4:
+        print('Want to pause\nPress enter to continue ...')
+        if input(sys.stdin):
+            print('got something on stdin')
+
     pm.call_post_suite(index)
 
     return tap
-- 
2.15.1

[PATCH net-next 1/1] tools: tc-testing: Can refer to $TESTID in test spec

[Brenda J. Butler]

At debug level 5 or above, the test script will pause just before
the post_suite functions are called.  This allows the tester to
inspect the system before it is torn down.

When processing the commands in the test cases, substitute
the test id for $TESTID.  This helps to make more flexible
tests.  For example, the testid can be given as a command
line argument.

As an example, if we wish to save the test output to a file
named for the test case, we can write in the test case:

"cmdUnderTest": "some test command | tee -a $TESTID.out"

Signed-off-by: Brenda J. Butler <bjb@mojatatu.com>
---
 tools/testing/selftests/tc-testing/tdc.py | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/tools/testing/selftests/tc-testing/tdc.py b/tools/testing/selftests/tc-testing/tdc.py
index 241eea37e4a4..c05b9f0f3db2 100755
--- a/tools/testing/selftests/tc-testing/tdc.py
+++ b/tools/testing/selftests/tc-testing/tdc.py
@@ -177,6 +177,7 @@ def prepare_env(args, pm, stage, prefix, cmdlist, output = None):
                 '"{}" did not complete successfully'.format(prefix))
 
 def run_one_test(pm, args, index, tidx):
+    global NAMES
     result = True
     tresult = ""
     tap = ""
@@ -184,6 +185,9 @@ def run_one_test(pm, args, index, tidx):
         print("\t====================\n=====> ", end="")
     print("Test " + tidx["id"] + ": " + tidx["name"])
 
+    # populate NAMES with TESTID for this test
+    NAMES['TESTID'] = tidx['id']
+
     pm.call_pre_case(index, tidx['id'])
     prepare_env(args, pm, 'setup', "-----> prepare stage", tidx["setup"])
 
@@ -227,6 +231,8 @@ def run_one_test(pm, args, index, tidx):
 
     index += 1
 
+    # remove TESTID from NAMES
+    del(NAMES['TESTID'])
     return tap
 
 def test_runner(pm, args, filtered_tests):
-- 
2.15.1

Re: [PATCH net-next 1/1] tools: tc-testing: Can pause just before post-suite

[David Miller]

From: "Brenda J. Butler" <bjb@mojatatu.com>
Date: Wed,  7 Mar 2018 09:27:14 -0500

> At debug level 5 or above, the test script will pause just before
> the post_suite functions are called.  This allows the tester to
> inspect the system before it is torn down.
> 
> Signed-off-by: Brenda J. Butler <bjb@mojatatu.com>

When someone adjusts the debug level, they expect that this is a control
over verbosity.

They don't expect it to potentially make the test hang at the TTY
waiting for user input.

Make a separate, appropriately named, knob for this.

Re: [PATCH net-next 1/1] tools: tc-testing: Can refer to $TESTID in test spec

[David Miller]

From: "Brenda J. Butler" <bjb@mojatatu.com>
Date: Wed,  7 Mar 2018 09:27:15 -0500

> At debug level 5 or above, the test script will pause just before
> the post_suite functions are called.  This allows the tester to
> inspect the system before it is torn down.

I've asked for changes in that patch, which means this needs updating
too.

Re: [PATCH net-next 1/1] tools: tc-testing: Can refer to $TESTID in test spec

[Brenda Butler]

The patches are independent and can be applied in any order.

I will change my process, to use send-email in separate invocations
for unrelated patches, so the patches are more obviously independent.
Sorry for the misleading submission.

Thanks for feedback.  I will resubmit the patches.

bjb

On Thu, Mar 8, 2018 at 12:45 PM, David Miller <davem@davemloft.net> wrote:
> From: "Brenda J. Butler" <bjb@mojatatu.com>
> Date: Wed,  7 Mar 2018 09:27:15 -0500
>
>> At debug level 5 or above, the test script will pause just before
>> the post_suite functions are called.  This allows the tester to
>> inspect the system before it is torn down.
>
> I've asked for changes in that patch, which means this needs updating
> too.