[PATCH] tests: xlate: print total no. of testfiles, tests and tests passed

[PATCH] tests: xlate: print total no. of testfiles, tests and tests passed

[Harsha Sharma]

Print errors and total no of tests and tests passed for testfile argument.
Print total no. of testfiles, total no. of tests and total no. of tests
passed for "all" argument.
Change testfile argument to take "extensions/test_file" as argument.

Signed-off-by: Harsha Sharma <harshasharmaiitr@gmail.com>
---
 xlate-test.py | 24 ++++++++++++++++++++----
 1 file changed, 20 insertions(+), 4 deletions(-)

diff --git a/xlate-test.py b/xlate-test.py
index 43c4be19..9927ac9c 100755
--- a/xlate-test.py
+++ b/xlate-test.py
@@ -9,7 +9,6 @@ from subprocess import Popen, PIPE
 
 keywords = ("iptables-translate", "ip6tables-translate")
 
-
 if sys.stdout.isatty():
     colors = {"magenta": "\033[95m", "green": "\033[92m", "yellow": "\033[93m",
               "red": "\033[91m", "end": "\033[0m"}
@@ -35,11 +34,14 @@ def green(string):
 
 def run_test(name, payload):
     test_passed = True
+    tests = 0
+    passed = 0
     result = []
     result.append(yellow("## " + name.replace(".txlate", "")))
 
     for line in payload:
         if line.startswith(keywords):
+            tests += 1
             process = Popen(shlex.split(line), stdout=PIPE, stderr=PIPE)
             (output, error) = process.communicate()
             if process.returncode == 0:
@@ -52,9 +54,12 @@ def run_test(name, payload):
                     result.append(magenta("res: ") + translation + "\n")
                     test_passed = False
                 elif args.all:
+                    passed += 1
                     result.append(green("Ok"))
                     result.append(magenta("src: ") + line.rstrip(" \n"))
                     result.append(magenta("res: ") + translation + "\n")
+                elif args.test:
+                    passed += 1
             else:
                 test_passed = False
                 result.append(red("Error: ") + "iptables-translate failure")
@@ -62,14 +67,25 @@ def run_test(name, payload):
 
     if not test_passed or args.all:
         print("\n".join(result))
+    if args.test:
+        print("1 test file, %d tests, %d tests passed" % (tests, passed))
+    else:
+        return tests, passed
 
 
 def load_test_files():
+    test_files = 0
+    total_tests = 0
+    total_passed = 0
     for test in sorted(os.listdir("extensions")):
         if test.endswith(".txlate"):
             with open("extensions/" + test, "r") as payload:
-                run_test(test, payload)
-
+                tests, passed = run_test(test, payload)
+                test_files += 1
+                total_tests += tests
+                total_passed += passed
+    if args.all:
+        print("%d test file, %d tests, %d tests passed" % (test_files, total_tests, total_passed))
 
 def main():
     if os.getuid() != 0:
@@ -78,7 +94,7 @@ def main():
         if not args.test.endswith(".txlate"):
             args.test += ".txlate"
         try:
-            with open("extensions/" + args.test, "r") as payload:
+            with open(args.test, "r") as payload:
                 run_test(args.test, payload)
         except IOError:
             print(red("Error: ") + "test file does not exist")
-- 
2.11.0

Re: [PATCH] tests: xlate: print total no. of testfiles, tests and tests passed

[Pablo Neira Ayuso]

On Wed, Oct 18, 2017 at 07:59:44PM +0530, Harsha Sharma wrote:
> Print errors and total no of tests and tests passed for testfile argument.
> Print total no. of testfiles, total no. of tests and total no. of tests
> passed for "all" argument.
> Change testfile argument to take "extensions/test_file" as argument.

Looks good, but general rule is:

        One logical change per patch.

I mean, this patch description says you're doing two things:

1) Print errors and total no of tests and tests passed for testfile
argument.

2) Change testfile argument to take "extensions/test_file" as
argument.

This means you have to send two patches, one for #1, and another one
for #2.

If you do so, git becomes way more useful when doing archeology, since
we know what patch added what single thing.

Re: [PATCH] tests: xlate: print total no. of testfiles, tests and tests passed

[Pablo Neira Ayuso]

On Wed, Oct 18, 2017 at 07:59:44PM +0530, Harsha Sharma wrote:
> Print errors and total no of tests and tests passed for testfile argument.
> Print total no. of testfiles, total no. of tests and total no. of tests
> passed for "all" argument.

BTW, could you also to the commit description an example on how the
output looks like, for people that don't want to apply and run it, so
we set clear expectations on what this is doing.

Thanks Harsha!

Re: [PATCH] tests: xlate: print total no. of testfiles, tests and tests passed

[Harsha Sharma]

On Thu, Oct 19, 2017 at 4:05 PM, Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> On Wed, Oct 18, 2017 at 07:59:44PM +0530, Harsha Sharma wrote:
>> Print errors and total no of tests and tests passed for testfile argument.
>> Print total no. of testfiles, total no. of tests and total no. of tests
>> passed for "all" argument.
>
> BTW, could you also to the commit description an example on how the
> output looks like, for people that don't want to apply and run it, so
> we set clear expectations on what this is doing.
>
> Thanks Harsha!

Thanks for your feedback.
I have sent two different patches.
Let me know if any change is required.
Thanks again :')

Regards,
Harsha Sharma