[PATCH v2] tests: xlate: print tests passed and error for testfile argument

[PATCH v2] tests: xlate: print tests passed and error for testfile argument

[Harsha Sharma]

Print tests passed with errors and warnings when run with only specified
test file.

Signed-off-by: Harsha Sharma <harshasharmaiitr@gmail.com>
---
Changes in v2:
 -Change subject
 -print the errors if no argument is passed

 xlate-test.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xlate-test.py b/xlate-test.py
index 43c4be19..0a582af1 100755
--- a/xlate-test.py
+++ b/xlate-test.py
@@ -51,7 +51,7 @@ def run_test(name, payload):
                     result.append(magenta("exp: ") + expected)
                     result.append(magenta("res: ") + translation + "\n")
                     test_passed = False
-                elif args.all:
+                elif args.all or args.test:
                     result.append(green("Ok"))
                     result.append(magenta("src: ") + line.rstrip(" \n"))
                     result.append(magenta("res: ") + translation + "\n")
@@ -60,7 +60,7 @@ def run_test(name, payload):
                 result.append(red("Error: ") + "iptables-translate failure")
                 result.append(error.decode("utf-8"))
 
-    if not test_passed or args.all:
+    if not test_passed or args.all or args.test:
         print("\n".join(result))
 
 
-- 
2.11.0

Re: [PATCH v2] tests: xlate: print tests passed and error for testfile argument

[Pablo Neira Ayuso]

On Tue, Oct 17, 2017 at 02:54:58PM +0530, Harsha Sharma wrote:
> Print tests passed with errors and warnings when run with only specified
> test file.

I would expect this works like this, for consistency with other test
infrastructure we have:

# python xlate-test.py extensions/libxt_ecn.txlate 
Error: test file does not exist

I can see here this works like this:

# python xlate-test.py libxt_ecn.txlate 

But as said, it is counterintuitive. Would you revisit this patch?

Thanks.

Re: [PATCH v2] tests: xlate: print tests passed and error for testfile argument

[Harsha Sharma]

On Tue, Oct 17, 2017 at 4:57 PM, Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> On Tue, Oct 17, 2017 at 02:54:58PM +0530, Harsha Sharma wrote:
>> Print tests passed with errors and warnings when run with only specified
>> test file.
>
> I would expect this works like this, for consistency with other test
> infrastructure we have:
>
> # python xlate-test.py extensions/libxt_ecn.txlate
> Error: test file does not exist
>
> I can see here this works like this:
>
> # python xlate-test.py libxt_ecn.txlate
>
> But as said, it is counterintuitive. Would you revisit this patch?
What I have tried to do int this patch is
for e.g.

# sudo python xlate-test.py libxt_ecn.txlate
will not result into any output as all tests are passed
but with this patch
# sudo python xlate-test.py libxt_ecn.txlate  (gives output)
## libxt_ecn
Ok
src: iptables-translate -A INPUT -m ecn --ecn-ip-ect 0
res: nft add rule ip filter INPUT ip ecn not-ect counter

Ok
src: iptables-translate -A INPUT -m ecn --ecn-ip-ect 1
res: nft add rule ip filter INPUT ip ecn ect1 counter

which is expected output as one should get to know if the tests are passed
or not .

Thanks. Please let me know if this is expected output or not.
>
> Thanks.

Re: [PATCH v2] tests: xlate: print tests passed and error for testfile argument

[Pablo Neira Ayuso]

On Tue, Oct 17, 2017 at 06:11:20PM +0530, Harsha Sharma wrote:
> On Tue, Oct 17, 2017 at 4:57 PM, Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> > On Tue, Oct 17, 2017 at 02:54:58PM +0530, Harsha Sharma wrote:
> >> Print tests passed with errors and warnings when run with only specified
> >> test file.
> >
> > I would expect this works like this, for consistency with other test
> > infrastructure we have:
> >
> > # python xlate-test.py extensions/libxt_ecn.txlate
> > Error: test file does not exist
> >
> > I can see here this works like this:
> >
> > # python xlate-test.py libxt_ecn.txlate
> >
> > But as said, it is counterintuitive. Would you revisit this patch?
> What I have tried to do int this patch is
> for e.g.
> 
> # sudo python xlate-test.py libxt_ecn.txlate
> will not result into any output as all tests are passed
> but with this patch
> # sudo python xlate-test.py libxt_ecn.txlate  (gives output)
> ## libxt_ecn
> Ok
> src: iptables-translate -A INPUT -m ecn --ecn-ip-ect 0
> res: nft add rule ip filter INPUT ip ecn not-ect counter
> 
> Ok
> src: iptables-translate -A INPUT -m ecn --ecn-ip-ect 1
> res: nft add rule ip filter INPUT ip ecn ect1 counter
> 
> which is expected output as one should get to know if the tests are passed
> or not .
>
> Thanks. Please let me know if this is expected output or not.

I see, thanks for explaining.

I would prefer you make this converge with the output of
iptables-test.py. It would be great if you can update the
xlate-tests.py so the output looks the same.

Thanks!

Re: [PATCH v2] tests: xlate: print tests passed and error for testfile argument

[Pablo Neira Ayuso]

On Tue, Oct 17, 2017 at 02:44:21PM +0200, Pablo Neira Ayuso wrote:
> On Tue, Oct 17, 2017 at 06:11:20PM +0530, Harsha Sharma wrote:
> > On Tue, Oct 17, 2017 at 4:57 PM, Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> > > On Tue, Oct 17, 2017 at 02:54:58PM +0530, Harsha Sharma wrote:
> > >> Print tests passed with errors and warnings when run with only specified
> > >> test file.
> > >
> > > I would expect this works like this, for consistency with other test
> > > infrastructure we have:
> > >
> > > # python xlate-test.py extensions/libxt_ecn.txlate
> > > Error: test file does not exist
> > >
> > > I can see here this works like this:
> > >
> > > # python xlate-test.py libxt_ecn.txlate
> > >
> > > But as said, it is counterintuitive. Would you revisit this patch?
> > What I have tried to do int this patch is
> > for e.g.
> > 
> > # sudo python xlate-test.py libxt_ecn.txlate
> > will not result into any output as all tests are passed
> > but with this patch
> > # sudo python xlate-test.py libxt_ecn.txlate  (gives output)
> > ## libxt_ecn
> > Ok
> > src: iptables-translate -A INPUT -m ecn --ecn-ip-ect 0
> > res: nft add rule ip filter INPUT ip ecn not-ect counter
> > 
> > Ok
> > src: iptables-translate -A INPUT -m ecn --ecn-ip-ect 1
> > res: nft add rule ip filter INPUT ip ecn ect1 counter
> > 
> > which is expected output as one should get to know if the tests are passed
> > or not .
> >
> > Thanks. Please let me know if this is expected output or not.
> 
> I see, thanks for explaining.
> 
> I would prefer you make this converge with the output of
> iptables-test.py. It would be great if you can update the
> xlate-tests.py so the output looks the same.

If you can fix xlate-tests.py to work with:

# python xlate-test.py extensions/libxt_ecn.txlate

that would be great too :-).