[Buildroot] [PATCH 0/5] a few more runtime test cases for python packages - series 1/n

[Buildroot] [PATCH 0/5] a few more runtime test cases for python packages - series 1/n

[Ricardo Martincoski]

One way to reduce the burden of manual tests and also detecting broken runtime
dependencies is to have more test cases (even simple ones) for python packages.
Well... here are some.

Complete run on gitlab CI:
https://gitlab.com/RicardoMartincoski/buildroot/pipelines/31209399

Regards,
Ricardo

Ricardo Martincoski (5):
  support/testing: add python-autobahn tests
  support/testing: add python-txaio tests
  support/testing: add python-txtorcon tests
  support/testing: add python-incremental tests
  support/testing: add python-twisted tests

 .gitlab-ci.yml                                | 10 ++++
 .../tests/package/test_python_autobahn.py     | 32 +++++++++++
 .../tests/package/test_python_incremental.py  | 34 ++++++++++++
 .../tests/package/test_python_twisted.py      | 54 +++++++++++++++++++
 .../tests/package/test_python_txaio.py        | 34 ++++++++++++
 .../tests/package/test_python_txtorcon.py     | 32 +++++++++++
 6 files changed, 196 insertions(+)
 create mode 100644 support/testing/tests/package/test_python_autobahn.py
 create mode 100644 support/testing/tests/package/test_python_incremental.py
 create mode 100644 support/testing/tests/package/test_python_twisted.py
 create mode 100644 support/testing/tests/package/test_python_txaio.py
 create mode 100644 support/testing/tests/package/test_python_txtorcon.py

-- 
2.17.1

[Buildroot] [PATCH 1/5] support/testing: add python-autobahn tests

[Ricardo Martincoski]

Add a simple test case that imports the module.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
---
 .gitlab-ci.yml                                |  2 ++
 .../tests/package/test_python_autobahn.py     | 32 +++++++++++++++++++
 2 files changed, 34 insertions(+)
 create mode 100644 support/testing/tests/package/test_python_autobahn.py

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index b220d544f8..001ea43f72 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -299,6 +299,8 @@ tests.package.test_ipython.TestIPythonPy2: *runtime_test
 tests.package.test_ipython.TestIPythonPy3: *runtime_test
 tests.package.test_python.TestPython2: *runtime_test
 tests.package.test_python.TestPython3: *runtime_test
+tests.package.test_python_autobahn.TestPythonPy2Autobahn: *runtime_test
+tests.package.test_python_autobahn.TestPythonPy3Autobahn: *runtime_test
 tests.package.test_python_cryptography.TestPythonPy2Cryptography: *runtime_test
 tests.package.test_python_cryptography.TestPythonPy3Cryptography: *runtime_test
 tests.package.test_rust.TestRust: *runtime_test
diff --git a/support/testing/tests/package/test_python_autobahn.py b/support/testing/tests/package/test_python_autobahn.py
new file mode 100644
index 0000000000..2bc0f0cccf
--- /dev/null
+++ b/support/testing/tests/package/test_python_autobahn.py
@@ -0,0 +1,32 @@
+from tests.package.test_python import TestPythonBase
+
+
+class TestPythonAutobahn(TestPythonBase):
+    def import_test(self):
+        cmd = self.interpreter + " -c 'import autobahn.wamp'"
+        _, exit_code = self.emulator.run(cmd)
+        self.assertEqual(exit_code, 0)
+
+
+class TestPythonPy2Autobahn(TestPythonAutobahn):
+    config = TestPythonBase.config + \
+        """
+        BR2_PACKAGE_PYTHON=y
+        BR2_PACKAGE_PYTHON_AUTOBAHN=y
+        """
+
+    def test_run(self):
+        self.login()
+        self.import_test()
+
+
+class TestPythonPy3Autobahn(TestPythonAutobahn):
+    config = TestPythonBase.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        BR2_PACKAGE_PYTHON_AUTOBAHN=y
+        """
+
+    def test_run(self):
+        self.login()
+        self.import_test()
-- 
2.17.1

[Buildroot] [PATCH 1/5] support/testing: add python-autobahn tests

[Yegor Yefremov]

On Wed, Sep 26, 2018 at 4:57 AM Ricardo Martincoski
<ricardo.martincoski@gmail.com> wrote:
>
> Add a simple test case that imports the module.
>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
> Cc: Asaf Kahlon <asafka7@gmail.com>
> Cc: Yegor Yefremov <yegorslists@googlemail.com>

Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>

> ---
>  .gitlab-ci.yml                                |  2 ++
>  .../tests/package/test_python_autobahn.py     | 32 +++++++++++++++++++
>  2 files changed, 34 insertions(+)
>  create mode 100644 support/testing/tests/package/test_python_autobahn.py
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index b220d544f8..001ea43f72 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -299,6 +299,8 @@ tests.package.test_ipython.TestIPythonPy2: *runtime_test
>  tests.package.test_ipython.TestIPythonPy3: *runtime_test
>  tests.package.test_python.TestPython2: *runtime_test
>  tests.package.test_python.TestPython3: *runtime_test
> +tests.package.test_python_autobahn.TestPythonPy2Autobahn: *runtime_test
> +tests.package.test_python_autobahn.TestPythonPy3Autobahn: *runtime_test
>  tests.package.test_python_cryptography.TestPythonPy2Cryptography: *runtime_test
>  tests.package.test_python_cryptography.TestPythonPy3Cryptography: *runtime_test
>  tests.package.test_rust.TestRust: *runtime_test
> diff --git a/support/testing/tests/package/test_python_autobahn.py b/support/testing/tests/package/test_python_autobahn.py
> new file mode 100644
> index 0000000000..2bc0f0cccf
> --- /dev/null
> +++ b/support/testing/tests/package/test_python_autobahn.py
> @@ -0,0 +1,32 @@
> +from tests.package.test_python import TestPythonBase
> +
> +
> +class TestPythonAutobahn(TestPythonBase):
> +    def import_test(self):
> +        cmd = self.interpreter + " -c 'import autobahn.wamp'"
> +        _, exit_code = self.emulator.run(cmd)
> +        self.assertEqual(exit_code, 0)
> +
> +
> +class TestPythonPy2Autobahn(TestPythonAutobahn):
> +    config = TestPythonBase.config + \
> +        """
> +        BR2_PACKAGE_PYTHON=y
> +        BR2_PACKAGE_PYTHON_AUTOBAHN=y
> +        """
> +
> +    def test_run(self):
> +        self.login()
> +        self.import_test()
> +
> +
> +class TestPythonPy3Autobahn(TestPythonAutobahn):
> +    config = TestPythonBase.config + \
> +        """
> +        BR2_PACKAGE_PYTHON3=y
> +        BR2_PACKAGE_PYTHON_AUTOBAHN=y
> +        """
> +
> +    def test_run(self):
> +        self.login()
> +        self.import_test()
> --
> 2.17.1
>

[Buildroot] [PATCH 2/5] support/testing: add python-txaio tests

[Ricardo Martincoski]

Add a simple test case that imports the module to use with twisted in
Python 2 and with asyncio in Python 3.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
---
 .gitlab-ci.yml                                |  2 ++
 .../tests/package/test_python_txaio.py        | 34 +++++++++++++++++++
 2 files changed, 36 insertions(+)
 create mode 100644 support/testing/tests/package/test_python_txaio.py

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 001ea43f72..609ea0cdb5 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -303,6 +303,8 @@ tests.package.test_python_autobahn.TestPythonPy2Autobahn: *runtime_test
 tests.package.test_python_autobahn.TestPythonPy3Autobahn: *runtime_test
 tests.package.test_python_cryptography.TestPythonPy2Cryptography: *runtime_test
 tests.package.test_python_cryptography.TestPythonPy3Cryptography: *runtime_test
+tests.package.test_python_txaio.TestPythonPy2Txaio: *runtime_test
+tests.package.test_python_txaio.TestPythonPy3Txaio: *runtime_test
 tests.package.test_rust.TestRust: *runtime_test
 tests.package.test_rust.TestRustBin: *runtime_test
 tests.package.test_syslog_ng.TestSyslogNg: *runtime_test
diff --git a/support/testing/tests/package/test_python_txaio.py b/support/testing/tests/package/test_python_txaio.py
new file mode 100644
index 0000000000..af93e031b5
--- /dev/null
+++ b/support/testing/tests/package/test_python_txaio.py
@@ -0,0 +1,34 @@
+from tests.package.test_python import TestPythonBase
+
+
+class TestPythonPy2Txaio(TestPythonBase):
+    config = TestPythonBase.config + \
+        """
+        BR2_PACKAGE_PYTHON=y
+        BR2_PACKAGE_PYTHON_TXAIO=y
+        BR2_PACKAGE_PYTHON_TWISTED=y
+        """
+
+    def test_run(self):
+        self.login()
+        cmd = self.interpreter + " -c 'import txaio;"
+        cmd += "txaio.use_twisted();"
+        cmd += "f0 = txaio.create_future()'"
+        _, exit_code = self.emulator.run(cmd)
+        self.assertEqual(exit_code, 0)
+
+
+class TestPythonPy3Txaio(TestPythonBase):
+    config = TestPythonBase.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        BR2_PACKAGE_PYTHON_TXAIO=y
+        """
+
+    def test_run(self):
+        self.login()
+        cmd = self.interpreter + " -c 'import txaio;"
+        cmd += "txaio.use_asyncio();"
+        cmd += "f0 = txaio.create_future()'"
+        _, exit_code = self.emulator.run(cmd)
+        self.assertEqual(exit_code, 0)
-- 
2.17.1

[Buildroot] [PATCH 2/5] support/testing: add python-txaio tests

[Yegor Yefremov]

On Wed, Sep 26, 2018 at 4:57 AM Ricardo Martincoski
<ricardo.martincoski@gmail.com> wrote:
>
> Add a simple test case that imports the module to use with twisted in
> Python 2 and with asyncio in Python 3.
>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
> Cc: Asaf Kahlon <asafka7@gmail.com>
> Cc: Yegor Yefremov <yegorslists@googlemail.com>

Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>

> ---
>  .gitlab-ci.yml                                |  2 ++
>  .../tests/package/test_python_txaio.py        | 34 +++++++++++++++++++
>  2 files changed, 36 insertions(+)
>  create mode 100644 support/testing/tests/package/test_python_txaio.py
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 001ea43f72..609ea0cdb5 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -303,6 +303,8 @@ tests.package.test_python_autobahn.TestPythonPy2Autobahn: *runtime_test
>  tests.package.test_python_autobahn.TestPythonPy3Autobahn: *runtime_test
>  tests.package.test_python_cryptography.TestPythonPy2Cryptography: *runtime_test
>  tests.package.test_python_cryptography.TestPythonPy3Cryptography: *runtime_test
> +tests.package.test_python_txaio.TestPythonPy2Txaio: *runtime_test
> +tests.package.test_python_txaio.TestPythonPy3Txaio: *runtime_test
>  tests.package.test_rust.TestRust: *runtime_test
>  tests.package.test_rust.TestRustBin: *runtime_test
>  tests.package.test_syslog_ng.TestSyslogNg: *runtime_test
> diff --git a/support/testing/tests/package/test_python_txaio.py b/support/testing/tests/package/test_python_txaio.py
> new file mode 100644
> index 0000000000..af93e031b5
> --- /dev/null
> +++ b/support/testing/tests/package/test_python_txaio.py
> @@ -0,0 +1,34 @@
> +from tests.package.test_python import TestPythonBase
> +
> +
> +class TestPythonPy2Txaio(TestPythonBase):
> +    config = TestPythonBase.config + \
> +        """
> +        BR2_PACKAGE_PYTHON=y
> +        BR2_PACKAGE_PYTHON_TXAIO=y
> +        BR2_PACKAGE_PYTHON_TWISTED=y
> +        """
> +
> +    def test_run(self):
> +        self.login()
> +        cmd = self.interpreter + " -c 'import txaio;"
> +        cmd += "txaio.use_twisted();"
> +        cmd += "f0 = txaio.create_future()'"
> +        _, exit_code = self.emulator.run(cmd)
> +        self.assertEqual(exit_code, 0)
> +
> +
> +class TestPythonPy3Txaio(TestPythonBase):
> +    config = TestPythonBase.config + \
> +        """
> +        BR2_PACKAGE_PYTHON3=y
> +        BR2_PACKAGE_PYTHON_TXAIO=y
> +        """
> +
> +    def test_run(self):
> +        self.login()
> +        cmd = self.interpreter + " -c 'import txaio;"
> +        cmd += "txaio.use_asyncio();"
> +        cmd += "f0 = txaio.create_future()'"
> +        _, exit_code = self.emulator.run(cmd)
> +        self.assertEqual(exit_code, 0)
> --
> 2.17.1
>

[Buildroot] [PATCH 3/5] support/testing: add python-txtorcon tests

[Ricardo Martincoski]

Add a simple test case that imports the module.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
---
NOTICE that TestPythonPy3Txtorcon currently fails on master branch like this:

Cannot compile usr/lib/python3.7/site-packages/twisted/conch/manhole.py
Traceback (most recent call last):
  File "/builds/RicardoMartincoski/buildroot/test-output/TestPythonPy3Txtorcon/host/lib/python3.7/py_compile.py", line 136, in compile
    _optimize=optimize)
  File "<frozen importlib._bootstrap_external>", line 763, in source_to_code
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "usr/lib/python3.7/site-packages/twisted/conch/manhole.py", line 154
    def write(self, data, async=False):
                              ^
SyntaxError: invalid syntax
---
 .gitlab-ci.yml                                |  2 ++
 .../tests/package/test_python_txtorcon.py     | 32 +++++++++++++++++++
 2 files changed, 34 insertions(+)
 create mode 100644 support/testing/tests/package/test_python_txtorcon.py

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 609ea0cdb5..fcbb066cf6 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -305,6 +305,8 @@ tests.package.test_python_cryptography.TestPythonPy2Cryptography: *runtime_test
 tests.package.test_python_cryptography.TestPythonPy3Cryptography: *runtime_test
 tests.package.test_python_txaio.TestPythonPy2Txaio: *runtime_test
 tests.package.test_python_txaio.TestPythonPy3Txaio: *runtime_test
+tests.package.test_python_txtorcon.TestPythonPy2Txtorcon: *runtime_test
+tests.package.test_python_txtorcon.TestPythonPy3Txtorcon: *runtime_test
 tests.package.test_rust.TestRust: *runtime_test
 tests.package.test_rust.TestRustBin: *runtime_test
 tests.package.test_syslog_ng.TestSyslogNg: *runtime_test
diff --git a/support/testing/tests/package/test_python_txtorcon.py b/support/testing/tests/package/test_python_txtorcon.py
new file mode 100644
index 0000000000..352ff67825
--- /dev/null
+++ b/support/testing/tests/package/test_python_txtorcon.py
@@ -0,0 +1,32 @@
+from tests.package.test_python import TestPythonBase
+
+
+class TestPythonTxtorcon(TestPythonBase):
+    def import_test(self):
+        cmd = self.interpreter + " -c 'import txtorcon'"
+        _, exit_code = self.emulator.run(cmd, timeout=30)
+        self.assertEqual(exit_code, 0)
+
+
+class TestPythonPy2Txtorcon(TestPythonTxtorcon):
+    config = TestPythonBase.config + \
+        """
+        BR2_PACKAGE_PYTHON=y
+        BR2_PACKAGE_PYTHON_TXTORCON=y
+        """
+
+    def test_run(self):
+        self.login()
+        self.import_test()
+
+
+class TestPythonPy3Txtorcon(TestPythonTxtorcon):
+    config = TestPythonBase.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        BR2_PACKAGE_PYTHON_TXTORCON=y
+        """
+
+    def test_run(self):
+        self.login()
+        self.import_test()
-- 
2.17.1

[Buildroot] [PATCH 3/5] support/testing: add python-txtorcon tests

[Yegor Yefremov]

On Wed, Sep 26, 2018 at 4:57 AM Ricardo Martincoski
<ricardo.martincoski@gmail.com> wrote:
>
> Add a simple test case that imports the module.
>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
> Cc: Asaf Kahlon <asafka7@gmail.com>
> Cc: Yegor Yefremov <yegorslists@googlemail.com>

Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>

> ---
> NOTICE that TestPythonPy3Txtorcon currently fails on master branch like this:
>
> Cannot compile usr/lib/python3.7/site-packages/twisted/conch/manhole.py
> Traceback (most recent call last):
>   File "/builds/RicardoMartincoski/buildroot/test-output/TestPythonPy3Txtorcon/host/lib/python3.7/py_compile.py", line 136, in compile
>     _optimize=optimize)
>   File "<frozen importlib._bootstrap_external>", line 763, in source_to_code
>   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
>   File "usr/lib/python3.7/site-packages/twisted/conch/manhole.py", line 154
>     def write(self, data, async=False):
>                               ^
> SyntaxError: invalid syntax
> ---
>  .gitlab-ci.yml                                |  2 ++
>  .../tests/package/test_python_txtorcon.py     | 32 +++++++++++++++++++
>  2 files changed, 34 insertions(+)
>  create mode 100644 support/testing/tests/package/test_python_txtorcon.py
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 609ea0cdb5..fcbb066cf6 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -305,6 +305,8 @@ tests.package.test_python_cryptography.TestPythonPy2Cryptography: *runtime_test
>  tests.package.test_python_cryptography.TestPythonPy3Cryptography: *runtime_test
>  tests.package.test_python_txaio.TestPythonPy2Txaio: *runtime_test
>  tests.package.test_python_txaio.TestPythonPy3Txaio: *runtime_test
> +tests.package.test_python_txtorcon.TestPythonPy2Txtorcon: *runtime_test
> +tests.package.test_python_txtorcon.TestPythonPy3Txtorcon: *runtime_test
>  tests.package.test_rust.TestRust: *runtime_test
>  tests.package.test_rust.TestRustBin: *runtime_test
>  tests.package.test_syslog_ng.TestSyslogNg: *runtime_test
> diff --git a/support/testing/tests/package/test_python_txtorcon.py b/support/testing/tests/package/test_python_txtorcon.py
> new file mode 100644
> index 0000000000..352ff67825
> --- /dev/null
> +++ b/support/testing/tests/package/test_python_txtorcon.py
> @@ -0,0 +1,32 @@
> +from tests.package.test_python import TestPythonBase
> +
> +
> +class TestPythonTxtorcon(TestPythonBase):
> +    def import_test(self):
> +        cmd = self.interpreter + " -c 'import txtorcon'"
> +        _, exit_code = self.emulator.run(cmd, timeout=30)
> +        self.assertEqual(exit_code, 0)
> +
> +
> +class TestPythonPy2Txtorcon(TestPythonTxtorcon):
> +    config = TestPythonBase.config + \
> +        """
> +        BR2_PACKAGE_PYTHON=y
> +        BR2_PACKAGE_PYTHON_TXTORCON=y
> +        """
> +
> +    def test_run(self):
> +        self.login()
> +        self.import_test()
> +
> +
> +class TestPythonPy3Txtorcon(TestPythonTxtorcon):
> +    config = TestPythonBase.config + \
> +        """
> +        BR2_PACKAGE_PYTHON3=y
> +        BR2_PACKAGE_PYTHON_TXTORCON=y
> +        """
> +
> +    def test_run(self):
> +        self.login()
> +        self.import_test()
> --
> 2.17.1
>

[Buildroot] [PATCH 4/5] support/testing: add python-incremental tests

[Ricardo Martincoski]

Add a simple test case that imports the module and asserts a version
string for a fake package is generated.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
---
 .gitlab-ci.yml                                |  2 ++
 .../tests/package/test_python_incremental.py  | 34 +++++++++++++++++++
 2 files changed, 36 insertions(+)
 create mode 100644 support/testing/tests/package/test_python_incremental.py

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index fcbb066cf6..ab3493686d 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -303,6 +303,8 @@ tests.package.test_python_autobahn.TestPythonPy2Autobahn: *runtime_test
 tests.package.test_python_autobahn.TestPythonPy3Autobahn: *runtime_test
 tests.package.test_python_cryptography.TestPythonPy2Cryptography: *runtime_test
 tests.package.test_python_cryptography.TestPythonPy3Cryptography: *runtime_test
+tests.package.test_python_incremental.TestPythonPy2Incremental: *runtime_test
+tests.package.test_python_incremental.TestPythonPy3Incremental: *runtime_test
 tests.package.test_python_txaio.TestPythonPy2Txaio: *runtime_test
 tests.package.test_python_txaio.TestPythonPy3Txaio: *runtime_test
 tests.package.test_python_txtorcon.TestPythonPy2Txtorcon: *runtime_test
diff --git a/support/testing/tests/package/test_python_incremental.py b/support/testing/tests/package/test_python_incremental.py
new file mode 100644
index 0000000000..acf743cdd2
--- /dev/null
+++ b/support/testing/tests/package/test_python_incremental.py
@@ -0,0 +1,34 @@
+from tests.package.test_python import TestPythonBase
+
+
+class TestPythonIncremental(TestPythonBase):
+    def str_test(self):
+        cmd = self.interpreter + " -c 'import incremental;"
+        cmd += "v = incremental.Version(\"package\", 1, 2, 3, release_candidate=4);"
+        cmd += "assert(str(v) == \"[package, version 1.2.3rc4]\")'"
+        _, exit_code = self.emulator.run(cmd, timeout=30)
+        self.assertEqual(exit_code, 0)
+
+
+class TestPythonPy2Incremental(TestPythonIncremental):
+    config = TestPythonBase.config + \
+        """
+        BR2_PACKAGE_PYTHON=y
+        BR2_PACKAGE_PYTHON_INCREMENTAL=y
+        """
+
+    def test_run(self):
+        self.login()
+        self.str_test()
+
+
+class TestPythonPy3Incremental(TestPythonIncremental):
+    config = TestPythonBase.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        BR2_PACKAGE_PYTHON_INCREMENTAL=y
+        """
+
+    def test_run(self):
+        self.login()
+        self.str_test()
-- 
2.17.1

[Buildroot] [PATCH 4/5] support/testing: add python-incremental tests

[Yegor Yefremov]

On Wed, Sep 26, 2018 at 4:57 AM Ricardo Martincoski
<ricardo.martincoski@gmail.com> wrote:
>
> Add a simple test case that imports the module and asserts a version
> string for a fake package is generated.
>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
> Cc: Asaf Kahlon <asafka7@gmail.com>
> Cc: Yegor Yefremov <yegorslists@googlemail.com>

Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>

> ---
>  .gitlab-ci.yml                                |  2 ++
>  .../tests/package/test_python_incremental.py  | 34 +++++++++++++++++++
>  2 files changed, 36 insertions(+)
>  create mode 100644 support/testing/tests/package/test_python_incremental.py
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index fcbb066cf6..ab3493686d 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -303,6 +303,8 @@ tests.package.test_python_autobahn.TestPythonPy2Autobahn: *runtime_test
>  tests.package.test_python_autobahn.TestPythonPy3Autobahn: *runtime_test
>  tests.package.test_python_cryptography.TestPythonPy2Cryptography: *runtime_test
>  tests.package.test_python_cryptography.TestPythonPy3Cryptography: *runtime_test
> +tests.package.test_python_incremental.TestPythonPy2Incremental: *runtime_test
> +tests.package.test_python_incremental.TestPythonPy3Incremental: *runtime_test
>  tests.package.test_python_txaio.TestPythonPy2Txaio: *runtime_test
>  tests.package.test_python_txaio.TestPythonPy3Txaio: *runtime_test
>  tests.package.test_python_txtorcon.TestPythonPy2Txtorcon: *runtime_test
> diff --git a/support/testing/tests/package/test_python_incremental.py b/support/testing/tests/package/test_python_incremental.py
> new file mode 100644
> index 0000000000..acf743cdd2
> --- /dev/null
> +++ b/support/testing/tests/package/test_python_incremental.py
> @@ -0,0 +1,34 @@
> +from tests.package.test_python import TestPythonBase
> +
> +
> +class TestPythonIncremental(TestPythonBase):
> +    def str_test(self):
> +        cmd = self.interpreter + " -c 'import incremental;"
> +        cmd += "v = incremental.Version(\"package\", 1, 2, 3, release_candidate=4);"
> +        cmd += "assert(str(v) == \"[package, version 1.2.3rc4]\")'"
> +        _, exit_code = self.emulator.run(cmd, timeout=30)
> +        self.assertEqual(exit_code, 0)
> +
> +
> +class TestPythonPy2Incremental(TestPythonIncremental):
> +    config = TestPythonBase.config + \
> +        """
> +        BR2_PACKAGE_PYTHON=y
> +        BR2_PACKAGE_PYTHON_INCREMENTAL=y
> +        """
> +
> +    def test_run(self):
> +        self.login()
> +        self.str_test()
> +
> +
> +class TestPythonPy3Incremental(TestPythonIncremental):
> +    config = TestPythonBase.config + \
> +        """
> +        BR2_PACKAGE_PYTHON3=y
> +        BR2_PACKAGE_PYTHON_INCREMENTAL=y
> +        """
> +
> +    def test_run(self):
> +        self.login()
> +        self.str_test()
> --
> 2.17.1
>

[Buildroot] [PATCH 4/5] support/testing: add python-incremental tests

[Thomas Petazzoni]

Hello,

On Tue, 25 Sep 2018 23:55:13 -0300, Ricardo Martincoski wrote:

> +class TestPythonIncremental(TestPythonBase):
> +    def str_test(self):
> +        cmd = self.interpreter + " -c 'import incremental;"
> +        cmd += "v = incremental.Version(\"package\", 1, 2, 3, release_candidate=4);"
> +        cmd += "assert(str(v) == \"[package, version 1.2.3rc4]\")'"

When I see this, I wonder if it wouldn't be nicer to do something like
this:

class PythonIncremental(TestPythonBase):
	python_prog = """
import incremental
incremental.Version("package", 1, 2, 3, release_candidate=4)
assert(str(v) == "[package, version 1.2.3rc4]")
"""

and have the base TestPythonBase class have some logic that executes
this as a Python script with the Python interpreter.

It would make the thing a bit more readable. Also, the TestPythonBase
class could push things a bit further, and have a default
implementation of test_run() that does self.login() + execute the
Python program in python_prog.

This way, those simple Python tests only need to provide the Buildroot
configuration and the small Python program to run.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH 4/5] support/testing: add python-incremental tests

[Ricardo Martincoski]

Hello,

On Thu, Oct 11, 2018 at 03:46 PM, Thomas Petazzoni wrote:

> On Tue, 25 Sep 2018 23:55:13 -0300, Ricardo Martincoski wrote:
> 
>> +class TestPythonIncremental(TestPythonBase):
>> +    def str_test(self):
>> +        cmd = self.interpreter + " -c 'import incremental;"
>> +        cmd += "v = incremental.Version(\"package\", 1, 2, 3, release_candidate=4);"
>> +        cmd += "assert(str(v) == \"[package, version 1.2.3rc4]\")'"
> 
> When I see this, I wonder if it wouldn't be nicer to do something like
> this:
> 
> class PythonIncremental(TestPythonBase):
> 	python_prog = """
> import incremental
> incremental.Version("package", 1, 2, 3, release_candidate=4)
> assert(str(v) == "[package, version 1.2.3rc4]")
> """
> 
> and have the base TestPythonBase class have some logic that executes
> this as a Python script with the Python interpreter.
> 
> It would make the thing a bit more readable. Also, the TestPythonBase
> class could push things a bit further, and have a default
> implementation of test_run() that does self.login() + execute the
> Python program in python_prog.
> 
> This way, those simple Python tests only need to provide the Buildroot
> configuration and the small Python program to run.

Good idea.
I am working on this, I will send a small series in the next few days: one
patch adding the common logic and expecting the sample script to be in a file
to be copied in build time and 6 patches each one changing one of the current
test cases.


Regards,
Ricardo

[Buildroot] [PATCH 5/5] support/testing: add python-twisted tests

[Ricardo Martincoski]

Use a minimal script to listen to a port and check using netstat.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
---
NOTICE that TestPythonPy3Twisted currently fails on master branch like this:
Cannot compile usr/lib/python3.7/site-packages/twisted/conch/manhole.py
Traceback (most recent call last):
  File "/builds/RicardoMartincoski/buildroot/test-output/TestPythonPy3Twisted/host/lib/python3.7/py_compile.py", line 136, in compile
    _optimize=optimize)
  File "<frozen importlib._bootstrap_external>", line 763, in source_to_code
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "usr/lib/python3.7/site-packages/twisted/conch/manhole.py", line 154
    def write(self, data, async=False):
                              ^
SyntaxError: invalid syntax
---
 .gitlab-ci.yml                                |  2 +
 .../tests/package/test_python_twisted.py      | 54 +++++++++++++++++++
 2 files changed, 56 insertions(+)
 create mode 100644 support/testing/tests/package/test_python_twisted.py

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index ab3493686d..4b8fee5f23 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -305,6 +305,8 @@ tests.package.test_python_cryptography.TestPythonPy2Cryptography: *runtime_test
 tests.package.test_python_cryptography.TestPythonPy3Cryptography: *runtime_test
 tests.package.test_python_incremental.TestPythonPy2Incremental: *runtime_test
 tests.package.test_python_incremental.TestPythonPy3Incremental: *runtime_test
+tests.package.test_python_twisted.TestPythonPy2Twisted: *runtime_test
+tests.package.test_python_twisted.TestPythonPy3Twisted: *runtime_test
 tests.package.test_python_txaio.TestPythonPy2Txaio: *runtime_test
 tests.package.test_python_txaio.TestPythonPy3Txaio: *runtime_test
 tests.package.test_python_txtorcon.TestPythonPy2Txtorcon: *runtime_test
diff --git a/support/testing/tests/package/test_python_twisted.py b/support/testing/tests/package/test_python_twisted.py
new file mode 100644
index 0000000000..88de30edee
--- /dev/null
+++ b/support/testing/tests/package/test_python_twisted.py
@@ -0,0 +1,54 @@
+from tests.package.test_python import TestPythonBase
+
+TEST_SCRIPT = """
+from twisted.internet import protocol, reactor, endpoints
+class F(protocol.Factory):
+    pass
+endpoints.serverFromString(reactor, "tcp:1234").listen(F())
+reactor.run()
+"""
+
+
+class TestPythonTwisted(TestPythonBase):
+    def import_test(self):
+        cmd = "printf '{}' > test.py".format(TEST_SCRIPT)
+        _, exit_code = self.emulator.run(cmd)
+        self.assertEqual(exit_code, 0)
+
+        cmd = "netstat -ltn 2>/dev/null | grep 0.0.0.0:1234"
+        _, exit_code = self.emulator.run(cmd)
+        self.assertEqual(exit_code, 1)
+
+        cmd = self.interpreter + " test.py &"
+        # give some time to setup the server
+        cmd += "sleep 5"
+        _, exit_code = self.emulator.run(cmd, timeout=10)
+        self.assertEqual(exit_code, 0)
+
+        cmd = "netstat -ltn 2>/dev/null | grep 0.0.0.0:1234"
+        _, exit_code = self.emulator.run(cmd)
+        self.assertEqual(exit_code, 0)
+
+
+class TestPythonPy2Twisted(TestPythonTwisted):
+    config = TestPythonBase.config + \
+        """
+        BR2_PACKAGE_PYTHON=y
+        BR2_PACKAGE_PYTHON_TWISTED=y
+        """
+
+    def test_run(self):
+        self.login()
+        self.import_test()
+
+
+class TestPythonPy3Twisted(TestPythonTwisted):
+    config = TestPythonBase.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        BR2_PACKAGE_PYTHON_TWISTED=y
+        """
+
+    def test_run(self):
+        self.login()
+        self.import_test()
-- 
2.17.1

[Buildroot] [PATCH 5/5] support/testing: add python-twisted tests

[Yegor Yefremov]

On Wed, Sep 26, 2018 at 4:57 AM Ricardo Martincoski
<ricardo.martincoski@gmail.com> wrote:
>
> Use a minimal script to listen to a port and check using netstat.
>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
> Cc: Asaf Kahlon <asafka7@gmail.com>
> Cc: Yegor Yefremov <yegorslists@googlemail.com>

Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>

So basically the whole stuff is waiting till the Twisted project
would release a new version or we apply
https://github.com/twisted/twisted/pull/966

Yegor

> ---
> NOTICE that TestPythonPy3Twisted currently fails on master branch like this:
> Cannot compile usr/lib/python3.7/site-packages/twisted/conch/manhole.py
> Traceback (most recent call last):
>   File "/builds/RicardoMartincoski/buildroot/test-output/TestPythonPy3Twisted/host/lib/python3.7/py_compile.py", line 136, in compile
>     _optimize=optimize)
>   File "<frozen importlib._bootstrap_external>", line 763, in source_to_code
>   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
>   File "usr/lib/python3.7/site-packages/twisted/conch/manhole.py", line 154
>     def write(self, data, async=False):
>                               ^
> SyntaxError: invalid syntax
> ---
>  .gitlab-ci.yml                                |  2 +
>  .../tests/package/test_python_twisted.py      | 54 +++++++++++++++++++
>  2 files changed, 56 insertions(+)
>  create mode 100644 support/testing/tests/package/test_python_twisted.py
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index ab3493686d..4b8fee5f23 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -305,6 +305,8 @@ tests.package.test_python_cryptography.TestPythonPy2Cryptography: *runtime_test
>  tests.package.test_python_cryptography.TestPythonPy3Cryptography: *runtime_test
>  tests.package.test_python_incremental.TestPythonPy2Incremental: *runtime_test
>  tests.package.test_python_incremental.TestPythonPy3Incremental: *runtime_test
> +tests.package.test_python_twisted.TestPythonPy2Twisted: *runtime_test
> +tests.package.test_python_twisted.TestPythonPy3Twisted: *runtime_test
>  tests.package.test_python_txaio.TestPythonPy2Txaio: *runtime_test
>  tests.package.test_python_txaio.TestPythonPy3Txaio: *runtime_test
>  tests.package.test_python_txtorcon.TestPythonPy2Txtorcon: *runtime_test
> diff --git a/support/testing/tests/package/test_python_twisted.py b/support/testing/tests/package/test_python_twisted.py
> new file mode 100644
> index 0000000000..88de30edee
> --- /dev/null
> +++ b/support/testing/tests/package/test_python_twisted.py
> @@ -0,0 +1,54 @@
> +from tests.package.test_python import TestPythonBase
> +
> +TEST_SCRIPT = """
> +from twisted.internet import protocol, reactor, endpoints
> +class F(protocol.Factory):
> +    pass
> +endpoints.serverFromString(reactor, "tcp:1234").listen(F())
> +reactor.run()
> +"""
> +
> +
> +class TestPythonTwisted(TestPythonBase):
> +    def import_test(self):
> +        cmd = "printf '{}' > test.py".format(TEST_SCRIPT)
> +        _, exit_code = self.emulator.run(cmd)
> +        self.assertEqual(exit_code, 0)
> +
> +        cmd = "netstat -ltn 2>/dev/null | grep 0.0.0.0:1234"
> +        _, exit_code = self.emulator.run(cmd)
> +        self.assertEqual(exit_code, 1)
> +
> +        cmd = self.interpreter + " test.py &"
> +        # give some time to setup the server
> +        cmd += "sleep 5"
> +        _, exit_code = self.emulator.run(cmd, timeout=10)
> +        self.assertEqual(exit_code, 0)
> +
> +        cmd = "netstat -ltn 2>/dev/null | grep 0.0.0.0:1234"
> +        _, exit_code = self.emulator.run(cmd)
> +        self.assertEqual(exit_code, 0)
> +
> +
> +class TestPythonPy2Twisted(TestPythonTwisted):
> +    config = TestPythonBase.config + \
> +        """
> +        BR2_PACKAGE_PYTHON=y
> +        BR2_PACKAGE_PYTHON_TWISTED=y
> +        """
> +
> +    def test_run(self):
> +        self.login()
> +        self.import_test()
> +
> +
> +class TestPythonPy3Twisted(TestPythonTwisted):
> +    config = TestPythonBase.config + \
> +        """
> +        BR2_PACKAGE_PYTHON3=y
> +        BR2_PACKAGE_PYTHON_TWISTED=y
> +        """
> +
> +    def test_run(self):
> +        self.login()
> +        self.import_test()
> --
> 2.17.1
>

[Buildroot] [PATCH 5/5] support/testing: add python-twisted tests

[Ricardo Martincoski]

Hello,

On Tue, Sep 25, 2018 at 11:55 PM, Ricardo Martincoski wrote:

[snip]
> +        cmd = self.interpreter + " test.py &"
> +        # give some time to setup the server
> +        cmd += "sleep 5"
> +        _, exit_code = self.emulator.run(cmd, timeout=10)

NOTE: results below include the entire series:
http://patchwork.ozlabs.org/project/buildroot/list/?series=67693

Turns out that 5 seconds is enough to setup the server when running in Python 2,
but not in Python 3.
https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/102456734 FAIL

So I will need to change the sleep time (and the timeout passed to .run()) to
something like this:
        cmd += "sleep 30"
        _, exit_code = self.emulator.run(cmd, timeout=35)
https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/102483479 PASS

> +        self.assertEqual(exit_code, 0)
> +
> +        cmd = "netstat -ltn 2>/dev/null | grep 0.0.0.0:1234"
> +        _, exit_code = self.emulator.run(cmd)
> +        self.assertEqual(exit_code, 0)
[snip]

Can I remove only this patch from the series (it is the last) and move it to my
next series adding more test cases for python packages? Or should I resend this
series (5 patches) right now?
It could also be fixed while applying.


Regards,
Ricardo

[Buildroot] [PATCH 5/5] support/testing: add python-twisted tests

[Thomas Petazzoni]

Hello,

On Tue, 25 Sep 2018 23:55:14 -0300, Ricardo Martincoski wrote:

> +TEST_SCRIPT = """
> +from twisted.internet import protocol, reactor, endpoints
> +class F(protocol.Factory):
> +    pass
> +endpoints.serverFromString(reactor, "tcp:1234").listen(F())
> +reactor.run()
> +"""

At some point, we will have to find a solution to store such
"artefacts" somewhere, instead of having them inline in their test
cases. I don't have a good idea about this.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH 5/5] support/testing: add python-twisted tests

[Arnout Vandecappelle]

On 11/10/18 20:48, Thomas Petazzoni wrote:
> Hello,
> 
> On Tue, 25 Sep 2018 23:55:14 -0300, Ricardo Martincoski wrote:
> 
>> +TEST_SCRIPT = """
>> +from twisted.internet import protocol, reactor, endpoints
>> +class F(protocol.Factory):
>> +    pass
>> +endpoints.serverFromString(reactor, "tcp:1234").listen(F())
>> +reactor.run()
>> +"""
> 
> At some point, we will have to find a solution to store such
> "artefacts" somewhere, instead of having them inline in their test
> cases. I don't have a good idea about this.

 I think it would be nice if we could add in the package directory a
subdirectory with files to be copied to the target for the runtime test, and a
python script for the test itself.

 As an in-between step, we can put the files-to-be-copied inside the
support/testing directory and explicitly enumerate the files-to-be-copied in the
test fixture.

 Regards,
 Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH 5/5] support/testing: add python-twisted tests

[Thomas Petazzoni]

Hello,

On Thu, 11 Oct 2018 23:27:10 +0200, Arnout Vandecappelle wrote:

> > At some point, we will have to find a solution to store such
> > "artefacts" somewhere, instead of having them inline in their test
> > cases. I don't have a good idea about this.  
> 
>  I think it would be nice if we could add in the package directory a
> subdirectory with files to be copied to the target for the runtime test, and a
> python script for the test itself.
> 
>  As an in-between step, we can put the files-to-be-copied inside the
> support/testing directory and explicitly enumerate the files-to-be-copied in the
> test fixture.

I am still hesitant between putting the tests + fixtures near/in the
packages themselves, and keeping the whole test suite thing separate.
One issue with putting the tests+fixture near/in the packages is that a
number of tests aren't package-related, so where do we put them? Isn't
that also going to put a lot of mess/cruft in our packages?

I don't have a good answer to that. Perhaps for now it makes more sense
to have those test fixtures in support/testing/, and see where to go
from there.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH 5/5] support/testing: add python-twisted tests

[Yann E. MORIN]

Arnout, All,

On 2018-10-11 23:27 +0200, Arnout Vandecappelle spake thusly:
> On 11/10/18 20:48, Thomas Petazzoni wrote:
> > Hello,
> > 
> > On Tue, 25 Sep 2018 23:55:14 -0300, Ricardo Martincoski wrote:
> > 
> >> +TEST_SCRIPT = """
> >> +from twisted.internet import protocol, reactor, endpoints
> >> +class F(protocol.Factory):
> >> +    pass
> >> +endpoints.serverFromString(reactor, "tcp:1234").listen(F())
> >> +reactor.run()
> >> +"""
> > 
> > At some point, we will have to find a solution to store such
> > "artefacts" somewhere, instead of having them inline in their test
> > cases. I don't have a good idea about this.
> 
>  I think it would be nice if we could add in the package directory a
> subdirectory with files to be copied to the target for the runtime test, and a
> python script for the test itself.

I'm not sure that would answer the question, but I already worked on
moving the packages' tests files inside the packages directories:
    https://git.buildroot.org/~ymorin/git/buildroot/log/?h=yem/package-tests

(Note that this branch is not entirely up-to-date, as I haven't updated
it since we added new package tests.)

Contrary to Thomas, I don't really mind that we have tests in two
locations. Per-paclages tests are, in y opinion, better to be
side-by-side with the package they test.

As for the test on the infra, they go where they make more sense: in the
test infra.

My 2cts.

Regards,
Yann E. MORIN.

>  As an in-between step, we can put the files-to-be-copied inside the
> support/testing directory and explicitly enumerate the files-to-be-copied in the
> test fixture.
> 
>  Regards,
>  Arnout
> 
> -- 
> Arnout Vandecappelle                          arnout at mind be
> Senior Embedded Software Architect            +32-16-286500
> Essensium/Mind                                http://www.mind.be
> G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
> LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
> GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 5/5] support/testing: add python-twisted tests

[Ricardo Martincoski]

Hello,

On Sat, Oct 13, 2018 at 11:43 AM, Yann E. MORIN wrote:

> On 2018-10-11 23:27 +0200, Arnout Vandecappelle spake thusly:
>> On 11/10/18 20:48, Thomas Petazzoni wrote:
>> > On Tue, 25 Sep 2018 23:55:14 -0300, Ricardo Martincoski wrote:
>> > 
>> >> +TEST_SCRIPT = """
>> >> +from twisted.internet import protocol, reactor, endpoints
>> >> +class F(protocol.Factory):
>> >> +    pass
>> >> +endpoints.serverFromString(reactor, "tcp:1234").listen(F())
>> >> +reactor.run()
>> >> +"""
>> > 
>> > At some point, we will have to find a solution to store such
>> > "artefacts" somewhere, instead of having them inline in their test
>> > cases. I don't have a good idea about this.
>> 
>>  I think it would be nice if we could add in the package directory a
>> subdirectory with files to be copied to the target for the runtime test, and a
>> python script for the test itself.
> 
> I'm not sure that would answer the question, but I already worked on
> moving the packages' tests files inside the packages directories:
>     https://git.buildroot.org/~ymorin/git/buildroot/log/?h=yem/package-tests
> 
> (Note that this branch is not entirely up-to-date, as I haven't updated
> it since we added new package tests.)
> 
> Contrary to Thomas, I don't really mind that we have tests in two
> locations. Per-paclages tests are, in y opinion, better to be
> side-by-side with the package they test.

I agree with Yann.

I think some structure like this (exact naming yet to be defined, of course) is
really nice:

package/
\-- pkg1
    |-- Config.in
    |-- fixtures
    |?? \-- sample_pkg1.py
    |-- pkg1.hash
    |-- pkg1.mk
    |-- S60pkg1
    \-- test_pkg1.py

> 
> As for the test on the infra, they go where they make more sense: in the
> test infra.

Makes sense to me.


Regards,
Ricardo

[Buildroot] [PATCH 5/5] support/testing: add python-twisted tests

[Ricardo Martincoski]

Hello,

On Thu, Oct 11, 2018 at 06:27 PM, Arnout Vandecappelle wrote:

> On 11/10/18 20:48, Thomas Petazzoni wrote:
>> On Tue, 25 Sep 2018 23:55:14 -0300, Ricardo Martincoski wrote:
>> 
>>> +TEST_SCRIPT = """
>>> +from twisted.internet import protocol, reactor, endpoints
>>> +class F(protocol.Factory):
>>> +    pass
>>> +endpoints.serverFromString(reactor, "tcp:1234").listen(F())
>>> +reactor.run()
>>> +"""
>> 
>> At some point, we will have to find a solution to store such
>> "artefacts" somewhere, instead of having them inline in their test
>> cases. I don't have a good idea about this.
> 
>  I think it would be nice if we could add in the package directory a
> subdirectory with files to be copied to the target for the runtime test, and a
> python script for the test itself.
> 
>  As an in-between step, we can put the files-to-be-copied inside the
> support/testing directory and explicitly enumerate the files-to-be-copied in the
> test fixture.

Good idea.
I am working on this in-between step.
I will send a small series in the next few days.


Regards,
Ricardo

[Buildroot] [PATCH 0/5] a few more runtime test cases for python packages - series 1/n

[Thomas Petazzoni]

Hello,

On Tue, 25 Sep 2018 23:55:09 -0300, Ricardo Martincoski wrote:

> Ricardo Martincoski (5):
>   support/testing: add python-autobahn tests
>   support/testing: add python-txaio tests
>   support/testing: add python-txtorcon tests
>   support/testing: add python-incremental tests
>   support/testing: add python-twisted tests

I have applied the entire series. I however have a few
comments/suggestions, I'll reply to the individual patches.

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH 0/5] a few more runtime test cases for python packages - series 1/n

[Thomas Petazzoni]

Hello,

On Thu, 11 Oct 2018 20:42:44 +0200, Thomas Petazzoni wrote:

> > Ricardo Martincoski (5):
> >   support/testing: add python-autobahn tests
> >   support/testing: add python-txaio tests
> >   support/testing: add python-txtorcon tests
> >   support/testing: add python-incremental tests
> >   support/testing: add python-twisted tests  
> 
> I have applied the entire series. I however have a few
> comments/suggestions, I'll reply to the individual patches.

I forgot to say that on the python-twisted tests, I did adjust the
sleep + timeout as you suggested before applying the patch.

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [next 00/15] a few more runtime test cases for python packages - series 2/n

[Ricardo Martincoski]

Hello,

One way to reduce the burden of manual tests and also detect broken runtime
dependencies is to have more test cases (even simple ones) for python packages.
Well... here are some.

All runtime-tests on GitLab CI:
https://gitlab.com/RicardoMartincoski/buildroot/pipelines/36800857

Regards,
Ricardo

Ricardo Martincoski (15):
  support/testing: add python-argh tests
  support/testing: add python-attrs tests
  support/testing: add python-automat tests
  support/testing: add python-bitstring tests
  support/testing: add python-cbor tests
  support/testing: add python-click tests
  support/testing: add python-constantly tests
  support/testing: add python-passlib tests
  support/testing: add python-pexpect tests
  support/testing: add python-pynacl tests
  support/testing: add python-pyyaml tests
  support/testing: add python-service-identity tests
  support/testing: add python-subprocess32 test
  support/testing: add python-treq tests
  support/testing: add python-ubjson tests

 .gitlab-ci.yml                                | 29 ++++++++++++
 .../tests/package/sample_python_argh.py       | 10 +++++
 .../tests/package/sample_python_attrs.py      | 15 +++++++
 .../tests/package/sample_python_automat.py    | 27 +++++++++++
 .../tests/package/sample_python_bitstring.py  |  6 +++
 .../tests/package/sample_python_cbor_dec.py   | 10 +++++
 .../tests/package/sample_python_cbor_enc.py   | 14 ++++++
 .../tests/package/sample_python_click.py      | 12 +++++
 .../tests/package/sample_python_constantly.py | 19 ++++++++
 .../tests/package/sample_python_passlib.py    |  5 +++
 .../tests/package/sample_python_pexpect.py    |  8 ++++
 .../tests/package/sample_python_pynacl.py     |  3 ++
 .../tests/package/sample_python_pyyaml_dec.py | 10 +++++
 .../tests/package/sample_python_pyyaml_enc.py | 14 ++++++
 .../package/sample_python_service_identity.py |  2 +
 .../package/sample_python_subprocess32.py     |  6 +++
 .../tests/package/sample_python_treq.py       | 16 +++++++
 .../tests/package/sample_python_ubjson_dec.py | 10 +++++
 .../tests/package/sample_python_ubjson_enc.py | 14 ++++++
 .../testing/tests/package/test_python_argh.py | 45 +++++++++++++++++++
 .../tests/package/test_python_attrs.py        | 21 +++++++++
 .../tests/package/test_python_automat.py      | 23 ++++++++++
 .../tests/package/test_python_bitstring.py    | 21 +++++++++
 .../testing/tests/package/test_python_cbor.py | 23 ++++++++++
 .../tests/package/test_python_click.py        | 44 ++++++++++++++++++
 .../tests/package/test_python_constantly.py   | 21 +++++++++
 .../tests/package/test_python_passlib.py      | 23 ++++++++++
 .../tests/package/test_python_pexpect.py      | 21 +++++++++
 .../tests/package/test_python_pynacl.py       | 27 +++++++++++
 .../tests/package/test_python_pyyaml.py       | 23 ++++++++++
 .../package/test_python_service_identity.py   | 23 ++++++++++
 .../tests/package/test_python_subprocess32.py | 11 +++++
 .../testing/tests/package/test_python_treq.py | 29 ++++++++++++
 .../tests/package/test_python_ubjson.py       | 23 ++++++++++
 34 files changed, 608 insertions(+)
 create mode 100644 support/testing/tests/package/sample_python_argh.py
 create mode 100644 support/testing/tests/package/sample_python_attrs.py
 create mode 100644 support/testing/tests/package/sample_python_automat.py
 create mode 100644 support/testing/tests/package/sample_python_bitstring.py
 create mode 100644 support/testing/tests/package/sample_python_cbor_dec.py
 create mode 100644 support/testing/tests/package/sample_python_cbor_enc.py
 create mode 100644 support/testing/tests/package/sample_python_click.py
 create mode 100644 support/testing/tests/package/sample_python_constantly.py
 create mode 100644 support/testing/tests/package/sample_python_passlib.py
 create mode 100644 support/testing/tests/package/sample_python_pexpect.py
 create mode 100644 support/testing/tests/package/sample_python_pynacl.py
 create mode 100644 support/testing/tests/package/sample_python_pyyaml_dec.py
 create mode 100644 support/testing/tests/package/sample_python_pyyaml_enc.py
 create mode 100644 support/testing/tests/package/sample_python_service_identity.py
 create mode 100644 support/testing/tests/package/sample_python_subprocess32.py
 create mode 100644 support/testing/tests/package/sample_python_treq.py
 create mode 100644 support/testing/tests/package/sample_python_ubjson_dec.py
 create mode 100644 support/testing/tests/package/sample_python_ubjson_enc.py
 create mode 100644 support/testing/tests/package/test_python_argh.py
 create mode 100644 support/testing/tests/package/test_python_attrs.py
 create mode 100644 support/testing/tests/package/test_python_automat.py
 create mode 100644 support/testing/tests/package/test_python_bitstring.py
 create mode 100644 support/testing/tests/package/test_python_cbor.py
 create mode 100644 support/testing/tests/package/test_python_click.py
 create mode 100644 support/testing/tests/package/test_python_constantly.py
 create mode 100644 support/testing/tests/package/test_python_passlib.py
 create mode 100644 support/testing/tests/package/test_python_pexpect.py
 create mode 100644 support/testing/tests/package/test_python_pynacl.py
 create mode 100644 support/testing/tests/package/test_python_pyyaml.py
 create mode 100644 support/testing/tests/package/test_python_service_identity.py
 create mode 100644 support/testing/tests/package/test_python_subprocess32.py
 create mode 100644 support/testing/tests/package/test_python_treq.py
 create mode 100644 support/testing/tests/package/test_python_ubjson.py

-- 
2.17.1

[Buildroot] [next 01/15] support/testing: add python-argh tests

[Ricardo Martincoski]

Use a simple script to check the basic usage. Since this package
provides command line arguments, override run_sample_scripts to call the
script with arguments and check the expected output.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
---
 .gitlab-ci.yml                                |  2 +
 .../tests/package/sample_python_argh.py       | 10 +++++
 .../testing/tests/package/test_python_argh.py | 45 +++++++++++++++++++
 3 files changed, 57 insertions(+)
 create mode 100644 support/testing/tests/package/sample_python_argh.py
 create mode 100644 support/testing/tests/package/test_python_argh.py

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 98f9463928..62f3768d3d 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -316,6 +316,8 @@ tests.package.test_ipython.TestIPythonPy2: *runtime_test
 tests.package.test_ipython.TestIPythonPy3: *runtime_test
 tests.package.test_python.TestPython2: *runtime_test
 tests.package.test_python.TestPython3: *runtime_test
+tests.package.test_python_argh.TestPythonPy2Argh: *runtime_test
+tests.package.test_python_argh.TestPythonPy3Argh: *runtime_test
 tests.package.test_python_autobahn.TestPythonPy2Autobahn: *runtime_test
 tests.package.test_python_autobahn.TestPythonPy3Autobahn: *runtime_test
 tests.package.test_python_crossbar.TestPythonPy3Crossbar: *runtime_test
diff --git a/support/testing/tests/package/sample_python_argh.py b/support/testing/tests/package/sample_python_argh.py
new file mode 100644
index 0000000000..48247bf604
--- /dev/null
+++ b/support/testing/tests/package/sample_python_argh.py
@@ -0,0 +1,10 @@
+import argh
+
+
+ at argh.arg("foo", help="help for foo")
+ at argh.arg("--bar", help="help for bar")
+def main(foo, bar=False):
+    print("{}, {}".format(foo, bar))
+
+
+argh.dispatch_command(main)
diff --git a/support/testing/tests/package/test_python_argh.py b/support/testing/tests/package/test_python_argh.py
new file mode 100644
index 0000000000..de4b4cbe74
--- /dev/null
+++ b/support/testing/tests/package/test_python_argh.py
@@ -0,0 +1,45 @@
+from tests.package.test_python import TestPythonPackageBase
+
+
+class TestPythonArgh(TestPythonPackageBase):
+    config = TestPythonPackageBase.config
+    sample_scripts = ["tests/package/sample_python_argh.py"]
+
+    def run_sample_scripts(self):
+        cmd = self.interpreter + " sample_python_argh.py -h"
+        output, exit_code = self.emulator.run(cmd)
+        self.assertIn("usage:", output[0])
+        self.assertEqual(exit_code, 0)
+
+        cmd = self.interpreter + " sample_python_argh.py 123"
+        output, exit_code = self.emulator.run(cmd)
+        self.assertEqual(output[0], "123, False")
+        self.assertEqual(exit_code, 0)
+
+        cmd = self.interpreter + " sample_python_argh.py --bar 456"
+        output, exit_code = self.emulator.run(cmd)
+        self.assertEqual(output[0], "456, True")
+        self.assertEqual(exit_code, 0)
+
+        cmd = self.interpreter + " sample_python_argh.py"
+        output, exit_code = self.emulator.run(cmd)
+        self.assertIn("usage:", output[0])
+        self.assertEqual(exit_code, 2)
+
+
+class TestPythonPy2Argh(TestPythonArgh):
+    __test__ = True
+    config = TestPythonArgh.config + \
+        """
+        BR2_PACKAGE_PYTHON=y
+        BR2_PACKAGE_PYTHON_ARGH=y
+        """
+
+
+class TestPythonPy3Argh(TestPythonArgh):
+    __test__ = True
+    config = TestPythonArgh.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        BR2_PACKAGE_PYTHON_ARGH=y
+        """
-- 
2.17.1

[Buildroot] [next 02/15] support/testing: add python-attrs tests

[Ricardo Martincoski]

Use a minimal script to check the basic usage creating a class with 2
attributes.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
---
 .gitlab-ci.yml                                |  2 ++
 .../tests/package/sample_python_attrs.py      | 15 +++++++++++++
 .../tests/package/test_python_attrs.py        | 21 +++++++++++++++++++
 3 files changed, 38 insertions(+)
 create mode 100644 support/testing/tests/package/sample_python_attrs.py
 create mode 100644 support/testing/tests/package/test_python_attrs.py

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 62f3768d3d..494894a782 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -318,6 +318,8 @@ tests.package.test_python.TestPython2: *runtime_test
 tests.package.test_python.TestPython3: *runtime_test
 tests.package.test_python_argh.TestPythonPy2Argh: *runtime_test
 tests.package.test_python_argh.TestPythonPy3Argh: *runtime_test
+tests.package.test_python_attrs.TestPythonPy2Attrs: *runtime_test
+tests.package.test_python_attrs.TestPythonPy3Attrs: *runtime_test
 tests.package.test_python_autobahn.TestPythonPy2Autobahn: *runtime_test
 tests.package.test_python_autobahn.TestPythonPy3Autobahn: *runtime_test
 tests.package.test_python_crossbar.TestPythonPy3Crossbar: *runtime_test
diff --git a/support/testing/tests/package/sample_python_attrs.py b/support/testing/tests/package/sample_python_attrs.py
new file mode 100644
index 0000000000..f224944914
--- /dev/null
+++ b/support/testing/tests/package/sample_python_attrs.py
@@ -0,0 +1,15 @@
+import attr
+
+
+ at attr.s
+class Obj(object):
+    x = attr.ib()
+    y = attr.ib(default=1)
+
+
+obj1 = Obj(2)
+assert(obj1.x == 2)
+assert(obj1.y == 1)
+obj2 = Obj(3, 4)
+assert(obj2.x == 3)
+assert(obj2.y == 4)
diff --git a/support/testing/tests/package/test_python_attrs.py b/support/testing/tests/package/test_python_attrs.py
new file mode 100644
index 0000000000..9f597034ae
--- /dev/null
+++ b/support/testing/tests/package/test_python_attrs.py
@@ -0,0 +1,21 @@
+from tests.package.test_python import TestPythonPackageBase
+
+
+class TestPythonPy2Attrs(TestPythonPackageBase):
+    __test__ = True
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON=y
+        BR2_PACKAGE_PYTHON_ATTRS=y
+        """
+    sample_scripts = ["tests/package/sample_python_attrs.py"]
+
+
+class TestPythonPy3Attrs(TestPythonPackageBase):
+    __test__ = True
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        BR2_PACKAGE_PYTHON_ATTRS=y
+        """
+    sample_scripts = ["tests/package/sample_python_attrs.py"]
-- 
2.17.1

[Buildroot] [next 03/15] support/testing: add python-automat tests

[Ricardo Martincoski]

Use a minimal script to check the basic usage by creating and using a
small state machine.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
---
 .gitlab-ci.yml                                |  2 ++
 .../tests/package/sample_python_automat.py    | 27 +++++++++++++++++++
 .../tests/package/test_python_automat.py      | 23 ++++++++++++++++
 3 files changed, 52 insertions(+)
 create mode 100644 support/testing/tests/package/sample_python_automat.py
 create mode 100644 support/testing/tests/package/test_python_automat.py

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 494894a782..7f311f74cf 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -322,6 +322,8 @@ tests.package.test_python_attrs.TestPythonPy2Attrs: *runtime_test
 tests.package.test_python_attrs.TestPythonPy3Attrs: *runtime_test
 tests.package.test_python_autobahn.TestPythonPy2Autobahn: *runtime_test
 tests.package.test_python_autobahn.TestPythonPy3Autobahn: *runtime_test
+tests.package.test_python_automat.TestPythonPy2Automat: *runtime_test
+tests.package.test_python_automat.TestPythonPy3Automat: *runtime_test
 tests.package.test_python_crossbar.TestPythonPy3Crossbar: *runtime_test
 tests.package.test_python_cryptography.TestPythonPy2Cryptography: *runtime_test
 tests.package.test_python_cryptography.TestPythonPy3Cryptography: *runtime_test
diff --git a/support/testing/tests/package/sample_python_automat.py b/support/testing/tests/package/sample_python_automat.py
new file mode 100644
index 0000000000..baf536f993
--- /dev/null
+++ b/support/testing/tests/package/sample_python_automat.py
@@ -0,0 +1,27 @@
+from automat import MethodicalMachine
+
+
+class Led(object):
+    _machine = MethodicalMachine()
+
+    @_machine.state()
+    def led_on(self):
+        "led is on"
+
+    @_machine.state(initial=True)
+    def led_off(self):
+        "led is off"
+
+    @_machine.input()
+    def turn_on(self):
+        "turn the led on"
+
+    @_machine.output()
+    def _light(self):
+        print("light")
+
+    led_off.upon(turn_on, enter=led_on, outputs=[_light])
+
+
+led = Led()
+led.turn_on()
diff --git a/support/testing/tests/package/test_python_automat.py b/support/testing/tests/package/test_python_automat.py
new file mode 100644
index 0000000000..00a7ed6526
--- /dev/null
+++ b/support/testing/tests/package/test_python_automat.py
@@ -0,0 +1,23 @@
+from tests.package.test_python import TestPythonPackageBase
+
+
+class TestPythonPy2Automat(TestPythonPackageBase):
+    __test__ = True
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON=y
+        BR2_PACKAGE_PYTHON_AUTOMAT=y
+        """
+    sample_scripts = ["tests/package/sample_python_automat.py"]
+    timeout = 30
+
+
+class TestPythonPy3Automat(TestPythonPackageBase):
+    __test__ = True
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        BR2_PACKAGE_PYTHON_AUTOMAT=y
+        """
+    sample_scripts = ["tests/package/sample_python_automat.py"]
+    timeout = 30
-- 
2.17.1

[Buildroot] [next 04/15] support/testing: add python-bitstring tests

[Ricardo Martincoski]

Add a simple test case to check the basic usage by checking the
corresponding representation of a 12-bit decimal number in hex, binary
and integer.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
---
 .gitlab-ci.yml                                |  2 ++
 .../tests/package/sample_python_bitstring.py  |  6 ++++++
 .../tests/package/test_python_bitstring.py    | 21 +++++++++++++++++++
 3 files changed, 29 insertions(+)
 create mode 100644 support/testing/tests/package/sample_python_bitstring.py
 create mode 100644 support/testing/tests/package/test_python_bitstring.py

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 7f311f74cf..2fc0fe7e6a 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -324,6 +324,8 @@ tests.package.test_python_autobahn.TestPythonPy2Autobahn: *runtime_test
 tests.package.test_python_autobahn.TestPythonPy3Autobahn: *runtime_test
 tests.package.test_python_automat.TestPythonPy2Automat: *runtime_test
 tests.package.test_python_automat.TestPythonPy3Automat: *runtime_test
+tests.package.test_python_bitstring.TestPythonPy2Bitstring: *runtime_test
+tests.package.test_python_bitstring.TestPythonPy3Bitstring: *runtime_test
 tests.package.test_python_crossbar.TestPythonPy3Crossbar: *runtime_test
 tests.package.test_python_cryptography.TestPythonPy2Cryptography: *runtime_test
 tests.package.test_python_cryptography.TestPythonPy3Cryptography: *runtime_test
diff --git a/support/testing/tests/package/sample_python_bitstring.py b/support/testing/tests/package/sample_python_bitstring.py
new file mode 100644
index 0000000000..2cc10dba65
--- /dev/null
+++ b/support/testing/tests/package/sample_python_bitstring.py
@@ -0,0 +1,6 @@
+import bitstring
+
+value = bitstring.BitArray("uint:12=42")
+assert(value.hex == "02a")
+assert(value.bin == "000000101010")
+assert(value.uint == 42)
diff --git a/support/testing/tests/package/test_python_bitstring.py b/support/testing/tests/package/test_python_bitstring.py
new file mode 100644
index 0000000000..63b3fdb69c
--- /dev/null
+++ b/support/testing/tests/package/test_python_bitstring.py
@@ -0,0 +1,21 @@
+from tests.package.test_python import TestPythonPackageBase
+
+
+class TestPythonPy2Bitstring(TestPythonPackageBase):
+    __test__ = True
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON=y
+        BR2_PACKAGE_PYTHON_BITSTRING=y
+        """
+    sample_scripts = ["tests/package/sample_python_bitstring.py"]
+
+
+class TestPythonPy3Bitstring(TestPythonPackageBase):
+    __test__ = True
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        BR2_PACKAGE_PYTHON_BITSTRING=y
+        """
+    sample_scripts = ["tests/package/sample_python_bitstring.py"]
-- 
2.17.1

[Buildroot] [next 05/15] support/testing: add python-cbor tests

[Ricardo Martincoski]

Add a simple test case to check the basic usage, storing a dict into a
file and then retrieving the dict from the file.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
---
 .gitlab-ci.yml                                |  2 ++
 .../tests/package/sample_python_cbor_dec.py   | 10 ++++++++
 .../tests/package/sample_python_cbor_enc.py   | 14 +++++++++++
 .../testing/tests/package/test_python_cbor.py | 23 +++++++++++++++++++
 4 files changed, 49 insertions(+)
 create mode 100644 support/testing/tests/package/sample_python_cbor_dec.py
 create mode 100644 support/testing/tests/package/sample_python_cbor_enc.py
 create mode 100644 support/testing/tests/package/test_python_cbor.py

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 2fc0fe7e6a..d6f2e6595f 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -326,6 +326,8 @@ tests.package.test_python_automat.TestPythonPy2Automat: *runtime_test
 tests.package.test_python_automat.TestPythonPy3Automat: *runtime_test
 tests.package.test_python_bitstring.TestPythonPy2Bitstring: *runtime_test
 tests.package.test_python_bitstring.TestPythonPy3Bitstring: *runtime_test
+tests.package.test_python_cbor.TestPythonPy2Cbor: *runtime_test
+tests.package.test_python_cbor.TestPythonPy3Cbor: *runtime_test
 tests.package.test_python_crossbar.TestPythonPy3Crossbar: *runtime_test
 tests.package.test_python_cryptography.TestPythonPy2Cryptography: *runtime_test
 tests.package.test_python_cryptography.TestPythonPy3Cryptography: *runtime_test
diff --git a/support/testing/tests/package/sample_python_cbor_dec.py b/support/testing/tests/package/sample_python_cbor_dec.py
new file mode 100644
index 0000000000..5ca5e3aa7c
--- /dev/null
+++ b/support/testing/tests/package/sample_python_cbor_dec.py
@@ -0,0 +1,10 @@
+import cbor
+
+with open("/tmp/data.cbor", "rb") as f:
+    serialized = f.read()
+data = cbor.loads(serialized)
+print(data)
+assert(data["name"] == "python-cbor")
+assert(data["versions"] == ["1", "2"])
+assert(data["group"]["is_a_package"] is True)
+assert(data["group"]["value"] == 42)
diff --git a/support/testing/tests/package/sample_python_cbor_enc.py b/support/testing/tests/package/sample_python_cbor_enc.py
new file mode 100644
index 0000000000..1bde92bb9f
--- /dev/null
+++ b/support/testing/tests/package/sample_python_cbor_enc.py
@@ -0,0 +1,14 @@
+import cbor
+
+data = {
+    "name": "python-cbor",
+    "versions": ["1", "2"],
+    "group": {
+        "is_a_package": True,
+        "value": 42
+    }
+}
+serialized = cbor.dumps(data)
+print(serialized)
+with open("/tmp/data.cbor", "wb") as f:
+    f.write(serialized)
diff --git a/support/testing/tests/package/test_python_cbor.py b/support/testing/tests/package/test_python_cbor.py
new file mode 100644
index 0000000000..9da3503763
--- /dev/null
+++ b/support/testing/tests/package/test_python_cbor.py
@@ -0,0 +1,23 @@
+from tests.package.test_python import TestPythonPackageBase
+
+
+class TestPythonPy2Cbor(TestPythonPackageBase):
+    __test__ = True
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON=y
+        BR2_PACKAGE_PYTHON_CBOR=y
+        """
+    sample_scripts = ["tests/package/sample_python_cbor_enc.py",
+                      "tests/package/sample_python_cbor_dec.py"]
+
+
+class TestPythonPy3Cbor(TestPythonPackageBase):
+    __test__ = True
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        BR2_PACKAGE_PYTHON_CBOR=y
+        """
+    sample_scripts = ["tests/package/sample_python_cbor_enc.py",
+                      "tests/package/sample_python_cbor_dec.py"]
-- 
2.17.1

[Buildroot] [next 06/15] support/testing: add python-click tests

[Ricardo Martincoski]

Use a simple script to check the basic usage. Since this package
provides command line arguments, override run_sample_scripts to call the
script with arguments and check the expected output.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
---
 .gitlab-ci.yml                                |  2 +
 .../tests/package/sample_python_click.py      | 12 +++++
 .../tests/package/test_python_click.py        | 44 +++++++++++++++++++
 3 files changed, 58 insertions(+)
 create mode 100644 support/testing/tests/package/sample_python_click.py
 create mode 100644 support/testing/tests/package/test_python_click.py

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index d6f2e6595f..eb6a187615 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -328,6 +328,8 @@ tests.package.test_python_bitstring.TestPythonPy2Bitstring: *runtime_test
 tests.package.test_python_bitstring.TestPythonPy3Bitstring: *runtime_test
 tests.package.test_python_cbor.TestPythonPy2Cbor: *runtime_test
 tests.package.test_python_cbor.TestPythonPy3Cbor: *runtime_test
+tests.package.test_python_click.TestPythonPy2Click: *runtime_test
+tests.package.test_python_click.TestPythonPy3Click: *runtime_test
 tests.package.test_python_crossbar.TestPythonPy3Crossbar: *runtime_test
 tests.package.test_python_cryptography.TestPythonPy2Cryptography: *runtime_test
 tests.package.test_python_cryptography.TestPythonPy3Cryptography: *runtime_test
diff --git a/support/testing/tests/package/sample_python_click.py b/support/testing/tests/package/sample_python_click.py
new file mode 100644
index 0000000000..759868d087
--- /dev/null
+++ b/support/testing/tests/package/sample_python_click.py
@@ -0,0 +1,12 @@
+import click
+
+
+ at click.command()
+ at click.argument("foo")
+ at click.option("--bar", is_flag=True, help="help for bar")
+def main(foo, bar):
+    click.echo("{}, {}".format(foo, bar))
+
+
+if __name__ == '__main__':
+    main()
diff --git a/support/testing/tests/package/test_python_click.py b/support/testing/tests/package/test_python_click.py
new file mode 100644
index 0000000000..db57d3aa9f
--- /dev/null
+++ b/support/testing/tests/package/test_python_click.py
@@ -0,0 +1,44 @@
+from tests.package.test_python import TestPythonPackageBase
+
+
+class TestPythonClick(TestPythonPackageBase):
+    sample_scripts = ["tests/package/sample_python_click.py"]
+
+    def run_sample_scripts(self):
+        cmd = self.interpreter + " sample_python_click.py --help"
+        output, exit_code = self.emulator.run(cmd)
+        self.assertIn("Usage:", output[0])
+        self.assertEqual(exit_code, 0)
+
+        cmd = self.interpreter + " sample_python_click.py 123"
+        output, exit_code = self.emulator.run(cmd)
+        self.assertEqual(output[0], "123, False")
+        self.assertEqual(exit_code, 0)
+
+        cmd = self.interpreter + " sample_python_click.py --bar 456"
+        output, exit_code = self.emulator.run(cmd)
+        self.assertEqual(output[0], "456, True")
+        self.assertEqual(exit_code, 0)
+
+        cmd = self.interpreter + " sample_python_click.py"
+        output, exit_code = self.emulator.run(cmd)
+        self.assertIn("Usage:", output[0])
+        self.assertEqual(exit_code, 2)
+
+
+class TestPythonPy2Click(TestPythonClick):
+    __test__ = True
+    config = TestPythonClick.config + \
+        """
+        BR2_PACKAGE_PYTHON=y
+        BR2_PACKAGE_PYTHON_CLICK=y
+        """
+
+
+class TestPythonPy3Click(TestPythonClick):
+    __test__ = True
+    config = TestPythonClick.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        BR2_PACKAGE_PYTHON_CLICK=y
+        """
-- 
2.17.1

[Buildroot] [next 07/15] support/testing: add python-constantly tests

[Ricardo Martincoski]

Add a simple test case to check the basic usage by creating a class with
two constants.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
---
 .gitlab-ci.yml                                |  2 ++
 .../tests/package/sample_python_constantly.py | 19 +++++++++++++++++
 .../tests/package/test_python_constantly.py   | 21 +++++++++++++++++++
 3 files changed, 42 insertions(+)
 create mode 100644 support/testing/tests/package/sample_python_constantly.py
 create mode 100644 support/testing/tests/package/test_python_constantly.py

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index eb6a187615..5133affa25 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -330,6 +330,8 @@ tests.package.test_python_cbor.TestPythonPy2Cbor: *runtime_test
 tests.package.test_python_cbor.TestPythonPy3Cbor: *runtime_test
 tests.package.test_python_click.TestPythonPy2Click: *runtime_test
 tests.package.test_python_click.TestPythonPy3Click: *runtime_test
+tests.package.test_python_constantly.TestPythonPy2Constantly: *runtime_test
+tests.package.test_python_constantly.TestPythonPy3Constantly: *runtime_test
 tests.package.test_python_crossbar.TestPythonPy3Crossbar: *runtime_test
 tests.package.test_python_cryptography.TestPythonPy2Cryptography: *runtime_test
 tests.package.test_python_cryptography.TestPythonPy3Cryptography: *runtime_test
diff --git a/support/testing/tests/package/sample_python_constantly.py b/support/testing/tests/package/sample_python_constantly.py
new file mode 100644
index 0000000000..570cf85b23
--- /dev/null
+++ b/support/testing/tests/package/sample_python_constantly.py
@@ -0,0 +1,19 @@
+from constantly import ValueConstant, Values
+
+
+class RESULT(Values):
+    OK = ValueConstant(0)
+    FAIL = ValueConstant(-1)
+
+    @classmethod
+    def get(cls, rc):
+        if rc == 0:
+            return cls.OK
+        else:
+            return cls.FAIL
+
+
+print(list(RESULT.iterconstants()))
+assert(RESULT.OK < RESULT.FAIL)
+assert(RESULT.OK.value > RESULT.FAIL.value)
+assert(RESULT.get(-5) == RESULT.FAIL)
diff --git a/support/testing/tests/package/test_python_constantly.py b/support/testing/tests/package/test_python_constantly.py
new file mode 100644
index 0000000000..5be01c4f8c
--- /dev/null
+++ b/support/testing/tests/package/test_python_constantly.py
@@ -0,0 +1,21 @@
+from tests.package.test_python import TestPythonPackageBase
+
+
+class TestPythonPy2Constantly(TestPythonPackageBase):
+    __test__ = True
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON=y
+        BR2_PACKAGE_PYTHON_CONSTANTLY=y
+        """
+    sample_scripts = ["tests/package/sample_python_constantly.py"]
+
+
+class TestPythonPy3Constantly(TestPythonPackageBase):
+    __test__ = True
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        BR2_PACKAGE_PYTHON_CONSTANTLY=y
+        """
+    sample_scripts = ["tests/package/sample_python_constantly.py"]
-- 
2.17.1

[Buildroot] [next 08/15] support/testing: add python-passlib tests

[Ricardo Martincoski]

Add a simple test case that creates a hash for a password and verifies
it against an incorrect and a correct password.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
---
 .gitlab-ci.yml                                |  2 ++
 .../tests/package/sample_python_passlib.py    |  5 ++++
 .../tests/package/test_python_passlib.py      | 23 +++++++++++++++++++
 3 files changed, 30 insertions(+)
 create mode 100644 support/testing/tests/package/sample_python_passlib.py
 create mode 100644 support/testing/tests/package/test_python_passlib.py

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 5133affa25..61c44412b3 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -337,6 +337,8 @@ tests.package.test_python_cryptography.TestPythonPy2Cryptography: *runtime_test
 tests.package.test_python_cryptography.TestPythonPy3Cryptography: *runtime_test
 tests.package.test_python_incremental.TestPythonPy2Incremental: *runtime_test
 tests.package.test_python_incremental.TestPythonPy3Incremental: *runtime_test
+tests.package.test_python_passlib.TestPythonPy2Passlib: *runtime_test
+tests.package.test_python_passlib.TestPythonPy3Passlib: *runtime_test
 tests.package.test_python_twisted.TestPythonPy2Twisted: *runtime_test
 tests.package.test_python_twisted.TestPythonPy3Twisted: *runtime_test
 tests.package.test_python_txaio.TestPythonPy2Txaio: *runtime_test
diff --git a/support/testing/tests/package/sample_python_passlib.py b/support/testing/tests/package/sample_python_passlib.py
new file mode 100644
index 0000000000..3ab348b9e4
--- /dev/null
+++ b/support/testing/tests/package/sample_python_passlib.py
@@ -0,0 +1,5 @@
+from passlib.hash import pbkdf2_sha256
+
+hash = pbkdf2_sha256.hash("password")
+assert(pbkdf2_sha256.verify("passWord", hash) is False)
+assert(pbkdf2_sha256.verify("password", hash) is True)
diff --git a/support/testing/tests/package/test_python_passlib.py b/support/testing/tests/package/test_python_passlib.py
new file mode 100644
index 0000000000..7c85882bf8
--- /dev/null
+++ b/support/testing/tests/package/test_python_passlib.py
@@ -0,0 +1,23 @@
+from tests.package.test_python import TestPythonPackageBase
+
+
+class TestPythonPy2Passlib(TestPythonPackageBase):
+    __test__ = True
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON=y
+        BR2_PACKAGE_PYTHON_PASSLIB=y
+        """
+    sample_scripts = ["tests/package/sample_python_passlib.py"]
+    timeout = 30
+
+
+class TestPythonPy3Passlib(TestPythonPackageBase):
+    __test__ = True
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        BR2_PACKAGE_PYTHON_PASSLIB=y
+        """
+    sample_scripts = ["tests/package/sample_python_passlib.py"]
+    timeout = 30
-- 
2.17.1

[Buildroot] [next 09/15] support/testing: add python-pexpect tests

[Ricardo Martincoski]

Add a simple test case to check the basic usage. Call 'login' and try
wrong user/password, expecting the 'Login incorrect' message.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
---
 .gitlab-ci.yml                                |  2 ++
 .../tests/package/sample_python_pexpect.py    |  8 +++++++
 .../tests/package/test_python_pexpect.py      | 21 +++++++++++++++++++
 3 files changed, 31 insertions(+)
 create mode 100644 support/testing/tests/package/sample_python_pexpect.py
 create mode 100644 support/testing/tests/package/test_python_pexpect.py

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 61c44412b3..99df1f85e2 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -339,6 +339,8 @@ tests.package.test_python_incremental.TestPythonPy2Incremental: *runtime_test
 tests.package.test_python_incremental.TestPythonPy3Incremental: *runtime_test
 tests.package.test_python_passlib.TestPythonPy2Passlib: *runtime_test
 tests.package.test_python_passlib.TestPythonPy3Passlib: *runtime_test
+tests.package.test_python_pexpect.TestPythonPy2Pexpect: *runtime_test
+tests.package.test_python_pexpect.TestPythonPy3Pexpect: *runtime_test
 tests.package.test_python_twisted.TestPythonPy2Twisted: *runtime_test
 tests.package.test_python_twisted.TestPythonPy3Twisted: *runtime_test
 tests.package.test_python_txaio.TestPythonPy2Txaio: *runtime_test
diff --git a/support/testing/tests/package/sample_python_pexpect.py b/support/testing/tests/package/sample_python_pexpect.py
new file mode 100644
index 0000000000..cfe395fc29
--- /dev/null
+++ b/support/testing/tests/package/sample_python_pexpect.py
@@ -0,0 +1,8 @@
+import pexpect
+
+p = pexpect.spawn(["login"])
+p.expect("login:")
+p.sendline("wrong")
+p.expect("Password:")
+p.sendline("wrong")
+p.expect("Login incorrect")
diff --git a/support/testing/tests/package/test_python_pexpect.py b/support/testing/tests/package/test_python_pexpect.py
new file mode 100644
index 0000000000..ba31cdbdbe
--- /dev/null
+++ b/support/testing/tests/package/test_python_pexpect.py
@@ -0,0 +1,21 @@
+from tests.package.test_python import TestPythonPackageBase
+
+
+class TestPythonPy2Pexpect(TestPythonPackageBase):
+    __test__ = True
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON=y
+        BR2_PACKAGE_PYTHON_PEXPECT=y
+        """
+    sample_scripts = ["tests/package/sample_python_pexpect.py"]
+
+
+class TestPythonPy3Pexpect(TestPythonPackageBase):
+    __test__ = True
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        BR2_PACKAGE_PYTHON_PEXPECT=y
+        """
+    sample_scripts = ["tests/package/sample_python_pexpect.py"]
-- 
2.17.1

[Buildroot] [next 10/15] support/testing: add python-pynacl tests

[Ricardo Martincoski]

Add a simple test case that minimally uses the module.
Add haveged to the target to generate enough entropy so pynacl ->
libsodium don't hang waiting for /dev/random.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
---
 .gitlab-ci.yml                                |  2 ++
 .../tests/package/sample_python_pynacl.py     |  3 +++
 .../tests/package/test_python_pynacl.py       | 27 +++++++++++++++++++
 3 files changed, 32 insertions(+)
 create mode 100644 support/testing/tests/package/sample_python_pynacl.py
 create mode 100644 support/testing/tests/package/test_python_pynacl.py

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 99df1f85e2..2a68e6485d 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -341,6 +341,8 @@ tests.package.test_python_passlib.TestPythonPy2Passlib: *runtime_test
 tests.package.test_python_passlib.TestPythonPy3Passlib: *runtime_test
 tests.package.test_python_pexpect.TestPythonPy2Pexpect: *runtime_test
 tests.package.test_python_pexpect.TestPythonPy3Pexpect: *runtime_test
+tests.package.test_python_pynacl.TestPythonPy2Pynacl: *runtime_test
+tests.package.test_python_pynacl.TestPythonPy3Pynacl: *runtime_test
 tests.package.test_python_twisted.TestPythonPy2Twisted: *runtime_test
 tests.package.test_python_twisted.TestPythonPy3Twisted: *runtime_test
 tests.package.test_python_txaio.TestPythonPy2Txaio: *runtime_test
diff --git a/support/testing/tests/package/sample_python_pynacl.py b/support/testing/tests/package/sample_python_pynacl.py
new file mode 100644
index 0000000000..d17c50ebe5
--- /dev/null
+++ b/support/testing/tests/package/sample_python_pynacl.py
@@ -0,0 +1,3 @@
+import nacl.utils
+
+nonce = nacl.utils.random(16)
diff --git a/support/testing/tests/package/test_python_pynacl.py b/support/testing/tests/package/test_python_pynacl.py
new file mode 100644
index 0000000000..729a887552
--- /dev/null
+++ b/support/testing/tests/package/test_python_pynacl.py
@@ -0,0 +1,27 @@
+from tests.package.test_python import TestPythonPackageBase
+
+
+class TestPythonPy2Pynacl(TestPythonPackageBase):
+    __test__ = True
+    # use haveged to generate enough entropy so pynacl -> libsodium don't hang waiting for /dev/random
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON=y
+        BR2_PACKAGE_PYTHON_PYNACL=y
+        BR2_PACKAGE_HAVEGED=y
+        """
+    sample_scripts = ["tests/package/sample_python_pynacl.py"]
+    timeout = 10
+
+
+class TestPythonPy3Pynacl(TestPythonPackageBase):
+    __test__ = True
+    # use haveged to generate enough entropy so pynacl -> libsodium don't hang waiting for /dev/random
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        BR2_PACKAGE_PYTHON_PYNACL=y
+        BR2_PACKAGE_HAVEGED=y
+        """
+    sample_scripts = ["tests/package/sample_python_pynacl.py"]
+    timeout = 10
-- 
2.17.1

[Buildroot] [next 11/15] support/testing: add python-pyyaml tests

[Ricardo Martincoski]

Add a simple test case to check the basic usage, storing a dict into a
file and then retrieving the dict from the file.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
---
 .gitlab-ci.yml                                |  2 ++
 .../tests/package/sample_python_pyyaml_dec.py | 10 ++++++++
 .../tests/package/sample_python_pyyaml_enc.py | 14 +++++++++++
 .../tests/package/test_python_pyyaml.py       | 23 +++++++++++++++++++
 4 files changed, 49 insertions(+)
 create mode 100644 support/testing/tests/package/sample_python_pyyaml_dec.py
 create mode 100644 support/testing/tests/package/sample_python_pyyaml_enc.py
 create mode 100644 support/testing/tests/package/test_python_pyyaml.py

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 2a68e6485d..a0659d6bfb 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -343,6 +343,8 @@ tests.package.test_python_pexpect.TestPythonPy2Pexpect: *runtime_test
 tests.package.test_python_pexpect.TestPythonPy3Pexpect: *runtime_test
 tests.package.test_python_pynacl.TestPythonPy2Pynacl: *runtime_test
 tests.package.test_python_pynacl.TestPythonPy3Pynacl: *runtime_test
+tests.package.test_python_pyyaml.TestPythonPy2Pyyaml: *runtime_test
+tests.package.test_python_pyyaml.TestPythonPy3Pyyaml: *runtime_test
 tests.package.test_python_twisted.TestPythonPy2Twisted: *runtime_test
 tests.package.test_python_twisted.TestPythonPy3Twisted: *runtime_test
 tests.package.test_python_txaio.TestPythonPy2Txaio: *runtime_test
diff --git a/support/testing/tests/package/sample_python_pyyaml_dec.py b/support/testing/tests/package/sample_python_pyyaml_dec.py
new file mode 100644
index 0000000000..9ab8931197
--- /dev/null
+++ b/support/testing/tests/package/sample_python_pyyaml_dec.py
@@ -0,0 +1,10 @@
+import yaml
+
+with open("/tmp/data.yml", "rb") as f:
+    serialized = f.read()
+data = yaml.load(serialized)
+print(data)
+assert(data["name"] == "python-pyyaml")
+assert(data["versions"] == ["1", "2"])
+assert(data["group"]["is_a_package"] is True)
+assert(data["group"]["value"] == 42)
diff --git a/support/testing/tests/package/sample_python_pyyaml_enc.py b/support/testing/tests/package/sample_python_pyyaml_enc.py
new file mode 100644
index 0000000000..30759949e2
--- /dev/null
+++ b/support/testing/tests/package/sample_python_pyyaml_enc.py
@@ -0,0 +1,14 @@
+import yaml
+
+data = {
+    "name": "python-pyyaml",
+    "versions": ["1", "2"],
+    "group": {
+        "is_a_package": True,
+        "value": 42
+    }
+}
+serialized = yaml.dump(data, default_flow_style=False)
+print(serialized)
+with open("/tmp/data.yml", "w") as f:
+    f.write(serialized)
diff --git a/support/testing/tests/package/test_python_pyyaml.py b/support/testing/tests/package/test_python_pyyaml.py
new file mode 100644
index 0000000000..cd45a57b86
--- /dev/null
+++ b/support/testing/tests/package/test_python_pyyaml.py
@@ -0,0 +1,23 @@
+from tests.package.test_python import TestPythonPackageBase
+
+
+class TestPythonPy2Pyyaml(TestPythonPackageBase):
+    __test__ = True
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON=y
+        BR2_PACKAGE_PYTHON_PYYAML=y
+        """
+    sample_scripts = ["tests/package/sample_python_pyyaml_enc.py",
+                      "tests/package/sample_python_pyyaml_dec.py"]
+
+
+class TestPythonPy3Pyyaml(TestPythonPackageBase):
+    __test__ = True
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        BR2_PACKAGE_PYTHON_PYYAML=y
+        """
+    sample_scripts = ["tests/package/sample_python_pyyaml_enc.py",
+                      "tests/package/sample_python_pyyaml_dec.py"]
-- 
2.17.1

[Buildroot] [next 12/15] support/testing: add python-service-identity tests

[Ricardo Martincoski]

Add a simple test case that imports the module.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
---
 .gitlab-ci.yml                                |  2 ++
 .../package/sample_python_service_identity.py |  2 ++
 .../package/test_python_service_identity.py   | 23 +++++++++++++++++++
 3 files changed, 27 insertions(+)
 create mode 100644 support/testing/tests/package/sample_python_service_identity.py
 create mode 100644 support/testing/tests/package/test_python_service_identity.py

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index a0659d6bfb..4166412df3 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -345,6 +345,8 @@ tests.package.test_python_pynacl.TestPythonPy2Pynacl: *runtime_test
 tests.package.test_python_pynacl.TestPythonPy3Pynacl: *runtime_test
 tests.package.test_python_pyyaml.TestPythonPy2Pyyaml: *runtime_test
 tests.package.test_python_pyyaml.TestPythonPy3Pyyaml: *runtime_test
+tests.package.test_python_service_identity.TestPythonPy2ServiceIdentity: *runtime_test
+tests.package.test_python_service_identity.TestPythonPy3ServiceIdentity: *runtime_test
 tests.package.test_python_twisted.TestPythonPy2Twisted: *runtime_test
 tests.package.test_python_twisted.TestPythonPy3Twisted: *runtime_test
 tests.package.test_python_txaio.TestPythonPy2Txaio: *runtime_test
diff --git a/support/testing/tests/package/sample_python_service_identity.py b/support/testing/tests/package/sample_python_service_identity.py
new file mode 100644
index 0000000000..3184b6c580
--- /dev/null
+++ b/support/testing/tests/package/sample_python_service_identity.py
@@ -0,0 +1,2 @@
+from service_identity import VerificationError  # noqa
+from service_identity.pyopenssl import verify_hostname  # noqa
diff --git a/support/testing/tests/package/test_python_service_identity.py b/support/testing/tests/package/test_python_service_identity.py
new file mode 100644
index 0000000000..6aeb77de8c
--- /dev/null
+++ b/support/testing/tests/package/test_python_service_identity.py
@@ -0,0 +1,23 @@
+from tests.package.test_python import TestPythonPackageBase
+
+
+class TestPythonPy2ServiceIdentity(TestPythonPackageBase):
+    __test__ = True
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON=y
+        BR2_PACKAGE_PYTHON_SERVICE_IDENTITY=y
+        """
+    sample_scripts = ["tests/package/sample_python_service_identity.py"]
+    timeout = 30
+
+
+class TestPythonPy3ServiceIdentity(TestPythonPackageBase):
+    __test__ = True
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        BR2_PACKAGE_PYTHON_SERVICE_IDENTITY=y
+        """
+    sample_scripts = ["tests/package/sample_python_service_identity.py"]
+    timeout = 30
-- 
2.17.1

[Buildroot] [next 13/15] support/testing: add python-subprocess32 test

[Ricardo Martincoski]

Add a simple test case to check the basic usage by calling 'ls' and
checking the output.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
---
 .gitlab-ci.yml                                        |  1 +
 .../tests/package/sample_python_subprocess32.py       |  6 ++++++
 .../testing/tests/package/test_python_subprocess32.py | 11 +++++++++++
 3 files changed, 18 insertions(+)
 create mode 100644 support/testing/tests/package/sample_python_subprocess32.py
 create mode 100644 support/testing/tests/package/test_python_subprocess32.py

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 4166412df3..743f4edc06 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -347,6 +347,7 @@ tests.package.test_python_pyyaml.TestPythonPy2Pyyaml: *runtime_test
 tests.package.test_python_pyyaml.TestPythonPy3Pyyaml: *runtime_test
 tests.package.test_python_service_identity.TestPythonPy2ServiceIdentity: *runtime_test
 tests.package.test_python_service_identity.TestPythonPy3ServiceIdentity: *runtime_test
+tests.package.test_python_subprocess32.TestPythonPy2Subprocess32: *runtime_test
 tests.package.test_python_twisted.TestPythonPy2Twisted: *runtime_test
 tests.package.test_python_twisted.TestPythonPy3Twisted: *runtime_test
 tests.package.test_python_txaio.TestPythonPy2Txaio: *runtime_test
diff --git a/support/testing/tests/package/sample_python_subprocess32.py b/support/testing/tests/package/sample_python_subprocess32.py
new file mode 100644
index 0000000000..8579745cc8
--- /dev/null
+++ b/support/testing/tests/package/sample_python_subprocess32.py
@@ -0,0 +1,6 @@
+import subprocess32
+
+output = subprocess32.check_output(["ls", "-l", "/dev/null"])
+print(output)
+assert("/dev/null" in output)
+assert("No such" not in output)
diff --git a/support/testing/tests/package/test_python_subprocess32.py b/support/testing/tests/package/test_python_subprocess32.py
new file mode 100644
index 0000000000..d2878046d1
--- /dev/null
+++ b/support/testing/tests/package/test_python_subprocess32.py
@@ -0,0 +1,11 @@
+from tests.package.test_python import TestPythonPackageBase
+
+
+class TestPythonPy2Subprocess32(TestPythonPackageBase):
+    __test__ = True
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON=y
+        BR2_PACKAGE_PYTHON_SUBPROCESS32=y
+        """
+    sample_scripts = ["tests/package/sample_python_subprocess32.py"]
-- 
2.17.1

[Buildroot] [next 14/15] support/testing: add python-treq tests

[Ricardo Martincoski]

Use a simple script to check the basic usage. The target has no https
server, so a connection from in the target to localhost must not
succeed.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
---
 .gitlab-ci.yml                                |  2 ++
 .../tests/package/sample_python_treq.py       | 16 ++++++++++
 .../testing/tests/package/test_python_treq.py | 29 +++++++++++++++++++
 3 files changed, 47 insertions(+)
 create mode 100644 support/testing/tests/package/sample_python_treq.py
 create mode 100644 support/testing/tests/package/test_python_treq.py

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 743f4edc06..61a6997178 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -348,6 +348,8 @@ tests.package.test_python_pyyaml.TestPythonPy3Pyyaml: *runtime_test
 tests.package.test_python_service_identity.TestPythonPy2ServiceIdentity: *runtime_test
 tests.package.test_python_service_identity.TestPythonPy3ServiceIdentity: *runtime_test
 tests.package.test_python_subprocess32.TestPythonPy2Subprocess32: *runtime_test
+tests.package.test_python_treq.TestPythonPy2Treq: *runtime_test
+tests.package.test_python_treq.TestPythonPy3Treq: *runtime_test
 tests.package.test_python_twisted.TestPythonPy2Twisted: *runtime_test
 tests.package.test_python_twisted.TestPythonPy3Twisted: *runtime_test
 tests.package.test_python_txaio.TestPythonPy2Txaio: *runtime_test
diff --git a/support/testing/tests/package/sample_python_treq.py b/support/testing/tests/package/sample_python_treq.py
new file mode 100644
index 0000000000..974fdcd8e1
--- /dev/null
+++ b/support/testing/tests/package/sample_python_treq.py
@@ -0,0 +1,16 @@
+from twisted.internet import reactor
+import treq
+
+
+def done(response):
+    print(response.code)
+    reactor.stop()
+
+
+def err(fail):
+    print(fail.value)
+    reactor.stop()
+
+
+treq.get("https://localhost").addCallback(done).addErrback(err)
+reactor.run()
diff --git a/support/testing/tests/package/test_python_treq.py b/support/testing/tests/package/test_python_treq.py
new file mode 100644
index 0000000000..7108b95696
--- /dev/null
+++ b/support/testing/tests/package/test_python_treq.py
@@ -0,0 +1,29 @@
+from tests.package.test_python import TestPythonPackageBase
+
+
+class TestPythonTreq(TestPythonPackageBase):
+    sample_scripts = ["tests/package/sample_python_treq.py"]
+
+    def run_sample_scripts(self):
+        cmd = self.interpreter + " sample_python_treq.py"
+        output, exit_code = self.emulator.run(cmd, timeout=20)
+        self.assertIn("Connection refused", output[0])
+        self.assertEqual(exit_code, 0)
+
+
+class TestPythonPy2Treq(TestPythonTreq):
+    __test__ = True
+    config = TestPythonTreq.config + \
+        """
+        BR2_PACKAGE_PYTHON=y
+        BR2_PACKAGE_PYTHON_TREQ=y
+        """
+
+
+class TestPythonPy3Treq(TestPythonTreq):
+    __test__ = True
+    config = TestPythonTreq.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        BR2_PACKAGE_PYTHON_TREQ=y
+        """
-- 
2.17.1

[Buildroot] [next 15/15] support/testing: add python-ubjson tests

[Ricardo Martincoski]

Add a simple test case to check the basic usage, storing a dict into a
file and then retrieving the dict from the file.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
---
 .gitlab-ci.yml                                |  2 ++
 .../tests/package/sample_python_ubjson_dec.py | 10 ++++++++
 .../tests/package/sample_python_ubjson_enc.py | 14 +++++++++++
 .../tests/package/test_python_ubjson.py       | 23 +++++++++++++++++++
 4 files changed, 49 insertions(+)
 create mode 100644 support/testing/tests/package/sample_python_ubjson_dec.py
 create mode 100644 support/testing/tests/package/sample_python_ubjson_enc.py
 create mode 100644 support/testing/tests/package/test_python_ubjson.py

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 61a6997178..5a100884f6 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -356,6 +356,8 @@ tests.package.test_python_txaio.TestPythonPy2Txaio: *runtime_test
 tests.package.test_python_txaio.TestPythonPy3Txaio: *runtime_test
 tests.package.test_python_txtorcon.TestPythonPy2Txtorcon: *runtime_test
 tests.package.test_python_txtorcon.TestPythonPy3Txtorcon: *runtime_test
+tests.package.test_python_ubjson.TestPythonPy2Ubjson: *runtime_test
+tests.package.test_python_ubjson.TestPythonPy3Ubjson: *runtime_test
 tests.package.test_rust.TestRust: *runtime_test
 tests.package.test_rust.TestRustBin: *runtime_test
 tests.package.test_syslog_ng.TestSyslogNg: *runtime_test
diff --git a/support/testing/tests/package/sample_python_ubjson_dec.py b/support/testing/tests/package/sample_python_ubjson_dec.py
new file mode 100644
index 0000000000..12ccdfd587
--- /dev/null
+++ b/support/testing/tests/package/sample_python_ubjson_dec.py
@@ -0,0 +1,10 @@
+import ubjson
+
+with open("/tmp/data.json", "rb") as f:
+    serialized = f.read()
+data = ubjson.loadb(serialized)
+print(data)
+assert(data["name"] == "python-ubjson")
+assert(data["versions"] == ["1", "2"])
+assert(data["group"]["is_a_package"] is True)
+assert(data["group"]["value"] == 42)
diff --git a/support/testing/tests/package/sample_python_ubjson_enc.py b/support/testing/tests/package/sample_python_ubjson_enc.py
new file mode 100644
index 0000000000..9a2140ef85
--- /dev/null
+++ b/support/testing/tests/package/sample_python_ubjson_enc.py
@@ -0,0 +1,14 @@
+import ubjson
+
+data = {
+    "name": "python-ubjson",
+    "versions": ["1", "2"],
+    "group": {
+        "is_a_package": True,
+        "value": 42
+    }
+}
+serialized = ubjson.dumpb(data)
+print(serialized)
+with open("/tmp/data.json", "wb") as f:
+    f.write(serialized)
diff --git a/support/testing/tests/package/test_python_ubjson.py b/support/testing/tests/package/test_python_ubjson.py
new file mode 100644
index 0000000000..85c707e292
--- /dev/null
+++ b/support/testing/tests/package/test_python_ubjson.py
@@ -0,0 +1,23 @@
+from tests.package.test_python import TestPythonPackageBase
+
+
+class TestPythonPy2Ubjson(TestPythonPackageBase):
+    __test__ = True
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON=y
+        BR2_PACKAGE_PYTHON_UBJSON=y
+        """
+    sample_scripts = ["tests/package/sample_python_ubjson_enc.py",
+                      "tests/package/sample_python_ubjson_dec.py"]
+
+
+class TestPythonPy3Ubjson(TestPythonPackageBase):
+    __test__ = True
+    config = TestPythonPackageBase.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        BR2_PACKAGE_PYTHON_UBJSON=y
+        """
+    sample_scripts = ["tests/package/sample_python_ubjson_enc.py",
+                      "tests/package/sample_python_ubjson_dec.py"]
-- 
2.17.1

[Buildroot] [next 00/15] a few more runtime test cases for python packages - series 2/n

[Thomas Petazzoni]

Hello,

On Fri, 16 Nov 2018 01:57:20 -0200, Ricardo Martincoski wrote:

> Ricardo Martincoski (15):
>   support/testing: add python-argh tests
>   support/testing: add python-attrs tests
>   support/testing: add python-automat tests
>   support/testing: add python-bitstring tests
>   support/testing: add python-cbor tests
>   support/testing: add python-click tests
>   support/testing: add python-constantly tests
>   support/testing: add python-passlib tests
>   support/testing: add python-pexpect tests
>   support/testing: add python-pynacl tests
>   support/testing: add python-pyyaml tests
>   support/testing: add python-service-identity tests
>   support/testing: add python-subprocess32 test
>   support/testing: add python-treq tests
>   support/testing: add python-ubjson tests

Entire series applied to next. Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com