[Buildroot] [next] listing of tests is broken

[Buildroot] [next] listing of tests is broken

[Yegor Yefremov]

./support/testing/run-tests -l

  File "/home/user/br_test/support/testing/infra/basetest.py", line
45, in __init__
    self.config += "\nBR2_JLEVEL={}\n".format(self.jlevel)
TypeError: unsupported operand type(s) for +=: 'NoneType' and 'str'

Any idea?

Yegor

[Buildroot] [next] listing of tests is broken

[Thomas Petazzoni]

Hello,

On Fri, 9 Feb 2018 09:44:26 +0100, Yegor Yefremov wrote:
> ./support/testing/run-tests -l
> 
>   File "/home/user/br_test/support/testing/infra/basetest.py", line
> 45, in __init__
>     self.config += "\nBR2_JLEVEL={}\n".format(self.jlevel)
> TypeError: unsupported operand type(s) for +=: 'NoneType' and 'str'
> 
> Any idea?

Yes, it is a fallout from the Rust tests, one of the class has no
"config" variable, which causes this breakage.

I have not yet decided if the right fix is to replace config = None by
config = list() in the base test class, or to fix the Rust test case.

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
http://bootlin.com

[Buildroot] [next] listing of tests is broken

[Yegor Yefremov]

On Fri, Feb 9, 2018 at 9:54 AM, Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
> Hello,
>
> On Fri, 9 Feb 2018 09:44:26 +0100, Yegor Yefremov wrote:
>> ./support/testing/run-tests -l
>>
>>   File "/home/user/br_test/support/testing/infra/basetest.py", line
>> 45, in __init__
>>     self.config += "\nBR2_JLEVEL={}\n".format(self.jlevel)
>> TypeError: unsupported operand type(s) for +=: 'NoneType' and 'str'
>>
>> Any idea?
>
> Yes, it is a fallout from the Rust tests, one of the class has no
> "config" variable, which causes this breakage.
>
> I have not yet decided if the right fix is to replace config = None by
> config = list() in the base test class, or to fix the Rust test case.

I'm for the first fix: config = list()

Yegor

[Buildroot] [next] listing of tests is broken

[Thomas Petazzoni]

Hello,

On Fri, 9 Feb 2018 10:04:37 +0100, Yegor Yefremov wrote:

> > I have not yet decided if the right fix is to replace config = None by
> > config = list() in the base test class, or to fix the Rust test case.  
> 
> I'm for the first fix: config = list()

Before going with this fix, I'd like to understand if what the Rust
tests are doing is legit or not. If it isn't, then we may not
necessarily want to paper over the problem.

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
http://bootlin.com

[Buildroot] [next] listing of tests is broken

[Ricardo Martincoski]

Hello,

It happens on the master branch too.

On Fri, Feb 09, 2018 at 08:06 AM, Thomas Petazzoni wrote:

> On Fri, 9 Feb 2018 10:04:37 +0100, Yegor Yefremov wrote:
> 
>> > I have not yet decided if the right fix is to replace config = None by
>> > config = list() in the base test class, or to fix the Rust test case.  
>> 
>> I'm for the first fix: config = list()
> 
> Before going with this fix, I'd like to understand if what the Rust
> tests are doing is legit or not. If it isn't, then we may not
> necessarily want to paper over the problem.

The correct fix IMO is to change the Rust test case.

With the current code, nose2 will collect 3 test cases from the file:
TestRustBase, TestRustBin and TestRust. It happens because all three have a
method called test_something. But TestRustBase is not a test case by itself.

So we should move the test_* method from the base class to the 2 derived classes
(yes, duplicating the code), like it is already done by the 'init' test cases.

An alternative solution is to rename the method in the base class to not start
with test_ and create a test_run method in each derived class calling it. The
method is short, so I don't think we need to do this.


With the listing of tests broken, also 'make .gitlab-ci.yml' is broken. Based
only on this I would say we should fix that on master branch. But see below...
And also the update to .gitlab-ci.yml must be committed. This way we fix
check-gitlab-ci.yml in GitLab:
https://gitlab.com/buildroot.org/buildroot/-/jobs/52000035

But after fixing those, the TestRustBin still fails at GitLab:
https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/52092045
from test_rust.py, line 39, like this:
>SystemError: Cargo init failed
The build.log ends in these messages:
>error: Failed to create project `hello-world` at `/tmp/tmp_US3dA-br2-testing-rust/hello-world`
>
>Caused by:
>  could not determine the current user, please set $USER
The same test case does not fail for me when it runs outside the Docker image,
so I guess we will need to tweak the Docker image.


I don't have the result yet for TestRust in GitLab. It is still running (105
minutes and counting) by the time I write this.
https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/52092044
But running outside the Docker image it also passes.


... Would it be the case to revert only the rust test case only from the master
branch? If yes, extra care must be taken when merging the branches to not remove
it from the next branch too.


Regards,
Ricardo

[Buildroot] [next] listing of tests is broken

[Thomas Petazzoni]

Hello,

On Sun, 11 Feb 2018 23:44:02 -0200, Ricardo Martincoski wrote:

> The correct fix IMO is to change the Rust test case.

Agreed.

> With the current code, nose2 will collect 3 test cases from the file:
> TestRustBase, TestRustBin and TestRust. It happens because all three have a
> method called test_something. But TestRustBase is not a test case by itself.
> 
> So we should move the test_* method from the base class to the 2 derived classes
> (yes, duplicating the code), like it is already done by the 'init' test cases.
> 
> An alternative solution is to rename the method in the base class to not start
> with test_ and create a test_run method in each derived class calling it. The
> method is short, so I don't think we need to do this.

Yes, duplicating would be fine for now.

> With the listing of tests broken, also 'make .gitlab-ci.yml' is broken. Based
> only on this I would say we should fix that on master branch. But see below...
> And also the update to .gitlab-ci.yml must be committed. This way we fix
> check-gitlab-ci.yml in GitLab:
> https://gitlab.com/buildroot.org/buildroot/-/jobs/52000035
> 
> But after fixing those, the TestRustBin still fails at GitLab:
> https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/52092045
> from test_rust.py, line 39, like this:
> >SystemError: Cargo init failed  
> The build.log ends in these messages:
> >error: Failed to create project `hello-world` at `/tmp/tmp_US3dA-br2-testing-rust/hello-world`
> >
> >Caused by:
> >  could not determine the current user, please set $USER  
> The same test case does not fail for me when it runs outside the Docker image,
> so I guess we will need to tweak the Docker image.
> 
> 
> I don't have the result yet for TestRust in GitLab. It is still running (105
> minutes and counting) by the time I write this.
> https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/52092044
> But running outside the Docker image it also passes.
> 
> ... Would it be the case to revert only the rust test case only from the master
> branch? If yes, extra care must be taken when merging the branches to not remove
> it from the next branch too.

I'd say, we fix the test enumeration problem, and then let the Rust
test case fail. Hopefully Eric Le Bihan (added in Cc) will chime in and
fix the Rust test failure.

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
http://bootlin.com

[Buildroot] [PATCH] support/testing: fix job check-gitlab-ci.yml

[Ricardo Martincoski]

Currently 'run-tests -l' is broken. It breaks 'make .gitlab-ci.yml' that
in turn breaks the job in GitLab.

TestRustBase is not a test case by itself, so it can't have a method
with the name starting with "test_" otherwise nose2 assumes it is a test
case.
Move the test_run method from the base class to the derived classes.

While at it, update .gitlab-ci.yml with the new test cases.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/52000035

Reported-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Eric Le Bihan <eric.le.bihan.dev@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
---
check-gitlab-ci.yml after the fix:
https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/52092002
which is part of a full run:
https://gitlab.com/RicardoMartincoski/buildroot/pipelines/17391318
---
 .gitlab-ci.yml                             |  2 ++
 support/testing/tests/package/test_rust.py | 18 ++++++++++++------
 2 files changed, 14 insertions(+), 6 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 30a4399..f76e53b 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -282,6 +282,8 @@ tests.package.test_ipython.TestIPythonPy2: *runtime_test
 tests.package.test_ipython.TestIPythonPy3: *runtime_test
 tests.package.test_python.TestPython2: *runtime_test
 tests.package.test_python.TestPython3: *runtime_test
+tests.package.test_rust.TestRust: *runtime_test
+tests.package.test_rust.TestRustBin: *runtime_test
 tests.toolchain.test_external.TestExternalToolchainBuildrootMusl: *runtime_test
 tests.toolchain.test_external.TestExternalToolchainBuildrootuClibc: *runtime_test
 tests.toolchain.test_external.TestExternalToolchainCCache: *runtime_test
diff --git a/support/testing/tests/package/test_rust.py b/support/testing/tests/package/test_rust.py
index 8035f8b..e6c0de2 100644
--- a/support/testing/tests/package/test_rust.py
+++ b/support/testing/tests/package/test_rust.py
@@ -53,12 +53,6 @@ class TestRustBase(infra.basetest.BRTest):
         self.b.build()
         shutil.rmtree(workdir)
 
-    def test_run(self):
-        self.build_test_prog()
-        self.login()
-        _, exit_code = self.emulator.run(self.crate)
-        self.assertEqual(exit_code, 0)
-
 
 class TestRustBin(TestRustBase):
     config = \
@@ -82,6 +76,12 @@ class TestRustBin(TestRustBase):
              BR2_PACKAGE_HOST_RUSTC=y
              """
 
+    def test_run(self):
+        self.build_test_prog()
+        self.login()
+        _, exit_code = self.emulator.run(self.crate)
+        self.assertEqual(exit_code, 0)
+
 
 class TestRust(TestRustBase):
     config = \
@@ -105,3 +105,9 @@ class TestRust(TestRustBase):
              BR2_PACKAGE_HOST_RUSTC=y
              BR2_PACKAGE_HOST_RUST=y
              """
+
+    def test_run(self):
+        self.build_test_prog()
+        self.login()
+        _, exit_code = self.emulator.run(self.crate)
+        self.assertEqual(exit_code, 0)
-- 
2.7.4

[Buildroot] [PATCH] support/testing: fix job check-gitlab-ci.yml

[Yegor Yefremov]

On Mon, Feb 12, 2018 at 10:02 PM, Ricardo Martincoski
<ricardo.martincoski@gmail.com> wrote:
> Currently 'run-tests -l' is broken. It breaks 'make .gitlab-ci.yml' that
> in turn breaks the job in GitLab.
>
> TestRustBase is not a test case by itself, so it can't have a method
> with the name starting with "test_" otherwise nose2 assumes it is a test
> case.
> Move the test_run method from the base class to the derived classes.
>
> While at it, update .gitlab-ci.yml with the new test cases.
>
> Fixes:
> https://gitlab.com/buildroot.org/buildroot/-/jobs/52000035
>
> Reported-by: Yegor Yefremov <yegorslists@googlemail.com>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
> Cc: Eric Le Bihan <eric.le.bihan.dev@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Yann E. MORIN <yann.morin.1998@free.fr>
> Cc: Yegor Yefremov <yegorslists@googlemail.com>

Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>

> ---
> check-gitlab-ci.yml after the fix:
> https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/52092002
> which is part of a full run:
> https://gitlab.com/RicardoMartincoski/buildroot/pipelines/17391318
> ---
>  .gitlab-ci.yml                             |  2 ++
>  support/testing/tests/package/test_rust.py | 18 ++++++++++++------
>  2 files changed, 14 insertions(+), 6 deletions(-)
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 30a4399..f76e53b 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -282,6 +282,8 @@ tests.package.test_ipython.TestIPythonPy2: *runtime_test
>  tests.package.test_ipython.TestIPythonPy3: *runtime_test
>  tests.package.test_python.TestPython2: *runtime_test
>  tests.package.test_python.TestPython3: *runtime_test
> +tests.package.test_rust.TestRust: *runtime_test
> +tests.package.test_rust.TestRustBin: *runtime_test
>  tests.toolchain.test_external.TestExternalToolchainBuildrootMusl: *runtime_test
>  tests.toolchain.test_external.TestExternalToolchainBuildrootuClibc: *runtime_test
>  tests.toolchain.test_external.TestExternalToolchainCCache: *runtime_test
> diff --git a/support/testing/tests/package/test_rust.py b/support/testing/tests/package/test_rust.py
> index 8035f8b..e6c0de2 100644
> --- a/support/testing/tests/package/test_rust.py
> +++ b/support/testing/tests/package/test_rust.py
> @@ -53,12 +53,6 @@ class TestRustBase(infra.basetest.BRTest):
>          self.b.build()
>          shutil.rmtree(workdir)
>
> -    def test_run(self):
> -        self.build_test_prog()
> -        self.login()
> -        _, exit_code = self.emulator.run(self.crate)
> -        self.assertEqual(exit_code, 0)
> -
>
>  class TestRustBin(TestRustBase):
>      config = \
> @@ -82,6 +76,12 @@ class TestRustBin(TestRustBase):
>               BR2_PACKAGE_HOST_RUSTC=y
>               """
>
> +    def test_run(self):
> +        self.build_test_prog()
> +        self.login()
> +        _, exit_code = self.emulator.run(self.crate)
> +        self.assertEqual(exit_code, 0)
> +
>
>  class TestRust(TestRustBase):
>      config = \
> @@ -105,3 +105,9 @@ class TestRust(TestRustBase):
>               BR2_PACKAGE_HOST_RUSTC=y
>               BR2_PACKAGE_HOST_RUST=y
>               """
> +
> +    def test_run(self):
> +        self.build_test_prog()
> +        self.login()
> +        _, exit_code = self.emulator.run(self.crate)
> +        self.assertEqual(exit_code, 0)
> --
> 2.7.4
>

[Buildroot] [PATCH] support/testing: fix job check-gitlab-ci.yml

[Thomas Petazzoni]

Hello,

On Mon, 12 Feb 2018 19:02:08 -0200, Ricardo Martincoski wrote:
> Currently 'run-tests -l' is broken. It breaks 'make .gitlab-ci.yml' that
> in turn breaks the job in GitLab.
> 
> TestRustBase is not a test case by itself, so it can't have a method
> with the name starting with "test_" otherwise nose2 assumes it is a test
> case.
> Move the test_run method from the base class to the derived classes.
> 
> While at it, update .gitlab-ci.yml with the new test cases.
> 
> Fixes:
> https://gitlab.com/buildroot.org/buildroot/-/jobs/52000035
> 
> Reported-by: Yegor Yefremov <yegorslists@googlemail.com>
> Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
> Cc: Eric Le Bihan <eric.le.bihan.dev@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Yann E. MORIN <yann.morin.1998@free.fr>
> Cc: Yegor Yefremov <yegorslists@googlemail.com>
> ---
> check-gitlab-ci.yml after the fix:
> https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/52092002
> which is part of a full run:
> https://gitlab.com/RicardoMartincoski/buildroot/pipelines/17391318
> ---
>  .gitlab-ci.yml                             |  2 ++
>  support/testing/tests/package/test_rust.py | 18 ++++++++++++------
>  2 files changed, 14 insertions(+), 6 deletions(-)

Applied to master, thanks.

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com