* [Buildroot] Various runtime test failures on Gitlab CI
@ 2019-08-10 13:19 Thomas Petazzoni
2019-08-10 13:52 ` Yann E. MORIN
2019-08-10 22:44 ` Ricardo Martincoski
0 siblings, 2 replies; 5+ messages in thread
From: Thomas Petazzoni @ 2019-08-10 13:19 UTC (permalink / raw)
To: buildroot
Ricardo, Fran?ois, Yegor, Yann,
The last Gitlab CI pipeline on master
(https://gitlab.com/buildroot.org/buildroot/pipelines/75533041/failures)
shows a number of failures in our runtime tests.
Here are the failing tests:
tests.package.test_python_treq.TestPythonPy3Treq
tests.package.test_python_treq.TestPythonPy2Treq
tests.package.test_python_crossbar.TestPythonPy3Crossbar
tests.package.test_perl_io_socket_ssl.TestPerlIOSocketSSL
tests.package.test_lua_http.TestLuajitHttp
tests.package.test_lua_http.TestLuaHttp
tests.init.test_systemd.TestInitSystemSystemdRwFull
And now, the details of the build failures:
- For tests.package.test_python_treq.TestPythonPy3Treq and
tests.package.test_python_treq.TestPythonPy2Treq, we're getting:
Traceback (most recent call last):
File "/builds/buildroot.org/buildroot/support/testing/tests/package/test_python.py", line 125, in test_run
self.run_sample_scripts()
File "/builds/buildroot.org/buildroot/support/testing/tests/package/test_python_treq.py", line 10, in run_sample_scripts
self.assertIn("Connection refused", output[0])
AssertionError: 'Connection refused' not found in ":0: UserWarning: You do not have a working installation of the service_identity module: 'cannot import name 'verify_ip_address' from 'service_identity.pyopenssl' (/usr/lib/python3.7/site-packages/service_identity/pyopenssl.pyc)'. Please install it from <https://pypi.python.org/pypi/service_identity> and make sure all of its dependencies are satisfied. Without the service_identity module, Twisted can perform only rudimentary TLS client hostname verification. Many valid certificate/hostname mappings may be rejected."
- For tests.package.test_python_crossbar.TestPythonPy3Crossbar, we're
getting
Traceback (most recent call last):
File "/builds/buildroot.org/buildroot/support/testing/tests/package/test_python.py", line 125, in test_run
self.run_sample_scripts()
File "/builds/buildroot.org/buildroot/support/testing/tests/package/test_python.py", line 120, in run_sample_scripts
self.assertEqual(exit_code, 0)
AssertionError: 1 != 0
- For tests.package.test_perl_io_socket_ssl.TestPerlIOSocketSSL,
tests.package.test_lua_http.TestLuajitHttp and
tests.package.test_lua_http.TestLuaHttp, we're getting:
======================================================================
ERROR: test_run (tests.package.test_perl_io_socket_ssl.TestPerlIOSocketSSL)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/builds/buildroot.org/buildroot/support/testing/tests/package/test_perl_io_socket_ssl.py", line 21, in test_run
self.module_test("IO::Socket::SSL")
File "/builds/buildroot.org/buildroot/support/testing/tests/package/test_perl.py", line 22, in module_test
_, exit_code = self.emulator.run(cmd)
File "/builds/buildroot.org/buildroot/support/testing/infra/emulator.py", line 109, in run
self.qemu.expect("# ", timeout=timeout)
File "/usr/lib/python2.7/dist-packages/pexpect/spawnbase.py", line 321, in expect
timeout, searchwindowsize, async)
File "/usr/lib/python2.7/dist-packages/pexpect/spawnbase.py", line 345, in expect_list
return exp.expect_loop(timeout)
File "/usr/lib/python2.7/dist-packages/pexpect/expect.py", line 107, in expect_loop
return self.timeout(e)
File "/usr/lib/python2.7/dist-packages/pexpect/expect.py", line 70, in timeout
raise TIMEOUT(msg)
TIMEOUT: Timeout exceeded.
- For tests.init.test_systemd.TestInitSystemSystemdRwFull, we're
getting:
Traceback (most recent call last):
File "/builds/buildroot.org/buildroot/support/testing/tests/init/test_systemd.py", line 157, in test_run
self.start_emulator("ext2", "zImage", "vexpress-v2p-ca9")
File "/builds/buildroot.org/buildroot/support/testing/tests/init/base.py", line 38, in start_emulator
self.emulator.login()
File "/builds/buildroot.org/buildroot/support/testing/infra/emulator.py", line 92, in login
raise SystemError("System does not boot")
SystemError: System does not boot
Could you have a look and fix those issues ?
Thanks!
Thomas
--
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 5+ messages in thread* [Buildroot] Various runtime test failures on Gitlab CI
2019-08-10 13:19 [Buildroot] Various runtime test failures on Gitlab CI Thomas Petazzoni
@ 2019-08-10 13:52 ` Yann E. MORIN
2019-08-10 17:01 ` Yann E. MORIN
2019-08-10 22:44 ` Ricardo Martincoski
1 sibling, 1 reply; 5+ messages in thread
From: Yann E. MORIN @ 2019-08-10 13:52 UTC (permalink / raw)
To: buildroot
Thomas, All,
On 2019-08-10 15:19 +0200, Thomas Petazzoni spake thusly:
> The last Gitlab CI pipeline on master
> (https://gitlab.com/buildroot.org/buildroot/pipelines/75533041/failures)
> shows a number of failures in our runtime tests.
[--SNIP--]
> - For tests.init.test_systemd.TestInitSystemSystemdRwFull, we're
> getting:
>
> Traceback (most recent call last):
> File "/builds/buildroot.org/buildroot/support/testing/tests/init/test_systemd.py", line 157, in test_run
> self.start_emulator("ext2", "zImage", "vexpress-v2p-ca9")
> File "/builds/buildroot.org/buildroot/support/testing/tests/init/base.py", line 38, in start_emulator
> self.emulator.login()
> File "/builds/buildroot.org/buildroot/support/testing/infra/emulator.py", line 92, in login
> raise SystemError("System does not boot")
> SystemError: System does not boot
This one works OK here:
$ ./support/testing/run-tests -o $(pwd)/runtests -k
tests.init.test_systemd.TestInitSystemSystemdRwFull
15:25:25 TestInitSystemSystemdRwFull Starting
15:25:26 TestInitSystemSystemdRwFull Building
15:37:55 TestInitSystemSystemdRwFull Building done
15:38:34 TestInitSystemSystemdRwFull Cleaning up
.
----------------------------------------------------------------------
Ran 1 test in 788.838s
OK
Excerpt of the run log:
10009000.uart: ttyAMA0 at MMIO 0x10009000 (irq = 38, base_baud = 0) is a PL011 rev1
console [ttyAMA0] enabled
[...]
[ ***] (2 of 2) A start job is running for /dev/ttyAMA0 (16s / 1min 31s)
[...]
[ OK ] Found device /dev/ttyAMA0.
[ OK ] Started Serial Getty on ttyAMA0.
while the one on gitlab fails to find the serial:
10009000.uart: ttyAMA0 at MMIO 0x10009000 (irq = 38, base_baud = 0) is a PL011 rev1
console [ttyAMA0] enabled
[...]
[ **] (1 of 2) A start job is running for /dev/ttyAMA0 (1min 28s / 1min 30s)
[...]
[ TIME ] Timed out waiting for device /dev/ttyAMA0.
[DEPEND] Dependency failed for Serial Getty on ttyAMA0.
So, it looks like on my machine, it took abouts 16s for the ttyAMA0
systemd slice to be up (wft? 16s?), while on the gitlab-ci it was not
found...
I'll investigate what makes it appear so late...
Regards,
Yann E. MORIN.
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
^ permalink raw reply [flat|nested] 5+ messages in thread* [Buildroot] Various runtime test failures on Gitlab CI
2019-08-10 13:52 ` Yann E. MORIN
@ 2019-08-10 17:01 ` Yann E. MORIN
2019-08-10 17:28 ` Yann E. MORIN
0 siblings, 1 reply; 5+ messages in thread
From: Yann E. MORIN @ 2019-08-10 17:01 UTC (permalink / raw)
To: buildroot
Thomas, All,
On 2019-08-10 15:52 +0200, Yann E. MORIN spake thusly:
> On 2019-08-10 15:19 +0200, Thomas Petazzoni spake thusly:
> > The last Gitlab CI pipeline on master
> > (https://gitlab.com/buildroot.org/buildroot/pipelines/75533041/failures)
> > shows a number of failures in our runtime tests.
> [--SNIP--]
> > - For tests.init.test_systemd.TestInitSystemSystemdRwFull, we're
> > getting:
> >
> > Traceback (most recent call last):
> > File "/builds/buildroot.org/buildroot/support/testing/tests/init/test_systemd.py", line 157, in test_run
> > self.start_emulator("ext2", "zImage", "vexpress-v2p-ca9")
> > File "/builds/buildroot.org/buildroot/support/testing/tests/init/base.py", line 38, in start_emulator
> > self.emulator.login()
> > File "/builds/buildroot.org/buildroot/support/testing/infra/emulator.py", line 92, in login
> > raise SystemError("System does not boot")
> > SystemError: System does not boot
>
> This one works OK here:
> [ OK ] Found device /dev/ttyAMA0.
> [ OK ] Started Serial Getty on ttyAMA0.
I was however able to reproduce a similar issue with another runtime
test, tests.init.test_systemd.TestInitSystemSystemdRwNetworkd which
failed because it too was unable to find ttyAMA0...
Regards,
Yann E. MORIN.
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
^ permalink raw reply [flat|nested] 5+ messages in thread* [Buildroot] Various runtime test failures on Gitlab CI
2019-08-10 17:01 ` Yann E. MORIN
@ 2019-08-10 17:28 ` Yann E. MORIN
0 siblings, 0 replies; 5+ messages in thread
From: Yann E. MORIN @ 2019-08-10 17:28 UTC (permalink / raw)
To: buildroot
All,
On 2019-08-10 19:01 +0200, Yann E. MORIN spake thusly:
> On 2019-08-10 15:52 +0200, Yann E. MORIN spake thusly:
> > On 2019-08-10 15:19 +0200, Thomas Petazzoni spake thusly:
> > > - For tests.init.test_systemd.TestInitSystemSystemdRwFull, we're
> > > getting:
[--SNIP--]
> > > SystemError: System does not boot
> > This one works OK here:
[--SNIP--]
> I was however able to reproduce a similar issue with another runtime
> test, tests.init.test_systemd.TestInitSystemSystemdRwNetworkd which
> failed because it too was unable to find ttyAMA0...
And restarting it again no longer fails...
So, it really is a timing issue, it seems. Meh...
Regards,
Yann E. MORIN.
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
^ permalink raw reply [flat|nested] 5+ messages in thread
* [Buildroot] Various runtime test failures on Gitlab CI
2019-08-10 13:19 [Buildroot] Various runtime test failures on Gitlab CI Thomas Petazzoni
2019-08-10 13:52 ` Yann E. MORIN
@ 2019-08-10 22:44 ` Ricardo Martincoski
1 sibling, 0 replies; 5+ messages in thread
From: Ricardo Martincoski @ 2019-08-10 22:44 UTC (permalink / raw)
To: buildroot
Hello,
I will send a small series to fix the python-*, lua-* and perl-* tests.
On Sat, Aug 10, 2019 at 10:19 AM, Thomas Petazzoni wrote:
[snip]
> - For tests.package.test_python_treq.TestPythonPy3Treq and
> tests.package.test_python_treq.TestPythonPy2Treq, we're getting:
>
> Traceback (most recent call last):
> File "/builds/buildroot.org/buildroot/support/testing/tests/package/test_python.py", line 125, in test_run
> self.run_sample_scripts()
> File "/builds/buildroot.org/buildroot/support/testing/tests/package/test_python_treq.py", line 10, in run_sample_scripts
> self.assertIn("Connection refused", output[0])
> AssertionError: 'Connection refused' not found in ":0: UserWarning: You do not have a working installation of the service_identity module: 'cannot import name 'verify_ip_address' from 'service_identity.pyopenssl' (/usr/lib/python3.7/site-packages/service_identity/pyopenssl.pyc)'. Please install it from <https://pypi.python.org/pypi/service_identity> and make sure all of its dependencies are satisfied. Without the service_identity module, Twisted can perform only rudimentary TLS client hostname verification. Many valid certificate/hostname mappings may be rejected."
The new warning message from treq at runtime confuses the test. I will improve
the test code to ignore it.
> - For tests.package.test_python_crossbar.TestPythonPy3Crossbar, we're
> getting
>
> Traceback (most recent call last):
> File "/builds/buildroot.org/buildroot/support/testing/tests/package/test_python.py", line 125, in test_run
> self.run_sample_scripts()
> File "/builds/buildroot.org/buildroot/support/testing/tests/package/test_python.py", line 120, in run_sample_scripts
> self.assertEqual(exit_code, 0)
> AssertionError: 1 != 0
python-autobahn now defaults to use msgpack when CPython is used. But it
also still supports umsgpack when the user explicitly selects it using an
environment variable.
python-crossbar selects python-u-msgpack on menuconfig.
I will change the test to explicitly use umsgpack.
> - For tests.package.test_perl_io_socket_ssl.TestPerlIOSocketSSL,
> tests.package.test_lua_http.TestLuajitHttp and
> tests.package.test_lua_http.TestLuaHttp, we're getting:
>
> ======================================================================
> ERROR: test_run (tests.package.test_perl_io_socket_ssl.TestPerlIOSocketSSL)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File "/builds/buildroot.org/buildroot/support/testing/tests/package/test_perl_io_socket_ssl.py", line 21, in test_run
> self.module_test("IO::Socket::SSL")
> File "/builds/buildroot.org/buildroot/support/testing/tests/package/test_perl.py", line 22, in module_test
> _, exit_code = self.emulator.run(cmd)
> File "/builds/buildroot.org/buildroot/support/testing/infra/emulator.py", line 109, in run
> self.qemu.expect("# ", timeout=timeout)
> File "/usr/lib/python2.7/dist-packages/pexpect/spawnbase.py", line 321, in expect
> timeout, searchwindowsize, async)
> File "/usr/lib/python2.7/dist-packages/pexpect/spawnbase.py", line 345, in expect_list
> return exp.expect_loop(timeout)
> File "/usr/lib/python2.7/dist-packages/pexpect/expect.py", line 107, in expect_loop
> return self.timeout(e)
> File "/usr/lib/python2.7/dist-packages/pexpect/expect.py", line 70, in timeout
> raise TIMEOUT(msg)
> TIMEOUT: Timeout exceeded.
All of these need entropy and are currently using armv7 kernel. I will switch to
use armv5 builtin kernel that already provides entropy.
Regards,
Ricardo
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2019-08-10 22:44 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-08-10 13:19 [Buildroot] Various runtime test failures on Gitlab CI Thomas Petazzoni
2019-08-10 13:52 ` Yann E. MORIN
2019-08-10 17:01 ` Yann E. MORIN
2019-08-10 17:28 ` Yann E. MORIN
2019-08-10 22:44 ` Ricardo Martincoski
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox