Re: [Buildroot] [PATCH 6/6] support/testing: test_luvi: make luvi test reproducible

["Yann E. MORIN" <yann.morin.1998@free.fr>]

Romain, All,

On 2021-11-12 12:00 +0100, Romain Naour spake thusly:
> As explained by Jörg [1], iteration with pairs() does not result in the
> same order since luajit 2.1.
> 
> From [2]
> "Table iteration with pairs() does not result in the same order?
> 
> The order of table iteration is explicitly undefined by the Lua
> language standard. Different Lua implementations or versions may use
> different orders for otherwise identical tables. Different ways of
> constructing a table may result in different orders, too. Due to
> improved VM security, LuaJIT 2.1 may even use a different order on
> separate VM invocations or when string keys are newly interned.
> 
> If your program relies on a deterministic order, it has a bug.
> Rewrite it, so it doesn't rely on the key order.
> Or sort the table keys, if you must."
> 
> [1] http://lists.busybox.net/pipermail/buildroot/2021-November/627938.html
> [2] https://luajit.org/faq.html
> 
> Signed-off-by: Romain Naour <romain.naour@gmail.com>
> Cc: Francois Perrad <francois.perrad@gadz.org>
> Cc: Jörg Krause <joerg.krause@embedded.rocks>
> ---
>  support/testing/tests/package/test_luvi.py | 10 +++++-----
>  1 file changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/support/testing/tests/package/test_luvi.py b/support/testing/tests/package/test_luvi.py
> index eb15ad31b5..27155a75b6 100644
> --- a/support/testing/tests/package/test_luvi.py
> +++ b/support/testing/tests/package/test_luvi.py
> @@ -23,13 +23,13 @@ class TestLuvi(infra.basetest.BRTest):
>          self.emulator.login()
>  
>      def version_test(self):
> -        cmd = "luvi -v"
> +        cmd = "luvi -v | sort"

The issue with this is that, should 'luvi -v' fail, the command will not
fail, because of the pipe. Indeed, the result of a pipe is the result of
the right-most command, in this case 'sort', which I don't think would
ever fail in practice in this case.

So, to properly catch that 'luvi -v' fails, you would sort in the python
code, something like (assuming the luvi version alwas comes first?):

    cmd = "luvi -v"
    output, exit_code = self.emulator.run(cmd)
    self.assertEqual(exit_code, 0)
    self.assertIn('luvi', output[0])
    output = sorted(output[1:])
    self.assertIn('libuv', output[0])
    self.assertIn('luvi', output[1])
    self.assertIn('rex', output[2])
    self.assertIn('ssl', output[3])
    self.assertIn('zlib', output[4])

Regards,
Yann E. MORIN.

>          output, exit_code = self.emulator.run(cmd)
> -        self.assertIn('luvi', output[0])
> -        self.assertIn('zlib', output[1])
> +        self.assertIn('libuv', output[0])
> +        self.assertIn('luvi', output[1])
>          self.assertIn('rex', output[2])
> -        self.assertIn('libuv', output[3])
> -        self.assertIn('ssl', output[4])
> +        self.assertIn('ssl', output[3])
> +        self.assertIn('zlib', output[4])
>  
>      def test_run(self):
>          self.login()
> -- 
> 2.31.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot