[Buildroot] [PATCH 1/1] support/testing: python-magic-wormhole: fix random failures

[Buildroot] [PATCH 1/1] support/testing: python-magic-wormhole: fix random failures

[Julien Olivain]

The magic-wormhole "receive" command can output "waiting" messages
when key receival or verification are longer than a predefined
timeout:
https://github.com/magic-wormhole/magic-wormhole/blob/0.13.0/src/wormhole/cli/cmd_receive.py#L135

The intent is to have an interactive user experience.

This behavior makes the runtime test unreliable as the test always
expect the sent message as the exact output. When the test execution
is slower, it sometimes get the "waiting" message instead of the
expected message.

Some test jobs are succeeding:
https://gitlab.com/buildroot.org/buildroot/-/jobs/4968059737
while some other are failing.

magic-wormhole can override those timers with environment variables.
See:
https://github.com/magic-wormhole/magic-wormhole/blob/0.13.0/src/wormhole/cli/cmd_receive.py#L26

This commit sets those environment variable to larger values
(100 seconds instread of 1 by default), to make sure the test will
always pass.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/4962923235

Reported-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
---
 support/testing/tests/package/test_python_magic_wormhole.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/support/testing/tests/package/test_python_magic_wormhole.py b/support/testing/tests/package/test_python_magic_wormhole.py
index abd34bca50..5a0f0277b0 100644
--- a/support/testing/tests/package/test_python_magic_wormhole.py
+++ b/support/testing/tests/package/test_python_magic_wormhole.py
@@ -50,7 +50,9 @@ class TestPythonPy3MagicWormhole(TestPythonPackageBase):
         cmd += "sleep 25"
         self.assertRunOk(cmd, timeout=30)
 
-        cmd = wormhole_cmd + " receive {}".format(code)
+        wormhole_env = "_MAGIC_WORMHOLE_TEST_KEY_TIMER=100 "
+        wormhole_env += "_MAGIC_WORMHOLE_TEST_VERIFY_TIMER=100 "
+        cmd = wormhole_env + wormhole_cmd + " receive {}".format(code)
         output, exit_code = self.emulator.run(cmd, timeout=35)
         self.assertEqual(exit_code, 0)
         self.assertEqual(output[0], text)
-- 
2.41.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/1] support/testing: python-magic-wormhole: fix random failures

[Romain Naour]

Hello Julien,

Le 31/08/2023 à 19:37, Julien Olivain a écrit :
> The magic-wormhole "receive" command can output "waiting" messages
> when key receival or verification are longer than a predefined
> timeout:
> https://github.com/magic-wormhole/magic-wormhole/blob/0.13.0/src/wormhole/cli/cmd_receive.py#L135
> 
> The intent is to have an interactive user experience.
> 
> This behavior makes the runtime test unreliable as the test always
> expect the sent message as the exact output. When the test execution
> is slower, it sometimes get the "waiting" message instead of the
> expected message.
> 
> Some test jobs are succeeding:
> https://gitlab.com/buildroot.org/buildroot/-/jobs/4968059737
> while some other are failing.
> 
> magic-wormhole can override those timers with environment variables.
> See:
> https://github.com/magic-wormhole/magic-wormhole/blob/0.13.0/src/wormhole/cli/cmd_receive.py#L26
> 
> This commit sets those environment variable to larger values
> (100 seconds instread of 1 by default), to make sure the test will
> always pass.
> 
> Fixes:
> https://gitlab.com/buildroot.org/buildroot/-/jobs/4962923235
> 
> Reported-by: Romain Naour <romain.naour@gmail.com>
> Signed-off-by: Julien Olivain <ju.o@free.fr>

Reviewed-by: Romain Naour <romain.naour@gmail.com>
Tested-by: Romain Naour <romain.naour@gmail.com>

Best regards,
Romain


> ---
>  support/testing/tests/package/test_python_magic_wormhole.py | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/support/testing/tests/package/test_python_magic_wormhole.py b/support/testing/tests/package/test_python_magic_wormhole.py
> index abd34bca50..5a0f0277b0 100644
> --- a/support/testing/tests/package/test_python_magic_wormhole.py
> +++ b/support/testing/tests/package/test_python_magic_wormhole.py
> @@ -50,7 +50,9 @@ class TestPythonPy3MagicWormhole(TestPythonPackageBase):
>          cmd += "sleep 25"
>          self.assertRunOk(cmd, timeout=30)
>  
> -        cmd = wormhole_cmd + " receive {}".format(code)
> +        wormhole_env = "_MAGIC_WORMHOLE_TEST_KEY_TIMER=100 "
> +        wormhole_env += "_MAGIC_WORMHOLE_TEST_VERIFY_TIMER=100 "
> +        cmd = wormhole_env + wormhole_cmd + " receive {}".format(code)
>          output, exit_code = self.emulator.run(cmd, timeout=35)
>          self.assertEqual(exit_code, 0)
>          self.assertEqual(output[0], text)

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/1] support/testing: python-magic-wormhole: fix random failures

[Thomas Petazzoni via buildroot]

On Thu, 31 Aug 2023 19:37:36 +0200
Julien Olivain <ju.o@free.fr> wrote:

> The magic-wormhole "receive" command can output "waiting" messages
> when key receival or verification are longer than a predefined
> timeout:
> https://github.com/magic-wormhole/magic-wormhole/blob/0.13.0/src/wormhole/cli/cmd_receive.py#L135
> 
> The intent is to have an interactive user experience.
> 
> This behavior makes the runtime test unreliable as the test always
> expect the sent message as the exact output. When the test execution
> is slower, it sometimes get the "waiting" message instead of the
> expected message.
> 
> Some test jobs are succeeding:
> https://gitlab.com/buildroot.org/buildroot/-/jobs/4968059737
> while some other are failing.
> 
> magic-wormhole can override those timers with environment variables.
> See:
> https://github.com/magic-wormhole/magic-wormhole/blob/0.13.0/src/wormhole/cli/cmd_receive.py#L26
> 
> This commit sets those environment variable to larger values
> (100 seconds instread of 1 by default), to make sure the test will
> always pass.
> 
> Fixes:
> https://gitlab.com/buildroot.org/buildroot/-/jobs/4962923235
> 
> Reported-by: Romain Naour <romain.naour@gmail.com>
> Signed-off-by: Julien Olivain <ju.o@free.fr>
> ---
>  support/testing/tests/package/test_python_magic_wormhole.py | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)

Applied to master, thanks.

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/1] support/testing: python-magic-wormhole: fix random failures

[Peter Korsgaard]

>>>>> "Julien" == Julien Olivain <ju.o@free.fr> writes:

 > The magic-wormhole "receive" command can output "waiting" messages
 > when key receival or verification are longer than a predefined
 > timeout:
 > https://github.com/magic-wormhole/magic-wormhole/blob/0.13.0/src/wormhole/cli/cmd_receive.py#L135

 > The intent is to have an interactive user experience.

 > This behavior makes the runtime test unreliable as the test always
 > expect the sent message as the exact output. When the test execution
 > is slower, it sometimes get the "waiting" message instead of the
 > expected message.

 > Some test jobs are succeeding:
 > https://gitlab.com/buildroot.org/buildroot/-/jobs/4968059737
 > while some other are failing.

 > magic-wormhole can override those timers with environment variables.
 > See:
 > https://github.com/magic-wormhole/magic-wormhole/blob/0.13.0/src/wormhole/cli/cmd_receive.py#L26

 > This commit sets those environment variable to larger values
 > (100 seconds instread of 1 by default), to make sure the test will
 > always pass.

 > Fixes:
 > https://gitlab.com/buildroot.org/buildroot/-/jobs/4962923235

 > Reported-by: Romain Naour <romain.naour@gmail.com>
 > Signed-off-by: Julien Olivain <ju.o@free.fr>

Committed to 2023.02.x and 2023.05.x, thanks.

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot