[U-Boot] [PATCH v2] test: py: Add an option to skip sleep test

[U-Boot] [PATCH v2] test: py: Add an option to skip sleep test

[Michal Simek]

Some QEMUs have a problem with time setup that's why
sleep test is failing. Introduce env__sleep_accurate
boardenv variable to have an option to skip sleep test.

Signed-off-by: Michal Simek <michal.simek@xilinx.com>
---

Changes in v2:
- Remove unrelated zc706 change

Changes in v1:
- Change subject line and patch description
- Move from RFC to regular patch
- Remove boardidentity marker and use variable instead

 test/py/tests/test_sleep.py | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/test/py/tests/test_sleep.py b/test/py/tests/test_sleep.py
index 64e057132622..ccef24d7f7b8 100644
--- a/test/py/tests/test_sleep.py
+++ b/test/py/tests/test_sleep.py
@@ -5,10 +5,23 @@
 import pytest
 import time
 
+"""
+Note: This test doesn't rely on boardenv_* configuration values but they can
+change test behavior.
+
+# Setup env__sleep_accurate to False if time is not accurate on your platform
+env__sleep_accurate = False
+
+"""
+
 def test_sleep(u_boot_console):
     """Test the sleep command, and validate that it sleeps for approximately
     the correct amount of time."""
 
+    sleep_skip = u_boot_console.config.env.get('env__sleep_accurate', True)
+    if not sleep_skip:
+        pytest.skip('sleep is not accurate')
+
     if u_boot_console.config.buildconfig.get('config_cmd_misc', 'n') != 'y':
         pytest.skip('sleep command not supported')
     # 3s isn't too long, but is enough to cross a few second boundaries.
-- 
1.9.1

[U-Boot] [PATCH v2] test: py: Add an option to skip sleep test

[Tom Rini]

On Fri, Dec 08, 2017 at 03:47:18PM +0100, Michal Simek wrote:

> Some QEMUs have a problem with time setup that's why
> sleep test is failing. Introduce env__sleep_accurate
> boardenv variable to have an option to skip sleep test.
> 
> Signed-off-by: Michal Simek <michal.simek@xilinx.com>

Reviewed-by: Tom Rini <trini@konsulko.com>

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20171208/81025006/attachment.sig>

[U-Boot] [PATCH v2] test: py: Add an option to skip sleep test

[Stephen Warren]

On 12/08/2017 07:47 AM, Michal Simek wrote:
> Some QEMUs have a problem with time setup that's why
> sleep test is failing. Introduce env__sleep_accurate
> boardenv variable to have an option to skip sleep test.

Nit: Commit descriptions are typically wrapped around 70-74 characters; 
the above is very narrow.

> diff --git a/test/py/tests/test_sleep.py b/test/py/tests/test_sleep.py

> +"""
> +Note: This test doesn't rely on boardenv_* configuration values but they can
> +change test behavior.
> +
> +# Setup env__sleep_accurate to False if time is not accurate on your platform
> +env__sleep_accurate = False
> +
> +"""

Nit: No need for that trailing empty line in the docstring.

Reviewed-by: Stephen Warren <swarren@nvidia.com>

(I was going to contemplate a more generic system implemented in 
conftest.py that could work with any test, but I guess this is simple 
enough and solves the only issue we have right now, so it's fine.)

[U-Boot] [PATCH v2] test: py: Add an option to skip sleep test

[Michal Simek]

2017-12-08 17:54 GMT+01:00 Stephen Warren <swarren@wwwdotorg.org>:

> On 12/08/2017 07:47 AM, Michal Simek wrote:
>
>> Some QEMUs have a problem with time setup that's why
>> sleep test is failing. Introduce env__sleep_accurate
>> boardenv variable to have an option to skip sleep test.
>>
>
> Nit: Commit descriptions are typically wrapped around 70-74 characters;
> the above is very narrow.
>
> diff --git a/test/py/tests/test_sleep.py b/test/py/tests/test_sleep.py
>>
>
> +"""
>> +Note: This test doesn't rely on boardenv_* configuration values but they
>> can
>> +change test behavior.
>> +
>> +# Setup env__sleep_accurate to False if time is not accurate on your
>> platform
>> +env__sleep_accurate = False
>> +
>> +"""
>>
>
> Nit: No need for that trailing empty line in the docstring.
>
> Reviewed-by: Stephen Warren <swarren@nvidia.com>
>
> (I was going to contemplate a more generic system implemented in
> conftest.py that could work with any test, but I guess this is simple
> enough and solves the only issue we have right now, so it's fine.)
>


I think also that generic solution would be better. I was checking xilinx
repo where we have some simple i2c
device tests which also needs to be run based on platforms. This test is
broken on qemu and some boards are also broken.
There is not an issue with travis because sleep tests are disabled now.
For my qemu testing it is not big deal and I can use -k not sleep or this
patch as temporary workaround till we have generic solution in place.

Thanks,
Michal

-- 
Michal Simek, Ing. (M.Eng), OpenPGP -> KeyID: FE3D1F91
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel - Microblaze cpu - http://www.monstr.eu/fdt/
Maintainer of Linux kernel - Xilinx Zynq ARM architecture
Microblaze U-BOOT custodian and responsible for u-boot arm zynq platform

[U-Boot] [U-Boot, v2] test: py: Add an option to skip sleep test

[Tom Rini]

On Fri, Dec 08, 2017 at 03:47:18PM +0100, Michal Simek wrote:

> Some QEMUs have a problem with time setup that's why
> sleep test is failing. Introduce env__sleep_accurate
> boardenv variable to have an option to skip sleep test.
> 
> Signed-off-by: Michal Simek <michal.simek@xilinx.com>
> Reviewed-by: Tom Rini <trini@konsulko.com>
> Reviewed-by: Stephen Warren <swarren@nvidia.com>

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20171213/06596f5a/attachment.sig>