[U-Boot] [PATCH] test/py: Provide custom IDs when parametrizing tests

[U-Boot] [PATCH] test/py: Provide custom IDs when parametrizing tests

[Stephen Warren]

From: Stephen Warren <swarren@nvidia.com>

When pytest generates the name for parametrized tests, simple parameter
values (ints, strings) get used directly, but more complex values such
as dicts are not handled. This yields test names such as:

    dfu[env__usb_dev_port0-env__dfu_config0]
    dfu[env__usb_dev_port0-env__dfu_config1]

Add some code to extract a custom fixture ID from the fixture values, so
that we end up with meaningful names such as:

    dfu[micro_b-emmc]
    dfu[devport2-ram]

If the boardenv file doesn't define custom names, the code falls back to
the old algorithm.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
---
 test/py/conftest.py       | 8 +++++++-
 test/py/tests/test_dfu.py | 3 +++
 test/py/tests/test_ums.py | 3 +++
 3 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/test/py/conftest.py b/test/py/conftest.py
index a4e54c66ceda..9c9426aebe10 100644
--- a/test/py/conftest.py
+++ b/test/py/conftest.py
@@ -225,7 +225,13 @@ def pytest_generate_tests(metafunc):
             # ... otherwise, see if there's a key that contains a list of
             # values to use instead.
             vals = subconfig.get(fn + 's', [])
-        metafunc.parametrize(fn, vals)
+        def fixture_id(index, val):
+            try:
+                return val["fixture_id"]
+            except:
+                return fn + str(index)
+        ids = [fixture_id(index, val) for (index, val) in enumerate(vals)]
+        metafunc.parametrize(fn, vals, ids=ids)
 
 @pytest.fixture(scope='function')
 def u_boot_console(request):
diff --git a/test/py/tests/test_dfu.py b/test/py/tests/test_dfu.py
index c09b90278dca..bb70008af300 100644
--- a/test/py/tests/test_dfu.py
+++ b/test/py/tests/test_dfu.py
@@ -21,6 +21,7 @@ For example:
 
 env__usb_dev_ports = (
     {
+        "fixture_id": "micro_b",
         "tgt_usb_ctlr": "0",
         "host_usb_dev_node": "/dev/usbdev-p2371-2180",
         # This parameter is optional /if/ you only have a single board
@@ -32,10 +33,12 @@ env__usb_dev_ports = (
 env__dfu_configs = (
     # eMMC, partition 1
     {
+        "fixture_id": "emmc",
         "alt_info": "/dfu_test.bin ext4 0 1;/dfu_dummy.bin ext4 0 1",
         "cmd_params": "mmc 0",
     },
 )
+
 b) udev rules to set permissions on devices nodes, so that sudo is not
 required. For example:
 
diff --git a/test/py/tests/test_ums.py b/test/py/tests/test_ums.py
index 21d40a972581..8c3ee2b053c4 100644
--- a/test/py/tests/test_ums.py
+++ b/test/py/tests/test_ums.py
@@ -29,6 +29,7 @@ env__mount_points = (
 
 env__usb_dev_ports = (
     {
+        "fixture_id": "micro_b",
         "tgt_usb_ctlr": "0",
         "host_ums_dev_node": "/dev/disk/by-path/pci-0000:00:14.0-usb-0:13:1.0-scsi-0:0:0:0",
     },
@@ -37,6 +38,7 @@ env__usb_dev_ports = (
 env__block_devs = (
     # eMMC; always present
     {
+        "fixture_id": "emmc",
         "type": "mmc",
         "id": "0",
         # The following two properties are optional.
@@ -48,6 +50,7 @@ env__block_devs = (
     },
     # SD card; present since I plugged one in
     {
+        "fixture_id": "sd",
         "type": "mmc",
         "id": "1"
     },
-- 
2.7.0

[U-Boot] [PATCH] test/py: Provide custom IDs when parametrizing tests

[Simon Glass]

Hi Stephen,

On 26 January 2016 at 15:26, Stephen Warren <swarren@wwwdotorg.org> wrote:
> From: Stephen Warren <swarren@nvidia.com>
>
> When pytest generates the name for parametrized tests, simple parameter
> values (ints, strings) get used directly, but more complex values such
> as dicts are not handled. This yields test names such as:
>
>     dfu[env__usb_dev_port0-env__dfu_config0]
>     dfu[env__usb_dev_port0-env__dfu_config1]
>
> Add some code to extract a custom fixture ID from the fixture values, so
> that we end up with meaningful names such as:
>
>     dfu[micro_b-emmc]
>     dfu[devport2-ram]
>
> If the boardenv file doesn't define custom names, the code falls back to
> the old algorithm.
>
> Signed-off-by: Stephen Warren <swarren@nvidia.com>
> ---
>  test/py/conftest.py       | 8 +++++++-
>  test/py/tests/test_dfu.py | 3 +++
>  test/py/tests/test_ums.py | 3 +++
>  3 files changed, 13 insertions(+), 1 deletion(-)

Reviewed-by: Simon Glass <sjg@chromium.org>

>
> diff --git a/test/py/conftest.py b/test/py/conftest.py
> index a4e54c66ceda..9c9426aebe10 100644
> --- a/test/py/conftest.py
> +++ b/test/py/conftest.py
> @@ -225,7 +225,13 @@ def pytest_generate_tests(metafunc):
>              # ... otherwise, see if there's a key that contains a list of
>              # values to use instead.

Is this function comment still accurate?

>              vals = subconfig.get(fn + 's', [])
> -        metafunc.parametrize(fn, vals)
> +        def fixture_id(index, val):
> +            try:
> +                return val["fixture_id"]
> +            except:
> +                return fn + str(index)
> +        ids = [fixture_id(index, val) for (index, val) in enumerate(vals)]
> +        metafunc.parametrize(fn, vals, ids=ids)
>
>  @pytest.fixture(scope='function')
>  def u_boot_console(request):
> diff --git a/test/py/tests/test_dfu.py b/test/py/tests/test_dfu.py
> index c09b90278dca..bb70008af300 100644
> --- a/test/py/tests/test_dfu.py
> +++ b/test/py/tests/test_dfu.py
> @@ -21,6 +21,7 @@ For example:
>
>  env__usb_dev_ports = (
>      {
> +        "fixture_id": "micro_b",
>          "tgt_usb_ctlr": "0",
>          "host_usb_dev_node": "/dev/usbdev-p2371-2180",
>          # This parameter is optional /if/ you only have a single board
> @@ -32,10 +33,12 @@ env__usb_dev_ports = (
>  env__dfu_configs = (
>      # eMMC, partition 1
>      {
> +        "fixture_id": "emmc",
>          "alt_info": "/dfu_test.bin ext4 0 1;/dfu_dummy.bin ext4 0 1",
>          "cmd_params": "mmc 0",
>      },
>  )
> +
>  b) udev rules to set permissions on devices nodes, so that sudo is not
>  required. For example:
>
> diff --git a/test/py/tests/test_ums.py b/test/py/tests/test_ums.py
> index 21d40a972581..8c3ee2b053c4 100644
> --- a/test/py/tests/test_ums.py
> +++ b/test/py/tests/test_ums.py
> @@ -29,6 +29,7 @@ env__mount_points = (
>
>  env__usb_dev_ports = (
>      {
> +        "fixture_id": "micro_b",
>          "tgt_usb_ctlr": "0",
>          "host_ums_dev_node": "/dev/disk/by-path/pci-0000:00:14.0-usb-0:13:1.0-scsi-0:0:0:0",
>      },
> @@ -37,6 +38,7 @@ env__usb_dev_ports = (
>  env__block_devs = (
>      # eMMC; always present
>      {
> +        "fixture_id": "emmc",
>          "type": "mmc",
>          "id": "0",
>          # The following two properties are optional.
> @@ -48,6 +50,7 @@ env__block_devs = (
>      },
>      # SD card; present since I plugged one in
>      {
> +        "fixture_id": "sd",
>          "type": "mmc",
>          "id": "1"
>      },
> --
> 2.7.0
>

Regards,
Simon

[U-Boot] [PATCH] test/py: Provide custom IDs when parametrizing tests

[Stephen Warren]

On 01/26/2016 04:08 PM, Simon Glass wrote:
> Hi Stephen,
>
> On 26 January 2016 at 15:26, Stephen Warren <swarren@wwwdotorg.org> wrote:
>> From: Stephen Warren <swarren@nvidia.com>
>>
>> When pytest generates the name for parametrized tests, simple parameter
>> values (ints, strings) get used directly, but more complex values such
>> as dicts are not handled. This yields test names such as:
>>
>>      dfu[env__usb_dev_port0-env__dfu_config0]
>>      dfu[env__usb_dev_port0-env__dfu_config1]
>>
>> Add some code to extract a custom fixture ID from the fixture values, so
>> that we end up with meaningful names such as:
>>
>>      dfu[micro_b-emmc]
>>      dfu[devport2-ram]
>>
>> If the boardenv file doesn't define custom names, the code falls back to
>> the old algorithm.
>>
>> Signed-off-by: Stephen Warren <swarren@nvidia.com>
>> ---
>>   test/py/conftest.py       | 8 +++++++-
>>   test/py/tests/test_dfu.py | 3 +++
>>   test/py/tests/test_ums.py | 3 +++
>>   3 files changed, 13 insertions(+), 1 deletion(-)
>
> Reviewed-by: Simon Glass <sjg@chromium.org>
>
>> diff --git a/test/py/conftest.py b/test/py/conftest.py

>> @@ -225,7 +225,13 @@ def pytest_generate_tests(metafunc):
>>               # ... otherwise, see if there's a key that contains a list of
>>               # values to use instead.
>
> Is this function comment still accurate?

Yes; that comment is referring to the list of values that the relevant 
test function iterates over, not the names/IDs of the values or anything 
like that.

[U-Boot] [PATCH] test/py: Provide custom IDs when parametrizing tests

[Simon Glass]

On 26 January 2016 at 16:18, Stephen Warren <swarren@wwwdotorg.org> wrote:
> On 01/26/2016 04:08 PM, Simon Glass wrote:
>>
>> Hi Stephen,
>>
>> On 26 January 2016 at 15:26, Stephen Warren <swarren@wwwdotorg.org> wrote:
>>>
>>> From: Stephen Warren <swarren@nvidia.com>
>>>
>>> When pytest generates the name for parametrized tests, simple parameter
>>> values (ints, strings) get used directly, but more complex values such
>>> as dicts are not handled. This yields test names such as:
>>>
>>>      dfu[env__usb_dev_port0-env__dfu_config0]
>>>      dfu[env__usb_dev_port0-env__dfu_config1]
>>>
>>> Add some code to extract a custom fixture ID from the fixture values, so
>>> that we end up with meaningful names such as:
>>>
>>>      dfu[micro_b-emmc]
>>>      dfu[devport2-ram]
>>>
>>> If the boardenv file doesn't define custom names, the code falls back to
>>> the old algorithm.
>>>
>>> Signed-off-by: Stephen Warren <swarren@nvidia.com>
>>> ---
>>>   test/py/conftest.py       | 8 +++++++-
>>>   test/py/tests/test_dfu.py | 3 +++
>>>   test/py/tests/test_ums.py | 3 +++
>>>   3 files changed, 13 insertions(+), 1 deletion(-)
>>
>>
>> Reviewed-by: Simon Glass <sjg@chromium.org>
>>
>>> diff --git a/test/py/conftest.py b/test/py/conftest.py
>
>
>>> @@ -225,7 +225,13 @@ def pytest_generate_tests(metafunc):
>>>               # ... otherwise, see if there's a key that contains a list
>>> of
>>>               # values to use instead.
>>
>>
>> Is this function comment still accurate?
>
>
> Yes; that comment is referring to the list of values that the relevant test
> function iterates over, not the names/IDs of the values or anything like
> that.
>

Applied to u-boot-dm, thanks!