[U-Boot] [PATCH V2 1/7] test/py: Implement pytest infrastructure

[Stephen Warren <swarren@wwwdotorg.org>]

On 01/04/2016 05:41 AM, Michal Simek wrote:
> On 18.12.2015 19:33, Stephen Warren wrote:
>> On 12/18/2015 07:50 AM, Michal Simek wrote:
>>> Hi Stephen,
>>>
>>>>>                Finally, the example scripts support two boards; my
>>>>>           home/laptop dev
>>>>>                setup that uses a Numato relay board to control the
>>>>> signals
>>>>>           to the
>>>>>                board I use there, and my work desktop dev setup that
>>>>> uses our
>>>>>                "PM342" debug board to controll the signals. The latter
>>>>> works
>>>>>                logically the same as the numato relay board, except it
>>>>>           contains
>>>>>                electronic switches driven by an FTDI chip.
>>>>>
>>>>>           I expect this is FTDI chip on the target right?
>>>>>
>>>>>
>>>>>       It's actually a separate common debug board. Most/all of our
>>>>>       development boards (and perhaps some production boards) have a
>>>>>       standardized connector into which the common debug board plugs.
>>>>>
>>>>>
>>>>>
>>>>> ok.
>>>>> I think my setup is not that far from what you are using and I expect
>>>>> that others SoCs will be very similar.
>>>>> Do you have any other testcases which you are running and you haven't
>>>>> sent?
>>>>
>>>> Not at present.
>>>>
>>>> As an FYI, I typically publish my local work-in-progress branch at:
>>>> git://github.com/swarren/u-boot.git tegra_dev
>>>
>>> I have looked at your patches and no problem to get it work on
>>> microblaze and zynq board. I do use kermit without any problem.
>>> I used cu on Microblaze.
>>
>> Great!
>
> btw: Is there any reason that you don't allow to clone your git repos?

Hmm. git protocol doesn't seem to work on github any more; try cloning 
over SSH if you have a github ID (git at github.com:swarren/u-boot.git) or 
HTTPS otherwise (https://github.com/swarren/u-boot.git).

...
>>> I will have more comments when I spend more time with it but it looks
>>> pretty good for start.
>
> Then I see incorrect timeout reporting with tftpboot
>
> Loading:
> *%08#################################################################
> 	 ######################
> 	 2.4 MiB/s

This looks like another case where an individual test needs to adjust 
the timeout used to wait for the prompt.

> Regarding board-identity parameter. If not setup you are using "na" but
> I think CONFIG_IDENT_STRING can be used instead.

I believe those two things are different. The test system's concept of 
board identity refers to the physical instance of the board (e.g. if you 
have 3 identical boards in order to test N branches/commits in parallel) 
whereas CONFIG_IDENT_STRING is something built into the U-Boot binary to 
identify the type (not instance) of board if I understand correctly.

> Also I would like to have this parameter available in test because for
> ethernet testing will be good to have several folders with golden images
> for testing.

I believe you can access uboot_console.config.board_identity. However, 
that would make the tests depend on your particular environment, so it's 
probably not a good idea to use that parameter at all.

> Also is there a way to run one particular test for easier developing. I
> know that I can simply remove all testing files but better option will
> be useful.

If you pass "-k testname" to the script, it'll only run test(s) that 
match that string. That's a standard pytest option.