[U-Boot] [PATCH V2 1/7] test/py: Implement pytest infrastructure

[Michal Simek <monstr@monstr.eu>]

On 18.12.2015 19:33, Stephen Warren wrote:
> On 12/18/2015 07:50 AM, Michal Simek wrote:
>> Hi Stephen,
>>
>>>>               Finally, the example scripts support two boards; my
>>>>          home/laptop dev
>>>>               setup that uses a Numato relay board to control the
>>>> signals
>>>>          to the
>>>>               board I use there, and my work desktop dev setup that
>>>> uses our
>>>>               "PM342" debug board to controll the signals. The latter
>>>> works
>>>>               logically the same as the numato relay board, except it
>>>>          contains
>>>>               electronic switches driven by an FTDI chip.
>>>>
>>>>          I expect this is FTDI chip on the target right?
>>>>
>>>>
>>>>      It's actually a separate common debug board. Most/all of our
>>>>      development boards (and perhaps some production boards) have a
>>>>      standardized connector into which the common debug board plugs.
>>>>
>>>>
>>>>
>>>> ok.
>>>> I think my setup is not that far from what you are using and I expect
>>>> that others SoCs will be very similar.
>>>> Do you have any other testcases which you are running and you haven't
>>>> sent?
>>>
>>> Not at present.
>>>
>>> As an FYI, I typically publish my local work-in-progress branch at:
>>> git://github.com/swarren/u-boot.git tegra_dev
>>
>> I have looked at your patches and no problem to get it work on
>> microblaze and zynq board. I do use kermit without any problem.
>> I used cu on Microblaze.
> 
> Great!


btw: Is there any reason that you don't allow to clone your git repos?

> 
>> - What I do miss is power off functionality because it is not practical
>> to keep board always on. On can be solved via reset script.
> 
> Yes, I would expect that the flash or reset script would turn the board
> on. It should be easy to add an extra hook script at the end which turns
> the board off. Or, whatever automation system you use to invoke test.py
> could simply do that right after running test.py.
> 
>> - Then place tests to separate folder for better separation.
> 
> You mean e.g. test/py/tests/ ?

yes.


>> - Is there any way to handle timeouts or stucks? For example to
>> recognize if sleep 60 fails or just takes long. It means setting up
>> timeouts will be good.
> 
> ubspawn.py:expect() does have a timeout capability, and
> uboot_console_base.py:ensure_spawned() sets this to 30s by default.
> There isn't currently any example of modifying or saving/restoring the
> timeout, or running commands that are expected to have a timeout,
> although either should be pretty easy to add. I expect the result would
> look something like this in a test:
> 
> with uboot_console.push_timeout(60000 + some_margin):
>     uboot_console.run_command("sleep 60")
> # Perhaps the actual time taken should be validated here too
> 
> with uboot_console.timeout_is_expected(10000):
>     # code that is expected to time out
> # Perhaps the following command would be integrated into the
> # timeout_is_expected() implementation, since I think it's the only
> # way you could recover from this situation?
> uboot_console.ctrlc()
> 
> ... both modelled after the existing uboot_console.disable_check() code.

I think this should be the part of sleep testing.

> 
>> I will have more comments when I spend more time with it but it looks
>> pretty good for start.

Then I see incorrect timeout reporting with tftpboot

Loading:
*%08#################################################################
	 ######################
	 2.4 MiB/s

Regarding board-identity parameter. If not setup you are using "na" but
I think CONFIG_IDENT_STRING can be used instead.
Also I would like to have this parameter available in test because for
ethernet testing will be good to have several folders with golden images
for testing.

Also is there a way to run one particular test for easier developing. I
know that I can simply remove all testing files but better option will
be useful.

Thanks,
Michal

-- 
Michal Simek, Ing. (M.Eng), OpenPGP -> KeyID: FE3D1F91
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel - Microblaze cpu - http://www.monstr.eu/fdt/
Maintainer of Linux kernel - Xilinx Zynq ARM architecture
Microblaze U-BOOT custodian and responsible for u-boot arm zynq platform


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20160104/27c9d594/attachment.sig>