[U-Boot] [PATCH V2 1/7] test/py: Implement pytest infrastructure

[Stephen Warren <swarren@wwwdotorg.org>]

On 12/16/2015 10:43 AM, Michal Simek wrote:
> Hi Stephen,
>
> 2015-12-16 17:27 GMT+01:00 Stephen Warren <swarren@wwwdotorg.org
> <mailto:swarren@wwwdotorg.org>>:
>
>     On 12/16/2015 08:11 AM, Michal Simek wrote:
>
>         On 9.12.2015 17:32, Stephen Warren wrote:
>
>             On 12/02/2015 03:18 PM, Stephen Warren wrote:
>
>                 This tool aims to test U-Boot by executing U-Boot shell
>                 commands using
>                 the
>                 console interface. A single top-level script exists to
>                 execute or attach
>                 to the U-Boot console, run the entire script of tests
>                 against it, and
>                 summarize the results. Advantages of this approach are:
>
>                 - Testing is performed in the same way a user or script
>                 would interact
>                      with U-Boot; there can be no disconnect.
>                 - There is no need to write or embed test-related code
>                 into U-Boot
>                 itself.
>                      It is asserted that writing test-related code in
>                 Python is simpler and
>                      more flexible that writing it all in C.
>                 - It is reasonably simple to interact with U-Boot in
>                 this way.
>
>                 A few simple tests are provided as examples. Soon, we
>                 should convert as
>                 many as possible of the other tests in test/* and
>                 test/cmd_ut.c too.
>
>                 In the future, I hope to publish (out-of-tree) the hook
>                 scripts, relay
>                 control utilities, and udev rules I will use for my own
>                 HW setup.
>
>
>             I finally got permission to publish these. Examples are at:
>             https://github.com/swarren/uboot-test-hooks
>
>
>         Interesting. What's the normal setup which you have for the board?
>         I see from your description that you use numato usb relay - I
>         expect one
>         with more channels for reset.
>         Then you are able to control boot mode. Is it also via the same
>         relay?
>         How do you power up the board?
>
>
>     In my current setup I leave the board on all the time (or rather,
>     manually turn on the power when I'm about to run the tests).
>     Automating control of the power source is a step I'll take later.
>
>
> ok.
>
>
>     For Tegra, there are two important signals: reset and "force recovery".
>
>
>
> Do you mean that these both signals are just connected out of chip?

Yes. Reset is typically driven into the PMIC, and the signal to request 
force recovery is driven into Tegra itself.

Typically there are push-buttons on development boards to control those 
two signals. I've simply wired my relays across those buttons to 
simulate the button press.

>     Each of these has a separate relay, so the system currently uses 2
>     relays per target board. The numato relay board I own has 8 relays,
>     although there are a number of different models.
>
>
> ok
>
>
>     On Tegra, when reset is pulsed:
>
>     - If force-recovery is connected, the SoC enters USB recovery mode.
>     In this state, SW can be downloaded over USB into RAM and executed.
>
>
> Is this bootrom feature?

Yes.

> For xilinx boards there is all the time jtag available. It means
> download can be done via jtag instead.

That sounds plausible. The only issue might be general system state; can 
you reset everything to POR defaults via JTAG before the download? If 
not, perhaps e.g. the eMMC controller was partially initialized by 
previous code, which might interfere with assumptions made by the new 
code that's downloaded?

>     - If force-recovery is not connected, the SoC boots normally, from
>     SW stored in flash (eMMC, SPI, ...)
>
>
>     The example scripts always use recovery mode to download U-Boot into
>     RAM rather than writing it to flash first and then resetting. This
>     saves wear cycles on the flash, but does mean the download happens
>     in the "reset" rather than "flash" script, which may make the
>     examples a bit different than for some other SoCs.
>
>
> Are you testing all boot modes? Because I expect these needs to be
> tested too. Do you use SPL? If yes, are you going to test it in this way?

With those example scripts, cold boot isn't being tested. However, (a) I 
could define a new board ID (or pick up environment variables) to cause 
that to be tested sometimes (b) I don't recall having seen any 
differences between cold boot and recovery mode boot in the past; we get 
a lot of quicker/lower-wear test coverage this way without too much 
additional risk.

SPL is in use. However, SPL on Tegra has a bit of a different job than 
it has on some other chips. The boot ROM always initializes SDRAM, and 
SPL actually runs on a different CPU and primarily has the job of 
booting the main CPU where the main U-Boot binary runs. For more 
information, see:

ftp://download.nvidia.com/tegra-public-appnotes/index.html

>     Finally, the example scripts support two boards; my home/laptop dev
>     setup that uses a Numato relay board to control the signals to the
>     board I use there, and my work desktop dev setup that uses our
>     "PM342" debug board to controll the signals. The latter works
>     logically the same as the numato relay board, except it contains
>     electronic switches driven by an FTDI chip.
>
> I expect this is FTDI chip on the target right?

It's actually a separate common debug board. Most/all of our development 
boards (and perhaps some production boards) have a standardized 
connector into which the common debug board plugs.

>
>
> thanks,
> Michal
>
>
>
> --
> Michal Simek, Ing. (M.Eng), OpenPGP -> KeyID: FE3D1F91
> w: www.monstr.eu <http://www.monstr.eu> p: +42-0-721842854
> Maintainer of Linux kernel - Microblaze cpu - http://www.monstr.eu/fdt/
> Maintainer of Linux kernel - Xilinx Zynq ARM architecture
> Microblaze U-BOOT custodian and responsible for u-boot arm zynq platform