[U-Boot] [PATCH V3 1/7] test/py: Implement pytest infrastructure

Simon Glass
2016-01-11T15:23:08Z

On 5 January 2016 at 15:58, Stephen Warren <swarren@wwwdotorg.org> wrote:
> This tool aims to test U-Boot by executing U-Boot shell commands using the
> console interface. A single top-level script exists to execute or attach
> to the U-Boot console, run the entire script of tests against it, and
> summarize the results. Advantages of this approach are:
>
> - Testing is performed in the same way a user or script would interact
> with U-Boot; there can be no disconnect.
> - There is no need to write or embed test-related code into U-Boot itself.
> It is asserted that writing test-related code in Python is simpler and
> more flexible that writing it all in C.
> - It is reasonably simple to interact with U-Boot in this way.
>
> A few simple tests are provided as examples. Soon, we should convert as
> many as possible of the other tests in test/* and test/cmd_ut.c too.
>
> The hook scripts, relay control utilities, and udev rules I use for my
> own HW setup are published at https://github.com/swarren/uboot-test-hooks.
>
> See README.md for more details!
>
> Signed-off-by: Stephen Warren <swarren@wwwdotorg.org>
> Signed-off-by: Stephen Warren <swarren@nvidia.com>
> Tested-by: Michal Simek <michal.simek@xilinx.com>
> Tested-by: Simon Glass <sjg@chromium.org>
> ---
> v3:
> - Rework HTML log generation so that TAB characters render as expected.
> Suggested by Michal Simek.
> - Move test scripts into a sub-directory.
> Suggested by Michal Simek.
> - s/uboot/u[-_]boot/g. Suggested by Simon Glass.
> - s/"/'/g. Suggested by Simon Glass.
> - Typo fixes.
> - Add more documentation. Suggested by Simon Glass.
> - Make "notes" in the log file be so that their formatting is
> preserved. This is useful for large notes such as exception dumps.
>
> v2:
> - Many fixes and tweaks have been squashed in. Separated out some of
> the tests into separate commits, and added some more tests.
> ---
> test/py/.gitignore | 1 +
> test/py/README.md | 300 ++++++++++++++++++++++++++++++
> test/py/conftest.py | 335 ++++++++++++++++++++++++++++++++++
> test/py/multiplexed_log.css | 88 +++++++++
> test/py/multiplexed_log.py | 335 ++++++++++++++++++++++++++++++++++
> test/py/pytest.ini | 11 ++
> test/py/test.py | 32 ++++
> test/py/tests/test_000_version.py | 20 ++
> test/py/tests/test_help.py | 9 +
> test/py/tests/test_unknown_cmd.py | 14 ++
> test/py/u_boot_console_base.py | 260 ++++++++++++++++++++++++++
> test/py/u_boot_console_exec_attach.py | 51 ++++++
> test/py/u_boot_console_sandbox.py | 48 +++++
> test/py/u_boot_spawn.py | 123 +++++++++++++
> 14 files changed, 1627 insertions(+)
> create mode 100644 test/py/.gitignore
> create mode 100644 test/py/README.md
> create mode 100644 test/py/conftest.py
> create mode 100644 test/py/multiplexed_log.css
> create mode 100644 test/py/multiplexed_log.py
> create mode 100644 test/py/pytest.ini
> create mode 100755 test/py/test.py
> create mode 100644 test/py/tests/test_000_version.py
> create mode 100644 test/py/tests/test_help.py
> create mode 100644 test/py/tests/test_unknown_cmd.py
> create mode 100644 test/py/u_boot_console_base.py
> create mode 100644 test/py/u_boot_console_exec_attach.py
> create mode 100644 test/py/u_boot_console_sandbox.py
> create mode 100644 test/py/u_boot_spawn.py

Looks good!

Acked-by: Simon Glass <sjg@chromium.org>

One comment about function / class comments. If you like at
buildman/patman they use this sort of format:

def RunCommit(self, commit_upto, brd, work_dir, do_config, force_build,
force_build_failures):
"""Build a particular commit.

If the build is already done, and we are not forcing a build, we skip
the build and just return the previously-saved results.

Args:
commit_upto: Commit number to build (0...n-1)
brd: Board object to build
work_dir: Directory to which the source will be checked out
do_config: True to run a make <board>_defconfig on the source
force_build: Force a build even if one was previously done
force_build_failures: Force a bulid if the previous result showed
failure

Returns:
tuple containing:
- CommandResult object containing the results of the build
- boolean indicating whether 'make config' is still needed
"""

So double quotes, and the first line describes the function in a very
simple way, a bit like a git commit subject. The arguments and return
value are specifically called out, as we do in the C code.

For consistency I'd suggest doing the same.

Regards,
Simon