From: Simon Glass <sjg@chromium.org>
To: u-boot@lists.denx.de
Subject: [PATCH v3 40/41] test: sandbox: Move sandbox test docs into doc/develop
Date: Wed, 3 Feb 2021 05:44:46 -0700 [thread overview]
Message-ID: <20210203124447.2458527-39-sjg@chromium.org> (raw)
In-Reply-To: <20210203124447.2458527-1-sjg@chromium.org>
At present some of the documentation about running sandbox tests is in the
sandbox docs. It makes more sense to put it in with the other testing
docs, with a link there from sandbox. Update the documentation
accordingly.
Also add a paragraph explaining why sandbox exists and the test philosophy
that it uses.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
(no changes since v1)
doc/arch/sandbox.rst | 44 +++++++----------------------------
doc/develop/tests_sandbox.rst | 44 +++++++++++++++++++++++++++++++++--
2 files changed, 51 insertions(+), 37 deletions(-)
diff --git a/doc/arch/sandbox.rst b/doc/arch/sandbox.rst
index 1ccb5344ac6..1638b050002 100644
--- a/doc/arch/sandbox.rst
+++ b/doc/arch/sandbox.rst
@@ -17,6 +17,12 @@ of the sandbox U-Boot. The purpose of running U-Boot under Linux is to test
all the generic code, not specific to any one architecture. The idea is to
create unit tests which we can run to test this upper level code.
+Sandbox allows development of many types of new features in a traditional way,
+rather than needing to test each iteration on real hardware. Many U-Boot
+features were developed on sandbox, including the core driver model, most
+uclasses, verified boot, bloblist, logging and dozens of others. Sandbox has
+enabled many large-scale code refactors as well.
+
CONFIG_SANDBOX is defined when building a native board.
The board name is 'sandbox' but the vendor name is unset, so there is a
@@ -486,42 +492,10 @@ Testing
-------
U-Boot sandbox can be used to run various tests, mostly in the test/
-directory. These include:
-
-command_ut:
- Unit tests for command parsing and handling
-compression:
- Unit tests for U-Boot's compression algorithms, useful for
- security checking. It supports gzip, bzip2, lzma and lzo.
-driver model:
- Run this pytest::
-
- ./test/py/test.py --bd sandbox --build -k ut_dm -v
-
-image:
- Unit tests for images:
- test/image/test-imagetools.sh - multi-file images
- test/image/test-fit.py - FIT images
-tracing:
- test/trace/test-trace.sh tests the tracing system (see README.trace)
-verified boot:
- See test/vboot/vboot_test.sh for this
-
-If you change or enhance any of the above subsystems, you shold write or
-expand a test and include it with your patch series submission. Test
-coverage in U-Boot is limited, as we need to work to improve it.
-
-Note that many of these tests are implemented as commands which you can
-run natively on your board if desired (and enabled).
-
-To run all tests use "make check".
-
-To run a single test in an existing sandbox build, you can use -T to use the
-test device tree, and -c to select the test:
-
- /tmp/b/sandbox/u-boot -T -c "ut dm pci_busdev"
+directory.
-This runs dm_test_pci_busdev() which is in test/dm/pci.c
+See :doc:`../develop/tests_sandbox` for more information and
+:doc:`../develop/testing` for information about testing generally.
Memory Map
diff --git a/doc/develop/tests_sandbox.rst b/doc/develop/tests_sandbox.rst
index f05de2f2b7a..9ab7e0237c6 100644
--- a/doc/develop/tests_sandbox.rst
+++ b/doc/develop/tests_sandbox.rst
@@ -1,7 +1,47 @@
.. SPDX-License-Identifier: GPL-2.0+
-Tests Under the Hood
-====================
+Sandbox tests
+=============
+
+Test Design
+-----------
+
+Most uclasses and many functions of U-Boot have sandbox tests. This allows much
+of the code to be checked in an developer-friendly environment.
+
+Sandbox provides a way to write and run unit tests. The traditional approach to
+unit tests is to build lots of little executables, one for each test or
+category of tests. With sandbox, so far as possible, all the tests share a
+small number of executables (e.g. 'u-boot' for sandbox, 'u-boot-spl' and
+'u-boot' for sandbox_spl) and can be run very quickly. The vast majority of
+tests can run on the 'sandbox' build,
+
+Available tests
+---------------
+
+Some of the available tests are:
+
+ - command_ut: Unit tests for command parsing and handling
+ - compression: Unit tests for U-Boot's compression algorithms, useful for
+ security checking. It supports gzip, bzip2, lzma and lzo.
+ - image: Unit tests for images:
+
+ - test/image/test-imagetools.sh - multi-file images
+ - test/py/tests/test-fit.py - FIT images
+ - tracing: test/trace/test-trace.sh tests the tracing system (see
+ README.trace)
+ - verified boot: test/py/tests/test_vboot.py
+
+If you change or enhance any U-Boot subsystem, you should write or expand a
+test and include it with your patch series submission. Test coverage in some
+older areas of U-Boot is still somewhat limited and we need to work to improve
+it.
+
+Note that many of these tests are implemented as commands which you can
+run natively on your board if desired (and enabled).
+
+To run all tests, use 'make check'.
+
Running sandbox tests directly
------------------------------
--
2.30.0.365.g02bc693789-goog
next prev parent reply other threads:[~2021-02-03 12:44 UTC|newest]
Thread overview: 47+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-02-03 12:44 [PATCH v3 00/41] test: Refactor tests to have a single test runner Simon Glass
2021-02-03 12:44 ` [PATCH v3 01/41] doc: Tidy up testing section Simon Glass
2021-02-03 12:44 ` [PATCH v3 02/41] doc: Document make tcheck Simon Glass
2021-02-03 12:44 ` [PATCH v3 03/41] sandbox: Drop the 'starting...' message unless testing Simon Glass
2021-02-03 12:44 ` [PATCH v3 04/41] test: Re-enable test_ofplatdata Simon Glass
2021-02-03 12:44 ` [PATCH v3 05/41] doc: Explain how to run tests without pytest Simon Glass
2021-02-03 12:44 ` [PATCH v3 06/41] doc: Document how sandbox_spl_tests are run Simon Glass
2021-02-03 14:40 ` Pratyush Yadav
2021-02-03 12:44 ` [PATCH v3 07/41] test: Correct setexpr test prefix Simon Glass
2021-02-03 12:44 ` [PATCH v3 08/41] test: Mark all driver model tests with a flag Simon Glass
2021-02-03 12:44 ` [PATCH v3 09/41] test: Rename test-main.c to test-dm.c Simon Glass
2021-02-03 12:44 ` [PATCH v3 10/41] test: Add an overall test runner Simon Glass
2021-02-03 12:44 ` [PATCH v3 11/41] test: Create pre/post-run functions Simon Glass
2021-02-03 12:44 ` [PATCH v3 12/41] test: Call test_pre/post_run() from driver model tests Simon Glass
2021-02-03 12:44 ` [PATCH v3 13/41] test: Move dm_extended_scan() to test_pre_run() Simon Glass
2021-02-03 12:44 ` [PATCH v3 14/41] test: Move do_autoprobe() " Simon Glass
2021-02-03 12:44 ` [PATCH v3 15/41] test: Move dm_scan_plat() " Simon Glass
2021-02-03 12:44 ` [PATCH v3 16/41] test: Drop mallinfo() work-around Simon Glass
2021-02-03 12:44 ` [PATCH v3 17/41] test: Move console silencing to test_pre_run() Simon Glass
2021-02-03 12:44 ` [PATCH v3 18/41] test: Move delay skipping " Simon Glass
2021-02-03 12:44 ` [PATCH v3 19/41] test: Handle driver model reinit in test_pre_run() Simon Glass
2021-02-03 12:44 ` [PATCH v3 20/41] test: Drop struct dm_test_state Simon Glass
2021-02-03 12:44 ` [PATCH v3 21/41] test: Move dm_test_init() into test-main.c Simon Glass
2021-02-03 12:44 ` [PATCH v3 22/41] test: Move dm_test_destroy() " Simon Glass
2021-02-03 12:44 ` [PATCH v3 23/41] test: Move test running into a separate function Simon Glass
2021-02-03 12:44 ` [PATCH v3 24/41] test: Use ut_run_test() to run driver model tests Simon Glass
2021-02-03 12:44 ` [PATCH v3 25/41] test: Drop dm_do_test() Simon Glass
2021-02-03 12:44 ` [PATCH v3 26/41] test: Add ut_run_test_live_flat() to run tests twice Simon Glass
2021-02-03 12:44 ` [PATCH v3 27/41] test: Use a local variable for test state Simon Glass
2021-02-03 12:44 ` [PATCH v3 28/41] test: Run driver-model tests using ut_run_list() Simon Glass
2021-02-03 12:44 ` [PATCH v3 29/41] test: Use return values in dm_test_run() Simon Glass
2021-02-03 12:44 ` [PATCH v3 30/41] test: Move the devicetree check into ut_run_list() Simon Glass
2021-02-03 12:44 ` [PATCH v3 31/41] test: Move restoring of driver model state to ut_run_list() Simon Glass
2021-02-03 12:44 ` [PATCH v3 32/41] test: log: Rename log main test file to log_ut.c Simon Glass
2021-02-03 12:44 ` [PATCH v3 33/41] test: Add a macros for finding tests in linker_lists Simon Glass
2021-02-03 12:44 ` [PATCH v3 34/41] test: Rename all linker lists to have a ut_ prefix Simon Glass
2021-02-03 12:44 ` [PATCH v3 35/41] test: Allow SPL to run any available test Simon Glass
2021-02-03 12:44 ` [PATCH v3 36/41] sandbox: Update os_find_u_boot() to find the .img file Simon Glass
2021-02-03 12:44 ` [PATCH v3 37/41] spl: Convert spl_fit to work with sandbox Simon Glass
2021-02-03 12:44 ` [PATCH v3 38/41] doc: Move coccinelle into its own section Simon Glass
2021-02-03 12:44 ` [PATCH v3 39/41] spl: test: Add a test for spl_load_simple_fit() Simon Glass
2021-02-03 12:44 ` Simon Glass [this message]
2021-02-03 12:44 ` [PATCH v3 41/41] doc: Explain briefly how to write new tests Simon Glass
2021-03-03 19:37 ` [PATCH v3 00/41] test: Refactor tests to have a single test runner Tom Rini
2021-03-03 20:18 ` Tom Rini
2021-03-03 20:37 ` Tom Rini
2021-03-04 3:06 ` Simon Glass
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210203124447.2458527-39-sjg@chromium.org \
--to=sjg@chromium.org \
--cc=u-boot@lists.denx.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.