Re: [Qemu-devel] [PATCH] docs: Add docs/devel/testing.rst

[Stefan Hajnoczi <stefanha@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 4783 bytes --]

On Mon, Jan 29, 2018 at 11:31:33AM +0800, Fam Zheng wrote:

Thanks for writing this!

I have only reviewed some parts in detail.

> +"Make check" testings
> +=====================
> +
> +The "make check" testing family includes most of the C based tests in QEMU. For
> +a quick help, run ``make check-help`` from the source tree.
> +
> +The usual way to run these tests is:
> +
> +.. code::
> +
> +  make check
> +
> +which includes QAPI schema tests, unit tests, and QTests. Different sub-types
> +of "make check" testings will be explained below.
> +
> +Before running tests, it is best to build QEMU programs first. Some tests
> +expect the executables to exist and will fail with obscure messages if cannot

s/if cannot/if they cannot/

> +find them.
> +
> +Unit tests
> +----------
> +
> +Unit tests, which can be invoked with ``make check-unit``, are simple C tests
> +that typically link to individual QEMU objects and exercise them by calling

I think this is clearer:
s/objects/object files/

Otherwise people might think "QEMU objects" is releated to QOM objects.

> +into the modules.

The term "modules" hasn't been defined.  I think the following is
clearer:

s/into the modules/exported functions/

> +
> +If you are writing new code in QEMU, consider adding a unit test, especially
> +for utility modules that are relatively stateless or have few dependencies. To
> +add a new unit test:
> +
> +1. Create a new source file. For example, ``tests/test-foo.c``.
> +
> +2. Write the test. Normally you would include the headers file which exports

s/headers/header/

> +   the module API, then verify the interface behaves as expected from your
> +   test. The test code should be organized with the glib testing framework.
> +   Copy and modify an existing test is usually a good idea.

s/Copy and modify/Copying and modifying/

> +
> +3. Add the test to ``tests/Makefile.include``. First, name the unit test
> +   program and add it to ``$(check-unit-y)``; then add a rule to build the
> +   executable. Optionally, you can add a magical variable to support ``gcov``.
> +   For example:
> +
> +.. code::
> +
> +  check-unit-y += tests/test-foo$(EXESUF)
> +  tests/test-foo$(EXESUF): tests/test-foo.o $(test-util-obj-y)
> +  ...
> +  gcov-files-test-foo-y = util/foo.c
> +
> +QTest
> +-----
> +
> +QTest is a testing framework that simplifies starting QEMU and interacting with

"simplifies starting QEMU" is not the most important aspect of qtest and
I would drop this completely.

The following single-sentence summary is a good opener: QTest is a
device emulation testing framework.

> +the virtual machine just like a guest kernel does. It can be very useful to
> +test hardware emulation, for example; it could also control certain aspects of
> +QEMU (such as virtual clock stepping), with a specially purposed "qtest"

s/a specially purposed/the special purpose/

> +protocol. Refer to the documentation in ``qtest.c`` file for more details of

s/in ``qtest.c`` file/in ``qtest.c``/

(If you want to keep "file" then a definite article is needed - "the".)

> +the protocol.
> +
> +QTest cases can be executed with
> +
> +.. code::
> +
> +   make check-qtest
> +
> +The QTest library is implemented by ``tests/libqtest.c`` and the API is defined
> +in ``tests/libqtest.h``.
> +
> +Consider adding a new QTest case when you are introducing a new virtual
> +hardware, or extending one if you are adding functionalities to an existing
> +virtual device.
> +
> +On top of libqtest, a higher level library, ``libqos``, was created to
> +encapsulate common tasks of device drivers, such as memory management and
> +communicating with system buses or devices. Many virtual device tests use

s/buses/busses/

> +libqos instead of directly calling into libqos.
> +
> +Steps to add a new QTest case are:
> +
> +1. Create a new source file for the test. (More than one file can be added as
> +   necessary.) For example, ``tests/test-foo-device.c``.  2. Write the test
> +   code with the glib and libqtest/libqos API. See also existing tests and the
> +   library headers for reference.
> +
> +3. Register the new test in ``tests/Makefile.include``. Add the test executable
> +   name to an appropriate ``check-qtest-*-y`` variable. For example:
> +
> +   ``check-qtest-generic-y = tests/test-foo-device$(EXESUF)``
> +
> +4. Add object dependencies of the executable in the Makefile, including the
> +   test source file(s) and other interesting objects. For example:
> +
> +   ``tests/test-foo-device$(EXESUF): tests/test-foo-device.o $(libqos-obj-y)``
> +
> +QAPI schema tests
> +-----------------
> +
> +The QAPI schema tests validate the QAPI parser used by QMP, by feeding
> +predefined input to the parser and compare the result with the reference

s/compare/comparing/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]