Re: [PATCH 19/23] piglit: Update the README file with the new way of running tests

[Daniel Vetter <daniel@ffwll.ch>]

On Fri, Nov 15, 2013 at 04:33:36PM +0000, Damien Lespiau wrote:
> Signed-off-by: Damien Lespiau <damien.lespiau@intel.com>

Hm, I'd keep the references to the real piglit somewhere, since our copy
here is just for convenience. Development should happen upstream.
-Daniel

> ---
>  README | 54 ++++++++++++++++++++++++++++++++----------------------
>  1 file changed, 32 insertions(+), 22 deletions(-)
> 
> diff --git a/README b/README
> index 246e24c..021888f 100644
> --- a/README
> +++ b/README
> @@ -24,38 +24,48 @@ tests/
>  	changes.  Hopefully this can cover the relevant cases we need to
>  	worry about, including backwards compatibility.
>  
> -	Note: The old automake based testrunner had to be scraped due to
> -	upstream changes which broke dynamic creation of the test list. Of
> -	course it is still possible to directly run tests, even when not always
> -	limiting tests to specific subtests (like piglit does).
> +	After having compiled the tests, one can run the test-suite with:
>  
> -	The more comfortable way to run tests is with piglit. First grab piglit
> -	from:
> +	$ sudo make run-tests
>  
> -	git://anongit.freedesktop.org/piglit
> +	As we have display tests, we need to be DRM master. As a result the
> +	test suite can only be run if no other DRM client is active.
> +	Similarly, some tests access debugfs, so we need to be root.
>  
> -	and build it (no need to install anything). Then we need to link up the
> -	i-g-t sources with piglit
> +	"make run-tests" create a $date-piglit-results.$n directory with the
> +	results of the run. More specifically:
> +	  - $date-piglit-results.$n/main JSON file with the test results
> +	  - $date-piglit-results.$n/html/index.html HTML summary of the run
>  
> -	piglit-sources $ cd bin
> -	piglit-sources/bin $ ln $i-g-t-sources igt -s
> +	Where $date is the date formated with `date +%Y%m%d` and $n the nth run
> +	of the day.
>  
> -	The tests in the i-g-t sources need to have been built already. Then we
> -	can run the testcases with (as usual as root, no other drm clients
> -	running):
> +	PIGLIT_FLAGS can be used to give options to the underlying piglit
> +	runner. For instance, to exclude test matching '^kms_':
>  
> -	piglit-sources # ./piglit-run.py tests/igt.tests <results-file>
> +	$ sudo make run-tests PIGLIT_FLAGS="-x ^kms_"
>  
> -	The testlist is built at runtime, so no need to update anything in
> -	piglit when adding new tests. See
> +	For the list of piglit options, run:
>  
> -	piglit-sources $ ./piglit-run.py -h
> +	$ ./piglit/piglit-run.py -h
>  
> -	for some useful options.
> +	Another useful feature is to be able to resume an interrupted run. To
> +	do that, make run-tests needs to know which run we are talking about:
>  
> -	Piglit only runs a default set of tests and is useful for regression
> -	testing. Other tests not run are:
> -	- tests that might hang the gpu, see HANG in Makefile.am
> +	$ sudo make run-tests RESUME=$date-piglit-results.$n
> +
> +	or, more succinctly:
> +
> +	$ sudo make run-tests R=$date-piglit-results.$n
> +
> +	It's possible to combine PIGLIT_FLAGS and RESUME. This is useful to
> +	resume runs where a specific test deterministically hang the machine:
> +
> +	$ sudo make run-tests PIGLIT_FLAGS="-x drv_module_reload" R=$date-piglit-results.$n
> +
> +	"make run-tests" only runs a default set of tests and is useful for
> +	regression testing. Other tests not run are:
> +	- tests that might hang the gpu, see HANG in tests/Makefile.sources
>  	- gem_stress, a stress test suite. Look at the source for all the
>  	  various options.
>  	- testdisplay is only run in the default mode. testdisplay has tons of
> -- 
> 1.8.3.1
> 
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/intel-gfx

-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch