Re: [PATCH i-g-t 3/5] tests: Use IGT_SIMULATION to tune the list of tests to run

[Daniel Vetter <daniel@ffwll.ch>]

On Tue, Mar 26, 2013 at 03:24:37PM +0000, Damien Lespiau wrote:
> There are two ways to run tests, directly with make check/test
> or through piglit.
> 
> When IGT_SIMULATION is set to '1', we substitute the list of tests in
> those two code paths with carefully selected tests. The stress tests and
> other horrors are left to torture the real hardware as they don't make
> too much sense in simulation.
> 
> Signed-off-by: Damien Lespiau <damien.lespiau@intel.com>

This feels a bit fragile since even just now I often fail to put a test
newly converted to the subtest stuff into the right single/multi make
target list. Also, this way we exclude a test by default, which also feels
like the wrong way round (most of the tests we're adding aren't heavy
stress-tests, but more excercise some corner-case).

So what about a sprinkling

drmtest_skip_on_simulation();

over the remaining testcases instead, which just calls exit(77);? Only
caveat is that for subtests we need to put it into the right spot to not
break subtest enumeration in piglit (it's a bit fragile).

Cheers, Daniel

> ---
>  tests/Makefile.am | 43 ++++++++++++++++++++++++++++++++++++++++---
>  1 file changed, 40 insertions(+), 3 deletions(-)
> 
> diff --git a/tests/Makefile.am b/tests/Makefile.am
> index 2fddfe8..33db962 100644
> --- a/tests/Makefile.am
> +++ b/tests/Makefile.am
> @@ -137,22 +137,59 @@ kernel_tests = \
>  	$(multi_kernel_tests) \
>  	$(NULL)
>  
> +# We maintain here a list of tests to run in our simulation environments
> +single_simulation_tests = \
> +	gem_mmap \
> +	gem_pread_after_blit \
> +	gem_ring_sync_loop \
> +	gem_ctx_basic \
> +	gem_pipe_control_store_loop \
> +	gem_storedw_loop_render \
> +	gem_storedw_loop_blt \
> +	gem_storedw_loop_bsd \
> +	gem_render_linear_blits \
> +	gem_tiled_blits \
> +	gem_cpu_reloc \
> +	$(NULL)
> +
> +multi_simulation_tests = \
> +	gem_exec_nop \
> +	gem_mmap_gtt \
> +	$(NULL)
> +
> +simulation_tests = \
> +	$(single_simulation_tests) \
> +	$(multi_simulation_tests) \
> +	$(NULL)
> +
>  TESTS = \
>  	$(NULL)
>  
>  test:
>  	@whoami | grep root || ( echo ERROR: not running as root; exit 1 )
>  	@./check_drm_clients
> -	@make TESTS="${kernel_tests}" check
> +	@if [ "${IGT_SIMULATION}" == 1 ]; then \
> +		make TESTS="${simulation_tests}" check; \
> +	else \
> +		make TESTS="${kernel_tests}" check; \
> +	fi
>  
>  list-single-tests:
>  	@echo TESTLIST
> -	@echo ${single_kernel_tests}
> +	@if [ "${IGT_SIMULATION}" == 1 ]; then \
> +		echo ${single_simulation_tests}; \
> +	else \
> +		echo ${single_kernel_tests}; \
> +	fi
>  	@echo END TESTLIST
>  
>  list-multi-tests:
>  	@echo TESTLIST
> -	@echo ${multi_kernel_tests}
> +	@if [ "${IGT_SIMULATION}" == 1 ]; then \
> +		echo ${multi_simulation_tests}; \
> +	else \
> +		echo ${multi_kernel_tests}; \
> +	fi
>  	@echo END TESTLIST
>  
>  HANG = \
> -- 
> 1.7.11.7
> 
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/intel-gfx

-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch