Re: [RFC i-g-t 0/4] Redundant test pruning

From: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>
To: Daniel Vetter <daniel@ffwll.ch>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>, Intel-gfx@lists.freedesktop.org
Subject: Re: [RFC i-g-t 0/4] Redundant test pruning
Date: Thu, 6 Jul 2017 12:31:24 +0100	[thread overview]
Message-ID: <cbf2283a-219f-fc56-dbb8-9a2c222a9d08@linux.intel.com> (raw)
In-Reply-To: <20170706092837.qmx2iemo6ytg44ks@phenom.ffwll.local>

On 06/07/2017 10:28, Daniel Vetter wrote:
> On Wed, Jul 05, 2017 at 02:30:43PM +0100, Tvrtko Ursulin wrote:
>>
>> On 27/06/2017 09:02, Tvrtko Ursulin wrote:
>>> On 26/06/2017 17:09, Daniel Vetter wrote:
>>>> On Fri, Jun 23, 2017 at 12:31:39PM +0100, Tvrtko Ursulin wrote:
>>>>> From: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>>>>>
>>>>> Small series which saves test execution time by removing the
>>>>> redundant tests.
>>>>>
>>>>> Tvrtko Ursulin (4):
>>>>>     igt: Remove default from the engine list
>>>>>     gem_exec_basic: Exercise the default engine selection
>>>>>     gem_sync: Add all and store_all subtests
>>>>>     extended.testlist: Remove some test-subtest combinations
>>>>
>>>> Ack on patches 1&2, but I'm not sold on patch 3. Atm gem_* takes a
>>>> ridiculous amount of machine time to run, you're adding more stuff. Are
>>>> those tests really drastially better at catching races if we run them 10x
>>>> longer? Is there no better way to exercise the races (lots more machines,
>>>> maybe slower ones, which is atm impossible since it just takes way, way
>>>> too long and we need an entire farm just for one machine).
>>>
>>> New gem_sync subtests were suggested by Chris after I send the first
>>> version of the series with the goal of getting the same coverage in
>>> faster time.
>>>
>>> If you look at patch 4, it removes 18 * 150s of gem_sync subtests, and
>>> adds 4 * 150s. So in total we are 35 minutes better of in the best case,
>>> a bit less on smaller machines.
>>>
>>> This is just for gem_sync, I forgot what did the saving for the series
>>> add up to. 1-2 hours maybe?
>>>
>>>> Also not sure how much curating extended.testlist is worth it, just make
>>>> the testcases faster :-) Like, roughly 100x faster overall for gem_*
>>>> ... >
>>>> But meanwhile ack on that one too.
>>>
>>> In which one, 3, or 4, or both?
>>
>> Ping on the series - do we want to try easy runtime reduction via this way
>> or should I drop it?
> 
> Go ahead. I'm still not happy with keeping tests around just because, but
> that's a larger topic.

Thanks, pushed.

Regards,

Tvrtko
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx