From mboxrd@z Thu Jan 1 00:00:00 1970 From: Damien Lespiau Subject: Re: [igt] Making the test-suite easier to run Date: Fri, 15 Nov 2013 17:41:28 +0000 Message-ID: <20131115174128.GK14757@strange.amr.corp.intel.com> References: <1384533220-20780-1-git-send-email-damien.lespiau@intel.com> <20131115172318.GY22741@phenom.ffwll.local> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by gabe.freedesktop.org (Postfix) with ESMTP id 945E91059F2 for ; Fri, 15 Nov 2013 09:41:30 -0800 (PST) Content-Disposition: inline In-Reply-To: <20131115172318.GY22741@phenom.ffwll.local> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: intel-gfx-bounces@lists.freedesktop.org Errors-To: intel-gfx-bounces@lists.freedesktop.org To: Daniel Vetter Cc: intel-gfx@lists.freedesktop.org List-Id: intel-gfx@lists.freedesktop.org On Fri, Nov 15, 2013 at 06:23:18PM +0100, Daniel Vetter wrote: > - I think we need to make it very clear that piglit should still be > developed upstream. So local changes shouldn't be allowed at all imo. Sure, I'll wait until my patches are upstreamed to resend the series without the local patches. > Afaics that would only affect tests/igt.tests - can't we fix that by > replicating the symlink, too? I was thinking to completely remove igt.tests from upstream piglit. There isn't anything special about the location of igt.tests, you just need to give a (absolute) path to it when launching piglit-run.py and not being in the piglit-run.py directory. As igt.test is really coupled with igt, it makes sense to carry it in our tree, IMO. > - The makefile target looks like a script. I think it'd be better to > extract it as a real script. Can do. > - I'm not too terribly sold on the convenience script. Imo it shouldn't be > more than executable documentation, since I'm a bit afraid that we'll > add neat features (like fancy resume with auto-blocking of crashing > tests) to it that would better be done in upstream piglit to benefit > everyone. Point taken. We can judge when adding a feature if it'd be better to do it upstream of if it's just a convenience wrapper for us. But what I really want is the convenience and well defined runs that everyone can replicate (a lof of details to remember, concurrent Vs not-concurrent, how to generate the HTML report, ...). And I'd like to add more: eg. give the 10 tests that take the longest time in the run so people can look at them and try to make them faster, ... Also I really want quicker subsets of the test-suite, people don't run the full test-suite everytime they work on a series and this just step 1/ towards that goal. Running a subset is better than running nothing at all :) -- Damien