Re: [PATCH i-g-t] scripts: Add support to retest tests wich changed results

[Daniel Vetter <daniel@ffwll.ch>]

On Fri, Nov 27, 2015 at 02:53:36PM +0000, Thomas Wood wrote:
> On 27 November 2015 at 12:40, Gabriel Feceoru <gabriel.feceoru@intel.com> wrote:
> > This script takes multiple test results and prints the diffs, useful
> > for detecting noise in subsequent executions.
> > It gives the option to re-run these tests.
> 
> This feature might be useful in Piglit itself. It sounds similar to
> the ideas mentioned in this thread:
> 
> http://lists.freedesktop.org/archives/piglit/2015-November/018197.html

Yeah agreed, imo this should be implemented in piglit proper. We could
have different knobs to retest just regressions, fixes, or all changes
even.
-Daniel

> 
> 
> >
> > Signed-off-by: Gabriel Feceoru <gabriel.feceoru@intel.com>
> > ---
> >  scripts/retest-diff.sh | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 72 insertions(+)
> >  create mode 100755 scripts/retest-diff.sh
> >
> > diff --git a/scripts/retest-diff.sh b/scripts/retest-diff.sh
> > new file mode 100755
> > index 0000000..aee1d05
> > --- /dev/null
> > +++ b/scripts/retest-diff.sh
> > @@ -0,0 +1,72 @@
> > +#!/bin/bash
> > +#
> > +# Copyright © 2015 Intel Corporation
> > +#
> > +# Permission is hereby granted, free of charge, to any person obtaining a
> > +# copy of this software and associated documentation files (the "Software"),
> > +# to deal in the Software without restriction, including without limitation
> > +# the rights to use, copy, modify, merge, publish, distribute, sublicense,
> > +# and/or sell copies of the Software, and to permit persons to whom the
> > +# Software is furnished to do so, subject to the following conditions:
> > +#
> > +# The above copyright notice and this permission notice (including the next
> > +# paragraph) shall be included in all copies or substantial portions of the
> > +# Software.
> > +#
> > +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> > +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> > +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
> > +# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> > +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> > +# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
> > +# IN THE SOFTWARE.
> > +
> > +ROOT="`dirname $0`"
> > +ROOT="`readlink -f $ROOT/..`"
> > +IGT_TEST_ROOT="$ROOT/tests"
> > +RESULTS="$ROOT/results"
> > +PIGLIT="$ROOT/piglit/piglit"
> > +
> > +if [ ! -d "$IGT_TEST_ROOT" ]; then
> > +       echo "Error: could not find tests directory."
> > +       exit 1
> > +fi
> > +
> > +function print_help {
> > +       echo "Usage: $0 [<Results Paths> ...]"
> > +       echo ""
> > +}
> > +
> > +
> > +if [ $# -lt 2 ]; then
> > +       print_help
> > +       exit 1
> > +fi
> > +
> > +RESULT_FILES=$@
> > +
> > +for result in $RESULT_FILES; do
> > +    if [ ! -e $result ]; then
> > +        echo "Wrong result paths"
> > +        exit 1
> > +    fi
> > +done
> > +
> > +
> > +TESTS=`$PIGLIT summary console -d $RESULT_FILES | grep igt | cut -d':' -f1`
> > +if [[ -z $TESTS ]]; then
> > +    exit 1
> > +else
> > +    RED='\033[0;31m'
> > +    NC='\033[0m'
> > +    printf "$RED $TESTS $NC " | sed -e 's/ /\n/g'
> > +fi
> > +
> > +
> > +read -p "Dow you want to run these tests [y/N]? " -n 2 -r
> > +if [[ $REPLY =~ ^[Yy]$ ]]; then
> > +    mkdir -p $RESULTS
> > +    TEST_LIST=($TESTS)
> > +    TEST_PARAMS=`printf "%s "  "${TEST_LIST[@]/#/-t }"`
> > +    sudo IGT_TEST_ROOT=$IGT_TEST_ROOT $PIGLIT run igt $RESULTS $TEST_PARAMS
> > +fi
> > --
> > 1.9.1
> >
> > _______________________________________________
> > Intel-gfx mailing list
> > Intel-gfx@lists.freedesktop.org
> > http://lists.freedesktop.org/mailman/listinfo/intel-gfx
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/intel-gfx

-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/intel-gfx