Re: [PATCH v2 2/2] cambridge: arrange to test each new baseline

[Ian Campbell <ian.campbell@citrix.com>]

On Mon, 2015-07-27 at 11:55 +0100, Ian Campbell wrote:
> Provide a new cr-daily-branch setting OSSTEST_BASELINES_ONLY which
> causes it to only attempt to test the current baseline (if it is
> untested)

There is a problem with this "if it is untested". This is implemented below
by setting skipidentical=true. However that is comparing ap-fetch-version
with ap-fetch-version-old which uses the main push gate results, i.e. it is
always going to see the same old and new revisions (they are both the
baseline) and skip it. Not sure how I missed this before!

I think I need to do something with check_tested to find a flight in the
database.

>  and never the tip version. Such tests will not result in any
> push.
> 
> Add a cronjob to Cambridge which runs in this manner, ensuring that
> there will usually be some sort of reasonably up to date baseline for
> any given branch which can be used for comparisons in adhoc testing or
> bisections.
> 
> This will also give us some data on the success of various branches on
> the set of machines in Cambridge, which can be useful/interesting.
> 
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
> ---
> v2: Wording tweak.
> ---
>  cr-daily-branch   | 13 ++++++++++++-
>  crontab-cambridge |  1 +
>  2 files changed, 13 insertions(+), 1 deletion(-)
> 
> diff --git a/cr-daily-branch b/cr-daily-branch
> index 7e3e69e..ed8f8c0 100755
> --- a/cr-daily-branch
> +++ b/cr-daily-branch
> @@ -85,7 +85,11 @@ check_tested () {
>  	  "$@"
>  }
>  
> -if [ "x$OSSTEST_NO_BASELINE" != xy ] ; then
> +if [ "x$OSSTEST_BASELINES_ONLY" = xy ] ; then
> +    force_baseline=baselines-only
> +    wantpush=false
> +    skipidentical=true
> +elif [ "x$OSSTEST_NO_BASELINE" != xy ] ; then
>  	testedflight=`check_tested --revision-$tree="$OLD_REVISION"`
>  
>  	if [ "x$testedflight" = x ]; then
> @@ -258,6 +262,13 @@ any, is the most recent actually tested revision.
>  
>  END
>      ;;
> +    baselines-only)
> +        #subject-prefix="[... ] "
> +        cat >> $heading <<END
> +This run is configured for baseline tests only.
> +
> +END
> +    ;;
>      '') # Not forcing a baseline, nothing to say.
>      ;;
>  esac
> diff --git a/crontab-cambridge b/crontab-cambridge
> index 2146311..64e4f27 100644
> --- a/crontab-cambridge
> +++ b/crontab-cambridge
> @@ -2,5 +2,6 @@ 
> PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
>  MAILTO=ian.jackson@citrix.com,ian.campbell@eu.citrix.com
>  # m            h       dom mon dow     command
>  4-59/30        *       * * *           cd testing.git && 
> BRANCHES=osstest              ./cr-for-branches branches -q "./cr-daily
> -branch --real"
> +11-59/30	*	* * *		cd testing.git && 
> OSSTEST_BASELINES_ONLY=y	./cr-for-branches branches -q "./cr
> -daily-branch --real"
>  22		8	* * *		cd testing.git && 
> BRANCHES=maintjobs		./cr-for-branches . -w ./cr-all-branch
> -statuses ''
>  3              4       * * *           savelog -c28 testing.git/tmp/cr
> -for-branches.log >/dev/null