Re: [Qemu-trivial] [PATCH v3 2/3] .travis.yml: run make check for all matrix targets

["Alex Bennée" <alex.bennee@linaro.org>]

David Gibson <david@gibson.dropbear.id.au> writes:

> On Thu, Jan 28, 2016 at 02:23:28PM +0000, Alex Bennée wrote:
>> We only ran make check once before it used to be an unreliable target.
>> It was only a stop gap measure and we should be able to revert it now.
>> This also stops us needing a large all-MMU build.
>>
>> We disable "make check" for a couple of the extra config targets which
>> are currently broken.
>>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>
> So, in general I like the idea of running make check more widely.
>
> However.. I was wondering - what's the rationale for having separate
> matrix builds for each target (or small group) rather than just doing
> one build with all the targets?

Each individual part of the matrix can be run in parallel with the
others so it makes sense to keep the build component small (as each
softmmu target rebuilds a significant chunk of the build).

Having said that there is a fair amount of repetition as we are
repeating all the generic qtests each time just so we can run the extra
${TARGET}-qtest binaries.

Travis does has an option for using ccache so it might be worth
experimenting with that to see if things are improved.

> I can't see any obvious benefit to splitting the build that way, but
> it does increase the total build time significantly - and will do so
> rather more so with make check added.

Elapsed and total are the ones to look at:

https://travis-ci.org/stsquad/qemu/builds/105401126

vs

https://travis-ci.org/qemu/qemu/builds/105711606

However it looks like Travis are having scaling growing pains because
there "old style" VM approach is running a lot faster than it used to.

>
>> ---
>>  .travis.yml | 15 ++++++++-------
>>  1 file changed, 8 insertions(+), 7 deletions(-)
>>
>> diff --git a/.travis.yml b/.travis.yml
>> index 4a0c23a..16be23f 100644
>> --- a/.travis.yml
>> +++ b/.travis.yml
>> @@ -40,7 +40,7 @@ notifications:
>>      on_failure: always
>>  env:
>>    global:
>> -    - TEST_CMD=""
>> +    - TEST_CMD="make check"
>>      - EXTRA_CONFIG=""
>>    matrix:
>>      # Group major targets together with their linux-user counterparts
>> @@ -73,17 +73,14 @@ script:
>>  matrix:
>>    # We manually include a number of additional build for non-standard bits
>>    include:
>> -    # Make check target (we only do this once)
>> -    - env:
>> -        - TARGETS=alpha-softmmu,arm-softmmu,aarch64-softmmu,cris-softmmu,i386-softmmu,x86_64-softmmu,m68k-softmmu,microblaze-softmmu,microblazeel-softmmu,mips-softmmu,mips64-softmmu,mips64el-softmmu,mipsel-softmmu,or32-softmmu,ppc-softmmu,ppc64-softmmu,ppcemb-softmmu,s390x-softmmu,sh4-softmmu,sh4eb-softmmu,sparc-softmmu,sparc64-softmmu,unicore32-softmmu,lm32-softmmu,moxie-softmmu,tricore-softmmu,xtensa-softmmu,xtensaeb-softmmu
>> -          TEST_CMD="make check"
>> -      compiler: gcc
>>      # Debug related options
>>      - env: TARGETS=i386-softmmu,x86_64-softmmu
>>             EXTRA_CONFIG="--enable-debug"
>>        compiler: gcc
>> +    # We currently disable "make check"
>>      - env: TARGETS=i386-softmmu,x86_64-softmmu
>>             EXTRA_CONFIG="--enable-debug --enable-tcg-interpreter"
>> +           TEST_CMD=""
>>        compiler: gcc
>>      # Disable a few of the optional features
>>      - env: TARGETS=i386-softmmu,x86_64-softmmu
>> @@ -104,11 +101,15 @@ matrix:
>>      - env: TARGETS=i386-softmmu,x86_64-softmmu
>>             EXTRA_CONFIG="--enable-trace-backends=simple"
>>        compiler: gcc
>> +    # We currently disable "make check"
>>      - env: TARGETS=i386-softmmu,x86_64-softmmu
>>             EXTRA_CONFIG="--enable-trace-backends=ftrace"
>> +           TEST_CMD=""
>>        compiler: gcc
>> +    # We currently disable "make check"
>>      - env: TARGETS=i386-softmmu,x86_64-softmmu
>> -          EXTRA_CONFIG="--enable-trace-backends=ust"
>> +           EXTRA_CONFIG="--enable-trace-backends=ust"
>> +           TEST_CMD=""
>>        compiler: gcc
>>      - env: TARGETS=i386-softmmu,x86_64-softmmu
>>             EXTRA_CONFIG="--enable-modules"


--
Alex Bennée