[U-Boot] [PATCH v2 8/8] travis-ci: Add test.py for various qemu platforms

Stephen Warren
2016-10-24T22:21:36Z

On 10/22/2016 06:56 AM, Tom Rini wrote:
> On Fri, Oct 21, 2016 at 12:24:07PM -0600, Stephen Warren wrote:
>> On 10/20/2016 01:40 PM, Tom Rini wrote:
>>> - Add a PPA for a more recent qemu (required for PowerPC to work)
>>> - Add tests to run test.py for various QEMU platforms. This relies on
>>> swarren's uboot-test-hooks repository to provide the abstractions.

>>> + # Clone uboot-test-hooks
>>> + - git clone --depth=1 git://github.com/swarren/uboot-test-hooks.git /tmp/uboot-test-hooks
>>
>> Does it make sense to rely on my copy of the repo being around all
>> the time? I have no intention of removing it, but when I created it
>> I'd assumed others would create their own clones for their own
>> infra-structure. I wonder if all the dependencies for U-Boot's
>> testing should be hosted on U-Boot infra-structure?
>
> A good point. The way I've been thinking of it is that we also need a
> repo with as many and varied configs as possible, which has become yours
> :). That said, we could move a copy of this over to the u-boot account
> on github if you'd like.

I think it's fine for now.

>> That all said, this should be fine; just something to ponder.
>>
>>> + - ln -s travis-ci /tmp/uboot-test-hooks/bin/`hostname`
>>
>> It may be useful to add the following too:
>>
>> - ln -s travis-ci /tmp/uboot-test-hooks/py/`hostname`
>>
>> That way, if we ever need per-board test configuration files, the
>> path to access them will already be available.
>
> A good point. I think we could get the network tests going, maybe with
> a little re-work to dynamically hash the file from the host side? But
> that's an aside.

Yes, assuming qemu networking works, I expect that would be possible.
Yes, I think it'd be pretty easy to dynamically create a test download
file and calculate the CRC from the per-board Python config files.

>>> + - env:
>>> + - CROSS_COMPILE="/usr/bin/arm-linux-gnueabihf-"
>>> + script:
>>> + - ./test/py/test.py --bd vexpress_ca15_tc2 --id qemu --build;
>>> + ./test/py/test.py --bd vexpress_ca9x4 --id qemu --build;
>>> + ./test/py/test.py --bd integratorcp_cm926ejs --id qemu --build;
>>
>> Does it make sense to make that 3 separate scripts? I assume that
>> would (a) allow the individual status of the 3 test.py invocations
>> to be seen separately (b) might allow/cause travis-ci to run them in
>> parallel; I'm not sure if this Travis config file is a sequential
>> list of commands, or a list of potentially parallel actions.
>
> So, the way travis-ci works is that you can (and we do) split tasks up
> to run in parallel. The hunk above is a series to run in sequence on
> the same host. The downside to moving this to parallel is the amount of
> duplication you need in each stanza.

The duplication doesn't seem like a terribly big deal given it will
allow finer-grained status on the main per-build page.

> We could fix that (at least in the
> case of non-buildman-fetched toolchains) if test.py used buildman to
> build, as that will determine CROSS_COMPILE correctly from the default
> .buildman file we create earlier on. Thanks!

I'll look into whether that's possible.

BTW, does/can Travis save the test-log.html from the test.py
invocations? That would make debugging any failures a little easier.