Re: [PATCH] tests/acceptance: Add boot tests for some of the QEMU advent calendar images

[Thomas Huth <thuth@redhat.com>]

On 25/01/2020 19.38, Philippe Mathieu-Daudé wrote:
> On 1/25/20 5:43 PM, Thomas Huth wrote:
>> On 24/01/2020 22.28, Wainer dos Santos Moschetta wrote:
>>>
>>> On 1/24/20 3:03 PM, Thomas Huth wrote:
>>>> The 2018 edition of the QEMU advent calendar 2018 featured Linux images
>>>> for various non-x86 machines. We can use them for a boot tests in our
>>>> acceptance test suite.
> 
> Excellent idea!
> 
>>>>
>>>> Let's also make sure that we build the corresponding machines in Travis,
>>>> and while we're there, drop the superfluous --python parameter (python3
>>>> is now the only supported version anyway).
> 
> I'd rather see this change as another commit.

I thought so, too, often it is good to separate stuff into separate
commits, but in this case, it's not source code but just the CI yml
file, and the change is really trivial, so it rather sounds like
superfluous churn to me in this case to do it in a separate patch...

>>> Yes, please, removal of --python was in my wish list.
>>>
>>>>
>>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>>> ---
>>>>   .travis.yml                            |  2 +-
>>>>   tests/acceptance/boot_linux_console.py | 96 ++++++++++++++++++++++++++
>>>>   2 files changed, 97 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/.travis.yml b/.travis.yml
>>>> index 6c1038a0f1..73ca12c921 100644
>>>> --- a/.travis.yml
>>>> +++ b/.travis.yml
>>>> @@ -268,7 +268,7 @@ matrix:
>>>>         # Acceptance (Functional) tests
>>>>       - env:
>>>> -        - CONFIG="--python=/usr/bin/python3
>>>> --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu,ppc-softmmu,ppc64-softmmu,m68k-softmmu,sparc-softmmu"
>>>>
>>>> +        -
>>>> CONFIG="--target-list=aarch64-softmmu,alpha-softmmu,arm-softmmu,m68k-softmmu,microblaze-softmmu,mips-softmmu,mips64el-softmmu,nios2-softmmu,or1k-softmmu,ppc-softmmu,ppc64-softmmu,s390x-softmmu,sparc-softmmu,x86_64-softmmu,xtensa-softmmu"
>>>>
>>>
>>>
>>> Perhaps use MAIN_SOFTMMU_TARGETS in only append the other targets, like:
>>>
>>> --target-list=${MAIN_SOFTMMU_TARGETS},alpha-softmmu,sparc-softmmu,....
>>
>> Not sure ... while it is a nice way to shorten the line here, it adds a
>> dependency to that variable ... and MAIN_SOFTMMU_TARGETS has been changed
>> a couple of times during the course of time, so we might risk to lose
>> some testing coverage here in case someone removes a target from
>> MAIN_SOFTMMU_TARGETS but forgets to add it here again...? I think we
>> should better use the explicit list here instead.
> 
> Maybe related: "Split enterprise vs. hobbyist acceptance test job"
> https://www.mail-archive.com/qemu-devel@nongnu.org/msg644683.html

I guess naming/splitting is hard in this case ... many targets contain
both, "enterprise" and "hobbyist" machines ... and as long as the Travis
job runs reasonably fast, it's also not really necessary to split it.
The only real ugliness is the very long line - I haven't found a way yet
to correctly wrap this in the YML file to multiple lines .... does
anybody know how to do this in YML?

> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Thanks!

 Thomas