Re: [Qemu-arm] [Qemu-devel] [RFC PATCH] tests/device-introspect: Test devices with all machines, not only with "none"

[Thomas Huth <thuth@redhat.com>]

On 27.04.2018 02:32, Eduardo Habkost wrote:
> On Thu, Apr 26, 2018 at 05:20:25PM +0200, Thomas Huth wrote:
>> On 26.04.2018 13:45, Markus Armbruster wrote:
>>> Thomas Huth <thuth@redhat.com> writes:
>> [...]
>>>> @@ -260,6 +263,26 @@ static void test_abstract_interfaces(void)
>>>>      qtest_end();
>>>>  }
>>>>  
>>>> +static void add_machine_test_case(const char *mname)
>>>> +{
>>>> +    char *path, *args;
>>>> +
>>>> +    /* Ignore blacklisted machines */
>>>> +    if (g_str_equal("xenfv", mname) || g_str_equal("xenpv", mname)) {
>>>> +        return;
>>>> +    }
>>>> +
>>>> +    path = g_strdup_printf("device/introspect/concrete-defaults-%s", mname);
>>>> +    args = g_strdup_printf("-machine %s", mname);
>>>> +    qtest_add_data_func(path, args, test_device_intro_concrete);
>>>
>>> This runs test_device_intro_concrete() with "-machine M" for all machine
>>> types M, in SPEED=slow mode.
>>>
>>>> +    g_free(path);
>>>> +
>>>> +    path = g_strdup_printf("device/introspect/concrete-nodefaults-%s", mname);
>>>> +    args = g_strdup_printf("-nodefaults -machine %s", mname);
>>>> +    qtest_add_data_func(path, args, test_device_intro_concrete);
>>>
>>> This runs test_device_intro_concrete() with "-nodefaults -machine M" for
>>> all machine types M, in SPEED=slow mode.
>>>
>>> Has "without -nodefaults" exposed additional bugs?
>>
>> After testing this with all machines, I had to discover that
>> "-nodefaults" does not work so easily: A lot of the embedded machines
>> (especially the ARM machines) simply refuse to work with "-nodefaults"
>> and exit immediately instead. E.g.:
>>
>> $ arm-softmmu/qemu-system-arm -nodefaults -nographic -M n810,accel=qtest
>> qemu-system-arm: missing SecureDigital device
>>
>> So we'd either need a rather big black list for the machines that do not
>> work, or simply drop the "-nodefaults" tests from this patch.
> 
> Or we could try to test all machines anyway, but not consider it
> an error if QEMU just does exit(1).  Can the qtest C API give us
> that information?

At a first glance, I haven't seen an easy way to do this. I guess we
could do some polling with waitid() or do something with SIGCHLD, but...

> (Or we could simply let -nodefaults aside by now, and do this
> after we implement this test case in Python.)

... I'd rather prefer that for now, assuming that the test will later
get replaced by the python test anyway.

 Thomas