Re: [Qemu-devel] Please read: make check framework

["Andreas Färber" <andreas.faerber@web.de>]

Am 10.01.2012 00:56, schrieb Anthony Liguori:
> On 01/09/2012 05:22 PM, Andreas Färber wrote:
>> Am 09.01.2012 22:42, schrieb Anthony Liguori:
>>> On 01/09/2012 02:57 PM, Andreas Färber wrote:
>>>> Am 09.01.2012 20:28, schrieb Anthony Liguori:
>>>>> We're still short on infrastructure right now so I expect that a
>>>>> lot of
>>>>> stuff will get merged without tests at first.  As we get more test
>>>>> infrastructure like qtest and qemu-test, I expect that to change.
>>>>>
>>>>> In terms of total coverage for any given feature, I think
>>>>> ultimately the
>>>>> goal is to move the responsibility for regression testing from the
>>>>> contributors of new features to the subsystems themselves.
>>>>>
>>>>> IOW, after we get going with our test infrastructure, when we do big
>>>>> sweeping changes like the memory API or QOM, I will have much less
>>>>> sympathy for breakages that are caused by subsystems with an
>>>>> incomplete
>>>>> test suite.
>>>>>
>>>>> So I think the most important thing for maintainers to start thinking
>>>>> about is how they can add the necessary infrastructure for testing
>>>>> their
>>>>> subsystems.
>>>>
>>>> Well, I already tried pointing out that qemu-test in the way previously
>>>> proposed does not help catching the recent regressions for PReP.
>>>
>>> Actually, qemu-test did catch it.  The only reason I didn't see it is
>>> that I didn't add qemu-system-powerpc to my regression script that I'm
>>> using to drive it.  But simply booting up a Linux guest would have
>>> caught it.
>>
>> Booting up a Linux kernel yes, but not the latest-and-greatest like you
>> proposed. Meaning, we need to be able to reference different branches or
>> trees instead of a single submodule (cf. qemu-test thread).
> 
> I don't understand what you mean.  Given the set of submodules currently
> used, I'm not aware of any target that has issues.

To my knowledge -M prep is not supported from 2.6 kernel on while
g3beige/mac99/pseries are. So yes, there's ppc support in binutils, yes
there'll probably be ppc support in uClibc but no 3.x kernel support
despite the target generally being supported by the kernel.

>  You asserted that it
> may not be the case that a single version of Linux, busybox, uclibc
> would all work on all targets.  But so far, I don't see any evidence to
> suggest that this isn't, in fact, possible.

Depends on what you consider possible, of course. ;)

At university we were stuck at 2.4.x for the Sharp Zaurus (armel) -
don't know the background there though, might be resolved by now.
Many mipsel devices still run 2.4.x but that's more of a binary-driver
issue, I guess.

ARM recently needed a binutils fix by Alex to avoid some segfault.
Toshiba AC100, Efika, etc. all have their respective downstream kernels.
Haven't checked the kernel for Cortex-R4F support yet but my board
doesn't really support it memory-wise (unless there's sufficient RAM to
boot via CAN, but point is, not all boards are manufactured for Linux).

Examples of potentially problematic targets not in upstream:

* V850 was recently scheduled for removal from the kernel; stable
binutils+gcc available, apparently supported by uClib.

* RX needs a binutils snapshot apparently (GNURX v11.03) and is not
mentioned in the uClibc FAQ, but supported by gcc 4.5; no kernel port
that I'm aware of (/me poking at it).

* RL78 needs binutils+gcc+newlib snapshot (GNRL78 v11.03), not mentioned
in uClibc FAQ; no kernel port that I'm aware of (and doesn't seem worth
the effort to me).

* STM8 not supported by binutils+gcc at all (llvm+clang port underway),
probably no uClibc/kernel support either then.

* 8051 needs sdcc iirc. Don't know about libc/kernel.

* z80? openrisc? mmix? avr32? iPhone?

You'll find plenty more examples when you look at embedded systems
(e.g., automotive) and/or relatively new architectures.

Doesn't mean we can't use qemu-test for some targets, just that we can't
use Linux 3.x and uClibc for everything. For many things yes.

In particular my criticism targetted the proposed use of submodules for
qemu-jeos.git: Doing the checkouts from a trivial shell script instead
would allow to checkout different commits/trees for different targets
and resolve part of the problem.

Andreas