Re: [Announcement] Yocto Project 1.5 Milestone 3 now available.

[Paul Eggleton <paul.eggleton@linux.intel.com>]

Hi Robert,

On Saturday 10 August 2013 10:20:54 Robert Berger wrote:
> On 08/10/2013 03:22 AM, Flanagan, Elizabeth wrote:
> > * Infrastructure for automated QA testing. More testcases will be
> > automated in the next milestone release.
> 
> Are we talking about test automation using autobuilder here?
> As far as I can see runtime testing of images using QEMU is
> planned/already used.

The new framework replaces the old, is written in python and makes writing 
tests much easier. As a result we are going through a process of adding many 
more runtime tests than we had previously.

> What I'm after is runtime testing on real hardware. Assuming an
> infrastructure where you can remotely turn on/off the power of the board
> and where you have access e.g. via a conserver to the serial port I
> guess this should be possible as well.
> 
> I'm wondering why this seems to be limited to QEMU, am I missing something?

We do definitely want automated testing on real hardware; it's just that that 
it presents a number of problems that need solving:

 * How do you deploy the image/kernel/bootloader onto the board in an 
automated manner? To different kinds of boards?
 * How do you manage access to the boards when you have multiple autobuilders 
potentially wanting to make use of them at around the same time?

We'll be working through these but the likelihood is that we won't have this 
part of it for 1.5. Once we do though, connecting the currently QEMU-based 
testing framework to it should be trivial.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre