toaster automated testing

toaster automated testing

[Damian, Alexandru]

[-- Attachment #1: Type: text/plain, Size: 687 bytes --]

Hello,

I've been progressing a bit on the automated Toaster testing. We need to
get this step in a reasonable shape in order to help with the massive
refactoring that we planned for 1.9 M1.

As part of this effort, I moved the existing automated UI testing from a
poky-branch to yocto-TTS, as it stand no chance for successful merging
upstream in Poky.

You can find it here:

https://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/tree/toasteruitest?h=yocto-tts/master

and I strongly suggest that any further development be based on this tree.

I'm working to integrate these tests properly in the TTS.

Cheers,
Alex

-- 
Alex Damian
Yocto Project
SSG / OTC

[-- Attachment #2: Type: text/html, Size: 2061 bytes --]

Re: toaster automated testing

[Richard Purdie]

On Fri, 2015-05-08 at 14:47 +0100, Damian, Alexandru wrote:
> I've been progressing a bit on the automated Toaster testing. We need
> to get this step in a reasonable shape in order to help with the
> massive refactoring that we planned for 1.9 M1.
>
> As part of this effort, I moved the existing automated UI testing from
> a poky-branch to yocto-TTS, as it stand no chance for successful
> merging upstream in Poky.

Why wouldn't we want to merge automated tests for toaster somewhere like
bitbake?

Cheers,

Richard

Re: toaster automated testing

[Damian, Alexandru]

[-- Attachment #1: Type: text/plain, Size: 1334 bytes --]

My thinking is that unit tests, that are to be run inside the django
application, should be in bitbake.

These tests are integration and smoke tests, with a long list of
dependencies (selenium, browsers, w3c-validator, various other tools) and I
wouldn't want to bring these dependencies into Bitbake. Furthermore, the
code structure is not integrated at all with current code in Toaster,
Bitbake, or oe-selftest framework. It doesn't fit in.

Unit tests that validate individual Toaster features and bug fixes are to
be implemented in Django unit test framework, and live inside Bitbake.

Cheers,
Alex

On Fri, May 8, 2015 at 2:52 PM, Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Fri, 2015-05-08 at 14:47 +0100, Damian, Alexandru wrote:
> > I've been progressing a bit on the automated Toaster testing. We need
> > to get this step in a reasonable shape in order to help with the
> > massive refactoring that we planned for 1.9 M1.
> >
> > As part of this effort, I moved the existing automated UI testing from
> > a poky-branch to yocto-TTS, as it stand no chance for successful
> > merging upstream in Poky.
>
> Why wouldn't we want to merge automated tests for toaster somewhere like
> bitbake?
>
> Cheers,
>
> Richard
>
>
>


-- 
Alex Damian
Yocto Project
SSG / OTC

[-- Attachment #2: Type: text/html, Size: 2401 bytes --]

Re: toaster automated testing

[Richard Purdie]

On Fri, 2015-05-08 at 15:21 +0100, Damian, Alexandru wrote:
> My thinking is that unit tests, that are to be run inside the django
> application, should be in bitbake.
>
> These tests are integration and smoke tests, with a long list of
> dependencies (selenium, browsers, w3c-validator, various other tools)
> and I wouldn't want to bring these dependencies into Bitbake.
> Furthermore, the code structure is not integrated at all with current
> code in Toaster, Bitbake, or oe-selftest framework. It doesn't fit in.
>
> Unit tests that validate individual Toaster features and bug fixes are
> to be implemented in Django unit test framework, and live inside
> Bitbake.


I agree they don't belong alongside the existing tests, we could put
them in a contrib style directory though (I'm open to suggestions) and
document that there are certain dependencies needed to use them?

Cheers,

Richard

Re: toaster automated testing

[Damian, Alexandru]

[-- Attachment #1: Type: text/plain, Size: 1310 bytes --]

On Fri, May 8, 2015 at 5:32 PM, Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Fri, 2015-05-08 at 15:21 +0100, Damian, Alexandru wrote:
> > My thinking is that unit tests, that are to be run inside the django
> > application, should be in bitbake.
> >
> > These tests are integration and smoke tests, with a long list of
> > dependencies (selenium, browsers, w3c-validator, various other tools)
> > and I wouldn't want to bring these dependencies into Bitbake.
> > Furthermore, the code structure is not integrated at all with current
> > code in Toaster, Bitbake, or oe-selftest framework. It doesn't fit in.
> >
> > Unit tests that validate individual Toaster features and bug fixes are
> > to be implemented in Django unit test framework, and live inside
> > Bitbake.
>
>
> I agree they don't belong alongside the existing tests, we could put
> them in a contrib style directory though (I'm open to suggestions) and
> document that there are certain dependencies needed to use them?
>
>
​Yep, this is do-able - they will have no relationship at code level with
existing infrastructure though; separate requirements.txt and README will
be provided to guide usage. Is this ok?



> Cheers,
>
> Richard
>
>


-- 
Alex Damian
Yocto Project
SSG / OTC

[-- Attachment #2: Type: text/html, Size: 2139 bytes --]