Re: announcing the toaster test framework

[Michael Wood <michael.g.wood@intel.com>]

On 27/04/15 18:22, Damian, Alexandru wrote:
> Hi,
>
> I put together a couple of scripts that will monitor the toaster 
> mailing list, and provide smoke testing on each patchset submitted to 
> review request.
>
> A very rough form of these scripts existed for some time, but I've 
> taken time to clean that up, and enough code to be able to add tests 
> very easily, and make everything a bit more reliable.
>
> It lives outside of the poky/ or bitbake/ trees, and you can check out 
> the code here, until moved to a proper repo.
>
>
> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=yocto-tts/master
>
>
> Currently, my installation is emailing tests results only to me, but 
> once I've been satisfied that it doesn't crash or burn the mailing 
> list, I will update the setup to mail results directly to the mailing 
> list.
>
>
> You can also use it to manually run tests on your branch prior to 
> submission.
>

I know this wasn't an RFC but I do have one question... you're 
encoraging people to write tests for this new framework, but when are we 
saying a test be sent to the "toaster test framework" and when should it 
be a unit test in Toaster?

...My thoughts are that if it's possible to write the unit test in 
toaster with django[1] (i.e no external tools) then that should be the 
primary place for tests, we should be in the habbit of running 
./manage.py test pre-submission of patches. Those tests are broken or 
missing at the moment, so I hope we can fix those first.

What do you think?

[1] e.g. some I wrote for the error-report-web:
http://git.yoctoproject.org/cgit/cgit.cgi/error-report-web/tree/Post/test.py



> Cheers,
> Alex
>
> -- 
> Alex Damian
> Yocto Project
> SSG / OTC
>
>