fsl-community-bsp: signing up for testing

fsl-community-bsp: signing up for testing

[Stephano Cetola]

[-- Attachment #1: Type: text/plain, Size: 879 bytes --]

I would like to open a discussion regarding the testing of the
fsl-community-bsp. I am currently building several images for both the
wandboard and udoo boards, and have access to both the mx28 and mx53 EVK
boards.

It would be great if there was a place for people in the community like
myself to post my experiences building for these boards. It would be a
place where people could sign up for specific boards, images, or even
specific packages that they will be able to test with each new release
(e.g. x11, qt, touch, gstreamer). I release that this can get complicated
rather quickly, so it may be wise to keep things as simple as possible.

I think that this would not only help with testing, but also clarify what
needs to be documented for both new users and for users already experienced
with Yocto and Freescale's meta layers.

Kind Regards,
Stephano

[-- Attachment #2: Type: text/html, Size: 975 bytes --]

Re: fsl-community-bsp: signing up for testing

[Trevor Woerner]

On 07/03/14 11:52, Stephano Cetola wrote:
...

All excellent points!

In my opinion, the testing needs to be split into two phases:
1) help/guidelines/docs on testing
2) reporting

I think we can agree that Daiane has done an excellent job on the
reporting side: to create forms to collect and aggregate the test result
information.

I think where we could use more work (or maybe this already exists,
perhaps I'm just not aware of it?) is on the "how to test" side.

If we had a set of procedures that were so "dumbed down" that a person
with little OE experience could follow them, perhaps it would be easier
to get more members of the community involved in testing?

Or, even better than documentation, what if we had build targets that
were functional and ready to run "out of the box" for various tests? Why
isn't it good enough to simply run "bitbake core-image-weston" or
"fsl-image-multimedia"? Why are tweaks always required to local.conf
(i.e. DISTRO_FEATURES, EXTRA_IMAGES, PREFERRED_PROVIDER)? If the
multimedia target _always_ requires the gstreamer packages to be added,
shouldn't that happen automatically when someone builds that target?

We should probably also look into those ptests to see if they can't help
with some of this testing/reporting.

Re: fsl-community-bsp: signing up for testing

[Daiane Angolini]

> In my opinion, the testing needs to be split into two phases:
> 1) help/guidelines/docs on testing
> 2) reporting

Continuous test? Or code freeze followed by test cycle?

>
> I think we can agree that Daiane has done an excellent job on the
> reporting side: to create forms to collect and aggregate the test result
> information.
>
> I think where we could use more work (or maybe this already exists,
> perhaps I'm just not aware of it?) is on the "how to test" side.

It looks like dumb, but "how to test" is always a bottle neck.

I personally does not believe in benchmarking, because they are used
as a global measurement tool, and in my point of view they only
provide limited results. The same way, I believe any test must be tied
with some boundary condition. And this is the definition of "how to
test" and "how to read the result"

Only to be remember. In test cycle from 1.6 it was the very first time
we (Otavio mainly) provided all the pre-built target image to be
tested. And the form has all the command lines to be used, or open
answers.

>
> If we had a set of procedures that were so "dumbed down" that a person
> with little OE experience could follow them, perhaps it would be easier
> to get more members of the community involved in testing?

We provided the pre-built images last time. And Otavio has been
keeping it online since them.

More "no OE experience" than download and burn sdcard is impossible.


>
> Or, even better than documentation, what if we had build targets that
> were functional and ready to run "out of the box" for various tests? Why
> isn't it good enough to simply run "bitbake core-image-weston" or
> "fsl-image-multimedia"? Why are tweaks always required to local.conf
> (i.e. DISTRO_FEATURES, EXTRA_IMAGES, PREFERRED_PROVIDER)? If the
> multimedia target _always_ requires the gstreamer packages to be added,
> shouldn't that happen automatically when someone builds that target?

fsl-image-machine-test was created focusing it.

>
> We should probably also look into those ptests to see if they can't help
> with some of this testing/reporting.


Absolutely agree!





Daiane

Re: fsl-community-bsp: signing up for testing

[Trevor Woerner]

On 07/04/14 08:59, Daiane Angolini wrote:
> We provided the pre-built images last time. And Otavio has been
> keeping it online since them. More "no OE experience" than download
> and burn sdcard is impossible. 

Oh yes, I completely forgot about those images! Thanks for the reminder.
These images are exactly what I was hoping for.

Out of curiosity, any chance the configuration and build information is
available for each image?

>> Or, even better than documentation, what if we had build targets that
>> were functional and ready to run "out of the box" for various tests? Why
>> isn't it good enough to simply run "bitbake core-image-weston" or
>> "fsl-image-multimedia"? Why are tweaks always required to local.conf
>> (i.e. DISTRO_FEATURES, EXTRA_IMAGES, PREFERRED_PROVIDER)? If the
>> multimedia target _always_ requires the gstreamer packages to be added,
>> shouldn't that happen automatically when someone builds that target?
> fsl-image-machine-test was created focusing it.

Thanks for this pointer :-)

Re: fsl-community-bsp: signing up for testing

[Otavio Salvador]

On Sat, Jul 5, 2014 at 10:49 AM, Trevor Woerner
<trevor.woerner@linaro.org> wrote:
> On 07/04/14 08:59, Daiane Angolini wrote:
>> We provided the pre-built images last time. And Otavio has been
>> keeping it online since them. More "no OE experience" than download
>> and burn sdcard is impossible.
>
> Oh yes, I completely forgot about those images! Thanks for the reminder.
> These images are exactly what I was hoping for.

Great this is helpful :-)

> Out of curiosity, any chance the configuration and build information is
> available for each image?

What are you looking for specifically?

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: fsl-community-bsp: signing up for testing

[Trevor Woerner]

Hi Otavio,

On 07/05/14 10:17, Otavio Salvador wrote:
> On Sat, Jul 5, 2014 at 10:49 AM, Trevor Woerner
> <trevor.woerner@linaro.org> wrote:
> Out of curiosity, any chance the configuration and build information is
> available for each image?
> What are you looking for specifically?

I would be interested in seeing a Weston/Wayland image and a directfb
image running. I'd like to be able to build them myself so I can tweak
them myself (e.g. maynard?).

I'm guessing neither of those would work with the imx53qsb board, but I
do have a Wandboard-dual which I assume would work?

Re: fsl-community-bsp: signing up for testing

[Otavio Salvador]

On Sun, Jul 6, 2014 at 5:43 PM, Trevor Woerner
<trevor.woerner@linaro.org> wrote:
> Hi Otavio,
>
> On 07/05/14 10:17, Otavio Salvador wrote:
>> On Sat, Jul 5, 2014 at 10:49 AM, Trevor Woerner
>> <trevor.woerner@linaro.org> wrote:
>> Out of curiosity, any chance the configuration and build information is
>> available for each image?
>> What are you looking for specifically?
>
> I would be interested in seeing a Weston/Wayland image and a directfb
> image running. I'd like to be able to build them myself so I can tweak
> them myself (e.g. maynard?).

The images are built there. What I think it is missing is a document
which says how we can use Wayland/Weston in i.MX6 specifically.

Daiane, any advice where we should put this information?

> I'm guessing neither of those would work with the imx53qsb board, but I
> do have a Wandboard-dual which I assume would work?

Yes, MX5 does not have Wayland support or DirectFB at this moment. Use
Wandboard for now...

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: fsl-community-bsp: signing up for testing

[Daiane Angolini]

Daiane


On Mon, Jul 7, 2014 at 9:16 AM, Otavio Salvador <otavio@ossystems.com.br> wrote:
> On Sun, Jul 6, 2014 at 5:43 PM, Trevor Woerner
> <trevor.woerner@linaro.org> wrote:
>> Hi Otavio,
>>
>> On 07/05/14 10:17, Otavio Salvador wrote:
>>> On Sat, Jul 5, 2014 at 10:49 AM, Trevor Woerner
>>> <trevor.woerner@linaro.org> wrote:
>>> Out of curiosity, any chance the configuration and build information is
>>> available for each image?
>>> What are you looking for specifically?
>>
>> I would be interested in seeing a Weston/Wayland image and a directfb
>> image running. I'd like to be able to build them myself so I can tweak
>> them myself (e.g. maynard?).
>
> The images are built there. What I think it is missing is a document
> which says how we can use Wayland/Weston in i.MX6 specifically.
>
> Daiane, any advice where we should put this information?

There is a tiny step-by-step for weston here (copied from some commit log)
https://github.com/Freescale/Documentation/blob/master/user-guide/source/weston.rst


However, Otavio, there is no way to documet "how we can use
Wayland/weston in i.MX6" unless you want to write another book, this
time more than 300 pages.

Maybe what you're talking about is a single page with the instructions
on how to recreate the pre-built images from OSSystems's repositories.




>
>> I'm guessing neither of those would work with the imx53qsb board, but I
>> do have a Wandboard-dual which I assume would work?
>
> Yes, MX5 does not have Wayland support or DirectFB at this moment. Use
> Wandboard for now...

Great reminder to me! Add a line in known issues from #RN telling that
wayland support or directfb support is not accelerated in any non-imx6
board, and the SW version is broken right now.


Daiane

Re: fsl-community-bsp: signing up for testing

[Otavio Salvador]

On Mon, Jul 7, 2014 at 9:28 AM, Daiane Angolini <daiane.list@gmail.com> wrote:
> On Mon, Jul 7, 2014 at 9:16 AM, Otavio Salvador <otavio@ossystems.com.br> wrote:
>> On Sun, Jul 6, 2014 at 5:43 PM, Trevor Woerner
>> <trevor.woerner@linaro.org> wrote:
>>> On 07/05/14 10:17, Otavio Salvador wrote:
>>>> On Sat, Jul 5, 2014 at 10:49 AM, Trevor Woerner
>>>> <trevor.woerner@linaro.org> wrote:
>>>> Out of curiosity, any chance the configuration and build information is
>>>> available for each image?
>>>> What are you looking for specifically?
>>>
>>> I would be interested in seeing a Weston/Wayland image and a directfb
>>> image running. I'd like to be able to build them myself so I can tweak
>>> them myself (e.g. maynard?).
>>
>> The images are built there. What I think it is missing is a document
>> which says how we can use Wayland/Weston in i.MX6 specifically.
>>
>> Daiane, any advice where we should put this information?
>
> There is a tiny step-by-step for weston here (copied from some commit log)
> https://github.com/Freescale/Documentation/blob/master/user-guide/source/weston.rst
>
>
> However, Otavio, there is no way to documet "how we can use
> Wayland/weston in i.MX6" unless you want to write another book, this
> time more than 300 pages.

Oh my ... not now please ...

> Maybe what you're talking about is a single page with the instructions
> on how to recreate the pre-built images from OSSystems's repositories.

This works fine.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: fsl-community-bsp: signing up for testing

[Daiane Angolini]

On Thu, Jul 3, 2014 at 12:52 PM, Stephano Cetola <stephanoc@gmail.com> wrote:
> I would like to open a discussion regarding the testing of the
> fsl-community-bsp. I am currently building several images for both the
> wandboard and udoo boards, and have access to both the mx28 and mx53 EVK
> boards.
>
> It would be great if there was a place for people in the community like
> myself to post my experiences building for these boards. It would be a place
> where people could sign up for specific boards, images, or even specific
> packages that they will be able to test with each new release (e.g. x11, qt,
> touch, gstreamer). I release that this can get complicated rather quickly,
> so it may be wise to keep things as simple as possible.

Do you know the test form I created for the test cycle?
I mean, would be this tool (form from GDoc) a good starting point?
(if not I can share the GDoc to your gmail account)

Daiane

Re: fsl-community-bsp: signing up for testing

[Stephano Cetola]

On Fri, Jul 4, 2014 at 5:50 AM, Daiane Angolini <daiane.list@gmail.com> wrote:
>
> Do you know the test form I created for the test cycle?

I have not seen this form.  Please do share it when you can.

I think that a GDoc form would be a great starting point.

Stephano

Re: fsl-community-bsp: signing up for testing

[Daiane Angolini]

On Fri, Jul 4, 2014 at 11:43 PM, Stephano Cetola <stephanoc@gmail.com> wrote:
> On Fri, Jul 4, 2014 at 5:50 AM, Daiane Angolini <daiane.list@gmail.com> wrote:
>>
>> Do you know the test form I created for the test cycle?
>
> I have not seen this form.  Please do share it when you can.
>
> I think that a GDoc form would be a great starting point.

Done.

I shared it privately to your g-account


Daiane