[Buildroot] Sample configurations / test suite ?

[Buildroot] Sample configurations / test suite ?

[Thomas Petazzoni]

Hello,

Currently, the defconfig that we accept in configs/ are defconfig for
various hardware platforms that should be as minimal as possible: the
bootloader(s), the kernel, Busybox, the right filesystem images and
that's it. Since I've been a fairly strong supporter of this approach,
I continue to believe it makes sense, since there is no way, for a
particular board, to decide which software components should or should
not be part of the default configuration for this board.

However, I am seeing two cases where we would want to have bigger, more
specific defconfig files:

 1) As part of a test suite. Things like building Xenomai, RTAI,
    building a kernel with the various combination (with or without DT,
    etc.) are not tested by our autobuilders, and we've recently had
    reports of Xenomai being broken for example. Having a set of
    configurations that are interesting to build would be very useful,
    and this set could be extended when we think it is necessary (for
    example after receiving a bug report). Those configurations could
    be built on a daily basis, they could have some custom post-image
    script to verify that the build has generated everything that was
    expected, etc.

 2) As part of a set of demonstration defconfigs. For example, Spenser
    Gilliland has recently posted on his blog a defconfig that
    demonstrates how to use OpenMAX on Raspberry Pi to do
    hardware-accelerated video decoding. Maybe it would make sense to
    have a way of storing those "demonstration" configurations
    somewhere, so that they gain in visibility and can more easily be
    re-used by users.

What do you think about this? Do you have ideas on how to implement
this? Should it be part of the Buildroot tree itself, or something
separate? If something separate, how do we keep Buildroot and this
separate tree in sync?

Best regards,

Thomas Petazzoni
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] Sample configurations / test suite ?

[Yann E. MORIN]

Thomas, All,

On 2013-06-27 00:43 +0200, Thomas Petazzoni spake thusly:
> Currently, the defconfig that we accept in configs/ are defconfig for
> various hardware platforms that should be as minimal as possible: the
> bootloader(s), the kernel, Busybox, the right filesystem images and
> that's it. Since I've been a fairly strong supporter of this approach,
> I continue to believe it makes sense, since there is no way, for a
> particular board, to decide which software components should or should
> not be part of the default configuration for this board.
> 
> However, I am seeing two cases where we would want to have bigger, more
> specific defconfig files:
> 
>  1) As part of a test suite. Things like building Xenomai, RTAI,
>     building a kernel with the various combination (with or without DT,
>     etc.) are not tested by our autobuilders, and we've recently had
>     reports of Xenomai being broken for example. Having a set of
>     configurations that are interesting to build would be very useful,
>     and this set could be extended when we think it is necessary (for
>     example after receiving a bug report). Those configurations could
>     be built on a daily basis, they could have some custom post-image
>     script to verify that the build has generated everything that was
>     expected, etc.
> 
>  2) As part of a set of demonstration defconfigs. For example, Spenser
>     Gilliland has recently posted on his blog a defconfig that
>     demonstrates how to use OpenMAX on Raspberry Pi to do
>     hardware-accelerated video decoding. Maybe it would make sense to
>     have a way of storing those "demonstration" configurations
>     somewhere, so that they gain in visibility and can more easily be
>     re-used by users.
> 
> What do you think about this? Do you have ideas on how to implement
> this? Should it be part of the Buildroot tree itself, or something
> separate? If something separate, how do we keep Buildroot and this
> separate tree in sync?

For what it's worth, I've started something along those lines. This is
for now a simple git-tree with a few (RPi-related for now) defconfigs
and a collection of scripts.

For lack of a better name, I've called it 'Buildroot.config'. You can
browse the tree there:
    http://ymorin.is-a-geek.org/git/buildroot.config/

The basic idea I've had for this was to automate building a known
Buildroot configuration, from cloning the Buildroot tree, up to
generating the final images.

It's still rough on the edges, and still lacks all the glue to make it
work seamlessly. I'm currently implementing a simple script that would
wrap all of this, but I still need to make some design decisions.

I wanted to wait yet a bit more before advertising it, but since you
raised the question, here it goes public! Well, Peter already knew, but
it's yet more public, now! :-)

Maybe this Buildroot.config tree is something we can improve with the
aim of making it the basis of this full-blown defconfig-for-tests
infrastructure. Please tell me what you think about it! :-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] Sample configurations / test suite ?

[Arnout Vandecappelle]

On 06/27/13 00:43, Thomas Petazzoni wrote:
> Hello,
>
> Currently, the defconfig that we accept in configs/ are defconfig for
> various hardware platforms that should be as minimal as possible: the
> bootloader(s), the kernel, Busybox, the right filesystem images and
> that's it. Since I've been a fairly strong supporter of this approach,
> I continue to believe it makes sense, since there is no way, for a
> particular board, to decide which software components should or should
> not be part of the default configuration for this board.
>
> However, I am seeing two cases where we would want to have bigger, more
> specific defconfig files:
>
>   1) As part of a test suite. Things like building Xenomai, RTAI,
>      building a kernel with the various combination (with or without DT,
>      etc.) are not tested by our autobuilders, and we've recently had
>      reports of Xenomai being broken for example. Having a set of
>      configurations that are interesting to build would be very useful,
>      and this set could be extended when we think it is necessary (for
>      example after receiving a bug report). Those configurations could
>      be built on a daily basis, they could have some custom post-image
>      script to verify that the build has generated everything that was
>      expected, etc.
>
>   2) As part of a set of demonstration defconfigs. For example, Spenser
>      Gilliland has recently posted on his blog a defconfig that
>      demonstrates how to use OpenMAX on Raspberry Pi to do
>      hardware-accelerated video decoding. Maybe it would make sense to
>      have a way of storing those "demonstration" configurations
>      somewhere, so that they gain in visibility and can more easily be
>      re-used by users.
>
> What do you think about this? Do you have ideas on how to implement
> this? Should it be part of the Buildroot tree itself, or something
> separate? If something separate, how do we keep Buildroot and this
> separate tree in sync?

  For both use cases, it makes the most sense if these defconfigs are 
part of the buildroot tree.

  The risk is that the configs/ directory becomes too large and unwieldy 
(people will have to browse it to find the defconfig they want). So 
perhaps this calls for changing it into a tree.

  Personally, I think it makes sense to move the defconfigs into the 
board/ directory. Many defconfigs already refer into there for kernel 
configs or specific patches, so it makes sense to put the defconfig in 
the same place.

  And while I'm on this subject, I think the structure of the board 
directory is not very clear. It would make sense to me to switch to the 
layout that U-Boot uses: board/<arch>/<soc>/<boardname>/ (although the 
<soc> level may be optional for us). You can expect people to know what 
the basic architecture of the processor is, but not always who the vendor 
is (which is probably why raspberrypi, beaglebone and gnublin don't have 
a vendor directory). Or sometimes there are multiple vendors for the same 
board (e.g. Beagleboard and SabreLite).

  But of course, I'm not going to produce patches so I should probably 
shut up :-)

  Regards,
  Arnout


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] Sample configurations / test suite ?

[Thomas Petazzoni]

Dear Arnout Vandecappelle,

On Mon, 01 Jul 2013 08:00:02 +0200, Arnout Vandecappelle wrote:

> > What do you think about this? Do you have ideas on how to implement
> > this? Should it be part of the Buildroot tree itself, or something
> > separate? If something separate, how do we keep Buildroot and this
> > separate tree in sync?
> 
>   For both use cases, it makes the most sense if these defconfigs are 
> part of the buildroot tree.

In order to keep those configurations consistent with the rest of
Buildroot, I agree that having them in the Buildroot tree is probably
the easiest option. However, I'm worried about the size of it: I was
not only thinking of defconfigs, but potentially additional artifacts
needed to make the build work.

>   The risk is that the configs/ directory becomes too large and unwieldy 
> (people will have to browse it to find the defconfig they want). So 
> perhaps this calls for changing it into a tree.
> 
>   Personally, I think it makes sense to move the defconfigs into the 
> board/ directory. Many defconfigs already refer into there for kernel 
> configs or specific patches, so it makes sense to put the defconfig in 
> the same place.

Funnily enough, the defconfigs *used* to be in the board/ directory
(which at the time was target/device). We had a discussion back in the
days on whether the defconfigs should remain with their board, or
grouped in the top-level configs/ directory.

See
http://lists.busybox.net/pipermail/buildroot/2009-October/029556.html
and my complaint
http://lists.busybox.net/pipermail/buildroot/2009-October/029559.html.

That said, after several years, I feel that configs/ was a pretty good
choice, I don't really feel the need of moving things back to board/,
especially considering the change it will cause to all users.

Moreover, I am not sure that those test suite / demos
configurations should be located in the same place as the minimal
defconfigs we have in configs/.

>   And while I'm on this subject, I think the structure of the board 
> directory is not very clear. It would make sense to me to switch to the 
> layout that U-Boot uses: board/<arch>/<soc>/<boardname>/ (although the 
> <soc> level may be optional for us). You can expect people to know what 
> the basic architecture of the processor is, but not always who the vendor 
> is (which is probably why raspberrypi, beaglebone and gnublin don't have 
> a vendor directory). Or sometimes there are multiple vendors for the same 
> board (e.g. Beagleboard and SabreLite).

Hmm, no strong opinion on this one. How many end users know which SoC
the RasberryPi is using?

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] Sample configurations / test suite ?

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

Hi,

 >> For both use cases, it makes the most sense if these defconfigs are 
 >> part of the buildroot tree.

 Thomas> In order to keep those configurations consistent with the rest of
 Thomas> Buildroot, I agree that having them in the Buildroot tree is probably
 Thomas> the easiest option. However, I'm worried about the size of it: I was
 Thomas> not only thinking of defconfigs, but potentially additional artifacts
 Thomas> needed to make the build work.

Yes, I also think they need to be in tree to really be useful.


 >> The risk is that the configs/ directory becomes too large and unwieldy 
 >> (people will have to browse it to find the defconfig they want). So 
 >> perhaps this calls for changing it into a tree.y
 >> 
 >> Personally, I think it makes sense to move the defconfigs into the 
 >> board/ directory. Many defconfigs already refer into there for kernel 
 >> configs or specific patches, so it makes sense to put the defconfig in 
 >> the same place.

 Thomas> Funnily enough, the defconfigs *used* to be in the board/ directory
 Thomas> (which at the time was target/device). We had a discussion back in the
 Thomas> days on whether the defconfigs should remain with their board, or
 Thomas> grouped in the top-level configs/ directory.

 Thomas> See
 Thomas> http://lists.busybox.net/pipermail/buildroot/2009-October/029556.html
 Thomas> and my complaint
 Thomas> http://lists.busybox.net/pipermail/buildroot/2009-October/029559.html.

Gah, 2009 ;)

I do still think having the available configs listed together (like
linux/barebox arch/$ARCH/configs, or u-boot's boards.cfg) is nice for
users.

 Thomas> That said, after several years, I feel that configs/ was a pretty good
 Thomas> choice, I don't really feel the need of moving things back to board/,
 Thomas> especially considering the change it will cause to all users.

 Thomas> Moreover, I am not sure that those test suite / demos
 Thomas> configurations should be located in the same place as the minimal
 Thomas> defconfigs we have in configs/.

 >> And while I'm on this subject, I think the structure of the board 
 >> directory is not very clear. It would make sense to me to switch to the 
 >> layout that U-Boot uses: board/<arch>/<soc>/<boardname>/ (although the 
 >> <soc> level may be optional for us). You can expect people to know what 
 >> the basic architecture of the processor is, but not always who the vendor 
 >> is (which is probably why raspberrypi, beaglebone and gnublin don't have 
 >> a vendor directory). Or sometimes there are multiple vendors for the same 
 >> board (e.g. Beagleboard and SabreLite).

U-boot actually nowadays normally use board/<vendor>/<boardname>, which
is in line with what we do.

-- 
Bye, Peter Korsgaard

[Buildroot] Sample configurations / test suite ?

[Arnout Vandecappelle]

On 07/01/13 09:19, Thomas Petazzoni wrote:
> Dear Arnout Vandecappelle,
>
> On Mon, 01 Jul 2013 08:00:02 +0200, Arnout Vandecappelle wrote:
>
>>> What do you think about this? Do you have ideas on how to implement
>>> this? Should it be part of the Buildroot tree itself, or something
>>> separate? If something separate, how do we keep Buildroot and this
>>> separate tree in sync?
>>
>>    For both use cases, it makes the most sense if these defconfigs are
>> part of the buildroot tree.
>
> In order to keep those configurations consistent with the rest of
> Buildroot, I agree that having them in the Buildroot tree is probably
> the easiest option. However, I'm worried about the size of it: I was
> not only thinking of defconfigs, but potentially additional artifacts
> needed to make the build work.

  I don't see that becoming an issue, honestly.

  When it does, we can still split it up I guess.


>>    The risk is that the configs/ directory becomes too large and unwieldy
>> (people will have to browse it to find the defconfig they want). So
>> perhaps this calls for changing it into a tree.
>>
>>    Personally, I think it makes sense to move the defconfigs into the
>> board/ directory. Many defconfigs already refer into there for kernel
>> configs or specific patches, so it makes sense to put the defconfig in
>> the same place.
>
> Funnily enough, the defconfigs *used* to be in the board/ directory
> (which at the time was target/device). We had a discussion back in the
> days on whether the defconfigs should remain with their board, or
> grouped in the top-level configs/ directory.
>
> See
> http://lists.busybox.net/pipermail/buildroot/2009-October/029556.html
> and my complaint
> http://lists.busybox.net/pipermail/buildroot/2009-October/029559.html.
>
> That said, after several years, I feel that configs/ was a pretty good
> choice, I don't really feel the need of moving things back to board/,
> especially considering the change it will cause to all users.

  Can't argue with the wisdom of years :-)


> Moreover, I am not sure that those test suite / demos
> configurations should be located in the same place as the minimal
> defconfigs we have in configs/.

  Good point. But then, that goes against the "putting all configs 
together" philosophy.

>
>>    And while I'm on this subject, I think the structure of the board
>> directory is not very clear. It would make sense to me to switch to the
>> layout that U-Boot uses: board/<arch>/<soc>/<boardname>/ (although the
>> <soc> level may be optional for us). You can expect people to know what
>> the basic architecture of the processor is, but not always who the vendor
>> is (which is probably why raspberrypi, beaglebone and gnublin don't have
>> a vendor directory). Or sometimes there are multiple vendors for the same
>> board (e.g. Beagleboard and SabreLite).
>
> Hmm, no strong opinion on this one. How many end users know which SoC
> the RasberryPi is using?

  And how many people know who is the vendor of the RPi? :-)


  Regards,
  Arnout

>
> Best regards,
>
> Thomas
>


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F