[Buildroot] Article for the German Linux magazine

[Buildroot] Article for the German Linux magazine

[Arnout Vandecappelle]

  Hi all,

  About two weeks ago, we got a mail on the list asking for a volunteer to write 
an introductory article about Buildroot for the IoT-themed issue of the German 
Linux magazine. I volunteered two rite such an article.

  I have an outline or rough draft below. It's just bullet points (though I 
ended up writing sentences for most of it). I'd like to get feedback on:

- Is there anything missing that I should mention in the article?

- Is there something in here which is incorrect?

- Is there something here that we actually don't want to draw attention to?

  Please *do not* pay attention to style or spelling or little details that are 
missing.


  Thank you for your help!


  Regards,
  Arnout


Buildroot for IoT

For IoT distro is important:
- getting a basic system up and running quickly;
- small footprint (sometimes only 64MB of flash)
- control over what you build;
- support for all major cloud services
- easy integration of own components;
- attention for security issues.

Buildroot is a build system that grew out of uClinux and busybox, so with a 
focus on very small systems.
Meanwhile, it supports all kinds of different use cases, ranging from (still) 
systems with only a few MB of flash, over docker images for cloud deployments, 
to complex systems with a graphical stack, database, etc.
The principles of Buildroot are
- simplicity;
- flexibility to build exactly the system you want;
- reproducibility: if you rebuild the same configuration years later, you should 
get the same result bit-for-bit;
- maintainability.

With Buildroot, you can build a system by choosing which packages you want and 
some other options in a menu system (based on that of the Linux kernel).
On that basis, Buildroot will download and build the source code of the selected 
packages and compile it into the root file system.
You can also add your own extra files, for example configuration files or your 
own application.
To get you started, there are sample configurations for about 230 boards that 
you can buy off-the-shelf, plus about 40 configurations for simulation in qemu.
However, it is easy enough to make your own configuration, as long as you know 
which CPU variant you have and which bootloader and kernel options (device tree) 
to use.

In practice, however, you will almost always want to add your own application.
For that, you can generate an SDK that you can use to build the application, 
e.g. in an IDE.
But it is also very easy to create a new package for your application.
If you use one of the supported build systems (meson, cmake, autotools, qmake, 
waf, cargo, go, python, rebar), a few lines are usually enough.

Apart from the flexibility of the choice of packages (e.g. 4 different init 
systems, 3 different C libraries, 15 different root file systems, more than 10 
different web servers or web frameworks), the flexibility is mainly in the 
possibilities to modify the root file system, to add configuration files, etc. 
Because of the large amount of freedom, the choice of packages is often limited 
to a few lines. However, with so much freedom, there is no standard way to 
implement some common use cases, e.g., readonly root file system with a separate 
partition for persistence, or a verified boot process with hardware TPM. It is 
possible to implement all these things, but you have to figure out how.

There are some projects that use buildroot to build a more managed (and thus 
less flexible) "distro": SkiffOS, dahliaOS, Batocera Linux, Home Assistant 
Operating System.

For security issues, buildroot has a system that automatically detects CVEs. The 
list of packages is updated weekly on 
http://autobuild.buildroot.net/stats/master.html. A mail is also sent to the 
people who signed up as developers for the package.
In addition, Buildroot offers the classical global hardening options: stack 
smashing protection, PIC/PIE, RELRO, buffer overflow detection.

Internally, Buildroot uses GNU make to drive the build, supported by shell 
scripts and some C code (mostly copied and adapted from elsewhere).
The infrastructure consists of about 5000 lines GNU make, 5000 lines shell 
scripts, and 20000 lines C code (mainly for the Kconfig infrastructure). The 
make code is not so easy to understand (because make is a strange language), but 
it is still manageable. Moreover, adjustments to the core code are rarely needed 
- most evolution is in the packages themselves.

Buildroot is maintained by volunteers. There are 4 maintainers, i.e. people with 
commit access. However, the biggest contribution is made by about 100 other 
people who add about 1500 changesets each release. Releases are made every 3 
months (20XX.02, 20XX.05, 20XX.08, 20XX.11). The last month of a cycle is a 
stability phase where only fixes are applied - but new evolutions still end up 
in the 'next' branch. Twice a year there is a developer meeting, although due to 
the known circumstances this has not taken place since February 2020. The 
community is mainly on the mailing list (lists.buildroot.org) and on IRC 
(#buildroot on OFTC).


_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Article for the German Linux magazine

[Romain Naour]

Hello Arnout,

Le 13/05/2022 à 10:45, Arnout Vandecappelle a écrit :
>  Hi all,
> 
>  About two weeks ago, we got a mail on the list asking for a volunteer to write
> an introductory article about Buildroot for the IoT-themed issue of the German
> Linux magazine. I volunteered two rite such an article.
> 
>  I have an outline or rough draft below. It's just bullet points (though I ended
> up writing sentences for most of it). I'd like to get feedback on:
> 
> - Is there anything missing that I should mention in the article?
> 
> - Is there something in here which is incorrect?
> 
> - Is there something here that we actually don't want to draw attention to?
> 
>  Please *do not* pay attention to style or spelling or little details that are
> missing.
> 
> 
>  Thank you for your help!
> 
> 
>  Regards,
>  Arnout
> 
> 
> Buildroot for IoT
> 
> For IoT distro is important:
> - getting a basic system up and running quickly;
> - small footprint (sometimes only 64MB of flash)
> - control over what you build;
> - support for all major cloud services
> - easy integration of own components;
> - attention for security issues.
> 
> Buildroot is a build system that grew out of uClinux and busybox, so with a
> focus on very small systems.
> Meanwhile, it supports all kinds of different use cases, ranging from (still)
> systems with only a few MB of flash, over docker images for cloud deployments,
> to complex systems with a graphical stack, database, etc.
> The principles of Buildroot are
> - simplicity;
> - flexibility to build exactly the system you want;
> - reproducibility: if you rebuild the same configuration years later, you should
> get the same result bit-for-bit;
> - maintainability.
> 
> With Buildroot, you can build a system by choosing which packages you want and
> some other options in a menu system (based on that of the Linux kernel).
> On that basis, Buildroot will download and build the source code of the selected
> packages and compile it into the root file system.
> You can also add your own extra files, for example configuration files or your
> own application.
> To get you started, there are sample configurations for about 230 boards that
> you can buy off-the-shelf, plus about 40 configurations for simulation in qemu.
> However, it is easy enough to make your own configuration, as long as you know
> which CPU variant you have and which bootloader and kernel options (device tree)
> to use.
> 
> In practice, however, you will almost always want to add your own application.
> For that, you can generate an SDK that you can use to build the application,
> e.g. in an IDE.
> But it is also very easy to create a new package for your application.
> If you use one of the supported build systems (meson, cmake, autotools, qmake,
> waf, cargo, go, python, rebar), a few lines are usually enough.
> 
> Apart from the flexibility of the choice of packages (e.g. 4 different init
> systems, 3 different C libraries, 15 different root file systems, more than 10
> different web servers or web frameworks), the flexibility is mainly in the
> possibilities to modify the root file system, to add configuration files, etc.
> Because of the large amount of freedom, the choice of packages is often limited
> to a few lines. However, with so much freedom, there is no standard way to
> implement some common use cases, e.g., readonly root file system with a separate
> partition for persistence, or a verified boot process with hardware TPM. It is
> possible to implement all these things, but you have to figure out how.
> 
> There are some projects that use buildroot to build a more managed (and thus
> less flexible) "distro": SkiffOS, dahliaOS, Batocera Linux, Home Assistant
> Operating System.

It would be nice if we can add recalbox project in this list :)

https://github.com/recalbox/recalbox-os

Best regards,
Romain

> 
> For security issues, buildroot has a system that automatically detects CVEs. The
> list of packages is updated weekly on
> http://autobuild.buildroot.net/stats/master.html. A mail is also sent to the
> people who signed up as developers for the package.
> In addition, Buildroot offers the classical global hardening options: stack
> smashing protection, PIC/PIE, RELRO, buffer overflow detection.
> 
> Internally, Buildroot uses GNU make to drive the build, supported by shell
> scripts and some C code (mostly copied and adapted from elsewhere).
> The infrastructure consists of about 5000 lines GNU make, 5000 lines shell
> scripts, and 20000 lines C code (mainly for the Kconfig infrastructure). The
> make code is not so easy to understand (because make is a strange language), but
> it is still manageable. Moreover, adjustments to the core code are rarely needed
> - most evolution is in the packages themselves.
> 
> Buildroot is maintained by volunteers. There are 4 maintainers, i.e. people with
> commit access. However, the biggest contribution is made by about 100 other
> people who add about 1500 changesets each release. Releases are made every 3
> months (20XX.02, 20XX.05, 20XX.08, 20XX.11). The last month of a cycle is a
> stability phase where only fixes are applied - but new evolutions still end up
> in the 'next' branch. Twice a year there is a developer meeting, although due to
> the known circumstances this has not taken place since February 2020. The
> community is mainly on the mailing list (lists.buildroot.org) and on IRC
> (#buildroot on OFTC).
> 
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Article for the German Linux magazine

[Arnout Vandecappelle]

On 13/05/2022 11:02, Romain Naour wrote:
> Hello Arnout,
> 
> Le 13/05/2022 à 10:45, Arnout Vandecappelle a écrit :

[snip]
>> There are some projects that use buildroot to build a more managed (and thus
>> less flexible) "distro": SkiffOS, dahliaOS, Batocera Linux, Home Assistant
>> Operating System.
> 
> It would be nice if we can add recalbox project in this list :)
> 
> https://github.com/recalbox/recalbox-os

  Ah, I thought that that one was superseded by batocera, but apparently both 
are still active. I'll add it.

  Regards,
  Arnout

[snip]
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Article for the German Linux magazine

[Baruch Siach via buildroot]

Hi Arnout,

On Fri, May 13 2022, Arnout Vandecappelle wrote:
>  About two weeks ago, we got a mail on the list asking for a volunteer to
> write an introductory article about Buildroot for the IoT-themed issue of the
> German Linux magazine. I volunteered two rite such an article.

Thanks.

>  I have an outline or rough draft below. It's just bullet points (though I
> ended up writing sentences for most of it). I'd like to get feedback on:
>
> - Is there anything missing that I should mention in the article?
>
> - Is there something in here which is incorrect?
>
> - Is there something here that we actually don't want to draw attention to?
>
>  Please *do not* pay attention to style or spelling or little details that are
> missing.
>
>
>  Thank you for your help!
>
>
>  Regards,
>  Arnout
>
>
> Buildroot for IoT
>
> For IoT distro is important:
> - getting a basic system up and running quickly;
> - small footprint (sometimes only 64MB of flash)
> - control over what you build;
> - support for all major cloud services
> - easy integration of own components;
> - attention for security issues.
>
> Buildroot is a build system that grew out of uClinux and busybox, so with a
> focus on very small systems.
> Meanwhile, it supports all kinds of different use cases, ranging from (still)
> systems with only a few MB of flash, over docker images for cloud deployments, 
> to complex systems with a graphical stack, database, etc.
> The principles of Buildroot are
> - simplicity;
> - flexibility to build exactly the system you want;
> - reproducibility: if you rebuild the same configuration years later, you
>   should get the same result bit-for-bit;

"bit-for-bit" is a bold statement. I'm not sure we are quite there
yet. Maybe just "should get the same result".

> - maintainability.
>
> With Buildroot, you can build a system by choosing which packages you want and
> some other options in a menu system (based on that of the Linux kernel).
> On that basis, Buildroot will download and build the source code of the
> selected packages and compile it into the root file system.

Might be worth mention that Buildroot also (optionally) builds the
toolchain for you, optimized specifically for your target platform
variant. You mention the SDK below, but it is framed as a tool for
external software (which it is).

Maybe also say something about cross compilation in general.

> You can also add your own extra files, for example configuration files or your
> own application.
> To get you started, there are sample configurations for about 230 boards that
> you can buy off-the-shelf, plus about 40 configurations for simulation in
> qemu.
> However, it is easy enough to make your own configuration, as long as you know
> which CPU variant you have and which bootloader and kernel options (device
> tree) to use.
>
> In practice, however, you will almost always want to add your own application.
> For that, you can generate an SDK that you can use to build the application,
> e.g. in an IDE.
> But it is also very easy to create a new package for your application.
> If you use one of the supported build systems (meson, cmake, autotools, qmake,
> waf, cargo, go, python, rebar), a few lines are usually enough.

Maybe mention BR2_EXTERNAL here. That is, we have facilities that are
meant to make integration of external software easier.

> Apart from the flexibility of the choice of packages (e.g. 4 different init
> systems, 3 different C libraries, 15 different root file systems, more than 10 
> different web servers or web frameworks), the flexibility is mainly in the
> possibilities to modify the root file system, to add configuration files,
> etc. Because of the large amount of freedom, the choice of packages is often
> limited to a few lines.

What does that mean?

> However, with so much freedom, there is no standard
> way to implement some common use cases, e.g., readonly root file system with a
> separate partition for persistence, or a verified boot process with hardware
> TPM. It is possible to implement all these things, but you have to figure out
> how.

Alternative phrasing that emphasizes Buildroot's flexibility:

  The flexibility that Buildroot provides allows you to integrate features that 
  are not standardized, like read only root filesystems with separate writable
  partitions, or secure boot. It does not save you from the hard work of
  figuring out any platform specific requirements, but Buildroot is not
  in your way.

> There are some projects that use buildroot to build a more managed (and thus
> less flexible) "distro": SkiffOS, dahliaOS, Batocera Linux, Home Assistant 
> Operating System.
>
> For security issues, buildroot has a system that automatically detects
> CVEs. The list of packages is updated weekly on 
> http://autobuild.buildroot.net/stats/master.html. A mail is also sent to the
> people who signed up as developers for the package.
> In addition, Buildroot offers the classical global hardening options: stack
> smashing protection, PIC/PIE, RELRO, buffer overflow detection.
>
> Internally, Buildroot uses GNU make to drive the build, supported by shell
> scripts and some C code (mostly copied and adapted from elsewhere).
> The infrastructure consists of about 5000 lines GNU make, 5000 lines shell
> scripts, and 20000 lines C code (mainly for the Kconfig infrastructure). The 
> make code is not so easy to understand (because make is a strange language),
> but it is still manageable. Moreover, adjustments to the core code are rarely
> needed - most evolution is in the packages themselves.

Core Buildroot make code is complex indeed. But most developer or users never 
see this code. Specific package build code is not hard to grasp. Maybe add an 
example package?

> Buildroot is maintained by volunteers. There are 4 maintainers, i.e. people
> with commit access. However, the biggest contribution is made by about 100
> other people who add about 1500 changesets each release. Releases are made
> every 3 months (20XX.02, 20XX.05, 20XX.08, 20XX.11). The last month of a cycle
> is a stability phase where only fixes are applied - but new evolutions still
> end up in the 'next' branch. Twice a year there is a developer meeting,
> although due to the known circumstances this has not taken place since
> February 2020. The community is mainly on the mailing list
> (lists.buildroot.org) and on IRC (#buildroot on OFTC).

A few more topics that might be worth mention:

  * Buildroot manual

  * LTS releases

  * Autobuilders

  * Tests infrastructure

  * License information

Hope this helps,
baruch

-- 
                                                     ~. .~   Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
   - baruch@tkos.co.il - tel: +972.52.368.4656, http://www.tkos.co.il -
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Article for the German Linux magazine

[Arnout Vandecappelle]

On 13/05/2022 12:33, Baruch Siach wrote:
> Hi Arnout,
> 
> On Fri, May 13 2022, Arnout Vandecappelle wrote:

[snip]
>> Apart from the flexibility of the choice of packages (e.g. 4 different init
>> systems, 3 different C libraries, 15 different root file systems, more than 10
>> different web servers or web frameworks), the flexibility is mainly in the
>> possibilities to modify the root file system, to add configuration files,
>> etc. Because of the large amount of freedom, the choice of packages is often
>> limited to a few lines.
> 
> What does that mean?

  Err, I have no idea... Maybe it means it was late at night when I wrote that? :-)


[snip]
> Hope this helps,

  All other points are very good, it will certainly help, thank you!

  Regards,
  Arnout

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Article for the German Linux magazine

[Alexander Dahl]

[-- Attachment #1.1: Type: text/plain, Size: 6114 bytes --]

Hei hei,

On Fri, May 13, 2022 at 10:45:20AM +0200, Arnout Vandecappelle wrote:
>  Hi all,
> 
>  About two weeks ago, we got a mail on the list asking for a volunteer to
> write an introductory article about Buildroot for the IoT-themed issue of
> the German Linux magazine. I volunteered two rite such an article.
> 
>  I have an outline or rough draft below. It's just bullet points (though I
> ended up writing sentences for most of it). I'd like to get feedback on:
> 
> - Is there anything missing that I should mention in the article?
> 
> - Is there something in here which is incorrect?
> 
> - Is there something here that we actually don't want to draw attention to?
> 
>  Please *do not* pay attention to style or spelling or little details that
> are missing.
> 
> 
>  Thank you for your help!
> 
> 
>  Regards,
>  Arnout
> 
> 
> Buildroot for IoT
> 
> For IoT distro is important:
> - getting a basic system up and running quickly;
> - small footprint (sometimes only 64MB of flash)
> - control over what you build;
> - support for all major cloud services
> - easy integration of own components;
> - attention for security issues.
> 
> Buildroot is a build system that grew out of uClinux and busybox, so with a
> focus on very small systems.
> Meanwhile, it supports all kinds of different use cases, ranging from
> (still) systems with only a few MB of flash, over docker images for cloud
> deployments, to complex systems with a graphical stack, database, etc.
> The principles of Buildroot are
> - simplicity;
> - flexibility to build exactly the system you want;
> - reproducibility: if you rebuild the same configuration years later, you
> should get the same result bit-for-bit;
> - maintainability.
> 
> With Buildroot, you can build a system by choosing which packages you want
> and some other options in a menu system (based on that of the Linux kernel).
> On that basis, Buildroot will download and build the source code of the
> selected packages and compile it into the root file system.
> You can also add your own extra files, for example configuration files or
> your own application.
> To get you started, there are sample configurations for about 230 boards
> that you can buy off-the-shelf, plus about 40 configurations for simulation
> in qemu.
> However, it is easy enough to make your own configuration, as long as you
> know which CPU variant you have and which bootloader and kernel options
> (device tree) to use.
> 
> In practice, however, you will almost always want to add your own application.
> For that, you can generate an SDK that you can use to build the application,
> e.g. in an IDE.
> But it is also very easy to create a new package for your application.
> If you use one of the supported build systems (meson, cmake, autotools,
> qmake, waf, cargo, go, python, rebar), a few lines are usually enough.
> 
> Apart from the flexibility of the choice of packages (e.g. 4 different init
> systems, 3 different C libraries, 15 different root file systems, more than
> 10 different web servers or web frameworks), the flexibility is mainly in
> the possibilities to modify the root file system, to add configuration
> files, etc. Because of the large amount of freedom, the choice of packages
> is often limited to a few lines. However, with so much freedom, there is no
> standard way to implement some common use cases, e.g., readonly root file
> system with a separate partition for persistence, or a verified boot process
> with hardware TPM. It is possible to implement all these things, but you
> have to figure out how.
> 
> There are some projects that use buildroot to build a more managed (and thus
> less flexible) "distro": SkiffOS, dahliaOS, Batocera Linux, Home Assistant
> Operating System.

fli4l [1] is also using buildroot, but development has stalled in the
last two years after we could not do on site meetings anymore. :-/

Greets
Alex

[1] http://www.fli4l.de/en/home/whats-that-fli4l/

> 
> For security issues, buildroot has a system that automatically detects CVEs.
> The list of packages is updated weekly on
> http://autobuild.buildroot.net/stats/master.html. A mail is also sent to the
> people who signed up as developers for the package.
> In addition, Buildroot offers the classical global hardening options: stack
> smashing protection, PIC/PIE, RELRO, buffer overflow detection.
> 
> Internally, Buildroot uses GNU make to drive the build, supported by shell
> scripts and some C code (mostly copied and adapted from elsewhere).
> The infrastructure consists of about 5000 lines GNU make, 5000 lines shell
> scripts, and 20000 lines C code (mainly for the Kconfig infrastructure). The
> make code is not so easy to understand (because make is a strange language),
> but it is still manageable. Moreover, adjustments to the core code are
> rarely needed - most evolution is in the packages themselves.
> 
> Buildroot is maintained by volunteers. There are 4 maintainers, i.e. people
> with commit access. However, the biggest contribution is made by about 100
> other people who add about 1500 changesets each release. Releases are made
> every 3 months (20XX.02, 20XX.05, 20XX.08, 20XX.11). The last month of a
> cycle is a stability phase where only fixes are applied - but new evolutions
> still end up in the 'next' branch. Twice a year there is a developer
> meeting, although due to the known circumstances this has not taken place
> since February 2020. The community is mainly on the mailing list
> (lists.buildroot.org) and on IRC (#buildroot on OFTC).
> 
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
/"\ ASCII RIBBON | »With the first link, the chain is forged. The first
\ / CAMPAIGN     | speech censured, the first thought forbidden, the
 X  AGAINST      | first freedom denied, chains us all irrevocably.«
/ \ HTML MAIL    | (Jean-Luc Picard, quoting Judge Aaron Satie)

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 150 bytes --]

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Article for the German Linux magazine

[Heiko Thiery]

[-- Attachment #1.1: Type: text/plain, Size: 93 bytes --]

Hi,

What about bootloaders and the generation of complete deployable Images?

--
Heiko

>
>

[-- Attachment #1.2: Type: text/html, Size: 389 bytes --]

[-- Attachment #2: Type: text/plain, Size: 150 bytes --]

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot