[Buildroot] [PATCH 0 of 3 v2 for 2014.08] Manual improvements part 4

[Buildroot] [PATCH 0 of 3 v2 for 2014.08] Manual improvements part 4

[Thomas De Schampheleire]

This short series starts improving the 'Project-specific customizations'
chapter in the manual. Subsequent patches are planned (even for 2014.08) but
since the release is almost there I'm sending out patches as soon as they
are available, to give enough time for review.

Further changes planned:
- reworking the 'Customizing the generated target filesystem' section to
  emphasize the recommendations of rootfs-overlay + post-build
- rework the rest of the chapter as there is some duplicate and inconsistent
  information.

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>

---
Thomas De Schampheleire (3)
  manual/user guide/customization: expand introduction text
  manual/user guide/customization: add section with recommended dir structure
  manual/user guide/customization: rework section on BR2_EXTERNAL

 docs/manual/customize-directory-structure.txt |  59 ++++++++++++
 docs/manual/customize-outside-br.txt          |  90 ++++++------------
 docs/manual/customize.txt                     |  39 ++++++-
 3 files changed, 121 insertions(+), 67 deletions(-)

[Buildroot] [PATCH 1 of 3 v2 for 2014.08] manual/user guide/customization: expand introduction text

[Thomas De Schampheleire]

This patch expands the introduction text of chapter 'Project-specific
customization' by listing the type of customizations that can be performed
by a project.

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>

---
v2: reword 'from a clean buildroot tree' --> 'after running make clean' to
    avoid confusion (Samuel)

 docs/manual/customize.txt |  25 +++++++++++++++++++++----
 1 files changed, 21 insertions(+), 4 deletions(-)

diff --git a/docs/manual/customize.txt b/docs/manual/customize.txt
--- a/docs/manual/customize.txt
+++ b/docs/manual/customize.txt
@@ -3,11 +3,28 @@
 
 == Project-specific customization
 
-The following sections describe the various way in which you can
-customize Buildroot for a given project.
+Typical actions you may need to perform for a given project are:
 
-For instructions on how to add new packages to Buildroot, refer to
-xref:adding-packages[]
+- configuring Buildroot (including build options and toolchain,
+  bootloader, kernel, package and filesystem image type selection)
+- configuring other components, like the Linux kernel and BusyBox
+- adding project-specific patches to some packages (using
+  +BR2_GLOBAL_PATCH_DIR+)
+- adding or overwriting files on the target filesystem (using
+  +BR2_ROOTFS_OVERLAY+)
+- modifying or deleting files on the target filesystem (using
+  +BR2_ROOTFS_POST_BUILD_SCRIPT+)
+- running arbitrary commands prior to generating the filesystem image
+  (using +BR2_ROOTFS_POST_BUILD_SCRIPT+)
+- running arbitrary commands after generating the filesystem image
+  (using +BR2_ROOTFS_POST_IMAGE_SCRIPT+)
+- adding project-specific packages
+
+This chapter describes how to make such project-specific customizations
+in Buildroot and how to store them in a way that you can build the same
+image in a reproducible way, even after running 'make clean'. By
+following the recommended strategy, you can even use the same Buildroot
+tree to build multiple distinct projects!
 
 include::customize-rootfs.txt[]
 

[Buildroot] [PATCH 2 of 3 v2 for 2014.08] manual/user guide/customization: add section with recommended dir structure

[Thomas De Schampheleire]

Although the user is free to choose where to put project-specific files, the
presentation 'Using Buildroot for real projects' [1] first described a
certain recommendation.
This patch briefly describes this recommendation in the form of a directory
structure overview, the details will be added in subsequent patches.

Since the section on BR2_EXTERNAL is logically connected to the directory
structure, this section is moved right after it.

[1] http://elinux.org/images/2/2a/Using-buildroot-real-project.pdf


Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>

---
v2: no changes

 docs/manual/customize-directory-structure.txt |  58 +++++++++++++++++++
 docs/manual/customize.txt                     |   6 +-
 2 files changed, 62 insertions(+), 2 deletions(-)

diff --git a/docs/manual/customize-directory-structure.txt b/docs/manual/customize-directory-structure.txt
new file mode 100644
--- /dev/null
+++ b/docs/manual/customize-directory-structure.txt
@@ -0,0 +1,58 @@
+// -*- mode:doc; -*-
+// vim: set syntax=asciidoc:
+
+=== Recommended directory structure
+
+When customizing Buildroot for your project, you will be creating one or
+more project-specific files that need to be stored somewhere. While most
+of these files could be placed in _any_ location as their path is to be
+specified in the Buildroot configuration, the Buildroot developers
+recommend a specific directory structure which is described in this
+section.
+
+Orthogonal to this directory structure, you can choose _where_ you place
+this structure itself: either inside the Buildroot tree, or outside of
+it using +BR2_EXTERNAL+. Both options are valid, the choice is up to you.
+
+-----
++-- board/
+|   +-- <company>/
+|       +-- <boardname>/
+|           +-- linux.config
+|           +-- busybox.config
+|           +-- <other configuration files>
+|           +-- post_build.sh
+|           +-- post_image.sh
+|           +-- rootfs_overlay/
+|           |   +-- etc/
+|           |   +-- <some file>
+|           +-- patches/
+|               +-- foo/
+|               |   +-- <some patch>
+|               +-- libbar/
+|                   +-- <some other patches>
+|
++-- configs/
+|   +-- <boardname>_defconfig
+|
++-- package/
+|   +-- <company>/
+|       +-- Config.in (if not using BR2_EXTERNAL)
+|       +-- <company>.mk (if not using BR2_EXTERNAL)
+|       +-- <boardname>/
+|           +-- package1/
+|           |    +-- Config.in
+|           |    +-- package1.mk
+|           +-- package2/
+|               +-- Config.in
+|               +-- package2.mk
+|
++-- Config.in (if using BR2_EXTERNAL)
++-- external.mk (if using BR2_EXTERNAL)
+------
+
+Details on the files shown above are given further in this chapter.
+
+Note: if you choose to place this structure outside of the Buildroot
+tree using +BR2_EXTERNAL+, the <company> and possibly <boardname>
+components may be superfluous and can be left out.
diff --git a/docs/manual/customize.txt b/docs/manual/customize.txt
--- a/docs/manual/customize.txt
+++ b/docs/manual/customize.txt
@@ -26,10 +26,12 @@
 following the recommended strategy, you can even use the same Buildroot
 tree to build multiple distinct projects!
 
+include::customize-directory-structure.txt[]
+
+include::customize-outside-br.txt[]
+
 include::customize-rootfs.txt[]
 
 include::customize-store.txt[]
 
 include::customize-packages.txt[]
-
-include::customize-outside-br.txt[]

[Buildroot] [PATCH 3 of 3 v2 for 2014.08] manual/user guide/customization: rework section on BR2_EXTERNAL

[Thomas De Schampheleire]

This patch reworks the section on BR2_EXTERNAL as follows:
- move note about upstreaming to the chapter introduction
- streamline the section with the previously added section 'Recommended
  directory structure', avoiding duplication.
- use $(BR2_EXTERNAL) rather than BR2_EXTERNAL when referring to file paths.
- some general rewording


Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>

---
v2: new patch

 docs/manual/customize-directory-structure.txt |   1 +
 docs/manual/customize-outside-br.txt          |  90 ++++++------------
 docs/manual/customize.txt                     |   8 +
 3 files changed, 38 insertions(+), 61 deletions(-)

diff --git a/docs/manual/customize-directory-structure.txt b/docs/manual/customize-directory-structure.txt
--- a/docs/manual/customize-directory-structure.txt
+++ b/docs/manual/customize-directory-structure.txt
@@ -1,6 +1,7 @@
 // -*- mode:doc; -*-
 // vim: set syntax=asciidoc:
 
+[[customize-dir-structure]]
 === Recommended directory structure
 
 When customizing Buildroot for your project, you will be creating one or
diff --git a/docs/manual/customize-outside-br.txt b/docs/manual/customize-outside-br.txt
--- a/docs/manual/customize-outside-br.txt
+++ b/docs/manual/customize-outside-br.txt
@@ -1,30 +1,25 @@
 // -*- mode:doc -*- ;
 
 [[outside-br-custom]]
-=== Keeping customizations outside Buildroot
+=== Keeping customizations outside of Buildroot
 
-The Buildroot community recommends and encourages upstreaming to the
-official Buildroot version the packages and board support that are
-written by developers. However, it is sometimes not possible or
-desirable because some of these packages or board support are highly
-specific or proprietary.
+As already briefly mentioned in xref:customize-dir-structure[], you can
+place project-specific customizations in two locations:
 
-In this case, Buildroot users are offered two choices:
+ * directly within the Buildroot tree, typically maintaining them using
+   branches in a version control system so that upgrading to a newer
+   Buildroot release is easy.
 
- * They can add their packages, board support and configuration files
-   directly within the Buildroot tree, and maintain them by using
-   branches in a version control system.
-
- * They can use the +BR2_EXTERNAL+ mechanism, which allows to keep
-   package recipes, board support and configuration files outside of
-   the Buildroot tree, while still having them nicely integrated in
-   the build logic. The following paragraphs give details on how to
-   use +BR2_EXTERNAL+.
+ * outside of the Buildroot tree, using the +BR2_EXTERNAL+ mechanism.
+   This mechanism allows to keep package recipes, board support and
+   configuration files outside of the Buildroot tree, while still
+   having them nicely integrated in the build logic. This section
+   explains how to use +BR2_EXTERNAL+.
 
 +BR2_EXTERNAL+ is an environment variable that can be used to point to
 a directory that contains Buildroot customizations. It can be passed
 to any Buildroot +make+ invocation. It is automatically saved in the
-hidden +.br-external+ file in the output directory. By doing this,
+hidden +.br-external+ file in the output directory. Thanks to this,
 there is no need to pass +BR2_EXTERNAL+ at every +make+ invocation. It
 can however be changed at any time by passing a new value, and can be
 removed by passing an empty value.
@@ -32,7 +27,7 @@
 *Note:* the +BR2_EXTERNAL+ path can be either an absolute or a relative path,
 but if it's passed as a relative path, it is important to note that it
 is interpreted relative to the main Buildroot source directory, *not*
-the Buildroot output directory.
+to the Buildroot output directory.
 
 Some examples:
 
@@ -40,7 +35,7 @@
  buildroot/ $ make BR2_EXTERNAL=/path/to/foobar menuconfig
 -----
 
-Starting from now on, external definitions from the +/path/to/foobar+
+From now on, external definitions from the +/path/to/foobar+
 directory will be used:
 
 -----
@@ -60,7 +55,7 @@
  buildroot/ $ make BR2_EXTERNAL= xconfig
 -----
 
-+BR2_EXTERNAL+ then allows three different things:
++BR2_EXTERNAL+ allows three different things:
 
  * One can store all the board-specific configuration files there,
    such as the kernel configuration, the root filesystem overlay, or
@@ -72,63 +67,36 @@
    filesystem overlay), or the +BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE+
    Buildroot option to
    +$(BR2_EXTERNAL)/board/<boardname>/kernel.config+ (to specify the
-   location of the kernel configuration file). To achieve this, it is
-   recommended but not mandatory, to store those details in
-   directories called +board/<boardname>/+ under +BR2_EXTERNAL+. This
-   matches the directory structure used within Buildroot.
+   location of the kernel configuration file).
 
  * One can store package recipes (i.e. +Config.in+ and
    +<packagename>.mk+), or even custom configuration options and make
-   logic. Buildroot automatically includes +BR2_EXTERNAL/Config.in+ to
+   logic. Buildroot automatically includes +$(BR2_EXTERNAL)/Config.in+ to
    make it appear in the top-level configuration menu, and includes
-   +BR2_EXTERNAL/external.mk+ with the rest of the makefile logic.
+   +$(BR2_EXTERNAL)/external.mk+ with the rest of the makefile logic.
    Providing those two files is mandatory, but they can be empty.
 +
 The main usage of this is to store package recipes. The recommended
-   way to do this is to write a +BR2_EXTERNAL/Config.in+ that looks
-   like:
+   way to do this is to write a +$(BR2_EXTERNAL)/Config.in+ file that
+   looks like:
 +
 ------
-source "$BR2_EXTERNAL/package/package1/Config.in"
-source "$BR2_EXTERNAL/package/package2/Config.in"
+source "$BR2_EXTERNAL/package/<boardname>/package1/Config.in"
+source "$BR2_EXTERNAL/package/<boardname>/package2/Config.in"
 ------
 +
-Then, have a +BR2_EXTERNAL/external.mk+ file that looks like:
+Then, have a +$(BR2_EXTERNAL)/external.mk+ file that looks like:
 +
 ------
-include $(sort $(wildcard $(BR2_EXTERNAL)/package/*/*.mk))
+include $(sort $(wildcard $(BR2_EXTERNAL)/package/*/*/*.mk))
 ------
 +
-And then in +BR2_EXTERNAL/package/package1+ and
-   +BR2_EXTERNAL/package/package2+ create normal Buildroot package
-   recipes, as explained in xref:adding-packages[].
+And then in +$(BR2_EXTERNAL)/package/<boardname>/package1+ and
+   +$(BR2_EXTERNAL)/package/<boardname>/package2+ create normal Buildroot
+   package recipes, as explained in xref:adding-packages[].
 
  * One can store Buildroot defconfigs in the +configs+ subdirectory of
-   +BR2_EXTERNAL+. Buildroot will automatically show them in the
+   +$(BR2_EXTERNAL)+. Buildroot will automatically show them in the
    output of +make help+ and allow them to be loaded with the normal
    +make <name>_defconfig+ command. They will be visible under the
-   +User-provided configs:+' label in the 'make help' output.
-
-In the end, a typical +BR2_EXTERNAL+ directory organization would
-generally be:
-
------
-$(BR2_EXTERNAL)/
-+-- Config.in
-+-- external.mk
-+-- board/
-|   +-- <boardname>/
-|       +-- linux.config
-|       +-- overlay/
-|           +-- etc/
-|               +-- <some file>
-+-- configs/
-|   +-- <boardname>_defconfig
-+-- package/
-    +-- package1/
-    |    +-- Config.in
-    |    +-- package1.mk
-    +-- package2/
-        +-- Config.in
-        +-- package2.mk
-------
+   +User-provided configs+' label in the 'make help' output.
diff --git a/docs/manual/customize.txt b/docs/manual/customize.txt
--- a/docs/manual/customize.txt
+++ b/docs/manual/customize.txt
@@ -20,6 +20,14 @@
   (using +BR2_ROOTFS_POST_IMAGE_SCRIPT+)
 - adding project-specific packages
 
+An important note regarding such 'project-specific' customizations:
+please carefully consider which changes are indeed project-specific and
+which changes are also useful to developers outside your project. The
+Buildroot community highly recommends and encourages the upstreaming of
+improvements, packages and board support to the official Buildroot
+project. Of course, it is sometimes not possible or desirable to
+upstream because the changes are highly specific or proprietary.
+
 This chapter describes how to make such project-specific customizations
 in Buildroot and how to store them in a way that you can build the same
 image in a reproducible way, even after running 'make clean'. By

[Buildroot] [PATCH 0 of 3 v2 for 2014.08] Manual improvements part 4

[Peter Korsgaard]

>>>>> "Thomas" == Thomas De Schampheleire <patrickdepinguin@gmail.com> writes:

 > This short series starts improving the 'Project-specific customizations'
 > chapter in the manual. Subsequent patches are planned (even for 2014.08) but
 > since the release is almost there I'm sending out patches as soon as they
 > are available, to give enough time for review.

 > Further changes planned:
 > - reworking the 'Customizing the generated target filesystem' section to
 >   emphasize the recommendations of rootfs-overlay + post-build
 > - rework the rest of the chapter as there is some duplicate and inconsistent
 >   information.

 > Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>

Committed series, thanks!

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH 0 of 3 v2 for 2014.08] Manual improvements part 4

[Thomas De Schampheleire]

Peter Korsgaard <jacmet@uclibc.org> schreef:
>>>>>> "Thomas" == Thomas De Schampheleire <patrickdepinguin@gmail.com> writes:
>
> > This short series starts improving the 'Project-specific customizations'
> > chapter in the manual. Subsequent patches are planned (even for 2014.08) but
> > since the release is almost there I'm sending out patches as soon as they
> > are available, to give enough time for review.
>
> > Further changes planned:
> > - reworking the 'Customizing the generated target filesystem' section to
> >   emphasize the recommendations of rootfs-overlay + post-build
> > - rework the rest of the chapter as there is some duplicate and inconsistent
> >   information.
>
> > Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
>
>Committed series, thanks!
>

Thanks! It is likely that this will be the last manual
 update for 2014.08, as I'm not sure I will get around
 doing the other planned changes before the release.

Best regards,
Thomas

[Buildroot] [PATCH 0 of 3 v2 for 2014.08] Manual improvements part 4

[Peter Korsgaard]

>>>>> "Thomas" == Thomas De Schampheleire <patrickdepinguin@gmail.com> writes:

 > Thanks! It is likely that this will be the last manual
 >  update for 2014.08, as I'm not sure I will get around
 >  doing the other planned changes before the release.

That's ok. I plan on releasing 2014.08 tomorrow, so there isn't much
time left anyway.

-- 
Bye, Peter Korsgaard