[Buildroot] Post-image script feature and other related fixes

[Buildroot] Post-image script feature and other related fixes

[Thomas Petazzoni]

Hello,

For quite a while, I had this post-image patch sitting around in one
of my customer's project, and the last mail from Stephan reminded me
that it was probably time to clean it up, update it and send it
upstream. That's the first two patches.

Then, in patches 3 and 4, I remove the package/customize stuff,
deprecated since a long time.

And in patch 5, I slightly update/improve the documentation about the
various solutions to customize the root filesystem.

Best regards,

Thomas

[Buildroot] [PATCH 1/5] Add a post-image script mechanism

[Thomas Petazzoni]

Just like we have a post-build script mechanism that gets executed
after the build of all packages but before the creation of the
filesystem images, let's introduce a post-image script mechanism, that
gets executed once all filesystem images have been generated.

This can for example be used to call a tool building a firmware image
from different images generated by Buildroot, or automatically extract
the tarball root filesystem image into some location exported by NFS,
or any other custom action.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 Makefile         |    9 +++++++++
 system/Config.in |   18 ++++++++++++++++++
 2 files changed, 27 insertions(+)

diff --git a/Makefile b/Makefile
index 6f8ed0e..88b5a85 100644
--- a/Makefile
+++ b/Makefile
@@ -353,6 +353,8 @@ endif
 
 include fs/common.mk
 
+TARGETS+=target-post-image
+
 TARGETS_CLEAN:=$(patsubst %,%-clean,$(TARGETS))
 TARGETS_SOURCE:=$(patsubst %,%-source,$(TARGETS) $(BASE_TARGETS))
 TARGETS_DIRCLEAN:=$(patsubst %,%-dirclean,$(TARGETS))
@@ -548,6 +550,13 @@ target-generatelocales: host-localedef
 	done
 endif
 
+target-post-image:
+ifneq ($(BR2_ROOTFS_POST_IMAGE_SCRIPT),)
+	@$(call MESSAGE,"Executing post-image script\(s\)")
+	@$(foreach s, $(call qstrip,$(BR2_ROOTFS_POST_IMAGE_SCRIPT)), \
+		$(s) $(BINARIES_DIR)$(sep))
+endif
+
 toolchain-eclipse-register:
 	./support/scripts/eclipse-register-toolchain `readlink -f $(O)` $(notdir $(TARGET_CROSS)) $(BR2_ARCH)
 
diff --git a/system/Config.in b/system/Config.in
index 69863c4..1ffe8a0 100644
--- a/system/Config.in
+++ b/system/Config.in
@@ -265,4 +265,22 @@ config BR2_ROOTFS_POST_BUILD_SCRIPT
 	  only argument. Make sure the exit code of those scripts are 0,
 	  otherwise make will stop after calling them.
 
+config BR2_ROOTFS_POST_IMAGE_SCRIPT
+	string "Custom script to run after creating filesystem images"
+	default ""
+	help
+	  Specify a space-separated list of scripts to be run after
+	  the build has finished and after Buildroot has packed the
+	  files into selected filesystem images.
+
+	  This can for example be used to call a tool building a
+	  firmware image from different images generated by Buildroot,
+	  or automatically extract the tarball root filesystem image
+	  into some location exported by NFS, or any other custom
+	  action.
+
+	  These scripts are called with the images directory name as
+	  first and only argument. The script is executed from the
+	  main Buildroot source directory as the current directory.
+
 endmenu
-- 
1.7.9.5

[Buildroot] [PATCH 2/5] docs/manual: mention the new post-image mechanism

[Thomas Petazzoni]

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 docs/manual/customize-rootfs.txt |   15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/docs/manual/customize-rootfs.txt b/docs/manual/customize-rootfs.txt
index 6399699..66e38dc 100644
--- a/docs/manual/customize-rootfs.txt
+++ b/docs/manual/customize-rootfs.txt
@@ -50,3 +50,18 @@ there are a few ways to customize the resulting target filesystem.
   target root filesystem in +package/customize/source+, and then
   enable this special package in the configuration system. _This
   method is marked as deprecated_.
+
+Note also that if want to perform some specific actions *after* all
+filesystem images have been created (for example to automatically
+extract your root filesystem tarball in a location exported by your
+NFS server, or to create a special firmware image that bundles your
+root filesystem and kernel image, or any other custom action), you can
+specify a list of scripts in the +BR2_ROOTFS_POST_IMAGE_SCRIPT+
+configuration option. Each of those scripts will be called, with the
+path to the +images+ output directory as first and unique argument,
+and will be executed with the main Buildroot source directory as the
+current directory. Also, just like for the _post-build scripts_
+mentionned above, you also have access to the following environment
+variables from your _post-image scripts_: +BUILDROOT_CONFIG+,
++HOST_DIR+, +STAGING_DIR+, +TARGET_DIR+, +BINARIES_DIR+ and
++BASE_DIR+.
-- 
1.7.9.5

[Buildroot] [PATCH 3/5] package/customize: remove

[Thomas Petazzoni]

This mechanism of root filesystem customization has been deprecated
since a long time, so let's remove it now.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 package/Config.in              |    1 -
 package/customize/Config.in    |    7 -------
 package/customize/customize.mk |   32 --------------------------------
 3 files changed, 40 deletions(-)
 delete mode 100644 package/customize/Config.in
 delete mode 100644 package/customize/customize.mk
 delete mode 100644 package/customize/source/.empty

diff --git a/package/Config.in b/package/Config.in
index ba03b3c..5c78784 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -1,7 +1,6 @@
 menu "Package Selection for the target"
 
 source "package/busybox/Config.in"
-source "package/customize/Config.in"
 
 # Audio and video applications
 source "package/multimedia/Config.in"
diff --git a/package/customize/Config.in b/package/customize/Config.in
deleted file mode 100644
index 8f8aaa2..0000000
--- a/package/customize/Config.in
+++ /dev/null
@@ -1,7 +0,0 @@
-config BR2_PACKAGE_CUSTOMIZE
-	bool "customize"
-	depends on BR2_DEPRECATED
-	help
-	  Add custom stuff to your buildroot.
-
-	  (edit package/customize/customize.mk)
diff --git a/package/customize/customize.mk b/package/customize/customize.mk
deleted file mode 100644
index d2b8518..0000000
--- a/package/customize/customize.mk
+++ /dev/null
@@ -1,32 +0,0 @@
-#############################################################
-#
-# Any custom stuff you feel like doing....
-#
-#############################################################
-CUST_DIR:=package/customize/source
-
-$(BUILD_DIR)/.customize:
-	rm -f $(BUILD_DIR)/series
-	(cd $(CUST_DIR); \
-	 /bin/ls -d * > $(BUILD_DIR)/series || \
-	 touch $(BUILD_DIR)/series )
-	for f in `cat $(BUILD_DIR)/series`; do \
-		cp -af $(CUST_DIR)/$$f $(TARGET_DIR); \
-	done
-	rm -f $(BUILD_DIR)/series
-	touch $@
-
-customize: $(BUILD_DIR)/.customize
-
-customize-clean:
-	rm -f $(BUILD_DIR)/.customize
-
-.PHONY: customize
-#############################################################
-#
-# Toplevel Makefile options
-#
-#############################################################
-ifeq ($(BR2_PACKAGE_CUSTOMIZE),y)
-TARGETS+=customize
-endif
diff --git a/package/customize/source/.empty b/package/customize/source/.empty
deleted file mode 100644
index e69de29..0000000
-- 
1.7.9.5

[Buildroot] [PATCH 4/5] docs/manual: no longer mention the package/customize thing

[Thomas Petazzoni]

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 docs/manual/customize-rootfs.txt |    6 ------
 1 file changed, 6 deletions(-)

diff --git a/docs/manual/customize-rootfs.txt b/docs/manual/customize-rootfs.txt
index 66e38dc..ce62412 100644
--- a/docs/manual/customize-rootfs.txt
+++ b/docs/manual/customize-rootfs.txt
@@ -45,12 +45,6 @@ there are a few ways to customize the resulting target filesystem.
       stored
     - +BASE_DIR+: the base output directory
 
-* A special package, 'customize', stored in +package/customize+ can be
-  used. You can put all the files that you want to see in the final
-  target root filesystem in +package/customize/source+, and then
-  enable this special package in the configuration system. _This
-  method is marked as deprecated_.
-
 Note also that if want to perform some specific actions *after* all
 filesystem images have been created (for example to automatically
 extract your root filesystem tarball in a location exported by your
-- 
1.7.9.5

[Buildroot] [PATCH 5/5] docs/manual: slightly reword the solutions to customize rootfs

[Thomas Petazzoni]

The order of the solutions to customize the root filesystem is
changed: we now mention the post-build script mechanism *before* the
custom root filesystem skeleton mechanism, because the former is
preferred over the latter.

In addition to this, we give a few more details about direct
customization of the root filesystem in output/target, and about the
custom target skeleton solution.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 docs/manual/customize-rootfs.txt |   29 ++++++++++++++++++-----------
 1 file changed, 18 insertions(+), 11 deletions(-)

diff --git a/docs/manual/customize-rootfs.txt b/docs/manual/customize-rootfs.txt
index ce62412..9de98f7 100644
--- a/docs/manual/customize-rootfs.txt
+++ b/docs/manual/customize-rootfs.txt
@@ -12,17 +12,11 @@ there are a few ways to customize the resulting target filesystem.
   simply make your changes here and run make afterwards - this will
   rebuild the target filesystem image. This method allows you to do
   anything to the target filesystem, but if you decide to completely
-  rebuild your toolchain and tools, these changes will be lost.
-  _Changes do not survive the +make clean+ command_.
-
-* Create your own 'target skeleton'. You can start with the default
-  skeleton available under +system/skeleton+ and then customize it to
-  suit your needs. The +BR2_ROOTFS_SKELETON_CUSTOM+ and
-  +BR2_ROOTFS_SKELETON_CUSTOM_PATH+ will allow you to specify the
-  location of your custom skeleton. These options can be found in the
-  +System configuration+ menu. At build time, the contents of the
-  skeleton are copied to output/target before any package
-  installation.
+  rebuild your toolchain and tools, these changes will be lost. This
+  solution is therefore only useful for quick tests only: _changes do
+  not survive the +make clean+ command_. Once you have validated your
+  changes, you should make sure that they will persist after a +make
+  clean+ by using one of the following methods.
 
 * In the Buildroot configuration, you can specify the path to a
   *post-build script*, that gets called 'after' Buildroot builds all the
@@ -45,6 +39,19 @@ there are a few ways to customize the resulting target filesystem.
       stored
     - +BASE_DIR+: the base output directory
 
+* Create your own 'target skeleton'. You can start with the default
+  skeleton available under +system/skeleton+ and then customize it to
+  suit your needs. The +BR2_ROOTFS_SKELETON_CUSTOM+ and
+  +BR2_ROOTFS_SKELETON_CUSTOM_PATH+ will allow you to specify the
+  location of your custom skeleton. These options can be found in the
+  +System configuration+ menu. At build time, the contents of the
+  skeleton are copied to output/target before any package
+  installation. Note that this method is *not recommended*, as it
+  duplicates the entire skeleton, which prevents from taking advantage
+  of the fixes or improvements brought to the default Buildroot
+  skeleton. The recommended method is to use the _post-build script_
+  mechanism described in the previous item.
+
 Note also that if want to perform some specific actions *after* all
 filesystem images have been created (for example to automatically
 extract your root filesystem tarball in a location exported by your
-- 
1.7.9.5

[Buildroot] [PATCH 1/5] Add a post-image script mechanism

[Stefan Fröberg]

16.1.2013 21:09, Thomas Petazzoni kirjoitti:
> Just like we have a post-build script mechanism that gets executed
> after the build of all packages but before the creation of the
> filesystem images, let's introduce a post-image script mechanism, that
> gets executed once all filesystem images have been generated.
>
> This can for example be used to call a tool building a firmware image
> from different images generated by Buildroot, or automatically extract
> the tarball root filesystem image into some location exported by NFS,
> or any other custom action.
>
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>  Makefile         |    9 +++++++++
>  system/Config.in |   18 ++++++++++++++++++
>  2 files changed, 27 insertions(+)
>
> diff --git a/Makefile b/Makefile
> index 6f8ed0e..88b5a85 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -353,6 +353,8 @@ endif
>  
>  include fs/common.mk
>  
> +TARGETS+=target-post-image
> +
>  TARGETS_CLEAN:=$(patsubst %,%-clean,$(TARGETS))
>  TARGETS_SOURCE:=$(patsubst %,%-source,$(TARGETS) $(BASE_TARGETS))
>  TARGETS_DIRCLEAN:=$(patsubst %,%-dirclean,$(TARGETS))
> @@ -548,6 +550,13 @@ target-generatelocales: host-localedef
>  	done
>  endif
>  
> +target-post-image:
> +ifneq ($(BR2_ROOTFS_POST_IMAGE_SCRIPT),)
> +	@$(call MESSAGE,"Executing post-image script\(s\)")
> +	@$(foreach s, $(call qstrip,$(BR2_ROOTFS_POST_IMAGE_SCRIPT)), \
> +		$(s) $(BINARIES_DIR)$(sep))
> +endif
> +
>  toolchain-eclipse-register:
>  	./support/scripts/eclipse-register-toolchain `readlink -f $(O)` $(notdir $(TARGET_CROSS)) $(BR2_ARCH)
>  
> diff --git a/system/Config.in b/system/Config.in
> index 69863c4..1ffe8a0 100644
> --- a/system/Config.in
> +++ b/system/Config.in
> @@ -265,4 +265,22 @@ config BR2_ROOTFS_POST_BUILD_SCRIPT
>  	  only argument. Make sure the exit code of those scripts are 0,
>  	  otherwise make will stop after calling them.
>  
> +config BR2_ROOTFS_POST_IMAGE_SCRIPT
> +	string "Custom script to run after creating filesystem images"
> +	default ""
> +	help
> +	  Specify a space-separated list of scripts to be run after
> +	  the build has finished and after Buildroot has packed the
> +	  files into selected filesystem images.
> +
> +	  This can for example be used to call a tool building a
> +	  firmware image from different images generated by Buildroot,
> +	  or automatically extract the tarball root filesystem image
> +	  into some location exported by NFS, or any other custom
> +	  action.
> +
> +	  These scripts are called with the images directory name as
> +	  first and only argument. The script is executed from the
> +	  main Buildroot source directory as the current directory.
> +
>  endmenu

Nice.  I know exactly what Im going to use this mechanism for.

Packing target binaries with UPX (http://upx.sourceforge.net/) at the
end of image creation.

Every byte counts :-)

Regards
Stefan

[Buildroot] [PATCH 1/5] Add a post-image script mechanism

[Yann E. MORIN]

Thomas, All,

On Wednesday 16 January 2013 Thomas Petazzoni wrote:
> Just like we have a post-build script mechanism that gets executed
> after the build of all packages but before the creation of the
> filesystem images, let's introduce a post-image script mechanism, that
> gets executed once all filesystem images have been generated.
[--SNIP--]
> diff --git a/Makefile b/Makefile
> index 6f8ed0e..88b5a85 100644
> --- a/Makefile
> +++ b/Makefile
[--SNIP--]
> @@ -548,6 +550,13 @@ target-generatelocales: host-localedef
>  	done
>  endif
>  
> +target-post-image:
> +ifneq ($(BR2_ROOTFS_POST_IMAGE_SCRIPT),)
> +	@$(call MESSAGE,"Executing post-image script\(s\)")
> +	@$(foreach s, $(call qstrip,$(BR2_ROOTFS_POST_IMAGE_SCRIPT)), \
> +		$(s) $(BINARIES_DIR)$(sep))

No need to pass $(BINARIES_DIR) as it now is in the environment.

Would it make sense to run this under fakeroot?

Also, I wonder if we could instead call these scripts for each image we
generate, so it will already be run under fakeroot.

Not sure how to pass the image type to the script, though.
Maybe, something like:
  post-image.sh <image-file> <image-type> <compression>

with image-type ? {tar,ext2,squashfs...}
and  compression ? {none,gz,bz2,lzma,xz...}

And the script would typically contain:

  #!/bin/sh
  img_type="${1}"
  img_file="${2}"
  img_comp="${3}"
  case "${img_type}" in
    tar)
      handle_tar "${img_file}" "#{img_comp}"
      ;;
    squashfs)
      handle_squash "${img_file}"
      ;;
    *)
      # Ignore other types
     ;;
  esac

At least, I find it more interesting.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 2/5] docs/manual: mention the new post-image mechanism

[Yann E. MORIN]

Thomas, All,

On Wednesday 16 January 2013 Thomas Petazzoni wrote:
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>  docs/manual/customize-rootfs.txt |   15 +++++++++++++++
>  1 file changed, 15 insertions(+)

Why did you split the doc-patch out of the feature-patch?

I would expect that documentation is committed in the same changeset as
the feature (or bug-fix) it relates to.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 4/5] docs/manual: no longer mention the package/customize thing

[Yann E. MORIN]

Thomas, All,

On Wednesday 16 January 2013 Thomas Petazzoni wrote:
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>  docs/manual/customize-rootfs.txt |    6 ------
>  1 file changed, 6 deletions(-)

Ditto. This should be folded with the previous cset.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 1/5] Add a post-image script mechanism

[Thomas Petazzoni]

Dear Yann E. MORIN,

On Wed, 16 Jan 2013 21:03:43 +0100, Yann E. MORIN wrote:

> No need to pass $(BINARIES_DIR) as it now is in the environment.

I'd prefer to pass $(BINARIES_DIR) as argument, in order to keep a
symmetry with what we do for the post-build script.

> Would it make sense to run this under fakeroot?

Why?

> Also, I wonder if we could instead call these scripts for each image
> we generate, so it will already be run under fakeroot.

No, I don't think it's a good idea. For example, in my use-case, it was
used to create a firmware image that bundles multiple images generated
by Buildroot. So getting called on a per-image basis doesn't make sense.

> Not sure how to pass the image type to the script, though.
> Maybe, something like:
>   post-image.sh <image-file> <image-type> <compression>
> 
> with image-type ? {tar,ext2,squashfs...}
> and  compression ? {none,gz,bz2,lzma,xz...}
> 
> And the script would typically contain:
> 
>   #!/bin/sh
>   img_type="${1}"
>   img_file="${2}"
>   img_comp="${3}"
>   case "${img_type}" in
>     tar)
>       handle_tar "${img_file}" "#{img_comp}"
>       ;;
>     squashfs)
>       handle_squash "${img_file}"
>       ;;
>     *)
>       # Ignore other types
>      ;;
>   esac
> 
> At least, I find it more interesting.

Unfortunately, not me. Calling a script once all images are generated
is *simple*, the script is free to do whatever it wants. Iterate over
all images, iterate over certain images only, etc. There is really no
need to over-engineer this thing.

I really don't see the point of putting more complexity into this.
Let's have something nicely symmetric between the post-build script
feature and the post-image script feature.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [PATCH 2/5] docs/manual: mention the new post-image mechanism

[Thomas Petazzoni]

Dear Yann E. MORIN,

On Wed, 16 Jan 2013 21:06:06 +0100, Yann E. MORIN wrote:
> Thomas, All,
> 
> On Wednesday 16 January 2013 Thomas Petazzoni wrote:
> > Signed-off-by: Thomas Petazzoni
> > <thomas.petazzoni@free-electrons.com> ---
> >  docs/manual/customize-rootfs.txt |   15 +++++++++++++++
> >  1 file changed, 15 insertions(+)
> 
> Why did you split the doc-patch out of the feature-patch?
> 
> I would expect that documentation is committed in the same changeset
> as the feature (or bug-fix) it relates to.

Simply because not having the documentation doesn't prevent the feature
from working, but it makes the patch smaller, so easier to review, in
my opinion.

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [PATCH 1/5] Add a post-image script mechanism

[Yann E. MORIN]

Thomas, All,

On Wednesday 16 January 2013 Thomas Petazzoni wrote:
> On Wed, 16 Jan 2013 21:03:43 +0100, Yann E. MORIN wrote:
> > No need to pass $(BINARIES_DIR) as it now is in the environment.
> I'd prefer to pass $(BINARIES_DIR) as argument, in order to keep a
> symmetry with what we do for the post-build script.
> 
> > Would it make sense to run this under fakeroot?
> Why?

For example, I need to create multiple partitions for my device, so I
need one file system image per partition:
  - vfat on /boot
  - squashfs on /
  - ext2 on /home

So I would :
  - configure Buildroot to build only a tarball
  - configure Buildroot to run my post-image script
  - extract the tarball
  - generate each file system images

But the untar would loose the device nodes and the files UIDs and GIDs,
forcing me do run fakeroot inside the script, saving/loading the fakeroot
DB, for every commands in my script.

Having Buildroot do the fakeroot would ensure this consistency.

> > Also, I wonder if we could instead call these scripts for each image
> > we generate, so it will already be run under fakeroot.
> 
> No, I don't think it's a good idea. For example, in my use-case, it was
> used to create a firmware image that bundles multiple images generated
> by Buildroot. So getting called on a per-image basis doesn't make sense.

OK, so it makes sense to run the scripts only once at the end. Agreed.

[--SNIP--]
> I really don't see the point of putting more complexity into this.
> Let's have something nicely symmetric between the post-build script
> feature and the post-image script feature.

Right, agreed.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 1/5] Add a post-image script mechanism

[Jeremy Rosen]

I think we have two different use cases here...


* we need a hook within fakeroot to make multiple images when each image is a different partition

* we need a hook post-image to bundle multiple images into a single binary files

that's a lot of hook, but i'm not sure how to not have all of them... 

    Regards

    J?r?my Rosen

fight key loggers : write some perl using vim

[Buildroot] [PATCH 3/5] package/customize: remove

[Arnout Vandecappelle]

On 16/01/13 20:09, Thomas Petazzoni wrote:
> This mechanism of root filesystem customization has been deprecated
> since a long time, so let's remove it now.
>
> Signed-off-by: Thomas Petazzoni<thomas.petazzoni@free-electrons.com>

Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

  It can also be removed from pkg-stats.

  And maybe it should be added to Config.in.legacy?

  Regards,
  Arnout
-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH 5/5] docs/manual: slightly reword the solutions to customize rootfs

[Yann E. MORIN]

On Wednesday 16 January 2013 Thomas Petazzoni wrote:
> The order of the solutions to customize the root filesystem is
> changed: we now mention the post-build script mechanism *before* the
> custom root filesystem skeleton mechanism, because the former is
> preferred over the latter.
> 
> In addition to this, we give a few more details about direct
> customization of the root filesystem in output/target, and about the
> custom target skeleton solution.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

> ---
>  docs/manual/customize-rootfs.txt |   29 ++++++++++++++++++-----------
>  1 file changed, 18 insertions(+), 11 deletions(-)
> 
> diff --git a/docs/manual/customize-rootfs.txt b/docs/manual/customize-rootfs.txt
> index ce62412..9de98f7 100644
> --- a/docs/manual/customize-rootfs.txt
> +++ b/docs/manual/customize-rootfs.txt
> @@ -12,17 +12,11 @@ there are a few ways to customize the resulting target filesystem.
>    simply make your changes here and run make afterwards - this will
>    rebuild the target filesystem image. This method allows you to do
>    anything to the target filesystem, but if you decide to completely
> -  rebuild your toolchain and tools, these changes will be lost.
> -  _Changes do not survive the +make clean+ command_.
> -
> -* Create your own 'target skeleton'. You can start with the default
> -  skeleton available under +system/skeleton+ and then customize it to
> -  suit your needs. The +BR2_ROOTFS_SKELETON_CUSTOM+ and
> -  +BR2_ROOTFS_SKELETON_CUSTOM_PATH+ will allow you to specify the
> -  location of your custom skeleton. These options can be found in the
> -  +System configuration+ menu. At build time, the contents of the
> -  skeleton are copied to output/target before any package
> -  installation.
> +  rebuild your toolchain and tools, these changes will be lost. This
> +  solution is therefore only useful for quick tests only: _changes do
> +  not survive the +make clean+ command_. Once you have validated your
> +  changes, you should make sure that they will persist after a +make
> +  clean+ by using one of the following methods.
>  
>  * In the Buildroot configuration, you can specify the path to a
>    *post-build script*, that gets called 'after' Buildroot builds all the
> @@ -45,6 +39,19 @@ there are a few ways to customize the resulting target filesystem.
>        stored
>      - +BASE_DIR+: the base output directory
>  
> +* Create your own 'target skeleton'. You can start with the default
> +  skeleton available under +system/skeleton+ and then customize it to
> +  suit your needs. The +BR2_ROOTFS_SKELETON_CUSTOM+ and
> +  +BR2_ROOTFS_SKELETON_CUSTOM_PATH+ will allow you to specify the
> +  location of your custom skeleton. These options can be found in the
> +  +System configuration+ menu. At build time, the contents of the
> +  skeleton are copied to output/target before any package
> +  installation. Note that this method is *not recommended*, as it
> +  duplicates the entire skeleton, which prevents from taking advantage
> +  of the fixes or improvements brought to the default Buildroot
> +  skeleton. The recommended method is to use the _post-build script_
> +  mechanism described in the previous item.
> +
>  Note also that if want to perform some specific actions *after* all
>  filesystem images have been created (for example to automatically
>  extract your root filesystem tarball in a location exported by your
> -- 
> 1.7.9.5
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
> 
> 

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 4/5] docs/manual: no longer mention the package/customize thing

[Yann E. MORIN]

On Wednesday 16 January 2013 Thomas Petazzoni wrote:
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

> ---
>  docs/manual/customize-rootfs.txt |    6 ------
>  1 file changed, 6 deletions(-)
> 
> diff --git a/docs/manual/customize-rootfs.txt b/docs/manual/customize-rootfs.txt
> index 66e38dc..ce62412 100644
> --- a/docs/manual/customize-rootfs.txt
> +++ b/docs/manual/customize-rootfs.txt
> @@ -45,12 +45,6 @@ there are a few ways to customize the resulting target filesystem.
>        stored
>      - +BASE_DIR+: the base output directory
>  
> -* A special package, 'customize', stored in +package/customize+ can be
> -  used. You can put all the files that you want to see in the final
> -  target root filesystem in +package/customize/source+, and then
> -  enable this special package in the configuration system. _This
> -  method is marked as deprecated_.
> -
>  Note also that if want to perform some specific actions *after* all
>  filesystem images have been created (for example to automatically
>  extract your root filesystem tarball in a location exported by your
> -- 
> 1.7.9.5
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
> 
> 

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 3/5] package/customize: remove

[Yann E. MORIN]

On Wednesday 16 January 2013 Thomas Petazzoni wrote:
> This mechanism of root filesystem customization has been deprecated
> since a long time, so let's remove it now.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

As Arnout suggested, it would be nice to add this to the legacy checks.
I'm afraid people who used that will be highly disapointed when they
see thir builds breaks without warning.

Otherwise, on the priniciple:
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

> ---
>  package/Config.in              |    1 -
>  package/customize/Config.in    |    7 -------
>  package/customize/customize.mk |   32 --------------------------------
>  3 files changed, 40 deletions(-)
>  delete mode 100644 package/customize/Config.in
>  delete mode 100644 package/customize/customize.mk
>  delete mode 100644 package/customize/source/.empty
> 
> diff --git a/package/Config.in b/package/Config.in
> index ba03b3c..5c78784 100644
> --- a/package/Config.in
> +++ b/package/Config.in
> @@ -1,7 +1,6 @@
>  menu "Package Selection for the target"
>  
>  source "package/busybox/Config.in"
> -source "package/customize/Config.in"
>  
>  # Audio and video applications
>  source "package/multimedia/Config.in"
> diff --git a/package/customize/Config.in b/package/customize/Config.in
> deleted file mode 100644
> index 8f8aaa2..0000000
> --- a/package/customize/Config.in
> +++ /dev/null
> @@ -1,7 +0,0 @@
> -config BR2_PACKAGE_CUSTOMIZE
> -	bool "customize"
> -	depends on BR2_DEPRECATED
> -	help
> -	  Add custom stuff to your buildroot.
> -
> -	  (edit package/customize/customize.mk)
> diff --git a/package/customize/customize.mk b/package/customize/customize.mk
> deleted file mode 100644
> index d2b8518..0000000
> --- a/package/customize/customize.mk
> +++ /dev/null
> @@ -1,32 +0,0 @@
> -#############################################################
> -#
> -# Any custom stuff you feel like doing....
> -#
> -#############################################################
> -CUST_DIR:=package/customize/source
> -
> -$(BUILD_DIR)/.customize:
> -	rm -f $(BUILD_DIR)/series
> -	(cd $(CUST_DIR); \
> -	 /bin/ls -d * > $(BUILD_DIR)/series || \
> -	 touch $(BUILD_DIR)/series )
> -	for f in `cat $(BUILD_DIR)/series`; do \
> -		cp -af $(CUST_DIR)/$$f $(TARGET_DIR); \
> -	done
> -	rm -f $(BUILD_DIR)/series
> -	touch $@
> -
> -customize: $(BUILD_DIR)/.customize
> -
> -customize-clean:
> -	rm -f $(BUILD_DIR)/.customize
> -
> -.PHONY: customize
> -#############################################################
> -#
> -# Toplevel Makefile options
> -#
> -#############################################################
> -ifeq ($(BR2_PACKAGE_CUSTOMIZE),y)
> -TARGETS+=customize
> -endif
> diff --git a/package/customize/source/.empty b/package/customize/source/.empty
> deleted file mode 100644
> index e69de29..0000000
> -- 
> 1.7.9.5
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
> 
> 

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 1/5] Add a post-image script mechanism

[Yann E. MORIN]

On Wednesday 16 January 2013 Thomas Petazzoni wrote:
> Just like we have a post-build script mechanism that gets executed
> after the build of all packages but before the creation of the
> filesystem images, let's introduce a post-image script mechanism, that
> gets executed once all filesystem images have been generated.
> 
> This can for example be used to call a tool building a firmware image
> from different images generated by Buildroot, or automatically extract
> the tarball root filesystem image into some location exported by NFS,
> or any other custom action.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

That is really usefull! Thank you! :-)

Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

> ---
>  Makefile         |    9 +++++++++
>  system/Config.in |   18 ++++++++++++++++++
>  2 files changed, 27 insertions(+)
> 
> diff --git a/Makefile b/Makefile
> index 6f8ed0e..88b5a85 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -353,6 +353,8 @@ endif
>  
>  include fs/common.mk
>  
> +TARGETS+=target-post-image
> +
>  TARGETS_CLEAN:=$(patsubst %,%-clean,$(TARGETS))
>  TARGETS_SOURCE:=$(patsubst %,%-source,$(TARGETS) $(BASE_TARGETS))
>  TARGETS_DIRCLEAN:=$(patsubst %,%-dirclean,$(TARGETS))
> @@ -548,6 +550,13 @@ target-generatelocales: host-localedef
>  	done
>  endif
>  
> +target-post-image:
> +ifneq ($(BR2_ROOTFS_POST_IMAGE_SCRIPT),)
> +	@$(call MESSAGE,"Executing post-image script\(s\)")
> +	@$(foreach s, $(call qstrip,$(BR2_ROOTFS_POST_IMAGE_SCRIPT)), \
> +		$(s) $(BINARIES_DIR)$(sep))
> +endif
> +
>  toolchain-eclipse-register:
>  	./support/scripts/eclipse-register-toolchain `readlink -f $(O)` $(notdir $(TARGET_CROSS)) $(BR2_ARCH)
>  
> diff --git a/system/Config.in b/system/Config.in
> index 69863c4..1ffe8a0 100644
> --- a/system/Config.in
> +++ b/system/Config.in
> @@ -265,4 +265,22 @@ config BR2_ROOTFS_POST_BUILD_SCRIPT
>  	  only argument. Make sure the exit code of those scripts are 0,
>  	  otherwise make will stop after calling them.
>  
> +config BR2_ROOTFS_POST_IMAGE_SCRIPT
> +	string "Custom script to run after creating filesystem images"
> +	default ""
> +	help
> +	  Specify a space-separated list of scripts to be run after
> +	  the build has finished and after Buildroot has packed the
> +	  files into selected filesystem images.
> +
> +	  This can for example be used to call a tool building a
> +	  firmware image from different images generated by Buildroot,
> +	  or automatically extract the tarball root filesystem image
> +	  into some location exported by NFS, or any other custom
> +	  action.
> +
> +	  These scripts are called with the images directory name as
> +	  first and only argument. The script is executed from the
> +	  main Buildroot source directory as the current directory.
> +
>  endmenu
> -- 
> 1.7.9.5
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
> 
> 

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 2/5] docs/manual: mention the new post-image mechanism

[Yann E. MORIN]

On Wednesday 16 January 2013 Thomas Petazzoni wrote:
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>  docs/manual/customize-rootfs.txt |   15 +++++++++++++++
>  1 file changed, 15 insertions(+)
> 
> diff --git a/docs/manual/customize-rootfs.txt b/docs/manual/customize-rootfs.txt
> index 6399699..66e38dc 100644
> --- a/docs/manual/customize-rootfs.txt
> +++ b/docs/manual/customize-rootfs.txt
> @@ -50,3 +50,18 @@ there are a few ways to customize the resulting target filesystem.
>    target root filesystem in +package/customize/source+, and then
>    enable this special package in the configuration system. _This
>    method is marked as deprecated_.
> +
> +Note also that if want to perform some specific actions *after* all

...if you want...

> +filesystem images have been created (for example to automatically
> +extract your root filesystem tarball in a location exported by your
> +NFS server, or to create a special firmware image that bundles your
> +root filesystem and kernel image, or any other custom action), you can
> +specify a list of scripts in the +BR2_ROOTFS_POST_IMAGE_SCRIPT+

... a space-separated list...

> +configuration option. Each of those scripts will be called, with the
> +path to the +images+ output directory as first and unique argument,
> +and will be executed with the main Buildroot source directory as the
> +current directory. Also, just like for the _post-build scripts_
> +mentionned above, you also have access to the following environment
> +variables from your _post-image scripts_: +BUILDROOT_CONFIG+,
> ++HOST_DIR+, +STAGING_DIR+, +TARGET_DIR+, +BINARIES_DIR+ and
> ++BASE_DIR+.

With those changes:
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 1/5] Add a post-image script mechanism

[Luca Ceresoli]

Thomas Petazzoni wrote:
> Just like we have a post-build script mechanism that gets executed
> after the build of all packages but before the creation of the
> filesystem images, let's introduce a post-image script mechanism, that
> gets executed once all filesystem images have been generated.
>
> This can for example be used to call a tool building a firmware image
> from different images generated by Buildroot, or automatically extract
> the tarball root filesystem image into some location exported by NFS,
> or any other custom action.
>
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Acked-by: Luca Ceresoli <luca@lucaceresoli.net>

[Buildroot] [PATCH 2/5] docs/manual: mention the new post-image mechanism

[Luca Ceresoli]

Thomas Petazzoni wrote:
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>   docs/manual/customize-rootfs.txt |   15 +++++++++++++++
>   1 file changed, 15 insertions(+)
>
> diff --git a/docs/manual/customize-rootfs.txt b/docs/manual/customize-rootfs.txt
> index 6399699..66e38dc 100644
> --- a/docs/manual/customize-rootfs.txt
> +++ b/docs/manual/customize-rootfs.txt
> @@ -50,3 +50,18 @@ there are a few ways to customize the resulting target filesystem.
>     target root filesystem in +package/customize/source+, and then
>     enable this special package in the configuration system. _This
>     method is marked as deprecated_.
> +
> +Note also that if want to perform some specific actions *after* all
> +filesystem images have been created (for example to automatically
> +extract your root filesystem tarball in a location exported by your
> +NFS server, or to create a special firmware image that bundles your
> +root filesystem and kernel image, or any other custom action), you can
> +specify a list of scripts in the +BR2_ROOTFS_POST_IMAGE_SCRIPT+
> +configuration option. Each of those scripts will be called, with the
> +path to the +images+ output directory as first and unique argument,
> +and will be executed with the main Buildroot source directory as the
> +current directory. Also, just like for the _post-build scripts_
> +mentionned above, you also have access to the following environment

s/mentionned/mentioned/

With that fix:
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>

[Buildroot] [PATCH 3/5] package/customize: remove

[Luca Ceresoli]

Thomas Petazzoni wrote:
> This mechanism of root filesystem customization has been deprecated
> since a long time, so let's remove it now.
>
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

With Arnout's suggestions applied:
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>

[Buildroot] [PATCH 4/5] docs/manual: no longer mention the package/customize thing

[Luca Ceresoli]

Thomas Petazzoni wrote:
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Acked-by: Luca Ceresoli <luca@lucaceresoli.net>

[Buildroot] [PATCH 5/5] docs/manual: slightly reword the solutions to customize rootfs

[Luca Ceresoli]

Thomas Petazzoni wrote:
> The order of the solutions to customize the root filesystem is
> changed: we now mention the post-build script mechanism *before* the
> custom root filesystem skeleton mechanism, because the former is
> preferred over the latter.
>
> In addition to this, we give a few more details about direct
> customization of the root filesystem in output/target, and about the
> custom target skeleton solution.
>
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Acked-by: Luca Ceresoli <luca@lucaceresoli.net>