[Buildroot] [RFC/PATCH 0/4] support building from read-only source tree

[Buildroot] [RFC/PATCH 0/4] support building from read-only source tree

[Nathan Lynch]

There are several places in Buildroot where files are copied from the
source tree to the output directory and then modified.  The most
obvious case is copying a .config or equivalent to the build
directory and then modifying it with sed, as with uClibc or busybox.

This works fine when the source file has write permissions; this is
preserved in the destination copy.  This is the behavior in the common
case of running from a git repository or a release tarball.

However, when the source file is read-only, the destination copy is
also not writable and such fixups fail.  This occurs when the the
Buildroot source tree is being hosted in a Perforce repository.
Perforce uses local file permissions to track whether a file is being
edited and checks out all regular files read-only by design.

The following changes were enough to get qemu_mips_malta_defconfig to
build and run successfully.  I submit these patches for discussion and
illustration of the effort required to support this use case.

Nathan Lynch (4):
  uClibc: ensure .oldconfig is writable
  ensure target fs is writable
  busybox: ensure $(BUSYBOX_BUILD_CONFIG) is writable
  linux: ensure buildroot_defconfig is writable

 Makefile                   |    1 +
 linux/linux.mk             |    3 ++-
 package/busybox/busybox.mk |    3 ++-
 toolchain/uClibc/uclibc.mk |    1 +
 4 files changed, 6 insertions(+), 2 deletions(-)

-- 
1.7.10.2

[Buildroot] [RFC/PATCH 1/4] uClibc: ensure .oldconfig is writable

[Nathan Lynch]

If $(UCLIBC_CONFIG_FILE) is read-only, the sed fixups to .oldconfig
fail.

Signed-off-by: Nathan Lynch <ntl@pobox.com>
---
 toolchain/uClibc/uclibc.mk |    1 +
 1 file changed, 1 insertion(+)

diff --git a/toolchain/uClibc/uclibc.mk b/toolchain/uClibc/uclibc.mk
index 0728135..f41e6b9 100644
--- a/toolchain/uClibc/uclibc.mk
+++ b/toolchain/uClibc/uclibc.mk
@@ -101,6 +101,7 @@ endif
 # Some targets may wish to provide their own UCLIBC_CONFIG_FILE...
 $(UCLIBC_DIR)/.oldconfig: $(UCLIBC_DIR)/.patched $(UCLIBC_CONFIG_FILE)
 	cp -f $(UCLIBC_CONFIG_FILE) $(UCLIBC_DIR)/.oldconfig
+	chmod u+w $(UCLIBC_DIR)/.oldconfig
 	$(SED) 's,^CROSS_COMPILER_PREFIX=.*,CROSS_COMPILER_PREFIX="$(TARGET_CROSS)",g' \
 		-e 's,# TARGET_$(UCLIBC_TARGET_ARCH) is not set,TARGET_$(UCLIBC_TARGET_ARCH)=y,g' \
 		-e 's,^TARGET_ARCH=".*",TARGET_ARCH=\"$(UCLIBC_TARGET_ARCH)\",g' \
-- 
1.7.10.2

[Buildroot] [RFC/PATCH 2/4] ensure target fs is writable

[Nathan Lynch]

If the source target skeleton is read-only, modifications to the
output target (such as creating /etc/hostname and /etc/issue) fail.

Signed-off-by: Nathan Lynch <ntl@pobox.com>
---
 Makefile |    1 +
 1 file changed, 1 insertion(+)

diff --git a/Makefile b/Makefile
index 9a9634d..38276cd 100644
--- a/Makefile
+++ b/Makefile
@@ -378,6 +378,7 @@ endif
 	if ! [ -d "$(TARGET_DIR)/bin" ]; then \
 		if [ -d "$(TARGET_SKELETON)" ]; then \
 			cp -fa $(TARGET_SKELETON)/* $(TARGET_DIR)/; \
+			chmod -R u+w $(TARGET_DIR)/; \
 		fi; \
 	fi
 	-find $(TARGET_DIR) -type d -name CVS -print0 -o -name .svn -print0 | xargs -0 rm -rf
-- 
1.7.10.2

[Buildroot] [RFC/PATCH 3/4] busybox: ensure $(BUSYBOX_BUILD_CONFIG) is writable

[Nathan Lynch]

If $(BUSYBOX_CONFIG_FILE) is read-only, the sed fixups to
$(BUSYBOX_BUILD_CONFIG) fail.

Signed-off-by: Nathan Lynch <ntl@pobox.com>
---
 package/busybox/busybox.mk |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/package/busybox/busybox.mk b/package/busybox/busybox.mk
index 1d988f8..50a612f 100644
--- a/package/busybox/busybox.mk
+++ b/package/busybox/busybox.mk
@@ -115,7 +115,8 @@ endef
 endif
 
 define BUSYBOX_COPY_CONFIG
-	cp -f $(BUSYBOX_CONFIG_FILE) $(BUSYBOX_BUILD_CONFIG)
+	cp -f $(BUSYBOX_CONFIG_FILE) $(BUSYBOX_BUILD_CONFIG) && \
+		chmod u+w $(BUSYBOX_BUILD_CONFIG)
 endef
 
 # Disable shadow passwords support if unsupported by the C library
-- 
1.7.10.2

[Buildroot] [RFC/PATCH 4/4] linux: ensure buildroot_defconfig is writable

[Nathan Lynch]

If $(KERNEL_SOURCE_CONFIG) is read-only, the rm of
$(KERNEL_ARCH_PATH)/configs/buildroot_defconfig will prompt the user.
Make it writable and use rm -f for good measure.

Signed-off-by: Nathan Lynch <ntl@pobox.com>
---
 linux/linux.mk |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/linux/linux.mk b/linux/linux.mk
index f165dda..211d054 100644
--- a/linux/linux.mk
+++ b/linux/linux.mk
@@ -138,8 +138,9 @@ endif
 
 define LINUX_CONFIGURE_CMDS
 	cp $(KERNEL_SOURCE_CONFIG) $(KERNEL_ARCH_PATH)/configs/buildroot_defconfig
+	chmod u+w $(KERNEL_ARCH_PATH)/configs/buildroot_defconfig
 	$(TARGET_MAKE_ENV) $(MAKE1) $(LINUX_MAKE_FLAGS) -C $(@D) buildroot_defconfig
-	rm $(KERNEL_ARCH_PATH)/configs/buildroot_defconfig
+	rm -f $(KERNEL_ARCH_PATH)/configs/buildroot_defconfig
 	$(if $(BR2_ARM_EABI),
 		$(call KCONFIG_ENABLE_OPT,CONFIG_AEABI,$(@D)/.config),
 		$(call KCONFIG_DISABLE_OPT,CONFIG_AEABI,$(@D)/.config))
-- 
1.7.10.2

[Buildroot] [RFC/PATCH 2/4] ensure target fs is writable

[Arnout Vandecappelle]

On 06/18/12 17:02, Nathan Lynch wrote:
> If the source target skeleton is read-only, modifications to the
> output target (such as creating /etc/hostname and /etc/issue) fail.
>
> Signed-off-by: Nathan Lynch<ntl@pobox.com>
> ---
>   Makefile |    1 +
>   1 file changed, 1 insertion(+)
>
> diff --git a/Makefile b/Makefile
> index 9a9634d..38276cd 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -378,6 +378,7 @@ endif
>   	if ! [ -d "$(TARGET_DIR)/bin" ]; then \
>   		if [ -d "$(TARGET_SKELETON)" ]; then \
>   			cp -fa $(TARGET_SKELETON)/* $(TARGET_DIR)/; \
> +			chmod -R u+w $(TARGET_DIR)/; \

  Maybe it's safer to do this only on directories, e.g. with
rsync -a --chmod=Du+w $(TARGET_SKELETON)/ $(TARGET_DIR)/
(untested)


  Regards,
  Arnout
-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [RFC/PATCH 3/4] busybox: ensure $(BUSYBOX_BUILD_CONFIG) is writable

[Arnout Vandecappelle]

On 06/18/12 17:02, Nathan Lynch wrote:
> If $(BUSYBOX_CONFIG_FILE) is read-only, the sed fixups to
> $(BUSYBOX_BUILD_CONFIG) fail.
>
> Signed-off-by: Nathan Lynch<ntl@pobox.com>
> ---
>   package/busybox/busybox.mk |    3 ++-
>   1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/package/busybox/busybox.mk b/package/busybox/busybox.mk
> index 1d988f8..50a612f 100644
> --- a/package/busybox/busybox.mk
> +++ b/package/busybox/busybox.mk
> @@ -115,7 +115,8 @@ endef
>   endif
>
>   define BUSYBOX_COPY_CONFIG
> -	cp -f $(BUSYBOX_CONFIG_FILE) $(BUSYBOX_BUILD_CONFIG)
> +	cp -f $(BUSYBOX_CONFIG_FILE) $(BUSYBOX_BUILD_CONFIG)&&  \
> +		chmod u+w $(BUSYBOX_BUILD_CONFIG)

  Just put it on two lines, without the && \ connector.


  Regards,
  Arnout

-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [RFC/PATCH 0/4] support building from read-only source tree

[Yann E. MORIN]

Nathan, All,

On 2012-06-18 10:02 -0500, Nathan Lynch spake thusly:
> There are several places in Buildroot where files are copied from the
> source tree to the output directory and then modified.  The most
> obvious case is copying a .config or equivalent to the build
> directory and then modifying it with sed, as with uClibc or busybox.
> 
> This works fine when the source file has write permissions; this is
> preserved in the destination copy.  This is the behavior in the common
> case of running from a git repository or a release tarball.
> 
> However, when the source file is read-only, the destination copy is
> also not writable and such fixups fail.  This occurs when the the
> Buildroot source tree is being hosted in a Perforce repository.
> Perforce uses local file permissions to track whether a file is being
> edited and checks out all regular files read-only by design.
> 
> The following changes were enough to get qemu_mips_malta_defconfig to
> build and run successfully.  I submit these patches for discussion and
> illustration of the effort required to support this use case.

Unfortunately, those patches mostly no longer apply, since there have
been some code churn (reoridering) in the meantime.

I'll take whatever I can, and adapt the remaining.

Regards,
Yann E. MORIN. 

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [RFC/PATCH 0/4] support building from read-only source tree

[Yann E. MORIN]

Nathan, All,

On 2013-05-26 16:45 +0200, Yann E. MORIN spake thusly:
> On 2012-06-18 10:02 -0500, Nathan Lynch spake thusly:
> > There are several places in Buildroot where files are copied from the
> > source tree to the output directory and then modified.  The most
> > obvious case is copying a .config or equivalent to the build
> > directory and then modifying it with sed, as with uClibc or busybox.
> 
> Unfortunately, those patches mostly no longer apply, since there have
> been some code churn (reoridering) in the meantime.
> 
> I'll take whatever I can, and adapt the remaining.

I've added them to my patchwork branch:
    https://www.gitorious.org/buildroot/buildroot/commits/yem-patchwork-fixes

Thank you!

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [RFC/PATCH 0/4] support building from read-only source tree

[Nathan Lynch]

On Sun, 2013-05-26 at 18:36 +0200, Yann E. MORIN wrote:
> Nathan, All,
> 
> On 2013-05-26 16:45 +0200, Yann E. MORIN spake thusly:
> > On 2012-06-18 10:02 -0500, Nathan Lynch spake thusly:
> > > There are several places in Buildroot where files are copied from the
> > > source tree to the output directory and then modified.  The most
> > > obvious case is copying a .config or equivalent to the build
> > > directory and then modifying it with sed, as with uClibc or busybox.
> > 
> > Unfortunately, those patches mostly no longer apply, since there have
> > been some code churn (reoridering) in the meantime.
> > 
> > I'll take whatever I can, and adapt the remaining.
> 
> I've added them to my patchwork branch:
>     https://www.gitorious.org/buildroot/buildroot/commits/yem-patchwork-fixes
> 
> Thank you!

Thank you for picking these up.  I haven't been sufficiently motivated
to pursue these (obviously); hopefully they will be useful to others.