* [Buildroot] [PATCH v12 1/3] core: split variables definition related to in/out-of-tree build from O itself
2016-10-17 21:05 [Buildroot] [PATCH v12 0/3] Relocatable SDK: preparatory changes Samuel Martin
@ 2016-10-17 21:05 ` Samuel Martin
2016-10-17 21:05 ` [Buildroot] [PATCH v12 2/3] core: reorder top-level Makefile and document things Samuel Martin
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: Samuel Martin @ 2016-10-17 21:05 UTC (permalink / raw)
To: buildroot
This change uncorrolates the CONFIG_DIR and NEED_WRAPPER definition from
the presence of the O variable in the command line.
Now, the condition used to set these variables is the value of O itself.
This change is a preparatory work since the O definition will need to
be moved around when we will make Buildroot run with absolute canonical
paths for both its root directory and the output location.
This will be addressed in a follow-up patch.
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
---
changes v9->v10:
- update A/R/T tags
changes v8->v9:
- none
changes v7->v8:
- new patch
---
Makefile | 14 +++++++++++---
1 file changed, 11 insertions(+), 3 deletions(-)
diff --git a/Makefile b/Makefile
index a860218..2c0ac23 100644
--- a/Makefile
+++ b/Makefile
@@ -109,10 +109,11 @@ endif
# Include some helper macros and variables
include support/misc/utils.mk
+# Set O variable if not already done on the command line;
+# or avoid confusing packages that can use the O=<dir> syntax for out-of-tree
+# build by preventing it from being forwarded to sub-make calls.
ifneq ("$(origin O)", "command line")
O := output
-CONFIG_DIR := $(TOPDIR)
-NEED_WRAPPER =
else
# other packages might also support Linux-style out of tree builds
# with the O=<dir> syntax (E.G. BusyBox does). As make automatically
@@ -125,9 +126,16 @@ MAKEOVERRIDES =
# Unfortunately some packages don't look at origin (E.G. uClibc 0.9.31+)
# To really make O go away, we have to override it.
override O := $(O)
-CONFIG_DIR := $(O)
# we need to pass O= everywhere we call back into the toplevel makefile
EXTRAMAKEARGS = O=$(O)
+endif
+
+# Set variables related to in-tree or out-of-tree build.
+ifeq ($(O),output)
+CONFIG_DIR := $(TOPDIR)
+NEED_WRAPPER =
+else
+CONFIG_DIR := $(O)
NEED_WRAPPER = y
endif
--
2.10.0
^ permalink raw reply related [flat|nested] 5+ messages in thread* [Buildroot] [PATCH v12 2/3] core: reorder top-level Makefile and document things
2016-10-17 21:05 [Buildroot] [PATCH v12 0/3] Relocatable SDK: preparatory changes Samuel Martin
2016-10-17 21:05 ` [Buildroot] [PATCH v12 1/3] core: split variables definition related to in/out-of-tree build from O itself Samuel Martin
@ 2016-10-17 21:05 ` Samuel Martin
2016-10-17 21:05 ` [Buildroot] [PATCH v12 3/3] core: re-enter make if $(CURDIR) or $(O) are not canonical paths Samuel Martin
2016-10-19 21:27 ` [Buildroot] [PATCH v12 0/3] Relocatable SDK: preparatory changes Thomas Petazzoni
3 siblings, 0 replies; 5+ messages in thread
From: Samuel Martin @ 2016-10-17 21:05 UTC (permalink / raw)
To: buildroot
This change only moves things around and comments what is done in the
top-level Makefile file, in order to prepare the next changes.
Note that moving the definition of $(O) before or after re-entering make
does not change anything on the buildroot behavior.
This change also renames the variable UMASK to REQ_UMASK.
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
changes v11->v12:
- update commit log (Arnout)
- reintroduce EXTRAMAKEARGS (Arnout)
- remove CUR_UMASK since only used once (Arnout)
- update A/R/T tags
changes v10->v11:
- fix typos in umask variable
changes v9->v10:
- new patch (Arnout)
- s/:=/=/ in CUR_MASK assignation (Arnout)
---
Makefile | 59 ++++++++++++++++++++++++++++++++++-------------------------
1 file changed, 34 insertions(+), 25 deletions(-)
diff --git a/Makefile b/Makefile
index 2c0ac23..717b2db 100644
--- a/Makefile
+++ b/Makefile
@@ -29,16 +29,46 @@ SHELL := $(shell if [ -x "$$BASH" ]; then echo $$BASH; \
else if [ -x /bin/bash ]; then echo /bin/bash; \
else echo sh; fi; fi)
-# Trick for always running with a fixed umask
-UMASK = 0022
-ifneq ($(shell umask),$(UMASK))
+# Set O variable if not already done on the command line;
+# or avoid confusing packages that can use the O=<dir> syntax for out-of-tree
+# build by preventing it from being forwarded to sub-make calls.
+ifneq ("$(origin O)", "command line")
+O := output
+else
+# Other packages might also support Linux-style out of tree builds
+# with the O=<dir> syntax (E.G. BusyBox does). As make automatically
+# forwards command line variable definitions those packages get very
+# confused. Fix this by telling make to not do so.
+MAKEOVERRIDES :=
+# Strangely enough O is still passed to submakes with MAKEOVERRIDES
+# (with make 3.81 atleast), the only thing that changes is the output
+# of the origin function (command line -> environment).
+# Unfortunately some packages don't look at origin (E.G. uClibc 0.9.31+)
+# To really make O go away, we have to override it.
+override O := $(O)
+endif
+
+# Check if the current Buildroot execution meets all the pre-requisites.
+# If they are not met, Buildroot will actually do its job in a sub-make meeting
+# its pre-requisites, which is:
+# 1- Permissive enough umask:
+# Wrong or too restrictive umask will prevent Buildroot and packages from
+# creating files and directories.
+
+REQ_UMASK = 0022
+
+# we need to pass O= everywhere we call back into the toplevel makefile
+EXTRAMAKEARGS = O=$(O)
+
+# Check Buildroot execution pre-requisites here.
+ifneq ($(shell umask),$(REQ_UMASK))
.PHONY: _all $(MAKECMDGOALS)
$(MAKECMDGOALS): _all
@:
_all:
- @umask $(UMASK) && $(MAKE) --no-print-directory $(MAKECMDGOALS)
+ @umask $(REQ_UMASK) && $(MAKE) --no-print-directory $(MAKECMDGOALS)
else # umask
@@ -109,27 +139,6 @@ endif
# Include some helper macros and variables
include support/misc/utils.mk
-# Set O variable if not already done on the command line;
-# or avoid confusing packages that can use the O=<dir> syntax for out-of-tree
-# build by preventing it from being forwarded to sub-make calls.
-ifneq ("$(origin O)", "command line")
-O := output
-else
-# other packages might also support Linux-style out of tree builds
-# with the O=<dir> syntax (E.G. BusyBox does). As make automatically
-# forwards command line variable definitions those packages get very
-# confused. Fix this by telling make to not do so
-MAKEOVERRIDES =
-# strangely enough O is still passed to submakes with MAKEOVERRIDES
-# (with make 3.81 atleast), the only thing that changes is the output
-# of the origin function (command line -> environment).
-# Unfortunately some packages don't look at origin (E.G. uClibc 0.9.31+)
-# To really make O go away, we have to override it.
-override O := $(O)
-# we need to pass O= everywhere we call back into the toplevel makefile
-EXTRAMAKEARGS = O=$(O)
-endif
-
# Set variables related to in-tree or out-of-tree build.
ifeq ($(O),output)
CONFIG_DIR := $(TOPDIR)
--
2.10.0
^ permalink raw reply related [flat|nested] 5+ messages in thread* [Buildroot] [PATCH v12 3/3] core: re-enter make if $(CURDIR) or $(O) are not canonical paths
2016-10-17 21:05 [Buildroot] [PATCH v12 0/3] Relocatable SDK: preparatory changes Samuel Martin
2016-10-17 21:05 ` [Buildroot] [PATCH v12 1/3] core: split variables definition related to in/out-of-tree build from O itself Samuel Martin
2016-10-17 21:05 ` [Buildroot] [PATCH v12 2/3] core: reorder top-level Makefile and document things Samuel Martin
@ 2016-10-17 21:05 ` Samuel Martin
2016-10-19 21:27 ` [Buildroot] [PATCH v12 0/3] Relocatable SDK: preparatory changes Thomas Petazzoni
3 siblings, 0 replies; 5+ messages in thread
From: Samuel Martin @ 2016-10-17 21:05 UTC (permalink / raw)
To: buildroot
When $(CURDIR) and/or $(O) contain symlinks in their paths, they can be
resolved differently, depending on each package build-system (whether it
uses the given paths or get the absolute canonical ones).
Using absolute canonical paths will help achieving reproducible builds and
will make easier tracking down host machine paths leaking into the host,
target or staging trees.
So, this change ensures the build takes place with the CURDIR and O
variables are set to their absolute canonical paths.
In order to recall the toplevel makefile with absolute canonical paths
for $(CURDIR) and $(O), we need to:
1- Compute the absolute canonical paths for $(CURDIR) and $(O) that will
be passed to the sub-make. This is achieved using the 'realpath' make
primitive. However, some care must be taken when manipulating O:
- the out-of-tree makefile wrapper happens a trailing "/.", we need
to strip this part away to not break the comparison driving the
sub-make call;
- the user can leave a trailing '/' to $(O);
- according to [1,2], realpath returns an empty string in case of
non-existing entry. So, to avoid passing an empty O= variable to
sub-make, it is necessary to define the output directory and create
it prior to call realpath on it (because on the first invocation,
$(O) usually does not yet exists), hence the trick doing the mkdir
right before calling realpath.
2- Update EXTRAMAKEARGS with the absolute canonical $(O) and use it
when call recalling the top-level makefile with umask and paths
correctly set.
3- Lastly, update the condition for setting the CONFIG_DIR and
NEED_WRAPPER variables.
Note:
* This change takes care of the makefile wrapper installed in $(O) to
avoid unneeded make recursion.
[1] https://www.gnu.org/software/make/manual/html_node/File-Name-Functions.html
[2] http://man7.org/linux/man-pages/man3/realpath.3.html
Reported-by: Matthew Weber <matt@thewebers.ws>
Cc: Matthew Weber <matt@thewebers.ws>
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
changes v11->v12:
- fix typos in commit logs (Arnout)
- add comment when defining CANONICAL_O (Arnout)
- only update EXTRAMAKEARGS, instead of re-introducing it after its removal
by mistake (spotted by Arnout)
- update A/R/T tags
changes v10->v11:
- none
changes v9->v10:
- only keep what is about checking umask and canonical paths for CWD and
O (Arnout)
- fix assignation (Arnout)
- remove the unrelated MAKEOVERRIDES cleanup (Arnout)
- fixes typos in commit log (Arnout)
changes v8->v9:
- none
changes v7->v8:
- keep @ at the beginning of the command (Yann)
- make ifneq condition easier to read/parsed (Yann)
- fix O definition before re-entering make (Reported by Matthew)
- use EXTRAMAKEARGS when re-entering make
- update the condition for CONFIG_DIR and NEED_WRAPPER
changes v6->v7:
- none
changes v5->v6:
- new patch
---
Makefile | 40 ++++++++++++++++++++++++++++++----------
1 file changed, 30 insertions(+), 10 deletions(-)
diff --git a/Makefile b/Makefile
index 717b2db..3ba1490 100644
--- a/Makefile
+++ b/Makefile
@@ -33,7 +33,7 @@ SHELL := $(shell if [ -x "$$BASH" ]; then echo $$BASH; \
# or avoid confusing packages that can use the O=<dir> syntax for out-of-tree
# build by preventing it from being forwarded to sub-make calls.
ifneq ("$(origin O)", "command line")
-O := output
+O := $(CURDIR)/output
else
# Other packages might also support Linux-style out of tree builds
# with the O=<dir> syntax (E.G. BusyBox does). As make automatically
@@ -50,27 +50,46 @@ endif
# Check if the current Buildroot execution meets all the pre-requisites.
# If they are not met, Buildroot will actually do its job in a sub-make meeting
-# its pre-requisites, which is:
+# its pre-requisites, which are:
# 1- Permissive enough umask:
# Wrong or too restrictive umask will prevent Buildroot and packages from
# creating files and directories.
+# 2- Absolute canonical CWD (i.e. $(CURDIR)):
+# Otherwise, some packages will use CWD as-is, others will compute its
+# absolute canonical path. This makes harder tracking and fixing host
+# machine path leaks.
+# 3- Absolute canonical output location (i.e. $(O)):
+# For the same reason as the one for CWD.
+
+# Remove the trailing '/.' from $(O) as it can be added by the makefile wrapper
+# installed in the $(O) directory.
+# Also remove the trailing '/' the user can set when on the command line.
+override O := $(patsubst %/,%,$(patsubst %.,%,$(O)))
+# Make sure $(O) actually exists before calling realpath on it; this is to
+# avoid empty CANONICAL_O in case on non-existing entry.
+CANONICAL_O := $(shell mkdir -p $(O) >/dev/null 2>&1)$(realpath $(O))
+
+CANONICAL_CURDIR = $(realpath $(CURDIR))
REQ_UMASK = 0022
-# we need to pass O= everywhere we call back into the toplevel makefile
-EXTRAMAKEARGS = O=$(O)
+# Make sure O= is passed (with its absolute canonical path) everywhere the
+# toplevel makefile is called back.
+EXTRAMAKEARGS := O=$(CANONICAL_O)
# Check Buildroot execution pre-requisites here.
-ifneq ($(shell umask),$(REQ_UMASK))
+ifneq ($(shell umask):$(CURDIR):$(O),$(REQ_UMASK):$(CANONICAL_CURDIR):$(CANONICAL_O))
.PHONY: _all $(MAKECMDGOALS)
$(MAKECMDGOALS): _all
@:
_all:
- @umask $(REQ_UMASK) && $(MAKE) --no-print-directory $(MAKECMDGOALS)
+ @umask $(REQ_UMASK) && \
+ $(MAKE) -C $(CANONICAL_CURDIR) --no-print-directory \
+ $(MAKECMDGOALS) $(EXTRAMAKEARGS)
-else # umask
+else # umask / $(CURDIR) / $(O)
# This is our default rule, so must come first
all:
@@ -140,8 +159,9 @@ endif
include support/misc/utils.mk
# Set variables related to in-tree or out-of-tree build.
-ifeq ($(O),output)
-CONFIG_DIR := $(TOPDIR)
+# Here, both $(O) and $(CURDIR) are absolute canonical paths.
+ifeq ($(O),$(CURDIR)/output)
+CONFIG_DIR := $(CURDIR)
NEED_WRAPPER =
else
CONFIG_DIR := $(O)
@@ -1038,4 +1058,4 @@ include docs/manual/manual.mk
.PHONY: $(noconfig_targets)
-endif #umask
+endif #umask / $(CURDIR) / $(O)
--
2.10.0
^ permalink raw reply related [flat|nested] 5+ messages in thread* [Buildroot] [PATCH v12 0/3] Relocatable SDK: preparatory changes
2016-10-17 21:05 [Buildroot] [PATCH v12 0/3] Relocatable SDK: preparatory changes Samuel Martin
` (2 preceding siblings ...)
2016-10-17 21:05 ` [Buildroot] [PATCH v12 3/3] core: re-enter make if $(CURDIR) or $(O) are not canonical paths Samuel Martin
@ 2016-10-19 21:27 ` Thomas Petazzoni
3 siblings, 0 replies; 5+ messages in thread
From: Thomas Petazzoni @ 2016-10-19 21:27 UTC (permalink / raw)
To: buildroot
Hello,
On Mon, 17 Oct 2016 23:05:40 +0200, Samuel Martin wrote:
> Samuel Martin (3):
> core: split variables definition related to in/out-of-tree build from
> O itself
> core: reorder top-level Makefile and document things
> core: re-enter make if $(CURDIR) or $(O) are not canonical paths
I've merged those three patches. Thanks a lot!
Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com
^ permalink raw reply [flat|nested] 5+ messages in thread