[PATCH v4 0/2] Makefile: libfdt: build only the strict necessary

[PATCH v4 0/2] Makefile: libfdt: build only the strict necessary

[Claudio Fontana]

v3 -> v4: NB! only useful when updating to latest dtc (not in QEMU yet)

* changed the approach to leverage new upstream dtc Makefile,
  needs dtc submodule update to include upstream dtc commit
  85e5d839847af54efab170f2b1331b2a6421e647.

* LIBFDT_srcdir does not exist anymore in upstream dtc: it is also
  not used anymore.

* LIBFDT_lib: need to avoid building libfdt.so at least for now, so pass as
  empty to avoid building the .so. This is to avoid breaking existing
  trees with objects already compiled without -fPIC.

* clean: no need to make dtc clean target, every artifact is captured by
  global clean rule

v2 -> v3:

* changed into a 2 patch series; in the second patch we remove the old
  compatibility gunks that were meant for removal some time after 4.1.

* renamed the libfdt PHONY rule to dtc/all, with the intent to make
  existing working trees forward and backward compatible across the change.

v1 -> v2:

* fix error generated when running UNCHECKED_GOALS without prior configure,
  for example during make docker-image-fedora. Without configure, DSOSUF is
  empty, and the module pattern rule in rules.mak that uses this variable
  can match too much; provide a default in the Makefile to avoid it.

* only attempt to build the archive when there is a non-empty list of objects.
  This could be done in general for the %.a: pattern in rules.mak, but maybe
  there are valid reasons to build an empty .a?

* removed some intermediate variables that did not add much value
  (LIBFDT_srcdir, LIBFDT_archive)

Tested locally with 3 VPATH configurations (no-, VPATH, VPATH in src subdir),
and with docker-image-fedora, docker-test-debug@fedora that failed before.

Claudio Fontana (2):
  Makefile: libfdt: build only the strict necessary
  Makefile: remove old compatibility gunks

 Makefile  | 32 ++++++++++++++++----------------
 configure |  6 +-----
 rules.mak |  2 ++
 3 files changed, 19 insertions(+), 21 deletions(-)

-- 
2.16.4

[PATCH v4 1/2] Makefile: dtc: build the libfdt target

[Claudio Fontana]

call the libfdt target from the dtc Makefile, which has been
changed to not require bison, flex, etc.

scripts/ symlink and tests directory creation are not necessary,
and neither is calling the clean rule explicitly.

Signed-off-by: Claudio Fontana <cfontana@suse.de>
---
 Makefile  | 10 +++++-----
 configure |  1 -
 2 files changed, 5 insertions(+), 6 deletions(-)

diff --git a/Makefile b/Makefile
index 34275f57c9..36a6454295 100644
--- a/Makefile
+++ b/Makefile
@@ -526,13 +526,14 @@ $(SOFTMMU_FUZZ_RULES): $(edk2-decompressed)
 $(TARGET_DIRS_RULES):
 	$(call quiet-command,$(MAKE) $(SUBDIR_MAKEFLAGS) -C $(dir $@) V="$(V)" TARGET_DIR="$(dir $@)" $(notdir $@),)
 
-DTC_MAKE_ARGS=-I$(SRC_PATH)/dtc VPATH=$(SRC_PATH)/dtc -C dtc V="$(V)" LIBFDT_srcdir=$(SRC_PATH)/dtc/libfdt
+# LIBFDT_lib="": avoid breaking existing trees with objects requiring -fPIC
+DTC_MAKE_ARGS=-I$(SRC_PATH)/dtc VPATH=$(SRC_PATH)/dtc -C dtc V="$(V)" LIBFDT_lib=""
 DTC_CFLAGS=$(CFLAGS) $(QEMU_CFLAGS)
-DTC_CPPFLAGS=-I$(BUILD_DIR)/dtc -I$(SRC_PATH)/dtc -I$(SRC_PATH)/dtc/libfdt
+DTC_CPPFLAGS=-I$(SRC_PATH)/dtc/libfdt
 
 .PHONY: dtc/all
-dtc/all: .git-submodule-status dtc/libfdt dtc/tests
-	$(call quiet-command,$(MAKE) $(DTC_MAKE_ARGS) CPPFLAGS="$(DTC_CPPFLAGS)" CFLAGS="$(DTC_CFLAGS)" LDFLAGS="$(QEMU_LDFLAGS)" ARFLAGS="$(ARFLAGS)" CC="$(CC)" AR="$(AR)" LD="$(LD)" $(SUBDIR_MAKEFLAGS) libfdt/libfdt.a,)
+dtc/all: .git-submodule-status dtc/libfdt
+	$(call quiet-command,$(MAKE) $(DTC_MAKE_ARGS) CPPFLAGS="$(DTC_CPPFLAGS)" CFLAGS="$(DTC_CFLAGS)" LDFLAGS="$(QEMU_LDFLAGS)" ARFLAGS="$(ARFLAGS)" CC="$(CC)" AR="$(AR)" LD="$(LD)" $(SUBDIR_MAKEFLAGS) libfdt,)
 
 dtc/%: .git-submodule-status
 	@mkdir -p $@
@@ -820,7 +821,6 @@ distclean: clean
 	rm -rf $$d || exit 1 ; \
         done
 	rm -Rf .sdk
-	if test -f dtc/version_gen.h; then $(MAKE) $(DTC_MAKE_ARGS) clean; fi
 
 KEYMAPS=da     en-gb  et  fr     fr-ch  is  lt  no  pt-br  sv \
 ar      de     en-us  fi  fr-be  hr     it  lv  nl         pl  ru     th \
diff --git a/configure b/configure
index 0d69c360c0..42554792ec 100755
--- a/configure
+++ b/configure
@@ -4281,7 +4281,6 @@ EOF
           mkdir -p dtc
           if [ "$pwd_is_source_path" != "y" ] ; then
               symlink "$source_path/dtc/Makefile" "dtc/Makefile"
-              symlink "$source_path/dtc/scripts" "dtc/scripts"
           fi
           fdt_cflags="-I\$(SRC_PATH)/dtc/libfdt"
           fdt_ldflags="-L\$(BUILD_DIR)/dtc/libfdt"
-- 
2.16.4

[PATCH v4 2/2] Makefile: remove old compatibility gunks

[Claudio Fontana]

Signed-off-by: Claudio Fontana <cfontana@suse.de>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 Makefile | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/Makefile b/Makefile
index 36a6454295..2873d59ea2 100644
--- a/Makefile
+++ b/Makefile
@@ -562,12 +562,6 @@ slirp/all: .git-submodule-status
 		CC="$(CC)" AR="$(AR)" 	LD="$(LD)" RANLIB="$(RANLIB)"	\
 		CFLAGS="$(QEMU_CFLAGS) $(CFLAGS)" LDFLAGS="$(QEMU_LDFLAGS)")
 
-# Compatibility gunk to keep make working across the rename of targets
-# for recursion, to be removed some time after 4.1.
-subdir-dtc: dtc/all
-subdir-capstone: capstone/all
-subdir-slirp: slirp/all
-
 $(filter %/all, $(TARGET_DIRS_RULES)): libqemuutil.a $(common-obj-y) \
 	$(qom-obj-y)
 
-- 
2.16.4

Re: [PATCH v4 0/2] Makefile: libfdt: build only the strict necessary

[Philippe Mathieu-Daudé]

Hi Claudio,

On 5/14/20 9:38 PM, Claudio Fontana wrote:
> v3 -> v4: NB! only useful when updating to latest dtc (not in QEMU yet)
> 
> * changed the approach to leverage new upstream dtc Makefile,
>    needs dtc submodule update to include upstream dtc commit
>    85e5d839847af54efab170f2b1331b2a6421e647.

I don't have this commit locally:

fatal: bad object 85e5d839847af54efab170f2b1331b2a6421e647

My remote is https://git.qemu.org/git/dtc.git, might it be de-synchronized?

> 
> * LIBFDT_srcdir does not exist anymore in upstream dtc: it is also
>    not used anymore.

Why not include the submodule update in the first patch?

> 
> * LIBFDT_lib: need to avoid building libfdt.so at least for now, so pass as
>    empty to avoid building the .so. This is to avoid breaking existing
>    trees with objects already compiled without -fPIC.
> 
> * clean: no need to make dtc clean target, every artifact is captured by
>    global clean rule
> 
> v2 -> v3:
> 
> * changed into a 2 patch series; in the second patch we remove the old
>    compatibility gunks that were meant for removal some time after 4.1.
> 
> * renamed the libfdt PHONY rule to dtc/all, with the intent to make
>    existing working trees forward and backward compatible across the change.
> 
> v1 -> v2:
> 
> * fix error generated when running UNCHECKED_GOALS without prior configure,
>    for example during make docker-image-fedora. Without configure, DSOSUF is
>    empty, and the module pattern rule in rules.mak that uses this variable
>    can match too much; provide a default in the Makefile to avoid it.
> 
> * only attempt to build the archive when there is a non-empty list of objects.
>    This could be done in general for the %.a: pattern in rules.mak, but maybe
>    there are valid reasons to build an empty .a?
> 
> * removed some intermediate variables that did not add much value
>    (LIBFDT_srcdir, LIBFDT_archive)
> 
> Tested locally with 3 VPATH configurations (no-, VPATH, VPATH in src subdir),
> and with docker-image-fedora, docker-test-debug@fedora that failed before.
> 
> Claudio Fontana (2):
>    Makefile: libfdt: build only the strict necessary
>    Makefile: remove old compatibility gunks
> 
>   Makefile  | 32 ++++++++++++++++----------------
>   configure |  6 +-----
>   rules.mak |  2 ++
>   3 files changed, 19 insertions(+), 21 deletions(-)
> 

Re: [PATCH v4 0/2] Makefile: libfdt: build only the strict necessary

[Claudio Fontana]

On 5/15/20 9:50 AM, Philippe Mathieu-Daudé wrote:
> Hi Claudio,
> 
> On 5/14/20 9:38 PM, Claudio Fontana wrote:
>> v3 -> v4: NB! only useful when updating to latest dtc (not in QEMU yet)
>>
>> * changed the approach to leverage new upstream dtc Makefile,
>>    needs dtc submodule update to include upstream dtc commit
>>    85e5d839847af54efab170f2b1331b2a6421e647.
> 
> I don't have this commit locally:
> 
> fatal: bad object 85e5d839847af54efab170f2b1331b2a6421e647
> 
> My remote is https://git.qemu.org/git/dtc.git, might it be de-synchronized?

Hi Philippe, maybe that commit is not in the qemu dtc.git yet, only in David's tree at:

https://github.com/dgibson/dtc.git

> 
>>
>> * LIBFDT_srcdir does not exist anymore in upstream dtc: it is also
>>    not used anymore.
> 
> Why not include the submodule update in the first patch?

Right, I am not sure how git.qemu.org/git/dtc.git is updated?

I guess once it is, I should include the submodule update.

Thanks!

Claudio

> 
>>
>> * LIBFDT_lib: need to avoid building libfdt.so at least for now, so pass as
>>    empty to avoid building the .so. This is to avoid breaking existing
>>    trees with objects already compiled without -fPIC.
>>
>> * clean: no need to make dtc clean target, every artifact is captured by
>>    global clean rule
>>
>> v2 -> v3:
>>
>> * changed into a 2 patch series; in the second patch we remove the old
>>    compatibility gunks that were meant for removal some time after 4.1.
>>
>> * renamed the libfdt PHONY rule to dtc/all, with the intent to make
>>    existing working trees forward and backward compatible across the change.
>>
>> v1 -> v2:
>>
>> * fix error generated when running UNCHECKED_GOALS without prior configure,
>>    for example during make docker-image-fedora. Without configure, DSOSUF is
>>    empty, and the module pattern rule in rules.mak that uses this variable
>>    can match too much; provide a default in the Makefile to avoid it.
>>
>> * only attempt to build the archive when there is a non-empty list of objects.
>>    This could be done in general for the %.a: pattern in rules.mak, but maybe
>>    there are valid reasons to build an empty .a?
>>
>> * removed some intermediate variables that did not add much value
>>    (LIBFDT_srcdir, LIBFDT_archive)
>>
>> Tested locally with 3 VPATH configurations (no-, VPATH, VPATH in src subdir),
>> and with docker-image-fedora, docker-test-debug@fedora that failed before.
>>
>> Claudio Fontana (2):
>>    Makefile: libfdt: build only the strict necessary
>>    Makefile: remove old compatibility gunks
>>
>>   Makefile  | 32 ++++++++++++++++----------------
>>   configure |  6 +-----
>>   rules.mak |  2 ++
>>   3 files changed, 19 insertions(+), 21 deletions(-)
>>
> 

Re: [PATCH v4 0/2] Makefile: libfdt: build only the strict necessary

[Philippe Mathieu-Daudé]

On 5/15/20 9:56 AM, Claudio Fontana wrote:
> On 5/15/20 9:50 AM, Philippe Mathieu-Daudé wrote:
>> Hi Claudio,
>>
>> On 5/14/20 9:38 PM, Claudio Fontana wrote:
>>> v3 -> v4: NB! only useful when updating to latest dtc (not in QEMU yet)
>>>
>>> * changed the approach to leverage new upstream dtc Makefile,
>>>     needs dtc submodule update to include upstream dtc commit
>>>     85e5d839847af54efab170f2b1331b2a6421e647.
>>
>> I don't have this commit locally:
>>
>> fatal: bad object 85e5d839847af54efab170f2b1331b2a6421e647
>>
>> My remote is https://git.qemu.org/git/dtc.git, might it be de-synchronized?
> 
> Hi Philippe, maybe that commit is not in the qemu dtc.git yet, only in David's tree at:
> 
> https://github.com/dgibson/dtc.git
> 
>>
>>>
>>> * LIBFDT_srcdir does not exist anymore in upstream dtc: it is also
>>>     not used anymore.
>>
>> Why not include the submodule update in the first patch?
> 
> Right, I am not sure how git.qemu.org/git/dtc.git is updated?
> 
> I guess once it is, I should include the submodule update.

OK. Maybe David want to make a libdtc release first, before to update 
QEMU submodule.

Meanwhile using David's remote, and the submodule update:

-- >8 --
diff --git a/dtc b/dtc
index 88f18909db..85e5d83984 160000
--- a/dtc
+++ b/dtc
@@ -1 +1 @@
-Subproject commit 88f18909db731a627456f26d779445f84e449536
+Subproject commit 85e5d839847af54efab170f2b1331b2a6421e647
---

for the series:
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>

> 
> Thanks!
> 
> Claudio
> 
>>
>>>
>>> * LIBFDT_lib: need to avoid building libfdt.so at least for now, so pass as
>>>     empty to avoid building the .so. This is to avoid breaking existing
>>>     trees with objects already compiled without -fPIC.
>>>
>>> * clean: no need to make dtc clean target, every artifact is captured by
>>>     global clean rule
>>>
>>> v2 -> v3:
>>>
>>> * changed into a 2 patch series; in the second patch we remove the old
>>>     compatibility gunks that were meant for removal some time after 4.1.
>>>
>>> * renamed the libfdt PHONY rule to dtc/all, with the intent to make
>>>     existing working trees forward and backward compatible across the change.
>>>
>>> v1 -> v2:
>>>
>>> * fix error generated when running UNCHECKED_GOALS without prior configure,
>>>     for example during make docker-image-fedora. Without configure, DSOSUF is
>>>     empty, and the module pattern rule in rules.mak that uses this variable
>>>     can match too much; provide a default in the Makefile to avoid it.
>>>
>>> * only attempt to build the archive when there is a non-empty list of objects.
>>>     This could be done in general for the %.a: pattern in rules.mak, but maybe
>>>     there are valid reasons to build an empty .a?
>>>
>>> * removed some intermediate variables that did not add much value
>>>     (LIBFDT_srcdir, LIBFDT_archive)
>>>
>>> Tested locally with 3 VPATH configurations (no-, VPATH, VPATH in src subdir),
>>> and with docker-image-fedora, docker-test-debug@fedora that failed before.
>>>
>>> Claudio Fontana (2):
>>>     Makefile: libfdt: build only the strict necessary
>>>     Makefile: remove old compatibility gunks
>>>
>>>    Makefile  | 32 ++++++++++++++++----------------
>>>    configure |  6 +-----
>>>    rules.mak |  2 ++
>>>    3 files changed, 19 insertions(+), 21 deletions(-)
>>>
>>
> 

Re: [PATCH v4 0/2] Makefile: libfdt: build only the strict necessary

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 3395 bytes --]

On Fri, May 15, 2020 at 09:56:25AM +0200, Claudio Fontana wrote:
> On 5/15/20 9:50 AM, Philippe Mathieu-Daudé wrote:
> > Hi Claudio,
> > 
> > On 5/14/20 9:38 PM, Claudio Fontana wrote:
> >> v3 -> v4: NB! only useful when updating to latest dtc (not in QEMU yet)
> >>
> >> * changed the approach to leverage new upstream dtc Makefile,
> >>    needs dtc submodule update to include upstream dtc commit
> >>    85e5d839847af54efab170f2b1331b2a6421e647.
> > 
> > I don't have this commit locally:
> > 
> > fatal: bad object 85e5d839847af54efab170f2b1331b2a6421e647
> > 
> > My remote is https://git.qemu.org/git/dtc.git, might it be de-synchronized?
> 
> Hi Philippe, maybe that commit is not in the qemu dtc.git yet, only in David's tree at:
> 
> https://github.com/dgibson/dtc.git

Oops, I forgot to push it to the master tree at
git://git.kernel.org/pub/scm/utils/dtc/dtc.git

Done now.  The qemu.org mirror should update within a day or so.

> >> * LIBFDT_srcdir does not exist anymore in upstream dtc: it is also
> >>    not used anymore.
> > 
> > Why not include the submodule update in the first patch?
> 
> Right, I am not sure how git.qemu.org/git/dtc.git is updated?
> 
> I guess once it is, I should include the submodule update.
> 
> Thanks!
> 
> Claudio
> 
> > 
> >>
> >> * LIBFDT_lib: need to avoid building libfdt.so at least for now, so pass as
> >>    empty to avoid building the .so. This is to avoid breaking existing
> >>    trees with objects already compiled without -fPIC.
> >>
> >> * clean: no need to make dtc clean target, every artifact is captured by
> >>    global clean rule
> >>
> >> v2 -> v3:
> >>
> >> * changed into a 2 patch series; in the second patch we remove the old
> >>    compatibility gunks that were meant for removal some time after 4.1.
> >>
> >> * renamed the libfdt PHONY rule to dtc/all, with the intent to make
> >>    existing working trees forward and backward compatible across the change.
> >>
> >> v1 -> v2:
> >>
> >> * fix error generated when running UNCHECKED_GOALS without prior configure,
> >>    for example during make docker-image-fedora. Without configure, DSOSUF is
> >>    empty, and the module pattern rule in rules.mak that uses this variable
> >>    can match too much; provide a default in the Makefile to avoid it.
> >>
> >> * only attempt to build the archive when there is a non-empty list of objects.
> >>    This could be done in general for the %.a: pattern in rules.mak, but maybe
> >>    there are valid reasons to build an empty .a?
> >>
> >> * removed some intermediate variables that did not add much value
> >>    (LIBFDT_srcdir, LIBFDT_archive)
> >>
> >> Tested locally with 3 VPATH configurations (no-, VPATH, VPATH in src subdir),
> >> and with docker-image-fedora, docker-test-debug@fedora that failed before.
> >>
> >> Claudio Fontana (2):
> >>    Makefile: libfdt: build only the strict necessary
> >>    Makefile: remove old compatibility gunks
> >>
> >>   Makefile  | 32 ++++++++++++++++----------------
> >>   configure |  6 +-----
> >>   rules.mak |  2 ++
> >>   3 files changed, 19 insertions(+), 21 deletions(-)
> >>
> > 
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]