Re: [PATCH v2] Makefile: dedup archives in $(LIBS) so link recipes don't repeat them

[Junio C Hamano <gitster@pobox.com>]

Harald Nordgren <haraldnordgren@gmail.com> writes:

> I think this would be quite nice to fix for all the macOS developers
> (I don't know how many we have who are active on this list), but when
> running repeated tests it does take up some space on the terminal:
>
> ````
> ❯ git rebase --keep-base -x 'make -s && cd t && prove -j8
> t345?-history*.sh && echo'
>
> Executing: make -s && cd t && prove -j8 t345?-history*.sh && echo
> GIT_VERSION=2.55.0.rc1.20.g1e31474ef6
> ld: warning: ignoring duplicate libraries: 'libgit.a',
> 'target/release/libgitcore.a'
> ld: warning: ignoring duplicate libraries: 'libgit.a',
> 'target/release/libgitcore.a'

While I am very sympathetic that it may be annoying, I have to
wonder if that is ultimately the linker's job to accept the same
library listed twice on the same command line, deside when it can
ignore the second one, and *silently* ignore it.

Imagine this situation.

 - There are two library archives, libA.a has a.o in it and libB.a
   has b.o in it, respectively.

 - The object file a.o defines a symbol that b.o needs, and b.o
   defines a symbol a.o needs (i.e., mutually dependent). libA.a and
   libB.a have other symbols in them. There are valid reasons why we
   do not want to combine them into a single libAB.a.

 - Now our program X uses both libraries and we build and try to link it this way:

   $(CC) -c x.c				# this builds x.o
   $(CC) -o programX x.o libA.a libB.a  # unfortunately does not work as-is

   which fails because x.o uses symbol from libA.a that is not in
   a.o (so a.o is not linked), and then x.o also uses something in
   b.o that is picked up from libB.a.  But b.o in turn needs a.o
   that we already skipped.  One way to make it work is to tweak the
   final link phase to read like this:

   $(CC) -o programX x.o libA.a libB.a libA.a

If your linker complains because we list libA.a twice, it would be
annoying.

I guess if we can assume GNU ld (e.g., gcc/clang), we can use

   $(CC) -o programX x.o -Wl,--start-group libA.a libB.a -Wl,end-group

to tell the linker that they need to be processed for circular
dependencies, but listing them twice is more portable and harmless
(i.e., if all the symbols are resolved by the time the linker sees
the second libA.a, then it would not pick up anything extra from
there) way to achieve the same thing.  

So from future-proofing and portability perspective (which is
another way to say maintainability we care about), I would very much
prefer to see this solved at the linker level, allowing the build
procedure to list the same library twice on the command line.

It seems that on the Internet various folks, including masonbuild
and CMake, have heard complaints from users enough and fixed the
linker by using -no_warn_duplicate_libraries option.  Their approach
translates to something like the following in our build environment.

 config.mak.uname | 4 ++++
 1 file changed, 4 insertions(+)

diff --git i/config.mak.uname w/config.mak.uname
index 8719e09f66..e29eaaf3fd 100644
--- i/config.mak.uname
+++ w/config.mak.uname
@@ -149,6 +149,10 @@ ifeq ($(uname_S),Darwin)
         ifeq ($(shell test "`expr "$(uname_R)" : '\([0-9][0-9]*\)\.'`" -ge 20 && echo 1),1)
 		OPEN_RETURNS_EINTR = UnfortunatelyYes
         endif
+
+	# NEEDSWORK: do this only for XCode 15 or later
+	BASIC_LDFLAGS += -Wl,-no_warn_duplicate_libraries
+
 	NO_MEMMEM = YesPlease
 	USE_ST_TIMESPEC = YesPlease
 	HAVE_DEV_TTY = YesPlease