Git configure/make does not honor ARFLAGS

Git configure/make does not honor ARFLAGS

[Jeffrey Walton]

I'm working on an old OS X machine. I needed to perform:

  AR=libtool
  ARFLAGS="-static -o"
  ...
  make configure
  ./configure ...
  make

However, it appears the Makefile does not respect ARFLAGS:

    $ grep -IR '$(AR)' *
    Makefile:    $(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $^
    Makefile:    $(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $^
    Makefile:    $(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $^

It was fixed with a quick "sed":

    sed -i "" 's|$(AR) rcs|$(AR) $(ARFLAGS)|g' Makefile

The Makefile might benefit from the following for users who need to
tweak things:

    ARFLAGS ?= rcs
    ...

    $(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^
    ...

Re: Git configure/make does not honor ARFLAGS

[Jeff King]

On Sun, Aug 30, 2015 at 05:34:59PM -0400, Jeffrey Walton wrote:

> I'm working on an old OS X machine. I needed to perform:
> 
>   AR=libtool
>   ARFLAGS="-static -o"
>   ...
>   make configure
>   ./configure ...
>   make

Hrm. Your "$(AR)" is not really "ar" then, is it? It has been a long
time since I played with libtool, but what is the reason that you are
calling libtool and not "ar" in the first place. Is it that you do not
have "ar" at all, and libtool performs some other procedure? If so, is
there a more ar-compatible wrapper that can be used?

> The Makefile might benefit from the following for users who need to
> tweak things:
> 
>     ARFLAGS ?= rcs
>     ...
> 
>     $(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^
>     ...

Yeah, that does sound reasonable (even if one does not set $(AR) to
something completely different, they might need slightly different
flags).

Care to send a patch?

-Peff

Re: Git configure/make does not honor ARFLAGS

[Eric Sunshine]

On Sun, Sep 13, 2015 at 6:17 AM, Jeff King <peff@peff.net> wrote:
> On Sun, Aug 30, 2015 at 05:34:59PM -0400, Jeffrey Walton wrote:
>> I'm working on an old OS X machine. I needed to perform:
>>
>>   AR=libtool
>>   ARFLAGS="-static -o"
>>   ...
>>   make configure
>>   ./configure ...
>>   make
>
> Hrm. Your "$(AR)" is not really "ar" then, is it? It has been a long
> time since I played with libtool, but what is the reason that you are
> calling libtool and not "ar" in the first place. Is it that you do not
> have "ar" at all, and libtool performs some other procedure? If so, is
> there a more ar-compatible wrapper that can be used?

This isn't GNU's libtool. It's Apple's libtool, an entirely different
beast, which is an 'ar' replacement and is needed when linking
Universal binaries containing code for more than one architecture,
such as 'ppc' and 'i386', so the same executable can run on multiple
architectures. This tool dates all the way back to at least NextStep
3.1 when NeXT ported NextStep to Intel hardware (i486) from NeXT
computers (m68k). The name "Universal" is an Apple invention, but back
in the NeXT days, they were called Multi-Architecture Binaries (MAB)
or, colloquially, just FAT (for "fat"); there was a corresponding
"lipo" command (short for "liposuction") to "thin" out "fat" binaries.
NeXT's libtool predates GNU's libtool by a few years: May 1993 vs.
July 1997, respectively. When an attempt is made to use 'ar' on
Universal object files, it errors out saying that it can't be used
with such files and recommends 'libtool' instead.

>> The Makefile might benefit from the following for users who need to
>> tweak things:
>>
>>     ARFLAGS ?= rcs
>>     ...
>>
>>     $(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^
>>     ...
>
> Yeah, that does sound reasonable (even if one does not set $(AR) to
> something completely different, they might need slightly different
> flags).

Re: Git configure/make does not honor ARFLAGS

[Jeffrey Walton]

On Sun, Sep 13, 2015 at 1:11 PM, Eric Sunshine <sunshine@sunshineco.com> wrote:
> On Sun, Sep 13, 2015 at 6:17 AM, Jeff King <peff@peff.net> wrote:
>> On Sun, Aug 30, 2015 at 05:34:59PM -0400, Jeffrey Walton wrote:
>>> I'm working on an old OS X machine. I needed to perform:
>>>
>>>   AR=libtool
>>>   ARFLAGS="-static -o"
>>>   ...
>>>   make configure
>>>   ./configure ...
>>>   make
>>
>> Hrm. Your "$(AR)" is not really "ar" then, is it? It has been a long
>> time since I played with libtool, but what is the reason that you are
>> calling libtool and not "ar" in the first place. Is it that you do not
>> have "ar" at all, and libtool performs some other procedure? If so, is
>> there a more ar-compatible wrapper that can be used?
>
> This isn't GNU's libtool. It's Apple's libtool, an entirely different
> beast, which is an 'ar' replacement and is needed when linking
> Universal binaries containing code for more than one architecture,
> such as 'ppc' and 'i386', so the same executable can run on multiple
> architectures. This tool dates all the way back to at least NextStep
> 3.1 when NeXT ported NextStep to Intel hardware (i486) from NeXT
> computers (m68k). The name "Universal" is an Apple invention, but back
> in the NeXT days, they were called Multi-Architecture Binaries (MAB)
> or, colloquially, just FAT (for "fat"); there was a corresponding
> "lipo" command (short for "liposuction") to "thin" out "fat" binaries.
> NeXT's libtool predates GNU's libtool by a few years: May 1993 vs.
> July 1997, respectively. When an attempt is made to use 'ar' on
> Universal object files, it errors out saying that it can't be used
> with such files and recommends 'libtool' instead.

Thanks Eric. You did a much better job than I would have done.

JW

Re: Git configure/make does not honor ARFLAGS

[Jeff King]

On Sun, Sep 13, 2015 at 01:11:46PM -0400, Eric Sunshine wrote:

> > Hrm. Your "$(AR)" is not really "ar" then, is it? It has been a long
> > time since I played with libtool, but what is the reason that you are
> > calling libtool and not "ar" in the first place. Is it that you do not
> > have "ar" at all, and libtool performs some other procedure? If so, is
> > there a more ar-compatible wrapper that can be used?
> 
> This isn't GNU's libtool. It's Apple's libtool, an entirely different
> beast, which is an 'ar' replacement and is needed when linking
> Universal binaries containing code for more than one architecture,
> such as 'ppc' and 'i386', so the same executable can run on multiple
> architectures. This tool dates all the way back to at least NextStep
> 3.1 when NeXT ported NextStep to Intel hardware (i486) from NeXT
> computers (m68k). The name "Universal" is an Apple invention, but back
> in the NeXT days, they were called Multi-Architecture Binaries (MAB)
> or, colloquially, just FAT (for "fat"); there was a corresponding
> "lipo" command (short for "liposuction") to "thin" out "fat" binaries.
> NeXT's libtool predates GNU's libtool by a few years: May 1993 vs.
> July 1997, respectively. When an attempt is made to use 'ar' on
> Universal object files, it errors out saying that it can't be used
> with such files and recommends 'libtool' instead.

Ah, OK. Today I learned something. :)

Jeffrey, can you produce a tested patch which works for you?

-Peff

Re: Git configure/make does not honor ARFLAGS

[Junio C Hamano]

On Sun, Sep 13, 2015 at 9:30 PM, Jeff King <peff@peff.net> wrote:
>
> Ah, OK. Today I learned something. :)
>
> Jeffrey, can you produce a tested patch which works for you?

If I am not mistaken, I think I already have one on 'pu' (I think I did that
as an afternoon-tea time hack or something).

Re: Git configure/make does not honor ARFLAGS

[Jeff King]

On Sun, Sep 13, 2015 at 09:57:08PM -0700, Junio C Hamano wrote:

> On Sun, Sep 13, 2015 at 9:30 PM, Jeff King <peff@peff.net> wrote:
> >
> > Ah, OK. Today I learned something. :)
> >
> > Jeffrey, can you produce a tested patch which works for you?
> 
> If I am not mistaken, I think I already have one on 'pu' (I think I did that
> as an afternoon-tea time hack or something).

Oh, indeed. ac179b4d9. Looks good to me.

My follow-up question was going to be: is this something we should be
setting in config.mak.uname for appropriate versions of Darwin? It
wasn't clear to me from Eric's description if this is something that
particular versions need, or just something that people who want to
build Universal binaries would choose to use.

-Peff

Re: Git configure/make does not honor ARFLAGS

[Junio C Hamano]

On Sun, Sep 13, 2015 at 9:59 PM, Jeff King <peff@peff.net> wrote:
>
> My follow-up question was going to be: is this something we should be
> setting in config.mak.uname for appropriate versions of Darwin? It
> wasn't clear to me from Eric's description if this is something that
> particular versions need, or just something that people who want to
> build Universal binaries would choose to use.

My preference is not to worry anything about config.mak.uname
ourselves, until somebody who does work on the ports proposes
to do something concrete.

Re: Git configure/make does not honor ARFLAGS

[Eric Sunshine]

On Sun, Sep 13, 2015 at 10:52:36PM -0700, Junio C Hamano wrote:
> On Sun, Sep 13, 2015 at 9:59 PM, Jeff King <peff@peff.net> wrote:
> >
> > My follow-up question was going to be: is this something we should be
> > setting in config.mak.uname for appropriate versions of Darwin? It
> > wasn't clear to me from Eric's description if this is something that
> > particular versions need, or just something that people who want to
> > build Universal binaries would choose to use.
> 
> My preference is not to worry anything about config.mak.uname
> ourselves, until somebody who does work on the ports proposes
> to do something concrete.

Normal 'ar' works for non-multi-architecture-binaries (MAB);
'libtool' is only needed when building Universal. Unfortunately,
there probably isn't a reliable way to auto-detect a Universal build.
Back in the NextStep days, projects would support MAB via a
TARGET_ARCHS variable:

    make TARGET_ARCHS='m68k i386 sparc hppa'

And, for project's which didn't understand that, you'd just have to
specify build flags which the Makefile did understand:

    make CFLAGS='-arch ppc -arch i386' LDFLAGS='-arch ppc -arch i386'

or, just make ad-hoc modifications to the Makefile if it didn't even
respect those variables. So, I don't think there's really a good way
to detect MAB builds.

On the other hand, as far as I know, it's *always* safe to replace
'ar' with 'libtool' on this platform, so we could just do it
unconditionally.

--- 8< ---
diff --git a/config.mak.uname b/config.mak.uname
index be5cbec..e7970dd 100644
--- a/config.mak.uname
+++ b/config.mak.uname
@@ -88,6 +88,8 @@ ifeq ($(uname_S),SCO_SV)
 	TAR = gtar
 endif
 ifeq ($(uname_S),Darwin)
+	AR = libtool
+	ARFLAGS = -static -o
 	NEEDS_CRYPTO_WITH_SSL = YesPlease
 	NEEDS_SSL_WITH_CRYPTO = YesPlease
 	NEEDS_LIBICONV = YesPlease
--- 8< ---

I've tested this on modern Mac OS X, Yosemite 10.10.5 (x86_64), and
ancient Snow Leopard 10.5.8 PowerPC (circa 2009), and it works fine
in both cases, so perhaps that's the way to go.

My one concern, however, would be people who've installed GNU libtool
and have that in PATH before Apple's tools.

Re: Git configure/make does not honor ARFLAGS

[Eric Sunshine]

On Wed, Sep 16, 2015 at 3:38 PM, Eric Sunshine <sunshine@sunshineco.com> wrote:
> On the other hand, as far as I know, it's *always* safe to replace
> 'ar' with 'libtool' on this platform, so we could just do it
> unconditionally.
>
> --- 8< ---
>  ifeq ($(uname_S),Darwin)
> +       AR = libtool
> +       ARFLAGS = -static -o
> --- 8< ---
>
> I've tested this on modern Mac OS X, Yosemite 10.10.5 (x86_64), and
> ancient Snow Leopard 10.5.8 PowerPC (circa 2009), and it works fine
> in both cases, so perhaps that's the way to go.
>
> My one concern, however, would be people who've installed GNU libtool
> and have that in PATH before Apple's tools.

Although, perhaps specifying the full path to 'libtool' would be
sufficient so as not to worry about GNU libtool being picked up by
accident. Apple's command resides at /usr/bin/libtool on both modern
and ancient Mac OS X, so maybe that's all we need.

Re: Git configure/make does not honor ARFLAGS

[Jeffrey Walton]

On Mon, Sep 14, 2015 at 12:59 AM, Jeff King <peff@peff.net> wrote:
> On Sun, Sep 13, 2015 at 09:57:08PM -0700, Junio C Hamano wrote:
>
>> On Sun, Sep 13, 2015 at 9:30 PM, Jeff King <peff@peff.net> wrote:
>> >
>> > Ah, OK. Today I learned something. :)
>> >
>> > Jeffrey, can you produce a tested patch which works for you?
>>
>> If I am not mistaken, I think I already have one on 'pu' (I think I did that
>> as an afternoon-tea time hack or something).
>
> Oh, indeed. ac179b4d9. Looks good to me.
>
> My follow-up question was going to be: is this something we should be
> setting in config.mak.uname for appropriate versions of Darwin? It
> wasn't clear to me from Eric's description if this is something that
> particular versions need, or just something that people who want to
> build Universal binaries would choose to use.
>
If you have something that needs testing, then please let me know. I'd
be happy to test it for you.

My OS X test environment includes OS X 10.5 PoweMac (PowerPC), 10.8
Intel x64 (ancient C++ stdlib that claims to be C++11), and 10.9 Intel
x64.

If desired, I can give you SSH access to a couple of the machines.
OpenSSL and Cryptlib uses them on occasion for testing, too.

Jeff

Re: Git configure/make does not honor ARFLAGS

[Junio C Hamano]

Jeffrey Walton <noloader@gmail.com> writes:

>> Oh, indeed. ac179b4d9. Looks good to me.
>>
>> My follow-up question was going to be: is this something we should be
>> setting in config.mak.uname for appropriate versions of Darwin? It
>> wasn't clear to me from Eric's description if this is something that
>> particular versions need, or just something that people who want to
>> build Universal binaries would choose to use.
>>
> If you have something that needs testing, then please let me know. I'd
> be happy to test it for you.

In your copy of Git that is recent enough, please run this:

    $ git fetch git://git.kernel.org/pub/scm/git/git.git pu
    $ git checkout -b jw/make-arflags-customizable ac179b4d9

This gives you a rather old maintenance track of Git based on v2.0.5
plus a patch that makes $(ARFLAGS) customizable.

Presumably you are building a more recent versions of Git yourself.
I do not know which branch you are on, but you should be able to
merge this brancch to anything newer than v2.0.x maintenance track.
E.g.

    $ git checkout master
    $ git merge jw/make-arflags-customizable

should merge cleanly and would let you say things like:

    $ make AR=libtool ARFLAGS='-static -o'

with any other build customization you usually use on the command
line (or in config.mak, if you prefer).

What Jeff King was wondering was if those on platform like you have
would benefit if they have something like the attached patch on top
of the ac179b4d (Makefile: allow $(ARFLAGS) specified from the
command line, 2015-09-10) patch.  Pieces of information that is
needed to determine if that is a good idea are:

 * What exact condition we should use in the "ifeq" to identify the
   class of platforms that want this custom AR/ARFLAGS.

 * Ask if everybody who shares that particular platform would always
   want "libtool" and "-static -o" as their custom AR/ARFLAGS.  If
   the answer is "no", i.e. on the same platform, more than one
   compiler toolchain may be available to the users and not
   everybody may want to use "libtool" and "-static -o" lie you do,
   then the approach to auto-determine this settings does not help
   people, and we should not bother with shipping customized
   config.mak.uname (instead we'd just tell people to do whatever
   they want with AR and ARFLAGS from the command line).

 * If it is worth shipping customized config.mak.uname, determine
   what things, other than AR/ARFLAGS, would help the uses of the
   class of platforms we identified above in the new "ifeq" section
   (e.g. perhaps they may want CC=some-other-cc).

 config.mak.uname | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/config.mak.uname b/config.mak.uname
index 943c439..247dfed 100644
--- a/config.mak.uname
+++ b/config.mak.uname
@@ -46,6 +46,10 @@ ifeq ($(uname_S),GNU/kFreeBSD)
 	DIR_HAS_BSD_GROUP_SEMANTICS = YesPlease
 	LIBC_CONTAINS_LIBINTL = YesPlease
 endif
+ifeq ($(uname_S),"whatever jeffrey uses")
+	AR = libtool
+	ARFLAGS = -static -o
+endif
 ifeq ($(uname_S),UnixWare)
 	CC = cc
 	NEEDS_SOCKET = YesPlease