./configure fails to link test program due to missing dependencies

./configure fails to link test program due to missing dependencies

[Henrik Holst]

Hej and hello!

I am tinkering with LFS 12.1 and I ran into a problem to compile git 2.44.0 with https/curl support due to an error in the ./configure script for the libcurl detection.

The relevant section of config.log:

configure:5462: checking for curl_global_init in -lcurl
configure:5485: gcc -o conftest -g -O2   conftest.c -lcurl   >&5
/usr/bin/ld: /usr/lib/gcc/x86_64-pc-linux-gnu/13.2.0/../../../../lib/libcurl.a(libcurl_la-content_encoding.o): in function `zstd_do_close':
content_encoding.c:(.text+0x78): undefined reference to `ZSTD_freeDStream'
/usr/bin/ld: /usr/lib/gcc/x86_64-pc-linux-gnu/13.2.0/../../../../lib/libcurl.a(libcurl_la-content_encoding.o): in function `zstd_do_write':
content_encoding.c:(.text+0x112): undefined reference to `ZSTD_decompressStream'
/usr/bin/ld: content_encoding.c:(.text+0x11a): undefined reference to `ZSTD_isError'
...

If I set LDFLAGS to whatever pkg-config --libs libcurl says on my system (actually: -lcurl -lssl -lcrypto -lzstd -lbrotlidec -lz) then it compiles just fine. If I add LDFLAGS to the configure environment it will accept that test, and then detect, as expected, the pkg-config settings for libcurl.

Should not ./configure FIRST check for a pkg-config environment without assuming that even the most trivial curl programs should compile without any additional dependencies like zstd etc?

Thank you for your help and have a great weekend!

Henrik Holst

[System Info]
git version:
git version 2.44.0
cpu: x86_64
no commit associated with this build
sizeof-long: 8
sizeof-size_t: 8
shell-path: /bin/sh
compiler info: gnuc: 13.2
libc info: glibc: 2.39
$SHELL (typically, interactive shell): /usr/bin/fish


[Enabled Hooks]
not run from a git repository - no hooks to show

Re: ./configure fails to link test program due to missing dependencies

[Junio C Hamano]

Henrik Holst <henrik.holst@outlook.com> writes:

> If I set LDFLAGS to whatever pkg-config --libs libcurl says on my system (actually: -lcurl -lssl -lcrypto -lzstd -lbrotlidec -lz) then it compiles just fine. If I add LDFLAGS to the configure environment it will accept that test, and then detect, as expected, the pkg-config settings for libcurl.
>
> Should not ./configure FIRST check for a pkg-config environment without assuming that even the most trivial curl programs should compile without any additional dependencies like zstd etc?

Looking at configure.ac, pkg-config is not used for any package.
Specifically for curl, it seems that "curl-config --libs" is used.

Presumably the reason behind the current behaviour is combination of
(1) ./configure is an after-thought in the build infrastructure for
this project, (2) pkg-config was not ubiquitous back when autoconf
support was written for this project, and (3) nobody considered
"upgrading" our use of "curl-config" and our manual detection of
dependency detection for other libraries to just use "pkg-config".

Patches welcome ;-)

Thanks.

Re: ./configure fails to link test program due to missing dependencies

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1837 bytes --]

On 2024-09-15 at 16:37:34, Junio C Hamano wrote:
> Henrik Holst <henrik.holst@outlook.com> writes:
> 
> > If I set LDFLAGS to whatever pkg-config --libs libcurl says on my system (actually: -lcurl -lssl -lcrypto -lzstd -lbrotlidec -lz) then it compiles just fine. If I add LDFLAGS to the configure environment it will accept that test, and then detect, as expected, the pkg-config settings for libcurl.
> >
> > Should not ./configure FIRST check for a pkg-config environment without assuming that even the most trivial curl programs should compile without any additional dependencies like zstd etc?
> 
> Looking at configure.ac, pkg-config is not used for any package.
> Specifically for curl, it seems that "curl-config --libs" is used.
> 
> Presumably the reason behind the current behaviour is combination of
> (1) ./configure is an after-thought in the build infrastructure for
> this project, (2) pkg-config was not ubiquitous back when autoconf
> support was written for this project, and (3) nobody considered
> "upgrading" our use of "curl-config" and our manual detection of
> dependency detection for other libraries to just use "pkg-config".

I'll also note that the traditional expectation is that users are using
dynamic libraries, where this would have worked because libcurl would
have been linked against all its dependencies, and not static libraries,
where all dependencies must be specified explicitly.

You almost certainly don't want to statically link libcurl or OpenSSL
because you'll have a security issue as soon as one of those packages
issues a security update, which will require that you recompile Git to
apply the update.  With dynamic libraries, security updates are applied
as soon as a new process is spawned.
-- 
brian m. carlson (they/them or he/him)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: ./configure fails to link test program due to missing dependencies

[Patrick Steinhardt]

On Sun, Sep 15, 2024 at 09:37:34AM -0700, Junio C Hamano wrote:
> Henrik Holst <henrik.holst@outlook.com> writes:
> 
> > If I set LDFLAGS to whatever pkg-config --libs libcurl says on my system (actually: -lcurl -lssl -lcrypto -lzstd -lbrotlidec -lz) then it compiles just fine. If I add LDFLAGS to the configure environment it will accept that test, and then detect, as expected, the pkg-config settings for libcurl.
> >
> > Should not ./configure FIRST check for a pkg-config environment without assuming that even the most trivial curl programs should compile without any additional dependencies like zstd etc?
> 
> Looking at configure.ac, pkg-config is not used for any package.
> Specifically for curl, it seems that "curl-config --libs" is used.
> 
> Presumably the reason behind the current behaviour is combination of
> (1) ./configure is an after-thought in the build infrastructure for
> this project, (2) pkg-config was not ubiquitous back when autoconf
> support was written for this project, and (3) nobody considered
> "upgrading" our use of "curl-config" and our manual detection of
> dependency detection for other libraries to just use "pkg-config".

I sometimes wonder whether we should move on and discard one of the
three build systems we have: plain GNU Make, autoconf and CMake. And
from these three I'd rather want to throw the autoconf-based thing away:

  - The Makefile is probably what most people use, so throwing it out is
    a no-go right now.

  - CMake is really useful because it has support for IDEs and
    alternatives to GNU Make like Ninja, which builds Git way faster
    than Makefiles. It also has support for out-of-tree builds, which I
    find rather useful.

So is there a path forward to move CMake support out of contrib/, make
it an officially supported way to build Git and then throw away the
autoconf-based infra? I'm not the biggest fan of CMake myself and very
much prefer Meson, but we already have it wired up and thus I'm trying
to be at least a bit pragmatic.

(I'd honestly prefer to end up with a single build system, but also
throwing our Makefiles out would be a step too far at this point in
time.)

Patrick

Re: ./configure fails to link test program due to missing dependencies

[Phillip Wood]

Hi Patrick

On 16/09/2024 08:50, Patrick Steinhardt wrote:
> 
> I sometimes wonder whether we should move on and discard one of the
> three build systems we have: plain GNU Make, autoconf and CMake. And
> from these three I'd rather want to throw the autoconf-based thing away:
> 
>    - The Makefile is probably what most people use, so throwing it out is
>      a no-go right now.
> 
>    - CMake is really useful because it has support for IDEs and
>      alternatives to GNU Make like Ninja, which builds Git way faster
>      than Makefiles. It also has support for out-of-tree builds, which I
>      find rather useful.
> 
> So is there a path forward to move CMake support out of contrib/, make
> it an officially supported way to build Git and then throw away the
> autoconf-based infra? I'm not the biggest fan of CMake myself and very
> much prefer Meson, but we already have it wired up and thus I'm trying
> to be at least a bit pragmatic.

We seem to get fairly regular bug reports about the configure script, 
presumably because most contributors are using the Makefile. It would 
certainly be nice if we could get the CMake support into a state where 
we could consider dropping the configure script.

Best Wishes

Phillip

Re: ./configure fails to link test program due to missing dependencies

[Junio C Hamano]

Phillip Wood <phillip.wood123@gmail.com> writes:

> We seem to get fairly regular bug reports about the configure script,
> presumably because most contributors are using the Makefile. It would
> certainly be nice if we could get the CMake support into a state where
> we could consider dropping the configure script.

While I would agree that two is better than having to support three
build procedures, I am not sure how improvement of CMake support
needs to be a prerequisite for removal of autoconf.

Re: ./configure fails to link test program due to missing dependencies

[Patrick Steinhardt]

On Wed, Sep 18, 2024 at 03:39:13PM -0700, Junio C Hamano wrote:
> Phillip Wood <phillip.wood123@gmail.com> writes:
> 
> > We seem to get fairly regular bug reports about the configure script,
> > presumably because most contributors are using the Makefile. It would
> > certainly be nice if we could get the CMake support into a state where
> > we could consider dropping the configure script.
> 
> While I would agree that two is better than having to support three
> build procedures, I am not sure how improvement of CMake support
> needs to be a prerequisite for removal of autoconf.

I'm mostly coming from the angle that autoconf is likely used by systems
that are not perfectly supported by our current, static configuration. I
don't want to make the life of such system integrators harder by having
to figure out what kind of arcane functions they have to set manually
now to make things build on their platform again.

I'm not really sure whether distros _do_ actually use autoconf. Checking
a few distros:

  - Arch doesn't.
  - Cygwin uses autoconf.
  - Debian doesn't.
  - FreeBSD uses autoconf.
  - Gentoo doesn't.
  - NixOS uses autoconf.
  - OpenBSD uses autoconf.
  - Ubuntu doesn't.

So basically, we'd be making the life harder of anybody who doesn't
conform to the "standard" way of doing things in Linux, which I think is
not exactly a nice thing to do.

And that's why I think we should have an alternative way to configure
and build Git that can act as a replacement for autoconf, with my vote
going to either CMake or Meson. They are a proper replacement for
autoconf that makes the downstream maintainer's jobs easier while also
bringing additional features to the table that we don't currently have.

Eli makes a couple of good remarks in [1] about things that both CMake
and Meson bring to the table in addition to that, while also mentioning
some of the benefits of Meson over CMake.

I would be okay to make Git work with such a build system myself. The
current CMake build instructions can be used to _build_ Git, but AFAIU
they cannot yet run the Git test suite. Dscho pointed me to a couple of
patches from Ævar that work into this direction, and I'd be happy to
revive them. I'd also be okay with picking Meson over CMake if that is
what people want. But my ultimate goal would then be that we have at
least one CI job build and test against such a build system and give it
the "official blessing" as an alternative way to build Git.

[1]: <0864bd25-d5c4-45ac-a59e-e6f7d24002de@gentoo.org>

Patrick

Re: ./configure fails to link test program due to missing dependencies

[Eli Schwartz]

[-- Attachment #1.1: Type: text/plain, Size: 5305 bytes --]

On 9/24/24 8:10 AM, Patrick Steinhardt wrote:

Thanks for the CC to this interesting thread. :)


> On Wed, Sep 18, 2024 at 03:39:13PM -0700, Junio C Hamano wrote:
>> Phillip Wood <phillip.wood123@gmail.com> writes:
>>
>>> We seem to get fairly regular bug reports about the configure script,
>>> presumably because most contributors are using the Makefile. It would
>>> certainly be nice if we could get the CMake support into a state where
>>> we could consider dropping the configure script.
>>
>> While I would agree that two is better than having to support three
>> build procedures, I am not sure how improvement of CMake support
>> needs to be a prerequisite for removal of autoconf.
> 
> I'm mostly coming from the angle that autoconf is likely used by systems
> that are not perfectly supported by our current, static configuration. I
> don't want to make the life of such system integrators harder by having
> to figure out what kind of arcane functions they have to set manually
> now to make things build on their platform again.
> 
> I'm not really sure whether distros _do_ actually use autoconf. Checking
> a few distros:
> 
>   - Arch doesn't.
>   - Cygwin uses autoconf.
>   - Debian doesn't.
>   - FreeBSD uses autoconf.
>   - Gentoo doesn't.
>   - NixOS uses autoconf.
>   - OpenBSD uses autoconf.
>   - Ubuntu doesn't.
> 
> So basically, we'd be making the life harder of anybody who doesn't
> conform to the "standard" way of doing things in Linux, which I think is
> not exactly a nice thing to do.


I think we'd probably like to use the configure script rather than the
raw Makefile, if the configure script was supported.

It would fix things like git failing to cross-compile because
curl-config doesn't work for that, assuming that --

Oh yeah, the configure script isn't maintained well and doesn't use
pkg-config. :)

See e.g. https://bugs.gentoo.org/738218 which mentions pkg-config, even.


> And that's why I think we should have an alternative way to configure
> and build Git that can act as a replacement for autoconf, with my vote
> going to either CMake or Meson. They are a proper replacement for
> autoconf that makes the downstream maintainer's jobs easier while also
> bringing additional features to the table that we don't currently have.


Let's say, rather, that they are an alternative for autoconf. And one of
the good qualities of them as an alternative for autoconf is that you
can actually build on Windows, without needing a mingw toolchain to run
a shell script.

An actually maintained autoconf script makes the downstream maintainer's
job easier in all cases...

... and also makes the upstream maintainer's job easier in some ways and
harder in other ways, because autoconf is hard/annoying to get right as
a maintainer. This is due to the complexities of m4 and mixing that
inline into "m4sh" -- which was a logical tradeoff in the past, when
Windows wasn't as relevant and the GNU project wanted to design a build
system that maximized the benefits for end users, including "do not need
to install any software to run ./configure", even if that sometimes
meant making maintainers' jobs harder. I've never seen a project with a
*well-maintained, correctly written* autotools build system where the
*unix* end users had complaints about the use of autotools. The
complaint is inevitably that autotools wasn't correctly used

Still I would prefer meson over autotools any day of the week. I'd also
prefer autotools over cmake, mind you.


> Eli makes a couple of good remarks in [1] about things that both CMake
> and Meson bring to the table in addition to that, while also mentioning
> some of the benefits of Meson over CMake.
> 
> I would be okay to make Git work with such a build system myself. The
> current CMake build instructions can be used to _build_ Git, but AFAIU
> they cannot yet run the Git test suite. Dscho pointed me to a couple of
> patches from Ævar that work into this direction, and I'd be happy to
> revive them. I'd also be okay with picking Meson over CMake if that is
> what people want. But my ultimate goal would then be that we have at
> least one CI job build and test against such a build system and give it
> the "official blessing" as an alternative way to build Git.


Like, erm, many people :D I spend vast portions of my day inside git. I
am not very good at C, though -- and the likelihood of git being
completely rewritten in python is quite low -- so I generally do not try
very hard to repay that by getting involved in git development (I have
some humble patches consisting of a single patch series, which I do feel
pretty proud of since it enabled a very useful workflow, but still:
ultimately amounts to a one-off event).

I do know build systems pretty well though! :) And I'd be happy to
collaborate on Meson and help maintain the build system support in the
long term, assuming the consensus is that people think it would be a
neat idea to add meson support (regardless of whether it serves as a
primary or secondary build system).


Although I'm uninterested in personally working on cmake, as you
probably predicted.


-- 
Eli Schwartz
Gentoo Developer and Meson Build maintainer

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: ./configure fails to link test program due to missing dependencies

[Paul Smith]

On Tue, 2024-09-24 at 09:59 -0400, Eli Schwartz wrote:
> Still I would prefer meson over autotools any day of the week. I'd
> also prefer autotools over cmake, mind you.

Just to point out that relying on ever-more-esoteric build tools is a
recipe for frustration for those of us who want to build from source. 
Yes, I know meson is somewhat popular and definitely has a following,
but in the grand scheme of things it's still rare to find projects
requiring it, especially at the base level of software.  I build a lot
of software and I have never once had to install it, for example.

I maintain a minimalist set of tools and libraries that we compile
locally from source so we can control the full toolchain.  Every time a
package we want to add requires some new, different build tool it's a
massive annoyance.  As of today everything builds with either make or
cmake (mainly for Windows support).

On the other hand, now that all the systems we use have "good enough"
native versions of Git we have stopped building it in our environment.
So in that sense it no longer matters to me directly :).

Re: ./configure fails to link test program due to missing dependencies

[Patrick Steinhardt]

On Tue, Sep 24, 2024 at 09:59:52AM -0400, Eli Schwartz wrote:
> On 9/24/24 8:10 AM, Patrick Steinhardt wrote:
> Still I would prefer meson over autotools any day of the week. I'd also
> prefer autotools over cmake, mind you.

Is that a typo or do you really prefer autotools over CMake? :)

> > Eli makes a couple of good remarks in [1] about things that both CMake
> > and Meson bring to the table in addition to that, while also mentioning
> > some of the benefits of Meson over CMake.
> > 
> > I would be okay to make Git work with such a build system myself. The
> > current CMake build instructions can be used to _build_ Git, but AFAIU
> > they cannot yet run the Git test suite. Dscho pointed me to a couple of
> > patches from Ævar that work into this direction, and I'd be happy to
> > revive them. I'd also be okay with picking Meson over CMake if that is
> > what people want. But my ultimate goal would then be that we have at
> > least one CI job build and test against such a build system and give it
> > the "official blessing" as an alternative way to build Git.
> 
> Like, erm, many people :D I spend vast portions of my day inside git. I
> am not very good at C, though -- and the likelihood of git being
> completely rewritten in python is quite low -- so I generally do not try
> very hard to repay that by getting involved in git development (I have
> some humble patches consisting of a single patch series, which I do feel
> pretty proud of since it enabled a very useful workflow, but still:
> ultimately amounts to a one-off event).
> 
> I do know build systems pretty well though! :) And I'd be happy to
> collaborate on Meson and help maintain the build system support in the
> long term, assuming the consensus is that people think it would be a
> neat idea to add meson support (regardless of whether it serves as a
> primary or secondary build system).

People with different skillsets can repay in different ways, and that's
not a bad thing. Not that anybody really has to repay anything. But in
any case, I would certainly appreciate a second pair of eyes from
somebody with expert knowledge on Meson once I have something to show.

I had a deeper look at the CMake build infra that we have and figured
that it does make a whole lot of assumptions that autoconf wouldn't
make. Those assumptions can of course be removed, and I'd be okay with
doing that eventually.

But for now I'm hacking up an initial iteration of Meson that is good
enough to build, install and hopefully test Git. The intent here isn't
quite to preclude us from using CMake as "official" build system, but
rather to make it possible to compare the different build systems with
actual code, where our choices will be plain Make, autoconf, Meson and
CMake.

That should hopefully lead to a more informed discussion.

> Although I'm uninterested in personally working on cmake, as you
> probably predicted.

Fair.

Patrick

Re: ./configure fails to link test program due to missing dependencies

[Eli Schwartz]

[-- Attachment #1.1: Type: text/plain, Size: 3674 bytes --]

On 9/25/24 12:36 AM, Patrick Steinhardt wrote:
> On Tue, Sep 24, 2024 at 09:59:52AM -0400, Eli Schwartz wrote:
>> On 9/24/24 8:10 AM, Patrick Steinhardt wrote:
>> Still I would prefer meson over autotools any day of the week. I'd also
>> prefer autotools over cmake, mind you.
> 
> Is that a typo or do you really prefer autotools over CMake? :)


POSIX sh (used by autotools) has a more powerful and capable type system
than CMakeLists.txt (this is not a typo either! compare CMake's
"semicolon;delimited;string" type to POSIX sh's "$@" type)


m4 is less painful to debug than successfully configuring, spending 4
hours compiling a ginormous project, then failing at the end with (this
is because of the type system again, since there is no type for
dependency-was-not-found they smuggle it along as a string value with
special meaning):
ld: cannot find -lCURL-NOTFOUND: No such file or directory


If you enable cmake's test system, but you do it inside a subdir e.g.
inside tests/CMakeLists.txt, you cannot run "ctest" (or make test)
except inside that subdir. ctest will exit 0, and no rules will be
generated to descend into the correct directory instead. This has bitten
me a *bunch* of times in Gentoo packaging and it always throws me for a
loop. I don't understand the point of such a design.


I have never had autotools refuse point-blank to detect that another
package is installed and usable for ***shared linkage*** because my
distro automatically removes static libraries when a corresponding
shared library exists, and

CMake Error at /usr/lib64/cmake/libssh2/Libssh2Config.cmake:92 (message):
  The imported target "Libssh2::libssh2_static" references the file

     "/usr/lib64/libssh2.a"

  but this file does not exist.  Possible reasons include:

  * The file was deleted, renamed, or moved to another location.

  * An install or uninstall procedure did not complete successfully.

  * The installation package was faulty and contained

     "/usr/lib64/cmake/libssh2/Libssh2Config-relwithdebinfo.cmake"

  but not all the files it references.


autotools projects have never harmed me by running the square of my make
-j$(nproc) count due to recursively running cmake inside of generated
Makefiles -- perhaps that isn't strictly the fault of CMake but cmake
does very little to discourage it: https://bugs.gentoo.org/921309


autotools doesn't have much in the way of built-in tooling for detecting
"packages" instead of "libraries" for arbitrary system dependencies. It
allows you to use pkg-config or code your own (foo-config scripts were
popular once and in some circles still are). You might think this is a
negative, but it's actually a positive compared to cmake, which includes
builtin dependency finders for e.g. zlib that break on a simple version
update because they locate the header file, open it up and use regular
expressions to extract a #define for the package version instead of
asking the preprocessor... a very brittle regex, too:
https://gitlab.kitware.com/cmake/cmake/-/issues/25200


autotools projects don't automatically detect your end-to-end
integration test's dummy project, integrate its results into your build,
and install some of your project files twice, once with bad values, but
only when you run the project tests (this one was very fun):
https://github.com/nlohmann/json/issues/2907#issuecomment-890580081


...


I'm probably biased, but some of these failure modes are *weird*. And
they basically never require the CMakeLists.txt to do something
considered non-idiomatic in order to trigger the issue.


-- 
Eli Schwartz


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: ./configure fails to link test program due to missing dependencies

[Patrick Steinhardt]

On Wed, Sep 25, 2024 at 02:02:34AM -0400, Eli Schwartz wrote:
> On 9/25/24 12:36 AM, Patrick Steinhardt wrote:
> > On Tue, Sep 24, 2024 at 09:59:52AM -0400, Eli Schwartz wrote:
> >> On 9/24/24 8:10 AM, Patrick Steinhardt wrote:
> >> Still I would prefer meson over autotools any day of the week. I'd also
> >> prefer autotools over cmake, mind you.
> > 
> > Is that a typo or do you really prefer autotools over CMake? :)
> 
> 
> POSIX sh (used by autotools) has a more powerful and capable type system
> than CMakeLists.txt (this is not a typo either! compare CMake's
> "semicolon;delimited;string" type to POSIX sh's "$@" type)
> 
> 
> m4 is less painful to debug than successfully configuring, spending 4
> hours compiling a ginormous project, then failing at the end with (this
> is because of the type system again, since there is no type for
> dependency-was-not-found they smuggle it along as a string value with
> special meaning):
> ld: cannot find -lCURL-NOTFOUND: No such file or directory
> 
> 
> If you enable cmake's test system, but you do it inside a subdir e.g.
> inside tests/CMakeLists.txt, you cannot run "ctest" (or make test)
> except inside that subdir. ctest will exit 0, and no rules will be
> generated to descend into the correct directory instead. This has bitten
> me a *bunch* of times in Gentoo packaging and it always throws me for a
> loop. I don't understand the point of such a design.
> 
> 
> I have never had autotools refuse point-blank to detect that another
> package is installed and usable for ***shared linkage*** because my
> distro automatically removes static libraries when a corresponding
> shared library exists, and
> 
> CMake Error at /usr/lib64/cmake/libssh2/Libssh2Config.cmake:92 (message):
>   The imported target "Libssh2::libssh2_static" references the file
> 
>      "/usr/lib64/libssh2.a"
> 
>   but this file does not exist.  Possible reasons include:
> 
>   * The file was deleted, renamed, or moved to another location.
> 
>   * An install or uninstall procedure did not complete successfully.
> 
>   * The installation package was faulty and contained
> 
>      "/usr/lib64/cmake/libssh2/Libssh2Config-relwithdebinfo.cmake"
> 
>   but not all the files it references.
> 
> 
> autotools projects have never harmed me by running the square of my make
> -j$(nproc) count due to recursively running cmake inside of generated
> Makefiles -- perhaps that isn't strictly the fault of CMake but cmake
> does very little to discourage it: https://bugs.gentoo.org/921309
> 
> 
> autotools doesn't have much in the way of built-in tooling for detecting
> "packages" instead of "libraries" for arbitrary system dependencies. It
> allows you to use pkg-config or code your own (foo-config scripts were
> popular once and in some circles still are). You might think this is a
> negative, but it's actually a positive compared to cmake, which includes
> builtin dependency finders for e.g. zlib that break on a simple version
> update because they locate the header file, open it up and use regular
> expressions to extract a #define for the package version instead of
> asking the preprocessor... a very brittle regex, too:
> https://gitlab.kitware.com/cmake/cmake/-/issues/25200
> 
> 
> autotools projects don't automatically detect your end-to-end
> integration test's dummy project, integrate its results into your build,
> and install some of your project files twice, once with bad values, but
> only when you run the project tests (this one was very fun):
> https://github.com/nlohmann/json/issues/2907#issuecomment-890580081
> 
> 
> ...
> 
> 
> I'm probably biased, but some of these failure modes are *weird*. And
> they basically never require the CMakeLists.txt to do something
> considered non-idiomatic in order to trigger the issue.

All of this is very valuable data to make my case for Meson instead of
CMake. Appreciated, thanks!

Patrick

Re: ./configure fails to link test program due to missing dependencies

[Phillip Wood]

Hi Patrick

On 25/09/2024 07:04, Patrick Steinhardt wrote:
> On Wed, Sep 25, 2024 at 02:02:34AM -0400, Eli Schwartz wrote:
>
>> I'm probably biased, but some of these failure modes are *weird*. And
>> they basically never require the CMakeLists.txt to do something
>> considered non-idiomatic in order to trigger the issue.
> 
> All of this is very valuable data to make my case for Meson instead of
> CMake. Appreciated, thanks!

One thing to bear in mind is why our CMakeLists.txt was introduced in 
the first place [1]. Visual Studio's CMake integration means that so 
long as git-for-windows is installed building git is simply a case of 
clicking on a button, there is no need to install extra software or 
plugins. I'm not sure the same is true for meson and I don't think we 
want to end up supporting both.

Best Wishes

Phillip

[1] 
https://lore.kernel.org/git/nycvar.QRO.7.76.6.2004251354390.18039@tvgsbejvaqbjf.bet/

Re: ./configure fails to link test program due to missing dependencies

[Patrick Steinhardt]

On Thu, Sep 26, 2024 at 02:55:52PM +0100, Phillip Wood wrote:
> Hi Patrick
> 
> On 25/09/2024 07:04, Patrick Steinhardt wrote:
> > On Wed, Sep 25, 2024 at 02:02:34AM -0400, Eli Schwartz wrote:
> > 
> > > I'm probably biased, but some of these failure modes are *weird*. And
> > > they basically never require the CMakeLists.txt to do something
> > > considered non-idiomatic in order to trigger the issue.
> > 
> > All of this is very valuable data to make my case for Meson instead of
> > CMake. Appreciated, thanks!
> 
> One thing to bear in mind is why our CMakeLists.txt was introduced in the
> first place [1]. Visual Studio's CMake integration means that so long as
> git-for-windows is installed building git is simply a case of clicking on a
> button, there is no need to install extra software or plugins. I'm not sure
> the same is true for meson and I don't think we want to end up supporting
> both.
> 
> Best Wishes
> 
> Phillip
> 
> [1] https://lore.kernel.org/git/nycvar.QRO.7.76.6.2004251354390.18039@tvgsbejvaqbjf.bet/

Fair enough. The final discussion about which build system to pick is of
course still to be had. Having worked with both CMake and Meson quite a
bit in the past I'm strongly in favor of Meson myself, and so I will try
to make a strong case for it. But points like this of course need to be
considered when we have the discussion.

The nice thing is that we'll then have all serious contenders (that I am
aware of) wired up, even though the level of support will be different
across them. But it should allow folks to come to a better understanding
of what they will be signing up for.

In any case, I'm now in a state where the Meson-based build works and
tests just fine, except for t0200, which requires a bit more plumbing to
set up xgettext infra. Once I'm done with that I'll go off and test with
both macOS and Windows to check how the experience is over there. I hope
to be done with that somewhen next week, at which point I'll send things
to the mailing list.

Patrick

Re: ./configure fails to link test program due to missing dependencies

[Phillip Wood]

Hi Patrick

On 26/09/2024 15:02, Patrick Steinhardt wrote:
> On Thu, Sep 26, 2024 at 02:55:52PM +0100, Phillip Wood wrote:
>> Hi Patrick
>>
>> On 25/09/2024 07:04, Patrick Steinhardt wrote:
>>> On Wed, Sep 25, 2024 at 02:02:34AM -0400, Eli Schwartz wrote:
>>>
>>>> I'm probably biased, but some of these failure modes are *weird*. And
>>>> they basically never require the CMakeLists.txt to do something
>>>> considered non-idiomatic in order to trigger the issue.
>>>
>>> All of this is very valuable data to make my case for Meson instead of
>>> CMake. Appreciated, thanks!
>>
>> One thing to bear in mind is why our CMakeLists.txt was introduced in the
>> first place [1]. Visual Studio's CMake integration means that so long as
>> git-for-windows is installed building git is simply a case of clicking on a
>> button, there is no need to install extra software or plugins. I'm not sure
>> the same is true for meson and I don't think we want to end up supporting
>> both.
>>
>> Best Wishes
>>
>> Phillip
>>
>> [1] https://lore.kernel.org/git/nycvar.QRO.7.76.6.2004251354390.18039@tvgsbejvaqbjf.bet/
> 
> Fair enough. The final discussion about which build system to pick is of
> course still to be had. Having worked with both CMake and Meson quite a
> bit in the past I'm strongly in favor of Meson myself, and so I will try
> to make a strong case for it. But points like this of course need to be
> considered when we have the discussion.

 From what Eli said, we should have a reasonable story for building with 
meson in Visual Studio

> The nice thing is that we'll then have all serious contenders (that I am
> aware of) wired up, even though the level of support will be different
> across them. But it should allow folks to come to a better understanding
> of what they will be signing up for.

Yes, that will be helpful

> In any case, I'm now in a state where the Meson-based build works and
> tests just fine, except for t0200, which requires a bit more plumbing to
> set up xgettext infra. Once I'm done with that I'll go off and test with
> both macOS and Windows to check how the experience is over there. I hope
> to be done with that somewhen next week, at which point I'll send things
> to the mailing list.

It sounds like you're making good progress, I look forward to seeing the 
patches

Best Wishes

Phillip

Re: ./configure fails to link test program due to missing dependencies

[Eli Schwartz]

[-- Attachment #1.1: Type: text/plain, Size: 2287 bytes --]

On 9/26/24 9:55 AM, Phillip Wood wrote:
> One thing to bear in mind is why our CMakeLists.txt was introduced in
> the first place [1]. Visual Studio's CMake integration means that so
> long as git-for-windows is installed building git is simply a case of
> clicking on a button, there is no need to install extra software or
> plugins. I'm not sure the same is true for meson and I don't think we
> want to end up supporting both.


I can't really offer suggestions on what may or may not come
preinstalled in Visual Studio. That thread does suggest the major
problem cmake was trying to solve is:

- having to install the git-for-windows sdk at all (is it still
  necessary? I guess so, because POSIX shell and perl and mingw
  runtimes. Unsure how either meson or cmake could solve this.)

- people who are *unfamiliar with the command line and want a GUI*


Meson has a trivially installable VS Code plugin that is supposed to
handle setting up the project for you. You can generate either ninja
projects or Visual Studio solutions. "One may need to install a plugin"
is hopefully not as big a barrier to entry as "install a bunch of stuff
then go to a shell and run make vcxproj". Is the criteria truly "must be
one button click"?

I'm aware that Visual Studio and VS Code are different IDEs but I know
nothing really about the former, SEO for distinguishing between the two
is *atrocious*, they use the same exact plugins marketplace, and I
figure using VS Code ought to be easy enough for such users if
necessary, anyway.

I think that discussion between Meson / CMake / GNU Make / etc. should
be based on technical merit with attention paid to ease of use without
coalescing down to "unfortunately, our choice has been made for us. We
must support a *specific* Windows IDE and use whichever build system is
preinstalled inside that IDE, because it must be a one-button solution
with no dependencies".

Stuff like "how painful is it for a Windows contributor to set up an SDK
and then also go mess around with Makefile targets and then load the
result into their IDE" is an interesting discussion to have but not
quite the same as saying "go to the marketplace and install such and
such plugin" is an obstacle.


-- 
Eli Schwartz


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: ./configure fails to link test program due to missing dependencies

[phillip.wood123]

Hi Eli

Thanks for this, it's useful to know how meson works with Visual Studio

On 26/09/2024 17:04, Eli Schwartz wrote:
> On 9/26/24 9:55 AM, Phillip Wood wrote:
>> One thing to bear in mind is why our CMakeLists.txt was introduced in
>> the first place [1]. Visual Studio's CMake integration means that so
>> long as git-for-windows is installed building git is simply a case of
>> clicking on a button, there is no need to install extra software or
>> plugins. I'm not sure the same is true for meson and I don't think we
>> want to end up supporting both.
> 
> 
> I can't really offer suggestions on what may or may not come
> preinstalled in Visual Studio. That thread does suggest the major
> problem cmake was trying to solve is:
> 
> - having to install the git-for-windows sdk at all (is it still
>    necessary? I guess so, because POSIX shell and perl and mingw
>    runtimes. Unsure how either meson or cmake could solve this.)

If you've got git-for-windows installed then it has the POSIX and perl 
bits that are required to run git and the CMake build uses those and 
downloads any libraries it needs with vcpkg so you don't need the sdk.

> - people who are *unfamiliar with the command line and want a GUI*
> 
> 
> Meson has a trivially installable VS Code plugin that is supposed to
> handle setting up the project for you. You can generate either ninja
> projects or Visual Studio solutions. "One may need to install a plugin"
> is hopefully not as big a barrier to entry as "install a bunch of stuff
> then go to a shell and run make vcxproj". Is the criteria truly "must be
> one button click"?

Personally I think so long as there is a simple way to build git without 
resorting to the command line that should be fine. It sounds like that's 
the case with meson.

> [...]
> Stuff like "how painful is it for a Windows contributor to set up an SDK
> and then also go mess around with Makefile targets and then load the
> result into their IDE" is an interesting discussion to have but not
> quite the same as saying "go to the marketplace and install such and
> such plugin" is an obstacle.

Agreed

Best Wishes

Phillip

Re: ./configure fails to link test program due to missing dependencies

[Junio C Hamano]

Phillip Wood <phillip.wood123@gmail.com> writes:

> Hi Patrick
>
> On 25/09/2024 07:04, Patrick Steinhardt wrote:
>> On Wed, Sep 25, 2024 at 02:02:34AM -0400, Eli Schwartz wrote:
>>
>>> I'm probably biased, but some of these failure modes are *weird*. And
>>> they basically never require the CMakeLists.txt to do something
>>> considered non-idiomatic in order to trigger the issue.
>> All of this is very valuable data to make my case for Meson instead
>> of
>> CMake. Appreciated, thanks!
>
> One thing to bear in mind is why our CMakeLists.txt was introduced in
> the first place [1]. Visual Studio's CMake integration means that so
> long as git-for-windows is installed building git is simply a case of
> clicking on a button, there is no need to install extra software or
> plugins. I'm not sure the same is true for meson and I don't think we
> want to end up supporting both.

Is CMake the _only_ thing that is integrated into Visual Studio?
Are there other possible candidates that could also be used to build
for non-Windows and is usable by this project?



> Best Wishes
>
> Phillip
>
> [1]
> https://lore.kernel.org/git/nycvar.QRO.7.76.6.2004251354390.18039@tvgsbejvaqbjf.bet/

Re: ./configure fails to link test program due to missing dependencies

[Johannes Schindelin]

Hi Junio,

On Thu, 26 Sep 2024, Junio C Hamano wrote:

> Phillip Wood <phillip.wood123@gmail.com> writes:
>
> > On 25/09/2024 07:04, Patrick Steinhardt wrote:
> >> On Wed, Sep 25, 2024 at 02:02:34AM -0400, Eli Schwartz wrote:
> >>
> >>> I'm probably biased, but some of these failure modes are *weird*.
> >>> And they basically never require the CMakeLists.txt to do something
> >>> considered non-idiomatic in order to trigger the issue.
> >>
> >> All of this is very valuable data to make my case for Meson instead
> >> of CMake. Appreciated, thanks!
> >
> > One thing to bear in mind is why our CMakeLists.txt was introduced in
> > the first place [1]. Visual Studio's CMake integration means that so
> > long as git-for-windows is installed building git is simply a case of
> > clicking on a button, there is no need to install extra software or
> > plugins. I'm not sure the same is true for meson and I don't think we
> > want to end up supporting both.
>
> Is CMake the _only_ thing that is integrated into Visual Studio? Are
> there other possible candidates that could also be used to build for
> non-Windows and is usable by this project?

There is one other build system that is highly integrated into Visual
Studio, and that is MSBuild, using `.vcxproj` files. I do not need three
guesses to find out what you think about porting Git over to that system.

Other than that, there really is only CMake support:
https://learn.microsoft.com/en-us/cpp/build/cmake-projects-in-visual-studio

Meson came up as an alternative, so the obvious question is whether it
could be used conveniently from within Visual Studio. It takes but one
look at https://mesonbuild.com/Using-with-Visual-Studio.html to see that
no, the instructions ask the developed to use a command-line interface,
which is the opposite of integrating well with an IDE.

In short: If we're serious that we want to stop treating Windows-based
developers as if they were unwanted here, we'll need to stick to CMake.

Ciao,
Johannes

Re: ./configure fails to link test program due to missing dependencies

[Eli Schwartz]

[-- Attachment #1.1: Type: text/plain, Size: 701 bytes --]

On 9/29/24 1:56 PM, Johannes Schindelin wrote:
> Meson came up as an alternative, so the obvious question is whether it
> could be used conveniently from within Visual Studio. It takes but one
> look at https://mesonbuild.com/Using-with-Visual-Studio.html to see that
> no, the instructions ask the developed to use a command-line interface,
> which is the opposite of integrating well with an IDE.
> 
> In short: If we're serious that we want to stop treating Windows-based
> developers as if they were unwanted here, we'll need to stick to CMake.

Hi,

I guess you didn't read the previous comments in this thread? Maybe you
should take more than one look. :)


-- 
Eli Schwartz


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: ./configure fails to link test program due to missing dependencies

[Phillip Wood]

On 29/09/2024 19:10, Eli Schwartz wrote:
> On 9/29/24 1:56 PM, Johannes Schindelin wrote:
>> Meson came up as an alternative, so the obvious question is whether it
>> could be used conveniently from within Visual Studio. It takes but one
>> look at https://mesonbuild.com/Using-with-Visual-Studio.html to see that
>> no, the instructions ask the developed to use a command-line interface,
>> which is the opposite of integrating well with an IDE.
>>
>> In short: If we're serious that we want to stop treating Windows-based
>> developers as if they were unwanted here, we'll need to stick to CMake.
> 
> Hi,
> 
> I guess you didn't read the previous comments in this thread? Maybe you
> should take more than one look. :)

We cannot expect everyone who wants to build git using meson in Visual 
Studio to read this thread and find the message that mentions installing 
a plugin. It is much more likely that they, like Johannes, will find the 
documentation on the meson website and conclude they need to run some 
commands on the commandline. That's a problem with the documentation, 
not the person reading it. Even if they do find the plugin [1] it is not 
clear that it helps - no where does it say "this enables you to build 
software with meson", instead it talks about syntax highlighting, code 
snippets and linting for meson files.

Best Wishes

Phillip

[1] 
https://marketplace.visualstudio.com/items?itemName=mesonbuild.mesonbuild

Re: ./configure fails to link test program due to missing dependencies

[Eli Schwartz]

[-- Attachment #1.1: Type: text/plain, Size: 1499 bytes --]

On 9/30/24 4:50 AM, Phillip Wood wrote:
> We cannot expect everyone who wants to build git using meson in Visual
> Studio to read this thread and find the message that mentions installing
> a plugin. It is much more likely that they, like Johannes, will find the
> documentation on the meson website and conclude they need to run some
> commands on the commandline. That's a problem with the documentation,
> not the person reading it. Even if they do find the plugin [1] it is not
> clear that it helps - no where does it say "this enables you to build
> software with meson", instead it talks about syntax highlighting, code
> snippets and linting for meson files.


Sure. And I'm happy to (help) improve the documentation. I've already
pushed a fix to the page Johannes linked, since it was many years out of
date (???) and didn't reflect the fact that meson will automatically
detect the Visual Studio toolset for you.

(Note that since meson will "automatically do the right thing" here,
that means it is practical for a plugin to run meson under the hood for
you... much like how cmake plugins handle things.)


As far as reading this thread goes, though, I assume that in such a
future, people who want to build git using meson in Visual Studio would
be optimally served by a slight reorganization to ./INSTALL to provide
guidance on where to find meson and what plugin to use, which provides
an additional entrypoint for clarification. :)


-- 
Eli Schwartz


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: ./configure fails to link test program due to missing dependencies

[Junio C Hamano]

Eli Schwartz <eschwartz@gentoo.org> writes:

> Sure. And I'm happy to (help) improve the documentation. I've already
> pushed a fix to the page Johannes linked, since it was many years out of
> date (???) and didn't reflect the fact that meson will automatically
> detect the Visual Studio toolset for you.
>
> (Note that since meson will "automatically do the right thing" here,
> that means it is practical for a plugin to run meson under the hood for
> you... much like how cmake plugins handle things.)
>
>
> As far as reading this thread goes, though, I assume that in such a
> future, people who want to build git using meson in Visual Studio would
> be optimally served by a slight reorganization to ./INSTALL to provide
> guidance on where to find meson and what plugin to use, which provides
> an additional entrypoint for clarification. :)

Yup, whenever a procedure changes, documentation to guide folks
along the procedure need to change.  Otherwise they will be lost.

And it is very good that we see people are improving the candidates
being offered that way, not just enumerating how it is superior to
the status quo (in their opinion) once adopted, but making sure it
would not inconvenience or alienate existing users who are used to
the way proposed for deprecation.

Re: ./configure fails to link test program due to missing dependencies

[Johannes Schindelin]

Hi Phillip,

On Mon, 30 Sep 2024, Phillip Wood wrote:

> On 29/09/2024 19:10, Eli Schwartz wrote:
> > On 9/29/24 1:56 PM, Johannes Schindelin wrote:
> > > Meson came up as an alternative, so the obvious question is whether it
> > > could be used conveniently from within Visual Studio. It takes but one
> > > look at https://mesonbuild.com/Using-with-Visual-Studio.html to see that
> > > no, the instructions ask the developed to use a command-line interface,
> > > which is the opposite of integrating well with an IDE.
> > >
> > > In short: If we're serious that we want to stop treating Windows-based
> > > developers as if they were unwanted here, we'll need to stick to CMake.
> >
> > I guess you didn't read the previous comments in this thread? Maybe you
> > should take more than one look. :)
>
> We cannot expect everyone who wants to build git using meson in Visual Studio
> to read this thread and find the message that mentions installing a plugin. It
> is much more likely that they, like Johannes, will find the documentation on
> the meson website and conclude they need to run some commands on the
> commandline. That's a problem with the documentation, not the person reading
> it. Even if they do find the plugin [1] it is not clear that it helps - no
> where does it say "this enables you to build software with meson", instead it
> talks about syntax highlighting, code snippets and linting for meson files.

I had actually read it. Not wanting to be xkcd 386, I had decided to
refrain from replying. But it would appear that I have to.

> [1] https://marketplace.visualstudio.com/items?itemName=mesonbuild.mesonbuild

Thank you for providing an actual link, it was missing in this thread
(handwaving is not really good enough, I would say).

On that page, we see a serious flaw in the argument made in
https://lore.kernel.org/git/71ed5967-0302-42bc-97c7-81886408d688@gentoo.org/:

	Visual Studio Code>Programming Languages>Meson

	[...]

	Meson for Visual Studio Code

Now, I am the first to admit that it is confusing that there is Visual
Studio Code and then there is Visual Studio, and those two products share
mostly the "Visual Studio" in their name, but little else.

Most notably, you cannot install VS Code Extensions into Visual Studio,
and vice versa Visual Studio extensions cannot be installed into VS Code.
See https://www.freecodecamp.org/news/visual-studio-vs-visual-studio-code/
or https://dev.to/hadil/the-difference-between-visual-studio-vs-visual-studio-code-35oh
for more detailed explanations.

And when you navigate on that Marketplace to the Visual Studio extensions
(sans "Code"), you will find that, frustratingly,
https://marketplace.visualstudio.com/search?term=meson&target=VS comes up
empty:

	Your search for 'meson' didn't match any extensions

In other words: Visual Studio, _the_ most prevalent IDE for Windows-based
C/C++ developers, has no Meson support worth talking about. None.

Now, how about instead suggesting VS Code to direct Windows-based
developers who are eager to contribute to Git? Alas, VS Code does not come
with a C/C++ compiler. Not even a canonical extension to compile C. Even
if there were, the user experience of having to scour around for plugins
and 3rd-party software to install, and _then_ somehow getting the needed
libraries like libcurl into that setup, _just_ to build Git (and we are
not even yet talking about running tests from Git's test suite that do not
eagerly lend themselves to be run from common test frameworks that would
be supported by VS Code _extensions_) is about 100x worse than telling
contributors to simply open their git.git checkout in Visual Studio and
let CMake do its thing.

Yes, I made up that "100x" from thin air, but it's easy to imagine that
some (most?) developers would contend that this is a few orders of
magnitude too generous.

Personally, I have to admit that I find very, very little consolation in
the suggestion that these days, Meson will "automatically do the right
thing", without requiring to be run in a VS Developer Prompt:
_These commands still have to be run in a Command Prompt_, i.e. completely
outside of Visual Studio, in a text-based terminal that Windows-based
developers will find off-putting to the point of rolling their eyes at the
suggestion.

In summary: From my perspective, these quite serious flaws put quite a
huge dent into the credibility of these arguments pro Meson (and contra
CMake). Forcing Windows-based developers away from CMake and toward Meson
would definitely make the developer experience substantially worse. To be
honest, I hoped that we would make the experience better. Certainly not
worse.

Ciao,
Johannes

Re: ./configure fails to link test program due to missing dependencies

[Patrick Steinhardt]

Hey Eli,

dropping the mailing list for a bit: I really want Meson to start become
a thing in Git. I very much feel that the current build infrastructure
is antiquated and has lots of issues. And while we do have CMake wired
up somewhat already, it neither is a replacement due to it lacking heaps
of features/autodetection, nor is it a direction I really want to go.

It also seems to be the right point in time: Junio hasn't really been a
fan of converting our build system in the past, but his response to my
ramblings was surprisingly positive. The session I hosted during the Git
contributor's summit also seemed positive overall, but I naturally still
anticipate some bikeshedding.

So I highly appreciate all the info that you've been posting in this
context, as it helps to solidify my stance quite a bit! Which brings me
to my ask: would you be willing to do an off-list review before I post
things to the Git mailing list? The intent here is to mostly make things
look as nice as possible and work out-of-the-box to hopefully sway the
list more into favor of Meson.

My current state is that I've got libgit.a set up while detecting many
of the important platform-specific bits, the Git executable links just
fine and I've got unit tests wired up and executing correctly. Still
missing is documentation, Perl modules, and fixing some last remaining
bugs around locales in t0200 and t7816.

Anyway, the current version of this can be found at [1]. Feel free to
have a look and provide any feedback, either on the merge request or as
a patch on top. It can be built with `meson setup -Dpython=disabled
-Dperl=disabled`, enabling these options breaks a bunch more stuff.

Thanks!

Patrick

[1]: https://gitlab.com/gitlab-org/git/-/merge_requests/217

Re: ./configure fails to link test program due to missing dependencies

[Patrick Steinhardt]

On Wed, Sep 25, 2024 at 09:15:36PM +0200, Patrick Steinhardt wrote:
> Hey Eli,
> 
> dropping the mailing list for a bit: I really want Meson to start become
> a thing in Git. I very much feel that the current build infrastructure
> is antiquated and has lots of issues. And while we do have CMake wired
> up somewhat already, it neither is a replacement due to it lacking heaps
> of features/autodetection, nor is it a direction I really want to go.

Well, so much for dropping the mailing list :P Anyway, it's not like
there's anything in here that I haven't already said in the context of
the Git contributor's summit.

Patrick

Re: ./configure fails to link test program due to missing dependencies

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

> I'm not really sure whether distros _do_ actually use autoconf. Checking
> a few distros:
>
>   - Arch doesn't.
>   - Cygwin uses autoconf.
>   - Debian doesn't.
>   - FreeBSD uses autoconf.
>   - Gentoo doesn't.
>   - NixOS uses autoconf.
>   - OpenBSD uses autoconf.
>   - Ubuntu doesn't.
>
> So basically, we'd be making the life harder of anybody who doesn't
> conform to the "standard" way of doing things in Linux, which I think is
> not exactly a nice thing to do.

If we stopped shipping configure (in our tarballs) and configure.in
(in the sources), would distros in the above list that do currently
use autoconf be able to build purely by having reasonable setting in
config.mak.uname already?

> And that's why I think we should have an alternative way to configure
> and build Git that can act as a replacement for autoconf, with my vote
> going to either CMake or Meson.

I guess the above question from me is a semi- tongue-in-cheek vote
for config.mak.uname as that alternative way.

> They are a proper replacement for autoconf that makes the
> downstream maintainer's jobs easier while also bringing additional
> features to the table that we don't currently have.
>
> Eli makes a couple of good remarks in [1] about things that both CMake
> and Meson bring to the table in addition to that, while also mentioning
> some of the benefits of Meson over CMake.
>
> I would be okay to make Git work with such a build system myself. The
> current CMake build instructions can be used to _build_ Git, but AFAIU
> they cannot yet run the Git test suite. Dscho pointed me to a couple of
> patches from Ævar that work into this direction, and I'd be happy to
> revive them. I'd also be okay with picking Meson over CMake if that is
> what people want. But my ultimate goal would then be that we have at
> least one CI job build and test against such a build system and give it
> the "official blessing" as an alternative way to build Git.

I already said that having to support two is better than having to
support three in another thread ;-).  If adding the fourth would
allow us to drop all other three eventually, that would be nice.

Our dependance of heavy use of GNU-ism in our Makefiles makes an
argument that make is the common denominator a fairly weak one, so
the single one that eventually we use does not have to be "make",
but it has to be something available widely and easy to learn.

Thanks.

Re: ./configure fails to link test program due to missing dependencies

[Paul Smith]

On Tue, 2024-09-24 at 10:39 -0700, Junio C Hamano wrote:
> Our dependance of heavy use of GNU-ism in our Makefiles makes an
> argument that make is the common denominator a fairly weak one, so
> the single one that eventually we use does not have to be "make",
> but it has to be something available widely and easy to learn.

Regardless of what one might imagine :), I am not advocating GNU Make
as the perfect solution: it certainly has downsides and disadvantages.

But, it also has benefits that should not be ignored: for example, it's
highly portable and it consists of a single binary that can be copied
anywhere and run from anywhere with no other prerequisites or need for
any setup or privileges.  Also it's extremely flexible since it just
runs shell scripts.  That also makes portability much more "do it
yourself" than other tools of course.

Meson is portable, but that's because it's written in Python: that
means you have to have a Python interpreter already available
(currently Python 3.7 or better), and the ability to add new modules to
it.  Admittedly this is not a super-high bar in 2024, but it's a non-
trivial requirement if you're trying to start from scratch.

Things like cmake provide abstractions that can make building code
simple, but it can be surprisingly difficult to get them to perform
more advanced tricks like generating source files, linker map files,
etc.  It can be done, but it's... not always easy.  And it has some
weird behaviors (for example how cached variables are handled will
certainly confuse you at first).

Re: ./configure fails to link test program due to missing dependencies

[Eli Schwartz]

[-- Attachment #1.1: Type: text/plain, Size: 3498 bytes --]

On 9/25/24 11:33 AM, Paul Smith wrote:
> On Tue, 2024-09-24 at 10:39 -0700, Junio C Hamano wrote:
>> Our dependance of heavy use of GNU-ism in our Makefiles makes an
>> argument that make is the common denominator a fairly weak one, so
>> the single one that eventually we use does not have to be "make",
>> but it has to be something available widely and easy to learn.
> 
> Regardless of what one might imagine :), I am not advocating GNU Make
> as the perfect solution: it certainly has downsides and disadvantages.


:)

I've read your article about why people should use autoconf!

(By the way: I had a bit of a... chuckle, when I read in your previous
email that as the GNU maintainer of Make, you build lots of projects
with Make or CMake, but not with GNU autoconf / automake. I assume that
was just bad wording?)


> But, it also has benefits that should not be ignored: for example, it's
> highly portable and it consists of a single binary that can be copied
> anywhere and run from anywhere with no other prerequisites or need for
> any setup or privileges.  Also it's extremely flexible since it just
> runs shell scripts.  That also makes portability much more "do it
> yourself" than other tools of course.
> 
> Meson is portable, but that's because it's written in Python: that
> means you have to have a Python interpreter already available
> (currently Python 3.7 or better), and the ability to add new modules to
> it.  Admittedly this is not a super-high bar in 2024, but it's a non-
> trivial requirement if you're trying to start from scratch.


FWIW: you don't need the ability to add new modules to python, you can
run meson by acquiring its sources (tarball or git clone, either one
works) and running meson as

$ python3 mesonsources/meson.py ....

No installation required.

You can also make a single-file executable using the "create_zipapp.py"
packer that ships in the meson sources. It uses python's ability to
execute a .zip archive by expecting the root of the zip file to contain

a) the file __main__.py containing the program entrypoint
b) any additional modules that should be available at runtime

You do still need python3, sure.

There are a few different tools available for creating single-file
executables that don't require a python interpreter. What they do is
create a self-extracting executable that includes its own python and
internalized support files. Meson uses https://pyinstaller.org to do
this in order to create the Windows .msi and macOS .dmg installer
bundles without requiring the user to install python. I've used it to
create Linux executable installers of meson too -- but I have no strong
feelings about linux executable installers existing, so I only bother
doing so in order to run tests on the packing process e.g. when I want
to verify that the Windows installers are ok without actually running
Windows.

It's not that hard to build a more or less standalone python that only
depends on "glibc from CentOS 7 or newer" to use as your base. There's
an unofficial project that hosts some precompiled versions at
https://gregoryszorc.com/docs/python-build-standalone/main/index.html

Single-file executables are alive and flourishing. :)

The same could probably be done for other operating systems and not just
"the big 3", but I lack direct personal experience with deploying
software to such systems so I can't really say for sure.


-- 
Eli Schwartz


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: ./configure fails to link test program due to missing dependencies

[Paul Smith]

On Wed, 2024-09-25 at 21:35 -0400, Eli Schwartz wrote:
> On 9/25/24 11:33 AM, Paul Smith wrote:
> > On Tue, 2024-09-24 at 10:39 -0700, Junio C Hamano wrote:
> > > Our dependance of heavy use of GNU-ism in our Makefiles makes an
> > > argument that make is the common denominator a fairly weak one,
> > > so
> > > the single one that eventually we use does not have to be "make",
> > > but it has to be something available widely and easy to learn.
> > 
> > Regardless of what one might imagine :), I am not advocating GNU
> > Make as the perfect solution: it certainly has downsides and
> > disadvantages.
> 
> :)
> 
> I've read your article about why people should use autoconf!
> 
> (By the way: I had a bit of a... chuckle, when I read in your
> previous email that as the GNU maintainer of Make, you build lots of
> projects with Make or CMake, but not with GNU autoconf / automake. I
> assume that was just bad wording?)

This is actually part of my $DAYJOB, not as the maintainer of GNU Make.
It's not even part of my job there, but I like to have modern tools and
I hate to mandate certain distributions and distro releases for
everyone, so I build a complete toolchain including its own sysroot
that everyone checks out from a Git repo to build our product.

At my $DAYJOB we use CMake, actually, so I'm also very familiar with
it.

But when I said "make" previously I meant to encompass autoconf
projects.  You're right, it's not really precise to compare make to
cmake since cmake build makefiles as well.  It's more accurate to
compare it to autoconf.

So, I should have said "autoconf and cmake" :)

However there are also projects that use raw makefiles and no build
generator tools.


I certainly didn't mean to imply that requiring Python is a show-
stopper for Git.  There are lots of options depending on needs.  But it
can't be argued that it's not more work for the builder than using GNU
Make.

Maybe this is will be deemed not worth worrying about given the
alternatives, and I wouldn't argue with that.  I just raise it so it
can be given consideration before a decision is made to (for example)
drop support for makefiles.

Re: ./configure fails to link test program due to missing dependencies

[Eli Schwartz]

[-- Attachment #1.1: Type: text/plain, Size: 1673 bytes --]

On 9/15/24 12:37 PM, Junio C Hamano wrote:
> Looking at configure.ac, pkg-config is not used for any package.
> Specifically for curl, it seems that "curl-config --libs" is used.
> 
> Presumably the reason behind the current behaviour is combination of
> (1) ./configure is an after-thought in the build infrastructure for
> this project, (2) pkg-config was not ubiquitous back when autoconf
> support was written for this project, and (3) nobody considered
> "upgrading" our use of "curl-config" and our manual detection of
> dependency detection for other libraries to just use "pkg-config".
> 
> Patches welcome ;-)


As an aside: I am skeptical of this explanation.

pkg-config's first git commit in 2005 is an import of a 2001 commit from
another version control system (probably GNU Arch -- author "Arch
Librarian"). The changelog it includes starts a year earlier, in 2000.
Wikipedia agrees, claiming the initial release of pkg-config was in 2000
and citing gtk-devel mailing list discussions from June of that year.

git added a configure.ac script 6 years later. Curl itself had installed
a .pc file for 7 months at that time, and OpenSSL had installed one for
nearly 4 years.

But git's configure script looked -- and still looks -- for -lcurl only
to set NO_CURL for the Makefile. In 2018, ./configure started using
curl-config, in order to set CURL_LDFLAGS.

I suppose it was copied by rote from the Makefile, which has less
flexibility to try pkg-config and fall back to curl-config.

Part 1 of the explanation is valid: ./configure was an after-thought. :)


-- 
Eli Schwartz
Gentoo Developer and Meson Build maintainer

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 963 bytes --]