Re: [RFC PATCH v4 00/19] Modernize the build system

[Taylor Blau <me@ttaylorr.com>]

On Thu, Oct 24, 2024 at 02:39:43PM +0200, Patrick Steinhardt wrote:
> Hi,
>
> this is the fourth version of my patch series that modernizes our build
> system. It refactors various parts of it to make it possible to perform
> out-of-tree builds in theory and then wires up Meson.

I was thinking a little bit about this topic last night and wanted to
collect my thoughts somewhere.

I think that there are a couple of things that I'm not 100% clear or
sold on, which are:

  - What is the eventual goal of this series? Do we plan to transition
    the existing make-driven builds to instead be built with Meson? Will
    we have both? Something else?

  - What is the eventual plan for CMake, which is maintained currently
    in the contrib directory? Will it be deprecated in favor of Meson?
    Will we continue to support it?

Let me expand a little on each of those in turn:

== What is the eventual goal?

From reading [PATCH 17/19], it seems the main arguments in favor of
using Meson are:

  - Ease of use: easy to use, discovering available options is easy. The
    scripting language is straight-forward to use.
  - IDE support: Supports generating build instructions for Xcode and Microsoft
    Visual Studio, a plugin exists for Visual Studio Code.
  - Out-of-tree builds: supported.
  - Cross-platform builds: supported.
  - Language support:
    - C: Supported for GCC, Clang, MSVC and other toolchains.
    - Rust: Supported for rustc.
  - Test integration: supported. Interactive tests are supported starting with
    Meson 1.5.0 via the `--interactive` flag.

I don't think that when reading these any of them stick out to me and
compel me to learn a new build system. I understand and am sympathetic
to the fact that GNU Make has odd syntax and can be cumbersome. But I
don't think that incrementally modifying our Makefile over time is
difficult, and it has certainly worked well over the years.

Certainly there is ample support for IDE integration with Make.
Out-of-tree builds and cross-platform builds could be supported in
theory as you note within the existing build system. Another suggestion
you make is that Meson has better native support for Rust, which I agree
may be important to consider in the future.

But I don't think that any of those three (out-of-tree builds,
cross-compilation, or Rust support) are insurmountable challenges in
Make. Certainly there is a lot of inertia there, but I don't think
that's a bad thing. Contributors are used to the existing build system,
it has worked well for us, works across many platforms and has (IMO)
stood the test of time.

I admittedly have a hard time squaring the benefits and goals we have
with Meson with the cost of learning a new build system, and/or moving
away from Make entirely.

I am entirely open to the possibility that there is something that I am
missing here, and that Meson really is a better choice for Git given our
current direction. But I think if that's true, then the series needs to
explain that more prominently.

== What is the eventual plan for CMake?

From [PATCH 18/19], you write:

> If this patch lands the expectation is that it will coexist
> with our other build systems for a while. Like this, distributions can
> slowly migrate over to Meson and report any findings they have to us
> such that we can continue to iterate. A potential cutoff date for other
> build systems may be Git 3.0.

I don't view this is a good intermediate state, and is in my view making
an existing problem that we have worse. On the "existing problem": I
think that landing CMake support in contrib was a mistake. In my view,
CMake support should have either been a first-class citizen in Git (such
that we don't consider a change "done" until it can be built by CMake),
or should have been maintained out-of-tree.

But I think we struck a worst-of-both-worlds balance by landing it in
contrib. It's maintained in the tree, so people expect to be able to
build the project with it because it comes with a bog-standard clone of
git.git.

But despite living in the project's tree, it is not a first-class
citizen, and subjectively it seems that we get an awful lot of mail
asking why something doesn't build in CMake, etc. (To your credit, I
think you have been one of the main people to help with that, often
fixing those bugs yourself, which is greatly appreciated).

I don't want to see the project have to pseudo-maintain three build
systems. It seems like doing so would be cumbersome, and error-prone. I
am already probably guilty of breaking CMake builds when I add a new
compilation unit to the project, because of a combination of my lack of
familiarity with CMake, and the fact that we don't have a project-wide
convention of treating it with the same care as we do the Makefile. I
think that having three build systems (even if they only co-existed
until Git 3.0) would make that problem worse.

I feel that if we are going to pursue Meson over CMake and/or Make, we
should have a clear plan to either get rid of CMake, keep it up-to-date,
or something else.

== Conclusion

To step back, I want to say that I appreciate your work on this series
and am certainly not opposed[^1] to the idea that we may need to make
significant changes to our build infrastructure to support the project's
goals.

But I think that what I'm missing currently is a clear picture of what
goals we *can't* achieve with the existing build system (or could, but
only at significant cost/awkwardness), and why Meson is the right choice
to address those gaps.

If the project can agree that pursuing Meson as a replacement for Make,
CMake, or both, then I think we would need further clarification on what
we want to do with CMake (and more generally how we want to support new
efforts to add additional build systems to Git in the future).

Anyway. If I have significantly missed the point here, please feel free
to correct me. In the meantime, I think discussing these points would be
useful to clarifying the direction of the series.

Thanks,
Taylor

[^1]: Both in my current capacity as interim-maintainer, but also as a
  regular contributor to the project, who would (in both cases) be
  interested in evolving the build system we use in furtherance of
  project goals.