Re: [RFC PATCH v2 23/24] Documentation: add comparison of build systems

[Patrick Steinhardt <ps@pks.im>]

On Wed, Oct 16, 2024 at 02:36:54PM +0100, phillip.wood123@gmail.com wrote:
> > > > +=== Test integration
> > > > +
> > > > +It should be possible to integrate tests into the build system such that it is
> > > > +possible to build and test Git within the build system. Features which are nice
> > > > +to have:
> > > > +
> > > > +  - Track build-time dependencies for respective tests. Unit tests have
> > > > +    different requirements than integration tests.
> > > > +  - Allow filtering of which tests to run.
> > > > +  - Allow interactive tests that drop the user into a shell with `test_pause` or
> > > > +    `debug`.
> > > 
> > > Does this last point mean we want to be able to selectively pass
> > > --interactive to the test script(s) being run?
> > 
> > What I mean by this is that when I see that a specific test fails, I
> > want to be able to execute only that single test such that things like
> > `test_pause` fail. What I don't mean is that the build system should
> > know to automatically rerun failing tests with that.
> > 
> > I've reformulated it to "Allow running tests such that utilities like
> > `test_pause` or `debug` work."
> 
> Ah so this is basically "there should be a way to disable parallelization
> and let the test access the terminal"?

Exactly.

With out-of-tree builds the development process is going to change a
bit, and it is going to be closer integrated with the build system.
Right now it's easy to manually execute a test and thus give it access
to the terminal. And while that shouldn't go away, you'd now have to set
up an additional environment variable to tell the test where the build
directory lives. There isn't really a way to avoid that.

But to me that also means that it's important that features we're all
used to remain easily accessible, and `test_pause` and `debug` are two
things that many developers probably use on a daily basis. I was worried
about that a when starting out on the Meson journey, but now I actually
have to say that I already favor the Meson workflow over the Makefile
workflow. Previously, I always used to do:

    $ make -C .. -j20 && ./t1234-something.sh

With Meson that becomes:

    $ meson test -i t1234-something

Which, at least to me, somehow feels way more enjoyable.

> > > > +- Ease of use: easy to use, discovering available options is easy. The
> > > > +  scripting language is straight-forward to use.
> > > > +- IDE support: Supports generating build instructions for Xcode and Microsoft
> > > > +  Visual Studio, a plugin exists for Visual Studio Code.
> > > 
> > > This is my main concern about meson - it means we either loose the nice
> > > integration on Windows that we have with CMake or we have to continue to
> > > maintain both. As I understand it Microsoft Visual Studio support requires
> > > the user to open a mingw terminal and run some to generate a build
> > > description which they can then use form the GUI which is what the CMake
> > > support was added to avoid. I guess they also need to install meson somehow
> > > as well.
> > 
> > I'm personally not particularly worried about having to generate the
> > MSVC solution from the command line once, as long as things just work
> > from thereon without requiring the developer to jump through hoops to
> > get it set up. It certainly doesn't seem like a particularly high
> > barrier to me, and should be a huge improvement compared to our current
> > Makefile.
> 
> It's an improvement on our Makefile but a regression compared to our
> CMakeList.txt which was specifically aimed making it easy for new
> contributors to get started without downloading any extra software or
> running terminal commands.

Yeah, that's fair enough.

> > I'm mostly there by now with the subprojects added in this version of
> > the patch series, which make it way easier to use MSVC without all deps
> > having been installed. But I still have to port over the SANE_TOOL_PATH
> > hack that we have in CMake.
> > 
> > I do understand that just clicking a button to import a CMakeLists.txt
> > is easier. It's mostly that I personally value the sanity that Meson
> > brings with it higher, which is of course a subjective opinion.
> 
> Right, I suspect the people who added support for building git in Visual
> Studio with CMake have different priorities. It's a real shame the meson
> there isn't a meson plugin for Visual Studio.

It certainly would be great to have such a plugin. There is one for
Visual Studio Code, but of course MSVC and VSC aren't the same. Which
reminds me, I should give that plugin a try.

In any case, my hope would be that eventually such a plugin does show
up. My gut feeling tells me that Meson has been picking up quite a bit
of steam over the last couple years, and more and more projects are
picking it up. And that makes me hopeful that we would eventually get
such a plugin.

It's of course wishful thinking, but as said, meanwhile we do have a way
for MSVC by generating the solution manually. So I'd already see it as a
net win if the official build system supports MSVC, even though not as
easily as with the click of a button.

Patrick