[PATCH] t/meson.build: fix up tests for 'seen' branch

[PATCH] t/meson.build: fix up tests for 'seen' branch

[Ramsay Jones]

Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
---

Hi Patrick,

So, having got the build to work, I noticed that 'ninja test' was running
less tests than 'make test': 1028 v. 1035. Having a fixed list of tests
to run has 'issues'. :) This patch fixes up the list for the 'seen' branch.

The 'new' tests in 'seen' are:

  t1017-cat-file-remote-object-info.sh
    branch: 'ej/cat-file-remote-object-info'
    commit: ef30c4557c ("cat-file: add remote-object-info to batch-command",
    2024-09-25)

  t5710-promisor-remote-capability.sh
    branch: 'cc/promisor-remote-capability'
    commit: 3cb155a011 ("Add 'promisor-remote' capability to protocol v2",
    2024-09-10)

  t6601-path-walk.sh
    branch: 'ds/path-walk'
    commit: d7e16a4ca9 ("t6601: add helper for testing path-walk API",
    2024-10-08)

  t8015-blame-default-ignore-revs.sh
    branch: 'am/git-blame-ignore-revs-by-default'
    commit: eca5438740 ("blame: respect .git-blame-ignore-revs automatically",
    2024-10-12)

  t8016-blame-override-ignore-revs.sh
    branch: 'am/git-blame-ignore-revs-by-default'
    commit: 0386bb36aa ("blame: introduce --override-ignore-revs to bypass
    ignore revisions list", 2024-10-12)

A 'new' test in 'next':

  t2408-worktree-relative.sh
    branch: 'cw/worktree-relative'
    commit: 717af916cd ("worktree: link worktrees with relative paths",
    2024-10-07)

A missing test which (as you know ;) ) is in maint:

  unit-tests/t-reftable-reader.c
    commit: 0a148a8eda ("reftable/reader: make table iterator reseekable",
    2024-09-16)

Hmm, this is going to be a PITA as far as maintenance is concerned! :(
If I am reading it correctly, the cmake solution uses file globbing
to get the list of test files to run - could meson do the same?

ATB,
Ramsay Jones



 t/meson.build | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/t/meson.build b/t/meson.build
index 152fd7dc13..a956e6db75 100644
--- a/t/meson.build
+++ b/t/meson.build
@@ -43,6 +43,7 @@ unit_test_programs = [
   'unit-tests/t-reftable-block.c',
   'unit-tests/t-reftable-merged.c',
   'unit-tests/t-reftable-pq.c',
+  'unit-tests/t-reftable-reader.c',
   'unit-tests/t-reftable-readwrite.c',
   'unit-tests/t-reftable-record.c',
   'unit-tests/t-reftable-stack.c',
@@ -166,6 +167,7 @@ integration_tests = [
   't1014-read-tree-confusing.sh',
   't1015-read-index-unmerged.sh',
   't1016-compatObjectFormat.sh',
+  't1017-cat-file-remote-object-info.sh',
   't1020-subdirectory.sh',
   't1021-rerere-in-workdir.sh',
   't1022-read-tree-partial-clone.sh',
@@ -295,6 +297,7 @@ integration_tests = [
   't2405-worktree-submodule.sh',
   't2406-worktree-repair.sh',
   't2407-worktree-heads.sh',
+  't2408-worktree-relative.sh',
   't2500-untracked-overwriting.sh',
   't2501-cwd-empty.sh',
   't3000-ls-files-others.sh',
@@ -716,6 +719,7 @@ integration_tests = [
   't5703-upload-pack-ref-in-want.sh',
   't5704-protocol-violations.sh',
   't5705-session-id-in-capabilities.sh',
+  't5710-promisor-remote-capability.sh',
   't5730-protocol-v2-bundle-uri-file.sh',
   't5731-protocol-v2-bundle-uri-git.sh',
   't5732-protocol-v2-bundle-uri-http.sh',
@@ -818,6 +822,7 @@ integration_tests = [
   't6500-gc.sh',
   't6501-freshen-objects.sh',
   't6600-test-reach.sh',
+  't6601-path-walk.sh',
   't6700-tree-depth.sh',
   't7001-mv.sh',
   't7002-mv-sparse-checkout.sh',
@@ -944,6 +949,8 @@ integration_tests = [
   't8012-blame-colors.sh',
   't8013-blame-ignore-revs.sh',
   't8014-blame-ignore-fuzzy.sh',
+  't8015-blame-default-ignore-revs.sh',
+  't8016-blame-override-ignore-revs.sh',
   't9001-send-email.sh',
   't9002-column.sh',
   't9003-help-autocorrect.sh',
-- 
2.47.0

Re: [PATCH] t/meson.build: fix up tests for 'seen' branch

[Patrick Steinhardt]

On Sun, Oct 20, 2024 at 05:00:44PM +0100, Ramsay Jones wrote:

Thanks, I'll pick these up.

> Hmm, this is going to be a PITA as far as maintenance is concerned! :(
> If I am reading it correctly, the cmake solution uses file globbing
> to get the list of test files to run - could meson do the same?

In theory we can, yes. But there's a big problem with it, both in Meson
and in CMake: the instructions to deduce source files only get executed
at configure time. Consequently, when new files get added, the build
instructions do not get updated accordingly and are thus broken. So
CMake does get around this, but not in a way that is feasible for use as
our main build system, and the same would be true for Meson.

For our integration-style tests I'd be okay with not listing the files
individually, such that we instead use e.g. prove(1) to run all tests
via a single test target. It would be a regression in functionality as
we now cannot easily run e.g. "meson test t0000*", but at least we would
not have to maintain the list of test scripts anymore.

But for our source files I don't really see an alternative to listing
them out explicitly. You don't want things like git-rebase(1) or
git-bisect(1) to be broken just because one of the commits happens to
add or remove a file. And this is true for whichever build system we
want to adopt as additional official buildsystem, whether that is CMake
or Meson.

I guess we have less churn here anyway, and we would notice breakage
more readily because things would stop compiling. So this is likely less
of a problem compared to our tests.

Patrick

Re: [PATCH] t/meson.build: fix up tests for 'seen' branch

[Taylor Blau]

On Mon, Oct 21, 2024 at 08:04:16AM +0200, Patrick Steinhardt wrote:
> On Sun, Oct 20, 2024 at 05:00:44PM +0100, Ramsay Jones wrote:
>
> Thanks, I'll pick these up.

Thanks. Please make sure to let me know which topic(s) that the new
version of ps/build depends on as a result so I can queue it properly.

> > Hmm, this is going to be a PITA as far as maintenance is concerned! :(
> > If I am reading it correctly, the cmake solution uses file globbing
> > to get the list of test files to run - could meson do the same?
>
> In theory we can, yes. But there's a big problem with it, both in Meson
> and in CMake: the instructions to deduce source files only get executed
> at configure time. Consequently, when new files get added, the build
> instructions do not get updated accordingly and are thus broken. So
> CMake does get around this, but not in a way that is feasible for use as
> our main build system, and the same would be true for Meson.
>
> For our integration-style tests I'd be okay with not listing the files
> individually, such that we instead use e.g. prove(1) to run all tests
> via a single test target. It would be a regression in functionality as
> we now cannot easily run e.g. "meson test t0000*", but at least we would
> not have to maintain the list of test scripts anymore.

I quite like the ability to run tNNNN-*.sh individually when spelled
like that (i.e. including the test number, but not the full script
name).

I find that for the areas that I have worked on most over the years that
I have the relevant test numbers committed by heart, but often don't
know the full script name. So it is nice to be able to refer to them by
number only.

Am I reading it write that this suggests that Meson would lose that
ability?

Thanks,
Taylor

Re: [PATCH] t/meson.build: fix up tests for 'seen' branch

[Eli Schwartz]

[-- Attachment #1.1: Type: text/plain, Size: 1494 bytes --]

On 10/21/24 5:14 PM, Taylor Blau wrote:
> On Mon, Oct 21, 2024 at 08:04:16AM +0200, Patrick Steinhardt wrote:
>> For our integration-style tests I'd be okay with not listing the files
>> individually, such that we instead use e.g. prove(1) to run all tests
>> via a single test target. It would be a regression in functionality as
>> we now cannot easily run e.g. "meson test t0000*", but at least we would
>> not have to maintain the list of test scripts anymore.
> 
> I quite like the ability to run tNNNN-*.sh individually when spelled
> like that (i.e. including the test number, but not the full script
> name).
> 
> I find that for the areas that I have worked on most over the years that
> I have the relevant test numbers committed by heart, but often don't
> know the full script name. So it is nice to be able to refer to them by
> number only.
> 
> Am I reading it write that this suggests that Meson would lose that
> ability?


If "meson test" simply ran the prove(1) utility then it would still be
possible to do:

```
meson test --test-args "--args --to --prove"
```

This would not actually do what you want, since prove(1) doesn't
implement "t0000*" style globbing, does it?

So your reading is correct -- meson's own globbing support relies on
meson knowing the list of all test names, which means you do have to
maintain that list somehow (and have that list update when you do "git
pull" and new tests materialize).


-- 
Eli Schwartz

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 963 bytes --]