[PATCH 0/2] cat-file related doc and test

[PATCH 0/2] cat-file related doc and test

[Eric Wong]

I was working on reducing syscalls required for cat-file --batch
(and readers) in a different patch, but noticed a bug in my
yet-to-be-published patch wasn't detected in our test suite,
only via 3rd-party Perl code.

Then I noticed Git.pm documentation was wrong..., so fixes
are in reverse order.

Eric Wong (2):
  Git.pm: use array in command_bidi_pipe example
  t9700: ensure cat-file info isn't buffered by default

 perl/Git.pm     |  4 ++--
 t/t9700/test.pl | 14 ++++++++++++++
 2 files changed, 16 insertions(+), 2 deletions(-)

[PATCH 1/2] Git.pm: use array in command_bidi_pipe example

[Eric Wong]

command_bidi_pipe takes the git command and optional arguments as an
array, not a string.  Make sure the documentation example is usable
code.

Signed-off-by: Eric Wong <e@80x24.org>
---
 perl/Git.pm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/perl/Git.pm b/perl/Git.pm
index 03bf570bf4..aebfe0c6e0 100644
--- a/perl/Git.pm
+++ b/perl/Git.pm
@@ -418,7 +418,7 @@ sub command_bidi_pipe {
 and it is the fourth value returned by C<command_bidi_pipe()>.  The call idiom
 is:
 
-	my ($pid, $in, $out, $ctx) = $r->command_bidi_pipe('cat-file --batch-check');
+	my ($pid, $in, $out, $ctx) = $r->command_bidi_pipe(qw(cat-file --batch-check));
 	print $out "000000000\n";
 	while (<$in>) { ... }
 	$r->command_close_bidi_pipe($pid, $in, $out, $ctx);
@@ -431,7 +431,7 @@ sub command_bidi_pipe {
 calling this function.  This may be useful in a query-response type of
 commands where caller first writes a query and later reads response, eg:
 
-	my ($pid, $in, $out, $ctx) = $r->command_bidi_pipe('cat-file --batch-check');
+	my ($pid, $in, $out, $ctx) = $r->command_bidi_pipe(qw(cat-file --batch-check));
 	print $out "000000000\n";
 	close $out;
 	while (<$in>) { ... }

Re: [PATCH 1/2] Git.pm: use array in command_bidi_pipe example

[Junio C Hamano]

Eric Wong <e@80x24.org> writes:

> command_bidi_pipe takes the git command and optional arguments as an
> array, not a string.  Make sure the documentation example is usable
> code.

Makes sense.

>
> Signed-off-by: Eric Wong <e@80x24.org>
> ---
>  perl/Git.pm | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/perl/Git.pm b/perl/Git.pm
> index 03bf570bf4..aebfe0c6e0 100644
> --- a/perl/Git.pm
> +++ b/perl/Git.pm
> @@ -418,7 +418,7 @@ sub command_bidi_pipe {
>  and it is the fourth value returned by C<command_bidi_pipe()>.  The call idiom
>  is:
>  
> -	my ($pid, $in, $out, $ctx) = $r->command_bidi_pipe('cat-file --batch-check');
> +	my ($pid, $in, $out, $ctx) = $r->command_bidi_pipe(qw(cat-file --batch-check));
>  	print $out "000000000\n";
>  	while (<$in>) { ... }
>  	$r->command_close_bidi_pipe($pid, $in, $out, $ctx);
> @@ -431,7 +431,7 @@ sub command_bidi_pipe {
>  calling this function.  This may be useful in a query-response type of
>  commands where caller first writes a query and later reads response, eg:
>  
> -	my ($pid, $in, $out, $ctx) = $r->command_bidi_pipe('cat-file --batch-check');
> +	my ($pid, $in, $out, $ctx) = $r->command_bidi_pipe(qw(cat-file --batch-check));
>  	print $out "000000000\n";
>  	close $out;
>  	while (<$in>) { ... }

[PATCH 2/2] t9700: ensure cat-file info isn't buffered by default

[Eric Wong]

While working on buffering changes to `git cat-file' in a
separate patch, I inadvertently made the output of --batch-check
and the `info' command of --batch-command buffered by default.

Buffering by default breaks some 3rd-party Perl scripts using
cat-file, but this breakage was not detected anywhere in our
test suite.  The easiest place to test this behavior is with
Git.pm, since (AFAIK) other equivalent way to test this behavior
from Bourne shell and/or awk would require racy sleeps,
non-portable FIFOs or tedious C code.

Signed-off-by: Eric Wong <e@80x24.org>
---
 t/t9700/test.pl | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/t/t9700/test.pl b/t/t9700/test.pl
index d8e85482ab..94a2e2c09d 100755
--- a/t/t9700/test.pl
+++ b/t/t9700/test.pl
@@ -154,6 +154,20 @@ sub adjust_dirsep {
 		     "abc\"\\ \x07\x08\x09\x0a\x0b\x0c\x0d\x01 ",
 		     'unquote escape sequences');
 
+# ensure --batch-check is unbuffered by default
+my ($pid, $in, $out, $ctx) = $r->command_bidi_pipe(qw(cat-file --batch-check));
+print $out $file1hash, "\n" or die $!;
+my $info = <$in>;
+is $info, "$file1hash blob 15\n", 'command_bidi_pipe w/ --batch-check';
+$r->command_close_bidi_pipe($pid, $in, $out, $ctx);
+
+# ditto with `info' with --batch-command
+($pid, $in, $out, $ctx) = $r->command_bidi_pipe(qw(cat-file --batch-command));
+print $out 'info ', $file1hash, "\n" or die $!;
+$info = <$in>;
+is $info, "$file1hash blob 15\n", 'command_bidi_pipe w/ --batch-command=info';
+$r->command_close_bidi_pipe($pid, $in, $out, $ctx);
+
 printf "1..%d\n", Test::More->builder->current_test;
 
 my $is_passing = eval { Test::More->is_passing };

Re: [PATCH 2/2] t9700: ensure cat-file info isn't buffered by default

[Junio C Hamano]

Eric Wong <e@80x24.org> writes:

> Buffering by default breaks some 3rd-party Perl scripts using
> cat-file, but this breakage was not detected anywhere in our
> test suite.  The easiest place to test this behavior is with
> Git.pm, since (AFAIK) other equivalent way to test this behavior
> from Bourne shell and/or awk would require racy sleeps,
> non-portable FIFOs or tedious C code.

Yes, using Perl is a good substitute for writing it in C in this
case.  I however question the choice to use t9700/test.pl here,
which is clearly stated that its purpose is to "test perl interface
which is Git.pm", and added tests are not testing anything in Git.pm
at all.

Using t9700/test.pl only because it happens to use "perl -MTest::More"
sounds a bit eh, suboptimal.

It seems that there are Perl snippets in other tests (including
t1006 that is specifically about cat-file).  How involved would it
be to implement these new tests without modifying unrelated test
scripts?

> Signed-off-by: Eric Wong <e@80x24.org>
> ---
>  t/t9700/test.pl | 14 ++++++++++++++
>  1 file changed, 14 insertions(+)
>
> diff --git a/t/t9700/test.pl b/t/t9700/test.pl
> index d8e85482ab..94a2e2c09d 100755
> --- a/t/t9700/test.pl
> +++ b/t/t9700/test.pl
> @@ -154,6 +154,20 @@ sub adjust_dirsep {
>  		     "abc\"\\ \x07\x08\x09\x0a\x0b\x0c\x0d\x01 ",
>  		     'unquote escape sequences');
>  
> +# ensure --batch-check is unbuffered by default
> +my ($pid, $in, $out, $ctx) = $r->command_bidi_pipe(qw(cat-file --batch-check));
> +print $out $file1hash, "\n" or die $!;
> +my $info = <$in>;
> +is $info, "$file1hash blob 15\n", 'command_bidi_pipe w/ --batch-check';
> +$r->command_close_bidi_pipe($pid, $in, $out, $ctx);
> +
> +# ditto with `info' with --batch-command
> +($pid, $in, $out, $ctx) = $r->command_bidi_pipe(qw(cat-file --batch-command));
> +print $out 'info ', $file1hash, "\n" or die $!;
> +$info = <$in>;
> +is $info, "$file1hash blob 15\n", 'command_bidi_pipe w/ --batch-command=info';
> +$r->command_close_bidi_pipe($pid, $in, $out, $ctx);
> +
>  printf "1..%d\n", Test::More->builder->current_test;
>  
>  my $is_passing = eval { Test::More->is_passing };

[PATCH v2 2/2] t1006: ensure cat-file info isn't buffered by default

[Eric Wong]

Junio C Hamano <gitster@pobox.com> wrote:
> Eric Wong <e@80x24.org> writes:
> 
> > Buffering by default breaks some 3rd-party Perl scripts using
> > cat-file, but this breakage was not detected anywhere in our
> > test suite.  The easiest place to test this behavior is with
> > Git.pm, since (AFAIK) other equivalent way to test this behavior
> > from Bourne shell and/or awk would require racy sleeps,
> > non-portable FIFOs or tedious C code.
> 
> Yes, using Perl is a good substitute for writing it in C in this
> case.  I however question the choice to use t9700/test.pl here,
> which is clearly stated that its purpose is to "test perl interface
> which is Git.pm", and added tests are not testing anything in Git.pm
> at all.
> 
> Using t9700/test.pl only because it happens to use "perl -MTest::More"
> sounds a bit eh, suboptimal.

*shrug*  I figure Test::More is common enough since it's part of
the Perl standard library; but I consider Perl a better scripting
language than sh by far and wish our whole test suite were Perl :>

> It seems that there are Perl snippets in other tests (including
> t1006 that is specifically about cat-file).  How involved would it
> be to implement these new tests without modifying unrelated test
> scripts?

> >  t/t9700/test.pl | 14 ++++++++++++++
> >  1 file changed, 14 insertions(+)

More code than that.  At least IPC::Open2 takes care of the nasty
portability bits, but getting the Perl quoting nested properly
inside sh was confusing :x

v2: moved test to t1006 to avoid Test::More,
    add select timeout in case a buffering bug does get introduced,
    updated commit message and clarified the bug it's supposed
    to guard against
    (I initially tried stdio buffering, but moved away from it for the
    patch I'm testing...)

----8<----
Subject: [PATCH] t1006: ensure cat-file info isn't buffered by default

While working on buffering changes to `git cat-file' in a
separate patch, I inadvertently made the output of --batch-check
and the `info' command of --batch-command buffered as if
opt->buffer_output is turned on by default.

Buffering by default breaks some 3rd-party Perl scripts using
cat-file, but this breakage was not detected anywhere in our
test suite.  Add a small Perl snippet to test this problem since
(AFAIK) other equivalent ways to test this behavior from Bourne
shell and/or awk would require racy sleeps, non-portable FIFOs
or tedious C code.

Signed-off-by: Eric Wong <e@80x24.org>
---
 t/t1006-cat-file.sh | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh
index e12b221972..ff9bf213aa 100755
--- a/t/t1006-cat-file.sh
+++ b/t/t1006-cat-file.sh
@@ -1294,4 +1294,34 @@ test_expect_success 'batch-command flush without --buffer' '
 	grep "^fatal:.*flush is only for --buffer mode.*" err
 '
 
+script='
+use warnings;
+use strict;
+use IPC::Open2;
+my ($opt, $oid, $expect, @pfx) = @ARGV;
+my @cmd = (qw(git cat-file), $opt);
+my $pid = open2(my $out, my $in, @cmd) or die "open2: @cmd";
+print $in @pfx, $oid, "\n" or die "print $!";
+my $rvec = "";
+vec($rvec, fileno($out), 1) = 1;
+select($rvec, undef, undef, 30) or die "no response to `@pfx $oid` from @cmd";
+my $info = <$out>;
+chop($info) eq "\n" or die "no LF";
+$info eq $expect or die "`$info` != `$expect`";
+close $in or die "close in $!";
+close $out or die "close out $!";
+waitpid $pid, 0;
+$? == 0 or die "\$?=$?";
+'
+
+expect="$hello_oid blob $hello_size"
+
+test_expect_success PERL '--batch-check is unbuffered by default' '
+	perl -e "$script" -- --batch-check $hello_oid "$expect"
+'
+
+test_expect_success PERL '--batch-command info is unbuffered by default' '
+	perl -e "$script" -- --batch-command $hello_oid "$expect" "info "
+'
+
 test_done

Re: [PATCH v2 2/2] t1006: ensure cat-file info isn't buffered by default

[Junio C Hamano]

Eric Wong <e@80x24.org> writes:

>> Yes, using Perl is a good substitute for writing it in C in this
>> case.  I however question the choice to use t9700/test.pl here,
>> which is clearly stated that its purpose is to "test perl interface
>> which is Git.pm", and added tests are not testing anything in Git.pm
>> at all.
>> 
>> Using t9700/test.pl only because it happens to use "perl -MTest::More"
>> sounds a bit eh, suboptimal.
>
> *shrug*  I figure Test::More is common enough since it's part of
> the Perl standard library; but I consider Perl a better scripting
> language than sh by far and wish our whole test suite were Perl :>

Oh, I think we (actually the author of t9700) considers it common
enough that we have PERL_TEST_MORE prerequisite to allow us to write
tests, assuming that it is available, and let us easily skip where
it is not available.  So I do not think I mind the dependency on
Test::More at all.  Moving the tests to t1006 and rewriting the
tests not to use Test::More are two separate and unrelated things,
and if you are more comfortable with Test::More (and more
importantly if it is natural to write Perl based tests using
Test::More), it is not necessary to switch away from it.

Re: [PATCH v2 2/2] t1006: ensure cat-file info isn't buffered by default

[Eric Wong]

Junio C Hamano <gitster@pobox.com> wrote:
> Eric Wong <e@80x24.org> writes:
> 
> >> Yes, using Perl is a good substitute for writing it in C in this
> >> case.  I however question the choice to use t9700/test.pl here,
> >> which is clearly stated that its purpose is to "test perl interface
> >> which is Git.pm", and added tests are not testing anything in Git.pm
> >> at all.
> >> 
> >> Using t9700/test.pl only because it happens to use "perl -MTest::More"
> >> sounds a bit eh, suboptimal.
> >
> > *shrug*  I figure Test::More is common enough since it's part of
> > the Perl standard library; but I consider Perl a better scripting
> > language than sh by far and wish our whole test suite were Perl :>
> 
> Oh, I think we (actually the author of t9700) considers it common
> enough that we have PERL_TEST_MORE prerequisite to allow us to write
> tests, assuming that it is available, and let us easily skip where
> it is not available.  So I do not think I mind the dependency on
> Test::More at all.  Moving the tests to t1006 and rewriting the
> tests not to use Test::More are two separate and unrelated things,
> and if you are more comfortable with Test::More (and more
> importantly if it is natural to write Perl based tests using
> Test::More), it is not necessary to switch away from it.

OK, fair enough.  Given t1006 is mostly sh, I prefer keeping v2
as-is because the Test::More->builder munging of test numbers in
t9700/test.pl is nasty too and I wouldn't enjoy duplicating
those bits in a hypothetical t1006/test.pl, either.

It would be nice to have first class support for Test::More in
our suite so we could just have t/t0006-cat-file.t and
t/t9700-perl-git.t implemented in Perl without sh at all, but
that's a separate discussion.

Re: [PATCH v2 2/2] t1006: ensure cat-file info isn't buffered by default

[Junio C Hamano]

Eric Wong <e@80x24.org> writes:

> OK, fair enough.  Given t1006 is mostly sh, I prefer keeping v2
> as-is because the Test::More->builder munging of test numbers in
> t9700/test.pl is nasty too and I wouldn't enjoy duplicating
> those bits in a hypothetical t1006/test.pl, either.

That is quite sensible.

Re: [PATCH v2 2/2] t1006: ensure cat-file info isn't buffered by default

[Jeff King]

On Tue, Jun 18, 2024 at 09:30:41PM +0000, Eric Wong wrote:

> +script='
> +use warnings;
> +use strict;
> +use IPC::Open2;
> +my ($opt, $oid, $expect, @pfx) = @ARGV;
> +my @cmd = (qw(git cat-file), $opt);
> +my $pid = open2(my $out, my $in, @cmd) or die "open2: @cmd";
> +print $in @pfx, $oid, "\n" or die "print $!";
> +my $rvec = "";
> +vec($rvec, fileno($out), 1) = 1;
> +select($rvec, undef, undef, 30) or die "no response to `@pfx $oid` from @cmd";
> +my $info = <$out>;
> +chop($info) eq "\n" or die "no LF";
> +$info eq $expect or die "`$info` != `$expect`";
> +close $in or die "close in $!";
> +close $out or die "close out $!";
> +waitpid $pid, 0;
> +$? == 0 or die "\$?=$?";
> +'
> +
> +expect="$hello_oid blob $hello_size"
> +
> +test_expect_success PERL '--batch-check is unbuffered by default' '
> +	perl -e "$script" -- --batch-check $hello_oid "$expect"
> +'

We often use "perl -e" for one-liners, etc, but this is pretty big.
Maybe:

  cat >foo.pl <<-\EOF
  ...
  EOF
  perl foo.pl -- ...

would be more readable? To be clear I don't think there's anything
incorrect about your usage, but it would match the style of our suite a
bit better.

Likewise, it would be usual in our suite for the helper to do the
minimum that needs to be in perl, and use our normal functions for
things like comparing output (rather than taking its own "expect"
argument).

So maybe:

diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh
index e12b221972..929d7a7579 100755
--- a/t/t1006-cat-file.sh
+++ b/t/t1006-cat-file.sh
@@ -1294,4 +1294,33 @@ test_expect_success 'batch-command flush without --buffer' '
 	grep "^fatal:.*flush is only for --buffer mode.*" err
 '
 
+# Copy a single line from stdin to the program specified
+# by @ARGV, and then wait for a response _without_ closing
+# the pipe.
+cat >run-and-wait.pl <<-\EOF
+use IPC::Open2;
+open2(my $out, my $in, @ARGV) or die "open2: @ARGV";
+print $in scalar(<STDIN>) or die "print $!";
+
+my $rvec = "";
+vec($rvec, fileno($out), 1) = 1;
+select($rvec, undef, undef, 30) or die "no response after 30 seconds";
+
+print scalar(<$out>);
+EOF
+
+test_expect_success PERL '--batch-check is unbuffered by default' '
+	echo "$hello_oid" |
+	perl run-and-wait.pl git cat-file --batch-check >out &&
+	echo "$hello_oid blob $hello_size" >expect &&
+	test_cmp expect out
+'
+
+test_expect_success PERL '--batch-command info is unbuffered by default' '
+	echo "info $hello_oid" |
+	perl run-and-wait.pl git cat-file --batch-command >out &&
+	echo "$hello_oid blob $hello_size" >expect &&
+	test_cmp expect out
+'
+
 test_done

I went for brevity above. Notably missing are:

  - the use of strict/warnings. I think we've shied away from these in
    the test suite because we want to run on any version of perl. In my
    experience most strict/warnings output is actually telling you about
    obvious garbage, but not always. IIRC perl got more strict about
    "()" around lists in some contexts a few years back, and code which
    used to be OK started generating warnings. OTOH, those warnings were
    probably a sign of problems-to-come, anyway. Without "FATAL",
    though, I think "use warnings" is not doing much good (nobody is
    ever going to see its output if the test isn't failing).

  - I dropped the close/waitpid. I guess maybe it is valuable to confirm
    that cat-file did not barf, but IMHO the important thing here is
    testing that it produced the single line of output we expected.

-Peff

Re: [PATCH v2 2/2] t1006: ensure cat-file info isn't buffered by default

[Eric Wong]

Jeff King <peff@peff.net> wrote:
> On Tue, Jun 18, 2024 at 09:30:41PM +0000, Eric Wong wrote:
> 
> > +script='

<snip>

> > +expect="$hello_oid blob $hello_size"
> > +
> > +test_expect_success PERL '--batch-check is unbuffered by default' '
> > +	perl -e "$script" -- --batch-check $hello_oid "$expect"
> > +'
> 
> We often use "perl -e" for one-liners, etc, but this is pretty big.
> Maybe:
> 
>   cat >foo.pl <<-\EOF
>   ...
>   EOF
>   perl foo.pl -- ...
> 
> would be more readable? To be clear I don't think there's anything
> incorrect about your usage, but it would match the style of our suite a
> bit better.

*shrug*  It doesn't save the nested quoting/expansion confusion;
but it's Junio's call.  I don't think a v3 is worth the effort.

> Likewise, it would be usual in our suite for the helper to do the
> minimum that needs to be in perl, and use our normal functions for
> things like comparing output (rather than taking its own "expect"
> argument).

<snip>

> +test_expect_success PERL '--batch-check is unbuffered by default' '
> +	echo "$hello_oid" |
> +	perl run-and-wait.pl git cat-file --batch-check >out &&
> +	echo "$hello_oid blob $hello_size" >expect &&
> +	test_cmp expect out

I prefer to avoid process spawning overhead from test_cmp;
but that's a small drop in a big bucket.

> I went for brevity above. Notably missing are:
> 
>   - the use of strict/warnings. I think we've shied away from these in
>     the test suite because we want to run on any version of perl. In my
>     experience most strict/warnings output is actually telling you about
>     obvious garbage, but not always. IIRC perl got more strict about
>     "()" around lists in some contexts a few years back, and code which
>     used to be OK started generating warnings. OTOH, those warnings were
>     probably a sign of problems-to-come, anyway. Without "FATAL",
>     though, I think "use warnings" is not doing much good (nobody is
>     ever going to see its output if the test isn't failing).

It may make problems easier to find if there are failures,
so I think the potential benefits outweight any downsides.

>   - I dropped the close/waitpid. I guess maybe it is valuable to confirm
>     that cat-file did not barf, but IMHO the important thing here is
>     testing that it produced the single line of output we expected.

I've found some unexpected bugs through excessive error checking
in the past, so much preferred to keep them.

Re: [PATCH v2 2/2] t1006: ensure cat-file info isn't buffered by default

[Jeff King]

On Fri, Jun 21, 2024 at 08:00:02PM +0000, Eric Wong wrote:

> > We often use "perl -e" for one-liners, etc, but this is pretty big.
> > Maybe:
> > 
> >   cat >foo.pl <<-\EOF
> >   ...
> >   EOF
> >   perl foo.pl -- ...
> > 
> > would be more readable? To be clear I don't think there's anything
> > incorrect about your usage, but it would match the style of our suite a
> > bit better.
> 
> *shrug*  It doesn't save the nested quoting/expansion confusion;
> but it's Junio's call.  I don't think a v3 is worth the effort.

It does allow you to use single quotes in the script, though I think you
managed without it.

> > +test_expect_success PERL '--batch-check is unbuffered by default' '
> > +	echo "$hello_oid" |
> > +	perl run-and-wait.pl git cat-file --batch-check >out &&
> > +	echo "$hello_oid blob $hello_size" >expect &&
> > +	test_cmp expect out
> 
> I prefer to avoid process spawning overhead from test_cmp;
> but that's a small drop in a big bucket.

If we care about that, I'd rather see us make test_cmp zero-process with
a shell helper than come up with ad-hoc solutions. I've tried to measure
something like that before, but couldn't come up with any conclusive
improvements (my findings were mostly that running Git itself accounts
for most of the process overhead).

-Peff

Re: [PATCH 2/2] t9700: ensure cat-file info isn't buffered by default

[Junio C Hamano]

Eric Wong <e@80x24.org> writes:

> While working on buffering changes to `git cat-file' in a
> separate patch, I inadvertently made the output of --batch-check
> and the `info' command of --batch-command buffered by default.

Here "buffered" means "as if opt->buffer_output is turned on", which
in turn means "the output goes through stdio"?  Just making sure
that my understanding of what the breakage was is in line with what
you wanted to convey.

Thanks.

Re: [PATCH 2/2] t9700: ensure cat-file info isn't buffered by default

[Phillip Wood]

Hi Eric

On 17/06/2024 11:43, Eric Wong wrote:
> +# ensure --batch-check is unbuffered by default
> +my ($pid, $in, $out, $ctx) = $r->command_bidi_pipe(qw(cat-file --batch-check));
> +print $out $file1hash, "\n" or die $!;

It's been a while since I did any perl scripting and I'm not clear 
whether $out is buffered or not and if it is whether it is guaranteed to 
be flushed when we print "\n". It might be worth adding a explicit flush 
so it is clear that any deadlocks come from cat-file and not our test code.

> +my $info = <$in>;

Is there an easy way to add a timeout to this read so that the failure 
mode isn't "the test hangs without printing anything"? I'm not sure that 
failure mode is easy to diagnose from our CI output as it is hard to 
tell which test caused the CI to timeout and it takes ages for the CI to 
time out.

Best Wishes

Phillip

> +is $info, "$file1hash blob 15\n", 'command_bidi_pipe w/ --batch-check';
> +$r->command_close_bidi_pipe($pid, $in, $out, $ctx);
> +
> +# ditto with `info' with --batch-command
> +($pid, $in, $out, $ctx) = $r->command_bidi_pipe(qw(cat-file --batch-command));
> +print $out 'info ', $file1hash, "\n" or die $!;
> +$info = <$in>;
> +is $info, "$file1hash blob 15\n", 'command_bidi_pipe w/ --batch-command=info';
> +$r->command_close_bidi_pipe($pid, $in, $out, $ctx);
> +
>   printf "1..%d\n", Test::More->builder->current_test;
>   
>   my $is_passing = eval { Test::More->is_passing };
> 

Re: [PATCH 2/2] t9700: ensure cat-file info isn't buffered by default

[Eric Wong]

Phillip Wood <phillip.wood123@gmail.com> wrote:
> Hi Eric
> 
> On 17/06/2024 11:43, Eric Wong wrote:
> > +# ensure --batch-check is unbuffered by default
> > +my ($pid, $in, $out, $ctx) = $r->command_bidi_pipe(qw(cat-file --batch-check));
> > +print $out $file1hash, "\n" or die $!;
> 
> It's been a while since I did any perl scripting and I'm not clear whether
> $out is buffered or not and if it is whether it is guaranteed to be flushed
> when we print "\n". It might be worth adding a explicit flush so it is clear
> that any deadlocks come from cat-file and not our test code.

Pipes and sockets created by Perl are always unbuffered since
5.8, at least.  If they were buffered, Git.pm users (including
git-svn) wouldn't have worked at all.

> > +my $info = <$in>;
> 
> Is there an easy way to add a timeout to this read so that the failure mode
> isn't "the test hangs without printing anything"? I'm not sure that failure
> mode is easy to diagnose from our CI output as it is hard to tell which test
> caused the CI to timeout and it takes ages for the CI to time out.

Yeah, select() has been added in v2.

Re: [PATCH 2/2] t9700: ensure cat-file info isn't buffered by default

[Phillip Wood]

On 19/06/2024 19:08, Eric Wong wrote:
> Phillip Wood <phillip.wood123@gmail.com> wrote:
>> Hi Eric
>>
>> On 17/06/2024 11:43, Eric Wong wrote:
>>> +# ensure --batch-check is unbuffered by default
>>> +my ($pid, $in, $out, $ctx) = $r->command_bidi_pipe(qw(cat-file --batch-check));
>>> +print $out $file1hash, "\n" or die $!;
>>
>> It's been a while since I did any perl scripting and I'm not clear whether
>> $out is buffered or not and if it is whether it is guaranteed to be flushed
>> when we print "\n". It might be worth adding a explicit flush so it is clear
>> that any deadlocks come from cat-file and not our test code.
> 
> Pipes and sockets created by Perl are always unbuffered since
> 5.8, at least.  If they were buffered, Git.pm users (including
> git-svn) wouldn't have worked at all.

Thanks for clarifying that

>>> +my $info = <$in>;
>>
>> Is there an easy way to add a timeout to this read so that the failure mode
>> isn't "the test hangs without printing anything"? I'm not sure that failure
>> mode is easy to diagnose from our CI output as it is hard to tell which test
>> caused the CI to timeout and it takes ages for the CI to time out.
> 
> Yeah, select() has been added in v2.

That's much nicer.

Thanks

Phillip