Re: Anyone know why git ls-remote output might be corrupted?

[Paul Smith <paul@mad-scientist.net>]

On Fri, 2023-06-02 at 18:12 -0700, Elijah Newren wrote:
> Sounds kind of like
> https://lore.kernel.org/git/6786526.72e2EbofS7@devpool47/ which also
> triggered for some other tooling and then was reduced down to some
> shell commands.  Unfortunately, the thread ended without a lot of
> resolution other than "don't mix stdout and stderr" and "if we slow
> down the network connection somehow, that'll avoid the problem".

As I was riding my bike home last week I had this exact same epiphany.
So, I examined my script and discovered that indeed, I was using
subprocess.PIPE for stderr, which is the Python equivalent of the
shell's 2>&1 operation.  So, both stdout and stderr are going to the
same place.

I also checked and indeed, the git ls-remote command does print to both
stdout and stderr as part of its "standard" behavior:

  $ git ls-remote --heads >/dev/null
  From git@git:myrepo

This is unexpected to me, although of course there's nothing inherently
wrong with it but usually you don't expect "regular" output to go to
stderr.  I suppose the idea is that people can run:

  $ git ls-remote --heads 2>/dev/null

if they want just the output without the header.

Anyway, I modified my scripting to keep stdout and stderr separate and
I haven't heard anything about things continuing to fail so I guess
that was the problem.

The mode of failure is very very bizarre to me: first why is it just
one "bogus" character embedded in the output?  Second why are 60 lines
just dropped?  Third where did the REST of the stderr output go?

And finally, why did this not fail 100% of the time on Linux?  I can
believe that Windows pipe behavior is not POSIX and maybe Python's
emulation of it is imperfect, but surely that is not the case on Linux
and we would ALWAYS get the full results of both stdout and stderr on
Linux, even if mixed together?!?!

The mysteries never end.

Thanks all; hopefully this helps someone else.