Re: data loss when doing ls-remote and piped to command

[Rolf Eike Beer <eb@emlix.com>]

[-- Attachment #1: Type: text/plain, Size: 2290 bytes --]

Am Mittwoch, 15. September 2021, 20:17:42 CEST schrieb Junio C Hamano:
> Rolf Eike Beer <eb@emlix.com> writes:
> > The given repository is a clone of the vanilla kernel.
> > 
> > /usr/bin/git --git-dir=/home/ebeer/repos/upstream/linux/.git ls-remote
> > origin 2>&1 | less
> > 
> > And I then see things like this:
> > 
> > 6f38b5d6cfd43dde3058a10c68baae9cf17af912        refs/tags/v5.0-rc2
> > 1c7fc5cbc33980acd13ae83d0b416db002fe95601e7f97f64b59514d936    
> > refs/tags/v5.7-rc2^{} d0709bb6da2ab6d49b11643e98abdf79b1a2817f       
> > refs/tags/v5.7-rc3
> Not offering any solution, just an observation of the problem and
> annotating the report.
> 
> What we see on the second line is the beginning of peeled
> v5.0-rc2^{} up to the "acd13" (that is, the first 19 bytes of the
> line), followed by the full line for peeled v5.7-rc2^{} (which
> begins with "ae83d").  12407 bytes in between are missing, which
> is even more puzzling as it is not a nice round number.
> 
> I wonder if this is "less" misconfigured and misbehaving.  Did the
> user after seeing v5.7-* tags scroll back with 'b' or something?

To quote myself:

>> This is not bound to less, we originally observed the error on a 
>> entirely different tool that tried to parse the output of ls-remote.

In fact when less opened I just started to scroll down until I visually 
noticed an error.

> If the output (including the 2>&1 redirection) is sent to a file and
> then "cat <that-file" is invoked, does the same thing happen?  How
> about "cat <that-file | less"?

The redirection seems to be an important part of it. I now did:

git ... 2>&1 | sha256sum

This gives different results basically on every run. I also noticed that 
having more tags makes it easier to reproduce, so a stable kernel in contrast 
to vanilla is a better trigger. Doing that without the stderr redirection gave 
the same result every time I tried.

Regards,

Eike
-- 
Rolf Eike Beer, emlix GmbH, http://www.emlix.com
Fon +49 551 30664-0, Fax +49 551 30664-11
Gothaer Platz 3, 37083 Göttingen, Germany
Sitz der Gesellschaft: Göttingen, Amtsgericht Göttingen HR B 3160
Geschäftsführung: Heike Jordan, Dr. Uwe Kracke – Ust-IdNr.: DE 205 198 055

emlix - smart embedded open source

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 313 bytes --]