[BUG] 2.46.0 t7701.09 fails on NonStop ia64

[BUG] 2.46.0 t7701.09 fails on NonStop ia64

[rsbecker]

Hi Team,

I think this is low priority but would like to understand the situation. It
only happens on NonStop ia64 (consistently), not x86.

The t7701.09 subtest fails missing files:

Total 3 (delta 0), reused 3 (delta 0), pack-reused 0 (from 0)
ls: cannot access '.git/objects/pack/pack-*.mtimes': No such file or
directory
test_line_count: line count for cruft.after != 2
not ok 9 - --max-cruft-size with pruning

The test directory contains:
./.git/objects/pack
./.git/objects/pack/pack-68c6c8c8538900694c32380ac1484201c8b60d8d.idx
./.git/objects/pack/pack-68c6c8c8538900694c32380ac1484201c8b60d8d.pack
./.git/objects/pack/pack-68c6c8c8538900694c32380ac1484201c8b60d8d.rev

but no mtimes files. I'm not sure how to proceed with this - currently
ignoring the failure but I
think it might be significant. This only fails on NonStop ia64 (which is
much older) and passes
on NonStop x86. There is no real file system reason why this would be a
difference between the
two platforms. There are some limit differences, but that's pretty much it.

Any assistance tracking this down would be appreciated. I can set up
breakpoints to catch
whatever is needed.

Regards,
Randall

Re: [BUG] 2.46.0 t7701.09 fails on NonStop ia64

[Taylor Blau]

On Thu, Aug 01, 2024 at 02:03:31PM -0400, rsbecker@nexbridge.com wrote:
> Hi Team,
>
> I think this is low priority but would like to understand the situation. It
> only happens on NonStop ia64 (consistently), not x86.
>
> The t7701.09 subtest fails missing files:

Hmm. Script t7701 only has 8 sub-tests, but I think you're referring to
t7704.9
>
> Total 3 (delta 0), reused 3 (delta 0), pack-reused 0 (from 0)
> ls: cannot access '.git/objects/pack/pack-*.mtimes': No such file or
> directory
> test_line_count: line count for cruft.after != 2
> not ok 9 - --max-cruft-size with pruning
>
> The test directory contains:
> ./.git/objects/pack
> ./.git/objects/pack/pack-68c6c8c8538900694c32380ac1484201c8b60d8d.idx
> ./.git/objects/pack/pack-68c6c8c8538900694c32380ac1484201c8b60d8d.pack
> ./.git/objects/pack/pack-68c6c8c8538900694c32380ac1484201c8b60d8d.rev

Interesting. Can you attach the full -vx output of this test, as well as
a ls -la from $GIT_DIR/objects/pack?

I suspect that this is a mtime resolution issue in whatever filesystem
is in use in your ia64 environment, but the full logs will help us
confirm that, or at least point is in a better direction.

Thanks in advance.

Thanks,
Taylor

RE: [BUG] 2.46.0 t7701.09 fails on NonStop ia64

[rsbecker]

On Thursday, August 1, 2024 3:23 PM, Taylor Blau wrote:
>On Thu, Aug 01, 2024 at 02:03:31PM -0400, rsbecker@nexbridge.com wrote:
>> Hi Team,
>>
>> I think this is low priority but would like to understand the
>> situation. It only happens on NonStop ia64 (consistently), not x86.
>>
>> The t7701.09 subtest fails missing files:
>
>Hmm. Script t7701 only has 8 sub-tests, but I think you're referring to
>t7704.9

Yes, t7704.9. Sorry.

>> Total 3 (delta 0), reused 3 (delta 0), pack-reused 0 (from 0)
>> ls: cannot access '.git/objects/pack/pack-*.mtimes': No such file or
>> directory
>> test_line_count: line count for cruft.after != 2 not ok 9 -
>> --max-cruft-size with pruning
>>
>> The test directory contains:
>> ./.git/objects/pack
>> ./.git/objects/pack/pack-68c6c8c8538900694c32380ac1484201c8b60d8d.idx
>> ./.git/objects/pack/pack-68c6c8c8538900694c32380ac1484201c8b60d8d.pack
>> ./.git/objects/pack/pack-68c6c8c8538900694c32380ac1484201c8b60d8d.rev
>
>Interesting. Can you attach the full -vx output of this test, as well as a ls -la from
>$GIT_DIR/objects/pack?
>
>I suspect that this is a mtime resolution issue in whatever filesystem is in use in your
>ia64 environment, but the full logs will help us confirm that, or at least point is in a
>better direction.

Here is the output as requested, with ls following. This is using OpenSSL 3.0 - OpenSSL 1.0.2 is further down with slightly different results.

++ git init max-cruft-size-prune
Initialized empty Git repository in /home/ituglib/randall/git/t/trash directory.t7704-repack-cruft/max-cruft-size-prune/.git/
++ cd max-cruft-size-prune
++ test_commit base
++ local notick=
++ local echo=echo
++ local append=
++ local author=
++ local signoff=
++ local indir=
++ local tag=light
++ test 1 '!=' 0
++ case "$1" in
++ break
++ indir=
++ local file=base.t
++ test -n ''
++ echo base
++ git add -- base.t
++ test -z ''
++ test_tick
++ test -z ''
++ test_tick=1112911993
++ GIT_COMMITTER_DATE='1112911993 -0700'
++ GIT_AUTHOR_DATE='1112911993 -0700'
++ export GIT_COMMITTER_DATE GIT_AUTHOR_DATE
++ git commit -m base
[master (root-commit) d1ff1c9] base
 Author: A U Thor <author@example.com>
 1 file changed, 1 insertion(+)
 create mode 100644 base.t
++ case "$tag" in
++ git tag base
+++ generate_random_blob foo 1048576
+++ test-tool genrandom foo 1048576
+++ git hash-object -w -t blob blob
+++ rm blob
++ foo=4f336f3df31054eabc05ac05f98bc024c8e05423
+++ generate_random_blob bar 1048576
+++ test-tool genrandom bar 1048576
+++ git hash-object -w -t blob blob
+++ rm blob
++ bar=bd44d8aa2d22eb47ff70cef4b0bb45d1549ee49c
+++ generate_random_blob baz 1048576
+++ test-tool genrandom baz 1048576
+++ git hash-object -w -t blob blob
+++ rm blob
++ baz=ad16bed54cc61cf036075879deeff95ae977514b
+++ test_oid_to_path 4f336f3df31054eabc05ac05f98bc024c8e05423
+++ local basename=336f3df31054eabc05ac05f98bc024c8e05423
+++ echo 4f/336f3df31054eabc05ac05f98bc024c8e05423
++ test-tool chmtime -10000 .git/objects/4f/336f3df31054eabc05ac05f98bc024c8e05423
++ git repack -d --cruft --max-cruft-size=1M
Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Writing objects: 100% (3/3), done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
Enumerating cruft objects: 6, done.
Counting objects: 100% (3/3), done.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
++ ls .git/objects/pack/pack-8f53f837597ebbdfc6f8ed173349e5cecbcfb97e.mtimes .git/objects/pack/pack-b8dc9aadaadc12c82b0053fdee0039ae1025a8f8.mtimes .git/objects/pack/pack-c2357b2b204fda52bc1f5515de94227e1db012af.mtimes
++ sort
+++ cat cruft.before
++ for cruft in $(cat cruft.before)
+++ test-tool chmtime --get -10000 .git/objects/pack/pack-8f53f837597ebbdfc6f8ed173349e5cecbcfb97e.mtimes
++ mtime=1722540725
++ echo .git/objects/pack/pack-8f53f837597ebbdfc6f8ed173349e5cecbcfb97e.mtimes 1722540725
++ for cruft in $(cat cruft.before)
+++ test-tool chmtime --get -10000 .git/objects/pack/pack-b8dc9aadaadc12c82b0053fdee0039ae1025a8f8.mtimes
++ mtime=1722540723
++ echo .git/objects/pack/pack-b8dc9aadaadc12c82b0053fdee0039ae1025a8f8.mtimes 1722540723
++ for cruft in $(cat cruft.before)
+++ test-tool chmtime --get -10000 .git/objects/pack/pack-c2357b2b204fda52bc1f5515de94227e1db012af.mtimes
++ mtime=1722540726
++ echo .git/objects/pack/pack-c2357b2b204fda52bc1f5515de94227e1db012af.mtimes 1722540726
++ git repack -d --cruft --max-cruft-size=1M --cruft-expiration=10.seconds.ago
Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Writing objects: 100% (3/3), done.
Total 3 (delta 0), reused 3 (delta 0), pack-reused 0 (from 0)
++ ls '.git/objects/pack/pack-*.mtimes'
++ sort
ls: cannot access '.git/objects/pack/pack-*.mtimes': No such file or directory
+++ cat cruft.after
++ test_line_count = 3 cruft.before
++ test 3 '!=' 3
+++ wc -l
++ test 3 = 3
++ test_line_count = 2 cruft.after
++ test 3 '!=' 3
+++ wc -l
++ test 0 = 2
++ echo 'test_line_count: line count for cruft.after != 2'
test_line_count: line count for cruft.after != 2
++ cat cruft.after
++ return 1
error: last command exited with $?=1
not ok 9 - --max-cruft-size with pruning

ls output with second resolution is here - the file system does not have nanosecond resolution despite showing zeros:

/home/ituglib/randall/git/t/trash directory.t7704-repack-cruft/max-cruft-size-prune/.git/objects/pack: ls -la --full-time
total 11
drwxrwxrwx 1 ITUGLIB.RANDALL ITUGLIB 4096 2024-08-01 16:18:55.000000000 -0600 .
drwxrwxrwx 1 ITUGLIB.RANDALL ITUGLIB 4096 2024-08-01 16:18:48.000000000 -0600 ..
-r--r--r-- 1 ITUGLIB.RANDALL ITUGLIB 1156 2024-08-01 16:18:52.000000000 -0600 pack-68c6c8c8538900694c32380ac1484201c8b60d8d.idx
-r--r--r-- 1 ITUGLIB.RANDALL ITUGLIB  217 2024-08-01 16:18:52.000000000 -0600 pack-68c6c8c8538900694c32380ac1484201c8b60d8d.pack
-r--r--r-- 1 ITUGLIB.RANDALL ITUGLIB   64 2024-08-01 16:18:52.000000000 -0600 pack-68c6c8c8538900694c32380ac1484201c8b60d8d.rev


When running with OpenSSL 1.0.2, the failure is different:
++ git init max-cruft-size-prune
Initialized empty Git repository in /home/ituglib/randall/jenkins/.jenkins/workspace/Git_Pipeline/t/trash directory.t7704-repack-cruft/max-cruft-size-prune/.git/
++ cd max-cruft-size-prune
++ test_commit base
++ local notick=
++ local echo=echo
++ local append=
++ local author=
++ local signoff=
++ local indir=
++ local tag=light
++ test 1 '!=' 0
++ case "$1" in
++ break
++ indir=
++ local file=base.t
++ test -n ''
++ echo base
++ git add -- base.t
++ test -z ''
++ test_tick
++ test -z ''
++ test_tick=1112911993
++ GIT_COMMITTER_DATE='1112911993 -0700'
++ GIT_AUTHOR_DATE='1112911993 -0700'
++ export GIT_COMMITTER_DATE GIT_AUTHOR_DATE
++ git commit -m base
[master (root-commit) d1ff1c9] base
 Author: A U Thor <author@example.com>
 1 file changed, 1 insertion(+)
 create mode 100644 base.t
++ case "$tag" in
++ git tag base
+++ generate_random_blob foo 1048576
+++ test-tool genrandom foo 1048576
+++ git hash-object -w -t blob blob
+++ rm blob
++ foo=4f336f3df31054eabc05ac05f98bc024c8e05423
+++ generate_random_blob bar 1048576
+++ test-tool genrandom bar 1048576
+++ git hash-object -w -t blob blob
+++ rm blob
++ bar=bd44d8aa2d22eb47ff70cef4b0bb45d1549ee49c
+++ generate_random_blob baz 1048576
+++ test-tool genrandom baz 1048576
+++ git hash-object -w -t blob blob
+++ rm blob
++ baz=ad16bed54cc61cf036075879deeff95ae977514b
+++ test_oid_to_path 4f336f3df31054eabc05ac05f98bc024c8e05423
+++ local basename=336f3df31054eabc05ac05f98bc024c8e05423
+++ echo 4f/336f3df31054eabc05ac05f98bc024c8e05423
++ test-tool chmtime -10000 .git/objects/4f/336f3df31054eabc05ac05f98bc024c8e05423
++ git repack -d --cruft --max-cruft-size=1M
Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Writing objects: 100% (3/3), done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
Enumerating cruft objects: 6, done.
Counting objects: 100% (3/3), done.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
++ ls .git/objects/pack/pack-8f53f837597ebbdfc6f8ed173349e5cecbcfb97e.mtimes .git/objects/pack/pack-b8dc9aadaadc12c82b0053fdee0039ae1025a8f8.mtimes .git/objects/pack/pack-c2357b2b204fda52bc1f5515de94227e1db012af.mtimes
++ sort
+++ cat cruft.before
++ for cruft in $(cat cruft.before)
+++ test-tool chmtime --get -10000 .git/objects/pack/pack-8f53f837597ebbdfc6f8ed173349e5cecbcfb97e.mtimes
++ mtime=1722531262
++ echo .git/objects/pack/pack-8f53f837597ebbdfc6f8ed173349e5cecbcfb97e.mtimes 1722531262
++ for cruft in $(cat cruft.before)
+++ test-tool chmtime --get -10000 .git/objects/pack/pack-b8dc9aadaadc12c82b0053fdee0039ae1025a8f8.mtimes
++ mtime=1722531261
++ echo .git/objects/pack/pack-b8dc9aadaadc12c82b0053fdee0039ae1025a8f8.mtimes 1722531261
++ for cruft in $(cat cruft.before)
+++ test-tool chmtime --get -10000 .git/objects/pack/pack-c2357b2b204fda52bc1f5515de94227e1db012af.mtimes
++ mtime=1722531263
++ echo .git/objects/pack/pack-c2357b2b204fda52bc1f5515de94227e1db012af.mtimes 1722531263
++ git repack -d --cruft --max-cruft-size=1M --cruft-expiration=10.seconds.ago
Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Writing objects: 100% (3/3), done.
Total 3 (delta 0), reused 3 (delta 0), pack-reused 0 (from 0)
Enumerating cruft objects: 1, done.
Traversing cruft objects: 1, done.
Counting objects: 100% (1/1), done.
Writing objects: 100% (1/1), done.
Total 1 (delta 0), reused 1 (delta 0), pack-reused 0 (from 0)
++ ls .git/objects/pack/pack-8f53f837597ebbdfc6f8ed173349e5cecbcfb97e.mtimes
++ sort
+++ cat cruft.after
++ for cruft in $(cat cruft.after)
+++ grep .git/objects/pack/pack-8f53f837597ebbdfc6f8ed173349e5cecbcfb97e.mtimes mtimes
+++ cut '-d ' -f2
++ old_mtime=1722531262
+++ test-tool chmtime --get .git/objects/pack/pack-8f53f837597ebbdfc6f8ed173349e5cecbcfb97e.mtimes
++ new_mtime=1722541267
++ test 1722531262 -lt 1722541267
++ test_line_count = 3 cruft.before
++ test 3 '!=' 3
+++ wc -l
++ test 3 = 3
++ test_line_count = 2 cruft.after
++ test 3 '!=' 3
+++ wc -l
++ test 1 = 2
++ echo 'test_line_count: line count for cruft.after != 2'
test_line_count: line count for cruft.after != 2
++ cat cruft.after
.git/objects/pack/pack-8f53f837597ebbdfc6f8ed173349e5cecbcfb97e.mtimes
++ return 1
error: last command exited with $?=1
not ok 9 - --max-cruft-size with pruning

Re: [BUG] 2.46.0 t7701.09 fails on NonStop ia64

[Taylor Blau]

On Thu, Aug 01, 2024 at 06:25:51PM -0400, rsbecker@nexbridge.com wrote:
> ls output with second resolution is here - the file system does not have nanosecond resolution despite showing zeros:
>
> /home/ituglib/randall/git/t/trash directory.t7704-repack-cruft/max-cruft-size-prune/.git/objects/pack: ls -la --full-time
> total 11
> drwxrwxrwx 1 ITUGLIB.RANDALL ITUGLIB 4096 2024-08-01 16:18:55.000000000 -0600 .
> drwxrwxrwx 1 ITUGLIB.RANDALL ITUGLIB 4096 2024-08-01 16:18:48.000000000 -0600 ..
> -r--r--r-- 1 ITUGLIB.RANDALL ITUGLIB 1156 2024-08-01 16:18:52.000000000 -0600 pack-68c6c8c8538900694c32380ac1484201c8b60d8d.idx
> -r--r--r-- 1 ITUGLIB.RANDALL ITUGLIB  217 2024-08-01 16:18:52.000000000 -0600 pack-68c6c8c8538900694c32380ac1484201c8b60d8d.pack
> -r--r--r-- 1 ITUGLIB.RANDALL ITUGLIB   64 2024-08-01 16:18:52.000000000 -0600 pack-68c6c8c8538900694c32380ac1484201c8b60d8d.rev

Ah, I suspect that this is even less interesting than imprecise mtime
resolution. The test expects that the packs are larger than 1M so that
we can exercise writing multiple cruft packs as part of the setup.

But that pack is only 217 bytes, which wouldn't trigger the split. I'm
suspicious that it's even packing the cruft objects at all, so I'm
curious if you can find $foo, $bar, and $baz in the cruft pack's .idx
file(s, if multiple) after the first 'git repack -d --cruft'.

Assuming they are there, it's possible that setting repack.cruftWindow
in the test repository would do the trick.

But I'm suspicious that that's even what's going on since
generate_random_blob()'s first argument is a seed, and all three objects
have different seeds, so I don't think they would even be considered
good delta candidates for one another. But it's also possible that your
generate_random_blob() behaves differently from my own.

Thanks,
Taylor

RE: [BUG] 2.46.0 t7701.09 fails on NonStop ia64

[rsbecker]

On Thursday, August 1, 2024 8:59 PM, Taylor Blau wrote:
>On Thu, Aug 01, 2024 at 06:25:51PM -0400, rsbecker@nexbridge.com wrote:
>> ls output with second resolution is here - the file system does not have
>nanosecond resolution despite showing zeros:
>>
>> /home/ituglib/randall/git/t/trash
>> directory.t7704-repack-cruft/max-cruft-size-prune/.git/objects/pack:
>> ls -la --full-time total 11 drwxrwxrwx 1 ITUGLIB.RANDALL ITUGLIB 4096 2024-08-
>01 16:18:55.000000000 -0600 .
>> drwxrwxrwx 1 ITUGLIB.RANDALL ITUGLIB 4096 2024-08-01
>16:18:48.000000000 -0600 ..
>> -r--r--r-- 1 ITUGLIB.RANDALL ITUGLIB 1156 2024-08-01
>> 16:18:52.000000000 -0600
>> pack-68c6c8c8538900694c32380ac1484201c8b60d8d.idx
>> -r--r--r-- 1 ITUGLIB.RANDALL ITUGLIB  217 2024-08-01 16:18:52.000000000 -
>0600 pack-68c6c8c8538900694c32380ac1484201c8b60d8d.pack
>> -r--r--r-- 1 ITUGLIB.RANDALL ITUGLIB   64 2024-08-01 16:18:52.000000000 -
>0600 pack-68c6c8c8538900694c32380ac1484201c8b60d8d.rev
>
>Ah, I suspect that this is even less interesting than imprecise mtime resolution. The
>test expects that the packs are larger than 1M so that we can exercise writing
>multiple cruft packs as part of the setup.
>
>But that pack is only 217 bytes, which wouldn't trigger the split. I'm suspicious that
>it's even packing the cruft objects at all, so I'm curious if you can find $foo, $bar,
>and $baz in the cruft pack's .idx file(s, if multiple) after the first 'git repack -d --cruft'.
>
>Assuming they are there, it's possible that setting repack.cruftWindow in the test
>repository would do the trick.
>
>But I'm suspicious that that's even what's going on since generate_random_blob()'s
>first argument is a seed, and all three objects have different seeds, so I don't think
>they would even be considered good delta candidates for one another. But it's also
>possible that your
>generate_random_blob() behaves differently from my own.

I will try to look tomorrow for $foo, $bar, and $baz. On generate_random_blob(), from OpenSSL's standpoint, on ia64, we use PRNGD. It should be of the same order of magnitude as anywhere else, but depends on the randomness measure - we are able to start it, so randomness is proper. On my x86 machine, I use the hardware randomizer, which is better (and FIPS compatible) and might be why we don't see it there.

RE: [BUG] 2.46.0 t7701.09 fails on NonStop ia64

[rsbecker]

On Thursday, August 1, 2024 8:59 PM, Taylor Blau wrote:
>On Thu, Aug 01, 2024 at 06:25:51PM -0400, rsbecker@nexbridge.com wrote:
>> ls output with second resolution is here - the file system does not have
>nanosecond resolution despite showing zeros:
>>
>> /home/ituglib/randall/git/t/trash
>> directory.t7704-repack-cruft/max-cruft-size-prune/.git/objects/pack:
>> ls -la --full-time total 11 drwxrwxrwx 1 ITUGLIB.RANDALL ITUGLIB 4096 2024-08-
>01 16:18:55.000000000 -0600 .
>> drwxrwxrwx 1 ITUGLIB.RANDALL ITUGLIB 4096 2024-08-01
>16:18:48.000000000 -0600 ..
>> -r--r--r-- 1 ITUGLIB.RANDALL ITUGLIB 1156 2024-08-01
>> 16:18:52.000000000 -0600
>> pack-68c6c8c8538900694c32380ac1484201c8b60d8d.idx
>> -r--r--r-- 1 ITUGLIB.RANDALL ITUGLIB  217 2024-08-01 16:18:52.000000000 -
>0600 pack-68c6c8c8538900694c32380ac1484201c8b60d8d.pack
>> -r--r--r-- 1 ITUGLIB.RANDALL ITUGLIB   64 2024-08-01 16:18:52.000000000 -
>0600 pack-68c6c8c8538900694c32380ac1484201c8b60d8d.rev
>
>Ah, I suspect that this is even less interesting than imprecise mtime resolution. The
>test expects that the packs are larger than 1M so that we can exercise writing
>multiple cruft packs as part of the setup.
>
>But that pack is only 217 bytes, which wouldn't trigger the split. I'm suspicious that
>it's even packing the cruft objects at all, so I'm curious if you can find $foo, $bar,
>and $baz in the cruft pack's .idx file(s, if multiple) after the first 'git repack -d --cruft'.
>
>Assuming they are there, it's possible that setting repack.cruftWindow in the test
>repository would do the trick.
>
>But I'm suspicious that that's even what's going on since generate_random_blob()'s
>first argument is a seed, and all three objects have different seeds, so I don't think
>they would even be considered good delta candidates for one another. But it's also
>possible that your
>generate_random_blob() behaves differently from my own.

The entire final .idx file is:
0000000 377   t   O   c  \0  \0  \0 002  \0  \0  \0  \0  \0  \0  \0  \0
0000020  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
*
0001500  \0  \0  \0 001  \0  \0  \0 001  \0  \0  \0 001  \0  \0  \0 002
0001520  \0  \0  \0 002  \0  \0  \0 002  \0  \0  \0 002  \0  \0  \0 002
*
0001600  \0  \0  \0 002  \0  \0  \0 003  \0  \0  \0 003  \0  \0  \0 003
0001620  \0  \0  \0 003  \0  \0  \0 003  \0  \0  \0 003  \0  \0  \0 003
*
0002000  \0  \0  \0 003  \0  \0  \0 003 316   a   1 002   ,   :   H 241
0002020   _   K   z   h 257 266 354 363 263 274 260 315 321 377 034 222
0002040   $ 256   ^   X 247   e   o 271 354 311   X   e 324   . 327 036
0002060 337 226   { 226 245   y 344   Z 030 270   % 027   2 321   h 004
0002100 262 345   j   U 003 302 375 225   d 335   = 272 207 227   ) 265
0002120  \0  \0  \0 212  \0  \0  \0  \f  \0  \0  \0 267   h 306 310 310
0002140   S 211  \0   i   L   2   8  \n 301   H   B 001 310 266  \r 215
0002160 273   y 312 253 265   V 225 350 246 212   h   Z   d   ? 313   f
0002200 372   Z 336   3
0002204

After the first repack, I have the following idx files. No foo/bar/baz inside.
The generate_random_blob() does generate the proper amount of bytes.
I tried changing 0xff to 0x00ff at the putchar just in case we had bad
sign extension - that wasn't it.

pack-68c6c8c8538900694c32380ac1484201c8b60d8d.idx:
0000000 377   t   O   c  \0  \0  \0 002  \0  \0  \0  \0  \0  \0  \0  \0
0000020  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
*
0001500  \0  \0  \0 001  \0  \0  \0 001  \0  \0  \0 001  \0  \0  \0 002
0001520  \0  \0  \0 002  \0  \0  \0 002  \0  \0  \0 002  \0  \0  \0 002
*
0001600  \0  \0  \0 002  \0  \0  \0 003  \0  \0  \0 003  \0  \0  \0 003
0001620  \0  \0  \0 003  \0  \0  \0 003  \0  \0  \0 003  \0  \0  \0 003
*
0002000  \0  \0  \0 003  \0  \0  \0 003 316   a   1 002   ,   :   H 241
0002020   _   K   z   h 257 266 354 363 263 274 260 315 321 377 034 222
0002040   $ 256   ^   X 247   e   o 271 354 311   X   e 324   . 327 036
0002060 337 226   { 226 245   y 344   Z 030 270   % 027   2 321   h 004
0002100 262 345   j   U 003 302 375 225   d 335   = 272 207 227   ) 265
0002120  \0  \0  \0 212  \0  \0  \0  \f  \0  \0  \0 267   h 306 310 310
0002140   S 211  \0   i   L   2   8  \n 301   H   B 001 310 266  \r 215
0002160 273   y 312 253 265   V 225 350 246 212   h   Z   d   ? 313   f
0002200 372   Z 336   3
0002204

pack-8f53f837597ebbdfc6f8ed173349e5cecbcfb97e.idx:
0000000 377   t   O   c  \0  \0  \0 002  \0  \0  \0  \0  \0  \0  \0  \0
0000020  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
*
0001260  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0 001
0001300  \0  \0  \0 001  \0  \0  \0 001  \0  \0  \0 001  \0  \0  \0 001
*
0002000  \0  \0  \0 001  \0  \0  \0 001 255 026 276 325   L 306 034 360
0002020   6  \a   X   y 336 357 371   Z 351   w   Q   K   # 236 346 377
0002040  \0  \0  \0  \f 217   S 370   7   Y   ~ 273 337 306 370 355 027
0002060   3   I 345 316 313 317 271   ~   9 033   9 316 363 306   G   -
0002100 207   f 224 323   1 372 021   W 037 335 206 372
0002114

pack-b8dc9aadaadc12c82b0053fdee0039ae1025a8f8.idx:
0000000 377   t   O   c  \0  \0  \0 002  \0  \0  \0  \0  \0  \0  \0  \0
0000020  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
*
0000500  \0  \0  \0  \0  \0  \0  \0 001  \0  \0  \0 001  \0  \0  \0 001
0000520  \0  \0  \0 001  \0  \0  \0 001  \0  \0  \0 001  \0  \0  \0 001
*
0002000  \0  \0  \0 001  \0  \0  \0 001   O   3   o   = 363 020   T 352
0002020 274 005 254 005 371 213 300   $ 310 340   T   #   $ 274   l 342
0002040  \0  \0  \0  \f 270 334 232 255 252 334 022 310   +  \0   S 375
0002060 356  \0   9 256 020   % 250 370 204 242   7 033   =   0 375 207
0002100 343 352 257 374 265 021   j 214   5 026 355 243
0002114

pack-c2357b2b204fda52bc1f5515de94227e1db012af.idx:
0000000 377   t   O   c  \0  \0  \0 002  \0  \0  \0  \0  \0  \0  \0  \0
0000020  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
*
0001360  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0 001
0001400  \0  \0  \0 001  \0  \0  \0 001  \0  \0  \0 001  \0  \0  \0 001
*
0002000  \0  \0  \0 001  \0  \0  \0 001 275   D 330 252   -   " 353   G
0002020 377   p 316 364 260 273   E 321   T 236 344 234 302 311   8   h
0002040  \0  \0  \0  \f 302   5   {   +       O 332   R 274 037   U 025
0002060 336 224   "   ~ 035 260 022 257 373   U   )   - 204   " 315   v
0002100   /   E 272   e   7 342 261 037   6 255   < 306
0002114

--Randall

Re: [BUG] 2.46.0 t7701.09 fails on NonStop ia64

[Taylor Blau]

On Fri, Aug 02, 2024 at 06:07:55PM -0400, rsbecker@nexbridge.com wrote:
> After the first repack, I have the following idx files. No foo/bar/baz inside.
> The generate_random_blob() does generate the proper amount of bytes.
> I tried changing 0xff to 0x00ff at the putchar just in case we had bad
> sign extension - that wasn't it.

Hmm. What is in these index files if not the three randomly generated
objects? Can you run show-index instead and share the results of that on
the pack(s) in your repository after the first repack, and share the
results with the list?

I think those would be more readable than a hexdump, especially if your
machine random source is different than mine (in which case I can't
guess the values of $foo, $bar, and $baz).

Thanks,
Taylor

RE: [BUG] 2.46.0 t7701.09 fails on NonStop ia64

[rsbecker]

>-----Original Message-----
>From: Taylor Blau <me@ttaylorr.com>
>Sent: Sunday, August 4, 2024 12:03 PM
>To: rsbecker@nexbridge.com
>Cc: git@vger.kernel.org
>Subject: Re: [BUG] 2.46.0 t7701.09 fails on NonStop ia64
>
>On Fri, Aug 02, 2024 at 06:07:55PM -0400, rsbecker@nexbridge.com wrote:
>> After the first repack, I have the following idx files. No foo/bar/baz inside.
>> The generate_random_blob() does generate the proper amount of bytes.
>> I tried changing 0xff to 0x00ff at the putchar just in case we had bad
>> sign extension - that wasn't it.
>
>Hmm. What is in these index files if not the three randomly generated objects? Can
>you run show-index instead and share the results of that on the pack(s) in your
>repository after the first repack, and share the results with the list?
>
>I think those would be more readable than a hexdump, especially if your machine
>random source is different than mine (in which case I can't guess the values of $foo,
>$bar, and $baz).

After git repack -d --cruft --max-cruft-size=1M &&...

./trash directory.t7704-repack-cruft/max-cruft-size-prune: git show-index < .git/objects/pack/pack-68c6c8c8538900694c32380ac1484201c8b60d8d.pack
fatal: unable to read index
./trash directory.t7704-repack-cruft/max-cruft-size-prune: git show-index < .git/objects/pack/pack-8f53f837597ebbdfc6f8ed173349e5cecbcfb97e.pack
fatal: corrupt index file
./trash directory.t7704-repack-cruft/max-cruft-size-prune: git show-index < .git/objects/pack/pack-b8dc9aadaadc12c82b0053fdee0039ae1025a8f8.pack
fatal: corrupt index file
./trash directory.t7704-repack-cruft/max-cruft-size-prune: git show-index < .git/objects/pack/pack-c2357b2b204fda52bc1f5515de94227e1db012af.pack
fatal: corrupt index file

Apparently, something is wrong. This is an ia64 Big Endian. Not sure that matters.

Re: [BUG] 2.46.0 t7701.09 fails on NonStop ia64

[Taylor Blau]

On Sun, Aug 04, 2024 at 12:15:41PM -0400, rsbecker@nexbridge.com wrote:
> After git repack -d --cruft --max-cruft-size=1M &&...
>
> ./trash directory.t7704-repack-cruft/max-cruft-size-prune: git show-index < .git/objects/pack/pack-68c6c8c8538900694c32380ac1484201c8b60d8d.pack
> fatal: unable to read index
> ./trash directory.t7704-repack-cruft/max-cruft-size-prune: git show-index < .git/objects/pack/pack-8f53f837597ebbdfc6f8ed173349e5cecbcfb97e.pack
> fatal: corrupt index file
> ./trash directory.t7704-repack-cruft/max-cruft-size-prune: git show-index < .git/objects/pack/pack-b8dc9aadaadc12c82b0053fdee0039ae1025a8f8.pack
> fatal: corrupt index file
> ./trash directory.t7704-repack-cruft/max-cruft-size-prune: git show-index < .git/objects/pack/pack-c2357b2b204fda52bc1f5515de94227e1db012af.pack
> fatal: corrupt index file
>
> Apparently, something is wrong. This is an ia64 Big Endian. Not sure that matters.

show-index expects the contents of the *.idx file on stdin, not the *.pack.

Thanks,
Taylor

RE: [BUG] 2.46.0 t7701.09 fails on NonStop ia64

[rsbecker]

On Sunday, August 4, 2024 12:20 PM, Taylor Blau wrote:
>On Sun, Aug 04, 2024 at 12:15:41PM -0400, rsbecker@nexbridge.com wrote:
>> After git repack -d --cruft --max-cruft-size=1M &&...
>>
>> ./trash directory.t7704-repack-cruft/max-cruft-size-prune: git show-index <
>.git/objects/pack/pack-68c6c8c8538900694c32380ac1484201c8b60d8d.pack
>> fatal: unable to read index
>> ./trash directory.t7704-repack-cruft/max-cruft-size-prune: git show-index <
>.git/objects/pack/pack-8f53f837597ebbdfc6f8ed173349e5cecbcfb97e.pack
>> fatal: corrupt index file
>> ./trash directory.t7704-repack-cruft/max-cruft-size-prune: git show-index <
>.git/objects/pack/pack-b8dc9aadaadc12c82b0053fdee0039ae1025a8f8.pack
>> fatal: corrupt index file
>> ./trash directory.t7704-repack-cruft/max-cruft-size-prune: git show-index <
>.git/objects/pack/pack-c2357b2b204fda52bc1f5515de94227e1db012af.pack
>> fatal: corrupt index file
>>
>> Apparently, something is wrong. This is an ia64 Big Endian. Not sure that matters.
>
>show-index expects the contents of the *.idx file on stdin, not the *.pack.

This is what I get for being 1/2 asleep on Sunday.

./trash directory.t7704-repack-cruft/max-cruft-size-prune: git show-index < .git/objects/pack/pack-68c6c8c8538900694c32380ac1484201c8b60d8d.idx
138 ce6131022c3a48a15f4b7a68afb6ecf3b3bcb0cd (03c2fd95)
12 d1ff1c9224ae5e58a7656fb9ecc95865d42ed71e (64dd3dba)
183 df967b96a579e45a18b8251732d16804b2e56a55 (879729b5)
./trash directory.t7704-repack-cruft/max-cruft-size-prune: git show-index <.git/objects/pack/pack-8f53f837597ebbdfc6f8ed173349e5cecbcfb97e.idx
12 ad16bed54cc61cf036075879deeff95ae977514b (239ee6ff)
./trash directory.t7704-repack-cruft/max-cruft-size-prune: git show-index <.git/objects/pack/pack-b8dc9aadaadc12c82b0053fdee0039ae1025a8f8.idx
12 4f336f3df31054eabc05ac05f98bc024c8e05423 (24bc6ce2)
./trash directory.t7704-repack-cruft/max-cruft-size-prune: git show-index <.git/objects/pack/pack-c2357b2b204fda52bc1f5515de94227e1db012af.idx
12 bd44d8aa2d22eb47ff70cef4b0bb45d1549ee49c (c2c93868)

Re: [BUG] 2.46.0 t7701.09 fails on NonStop ia64

[Taylor Blau]

On Sun, Aug 04, 2024 at 12:38:38PM -0400, rsbecker@nexbridge.com wrote:
> ./trash directory.t7704-repack-cruft/max-cruft-size-prune: git show-index < .git/objects/pack/pack-68c6c8c8538900694c32380ac1484201c8b60d8d.idx
> 138 ce6131022c3a48a15f4b7a68afb6ecf3b3bcb0cd (03c2fd95)
> 12 d1ff1c9224ae5e58a7656fb9ecc95865d42ed71e (64dd3dba)
> 183 df967b96a579e45a18b8251732d16804b2e56a55 (879729b5)
> ./trash directory.t7704-repack-cruft/max-cruft-size-prune: git show-index <.git/objects/pack/pack-8f53f837597ebbdfc6f8ed173349e5cecbcfb97e.idx
> 12 ad16bed54cc61cf036075879deeff95ae977514b (239ee6ff)
> ./trash directory.t7704-repack-cruft/max-cruft-size-prune: git show-index <.git/objects/pack/pack-b8dc9aadaadc12c82b0053fdee0039ae1025a8f8.idx
> 12 4f336f3df31054eabc05ac05f98bc024c8e05423 (24bc6ce2)
> ./trash directory.t7704-repack-cruft/max-cruft-size-prune: git show-index <.git/objects/pack/pack-c2357b2b204fda52bc1f5515de94227e1db012af.idx
> 12 bd44d8aa2d22eb47ff70cef4b0bb45d1549ee49c (c2c93868)

OK, so it looks like the first repack is doing what we expect:
presumably the first pack containing three objects holds the result of
"test_commit base" (and contains the blob, tree, and commit objects for
commit "base"). Then I'm assuming that the remaining three packs hold
$foo, $bar, and $baz in some unspecified order.

So I think that everything up through the first repack is doing the
right thing.

(As an aside, it took me a while to remember what was going on with this
test. Since it was confusing to me as its author, I figured it may be
helpful to spell out some of the details. We're just making sure that
pruning a cruft object works with multiple cruft packs, and we're
adjusting the mtimes of the *.mtimes files themselves to ensure that
they are rewritten during the pruning repack. Those adjustments have no
bearing on what actually gets pruned, unlike when we update the mtime of
the loose copy of $foo, which does affect pruning).

Anyway... in your setup it looks like all three objects are gone, and we
expected only $foo to be pruned. So I suspect what's going on is that
this test takes longer than 10 seconds to run on your machine, in which
case it would fail as all objects would have passed the 10-second grace
period.

If you apply:

--- 8< ---
diff --git a/t/t7704-repack-cruft.sh b/t/t7704-repack-cruft.sh
index 71e1ef3a10..959e6e2648 100755
--- a/t/t7704-repack-cruft.sh
+++ b/t/t7704-repack-cruft.sh
@@ -330,7 +330,7 @@ test_expect_success '--max-cruft-size with pruning' '
 		# repack (and prune) with a --max-cruft-size to ensure
 		# that we appropriately split the resulting set of packs
 		git repack -d --cruft --max-cruft-size=1M \
-			--cruft-expiration=10.seconds.ago &&
+			--cruft-expiration=1000.seconds.ago &&
 		ls $packdir/pack-*.mtimes | sort >cruft.after &&

 		for cruft in $(cat cruft.after)
--- >8 ---

, does it still fail?

(There's really no reason to have --cruft-expiration as short as 10
seconds here, since we backdate the cruft mtimes by 10,000 seconds.)

Thanks,
Taylor

RE: [BUG] 2.46.0 t7701.09 fails on NonStop ia64

[rsbecker]

On Sunday, August 4, 2024 2:13 PM, Taylor Blau wrote:
>On Sun, Aug 04, 2024 at 12:38:38PM -0400, rsbecker@nexbridge.com wrote:
>> ./trash directory.t7704-repack-cruft/max-cruft-size-prune: git
>> show-index <
>> .git/objects/pack/pack-68c6c8c8538900694c32380ac1484201c8b60d8d.idx
>> 138 ce6131022c3a48a15f4b7a68afb6ecf3b3bcb0cd (03c2fd95)
>> 12 d1ff1c9224ae5e58a7656fb9ecc95865d42ed71e (64dd3dba)
>> 183 df967b96a579e45a18b8251732d16804b2e56a55 (879729b5) ./trash
>> directory.t7704-repack-cruft/max-cruft-size-prune: git show-index
>> <.git/objects/pack/pack-8f53f837597ebbdfc6f8ed173349e5cecbcfb97e.idx
>> 12 ad16bed54cc61cf036075879deeff95ae977514b (239ee6ff) ./trash
>> directory.t7704-repack-cruft/max-cruft-size-prune: git show-index
>> <.git/objects/pack/pack-b8dc9aadaadc12c82b0053fdee0039ae1025a8f8.idx
>> 12 4f336f3df31054eabc05ac05f98bc024c8e05423 (24bc6ce2) ./trash
>> directory.t7704-repack-cruft/max-cruft-size-prune: git show-index
>> <.git/objects/pack/pack-c2357b2b204fda52bc1f5515de94227e1db012af.idx
>> 12 bd44d8aa2d22eb47ff70cef4b0bb45d1549ee49c (c2c93868)
>
>OK, so it looks like the first repack is doing what we expect:
>presumably the first pack containing three objects holds the result of "test_commit
>base" (and contains the blob, tree, and commit objects for commit "base"). Then
>I'm assuming that the remaining three packs hold $foo, $bar, and $baz in some
>unspecified order.
>
>So I think that everything up through the first repack is doing the right thing.
>
>(As an aside, it took me a while to remember what was going on with this test. Since
>it was confusing to me as its author, I figured it may be helpful to spell out some of
>the details. We're just making sure that pruning a cruft object works with multiple
>cruft packs, and we're adjusting the mtimes of the *.mtimes files themselves to
>ensure that they are rewritten during the pruning repack. Those adjustments have
>no bearing on what actually gets pruned, unlike when we update the mtime of the
>loose copy of $foo, which does affect pruning).
>
>Anyway... in your setup it looks like all three objects are gone, and we expected only
>$foo to be pruned. So I suspect what's going on is that this test takes longer than 10
>seconds to run on your machine, in which case it would fail as all objects would
>have passed the 10-second grace period.
>
>If you apply:
>
>--- 8< ---
>diff --git a/t/t7704-repack-cruft.sh b/t/t7704-repack-cruft.sh index
>71e1ef3a10..959e6e2648 100755
>--- a/t/t7704-repack-cruft.sh
>+++ b/t/t7704-repack-cruft.sh
>@@ -330,7 +330,7 @@ test_expect_success '--max-cruft-size with pruning' '
> 		# repack (and prune) with a --max-cruft-size to ensure
> 		# that we appropriately split the resulting set of packs
> 		git repack -d --cruft --max-cruft-size=1M \
>-			--cruft-expiration=10.seconds.ago &&
>+			--cruft-expiration=1000.seconds.ago &&
> 		ls $packdir/pack-*.mtimes | sort >cruft.after &&
>
> 		for cruft in $(cat cruft.after)
>--- >8 ---
>
>, does it still fail?
>
>(There's really no reason to have --cruft-expiration as short as 10 seconds here,
>since we backdate the cruft mtimes by 10,000 seconds.)

This worked. Even taking it down to 100 seconds also works. I suspect that the reason this previously passed was that we have an unrelated heavy use looping process on the box right now so things are taking longer. Running verbose also takes longer.