2.2.0-rc behavior changes (2/2)

2.2.0-rc behavior changes (2/2)

[Bryan Turner]

I've been running a test suite we use to verify Git behaviors across
versions, and the 2.2.0 RCs (0 and 1 both) have a couple of small
behavioral differences. I'm sending them in separate e-mails just to
make the contents easier to grok.

Important: It's entirely possible neither of these is a _bug_; they
may both be intentional changes in behavior.

Second change: git gc --auto now fails if there are loose empty blobs.

We have a test which just touched empty files in objects/17 to trigger
the gc --auto preconditions (Note: I realize this is completely
invalid, and I've changed the test to no longer do this; I'm only
surfacing the behavioral change).

On Xubuntu 14.10 I can reproduce this using bash with the following steps:
git init gc
cd gc
echo Hello, world > file.txt
git add file.txt
git commit -m "Initial commit"
mkdir .git/objects/17
git config gc.auto 2
git config --bool gc.autodetach false
for i in $(seq 1 20); do touch .git/objects/17/$(head -n 4096
/dev/urandom | openssl sha1 | cut -c 10-47); done

(openssl sha1 on my machine prefixes the SHA-1s with "(stdin)= ", so
the cut is both to shorten the SHA-1 and to drop that prefix)

In 2.1.x and prior, git gc --auto appears to ignore those objects and
exit 0, although it does note that there are still too many loose
objects:
bturner@felurian:~/tmp/gc$ git version
git version 2.1.0
bturner@felurian:~/tmp/gc$ git gc --auto
Auto packing the repository for optimum performance.
See "git help gc" for manual housekeeping.
Nothing new to pack.
warning: There are too many unreachable loose objects; run 'git prune'
to remove them.

In the 2.2.0 RCs git gc --auto exits with 255 and the following errors:
bturner@felurian:~/tmp/gc$ /opt/git-2.2.0-rc1/bin/git gc --auto
Auto packing the repository for optimum performance.
See "git help gc" for manual housekeeping.
Nothing new to pack.
error: object file
.git/objects/17/02d54e8fba95ef9968a0c9b183fe22ec551c86 is empty
fatal: unable to get object info for 1702d54e8fba95ef9968a0c9b183fe22ec551c86
error: failed to run prune

Making git gc more sensitive to invalid objects may be a good thing. I
only point out this behavior change because the change it bisects to
doesn't really cite this as an intentional change.

This change bisects to:

bturner@felurian:~/Development/git/git$ git bisect good
d3038d22f91aad9620bd8e6fc43fc67c16219738 is the first bad commit
commit d3038d22f91aad9620bd8e6fc43fc67c16219738
Author: Jeff King <peff@peff.net>
Date:   Wed Oct 15 18:41:35 2014 -0400

    prune: keep objects reachable from recent objects

Best regards,
Bryan Turner

Re: 2.2.0-rc behavior changes (2/2)

[Jeff King]

On Mon, Nov 10, 2014 at 07:51:00PM +1100, Bryan Turner wrote:

> Second change: git gc --auto now fails if there are loose empty blobs.
> 
> We have a test which just touched empty files in objects/17 to trigger
> the gc --auto preconditions (Note: I realize this is completely
> invalid, and I've changed the test to no longer do this; I'm only
> surfacing the behavioral change).

This is expected. As you noticed here:

> error: object file
> .git/objects/17/02d54e8fba95ef9968a0c9b183fe22ec551c86 is empty
> fatal: unable to get object info for 1702d54e8fba95ef9968a0c9b183fe22ec551c86
> error: failed to run prune

the error comes from "git prune" failing. It is using unreachable but
"recent" objects as graph tips to keep around. If we can't load a tip
object, we abort the prune, because we would not want to delete objects
that might have been referenced (e.g., if it were a real corrupted
object that we could not read).

The second behavior (die on broken objects) has been around for a while.
The new change (in the commit you bisected to) is that we are
considering more objects.

I'll admit I didn't really consider the impact on sticking broken object
files into the object directory, but I think the code is doing the
sensible and safe thing.

-Peff

Re: 2.2.0-rc behavior changes (2/2)

[Bryan Turner]

On Mon, Nov 10, 2014 at 8:31 PM, Jeff King <peff@peff.net> wrote:
> On Mon, Nov 10, 2014 at 07:51:00PM +1100, Bryan Turner wrote:
>
>> Second change: git gc --auto now fails if there are loose empty blobs.
>>
>> We have a test which just touched empty files in objects/17 to trigger
>> the gc --auto preconditions (Note: I realize this is completely
>> invalid, and I've changed the test to no longer do this; I'm only
>> surfacing the behavioral change).
>
> This is expected. As you noticed here:
>
>> error: object file
>> .git/objects/17/02d54e8fba95ef9968a0c9b183fe22ec551c86 is empty
>> fatal: unable to get object info for 1702d54e8fba95ef9968a0c9b183fe22ec551c86
>> error: failed to run prune
>
> the error comes from "git prune" failing. It is using unreachable but
> "recent" objects as graph tips to keep around. If we can't load a tip
> object, we abort the prune, because we would not want to delete objects
> that might have been referenced (e.g., if it were a real corrupted
> object that we could not read).
>
> The second behavior (die on broken objects) has been around for a while.
> The new change (in the commit you bisected to) is that we are
> considering more objects.
>
> I'll admit I didn't really consider the impact on sticking broken object
> files into the object directory, but I think the code is doing the
> sensible and safe thing.

I agree. I wasn't aware this particular test was using such a
questionable mechanism to trigger automatic garbage collection. I
think the change to make git gc --auto more sensitive to this type of
thing is definitely an improvement. As noted, I just wanted to surface
the behavior change.

Thanks again for your quick response!

>
> -Peff