Re: Bug report: Strange behavior with `git gc` and `reference-transaction` hook

[Patrick Steinhardt <ps@pks.im>]

[-- Attachment #1: Type: text/plain, Size: 4393 bytes --]

On Thu, Nov 18, 2021 at 01:08:41PM -0500, Jeff King wrote:
> [+cc pks]
> 
> On Wed, Nov 17, 2021 at 04:52:46PM -0800, Bryan Turner wrote:
> 
> > > The expected behavior would be that the latest reference transaction
> > > hook refers to the state of the references on disk. That is, either
> > > `master` should point to 0 (be deleted), or it should have said that
> > > `master` pointed to `e197d1`.
> > >
> > > But if we actually examine `master`, it's set to `e197d1`, just as you
> > > would expect. The GC should have been a no-op overall.
> > 
> > One of the subtasks of "git gc" is "git pack-refs". If you inspect in
> > more detail, I suspect you'll find that "refs/heads/master" was loose
> > before "git gc" ran (as in, there was a file
> > "$GIT_DIR/refs/heads/master") and "packed-refs" either didn't have a
> > "refs/heads/master" entry or had a different hash. (Loose refs always
> > "win" over packed, since ref updates only write loose refs.)
> 
> It seems totally broken to me that we'd trigger the
> reference-transaction hook for ref packing. The point of the hook is to
> track logical updates to the refs. But during ref packing that does not
> change at all; the value remains the same. So I don't think we should be
> triggering the hook at all, let alone with confusing values.
> 
> This snippet shows a simple case that I think is wrong:
> 
> -- >8 --
> git init -q repo
> cd repo
> 
> cat >.git/hooks/reference-transaction <<\EOF
> #!/bin/sh
> echo >&2 "==> reference-transaction $*"
> sed 's/^/  /'
> EOF
> chmod +x .git/hooks/reference-transaction
> 
> echo >&2 "running commit..."
> git commit --allow-empty -qm foo
> echo >&2 "running pack-refs..."
> git pack-refs --all --prune
> -- >8 --
> 
> It produces:
> 
>   running commit...
>   ==> reference-transaction prepared
>     0000000000000000000000000000000000000000 77bcab0d950aee3021e8aa13a15d40e7a9a5f71b HEAD
>     0000000000000000000000000000000000000000 77bcab0d950aee3021e8aa13a15d40e7a9a5f71b refs/heads/main
>   ==> reference-transaction committed
>     0000000000000000000000000000000000000000 77bcab0d950aee3021e8aa13a15d40e7a9a5f71b HEAD
>     0000000000000000000000000000000000000000 77bcab0d950aee3021e8aa13a15d40e7a9a5f71b refs/heads/main
>   running pack-refs...
>   ==> reference-transaction prepared
>     0000000000000000000000000000000000000000 77bcab0d950aee3021e8aa13a15d40e7a9a5f71b refs/heads/main
>   ==> reference-transaction committed
>     0000000000000000000000000000000000000000 77bcab0d950aee3021e8aa13a15d40e7a9a5f71b refs/heads/main
>   ==> reference-transaction prepared
>     77bcab0d950aee3021e8aa13a15d40e7a9a5f71b 0000000000000000000000000000000000000000 refs/heads/main
>   ==> reference-transaction committed
>     77bcab0d950aee3021e8aa13a15d40e7a9a5f71b 0000000000000000000000000000000000000000 refs/heads/main
> 
> I think the final four invocations should be skipped entirely. They're
> pointless at best (nothing actually changed), and extremely misleading
> at worst (they look like the ref ended up deleted!).
> 
> -Peff

Yeah, I agree that this is something that is totally misleading.

For what it's worth, we also hit a similar case in production at GitLab,
where we use the hook to do voting on ref updates across different
nodes. Sometimes we observed different votes on a subset of nodes, and
it took me quite some time to figure out that this was dependent on
whether a ref was packed or not. We're now filtering out transactions
which consist only of force-deletions [1], which are _likely_ to be
cleanups of such packed refs. But this is very clearly a hack, and I
agree that calling the hook for 

In the end, these really are special cases of how the "files" backend
works and thus are implementation details which shouldn't be exposed to
the user at all. With these implementation details exposed, we'll start
to see different behaviour of when the hook is executed depending on
which ref backend you use, which is even worse compared to the current
state where it's at least consistently misleading.

As Peff said, the hook should really only track logical changes and not
expose any implementation details.

Patrick

[1]: https://gitlab.com/gitlab-org/gitaly/-/blob/3ef55853e9e161204464868390d97d1a1577042d/internal/gitaly/hook/referencetransaction.go#L58

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]