Re: [RFC PATCH 0/4] sign a SHA-256 digest of the tree in commits and tags

["brian m. carlson" <sandals@crustytoothpaste.net>]

[-- Attachment #1: Type: text/plain, Size: 5161 bytes --]

On 2026-10-06 at 16:16:07, Kristoffer Haugsbakk wrote:
> And that was okay. The Git repo seems to have a bit of cruft, and
> git-fast-export(1) fails on the first error then suggests a fix so that
> you can continue on to the next error. But that’s fine for a one-shot
> program. For anyone interested:
> 
>     git fast-export --all --reencode=yes --mark-tags \
>         --signed-tags=verbatim \
>         --tag-of-filtered-object=rewrite >SHA1HERE
> 
> Some real loss of fidelity was there though:
> 
> 1. You can’t for some reason export refs that point to blobs or trees
> 2. Your Git notes will be effectively lost since they will retain their
>    SHA-1 filenames. (This is mentioned in hash-function-transition)
> 
> Then you import it with
> 
>     git fast-import
> 
> But to no one’s surprise (here) this does not work because of the SHA-1
> collision submodule.

We actually have support for rewriting submodules in fast-export and
fast-import.  It's a little fussy because you have to rewrite all the
submodules before you rewrite the main repository, but it works.  Here's
a command to handle git.git:

----
#!/bin/sh

temp=$(mktemp -d)
trap 'rm -fr "$temp"' EXIT

GIT_LOCATION="$1"
RESULT="$2"

git -C "$GIT_LOCATION/sha1collisiondetection" fast-export --signed-tags=verbatim --tag-of-filtered-object=drop --export-marks="$temp/sha1dc-sha1.marks" --all >"$temp/sha1dc.export"

git init --bare --object-format=sha256 "$temp/sha1dc"
git -C "$temp/sha1dc" fast-import --export-marks="$temp/sha1dc-sha256.marks" < "$temp/sha1dc.export"

git -C "$GIT_LOCATION" fast-export --reencode=no --signed-tags=verbatim --tag-of-filtered-object=drop --branches --tags >"$temp/git.export"
git init --object-format=sha256 "$RESULT"
git -C "$RESULT" fast-import --rewrite-submodules-from=sha1dc:"$temp/sha1dc-sha1.marks" --rewrite-submodules-to=sha1dc:"$temp/sha1dc-sha256.marks" <"$temp/git.export"
----

And here's an example running it right now (my main branch following
`master` is `dev`):

----
% ./convert-git ~/checkouts/git git-sha256.git
[elided]
% git -C git-sha256.git log -1 --format=oneline dev
05370fd7088edf77bfcd09c8c909450e764a9d10d8304dc333f39f01697c5a84 4th batch for -rc1
----

The downside is that it doesn't produce the same results as the true
interoperability code and it's much slower, and, as I pointed out above,
the user experience is poor.  The advantage is that it's been available
since the original SHA-256 work in about 2.30 or so, so you can totally
make it work almost anywhere.  The above script could also probably be
nicely converted into a generic script that would work on any repository
without too much effort.

> Okay, dropping that exercise for a second. I would personally be okay
> with trying out this migration on my existing repos that are “local
> only”. It would clearly be in my interest to find any bugs that are
> particular to my workflows. But for that I would that migration where
> you keep a mapping of SHA-1 to SHA-256. Or else I will lose Git notes
> forever (which I use a lot).
> 
> But reading brian’s cousin response:
> <asQrWAKQXV9zn1Vq@fruit.crustytoothpaste.net> ... it seems that there is
> not enough in git(1) or anywhere else to do that.

The interoperability work doesn't rewrite notes because it only happens
when cloning or fetching from a repository and notes aren't usually
copied in that case.  In-place rewriting is not yet implemented,
although that's a thing I'd like to work on.  Hooking notes into that
shouldn't be very difficult to do.

The reason more of the interoperability work has not gone upstream is
because the pluggable ODB work has really ended up breaking a lot of
things[0], so sending almost anything requires a bunch of rebasing and
fixing, and I'm presently very burnt out, so I'm doing very little
coding in my free time and doing more cycling, reading, and Factorio:
Space Age.

> The above scenario would be very hyperbolic and too cynical if not for
> the context: one person is leading the direct implementation work[2] in
> their spare time. In order to migrate Git from a to-be government-wide
> banned hash algorithm. That seems like an institutional malfunction.
> Somewhere.

This is the problem with open source, unfortunately.  In the ideal
world, would other people and very especially major companies help out
more?  Sure.  But macOS and FreeBSD also ship one person's bc/dc
implementation as a core part of the OS, there's only one maintainer
each for bash and ncurses, and a lot of other cases.  This is basically
https://xkcd.com/2347/, which, as we all know, is a widespread problem.

As I said elsewhere, everyone is interested in scaling Git to larger and
larger repositories and improving performance, but little else gets
attention.  Those are things I _don't_ really want to work on, which is
why my job is not working on Git.

[0] To be clear, I think it's a great project and I'm very happy to see
the work come in, but it has impacts throughout the codebase.
-- 
brian m. carlson (they/them)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 325 bytes --]