Re: git fails to checkout SHA1 submodule in SHA256 repo with --depth=1

["brian m. carlson" <sandals@crustytoothpaste.net>]

[-- Attachment #1: Type: text/plain, Size: 3961 bytes --]

On 2025-11-13 at 10:15:30, Martin Wilck wrote:
> Let me just explain the use case: The distribution ((open)SUSE) has
> switched to git for version control of its packages. We have chosen
> SHA256, because we'll need to support the distribution for many years
> to come, much longer than SHA1 is going to be considered good enough.
> 
> We can store the source code of the package e.g. in the form of
> tarballs (and we do). But it's convenient and efficient, and thus
> tempting for developers, to simply link to an existing repository
> hosting the sources, using a submodule. And upstream repos still use
> SHA1. This is what lead us to experiment with this sort of mixed
> repository.
> 
> I get it that the concept is flawed and unsupported. Up to now, that
> wasn't obvious to me.

It wasn't obvious to a lot of people because the assumption is only
documented in the transition plan.  Mixed algorithm submodules are
something we should have thought about earlier on, but we didn't.

The original transition plan was to have full interoperability support
early on, in which case this wouldn't have been a problem, but for
technical reasons it ended up being much easier to have SHA-256-only
repositories, so we finished that first.  I then lost interest in the
project for many years (having kind of burnt out on the work) and, with
the exception of one set of patches that were sent in, nobody else
picked it up either.  So this ideally would have been implemented sooner
under the original plan, but it wasn't.  This is one of the pitfalls of
open source projects, as we all know.

> So what we can do now is either keep storing tarballs, or wait until
> there's a full solution for migration between / interoperability of
> different hash algorithms, and until the source code repos we're
> interested in have been fully migrated to SHA256. In some special
> cases, where (open)SUSE owns the source repositories, we may be able to
> simply migrate to a SHA256 forge. We can also invent a "poor man's
> submodule" mechanism to link to sources on some external repository
> from ours [1].

There is work underway for SHA-1/SHA-256 interoperability.  However, it
is a thing I'm working on primarily in my free time (although also a
little at work) and nobody else has stepped up to contribute, so it will
take some time to complete and is not expected to be included in Git
3.0 without additional assistance.

To be clear, the existing interoperability code does work for
repositories without submodules and you may find it on the
`sha256-interop` branch at https://github.com/bk2204/git.git, but it
isn't production ready.  I might use it myself, but I would not
recommend others do so in its current state.

For right now, I would say that you should keep storing tarballs.

> Another question: If I, in the current repo [2], create a commit on top
> removing the submodule and replacing it by a tarball, would the
> repository remain broken, as it would still have the deprecated
> SHA256/SHA1 combination in the history? Should I expect errors if run
> e.g. "git rebase" or "git bisect" in a repository like this? IOW, do I
> need to rewrite the history of this repo, eliminating all instances of
> such mixed-hash submodules, to be on the safe side?

If the mismatched submodule remains in the history, then it will be
broken, so you'll almost certainly want to rewrite the history.

When the interoperability work is done and fully functional, a
repository in this situation will not work at all with dual algorithms.
The reason for that is that the SHA-256 main repository will try to use
the submodule value as a SHA-256 submodule (using the incorrect padded
SHA-1 value as a SHA-256 value) and it won't be able to create the
appropriate SHA-1/SHA-256 mapping since that object ID doesn't really
exist in the submodule.
-- 
brian m. carlson (they/them)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]