discussion: sha1/sha256 repositories and convert_commit_object

discussion: sha1/sha256 repositories and convert_commit_object

[Adam Majer]

Hi,

I'm been experimenting with having both OID types in a repository. 
First, I tried nodejs, which failed due to missing submodules, so I 
could not import it.. Then I tried linux. This proved somewhat more 
problematic.

I've been using convert_commit_object to convert a commit from one OID 
type to another. The current implementation of convert_commit_object 
function has some comments about requiring to convert all types of OID 
references. These includes mergetag and it just bails on other, unknown 
headers.

In the linux tree, some dozen or so mergetags contain references that do 
not seem to be fetchable from anywhere.

Also, a number of commits in linux history contains these unknown headers:

ab5af2903baa472930c94a421efdd22a49036213:
   predecessor bc6a71940d5182103469a479d12f4266ac155a39
   predecessor-op rebase

3a413ece2504c70aa34a20be4dafec04e8c741f9:
   predecessor 86d97728d053d6584be5592480ff3687d3d616ab
   predecessor-op amend

And the objects references here are unavailable.


How can we deal with these if we ever want to have the sha1-sha256 
interop? If we require stubborn adherence to 1-1 mappings of all OIDs, 
these repositories cannot ever be imported into sha256 repositories with 
their histories. Even recent commits cause issues here as git is not 
holding on to referenced objects.

Would it be possible to have a little different approach? For example, 
mergetag headers and other unknown headers, maybe we can consider these 
as just text and copy verbatim in convert_commit_object? I understand 
this would result in different commit ids, depending if coming 
originally from sha256 or sha1 repo, but since git is not really 
preserving references to these objects anyway, this may call for a 
little flexibility.

Best regards,
- Adam

Re: discussion: sha1/sha256 repositories and convert_commit_object

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 2925 bytes --]

On 2026-10-21 at 17:37:13, Adam Majer wrote:
> Hi,

Hey,

> I'm been experimenting with having both OID types in a repository. First, I
> tried nodejs, which failed due to missing submodules, so I could not import
> it.. Then I tried linux. This proved somewhat more problematic.
> 
> I've been using convert_commit_object to convert a commit from one OID type
> to another. The current implementation of convert_commit_object function has
> some comments about requiring to convert all types of OID references. These
> includes mergetag and it just bails on other, unknown headers.
> 
> In the linux tree, some dozen or so mergetags contain references that do not
> seem to be fetchable from anywhere.

I had noticed that the mergetags were not in Linus's repository, but I
hadn't tried other repositories on kernel.org.

> Also, a number of commits in linux history contains these unknown headers:
> 
> ab5af2903baa472930c94a421efdd22a49036213:
>   predecessor bc6a71940d5182103469a479d12f4266ac155a39
>   predecessor-op rebase
> 
> 3a413ece2504c70aa34a20be4dafec04e8c741f9:
>   predecessor 86d97728d053d6584be5592480ff3687d3d616ab
>   predecessor-op amend
> 
> And the objects references here are unavailable.

We won't try to convert arbitrary headers like this.  However, I'm fine
with us skipping those headers so that their commits can be converted.

> How can we deal with these if we ever want to have the sha1-sha256 interop?
> If we require stubborn adherence to 1-1 mappings of all OIDs, these
> repositories cannot ever be imported into sha256 repositories with their
> histories. Even recent commits cause issues here as git is not holding on to
> referenced objects.
> 
> Would it be possible to have a little different approach? For example,
> mergetag headers and other unknown headers, maybe we can consider these as
> just text and copy verbatim in convert_commit_object? I understand this
> would result in different commit ids, depending if coming originally from
> sha256 or sha1 repo, but since git is not really preserving references to
> these objects anyway, this may call for a little flexibility.

I don't think I converted mergetag entries in my original series.  The
problem is that if we don't convert them, then we also can't validate
them because we don't know what data they point to.  We don't make
assumptions based on the size of the hash because in the future we might
also have SHA3-256 or BLAKE2b-256.

That means that the original point of the mergetags, to preserve the
signature and contents of the original tag, would be useless.  That
doesn't mean we couldn't skip converting them, but it does mean that if
we do so, we should probably just stop writing those headers because
they're no longer fit for purpose and we should consider them a failed
implementation.
-- 
brian m. carlson (they/them)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 325 bytes --]

Re: discussion: sha1/sha256 repositories and convert_commit_object

[Adam Majer]

On October 22, 2026 1:21:20 a.m. GMT+03:00, "brian m. carlson" <sandals@crustytoothpaste.net> wrote:
>On 2026-10-21 at 17:37:13, Adam Majer wrote

>I had noticed that the mergetags were not in Linus's repository, but I
>hadn't tried other repositories on kernel.org.

There is about a dozen or so mertags that I cannot find their objects. They have references to other repos, but trying those, they remained unresolved. One mergetag had some 3rd party url, which wasn't working, but http:// access allowed me to resolve it. One from a dozen missing resolved. 



>> Would it be possible to have a little different approach? For example,
>> mergetag headers and other unknown headers, maybe we can consider these as
>> just text and copy verbatim in convert_commit_object? I understand this
>> would result in different commit ids, depending if coming originally from
>> sha256 or sha1 repo, but since git is not really preserving references to
>> these objects anyway, this may call for a little flexibility.
>
>I don't think I converted mergetag entries in my original series.  The
>problem is that if we don't convert them, then we also can't validate
>them because we don't know what data they point to.  We don't make
>assumptions based on the size of the hash because in the future we might
>also have SHA3-256 or BLAKE2b-256.

I'm not sure that is quite correct. Any supported hash function should be unique enough such that we can just try all of them. So, a mergetag or proprietary headers, if we just treat them as text in conversion function, they can be retrieved later using the original function. Hash function doesn't matter for their existence or validity. I'm assuming here that the original repo containing them is not replaced in the meantime.

Basically, I'm assuming that a repo url keeps their original hash function.

The most important part here is to be consistent. These shouldn't be converted when available and skipped otherwise, or history becomes nondeterministic and depends on ingenuity of converting person to find these objects. 


>That means that the original point of the mergetags, to preserve the
>signature and contents of the original tag, would be useless.  That
>doesn't mean we couldn't skip converting them, but it does mean that if
>we do so, we should probably just stop writing those headers because
>they're no longer fit for purpose and we should consider them a failed
>implementation.

My problem is these mergetags have url where they were published. But they are quite missing on the git. kernel.org in these repos.

We can still convert the objects, but references would go via loose object table.


As for unavailable submodules, like in NodeJS repo from over a decade ago, I think they'll be unconvertible. It's not really possible to skip these without making a mess of history. Or perhaps you have an idea here? 

- Adam