From: Adrian Ratiu <adrian.ratiu@collabora.com>
To: git@vger.kernel.org
Cc: Junio C Hamano <gitster@pobox.com>,
Patrick Steinhardt <ps@pks.im>,
"Eric W . Biederman" <ebiederm@xmission.com>,
"brian m . carlson" <sandals@crustytoothpaste.net>,
Adrian Ratiu <adrian.ratiu@collabora.com>
Subject: [PATCH v2 0/3] compatObjectFormat: fix a BUG(), wrong ID and unknown headers
Date: Thu, 19 Nov 2026 16:18:19 +0200 [thread overview]
Message-ID: <cover.1794932695.git.adrian.ratiu@collabora.com> (raw)
In-Reply-To: <20261112195740.2834341-1-adrian.ratiu@collabora.com>
Hello all,
This is v2 of the compatObjectFormat fixes found while testing sha1-sha256
interop between Git, Gitoxide and Jujutsu.
Because patch 3 drops a check that 08a45903cb (object-file-convert: convert
commits that embed signed tags, 2023-10-01) added on purpose, I'd still welcome
a look from Eric or Brian.
Changes in v2:
* Check repo_oid_to_algop() for "< 0" and reword the note on which
rev-parse forms are covered. The test now writes the object before
enabling compat instead of copying it into the object directory by
hand. (Patrick)
* Match the "tree ", "parent " and "mergetag " headers by their key,
like read_commit_extra_header_lines() does. (Patrick)
* Minor casing, spacing, doc and commit msg fixes (Patrick)
v1: https://lore.kernel.org/git/20261112195740.2834341-1-adrian.ratiu@collabora.com/
Many thanks,
Adrian
Adrian Ratiu (3):
setup: reject compatObjectFormat == objectFormat
rev-parse: die when --output-object-format cannot map an object
object-file-convert: copy unknown commit headers unchanged
builtin/rev-parse.c | 8 +++++---
object-file-convert.c | 35 ++++++++++++++++++++++-------------
setup.c | 7 +++++++
t/t1016-compatObjectFormat.sh | 34 ++++++++++++++++++++++++++++++++++
t/t1302-repo-version.sh | 14 ++++++++++++++
t/t1500-rev-parse.sh | 14 ++++++++++++++
6 files changed, 96 insertions(+), 16 deletions(-)
Range-diff against v1:
1: 53c814a1e1 ! 1: f74a0fd67a setup: reject compatObjectFormat == objectFormat
@@ setup.c: int verify_repository_format(const struct repository_format *format,
+ if (format->compat_hash_algo == format->hash_algo) {
+ strbuf_addf(err, _("'%s' is '%s', the same as the object format"),
-+ "extensions.compatobjectformat",
++ "extensions.compatObjectFormat",
+ hash_algos[format->hash_algo].name);
+ return -1;
+ }
@@ t/t1302-repo-version.sh: abort 0 noop-v1
+ test_expect_success "compatObjectFormat equal to objectFormat $hash" "
+ test_when_finished 'rm -rf same' &&
+ git init --object-format=$hash same &&
-+ git -C same config core.repositoryformatversion 1 &&
-+ git -C same config extensions.compatObjectFormat $hash &&
++ git -C same config set core.repositoryformatversion 1 &&
++ git -C same config set extensions.compatObjectFormat $hash &&
+ test_must_fail git -C same rev-parse --git-dir 2>err &&
-+ test_grep \"'extensions.compatobjectformat' is '$hash', the same as the object format\" err &&
-+ git config --file=same/.git/config --unset extensions.compatObjectFormat &&
++ test_grep \"'extensions.compatObjectFormat' is '$hash', the same as the object format\" err &&
++ git config unset --file=same/.git/config extensions.compatObjectFormat &&
+ git -C same rev-parse --git-dir
+ "
+done
2: 066fc59207 ! 2: 5b91372748 rev-parse: die when --output-object-format cannot map an object
@@ Commit message
rev-parse: die when --output-object-format cannot map an object
"git rev-parse --output-object-format=<compat-format>" should print
- an object compat id when extensions.compatObjectFormat is enabled.
+ the compat ID of an object when extensions.compatObjectFormat is
+ enabled.
- Git however doesn't record a compat id for every object, e.g. for
+ Git however doesn't record a compat ID for every object, e.g. for
objects received by a fetch that goes through index-pack.
- For such objects, rev-parse silently prints the wrong id: it prints
- the storage id where the compat id is expected and exits with 0.
+ For such objects, rev-parse silently prints the wrong ID: it prints
+ the storage ID where the compat ID is expected and exits with 0.
This happens because rev-parse ignores the repo_oid_to_algop() return
value. When no mapping is found, that function fails and leaves the
- id unchanged, so the storage id is what gets printed.
+ ID unchanged, so the storage ID is what gets printed.
Fix it to die with the same message as pack-objects. The other callers
of repo_oid_to_algop() already check its result.
- This only covers a single revision argument, the one case where
- rev-parse converts its output. Other forms, such as "A..B" or "A^@",
- print storage ids whatever --output-object-format says. That is left
- for a separate change.
-
- The test copies a loose object into the repository by hand, so it
- doesn't depend on whether fetch writes compat mappings.
+ Note that this only covers the case where the user asks for a single
+ revision, as it is the only case where rev-parse passes the desired
+ output hash explicitly. All the other cases, like for example when the
+ user passes a range, do not use the desired output hash at all. Fixing
+ that would be a bigger change though, so it is left for a future patch
+ series.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
@@ builtin/rev-parse.c: int cmd_rev_parse(int argc,
- output_algo, &oid);
+ if (output_algo &&
+ repo_oid_to_algop(the_repository, &oid,
-+ output_algo, &oid))
++ output_algo, &oid) < 0)
+ die(_("cannot map object %s to %s"),
+ oid_to_hex(&oid), output_algo->name);
if (verify)
@@ t/t1500-rev-parse.sh: test_expect_success RUST 'rev-parse --show-object-format i
'
+test_expect_success RUST 'rev-parse --output-object-format with unmapped object' '
-+ test_when_finished "rm -rf plain compat" &&
-+ git init --object-format=sha256 plain &&
-+ git init --object-format=sha256 compat &&
-+ git -C compat config extensions.compatobjectformat sha1 &&
-+ oid=$(echo unmapped | git -C plain hash-object -w --stdin) &&
-+ # copy the object by hand, so that no compat mapping is written for it
-+ path=$(test_oid_to_path $oid) &&
-+ mkdir compat/.git/objects/${path%/*} &&
-+ cp plain/.git/objects/$path compat/.git/objects/$path &&
++ test_when_finished "rm -rf unmapped" &&
++ git init --object-format=sha256 unmapped &&
++ # write the object before enabling compat, so it gets no compat mapping
++ oid=$(echo unmapped | git -C unmapped hash-object -w --stdin) &&
++ git -C unmapped config set extensions.compatObjectFormat sha1 &&
+ echo $oid >expect &&
-+ git -C compat rev-parse --output-object-format=sha256 $oid >actual &&
++ git -C unmapped rev-parse --output-object-format=sha256 $oid >actual &&
+ test_cmp expect actual &&
-+ test_must_fail git -C compat rev-parse --output-object-format=sha1 $oid >actual 2>err &&
++ test_must_fail git -C unmapped rev-parse --output-object-format=sha1 $oid >actual 2>err &&
+ test_must_be_empty actual &&
+ test_grep "cannot map object $oid to sha1" err
+'
3: a171e86b54 ! 3: 26080a7a38 object-file-convert: copy unknown commit headers unchanged
@@ Commit message
don't know:
1. When a commit is created with commit_tree_extended(), as
- "git commit" does, convert_commit_extra_headers() copies unknown
+ git-commit(1) does, convert_commit_extra_headers() copies unknown
headers unchanged.
- 2. When an existing commit object is converted (git hash-object -w)
- or when it's read by its compat id, convert_commit_object() fails
+ 2. When an existing commit object is converted (git-hash-object(1) -w)
+ or when it's read by its compat ID, convert_commit_object() fails
on them instead.
- So git hash-object refuses such a commit, while git commit --amend
- records a compat id for it that can't be read back:
+ So git-hash-object(1) refuses such a commit, while "git commit --amend"
+ records a compat ID for it that can't be read back:
"Failed to convert object from sha256 to sha1".
The rejection was added in 08a45903cb (object-file-convert: convert
commits that embed signed tags, 2023-10-01). Git can't know what an
- unknown header holds. It might hold an object id that needs
+ unknown header holds. It might hold an object ID that needs
converting, so convert_commit_object() rejects all of them.
Some tools rely on such headers though. Jujutsu, for example, writes
@@ Commit message
converted.
Make convert_commit_object() copy unknown headers and their
- continuation lines unchanged, just like convert_commit_extra_headers()
+ continuation lines unchanged, just like convert_commit_extra_headers(),
so both give the same compat object.
The author, committer, encoding and gpgsig headers were already copied
the same, so their separate cases are no longer needed.
- A "tree", "parent" or "mergetag" line with nothing after the keyword
- used to be rejected as unknown and is now copied as well. Such a
- commit is malformed in both formats anyway.
+ The "tree ", "parent " and "mergetag " cases only matched a line with
+ something after the space. That didn't matter while all other lines
+ were rejected, but they would now be copied. For a "mergetag " line
+ whose tag starts on the next line, the object ID in the tag would be
+ copied unconverted, even though Git reads that header as a mergetag and
+ convert_commit_extra_headers() refuses to convert it.
+
+ Match these headers by their key alone instead, the way
+ read_commit_extra_header_lines() does. Such a line with an empty or bad
+ value is then rejected, as before. A "tree", "parent" or "mergetag"
+ line without a space is not one of these headers to Git, so it is
+ copied like any other unknown header, which is also what
+ convert_commit_extra_headers() does.
Alternatives considered:
@@ Commit message
Both paths would then agree, but commits from tools that add
their own headers could not be used in a compat repo.
- - Copy only headers known to hold no object ids. Every tool would
+ - Copy only headers known to hold no object IDs. Every tool would
first need to get its headers added to that list.
- The test writes a commit with an unknown header with "git hash-object",
- amends it with "git commit", and checks that both commits map to the
- same objects in the SHA-1 and the SHA-256 repo.
+ - Convert values that look like object IDs. Any other hex string
+ could be taken for one, and Git shouldn't interpret headers it
+ doesn't know.
+
+ - Also reject lines that don't look like "<key> <value>". This would
+ make the two paths disagree again, because
+ convert_commit_extra_headers() copies such lines, so
+ "git commit --amend" would record compat IDs that can't be read
+ back.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
## object-file-convert.c ##
+@@ object-file-convert.c: static int convert_commit_object(struct repository *repo,
+ if (!eol)
+ return error(_("bad %s in commit"), "line");
+
+- if (((bufptr + 5) < eol) && !memcmp(bufptr, "tree ", 5))
++ if (((bufptr + 5) <= eol) && !memcmp(bufptr, "tree ", 5))
+ {
+ if (((bufptr + tree_entry_len) != eol) ||
+ parse_oid_hex_algop(bufptr + 5, &oid, &p, from) ||
+@@ object-file-convert.c: static int convert_commit_object(struct repository *repo,
+ "tree", oid_to_hex(&oid));
+ strbuf_addf(out, "tree %s\n", oid_to_hex(&mapped_oid));
+ }
+- else if (((bufptr + 7) < eol) && !memcmp(bufptr, "parent ", 7))
++ else if (((bufptr + 7) <= eol) && !memcmp(bufptr, "parent ", 7))
+ {
+ if (((bufptr + parent_entry_len) != eol) ||
+ parse_oid_hex_algop(bufptr + 7, &oid, &p, from) ||
+@@ object-file-convert.c: static int convert_commit_object(struct repository *repo,
+
+ strbuf_addf(out, "parent %s\n", oid_to_hex(&mapped_oid));
+ }
+- else if (((bufptr + 9) < eol) && !memcmp(bufptr, "mergetag ", 9))
++ else if (((bufptr + 9) <= eol) && !memcmp(bufptr, "mergetag ", 9))
+ {
+ struct strbuf tag = STRBUF_INIT, new_tag = STRBUF_INIT;
+
@@ object-file-convert.c: static int convert_commit_object(struct repository *repo,
strbuf_release(&tag);
strbuf_release(&new_tag);
@@ object-file-convert.c: static int convert_commit_object(struct repository *repo,
- /* Unknown line fail it might embed an oid */
- return -1;
+ /*
-+ * Copy all other headers unchanged. author, committer, encoding and gpgsig
-+ * hold no object ids. For unknown headers we can't tell, so copy them too,
-+ * like convert_commit_extra_headers() does when the commit is written. Both
-+ * must give the same compat object.
++ * Copy all other headers unchanged. author, committer,
++ * encoding and gpgsig hold no object IDs. For unknown
++ * headers we can't tell, so copy them too, like
++ * convert_commit_extra_headers() does when the commit
++ * is written. Both must give the same compat object.
++ *
++ * This means that a header holding object IDs is
++ * copied without converting them. Any such header added
++ * to Git needs its own case above, like parent or
++ * mergetag, and the same conversion in
++ * convert_commit_extra_headers().
+ *
-+ * This means that a header holding object ids is copied without converting
-+ * them. Any such header added to Git needs its own case above, like parent
-+ * or mergetag, and the same conversion in convert_commit_extra_headers().
++ * The cases above match a header by its key alone, as
++ * read_commit_extra_header_lines() does, so that e.g. a
++ * "mergetag " line with nothing after the space is
++ * still handled as a mergetag instead of being copied.
+ */
+ strbuf_add(out, bufptr, (eol - bufptr) + 1);
}
@@ object-file-convert.c: static int convert_commit_object(struct repository *repo,
## t/t1016-compatObjectFormat.sh ##
@@ t/t1016-compatObjectFormat.sh: do
- eval signedtag3_${hash}_oid=$(git hash-object -t tag -w ../${hash}_signedtag3) &&
eval signedtag4_${hash}_oid=$(git hash-object -t tag -w ../${hash}_signedtag4)
'
+
+ test_expect_success "create a $hash commit with an unknown header" '
+ git cat-file commit $(commit2_oid $hash) >../${hash}_commit2 &&
+ awk "{ print } /^committer / { print \"x-unknown one\"; print \" two\" }" \
@@ t/t1016-compatObjectFormat.sh: do
+ eval unknownheader_${hash}_oid=$oid &&
+ git branch unknownheader $oid
+ '
++
+ test_expect_success "amend a $hash commit with an unknown header" '
+ git checkout unknownheader &&
+ git commit --amend -m "Amend a commit with an unknown header" &&
@@ t/t1016-compatObjectFormat.sh: do
+ oid=$(git rev-parse HEAD) &&
+ eval unknownheader2_${hash}_oid=$oid
+ '
-
++
++ test_expect_success "reject a $hash commit with an empty mergetag line" '
++ git cat-file tag committag >../${hash}_committag &&
++ {
++ sed -n "/^committer /q;p" ../${hash}_commit2 &&
++ grep "^committer " ../${hash}_commit2 &&
++ echo "mergetag " &&
++ sed "s/^/ /" ../${hash}_committag &&
++ echo &&
++ echo message
++ } >../${hash}_emptymergetag &&
++ test_must_fail git hash-object -t commit -w ../${hash}_emptymergetag 2>err &&
++ test_grep "Failed to convert object" err
++ '
++
test_expect_success 'rev-parse maps oid of object borrowed from alternate' '
for repo in alt borrow
+ do
@@ t/t1016-compatObjectFormat.sh: compare_oids 'blob' more "$more_sha1_oid" "$more_sha256_oid"
compare_oids 'blob' another "$another_sha1_oid" "$another_sha256_oid"
compare_oids 'tree' tree2 "$tree2_sha1_oid" "$tree2_sha256_oid"
--
2.56.1
next prev parent reply other threads:[~2026-11-19 14:18 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-12 19:57 [PATCH 0/3] compatObjectFormat: fix a BUG(), wrong id and unknown headers Adrian Ratiu
2026-11-12 19:57 ` [PATCH 1/3] setup: reject compatObjectFormat == objectFormat Adrian Ratiu
2026-11-16 7:36 ` Patrick Steinhardt
2026-11-17 10:42 ` Adrian Ratiu
2026-11-12 19:57 ` [PATCH 2/3] rev-parse: die when --output-object-format cannot map an object Adrian Ratiu
2026-11-16 7:37 ` Patrick Steinhardt
2026-11-17 11:55 ` Adrian Ratiu
2026-11-12 19:57 ` [PATCH 3/3] object-file-convert: copy unknown commit headers unchanged Adrian Ratiu
2026-11-16 7:37 ` Patrick Steinhardt
2026-11-17 15:27 ` Adrian Ratiu
2026-11-19 14:18 ` Adrian Ratiu [this message]
2026-11-19 14:18 ` [PATCH v2 1/3] setup: reject compatObjectFormat == objectFormat Adrian Ratiu
2026-11-19 14:18 ` [PATCH v2 2/3] rev-parse: die when --output-object-format cannot map an object Adrian Ratiu
2026-11-19 14:18 ` [PATCH v2 3/3] object-file-convert: copy unknown commit headers unchanged Adrian Ratiu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover.1794932695.git.adrian.ratiu@collabora.com \
--to=adrian.ratiu@collabora.com \
--cc=ebiederm@xmission.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=ps@pks.im \
--cc=sandals@crustytoothpaste.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox