From: Adrian Ratiu <adrian.ratiu@collabora.com>
To: git@vger.kernel.org
Cc: Junio C Hamano <gitster@pobox.com>,
Patrick Steinhardt <ps@pks.im>,
"Eric W . Biederman" <ebiederm@xmission.com>,
"brian m . carlson" <sandals@crustytoothpaste.net>,
Adrian Ratiu <adrian.ratiu@collabora.com>
Subject: [PATCH v2 0/3] compatObjectFormat: fix a BUG(), wrong ID and unknown headers
Date: Thu, 19 Nov 2026 16:18:19 +0200 [thread overview]
Message-ID: <cover.1794932695.git.adrian.ratiu@collabora.com> (raw)
In-Reply-To: <20261112195740.2834341-1-adrian.ratiu@collabora.com>
Hello all,
This is v2 of the compatObjectFormat fixes found while testing sha1-sha256
interop between Git, Gitoxide and Jujutsu.
Because patch 3 drops a check that 08a45903cb (object-file-convert: convert
commits that embed signed tags, 2023-10-01) added on purpose, I'd still welcome
a look from Eric or Brian.
Changes in v2:
* Check repo_oid_to_algop() for "< 0" and reword the note on which
rev-parse forms are covered. The test now writes the object before
enabling compat instead of copying it into the object directory by
hand. (Patrick)
* Match the "tree ", "parent " and "mergetag " headers by their key,
like read_commit_extra_header_lines() does. (Patrick)
* Minor casing, spacing, doc and commit msg fixes (Patrick)
v1: https://lore.kernel.org/git/20261112195740.2834341-1-adrian.ratiu@collabora.com/
Many thanks,
Adrian
Adrian Ratiu (3):
setup: reject compatObjectFormat == objectFormat
rev-parse: die when --output-object-format cannot map an object
object-file-convert: copy unknown commit headers unchanged
builtin/rev-parse.c | 8 +++++---
object-file-convert.c | 35 ++++++++++++++++++++++-------------
setup.c | 7 +++++++
t/t1016-compatObjectFormat.sh | 34 ++++++++++++++++++++++++++++++++++
t/t1302-repo-version.sh | 14 ++++++++++++++
t/t1500-rev-parse.sh | 14 ++++++++++++++
6 files changed, 96 insertions(+), 16 deletions(-)
Range-diff against v1:
1: 53c814a1e1 ! 1: f74a0fd67a setup: reject compatObjectFormat == objectFormat
@@ setup.c: int verify_repository_format(const struct repository_format *format,
+ if (format->compat_hash_algo == format->hash_algo) {
+ strbuf_addf(err, _("'%s' is '%s', the same as the object format"),
-+ "extensions.compatobjectformat",
++ "extensions.compatObjectFormat",
+ hash_algos[format->hash_algo].name);
+ return -1;
+ }
@@ t/t1302-repo-version.sh: abort 0 noop-v1
+ test_expect_success "compatObjectFormat equal to objectFormat $hash" "
+ test_when_finished 'rm -rf same' &&
+ git init --object-format=$hash same &&
-+ git -C same config core.repositoryformatversion 1 &&
-+ git -C same config extensions.compatObjectFormat $hash &&
++ git -C same config set core.repositoryformatversion 1 &&
++ git -C same config set extensions.compatObjectFormat $hash &&
+ test_must_fail git -C same rev-parse --git-dir 2>err &&
-+ test_grep \"'extensions.compatobjectformat' is '$hash', the same as the object format\" err &&
-+ git config --file=same/.git/config --unset extensions.compatObjectFormat &&
++ test_grep \"'extensions.compatObjectFormat' is '$hash', the same as the object format\" err &&
++ git config unset --file=same/.git/config extensions.compatObjectFormat &&
+ git -C same rev-parse --git-dir
+ "
+done
2: 066fc59207 ! 2: 5b91372748 rev-parse: die when --output-object-format cannot map an object
@@ Commit message
rev-parse: die when --output-object-format cannot map an object
"git rev-parse --output-object-format=<compat-format>" should print
- an object compat id when extensions.compatObjectFormat is enabled.
+ the compat ID of an object when extensions.compatObjectFormat is
+ enabled.
- Git however doesn't record a compat id for every object, e.g. for
+ Git however doesn't record a compat ID for every object, e.g. for
objects received by a fetch that goes through index-pack.
- For such objects, rev-parse silently prints the wrong id: it prints
- the storage id where the compat id is expected and exits with 0.
+ For such objects, rev-parse silently prints the wrong ID: it prints
+ the storage ID where the compat ID is expected and exits with 0.
This happens because rev-parse ignores the repo_oid_to_algop() return
value. When no mapping is found, that function fails and leaves the
- id unchanged, so the storage id is what gets printed.
+ ID unchanged, so the storage ID is what gets printed.
Fix it to die with the same message as pack-objects. The other callers
of repo_oid_to_algop() already check its result.
- This only covers a single revision argument, the one case where
- rev-parse converts its output. Other forms, such as "A..B" or "A^@",
- print storage ids whatever --output-object-format says. That is left
- for a separate change.
-
- The test copies a loose object into the repository by hand, so it
- doesn't depend on whether fetch writes compat mappings.
+ Note that this only covers the case where the user asks for a single
+ revision, as it is the only case where rev-parse passes the desired
+ output hash explicitly. All the other cases, like for example when the
+ user passes a range, do not use the desired output hash at all. Fixing
+ that would be a bigger change though, so it is left for a future patch
+ series.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
@@ builtin/rev-parse.c: int cmd_rev_parse(int argc,
- output_algo, &oid);
+ if (output_algo &&
+ repo_oid_to_algop(the_repository, &oid,
-+ output_algo, &oid))
++ output_algo, &oid) < 0)
+ die(_("cannot map object %s to %s"),
+ oid_to_hex(&oid), output_algo->name);
if (verify)
@@ t/t1500-rev-parse.sh: test_expect_success RUST 'rev-parse --show-object-format i
'
+test_expect_success RUST 'rev-parse --output-object-format with unmapped object' '
-+ test_when_finished "rm -rf plain compat" &&
-+ git init --object-format=sha256 plain &&
-+ git init --object-format=sha256 compat &&
-+ git -C compat config extensions.compatobjectformat sha1 &&
-+ oid=$(echo unmapped | git -C plain hash-object -w --stdin) &&
-+ # copy the object by hand, so that no compat mapping is written for it
-+ path=$(test_oid_to_path $oid) &&
-+ mkdir compat/.git/objects/${path%/*} &&
-+ cp plain/.git/objects/$path compat/.git/objects/$path &&
++ test_when_finished "rm -rf unmapped" &&
++ git init --object-format=sha256 unmapped &&
++ # write the object before enabling compat, so it gets no compat mapping
++ oid=$(echo unmapped | git -C unmapped hash-object -w --stdin) &&
++ git -C unmapped config set extensions.compatObjectFormat sha1 &&
+ echo $oid >expect &&
-+ git -C compat rev-parse --output-object-format=sha256 $oid >actual &&
++ git -C unmapped rev-parse --output-object-format=sha256 $oid >actual &&
+ test_cmp expect actual &&
-+ test_must_fail git -C compat rev-parse --output-object-format=sha1 $oid >actual 2>err &&
++ test_must_fail git -C unmapped rev-parse --output-object-format=sha1 $oid >actual 2>err &&
+ test_must_be_empty actual &&
+ test_grep "cannot map object $oid to sha1" err
+'
3: a171e86b54 ! 3: 26080a7a38 object-file-convert: copy unknown commit headers unchanged
@@ Commit message
don't know:
1. When a commit is created with commit_tree_extended(), as
- "git commit" does, convert_commit_extra_headers() copies unknown
+ git-commit(1) does, convert_commit_extra_headers() copies unknown
headers unchanged.
- 2. When an existing commit object is converted (git hash-object -w)
- or when it's read by its compat id, convert_commit_object() fails
+ 2. When an existing commit object is converted (git-hash-object(1) -w)
+ or when it's read by its compat ID, convert_commit_object() fails
on them instead.
- So git hash-object refuses such a commit, while git commit --amend
- records a compat id for it that can't be read back:
+ So git-hash-object(1) refuses such a commit, while "git commit --amend"
+ records a compat ID for it that can't be read back:
"Failed to convert object from sha256 to sha1".
The rejection was added in 08a45903cb (object-file-convert: convert
commits that embed signed tags, 2023-10-01). Git can't know what an
- unknown header holds. It might hold an object id that needs
+ unknown header holds. It might hold an object ID that needs
converting, so convert_commit_object() rejects all of them.
Some tools rely on such headers though. Jujutsu, for example, writes
@@ Commit message
converted.
Make convert_commit_object() copy unknown headers and their
- continuation lines unchanged, just like convert_commit_extra_headers()
+ continuation lines unchanged, just like convert_commit_extra_headers(),
so both give the same compat object.
The author, committer, encoding and gpgsig headers were already copied
the same, so their separate cases are no longer needed.
- A "tree", "parent" or "mergetag" line with nothing after the keyword
- used to be rejected as unknown and is now copied as well. Such a
- commit is malformed in both formats anyway.
+ The "tree ", "parent " and "mergetag " cases only matched a line with
+ something after the space. That didn't matter while all other lines
+ were rejected, but they would now be copied. For a "mergetag " line
+ whose tag starts on the next line, the object ID in the tag would be
+ copied unconverted, even though Git reads that header as a mergetag and
+ convert_commit_extra_headers() refuses to convert it.
+
+ Match these headers by their key alone instead, the way
+ read_commit_extra_header_lines() does. Such a line with an empty or bad
+ value is then rejected, as before. A "tree", "parent" or "mergetag"
+ line without a space is not one of these headers to Git, so it is
+ copied like any other unknown header, which is also what
+ convert_commit_extra_headers() does.
Alternatives considered:
@@ Commit message
Both paths would then agree, but commits from tools that add
their own headers could not be used in a compat repo.
- - Copy only headers known to hold no object ids. Every tool would
+ - Copy only headers known to hold no object IDs. Every tool would
first need to get its headers added to that list.
- The test writes a commit with an unknown header with "git hash-object",
- amends it with "git commit", and checks that both commits map to the
- same objects in the SHA-1 and the SHA-256 repo.
+ - Convert values that look like object IDs. Any other hex string
+ could be taken for one, and Git shouldn't interpret headers it
+ doesn't know.
+
+ - Also reject lines that don't look like "<key> <value>". This would
+ make the two paths disagree again, because
+ convert_commit_extra_headers() copies such lines, so
+ "git commit --amend" would record compat IDs that can't be read
+ back.
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
## object-file-convert.c ##
+@@ object-file-convert.c: static int convert_commit_object(struct repository *repo,
+ if (!eol)
+ return error(_("bad %s in commit"), "line");
+
+- if (((bufptr + 5) < eol) && !memcmp(bufptr, "tree ", 5))
++ if (((bufptr + 5) <= eol) && !memcmp(bufptr, "tree ", 5))
+ {
+ if (((bufptr + tree_entry_len) != eol) ||
+ parse_oid_hex_algop(bufptr + 5, &oid, &p, from) ||
+@@ object-file-convert.c: static int convert_commit_object(struct repository *repo,
+ "tree", oid_to_hex(&oid));
+ strbuf_addf(out, "tree %s\n", oid_to_hex(&mapped_oid));
+ }
+- else if (((bufptr + 7) < eol) && !memcmp(bufptr, "parent ", 7))
++ else if (((bufptr + 7) <= eol) && !memcmp(bufptr, "parent ", 7))
+ {
+ if (((bufptr + parent_entry_len) != eol) ||
+ parse_oid_hex_algop(bufptr + 7, &oid, &p, from) ||
+@@ object-file-convert.c: static int convert_commit_object(struct repository *repo,
+
+ strbuf_addf(out, "parent %s\n", oid_to_hex(&mapped_oid));
+ }
+- else if (((bufptr + 9) < eol) && !memcmp(bufptr, "mergetag ", 9))
++ else if (((bufptr + 9) <= eol) && !memcmp(bufptr, "mergetag ", 9))
+ {
+ struct strbuf tag = STRBUF_INIT, new_tag = STRBUF_INIT;
+
@@ object-file-convert.c: static int convert_commit_object(struct repository *repo,
strbuf_release(&tag);
strbuf_release(&new_tag);
@@ object-file-convert.c: static int convert_commit_object(struct repository *repo,
- /* Unknown line fail it might embed an oid */
- return -1;
+ /*
-+ * Copy all other headers unchanged. author, committer, encoding and gpgsig
-+ * hold no object ids. For unknown headers we can't tell, so copy them too,
-+ * like convert_commit_extra_headers() does when the commit is written. Both
-+ * must give the same compat object.
++ * Copy all other headers unchanged. author, committer,
++ * encoding and gpgsig hold no object IDs. For unknown
++ * headers we can't tell, so copy them too, like
++ * convert_commit_extra_headers() does when the commit
++ * is written. Both must give the same compat object.
++ *
++ * This means that a header holding object IDs is
++ * copied without converting them. Any such header added
++ * to Git needs its own case above, like parent or
++ * mergetag, and the same conversion in
++ * convert_commit_extra_headers().
+ *
-+ * This means that a header holding object ids is copied without converting
-+ * them. Any such header added to Git needs its own case above, like parent
-+ * or mergetag, and the same conversion in convert_commit_extra_headers().
++ * The cases above match a header by its key alone, as
++ * read_commit_extra_header_lines() does, so that e.g. a
++ * "mergetag " line with nothing after the space is
++ * still handled as a mergetag instead of being copied.
+ */
+ strbuf_add(out, bufptr, (eol - bufptr) + 1);
}
@@ object-file-convert.c: static int convert_commit_object(struct repository *repo,
## t/t1016-compatObjectFormat.sh ##
@@ t/t1016-compatObjectFormat.sh: do
- eval signedtag3_${hash}_oid=$(git hash-object -t tag -w ../${hash}_signedtag3) &&
eval signedtag4_${hash}_oid=$(git hash-object -t tag -w ../${hash}_signedtag4)
'
+
+ test_expect_success "create a $hash commit with an unknown header" '
+ git cat-file commit $(commit2_oid $hash) >../${hash}_commit2 &&
+ awk "{ print } /^committer / { print \"x-unknown one\"; print \" two\" }" \
@@ t/t1016-compatObjectFormat.sh: do
+ eval unknownheader_${hash}_oid=$oid &&
+ git branch unknownheader $oid
+ '
++
+ test_expect_success "amend a $hash commit with an unknown header" '
+ git checkout unknownheader &&
+ git commit --amend -m "Amend a commit with an unknown header" &&
@@ t/t1016-compatObjectFormat.sh: do
+ oid=$(git rev-parse HEAD) &&
+ eval unknownheader2_${hash}_oid=$oid
+ '
-
++
++ test_expect_success "reject a $hash commit with an empty mergetag line" '
++ git cat-file tag committag >../${hash}_committag &&
++ {
++ sed -n "/^committer /q;p" ../${hash}_commit2 &&
++ grep "^committer " ../${hash}_commit2 &&
++ echo "mergetag " &&
++ sed "s/^/ /" ../${hash}_committag &&
++ echo &&
++ echo message
++ } >../${hash}_emptymergetag &&
++ test_must_fail git hash-object -t commit -w ../${hash}_emptymergetag 2>err &&
++ test_grep "Failed to convert object" err
++ '
++
test_expect_success 'rev-parse maps oid of object borrowed from alternate' '
for repo in alt borrow
+ do
@@ t/t1016-compatObjectFormat.sh: compare_oids 'blob' more "$more_sha1_oid" "$more_sha256_oid"
compare_oids 'blob' another "$another_sha1_oid" "$another_sha256_oid"
compare_oids 'tree' tree2 "$tree2_sha1_oid" "$tree2_sha256_oid"
--
2.56.1
next prev parent reply other threads:[~2026-11-19 14:18 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-12 19:57 [PATCH 0/3] compatObjectFormat: fix a BUG(), wrong id and unknown headers Adrian Ratiu
2026-11-12 19:57 ` [PATCH 1/3] setup: reject compatObjectFormat == objectFormat Adrian Ratiu
2026-11-16 7:36 ` Patrick Steinhardt
2026-11-17 10:42 ` Adrian Ratiu
2026-11-12 19:57 ` [PATCH 2/3] rev-parse: die when --output-object-format cannot map an object Adrian Ratiu
2026-11-16 7:37 ` Patrick Steinhardt
2026-11-17 11:55 ` Adrian Ratiu
2026-11-12 19:57 ` [PATCH 3/3] object-file-convert: copy unknown commit headers unchanged Adrian Ratiu
2026-11-16 7:37 ` Patrick Steinhardt
2026-11-17 15:27 ` Adrian Ratiu
2026-11-19 14:18 ` Adrian Ratiu [this message]
2026-11-19 14:18 ` [PATCH v2 1/3] setup: reject compatObjectFormat == objectFormat Adrian Ratiu
2026-11-19 14:18 ` [PATCH v2 2/3] rev-parse: die when --output-object-format cannot map an object Adrian Ratiu
2026-11-19 14:18 ` [PATCH v2 3/3] object-file-convert: copy unknown commit headers unchanged Adrian Ratiu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover.1794932695.git.adrian.ratiu@collabora.com \
--to=adrian.ratiu@collabora.com \
--cc=ebiederm@xmission.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=ps@pks.im \
--cc=sandals@crustytoothpaste.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.