[PATCH v3] fast-import: allow 'from' to reference new commits and tags by oid

["Marc Becker via GitGitGadget" <gitgitgadget@gmail.com>]

From: Marc Becker <becm@gmx.de>

During a 'git fast-import' session, newly created objects are written
to an in-progress packfile and recorded in fast-import's internal
object table.  Until a 'checkpoint' command or the end of the session
finalizes and indexes that packfile, these objects are not visible to
the repository's object database via odb_read_object_peeled().

Commands such as 'ls' (via parse_treeish_dataref() and dereference())
and 'cat-blob' already account for this when given a hex object name:
they look up the object in fast-import's internal object table with
find_object() first and read it from the in-progress packfile with
gfi_unpack_entry() when its '.pack_id' is not MAX_PACK_ID.

By contrast, when 'from <committish>' (in a 'commit' or 'reset'
command) is given a hex object name instead of a mark or branch name,
parse_objectish() parses the hex string into 'b->oid' and calls
parse_from_existing(), which unconditionally looks up the object via
odb_read_object_peeled().  If the object is a commit or tag created in
the current pack cycle, odb_read_object_peeled() returns NULL and
fast-import dies with "fatal: not a valid commit: <oid>".

Teach parse_from_existing() to look up 'b->oid' with find_object()
first, peel any tag objects in the object table using dereference(),
and unpack the target commit with gfi_unpack_entry() if it resides in
the in-progress packfile, before falling back to
odb_read_object_peeled() for objects in the repository's object
database.

Signed-off-by: Marc Becker <becm@gmx.de>
---
    fast-import: fix lookup for new commits
    
    Changes since v2:
    
     * use LLM-generated commit message (description is on point) by Junio
     * provide hash-specific OIDs in test code via test_oid

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2252%2Fbecm%2Ffix-fast-import-commit-from-existing-v3
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2252/becm/fix-fast-import-commit-from-existing-v3
Pull-Request: https://github.com/gitgitgadget/git/pull/2252

Range-diff vs v2:

 1:  d834e2fc47 ! 1:  25113432c6 fast-import: fix lookup for new commits and tags
     @@ Metadata
      Author: Marc Becker <becm@gmx.de>
      
       ## Commit message ##
     -    fast-import: fix lookup for new commits and tags
     +    fast-import: allow 'from' to reference new commits and tags by oid
      
     -    Queries for commits created during the same 'git fast-import' session
     -    result in a crash:
     -        < reset refs/heads/newbranch
     -        < from <new-commit-oid>
     -        > fatal: not a valid commit: <new-commit-oid>
     +    During a 'git fast-import' session, newly created objects are written
     +    to an in-progress packfile and recorded in fast-import's internal
     +    object table.  Until a 'checkpoint' command or the end of the session
     +    finalizes and indexes that packfile, these objects are not visible to
     +    the repository's object database via odb_read_object_peeled().
      
     -    Getting file content for commit has no issue (working implementation):
     -        < ls <new-commit-oid> file-in-new-commit.txt
     -        > {file content}
     +    Commands such as 'ls' (via parse_treeish_dataref() and dereference())
     +    and 'cat-blob' already account for this when given a hex object name:
     +    they look up the object in fast-import's internal object table with
     +    find_object() first and read it from the in-progress packfile with
     +    gfi_unpack_entry() when its '.pack_id' is not MAX_PACK_ID.
      
     -    During import, new objects may not (yet) be reachable via the ODB layer.
     -    Try to resolve requested tag/commit via object cache first
     -    to avoid crash.
     -    Add test to 't/t9300.sh' with references to a pending tag and commit.
     +    By contrast, when 'from <committish>' (in a 'commit' or 'reset'
     +    command) is given a hex object name instead of a mark or branch name,
     +    parse_objectish() parses the hex string into 'b->oid' and calls
     +    parse_from_existing(), which unconditionally looks up the object via
     +    odb_read_object_peeled().  If the object is a commit or tag created in
     +    the current pack cycle, odb_read_object_peeled() returns NULL and
     +    fast-import dies with "fatal: not a valid commit: <oid>".
     +
     +    Teach parse_from_existing() to look up 'b->oid' with find_object()
     +    first, peel any tag objects in the object table using dereference(),
     +    and unpack the target commit with gfi_unpack_entry() if it resides in
     +    the in-progress packfile, before falling back to
     +    odb_read_object_peeled() for objects in the repository's object
     +    database.
      
          Signed-off-by: Marc Becker <becm@gmx.de>
      
     @@ t/t9300-fast-import.sh: test_expect_success 'Y: rewrite submodules' '
       	)
       '
       
     -+cat >Y-reference-pending-sha256 <<\Y_INPUT_END
     -+commit refs/heads/main
     -+committer A U Thor <author@example.com> 1712563200 +0000
     -+data 14
     -+commit message
     -+M 644 inline test file
     -+data 14
     -+dummy content
     -+
     -+tag newtag
     -+from 17e650f097a4db6fe069c96935360d3ccc9b35c4018728c5f311a4217c2ced3d
     -+tagger A U Thor <author@example.com> 1712563300 +0000
     -+data 11
     -+tag message
     -+# no emty line after tag data!
     -+reset refs/heads/other
     -+from 85edc5eac315bc680677b2fc71622bd77476f057eb8dc8ed6a497803a24f1f46
     -+
     -+commit refs/heads/further
     -+committer A U Thor <author@example.com> 1712563400 +0000
     -+data 21
     -+what about 2nd commit
     -+from 17e650f097a4db6fe069c96935360d3ccc9b35c4018728c5f311a4217c2ced3d
     -+M 644 inline other test file
     -+data 19
     -+more dummy content
     -+
     -+done
     -+Y_INPUT_END
     -+
     -+cat >Y-reference-pending-sha1 <<\Y_INPUT_END
     -+commit refs/heads/main
     -+committer A U Thor <author@example.com> 1712563200 +0000
     -+data 14
     -+commit message
     -+M 644 inline test file
     -+data 14
     -+dummy content
     -+
     -+tag newtag
     -+from 85b46dd918355ff17b4109ec738723d42f235a04
     -+tagger A U Thor <author@example.com> 1712563300 +0000
     -+data 11
     -+tag message
     -+# no emty line after tag data!
     -+reset refs/heads/other
     -+from ec6c4db71feb5f66615214dad2324ce030e777a7
     -+
     -+commit refs/heads/further
     -+committer A U Thor <author@example.com> 1712563400 +0000
     -+data 21
     -+what about 2nd commit
     -+from 85b46dd918355ff17b4109ec738723d42f235a04
     -+M 644 inline other test file
     -+data 19
     -+more dummy content
     -+
     -+done
     -+Y_INPUT_END
     -+
      +test_expect_success 'Y: reference pending tag or commit' '
     ++	test_oid_cache <<-EOF &&
     ++		pending_commit sha1:85b46dd918355ff17b4109ec738723d42f235a04
     ++		pending_commit sha256:17e650f097a4db6fe069c96935360d3ccc9b35c4018728c5f311a4217c2ced3d
     ++		pending_tag sha1:ec6c4db71feb5f66615214dad2324ce030e777a7
     ++		pending_tag sha256:85edc5eac315bc680677b2fc71622bd77476f057eb8dc8ed6a497803a24f1f46
     ++		EOF
     ++
      +	git init reference-pending &&
      +	(
      +		cd reference-pending &&
     -+		git fast-import < "../Y-reference-pending-$(git rev-parse --show-object-format)"
     ++		git fast-import <<-EOF
     ++			commit refs/heads/main
     ++			committer A U Thor <author@example.com> 1712563200 +0000
     ++			data 14
     ++			commit message
     ++			M 644 inline test file
     ++			data 14
     ++			dummy content
     ++
     ++			tag newtag
     ++			from $(test_oid pending_commit)
     ++			tagger A U Thor <author@example.com> 1712563300 +0000
     ++			data 11
     ++			tag message
     ++			# no emty line after tag data!
     ++			reset refs/heads/other
     ++			from $(test_oid pending_tag)
     ++
     ++			commit refs/heads/further
     ++			committer A U Thor <author@example.com> 1712563400 +0000
     ++			data 21
     ++			what about 2nd commit
     ++			from $(test_oid pending_commit)
     ++			M 644 inline other test file
     ++			data 19
     ++			more dummy content
     ++
     ++			done
     ++			EOF
      +	)
      +'
      +


 builtin/fast-import.c  | 19 +++++++++++++++----
 t/t9300-fast-import.sh | 43 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 58 insertions(+), 4 deletions(-)

diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index fbd919982c..574a03c484 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -292,6 +292,9 @@ static void fast_import_state_init(struct fast_import_state *state,
 	state->option = option;
 }
 
+static struct object_entry *dereference(struct object_entry *oe,
+					struct object_id *oid);
+
 static void parse_argv(struct fast_import_state *state);
 static void parse_get_mark(struct fast_import_state *state, const char *p);
 static void parse_cat_blob(struct fast_import_state *state, const char *p);
@@ -2658,12 +2661,20 @@ static void parse_from_existing(struct branch *b)
 		oidclr(&b->branch_tree.versions[1].oid, the_repository->hash_algo);
 	} else {
 		unsigned long size;
-		size_t size_st = 0;
 		char *buf;
+		struct object_entry *e = find_object(&b->oid);
 
-		buf = odb_read_object_peeled(the_repository->objects, &b->oid,
-					     OBJ_COMMIT, &size_st, &b->oid);
-		size = cast_size_t_to_ulong(size_st);
+		while (e && e->type == OBJ_TAG)
+			e = dereference(e, &b->oid);
+
+		if (e && e->type == OBJ_COMMIT && e->pack_id != MAX_PACK_ID) {
+			buf = gfi_unpack_entry(e, &size);
+		} else {
+			size_t size_st = 0;
+			buf = odb_read_object_peeled(the_repository->objects, &b->oid,
+						     OBJ_COMMIT, &size_st, &b->oid);
+			size = cast_size_t_to_ulong(size_st);
+		}
 		parse_from_commit(b, buf, size);
 		free(buf);
 	}
diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh
index d9de2ef0d8..9001e2572a 100755
--- a/t/t9300-fast-import.sh
+++ b/t/t9300-fast-import.sh
@@ -3938,4 +3938,47 @@ test_expect_success 'Y: rewrite submodules' '
 	)
 '
 
+test_expect_success 'Y: reference pending tag or commit' '
+	test_oid_cache <<-EOF &&
+		pending_commit sha1:85b46dd918355ff17b4109ec738723d42f235a04
+		pending_commit sha256:17e650f097a4db6fe069c96935360d3ccc9b35c4018728c5f311a4217c2ced3d
+		pending_tag sha1:ec6c4db71feb5f66615214dad2324ce030e777a7
+		pending_tag sha256:85edc5eac315bc680677b2fc71622bd77476f057eb8dc8ed6a497803a24f1f46
+		EOF
+
+	git init reference-pending &&
+	(
+		cd reference-pending &&
+		git fast-import <<-EOF
+			commit refs/heads/main
+			committer A U Thor <author@example.com> 1712563200 +0000
+			data 14
+			commit message
+			M 644 inline test file
+			data 14
+			dummy content
+
+			tag newtag
+			from $(test_oid pending_commit)
+			tagger A U Thor <author@example.com> 1712563300 +0000
+			data 11
+			tag message
+			# no emty line after tag data!
+			reset refs/heads/other
+			from $(test_oid pending_tag)
+
+			commit refs/heads/further
+			committer A U Thor <author@example.com> 1712563400 +0000
+			data 21
+			what about 2nd commit
+			from $(test_oid pending_commit)
+			M 644 inline other test file
+			data 19
+			more dummy content
+
+			done
+			EOF
+	)
+'
+
 test_done

base-commit: 6de20f6092dcf9bdb1c8efe03db4b70c82b423dd
-- 
gitgitgadget