[PATCH V1][RFC] t/perf/p3400: speed up setup using fast-import

[PATCH V1][RFC] t/perf/p3400: speed up setup using fast-import

[Tian Yuchen]

The setup phase in 't/perf/p3400-rebase.sh' generates 100 commits to
simulate a noisy history. It currently uses a shell loop that invokes
'git add', 'git commit', 'test_seq', and 'sort' in each iteration.
This makes the whole process slow.

Optimize the setup by using 'git fast-import' to generate the commit
history in a single stream. Additionally, pre-compute the forward and
reversed file contents to avoid repetitive execution of 'seq' and 'sort'.

Performance enhancement:
  Before: 29.045s
  After:  18.081s

Measured on Lenovo Yoga 2020, Ubuntu 24.04.

Signed-off-by: Tian Yuchen <a3205153416@gmail.com>
---
 t/perf/p3400-rebase.sh | 53 +++++++++++++++++++++++++++++-------------
 1 file changed, 37 insertions(+), 16 deletions(-)

diff --git a/t/perf/p3400-rebase.sh b/t/perf/p3400-rebase.sh
index e6b0277729..9251a09b4f 100755
--- a/t/perf/p3400-rebase.sh
+++ b/t/perf/p3400-rebase.sh
@@ -9,25 +9,46 @@ test_expect_success 'setup rebasing on top of a lot of changes' '
 	git checkout -f -B base &&
 	git checkout -B to-rebase &&
 	git checkout -B upstream &&
-	for i in $(test_seq 100)
-	do
-		# simulate huge diffs
-		echo change$i >unrelated-file$i &&
-		test_seq 1000 >>unrelated-file$i &&
-		git add unrelated-file$i &&
-		test_tick &&
-		git commit -m commit$i unrelated-file$i &&
-		echo change$i >unrelated-file$i &&
-		test_seq 1000 | sort -nr >>unrelated-file$i &&
-		git add unrelated-file$i &&
-		test_tick &&
-		git commit -m commit$i-reverse unrelated-file$i ||
-		return 1
-	done &&
+
+	test_seq 1000 >content_fwd &&
+	test_seq 1000 | sort -nr >content_rev &&
+
+	(
+		for i in $(test_seq 100)
+		do
+			echo "commit refs/heads/upstream" &&
+			echo "committer WGYDY <author@mock.com> $i +0000" &&
+			echo "data <<EOF" &&
+			echo "commit$i" &&
+			echo "EOF" &&
+			
+			if test "$i" = 1; then
+				echo "from refs/heads/upstream^0"
+			fi &&
+
+			echo "M 100644 inline unrelated-file$i" &&
+			echo "data <<EOF" &&
+			echo "change$i" &&
+			cat content_fwd &&
+			echo "EOF" &&
+
+			echo "commit refs/heads/upstream" &&
+			echo "committer WGYDY <author@mock.com> $i +0000" &&
+			echo "data <<EOF" &&
+			echo "commit$i-reversed" &&
+			echo "EOF" &&
+			echo "M 100644 inline unrelated-file$i" &&
+			echo "data <<EOF" &&
+			echo "change$i" &&
+			cat content_rev &&
+			echo "EOF" || return 1
+		done
+	) | git fast-import &&
+	
+	git checkout -f upstream &&
 	git checkout to-rebase &&
 	test_commit our-patch interesting-file
 '
-
 test_perf 'rebase on top of a lot of unrelated changes' '
 	git rebase --onto upstream HEAD^ &&
 	git rebase --onto base HEAD^
-- 
2.43.0

Re: [PATCH V1][RFC] t/perf/p3400: speed up setup using fast-import

[Tian Yuchen]

I noticed that while the setup phase is much faster (as shown in the commit
message), the actual rebase performance test regressed slightly by about 3
seconds (from ~2s to ~5s). I don't quite understand what led to this outcome.

Additionally, I haven't delved deeply into the technical details of
`git fast-import`,
and I'm not entirely sure if my approach is reasonable and compliant with the
specifications.

I would greatly appreciate any guidance from those knowledgeable in these
area!

Regards,
Yuchen

Re: [PATCH V1][RFC] t/perf/p3400: speed up setup using fast-import

[Tian Yuchen]

Tian Yuchen <a3205153416@gmail.com> writes：
>
> I noticed that while the setup phase is much faster (as shown in the commit
> message), the actual rebase performance test regressed slightly by about 3
> seconds (from ~2s to ~5s). I don't quite understand what led to this outcome.
>
> Additionally, I haven't delved deeply into the technical details of
> `git fast-import`,
> and I'm not entirely sure if my approach is reasonable and compliant with the
> specifications.
>
> I would greatly appreciate any guidance from those knowledgeable in these
> area!
>
> Regards,
> Yuchen

I highly suspect that git fast-import generate the packfile that is NOT delta
compressed. According to git fast-import documentation:

>...For this reason it is strongly recommended that users repack the repository with
>'git repack -a -d' after fast-import completes, allowing Git to reorganize the packfiles
>for faster data access...

Could this be the reason why it's even slower than handling “loose
objects”? I'm working
on it currently.

Regards,
Yuchen

Re: [PATCH V1][RFC] t/perf/p3400: speed up setup using fast-import

[Tian Yuchen]

>...For this reason it is strongly recommended that users repack the repository with
>'git repack -a -d' after fast-import completes, allowing Git to reorganize the packfiles
>for faster data access...

After adding 'git repack -a -d', the time does changed:
                original(no change)        patch v1          NEW
     real           29.845                       18.081           22.231
rebase            13.34                         14.94            12.78

It seems that the 'git repack -a -d' line does reduce runtime.
However, the total duration
has seen a noticeable performance regression. I'm still trying to figure it out.

With regards,
Yuchen

[PATCH v2] t/perf/p3400: speed up setup using fast-import

[Tian Yuchen]

The setup phase in 't/perf/p3400-rebase.sh' generates 100 commits to
simulate a noisy history. It currently uses a shell loop that invokes
'git add', 'git commit', 'test_seq', and 'sort' in each iteration.
This incurs significant overhead due to repeated process spawning.

Optimize the setup by using 'git fast-import' to generate the commit
history in a single stream. Additionally, pre-compute the forward and
reversed file contents to avoid repetitive execution of 'seq' and 'sort'.

To ensure the test measures rebase performance against a consistent
object layout (rather than the suboptimal pack/loose objects created
by the raw import), perform a full repack (`git repack -a -d`) at the
end of the setup.

This reduces the setup time significantly while maintaining the validity
of the subsequent performance tests.

Performance enhancement:
          Real         Rebase
  Before: 29.045s      13.34s
  After:  22.231s      12.78s

Measured on Lenovo Yoga 2020, Ubuntu 24.04.

Signed-off-by: Tian Yuchen <a3205153416@gmail.com>
---

Changes since v1:
- Added 'git repack -a -d' at the end of the setup phase.
- This fixes a performance regression observed in the subsequent rebase
  test, ensuring the object layout is normalized before testing.
- Updated setup timing in the commit message to reflect the repack overhead.

 t/perf/p3400-rebase.sh | 54 +++++++++++++++++++++++++++++-------------
 1 file changed, 38 insertions(+), 16 deletions(-)

diff --git a/t/perf/p3400-rebase.sh b/t/perf/p3400-rebase.sh
index e6b0277729..9f4251aed6 100755
--- a/t/perf/p3400-rebase.sh
+++ b/t/perf/p3400-rebase.sh
@@ -9,25 +9,47 @@ test_expect_success 'setup rebasing on top of a lot of changes' '
 	git checkout -f -B base &&
 	git checkout -B to-rebase &&
 	git checkout -B upstream &&
-	for i in $(test_seq 100)
-	do
-		# simulate huge diffs
-		echo change$i >unrelated-file$i &&
-		test_seq 1000 >>unrelated-file$i &&
-		git add unrelated-file$i &&
-		test_tick &&
-		git commit -m commit$i unrelated-file$i &&
-		echo change$i >unrelated-file$i &&
-		test_seq 1000 | sort -nr >>unrelated-file$i &&
-		git add unrelated-file$i &&
-		test_tick &&
-		git commit -m commit$i-reverse unrelated-file$i ||
-		return 1
-	done &&
+
+	test_seq 1000 >content_fwd &&
+	test_seq 1000 | sort -nr >content_rev &&
+
+	(
+		for i in $(test_seq 100)
+		do
+			echo "commit refs/heads/upstream" &&
+			echo "committer WGYDY <author@mock.com> $i +0000" &&
+			echo "data <<EOF" &&
+			echo "commit$i" &&
+			echo "EOF" &&
+			
+			if test "$i" = 1; then
+				echo "from refs/heads/upstream^0"
+			fi &&
+
+			echo "M 100644 inline unrelated-file$i" &&
+			echo "data <<EOF" &&
+			echo "change$i" &&
+			cat content_fwd &&
+			echo "EOF" &&
+
+			echo "commit refs/heads/upstream" &&
+			echo "committer WGYDY <author@mock.com> $i +0000" &&
+			echo "data <<EOF" &&
+			echo "commit$i-reversed" &&
+			echo "EOF" &&
+			echo "M 100644 inline unrelated-file$i" &&
+			echo "data <<EOF" &&
+			echo "change$i" &&
+			cat content_rev &&
+			echo "EOF" || return 1
+		done
+	) | git fast-import &&
+
+	git repack -a -d &&
+	git checkout -f upstream &&
 	git checkout to-rebase &&
 	test_commit our-patch interesting-file
 '
-
 test_perf 'rebase on top of a lot of unrelated changes' '
 	git rebase --onto upstream HEAD^ &&
 	git rebase --onto base HEAD^
-- 
2.43.0

Re: [PATCH v2] t/perf/p3400: speed up setup using fast-import

[Johannes Sixt]

Am 28.01.26 um 17:07 schrieb Tian Yuchen:
> +	test_seq 1000 >content_fwd &&
> +	test_seq 1000 | sort -nr >content_rev &&

Remove another pipe with

	sort -nr content_fwd >content_rev &&

> +
> +	(
> +		for i in $(test_seq 100)
> +		do
> +			...
> +			cat content_rev &&
> +			echo "EOF" || return 1

This is an incorrect use of `return` outside a function. This must be
`|| exit 1` (or just `|| exit`) to actually break out of the loop.

Of course, the error code is ignored, because it is in the upstream of a
pipe, so in order to fail the complete command, it would be necessary
that the original error left an incorrect fast-import stream such that
the down-stream `git fast-import` fails. Otherwise, the entire command
can complete successfully in an unexpected way.

Maybe instead of a pipe, dump the stream into a temporary file, so that
the loop's exit code can be captured?

> +		done
> +	) | git fast-import &&
-- Hannes

Re: [PATCH v2] t/perf/p3400: speed up setup using fast-import

[Johannes Sixt]

Am 30.01.26 um 07:41 schrieb Johannes Sixt:
> Am 28.01.26 um 17:07 schrieb Tian Yuchen:
>> +
>> +	(
>> +		for i in $(test_seq 100)
>> +		do
>> +			...
>> +			cat content_rev &&
>> +			echo "EOF" || return 1
> 
> This is an incorrect use of `return` outside a function. This must be
> `|| exit 1` (or just `|| exit`) to actually break out of the loop.

I am wrong here, because my earlier tests were too narrow. This use of
`return` actually does work as intended, because it happens inside a
function. Neither the subshell nor being in the upstream of a pipe
negates this fact.

The rest of the comment still stands, though.

> Of course, the error code is ignored, because it is in the upstream of a
> pipe, so in order to fail the complete command, it would be necessary
> that the original error left an incorrect fast-import stream such that
> the down-stream `git fast-import` fails. Otherwise, the entire command
> can complete successfully in an unexpected way.
> 
> Maybe instead of a pipe, dump the stream into a temporary file, so that
> the loop's exit code can be captured?
> 
>> +		done
>> +	) | git fast-import &&
-- Hannes

Re: [PATCH v2] t/perf/p3400: speed up setup using fast-import

[Junio C Hamano]

Johannes Sixt <j6t@kdbg.org> writes:

> Am 28.01.26 um 17:07 schrieb Tian Yuchen:
>> +	test_seq 1000 >content_fwd &&
>> +	test_seq 1000 | sort -nr >content_rev &&
>
> Remove another pipe with
>
> 	sort -nr content_fwd >content_rev &&

I agree with the outcome (i.e., we do not run test_seq twice) but
not necessarily with the motivation (i.e., we do not assume test_seq
may fail and care about its failure code, so no need to avoid pipes
for the sake of avoiding pipes).

>> +	(
>> +		for i in $(test_seq 100)
>> +		do
>> +			...
>> +			cat content_rev &&
>> +			echo "EOF" || return 1
>
> This is an incorrect use of `return` outside a function. This must be
> `|| exit 1` (or just `|| exit`) to actually break out of the loop.

Good.

> Of course, the error code is ignored, because it is in the upstream of a
> pipe, so in order to fail the complete command, it would be necessary
> that the original error left an incorrect fast-import stream such that
> the down-stream `git fast-import` fails. Otherwise, the entire command
> can complete successfully in an unexpected way.
>
> Maybe instead of a pipe, dump the stream into a temporary file, so that
> the loop's exit code can be captured?

I like this suggestion.  A temporary file, unless it is excessively
large, has an additional advantage that it allows you to inspect
after "git fast-import" dies on it.

>> +		done
>> +	) | git fast-import &&
> -- Hannes

Re: [PATCH v2] t/perf/p3400: speed up setup using fast-import

[Johannes Sixt]

Am 30.01.26 um 17:39 schrieb Tian Yuchen:
> I will send V4 shortly.
In this case, please also undo the unnecessary removal of a blank line
at the end of the patch.

-- Hannes

Re: [PATCH v2] t/perf/p3400: speed up setup using fast-import

[Phillip Wood]

On 28/01/2026 16:07, Tian Yuchen wrote:
> The setup phase in 't/perf/p3400-rebase.sh' generates 100 commits to
> simulate a noisy history. It currently uses a shell loop that invokes
> 'git add', 'git commit', 'test_seq', and 'sort' in each iteration.
> This incurs significant overhead due to repeated process spawning.
> 
> Optimize the setup by using 'git fast-import' to generate the commit
> history in a single stream. Additionally, pre-compute the forward and
> reversed file contents to avoid repetitive execution of 'seq' and 'sort'.
> 
> To ensure the test measures rebase performance against a consistent
> object layout (rather than the suboptimal pack/loose objects created
> by the raw import), perform a full repack (`git repack -a -d`) at the
> end of the setup.
> 
> This reduces the setup time significantly while maintaining the validity
> of the subsequent performance tests.
> 
> Performance enhancement:
>            Real         Rebase
>    Before: 29.045s      13.34s
>    After:  22.231s      12.78s

That's a nice speedup in the test setup

> diff --git a/t/perf/p3400-rebase.sh b/t/perf/p3400-rebase.sh
> index e6b0277729..9f4251aed6 100755
> --- a/t/perf/p3400-rebase.sh
> +++ b/t/perf/p3400-rebase.sh
> @@ -9,25 +9,47 @@ test_expect_success 'setup rebasing on top of a lot of changes' '
>   	git checkout -f -B base &&
>   	git checkout -B to-rebase &&
>   	git checkout -B upstream &&
> -	for i in $(test_seq 100)
> -	do
> -		# simulate huge diffs
> -		echo change$i >unrelated-file$i &&
> -		test_seq 1000 >>unrelated-file$i &&
> -		git add unrelated-file$i &&
> -		test_tick &&
> -		git commit -m commit$i unrelated-file$i &&
> -		echo change$i >unrelated-file$i &&
> -		test_seq 1000 | sort -nr >>unrelated-file$i &&
> -		git add unrelated-file$i &&
> -		test_tick &&
> -		git commit -m commit$i-reverse unrelated-file$i ||
> -		return 1
> -	done &&
> +
> +	test_seq 1000 >content_fwd &&
> +	test_seq 1000 | sort -nr >content_rev &&
> +
> +	(
> +		for i in $(test_seq 100)
> +		do
> +			echo "commit refs/heads/upstream" &&
> +			echo "committer WGYDY <author@mock.com> $i +0000" &&

You can keep the same author and committer as the original with

	test_tick &&
	echo "author $GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL> $GIT_AUTHOR_DATE" &&
	echo "committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 
$GIT_COMMITTER_DATE" &&

here and below

> +			echo "data <<EOF" &&
> +			echo "commit$i" &&
> +			echo "EOF" &&
> +			
> +			if test "$i" = 1; then
> +				echo "from refs/heads/upstream^0"
> +			fi &&
> +
> +			echo "M 100644 inline unrelated-file$i" &&
> +			echo "data <<EOF" &&
> +			echo "change$i" &&
> +			cat content_fwd &&
> +			echo "EOF" &&
> +
> +			echo "commit refs/heads/upstream" &&
> +			echo "committer WGYDY <author@mock.com> $i +0000" &&
> +			echo "data <<EOF" &&
> +			echo "commit$i-reversed" &&

The commit message in the original is "commit$i-reverse", not "reversed"

> +			echo "EOF" &&
> +			echo "M 100644 inline unrelated-file$i" &&
> +			echo "data <<EOF" &&
> +			echo "change$i" &&
> +			cat content_rev &&
> +			echo "EOF" || return 1
> +		done

As Johannes pointed out we'll ignore the any failure above. We can 
address that by adding "echo done" here and adding "--done" to "git 
fast-import" below. That will cause "git fast-import" to fail because if 
there is an error in the loop as the last line of input to fast-import 
will not be "done"

Thanks

Phillip

> +	) | git fast-import &&
> +
> +	git repack -a -d &&
> +	git checkout -f upstream &&
>   	git checkout to-rebase &&
>   	test_commit our-patch interesting-file
>   '
> -
>   test_perf 'rebase on top of a lot of unrelated changes' '
>   	git rebase --onto upstream HEAD^ &&
>   	git rebase --onto base HEAD^

Re: [PATCH v2] t/perf/p3400: speed up setup using fast-import

[Tian Yuchen]

Hi Johannes, Hi Phillip,

Thanks a lot for the detailed review and the great suggestions!

@Johannes: I will adopt the 'sort -nr content_fwd' optimization to
save a process.

@Phillip: Your point about the pipe swallowing the exit code is very
insightful. I will
 implement the '--done' protocol to ensure any failure in the loop
causes the test to fail.

I will also update the author/committer generation to use test_tick
and the standard
environment variables, and fix the "reversed" typo.

I will prepare and send a v3 patch shortly incorporating these changes.

Thanks,
Tian Yuchen

[PATCH v3] t/perf/p3400: speed up setup using fast-import

[Tian Yuchen]

The setup phase in 't/perf/p3400-rebase.sh' generates 100 commits to
simulate a noisy history. It currently uses a shell loop that invokes
'git add', 'git commit', 'test_seq', and 'sort' in each iteration.
This incurs significant overhead due to repeated process spawning.

Optimize the setup by using 'git fast-import' to generate the commit
history in a single stream. Additionally, pre-compute the forward and
reversed file contents to avoid repetitive execution of 'seq' and 'sort'.

To ensure the test measures rebase performance against a consistent
object layout (rather than the suboptimal pack/loose objects created
by the raw import), perform a full repack (`git repack -a -d`) at the
end of the setup.

This reduces the setup time significantly while maintaining the validity
of the subsequent performance tests.

Performance enhancement (Average value of 5 tests):
           Real        Rebase
  Before: 29.045s      13.34s
  After:  22.431s      12.98s

Measured on Lenovo Yoga 2020, Ubuntu 24.04.

Signed-off-by: Tian Yuchen <a3205153416@gmail.com>
---

Changes since v2:
- Optimized `content_rev` generation by sorting `content_fwd` directly (Johannes Sixt).
- Used `test_tick` and standard `$GIT_COMMITTER_*` variables for consistency (Phillip Wood).
- Fixed typo "reversed" -> "reverse" to match original test (Phillip Wood).
- Added "done" command and used `git fast-import --done` to prevent errors in the loop being ignored by the pipe (Phillip Wood).

 t/perf/p3400-rebase.sh | 56 ++++++++++++++++++++++++++++++------------
 1 file changed, 40 insertions(+), 16 deletions(-)

diff --git a/t/perf/p3400-rebase.sh b/t/perf/p3400-rebase.sh
index e6b0277729..6bb58282d6 100755
--- a/t/perf/p3400-rebase.sh
+++ b/t/perf/p3400-rebase.sh
@@ -9,25 +9,49 @@ test_expect_success 'setup rebasing on top of a lot of changes' '
 	git checkout -f -B base &&
 	git checkout -B to-rebase &&
 	git checkout -B upstream &&
-	for i in $(test_seq 100)
-	do
-		# simulate huge diffs
-		echo change$i >unrelated-file$i &&
-		test_seq 1000 >>unrelated-file$i &&
-		git add unrelated-file$i &&
-		test_tick &&
-		git commit -m commit$i unrelated-file$i &&
-		echo change$i >unrelated-file$i &&
-		test_seq 1000 | sort -nr >>unrelated-file$i &&
-		git add unrelated-file$i &&
-		test_tick &&
-		git commit -m commit$i-reverse unrelated-file$i ||
-		return 1
-	done &&
+
+	test_seq 1000 >content_fwd &&
+	sort -nr content_fwd >content_rev &&
+
+	(
+		for i in $(test_seq 100)
+		do
+			test_tick &&
+			echo "commit refs/heads/upstream" &&
+			echo "committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE" &&
+			echo "data <<EOF" &&
+			echo "commit$i" &&
+			echo "EOF" &&
+			
+			if test "$i" = 1; then
+				echo "from refs/heads/upstream^0"
+			fi &&
+
+			echo "M 100644 inline unrelated-file$i" &&
+			echo "data <<EOF" &&
+			echo "change$i" &&
+			cat content_fwd &&
+			echo "EOF" &&
+
+			echo "commit refs/heads/upstream" &&
+			echo "committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE" &&
+			echo "data <<EOF" &&
+			echo "commit$i-reverse" &&
+			echo "EOF" &&
+			echo "M 100644 inline unrelated-file$i" &&
+			echo "data <<EOF" &&
+			echo "change$i" &&
+			cat content_rev &&
+			echo "EOF" || return 1
+		done &&
+		echo "done"
+	) | git fast-import --done &&
+
+	git repack -a -d &&
+	git checkout -f upstream &&
 	git checkout to-rebase &&
 	test_commit our-patch interesting-file
 '
-
 test_perf 'rebase on top of a lot of unrelated changes' '
 	git rebase --onto upstream HEAD^ &&
 	git rebase --onto base HEAD^
-- 
2.43.0

[PATCH v4] t/perf/p3400: speed up setup using fast-import

[Tian Yuchen]

The setup phase in 't/perf/p3400-rebase.sh' generates 100 commits to
simulate a noisy history. It currently uses a shell loop that invokes
'git add', 'git commit', 'test_seq', and 'sort' in each iteration.
This incurs significant overhead due to repeated process spawning.

Optimize the setup by using 'git fast-import' to generate the commit
history. Additionally, pre-compute the forward and reversed file contents
to avoid repetitive execution of 'seq' and 'sort'.

To ensure the test measures rebase performance against a consistent
object layout (rather than the suboptimal pack/loose objects created
by the raw import), perform a full repack (`git repack -a -d`) at the
end of the setup.

This reduces the setup time significantly while maintaining the validity
of the subsequent performance tests.

Performance enhancement (Average value of 5 tests):
            Real        Rebase
  Before:  29.045s      13.34s
   After:  21.989s      12.84s

Measured on Lenovo Yoga 2020, Ubuntu 24.04.

Signed-off-by: Tian Yuchen <a3205153416@gmail.com>
---

Changes since v3:
- Refactored the fast-import generation to write to a temporary file 
  (`fast_import_stream`) instead of using a pipe. This allows for better 
  debugging if fast-import fails, as suggested by Johannes Sixt and 
  endorsed by Junio C Hamano.
- Switched from `return 1` to `exit 1` inside the subshell to correctly 
  propagate exit codes.
- Undid the unnecessary removal of a blank line.

 t/perf/p3400-rebase.sh | 53 ++++++++++++++++++++++++++++++------------
 1 file changed, 38 insertions(+), 15 deletions(-)

diff --git a/t/perf/p3400-rebase.sh b/t/perf/p3400-rebase.sh
index e6b0277729..425f484681 100755
--- a/t/perf/p3400-rebase.sh
+++ b/t/perf/p3400-rebase.sh
@@ -9,21 +9,44 @@ test_expect_success 'setup rebasing on top of a lot of changes' '
 	git checkout -f -B base &&
 	git checkout -B to-rebase &&
 	git checkout -B upstream &&
-	for i in $(test_seq 100)
-	do
-		# simulate huge diffs
-		echo change$i >unrelated-file$i &&
-		test_seq 1000 >>unrelated-file$i &&
-		git add unrelated-file$i &&
-		test_tick &&
-		git commit -m commit$i unrelated-file$i &&
-		echo change$i >unrelated-file$i &&
-		test_seq 1000 | sort -nr >>unrelated-file$i &&
-		git add unrelated-file$i &&
-		test_tick &&
-		git commit -m commit$i-reverse unrelated-file$i ||
-		return 1
-	done &&
+	test_seq 1000 >content_fwd &&
+	sort -nr content_fwd >content_rev &&
+	(
+		for i in $(test_seq 100)
+		do
+			test_tick &&
+			echo "commit refs/heads/upstream" &&
+			echo "committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE" &&
+			echo "data <<EOF" &&
+			echo "commit$i" &&
+			echo "EOF" &&
+			
+			if test "$i" = 1; then
+				echo "from refs/heads/upstream^0"
+			fi &&
+
+			echo "M 100644 inline unrelated-file$i" &&
+			echo "data <<EOF" &&
+			echo "change$i" &&
+			cat content_fwd &&
+			echo "EOF" &&
+
+			echo "commit refs/heads/upstream" &&
+			echo "committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE" &&
+			echo "data <<EOF" &&
+			echo "commit$i-reverse" &&
+			echo "EOF" &&
+			echo "M 100644 inline unrelated-file$i" &&
+			echo "data <<EOF" &&
+			echo "change$i" &&
+			cat content_rev &&
+			echo "EOF" || exit 1
+		done
+	) >fast_import_stream &&
+
+	git fast-import <fast_import_stream &&
+	git repack -a -d &&
+	git checkout -f upstream &&
 	git checkout to-rebase &&
 	test_commit our-patch interesting-file
 '
-- 
2.43.0

Re: [PATCH v3] t/perf/p3400: speed up setup using fast-import

[Junio C Hamano]

Tian Yuchen <a3205153416@gmail.com> writes:

> Changes since v2:
> - Optimized `content_rev` generation by sorting `content_fwd` directly (Johannes Sixt).
> - Used `test_tick` and standard `$GIT_COMMITTER_*` variables for consistency (Phillip Wood).
> - Fixed typo "reversed" -> "reverse" to match original test (Phillip Wood).
> - Added "done" command and used `git fast-import --done` to prevent errors in the loop being ignored by the pipe (Phillip Wood).

Thanks.  Will replace.