Re: [PATCH v2 3/3] grep: prefetch necessary blobs

[Derrick Stolee <stolee@gmail.com>]

On 4/17/2026 8:32 PM, Elijah Newren via GitGitGadget wrote:
> From: Elijah Newren <newren@gmail.com>
> 
> In partial clones, `git grep` fetches necessary blobs on-demand one
> at a time, which can be very slow.  In partial clones, add an extra
> preliminary walk over the tree similar to grep_tree() which collects
> the blobs of interest, and then prefetches them.

A log of the code is about walking trees to find blobs matching
the input pathspec, with this being the core method:

> +static void collect_blob_oids_for_tree(struct repository *repo,
> +				       const struct pathspec *pathspec,
> +				       struct tree_desc *tree,
> +				       struct strbuf *base,
> +				       int tn_len,
> +				       struct oidset *blob_oids)

And in your test, you set up a repo to have three blobs with
matches in two of the files:

> +test_expect_success 'grep of revision in partial clone does bulk prefetch' '
> +	test_when_finished "rm -rf grep-partial-src grep-partial" &&
> +
> +	git init grep-partial-src &&
> +	(
> +		cd grep-partial-src &&
> +		git config uploadpack.allowfilter 1 &&
> +		git config uploadpack.allowanysha1inwant 1 &&
> +		echo "needle in haystack" >searchme &&
> +		echo "no match here" >other &&
> +		mkdir subdir &&
> +		echo "needle again" >subdir/deep &&
> +		git add . &&
> +		git commit -m "initial"
> +	) &&

But then the command downloads all of the blobs, not using a
pathspec:

> +	# grep HEAD should batch-prefetch all blobs in one request.
> +	GIT_TRACE2_EVENT="$(pwd)/grep-trace" \
> +		git -C grep-partial grep -c "needle" HEAD >result &&
> +
> +	# Should find matches in two files.
> +	test_line_count = 2 result &&
> +
> +	# Should have prefetched all 3 objects at once
> +	test_trace2_data promisor fetch_count 3 <grep-trace
> +'
I think your code is correct, but I'd like to see a test
here that demonstrates a pathspec filter on the 'grep'
command to help filter out a blob that has a matching string.

Perhaps something like:

* matches.txt (has needle)
* nomatch.txt (does not have needle)
* matches.md (has needle)

and then 'git grep -c "needle" HEAD -- *.txt' would
download two blobs and find one match. A second run without
the pathspec would download one blob and find two matches.

Does that make sense as a test?

Thanks,
-Stolee