[PATCH 0/2] upload-pack.c: fix partial clone allowed filter regression

[PATCH 0/2] upload-pack.c: fix partial clone allowed filter regression

[Taylor Blau]

Hi,

Here's a regression fix that I noticed this morning while deploying 2.28
out to GitHub. The gist is that we accidentally call
'string_list_clear(..., 1)' when freeing the list of allowed/banned
object filters, but the ->util pointer either contains '0', or '1'. So,
the free() of the util pointer is bogus, and we end up crashing.

The fix in the second patch is straightforward, but it doesn't work
without the first patch, which teaches 'git clone' to match 'git fetch'
and not ignore an error from 'transport_fetch_refs()'.

Unfortunately, this regression has been in Git since 2.28. Fortunately,
nobody has seemed to notice, so I doubt that it's bitten anybody out in
the wild. But, it's still worth fixing, since it is so obviously broken.

Thanks,
Taylor

Taylor Blau (2):
  builtin/clone.c: don't ignore transport_fetch_refs() errors
  upload-pack.c: don't free allowed_filters util pointers

 builtin/clone.c          | 15 +++++++++++----
 t/t5616-partial-clone.sh | 10 +++++++++-
 upload-pack.c            |  2 +-
 3 files changed, 21 insertions(+), 6 deletions(-)

--
2.29.2.533.g07db1f5344

[PATCH 1/2] builtin/clone.c: don't ignore transport_fetch_refs() errors

[Taylor Blau]

If 'git clone' couldn't execute 'transport_fetch_refs()' (e.g., because
of an error on the remote's side in 'git upload-pack'), then it will
silently ignore it.

Even though this has been the case at least since clone was ported to C
(way back in 8434c2f1af (Build in clone, 2008-04-27)), 'git fetch'
doesn't ignore these and reports any failures it sees.

That suggests that ignoring the return value in 'git clone' is simply an
oversight that should be corrected. That's exactly what this patch does.
(Noticing and fixing this is no coincidence, we'll want it in the next
patch in order to demonstrate a regression in 'git upload-pack' via a
'git clone'.)

There's no additional logging here, but that matches how 'git fetch'
handles the same case. An assumption there is that whichever part of
transport_fetch_refs() fails will complain loudly, so any additional
logging here is redundant.

Co-authored-by: Jeff King <peff@peff.net>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Taylor Blau <me@ttaylorr.com>
---
 builtin/clone.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/builtin/clone.c b/builtin/clone.c
index a0841923cf..a5630337e4 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -1293,8 +1293,11 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 				break;
 			}
 
-		if (!is_local && !complete_refs_before_fetch)
-			transport_fetch_refs(transport, mapped_refs);
+		if (!is_local && !complete_refs_before_fetch) {
+			err = transport_fetch_refs(transport, mapped_refs);
+			if (err)
+				goto cleanup;
+		}
 
 		remote_head = find_ref_by_name(refs, "HEAD");
 		remote_head_points_at =
@@ -1339,8 +1342,11 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 
 	if (is_local)
 		clone_local(path, git_dir);
-	else if (refs && complete_refs_before_fetch)
-		transport_fetch_refs(transport, mapped_refs);
+	else if (refs && complete_refs_before_fetch) {
+		err = transport_fetch_refs(transport, mapped_refs);
+		if (err)
+			goto cleanup;
+	}
 
 	update_remote_refs(refs, mapped_refs, remote_head_points_at,
 			   branch_top.buf, reflog_msg.buf, transport,
@@ -1367,6 +1373,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 	junk_mode = JUNK_LEAVE_REPO;
 	err = checkout(submodule_progress);
 
+cleanup:
 	free(remote_name);
 	strbuf_release(&reflog_msg);
 	strbuf_release(&branch_top);
-- 
2.29.2.533.g07db1f5344

[PATCH 2/2] upload-pack.c: don't free allowed_filters util pointers

[Taylor Blau]

To keep track of which object filters are allowed or not, 'git
upload-pack' stores the name of each filter in a string_list, and sets
it ->util pointer to be either 0 or 1, indicating whether it is banned
or allowed.

Later on, we attempt to clear that list, but we incorrectly ask for the
util pointers to be free()'d, too. This behavior (introduced back in
6dd3456a8c (upload-pack.c: allow banning certain object filter(s),
2020-08-03)) leads to an invalid free, and causes us to crash.

In order to trigger this, one needs to fetch from a server that (a) has
at least one object filter allowed, and (b) issue a fetch that contains
a subset of the allowed filters (i.e., we cannot ask for a banned
filter, since this causes us to die() before we hit the bogus
string_list_clear()).

In that case, whatever banned filters exist will cause a noop free()
(since those ->util pointers are set to 0), but the first allowed filter
we try to free will crash us.

We never noticed this in the tests because we didn't have an example of
setting 'uploadPackFilter' configuration variables and then following up
with a valid fetch. The first new 'git clone' prevents further
regression here. For good measure on top, add a test which checks the
same behavior at a tree depth greater than 0.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
---
 t/t5616-partial-clone.sh | 10 +++++++++-
 upload-pack.c            |  2 +-
 2 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/t/t5616-partial-clone.sh b/t/t5616-partial-clone.sh
index f4d49d8335..5da945cf15 100755
--- a/t/t5616-partial-clone.sh
+++ b/t/t5616-partial-clone.sh
@@ -281,7 +281,15 @@ test_expect_success 'upload-pack limits tree depth filters' '
 	test_config -C srv.bare uploadpackfilter.tree.maxDepth 0 &&
 	test_must_fail ok=sigpipe git clone --no-checkout --filter=tree:1 \
 		"file://$(pwd)/srv.bare" pc3 2>err &&
-	test_i18ngrep "tree filter allows max depth 0, but got 1" err
+	test_i18ngrep "tree filter allows max depth 0, but got 1" err &&
+
+	git clone --no-checkout --filter=tree:0 "file://$(pwd)/srv.bare" pc4 &&
+
+	test_config -C srv.bare uploadpackfilter.tree.maxDepth 5 &&
+	git clone --no-checkout --filter=tree:5 "file://$(pwd)/srv.bare" pc5 &&
+	test_must_fail ok=sigpipe git clone --no-checkout --filter=tree:6 \
+		"file://$(pwd)/srv.bare" pc6 2>err &&
+	test_i18ngrep "tree filter allows max depth 5, but got 6" err
 '
 
 test_expect_success 'partial clone fetches blobs pointed to by refs even if normally filtered out' '
diff --git a/upload-pack.c b/upload-pack.c
index 5dc8e1f844..d89c7e4a02 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -154,7 +154,7 @@ static void upload_pack_data_clear(struct upload_pack_data *data)
 	string_list_clear(&data->deepen_not, 0);
 	object_array_clear(&data->extra_edge_obj);
 	list_objects_filter_release(&data->filter_options);
-	string_list_clear(&data->allowed_filters, 1);
+	string_list_clear(&data->allowed_filters, 0);
 
 	free((char *)data->pack_objects_hook);
 }
-- 
2.29.2.533.g07db1f5344

Re: [PATCH 2/2] upload-pack.c: don't free allowed_filters util pointers

[Jeff King]

On Thu, Dec 03, 2020 at 01:55:18PM -0500, Taylor Blau wrote:

> We never noticed this in the tests because we didn't have an example of
> setting 'uploadPackFilter' configuration variables and then following up
> with a valid fetch. The first new 'git clone' prevents further
> regression here. For good measure on top, add a test which checks the
> same behavior at a tree depth greater than 0.

Very nice. The fix is obviously the right thing, but I like the
improvements to the tests here especially.

-Peff