Re: [PATCH v2 0/4] upload-pack: custom allowed object filters

[Taylor Blau <me@ttaylorr.com>]

On Fri, Jul 24, 2020 at 03:51:26PM -0400, Jeff King wrote:
> On Fri, Jul 24, 2020 at 12:51:33PM -0400, Taylor Blau wrote:
>
> > Let me double check my understanding... I think that you are suggesting
> > the following three things:
> >
> >   - Write the same message as an err packet over the wire as we do when
> >     'die()'ing from inside of upload-pack.c
> >
> >   - Don't mark said message(s) for translation, matching what we do in
> >     the rest of upload-pack.c.
> >
> >   - Re-introduce the 'test_must_fail ok=sigpipe' and stop grepping
> >     stderr for the right message.
> >
> > Do I have that right?
>
> I'm not Gábor, but that is the sequence I think is best.
>
> Between the die() and the ERR, the ERR packets are way more useful in
> practice, since they actually go back to the client. I'd even suggest we
> do away with the die() messages entirely (since they're either redundant
> or go nowhere, depending on the protocol), but I think it would make
> sense to wait until the raciness issues are fixed (until then, they
> _might_ help in the redundant cases, which is what the test here is
> relying on).

All sounds good. I'll wait for an ACK from Gábor to make sure that this
is the direction that they want, too.

In the meantime, this is what I have locally. I'll send a series which
has this as its range-diff if Gábor approves of the direction.

1:  6a77af563e = 1:  6a77af563e list_objects_filter_options: introduce 'list_object_filter_config_name'
2:  6dbf58441d ! 2:  80d19481d8 upload-pack.c: allow banning certain object filter(s)
    @@ t/t5616-partial-clone.sh: test_expect_success 'implicitly construct combine: fil

     +test_expect_success 'upload-pack fails banned object filters' '
     +	test_config -C srv.bare uploadpackfilter.blob:none.allow false &&
    -+	test_must_fail git clone --no-checkout --filter=blob:none \
    -+		"file://$(pwd)/srv.bare" pc3 2>err &&
    -+	test_i18ngrep "filter '\''blob:none'\'' not supported" err
    ++	test_must_fail ok=sigpipe git clone --no-checkout --filter=blob:none \
    ++		"file://$(pwd)/srv.bare" pc3
     +'
     +
     +test_expect_success 'upload-pack fails banned combine object filters' '
    @@ t/t5616-partial-clone.sh: test_expect_success 'implicitly construct combine: fil
     +	test_config -C srv.bare uploadpackfilter.combine.allow true &&
     +	test_config -C srv.bare uploadpackfilter.tree.allow true &&
     +	test_config -C srv.bare uploadpackfilter.blob:none.allow false &&
    -+	test_must_fail git clone --no-checkout --filter=tree:1 \
    -+		--filter=blob:none "file://$(pwd)/srv.bare" pc3 2>err &&
    -+	test_i18ngrep "filter '\''blob:none'\'' not supported" err
    ++	test_must_fail ok=sigpipe git clone --no-checkout --filter=tree:1 \
    ++		--filter=blob:none "file://$(pwd)/srv.bare" pc3
     +'
     +
     +test_expect_success 'upload-pack fails banned object filters with fallback' '
     +	test_config -C srv.bare uploadpackfilter.allow false &&
    -+	test_must_fail git clone --no-checkout --filter=blob:none \
    -+		"file://$(pwd)/srv.bare" pc3 2>err &&
    -+	test_i18ngrep "filter '\''blob:none'\'' not supported" err
    ++	test_must_fail ok=sigpipe git clone --no-checkout --filter=blob:none \
    ++		"file://$(pwd)/srv.bare" pc3
     +'
     +
      test_expect_success 'partial clone fetches blobs pointed to by refs even if normally filtered out' '
    @@ upload-pack.c: static int process_deepen_not(const char *line, struct string_lis
     +{
     +	struct list_objects_filter_options *banned = banned_filter(data,
     +								   &data->filter_options);
    ++	struct strbuf buf = STRBUF_INIT;
     +	if (!banned)
     +		return;
     +
    -+	die(_("git upload-pack: filter '%s' not supported"),
    -+	    list_object_filter_config_name(banned->choice));
    ++	strbuf_addf(&buf, "git upload-pack: filter '%s' not supported",
    ++		    list_object_filter_config_name(banned->choice));
    ++
    ++	packet_writer_error(&data->writer, "%s\n", buf.buf);
    ++	die("%s", buf.buf);
     +}
     +
      static void receive_needs(struct upload_pack_data *data,
3:  bacdea47d9 = 3:  790552c7c2 upload-pack.c: pass 'struct list_objects_filter_options *'
4:  79af94a41b ! 4:  eb9a81eb1f upload-pack.c: introduce 'uploadpackfilter.tree.maxDepth'
    @@ Documentation/config/uploadpack.txt: uploadpackfilter.<filter>.allow::

      ## t/t5616-partial-clone.sh ##
     @@ t/t5616-partial-clone.sh: test_expect_success 'upload-pack fails banned object filters with fallback' '
    - 	test_i18ngrep "filter '\''blob:none'\'' not supported" err
    + 		"file://$(pwd)/srv.bare" pc3
      '

     +test_expect_success 'upload-pack limits tree depth filters' '
    @@ upload-pack.c: static int allows_filter_choice(struct upload_pack_data *data,
      }

     @@ upload-pack.c: static void die_if_using_banned_filter(struct upload_pack_data *data)
    - {
    - 	struct list_objects_filter_options *banned = banned_filter(data,
    - 								   &data->filter_options);
    -+	struct strbuf buf = STRBUF_INIT;
    - 	if (!banned)
    - 		return;

    --	die(_("git upload-pack: filter '%s' not supported"),
    --	    list_object_filter_config_name(banned->choice));
    -+	strbuf_addf(&buf, _("filter '%s' not supported"),
    -+		    list_object_filter_config_name(banned->choice));
    + 	strbuf_addf(&buf, "git upload-pack: filter '%s' not supported",
    + 		    list_object_filter_config_name(banned->choice));
     +	if (banned->choice == LOFC_TREE_DEPTH &&
     +	    data->tree_filter_max_depth != ULONG_MAX)
     +		strbuf_addf(&buf, _(" (maximum depth: %lu, but got: %lu)"),
     +			    data->tree_filter_max_depth,
     +			    banned->tree_exclude_depth);
    -+	die("%s", buf.buf);
    - }

    - static void receive_needs(struct upload_pack_data *data,
    + 	packet_writer_error(&data->writer, "%s\n", buf.buf);
    + 	die("%s", buf.buf);
     @@ upload-pack.c: static int parse_object_filter_config(const char *var, const char *value,
      	if (!strcmp(key, "allow"))
      		string_list_insert(&data->allowed_filters, buf.buf)->util =

> -Peff

Thanks,
Taylor