[PATCH v6 0/3] object checking related additions and fixes for bundles in fetches

["blanet via GitGitGadget" <gitgitgadget@gmail.com>]

While attempting to fix a reference negotiation bug in bundle-uri, we
identified that the fetch process lacks some crucial object validation
checks when processing bundles. The primary issues are:

 1. In the bundle-uri scenario, object IDs were not validated before writing
    bundle references. This was the root cause of the original negotiation
    bug in bundle-uri and could lead to potential repository corruption.
 2. The existing "fetch.fsckObjects" and "transfer.fsckObjects"
    configurations were not applied when directly fetching bundles or
    fetching with bundle-uri enabled. In fact, there were no object
    validation supports for unbundle.

The first patch addresses the bundle-uri negotiation issue by removing the
REF_SKIP_OID_VERIFICATION flag when writing bundle references.

Patches 2 through 3 extend verify_bundle_flags for bundle.c:unbundle to add
support for object validation (fsck) in fetch scenarios, mainly following
the suggestions from Junio and Patrick on the mailing list.

Xing Xin (3):
  bundle-uri: verify oid before writing refs
  fetch-pack: expose fsckObjects configuration logic
  unbundle: support object verification for fetches

 bundle-uri.c                |   5 +-
 bundle.c                    |   5 +
 bundle.h                    |   1 +
 fetch-pack.c                |  17 ++--
 fetch-pack.h                |   5 +
 t/t5558-clone-bundle-uri.sh | 186 +++++++++++++++++++++++++++++++++++-
 t/t5607-clone-bundle.sh     |  33 +++++++
 transport.c                 |   2 +-
 8 files changed, 240 insertions(+), 14 deletions(-)


base-commit: b9cfe4845cb2562584837bc0101c0ab76490a239
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1730%2Fblanet%2Fxx%2Fbundle-uri-bug-using-bundle-list-v6
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1730/blanet/xx/bundle-uri-bug-using-bundle-list-v6
Pull-Request: https://github.com/gitgitgadget/git/pull/1730

Range-diff vs v5:

 1:  e958a3ab20c = 1:  e958a3ab20c bundle-uri: verify oid before writing refs
 2:  d21c236b8de = 2:  d21c236b8de fetch-pack: expose fsckObjects configuration logic
 3:  0a18d7839be < -:  ----------- unbundle: extend options to support object verification
 4:  eb9f21f16b5 ! 3:  53395e8c08a unbundle: use VERIFY_BUNDLE_FSCK_FOLLOW_FETCH for fetches
     @@ Metadata
      Author: Xing Xin <xingxin.xx@bytedance.com>
      
       ## Commit message ##
     -    unbundle: use VERIFY_BUNDLE_FSCK_FOLLOW_FETCH for fetches
     +    unbundle: support object verification for fetches
      
     -    This commit passes `VERIFY_BUNDLE_FSCK_FOLLOW_FETCH` to `unbundle` in
     -    the fetching process, including:
     +    This commit extends object verification support for fetches in
     +    `bundle.c:unbundle` by adding the `VERIFY_BUNDLE_FSCK_FOLLOW_FETCH`
     +    option to `verify_bundle_flags`. When this option is enabled,
     +    `bundle.c:unbundle` invokes `fetch-pack.c:fetch_pack_fsck_objects` to
     +    determine whether to append the "--fsck-objects" flag to
     +    "git-index-pack".
     +
     +    `VERIFY_BUNDLE_FSCK_FOLLOW_FETCH` is now passed to `unbundle` in the
     +    fetching process, including:
      
          - `transport.c:fetch_refs_from_bundle` for direct bundle fetches.
          - `bundle-uri.c:unbundle_from_file` for bundle-uri enabled fetches.
     @@ bundle-uri.c: static int unbundle_from_file(struct repository *r, const char *fi
       
       	/*
      
     + ## bundle.c ##
     +@@
     + #include "list-objects-filter-options.h"
     + #include "connected.h"
     + #include "write-or-die.h"
     ++#include "fetch-pack.h"
     + 
     + static const char v2_bundle_signature[] = "# v2 git bundle\n";
     + static const char v3_bundle_signature[] = "# v3 git bundle\n";
     +@@ bundle.c: int unbundle(struct repository *r, struct bundle_header *header,
     + 	if (header->filter.choice)
     + 		strvec_push(&ip.args, "--promisor=from-bundle");
     + 
     ++	if (flags & VERIFY_BUNDLE_FSCK_FOLLOW_FETCH)
     ++		if (fetch_pack_fsck_objects())
     ++			strvec_push(&ip.args, "--fsck-objects");
     ++
     + 	if (extra_index_pack_args) {
     + 		strvec_pushv(&ip.args, extra_index_pack_args->v);
     + 		strvec_clear(extra_index_pack_args);
     +
     + ## bundle.h ##
     +@@ bundle.h: int create_bundle(struct repository *r, const char *path,
     + enum verify_bundle_flags {
     + 	VERIFY_BUNDLE_VERBOSE = (1 << 0),
     + 	VERIFY_BUNDLE_QUIET = (1 << 1),
     ++	VERIFY_BUNDLE_FSCK_FOLLOW_FETCH = (1 << 2),
     + };
     + 
     + int verify_bundle(struct repository *r, struct bundle_header *header,
     +
       ## t/t5558-clone-bundle-uri.sh ##
      @@ t/t5558-clone-bundle-uri.sh: test_expect_success 'create bundle' '
       		git bundle create B.bundle topic &&

-- 
gitgitgadget