* [PATCH v3 0/2] fetch: make submodule fetch errors configurable
@ 2026-07-10 12:26 Paulius Zaleckas
2026-07-10 12:26 ` [PATCH v3 1/2] submodule: fix premature failure in recursive submodule fetch Paulius Zaleckas
` (2 more replies)
0 siblings, 3 replies; 10+ messages in thread
From: Paulius Zaleckas @ 2026-07-10 12:26 UTC (permalink / raw)
To: git; +Cc: Paulius Zaleckas
When fetching with --recurse-submodules, git currently exits with a
non-zero status if any submodule references an OID that is not reachable
from the submodule's remote. This situation arises naturally when an
upstream branch is still in preparation (e.g. a topic branch in a merge
window): the local branch does not depend on the missing commit, so a
hard failure is unnecessarily disruptive.
Patch 1 fixes a pre-existing NEEDSWORK in submodule.c where a phase-1
fetch failure was recorded immediately, even when a phase-2 OID-based
retry was about to be scheduled. After this fix the existing fatal
behaviour is preserved but the logic is now structured so that errors
are only recorded when the phase-2 retry actually fails, or when there
is no phase-2 retry to fall back on.
Patch 2 introduces fetch.submoduleErrors (fail|warn) and
--submodule-errors=(fail|warn) to let users opt into non-fatal
behaviour. The default remains fail for full backwards compatibility.
Changes in v3:
- Report a phase-1 failure also when the gitlink commits are already
present locally, instead of silently succeeding
- Route "Could not access submodule" through record_fetch_error() so it
shows up in the error summary and honors the warn mode
- Forward --submodule-errors to child fetches so it takes effect for
fetch --all/--multiple and nested submodule recursion
- Add tests for all of the above
- Documentation: don't imply git pull takes --submodule-errors, minor
wording and placement fixes
Changes in v2:
- Fix option synopsis to use (fail|warn) instead of <fail|warn>
- Add --submodule-errors documentation to Documentation/fetch-options.adoc
Paulius Zaleckas (2):
submodule: fix premature failure in recursive submodule fetch
fetch: add fetch.submoduleErrors to make submodule fetch errors
non-fatal
Documentation/config/fetch.adoc | 14 +++
Documentation/fetch-options.adoc | 8 ++
builtin/fetch.c | 41 ++++++++-
submodule.c | 58 ++++++++----
submodule.h | 7 +-
t/t5526-fetch-submodules.sh | 148 +++++++++++++++++++++++++++++++
6 files changed, 259 insertions(+), 17 deletions(-)
--
2.54.0
^ permalink raw reply [flat|nested] 10+ messages in thread* [PATCH v3 1/2] submodule: fix premature failure in recursive submodule fetch 2026-07-10 12:26 [PATCH v3 0/2] fetch: make submodule fetch errors configurable Paulius Zaleckas @ 2026-07-10 12:26 ` Paulius Zaleckas 2026-07-10 12:26 ` [PATCH v3 2/2] fetch: add fetch.submoduleErrors to make submodule fetch errors non-fatal Paulius Zaleckas 2026-07-14 13:29 ` [PATCH v4 0/2] fetch: make submodule fetch errors configurable Paulius Zaleckas 2 siblings, 0 replies; 10+ messages in thread From: Paulius Zaleckas @ 2026-07-10 12:26 UTC (permalink / raw) To: git Cc: Paulius Zaleckas, Jonathan Tan, Elijah Newren, Glen Choo, Patrick Steinhardt, Junio C Hamano When git fetch --recurse-submodules encounters a failure fetching a submodule's refs (phase 1), it immediately marks the overall operation as failed, even though a subsequent OID-based fetch (phase 2) is about to be attempted for any missing commits. If phase 2 succeeds, the overall result should be success, but the prematurely set failure flag makes it look like an error. Restructure fetch_finish() so that a phase-1 failure does not record an error immediately. Instead, the decision is deferred: - If missing commits trigger a phase-2 (OID-based) retry and that retry succeeds, no error is recorded. - If the phase-2 retry also fails, the error is recorded then. - If the submodule was fetched unconditionally (RECURSE_SUBMODULES_ON) and is not in the changed list, a phase-1 failure is recorded right away since there is no OID retry to fall back on. - If phase 1 fails but all required commits are already present locally, there is no retry to defer to; the failure is still recorded, since the fetch itself went wrong (e.g. a transport error) even though the wanted commits happen to be available. This resolves the NEEDSWORK comment added by bd5e567dc7 (submodule: explain first attempt failure clearly, 2019-03-13). Extract the common error-recording logic into a helper record_fetch_error() and use it in fetch_start_failure() and for the "Could not access submodule" error in get_fetch_task_from_index() as well; the latter now also lists the submodule in the final error summary. Add a test ensuring a failed submodule fetch is still reported when the gitlinked commits happen to be present locally. Signed-off-by: Paulius Zaleckas <paulius.zaleckas@gmail.com> --- submodule.c | 52 +++++++++++++++++++-------- t/t5526-fetch-submodules.sh | 72 +++++++++++++++++++++++++++++++++++++ 2 files changed, 110 insertions(+), 14 deletions(-) diff --git a/submodule.c b/submodule.c index fd91201a92..8bcef68a42 100644 --- a/submodule.c +++ b/submodule.c @@ -1562,6 +1562,13 @@ static struct fetch_task *fetch_task_create(struct submodule_parallel_fetch *spf return NULL; } +static void record_fetch_error(struct submodule_parallel_fetch *spf, + const char *name) +{ + spf->result = 1; + strbuf_addf(&spf->submodules_with_errors, "\t%s\n", name); +} + static struct fetch_task * get_fetch_task_from_index(struct submodule_parallel_fetch *spf, struct strbuf *err) @@ -1599,7 +1606,7 @@ get_fetch_task_from_index(struct submodule_parallel_fetch *spf, ce->name); if (S_ISGITLINK(ce->ce_mode) && !is_empty_dir(empty_submodule_path.buf)) { - spf->result = 1; + record_fetch_error(spf, ce->name); strbuf_addf(err, _("Could not access submodule '%s'\n"), ce->name); @@ -1753,7 +1760,7 @@ static int fetch_start_failure(struct strbuf *err UNUSED, struct submodule_parallel_fetch *spf = cb; struct fetch_task *task = task_cb; - spf->result = 1; + record_fetch_error(spf, task->sub->name); fetch_task_free(task); return 0; @@ -1779,18 +1786,12 @@ static int fetch_finish(int retvalue, struct strbuf *err UNUSED, if (!task || !task->sub) BUG("callback cookie bogus"); - if (retvalue) { + if (retvalue && task->commits) { /* - * NEEDSWORK: This indicates that the overall fetch - * failed, even though there may be a subsequent fetch - * by commit hash that might work. It may be a good - * idea to not indicate failure in this case, and only - * indicate failure if the subsequent fetch fails. + * This is the second pass (OID-based fetch) and it failed. + * The commits are genuinely unavailable from the remote. */ - spf->result = 1; - - strbuf_addf(&spf->submodules_with_errors, "\t%s\n", - task->sub->name); + record_fetch_error(spf, task->sub->name); } /* Is this the second time we process this submodule? */ @@ -1798,9 +1799,17 @@ static int fetch_finish(int retvalue, struct strbuf *err UNUSED, goto out; it = string_list_lookup(&spf->changed_submodule_names, task->sub->name); - if (!it) - /* Could be an unchanged submodule, not contained in the list */ + if (!it) { + /* + * This submodule is not in the changed list (e.g. it was + * fetched because RECURSE_SUBMODULES_ON fetches all populated + * submodules). A phase 1 failure here has no OID-based retry + * to fall back on, so it is a genuine error. + */ + if (retvalue) + record_fetch_error(spf, task->sub->name); goto out; + } cs_data = it->util; oid_array_filter(&cs_data->new_commits, @@ -1809,6 +1818,11 @@ static int fetch_finish(int retvalue, struct strbuf *err UNUSED, /* Are there commits we want, but do not exist? */ if (cs_data->new_commits.nr) { + /* + * Schedule an OID-based phase 2 fetch to retrieve the missing + * commits directly. Defer any error from phase 1: if phase 2 + * succeeds, the overall operation should still succeed. + */ task->commits = &cs_data->new_commits; ALLOC_GROW(spf->oid_fetch_tasks, spf->oid_fetch_tasks_nr + 1, @@ -1818,6 +1832,16 @@ static int fetch_finish(int retvalue, struct strbuf *err UNUSED, return 0; } + /* + * All required commits are already present locally (they were either + * fetched by phase 1 or existed beforehand), so there is no phase 2 + * retry to defer to. If phase 1 failed, the fetch itself went wrong + * (e.g. a transport error) and must still be reported, even though + * the gitlinked commits are available. + */ + if (retvalue) + record_fetch_error(spf, task->sub->name); + out: fetch_task_free(task); return 0; diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh index 1242ee9185..188c674c89 100755 --- a/t/t5526-fetch-submodules.sh +++ b/t/t5526-fetch-submodules.sh @@ -1262,4 +1262,76 @@ test_expect_success "fetch --all with --no-recurse-submodules only fetches super ! grep "Fetching submodule" fetch-log ' +# Create an isolated environment for submodule fetch error tests. +# +# Sets up sub_bare (the submodule upstream), super_bare (the superproject +# upstream), super_work (a working clone of super_bare with an initialized +# submodule), and clone (a clone of super_bare with an initialized submodule +# at a reachable commit). The caller can then create an unreachable commit +# and push the superproject to put the clone one commit behind a state it +# cannot fully fetch. +# +# Usage: create_err_env <envdir> +create_err_env () { + local envdir="$1" && + mkdir "$envdir" && + + git init --bare "$envdir/sub_bare" && + git clone "$envdir/sub_bare" "$envdir/sub_work" && + test_commit -C "$envdir/sub_work" "${envdir}_base" && + git -C "$envdir/sub_work" push && + + git init --bare "$envdir/super_bare" && + git clone "$envdir/super_bare" "$envdir/super_work" && + git -C "$envdir/super_work" submodule add \ + "$pwd/$envdir/sub_bare" sub && + git -C "$envdir/super_work" commit -m "add submodule" && + git -C "$envdir/super_work" push && + + git clone "$envdir/super_bare" "$envdir/clone" && + git -C "$envdir/clone" submodule update --init +} + +# Push a commit to <envdir>/super_bare that records a submodule SHA that is +# present locally in super_work/sub but NOT pushed to sub_bare, making the +# submodule commit unreachable from clone's sub remote. +push_unreachable_commit () { + local envdir="$1" && + git -C "$envdir/super_work/sub" commit --allow-empty -m "unreachable" && + git -C "$envdir/super_work" add sub && + git -C "$envdir/super_work" commit -m "point sub to unreachable commit" && + git -C "$envdir/super_work" push +} + +test_expect_success 'setup for submodule fetch error tests' ' + git config --global protocol.file.allow always +' + +test_expect_success 'failed submodule fetch is fatal even when its commits are present locally' ' + # Create the same commit (unreferenced, via commit-tree with fixed + # dates) in both super_work/sub and clone/sub, point the gitlink at + # it, and break clone/sub'\''s remote. The commit exists in clone/sub + # but is unreachable, so the submodule stays in the changed list; the + # fetch failure must still be reported even though there is nothing + # left to fetch by commit hash. + test_when_finished "rm -fr env_phase1" && + create_err_env env_phase1 && + commit=$(GIT_AUTHOR_DATE="1234567890 +0000" \ + GIT_COMMITTER_DATE="1234567890 +0000" \ + git -C env_phase1/super_work/sub commit-tree \ + "HEAD^{tree}" -p HEAD -m present) && + present=$(GIT_AUTHOR_DATE="1234567890 +0000" \ + GIT_COMMITTER_DATE="1234567890 +0000" \ + git -C env_phase1/clone/sub commit-tree \ + "HEAD^{tree}" -p HEAD -m present) && + test "$commit" = "$present" && + git -C env_phase1/super_work/sub checkout "$commit" && + git -C env_phase1/super_work add sub && + git -C env_phase1/super_work commit -m "gitlink to locally-present commit" && + git -C env_phase1/super_work push && + git -C env_phase1/clone/sub remote set-url origin "$pwd/env_phase1/missing" && + test_must_fail git -C env_phase1/clone fetch --recurse-submodules 2>err && + grep "Errors during submodule fetch" err +' + test_done -- 2.54.0 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v3 2/2] fetch: add fetch.submoduleErrors to make submodule fetch errors non-fatal 2026-07-10 12:26 [PATCH v3 0/2] fetch: make submodule fetch errors configurable Paulius Zaleckas 2026-07-10 12:26 ` [PATCH v3 1/2] submodule: fix premature failure in recursive submodule fetch Paulius Zaleckas @ 2026-07-10 12:26 ` Paulius Zaleckas 2026-07-10 22:21 ` Junio C Hamano 2026-07-14 13:29 ` [PATCH v4 0/2] fetch: make submodule fetch errors configurable Paulius Zaleckas 2 siblings, 1 reply; 10+ messages in thread From: Paulius Zaleckas @ 2026-07-10 12:26 UTC (permalink / raw) To: git Cc: Paulius Zaleckas, Glen Choo, Ævar Arnfjörð Bjarmason, Patrick Steinhardt, Junio C Hamano When fetching with --recurse-submodules, a submodule commit that is not yet reachable from any of the submodule's remote refs causes the entire fetch to fail. This is overly strict when the missing commit belongs to an upstream branch that is still being prepared (e.g. an in-progress merge topic): the local branch does not need that commit, so there is no reason to treat its absence as fatal. Add a new config key fetch.submoduleErrors (values: fail/warn) and a corresponding --submodule-errors=(fail|warn) command-line option that control this behaviour. The default remains fail (existing behaviour); setting the value to warn causes submodule fetch failures to be reported on stderr without affecting the overall exit status of git fetch / git pull. Forward the option to child fetches in add_options_to_argv() so that it also takes effect for `git fetch --all` / `--multiple` (where per-remote child processes handle the submodule recursion themselves) and for nested submodule recursion. Signed-off-by: Paulius Zaleckas <paulius.zaleckas@gmail.com> --- Documentation/config/fetch.adoc | 14 ++++++ Documentation/fetch-options.adoc | 8 ++++ builtin/fetch.c | 41 ++++++++++++++++- submodule.c | 8 +++- submodule.h | 7 ++- t/t5526-fetch-submodules.sh | 76 ++++++++++++++++++++++++++++++++ 6 files changed, 150 insertions(+), 4 deletions(-) diff --git a/Documentation/config/fetch.adoc b/Documentation/config/fetch.adoc index 04ac90912d..5c9c942a70 100644 --- a/Documentation/config/fetch.adoc +++ b/Documentation/config/fetch.adoc @@ -10,6 +10,20 @@ reference. Defaults to `on-demand`, or to the value of `submodule.recurse` if set. +`fetch.submoduleErrors`:: + Controls how errors from submodule fetches are handled when + `--recurse-submodules` is in effect. When set to `fail` (the default), + any submodule fetch error causes the overall `git fetch` or `git pull` + to exit with a non-zero status. When set to `warn`, submodule fetch + errors are reported to standard error but do not affect the exit + status of the command. This is useful when working in repositories + where some branches reference submodule commits that are not yet + available on the submodule remote, but those commits are not needed + for the currently checked-out branch. ++ +The value of this option can be overridden by the `--submodule-errors` +option of linkgit:git-fetch[1]. + `fetch.fsckObjects`:: If it is set to true, git-fetch-pack will check all fetched objects. See `transfer.fsckObjects` for what's diff --git a/Documentation/fetch-options.adoc b/Documentation/fetch-options.adoc index 035f780e58..78525f6848 100644 --- a/Documentation/fetch-options.adoc +++ b/Documentation/fetch-options.adoc @@ -294,6 +294,14 @@ ifndef::git-pull[] `--no-recurse-submodules`:: Disable recursive fetching of submodules (this has the same effect as using the `--recurse-submodules=no` option). + +`--submodule-errors=(fail|warn)`:: + Control how errors from submodule fetches are handled when + `--recurse-submodules` is in effect. When set to `fail` (the default), + any submodule fetch error causes the overall `git fetch` to exit with a + non-zero status. When set to `warn`, submodule fetch errors are reported + to standard error but do not affect the exit status of the command. Can + also be configured via `fetch.submoduleErrors`. See linkgit:git-config[1]. endif::git-pull[] `--set-upstream`:: diff --git a/builtin/fetch.c b/builtin/fetch.c index c1d7c672f4..40daaf5cc7 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -110,6 +110,7 @@ struct fetch_config { int recurse_submodules; int parallel; int submodule_fetch_jobs; + int submodule_errors; }; static int git_fetch_config(const char *k, const char *v, @@ -152,6 +153,19 @@ static int git_fetch_config(const char *k, const char *v, return 0; } + if (!strcmp(k, "fetch.submoduleerrors")) { + if (!v) + return config_error_nonbool(k); + else if (!strcasecmp(v, "fail")) + fetch_config->submodule_errors = SUBMODULE_ERRORS_FAIL; + else if (!strcasecmp(v, "warn")) + fetch_config->submodule_errors = SUBMODULE_ERRORS_WARN; + else + die(_("invalid value for '%s': '%s'"), + "fetch.submoduleErrors", v); + return 0; + } + if (!strcmp(k, "fetch.parallel")) { fetch_config->parallel = git_config_int(k, v, ctx->kvi); if (fetch_config->parallel < 0) @@ -2205,6 +2219,8 @@ static void add_options_to_argv(struct strvec *argv, strvec_push(argv, "--no-recurse-submodules"); else if (config->recurse_submodules == RECURSE_SUBMODULES_ON_DEMAND) strvec_push(argv, "--recurse-submodules=on-demand"); + if (config->submodule_errors == SUBMODULE_ERRORS_WARN) + strvec_push(argv, "--submodule-errors=warn"); if (tags == TAGS_SET) strvec_push(argv, "--tags"); else if (tags == TAGS_UNSET) @@ -2464,6 +2480,19 @@ static int fetch_one(struct remote *remote, int argc, const char **argv, return exit_code; } +static int option_parse_submodule_errors(const struct option *opt, + const char *arg, int unset) +{ + int *v = opt->value; + if (unset || !strcasecmp(arg, "fail")) + *v = SUBMODULE_ERRORS_FAIL; + else if (!strcasecmp(arg, "warn")) + *v = SUBMODULE_ERRORS_WARN; + else + die(_("invalid value for '%s': '%s'"), "--submodule-errors", arg); + return 0; +} + int cmd_fetch(int argc, const char **argv, const char *prefix, @@ -2477,6 +2506,7 @@ int cmd_fetch(int argc, .recurse_submodules = RECURSE_SUBMODULES_DEFAULT, .parallel = 1, .submodule_fetch_jobs = -1, + .submodule_errors = SUBMODULE_ERRORS_FAIL, }; const char *submodule_prefix = ""; const char *bundle_uri; @@ -2491,6 +2521,7 @@ int cmd_fetch(int argc, int max_jobs = -1; int recurse_submodules_cli = RECURSE_SUBMODULES_DEFAULT; int recurse_submodules_default = RECURSE_SUBMODULES_ON_DEMAND; + int submodule_errors_cli = -1; /* -1: not set on command line */ int fetch_write_commit_graph = -1; int stdin_refspecs = 0; int negotiate_only = 0; @@ -2527,6 +2558,10 @@ int cmd_fetch(int argc, OPT_CALLBACK_F(0, "recurse-submodules", &recurse_submodules_cli, N_("on-demand"), N_("control recursive fetching of submodules"), PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules), + OPT_CALLBACK_F(0, "submodule-errors", &submodule_errors_cli, + N_("(fail|warn)"), + N_("control how submodule fetch errors are handled"), + 0, option_parse_submodule_errors), OPT_BOOL(0, "dry-run", &dry_run, N_("dry run")), OPT_BOOL(0, "porcelain", &porcelain, N_("machine-readable output")), @@ -2616,6 +2651,9 @@ int cmd_fetch(int argc, if (recurse_submodules_cli != RECURSE_SUBMODULES_DEFAULT) config.recurse_submodules = recurse_submodules_cli; + if (submodule_errors_cli != -1) + config.submodule_errors = submodule_errors_cli; + if (negotiate_only) { switch (recurse_submodules_cli) { case RECURSE_SUBMODULES_OFF: @@ -2833,7 +2871,8 @@ int cmd_fetch(int argc, config.recurse_submodules, recurse_submodules_default, verbosity < 0, - max_children); + max_children, + config.submodule_errors); trace2_region_leave_printf("fetch", "recurse-submodule", the_repository, "%s", submodule_prefix); strvec_clear(&options); } diff --git a/submodule.c b/submodule.c index 8bcef68a42..da4ace751f 100644 --- a/submodule.c +++ b/submodule.c @@ -1409,6 +1409,7 @@ struct submodule_parallel_fetch { int oid_fetch_tasks_nr, oid_fetch_tasks_alloc; struct strbuf submodules_with_errors; + int submodule_errors; }; #define SPF_INIT { \ .args = STRVEC_INIT, \ @@ -1565,7 +1566,8 @@ static struct fetch_task *fetch_task_create(struct submodule_parallel_fetch *spf static void record_fetch_error(struct submodule_parallel_fetch *spf, const char *name) { - spf->result = 1; + if (spf->submodule_errors == SUBMODULE_ERRORS_FAIL) + spf->result = 1; strbuf_addf(&spf->submodules_with_errors, "\t%s\n", name); } @@ -1851,7 +1853,8 @@ int fetch_submodules(struct repository *r, const struct strvec *options, const char *prefix, int command_line_option, int default_option, - int quiet, int max_parallel_jobs) + int quiet, int max_parallel_jobs, + int submodule_errors) { struct submodule_parallel_fetch spf = SPF_INIT; const struct run_process_parallel_opts opts = { @@ -1871,6 +1874,7 @@ int fetch_submodules(struct repository *r, spf.default_option = default_option; spf.quiet = quiet; spf.prefix = prefix; + spf.submodule_errors = submodule_errors; if (!r->worktree) goto out; diff --git a/submodule.h b/submodule.h index b10e16e6c0..c80b687d2a 100644 --- a/submodule.h +++ b/submodule.h @@ -90,12 +90,17 @@ int should_update_submodules(void); */ const struct submodule *submodule_from_ce(const struct cache_entry *ce); void check_for_new_submodule_commits(struct object_id *oid); +/* Values for the submodule_errors parameter of fetch_submodules(). */ +#define SUBMODULE_ERRORS_FAIL 0 /* submodule fetch errors are fatal (default) */ +#define SUBMODULE_ERRORS_WARN 1 /* submodule fetch errors are non-fatal warnings */ + int fetch_submodules(struct repository *r, const struct strvec *options, const char *prefix, int command_line_option, int default_option, - int quiet, int max_parallel_jobs); + int quiet, int max_parallel_jobs, + int submodule_errors); unsigned is_submodule_modified(const char *path, int ignore_untracked); int submodule_uses_gitfile(const char *path); diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh index 188c674c89..b5db8fb5c2 100755 --- a/t/t5526-fetch-submodules.sh +++ b/t/t5526-fetch-submodules.sh @@ -1307,6 +1307,57 @@ test_expect_success 'setup for submodule fetch error tests' ' git config --global protocol.file.allow always ' +test_expect_success 'fetch --recurse-submodules fails when submodule commit is unreachable (default)' ' + test_when_finished "rm -fr env_default" && + create_err_env env_default && + push_unreachable_commit env_default && + test_must_fail git -C env_default/clone fetch --recurse-submodules 2>err && + grep "Errors during submodule fetch" err +' + +test_expect_success 'fetch.submoduleErrors=warn: unreachable submodule commit is non-fatal' ' + test_when_finished "rm -fr env_warn_cfg" && + create_err_env env_warn_cfg && + push_unreachable_commit env_warn_cfg && + git -C env_warn_cfg/clone -c fetch.submoduleErrors=warn \ + fetch --recurse-submodules 2>err && + grep "Errors during submodule fetch" err +' + +test_expect_success '--submodule-errors=warn: unreachable submodule commit is non-fatal' ' + test_when_finished "rm -fr env_warn_cli" && + create_err_env env_warn_cli && + push_unreachable_commit env_warn_cli && + git -C env_warn_cli/clone fetch --recurse-submodules \ + --submodule-errors=warn 2>err && + grep "Errors during submodule fetch" err +' + +test_expect_success '--submodule-errors=fail: unreachable submodule commit is fatal' ' + test_when_finished "rm -fr env_fail_cli" && + create_err_env env_fail_cli && + push_unreachable_commit env_fail_cli && + test_must_fail git -C env_fail_cli/clone fetch --recurse-submodules \ + --submodule-errors=fail 2>err && + grep "Errors during submodule fetch" err +' + +test_expect_success 'fetch.submoduleErrors=warn does not suppress successful fetch' ' + # A new reachable submodule commit (pushed to sub_bare) should be + # fetched without any error summary. + test_when_finished "rm -fr env_ok" && + create_err_env env_ok && + test_commit -C env_ok/sub_work reachable_ok && + git -C env_ok/sub_work push && + git -C env_ok/super_work submodule update --remote && + git -C env_ok/super_work add sub && + git -C env_ok/super_work commit -m "point sub to reachable commit" && + git -C env_ok/super_work push && + git -C env_ok/clone -c fetch.submoduleErrors=warn \ + fetch --recurse-submodules 2>err && + ! grep "Errors during submodule fetch" err +' + test_expect_success 'failed submodule fetch is fatal even when its commits are present locally' ' # Create the same commit (unreferenced, via commit-tree with fixed # dates) in both super_work/sub and clone/sub, point the gitlink at @@ -1334,4 +1385,29 @@ test_expect_success 'failed submodule fetch is fatal even when its commits are p grep "Errors during submodule fetch" err ' +test_expect_success '--submodule-errors=warn is honored by fetch --all' ' + # A second remote forces fetch_multiple(), which hands the submodule + # recursion off to per-remote child processes; the option must be + # forwarded to them. + test_when_finished "rm -fr env_all" && + create_err_env env_all && + push_unreachable_commit env_all && + git -C env_all/clone remote add second "$pwd/env_all/super_bare" && + git -C env_all/clone fetch --all --recurse-submodules \ + --submodule-errors=warn 2>err && + grep "Errors during submodule fetch" err +' + +test_expect_success 'fetch.submoduleErrors=warn: inaccessible submodule is non-fatal' ' + test_when_finished "rm -fr env_access" && + create_err_env env_access && + rm env_access/clone/sub/.git && + rm -r env_access/clone/.git/modules/sub && + git -C env_access/clone -c fetch.submoduleErrors=warn \ + fetch --recurse-submodules 2>err && + grep "Could not access submodule" err && + test_must_fail git -C env_access/clone fetch --recurse-submodules 2>err && + grep "Could not access submodule" err +' + test_done -- 2.54.0 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH v3 2/2] fetch: add fetch.submoduleErrors to make submodule fetch errors non-fatal 2026-07-10 12:26 ` [PATCH v3 2/2] fetch: add fetch.submoduleErrors to make submodule fetch errors non-fatal Paulius Zaleckas @ 2026-07-10 22:21 ` Junio C Hamano 2026-07-14 13:29 ` Paulius Zaleckas 0 siblings, 1 reply; 10+ messages in thread From: Junio C Hamano @ 2026-07-10 22:21 UTC (permalink / raw) To: Paulius Zaleckas Cc: git, Glen Choo, Ævar Arnfjörð Bjarmason, Patrick Steinhardt Paulius Zaleckas <paulius.zaleckas@gmail.com> writes: > When fetching with --recurse-submodules, a submodule commit that is not > yet reachable from any of the submodule's remote refs causes the entire > fetch to fail. This is overly strict when the missing commit belongs to > an upstream branch that is still being prepared (e.g. an in-progress > merge topic): the local branch does not need that commit, so there is no > reason to treat its absence as fatal. > > Add a new config key fetch.submoduleErrors (values: fail/warn) and a > corresponding --submodule-errors=(fail|warn) command-line option that > control this behaviour. The default remains fail (existing behaviour); > setting the value to warn causes submodule fetch failures to be reported > on stderr without affecting the overall exit status of git fetch / git > pull. > > Forward the option to child fetches in add_options_to_argv() so that it > also takes effect for `git fetch --all` / `--multiple` (where per-remote > child processes handle the submodule recursion themselves) and for > nested submodule recursion. > > Signed-off-by: Paulius Zaleckas <paulius.zaleckas@gmail.com> > --- > Documentation/config/fetch.adoc | 14 ++++++ > Documentation/fetch-options.adoc | 8 ++++ > builtin/fetch.c | 41 ++++++++++++++++- > submodule.c | 8 +++- > submodule.h | 7 ++- > t/t5526-fetch-submodules.sh | 76 ++++++++++++++++++++++++++++++++ > 6 files changed, 150 insertions(+), 4 deletions(-) > > diff --git a/Documentation/config/fetch.adoc b/Documentation/config/fetch.adoc > index 04ac90912d..5c9c942a70 100644 > --- a/Documentation/config/fetch.adoc > +++ b/Documentation/config/fetch.adoc > @@ -10,6 +10,20 @@ > reference. > Defaults to `on-demand`, or to the value of `submodule.recurse` if set. > > +`fetch.submoduleErrors`:: > + Controls how errors from submodule fetches are handled when > + `--recurse-submodules` is in effect. When set to `fail` (the default), > + any submodule fetch error causes the overall `git fetch` or `git pull` > + to exit with a non-zero status. When set to `warn`, submodule fetch > + errors are reported to standard error but do not affect the exit > + status of the command. This is useful when working in repositories > + where some branches reference submodule commits that are not yet > + available on the submodule remote, but those commits are not needed > + for the currently checked-out branch. > ++ > +The value of this option can be overridden by the `--submodule-errors` > +option of linkgit:git-fetch[1]. > + > `fetch.fsckObjects`:: > If it is set to true, git-fetch-pack will check all fetched > objects. See `transfer.fsckObjects` for what's > diff --git a/Documentation/fetch-options.adoc b/Documentation/fetch-options.adoc > index 035f780e58..78525f6848 100644 > --- a/Documentation/fetch-options.adoc > +++ b/Documentation/fetch-options.adoc > @@ -294,6 +294,14 @@ ifndef::git-pull[] > `--no-recurse-submodules`:: > Disable recursive fetching of submodules (this has the same effect as > using the `--recurse-submodules=no` option). > + > +`--submodule-errors=(fail|warn)`:: > + Control how errors from submodule fetches are handled when > + `--recurse-submodules` is in effect. When set to `fail` (the default), > + any submodule fetch error causes the overall `git fetch` to exit with a > + non-zero status. When set to `warn`, submodule fetch errors are reported > + to standard error but do not affect the exit status of the command. Can > + also be configured via `fetch.submoduleErrors`. See linkgit:git-config[1]. > endif::git-pull[] > > `--set-upstream`:: > diff --git a/builtin/fetch.c b/builtin/fetch.c > index c1d7c672f4..40daaf5cc7 100644 > --- a/builtin/fetch.c > +++ b/builtin/fetch.c > @@ -110,6 +110,7 @@ struct fetch_config { > int recurse_submodules; > int parallel; > int submodule_fetch_jobs; > + int submodule_errors; > }; > > static int git_fetch_config(const char *k, const char *v, > @@ -152,6 +153,19 @@ static int git_fetch_config(const char *k, const char *v, > return 0; > } > > + if (!strcmp(k, "fetch.submoduleerrors")) { > + if (!v) > + return config_error_nonbool(k); > + else if (!strcasecmp(v, "fail")) > + fetch_config->submodule_errors = SUBMODULE_ERRORS_FAIL; > + else if (!strcasecmp(v, "warn")) > + fetch_config->submodule_errors = SUBMODULE_ERRORS_WARN; > + else > + die(_("invalid value for '%s': '%s'"), > + "fetch.submoduleErrors", v); > + return 0; > + } > + > if (!strcmp(k, "fetch.parallel")) { > fetch_config->parallel = git_config_int(k, v, ctx->kvi); > if (fetch_config->parallel < 0) > @@ -2205,6 +2219,8 @@ static void add_options_to_argv(struct strvec *argv, > strvec_push(argv, "--no-recurse-submodules"); > else if (config->recurse_submodules == RECURSE_SUBMODULES_ON_DEMAND) > strvec_push(argv, "--recurse-submodules=on-demand"); > + if (config->submodule_errors == SUBMODULE_ERRORS_WARN) > + strvec_push(argv, "--submodule-errors=warn"); > if (tags == TAGS_SET) > strvec_push(argv, "--tags"); > else if (tags == TAGS_UNSET) If (config->submodule_errors != SUBMODULE_ERRORS_WARN), then the argv[] would not see any --submodule-errors=<anything> to propagate down. Specifically, this function is called when recurse-submodules is not disabled, and prepares argv[] used to call fetch_submodules(). > int cmd_fetch(int argc, > const char **argv, > const char *prefix, > @@ -2477,6 +2506,7 @@ int cmd_fetch(int argc, > .recurse_submodules = RECURSE_SUBMODULES_DEFAULT, > .parallel = 1, > .submodule_fetch_jobs = -1, > + .submodule_errors = SUBMODULE_ERRORS_FAIL, > }; Here, .submodule_errors member is initialized to SUBMODULE_ERRORS_FAIL (i.e. 0). > @@ -2491,6 +2521,7 @@ int cmd_fetch(int argc, > int max_jobs = -1; > int recurse_submodules_cli = RECURSE_SUBMODULES_DEFAULT; > int recurse_submodules_default = RECURSE_SUBMODULES_ON_DEMAND; > + int submodule_errors_cli = -1; /* -1: not set on command line */ > int fetch_write_commit_graph = -1; > int stdin_refspecs = 0; > int negotiate_only = 0; > @@ -2527,6 +2558,10 @@ int cmd_fetch(int argc, > OPT_CALLBACK_F(0, "recurse-submodules", &recurse_submodules_cli, N_("on-demand"), > N_("control recursive fetching of submodules"), > PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules), > + OPT_CALLBACK_F(0, "submodule-errors", &submodule_errors_cli, > + N_("(fail|warn)"), > + N_("control how submodule fetch errors are handled"), > + 0, option_parse_submodule_errors), And command line option "--submodule-errors={warn,fail}" may update the local variable submodule_errors_cli (initialied to -1) to one of SUBMODULE_ERRORS_{WARN,FAIL}. These are different from -1, so we can reliably tell if we saw a command line override, which is good. > OPT_BOOL(0, "dry-run", &dry_run, > N_("dry run")), > OPT_BOOL(0, "porcelain", &porcelain, N_("machine-readable output")), > @@ -2616,6 +2651,9 @@ int cmd_fetch(int argc, > if (recurse_submodules_cli != RECURSE_SUBMODULES_DEFAULT) > config.recurse_submodules = recurse_submodules_cli; > > + if (submodule_errors_cli != -1) > + config.submodule_errors = submodule_errors_cli; And we override what we read from the configuration if we got a command line override. And the value in config.submodule_errors is used much later, in a call to add_options_to_argv() we saw earlier, but this patch does not touch the caller so we do not see the calling site. I do not do submodules, so my expectation here may be a bit skewed, but what happens when we configure fetch.submoduleErrors to warn, but override it from the command line to fail? .submodule_errors is set to SUBMODULE_ERRORS_FAIL here? As we saw, add_options_to_argv() stuff --submodule-error=<setting> only when config.submodule_errors is set to SUBMODULE_ERRORS_WARN, so we do not pass command line override. Is this desirable? Don't we want to pass down not just --submodule-error=warn but --submodule-error=fail if that is what was given from the command line? Or does it not matter because fail is the default? Thanks. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v3 2/2] fetch: add fetch.submoduleErrors to make submodule fetch errors non-fatal 2026-07-10 22:21 ` Junio C Hamano @ 2026-07-14 13:29 ` Paulius Zaleckas 0 siblings, 0 replies; 10+ messages in thread From: Paulius Zaleckas @ 2026-07-14 13:29 UTC (permalink / raw) To: Junio C Hamano Cc: Paulius Zaleckas, git, Glen Choo, Ævar Arnfjörð Bjarmason, Patrick Steinhardt Junio C Hamano <gitster@pobox.com> writes: > Don't we want to pass down not just > --submodule-error=warn but --submodule-error=fail if that is what > was given from the command line? Or does it not matter because fail > is the default? Good catch, it does matter: the per-remote children of "fetch --all" re-read the repository configuration, so a configured warn silently won over an explicit --submodule-errors=fail. Fixed in v4 by forwarding the resolved value whenever it was set explicitly, in either direction; when nothing is set, nothing is forwarded. Added a test. Thanks. ^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v4 0/2] fetch: make submodule fetch errors configurable 2026-07-10 12:26 [PATCH v3 0/2] fetch: make submodule fetch errors configurable Paulius Zaleckas 2026-07-10 12:26 ` [PATCH v3 1/2] submodule: fix premature failure in recursive submodule fetch Paulius Zaleckas 2026-07-10 12:26 ` [PATCH v3 2/2] fetch: add fetch.submoduleErrors to make submodule fetch errors non-fatal Paulius Zaleckas @ 2026-07-14 13:29 ` Paulius Zaleckas 2026-07-14 13:29 ` [PATCH v4 1/2] submodule: fix premature failure in recursive submodule fetch Paulius Zaleckas 2026-07-14 13:29 ` [PATCH v4 2/2] fetch: add fetch.submoduleErrors to make submodule fetch errors non-fatal Paulius Zaleckas 2 siblings, 2 replies; 10+ messages in thread From: Paulius Zaleckas @ 2026-07-14 13:29 UTC (permalink / raw) To: git; +Cc: Junio C Hamano, Paulius Zaleckas When fetching with --recurse-submodules, git currently exits with a non-zero status if any submodule references an OID that is not reachable from the submodule's remote. This situation arises naturally when an upstream branch is still in preparation (e.g. a topic branch in a merge window): the local branch does not depend on the missing commit, so a hard failure is unnecessarily disruptive. Patch 1 fixes a pre-existing NEEDSWORK in submodule.c where a phase-1 fetch failure was recorded immediately, even when a phase-2 OID-based retry was about to be scheduled. After this fix the existing fatal behaviour is preserved but the logic is now structured so that errors are only recorded when the phase-2 retry actually fails, or when there is no phase-2 retry to fall back on. Patch 2 introduces fetch.submoduleErrors (fail|warn) and --submodule-errors=(fail|warn) to let users opt into non-fatal behaviour. The default remains fail for full backwards compatibility. Changes in v4: - Forward an explicit --submodule-errors=fail to child fetches as well, so the command line overrides fetch.submoduleErrors=warn config in the per-remote children of fetch --all/--multiple (noticed by Junio) Changes in v3: - Report a phase-1 failure also when the gitlink commits are already present locally, instead of silently succeeding - Route "Could not access submodule" through record_fetch_error() so it shows up in the error summary and honors the warn mode - Forward --submodule-errors to child fetches so it takes effect for fetch --all/--multiple and nested submodule recursion - Add tests for all of the above - Documentation: don't imply git pull takes --submodule-errors, minor wording and placement fixes Changes in v2: - Fix option synopsis to use (fail|warn) instead of <fail|warn> - Add --submodule-errors documentation to Documentation/fetch-options.adoc Paulius Zaleckas (2): submodule: fix premature failure in recursive submodule fetch fetch: add fetch.submoduleErrors to make submodule fetch errors non-fatal Documentation/config/fetch.adoc | 14 +++ Documentation/fetch-options.adoc | 8 ++ builtin/fetch.c | 46 ++++++++- submodule.c | 58 ++++++++--- submodule.h | 7 +- t/t5526-fetch-submodules.sh | 161 +++++++++++++++++++++++++++++++ 6 files changed, 277 insertions(+), 17 deletions(-) -- 2.54.0 ^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v4 1/2] submodule: fix premature failure in recursive submodule fetch 2026-07-14 13:29 ` [PATCH v4 0/2] fetch: make submodule fetch errors configurable Paulius Zaleckas @ 2026-07-14 13:29 ` Paulius Zaleckas 2026-07-14 13:29 ` [PATCH v4 2/2] fetch: add fetch.submoduleErrors to make submodule fetch errors non-fatal Paulius Zaleckas 1 sibling, 0 replies; 10+ messages in thread From: Paulius Zaleckas @ 2026-07-14 13:29 UTC (permalink / raw) To: git Cc: Junio C Hamano, Paulius Zaleckas, Elijah Newren, Patrick Steinhardt, Glen Choo, Jonathan Tan When git fetch --recurse-submodules encounters a failure fetching a submodule's refs (phase 1), it immediately marks the overall operation as failed, even though a subsequent OID-based fetch (phase 2) is about to be attempted for any missing commits. If phase 2 succeeds, the overall result should be success, but the prematurely set failure flag makes it look like an error. Restructure fetch_finish() so that a phase-1 failure does not record an error immediately. Instead, the decision is deferred: - If missing commits trigger a phase-2 (OID-based) retry and that retry succeeds, no error is recorded. - If the phase-2 retry also fails, the error is recorded then. - If the submodule was fetched unconditionally (RECURSE_SUBMODULES_ON) and is not in the changed list, a phase-1 failure is recorded right away since there is no OID retry to fall back on. - If phase 1 fails but all required commits are already present locally, there is no retry to defer to; the failure is still recorded, since the fetch itself went wrong (e.g. a transport error) even though the wanted commits happen to be available. This resolves the NEEDSWORK comment added by bd5e567dc7 (submodule: explain first attempt failure clearly, 2019-03-13). Extract the common error-recording logic into a helper record_fetch_error() and use it in fetch_start_failure() and for the "Could not access submodule" error in get_fetch_task_from_index() as well; the latter now also lists the submodule in the final error summary. Add a test ensuring a failed submodule fetch is still reported when the gitlinked commits happen to be present locally. Signed-off-by: Paulius Zaleckas <paulius.zaleckas@gmail.com> --- submodule.c | 52 +++++++++++++++++++-------- t/t5526-fetch-submodules.sh | 72 +++++++++++++++++++++++++++++++++++++ 2 files changed, 110 insertions(+), 14 deletions(-) diff --git a/submodule.c b/submodule.c index fd91201a92..8bcef68a42 100644 --- a/submodule.c +++ b/submodule.c @@ -1562,6 +1562,13 @@ static struct fetch_task *fetch_task_create(struct submodule_parallel_fetch *spf return NULL; } +static void record_fetch_error(struct submodule_parallel_fetch *spf, + const char *name) +{ + spf->result = 1; + strbuf_addf(&spf->submodules_with_errors, "\t%s\n", name); +} + static struct fetch_task * get_fetch_task_from_index(struct submodule_parallel_fetch *spf, struct strbuf *err) @@ -1599,7 +1606,7 @@ get_fetch_task_from_index(struct submodule_parallel_fetch *spf, ce->name); if (S_ISGITLINK(ce->ce_mode) && !is_empty_dir(empty_submodule_path.buf)) { - spf->result = 1; + record_fetch_error(spf, ce->name); strbuf_addf(err, _("Could not access submodule '%s'\n"), ce->name); @@ -1753,7 +1760,7 @@ static int fetch_start_failure(struct strbuf *err UNUSED, struct submodule_parallel_fetch *spf = cb; struct fetch_task *task = task_cb; - spf->result = 1; + record_fetch_error(spf, task->sub->name); fetch_task_free(task); return 0; @@ -1779,18 +1786,12 @@ static int fetch_finish(int retvalue, struct strbuf *err UNUSED, if (!task || !task->sub) BUG("callback cookie bogus"); - if (retvalue) { + if (retvalue && task->commits) { /* - * NEEDSWORK: This indicates that the overall fetch - * failed, even though there may be a subsequent fetch - * by commit hash that might work. It may be a good - * idea to not indicate failure in this case, and only - * indicate failure if the subsequent fetch fails. + * This is the second pass (OID-based fetch) and it failed. + * The commits are genuinely unavailable from the remote. */ - spf->result = 1; - - strbuf_addf(&spf->submodules_with_errors, "\t%s\n", - task->sub->name); + record_fetch_error(spf, task->sub->name); } /* Is this the second time we process this submodule? */ @@ -1798,9 +1799,17 @@ static int fetch_finish(int retvalue, struct strbuf *err UNUSED, goto out; it = string_list_lookup(&spf->changed_submodule_names, task->sub->name); - if (!it) - /* Could be an unchanged submodule, not contained in the list */ + if (!it) { + /* + * This submodule is not in the changed list (e.g. it was + * fetched because RECURSE_SUBMODULES_ON fetches all populated + * submodules). A phase 1 failure here has no OID-based retry + * to fall back on, so it is a genuine error. + */ + if (retvalue) + record_fetch_error(spf, task->sub->name); goto out; + } cs_data = it->util; oid_array_filter(&cs_data->new_commits, @@ -1809,6 +1818,11 @@ static int fetch_finish(int retvalue, struct strbuf *err UNUSED, /* Are there commits we want, but do not exist? */ if (cs_data->new_commits.nr) { + /* + * Schedule an OID-based phase 2 fetch to retrieve the missing + * commits directly. Defer any error from phase 1: if phase 2 + * succeeds, the overall operation should still succeed. + */ task->commits = &cs_data->new_commits; ALLOC_GROW(spf->oid_fetch_tasks, spf->oid_fetch_tasks_nr + 1, @@ -1818,6 +1832,16 @@ static int fetch_finish(int retvalue, struct strbuf *err UNUSED, return 0; } + /* + * All required commits are already present locally (they were either + * fetched by phase 1 or existed beforehand), so there is no phase 2 + * retry to defer to. If phase 1 failed, the fetch itself went wrong + * (e.g. a transport error) and must still be reported, even though + * the gitlinked commits are available. + */ + if (retvalue) + record_fetch_error(spf, task->sub->name); + out: fetch_task_free(task); return 0; diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh index 1242ee9185..188c674c89 100755 --- a/t/t5526-fetch-submodules.sh +++ b/t/t5526-fetch-submodules.sh @@ -1262,4 +1262,76 @@ test_expect_success "fetch --all with --no-recurse-submodules only fetches super ! grep "Fetching submodule" fetch-log ' +# Create an isolated environment for submodule fetch error tests. +# +# Sets up sub_bare (the submodule upstream), super_bare (the superproject +# upstream), super_work (a working clone of super_bare with an initialized +# submodule), and clone (a clone of super_bare with an initialized submodule +# at a reachable commit). The caller can then create an unreachable commit +# and push the superproject to put the clone one commit behind a state it +# cannot fully fetch. +# +# Usage: create_err_env <envdir> +create_err_env () { + local envdir="$1" && + mkdir "$envdir" && + + git init --bare "$envdir/sub_bare" && + git clone "$envdir/sub_bare" "$envdir/sub_work" && + test_commit -C "$envdir/sub_work" "${envdir}_base" && + git -C "$envdir/sub_work" push && + + git init --bare "$envdir/super_bare" && + git clone "$envdir/super_bare" "$envdir/super_work" && + git -C "$envdir/super_work" submodule add \ + "$pwd/$envdir/sub_bare" sub && + git -C "$envdir/super_work" commit -m "add submodule" && + git -C "$envdir/super_work" push && + + git clone "$envdir/super_bare" "$envdir/clone" && + git -C "$envdir/clone" submodule update --init +} + +# Push a commit to <envdir>/super_bare that records a submodule SHA that is +# present locally in super_work/sub but NOT pushed to sub_bare, making the +# submodule commit unreachable from clone's sub remote. +push_unreachable_commit () { + local envdir="$1" && + git -C "$envdir/super_work/sub" commit --allow-empty -m "unreachable" && + git -C "$envdir/super_work" add sub && + git -C "$envdir/super_work" commit -m "point sub to unreachable commit" && + git -C "$envdir/super_work" push +} + +test_expect_success 'setup for submodule fetch error tests' ' + git config --global protocol.file.allow always +' + +test_expect_success 'failed submodule fetch is fatal even when its commits are present locally' ' + # Create the same commit (unreferenced, via commit-tree with fixed + # dates) in both super_work/sub and clone/sub, point the gitlink at + # it, and break clone/sub'\''s remote. The commit exists in clone/sub + # but is unreachable, so the submodule stays in the changed list; the + # fetch failure must still be reported even though there is nothing + # left to fetch by commit hash. + test_when_finished "rm -fr env_phase1" && + create_err_env env_phase1 && + commit=$(GIT_AUTHOR_DATE="1234567890 +0000" \ + GIT_COMMITTER_DATE="1234567890 +0000" \ + git -C env_phase1/super_work/sub commit-tree \ + "HEAD^{tree}" -p HEAD -m present) && + present=$(GIT_AUTHOR_DATE="1234567890 +0000" \ + GIT_COMMITTER_DATE="1234567890 +0000" \ + git -C env_phase1/clone/sub commit-tree \ + "HEAD^{tree}" -p HEAD -m present) && + test "$commit" = "$present" && + git -C env_phase1/super_work/sub checkout "$commit" && + git -C env_phase1/super_work add sub && + git -C env_phase1/super_work commit -m "gitlink to locally-present commit" && + git -C env_phase1/super_work push && + git -C env_phase1/clone/sub remote set-url origin "$pwd/env_phase1/missing" && + test_must_fail git -C env_phase1/clone fetch --recurse-submodules 2>err && + grep "Errors during submodule fetch" err +' + test_done -- 2.54.0 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v4 2/2] fetch: add fetch.submoduleErrors to make submodule fetch errors non-fatal 2026-07-14 13:29 ` [PATCH v4 0/2] fetch: make submodule fetch errors configurable Paulius Zaleckas 2026-07-14 13:29 ` [PATCH v4 1/2] submodule: fix premature failure in recursive submodule fetch Paulius Zaleckas @ 2026-07-14 13:29 ` Paulius Zaleckas 2026-07-14 15:34 ` Junio C Hamano 2026-07-14 17:14 ` Junio C Hamano 1 sibling, 2 replies; 10+ messages in thread From: Paulius Zaleckas @ 2026-07-14 13:29 UTC (permalink / raw) To: git Cc: Junio C Hamano, Paulius Zaleckas, Ævar Arnfjörð Bjarmason, Glen Choo, Patrick Steinhardt When fetching with --recurse-submodules, a submodule commit that is not yet reachable from any of the submodule's remote refs causes the entire fetch to fail. This is overly strict when the missing commit belongs to an upstream branch that is still being prepared (e.g. an in-progress merge topic): the local branch does not need that commit, so there is no reason to treat its absence as fatal. Add a new config key fetch.submoduleErrors (values: fail/warn) and a corresponding --submodule-errors=(fail|warn) command-line option that control this behaviour. The default remains fail (existing behaviour); setting the value to warn causes submodule fetch failures to be reported on stderr without affecting the overall exit status of git fetch / git pull. Forward the option to child fetches in add_options_to_argv() so that it also takes effect for `git fetch --all` / `--multiple` (where per-remote child processes handle the submodule recursion themselves) and for nested submodule recursion. The resolved value is forwarded whenever it was set explicitly, in either direction: the per-remote children re-read the repository configuration, so a command-line --submodule-errors=fail must be passed down to them to override fetch.submoduleErrors=warn from the configuration. When neither the configuration nor the command line sets a value, nothing is forwarded and the child processes fall back to their own configuration. Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Paulius Zaleckas <paulius.zaleckas@gmail.com> --- Documentation/config/fetch.adoc | 14 +++++ Documentation/fetch-options.adoc | 8 +++ builtin/fetch.c | 46 ++++++++++++++++- submodule.c | 8 ++- submodule.h | 7 ++- t/t5526-fetch-submodules.sh | 89 ++++++++++++++++++++++++++++++++ 6 files changed, 168 insertions(+), 4 deletions(-) diff --git a/Documentation/config/fetch.adoc b/Documentation/config/fetch.adoc index 04ac90912d..5c9c942a70 100644 --- a/Documentation/config/fetch.adoc +++ b/Documentation/config/fetch.adoc @@ -10,6 +10,20 @@ reference. Defaults to `on-demand`, or to the value of `submodule.recurse` if set. +`fetch.submoduleErrors`:: + Controls how errors from submodule fetches are handled when + `--recurse-submodules` is in effect. When set to `fail` (the default), + any submodule fetch error causes the overall `git fetch` or `git pull` + to exit with a non-zero status. When set to `warn`, submodule fetch + errors are reported to standard error but do not affect the exit + status of the command. This is useful when working in repositories + where some branches reference submodule commits that are not yet + available on the submodule remote, but those commits are not needed + for the currently checked-out branch. ++ +The value of this option can be overridden by the `--submodule-errors` +option of linkgit:git-fetch[1]. + `fetch.fsckObjects`:: If it is set to true, git-fetch-pack will check all fetched objects. See `transfer.fsckObjects` for what's diff --git a/Documentation/fetch-options.adoc b/Documentation/fetch-options.adoc index 035f780e58..78525f6848 100644 --- a/Documentation/fetch-options.adoc +++ b/Documentation/fetch-options.adoc @@ -294,6 +294,14 @@ ifndef::git-pull[] `--no-recurse-submodules`:: Disable recursive fetching of submodules (this has the same effect as using the `--recurse-submodules=no` option). + +`--submodule-errors=(fail|warn)`:: + Control how errors from submodule fetches are handled when + `--recurse-submodules` is in effect. When set to `fail` (the default), + any submodule fetch error causes the overall `git fetch` to exit with a + non-zero status. When set to `warn`, submodule fetch errors are reported + to standard error but do not affect the exit status of the command. Can + also be configured via `fetch.submoduleErrors`. See linkgit:git-config[1]. endif::git-pull[] `--set-upstream`:: diff --git a/builtin/fetch.c b/builtin/fetch.c index c1d7c672f4..41122e17b3 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -110,6 +110,7 @@ struct fetch_config { int recurse_submodules; int parallel; int submodule_fetch_jobs; + int submodule_errors; }; static int git_fetch_config(const char *k, const char *v, @@ -152,6 +153,19 @@ static int git_fetch_config(const char *k, const char *v, return 0; } + if (!strcmp(k, "fetch.submoduleerrors")) { + if (!v) + return config_error_nonbool(k); + else if (!strcasecmp(v, "fail")) + fetch_config->submodule_errors = SUBMODULE_ERRORS_FAIL; + else if (!strcasecmp(v, "warn")) + fetch_config->submodule_errors = SUBMODULE_ERRORS_WARN; + else + die(_("invalid value for '%s': '%s'"), + "fetch.submoduleErrors", v); + return 0; + } + if (!strcmp(k, "fetch.parallel")) { fetch_config->parallel = git_config_int(k, v, ctx->kvi); if (fetch_config->parallel < 0) @@ -2205,6 +2219,10 @@ static void add_options_to_argv(struct strvec *argv, strvec_push(argv, "--no-recurse-submodules"); else if (config->recurse_submodules == RECURSE_SUBMODULES_ON_DEMAND) strvec_push(argv, "--recurse-submodules=on-demand"); + if (config->submodule_errors == SUBMODULE_ERRORS_FAIL) + strvec_push(argv, "--submodule-errors=fail"); + else if (config->submodule_errors == SUBMODULE_ERRORS_WARN) + strvec_push(argv, "--submodule-errors=warn"); if (tags == TAGS_SET) strvec_push(argv, "--tags"); else if (tags == TAGS_UNSET) @@ -2464,6 +2482,19 @@ static int fetch_one(struct remote *remote, int argc, const char **argv, return exit_code; } +static int option_parse_submodule_errors(const struct option *opt, + const char *arg, int unset) +{ + int *v = opt->value; + if (unset || !strcasecmp(arg, "fail")) + *v = SUBMODULE_ERRORS_FAIL; + else if (!strcasecmp(arg, "warn")) + *v = SUBMODULE_ERRORS_WARN; + else + die(_("invalid value for '%s': '%s'"), "--submodule-errors", arg); + return 0; +} + int cmd_fetch(int argc, const char **argv, const char *prefix, @@ -2477,6 +2508,7 @@ int cmd_fetch(int argc, .recurse_submodules = RECURSE_SUBMODULES_DEFAULT, .parallel = 1, .submodule_fetch_jobs = -1, + .submodule_errors = -1, /* unset */ }; const char *submodule_prefix = ""; const char *bundle_uri; @@ -2491,6 +2523,7 @@ int cmd_fetch(int argc, int max_jobs = -1; int recurse_submodules_cli = RECURSE_SUBMODULES_DEFAULT; int recurse_submodules_default = RECURSE_SUBMODULES_ON_DEMAND; + int submodule_errors_cli = -1; /* -1: not set on command line */ int fetch_write_commit_graph = -1; int stdin_refspecs = 0; int negotiate_only = 0; @@ -2527,6 +2560,10 @@ int cmd_fetch(int argc, OPT_CALLBACK_F(0, "recurse-submodules", &recurse_submodules_cli, N_("on-demand"), N_("control recursive fetching of submodules"), PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules), + OPT_CALLBACK_F(0, "submodule-errors", &submodule_errors_cli, + N_("(fail|warn)"), + N_("control how submodule fetch errors are handled"), + 0, option_parse_submodule_errors), OPT_BOOL(0, "dry-run", &dry_run, N_("dry run")), OPT_BOOL(0, "porcelain", &porcelain, N_("machine-readable output")), @@ -2616,6 +2653,9 @@ int cmd_fetch(int argc, if (recurse_submodules_cli != RECURSE_SUBMODULES_DEFAULT) config.recurse_submodules = recurse_submodules_cli; + if (submodule_errors_cli != -1) + config.submodule_errors = submodule_errors_cli; + if (negotiate_only) { switch (recurse_submodules_cli) { case RECURSE_SUBMODULES_OFF: @@ -2819,11 +2859,14 @@ int cmd_fetch(int argc, if (!result && remote && (config.recurse_submodules != RECURSE_SUBMODULES_OFF)) { struct strvec options = STRVEC_INIT; int max_children = max_jobs; + int submodule_errors = config.submodule_errors; if (max_children < 0) max_children = config.submodule_fetch_jobs; if (max_children < 0) max_children = config.parallel; + if (submodule_errors < 0) + submodule_errors = SUBMODULE_ERRORS_FAIL; add_options_to_argv(&options, &config); trace2_region_enter_printf("fetch", "recurse-submodule", the_repository, "%s", submodule_prefix); @@ -2833,7 +2876,8 @@ int cmd_fetch(int argc, config.recurse_submodules, recurse_submodules_default, verbosity < 0, - max_children); + max_children, + submodule_errors); trace2_region_leave_printf("fetch", "recurse-submodule", the_repository, "%s", submodule_prefix); strvec_clear(&options); } diff --git a/submodule.c b/submodule.c index 8bcef68a42..da4ace751f 100644 --- a/submodule.c +++ b/submodule.c @@ -1409,6 +1409,7 @@ struct submodule_parallel_fetch { int oid_fetch_tasks_nr, oid_fetch_tasks_alloc; struct strbuf submodules_with_errors; + int submodule_errors; }; #define SPF_INIT { \ .args = STRVEC_INIT, \ @@ -1565,7 +1566,8 @@ static struct fetch_task *fetch_task_create(struct submodule_parallel_fetch *spf static void record_fetch_error(struct submodule_parallel_fetch *spf, const char *name) { - spf->result = 1; + if (spf->submodule_errors == SUBMODULE_ERRORS_FAIL) + spf->result = 1; strbuf_addf(&spf->submodules_with_errors, "\t%s\n", name); } @@ -1851,7 +1853,8 @@ int fetch_submodules(struct repository *r, const struct strvec *options, const char *prefix, int command_line_option, int default_option, - int quiet, int max_parallel_jobs) + int quiet, int max_parallel_jobs, + int submodule_errors) { struct submodule_parallel_fetch spf = SPF_INIT; const struct run_process_parallel_opts opts = { @@ -1871,6 +1874,7 @@ int fetch_submodules(struct repository *r, spf.default_option = default_option; spf.quiet = quiet; spf.prefix = prefix; + spf.submodule_errors = submodule_errors; if (!r->worktree) goto out; diff --git a/submodule.h b/submodule.h index b10e16e6c0..c80b687d2a 100644 --- a/submodule.h +++ b/submodule.h @@ -90,12 +90,17 @@ int should_update_submodules(void); */ const struct submodule *submodule_from_ce(const struct cache_entry *ce); void check_for_new_submodule_commits(struct object_id *oid); +/* Values for the submodule_errors parameter of fetch_submodules(). */ +#define SUBMODULE_ERRORS_FAIL 0 /* submodule fetch errors are fatal (default) */ +#define SUBMODULE_ERRORS_WARN 1 /* submodule fetch errors are non-fatal warnings */ + int fetch_submodules(struct repository *r, const struct strvec *options, const char *prefix, int command_line_option, int default_option, - int quiet, int max_parallel_jobs); + int quiet, int max_parallel_jobs, + int submodule_errors); unsigned is_submodule_modified(const char *path, int ignore_untracked); int submodule_uses_gitfile(const char *path); diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh index 188c674c89..504ab200ef 100755 --- a/t/t5526-fetch-submodules.sh +++ b/t/t5526-fetch-submodules.sh @@ -1307,6 +1307,57 @@ test_expect_success 'setup for submodule fetch error tests' ' git config --global protocol.file.allow always ' +test_expect_success 'fetch --recurse-submodules fails when submodule commit is unreachable (default)' ' + test_when_finished "rm -fr env_default" && + create_err_env env_default && + push_unreachable_commit env_default && + test_must_fail git -C env_default/clone fetch --recurse-submodules 2>err && + grep "Errors during submodule fetch" err +' + +test_expect_success 'fetch.submoduleErrors=warn: unreachable submodule commit is non-fatal' ' + test_when_finished "rm -fr env_warn_cfg" && + create_err_env env_warn_cfg && + push_unreachable_commit env_warn_cfg && + git -C env_warn_cfg/clone -c fetch.submoduleErrors=warn \ + fetch --recurse-submodules 2>err && + grep "Errors during submodule fetch" err +' + +test_expect_success '--submodule-errors=warn: unreachable submodule commit is non-fatal' ' + test_when_finished "rm -fr env_warn_cli" && + create_err_env env_warn_cli && + push_unreachable_commit env_warn_cli && + git -C env_warn_cli/clone fetch --recurse-submodules \ + --submodule-errors=warn 2>err && + grep "Errors during submodule fetch" err +' + +test_expect_success '--submodule-errors=fail: unreachable submodule commit is fatal' ' + test_when_finished "rm -fr env_fail_cli" && + create_err_env env_fail_cli && + push_unreachable_commit env_fail_cli && + test_must_fail git -C env_fail_cli/clone fetch --recurse-submodules \ + --submodule-errors=fail 2>err && + grep "Errors during submodule fetch" err +' + +test_expect_success 'fetch.submoduleErrors=warn does not suppress successful fetch' ' + # A new reachable submodule commit (pushed to sub_bare) should be + # fetched without any error summary. + test_when_finished "rm -fr env_ok" && + create_err_env env_ok && + test_commit -C env_ok/sub_work reachable_ok && + git -C env_ok/sub_work push && + git -C env_ok/super_work submodule update --remote && + git -C env_ok/super_work add sub && + git -C env_ok/super_work commit -m "point sub to reachable commit" && + git -C env_ok/super_work push && + git -C env_ok/clone -c fetch.submoduleErrors=warn \ + fetch --recurse-submodules 2>err && + ! grep "Errors during submodule fetch" err +' + test_expect_success 'failed submodule fetch is fatal even when its commits are present locally' ' # Create the same commit (unreferenced, via commit-tree with fixed # dates) in both super_work/sub and clone/sub, point the gitlink at @@ -1334,4 +1385,42 @@ test_expect_success 'failed submodule fetch is fatal even when its commits are p grep "Errors during submodule fetch" err ' +test_expect_success '--submodule-errors=warn is honored by fetch --all' ' + # A second remote forces fetch_multiple(), which hands the submodule + # recursion off to per-remote child processes; the option must be + # forwarded to them. + test_when_finished "rm -fr env_all" && + create_err_env env_all && + push_unreachable_commit env_all && + git -C env_all/clone remote add second "$pwd/env_all/super_bare" && + git -C env_all/clone fetch --all --recurse-submodules \ + --submodule-errors=warn 2>err && + grep "Errors during submodule fetch" err +' + +test_expect_success '--submodule-errors=fail overrides warn config for fetch --all' ' + # The per-remote child processes re-read the repository config, so + # the command-line override must be forwarded to them explicitly. + test_when_finished "rm -fr env_override" && + create_err_env env_override && + push_unreachable_commit env_override && + git -C env_override/clone remote add second "$pwd/env_override/super_bare" && + git -C env_override/clone config fetch.submoduleErrors warn && + test_must_fail git -C env_override/clone fetch --all --recurse-submodules \ + --submodule-errors=fail 2>err && + grep "Errors during submodule fetch" err +' + +test_expect_success 'fetch.submoduleErrors=warn: inaccessible submodule is non-fatal' ' + test_when_finished "rm -fr env_access" && + create_err_env env_access && + rm env_access/clone/sub/.git && + rm -r env_access/clone/.git/modules/sub && + git -C env_access/clone -c fetch.submoduleErrors=warn \ + fetch --recurse-submodules 2>err && + grep "Could not access submodule" err && + test_must_fail git -C env_access/clone fetch --recurse-submodules 2>err && + grep "Could not access submodule" err +' + test_done -- 2.54.0 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH v4 2/2] fetch: add fetch.submoduleErrors to make submodule fetch errors non-fatal 2026-07-14 13:29 ` [PATCH v4 2/2] fetch: add fetch.submoduleErrors to make submodule fetch errors non-fatal Paulius Zaleckas @ 2026-07-14 15:34 ` Junio C Hamano 2026-07-14 17:14 ` Junio C Hamano 1 sibling, 0 replies; 10+ messages in thread From: Junio C Hamano @ 2026-07-14 15:34 UTC (permalink / raw) To: Paulius Zaleckas Cc: git, Ævar Arnfjörð Bjarmason, Glen Choo, Patrick Steinhardt Paulius Zaleckas <paulius.zaleckas@gmail.com> writes: > t/t5526-fetch-submodules.sh | 89 ++++++++++++++++++++++++++++++++ > 6 files changed, 168 insertions(+), 4 deletions(-) In addition to what was pointed out by Ramsay in his squashable patch <387a34d5-fdf5-4513-9aaf-4e73d9304c1d@ramsayjones.plus.com> this round adds another use of raw grep that is caught by the test framework. commit 8f7761ee72b3669c1aee98142852437d016c785c Author: Junio C Hamano <gitster@pobox.com> Date: Tue Jul 14 08:31:57 2026 -0700 fixup! fetch: add fetch.submoduleErrors to make submodule fetch errors non-fatal diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh index 614d45ab71..19d17440cf 100755 --- a/t/t5526-fetch-submodules.sh +++ b/t/t5526-fetch-submodules.sh @@ -1408,7 +1408,7 @@ test_expect_success '--submodule-errors=fail overrides warn config for fetch --a git -C env_override/clone config fetch.submoduleErrors warn && test_must_fail git -C env_override/clone fetch --all --recurse-submodules \ --submodule-errors=fail 2>err && - grep "Errors during submodule fetch" err + test_grep "Errors during submodule fetch" err ' test_expect_success 'fetch.submoduleErrors=warn: inaccessible submodule is non-fatal' ' ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH v4 2/2] fetch: add fetch.submoduleErrors to make submodule fetch errors non-fatal 2026-07-14 13:29 ` [PATCH v4 2/2] fetch: add fetch.submoduleErrors to make submodule fetch errors non-fatal Paulius Zaleckas 2026-07-14 15:34 ` Junio C Hamano @ 2026-07-14 17:14 ` Junio C Hamano 1 sibling, 0 replies; 10+ messages in thread From: Junio C Hamano @ 2026-07-14 17:14 UTC (permalink / raw) To: Paulius Zaleckas Cc: git, Ævar Arnfjörð Bjarmason, Glen Choo, Patrick Steinhardt Paulius Zaleckas <paulius.zaleckas@gmail.com> writes: > + if (!strcmp(k, "fetch.submoduleerrors")) { > + if (!v) > + return config_error_nonbool(k); > + else if (!strcasecmp(v, "fail")) > + fetch_config->submodule_errors = SUBMODULE_ERRORS_FAIL; > + else if (!strcasecmp(v, "warn")) > + fetch_config->submodule_errors = SUBMODULE_ERRORS_WARN; > + else > + die(_("invalid value for '%s': '%s'"), > + "fetch.submoduleErrors", v); > + return 0; > + } Two points. * Do not use strcasecmp() on the value. While "fetch.submoduleerrors" may be case-insenstive, the value does not have to be. We do not want to encourage users to write "[fetch] submoduleErrors = Fail", as some people may want to write third-party add-on scripts that parse "git config --get fetch.submoduleerrors" output. For example: error_handling=$(git config --get fetch.submoduleErrors) case "$error_handling" in fail) ... do something ... ;; warn) ... do something else ... ;; esac We should not force them to write extra code to handle the value case-insensitively. * Since you need to convert between the enum and the string here, in option_parse_submodule_errors(), and in add_options_to_argv(), defining a pair of parse/format functions would be cleaner. /* really private - use accessors to parse and format */ static const char *submodule_errors_[] = { [SUBMODULE_ERRORS_FAIL] = "fail", [SUBMODULE_ERRORS_WARN] = "warn", }; static const char *submodule_error(int num) { assert(0 <= num && num < ARRAY_SIZE(submodule_errors_)); return submodule_errors[num]; } static int parse_submodule_error(const char *name) { for (int num = 0; num < ARRAY_SIZE(submodule_errors_); num++) if (!strcmp(submodule_errors_[num], name)) return num; return -1; } The configuration parsing block would then become: if (!strcmp(k, "fetch.submoduleerrors")) { int num; if (!v) return config_error_nonbool(k); num = parse_submodule_error(v); if (num < 0) die(_("invalid value..."), ...); fetch_config->submodule_errors = num; return 0; } This approach is much more maintainable. You only need to keep the submodule_errors_[] array up to date with respect to the error-handling preprocessor macros. Some reviewers might suggest converting these macros into a proper enum. I would not object to that change, but I would not bother doing it myself as I do not personally care much about the distinction between an enum and a preprocessor macro in this context. > @@ -2205,6 +2219,10 @@ static void add_options_to_argv(struct strvec *argv, > strvec_push(argv, "--no-recurse-submodules"); > else if (config->recurse_submodules == RECURSE_SUBMODULES_ON_DEMAND) > strvec_push(argv, "--recurse-submodules=on-demand"); > + if (config->submodule_errors == SUBMODULE_ERRORS_FAIL) > + strvec_push(argv, "--submodule-errors=fail"); > + else if (config->submodule_errors == SUBMODULE_ERRORS_WARN) > + strvec_push(argv, "--submodule-errors=warn"); This part then becomes: if (config->submodule_errors < 0) ; /* nothing */ else { const char *name = submodule_error(config->submodule_errors); strvec_push(argv, "--submodule-errors=%s", name); } This is, again, much more miantainable. > +static int option_parse_submodule_errors(const struct option *opt, > + const char *arg, int unset) > +{ > + int *v = opt->value; > + if (unset || !strcasecmp(arg, "fail")) > + *v = SUBMODULE_ERRORS_FAIL; > + else if (!strcasecmp(arg, "warn")) > + *v = SUBMODULE_ERRORS_WARN; > + else > + die(_("invalid value for '%s': '%s'"), "--submodule-errors", arg); > + return 0; > +} Updating this function is left as an exercise ;-) ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2026-07-14 17:14 UTC | newest] Thread overview: 10+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-07-10 12:26 [PATCH v3 0/2] fetch: make submodule fetch errors configurable Paulius Zaleckas 2026-07-10 12:26 ` [PATCH v3 1/2] submodule: fix premature failure in recursive submodule fetch Paulius Zaleckas 2026-07-10 12:26 ` [PATCH v3 2/2] fetch: add fetch.submoduleErrors to make submodule fetch errors non-fatal Paulius Zaleckas 2026-07-10 22:21 ` Junio C Hamano 2026-07-14 13:29 ` Paulius Zaleckas 2026-07-14 13:29 ` [PATCH v4 0/2] fetch: make submodule fetch errors configurable Paulius Zaleckas 2026-07-14 13:29 ` [PATCH v4 1/2] submodule: fix premature failure in recursive submodule fetch Paulius Zaleckas 2026-07-14 13:29 ` [PATCH v4 2/2] fetch: add fetch.submoduleErrors to make submodule fetch errors non-fatal Paulius Zaleckas 2026-07-14 15:34 ` Junio C Hamano 2026-07-14 17:14 ` Junio C Hamano
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox