From: "Ravi Mistry via GitGitGadget" <gitgitgadget@gmail.com>
To: git@vger.kernel.org
Cc: Vaidas Pilkauskas <vaidas.pilkauskas@shopify.com>,
Patrick Steinhardt <ps@pks.im>,
"brian m. carlson" <sandals@crustytoothpaste.net>,
Ravi Mistry <rmistry@google.com>,
Ravi Mistry <rmistry@google.com>
Subject: [PATCH v4 5/5] http: default http.retryAfter to 10 seconds
Date: Thu, 19 Nov 2026 15:58:03 +0000 [thread overview]
Message-ID: <7d4335839dce4d0d9030c9b23b6f473fa127168d.1795103883.git.gitgitgadget@gmail.com> (raw)
In-Reply-To: <pull.2287.v4.git.1795103883.gitgitgadget@gmail.com>
From: Ravi Mistry <rmistry@google.com>
When http.maxRetries is enabled and a server returns HTTP 429 (or an
opt-in status code via http.retryStatusCodes) without a Retry-After
header, http.retryAfter currently defaults to 0. That causes Git to
retry immediately in a tight loop on every attempt, defeating
exponential backoff and continuing to hammer a rate-limiting server
right after it asked the client to back off.
Default http.retryAfter to 10 seconds (backing off as 10s, 20s, 40s,
..., capped at http.maxRetryTime) so that overloaded Git servers get
meaningful breathing room while keeping interactive commands reasonably
responsive. Callers or tests that want immediate retries can still set
http.retryAfter=0 explicitly.
Because libcurl's CURLINFO_RETRY_AFTER returns 0 when Retry-After is
absent or malformed and <= 0 when the server sends "Retry-After: 0" or a
past HTTP-date, inspect the Retry-After response header via
curl_easy_header() (enabled by default since libcurl 7.84.0) when
CURLINFO_RETRY_AFTER <= 0 so that an explicit "Retry-After: 0" or past
HTTP-date continues to retry immediately rather than falling back to the
new 10-second default.
Also drop the word "configured" from the http.maxRetryTime error message
and config documentation, since http.retryAfter can now exceed a lowered
http.maxRetryTime even when http.retryAfter was not explicitly set.
Signed-off-by: Ravi Mistry <rmistry@google.com>
---
Documentation/config/http.adoc | 26 +++++++++---------
git-curl-compat.h | 8 ++++++
http.c | 48 +++++++++++++++++++++++++++++++---
t/t5584-http-429-retry.sh | 34 +++++++++++++++++++++++-
4 files changed, 99 insertions(+), 17 deletions(-)
diff --git a/Documentation/config/http.adoc b/Documentation/config/http.adoc
index 72e9bf011d..051c29d7c4 100644
--- a/Documentation/config/http.adoc
+++ b/Documentation/config/http.adoc
@@ -332,16 +332,18 @@ http.retryAfter::
set of retryable status codes is configured by
`http.retryStatusCodes`, which includes only HTTP 429 (Too Many
Requests) by default.
- Defaults to 0, which retries immediately without waiting on every
- attempt (unless a Retry-After header is present). When set to a
- positive value and no Retry-After header is present, the first
- retry waits `http.retryAfter` seconds and the delay doubles on
- each subsequent retry attempt (`http.retryAfter * 2^(attempt - 1)`),
- capped at `http.maxRetryTime`. When a Retry-After header is
- present, its value takes precedence over this setting; however,
- automatic use of the server-provided `Retry-After` header requires
- libcurl 7.66.0 or later. On older versions, configure this setting
- manually to control the retry delay. Can be overridden by the
+ Defaults to 10. When positive and no Retry-After header is
+ present, the first retry waits `http.retryAfter` seconds and the
+ delay doubles on each subsequent retry attempt
+ (`http.retryAfter * 2^(attempt - 1)`), capped at
+ `http.maxRetryTime`. Set to 0 to retry immediately without waiting
+ on every attempt (unless a Retry-After header is present). When a
+ Retry-After header is present, its value takes precedence over
+ this setting; however, automatic use of the server-provided
+ `Retry-After` header requires libcurl 7.66.0 or later (7.84.0 or
+ later for `Retry-After: 0`). On versions older than 7.66.0,
+ `Retry-After` headers are ignored and `http.retryAfter` is always
+ used. Can be overridden by the
`GIT_HTTP_RETRY_AFTER` environment variable.
See also `http.maxRetries`, `http.maxRetryTime`, and
`http.retryStatusCodes`.
@@ -362,8 +364,8 @@ http.maxRetryTime::
handling a retryable HTTP status code. The set of retryable
status codes is configured by `http.retryStatusCodes`, which
includes only HTTP 429 (Too Many Requests) by default.
- If the server requests a delay (via Retry-After header) or if the
- configured `http.retryAfter` value itself exceeds this maximum,
+ If the server requests a delay (via Retry-After header) or if
+ `http.retryAfter` itself exceeds this maximum,
Git will fail immediately rather than waiting. Exponentially
backed-off delays computed from `http.retryAfter`, however, are
capped at this maximum rather than causing an immediate failure.
diff --git a/git-curl-compat.h b/git-curl-compat.h
index 25678b5dbd..c72a2a766c 100644
--- a/git-curl-compat.h
+++ b/git-curl-compat.h
@@ -52,6 +52,14 @@
#define GIT_CURL_HAVE_CURLINFO_RETRY_AFTER 1
#endif
+/**
+ * curl_easy_header() was added in 7.83.0 and enabled by default in 7.84.0,
+ * released in June 2022.
+ */
+#if LIBCURL_VERSION_NUM >= 0x075400
+#define GIT_CURL_HAVE_CURL_EASY_HEADER 1
+#endif
+
/**
* CURLOPT_PROTOCOLS_STR and CURLOPT_REDIR_PROTOCOLS_STR were added in 7.85.0,
* released in August 2022.
diff --git a/http.c b/http.c
index f7f4af7a31..1db5f7a69a 100644
--- a/http.c
+++ b/http.c
@@ -153,7 +153,7 @@ static char *http_ssl_backend;
static int http_schannel_check_revoke = 1;
-static long http_retry_after = 0;
+static long http_retry_after = 10;
static long http_max_retries = 0;
static long http_max_retry_time = 300;
static char *http_retry_status_codes;
@@ -334,6 +334,37 @@ static void closedown_active_slot(struct active_request_slot *slot)
slot->in_use = 0;
}
+#ifdef GIT_CURL_HAVE_CURL_EASY_HEADER
+static int has_zero_or_past_retry_after(CURL *curl)
+{
+ struct curl_header *hdr;
+ const char *p;
+
+ if (curl_easy_header(curl, "Retry-After", 0, CURLH_HEADER, -1,
+ &hdr) != CURLHE_OK)
+ return 0;
+ if (hdr->amount > 1 &&
+ curl_easy_header(curl, "Retry-After", hdr->amount - 1,
+ CURLH_HEADER, -1, &hdr) != CURLHE_OK)
+ return 0;
+
+ p = hdr->value;
+ while (isspace(*p))
+ p++;
+ if (!*p)
+ return 0;
+ if (curl_getdate(p, NULL) != (time_t)-1)
+ return 1;
+ if (*p != '0')
+ return 0;
+ while (*p == '0')
+ p++;
+ while (isspace(*p))
+ p++;
+ return !*p;
+}
+#endif
+
static void finish_active_slot(struct active_request_slot *slot)
{
closedown_active_slot(slot);
@@ -356,8 +387,17 @@ static void finish_active_slot(struct active_request_slot *slot)
{
curl_off_t retry_after;
if (curl_easy_getinfo(slot->curl, CURLINFO_RETRY_AFTER,
- &retry_after) == CURLE_OK && retry_after > 0)
- slot->results->retry_after = (long)retry_after;
+ &retry_after) == CURLE_OK) {
+ if (retry_after > 0)
+ slot->results->retry_after = (long)retry_after;
+ else if (retry_after < 0)
+ slot->results->retry_after = 0;
+#ifdef GIT_CURL_HAVE_CURL_EASY_HEADER
+ else if (slot->http_code >= 400 &&
+ has_zero_or_past_retry_after(slot->curl))
+ slot->results->retry_after = 0;
+#endif
+ }
}
#endif
}
@@ -2460,7 +2500,7 @@ static long handle_rate_limit_retry(long slot_retry_after, size_t attempt)
/* No Retry-After header provided, use configured default */
if (http_retry_after > http_max_retry_time) {
- error(_("configured http.retryAfter exceeds http.maxRetryTime (%ld > %ld seconds)"),
+ error(_("http.retryAfter exceeds http.maxRetryTime (%ld > %ld seconds)"),
http_retry_after, http_max_retry_time);
trace2_data_string("http", the_repository,
"http/429-error", "config-exceeds-max-retry-time");
diff --git a/t/t5584-http-429-retry.sh b/t/t5584-http-429-retry.sh
index afdea04b4e..43ed9b0de1 100755
--- a/t/t5584-http-429-retry.sh
+++ b/t/t5584-http-429-retry.sh
@@ -8,6 +8,14 @@ test_description='test HTTP 429 Too Many Requests retry logic'
start_httpd
+test_lazy_prereq CURL_EASY_HEADER - <<\EOT
+ build_option libcurl |
+ awk -F. '
+ $1 > 7 || ($1 == 7 && $2 >= 84) { ok = 1 }
+ END { exit !ok }
+ '
+EOT
+
test_expect_success 'setup test repository' '
test_commit initial &&
git clone --bare . "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
@@ -53,12 +61,14 @@ test_expect_success 'HTTP 429 with Retry-After is retried and succeeds' '
test_expect_success 'http.maxRetries=1 retries once and succeeds' '
test_config http.maxRetries 1 &&
+ test_config http.retryAfter 0 &&
git ls-remote "$HTTPD_URL/http_429/max-retries-one/none/repo.git" >output 2>err &&
test_grep "refs/heads/" output
'
test_expect_success 'http.maxRetries=2 fails after 3 failures and succeeds on 2 failures' '
test_config http.maxRetries 2 &&
+ test_config http.retryAfter 0 &&
test_must_fail git ls-remote "$HTTPD_URL/http_429/max-retries-two-fail-3/none/repo.git" 2>err &&
test_grep "rate limited" err &&
git ls-remote "$HTTPD_URL/http_429/max-retries-two-fail-2/none/repo.git" >output 2>err &&
@@ -72,6 +82,26 @@ test_expect_success 'Trace2 logs actual CURLINFO_RETRY_AFTER value instead of -1
test_grep "\"key\":\"http/429-retry-after\",\"value\":\"1\"" trace-retry-after.event
'
+test_expect_success 'http.retryAfter defaults to 10 seconds when unset' '
+ test_config http.maxRetries 1 &&
+ test_config http.maxRetryTime 9 &&
+ test_must_fail git ls-remote "$HTTPD_URL/http_429/default-retry-after-10/none/repo.git" 2>err &&
+ test_grep "http.retryAfter exceeds http.maxRetryTime (10 > 9 seconds)" err
+'
+
+test_expect_success CURL_EASY_HEADER 'HTTP 429 with Retry-After: 0 retries immediately' '
+ test_config http.maxRetries 3 &&
+ start=$(test-tool date getnanos) &&
+ GIT_TRACE2_EVENT="$(pwd)/trace-retry-zero.event" \
+ git ls-remote "$HTTPD_URL/http_429/retry-after-zero/0/repo.git" >output 2>err &&
+ duration=$(test-tool date getnanos $start) &&
+ duration_int=${duration%.*} &&
+ test "$duration_int" -lt 5 &&
+ test_grep "refs/heads/" output &&
+ test_grep ! -i "waiting.*retry" err &&
+ test_grep "\"key\":\"http/429-retry-after\",\"value\":\"0\"" trace-retry-zero.event
+'
+
test_expect_success 'HTTP 429 without Retry-After uses configured default' '
# Enable retries and configure default delay
test_config http.maxRetries 3 &&
@@ -161,7 +191,7 @@ test_expect_success 'HTTP 429 fails if configured http.retryAfter exceeds http.m
# Should fail quickly (no 100 second wait)
duration_int=${duration%.*} &&
test "$duration_int" -lt 99 &&
- test_grep "configured http.retryAfter.*exceeds.*http.maxRetryTime" err
+ test_grep "http.retryAfter.*exceeds.*http.maxRetryTime" err
'
test_expect_success 'HTTP 429 with Retry-After HTTP-date format' '
@@ -349,6 +379,7 @@ test_expect_success 'HTTP 503 honors Retry-After header when present' '
test_expect_success 'exhausted 502 retries report HTTP 502 error, not rate limited' '
test_config http.maxRetries 2 &&
+ test_config http.retryAfter 0 &&
test_config http.retryStatusCodes "429,502" &&
test_must_fail git ls-remote "$HTTPD_URL/http_429/exhausted-fail-3-status-502/none/repo.git" 2>err &&
test_grep "The requested URL returned error: 502" err &&
@@ -357,6 +388,7 @@ test_expect_success 'exhausted 502 retries report HTTP 502 error, not rate limit
test_expect_success 'GIT_HTTP_RETRY_STATUS_CODES and urlmatch http.<url>.retryStatusCodes' '
test_config http.maxRetries 2 &&
+ test_config http.retryAfter 0 &&
GIT_HTTP_RETRY_STATUS_CODES="429,502" \
git ls-remote "$HTTPD_URL/http_429/env-status-502/none/repo.git" >output 2>err &&
test_grep "refs/heads/" output &&
--
gitgitgadget
prev parent reply other threads:[~2026-11-19 15:58 UTC|newest]
Thread overview: 49+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-10-31 15:57 [PATCH 0/4] http: exponential backoff, configurable status codes, and post_rpc() retries Ravi Mistry via GitGitGadget
2026-10-31 15:57 ` [PATCH 1/4] http: fix retry count and Retry-After timing Ravi Mistry via GitGitGadget
2026-11-03 1:03 ` Junio C Hamano
2026-11-03 19:05 ` Ravi Mistry
2026-10-31 15:57 ` [PATCH 2/4] http: add exponential backoff without Retry-After Ravi Mistry via GitGitGadget
2026-11-03 13:21 ` Junio C Hamano
2026-11-03 19:05 ` Ravi Mistry
2026-10-31 15:57 ` [PATCH 3/4] http: add http.retryStatusCodes config option Ravi Mistry via GitGitGadget
2026-11-03 13:41 ` Junio C Hamano
2026-11-03 19:06 ` Ravi Mistry
2026-10-31 15:57 ` [PATCH 4/4] remote-curl: retry transient HTTP errors in post_rpc() Ravi Mistry via GitGitGadget
2026-11-03 14:47 ` Junio C Hamano
2026-11-03 18:57 ` [PATCH v2 0/4] http: exponential backoff, configurable status codes, and post_rpc() retries Ravi Mistry via GitGitGadget
2026-11-03 18:57 ` [PATCH v2 1/4] http: fix retry count and Retry-After timing Ravi Mistry via GitGitGadget
2026-11-03 18:57 ` [PATCH v2 2/4] http: add exponential backoff without Retry-After Ravi Mistry via GitGitGadget
2026-11-04 22:05 ` brian m. carlson
2026-11-05 0:10 ` Ravi Mistry
2026-11-05 17:45 ` Junio C Hamano
2026-11-05 22:32 ` brian m. carlson
2026-11-06 16:27 ` Ravi Mistry
2026-11-03 18:57 ` [PATCH v2 3/4] http: add http.retryStatusCodes config option Ravi Mistry via GitGitGadget
2026-11-03 20:56 ` Junio C Hamano
2026-11-03 18:57 ` [PATCH v2 4/4] remote-curl: retry transient HTTP errors in post_rpc() Ravi Mistry via GitGitGadget
2026-11-07 3:40 ` [PATCH v3 0/5] http: exponential backoff, configurable status codes, and post_rpc() retries Ravi Mistry via GitGitGadget
2026-11-07 3:40 ` [PATCH v3 1/5] http: fix retry count and Retry-After timing Ravi Mistry via GitGitGadget
2026-11-18 12:00 ` Patrick Steinhardt
2026-11-18 21:52 ` Ravi Mistry
2026-11-07 3:40 ` [PATCH v3 2/5] http: add exponential backoff without Retry-After Ravi Mistry via GitGitGadget
2026-11-18 12:00 ` Patrick Steinhardt
2026-11-18 21:53 ` Ravi Mistry
2026-11-07 3:40 ` [PATCH v3 3/5] http: add http.retryStatusCodes config option Ravi Mistry via GitGitGadget
2026-11-18 12:00 ` Patrick Steinhardt
2026-11-18 21:53 ` Ravi Mistry
2026-11-19 11:28 ` Patrick Steinhardt
2026-11-19 12:23 ` Ravi Mistry
2026-11-07 3:40 ` [PATCH v3 4/5] remote-curl: retry transient HTTP errors in post_rpc() Ravi Mistry via GitGitGadget
2026-11-18 12:01 ` Patrick Steinhardt
2026-11-18 21:53 ` Ravi Mistry
2026-11-07 3:40 ` [PATCH v3 5/5] http: default http.retryAfter to 10 seconds Ravi Mistry via GitGitGadget
2026-11-18 12:01 ` Patrick Steinhardt
2026-11-18 21:54 ` Ravi Mistry
2026-11-07 4:37 ` [PATCH v3 0/5] http: exponential backoff, configurable status codes, and post_rpc() retries Junio C Hamano
2026-11-10 3:58 ` Ravi Mistry
2026-11-19 15:57 ` [PATCH v4 " Ravi Mistry via GitGitGadget
2026-11-19 15:57 ` [PATCH v4 1/5] http: fix retry count and Retry-After timing Ravi Mistry via GitGitGadget
2026-11-19 15:58 ` [PATCH v4 2/5] http: add exponential backoff without Retry-After Ravi Mistry via GitGitGadget
2026-11-19 15:58 ` [PATCH v4 3/5] http: add http.retryStatusCodes config option Ravi Mistry via GitGitGadget
2026-11-19 15:58 ` [PATCH v4 4/5] remote-curl: retry transient HTTP errors in post_rpc() Ravi Mistry via GitGitGadget
2026-11-19 15:58 ` Ravi Mistry via GitGitGadget [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=7d4335839dce4d0d9030c9b23b6f473fa127168d.1795103883.git.gitgitgadget@gmail.com \
--to=gitgitgadget@gmail.com \
--cc=git@vger.kernel.org \
--cc=ps@pks.im \
--cc=rmistry@google.com \
--cc=sandals@crustytoothpaste.net \
--cc=vaidas.pilkauskas@shopify.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox