[PATCH 0/4] http: exponential backoff, configurable status codes, and post_rpc() retries

["Ravi Mistry via GitGitGadget" <gitgitgadget@gmail.com>]

Commit 640657ffd0 (http: add support for HTTP 429 rate limit retries,
2026-03-17) added HTTP 429 retry support using http.maxRetries,
http.retryAfter, and http.maxRetryTime.

In practice, when running Git against hosting backends and reverse proxies
under high concurrency, four gaps remain:

 1. Off-by-one retry count and Retry-After Trace2 timing: Because
    http_request_recoverable() pre-decrements the retry counter
    (--rate_limit_retries) in its loop condition, setting http.maxRetries=1
    performs 0 retries (and N configured retries performs N-1 retries). In
    addition, the Retry-After header is extracted after Trace2 logging has
    already run, so the http/429-retry-after Trace2 event always logs -1.

 2. No exponential backoff when Retry-After is absent: When a server does
    not send a Retry-After header, Git waits the same fixed http.retryAfter
    delay on every retry. If that delay is short (e.g., 1s), Git burns
    through all of its retries in just a few seconds before a busy server
    can recover. If that delay is long, even a brief glitch has to wait a
    long time on the first retry. Doubling the delay on each attempt (1s,
    2s, 4s, ...) keeps the first retry fast while giving the server more
    time to recover on later attempts, matching curl's own --retry behavior.

 3. Transient 5xx errors cannot be retried: Reverse proxies, load balancers,
    and Git backends frequently surface transient overload or connection
    resets as 502 Bad Gateway, 503 Service Unavailable, or 504 Gateway
    Timeout. Because 5xx errors are not universally transient across all
    environments, 429 should remain the default, with an opt-in config to
    specify additional status codes.

 4. HTTP POST requests are not retried: Git's existing 429 retry logic only
    covers HTTP GET requests (such as the initial /info/refs handshake).
    However, the actual data transfer during fetch and clone, as well as
    on-demand blob downloads in partial clones (during checkout, diff, or
    blame), are sent as HTTP POST requests (/git-upload-pack) via
    post_rpc(), which only retries authentication and immediately aborts on
    429 or 5xx.

This 4-patch series addresses all four gaps:

 * Patch 1/4 fixes the off-by-one retry count bug and extracts Retry-After
   in finish_active_slot() before Trace2 logging runs.
 * Patch 2/4 doubles http.retryAfter on each subsequent retry attempt (1s,
   2s, 4s, capped at http.maxRetryTime) when Retry-After is absent.
 * Patch 3/4 adds http.retryStatusCodes (and GIT_HTTP_RETRY_STATUS_CODES),
   defaulting to "429", so environments can opt into retrying codes like
   "429,502,503,504".
 * Patch 4/4 extracts a shared http_retry_wait() helper and extends retry
   support to HTTP POST requests in probe_rpc() and post_rpc() (while fixing
   a missing Content-Encoding: gzip header on retry).

Prior discussions:

 * https://lore.kernel.org/git/20201012184806.166251-3-smcallis@google.com/
 * https://lore.kernel.org/git/pull.2008.v6.git.1773752435.gitgitgadget@gmail.com/

Ravi Mistry (4):
  http: fix retry count and Retry-After timing
  http: add exponential backoff without Retry-After
  http: add http.retryStatusCodes config option
  remote-curl: retry transient HTTP errors in post_rpc()

 Documentation/config/http.adoc |  42 ++++++---
 http.c                         | 122 ++++++++++++++++++--------
 http.h                         |  12 ++-
 remote-curl.c                  |  75 ++++++++--------
 t/lib-httpd/http-429.sh        |  63 ++++++++++++--
 t/t5584-http-429-retry.sh      | 155 +++++++++++++++++++++++++++++++++
 6 files changed, 373 insertions(+), 96 deletions(-)


base-commit: 8d1b215c01512b9658d4525c54ff9b9245f9a7e8
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2287%2Frmistry%2Fhttp-exponential-retries-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2287/rmistry/http-exponential-retries-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/2287
-- 
gitgitgadget