All of lore.kernel.org
 help / color / mirror / Atom feed
From: Junio C Hamano <gitster@pobox.com>
To: Phil Hord <phil.hord@gmail.com>
Cc: git@vger.kernel.org
Subject: Re: [RFC PATCH 1/2] fetch-prune: optimize dangling-ref reporting
Date: Wed, 18 Jun 2025 14:50:49 -0700	[thread overview]
Message-ID: <xmqqzfe4d8hy.fsf@gitster.g> (raw)
In-Reply-To: <20250618211024.2332525-2-phil.hord@gmail.com> (Phil Hord's message of "Wed, 18 Jun 2025 14:08:39 -0700")

Phil Hord <phil.hord@gmail.com> writes:

> From: Phil Hord <phil.hord@gmail.com>
>
> When pruning during `git fetch` we check each pruned ref against the
> ref_store one at a time to decide whether to report it as dangling.
> This causes every local ref to be scanned for each ref being pruned.
>
> If there are N refs in the repo and M refs being pruned, this code is
> O(M*N). However, `git remote prune` uses a very similar function that
> is only O(N*log(M)).
>
> Remove the wasteful ref scanning for each pruned ref and use the faster
> version already available in refs_warn_dangling_symrefs.
>
> In a repo with 126,000 refs, where I was pruning 28,000 refs, this
> code made about 3.6 billion calls to strcmp and consumed 410 seconds
> of CPU. (Invariably in that time, my remote would timeout and the
> fetch would fail anyway.)
>
> After this change, the same operation completes in under 4 seconds.

Nice.

  reply	other threads:[~2025-06-18 21:50 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-06-18 21:08 [RFC PATCH 0/2] fetch --prune performance problem Phil Hord
2025-06-18 21:08 ` [RFC PATCH 1/2] fetch-prune: optimize dangling-ref reporting Phil Hord
2025-06-18 21:50   ` Junio C Hamano [this message]
2025-06-18 23:18   ` Jacob Keller
2025-06-19  4:00   ` Jeff King
2025-06-19 11:01     ` Lidong Yan
2025-06-19 14:41       ` Lidong Yan
2025-06-18 21:08 ` [RFC PATCH 2/2] refs: remove old refs_warn_dangling_symref Phil Hord
2025-06-18 23:15 ` [RFC PATCH 0/2] fetch --prune performance problem Jacob Keller
2025-06-19  3:37   ` Jeff King
2025-06-19 17:18     ` Junio C Hamano
     [not found]     ` <CABURp0p4d0JPg=-cW1OZdFQJ+vNT_0PDd9Rv3oz6toFGqGv5=g@mail.gmail.com>
2025-06-23 23:32       ` Jacob Keller
2025-06-23 23:41         ` Junio C Hamano
     [not found]         ` <CABURp0q-1FGmD+PJeSQ=xvyDN6ZYn1O7Fh8i1OojfD2WQCqgcw@mail.gmail.com>
2025-06-23 23:46           ` Jacob Keller

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=xmqqzfe4d8hy.fsf@gitster.g \
    --to=gitster@pobox.com \
    --cc=git@vger.kernel.org \
    --cc=phil.hord@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.