All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com>
To: git@vger.kernel.org
Cc: peff@peff.net, Junio C Hamano <gitster@pobox.com>
Subject: [PATCH v2 0/1] send-pack: set core.warnAmbiguousRefs=false
Date: Tue, 06 Nov 2018 12:34:46 -0800 (PST)	[thread overview]
Message-ID: <pull.68.v2.git.gitgitgadget@gmail.com> (raw)
In-Reply-To: <pull.68.git.gitgitgadget@gmail.com>

I've been looking into the performance of git push for very large repos. Our
users are reporting that 60-80% of git push time is spent during the
"Enumerating objects" phase of git pack-objects.

A git push process runs several processes during its run, but one includes 
git send-pack which calls git pack-objects and passes the known have/wants
into stdin using object ids. However, the default setting for 
core.warnAmbiguousRefs requires git pack-objects to check for ref names
matching the ref_rev_parse_rules array in refs.c. This means that every
object is triggering at least six "file exists?" queries.

When there are a lot of refs, this can add up significantly! My PerfView
trace for a simple push measured 3 seconds spent checking these paths.

The fix is to set the global warn_on_object_refname_ambiguity to 0 for the
section that is performing these object reads.

In addition to this patch submission, we are looking into merging it into
our fork sooner [1].

[1] https://github.com/Microsoft/git/pull/67

Changes in V2: Instead of using the "-c" flag from send-pack, just set the
global. I left the name of the cover letter the same to not confuse anyone
viewing the message without threading.

Derrick Stolee (1):
  pack-objects: ignore ambiguous object warnings

 builtin/pack-objects.c | 6 ++++++
 1 file changed, 6 insertions(+)


base-commit: cae598d9980661a978e2df4fb338518f7bf09572
Published-As: https://github.com/gitgitgadget/git/releases/tags/pr-68%2Fderrickstolee%2Fsend-pack-config-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-68/derrickstolee/send-pack-config-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/68

Range-diff vs v1:

 1:  1ef2c51550 < -:  ---------- send-pack: set core.warnAmbiguousRefs=false
 -:  ---------- > 1:  002868ee6b pack-objects: ignore ambiguous object warnings

-- 
gitgitgadget

  parent reply	other threads:[~2018-11-06 20:34 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-11-06 19:13 [PATCH 0/1] send-pack: set core.warnAmbiguousRefs=false Derrick Stolee via GitGitGadget
2018-11-06 19:13 ` [PATCH 1/1] " Derrick Stolee via GitGitGadget
2018-11-06 19:44 ` [PATCH 0/1] " Jeff King
2018-11-06 19:51   ` Jeff King
2018-11-06 20:16     ` Derrick Stolee
2018-11-06 20:00   ` Derrick Stolee
2018-11-06 20:34 ` Derrick Stolee via GitGitGadget [this message]
2018-11-06 20:34   ` [PATCH v2 1/1] pack-objects: ignore ambiguous object warnings Derrick Stolee via GitGitGadget
2018-11-06 21:12     ` Jeff King
2018-11-07  1:54       ` Junio C Hamano
2018-11-07  8:52         ` Junio C Hamano
2018-11-07  9:21           ` Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=pull.68.v2.git.gitgitgadget@gmail.com \
    --to=gitgitgadget@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=peff@peff.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.