From: s@kazlauskas.me
To: git@vger.kernel.org
Subject: Fetching new refs gets progressively slower
Date: Sun, 9 Jul 2017 13:25:06 +0300 [thread overview]
Message-ID: <20170709102506.GA32425@kumabox> (raw)
I have a weird issue where fetching a large number of refs will start off with
lines like these:
* [new ref] refs/pull/10000/head -> origin/pr/10000
* [new ref] refs/pull/10001/head -> origin/pr/10001
going fairly fast, and then progressively getting slower and slower. By the time
git is working on 40 thousandth such ref, it seems like it is only handling
about 3-5 such “new ref”s.
These are the steps I used to reproduce:
$ git clone git@github.com:rust-lang/rust
$ # edit .git/config to add
$ # `fetch = +refs/pull/*/head:refs/remotes/origin/pr/*` under origin remote
$ git fetch
I tried this on three distinct file systems: zfs, ext4 and tmpfs, and on two
distinct systems (both with different SSDs in it). Both exhibit the
approximately the same behaviour. Both on a fairly recent version of Linux.
Here’s some timings:
System 1 (ext4) 97% (1047.74 real, 700.11 kernel, 319.89 user); 599476k resident
System 1 (tmpf) 97% (963.78 real, 647.51 kernel, 292.77 user); 600052k resident
System 1 (zfs) 98% (2116.66 real, 1715.86 kernel, 370.32 user); 531232k resident
System 2 (ext4) 97% (1036.56 real, 710.54 kernel, 300.81 user); 602160k resident
Git version is same on both systems: 2.13.1
I did not investigate the issue more throughoutly, but I suspect that git will
end up doing something resembling listing the contents of the directory of refs
for each “new ref” it is creating.
S.
next reply other threads:[~2017-07-09 10:25 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-07-09 10:25 s [this message]
2017-07-09 11:29 ` Fetching new refs gets progressively slower Jeff King
2017-08-17 15:12 ` [PATCH] files-backend: cheapen refname_available check when locking refs Michael Haggerty
2017-08-17 15:22 ` Jeff King
2017-08-17 17:56 ` Brandon Williams
2017-08-17 21:37 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170709102506.GA32425@kumabox \
--to=s@kazlauskas.me \
--cc=git@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).