From: Jeff King <peff@peff.net>
To: Duy Nguyen <pclouds@gmail.com>
Cc: "Michael Haggerty" <mhagger@alum.mit.edu>,
"Junio C Hamano" <gitster@pobox.com>,
"Johannes Sixt" <j6t@kdbg.org>,
"Torsten Bögershausen" <tboegi@web.de>,
"Git Mailing List" <git@vger.kernel.org>
Subject: Re: [PATCH v4 00/32] Lockfile correctness and refactoring
Date: Wed, 10 Sep 2014 06:30:14 -0400 [thread overview]
Message-ID: <20140910103014.GA15461@peff.net> (raw)
In-Reply-To: <CACsJy8AO5Ni1BNJGa1Wy5ZdX=HNE1UuVM-gnwtHBw01sfcZNrg@mail.gmail.com>
On Wed, Sep 10, 2014 at 05:25:36PM +0700, Duy Nguyen wrote:
> On Wed, Sep 10, 2014 at 3:13 PM, Jeff King <peff@peff.net> wrote:
> > I was running pack-refs on a repository with a very large number of
> > loose refs (about 1.8 million). Needless to say, this ran very slowly
> > and thrashed the disk, as that's almost 7G using 4K inodes. But it did
> > eventually generate a packed-refs file, at which point it tried to prune
> > the loose refs.
>
> Urghh.. ref advertisment for fetch/push would be unbelievably large.
> Gotta look at the "git protocol v2" again soon..
Yes, we don't let normal fetchers see these repos. They're only for
holding shared objects and the ref tips to keep them reachable. So we
never fetch from them, even locally. We only fetch to them from normal
repos (and never push to or from them at all).
It's still rather painful just to do normal things, though. Every git
operation loads the whole packed-refs file into memory. I'm biding my
time on the ref-backend patches that are being worked on. :)
-Peff
next prev parent reply other threads:[~2014-09-10 10:30 UTC|newest]
Thread overview: 66+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-09-06 7:50 [PATCH v4 00/32] Lockfile correctness and refactoring Michael Haggerty
2014-09-06 7:50 ` [PATCH v4 01/32] unable_to_lock_die(): rename function from unable_to_lock_index_die() Michael Haggerty
2014-09-06 7:50 ` [PATCH v4 02/32] api-lockfile: expand the documentation Michael Haggerty
2014-09-09 22:40 ` Junio C Hamano
2014-09-06 7:50 ` [PATCH v4 03/32] rollback_lock_file(): do not clear filename redundantly Michael Haggerty
2014-09-11 19:13 ` Ronnie Sahlberg
2014-09-06 7:50 ` [PATCH v4 04/32] rollback_lock_file(): exit early if lock is not active Michael Haggerty
2014-09-11 19:13 ` Ronnie Sahlberg
2014-09-06 7:50 ` [PATCH v4 05/32] rollback_lock_file(): set fd to -1 Michael Haggerty
2014-09-06 7:50 ` [PATCH v4 06/32] lockfile: unlock file if lockfile permissions cannot be adjusted Michael Haggerty
2014-09-09 22:39 ` Junio C Hamano
2014-09-12 11:03 ` Michael Haggerty
2014-09-06 7:50 ` [PATCH v4 07/32] hold_lock_file_for_append(): release lock on errors Michael Haggerty
2014-09-09 22:41 ` Junio C Hamano
2014-09-12 11:04 ` Michael Haggerty
2014-09-06 7:50 ` [PATCH v4 08/32] lock_file(): always add lock_file object to lock_file_list Michael Haggerty
2014-09-06 7:50 ` [PATCH v4 09/32] lockfile.c: document the various states of lock_file objects Michael Haggerty
2014-09-11 19:57 ` Ronnie Sahlberg
2014-09-06 7:50 ` [PATCH v4 10/32] cache.h: define constants LOCK_SUFFIX and LOCK_SUFFIX_LEN Michael Haggerty
2014-09-11 22:15 ` Ronnie Sahlberg
2014-09-12 16:44 ` Michael Haggerty
2014-09-11 22:42 ` Ronnie Sahlberg
2014-09-12 17:13 ` Michael Haggerty
2014-09-12 17:32 ` Ronnie Sahlberg
2014-09-06 7:50 ` [PATCH v4 11/32] delete_ref_loose(): don't muck around in the lock_file's filename Michael Haggerty
2014-09-13 7:41 ` Johannes Sixt
2014-09-14 6:27 ` Michael Haggerty
2014-09-14 6:38 ` Michael Haggerty
2014-09-14 14:49 ` Johannes Sixt
2014-09-06 7:50 ` [PATCH v4 12/32] prepare_index(): declare return value to be (const char *) Michael Haggerty
2014-09-06 7:50 ` [PATCH v4 13/32] write_packed_entry_fn(): convert cb_data into a (const int *) Michael Haggerty
2014-09-11 19:55 ` Ronnie Sahlberg
2014-09-06 7:50 ` [PATCH v4 14/32] lock_file(): exit early if lockfile cannot be opened Michael Haggerty
2014-09-11 22:49 ` Ronnie Sahlberg
2014-09-06 7:50 ` [PATCH v4 15/32] remove_lock_file(): call rollback_lock_file() Michael Haggerty
2014-09-06 7:50 ` [PATCH v4 16/32] commit_lock_file(): inline temporary variable Michael Haggerty
2014-09-06 7:50 ` [PATCH v4 17/32] commit_lock_file(): die() if called for unlocked lockfile object Michael Haggerty
2014-09-06 7:50 ` [PATCH v4 18/32] commit_lock_file(): if close fails, roll back Michael Haggerty
2014-09-06 7:50 ` [PATCH v4 19/32] commit_lock_file(): rollback lock file on failure to rename Michael Haggerty
2014-09-10 7:55 ` Jeff King
2014-09-10 12:55 ` Duy Nguyen
2014-09-06 7:50 ` [PATCH v4 20/32] api-lockfile: document edge cases Michael Haggerty
2014-09-06 7:50 ` [PATCH v4 21/32] dump_marks(): remove a redundant call to rollback_lock_file() Michael Haggerty
2014-09-06 7:50 ` [PATCH v4 22/32] git_config_set_multivar_in_file(): avoid " Michael Haggerty
2014-09-06 7:50 ` [PATCH v4 23/32] lockfile: avoid transitory invalid states Michael Haggerty
2014-09-06 7:50 ` [PATCH v4 24/32] struct lock_file: declare some fields volatile Michael Haggerty
2014-09-06 7:50 ` [PATCH v4 25/32] try_merge_strategy(): remove redundant lock_file allocation Michael Haggerty
2014-09-06 7:50 ` [PATCH v4 26/32] try_merge_strategy(): use a statically-allocated lock_file object Michael Haggerty
2014-09-06 7:50 ` [PATCH v4 27/32] commit_lock_file(): use a strbuf to manage temporary space Michael Haggerty
2014-09-06 7:50 ` [PATCH v4 28/32] Change lock_file::filename into a strbuf Michael Haggerty
2014-09-06 7:50 ` [PATCH v4 29/32] resolve_symlink(): use a strbuf for internal scratch space Michael Haggerty
2014-09-06 7:50 ` [PATCH v4 30/32] resolve_symlink(): take a strbuf parameter Michael Haggerty
2014-09-06 7:50 ` [PATCH v4 31/32] trim_last_path_elm(): replace last_path_elm() Michael Haggerty
2014-09-06 7:50 ` [PATCH v4 32/32] Extract a function commit_lock_file_to() Michael Haggerty
2014-09-07 14:21 ` [PATCH v4 00/32] Lockfile correctness and refactoring Torsten Bögershausen
2014-09-12 12:50 ` Michael Haggerty
2014-09-08 22:35 ` Junio C Hamano
2014-09-10 8:13 ` Jeff King
2014-09-10 10:25 ` Duy Nguyen
2014-09-10 10:30 ` Jeff King [this message]
2014-09-10 16:51 ` Junio C Hamano
2014-09-10 19:11 ` Jeff King
2014-09-12 11:28 ` Michael Haggerty
2014-09-12 11:13 ` Michael Haggerty
2014-09-12 14:21 ` Michael Haggerty
2014-09-13 18:51 ` Jeff King
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20140910103014.GA15461@peff.net \
--to=peff@peff.net \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=j6t@kdbg.org \
--cc=mhagger@alum.mit.edu \
--cc=pclouds@gmail.com \
--cc=tboegi@web.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).